# Reddit Scraper | Posts, Search, Comments & User Data (`scrapepilot/reddit-scraper-posts-search-comments-user-data`) Actor

Extract structured Reddit data from subreddits, search results, single posts, and user profiles. Get titles, text, scores, upvote ratio, comments, authors, flairs, timestamps, and more in clean JSON. Built for research, monitoring, trend tracking, and automation

- **URL**: https://apify.com/scrapepilot/reddit-scraper-posts-search-comments-user-data.md
- **Developed by:** [Scrape Pilot](https://apify.com/scrapepilot) (community)
- **Categories:** Developer tools, Lead generation, Social media
- **Stats:** 2 total users, 0 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

$5.99/month + usage

To use this Actor, you pay a monthly rental fee to the developer. The rent is subtracted from your prepaid usage every month after the free trial period.You also pay for the Apify platform usage, which gets cheaper the higher Apify subscription plan you have.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#rental-actors

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## 🚀 Reddit Posts & Comments Scraper

<div align="center">

![GitHub stars](https://img.shields.io/github/stars/yourusername/reddit-scraper?style=for-the-badge)
![GitHub forks](htthttpsps://img.shields.io/github/forks/yourusername/reddit-scraper?style=for-the-badge)
![GitHub issues](https://img.shields.io/github/issues/yourusername/reddit-scraper?style=for-the-badge)
![License](https://img.shields.io/github/license/yourusername/reddit-scraper?style=for-the-badge)
![Python Version](https://img.shields.io/badge/python-3.8%2B-blue?style=for-the-badge)

**A powerful reddit scraper tool to extract posts, comments, and metadata from Reddit for data analysis, market research, and sentiment tracking.**

[Features](#-features) • [Installation](#-installation) • [Usage](#-usage) • [Documentation](#-documentation) • [Contributing](#-contributing)

</div>

***

httpshttpshttpshttpshttpshttpshttpshttpshttpshttpshttpshttpshttpshttpshttpshttps

### 📋 Table of Contents

- [About](#-about)
- [Features](#-features)
- [Installation](#-installation)
- [Quick Start](#-quick-start)
- [Configuration](#-configuration)
- [Input/Output Format](#-inputoutput-format)
- [API Reference](#-api-reference)
- [Examples](#-examples)
- [Proxy Support](#-proxy-support)
- [Rate Limiting](#-rate-limiting)
- [Troubleshooting](#-troubleshooting)
- [Contributing](#-contributing)
- [License](#-license)
- [FAQ](#-faq)

***

### 📖 About

The **Reddit Posts & Comments Scraper** is a professional-grade **reddit scraper** tool designed to efficiently extract public posts, comments, and metadata from Reddit subreddits. Whether you're conducting market research, sentiment analysis, or building data-driven applications, this **reddit scraper** provides reliable and structured data extraction capabilities.

This tool is built with scalability and compliance in mind, respecting Reddit's API guidelines while delivering high-performance data extraction for developers, researchers, and businesses.

***

### ✨ Features

| Feature | Description |
|---------|-------------|
| 🎯 **Targeted Scraping** | Extract posts from specific subreddits with custom filters |
| 💬 **Comment Extraction** | Optional comment scraping for deeper insights |
| 🔒 **Proxy Support** | Residential & datacenter proxy configuration included |
| 📊 **Rich Metadata** | Get scores, upvote ratios, authors, flairs, and more |
| 🔄 **Multiple Sort Options** | Sort by hot, new, top, rising, and controversial |
| ⏱️ **Time Filtering** | Filter posts by hour, day, week, month, year, or all time |
| 📁 **Multiple Formats** | Export data in JSON, CSV, or XML formats |
| 🚀 **High Performance** | Optimized for large-scale data extraction |
| 🛡️ **Rate Limiting** | Built-in rate limiting to avoid IP bans |
| 📝 **Detailed Logging** | Comprehensive logging for debugging and monitoring |

***

### ⚡ Quick Start

#### Basic Usage

```python
from reddit_scraper import RedditScraper

## Initialize the scraper
scraper = RedditScraper()

## Define your configuration
config = {
    "include_comments": False,
    "subreddit": "technology",
    "sort": "hot",
    "time_filter": "all",
    "max_results": 25
}

## Run the scraper
results = scraper.scrape(config)

## Export to JSON
scraper.export_to_json(results, "output.json")

## Export to CSV
scraper.export_to_csv(results, "output.csv")
```

#### Command Line Usage

```bash
## Basic scrape
python reddit_scraper.py --subreddit technology --max-results 25

## With comments
python reddit_scraper.py --subreddit technology --include-comments --max-results 50

## With custom sort and time filter
python reddit_scraper.py --subreddit technology --sort top --time-filter week --max-results 100

## With proxy configuration
python reddit_scraper.py --subreddit technology --use-proxy --proxy-group RESIDENTIAL
```

***

### ⚙️ Configuration

#### Input Parameters

| Parameter | Type | Required | Default | Description |
|-----------|------|----------|---------|-------------|
| `subreddit` | string | ✅ Yes | - | Target subreddit name (e.g., "technology") |
| `include_comments` | boolean | ❌ No | `false` | Whether to scrape comments for each post |
| `sort` | string | ❌ No | `"hot"` | Sort order: `hot`, `new`, `top`, `rising`, `controversial` |
| `time_filter` | string | ❌ No | `"all"` | Time range: `hour`, `day`, `week`, `month`, `year`, `all` |
| `max_results` | integer | ❌ No | `25` | Maximum number of posts to scrape (1-1000) |
| `proxyConfiguration.useApifyProxy` | boolean | ❌ No | `false` | Enable Apify proxy service |
| `proxyConfiguration.apifyProxyGroups` | array | ❌ No | `[]` | Proxy groups: `RESIDENTIAL`, `DATACENTER` |

#### Example Configuration

```json
{
  "include_comments": false,
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": ["RESIDENTIAL"]
  },
  "subreddit": "technology",
  "sort": "hot",
  "time_filter": "all",
  "max_results": 25
}
```

***

### 📥 Input/Output Format

#### Input Example

```json
{
  "include_comments": false,
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": ["RESIDENTIAL"]
  },
  "subreddit": "technology",
  "sort": "hot",
  "time_filter": "all",
  "max_results": 25
}
```

#### Output Example

```json
[
  {
    "post_id": "1rt52qa",
    "title": "Meta planning sweeping layoffs as AI costs mount",
    "text": null,
    "score": 4506,
    "upvote_ratio": 0.97,
    "url": "https://www.reuters.com/business/world-at-work/meta-planning-sweeping-layoffs-ai-costs-mount-2026-03-14/",
    "permalink": "https://www.reddit.com/r/technology/comments/1rt52qa/meta_planning_sweeping_layoffs_as_ai_costs_mount/",
    "author": "joe4942",
    "subreddit": "technology",
    "flair": "Business",
    "num_comments": 569,
    "awards": 0,
    "is_video": false,
    "domain": "reuters.com",
    "thumbnail": "https://external-preview.redd.it/...",
    "created_at": "1773448769"
  }
]
```

#### Output Fields Description

| Field | Type | Description |
|-------|------|-------------|
| `post_id` | string | Unique Reddit post identifier |
| `title` | string | Post title |
| `text` | string/null | Self-post text content (null for link posts) |
| `score` | integer | Total upvotes minus downvotes |
| `upvote_ratio` | float | Percentage of upvotes (0.0 - 1.0) |
| `url` | string | Original link URL (for link posts) |
| `permalink` | string | Reddit post permalink |
| `author` | string | Post author username |
| `subreddit` | string | Subreddit name |
| `flair` | string/null | Post flair text |
| `num_comments` | integer | Number of comments on the post |
| `awards` | integer | Total awards received |
| `is_video` | boolean | Whether the post is a video |
| `domain` | string/null | Domain of the linked content |
| `thumbnail` | string/null | Thumbnail image URL |
| `created_at` | string | Unix timestamp of post creation |

***

### 🔌 API Reference

#### Class: `RedditScraper`

##### Constructor

```python
scraper = RedditScraper(api_credentials=None, rate_limit=True)
```

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `api_credentials` | dict | `None` | Reddit API credentials (client\_id, client\_secret) |
| `rate_limit` | boolean | `True` | Enable automatic rate limiting |

##### Methods

| Method | Parameters | Returns | Description |
|--------|------------|---------|-------------|
| `scrape(config)` | `config: dict` | `list` | Main scraping method |
| `export_to_json(data, filename)` | `data: list, filename: str` | `bool` | Export data to JSON file |
| `export_to_csv(data, filename)` | `data: list, filename: str` | `bool` | Export data to CSV file |
| `export_to_xml(data, filename)` | `data: list, filename: str` | `bool` | Export data to XML file |
| `validate_config(config)` | `config: dict` | `bool` | Validate configuration parameters |
| `get_subreddit_info(name)` | `name: str` | `dict` | Get subreddit metadata |

***

### 💡 Examples

#### Example 1: Scrape Top Posts from r/technology

```python
config = {
    "subreddit": "technology",
    "sort": "top",
    "time_filter": "week",
    "max_results": 50
}

results = scraper.scrape(config)
print(f"Scraped {len(results)} posts")
```

#### Example 2: Scrape with Comments

```python
config = {
    "subreddit": "programming",
    "include_comments": True,
    "sort": "hot",
    "max_results": 10
}

results = scraper.scrape(config)

for post in results:
    print(f"Post: {post['title']}")
    print(f"Comments: {len(post.get('comments', []))}")
```

#### Example 3: Multiple Subreddits

```python
subreddits = ["technology", "programming", "artificial"]

for subreddit in subreddits:
    config = {
        "subreddit": subreddit,
        "max_results": 25
    }
    results = scraper.scrape(config)
    scraper.export_to_json(results, f"{subreddit}_posts.json")
```

#### Example 4: With Proxy Configuration

```python
config = {
    "subreddit": "technology",
    "proxyConfiguration": {
        "useApifyProxy": True,
        "apifyProxyGroups": ["RESIDENTIAL"]
    },
    "max_results": 100
}

results = scraper.scrape(config)
```

***

### 🔐 Proxy Support

This **reddit scraper** supports advanced proxy configurations to avoid rate limiting and IP bans.

#### Supported Proxy Types

| Proxy Type | Description | Best For |
|------------|-------------|----------|
| `RESIDENTIAL` | Real user IP addresses | High-volume scraping |
| `DATACENTER` | Datacenter IP addresses | Fast, cost-effective scraping |

#### Proxy Configuration

```json
{
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": ["RESIDENTIAL"],
    "apifyProxyCountry": "US"
  }
}
```

#### Environment Variables

```bash
## .env file
APIFY_API_TOKEN=your_apify_token_here
PROXY_ENABLED=true
PROXY_GROUP=RESIDENTIAL
```

***

### ⏱️ Rate Limiting

To ensure responsible usage and avoid bans, this **reddit scraper** includes built-in rate limiting:

| Action | Rate Limit | Recommendation |
|--------|------------|----------------|
| API Requests | 60/minute | Use proxy for higher limits |
| Post Scraping | 100/minute | Enable delays between requests |
| Comment Scraping | 50/minute | Use residential proxies |

#### Rate Limit Configuration

```python
scraper = RedditScraper(
    rate_limit=True,
    rate_limit_delay=1.0,  # seconds between requests
    max_retries=3
)
```

***

### 🛠️ Troubleshooting

#### Common Issues

| Issue | Solution |
|-------|----------|
| **429 Too Many Requests** | Enable proxy, increase delay between requests |
| **403 Forbidden** | Check subreddit privacy settings, use API credentials |
| **Empty Results** | Verify subreddit name, check sort/time\_filter values |
| **Connection Timeout** | Enable proxy, check network connection |
| **Invalid JSON Output** | Validate input configuration format |

#### Debug Mode

```bash
## Enable verbose logging
python reddit_scraper.py --subreddit technology --debug

## Check API status
python reddit_scraper.py --status-check
```

#### Log Files

Logs are saved in `./logs/scraper.log` by default. Configure log level:

```python
import logging
logging.basicConfig(level=logging.DEBUG)
```

***

### 🤝 Contributing

We welcome contributions! Here's how you can help:

1. **Fork** the repository
2. **Create** a feature branch (`git checkout -b feature/amazing-feature`)
3. **Commit** your changes (`git commit -m 'Add amazing feature'`)
4. **Push** to the branch (`git push origin feature/amazing-feature`)
5. **Open** a Pull Request

#### Development Setup

```bash
## Clone your fork
git clone https://github.com/yourusername/reddit-scraper.git

## Install dev dependencies
pip install -r requirements-dev.txt

## Run tests
pytest tests/

## Run linting
flake8 .
black .
```

#### seo keyword

***

reddit scraper, reddit post scraper, subreddit scraper, reddit search scraper, reddit comments scraper,
reddit user scraper, reddit data extractor, reddit api scraper, apify reddit scraper, social listening scraper,
community analysis scraper, osint reddit scrape
-----------------------------------------------

#### Code Style

- Follow PEP 8 guidelines
- Add docstrings for all functions
- Write unit tests for new features
- Update documentation for changes

***

# Actor input Schema

## `subreddit` (type: `string`):

Subreddit to scrape. E.g: technology, worldnews. No r/ needed.

## `query` (type: `string`):

Search Reddit for keywords. Leave empty to browse subreddit.

## `sort` (type: `string`):

How to sort posts

## `time_filter` (type: `string`):

Time range for Top sort

## `max_results` (type: `integer`):

Maximum posts to return (1-500)

## `include_comments` (type: `boolean`):

Fetch top 10 comments for each post

## `proxyConfiguration` (type: `object`):

Use Apify Proxy to avoid rate limits. RESIDENTIAL recommended.

## Actor input object example

```json
{
  "subreddit": "technology",
  "sort": "hot",
  "time_filter": "all",
  "max_results": 25,
  "include_comments": false,
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": [
      "RESIDENTIAL"
    ]
  }
}
```

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "subreddit": "technology",
    "proxyConfiguration": {
        "useApifyProxy": true,
        "apifyProxyGroups": [
            "RESIDENTIAL"
        ]
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("scrapepilot/reddit-scraper-posts-search-comments-user-data").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "subreddit": "technology",
    "proxyConfiguration": {
        "useApifyProxy": True,
        "apifyProxyGroups": ["RESIDENTIAL"],
    },
}

# Run the Actor and wait for it to finish
run = client.actor("scrapepilot/reddit-scraper-posts-search-comments-user-data").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "subreddit": "technology",
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": [
      "RESIDENTIAL"
    ]
  }
}' |
apify call scrapepilot/reddit-scraper-posts-search-comments-user-data --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "command": "npx",
            "args": [
                "mcp-remote",
                "https://mcp.apify.com/?tools=scrapepilot/reddit-scraper-posts-search-comments-user-data",
                "--header",
                "Authorization: Bearer <YOUR_API_TOKEN>"
            ]
        }
    }
}

```

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/1hHSJFPg0DwPASXT9/builds/jXP2EOkztflDOk6yw/openapi.json