👽 Reddit Post & Subreddit Extractor — Reddit Scraper for Posts, Comments, Communities & Users

Reddit Post & Subreddit Extractor is a production-grade Apify Actor for Reddit data extraction, subreddit scraping, comment thread collection and social listening. Give it search keywords, direct Reddit URLs, or a subreddit to pull in full, and it returns clean, structured posts, comments, communities and user profiles — written to your Apify Dataset in real time, ready for sentiment analysis, trend research, LLM training data, and RAG pipelines.

💡 Need a custom version, private integration, or a tailored data pipeline? Email hello.dataminds@gmail.com.

No Reddit account, no API key, and no OAuth app registration — the Actor reads the same public pages any visitor can see, applies your filters, and streams every finished item straight to your Dataset. Built for market researchers, social listening teams, content & trend analysts, academic researchers and community managers who need Reddit data without scrolling and copying by hand.

---

📑 Table of contents

• What is Reddit Post & Subreddit Extractor?
• Main features
• Who is this Actor for?
• Built for generative AI, LLMs, and RAG
• Search & source configuration
• Filtering & comment collection
• Output format (Dataset)
• Example output (JSON)
• Quick start
• Input parameters reference
• Integrations: LangChain, n8n, Make, warehouses
• Frequently asked questions (FAQ)
• Troubleshooting
• Help, support & custom builds
• Is web scraping legal?
• SEO keywords targeted

---

🏠 What is Reddit Post & Subreddit Extractor?

Reddit is one of the richest sources of public opinion, trend and community data online — and one of the most tedious to collect by hand. Posts are paginated, comment threads nest dozens deep, and pulling a whole subreddit's history one page at a time doesn't scale.

This Actor turns Reddit into a ready-to-use dataset:

• Searches Reddit for your keywords — posts, comments, or communities — with sort and time-range control.
• Opens direct URLs and classifies each one automatically: post, subreddit, user profile, or search-result page.
• Pulls a subreddit in full with the Full Subreddit Scrape option, going well past what a single listing page returns.
• Collects the full post profile — title, body, media flags, flair, score, comment count, and derived engagement metrics.
• Walks comment threads on request, including deeper "load more" replies, with author, score, depth, and thread position.
• Looks up community and user data — subscriber counts, active users, karma breakdown, account flags.
• Streams every finished item to your Dataset — export to JSON, CSV, Excel, XML, or pull it through the Apify API.

If you have ever needed "every post about X on r/Y from the last month, with comments" — this is the Actor.

---

✨ Main features

• 🔎 Keyword search — one term or many, each with its own independent limits, sort order (relevance, hot, top, new, comments) and time window.
• 🎯 Community-scoped search — restrict every keyword to a single subreddit with withinCommunity.
• 🔗 Direct URL scraping — post, subreddit, user profile, and search-result URLs, classified automatically.
• 📚 Full Subreddit Scrape — pass a name, r/name, or full URL and collect far more posts than one listing page allows.
• 💬 Comment threads on request — full text, author, score, depth, submitter flag, sticky flag, and thread position.
• 🗨️ Comment & community search — searchComments and searchCommunities widen a keyword run beyond posts.
• 👤 User & community profiles — karma breakdown, subscriber and active-user counts, descriptions, mod/gold/verified flags.
• 📅 Date-range filtering — separate windows for posts (postedAfter / postedBefore) and comments (commentedAfter / commentedBefore).
• 🏷️ Flair and NSFW filters — keep only flaired posts, and opt in to 18+ content explicitly.
• 📈 Built-in engagement metrics — ageHours, scorePerHour, comments-per-hour, upvote ratio, and an isHighEngagement flag computed for you.
• 🚦 Blocking handled for you — the run adapts on its own when a site pushes back, with no configuration required.
• ♻️ Automatic retries — transient failures are retried instead of dropped.
• 📦 Live Dataset writes — every item lands the moment it is ready, not at the end of the run.
• 🗂️ Five prebuilt table views — Overview, Posts, Comments, Communities, Users, so each data type reads cleanly in the Console.
• 📊 Run summary — item counts by type and the network route used, stored in the key-value store as run-summary.
• 🧾 Readable run log — every saved item is echoed live with its title, community, and score.

---

👥 Who is this Actor for?

• 📊 Market & trend researchers — track what a community says about a brand, product or topic over time.
• 📣 Social listening teams — monitor sentiment and volume across subreddits, daily or weekly.
• 🎓 Academic & data researchers — build reproducible, structured Reddit corpora for analysis.
• 🤖 AI engineers — collect grounded, human-written discussion text for RAG, fine-tuning, or evaluation sets.
• 🗞️ Content & SEO teams — find trending posts, questions, and pain points in a niche before writing.
• 🛡️ Community managers — audit a subreddit's post and comment activity without a scraping stack.
• 🧑‍💻 Developers — a dependable Reddit data source with no OAuth app, rate-limit juggling, or infrastructure to maintain.

---

🤖 Built for generative AI, LLMs, and RAG

Reddit text is conversational, opinionated, and dense with real-world language — exactly what modern AI pipelines are hungry for. Every row of this Actor's output is chunk-ready:

• RAG pipelines — index post bodies and comment threads so an assistant can answer "what do people actually say about X?"
• Sentiment & opinion mining — feed title + body + score into classification or scoring models.
• Trend detection — scorePerHour, ageHours, and isHighEngagement give a velocity signal out of the box.
• Vector database ingestion — embed and upsert into Pinecone, Qdrant, Weaviate, Chroma, Milvus, or pgvector, filtering by communityName or createdAt.
• Custom GPT & Claude Project knowledge files — export the Dataset as JSON and upload it directly.
• Fine-tuning & evaluation sets — build instruction or Q&A pairs from question posts and their top comments.
• AI agent grounding — give an agent live community context instead of stale training data.

Practical AI use cases

1. Voice-of-customer mining — search your product name across Reddit, collect posts plus comments, and cluster complaints by theme.
2. Competitive intelligence — monitor a competitor's subreddit for launch reactions and churn signals.
3. Support knowledge base — turn a community's most-upvoted troubleshooting threads into a retrieval corpus.
4. Content research — pull the top questions in a niche, then draft articles that answer them.
5. Academic discourse analysis — build a date-bounded corpus of one subreddit for NLP research.
6. Trend dashboards — schedule a daily run and chart post volume and engagement per community.
7. LLM evaluation data — sample real user questions to test how well a model answers them.

---

🧭 Search & source configuration

The Actor accepts three kinds of source, and you can mix all three in one run:

What you provide	What happens
best laptops 2025 in searchTerms	Every matching post Reddit returns for that keyword
AI news, machine learning, LLM	Three separate searches, each with its own independent limits
A post URL (/r/.../comments/...) in startUrls	That specific post, with its comments if enabled
A subreddit URL (/r/technology/, /r/technology/top/)	A listing walk of that subreddit
A user profile URL (/user/...)	Profile data, submitted posts, and comments
A search-result URL (/search?q=...)	The same results a Reddit search page shows
AskReddit, r/AskReddit, or a full link in subredditUrls	A far deeper pull of that subreddit's posts

• searchPosts / searchComments / searchCommunities decide which result types a keyword run collects. Posts are on by default.
• withinCommunity scopes every keyword to one subreddit — the equivalent of subreddit:name on Reddit's own search.
• searchSort (relevance, hot, top, new, comments) and searchTime (all, hour, day, week, month, year) shape which results come back first.
• maxPostsCount is applied per source, not as a shared pool — three terms with maxPostsCount: 25 can return 75 posts.

💡 Tip: start with a small maxPostsCount to sample the data shape, then raise it once the output looks right.

---

🧹 Filtering & comment collection

Filters run before an item is saved, so a narrow window keeps your Dataset (and your bill) small:

• postedAfter / postedBefore — keep only posts created inside this UTC window.
• commentedAfter / commentedBefore — the same window, applied to comments.
• onlyWithFlair — drop posts that carry no flair, useful in heavily-categorised subreddits.
• includeNSFW — off by default; 18+ content is skipped unless you opt in.

Comment collection is opt-in and independently budgeted:

• crawlCommentsPerPost — fetch the comment thread for every post the run finds.
• maxCommentsPerPost — cap per post, so one viral thread can't consume the whole run.
• maxCommentsCount — cap for comments found through keyword search (searchComments).
• postConcurrency — how many comment threads are fetched in parallel.

---

📤 Output format (Dataset)

Results land in the default Apify Dataset. You can:

• Browse rows in the Apify Console with five prebuilt views.
• Export to JSON, JSONL, CSV, Excel, XML, HTML, or RSS.
• Pull data programmatically via the Apify API.
• Stream rows into AI, ETL, or analytics pipelines while the run is still active.

View	What it shows
🏠 Overview	dataType, title/name, community, author, score, comment count, subscribers, karma, created date, links
📝 Posts	Title, body, flair, post type, score, upvotes, upvote ratio, comment count, NSFW/video/gallery flags, domain, engagement
💬 Comments	Comment body, author, score, depth, community, post id, permalink, submitter and sticky flags
📚 Communities	Prefixed name, title, public description, subscribers, active users, over-18 flag, subreddit type
👤 Users	Username, total/comment/link karma, mod / gold / verified flags, profile subreddit and its subscribers

Every row carries a dataType (post, comment, community, or user) so mixed exports stay easy to split. Key fields:

Field	Description
dataType	Which kind of record this row is — post, comment, community, user.
id, parsedId	Reddit's fullname (t3_…) and the bare id.
title, body	Post title and selftext, or the comment body.
communityName, authorName	Subreddit (r/…) and author the item belongs to.
flair, postType, domain	Post flair, link/text/media type, and the outbound domain for link posts.
score, upVotes, upvoteRatio, commentsCount	Engagement counters as Reddit reports them.
ageHours, scorePerHour, isHighEngagement	Derived velocity metrics computed by the Actor.
nsfw, isVideo, isGallery	Content flags.
depth, postId, isSubmitter, stickied	Comment-only thread position fields.
subscribers, activeUserCount, publicDescription, subredditType, over18	Community-only fields.
username, totalKarma, commentKarma, linkKarma, isMod, isGold, verified	User-only fields.
postUrl, permalink	Direct links back to the source on Reddit.
createdAt, crawledAt	When Reddit created the item, and when this run read it.
searchTerm	Which of your search terms produced this row (null for URL and subreddit sources).

📊 A run summary — item counts by type and the network route used — is stored in the run's key-value store as run-summary.

---

🧪 Example output (JSON)

A post row, exactly as it lands in the Dataset:

{
  "dataType": "post",
  "id": "t3_1vcc9rg",
  "parsedId": "1vcc9rg",
  "title": "Reddit Stock Collapses 23% as AI Eats Away at User Growth",
  "communityName": "r/technology",
  "authorName": "HumbleRestaurant790",
  "postType": "link",
  "flair": "Business",
  "score": 14899,
  "upVotes": 14899,
  "upvoteRatio": 0.93,
  "commentsCount": 2980,
  "postUrl": "https://www.reddit.com/r/technology/comments/1vcc9rg/...",
  "permalink": "/r/technology/comments/1vcc9rg/...",
  "domain": "finance.yahoo.com",
  "nsfw": false,
  "isVideo": false,
  "isGallery": false,
  "ageHours": 8.66,
  "scorePerHour": 1720.44,
  "isHighEngagement": true,
  "createdAt": "2026-08-01T03:38:10+00:00",
  "crawledAt": "2026-08-01T12:17:59.705129+00:00",
  "searchTerm": null
}

A comment row from the same run:

{
  "dataType": "comment",
  "id": "t1_m8xk20p",
  "body": "Growth stalling was obvious the moment third-party apps died.",
  "authorName": "quiet_dev",
  "communityName": "r/technology",
  "postId": "t3_1vcc9rg",
  "score": 1042,
  "depth": 0,
  "isSubmitter": false,
  "stickied": false,
  "permalink": "/r/technology/comments/1vcc9rg/.../m8xk20p/",
  "createdAt": "2026-08-01T04:12:44+00:00",
  "crawledAt": "2026-08-01T12:18:03.118904+00:00",
  "searchTerm": null
}

---

🚀 Quick start

Run in Apify Console

1. Log in at console.apify.com → Actors.
2. Open Reddit Post & Subreddit Extractor.
3. Enter a search term, paste a Reddit URL, or add a subreddit to Full Subreddit Scrape.
4. Set Max posts to a small number for a first test.
5. (Optional) turn on Scrape comments for each post.
6. Click Start and watch each item appear live in the log.
7. Open the Output tab, switch between the five views, and export to JSON / CSV / Excel.

Run via API

curl -X POST "https://api.apify.com/v2/acts/YOUR_ACTOR_ID/runs?token=YOUR_APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "searchTerms": ["artificial intelligence"],
    "searchSort": "top",
    "searchTime": "week",
    "maxPostsCount": 25,
    "proxyConfiguration": { "useApifyProxy": false }
  }'

Fetch the results once the run finishes:

$curl "https://api.apify.com/v2/datasets/YOUR_DATASET_ID/items?token=YOUR_APIFY_TOKEN&format=csv"

Python client

from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")
run = client.actor("YOUR_ACTOR_ID").call(run_input={
    "startUrls": [{"url": "https://www.reddit.com/r/technology/"}],
    "crawlCommentsPerPost": True,
    "maxCommentsPerPost": 20,
})

for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item["dataType"], item.get("title") or item.get("body"))

---

🛠️ Input parameters reference

Parameter	Type	Default	Description
searchTerms	array	[]	Keywords to search, one per entry — each gets its own limits.
searchPosts	boolean	true	Include posts in keyword search results.
searchComments	boolean	false	Include comments in keyword search results.
searchCommunities	boolean	false	Include communities in keyword search results.
withinCommunity	string	—	Restrict every keyword search to one subreddit.
searchSort	string	new	relevance, hot, top, new, comments.
searchTime	string	all	all, hour, day, week, month, year.
startUrls	array	[]	Reddit URLs to scrape directly — post, subreddit, user, or search pages. Bulk input supported.
subredditUrls	array	[]	Subreddits to scrape in full — name, r/name, or full URL.
postedAfter / postedBefore	string	—	Keep only posts created in this window (UTC).
commentedAfter / commentedBefore	string	—	Keep only comments created in this window (UTC).
onlyWithFlair	boolean	false	Keep only posts that have a flair.
includeNSFW	boolean	false	Include NSFW (18+) content.
crawlCommentsPerPost	boolean	false	Fetch the full comment thread for every post found.
maxCommentsPerPost	integer	10	Max comments per individual post.
maxPostsCount	integer	10	Max posts per search term / URL / subreddit.
maxCommentsCount	integer	10	Max comments from keyword comment-search.
maxCommunitiesCount	integer	2	Max communities from keyword community-search.
proxyConfiguration	object	no proxy	Optional proxy configuration for the run.
postConcurrency	integer	3	Comment threads fetched in parallel.
maxRetries	integer	3	Attempts per request before it is skipped.
requestDelayMin	number	1.0	Lower bound of the random pause between requests (seconds).
requestDelayMax	number	2.5	Upper bound of the random pause between requests (seconds).

---

🔌 Integrations: LangChain, n8n, Make, warehouses

This Actor's Dataset output plugs directly into the modern data and AI stack:

• 🦜 LangChain — load rows with ApifyDatasetLoader and map body → page_content, everything else → metadata.
• 🦙 LlamaIndex — build a VectorStoreIndex over post and comment text for community-grounded Q&A.
• 🧠 OpenAI Embeddings & Assistants — chunk title + body, embed, and upload as knowledge files.
• 🤖 Anthropic Claude — drop the Dataset JSON into Claude Projects or the Files API as RAG context.
• 📍 Pinecone, Qdrant, Weaviate, Chroma, Milvus, pgvector — filter by communityName, createdAt, or score at query time.
• 📊 n8n, Make.com, Zapier — push new rows into Slack, Airtable, Sheets, or Notion the moment a run ends.
• 📦 Snowflake, BigQuery, PostgreSQL, MongoDB, S3 — load Dataset JSON/CSV into your warehouse or lake.
• ⏰ Apify Schedules & Webhooks — run hourly, daily, or weekly and notify your own service on finish.
• 🧰 Apify API & SDKs — orchestrate runs from your own services in any language.

---

❓ Frequently asked questions (FAQ)

What is the best way to scrape Reddit posts?

Give this Actor a search term, a subreddit, or a direct URL and press Start. It handles pagination, filtering, and comment collection for you, and writes clean rows to a Dataset you can export or query through the API — no Reddit API key, OAuth app, or login involved.

Do I need a Reddit API key or a developer account?

No. The Actor reads publicly available pages, so there is nothing to register and no per-app rate quota to manage.

Can I scrape an entire subreddit?

Yes — put the subreddit in subredditUrls (Full Subreddit Scrape) and raise maxPostsCount. It collects far more posts than a single listing page returns.

Does it collect comments?

Turn on crawlCommentsPerPost to fetch the full comment thread for every post found, and cap it with maxCommentsPerPost. searchComments additionally lets you search for comments matching a keyword.

How many posts can I collect in one run?

maxPostsCount applies per search term, URL, or subreddit — each source gets its own independent budget, not a shared pool.

Do I need a proxy?

No. Leave the proxy input empty and the run handles blocking on its own. Set one only when you want a specific country or route.

Can I filter by date?

Yes — postedAfter / postedBefore for posts, commentedAfter / commentedBefore for comments. Both take UTC dates.

How do I export the results?

Open the run in Apify Console → Output tab → Export, and choose JSON, CSV, Excel, XML, HTML, or RSS. You can also pull rows through the Apify Dataset API.

Can I schedule recurring runs?

Yes — use Apify Schedules to run hourly, daily, or weekly, and pipe new rows into your warehouse, vector DB, or Slack channel.

Can I feed the results into my own pipeline?

Yes — use webhooks or the API, or connect Make / Zapier / n8n to push new rows into a database, spreadsheet, or vector store.

What if I need a field that isn't here?

Email hello.dataminds@gmail.com — custom fields and private builds are available.

---

🛠️ Troubleshooting

Symptom	Likely cause	Fix
No items saved	Reddit throttling the run, or every item filtered out	Widen the date range and retry with a smaller maxPostsCount
Few comments found	Comment collection off, or the per-post cap is low	Enable crawlCommentsPerPost and raise maxCommentsPerPost
Fewer results than requested	The term or subreddit simply holds fewer matching items	Broaden the keyword, raise searchTime, or widen postedAfter / postedBefore
Run feels slow	Deep comment threads	Lower maxCommentsPerPost, raise postConcurrency, or turn off crawlCommentsPerPost
searchComments returns little	Reddit's own comment search is limited	Results are approximated by scanning comment-rich posts — expect slower, less exhaustive coverage than post search
NSFW posts missing	includeNSFW is off by default	Set includeNSFW: true
Nothing happens at all	No source given	Provide at least one of searchTerms, startUrls, or subredditUrls

---

💬 Help, support & custom builds

For custom solutions, private integrations, white-label scrapers, or implementation help:

📧 hello.dataminds@gmail.com

• 🐛 Found a bug or need a new field? Open an issue on the Actor's Issues tab.
• ⭐ Enjoying it? Leave a review — it genuinely helps.

Useful Apify documentation:

• 📘 Apify documentation
• 🤖 Actors overview
• 🔌 Apify API reference
• 📚 Apify Academy — web scraping
• 🧰 Apify SDKs

---

⚖️ Is web scraping legal?

This Actor collects publicly available Reddit posts, comments, and profile data — the same information any visitor can see without logging in, with no credentials of any kind used. Scraping public data is generally lawful, but you remain responsible for how you use it:

• ✅ Do not collect data behind authentication or paywalls.
• ✅ Respect Reddit's terms of service and reasonable request rates.
• ✅ Handle personal data (a username's linked activity, for example) in line with GDPR, CCPA, and other applicable law.
• ✅ Respect copyright in the content you store or republish.

See Apify's guide on the legality of web scraping for background. This is not legal advice.

---

🔍 SEO keywords targeted

This README is intentionally keyword-rich for discovery on Google Search, Apify Store search, and Bing. Primary and secondary terms covered include:

Primary: reddit scraper · reddit post scraper · subreddit scraper · reddit data extractor · reddit comment scraper · reddit crawler · reddit scraping api · reddit api alternative

Data types: reddit post data · reddit comment threads · subreddit statistics · reddit user profile scraper · reddit karma scraper · community subscriber data · reddit flair filter · reddit engagement metrics

AI & LLM: reddit data for LLM training · RAG pipeline data source · reddit sentiment analysis data · vector database ingestion · embedding pipeline · custom GPT knowledge file · Claude Projects data · LangChain data loader · AI agent grounding

Use cases: social listening tool · market research tool · reddit trend analysis · voice of customer mining · competitive intelligence · academic reddit dataset · content research tool · reddit lead research

Capabilities: no api key reddit scraper · bulk reddit url scraper · reddit search scraper · date range filtering · nsfw filtering · anti-blocking scraper · automatic block handling · live dataset streaming

Integrations: n8n · Make.com · Zapier · Pinecone ingestion · Qdrant · Weaviate · Snowflake · BigQuery · Google Sheets · Airtable · Slack alerts