👽 Reddit Scraper - Posts, Comments, Search & Subreddits — No API Key, No Login

Reddit Scraper - Posts, Comments, Search & Subreddits is a production-grade Apify Actor that turns Reddit into structured data — posts, full comment threads, subreddits, user profiles and keyword search results — with no Reddit account, no API key, no OAuth app and no login. Paste URLs in bulk, type keywords, or name a subreddit to mine in full; every finished record is written to your Apify Dataset in real time, one tidy section per data type.

💡 Need a custom version, private integration, or a tailored data pipeline? Email hello.dataminds@gmail.com.

Where Reddit's official API means registering an app, holding a token and living inside a quota, this Actor reads the same public pages a visitor sees and hands you clean JSON. Built for social listening, market and trend research, community analytics, AI training corpora and lead research — at whatever scale your run budget allows.

---

📑 Table of contents

• What is Reddit Scraper - Posts, Comments, Search & Subreddits?
• Main features
• Who is this Actor for?
• Built for generative AI, LLMs, and RAG
• Bulk URLs, keywords & full subreddits
• Filters, comment depth & limits
• Output format (Dataset)
• Example output (JSON)
• Quick start
• Input parameters reference
• Integrations: MCP, LangChain, n8n, warehouses
• Frequently asked questions (FAQ)
• Troubleshooting
• Help, support & custom builds
• Is web scraping legal?
• SEO keywords targeted

---

🏠 What is Reddit Scraper - Posts, Comments, Search & Subreddits?

Reddit holds millions of candid, human-written opinions — and almost none of it is convenient to collect. Listings paginate, threads nest, "load more" hides the replies that matter, and the official API asks for an app registration before you read a single row.

This Actor removes all of that:

• No credentials at all — no login, no API key, no OAuth app, no token to rotate.
• Bulk URL input — drop in hundreds of post, subreddit, user, or search links at once; each is classified automatically.
• Keyword search — search posts, comments, or communities, with sort order and time window under your control.
• Full subreddit mining — go far past what one listing page returns.
• Real comment threads — including deeper "load more" replies, with author, score, depth and thread position.
• One section per data type — posts, comments, communities and users each get their own prebuilt table view.
• Live Dataset writes — export JSON, JSONL, CSV, Excel, XML, or pull through the Apify API while the run is still going.

If you have ever wanted "this subreddit, these keywords, with the comments, as a spreadsheet" — that is exactly what this Actor produces.

---

✨ Main features

• 🔓 Zero credentials — no Reddit account, API key, OAuth app, or login of any kind.
• 🧷 Bulk URL input — posts, subreddits, user profiles and search pages in one list, each classified on sight.
• 💬 Keyword search across three result types — posts (searchPosts), comments (searchComments) and communities (searchCommunities).
• 🧭 Search control — searchSort (relevance, hot, top, new, comments) and searchTime (hour → all).
• 🎯 Single-community scope — withinCommunity keeps a keyword run inside one subreddit.
• 🏔️ Full subreddit mining — plain name, r/name, or full URL in subredditUrls.
• 🧵 Deep comment threads — crawlCommentsPerPost with a per-post cap and parallel fetching.
• 🗓️ Independent date windows — one for posts, one for comments, both in UTC.
• 🏷️ Flair-only and NSFW switches — keep the corpus clean, or explicitly opt in to 18+ content.
• 📈 Engagement metrics included — ageHours, scorePerHour, upvote ratio, and an isHighEngagement flag.
• 🚦 Blocking handled for you — the run adapts on its own when a site pushes back, with no configuration required.
• ♻️ Automatic retries — transient failures never abort the run.
• 🗂️ Five prebuilt views — Overview, Posts, Comments, Communities, Users.
• 📦 Streamed output — rows appear in your Dataset as they are produced, so downstream jobs can start immediately.
• 📊 Run summary in the key-value store — counts per data type plus the network route actually used.

---

👥 Who is this Actor for?

• 📣 Social listening & brand teams — watch mentions, complaints and launch reactions across communities.
• 📊 Market researchers — quantify how often a topic, product or competitor comes up, and how hard it lands.
• 🤖 AI & data engineers — assemble conversational corpora for RAG, fine-tuning, or evaluation.
• 🎓 Academics — build reproducible, date-bounded discourse datasets.
• 🗞️ Content & SEO teams — mine real questions and phrasing straight from the audience.
• 🛍️ Product & growth teams — find feature requests and churn reasons in the wild.
• 🧑‍💻 Developers — a Reddit data source with no API registration, quota juggling, or scraping stack to maintain.

---

🤖 Built for generative AI, LLMs, and RAG

Reddit's value to an AI pipeline is that it is unpolished — real questions, real answers, real language. Every row here is chunk-ready:

• RAG over community knowledge — index posts plus their comment threads so an assistant can cite what people actually said.
• Fine-tuning & instruction data — pair question posts with their highest-scoring replies.
• Sentiment, intent & topic models — body + score + communityName is enough to train or evaluate.
• Vector store ingestion — embed and upsert into Pinecone, Qdrant, Weaviate, Chroma, Milvus, or pgvector.
• Custom GPT / Claude Project knowledge files — export the Dataset JSON and upload it as-is.
• Agent tooling via MCP — expose this Actor as a tool to Claude, Cursor and other MCP clients through Apify's MCP server.
• Evaluation sets — sample genuine user questions to benchmark model answers.

Practical AI use cases

1. Support assistant grounding — mine the top troubleshooting threads of a product's subreddit into a retrieval index.
2. Churn & complaint analysis — collect a month of posts mentioning your brand, then cluster the negatives.
3. Question mining for content — pull the most-commented questions in a niche and answer them on your site.
4. Community trend dashboard — schedule daily runs, chart scorePerHour by subreddit.
5. Discourse research corpus — one subreddit, one date window, posts plus comments, reproducible.
6. Agent research tool — let an MCP-connected agent query Reddit live during a research task.
7. Training-data curation — filter by flair and score to keep only high-signal threads.

---

🔗 Bulk URLs, keywords & full subreddits

Three input styles, freely mixable in one run:

Input	Where it goes	What comes back
https://www.reddit.com/r/technology/comments/…	startUrls	That post, plus its comments when enabled
https://www.reddit.com/r/technology/top/	startUrls	A listing walk of that subreddit
https://www.reddit.com/user/spez/	startUrls	Profile data, submitted posts and comments
https://www.reddit.com/search?q=gpu+prices	startUrls	The same results the search page shows
gpu prices, graphics card deals	searchTerms	Two independent searches, each with its own budget
AskReddit · r/AskReddit · full link	subredditUrls	A deep pull of that subreddit's posts

• Paste hundreds of URLs at once — startUrls accepts bulk input and every entry is budgeted separately.
• maxPostsCount applies per source, so five subreddits at maxPostsCount: 50 can return 250 posts.
• withinCommunity narrows keyword searches to a single subreddit without changing the terms themselves.

💡 Tip: run one URL with maxPostsCount: 5 first to confirm the field shape, then scale the same input up.

---

🧹 Filters, comment depth & limits

Filters are applied before a record is saved, so tightening them cuts both noise and cost:

Control	Effect
postedAfter / postedBefore	Keep only posts created inside this UTC window
commentedAfter / commentedBefore	The same window applied to comments
onlyWithFlair	Drop posts with no flair — useful in tightly categorised subreddits
includeNSFW	Off by default; 18+ content is skipped unless you opt in
crawlCommentsPerPost	Fetch the comment thread of every post the run finds
maxCommentsPerPost	Per-post ceiling so one viral thread can't eat the run
maxCommentsCount	Ceiling for comments discovered through keyword search
maxCommunitiesCount	Ceiling for communities discovered through keyword search
postConcurrency	How many comment threads are fetched in parallel

---

📤 Output format (Dataset)

Results land in the default Apify Dataset, with five prebuilt views so each data type reads as its own clean table:

View	What it shows
🏠 Overview	dataType, title/name, community, author, score, comment count, subscribers, karma, created date, links
📝 Posts	Title, body, flair, post type, score, upvotes, upvote ratio, comments, NSFW/video/gallery flags, domain, engagement
💬 Comments	Body, author, score, depth, community, post id, permalink, submitter and sticky flags
📚 Communities	Prefixed name, title, public description, subscribers, active users, over-18 flag, subreddit type
👤 Users	Username, total/comment/link karma, mod / gold / verified flags, profile subreddit and its subscribers

Every row carries dataType (post, comment, community, user), so a mixed export splits cleanly. Key fields:

Field	Description
dataType	Record kind — post, comment, community, or user.
id, parsedId	Reddit fullname (t3_…, t1_…) and the bare id.
title, body	Post title and selftext, or the comment text.
communityName, authorName	Subreddit (r/…) and author.
flair, postType, domain	Flair, link/text/media type, outbound domain for link posts.
score, upVotes, upvoteRatio, commentsCount	Engagement counters as Reddit reports them.
ageHours, scorePerHour, isHighEngagement	Velocity metrics computed by the Actor.
nsfw, isVideo, isGallery	Content flags.
depth, postId, isSubmitter, stickied	Comment-only thread-position fields.
subscribers, activeUserCount, publicDescription, subredditType, over18	Community-only fields.
username, totalKarma, commentKarma, linkKarma, isMod, isGold, verified	User-only fields.
postUrl, permalink	Direct links back to Reddit.
createdAt, crawledAt	Reddit's creation time, and when this run read the item.
searchTerm	Which keyword produced the row (null for URL and subreddit sources).

📊 A run summary — counts per data type and the network route used — is stored in the run's key-value store as run-summary, and linked from the run's Output tab alongside the Dataset.

---

🧪 Example output (JSON)

A community record:

{
  "dataType": "community",
  "id": "t5_2qh16",
  "name": "technology",
  "namePrefixed": "r/technology",
  "title": "Technology",
  "publicDescription": "Subreddit dedicated to the news and discussions about the creation and use of technology.",
  "subscribers": 18402911,
  "activeUserCount": 6215,
  "over18": false,
  "subredditType": "public",
  "createdAt": "2008-01-25T05:15:47+00:00",
  "crawledAt": "2026-08-01T12:19:44.220118+00:00",
  "searchTerm": "technology"
}

A user record from the same run:

{
  "dataType": "user",
  "id": "t2_9k1c8",
  "username": "quiet_dev",
  "totalKarma": 148902,
  "commentKarma": 121447,
  "linkKarma": 27455,
  "isMod": false,
  "isGold": true,
  "verified": true,
  "subredditDisplayName": "u_quiet_dev",
  "subredditSubscribers": 412,
  "createdAt": "2014-03-02T19:44:10+00:00",
  "crawledAt": "2026-08-01T12:19:51.884210+00:00"
}

---

🚀 Quick start

Run in Apify Console

1. Log in at console.apify.com → Actors.
2. Open Reddit Scraper - Posts, Comments, Search & Subreddits.
3. Paste your Reddit URLs (bulk is fine), type search terms, or name a subreddit to mine in full.
4. Set Max posts per source low for the first run.
5. (Optional) enable Pull comments for every post found.
6. Press Start and watch records stream into the log.
7. Open the Output tab, switch views, and export to JSON / CSV / Excel.

Run via API

curl -X POST "https://api.apify.com/v2/acts/YOUR_ACTOR_ID/runs?token=YOUR_APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "startUrls": [
      { "url": "https://www.reddit.com/r/technology/top/" },
      { "url": "https://www.reddit.com/r/gadgets/" }
    ],
    "maxPostsCount": 50,
    "crawlCommentsPerPost": true,
    "maxCommentsPerPost": 15,
    "proxyConfiguration": { "useApifyProxy": false }
  }'

Fetch the results once the run finishes:

$curl "https://api.apify.com/v2/datasets/YOUR_DATASET_ID/items?token=YOUR_APIFY_TOKEN&format=csv"

Python client

from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")
run = client.actor("YOUR_ACTOR_ID").call(run_input={
    "searchTerms": ["mechanical keyboard"],
    "withinCommunity": "MechanicalKeyboards",
    "searchSort": "top",
    "searchTime": "month",
    "maxPostsCount": 40,
})

for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    if item["dataType"] == "post":
        print(item["score"], item["title"])

---

🛠️ Input parameters reference

Parameter	Type	Default	Description
startUrls	array	[]	Reddit URLs to scrape — posts, subreddits, user profiles, search pages. Bulk input supported.
searchTerms	array	[]	Keywords to search, one per entry — each gets its own budget.
searchPosts	boolean	true	Include posts in keyword search results.
searchComments	boolean	false	Include comments in keyword search results.
searchCommunities	boolean	false	Include communities in keyword search results.
withinCommunity	string	—	Keep every keyword search inside one subreddit.
searchSort	string	new	relevance, hot, top, new, comments.
searchTime	string	all	all, hour, day, week, month, year.
subredditUrls	array	[]	Subreddits to mine in full — name, r/name, or full URL.
onlyWithFlair	boolean	false	Keep only posts that carry a flair.
includeNSFW	boolean	false	Allow NSFW (18+) content.
postedAfter / postedBefore	string	—	Keep only posts created in this UTC window.
commentedAfter / commentedBefore	string	—	Keep only comments created in this UTC window.
crawlCommentsPerPost	boolean	false	Pull the comment thread for every post found.
maxCommentsPerPost	integer	10	Ceiling on comments per individual post.
maxPostsCount	integer	10	Max posts per source (term / URL / subreddit).
maxCommentsCount	integer	10	Max comments from keyword comment-search.
maxCommunitiesCount	integer	2	Max communities from keyword community-search.
proxyConfiguration	object	no proxy	Optional proxy configuration for the run.
maxRetries	integer	3	Attempts per request before it is skipped.
requestDelayMin	number	1.0	Shortest pause between requests (seconds).
requestDelayMax	number	2.5	Longest pause between requests (seconds).
postConcurrency	integer	3	Parallel comment-thread fetches.

---

🔌 Integrations: MCP, LangChain, n8n, warehouses

• 🧩 MCP & AI agents — every Apify Actor, this one included, can be exposed as a tool to Claude, Cursor, and other MCP clients via Apify's MCP server — no extra setup on your side.
• 🦜 LangChain — ApifyDatasetLoader maps body → page_content and the rest → metadata.
• 🦙 LlamaIndex — build a VectorStoreIndex over posts and comment threads.
• 🧠 OpenAI Embeddings & Assistants — chunk, embed, and upload as Assistant or Custom GPT knowledge.
• 🤖 Anthropic Claude — attach the Dataset JSON to Claude Projects or the Files API.
• 📍 Pinecone, Qdrant, Weaviate, Chroma, Milvus, pgvector — filter by community, date, or score at query time.
• 🔄 n8n, Make.com, Zapier — route new rows into Slack, Airtable, Sheets, Notion, or a CRM.
• 📦 Snowflake, BigQuery, PostgreSQL, MongoDB, S3 — bulk-load Dataset exports.
• ⏰ Schedules & webhooks — recurring runs plus a callback to your service when each finishes.

---

❓ Frequently asked questions (FAQ)

Do I need a Reddit API key, an app registration, or a login?

No — none of the three. The Actor reads public pages the same way a logged-out visitor does, so there is nothing to register, no token to rotate, and no per-app quota.

How is this different from using Reddit's official API?

The official API requires an OAuth app, enforces per-app rate limits, and restricts some endpoints. This Actor needs no credentials, accepts bulk URLs, resolves nested comments for you, and hands back a Dataset you can export or stream anywhere.

Can I paste a large list of URLs at once?

Yes — startUrls is designed for bulk input. Each URL is classified and budgeted independently, and one bad link never stops the run.

Can I mine a whole subreddit?

Yes — put it in subredditUrls and raise maxPostsCount. This goes far past what a single listing page returns.

How deep do comments go?

crawlCommentsPerPost expands each post's thread, including deeper "load more" replies, up to maxCommentsPerPost. Every comment carries its depth and postId, so you can rebuild the tree.

Can I search inside one subreddit only?

Yes — set withinCommunity and every keyword search stays inside that community.

Why did I get fewer rows than my limit?

Limits are ceilings, not quotas — the source may simply hold fewer matching items after your date, flair and NSFW filters. Widen searchTime or the date window to see more.

Do I need to configure a proxy?

No. Leave the proxy input empty and the run handles blocking on its own. Set one only when you want a specific country or route.

Can I use this from an AI agent?

Yes — through Apify's MCP server this Actor becomes a callable tool in Claude, Cursor, and other MCP clients.

How do I export or schedule the results?

Export from the run's Output tab (JSON, JSONL, CSV, Excel, XML, RSS) or through the Dataset API. Use Apify Schedules for recurring runs.

What if I need a field that isn't here?

Email hello.dataminds@gmail.com — custom fields and private builds are available.

---

🛠️ Troubleshooting

Symptom	Likely cause	Fix
No records saved	No source provided, or everything filtered out	Supply at least one of startUrls, searchTerms, subredditUrls; widen the date window
Run slows down mid-way	The target started rate-limiting the run	Expected — lower postConcurrency if it repeats
Comments missing	crawlCommentsPerPost off, or cap too low	Enable it and raise maxCommentsPerPost
Keyword search returns little	Narrow searchTime, or the term is rare	Widen searchTime, drop withinCommunity, or switch searchSort to relevance
searchComments is slow / thin	Reddit's comment search itself is limited	Results are approximated by scanning comment-rich posts; prefer post search when possible
NSFW posts absent	includeNSFW defaults to off	Set includeNSFW: true
One URL produced nothing	Deleted, private, or quarantined page	Check the log line for that URL; the run continues regardless

---

💬 Help, support & custom builds

For custom solutions, private integrations, white-label scrapers, or implementation help:

📧 hello.dataminds@gmail.com

• 🐛 Found a bug or need a new field? Open an issue on the Actor's Issues tab.
• ⭐ Enjoying it? Leave a review — it genuinely helps.

Useful Apify documentation:

• 📘 Apify documentation
• 🤖 Actors overview
• 🔌 Apify API reference
• 📚 Apify Academy — web scraping
• 🧰 Apify SDKs

---

⚖️ Is web scraping legal?

This Actor reads publicly available Reddit pages — posts, comments, communities and profiles that any logged-out visitor can see. No login, credentials, or private data are involved. Scraping public data is generally lawful, but the responsibility for use is yours:

• ✅ Never collect content behind authentication or paywalls.
• ✅ Respect Reddit's terms of service and sensible request rates.
• ✅ Treat usernames and their linked activity as personal data under GDPR, CCPA, and equivalents.
• ✅ Respect copyright when storing or republishing what you collect.

See Apify's guide on the legality of web scraping for background. This is not legal advice.

---

🔍 SEO keywords targeted

This README is intentionally keyword-rich for discovery on Google Search, Apify Store search, and Bing. Primary and secondary terms covered include:

Primary: reddit scraper · reddit scraper no api key · reddit scraper without login · reddit post scraper · reddit comment scraper · subreddit scraper · reddit search scraper · reddit crawler

Data types: reddit posts · nested comment threads · subreddit statistics · reddit user profiles · karma breakdown · community subscribers · post flair · upvote ratio · engagement velocity

AI & LLM: reddit data for AI agents · reddit mcp integration · LLM training corpus · RAG data source · fine-tuning dataset · vector database ingestion · embedding pipeline · LangChain loader · LlamaIndex reader · Claude Projects knowledge

Use cases: social listening · brand monitoring · market research · competitor analysis · voice of customer · academic discourse dataset · content and SEO research · reddit lead generation tool

Capabilities: bulk reddit url scraper · full subreddit scrape · keyword search scraping · date range filter · nsfw filter · flair filter · anti-blocking · automatic block handling · live dataset streaming

Integrations: MCP server · n8n · Make.com · Zapier · Pinecone · Qdrant · Weaviate · Snowflake · BigQuery · Airtable · Google Sheets · Slack