# Lemmy Scraper - Posts, Comments, Communities & Search (`navy_currant/lemmy-fediverse-scraper`) Actor

Scrape Lemmy (any instance) via the official public REST API v3: a site-wide feed by sort/type, posts from named communities, recent comments, and full-text search over communities/posts/users/comments. Federated Reddit-alternative data, no credentials, no proxy required.

- **URL**: https://apify.com/navy\_currant/lemmy-fediverse-scraper.md
- **Developed by:** [Ishaan](https://apify.com/navy_currant) (community)
- **Categories:** Social media
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $5.00 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Lemmy Scraper — Posts, Comments, Communities & Search

Scrape **Lemmy** (any instance) using the official **public REST API v3** — no account, no API key, no proxy spend required. Lemmy is the federated, open-source **Reddit alternative**; point this actor at `lemmy.world` (default, largest) or any other instance.

### What it does

- **Feed** — a site-wide post feed by sort (`Active`/`Hot`/`New`/`TopDay`…) and listing type (`All` federated feed or `Local` only)
- **Community** — pull posts from any named community, e.g. `technology` or `technology@lemmy.world`, across many communities in one run
- **Comments** — recent comments site-wide or scoped to specific communities
- **Search** — find **communities**, **posts**, **users**, or **comments** by keyword (or `All` at once)
- **Unified output** — posts, comments, communities and users in one dataset with consistent field names, ready for analysis or AI pipelines

### Why use this actor

| Competitor actor | Problem | This actor |
|---|---|---|
| Single-community Lemmy scrapers | **One community per run** — instance-wide needs many runs | Site-wide **feed** + multi-community + comments + search in one |
| Post-only scrapers | No comments, no community/user discovery | Posts **+** comments **+** communities **+** users |
| Roll-your-own API calls | Cursor vs page pagination, view-object flattening, charge logic to maintain | One input, clean flat schema, pay per result |

The Fediverse is a fast-growing, **open** social network. Lemmy in particular is the leading federated **Reddit alternative** — a natural input for **social listening**, brand monitoring, academic research, and **RAG pipelines** that want public discourse without platform lock-in or API keys.

### Output fields

**Posts** (`type: "post"`): `id`, `ap_id`, `title`, `body`, `external_url`, `thumbnail_url`, `embed_title`, `embed_description`, `published`, `updated`, `nsfw`, `locked`, `featured_community`, `featured_local`, `language_id`, `score`, `upvotes`, `downvotes`, `comments_count`, `newest_comment_time`, `community_id`, `community_name`, `community_title`, `community_actor_id`, `creator_id`, `creator_name`, `creator_display_name`, `creator_actor_id`, `creator_instance_id`

**Comments** (`type: "comment"`): `id`, `ap_id`, `content`, `path`, `published`, `updated`, `distinguished`, `language_id`, `score`, `upvotes`, `downvotes`, `child_count`, `post_id`, `post_title`, `community_id`, `community_name`, `community_actor_id`, `creator_id`, `creator_name`, `creator_display_name`, `creator_actor_id`

**Communities** (`type: "community"`): `id`, `name`, `title`, `actor_id`, `description`, `published`, `nsfw`, `posting_restricted_to_mods`, `instance_id`, `icon`, `banner`, `subscribers`, `posts_count`, `comments_count`, `users_active_day`, `users_active_week`, `users_active_month`, `users_active_half_year`

**Users** (`type: "user"`): `id`, `name`, `display_name`, `actor_id`, `bio`, `published`, `banned`, `bot_account`, `instance_id`, `avatar`, `banner`, `matrix_user_id`, `post_count`, `comment_count`

### Usage examples

Site-wide hot feed:

```json
{ "mode": "feed", "instance": "lemmy.world", "postSort": "Hot", "listingType": "All", "maxResultsPerTerm": 500 }
```

Posts from two communities:

```json
{
  "mode": "community",
  "instance": "lemmy.world",
  "communities": ["technology", "selfhosted@lemmy.world"],
  "postSort": "TopWeek",
  "maxResultsPerTerm": 300
}
```

Recent comments in a community:

```json
{ "mode": "comments", "communities": ["technology"], "commentSort": "New", "maxResultsPerTerm": 200 }
```

Find communities about a topic:

```json
{ "mode": "search", "searchTerms": ["selfhosted", "privacy"], "searchType": "Communities" }
```

### Pricing

**Pay per event:** $0.003 per result.

Example: 5,000 items ≈ **$15**.

### Technical notes

- Uses each instance's official Lemmy REST API v3 (`/post/list`, `/comment/list`, `/search`) — free, public, no key
- **Zero proxy cost** — official API, no anti-bot bypass
- Handles both pagination models for you: `/post/list` uses **opaque cursor** pagination (`next_page` → `page_cursor`), while `/comment/list` and `/search` use classic 1-indexed `page`
- Polite rate limiting between pages
- Automatically stops when the user's spending limit is reached — charge respected per result, no overrun

#### Honest scope / limits

- **Feed / community post ordering** honors Lemmy's server-side `sort`; `Top*` windows are relative to run time (snapshot, not live).
- **Comments mode** returns comments as a flat list with a `path` field (materialized tree position) — reconstruct threads client-side from `path` if needed.
- **Search** depends on what the chosen instance has federated and indexed; a term may return fewer rows on a small instance than on `lemmy.world`.
- Snapshot at run time; schedule recurring runs for live updates.

### Common use cases

- **Social listening / brand monitoring** — track a community or keyword across the Fediverse
- **RAG pipelines** — feed fresh, open public discourse into retrieval systems
- **Community discovery** — find and rank Lemmy communities by activity/subscribers for a topic
- **Academic / network research** — collect public posts, comments and community metadata at scale

### Data compliance

This actor accesses only **public** Lemmy data through each instance's official, publicly documented REST API. No login credentials are used. Public posts, comments and communities are intentionally public; suitable for research and business use. Respect each instance's terms and the authors' content.

# Actor input Schema

## `mode` (type: `string`):

What to scrape. feed = site-wide posts by sort/type; community = posts from named communities; comments = recent comments (site-wide or per community); search = find communities/posts/users/comments by keyword.

## `instance` (type: `string`):

Instance host to query (Lemmy is federated — no global API). Default lemmy.world (largest). Any Lemmy instance exposes the same /api/v3 REST API.

## `communities` (type: `array`):

Community names to pull posts for, e.g. 'technology' (local) or 'technology@lemmy.world' (federated). Each runs as a separate query.

## `searchTerms` (type: `array`):

Keywords to search. Each term runs as a separate query.

## `postSort` (type: `string`):

How to order posts.

## `listingType` (type: `string`):

All = the whole federated feed the instance sees; Local = only communities hosted on this instance.

## `commentSort` (type: `string`):

How to order comments.

## `searchType` (type: `string`):

What to search for.

## `maxResultsPerTerm` (type: `integer`):

Maximum results per community / search term (feed mode is a single query). Higher values increase cost and run time.

## Actor input object example

```json
{
  "mode": "feed",
  "instance": "lemmy.world",
  "communities": [
    "technology"
  ],
  "searchTerms": [
    "selfhosted"
  ],
  "postSort": "Active",
  "listingType": "All",
  "commentSort": "New",
  "searchType": "Communities",
  "maxResultsPerTerm": 200
}
```

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "instance": "lemmy.world",
    "communities": [
        "technology"
    ],
    "searchTerms": [
        "selfhosted"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("navy_currant/lemmy-fediverse-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "instance": "lemmy.world",
    "communities": ["technology"],
    "searchTerms": ["selfhosted"],
}

# Run the Actor and wait for it to finish
run = client.actor("navy_currant/lemmy-fediverse-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "instance": "lemmy.world",
  "communities": [
    "technology"
  ],
  "searchTerms": [
    "selfhosted"
  ]
}' |
apify call navy_currant/lemmy-fediverse-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "command": "npx",
            "args": [
                "mcp-remote",
                "https://mcp.apify.com/?tools=navy_currant/lemmy-fediverse-scraper",
                "--header",
                "Authorization: Bearer <YOUR_API_TOKEN>"
            ]
        }
    }
}

```

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/n72ZphDhbCk9RBx3y/builds/0y6CeX2fMRz9TqDOO/openapi.json