# Beehiiv Newsletter Scraper (`khadinakbar/beehiiv-newsletter-scraper`) Actor

Scrape Beehiiv newsletters for posts, authors, newsletter metadata, and content discovery. MCP/API-ready.

- **URL**: https://apify.com/khadinakbar/beehiiv-newsletter-scraper.md
- **Developed by:** [Khadin Akbar](https://apify.com/khadinakbar) (community)
- **Categories:** Lead generation, MCP servers, AI
- **Stats:** 1 total users, 0 monthly users, 0.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $2.00 / 1,000 newsletter record scrapeds

This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Beehiiv Newsletter Scraper

Beehiiv Newsletter Scraper is an Apify Actor for public Beehiiv newsletters. It accepts Beehiiv publication homepage URLs and direct post URLs, then returns one dataset record per newsletter publication or post, depending on the selected scrape mode. Each record can include publication metadata, author details, social links, post titles, dates, previews, premium status, read time, tags, and full post content when requested. The result is a structured dataset that can be used in Apify runs, Apify API workflows, or through Apify MCP.

### Best fit and connected workflows

Use this Actor when your starting point is a Beehiiv newsletter homepage or a list of Beehiiv post URLs and you want structured records for discovery, research, or downstream analysis.

It fits well when you need:

- publication metadata for a Beehiiv newsletter directory
- a post list with titles, dates, and previews from a newsletter homepage
- full post text for selected public posts
- a dataset that distinguishes newsletter records from post records with `record_type`
- Apify Actor output that can be consumed through Apify MCP or the Apify API

Beehiiv Newsletter Scraper is designed as a focused standalone workflow.

### Practical scenario

Maya is building a research sheet for a Beehiiv newsletter she follows. She starts with the publication homepage URL and chooses `metadata_posts`, which returns the newsletter record plus a list of post records. From the dataset, she reads `publication_name`, `author_name`, `post_count`, `post_title`, `post_date`, and `post_preview`. She uses `post_date` and `post_preview` to decide which posts are most relevant, then opens the matched `post_url` for deeper reading or reruns the Actor in `full` mode for selected posts.

### Input fields

| Field | Type | Purpose |
|---|---|---|
| `publicationUrls` | array | Beehiiv publication homepage URLs. Use this for newsletter metadata and, based on `scrapeMode`, post discovery and scraping. |
| `postUrls` | array | Direct Beehiiv post URLs. Use this for individual post scraping. |
| `scrapeMode` | string | Controls the depth of extraction: `metadata`, `metadata_posts`, or `full`. |
| `maxPostsPerPublication` | integer | Maximum number of posts to scrape per publication. |
| `maxResults` | integer | Maximum total number of dataset records across all inputs. |

#### Focused input example

```json
{
  "publicationUrls": [
    {
      "url": "https://aibreakfast.beehiiv.com"
    }
  ],
  "scrapeMode": "metadata_posts",
  "maxPostsPerPublication": 10,
  "maxResults": 20
}
```

### Output fields

| Field | Type | Meaning |
|---|---|---|
| `record_type` | string | Record discriminator: `newsletter` or `post`. |
| `publication_url` | string or null | Beehiiv publication homepage URL. |
| `publication_name` | string or null | Newsletter display name. |
| `description` | string or null | Newsletter tagline, short description, or post excerpt. |
| `author_name` | string or null | Newsletter author or creator name. |
| `author_bio` | string or null | Author bio when available. |
| `logo_url` | string or null | Publication logo image URL. |
| `cover_image_url` | string or null | Main OG cover image URL. |
| `website_url` | string or null | Canonical publication URL. |
| `social_links` | object | Social links found on the publication page. |
| `post_count` | integer or null | Number of post links found on the publication homepage. |
| `post_url` | string or null | Individual post URL. |
| `post_title` | string or null | Post title. |
| `post_subtitle` | string or null | Post subtitle or secondary heading. |
| `post_date` | string or null | ISO 8601 post publication timestamp. |
| `post_preview` | string or null | Short post preview sourced from the OG description meta tag. |
| `post_content` | string or null | Full plain-text post content, populated in `full` mode. |
| `post_slug` | string or null | Slug after `/p/`. |
| `is_premium` | boolean or null | Premium or paywalled status signal. |
| `read_time_minutes` | integer or null | Estimated reading time displayed on the post page. |
| `tags` | array | Post tags or topic categories. |
| `scraped_at` | string | ISO 8601 scrape timestamp. |
| `source_url` | string | URL fetched to produce the record. |

#### Illustrative output record

```json
{
  "record_type": "post",
  "publication_url": "https://aibreakfast.beehiiv.com",
  "publication_name": "AI Breakfast",
  "description": "A newsletter about AI developments.",
  "author_name": "Devansh Mehta",
  "author_bio": null,
  "logo_url": "https://media.beehiiv.com/cdn-cgi/image/logo.png",
  "cover_image_url": "https://media.beehiiv.com/cdn-cgi/image/cover.png",
  "website_url": "https://aibreakfast.beehiiv.com",
  "social_links": {
    "twitter": "https://twitter.com/aibreakfast"
  },
  "post_count": null,
  "post_url": "https://aibreakfast.beehiiv.com/p/gpt-5-is-here",
  "post_title": "GPT-5 Just Dropped - Here's What You Need to Know",
  "post_subtitle": "Plus: Claude 4 leaks and Google's Gemini 2.0 update",
  "post_date": "2024-05-15T08:00:00.000Z",
  "post_preview": "Today we dive into GPT-5...",
  "post_content": null,
  "post_slug": "gpt-5-is-here",
  "is_premium": false,
  "read_time_minutes": 5,
  "tags": [
    "AI",
    "GPT"
  ],
  "scraped_at": "2026-04-08T10:01:00.000Z",
  "source_url": "https://aibreakfast.beehiiv.com/p/gpt-5-is-here"
}
```

### How it works

The Actor is configured for two input paths:

- `publicationUrls` for Beehiiv newsletter homepages
- `postUrls` for direct Beehiiv post pages

It supports three scrape modes:

- `metadata` returns newsletter information only
- `metadata_posts` returns newsletter information plus a post list with dates and previews
- `full` returns newsletter information plus full plain-text post content

The Actor writes scraped records to the default dataset. The dataset schema distinguishes newsletter records from post records with `record_type`, and each record includes `scraped_at` and `source_url` for provenance.

### Pricing

This Actor uses pay per event pricing on the Apify platform, plus standard Apify platform usage. Review the live Pricing tab for the current pricing details.

Two event types are charged:

- Actor start
- Newsletter Record Scraped, charged per newsletter or post record scraped

A simple example in words: if a run starts once and scrapes ten records, the total includes one start event plus ten record events.

### Use with AI agents (MCP)

This Actor is available as an Apify Actor usable through Apify MCP. The Actor identity is `khadinakbar/beehiiv-newsletter-scraper`.

Tool description:

- scrape Beehiiv publication pages or direct post URLs and return structured newsletter metadata and post records for agent workflows

> Scrape the Beehiiv newsletter at https://aibreakfast.beehiiv.com in metadata\_posts mode, then return the publication name, author, post titles, post dates, and previews in a compact summary.

Output interpretation:

- Use `record_type` to separate publication metadata from individual posts.
- Use `publication_name` and `author_name` for newsletter identity.
- Use `post_title`, `post_date`, `post_preview`, and `post_url` to build reading or routing steps.
- Use `post_content` only when `scrapeMode` is `full`.
- Use `scraped_at` and `source_url` for provenance and traceability.

Scope and pagination:

- `publicationUrls` can trigger publication scraping and post discovery.
- `maxPostsPerPublication` controls how many posts are collected per newsletter.
- `maxResults` caps the total number of returned records across all inputs.

Cost guidance:

- `metadata` is the lightest extraction path.
- `metadata_posts` adds post discovery fields such as titles, dates, and previews.
- `full` returns the most detailed records and is suited to research, archiving, and model preparation.

### Apify API example

```javascript
import { ApifyClient } from "apify-client";

const client = new ApifyClient({
  token: process.env.APIFY_TOKEN
});

const run = await client.actor("khadinakbar/beehiiv-newsletter-scraper").call({
  publicationUrls: [
    { url: "https://aibreakfast.beehiiv.com" }
  ],
  scrapeMode: "metadata_posts",
  maxPostsPerPublication: 10,
  maxResults: 20
});

const datasetItems = await client.dataset(run.defaultDatasetId).listItems();
console.log(datasetItems.items);
```

### Best results and outcome guidance

- Start with `publicationUrls` when you want the newsletter homepage record and a discovered post list.
- Use `postUrls` when you already have the exact Beehiiv post pages you want.
- Choose `metadata_posts` for most workflows that need both identity fields and post discovery.
- Choose `full` when you need the plain-text body of public posts.
- Increase `maxPostsPerPublication` for deeper archive collection within a single publication.
- Use `maxResults` to keep the run focused when you are batching multiple newsletters.

### Focused standalone workflow

Beehiiv Newsletter Scraper is designed as a focused standalone workflow for the public input and structured output contract described above.

### Design note

I found that the dataset contract uses `record_type` as the main discriminator, with `newsletter` and `post` as the two visible record shapes, which makes the output straightforward to filter in downstream workflows.

### FAQ

#### Which input should I use for a newsletter homepage?

Use `publicationUrls` when you have a Beehiiv newsletter homepage URL and want publication metadata, post discovery, or post scraping from that homepage.

#### Which input should I use for a single article link?

Use `postUrls` when you already have a direct Beehiiv post URL and want a record for that specific post.

#### How do I get full article text?

Set `scrapeMode` to `full`. In `metadata` and `metadata_posts`, `post_content` stays empty while the Actor focuses on lighter extraction.

#### How does the Actor separate newsletter records from post records?

Use the `record_type` field. Newsletter metadata records use `newsletter`, and individual post records use `post`.

#### How can I limit run size across many newsletters?

Use `maxResults` to cap the total number of returned records, and combine it with `maxPostsPerPublication` for per-publication control.

#### Can this Actor be used in Apify MCP workflows?

Yes. It is an Apify Actor with MCP-ready output, so agents can call it and read the dataset records directly.

### Responsible use

Use this Actor on public Beehiiv pages and review your own data handling practices before storing or redistributing content. Respect publisher terms, content rights, privacy obligations, and any applicable laws when working with newsletter data.

# Actor input Schema

## `publicationUrls` (type: `array`):

Use this when you have Beehiiv newsletter homepage URLs (e.g. https://aibreakfast.beehiiv.com). The actor will scrape publication metadata and, based on scrapeMode, also enumerate and scrape posts. Use postUrls instead when you have direct post links.

## `postUrls` (type: `array`):

Use this when you have direct Beehiiv post URLs (e.g. https://aibreakfast.beehiiv.com/p/some-post-slug). Scrapes individual posts directly. Use publicationUrls instead to discover all posts from a newsletter homepage.

## `scrapeMode` (type: `string`):

Controls how much data to extract. 'metadata' = newsletter info only (fastest, cheapest). 'metadata\_posts' = newsletter info + full post list with dates and previews (recommended for most use cases). 'full' = everything including full post content (best for LLM training, research, archiving).

## `maxPostsPerPublication` (type: `integer`):

Maximum number of posts to scrape per newsletter publication. Lower values reduce cost and run time. Set to 500 for full archive scraping.

## `maxResults` (type: `integer`):

Maximum total number of records (newsletter + post records combined) to return across all input publications. The actor stops as soon as this limit is reached.

## Actor input object example

```json
{
  "publicationUrls": [
    {
      "url": "https://aibreakfast.beehiiv.com"
    }
  ],
  "scrapeMode": "metadata_posts",
  "maxPostsPerPublication": 10,
  "maxResults": 20
}
```

# Actor output Schema

## `results` (type: `string`):

Scraped newsletter and post records. Filter by record\_type: 'newsletter' or 'post'. Fields include publication\_name, publication\_url, post\_title, post\_url, post\_preview, author, post\_date, scraped\_at, and more.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "publicationUrls": [
        {
            "url": "https://aibreakfast.beehiiv.com"
        }
    ],
    "scrapeMode": "metadata_posts",
    "maxPostsPerPublication": 10,
    "maxResults": 20
};

// Run the Actor and wait for it to finish
const run = await client.actor("khadinakbar/beehiiv-newsletter-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "publicationUrls": [{ "url": "https://aibreakfast.beehiiv.com" }],
    "scrapeMode": "metadata_posts",
    "maxPostsPerPublication": 10,
    "maxResults": 20,
}

# Run the Actor and wait for it to finish
run = client.actor("khadinakbar/beehiiv-newsletter-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "publicationUrls": [
    {
      "url": "https://aibreakfast.beehiiv.com"
    }
  ],
  "scrapeMode": "metadata_posts",
  "maxPostsPerPublication": 10,
  "maxResults": 20
}' |
apify call khadinakbar/beehiiv-newsletter-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "command": "npx",
            "args": [
                "mcp-remote",
                "https://mcp.apify.com/?tools=khadinakbar/beehiiv-newsletter-scraper",
                "--header",
                "Authorization: Bearer <YOUR_API_TOKEN>"
            ]
        }
    }
}

```

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/acts/qK1jqOSUmQsIdvBit/builds/ZLfPLhDFiAxumT8YJ/openapi.json