# Website to Markdown Scraper — LLM & RAG Ready (`perforated_hummingbird/url-to-markdown`) Actor

Scrape any website into clean, LLM-ready Markdown. This URL-to-Markdown converter strips ads, nav, and boilerplate with Mozilla Readability — feed your AI models and RAG pipelines only real content. Batch URLs, optional JavaScript rendering, pay only for pages scraped successfully.

- **URL**: https://apify.com/perforated\_hummingbird/url-to-markdown.md
- **Developed by:** [Damon Williams](https://apify.com/perforated_hummingbird) (community)
- **Categories:** Developer tools, AI
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $10.00 / 1,000 page scrapeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Website to Markdown Scraper — Clean, LLM-Ready Text

**Scrape any website into clean Markdown.** Give this URL-to-Markdown converter a
web page address and get back just the content — no ads, no navigation bars, no
cookie banners, no scripts. It's the fastest way to turn web pages into
**LLM-ready Markdown** for RAG pipelines, AI agents, and knowledge bases.

Built on the principle that junk in the input means junk in the output: if you're
feeding web content to an AI model, you want the article, not the page furniture.

***

### Why use this website-to-Markdown scraper

Most web scrapers hand you raw HTML and leave the cleanup to you. This one does
the cleanup for you and returns proper Markdown:

- **Readable Markdown, not HTML soup.** Uses Mozilla's Readability engine — the
  same technology behind Firefox's Reader View — to find the actual article and
  discard the rest.
- **Proper GitHub-flavoured Markdown.** Headings, lists, tables, code blocks, and
  links are all preserved and ready to paste straight into a prompt.
- **Handles modern JavaScript sites.** Flip on **Render JavaScript** and each page
  loads in a real browser first, so content built by JavaScript is captured too.
- **Batch URL scraping.** Pass a list of URLs and get one clean Markdown record
  per page in a single run.
- **You only pay for what works.** Failed pages are recorded so you can see what
  happened — and you are **never charged** for them.

### What you can use it for

- Feeding articles and documentation into **LLMs, RAG pipelines, and AI agents**
- Building **research databases and knowledge bases** from web sources
- **Content archiving** in a durable, human-readable Markdown format
- Converting documentation sites into clean Markdown for **AI training or
  reference data**
- Powering **AI writing and summarization tools** that need clean source text

***

### Input

| Field | Type | Description |
| --- | --- | --- |
| `startUrls` | array | One or more page URLs to convert. Each successful page is charged once. |
| `url` | string | Shortcut for a single page. Can be used instead of, or alongside, `startUrls`. |
| `renderJs` | boolean | Load each page in a real browser so JavaScript-built content is captured. Slower. Default `false`. |
| `renderWaitMs` | integer | With `renderJs` on, wait this many extra milliseconds after load for late content. Default `0`. |
| `includeLinks` | boolean | Keep hyperlinks in the Markdown. Turn off for cleaner plain prose. Default `true`. |

**Example input**

```json
{
  "startUrls": [
    "https://en.wikipedia.org/wiki/Markdown",
    "https://example.com"
  ],
  "renderJs": false,
  "includeLinks": true
}
```

### Output

One clean Markdown record per page:

```json
{
  "url": "https://en.wikipedia.org/wiki/Markdown",
  "title": "Markdown",
  "excerpt": "Markdown is a lightweight markup language...",
  "siteName": "Wikimedia Foundation, Inc.",
  "wordCount": 3092,
  "markdown": "Markdown is a lightweight markup language for creating formatted text...",
  "scrapedAt": "2026-07-23T20:31:00.000Z"
}
```

If a page can't be scraped, you get a record with an `error` field explaining
why — and that page is **not** charged.

***

### Pricing

Pay per event — you are charged **once per page successfully scraped**. No
subscription, and **failed pages are never charged**.

| What you run | Charged as |
| --- | --- |
| Standard scrape (default) | Page scraped — the low rate |
| With **Render JavaScript** on | Page rendered — a higher rate |

Rendering runs a real browser and uses far more computing power, so it's priced
separately. If you don't turn it on, you never pay the rendering rate.

***

### When to turn on "Render JavaScript"

Leave it **off** by default — it's faster and works for most articles, blogs,
documentation, and news sites.

Turn it **on** when a page comes back nearly empty. That usually means the site
builds its content with JavaScript after loading. The difference on such a page
is dramatic:

| Mode | Result on a JavaScript-built page |
| --- | --- |
| Default (off) | ~3 words — just navigation links |
| Render JavaScript (on) | ~190 words — the full content |

***

### Frequently asked questions

**How do I convert a website to Markdown for an LLM?**
Paste one or more URLs into the input and run the Actor. You get back clean,
GitHub-flavoured Markdown for each page — ads, menus, and boilerplate removed —
ready to drop straight into a prompt, a RAG pipeline, or a vector database.

**How is this different from a normal HTML scraper?**
A normal scraper returns raw HTML and leaves you to strip out the noise. This
Actor runs Mozilla Readability to isolate the real article first, then converts
only that to Markdown — so you get readable text, not tag soup.

**Does it handle JavaScript-rendered pages?**
Yes. Turn on **Render JavaScript** and each page loads in a real browser before
extraction, so single-page apps and JavaScript-built content are captured.

**Can I scrape multiple URLs at once?**
Yes. Pass a list in `startUrls` and the Actor returns one Markdown record per
page in a single run. Batch scraping is fully supported.

**Do I pay for pages that fail?**
No. You are only charged for pages that are successfully scraped and converted.
Failed pages are returned with an `error` field and are never billed.

**What Markdown format does it output?**
GitHub-flavoured Markdown (GFM) — with headings, lists, tables, code blocks, and
(optionally) links preserved.

**Is it suitable for RAG pipelines and AI agents?**
Yes — that's exactly what it's built for. Clean Markdown means less noise in your
embeddings and better answers from your model.

***

### Good to know

- Only `http` and `https` pages are supported.
- Pages are capped at 5 MB and time out after 20 seconds (30 with rendering) to
  keep runs fast and predictable.
- Internal and private network addresses are refused for security.
- A small number of sites block automated traffic; that's a site policy, and
  those pages are reported as errors rather than charged.

# Actor input Schema

## `startUrls` (type: `array`):

One or more web page URLs to convert to Markdown. Each successfully scraped page is charged once.

## `url` (type: `string`):

Shortcut for scraping just one page. You can use this instead of, or in addition to, the list above.

## `renderJs` (type: `boolean`):

Load each page in a real browser so content built by JavaScript is captured. Slower, but works on modern app-style sites.

## `renderWaitMs` (type: `integer`):

When 'Render JavaScript' is on, wait this many extra milliseconds after the page loads so late content can appear.

## `includeLinks` (type: `boolean`):

If on, hyperlinks are kept in the Markdown. Turn off for cleaner plain prose.

## Actor input object example

```json
{
  "startUrls": [
    "https://en.wikipedia.org/wiki/Large_language_model",
    "https://en.wikipedia.org/wiki/Markdown"
  ],
  "renderJs": false,
  "renderWaitMs": 0,
  "includeLinks": true
}
```

# Actor output Schema

## `pages` (type: `string`):

Every scraped page with its clean Markdown and metadata, as JSON.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrls": [
        "https://en.wikipedia.org/wiki/Large_language_model",
        "https://en.wikipedia.org/wiki/Markdown"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("perforated_hummingbird/url-to-markdown").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "startUrls": [
        "https://en.wikipedia.org/wiki/Large_language_model",
        "https://en.wikipedia.org/wiki/Markdown",
    ] }

# Run the Actor and wait for it to finish
run = client.actor("perforated_hummingbird/url-to-markdown").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrls": [
    "https://en.wikipedia.org/wiki/Large_language_model",
    "https://en.wikipedia.org/wiki/Markdown"
  ]
}' |
apify call perforated_hummingbird/url-to-markdown --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "command": "npx",
            "args": [
                "mcp-remote",
                "https://mcp.apify.com/?tools=perforated_hummingbird/url-to-markdown",
                "--header",
                "Authorization: Bearer <YOUR_API_TOKEN>"
            ]
        }
    }
}

```

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/BRnbJQknIpTz9NzdD/builds/nDKtEl4Zm0hgThybh/openapi.json