URL to Clean Markdown for LLMs & RAG

Turn a list of URLs into clean Markdown, ready for RAG ingestion and LLM context. Give it the pages you already have; it fetches each one, strips the nav, ads and boilerplate, and returns the main content as Markdown — plus title, byline, date and word count.

This is the "give me the readable text of this page" job, in bulk. Not a crawler, not a search scraper: you supply the exact URLs, and you pay only per page it actually extracts.

Why this and not a crawler

	This actor	Site crawlers / search scrapers
Input	The exact URLs you already have	A start URL or a search query
Work	Fetch + clean-extract each page	Discover, crawl, paginate, render
Cost	Per page extracted ($0.003)	Per page crawled, browser time, proxies
Best for	RAG ingestion of a known URL set	Mapping an unknown site

If you already know which pages you want in your vector store, crawling them is overkill. This does exactly the extraction step, cheaply and in parallel.

Input

field	meaning
URLs	The pages to convert. Paste a list, or wire this to another actor's output.
Keep hyperlinks	Keep in-text links in the Markdown. Turn off for the cleanest embedding text.
Only output pages that extracted	Off also writes a row for each failed URL with its error, so nothing is silently dropped.
Maximum URLs	Hard cap on how many are processed.
Per-URL timeout / Parallel fetches	Tune speed vs. politeness on slow sites.

Output

One dataset item per URL:

{
  "url": "https://example.com/article",
  "final_url": "https://example.com/article",
  "http_status": 200,
  "ok": true,
  "title": "Article title",
  "byline": "Jane Doe",
  "date": "2026-05-01",
  "markdown": "# Article title\n\nMain content…",
  "word_count": 1234,
  "error": null
}

A URL that fails to fetch or has no extractable main content gets ok: false with the reason in error — and is not charged.

Pricing

Pay-per-event: $0.003 per page successfully extracted. Failed fetches and pages with no main content cost nothing. Duplicate URLs in your input are de-duplicated, so the same page is never charged twice in a run.

Use cases

• RAG ingestion — convert a known list of documentation, blog or news URLs into clean Markdown chunks for a vector store.
• LLM context building — feed an agent the readable text of specific pages instead of raw HTML, cutting token cost and prompt noise.
• Dataset construction — bulk HTML-to-Markdown conversion for fine-tuning or evaluation corpora, with per-page metadata.
• Content monitoring — re-extract the same URL set on a schedule and diff the Markdown.

FAQ

Does it crawl or follow links? No. It converts exactly the URLs you supply. That is the point: no crawl budget, no surprise pages, no charge for pages you did not ask for.

How is this different from a local HTML-to-Markdown library? A library converts HTML you already fetched. This fetches at scale in parallel, applies main-content extraction (Readability-style boilerplate removal), and returns structured metadata — with failures reported instead of silently producing junk text.

What happens to pages that need JavaScript? They return ok: false with the reason, and are not charged. No browser is rendered, which is what keeps the price at a third of a cent per page.

Can I chain it to another Actor? Yes — wire any Actor's dataset of URLs into the URLs input and this becomes the extraction stage of your pipeline.

Is robots.txt respected? Fetches are plain HTTP GETs of URLs you supply; you remain responsible for having the right to retrieve them.

How it works

Extraction is trafilatura — a well-tested content extractor — the same engine behind our live clean-read micro-service. It reads the fetched HTML and returns the main article; it does not execute page scripts or render a browser, which is what keeps it fast and cheap. Pages that hard-require JavaScript to render their content, or that block datacenter traffic, will come back as ok: false rather than as partial text.