# Smart Article Extractor - Clean Text & Markdown from Any URL (`ninhothedev/smart-article-extractor`) Actor

$1/1K 🔥 Smart article extractor! Clean title, author, date, text & Markdown from any article URL — perfect for RAG & LLMs. No key. JSON, CSV, Excel or API in seconds. Feed news & blogs into AI in seconds ⚡

- **URL**: https://apify.com/ninhothedev/smart-article-extractor.md
- **Developed by:** [ninhothedev](https://apify.com/ninhothedev) (community)
- **Categories:** AI, News, Developer tools
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $1.00 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Smart Article Extractor 📰

**Turn any article URL into clean title, author, date, text and Markdown — no API key, no login, no config.**

Point it at any news story, blog post, or documentation page and get back a tidy, structured object ready for **RAG pipelines, LLM ingestion, news monitoring, and datasets**. Built on the battle-tested [`trafilatura`](https://trafilatura.readthedocs.io/) engine with a readability fallback, so it works across thousands of sites out of the box.

- ✅ **No key required** — just paste URLs and run
- ✅ **Clean text + Markdown** — perfect for feeding LLMs
- ✅ **Rich metadata** — author, publish date (ISO), description, site name, tags, main image, language
- ✅ **Word count + reading time** — built in
- ✅ **Robust** — Chrome-124 fetch with redirect follow, `urllib` fallback, and a BeautifulSoup readability fallback when needed
- ✅ **Cheap** — around **$1 per 1,000 articles**

***

### What it does

For every article URL you provide, the actor:

1. Fetches the page HTML (curl\_cffi with a real Chrome fingerprint, following redirects; `urllib` fallback).
2. Extracts the **main article content** with `trafilatura` — stripping nav, ads, comments, and boilerplate.
3. Falls back to a readability heuristic (largest `<article>`/`<main>`/`<p>`-dense block) if needed.
4. Returns one clean structured item per article.

### Use cases

- **RAG / LLM ingestion** — feed clean Markdown into vector stores and prompts without HTML noise.
- **News monitoring** — track coverage across outlets and pull the full body text.
- **Content archiving** — snapshot articles as durable, structured records.
- **Datasets** — build training/eval corpora with consistent fields.

### Input

| Field | Type | Default | Description |
|-------|------|---------|-------------|
| `mode` | select | `extract` | Operation mode (currently `extract`). |
| `urls` | array | — | Article URLs to extract. |
| `includeMarkdown` | boolean | `true` | Also return the body as clean Markdown. |
| `maxItems` | integer | `100` | Max articles per run (1–1000). |

Example:

```json
{
  "mode": "extract",
  "urls": [
    "https://www.bbc.com/news/articles/cewrx0ldlkko",
    "https://apnews.com/article/some-story"
  ],
  "includeMarkdown": true,
  "maxItems": 100
}
```

### Output

Each dataset item (all fields nullable):

```json
{
  "url": "https://www.bbc.com/news/articles/cewrx0ldlkko",
  "final_url": "https://www.bbc.com/news/articles/cewrx0ldlkko",
  "title": "BBC visits Iran school where strike killed 120 children",
  "author": "Nawal Al-Maghafi",
  "date": "2026-07-23T00:00:00",
  "description": "Nearly five months on, grief is still raw...",
  "text": "Clean article body text...",
  "markdown": "# Heading\n\nClean article body in Markdown...",
  "language": "en",
  "sitename": "BBC News",
  "tags": ["iran", "middle east"],
  "image": "https://ichef.bbci.co.uk/news/1024/.../image.png",
  "word_count": 1772,
  "reading_time_min": 9,
  "source": "article-extractor",
  "scraped_at": "2026-07-23T13:02:39.924934+00:00"
}
```

`text` and `markdown` are capped at 50,000 characters. URLs that fail or contain no readable article are skipped with a warning.

### Pricing

Pay-as-you-go on Apify platform usage — roughly **$1 per 1,000 articles**. No subscription, no key.

### Related actors

- [Website Content Crawler](https://apify.com/ninhothedev/website-content-crawler) — crawl an entire site and extract content.
- [HTML to Markdown Converter](https://apify.com/ninhothedev/html-to-markdown-converter) — convert raw HTML to Markdown.
- [RSS Feed Finder](https://apify.com/ninhothedev/rss-feed-finder) — discover feeds for any site.
- [PDF Text Extractor](https://apify.com/ninhothedev/pdf-text-extractor) — pull clean text from PDFs.

### Keywords

article extractor, web scraper, clean text, markdown, RAG, LLM, AI content feed, news scraper, readability, trafilatura, boilerplate removal, content extraction, article parser, text extraction, dataset builder, no API key.

***

Built by [ninhothedev](https://apify.com/ninhothedev).

# Actor input Schema

## `mode` (type: `string`):

Operation mode. 'extract' fetches each URL and returns a clean article object (title, author, date, text, markdown).

## `urls` (type: `array`):

List of article URLs to extract. Each URL is fetched and its main content is cleaned. Non-article pages are skipped.

## `includeMarkdown` (type: `boolean`):

If true, also return the article body as clean Markdown (great for LLM/RAG ingestion). Disable to shrink output.

## `maxItems` (type: `integer`):

Maximum number of articles to extract in one run (1-1000). Extra URLs are ignored.

## Actor input object example

```json
{
  "mode": "extract",
  "urls": [
    "https://apnews.com",
    "https://www.bbc.com/news"
  ],
  "includeMarkdown": true,
  "maxItems": 100
}
```

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "urls": [
        "https://apnews.com",
        "https://www.bbc.com/news"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("ninhothedev/smart-article-extractor").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "urls": [
        "https://apnews.com",
        "https://www.bbc.com/news",
    ] }

# Run the Actor and wait for it to finish
run = client.actor("ninhothedev/smart-article-extractor").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "urls": [
    "https://apnews.com",
    "https://www.bbc.com/news"
  ]
}' |
apify call ninhothedev/smart-article-extractor --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "command": "npx",
            "args": [
                "mcp-remote",
                "https://mcp.apify.com/?tools=ninhothedev/smart-article-extractor",
                "--header",
                "Authorization: Bearer <YOUR_API_TOKEN>"
            ]
        }
    }
}

```

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/acts/e5anyHuqPf7x0EkIv/builds/5RnhjDGqpW5M9dI0w/openapi.json