Under maintenance

Pricing

from $1.50 / 1,000 results

Try for free

Go to Apify Store

Website Content RAG Cleaner

Under maintenance

Try for free

Extract public web pages into clean, structured content ready for retrieval, summaries, and AI workflows.

Pricing

from $1.50 / 1,000 results

Rating

0.0

(0)

Developer

Daniel Ferreira de Almeida

Actor stats

Bookmarked

Total users

Monthly active users

13 hours ago

Last modified

What it does

This Actor fetches one or more public URLs, or consumes provided HTML, and returns:

the resolved page title and final URL
cleaned plain text
markdown-style content
optional raw HTML
extracted links
chunked text for retrieval pipelines
basic run metadata

Best for

knowledge base ingestion
RAG pipelines
AI search indexes
support and sales enablement
document-style content extraction

Input

urls — public URLs to process
html — optional single HTML document for offline validation
htmlByUrl — optional offline URL→HTML map
includeRawHtml — include the original HTML in output
includeLinks — include extracted links
maxTextChars — cap on cleaned text length
chunkSize — chunk size for downstream processing

Example input

{
  "urls": ["https://example.com/article"],
  "html": "<!doctype html><html><head><title>Example</title></head><body><h1>Example</h1><p>Hello world.</p></body></html>",
  "includeRawHtml": false,
  "includeLinks": true,
  "maxTextChars": 5000,
  "chunkSize": 400
}

Output

Each item contains:

url
finalUrl
title
description
text
markdown
chunks
links
wordCount
status
checkedAt

Validation

The local smoke test uses a fixture HTML document so the Actor can be validated without network dependence.

Notes

Safe for platform testing with fixture HTML.
Designed for clean ingestion into downstream retrieval systems.

RAG Web Browser

scraper-engine/rag-web-browser

RAG Web Browser websites and extract relevant content for Retrieval-Augmented Generation (RAG). Collect clean text, metadata, links, and documents from web pages, then export structured data to JSON, CSV, Excel, or XML for AI assistants, knowledge bases, and semantic search.

Scraper Engine

Website RAG Knowledge Builder

glowing_glove/website-rag-knowledge-builder

Crawl public website pages and build clean RAG-ready knowledge records with page summaries, key facts, FAQs, links, and retrieval chunks.

Ushba Khan

Ai Content Scraper Cleaner

dashjeevanthedev/ai-content-scraper-cleaner

AI Content Scraper & Cleaner — Scrapes structured content (documentation, articles, FAQs, blog posts) and converts it into clean, normalized JSON datasets for LLM training. Extracts text, detects content types, estimates tokens, and removes boilerplate to produce ready-to-use training data.

JEEVAN JYOTI DASH

AI Web Reader (RAG Ready)

viinaysonii/ai-web-reader-rag-ready

Convert any webpage into clean, structured, AI-ready Markdown. Removes ads, images, and UI noise, normalizes content, and outputs data optimized for LLMs, RAG pipelines, and AI agents. Fast, scalable, and built for real-world AI workflows.

Web Content Scraper - Clean Text, Metadata & AI Data

unrivaled_fortress/web-content-intelligence

Scrape public webpages and extract clean text, metadata, headings, links and structured data ready for AI and LLM consumption. Web scraping for RAG pipelines and content monitoring. Pay per result. Try free — no API key, instant results.

David Ahn

AI Web Crawler

hounderd/ai-web-crawler

Crawl websites and extract clean, LLM-ready markdown content with stealth browser rendering, anti-bot hardening, smart content filtering, and structured metadata extraction. Built for RAG pipelines, AI agents, and data workflows.

Hounderd

AI Website Content Extractor

scrapeai/ai-website-content-extractor

Crawl website pages, strip noise, and convert the main content to clean Markdown for RAG/LLM training.

ScrapeAI

5.0

AI-Ready Webpage Extractor

s3nafps/ai-ready-webpage-extractor

Convert public web pages into clean Markdown, text, links, tables, images, metadata, and JSON-LD for AI workflows.

mohamed senator

YouTube Transcript Scraper

fetch_cat/youtube-transcript-scraper

Extract transcripts and timestamped caption segments from public YouTube videos. Export text for summaries, RAG, content research, and AI-agent workflows.