llms.txt Checker, Generator & Monitor: AI Search Readiness.

Audit, generate and monitor your llms.txt in one Actor, so ChatGPT and other AI assistants can read your site and keep reading it. Score what you have today, draft a publishable file from your own pages, or re-check on a schedule and get told only what changed.

1. Why use llms.txt Suite

AI assistants increasingly answer questions about products by reading websites directly, and llms.txt is how a site tells them which pages matter. Getting that right is three separate jobs: find out where you stand, produce the file, and notice when it drifts.

Doing them with three different tools means three formats, three sets of results and no continuity between them. The audit that says "no llms.txt" hands you the writing work; the file you write has no baseline to be measured against later; and nobody notices six months on when a site restructure leaves half the listed URLs pointing at 404s.

This Actor covers all three modes with one input shape and one output contract, so the score you generate against is the score you are later monitored on.

2. Key features

• Three modes in one Actor: audit scores what exists, generate drafts a new file, monitor diffs against a stored baseline.
• 0–100 readiness score with named checks — each returns its own pass/fail and a detail line.
• Concrete recommendations, not just a number.
• Missing-section detection against the llms.txt structure.
• Link validation. audit and monitor can fetch a sample of the links your llms.txt lists and report the dead ones.
• robots.txt respected by default in every mode.
• Optional llms-full.txt generation alongside the index file.
• Change detection with status, distinguishing missing, broken, stale and changed from ok.
• Charged per completed mode, and never on a failed run.

3. Who it's for

For marketing and SEO. Audit your domain, generate the file the audit says is missing, then monitor monthly — one tool covering the whole loop rather than a one-off score you never revisit.

For documentation teams. Generate from your docs sitemap so assistants answering user questions land on current pages, then monitor after each restructure to catch links that have gone stale.

For agencies. Audit a client portfolio, generate drafts for the ones scoring badly, and use monitor to show measurable improvement at the next review.

For developers and AI agents. Run monitor in a scheduled job and alert only when changed is true, so a site restructure that breaks your llms.txt surfaces immediately rather than at the next manual audit.

4. How to use it

1. Open the Actor and click Try for free.
2. Choose a Mode: audit to score, generate to draft, monitor to compare against the last check.
3. Put your site in Site URL — a bare domain or any page on it.
4. Adjust Max pages if generating for a large documentation site.
5. Click Start and read the result in the Output tab.

5. Input parameters

Field	Type	Required	Default	Description
mode	string	Yes	audit	audit, generate or monitor.
siteUrl	string	Yes	https://docs.stripe.com	The site to process. A bare domain or any page on it.
maxPages	integer	No	50	How many pages generate reads, discovered via sitemap.xml first.
includeFullText	boolean	No	false	When true, generate also assembles llms-full.txt with the same pages' body text.
validateLinks	boolean	No	true	When true, audit and monitor fetch a sample of the links listed in llms.txt to find dead ones.
respectRobots	boolean	No	true	When true, every mode honours the site's robots.txt.

6. Output

The shape depends on the mode. Every result also carries ms, the run duration.

audit

{
  "mode": "audit",
  "siteUrl": "https://docs.stripe.com",
  "hasLlmsTxt": true,
  "score": 78,
  "checks": [
    { "name": "llms.txt present", "passed": true, "detail": "Found at /llms.txt" },
    { "name": "Links resolve", "passed": false, "detail": "2 of 20 sampled links returned 404" }
  ],
  "missingSections": ["Optional"],
  "recommendations": ["Fix or remove the 2 dead links listed in checks."],
  "ms": 3120
}

generate

{
  "mode": "generate",
  "siteUrl": "https://example.com",
  "llmsTxt": "# Example\n\n> Summary...\n\n## Docs\n\n- [Getting started](https://example.com/docs)\n",
  "pagesIncluded": ["https://example.com/docs", "https://example.com/pricing"],
  "fileUrl": "https://api.apify.com/v2/key-value-stores/<storeId>/records/llms.txt",
  "ms": 8450
}

monitor

{
  "mode": "monitor",
  "siteUrl": "https://example.com",
  "previousScore": 78,
  "currentScore": 64,
  "changed": true,
  "diff": ["Score fell 78 -> 64", "3 links now return 404"],
  "status": "broken",
  "ms": 2980
}

A run that could not complete returns the same shape with an error field and is not charged.

7. Output fields

Field	Mode	Meaning
mode	all	Which mode produced this result.
siteUrl	all	The site processed.
ms	all	Run duration in milliseconds.
error	all	Present only on failure. The run is not charged when it appears.
hasLlmsTxt	audit	Whether a file was found at all.
score	audit	0–100 readiness score.
checks[]	audit	Each named check with passed and a detail line.
missingSections	audit	Expected sections absent from the file.
recommendations	audit	Concrete actions to raise the score.
llmsTxt	generate	The generated file, complete and ready to publish.
llmsFullTxt	generate	The full-text variant, when includeFullText was true.
pagesIncluded	generate	The page URLs that made it into the file.
fileUrl / fileUrlFull	generate	Direct download links to the generated file(s).
previousScore	monitor	The stored baseline score, or null on a first run.
currentScore	monitor	The score right now.
changed	monitor	Whether anything material moved since the baseline.
diff[]	monitor	Human-readable list of what changed.
status	monitor	ok, missing, broken, stale or changed.

8. How it works

Audit fetches the site's llms.txt, parses it structurally, and runs a set of named checks — presence, structure, sections, and optionally whether the listed links still resolve. Each check contributes to the score and returns its own detail line, so the number is always explainable.

Generate discovers pages via sitemap.xml first, falling back to link-following, and reads each page's own title and description. Nothing is invented: the file is assembled from what the site already publishes. The result is written to the run's key-value store and returned both inline and as a download URL.

Monitor re-audits and compares against a stored baseline for that site URL — a score and a content hash, nothing more. The internal hash is stripped from the output before it reaches your dataset.

Charging is per mode and only on success. A run that fails writes its result with an error and is not billed.

9. API & MCP usage

cURL

curl -X POST "https://api.apify.com/v2/acts/apifmcpfactory~llms-txt-suite/run-sync-get-dataset-items?token=YOUR_APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"mode":"audit","siteUrl":"https://docs.stripe.com"}'

As an Apify MCP tool. This Actor is callable directly by AI agents such as Claude and Cursor. Usage bills through your own Apify account.

https://mcp.apify.com?tools=apifmcpfactory/llms-txt-suite

Claude Desktop (claude_desktop_config.json):

{
    "mcpServers": {
        "llms-txt-suite": {
            "command": "npx",
            "args": [
                "mcp-remote",
                "https://mcp.apify.com?tools=apifmcpfactory/llms-txt-suite",
                "--header",
                "Authorization: Bearer YOUR_APIFY_TOKEN"
            ]
        }
    }
}

As a standalone MCP server. Auditing and generation also run as a dedicated hosted MCP server, with audit_llms_txt, generate_llms_txt and bulk_audit_llms_txt for up to 50 sites. See the product page for that endpoint.

10. Pricing

Priced per completed mode, because the three do very different amounts of work:

Mode	Price	What it covers
audit	$0.01	One completed AI-readiness audit of one site.
generate	$0.03	One completed llms.txt draft of one site.
monitor	$0.005	One completed check against the stored baseline.

Plus a small per-run start fee. A run that fails is not charged — no usable result, no charge.

11. Limits & performance

Memory	256 MB
Pages read by generate	50 by default
Link validation	A sample of listed links, not all of them
Baseline storage	Score and content hash, keyed by site URL

12. Limitations

• Readability, not ranking. A high score means assistants can read your site. It does not promise they will cite it.
• Generated files are drafts. Sections come from URL structure and summaries from your own metadata; review before publishing.
• Link validation samples. It checks a sample of listed links, so a clean result is strong evidence rather than proof that every link resolves.
• Monitor needs a baseline. The first run on a site has nothing to compare against and returns previousScore: null.
• Scoring is our published model, not an industry standard. The checks are visible in every result so you can judge them.
• JavaScript-rendered content is not executed. Titles and descriptions must be in the served HTML.
• Public pages only. Nothing behind a login or paywall is fetched.

13. FAQ

What is llms.txt? A plain-text file at your domain root listing your key pages for AI assistants, in the way robots.txt speaks to search crawlers.

Which mode should I start with? audit. It tells you whether you need generate at all.

How is the score calculated? From named checks returned in the checks array of every audit, each with its own pass/fail and detail. The arithmetic is visible in the result.

Does generate use AI to write the file? No. Titles and summaries come from your own pages' HTML.

How do I publish the generated file? Copy the llmsTxt value, or download it from fileUrl, and serve it at /llms.txt on your domain.

What does monitor compare against? A stored baseline for that site URL — a score and a content hash, both derived from your own public llms.txt.

What do the monitor statuses mean? ok — nothing material changed. missing — the file has gone. broken — listed links no longer resolve. stale — the file no longer reflects the site. changed — it moved, without a more specific diagnosis.

Am I charged if a run fails? No. Charging happens only on a completed mode.

Compliance

Public pages only — this Actor never accesses anything behind a login or paywall. respectRobots (on by default) makes every mode honour a site's robots.txt. Nothing personal is stored: the only state kept between runs is the Monitor baseline (a score and a content hash, keyed by site URL), derived entirely from the site's own public llms.txt.

More tools from MCP Factory

• Website Technology Detector — detect any site's CMS, framework and analytics stack.
• PDF Toolkit — merge, split, compress and convert PDFs.
• EU VAT Validator — validate EU VAT numbers against the official VIES registry.

— A Howth Technology Factory tool. Public sources, nothing personal stored.