# LLMS.txt Auditor (`junipr/llms-txt-auditor`) Actor

Check for llms.txt / AI crawler policy guidance and produce structured readiness reports.

- **URL**: https://apify.com/junipr/llms-txt-auditor.md
- **Developed by:** [junipr](https://apify.com/junipr) (community)
- **Categories:** SEO tools
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $4.90 / 1,000 page auditeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## LLMS.txt Auditor

### Store Positioning

**Store title:** LLMS.txt Auditor

**Short description:** Check for llms.txt / AI crawler policy guidance and produce structured readiness reports.

**SEO title:** LLMS.txt Auditor — technical SEO, web, and domain audit

**SEO description:** Check for llms.txt / AI crawler policy guidance and produce structured readiness reports. Use it to find crawlability, indexability, security, metadata, and page-quality issues with evidence-backed rows and audit reports.

**Categories:** SEO\_TOOLS

**Keywords:** llms, txt, auditor, structured data, web/domain audit

### Fixed-Inclusive PPE Pricing

This actor uses pay-per-event pricing. Event prices include Apify platform usage; users are not expected to pay a separate platform-usage pass-through charge for the configured pricing model.

- Tier: W1 — Web/domain audit
- Primary event: `page-audited` at $0.00490 base
- Default max charge: $10.00
- Store discounts: FREE/BRONZE base, SILVER discounted, GOLD deepest approved discount

Event set:

- `actor-start`: base $0.00500, GOLD $0.00400. Llms Txt Auditor: charged when actor start is completed. The price includes Apify platform usage; no separate usage pass-through is intended.
- `page-audited`: base $0.00490, GOLD $0.00392. Llms Txt Auditor: charged when page audited is completed. The price includes Apify platform usage; no separate usage pass-through is intended.
- `record-extracted`: base $0.00372, GOLD $0.00298. Llms Txt Auditor: charged when record extracted is completed. The price includes Apify platform usage; no separate usage pass-through is intended.
- `finding-emitted`: base $0.00372, GOLD $0.00298. Llms Txt Auditor: charged when finding emitted is completed. The price includes Apify platform usage; no separate usage pass-through is intended.
- `audit-report-generated`: base $0.05000, GOLD $0.04000. Llms Txt Auditor: charged when audit report generated is completed. The price includes Apify platform usage; no separate usage pass-through is intended.

### Public Task Concepts

- Audit public URLs for LLMS.txt Auditor issues
- Find LLMS.txt Auditor problems across a website
- Generate a technical LLMS.txt Auditor report
- Audit llms issues across supplied public inputs
- Extract evidence-backed llms rows for client QA

### What It Does

Check for llms.txt / AI crawler policy guidance and produce structured readiness reports.

This local package is part of the Junipr Apify actors 171-200 premium build for ChatGPT review.

### What It Does Not Do

- It does not call live Apify APIs.
- It does not publish, upload Store assets, create public tasks, or configure live PPE.
- It does not claim publish readiness.
- It does not make unbounded network calls in the supplied examples path.

### Input Fields

Seed shape: inputItems, maxItems, includeReport

The tiny fixture is available at `examples/input.tiny.json` and mirrored at `fixtures/input.tiny.json` for the local runner.

### Output Fields

- `domain`
- `checkedUrl`
- `statusCode`
- `found`
- `contentType`
- `policySignals`
- `allowSignals`
- `disallowSignals`
- `issues`
- `readinessStatus`

### Public Task Examples

#### Audit llms.txt policy content for a website

- Search intent: audit llms.txt policy content for a website
- Specific input: Fixture input shaped for audit llms.txt policy content for a website.
- Expected output: Dataset rows with domain, checkedUrl, statusCode, found plus local KVS report evidence.
- Why run it: Publishers, SEO teams, legal/content teams, agencies would run this to get a bounded local proof before any live Apify review or production use.

#### Check AI crawler allow and disallow rules

- Search intent: check ai crawler allow and disallow rules
- Specific input: Fixture input shaped for check ai crawler allow and disallow rules.
- Expected output: Dataset rows with domain, checkedUrl, statusCode, found plus local KVS report evidence.
- Why run it: Publishers, SEO teams, legal/content teams, agencies would run this to get a bounded local proof before any live Apify review or production use.

#### Verify contact metadata in an llms.txt file

- Search intent: verify contact metadata in an llms.txt file
- Specific input: Fixture input shaped for verify contact metadata in an llms.txt file.
- Expected output: Dataset rows with domain, checkedUrl, statusCode, found plus local KVS report evidence.
- Why run it: Publishers, SEO teams, legal/content teams, agencies would run this to get a bounded local proof before any live Apify review or production use.

#### Prepare content-use policy rows for legal review

- Search intent: prepare content-use policy rows for legal review
- Specific input: Fixture input shaped for prepare content-use policy rows for legal review.
- Expected output: Dataset rows with domain, checkedUrl, statusCode, found plus local KVS report evidence.
- Why run it: Publishers, SEO teams, legal/content teams, agencies would run this to get a bounded local proof before any live Apify review or production use.

#### Review llms.txt readiness before public release

- Search intent: review llms.txt readiness before public release
- Specific input: Fixture input shaped for review llms.txt readiness before public release.
- Expected output: Dataset rows with domain, checkedUrl, statusCode, found plus local KVS report evidence.
- Why run it: Publishers, SEO teams, legal/content teams, agencies would run this to get a bounded local proof before any live Apify review or production use.

### Tests

- Actor-specific fixture tests live in `test/llms-txt-auditor.test.ts`.
- Node-based local suites validate dataset schema conformance, output samples, local smoke runs, and PPE guard behavior.

### FAQ

#### Is this live on Apify?

No. This is a local package for ChatGPT review only.

#### Is it publish-ready?

No. It needs later live tiny-run validation and console configuration before any publication decision.

# Actor input Schema

## `inputItems` (type: `array`):

Actor-specific records or URLs to process.

## `domains` (type: `array`):

Domain fixture records or domain strings.

## `urls` (type: `array`):

URL fixture records or URL strings.

## `maxItems` (type: `integer`):

Hard cap for processed items.

## `maxDomains` (type: `integer`):

Hard cap for processed domains.

## `timeoutMs` (type: `integer`):

Local/live request timeout cap in milliseconds.

## `includeReport` (type: `boolean`):

Write KVS Markdown and JSON report outputs.

## `includeSummary` (type: `boolean`):

Write summary KVS output.

## `strictMode` (type: `boolean`):

Use stricter validation thresholds.

## `fixtureMode` (type: `boolean`):

Use local deterministic fixtures. Live lookups remain off by default.

## `maxChargeUsd` (type: `number`):

Local PPE guard cap before paid rows stop gracefully.

## Actor input object example

```json
{
  "maxItems": 50,
  "maxDomains": 50,
  "timeoutMs": 5000,
  "includeReport": true,
  "includeSummary": true,
  "strictMode": false,
  "fixtureMode": true,
  "maxChargeUsd": 1
}
```

# Actor output Schema

## `dataset` (type: `string`):

Structured rows emitted by the actor.

## `report` (type: `string`):

Run summary report stored in the default key-value store.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {};

// Run the Actor and wait for it to finish
const run = await client.actor("junipr/llms-txt-auditor").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {}

# Run the Actor and wait for it to finish
run = client.actor("junipr/llms-txt-auditor").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{}' |
apify call junipr/llms-txt-auditor --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "command": "npx",
            "args": [
                "mcp-remote",
                "https://mcp.apify.com/?tools=junipr/llms-txt-auditor",
                "--header",
                "Authorization: Bearer <YOUR_API_TOKEN>"
            ]
        }
    }
}

```

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/acts/3evqKD26M1d3K19eO/builds/O61h8n4cXeAQcT6mY/openapi.json