# Structured Data Scraper & Validator (`taroyamada/structured-data-validator`) Actor Crawl websites to extract JSON-LD and Microdata, validate schema markup syntax, and flag missing fields across massive URL lists. - **URL**: https://apify.com/taroyamada/structured-data-validator.md - **Developed by:** [naoki anzai](https://apify.com/taroyamada) (community) - **Categories:** SEO tools, Developer tools - **Stats:** 7 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $9.00 / 1,000 results This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Structured Data Validator API | Schema Coverage & Errors ### After this run Turn this Actor's output into a capped paid report with [Website RAG Readiness Audit Report](https://apify.com/taroyamada/website-rag-readiness-audit?utm_source=apify_internal&utm_medium=readme_after_run&utm_campaign=v37_internal_flow&utm_content=structured-data-validator__website-rag-readiness-audit). Use it when AI builders, documentation teams, support teams, and technical marketers need to decide whether public website pages are clean and complete enough for RAG ingestion. - First report: $9 / `website_rag_snapshot_report`; set `maxChargeUsd` to $9. - Deeper report: $29 / `website_rag_readiness_report`; use only when the first result needs competitor or action-depth. - This is an internal Apify flow aid. It is not revenue proof until accounted paid usage appears. Crawling websites to extract structured data is essential for maintaining robust organic search visibility. Broken markup often goes unnoticed until traffic and rich snippets drop, costing you valuable clicks. This schema validation tool acts as a rigorous web scraper that parses every page to evaluate JSON-LD and Microdata quality at scale. Instead of manually checking individual URLs in external tools, you can schedule automated schema audits across thousands of pages to ensure semantic web standards are perfectly maintained. Built for technical SEO and recurring compliance workflows, this scraper identifies missing schema fields, exposes invalid JSON syntaxes, and flags recommended Schema.org properties you might have overlooked. Every page scraped receives a clear 0-100 quality score, giving SEO and operations teams immediate, actionable insight into technical website health. Easily schedule daily or weekly runs to monitor critical landing pages, product pages, or massive URL lists. By extracting the exact schema output mapped against search engine guidelines, you can rapidly export a comprehensive health report, catch syntax errors early, and secure your organic SERP real estate without manual intervention. ### Store Quickstart - Start with `Quickstart (Dataset)` to validate the score and error model on two public pages. - For full audits, use `Batch Validation` for multi-page quality scoring. - For recurring monitoring, use `Webhook Alert` to catch schema errors immediately. ### Key Features - 🔍 **JSON-LD + Microdata extraction** — Both formats supported - 📊 **Quality scoring** — 0-100 with A-F grade per page - ⚠️ **Error detection** — Missing `@type`, invalid JSON, missing `@context` - 💡 **Warnings** — Sparse data, missing recommended properties - 📋 **Bulk processing** — Check up to 200 URLs per run - 🪝 **Webhook delivery** — Integrate into SEO monitoring workflows ### Use Cases | Who | Why | |-----|-----| | Developers | Automate recurring data fetches without building custom scrapers | | Data teams | Pipe structured output into analytics warehouses | | Ops teams | Monitor changes via webhook alerts | | Product managers | Track competitor/market signals without engineering time | ### Input | Field | Type | Default | Description | |-------|------|---------|-------------| | urls | array | prefilled | List of page URLs to check for Schema.org structured data (JSON-LD, Microdata). Max 200. | | delivery | string | `"dataset"` | How to deliver results. 'dataset' saves to Apify Dataset (recommended), 'webhook' sends to a URL. | | webhookUrl | string | — | Webhook URL to send results to (only used when delivery is 'webhook'). Works with Slack, Discord, or any HTTP endpoint. | | concurrency | integer | `3` | Maximum number of parallel requests. Higher = faster but may trigger rate limits. | | dryRun | boolean | `false` | If true, runs without saving results or sending webhooks. Useful for testing. | #### Input Example ```json { "urls": ["https://www.google.com", "https://github.com", "https://schema.org"], "concurrency": 3 } ```` ### Input Examples #### Example: Single page validation ```json { "urls": [ "https://www.example.com/product/sku-1234" ], "includeWarnings": true } ``` #### Example: Sitemap-driven crawl ```json { "sitemapUrl": "https://www.example.com/sitemap.xml", "limit": 100, "vocabularies": [ "JSON-LD", "Microdata" ] } ``` #### Example: Audit batch with null-rate report ```json { "urls": [ "https://shop.example.com/", "https://shop.example.com/about" ], "emitNullRateReport": true } ``` ### Output | Field | Type | Description | |-------|------|-------------| | `meta` | object | | | `results` | array | | | `results[].url` | string (url) | | | `results[].jsonLd` | array | | | `results[].microdata` | array | | | `results[].errors` | array | | | `results[].warnings` | array | | | `results[].score` | object | | | `results[].error` | null | | | `results[].checkedAt` | timestamp | | #### Output Example ```json { "url": "https://www.google.com", "jsonLd": [ { "type": "WebSite", "context": "https://schema.org", "name": "Google", "_keyCount": 7 } ], "microdata": [], "errors": [], "warnings": [], "score": { "total": 80, "grade": "A" } } ``` ### API Usage Run this actor programmatically using the Apify API. Replace `YOUR_API_TOKEN` with your token from [Apify Console → Settings → Integrations](https://console.apify.com/account/integrations). #### cURL ```bash curl -X POST "https://api.apify.com/v2/acts/taroyamada~structured-data-validator/run-sync-get-dataset-items?token=YOUR_API_TOKEN" \ -H "Content-Type: application/json" \ -d '{ "urls": ["https://www.google.com", "https://github.com", "https://schema.org"], "concurrency": 3 }' ``` #### Python ```python from apify_client import ApifyClient client = ApifyClient("YOUR_API_TOKEN") run = client.actor("taroyamada/structured-data-validator").call(run_input={ "urls": ["https://www.google.com", "https://github.com", "https://schema.org"], "concurrency": 3 }) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) ``` #### JavaScript / Node.js ```javascript import { ApifyClient } from 'apify-client'; const client = new ApifyClient({ token: 'YOUR_API_TOKEN' }); const run = await client.actor('taroyamada/structured-data-validator').call({ "urls": ["https://www.google.com", "https://github.com", "https://schema.org"], "concurrency": 3 }); const { items } = await client.dataset(run.defaultDatasetId).listItems(); console.log(items); ``` ### Tips & Limitations - Keep concurrency ≤ 5 when auditing production sites to avoid WAF rate-limit triggers. - Use webhook delivery for recurring cron runs — push only deltas to downstream systems. - Enable `dryRun` for cheap validation before committing to a paid cron schedule. - Results are dataset-first; use Apify API `run-sync-get-dataset-items` for instant JSON in CI pipelines. - Run a tiny URL count first, review the sample, then scale up — pay-per-event means you only pay for what you use. ### See also (SEO audit cluster) - [Sitemap Scraper & XML Analyzer](https://apify.com/taroyamada/sitemap-analyzer) — Parse and audit XML sitemaps as a discovery layer feeding structured-data validation. ### FAQ **Is there a rate limit?** Built-in concurrency throttling keeps requests polite. For most public APIs this actor can run 1–10 parallel requests without issues. **What happens when the input URL is unreachable?** The actor records an error row with the failure reason — successful URLs keep processing. **Can I schedule recurring runs?** Yes — use Apify Schedules to run this actor on a cron (hourly, daily, weekly). Combine with webhook delivery for change alerts. **Does this actor respect robots.txt?** Yes — requests use a standard User-Agent and honor site rate limits. For aggressive audits, set a higher `concurrency` only on your own properties. **Can I integrate with Google Sheets or Airtable?** Use webhook delivery with a Zapier/Make/n8n catcher, or call the Apify REST API from Apps Script / Airtable automations. ### Complete Your Website Health Audit **Website Health Suite** — Build a comprehensive compliance and trust monitoring workflow: **1. Link & URL Health** - [🔗 Broken Link Checker](https://apify.com/taroyamada/broken-link-checker) — Find broken links across your entire site structure - [🔗 Bulk URL Health Checker](https://apify.com/taroyamada/bulk-url-health-checker) — Validate HTTP status, redirects, SSL, and response times **2. SEO & Metadata Quality** (you are here) - [Schema.org Validator](https://apify.com/taroyamada/structured-data-validator) — Validate JSON-LD and Microdata with quality scoring - [🏷️ Meta Tag Analyzer](https://apify.com/taroyamada/meta-tag-analyzer) — Audit title tags, Open Graph, Twitter Cards, and hreflang **3. Security & Email Deliverability** - [DNS/DMARC Security Checker](https://apify.com/taroyamada/dns-dmarc-security-checker) — Audit SPF, DKIM, DMARC, and MX records **4. Historical Data & Recovery** - [📚 Wayback Machine Checker](https://apify.com/taroyamada/wayback-machine-checker) — Find archived snapshots for content recovery **Recommended workflow**: Weekly schema validation → Fix errors/warnings → Validate metadata with Meta Tag Analyzer → Monitor with webhooks → Track rich snippet performance in Search Console. **Other Website Tools:** - [Sitemap Analyzer](https://apify.com/taroyamada/sitemap-analyzer) — SEO sitemap audit - [Site Governance Monitor](https://apify.com/taroyamada/site-governance-monitor) — Robots.txt and schema monitoring - [Domain Trust Monitor](https://apify.com/taroyamada/domain-trust-monitor) — SSL expiry and security headers ### Cost **Pay Per Event**: - `actor-start`: $0.01 (flat fee per run) - `dataset-item`: $0.003 per output item **Example**: 1,000 items = $0.01 + (1,000 × $0.003) = **$3.01** No subscription required — you only pay for what you use. > 💾 **Save it for later**: click the bookmark icon at the top of the Apify Store page if you'd like to come back to it. Bookmarks help other engineers find this actor via Apify's discovery surfaces. ### ⭐ Was Structured Data Scraper & Validator useful for your JSON-LD / Schema.org audit? If this actor saved you time, **[please leave a 5★ rating on Apify Store](https://apify.com/taroyamada/structured-data-validator/reviews)** — it takes 10 seconds, helps other engineers and analysts discover it, and keeps updates free. Have a feature request, bug, or sample workflow you'd like to share? **[Open an issue](https://apify.com/taroyamada/structured-data-validator/issues)** — we read every one and use them to prioritise the next release. ### Related report Actors Use these follow-on Actors when you want a capped, decision-ready report instead of more raw rows. They use public or user-provided inputs, respect `maxChargeUsd`, and do not promise rankings, revenue, conversion lifts, or sales outcomes. - [Website RAG Readiness Audit](https://apify.com/taroyamada/website-rag-readiness-audit) - add retrieval risk, source coverage, and cleanup actions after technical page checks. ### Related paid report workflows If this Actor gave you raw rows or source context, these follow-on report Actors are designed for a small capped paid run. They help make a decision, not just collect more data. - [Website RAG Readiness Audit Report](https://apify.com/taroyamada/website-rag-readiness-audit) - decide whether public website pages are clean and complete enough for RAG ingestion. Entry $9 / `website_rag_snapshot_report`; premium $29 / `website_rag_readiness_report`. Keep `maxChargeUsd` equal to the selected tier. Internal links are traffic aids only; real proof requires accounted paid usage. ### Related report Actors Use these follow-on Actors when you want a capped, decision-ready report instead of more raw rows. They use public or user-provided inputs, respect `maxChargeUsd`, and do not promise rankings, revenue, conversion lifts, or sales outcomes. - [Website RAG Readiness Audit](https://apify.com/taroyamada/website-rag-readiness-audit) - add retrieval risk, source coverage, and cleanup actions after technical page checks. # Actor input Schema ## `urls` (type: `array`): List of page URLs to check for Schema.org structured data (JSON-LD, Microdata). Max 200. ## `delivery` (type: `string`): How to deliver results. 'dataset' saves to Apify Dataset (recommended), 'webhook' sends to a URL. ## `webhookUrl` (type: `string`): Webhook URL to send results to (only used when delivery is 'webhook'). Works with Slack, Discord, or any HTTP endpoint. ## `concurrency` (type: `integer`): Maximum number of parallel requests. Higher = faster but may trigger rate limits. ## `dryRun` (type: `boolean`): If true, runs without saving results or sending webhooks. Useful for testing. ## `maxChargeUsd` (type: `number`): Safety cap for this run. Billable rows beyond this cap are returned as no-charge limit\_reached summaries. ## Actor input object example ```json { "urls": [ "https://www.google.com", "https://github.com" ], "delivery": "dataset", "concurrency": 3, "dryRun": false, "maxChargeUsd": 1 } ``` # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "urls": [ "https://www.google.com", "https://github.com" ] }; // Run the Actor and wait for it to finish const run = await client.actor("taroyamada/structured-data-validator").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "urls": [ "https://www.google.com", "https://github.com", ] } # Run the Actor and wait for it to finish run = client.actor("taroyamada/structured-data-validator").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "urls": [ "https://www.google.com", "https://github.com" ] }' | apify call taroyamada/structured-data-validator --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=taroyamada/structured-data-validator", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Structured Data Scraper & Validator", "description": "Crawl websites to extract JSON-LD and Microdata, validate schema markup syntax, and flag missing fields across massive URL lists.", "version": "0.1", "x-build-id": "iDzhGcoEOLDLfzraU" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/taroyamada~structured-data-validator/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-taroyamada-structured-data-validator", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/taroyamada~structured-data-validator/runs": { "post": { "operationId": "runs-sync-taroyamada-structured-data-validator", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/taroyamada~structured-data-validator/run-sync": { "post": { "operationId": "run-sync-taroyamada-structured-data-validator", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "required": [ "urls" ], "properties": { "urls": { "title": "URLs to Validate", "type": "array", "description": "List of page URLs to check for Schema.org structured data (JSON-LD, Microdata). Max 200.", "items": { "type": "string" } }, "delivery": { "title": "Delivery Mode", "enum": [ "dataset", "webhook" ], "type": "string", "description": "How to deliver results. 'dataset' saves to Apify Dataset (recommended), 'webhook' sends to a URL.", "default": "dataset" }, "webhookUrl": { "title": "Webhook URL", "type": "string", "description": "Webhook URL to send results to (only used when delivery is 'webhook'). Works with Slack, Discord, or any HTTP endpoint." }, "concurrency": { "title": "Concurrency", "minimum": 1, "maximum": 10, "type": "integer", "description": "Maximum number of parallel requests. Higher = faster but may trigger rate limits.", "default": 3 }, "dryRun": { "title": "Dry Run", "type": "boolean", "description": "If true, runs without saving results or sending webhooks. Useful for testing.", "default": false }, "maxChargeUsd": { "title": "Maximum charge (USD)", "minimum": 0, "type": "number", "description": "Safety cap for this run. Billable rows beyond this cap are returned as no-charge limit_reached summaries.", "default": 1 } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```