# Sitemap URL Extractor - XML Sitemap Scraper (`benthepythondev/sitemap-url-extractor`) Actor

Extract URLs from XML sitemaps and sitemap indexes. Get URL, lastmod, changefreq, priority and source sitemap.

- **URL**: https://apify.com/benthepythondev/sitemap-url-extractor.md
- **Developed by:** [Ben](https://apify.com/benthepythondev) (community)
- **Categories:** SEO tools, Automation, Business
- **Stats:** 4 total users, 3 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.80 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Sitemap URL Extractor - XML Sitemap Scraper

Extract URLs from XML sitemaps and sitemap indexes in one run. Get page URL, last modified date, change frequency, priority and the source sitemap each URL came from. Export to JSON/CSV/Excel, run on a schedule, call via API, or connect to Make, Zapier or n8n.

### What is the Sitemap URL Extractor?

The Sitemap URL Extractor turns XML sitemap files into clean datasets. Paste one or more sitemap URLs and the actor returns every URL it can find, including URLs inside nested sitemap indexes. It is useful for SEO audits, migration checks, website inventory, crawling preparation, content monitoring and competitor research.

#### What data does it extract?

- **URL:** the page or asset URL listed in the sitemap.
- **Last modified:** `lastmod` when provided by the site.
- **Change frequency:** `changefreq` when present.
- **Priority:** sitemap priority when present.
- **Source sitemap:** the exact sitemap file that contained the URL.

### Input

Paste one or more sitemap XML URLs into `sitemapUrls`. Keep `includeNested` enabled to follow sitemap indexes automatically. Use `maxUrls` to cap large websites.

#### Example input

```json
{
  "sitemapUrls": ["https://apify.com/sitemap.xml"],
  "includeNested": true,
  "maxUrls": 1000
}
```

### Output

Each dataset row can include:

```json
{
  "url": "https://example.com/page",
  "lastmod": "2026-07-01",
  "changefreq": "weekly",
  "priority": "0.8",
  "source_sitemap": "https://example.com/post-sitemap.xml"
}
```

### Use cases

🔎 **SEO audits:** Export all indexed sitemap URLs and check which pages a site wants search engines to crawl.

🚚 **Website migrations:** Compare old and new sitemap exports to catch missing URLs, redirects or forgotten content.

🗂️ **Content inventory:** Build a structured list of pages for editorial, analytics or internal search projects.

🕷️ **Crawler seed lists:** Use sitemap URLs as clean start URLs for a larger crawler.

📊 **Competitor research:** Export competitor sitemap structures to understand content depth, landing pages and publishing cadence.

### Workflow ideas

Run this actor before a crawl to collect a reliable URL seed list. Send the output into a website crawler, status-code checker, SEO audit tool or spreadsheet. For migrations, run it on the old and new domains, then compare URL sets. For publishers, schedule it weekly and track new URLs by comparing the latest run with previous datasets.

Because each row includes the `source_sitemap`, you can see which section of a sitemap index produced the URL. That helps separate blog posts, products, categories, docs, images or localized pages when a site uses multiple nested sitemap files.

### Reliability notes

The actor expects valid XML sitemaps using the standard sitemap protocol. If a website blocks requests, returns HTML instead of XML, or uses a non-standard sitemap format, the actor logs a warning and continues with the next sitemap.

Very large sitemap indexes can contain hundreds of thousands of URLs. Use `maxUrls` during testing and increase it gradually for production exports.

### Data quality tips

Sitemaps are declarations from the website owner, not proof that a page is live, indexed or canonical. A URL in a sitemap can still redirect, return 404, be blocked by robots.txt or have a `noindex` tag. For a complete SEO audit, use this actor as the URL discovery step and then pass the output into a crawler, status checker or page analyzer.

When comparing two sitemap exports, normalize trailing slashes, protocols and hostnames before diffing. Many migration issues are caused by small URL-shape differences. Keep the `source_sitemap` field because it often reveals which site section changed: blog, docs, products, categories, localized pages or image assets.

For large ecommerce sites, run with a small `maxUrls` first to confirm the sitemap structure. Then increase the limit once you know the nested sitemap index is behaving as expected.

### Pricing

This actor uses pay-per-result pricing. You are charged for each sitemap URL pushed to the dataset, plus a tiny actor-start event.

### FAQ

#### Does it follow sitemap indexes?

Yes. Keep `includeNested` enabled and the actor follows nested sitemap files.

#### Does it crawl the pages?

No. It extracts URLs from sitemap XML. Use a crawler or status checker separately if you need page content or HTTP status codes.

#### Can I extract multiple sitemaps?

Yes. Add multiple values to `sitemapUrls`.

#### Does it support image or video sitemaps?

It extracts the main URL entries. Specialized image/video metadata is not the focus of this actor.

#### Can I schedule sitemap monitoring?

Yes. Save a task and compare datasets over time.

#### Is it legal?

This actor reads public sitemap files. Use the output responsibly and respect website terms, robots guidance and applicable rules.

#### Why are some fields null?

Sitemap fields like `lastmod`, `priority` and `changefreq` are optional. Many sites only provide the URL.

#### Can it extract URLs from robots.txt?

This actor expects sitemap XML URLs. Use a robots/sitemap analyzer first if you need to discover sitemap URLs from a domain's robots.txt file.

#### Can I use it for competitor research?

Yes. Public sitemaps often reveal product pages, category structures, blog depth and localized landing pages.

#### Does it respect max URL limits?

Yes. The actor stops pushing rows once `maxUrls` is reached, even when a sitemap index contains more nested files.

### You might also like

- Robots Sitemap Analyzer - inspect robots.txt and sitemap references
- Website Contact Extractor - enrich domains from URL lists
- RSS Feed Scraper - extract feed articles
- Tech Stack Detector - analyze website technologies

### Keywords

sitemap scraper, sitemap URL extractor, XML sitemap parser, sitemap index scraper, SEO URL export, sitemap API, website URL inventory, crawl seed list, sitemap to CSV, sitemap monitoring, SEO audit scraper.

# Actor input Schema

## `sitemapUrls` (type: `array`):

Sitemap XML URLs to extract. Sitemap indexes are followed when includeNested is true.

## `includeNested` (type: `boolean`):

Follow sitemap indexes and extract URLs from nested sitemaps.

## `maxUrls` (type: `integer`):

Maximum number of URLs to push.

## Actor input object example

```json
{
  "sitemapUrls": [
    "https://apify.com/sitemap.xml"
  ],
  "includeNested": true,
  "maxUrls": 100
}
```

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "sitemapUrls": [
        "https://apify.com/sitemap.xml"
    ],
    "includeNested": true,
    "maxUrls": 100
};

// Run the Actor and wait for it to finish
const run = await client.actor("benthepythondev/sitemap-url-extractor").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "sitemapUrls": ["https://apify.com/sitemap.xml"],
    "includeNested": True,
    "maxUrls": 100,
}

# Run the Actor and wait for it to finish
run = client.actor("benthepythondev/sitemap-url-extractor").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "sitemapUrls": [
    "https://apify.com/sitemap.xml"
  ],
  "includeNested": true,
  "maxUrls": 100
}' |
apify call benthepythondev/sitemap-url-extractor --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "command": "npx",
            "args": [
                "mcp-remote",
                "https://mcp.apify.com/?tools=benthepythondev/sitemap-url-extractor",
                "--header",
                "Authorization: Bearer <YOUR_API_TOKEN>"
            ]
        }
    }
}

```

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/acts/LjvKMYRXTf48rNrkx/builds/sfSaczE3sFRd7koNa/openapi.json