# Apify Creator Scraper · Bulk Profile & Actor Data Export (`corent1robert/apify-users-sitemap-scraper`) Actor

Extract every public Apify creator profile in one run.
Get name, username, published actors, and usage stats —
ready to export as a lead list or enrich in your CRM

- **URL**: https://apify.com/corent1robert/apify-users-sitemap-scraper.md
- **Developed by:** [Corentin Robert](https://apify.com/corent1robert) (community)
- **Categories:** Lead generation, Automation, Developer tools
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $6.99 / 1,000 results

This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Apify Creator Scraper · Bulk Store profiles & actor signals

**Export public Apify Store creator profiles at scale** — one **dataset row per creator** with identity, **optional public email**, social links, usage stats, and a **server-rendered sample** of published Actors. Use it for **lead research**, **partner mapping**, **competitive snapshots**, or **CRM enrichment**, then export **JSON** or **CSV** from the Apify Dataset.

The Actor reads **only what is already public** on `apify.com` profile pages (HTML + embedded Next.js data). It is **not** an official Apify product or API. Respect [Apify’s Terms of Service](https://apify.com/legal) and **fair use** — do not overload the site or use the data in ways that violate privacy or platform rules.

***

### Table of contents

- [Why use this?](#why-use-this)
- [How it works (under the hood)](#how-it-works-under-the-hood)
- [Quick start on Apify](#quick-start-on-apify)
- [Dataset views & key fields](#dataset-views--key-fields)
- [Understanding `email` and `publicActors`](#understanding-email-and-publicactors)
- [Failures, speed, and tuning](#failures-speed-and-tuning)
- [Input](#input)
- [Output (dataset & stores)](#output-dataset--stores)
- [Example inputs](#example-inputs)
- [Logs during a run](#logs-during-a-run)
- [Limits](#limits)
- [Support & contact](#support--contact)
- [Developers: local run, tests, deploy](#developers-local-run-tests-deploy)

***

### Why use this?

| Without this Actor | With this Actor |
|--------------------|-----------------|
| Open each **`apify.com/{username}`** profile by hand | **Thousands of profiles in one run** (sitemap or your own URL list) |
| Copy-paste name, bio, and stats into a sheet | **Structured rows** — usernames, links, stats, timestamps |
| Guess who left a **public email** on their profile | **`email` column** when the creator chose to show it — filter non-empty in export |

**Who it’s for**

- **SDRs / growth** — build lists of creators who publish on the Store and surface **public** contact hints (email, website, LinkedIn, X).
- **Partnerships & BD** — map **actor counts** and activity-style stats without manual browsing.
- **Research & analytics** — snapshot **public** profile metadata and SSR actor samples for analysis you’re allowed to do.

**Reality check:** **`email` is often empty.** Most profiles do **not** expose an email. When present, it is **only** the address the user made **public** on Apify — not a guess or enrichment. **`publicActors`** is a **partial** list from the HTML (see below), not the full catalog behind “Load more” in the browser.

***

### How it works (under the hood)

1. **Input** — Either **no profile list** (empty input) or an array **`profileUrls`** — one `https://apify.com/{username}` per line (scheme optional). URLs are normalized and deduplicated.
2. **URL discovery** — If **`profileUrls`** is empty, the Actor fetches the official sitemap [`https://apify.com/sitemap/users.xml`](https://apify.com/sitemap/users.xml), parses `<loc>` entries, keeps only single-segment profile URLs, applies an internal cap (see [Limits](#limits)), then scrapes each page.
3. **Fetch** — Each profile page is requested with **`got-scraping`** (browser-like headers). Multiple pages run in **parallel** with configurable concurrency (see [Failures, speed, and tuning](#failures-speed-and-tuning)).
4. **Parse** — Public data is extracted from **embedded Flight / RSC-style JSON** in the HTML (`user`, `actorsTotalUsers`, `actors` array when present). No browser automation — **static HTTP + string parse** only.
5. **Output** — Each profile becomes **one dataset item** (success or error row). A **`RUN_LOG`** key in the default key-value store lists short failure lines for troubleshooting. The **Output** tab can link dataset URLs (including curated **views**) when `output_schema.json` is configured.

***

### Quick start on Apify

1. Open the Actor in the **Apify Console**.
2. **Full sitemap run:** leave **Profile URLs** **empty** (or use `{}` in JSON input).
3. **Targeted run:** paste **one profile URL per line** under **Profile URLs**.
4. Click **Start**.
5. Open **Dataset** → tab **Creators (lead-ready)** for a compact table, or **Profile detail (all fields)** for every column including **`publicActors`**. Export **JSON** or **CSV** as needed.

For **CLI** runs: `apify login` is optional for this Actor (no Apify Proxy in input); use `apify run --input-file=input.json` so local input is not empty after a clean checkout.

***

### Dataset views & key fields

The Console exposes two **views** (column presets). Raw items always contain the full successful payload; views only change **which columns** you see first.

#### Creators (lead-ready) — `?view=results`

Optimized for scanning and export: **contact and identity first**, no bulky **`publicActors`** JSON in the grid.

| Field | Why it matters |
|--------|----------------|
| **`profileUrl`** | Canonical public profile link. |
| **`username`** | Store username. |
| **`name`** | Display name. |
| **`email`** | **Public** profile email only — often `null`. |
| **`websiteUrl`**, **`linkedinUrl`**, **`githubUsername`**, **`twitterUsername`**, **`discordUserId`** | Other **public** links / handles. |
| **`pictureUrl`**, **`bio`** | Quick context for outreach or filtering. |
| **`actorsTotalUsers`**, **`publicActorsSampleCount`** | Scale of published actors vs. how many names appear in this row’s sample. |
| **`statsActiveUsers30Days`**, **`statsRunSuccessRate`**, **`statsIssueResponseTimeDays`** | Usage-style stats when present in the payload. |
| **`scrapedAt`** | When the row was written. |
| **`httpOk`**, **`error`** | Set on **failure** rows (see [Output](#output-dataset--stores)). |

#### Profile detail (all fields) — `?view=profileDetail`

Same **lead block** at the front, then **IDs**, **readme**, **SEO meta** (`pageTitle`, `metaDescription`, `canonicalUrl`), and the full **`publicActors`** array (sample from HTML).

***

### Understanding `email` and `publicActors`

- **`email`** — Mapped from Apify’s **`profile.publicEmail`**. If the creator did not enable a public email on their profile, the value is **`null`**. Filter **`email` is not empty** in Sheets or SQL when you only want rows **with** a visible address.
- **`publicActors`** — Parsed from **server-rendered** HTML. It often contains **fewer** actors than **`actorsTotalUsers`** because the rest loads in the **client** after page load. Treat **`actorsTotalUsers`** as the **total** signal and **`publicActors`** as a **capped sample** for context (names, titles, categories, etc.).

***

### Failures, speed, and tuning

Defaults are tuned for **throughput** while staying reasonable toward `apify.com`: **12** parallel fetches and **no** fixed delay between requests.

If you see **many timeouts**, **429**s, or **blocked** responses:

1. Open **`src/main.js`** and lower **`MAX_CONCURRENCY`** (e.g. from `12` to `6` or `4`).
2. Optionally set a small **`DELAY_MS`** between requests (e.g. `50`–`150`) to reduce burst load.

These are **source constants** (not Console input fields). Rebuild / **`apify push`** after changes.

***

### Input

| Field | Required | Description |
|--------|----------|-------------|
| **profileUrls** | No | String list: one **`https://apify.com/{username}`** per line (`apify.com/user` also works). **Leave empty** to use the **full users sitemap** (capped per run — see [Limits](#limits)). |

***

### Output (dataset & stores)

**Dataset** — one object per profile URL processed:

- **Success rows** — profile fields from embedded JSON + meta + stats + **`publicActors`** sample (see [Dataset views & key fields](#dataset-views--key-fields)).
- **Failure rows** — at minimum **`profileUrl`**, **`scrapedAt`**, **`error`**, **`httpOk`**: `false`; **`pageTitle`** may be present when HTML was partially readable.

**Key-value store** — key **`RUN_LOG`**: short text log (recent failures and progress-style lines).

**Apify Output tab** — links may include: raw dataset items, **Creators (lead-ready)** view, **Profile detail (all fields)** view, run metadata, and **`RUN_LOG`** — per `output_schema.json`.

***

### Example inputs

**Full sitemap run (default cap applies)**

```json
{}
```

**Targeted profiles**

```json
{
  "profileUrls": [
    "https://apify.com/apify",
    "apify.com/zdroj"
  ]
}
```

***

### Logs during a run

- **Progress** — Periodic **INFO** lines with counts, success/failure totals, and an **ETA** while the run is active.
- **Warnings** — Failed profiles log a **short** reason (full detail is in the dataset row’s **`error`** field).

The **dataset** is the source of truth; logs are for monitoring.

***

### Limits

- **Sitemap mode:** up to **50,000** profile URLs per run after filtering (internal cap in code; sitemap may grow over time).
- **Public pages only** — private or restricted profiles may yield **error** rows or missing payloads.
- **Layout changes** — if Apify changes how profile data is embedded in HTML, parsing may need an Actor update.

***

### Support & contact

**Questions or issues?** [Corentin Robert](mailto:corentin@outreacher.fr).

***

### Developers: local run, tests, deploy

```bash
npm install
npm run apify:run    # apify run --input-file=input.json
npm test             # offline unit tests (sitemap + profile parse)
apify push
```

Bump **`.actor/actor.json`** **`version`** (`MAJOR.MINOR` or patch per your convention) when you publish a new build.

# Actor input Schema

## `profileUrls` (type: `array`):

One **URL per line**, e.g. `https://apify.com/apify` or `apify.com/username`. Leave **empty** for a full sitemap run.

## Actor input object example

```json
{
  "profileUrls": [
    "https://apify.com/apify"
  ]
}
```

# Actor output Schema

## `dataset` (type: `string`):

All creator rows — same fields as **All columns** in the Console.

## `resultsView` (type: `string`):

Curated columns — **email** and social links first.

## `profileDetailView` (type: `string`):

Every field including `readme` and `publicActors` sample.

## `apiRun` (type: `string`):

Apify **GET** for run metadata. `Authorization: Bearer YOUR_API_TOKEN`.

## `runLog` (type: `string`):

Failures only — username and short error for troubleshooting.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "profileUrls": [
        "https://apify.com/apify"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("corent1robert/apify-users-sitemap-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "profileUrls": ["https://apify.com/apify"] }

# Run the Actor and wait for it to finish
run = client.actor("corent1robert/apify-users-sitemap-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "profileUrls": [
    "https://apify.com/apify"
  ]
}' |
apify call corent1robert/apify-users-sitemap-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "command": "npx",
            "args": [
                "mcp-remote",
                "https://mcp.apify.com/?tools=corent1robert/apify-users-sitemap-scraper",
                "--header",
                "Authorization: Bearer <YOUR_API_TOKEN>"
            ]
        }
    }
}

```

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/S6EJHIY6hqrx9aDDX/builds/5vPpRNsfT1npDdQIp/openapi.json