# Similarweb Scraper — Website Traffic & Competitors (`trakk/similarweb-scraper`) Actor

Get traffic & competitor intelligence for any website — global/country ranks, monthly visits, engagement, MoM growth, traffic sources, top keywords, top countries, AI-referral traffic, similar sites, screenshots & WHOIS. No login needed. Export to JSON, CSV or Excel.

- **URL**: https://apify.com/trakk/similarweb-scraper.md
- **Developed by:** [DataForge](https://apify.com/trakk) (community)
- **Categories:** SEO tools, Lead generation, E-commerce
- **Stats:** 13 total users, 9 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.75 / 1,000 base data

This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## 🔍 Similarweb Scraper — Website Traffic & Competitor Intelligence

> **Know how much traffic *any* website gets — and where it comes from.** 📊
> Paste a list of domains, press **Start**, and get clean, structured data: global
> rankings, monthly visits, traffic sources, top keywords, top countries, the AI
> chatbots sending visitors, similar/competitor sites, and domain WHOIS.
> **No Similarweb account, no login, no API key needed.** ✅

Feed it one domain to try it, or thousands to run a whole market analysis. One
website in → **one clean row out**. Export to JSON, CSV, Excel, or Google Sheets. 📤

***

### ⚡ Quick start (30 seconds)

1. Paste your domains into **Domains** (e.g. `github.com`, `nytimes.com`).
2. Pick a **Mode** — start with **All-in-one** to get everything.
3. Press **Start**. 🚀

```json
{
  "domains": ["github.com", "nytimes.com"],
  "mode": "all",
  "maxConcurrency": 10
}
```

That's it. Each domain becomes one row in the **Output** tab. Download it in any
format you like.

***

### 📦 What you can collect — pick a Mode

Each run uses **one mode**. Here's what each returns:

| Mode | 🏷️ Value | What you get |
|---|---|---|
| ⭐ **All-in-one** | Everything in one row | Base data **+** similar sites **+** WHOIS **+** keywords, merged per domain |
| 📈 **Base data** | Traffic & rankings | Ranks, monthly visits, engagement, month-over-month growth, traffic sources, top keywords, top countries, AI traffic, screenshot |
| 🔗 **Similar sites** | Competitors | Related/competitor sites with rank, category, visits & similarity grade, topic tags, related mobile apps |
| 🔎 **WHOIS + keywords** | Domain & on-page SEO | Registrar, registration/expiry dates, nameservers, DNSSEC + on-page keyword density |

> 💡 **Not sure which to pick?** Use **All-in-one** — it returns all of the above
> in a single row per domain.

***

### 📈 Mode 1 — Base data

The core traffic report. Example output for `github.com`:

```json
{
  "domain": "github.com",
  "siteName": "github.com",
  "category": "computers_electronics_and_technology/programming_and_developer_software",
  "globalRank": 47,
  "countryRank": { "countryCode": "US", "rank": 76 },
  "engagement": {
    "visits": 615239605,
    "bounceRate": 0.363,
    "pagesPerVisit": 5.87,
    "timeOnSite": 383.3
  },
  "estimatedMonthlyVisits": {
    "2026-04-01": 631004092,
    "2026-05-01": 636075828,
    "2026-06-01": 615239605
  },
  "trafficChangeMoM": -0.0328,
  "screenshot": "https://site-images.similarcdn.com/image?url=github.com&t=1&h=...",
  "trafficSources": {
    "direct": 0.521, "searchOrganic": 0.253, "referrals": 0.099,
    "socialOrganic": 0.091, "genAi": 0.020
  },
  "topCountryShares": [
    { "countryCode": "US", "share": 0.193 },
    { "countryCode": "CN", "share": 0.113 }
  ],
  "topKeywords": [
    { "name": "github", "volume": 9009470, "cpc": 1.54 }
  ],
  "aiTraffic": [
    { "name": "chatgpt.com", "rank": 1 },
    { "name": "gemini.google.com", "rank": 2 },
    { "name": "claude.ai", "rank": 3 }
  ],
  "status": "ok"
}
```

#### 📖 What each field means

| Field | Meaning |
|---|---|
| `globalRank` | The site's rank worldwide (1 = most visited on the internet) 🌍 |
| `countryRank` | Rank inside its top country |
| `engagement.visits` | Total visits in the latest month 👥 |
| `engagement.bounceRate` | Share of visitors who leave after one page (0.36 = 36%) |
| `engagement.pagesPerVisit` | Average pages viewed per visit |
| `engagement.timeOnSite` | Average time on site, in seconds ⏱️ |
| `estimatedMonthlyVisits` | Visit totals for the last 3 months (trend) 📅 |
| `trafficChangeMoM` | Month-over-month change (−0.0328 = **−3.3%**) 📉📈 |
| `screenshot` | 🖼️ **URL** of a homepage screenshot (opens in browser; not a saved file) |
| `trafficSources` | Where visits come from — direct, search, social, referrals, paid, mail, and **`genAi`** (AI chatbots) |
| `topCountryShares` | Top countries by visit share 🗺️ |
| `topKeywords` | Top search keywords with monthly `volume` and `cpc` (cost-per-click, $) 🔑 |
| `aiTraffic` | 🤖 Which AI platforms (ChatGPT, Gemini, Claude, Perplexity…) send visitors, ranked |

***

### 🔗 Mode 2 — Similar sites

Find competitors and alternatives. Example for `github.com`:

```json
{
  "domain": "github.com",
  "category": "Computers_Electronics_and_Technology/Programming_and_Developer_Software",
  "totalVisits": 513197610,
  "tags": [{ "Tag": "git" }, { "Tag": "open source" }, { "Tag": "programming" }],
  "similarSites": [
    {
      "site": "stackoverflow.com",
      "category": "Computers_Electronics_and_Technology/Programming_and_Developer_Software",
      "similarityRank": 1,
      "totalVisits": 69124392,
      "grade": 0.94
    },
    { "site": "sourceforge.net", "similarityRank": 2, "grade": 0.74 }
  ],
  "relatedApps": [
    { "Title": "GitHub", "Type": "ANDROID", "Ranking": 2,
      "Url": "https://play.google.com/store/apps/details?id=com.github.android" }
  ],
  "status": "ok"
}
```

#### 📖 What each field means

| Field | Meaning |
|---|---|
| `similarSites[].site` | A competitor/alternative domain |
| `similarSites[].similarityRank` | How similar it is (1 = most similar) |
| `similarSites[].grade` | Similarity score, 0–1 (higher = closer match) ⭐ |
| `similarSites[].totalVisits` | That competitor's monthly visits |
| `tags` | Topics/interests associated with the site 🏷️ |
| `relatedApps` | Related mobile apps (Android & iOS) with store links 📱 |

***

### 🔎 Mode 3 — WHOIS + keywords

Domain registration facts + what the homepage is "about". Example for `github.com`:

```json
{
  "domain": "github.com",
  "whois": {
    "registrar": "MarkMonitor Inc.",
    "registrationDate": "2007-10-09T18:20:50Z",
    "expirationDate": "2026-10-09T18:20:50Z",
    "nameServers": ["DNS1.P08.NSONE.NET", "NS-421.AWSDNS-52.COM"],
    "dnssec": false
  },
  "keywordDensity": {
    "pageTitle": "GitHub · Build and ship software",
    "totalTokens": 855,
    "keywords": [
      { "token": "github", "count": 53, "frequency": 0.062 },
      { "token": "copilot", "count": 20, "frequency": 0.023 }
    ]
  },
  "status": "ok"
}
```

#### 📖 What each field means

| Field | Meaning |
|---|---|
| `whois.registrar` | Company the domain is registered with 🏢 |
| `whois.registrationDate` / `expirationDate` | When the domain was registered / expires 📆 |
| `whois.nameServers` | The domain's DNS servers |
| `whois.dnssec` | Whether DNSSEC security is enabled 🔒 |
| `keywordDensity.keywords` | Most frequent words on the homepage, with `count` and `frequency` (share of all words) 🔑 |

***

### ⭐ Mode 4 — All-in-one

Everything above, merged into **one row per domain** — `globalRank`, `engagement`,
`trafficSources`, `topKeywords`, `aiTraffic`, `screenshot`, `similarSites`,
`relatedApps`, `whois`, `keywordDensity`. Run this when you want the full picture
in a single export. 🧩

***

### ⌨️ Input options

| Field | Description |
|---|---|
| **Domains** 🌐 | Websites to analyse. Accepts `example.com`, `www.example.com`, or `https://example.com/path` — everything is normalised to a bare host automatically. |
| **Dataset mode** 🗂️ | `all`, `base_data`, `similar_sites`, or `aitdk`. One mode per run. |
| **Max concurrency** ⚡ | How many domains to process at the same time (higher = faster). |

***

### ⚙️ Speed & memory per mode

Different modes do different amounts of work, so they have different memory
recommendations. Pick the memory when you start a run (default **512 MB**):

| Mode | 🧠 Recommended memory | ⚡ Speed (50 domains) | Why |
|---|---|---|---|
| 📈 **Base data** | **256 MB** | ~28 sec | One fast source |
| 🔗 **Similar sites** | **256 MB** | ~32 sec | One fast source |
| 🔎 **WHOIS + keywords** | **512 MB** | ~60 sec | WHOIS/RDAP is slow + parses homepage HTML |
| ⭐ **All-in-one** | **512 MB** | ~40 sec (high concurrency) | Fetches all sources at once |

> 💡 **Faster runs:** speed scales with **Max concurrency** — raising it processes
> more domains at the same time. For big jobs, bump concurrency up (to 50) and,
> for the heavier modes, keep memory at 512 MB.
>
> 💰 **Cheaper runs:** for **Base data** and **Similar sites** you can drop memory
> to **256 MB** to roughly halve the compute cost with no downside.

***

### 🧪 Recipes — what people use it for

- 📊 **Benchmark competitors** → run **Base data** on your domain + rivals, compare `globalRank`, `engagement`, and `trafficSources` side by side.
- 🕵️ **Discover new competitors** → run **Similar sites** and sort `similarSites` by `grade`.
- 🤖 **Track AI-driven traffic** → the `aiTraffic` list shows which chatbots (ChatGPT / Gemini / Claude / Perplexity) send visitors — a fast-growing channel most tools ignore.
- 🔑 **Keyword & SEO research** → pull `topKeywords` (volume + CPC) from Base data, or homepage `keywordDensity` from WHOIS + keywords.
- 💼 **Score sales leads** → rank prospects by real traffic and rank instead of guessing.
- 📈 **Validate growth claims** → `trafficChangeMoM` and the 3-month `estimatedMonthlyVisits` trend show if a site is really growing.

***

### 📤 Output & export

Every domain becomes **one row** in the dataset. From the run's **Output** tab you
can:

- 👀 Preview the data in a table
- 💾 Download as **JSON, CSV, Excel, or HTML**
- 🔗 Pull it via the Apify API, or push to **Google Sheets / Zapier / Make** and your CRM

Every row carries a **`status`** field so you always know what happened:

| `status` | Meaning |
|---|---|
| ✅ `ok` | Data collected successfully |
| ⚠️ `partial` | Some of the data was available, some wasn't (WHOIS/all modes) |
| ❌ `failed` | No data for this domain — check the `error` field (e.g. `no_data`, `not_found`) |

> 🛡️ **We never invent numbers.** If a source has no data for a domain, the row is
> marked `failed` — you always get honest results.

***

### 💰 Pricing & FAQ

**How much does it cost?** The run cost is **all-in** — no hidden add-ons. Cost
scales with the number of domains you analyse, so a quick 10-domain check is cheap
and a 5,000-domain sweep is predictable.

**Which sites work?** Any public website. Very new or tiny sites may return
`failed` / `partial` — Similarweb itself has limited data for low-traffic domains.

**How fresh is the data?** Traffic figures reflect Similarweb's latest monthly
snapshot; each Base data row includes a `snapshotDate`.

**Are competitors always included?** The `similarSites` list (Similar sites / All
modes) is your competitor list. The Base data `competitors` field is often empty —
use **Similar sites** mode for reliable competitor discovery.

**Can I get all four datasets at once?** Yes — use **All-in-one** mode, or run the
other modes separately and join on `domain`.

**Are the screenshots downloaded?** No — `screenshot` is a **URL** to a hosted
image. Open it in a browser or download it yourself; nothing is stored on your side.

# Actor input Schema

## `domains` (type: `array`):

Websites to analyse. Accepts example.com, www.example.com or https://example.com/path — everything is normalised to a bare host.

## `mode` (type: `string`):

Which data source to collect for every domain.

## `maxConcurrency` (type: `integer`):

How many domains to fetch in parallel.

## `proxyConfiguration` (type: `object`):

Proxy settings. Apify Residential is strongly recommended for reliable, captcha-free results.

## Actor input object example

```json
{
  "domains": [
    "github.com",
    "nytimes.com"
  ],
  "mode": "base_data",
  "maxConcurrency": 10,
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": [
      "RESIDENTIAL"
    ]
  }
}
```

# Actor output Schema

## `all` (type: `string`):

No description

## `base_data` (type: `string`):

No description

## `similar_sites` (type: `string`):

No description

## `aitdk` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "domains": [
        "github.com"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("trakk/similarweb-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "domains": ["github.com"] }

# Run the Actor and wait for it to finish
run = client.actor("trakk/similarweb-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "domains": [
    "github.com"
  ]
}' |
apify call trakk/similarweb-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "command": "npx",
            "args": [
                "mcp-remote",
                "https://mcp.apify.com/?tools=trakk/similarweb-scraper",
                "--header",
                "Authorization: Bearer <YOUR_API_TOKEN>"
            ]
        }
    }
}

```

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/BBX2Pjax6ghxvPBqV/builds/oBaNOkQ2yZdmcBoRh/openapi.json