# Google Search Scraper — SERP, AI Overview, Ads & Operators (`memo23/google-search-scraper`) Actor

Scrape Google Search SERP — organic results, paid ads, related queries, People Also Ask, knowledge panels — with built-in anti-block defenses (AdsBot UA, udm=14, multi-selector parsing, dual-engine HTTP).

- **URL**: https://apify.com/memo23/google-search-scraper.md
- **Developed by:** [Muhamed Didovic](https://apify.com/memo23) (community)
- **Categories:** SEO tools, AI, Automation
- **Stats:** 32 total users, 29 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: 5.00 out of 5 stars

## Pricing

from $3.00 / 1,000 serp pages

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Google Search Results Scraper

Scrape Google Search result pages (SERPs) and get clean, structured JSON — organic
results, related searches, People Also Ask, knowledge panels, and total result
counts — for any query, country, and language.

Built for reliability: a dual-engine HTTP fetcher with real browser TLS
impersonation, an 8-deep self-healing selector chain that survives Google's layout
rotations, and structural soft-block detection that won't false-reject a valid SERP.

![How the Google Search Scraper works](https://raw.githubusercontent.com/muhamed-didovic/muhamed-didovic.github.io/main/assets/how-it-works-google-search.png)

### What it extracts

- **Organic results** — position, title, URL, displayed URL, description, date,
  emphasized keywords, sitelinks, and inline product info (rating / reviews / price)
- **Related searches** — the "people also search for" / bottom-of-page queries
- **People Also Ask** — the expandable Q\&A questions (with answer text when present)
- **Knowledge panel** — title, subtitle, description, fact attributes (Born, Died,
  Address, Height, …), and source link
- **AI Overview** — detection + the cited **source domains** for GEO/AEO tracking
  (the generated prose itself is not available over the proxy — see **Limitations**)
- **Total results** — the "About N results" count
- **Contact emails (opt-in)** — turn a SERP into leads: scrape each result's own
  site for a contact email + people. See **Lead enrichment** below.
- **Perplexity AI answer (opt-in, BYOK)** — a grounded AI answer with citations per
  query, via Perplexity's official API with your key. See **Perplexity AI answer**.

### Input

| Field | Type | Default | Notes |
|---|---|---|---|
| `queries` | string\[] | — | One or more search terms. **Required.** |
| `maxPagesPerQuery` | integer | 1 | Pages to fetch per query (~10 results each). |
| `resultsPerPage` | integer | 10 | See the note under **Limitations** — Google caps this at ~10. |
| `countryCode` | string | `US` | Two-letter country (sets the Google domain + `gl`). |
| `languageCode` | string | `en` | Interface/results language (`hl`). |
| `safeSearch` | `off`/`medium`/`high` | `off` | SafeSearch level. |
| `includeRelated` | boolean | `true` | Extract related searches + People Also Ask. |
| `includeKnowledgePanel` | boolean | `true` | Extract the knowledge panel. |
| `includeAiOverview` | boolean | `true` | Detect AI Overview + extract its cited sources. Needs `appendUdm14: false`. |
| `includeAds` | boolean | `true` | Attempt paid-ad extraction (see **Limitations**). |
| `appendUdm14` | boolean | `true` | Use Google's clean "Web" layout. **Set `false` to see AI Overviews.** Recommended on for max organic stability. |
| `useAdsBotUA` | boolean | `false` | Send the AdsBot-Google user agent. |
| `maxConcurrency` | integer | 5 | Parallel requests. |
| `maxRequestRetries` | integer | 5 | Retries per page before giving up. |
| `proxy` | object | GOOGLE\_SERP | Proxy config. **Leave default** — see **Limitations**. |

#### Advanced search filters

Google search operators exposed as structured fields (all folded into the query /
URL — you can also type the operators directly inside a `queries` entry):

| Field | Type | Operator / param | Notes |
|---|---|---|---|
| `site` | string | `site:` | Restrict to one site. Wins over `relatedToSite`. |
| `relatedToSite` | string | `related:` | Pages related to a site. |
| `wordsInTitle` | string\[] | `intitle:` | Multi-word entries auto-quoted. |
| `wordsInText` | string\[] | `intext:` | Words required in body text. |
| `wordsInUrl` | string\[] | `inurl:` | Words required in the URL. |
| `fileTypes` | string\[] | `filetype:` | OR-combined (`pdf`, `docx`, …). |
| `forceExactMatch` | boolean | `"…"` | Wrap the query in quotes. |
| `searchLanguage` | string | `lr=lang_XX` | Results language (distinct from `hl`). |
| `locationUule` | string | `uule=` | Exact location code. |
| `quickDateRange` | string | `tbs=qdr:` | Relative recency: `h`, `d10`, `w`, `m6`, `y1`. |
| `afterDate` / `beforeDate` | string | `tbs=cdr:` | Absolute bounds (`YYYY-MM-DD` or `MM/DD/YYYY`). |
| `includeUnfilteredResults` | boolean | `filter=0` | Include Google's omitted near-duplicates (off by default). |
| `saveHtml` | boolean | — | Attach raw SERP HTML to each row under `html` (large). |
| `saveHtmlToKeyValueStore` | boolean | — | Save HTML to the KV store; add `htmlSnapshotUrl` to each row. |

#### Lead enrichment (opt-in)

Turn search results into leads. When `enrichEmails: true`, the top business domains
in each SERP get their **own site** scraped for a contact email + people; the hits
are attached to those organic results as `contactEmail` and `emailEnrichment`.

| Field | Type | Default | Notes |
|---|---|---|---|
| `enrichEmails` | boolean | `false` | Enable contact-email enrichment. |
| `maxEnrichedDomainsPerPage` | integer | `3` | Cap unique business domains enriched per page. |

- Best on **business-intent** queries (`"commercial plumbers chicago"`, `"B2B SaaS vendors"`).
- Non-business domains (search engines, social, Wikipedia, big marketplaces) are skipped;
  domains are de-duped per page.
- Best-effort and free (direct site scrape) — enrichment never fails a scrape.

#### Perplexity AI answer (opt-in, bring-your-own-key)

Attach a Perplexity (Sonar) AI answer to each query — grounded, with citations.
This calls Perplexity's **official API with your own key**, so **you are billed by
Perplexity** per request (it is not scraping). The answer lands on the page-1 row
under `perplexity` `{ answer, model, citations, searchResults?, relatedQuestions? }`.

| Field | Type | Default | Notes |
|---|---|---|---|
| `perplexitySearch` | boolean | `false` | Enable the Perplexity answer. |
| `perplexityApiKey` | string (secret) | — | Your key from [docs.perplexity.ai](https://docs.perplexity.ai). Required. |
| `perplexitySearchRecency` | `day`/`week`/`month`/`year` | — | Restrict grounding to recent content. |
| `perplexityRelatedQuestions` | boolean | `false` | Also return follow-up questions. |

Inert unless both the toggle is on **and** a key is supplied; a Perplexity error
never breaks the SERP scrape.

#### Example input

```json
{
  "queries": ["apify pricing 2026", "best web scraping tools"],
  "maxPagesPerQuery": 2,
  "countryCode": "US",
  "languageCode": "en"
}
```

### Output

One dataset item per `(query, page)`:

```json
{
  "query": "apify pricing 2026",
  "page": 1,
  "searchUrl": "http://www.google.com/search?q=apify+pricing+2026&...",
  "organicResults": [
    {
      "type": "organic",
      "position": 1,
      "title": "Apify pricing - plans for data collection at any scale",
      "url": "https://apify.com/pricing",
      "displayedUrl": "https://apify.com › pricing",
      "description": "...",
      "siteLinks": []
    }
  ],
  "paidResults": [],
  "relatedQueries": [{ "title": "apify free plan", "url": "https://www.google.com/search?q=..." }],
  "peopleAlsoAsk": [{ "question": "Is Apify free to use?", "answer": "...", "url": "https://..." }],
  "knowledgePanel": null,
  "aiOverview": { "detected": true, "textAvailable": false, "sources": [{ "url": "https://ibm.com/...", "domain": "ibm.com" }] },
  "totalResults": 1230000,
  "selectorUsed": "div.tF2Cxc",
  "scrapedAt": "2026-06-25T09:00:00.000Z"
}
```

`selectorUsed` tells you which parser variant matched — handy for spotting future
Google layout drift.

### How it works

- **Dual-engine fetch** — tries [`impit`](https://github.com/apify/impit) (Rust,
  real Chrome TLS + HTTP/2 fingerprint) first, falling back to `got-scraping`. Two
  independent fingerprints = two chances against bot detection.
- **`udm=14` clean layout** — by default the scraper requests Google's "Web" tab,
  which strips the AI Overview and returns stable, parseable markup.
- **Self-healing selectors** — organic results run through an 8-deep selector chain
  (2026 containers first, classic layouts as fallback, plus a class-agnostic
  `a:has(h3)` structural net), so a single Google class rename won't zero out a run.
- **Structural soft-block detection** — a page is only treated as blocked if it
  *doesn't* structurally look like a SERP, so benign `/sorry/` footer links never
  cause a false "blocked".

### Limitations & notes

- **Use the default GOOGLE\_SERP proxy.** Google aggressively blocks datacenter and
  residential IPs for SERP scraping (JS-challenge / CAPTCHA interstitials). This
  actor is built around Apify's **GOOGLE\_SERP** proxy group, which is purpose-built
  for Google and applied automatically. **Overriding `proxy` to RESIDENTIAL or
  datacenter groups will fail** with soft-blocks — leave the proxy at its default.

- **Paid ads (`paidResults`) are usually empty.** Google does not serve ads to
  SERP-proxy IP ranges, so the ads container comes back empty regardless of the
  `includeAds` toggle. Reliable ad capture requires a separate ad-specialized proxy
  with retries, which this actor does not include. Treat `includeAds` as
  best-effort. (Organic results, related searches, PAA, and knowledge panels are
  unaffected.)

- **AI Overview prose is not available over the proxy.** Google server-renders the
  AI Overview *shell* (label + citation cards) but generates the answer text
  client-side, so a no-JS request receives a placeholder. `aiOverview` therefore
  returns `detected` + the cited `sources` (useful for GEO/AEO — which domains
  Google cites) with `textAvailable: false`. The prose itself needs a JS-rendering
  path, the same wall as paid ads. Requires `appendUdm14: false` (the default
  `udm=14` strips AI Overviews entirely).

- **`resultsPerPage` is effectively ~10.** Google ignores the `num` parameter on
  most modern SERP layouts and returns ~10 results per page. To collect more, raise
  `maxPagesPerQuery` rather than `resultsPerPage`. Pagination always steps by the
  real page size, so no results are skipped if you set a larger value.

- **Knowledge-panel images** are lazy-loaded by Google's JS, so `imageUrl` may be
  absent even when the panel is otherwise fully populated.

- **People Also Ask answers** are only present when Google pre-expands the card;
  for most queries you'll get the question and destination URL but no answer text.

- **AI Overview** is intentionally excluded by `appendUdm14: true`. Set it to
  `false` if you want the default SERP layout (which can include the AI Overview and
  more ad/PAA surfaces), at the cost of slightly less stable organic markup.

### Tips

- Keep `appendUdm14` on for the most reliable organic extraction.
- For deep coverage, set `maxPagesPerQuery` (e.g. 5) instead of `resultsPerPage`.
- Use `countryCode` + `languageCode` together for accurate localized results
  (e.g. `DE` + `de`, `JP` + `ja`).
- Google search operators work inside `queries` (`site:`, `intitle:`, `"exact"`,
  `OR`, `-exclude`, `filetype:`).

### 🤖 For AI Agents & LLM Apps

Compact reference for AI agents calling this actor via the [Apify MCP server](https://mcp.apify.com) or the Apify API (actor: `memo23/google-search-scraper`).

**Purpose:** Scrape structured Google SERPs for any query, country and language — organic results, related searches, People Also Ask, knowledge panel, AI-Overview cited sources and total counts — with a self-healing selector chain that survives Google's layout rotations.

**Minimal input:**

```json
{
  "queries": ["best web scraping tools"],
  "maxPagesPerQuery": 1
}
```

**Output:** one row per (query, page) — query, page, searchUrl, organicResults {type, position, title, url, displayedUrl, description, siteLinks}, paidResults, relatedQueries {title, url}, peopleAlsoAsk {question, answer, url}, knowledgePanel, aiOverview {detected, textAvailable, sources {url, domain}}, totalResults, selectorUsed, scrapedAt.

**Behaviors an agent should know:**

- `queries` is the only required field; one row per query × page. To collect more results, raise `maxPagesPerQuery` (default 1) — Google caps `resultsPerPage` at ~10 regardless of the value set.
- Leave `proxy` at its default GOOGLE\_SERP group; overriding to residential/datacenter groups fails with soft-blocks.
- `appendUdm14` (default true) gives stable organic markup but strips AI Overview; set it false (with `includeAiOverview`) to get the AI-Overview `sources` — the generated prose is never available over the proxy.
- `paidResults` is usually empty (Google does not serve ads to SERP-proxy IPs); treat `includeAds` as best-effort.
- `enrichEmails` (opt-in, best-effort, free) scrapes top business domains' own sites for `contactEmail`; `perplexitySearch` (opt-in, bring-your-own-key) attaches a Perplexity answer and is billed by Perplexity, not this actor.
- Pay-per-event billing — see the Pricing tab on the actor page.

# Actor input Schema

## `queries` (type: `array`):

One or more search queries. Each query produces one or more SERP rows.

## `resultsPerPage` (type: `integer`):

How many organic results per SERP page (Google's effective cap is 10 on most pages since Sept 2025).

## `maxPagesPerQuery` (type: `integer`):

Paginate through this many SERP pages per query.

## `countryCode` (type: `string`):

Two-letter country code (gl param). Affects which results Google considers locally relevant.

## `languageCode` (type: `string`):

Two-letter language code (hl param). Controls the SERP UI language.

## `safeSearch` (type: `string`):

Safe search level.

## `includeAds` (type: `boolean`):

Parse and return sponsored results alongside organic.

## `includeRelated` (type: `boolean`):

Include 'Related searches' and 'People also ask' boxes.

## `includeKnowledgePanel` (type: `boolean`):

Parse the right-rail knowledge panel if present.

## `includeAiOverview` (type: `boolean`):

Detect the AI Overview block and extract its cited source links (great for GEO/AEO). Note: the generated prose is created client-side and is NOT available over the HTTP proxy — only detection + sources are returned. Requires appendUdm14 = false (udm=14 strips AI Overviews).

## `useAdsBotUA` (type: `boolean`):

Use AdsBot-Google UA to attempt to receive the legacy SERP layout. As of May 2026 Google serves a 'browser unsupported' page to this UA on most IPs — leave OFF unless you have a specific proxy where it works.

## `appendUdm14` (type: `boolean`):

Append \&udm=14 to disable AI Overviews. Stacks with AdsBot UA for maximum parser stability.

## `maxConcurrency` (type: `integer`):

How many queries to process in parallel.

## `maxRequestRetries` (type: `integer`):

Retries per SERP page request when soft-blocked.

## `proxy` (type: `object`):

Apify GOOGLE\_SERP proxy group is strongly recommended for Google scraping.

## `site` (type: `string`):

Limit results to one site, e.g. `example.com` (applies the `site:` operator). Takes precedence over 'Related to site'.

## `relatedToSite` (type: `string`):

Find pages related to a site, e.g. `example.com` (applies the `related:` operator). Ignored if 'Site' is set.

## `wordsInTitle` (type: `array`):

Each entry must appear in the page title (`intitle:`). Multi-word entries are auto-quoted.

## `wordsInText` (type: `array`):

Each entry must appear in the page body text (`intext:`).

## `wordsInUrl` (type: `array`):

Each entry must appear in the page URL (`inurl:`).

## `fileTypes` (type: `array`):

Restrict to file types, OR-combined (e.g. `pdf`, `docx`). Applies the `filetype:` operator.

## `forceExactMatch` (type: `boolean`):

Wrap the query in quotes for an exact-phrase match.

## `searchLanguage` (type: `string`):

Restrict results to a language via the `lr` parameter, e.g. `de` or `lang_de`. Distinct from the `hl` interface language.

## `locationUule` (type: `string`):

Google `uule` code for an exact location. Use a UULE generator to produce it.

## `quickDateRange` (type: `string`):

Relative recency filter (`tbs=qdr:`). Examples: `h` (past hour), `d10` (10 days), `w`, `m6`, `y1`. Wins over Before/After if both set.

## `afterDate` (type: `string`):

Only results after this date. Absolute date `YYYY-MM-DD` or `MM/DD/YYYY`.

## `beforeDate` (type: `string`):

Only results before this date. Absolute date `YYYY-MM-DD` or `MM/DD/YYYY`.

## `includeUnfilteredResults` (type: `boolean`):

Include the near-duplicate results Google normally omits (`filter=0`). Off by default for cleaner output.

## `saveHtml` (type: `boolean`):

Attach the raw SERP HTML to each dataset row under `html`. Makes the dataset large — use for debugging.

## `saveHtmlToKeyValueStore` (type: `boolean`):

Save each SERP's raw HTML to the key-value store and add a `htmlSnapshotUrl` to the dataset row.

## `enrichEmails` (type: `boolean`):

Opt-in, best-effort lead enrichment: for the top business domains in each SERP, scrape the result's OWN site for contact emails and people. Adds `contactEmail` + `emailEnrichment` to those organic results. Skips non-business domains (search/social/Wikipedia/marketplaces). Best on business-intent queries (e.g. 'plumbers chicago', 'B2B SaaS vendors').

## `qualifyByPayment` (type: `boolean`):

Requires "Enrich with contact emails". Scans each business's website — reusing the pages already fetched for email discovery, so no extra cost or time — for payment processors and e-commerce platforms (Stripe, Shopify, PayPal, Paddle, Lemon Squeezy, WooCommerce, Square, Chargebee and more). Adds takesPayments (is this a real paying business?), paymentProcessors (which stack), stripeLiveKey (the public key if exposed) and paymentConfidence. Turn raw contacts into monetization-qualified leads. No extra charge — included with each enriched company.

## `maxEnrichedDomainsPerPage` (type: `integer`):

Cap how many unique business domains to enrich per SERP page (bounds cost/time). Only applies when 'Enrich results with contact emails' is on.

## `perplexitySearch` (type: `boolean`):

Also fetch a Perplexity (Sonar) AI answer per query, attached to the page-1 row as `perplexity`. Bring your own Perplexity API key below — you are billed by Perplexity per request.

## `perplexityApiKey` (type: `string`):

Your Perplexity API key (from docs.perplexity.ai). Required for the Perplexity answer; runs are billed to YOUR Perplexity account. Leave blank to disable.

## `perplexitySearchRecency` (type: `string`):

Restrict Perplexity's web grounding to recent content.

## `perplexityRelatedQuestions` (type: `boolean`):

Ask Perplexity to also return follow-up questions.

## Actor input object example

```json
{
  "queries": [
    "best CRM software 2026",
    "apify scraper alternatives"
  ],
  "resultsPerPage": 10,
  "maxPagesPerQuery": 1,
  "countryCode": "US",
  "languageCode": "en",
  "safeSearch": "off",
  "includeAds": true,
  "includeRelated": true,
  "includeKnowledgePanel": true,
  "includeAiOverview": true,
  "useAdsBotUA": false,
  "appendUdm14": true,
  "maxConcurrency": 5,
  "maxRequestRetries": 5,
  "proxy": {
    "useApifyProxy": true,
    "apifyProxyGroups": [
      "GOOGLE_SERP"
    ]
  },
  "forceExactMatch": false,
  "includeUnfilteredResults": false,
  "saveHtml": false,
  "saveHtmlToKeyValueStore": false,
  "enrichEmails": false,
  "qualifyByPayment": false,
  "maxEnrichedDomainsPerPage": 3,
  "perplexitySearch": false,
  "perplexityRelatedQuestions": false
}
```

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "queries": [
        "best CRM software 2026",
        "apify scraper alternatives"
    ],
    "proxy": {
        "useApifyProxy": true,
        "apifyProxyGroups": [
            "GOOGLE_SERP"
        ]
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("memo23/google-search-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "queries": [
        "best CRM software 2026",
        "apify scraper alternatives",
    ],
    "proxy": {
        "useApifyProxy": True,
        "apifyProxyGroups": ["GOOGLE_SERP"],
    },
}

# Run the Actor and wait for it to finish
run = client.actor("memo23/google-search-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "queries": [
    "best CRM software 2026",
    "apify scraper alternatives"
  ],
  "proxy": {
    "useApifyProxy": true,
    "apifyProxyGroups": [
      "GOOGLE_SERP"
    ]
  }
}' |
apify call memo23/google-search-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "command": "npx",
            "args": [
                "mcp-remote",
                "https://mcp.apify.com/?tools=memo23/google-search-scraper",
                "--header",
                "Authorization: Bearer <YOUR_API_TOKEN>"
            ]
        }
    }
}

```

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/acts/5whUZMSnCaBMa6VpS/builds/DXkK9KixgHri3pCWy/openapi.json