scraperapi-mcp-server
OfficialThe ScraperAPI MCP server enables web scraping, structured data extraction, site crawling, and AI-powered parsing — all bypassing anti-bot protections (CAPTCHAs, IP blocks).
General Web Scraping
Fetch any web page or image with optional JavaScript rendering, geo-targeting, premium proxies, device emulation, and output formats (markdown, text, CSV, JSON)
Structured Data Endpoints (SDEs) Pre-parsed, structured data from major platforms — no manual HTML parsing needed:
Google: Search (SERP), News, Jobs, Shopping, Maps
Amazon: Product details (by ASIN), search results, seller offers
Walmart: Search, product details (by ID), category browsing, customer reviews
eBay: Search, product details (by item ID)
Redfin: For-sale/rental property listings, search results, agent profiles
Web Crawling
Start asynchronous crawl jobs from a seed URL, following links to a configurable depth or credit budget
Control URL inclusion/exclusion via regex, receive results via webhooks, and poll or cancel jobs
AI-Powered Custom Parsing
Create reusable AI parsers from 1–3 example URLs to extract structured JSON from similar pages
Manage parsers: create, retrieve status/details, apply to new URLs, list, update fields, or delete
Configuration
Behavior is customizable via environment variables: API key, request timeout, rate limits, and image size limits
Provides tools for retrieving structured product details, search results, and seller offers from Amazon, enabling programmatic product research and price monitoring.
Provides tools for retrieving structured listing details and search results from eBay, useful for product discovery and market research.
Provides tools for retrieving structured Google Search, Jobs, and Shopping results, enabling SERP analysis, job listing aggregation, and product price comparison.
Provides a tool for retrieving structured local business/place results from Google Maps based on a query and geographic coordinates.
Provides a tool for retrieving structured news article results from Google News, with support for date filtering and time windows.
Provides tools for retrieving structured product details, search results, category browsing, and customer reviews from Walmart, enabling product research and sentiment analysis.
ScraperAPI MCP server
The ScraperAPI MCP server enables LLM clients to retrieve and process web scraping requests using the ScraperAPI services.
This is the self-hosted (local) server. A hosted (remote) version is also available.
Table of Contents
Related MCP server: ma-browser
Features
Full implementation of the Model Context Protocol specification
Seamless integration with ScraperAPI for web scraping
Simple setup with Python or Docker
Architecture
┌───────────────┐ ┌───────────────────────┐ ┌───────────────┐
│ LLM Client │────▶│ Scraper MCP Server │────▶│ AI Model │
└───────────────┘ └───────────────────────┘ └───────────────┘
│
▼
┌──────────────────┐
│ ScraperAPI API │
└──────────────────┘Installation
The ScraperAPI MCP Server is designed to run as a local server on your machine, your LLM client will launch it automatically when configured.
Prerequisites
Python 3.11+
Docker (optional)
Using Python
Install the package:
pip install scraperapi-mcp-serverAdd this to your client configuration file:
{
"mcpServers": {
"ScraperAPI": {
"command": "python",
"args": ["-m", "scraperapi_mcp_server"],
"env": {
"API_KEY": "<YOUR_SCRAPERAPI_API_KEY>"
}
}
}
}Using Docker
Add this to your client configuration file:
{
"mcpServers": {
"ScraperAPI": {
"command": "docker",
"args": [
"run",
"-i",
"-e",
"API_KEY=${API_KEY}",
"--rm",
"scraperapi-mcp-server"]
}
}
}If your command is not working (for example, you see a package not found error when trying to start the server), double-check the path you are using. To find the correct path, activate your virtual environment first, then run:
which <YOUR_COMMAND>Available tools
API
Retrieve the content of a web page, or download an image, from a URL.
Parameter | Type | Description | Required |
| string | Target URL to scrape | Yes |
| boolean | Enable JavaScript rendering for dynamic pages (default: | No |
| string | ISO 2-letter country code for geo-targeting | No |
| boolean | Use premium residential/mobile proxies (default: | No |
| boolean | Advanced anti-bot bypass; incompatible with | No |
| string |
| No |
| string |
| No |
| boolean | Auto-parse supported sites into structured data (default: | No |
Returns: the scraped content as a string, or image data for image URLs.
SDEs
Structured Data Endpoints (SDEs) return pre-parsed JSON (or CSV) instead of raw HTML. Every SDE tool also accepts output_format (json by default, or csv), tld, and country_code, shown in each tool's table.
Parameter | Type | Description | Required |
| string | Search query, as typed into Google | Yes |
| integer | Number of results to return on the page | No |
| integer | Zero-based result offset for pagination | No |
| string | Interface/host language code (e.g. | No |
| string | Country edition to search (2-letter code) | No |
| string | Google | No |
| string | Start of a custom date range, | No |
| string | End of a custom date range, | No |
| string | Recent window: | No |
| boolean | Include raw HTML alongside parsed data (default: | No |
| string | Raw Google | No |
| string |
| No |
| string | Top-level domain (e.g. | No |
| string | ISO 2-letter country code for geo-targeting | No |
Parameter | Type | Description | Required |
| string | Search query | Yes |
| integer | Number of results to return on the page | No |
| integer | Zero-based result offset for pagination | No |
| string | Interface/host language code | No |
| string | Country edition to search (2-letter code) | No |
| string | Google | No |
| string | Start of a custom date range, | No |
| string | End of a custom date range, | No |
| string | Recent window: | No |
| string |
| No |
| string | Top-level domain (e.g. | No |
| string | ISO 2-letter country code for geo-targeting | No |
Parameter | Type | Description | Required |
| string | Search query | Yes |
| integer | Number of results to return on the page | No |
| integer | Zero-based result offset for pagination | No |
| string | Interface/host language code | No |
| string | Country edition to search (2-letter code) | No |
| string | Google | No |
| string |
| No |
| string | Top-level domain (e.g. | No |
| string | ISO 2-letter country code for geo-targeting | No |
Parameter | Type | Description | Required |
| string | Search query | Yes |
| integer | Number of results to return on the page | No |
| integer | Zero-based result offset for pagination | No |
| string | Interface/host language code | No |
| string | Country edition to search (2-letter code) | No |
| string | Google | No |
| boolean | Include raw HTML alongside parsed data (default: | No |
| string |
| No |
| string | Top-level domain (e.g. | No |
| string | ISO 2-letter country code for geo-targeting | No |
Parameter | Type | Description | Required |
| string | Search query (e.g. | Yes |
| number | Latitude of the map center, in decimal degrees | Yes |
| number | Longitude of the map center, in decimal degrees | Yes |
| integer | Map zoom level (roughly 3=country, 10=city, 15=street) | No |
| boolean | Include raw HTML alongside parsed data (default: | No |
| string |
| No |
| string | Top-level domain (e.g. | No |
| string | ISO 2-letter country code for geo-targeting | No |
Amazon
Parameter | Type | Description | Required |
| string | 10-character Amazon product identifier (e.g. | Yes |
| string | Language code for localized content (e.g. | No |
| boolean | Include raw HTML alongside parsed data (default: | No |
| string |
| No |
| string | Amazon TLD (e.g. | No |
| string | ISO 2-letter country code for geo-targeting | No |
Parameter | Type | Description | Required |
| string | Search query, as typed into Amazon | Yes |
| integer | 1-based results page number | No |
| string | Sort order (e.g. | No |
| string | Restrict search to a department/category (e.g. | No |
| string | Amazon | No |
| string | Language code for localized content | No |
| string |
| No |
| string | Amazon TLD (e.g. | No |
| string | ISO 2-letter country code for geo-targeting | No |
Parameter | Type | Description | Required |
| string | 10-character Amazon product identifier | Yes |
| string | Comma-separated condition filters (e.g. | No |
| boolean | Include only New-condition offers | No |
| boolean | Include Used - Like New offers | No |
| boolean | Include Used - Very Good offers | No |
| boolean | Include Used - Good offers | No |
| boolean | Include Used - Acceptable offers | No |
| string | Language code for localized content | No |
| string |
| No |
| string | Amazon TLD (e.g. | No |
| string | ISO 2-letter country code for geo-targeting | No |
Walmart
Parameter | Type | Description | Required |
| string | Product search query, as typed into Walmart | Yes |
| integer | 1-based results page number | No |
| string |
| No |
| string | Walmart TLD (e.g. | No |
| string | ISO 2-letter country code for geo-targeting | No |
Parameter | Type | Description | Required |
| string | Walmart product ID | Yes |
| string |
| No |
| string | Walmart TLD (e.g. | No |
| string | ISO 2-letter country code for geo-targeting | No |
Parameter | Type | Description | Required |
| string | Walmart category ID | Yes |
| integer | 1-based results page number | No |
| string |
| No |
| string | Walmart TLD (e.g. | No |
| string | ISO 2-letter country code for geo-targeting | No |
Parameter | Type | Description | Required |
| string | Walmart product ID | Yes |
| integer | 1-based results page number | No |
| string | Sort order (e.g. | No |
| string | Comma-separated star ratings to filter by (e.g. | No |
| boolean | Include only reviews with a "Verified Purchase" badge (default: | No |
| string |
| No |
| string | Walmart TLD (e.g. | No |
| string | ISO 2-letter country code for geo-targeting | No |
eBay
Parameter | Type | Description | Required |
| string | Search keywords | Yes |
| integer | 1-based results page number | No |
| integer | Number of items per page | No |
| string | Restrict results to a specific seller | No |
| string | Comma-separated: | No |
| string |
| No |
| string | Comma-separated: | No |
| string |
| No |
| string |
| No |
| string | eBay TLD (e.g. | No |
| string | ISO 2-letter country code for geo-targeting | No |
Parameter | Type | Description | Required |
| string | eBay item ID | Yes |
| string |
| No |
| string | eBay TLD (e.g. | No |
| string | ISO 2-letter country code for geo-targeting | No |
Redfin
Every Redfin tool takes a full Redfin URL matching the tool (property, search, or agent page).
Parameter | Type | Description | Required |
| string | Full Redfin for-sale property URL | Yes |
| boolean | Return raw extracted JSON instead of parsed data (default: | No |
| string |
| No |
| string | Redfin TLD ( | No |
| string | ISO 2-letter country code for geo-targeting | No |
Parameter | Type | Description | Required |
| string | Full Redfin rental property URL | Yes |
| boolean | Return raw extracted JSON instead of parsed data (default: | No |
| string |
| No |
| string | Redfin TLD ( | No |
| string | ISO 2-letter country code for geo-targeting | No |
Parameter | Type | Description | Required |
| string | Full Redfin search-results URL (with filters) | Yes |
| string |
| No |
| string | Redfin TLD ( | No |
| string | ISO 2-letter country code for geo-targeting | No |
Parameter | Type | Description | Required |
| string | Full Redfin agent profile URL | Yes |
| string |
| No |
| string | Redfin TLD ( | No |
| string | ISO 2-letter country code for geo-targeting | No |
Crawler
The crawler is asynchronous: crawler_job_start returns a job id immediately, then you poll crawler_job_status until the crawl finishes. Per-page results can also be pushed to a callback_url webhook.
Parameter | Type | Description | Required |
| string | The URL where crawling begins (depth 0) | Yes |
| string | Regex selecting which links to follow. Use | Yes |
| integer | Maximum crawl depth (start URL is depth 0). Provide | No* |
| integer | Maximum ScraperAPI credits the crawl may consume. Provide | No* |
| string | Regex for URLs to exclude from crawling | No |
| object | Per-scrape controls applied to each page (e.g. | No |
| string | Webhook URL to receive per-page results and the final summary | No |
| object | Arbitrary metadata to attach to the job | No |
| object | Recurring schedule: | No |
| boolean | For scheduled projects, whether the schedule is enabled (default: | No |
* Provide either max_depth or crawl_budget.
Returns: JSON with the job id and initial status (e.g. {"status": "initiated", "jobId": "..."}).
Parameter | Type | Description | Required |
| string | The job id returned by | Yes |
Returns: JSON with the job's page counts (done/failed/active).
Parameter | Type | Description | Required |
| string | The job id returned by | Yes |
AI Parser
Build a reusable parser from a few example URLs, then apply it to any similar page for structured extraction. Two-phase: ai_parser_create returns a parser id immediately and generation runs in the background — poll ai_parser_get_details until its status is FINISHED, then call ai_parser_parse_url.
Parameter | Type | Description | Required |
| string | A name for the parser | Yes |
| array<string> | 1–3 example URLs of the same page type (same structure) | Yes |
| object | ScraperAPI fetch options for the example pages: | No |
| array | Pre-declared fields to extract: | No |
Returns: JSON with the new parser's id and version. Generation is asynchronous — poll ai_parser_get_details until status is FINISHED.
Parameter | Type | Description | Required |
| string | The parser id returned by | Yes |
| integer | Specific parser version (default: latest) | No |
Returns: JSON with the parser's status (GENERATING/FINISHED/FAILED), fields, and example results.
Parameter | Type | Description | Required |
| string | The parser id to run | Yes |
| string | The URL to scrape and parse | Yes |
| integer | Specific parser version (default: latest) | No |
Returns: JSON {"parser": ..., "version": ..., "result": {...}}. Costs 1 credit per call.
No parameters. Returns a JSON array of parser summaries (id, name, status, version, generation time).
Parameter | Type | Description | Required |
| string | The parser id to delete | Yes |
Parameter | Type | Description | Required |
| string | The parser id to update | Yes |
| integer | Version to base the update on (default: latest) | No |
| array | Fields to add: | No |
| array | Fields to modify (triggers async regeneration) | No |
| array | Renames: | No |
| array<string> | Field names to remove (applied immediately) | No |
Prompt templates
Please scrape this URL
<URL>. If you receive a 500 server error identify the website's geo-targeting and add the corresponding country_code to overcome geo-restrictions. If errors continues, upgrade the request to use premium proxies by adding premium=true. For persistent failures, activate ultra_premium=true to use enhanced anti-blocking measures.Can you scrape URL
<URL>to extract<SPECIFIC_DATA>? If the request returns missing/incomplete<SPECIFIC_DATA>, set render=true to enable JS Rendering.
Configuration
Settings
Configure the server through environment variables. Only API_KEY is required.
Variable | Default | Description |
| — | Required. Your ScraperAPI API key. |
|
| Per-request timeout, in seconds. |
|
| Maximum tool calls allowed per rate-limit window. |
|
| Length of the rate-limit window, in seconds. |
|
| Maximum size of an image the |
Client Setup
Use the JSON configuration file from the Installation section. Below are steps for common clients.
Claude Desktop:
Open Claude Desktop and click the settings icon
Select the "Developer" tab
Click "Edit Config" and paste the JSON configuration file
Claude Code:
Add the server manually to your
.claude/settings.jsonwith the JSON configuration file, or run:claude mcp add scraperapi -e API_KEY=<YOUR_SCRAPERAPI_API_KEY> -- python -m scraperapi_mcp_server
Open Cursor
Access the Settings Menu
Open Cursor Settings
Go to Tools & Integrations section
Click '+ Add MCP Server'
Choose Manual and paste the JSON configuration file
More here
Open Windsurf
Access the Settings Menu
Click on the Cascade settings
Click on the MCP server section
Click on the gear icon, the
mcp_config.jsonfile will open
More here
Open VS Code and click the Cline icon in the activity bar to open the Cline panel
Click the MCP Servers icon in the top navigation bar of the Cline pane
Select the "Configure" tab
Click "Configure MCP Servers" at the bottom of the pane — this opens
cline_mcp_settings.json
More here
Development
Local setup
Clone the repository:
git clone https://github.com/scraperapi/scraperapi-mcp cd scraperapi-mcpInstall dependencies:
Using Poetry:
poetry installUsing pip:
# Create virtual environment and activate it python -m venv .venv source .venv/bin/activate # MacOS/Linux # OR .venv/Scripts/activate # Windows # Install the local package in editable mode pip install -e .Using Docker:
# Build the Docker image locally docker build -t scraperapi-mcp-server .
Run the server
Using Python:
python -m scraperapi_mcp_serverUsing Docker:
# Run the Docker container with your API key docker run -e API_KEY=<YOUR_SCRAPERAPI_API_KEY> scraperapi-mcp-server
Debug
python3 -m scraperapi_mcp_server --debugTesting
This project uses pytest for testing.
Install Test Dependencies
Using Poetry:
poetry install --with devUsing pip:
pip install -e . pip install pytest pytest-mock pytest-asyncio
Running Tests
# Run All Tests
pytest
# Run Specific Test
pytest <TEST_FILE_PATH>Maintenance
Resources
Unclaimed servers have limited discoverability.
Looking for Admin?
If you are the server author, to access and configure the admin panel.
Related MCP Servers
- AlicenseBqualityDmaintenanceA local web scraping MCP server with RAG capabilities that provides intelligent web search, content extraction, and screenshot tools without requiring API keys.Last updated428MIT
- Alicense-qualityDmaintenanceMCP server that lets AI agents use your real browser as an API, accessing any website with your login state, no keys or scrapers needed.Last updatedMIT
- Alicense-qualityDmaintenanceAn MCP server that gives AI tools a reliable way to fetch content from the web, handling anti-bot protection and JavaScript-rendered pages.Last updated836MIT
Related MCP Connectors
Driflyte MCP server which lets AI assistants query topic-specific knowledge from web and GitHub.
Hosted MCP server connecting claude.ai, ChatGPT and other AI apps to your own computer
An MCP server that gives your AI access to the source code and docs of all public github repos
Latest Blog Posts
- Who's Calling? MCP Hosts Are an Identity Blind Spot (And the Spec Knows It)By Om-Shree-0709 on .mcpAgent IdentityOAuth 2.1
- Your AI Chatbot Just Exposed Your CEO's Salary to an InternBy Om-Shree-0709 on .Agent IdentityMCP SecurityOAuth Delegation
- Why MCP Servers Need Execution Sandboxing (And Why Your Current Stack Isn't Enough)By Om-Shree-0709 on .Agentic AiPrompt InjectionWebAssembly
MCP directory API
We provide all the information about MCP servers via our MCP API.
curl -X GET 'https://glama.ai/api/mcp/v1/servers/scraperapi/scraperapi-mcp'
If you have feedback or need assistance with the MCP directory API, please join our Discord server