# Bing Search Scraper (`tri_angle/bing-search-scraper`) Actor Scrape search results from Bing.com. You can get the total number of results, organic results, paid results, people also ask, related queries and more. - **URL**: https://apify.com/tri\_angle/bing-search-scraper.md - **Developed by:** [Tri⟁angle](https://apify.com/tri_angle) (Apify) - **Categories:** SEO tools - **Stats:** 611 total users, 44 monthly users, 100.0% runs succeeded, 11 bookmarks - **User rating**: 4.33 out of 5 stars ## Pricing Pay per event This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ### 🔎 What is a Bing Search Scraper? Our SERP scraper crawls Bing Search Results Pages (SERPs) and extracts data from those web pages in structured formats such as JSON, XML, CSV, or Excel. With this SERP Scraper API, you will be able to extract the following Bing data from each Bing page:
🌱 Organic results
🛍 Paid results
🙋‍♀️ People Also Ask
🔍 Additional attributes
### 💯 How many results can you scrape with Bing Search Scraper? Bing Search Scraper can return **up to 2000 results for one keyword**. This is due to the limitations of Bing itself: although it shows as if it has millions of results for a given search query, Bing will not give you new results past page 200 (if we have 10 results per page). While we regularly run Actor tests to keep the benchmarks in check, the results may also fluctuate without our knowing. The best way to know for sure for your particular use case is to do a test run yourself. ### 💸 How much will scraping Bing cost you? When it comes to scraping, it can be challenging to estimate the resources needed to extract data as use cases may vary significantly. That's why the best course of action is to run a test scrape with a small sample of input data and limited output. You’ll get your price per scrape, which you’ll then multiply by the number of scrapes you intend to do. ### ⬇️ Bing Search Scraper data input The scraper gives you really good control over what kind of Bing Search results you'll get. You can specify the following settings: - Query phrases or raw Bing search URLs 🔍 - Search market 🗺 - Language of search 🇬🇧 - Number of results per page 🔟 #### How to scrape Bing Search results by URL or keyword There are two ways you can scrape Bing search pages: either by URL or by search term. 1. Scraping **by URL** will get you data from Bing Search results page. You'll get Bing data for the query thats included in the URL. You can add as many URLs as you want. 2. Scraping **by search term** will also get you data from Bing Search results page. You can also add as many search terms as you want. ##### Example input for scraping Bing Search search term It is also easy to get Bing search data by search term. Just enter the search term and a number of Bing pages to scrape. With this option, you also can: - scrape by multiple keywords in parallel by adding more search terms and separating them by a new line - indicate how many results you want to see per each Bing page (10-50) - indicate the market of search (locale) and language Here's is example of an input in JSON: ```json { "ignoreSoftBlocking": false, "maxPagesPerQuery": 1, "queries": "apify\ncrawlee", "resultsPerPage": 50, "saveHtml": false, "saveHtmlSnapshotUrls": false, "marketCode": "en-US", "languageCode": "en" } ```` ##### Scrape Bing Search results by URL To input URLs instead, simply replace `queries` with full URLs: ```json "queries": "https://www.bing.com/search?q=apify&form=QBLH&sp=-1&ghc=1&lq=0&pq=apify&sc=11-5&qs=n&sk=&cvid=E54654768E75474GA4316BFD06FF2CA715&ghsh=0&ghacc=0&ghpl=", ``` Note that the Actor will extract only the query from the URL (in this example, "apify"), while all other settings (e.g., language) will be taken from your input. ### ⬆️ Bing Search data output The scraper stores its result in the default [dataset](https://apify.com/docs/storage#dataset) associated with the scraper run, from which you can export it to various formats, such as JSON, XML, CSV, or Excel. #### Output example For each Bing Search results page, the dataset will contain a single record. Here is example in JSON: ```json { "searchQuery": { "term": "apify", "resultsPerPage": 10, "page": 1, "url": "https://www.bing.com/search?q=apify&mkt=en-US&setLang=en&count=10&first=1", "marketCode": "en-US", "languageCode": "en" }, "html": null, "htmlSnapshotUrl": null, "resultsTotal": 30100, "organicResults": [ { "iconUrl": "https://th.bing.com/th?id=ODLS.f0918ea1-202b-4bbd-abfe-46cd7093840f&w=32&h=32&qlt=90&pcl=fffffa&o=6&cb=13&pid=1.2", "displayedUrl": "https://apify.com", "title": "Apify: Full-stack web scraping and data extraction platform", "url": "https://apify.com/", "description": "Cloud platform for web scraping, browser automation, and data for AI. Use 2,000+ ready-made tools, code templates, or order a custom solution.", "emphasizedKeywords": ["web"], "date": null, "type": "organic", "position": 1, "productInfo": {}, "siteLinks": [ { "url": "https://apify.com/pricing", "title": "Plans & Pricing", "description": "Plans and pricing" } // ... and more ] } // ... and many more ], "paidResults": [], "peopleAlsoAsk": [ { "url": "https://docs.apify.com/platform", "question": "What is apify and how does it work?", "answer": "Apify is a cloud platform that helps you build reliable web scrapers, fast, and automate anything you can do manually in a web browser. Actors are serverless cloud programs running on the Apify platform that can easily crawl websites with millions of pages, but also perform arbitrary computing jobs such as sending emails or data transformations." } // ... and more ], "relatedQueries": [ { "title": "apify login", "url": "https://www.bing.com/search?q=apify+login&FORM=QSRE1" } // ... and more ] } ```
You can **download the results directly from the platform using a button** or from the [Get dataset items](https://www.apify.com/docs/api/v2#/reference/datasets/item-collection/get-items) API endpoint: ```json ``` where `[DATASET_ID]` is the ID of the dataset and `[FORMAT]`can be `csv`, `html`, `xlsx`, `xml`, `rss` or `json`. ### 🔒 Soft Blocking in Bing Scraper When scraping Bing, you may encounter soft blocking, where Bing gently degrades the quality of results rather than issuing a hard error. Typical signs include: - Limited results: Bing may ignore requests for more than the default count and only return 10 results on page 1, regardless of your resultsPerPage setting. - Empty pages: Sometimes Bing displays a page as having no results, even though there are results for your query. - Unexpected results: Occasionally, Bing may return results unrelated to your search terms when soft blocking kicks in. This actor automatically retries whenever it detects soft blocking and, by default, fires multiple concurrent requests with different extra url parameters for page 1 to improve efficiency. Once it successfully fetches a non soft blocked page, it cancels any other queued requests for that page that haven’t yet begun processing. If you’d like to disable these features, you can by disabling `ignoreSoftBlocking`. ### ❓FAQ #### 📚 Resources on how to use Bing SERP API to scrape Bing - [Is web scraping legal?](https://blog.apify.com/is-web-scraping-legal/) - blogpost on ethical scraping. - [Platform pricing page](https://apify.com/pricing/actors) with pricing specifications. ##### How do I scrape millions of results from Bing? Please note that, although Bing always shows that it has found millions of results, **Bing will never display more than a few thousands results for a single search query** 🤥 If your goal is to get as many results as possible, try creating many similar queries and combine different parameters and locations. ##### Is it legal to scrape Bing search results? Web scraping is legal if you are extracting publicly available data, but you should respect boundaries such as personal data and intellectual property regulations. Rule of thumb is: you should only scrape personal data if you have a legitimate reason to do so, factoring in Bing's [Terms of Use](https://learn.microsoft.com/en-us/microsoftsearch/security-for-search) as well. If you're unsure whether your reason is legitimate, consult your lawyers. We also recommend that you read our blog post on the subject: [is web scraping legal?](https://blog.apify.com/is-web-scraping-legal/) ##### How can I use data scraped from Bing Search? Bing SERP API has a lot to offer in terms of how extracted Bing data can be applied: 🔸 **Use it for search engine optimization (SEO)** and monitor how your website performs on Bing for certain queries over time. 🔹 **Monitor how frequently a search term has been used** on Bing, and how it compares with total search volume. 🔸 **Analyze display ads** for a given set of keywords. 🔹 **Monitor your competition** in both organic and paid results. 🔸 **Build a URL list for certain keywords**. This is useful if, for example, you need good relevant starting points when scraping web pages containing specific phrases. 🔹 **Analyze Bing algorithm** and identify its main trends ##### Can I use Bing Scraper with the API? The Apify API gives you programmatic access to the Apify platform. The API is organized around RESTful HTTP endpoints that enable you to manage, schedule, and run Apify Actors. The API also lets you access any datasets, monitor actor performance, fetch results, create and update versions, and more. To access the API using Node.js, use the `apify-client` NPM package. To access the API using Python, use the `apify-client` PyPi package. # Actor input Schema ## `queries` (type: `string`): Enter one or more keywords or full search URLs. ## `resultsPerPage` (type: `integer`): How many results to fetch on each page. More results per page can speed up total crawl time but may take longer per request. ## `maxPagesPerQuery` (type: `integer`): Maximum number of pages to retrieve for each query. ## `marketCode` (type: `string`): Select the market for your search results. You can also use a lowercase country code (e.g. "us"), which will be automatically mapped to the appropriate market code (e.g. "en-US"). ## `languageCode` (type: `string`): Select language for search results. If left blank, it will default to the language associated with the selected market. ## `saveHtml` (type: `boolean`): Store the raw HTML of each results page in the output. Useful for debugging or custom parsing, but increases dataset size. ## `saveHtmlSnapshotUrls` (type: `boolean`): Save each page's HTML to the key-value store and include snapshot URLs in the output. ## `ignoreSoftBlocking` (type: `boolean`): If enabled, the crawler creates a single request for the first page and does not retry on soft blocked results. If disabled, the crawler will try to detect soft blocking and automatically retry. It will also create multiple requests for the first page with different extra URL parameters until it obtains a full, valid response. This may significantly increase runtime due to multiple attempts. ## Actor input object example ```json { "queries": "apify\nhttps://www.bing.com/search?q=crawlee", "resultsPerPage": 10, "maxPagesPerQuery": 1, "marketCode": "", "languageCode": "", "saveHtml": false, "saveHtmlSnapshotUrls": false, "ignoreSoftBlocking": false } ``` # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "queries": `apify https://www.bing.com/search?q=crawlee` }; // Run the Actor and wait for it to finish const run = await client.actor("tri_angle/bing-search-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "queries": """apify https://www.bing.com/search?q=crawlee""" } # Run the Actor and wait for it to finish run = client.actor("tri_angle/bing-search-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "queries": "apify\\nhttps://www.bing.com/search?q=crawlee" }' | apify call tri_angle/bing-search-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=tri_angle/bing-search-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Bing Search Scraper", "description": "Scrape search results from Bing.com. You can get the total number of results, organic results, paid results, people also ask, related queries and more.", "version": "0.0", "x-build-id": "aYpQOVFrFzaIeWy2y" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/tri_angle~bing-search-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-tri_angle-bing-search-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/tri_angle~bing-search-scraper/runs": { "post": { "operationId": "runs-sync-tri_angle-bing-search-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/tri_angle~bing-search-scraper/run-sync": { "post": { "operationId": "run-sync-tri_angle-bing-search-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "required": [ "queries" ], "properties": { "queries": { "title": "Search term(s)", "pattern": "[^\\s]+", "type": "string", "description": "Enter one or more keywords or full search URLs." }, "resultsPerPage": { "title": "Results per page", "minimum": 1, "maximum": 50, "type": "integer", "description": "How many results to fetch on each page. More results per page can speed up total crawl time but may take longer per request.", "default": 10 }, "maxPagesPerQuery": { "title": "Max pages per search", "minimum": 1, "type": "integer", "description": "Maximum number of pages to retrieve for each query.", "default": 1 }, "marketCode": { "title": "Market", "enum": [ "", "es-AR", "en-AU", "de-AT", "nl-BE", "fr-BE", "pt-BR", "en-CA", "fr-CA", "es-CL", "da-DK", "fi-FI", "fr-FR", "de-DE", "zh-HK", "en-IN", "en-ID", "it-IT", "ja-JP", "ko-KR", "en-MY", "es-MX", "nl-NL", "en-NZ", "no-NO", "zh-CN", "pl-PL", "en-PH", "ru-RU", "en-ZA", "es-ES", "sv-SE", "fr-CH", "de-CH", "zh-TW", "tr-TR", "en-GB", "en-US", "es-US", "ar", "au", "at", "be", "br", "ca", "cl", "dk", "fi", "fr", "de", "hk", "in", "id", "it", "jp", "kr", "my", "mx", "nl", "nz", "no", "cn", "pl", "ph", "ru", "za", "es", "se", "ch", "tw", "tr", "gb", "us" ], "type": "string", "description": "Select the market for your search results. You can also use a lowercase country code (e.g. \"us\"), which will be automatically mapped to the appropriate market code (e.g. \"en-US\").", "default": "" }, "languageCode": { "title": "Language", "enum": [ "", "ar", "eu", "bn", "bg", "ca", "zh-hans", "zh-hant", "hr", "cs", "da", "nl", "en", "en-gb", "et", "fi", "fr", "gl", "de", "gu", "he", "hi", "hu", "is", "it", "jp", "kn", "ko", "lv", "lt", "ms", "ml", "mr", "nb", "pl", "pt-br", "pt-pt", "pa", "ro", "ru", "sr", "sk", "sl", "es", "sv", "ta", "te", "th", "tr", "uk", "vi" ], "type": "string", "description": "Select language for search results. If left blank, it will default to the language associated with the selected market.", "default": "" }, "saveHtml": { "title": "Save HTML to output", "type": "boolean", "description": "Store the raw HTML of each results page in the output. Useful for debugging or custom parsing, but increases dataset size.", "default": false }, "saveHtmlSnapshotUrls": { "title": "Save HTML snapshot URLs to output", "type": "boolean", "description": "Save each page's HTML to the key-value store and include snapshot URLs in the output.", "default": false }, "ignoreSoftBlocking": { "title": "Ignore soft blocking", "type": "boolean", "description": "If enabled, the crawler creates a single request for the first page and does not retry on soft blocked results. If disabled, the crawler will try to detect soft blocking and automatically retry. It will also create multiple requests for the first page with different extra URL parameters until it obtains a full, valid response. This may significantly increase runtime due to multiple attempts.", "default": false } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```