# Rightmove Agents Scraper (`dhrumil/rightmove-agents-scraper`) Actor Scrape rightmove.co.uk to crawl millions of real estate agents from United Kingdom. You can provide multiple search result listings to scrape/monitor. - **URL**: https://apify.com/dhrumil/rightmove-agents-scraper.md - **Developed by:** [Dhrumil Bhankhar](https://apify.com/dhrumil) (community) - **Categories:** Real estate, Lead generation, Automation - **Stats:** 48 total users, 0 monthly users, 100.0% runs succeeded, 1 bookmarks - **User rating**: 5.00 out of 5 stars ## Pricing $40.00/month + usage To use this Actor, you pay a monthly rental fee to the developer. The rent is subtracted from your prepaid usage every month after the free trial period.You also pay for the Apify platform usage, which gets cheaper the higher Apify subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#rental-actors ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ### 🏑 What is Rightmove Agents Scraper? This Rightmove agents scraper will enable you scrape any agent from [rightmove.co.uk](https://www.rightmove.co.uk/). You can simply take your listing url from browser and enter it into this actor. This actor will crawl through all pages of particular listing and generate dataset for you. Listing url is something you get when you perform the search on rightmove site. Example listing urls : - https://www.rightmove.co.uk/estate-agents/find.html?locationIdentifier=OUTCODE%5E1 - https://www.rightmove.co.uk/estate-agents/find.html?locationIdentifier=OUTCODE%5E1&branchType=LETTINGS ### πŸšͺ What can this Rightmove Scraper do? πŸ“ˆ Extract Rightmove market data listings on Rightmove πŸ‘€ This actor is not just scraper but also has monitoring capability. You can turn on monitoring mode and it will give you only newly added agents compared to your previous scrapes. πŸ“© This actor also helps yu to identify which agents are not listed anymore. Please refer to [Identifying delisted agents](#identifying-delisted-agents) ⬇️ Download Rightmove real estate data in Excel, CSV, JSON, and other formats ### πŸ“š How do I start scraping with this scraper? 1. Register for your free Apify account [here](https://apify.com/pricing?fpr=z2di7) 2. You don't need to provide your credit card details for free acount. Just click on "Get Started" button on above link and complete the registration only. 3. Free account comes with reasonable credits to try out this actor. This actor also comes with free trial of 3 days without any commitment/upfront charge. 4. Run this actor and verify the scraped data. Apify has huge integration possibilities. You can download the data or push the data into any 3rd party platform directly. ### 🌳 What Rightmove data can I extract using this tool? | πŸ“ | πŸ“ | | ------------------ | ------------------------------- | | Agent Name | Profile URL | | Branch Name | Whether Letting Agent ? | | Phone | Whether Sales Agent ? | | Direct Phone | Address | | Letting Phone | Sales Phone | | Main Phone | Postcode | | Company Name | Whether Estate Agent ? | | Latitude | Longitude | | Number of Sales Properties | Number of Lettings Properties | | Lettings listing URL | Sales Listing URL | ### ⬇️ Input For simple usecase, you just need to provide browser url of rightmove search result page & that's all. You can leave other fields as they are to be sensible defaults. #### Input example ```json { "listUrls": [ { "url": "https://www.rightmove.co.uk/estate-agents/find.html?locationIdentifier=OUTCODE%5E1" } ], "agentUrls": [ { "url": "https://www.rightmove.co.uk/estate-agents/agent/Savills/Aberdeen-Industrial-214091.html" } ], "monitoringMode": false, "fullAgentDetails" : true, "enableDelistingTracker" : false, "addEmptyTrackerRecord" : false, "deduplicateAtTaskLevel" : false } ```` You can either provide `listUrls` to search properties from or provide `propertyUrls` directly to crawl. Understading monitoring mode : - `monitoringMode` : This option when turned on will only scrape newly added property listings compared to previously scraped properties by this actor. It's important to turn off fullScrape setting if you are using this mode. If you keep fullScrape on, it will re-scrape complete listing again. - `enableDelistingTracker` : This option when turned on will start tracking date against each property under Apify Key Value store. This KV store can be queried later to find out which properties are delisted. - `addEmptyTrackerRecord` : This option when turned on will add empty record having only id of property to Apify dataset. This helps you identify whether property is still listed compared to your own database in incremental mode. - `fullAgentDetails` : By default it will scrape every single agent page to get full details of agent. If you don't need full details but basic details is enough for you, it's **highly encouraged** to turn off this setting. It will run the scraping a lot of faster and save the cost and resources. - `deduplicateAtTaskLevel` : By default scraper will deduplicate and monitor successive updates at account level. In case, you have configured many tasks from this actor and you want de-duplication / successive updates at task level, enble this setting. Enabling this means that two different tasks are treated as two isolated scraping when it comes to monitoring mode. ### ⬆️ Output The scraped data is stored in the dataset of each run. The data can be viewed or downloaded in many popular formats, such as *JSON, CSV, Excel, XML, RSS, and HTML*. #### Output example The result for scraping a single property like this: ```json { "id": "Aberdeen-Industrial-214091.html", "url": "https://www.rightmove.co.uk/estate-agents/agent/Savills/Aberdeen-Industrial-214091.html", "name": "Aberdeen Industrial", "branchName": "Savills, Aberdeen Industrial", "brandName": "Savills", "estateAgent": false, "lettings": true, "sales": true, "phone": "01224 007695", "directPhone": "01224 971111", "address": "37 Albyn Place\r\nAberdeen\r\nAB10 1YN", "lettingsTelephone": "01224 007695", "mainTelephone": "01224 971111", "salesTelephone": "01224 007694", "postcode": "AB10 1YN", "companyName": "Savills ", "coordinates": { "latitude": "57.14267347229752", "longitude": "-2.124575889305664" }, "lettingsProperties": 20, "salesProperties": 6, "lettingsPropertiesUrl": "https://www.rightmove.co.uk/commercial-property-to-let/find/Savills/Aberdeen-Industrial.html?locationIdentifier=BRANCH%5E214091&propertyStatus=all&includeLetAgreed=true&_includeLetAgreed=on", "salesPropertiesUrl": "https://www.rightmove.co.uk/commercial-property-for-sale/find/Savills/Aberdeen-Industrial.html?locationIdentifier=BRANCH%5E214091&includeSSTC=true&_includeSSTC=on" } ``` ### ❓Limitations Since Rightmove allows only 1000 agents per listing/search result, you might want to break down your listing urls into smaller area if it has more than 1K results. Good News is that even if multiple list urls contains overlapping results, they will get deduplicated within same run data. ### 🀝 Related Actors - [Rightmove Scraper](https://apify.com/dhrumil/rightmove-scraper?fpr=z2di7): Scrape sale/rent listings from rightmove.co.uk. - [Zoopla Scraper](https://apify.com/dhrumil/zoopla-scraper?fpr=z2di7): Scrape property sale/rent listings, prices, and details from Zoopla. - [OnTheMarket Scraper](https://apify.com/dhrumil/onthemarket-scraper?fpr=z2di7) : Scrape property sale/rent listings, prices, and details from OnTheMarket. - [Rightmove Full Address & House Number Finder](https://apify.com/dhrumil/rightmove-full-address-house-number-finder?fpr=z2di7): Find full addresses and house numbers for Rightmove listings. - [Rightmove Land Registry Full Address & House Number Finder](https://apify.com/dhrumil/rightmove-landregistry-full-address-house-number-finder?fpr=z2di7): Get full address and house number using Land Registry data for Rightmove properties. - [Rightmove EPC Full Address Finder](https://apify.com/dhrumil/rightmove-epc-full-address-finder?fpr=z2di7): Find EPC and full address details for Rightmove listings. - [Rightmove Commercial Scraper](https://apify.com/dhrumil/rightmove-commercial-scraper?fpr=z2di7): Scrape commercial property listings from Rightmove. - [Rightmove Sold House Properties Scraper](https://apify.com/dhrumil/rightmove-sold-house-properties-scraper?fpr=z2di7): Scrape sold house price data from Rightmove. - [Zoopla Sold Properties Scraper](https://apify.com/dhrumil/zoopla-sold-properties-scraper?fpr=z2di7): Scrape sold property data from Zoopla. - [Zoopla Commercial Properties Scraper](https://apify.com/dhrumil/zoopla-commercial-properties-scraper?fpr=z2di7): Scrape commercial property listings from Zoopla. ### πŸ”Ž Identifying delisted agents This actor provides you monitoring mode configuration using which you can get only incremental updates about newly added properties. In case, you also want to identify which agents are delisted from platform, you can use any of the following techniques with the help of this actor. 1. Running Always without monitoring mode : Run this actor always without monitoring mode enabled and cross check the new incoming batch of data with your existing database. If any property that exists in yoru database but not in newly scraped data batch, that means it's not listed anymore 2. Use Key Value Store generated by scraper : If your are monitoring very large batch of data and you don't want to scrape everything all the time, this method involves bit of technicality but achieves the goal efectively. Apify has storage feature called [Key-value store](https://docs.apify.com/api/v2/#/reference/key-value-stores/key-collection/get-list-of-keys). When you run this scrape, this scraper stores every single property in key value store along with timestamp in `rightmove-agents` store. Inside this store, key is property id itself and value is timestamp like this ``` { lastSeen : '2023-11-02T05:59:25.763Z'} ``` Whenever you run this scraper, it will update the timestamp against particular id if it finds property on the platform. e.g. if we have 2 proprties with id `prop1` and `prop2` and we scraped them both on November 1, key value storage would look like this : ``` prop1 -> { lastSeen : '2023-11-01T05:59:25.763Z'} prop2 -> { lastSeen : '2023-11-01T05:59:25.763Z'} ``` Now if you run this scraper again on December 1 and prop1 is not on the platform anymore but prop2 is still there, key value storage would change like this : ``` prop1 -> { lastSeen : '2023-11-01T05:59:25.763Z'} prop2 -> { lastSeen : '2023-12-01T05:59:25.763Z'} ``` That means if any property has `lastSeen` less than latest batch you loaded, that property is delisted now. You can directly iterate through whole Key value storage using Apify key value storage API to identify this. Please refer to [this](https://docs.apify.com/api/v2/#/reference/key-value-stores/key-collection/get-list-of-keys) API documentation to do the same. Please remember store name generated by this scrape will be `rightmove-agents`. Alternatively, you can iterate through your existing database active agents and use [this](https://docs.apify.com/api/v2/#/reference/key-value-stores/record/get-record) API to identify listing status. For this approach to work, it's important that you enable this feature via `enableDelistingTracker` (Enable Delisting tracker) input. ### πŸ™‹β€β™€οΈ For custom solutions In case you need some custom solution, you can contact me : Or learn more about me on github : # Actor input Schema ## `listUrls` (type: `array`): Any rightmove agent listing url which has list of agents ## `agentUrls` (type: `array`): Any rightmove agent url to scrape ## `monitoringMode` (type: `boolean`): If checked, it will only scrape newly listed agents compared to what has been scraped in previous runs. Please turn off full scrape setting above in case you are tunring this on. ## `deduplicateAtTaskLevel` (type: `boolean`): If this setting is enabled, deduplication and successive updates via monitoring mode works at task level instead of account level. Helpful only if you are using multiple tasks of this scraper ## `fullAgentDetails` (type: `boolean`): If checked, it will scrape each individual agent to get more details about agent. If you need only limited information, turning off this option would scrape a lot more faster. ## `enableDelistingTracker` (type: `boolean`): If checked, it will track every single agent with date so that you can find out which agent is delisted. ( Enabling this has impact on billing ) ## `addEmptyTrackerRecord` (type: `boolean`): If checked, it will add tracker ID record in incremental mode so that you can identify delisted properties if you have custom loader ## `email` (type: `string`): Email address to send notifications or reports to. ## `proxy` (type: `object`): Select proxies to be used by your crawler. ## Actor input object example ```json { "listUrls": [ { "url": "https://www.rightmove.co.uk/estate-agents/find.html?locationIdentifier=OUTCODE%5E1" } ], "agentUrls": [], "monitoringMode": false, "deduplicateAtTaskLevel": false, "fullAgentDetails": true, "enableDelistingTracker": false, "addEmptyTrackerRecord": false, "email": "", "proxy": { "useApifyProxy": true } } ``` # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "listUrls": [ { "url": "https://www.rightmove.co.uk/estate-agents/find.html?locationIdentifier=OUTCODE%5E1" } ], "agentUrls": [], "proxy": { "useApifyProxy": true } }; // Run the Actor and wait for it to finish const run = await client.actor("dhrumil/rightmove-agents-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`πŸ’Ύ Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // πŸ“š Want to learn more πŸ“–? Go to β†’ https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "listUrls": [{ "url": "https://www.rightmove.co.uk/estate-agents/find.html?locationIdentifier=OUTCODE%5E1" }], "agentUrls": [], "proxy": { "useApifyProxy": True }, } # Run the Actor and wait for it to finish run = client.actor("dhrumil/rightmove-agents-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("πŸ’Ύ Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # πŸ“š Want to learn more πŸ“–? Go to β†’ https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "listUrls": [ { "url": "https://www.rightmove.co.uk/estate-agents/find.html?locationIdentifier=OUTCODE%5E1" } ], "agentUrls": [], "proxy": { "useApifyProxy": true } }' | apify call dhrumil/rightmove-agents-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=dhrumil/rightmove-agents-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Rightmove Agents Scraper", "description": "Scrape rightmove.co.uk to crawl millions of real estate agents from United Kingdom. You can provide multiple search result listings to scrape/monitor.", "version": "0.0", "x-build-id": "dmMlmDKAVXagByhZX" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/dhrumil~rightmove-agents-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-dhrumil-rightmove-agents-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/dhrumil~rightmove-agents-scraper/runs": { "post": { "operationId": "runs-sync-dhrumil-rightmove-agents-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/dhrumil~rightmove-agents-scraper/run-sync": { "post": { "operationId": "run-sync-dhrumil-rightmove-agents-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "properties": { "listUrls": { "title": "Agent List URLs", "type": "array", "description": "Any rightmove agent listing url which has list of agents", "items": { "type": "object", "required": [ "url" ], "properties": { "url": { "type": "string", "title": "URL of a web page", "format": "uri" } } } }, "agentUrls": { "title": "Agent URLs", "type": "array", "description": "Any rightmove agent url to scrape", "items": { "type": "object", "required": [ "url" ], "properties": { "url": { "type": "string", "title": "URL of a web page", "format": "uri" } } } }, "monitoringMode": { "title": "Run in monitoring mode where only newly listed agents compared to previous runs will be scraped", "type": "boolean", "description": "If checked, it will only scrape newly listed agents compared to what has been scraped in previous runs. Please turn off full scrape setting above in case you are tunring this on.", "default": false }, "deduplicateAtTaskLevel": { "title": "Deduplicate and monitor at task level (Applicable only if you are running as task)", "type": "boolean", "description": "If this setting is enabled, deduplication and successive updates via monitoring mode works at task level instead of account level. Helpful only if you are using multiple tasks of this scraper", "default": false }, "fullAgentDetails": { "title": "Include Full Agent Details", "type": "boolean", "description": "If checked, it will scrape each individual agent to get more details about agent. If you need only limited information, turning off this option would scrape a lot more faster.", "default": true }, "enableDelistingTracker": { "title": "Enable Delisting tracker with KV Store", "type": "boolean", "description": "If checked, it will track every single agent with date so that you can find out which agent is delisted. ( Enabling this has impact on billing )", "default": false }, "addEmptyTrackerRecord": { "title": "Add tracker ID record in incremental mode", "type": "boolean", "description": "If checked, it will add tracker ID record in incremental mode so that you can identify delisted properties if you have custom loader", "default": false }, "email": { "title": "Email Address (In case of any error or notification)", "type": "string", "description": "Email address to send notifications or reports to.", "default": "" }, "proxy": { "title": "Proxy configuration", "type": "object", "description": "Select proxies to be used by your crawler.", "default": { "useApifyProxy": true } } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```