# Rew Agent Scraper (`getdataforme/rew-agent-scraper`) Actor Scrape comprehensive real estate agent profiles from REW, including contact details, experience, awards, and recommendations. Ideal for market research, lead generation, and competitive analysis.... - **URL**: https://apify.com/getdataforme/rew-agent-scraper.md - **Developed by:** [GetDataForMe](https://apify.com/getdataforme) (community) - **Categories:** Agents, Lead generation, Real estate - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $10.00 / 1,000 results This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Rew Agent Scraper ### Introduction The Rew Agent Scraper is a powerful Apify Actor designed to extract detailed information about real estate agents from REW (Real Estate Web) listings. It efficiently scrapes agent profiles, including contact details, experience, languages, awards, and recommendations, enabling users to gather comprehensive data for market research, lead generation, and competitive analysis. This tool leverages CheerioCrawler for fast, reliable scraping while respecting website terms and using proxies to ensure ethical and uninterrupted data collection. ### Features - **Comprehensive Data Extraction**: Scrapes key agent details such as name, role, brokerage, experience, languages, awards, recommendations, phone number, and profile image. - **Customizable Search**: Allows users to specify search keywords to target specific locations or agent types on REW. - **Proxy Support**: Integrates with Apify Proxy for residential IPs to avoid IP bans and ensure high success rates. - **Scalable Crawling**: Configurable request limits to control crawl depth and manage API costs. - **High-Quality Output**: Delivers structured JSON data that's easy to integrate into databases, spreadsheets, or analytics tools. - **Error-Resilient**: Built-in handling for common scraping challenges like rate limits and page changes. - **Fast Performance**: Utilizes CheerioCrawler for efficient HTML parsing and quick data retrieval. ### Input Parameters | Parameter | Type | Required | Description | Example | |-----------|------|----------|-------------|---------| | searchKeyword | string | Yes | The search keyword to find agents on REW, such as a location or agent name. | "newmarket-on" | | maxLimit | integer | No | Maximum number of requests the crawler can make during the run. Helps control costs and avoid overloading the site. | 100 | | proxyConfiguration | object | Yes | Configuration for proxy servers to mask the scraper's IP address. Recommended to use Apify Proxy with residential groups for better success. | {"useApifyProxy": true, "apifyProxyGroups": ["RESIDENTIAL"]} | ### Example Usage To run the Rew Agent Scraper, provide the following input JSON: ```json { "searchKeyword": "newmarket-on", "maxLimit": 100, "proxyConfiguration": { "useApifyProxy": true, "apifyProxyGroups": ["RESIDENTIAL"] } } ```` This will scrape agent data from REW based on the keyword. The output will be a JSON array of agent objects, like this: ```json [ { "url": "https://www.rew.ca/agents/84235/dave-elfassy", "agentId": "84235", "name": "Elfassy, Dave", "role": "Broker", "brokerage": "SUTTON GROUP-ADMIRAL REALTY INC.", "experience": "19 years experience", "languages": "Also speaks Cantonese, French, Mandarin, Russian, Slovenian, Italian, Spanish, Hebrew, Arabic, Persian, Portuguese", "awards": "6 awards", "recommendations": "812 Recommendations", "phone": "4168991199", "image": "https://assets.rew.ca/property-agent/image/84235/80bcd10a1af61fef.jpg?auto=format&trim=&fit=crop&w=150&h=150&mask=ellipse&crop=faces&fm=png", "scrapedFrom": "https://www.rew.ca/agents/areas/newmarket-on" } ] ``` ### Use Cases - **Market Research**: Analyze agent distributions, experience levels, and specializations in specific regions to identify trends. - **Lead Generation**: Collect contact information for outreach to potential clients or partnerships in real estate. - **Competitive Intelligence**: Monitor competitors' agents, their awards, and recommendations to benchmark performance. - **Price Monitoring**: Track agent profiles for changes in brokerage or roles, aiding in pricing strategies. - **Content Aggregation**: Build databases of real estate professionals for directories or marketing campaigns. - **Academic Research**: Study language diversity, experience correlations, or award patterns in real estate markets. ### Installation and Usage 1. Search for "Rew Agent Scraper" in the Apify Store. 2. Click "Try for free" or "Run". 3. Configure input parameters as described above. 4. Click "Start" to begin extraction. 5. Monitor progress in the log. 6. Export results in your preferred format (JSON, CSV, Excel). ### Output Format The Actor outputs a JSON array of objects, each representing an agent profile. Key fields include: - `url`: Direct link to the agent's REW profile. - `agentId`: Unique identifier for the agent. - `name`: Full name of the agent. - `role`: Position, e.g., "Broker". - `brokerage`: Associated real estate company. - `experience`: Years or description of experience. - `languages`: Additional languages spoken. - `awards`: Number and details of awards. - `recommendations`: Count of client recommendations. - `phone`: Contact phone number. - `image`: URL to the agent's profile picture. - `scrapedFrom`: Source URL from which the data was extracted. Data is structured for easy parsing and integration. ### Error Handling The Actor includes robust error handling for common issues: - **Rate Limiting**: Automatically respects REW's rate limits; use proxies to minimize blocks. - **Page Not Found**: Skips invalid URLs and logs errors without stopping the crawl. - **Network Issues**: Retries failed requests up to 3 times with exponential backoff. - **Data Parsing Errors**: Logs warnings for malformed data but continues processing. If errors persist, check the run logs for details and adjust `maxLimit` or proxy settings. ### Rate Limiting and Best Practices - REW may impose rate limits; the Actor uses delays between requests to comply. - Best practices: Start with low `maxLimit` (e.g., 50) to test, then scale up. Use residential proxies for higher success rates. - Avoid overloading by running during off-peak hours. Monitor Apify credits, as high limits consume more. - For large datasets, export incrementally to prevent memory issues. ### Limitations - Data accuracy depends on REW's site structure; updates may require Actor maintenance. - Phone numbers and images are scraped as-is; verify for completeness. - Not intended for real-time monitoring; best for batch extractions. - Respects robots.txt and terms of service; ethical use only. ### Support For custom/simplified outputs or bug reports, please contact: - Email: support@getdataforme.com - Subject line: "custom support" - Contact form: https://getdataforme.com/contact/ We're here to help you get the most out of this Actor! *** # Actor input Schema ## `searchKeyword` (type: `string`): Enter the search keyword for which you want to scrape recipes from REWE. ## `maxLimit` (type: `integer`): Maximum number of requests that can be made by this crawler. ## `proxyConfiguration` (type: `object`): Specifies proxy servers that will be used by the scraper in order to hide its origin. ## Actor input object example ```json { "searchKeyword": "newmarket-on", "maxLimit": 100, "proxyConfiguration": { "useApifyProxy": true, "apifyProxyGroups": [ "RESIDENTIAL" ] } } ``` # Actor output Schema ## `overview` (type: `string`): No description # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "searchKeyword": "newmarket-on", "proxyConfiguration": { "useApifyProxy": true, "apifyProxyGroups": [ "RESIDENTIAL" ] } }; // Run the Actor and wait for it to finish const run = await client.actor("getdataforme/rew-agent-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "searchKeyword": "newmarket-on", "proxyConfiguration": { "useApifyProxy": True, "apifyProxyGroups": ["RESIDENTIAL"], }, } # Run the Actor and wait for it to finish run = client.actor("getdataforme/rew-agent-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "searchKeyword": "newmarket-on", "proxyConfiguration": { "useApifyProxy": true, "apifyProxyGroups": [ "RESIDENTIAL" ] } }' | apify call getdataforme/rew-agent-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=getdataforme/rew-agent-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Rew Agent Scraper", "description": "Scrape comprehensive real estate agent profiles from REW, including contact details, experience, awards, and recommendations. Ideal for market research, lead generation, and competitive analysis....", "version": "0.0", "x-build-id": "uNxybhOtSYJjbhUZi" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/getdataforme~rew-agent-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-getdataforme-rew-agent-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/getdataforme~rew-agent-scraper/runs": { "post": { "operationId": "runs-sync-getdataforme-rew-agent-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/getdataforme~rew-agent-scraper/run-sync": { "post": { "operationId": "run-sync-getdataforme-rew-agent-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "required": [ "searchKeyword", "proxyConfiguration" ], "properties": { "searchKeyword": { "title": "Enter the search keyword", "type": "string", "description": "Enter the search keyword for which you want to scrape recipes from REWE." }, "maxLimit": { "title": "Max Requests per Crawl", "type": "integer", "description": "Maximum number of requests that can be made by this crawler.", "default": 100 }, "proxyConfiguration": { "title": "Proxy configuration", "type": "object", "description": "Specifies proxy servers that will be used by the scraper in order to hide its origin.", "default": { "useApifyProxy": true, "apifyProxyGroups": [ "RESIDENTIAL" ] } } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```