# Apify Scraper (`dz_omar/apify-scraper`) Actor Extract actor data from the Apify Store, including pricing, stats, and user info. Supports multiple queries, filters, and crash-safe state persistence. 🧑‍💻 Open-source and ideal for research, analysis, and discovering tools. - **URL**: https://apify.com/dz\_omar/apify-scraper.md - **Developed by:** [FlowExtract API](https://apify.com/dz_omar) (community) - **Categories:** Developer tools, Automation, Open source - **Stats:** 41 total users, 2 monthly users, 0.0% runs succeeded, 6 bookmarks - **User rating**: 5.00 out of 5 stars ## Pricing Pay per usage This Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage, which gets cheaper the higher subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-usage ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## 🔍 Apify Scraper ### ✨ Overview Apify Store Scraper is a powerful actor for extracting comprehensive data from the Apify Store marketplace. Whether you need to research available actors, monitor pricing changes, or gather insights on developer tools, this scraper provides a reliable solution with advanced features for data persistence and batch processing. ### 📊 What Can It Scrape? The Apify Store Scraper extracts detailed information from actor listings, including: - **📝 Basic Information**: Title, ID, URL, description, name - **👤 Developer Details**: Username, user full name, profile picture URL - **📈 Statistics**: Total runs, users, builds, and performance metrics - **🏷️ Categorization**: Categories, tags, and management type (Apify Official or Community) - **⭐ Ratings & Popularity**: Review ratings, bookmark counts - **💰 Pricing Information**: Pricing model, cost per unit, trial periods ### 🚀 Features - **🔄 Multi-Query Support**: Process multiple search terms in a single run - **💾 State Persistence**: Automatically save progress and resume after migrations or failures - **🔍 Configurable Filtering**: Filter by search terms, categories, pricing models, and more - **⏱️ Intelligent Rate Limiting**: Configurable delays between requests to respect server limits - **📄 Result Pagination**: Handles all result pages automatically for complete data collection - **⚙️ Customizable Output**: Control exactly what data you want to collect ### 🛠️ Usage #### Input Parameters | Parameter | Type | Description | |-----------|------|-------------| | `search` | Array | List of search terms to process sequentially | | `sortBy` | String | How to sort results: relevance, popularity, newest, or lastUpdate | | `category` | String | Filter by specific category (AI, Social Media, E-commerce, etc.) | | `pricingModel` | String | Filter by pricing type (Free, Pay Per Result, Monthly Rental, etc.) | | `managedBy` | String | Filter by developer type (Apify Official or Community) | | `limit` | Number | Items per request (max 100) | | `maxResults` | Number | Maximum total results to fetch (0 for unlimited) | | `batchDelay` | Number | Milliseconds to wait between API requests | #### Example Input ```json { "search": ["YouTube Scraper", "github", "e-commerce"], "sortBy": "popularity", "category": "SOCIAL_MEDIA", "pricingModel": "", "managedBy": "", "limit": 24, "maxResults": 100, "batchDelay": 100 } ```` #### Example Output ```json { "id": "y1IMcEPawMQPafm02", "title": "Youtube Video Downloader", "url": "https://apify.com/epctex/youtube-video-downloader", "username": "epctex", "description": "Effortlessly download YouTube videos of your preferred quality with our user-friendly Video Downloader. Try it now!", "categories": ["VIDEOS", "SOCIAL_MEDIA", "AUTOMATION"], "price": 30, "stats": { "totalRuns": 246731, "lastRun": "2025-04-25T09:03:27.155Z" } } ``` ### 💡 Use Cases - **🔎 Market Research**: Analyze available actors in specific niches - **👀 Competitor Analysis**: Monitor pricing models and popularity trends - **🛠️ Tool Discovery**: Find the most effective actors for your specific needs - **📊 Performance Monitoring**: Track statistics and user ratings over time - **🔌 Integration Planning**: Gather data for making informed decisions about which actors to use in your projects ### 🤔 Why Scrape the Apify Store? The Apify Store is a growing marketplace for web scraping and automation tools. By extracting data systematically, you can: 1. **🧠 Make Informed Decisions**: Compare actors based on performance metrics and user ratings 2. **🔄 Stay Updated**: Monitor new actors and updates to existing ones 3. **⏱️ Save Time**: Quickly filter through thousands of options to find exactly what you need 4. **📈 Analyze Trends**: Track changes in pricing, popularity, and feature sets over time ### 🚀 Getting Started 1. **Click "Try for free"** or use your existing Apify account 2. **Set your input parameters** to customize your search 3. **Run the actor** and wait for it to complete 4. **View your results** in the Dataset tab or download as JSON, CSV, or Excel ### 🔧 Advanced Usage #### 🔄 Resuming Failed Runs The actor features built-in state persistence. If a run fails or gets migrated, it will automatically resume from where it left off, saving you time and resources. #### 🔍 Multiple Search Terms Use the `search` array to process multiple queries in a single run. Each term will be processed sequentially, with results combined in the final dataset. #### ⏱️ Rate Limiting The `batchDelay` parameter helps you control how aggressively the actor makes requests. Higher values are more conservative but slower, while lower values are faster but may increase the risk of temporary blocks. ### ⚠️ Limitations - The actor only extracts publicly available information from the Apify Store - Request rates are limited to respect Apify's servers - Some detailed actor statistics may not be available for all listings ### 📦 Open Source This actor is open source! Feel free to: - 🔧 Fork and modify it to suit your specific needs - 🐛 Submit bug reports and feature requests - 🤝 Contribute improvements or optimizations - 🌐 Share with others who might benefit from it Check out the [GitHub repository](https://github.com/DZ-ABDLHAKIM/Apify-Scraper) for the latest code and updates. ### 🤝 Support & Contact For assistance or custom implementations: - 📧 Email: - 🐙 GitHub: [DZ-ABDLHAKIM](https://github.com/DZ-ABDLHAKIM) - 🐦 Twitter: [@DZ\_45Omar](https://x.com/DZ_45Omar) - 🔧 Apify: [dz\_omar](https://apify.com/dz_omar) ### 🙏 Acknowledgements Special thanks to all contributors and the Apify community for their support and inspiration. This project aims to make data collection from the Apify Store accessible and efficient for everyone. # Actor input Schema ## `search` (type: `array`): Enter one or more keywords to search Actors (e.g., 'YouTube Scraper', 'GitHub'). ## `sortBy` (type: `string`): Choose how the results should be sorted. ## `category` (type: `string`): 🎯 Narrow down results by selecting a specific category. ## `pricingModel` (type: `string`): Filter results by pricing type (free, monthly, etc.). ## `managedBy` (type: `string`): Filter by the developer type (Apify official or community). ## `limit` (type: `integer`): Number of items to fetch in each API call (max 100). ## `maxResults` (type: `integer`): Maximum total number of results to fetch (0 = unlimited). ## `batchDelay` (type: `integer`): Pause (in milliseconds) between each search query to reduce rate limits or load. ## Actor input object example ```json { "search": [ "YouTube Scraper" ], "sortBy": "relevance", "category": "", "pricingModel": "", "managedBy": "", "limit": 24, "maxResults": 1, "batchDelay": 100 } ``` # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = {}; // Run the Actor and wait for it to finish const run = await client.actor("dz_omar/apify-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = {} # Run the Actor and wait for it to finish run = client.actor("dz_omar/apify-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{}' | apify call dz_omar/apify-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=dz_omar/apify-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Apify Scraper", "description": "Extract actor data from the Apify Store, including pricing, stats, and user info. Supports multiple queries, filters, and crash-safe state persistence. 🧑‍💻 Open-source and ideal for research, analysis, and discovering tools.", "version": "0.0", "x-build-id": "FqT6dsnTMXxRCjOfK" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/dz_omar~apify-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-dz_omar-apify-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/dz_omar~apify-scraper/runs": { "post": { "operationId": "runs-sync-dz_omar-apify-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/dz_omar~apify-scraper/run-sync": { "post": { "operationId": "run-sync-dz_omar-apify-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "properties": { "search": { "title": "🔎 Search Terms", "type": "array", "description": "Enter one or more keywords to search Actors (e.g., 'YouTube Scraper', 'GitHub').", "default": [ "YouTube Scraper" ], "items": { "type": "string" } }, "sortBy": { "title": "📊 Sort By", "enum": [ "relevance", "popularity", "newest", "lastUpdate" ], "type": "string", "description": "Choose how the results should be sorted.", "default": "relevance" }, "category": { "title": "📁 Category", "enum": [ "", "AI", "AGENTS", "AUTOMATION", "DEVELOPER_TOOLS", "ECOMMERCE", "JOBS", "LEAD_GENERATION", "NEWS", "SEO_TOOLS", "SOCIAL_MEDIA", "TRAVEL", "VIDEOS", "REAL_ESTATE", "INTEGRATIONS", "OTHER", "OPEN_SOURCE" ], "type": "string", "description": "🎯 Narrow down results by selecting a specific category.", "default": "" }, "pricingModel": { "title": "💰 Pricing Model", "enum": [ "", "PRICE_PER_DATASET_ITEM", "PAY_PER_EVENT", "FREE", "FLAT_PRICE_PER_MONTH" ], "type": "string", "description": "Filter results by pricing type (free, monthly, etc.).", "default": "" }, "managedBy": { "title": "🛠 Managed By", "enum": [ "", "APIFY", "COMMUNITY" ], "type": "string", "description": "Filter by the developer type (Apify official or community).", "default": "" }, "limit": { "title": "📦 Items Per Request", "minimum": 1, "maximum": 100, "type": "integer", "description": "Number of items to fetch in each API call (max 100).", "default": 24 }, "maxResults": { "title": "📈 Max Results", "minimum": 0, "type": "integer", "description": "Maximum total number of results to fetch (0 = unlimited).", "default": 1 }, "batchDelay": { "title": "⏱ Delay Between Searches", "minimum": 0, "maximum": 10000, "type": "integer", "description": "Pause (in milliseconds) between each search query to reduce rate limits or load.", "default": 100 } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```