# Shopify Store Scraper (`scrapeflow/shopify-store-scraper`) Actor πŸ›οΈ Shopify Store Scraper extracts products, collections, prices, variants, inventory, images, reviews & SEO metadata from any Shopify store. πŸ” Ideal for competitor research, price tracking & SEO audits. πŸ“€ Export CSV/JSON for ETL, BI or CRM. ⚑ Fast, reliable, scalable. - **URL**: https://apify.com/scrapeflow/shopify-store-scraper.md - **Developed by:** [ScrapeFlow](https://apify.com/scrapeflow) (community) - **Categories:** Automation, Lead generation, Social media - **Stats:** 5 total users, 1 monthly users, 100.0% runs succeeded, 1 bookmarks - **User rating**: No ratings yet ## Pricing $19.99/month + usage To use this Actor, you pay a monthly rental fee to the developer. The rent is subtracted from your prepaid usage every month after the free trial period.You also pay for the Apify platform usage, which gets cheaper the higher Apify subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#rental-actors ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ### Shopify Store Scraper The Shopify Store Scraper is a fast, reliable Shopify web scraping tool that extracts structured store-level and product-level data from publicly accessible Shopify stores. It solves the tedious work of gathering catalogs, prices, variants, inventory, collections, app usage, and optional Yotpo reviews into clean JSON/CSV for analysis. Built for marketers, developers, data analysts, and researchers, this Shopify product scraper and Shopify store data extractor enables scalable competitive analysis, price monitoring, and SEO intelligence across any number of stores. πŸš€ ### What data / output can you get? Below are examples of structured fields the scraper saves to the Apify dataset. Exports are available as JSON or CSV. | Data field | Description | Example value | | --- | --- | --- | | store_domain | Store domain without protocol | www.allbirds.com | | store_url | Full store URL | https://www.allbirds.com | | shop_name | Shopify shop name | shop11044168.myshopify.com | | country | Store country code (from meta.json) | US | | currency | Store currency code (from meta.json) | USD | | locale | Store locale (from meta.json or HTML) | en-US | | theme_name | Active theme name parsed from HTML | rc-cm-2025-12-26_20-35 Update | | theme_id | Active theme ID parsed from HTML | 128596836432 | | apps_installed | Detected apps from HTML (e.g., Elevar, JsEncrypt) | ["Elevar Conversion Tracking","JsEncrypt"] | | total_products | Count of all products via /products.json pagination | 681 | | total_variants | Count of all variants across products | 7276 | | price_min / price_avg / price_max | Min/avg/max prices across variants | 2.0 / 79.34 / 500.0 | | inventory_availability_pct | Percentage of variants in stock | 77.6 | | collections_count | Number of collections via /collections.json | 250 | | top_products | Top 10 newest products with basic info (+ optional reviews) | [{"title":"Allbirds Laces - Warm Red - Round","url":"https://www.allbirds.com/products/allbirds-laces-warm-red-round","price":8.0,"reviews":0,"rating":0.0}] | Bonus output: When review extraction is enabled, a detailed_reviews array is included with per-product review stats and items from Yotpo. You can download results from the Apify dataset as JSON or CSV and integrate via the Apify API. ### Key features - πŸ”„ Smart proxy fallback & retries: Automatically moves from direct β†’ datacenter β†’ residential proxies with retry logic to keep scraping moving even when facing 403/429/503 blocks. - πŸ“Š Comprehensive data extraction: Collects store metadata, collections, full product catalogs, variants, prices, images, and availability to power Shopify competitor analysis scraper workflows. - ⭐ Optional Yotpo review scraping: Toggle extractReviews to enrich top/newest products with review counts, average ratings, and review details for deeper product insights. - πŸ“ˆ Pricing & inventory analytics: Outputs min/avg/max prices, total/available variants, and inventory availability percentage for Shopify price monitoring scraper and inventory tracking use cases. - 🧩 App detection: Identifies installed apps like Elevar Conversion Tracking and JsEncrypt from the store HTML for app discovery and tech-stack research. - πŸ’Ύ Live dataset saving: Pushes results as they’re collected, so partial data is preserved even if a run is interrupted. - πŸ›‘οΈ Robust error handling: Continues to the next store if one fails and logs an error entry to the dataset for full observability. - πŸ‘¨β€πŸ’» Developer-friendly: Built on Apify SDK for Python with clean JSON output for ETL/BI pipelines, APIs, and scripts. ### How to use Shopify Store Scraper - step by step 1. Sign in to the Apify Console and open the Shopify Store Scraper actor. 2. Add input in the INPUT tab: - Paste one or more Shopify store URLs into startUrls. - Set extractReviews to true or false depending on whether you want Yotpo reviews. - Optionally configure proxyConfiguration (e.g., useApifyProxy) for better resilience. 3. Start the run by clicking Start. 4. Monitor progress in the LOG tab. You’ll see notes about product page counts and any proxy fallback events. 5. Wait for completion. The actor processes stores sequentially and saves output items as they are scraped. 6. View results in the OUTPUT tab. Each store (or error) is a separate dataset item. 7. Export your data as JSON or CSV, or access it programmatically via the Apify API. Pro tip: Chain this Shopify store crawler with your ETL/BI or CRM using the Apify API to download products from Shopify store catalogs at scale. ### Use cases | Use case | Description | | --- | --- | | Competitive analysis & benchmarking | Track catalogs, prices, and variants across competitors to extract Shopify product details and compare assortments. | | Market research & category insights | Analyze categories, vendors, and collections to discover gaps and opportunities with a Shopify collections scraper. | | Price monitoring & alerts | Monitor min/avg/max pricing across Shopify stores to power pricing dashboards and alerts. | | Inventory availability tracking | Measure inventory availability percentages and variant counts with a Shopify inventory scraper. | | Theme & app intelligence | Identify theme IDs/versions and detect apps used by stores for technology research. | | Review analytics (Yotpo) | Enrich top products with review counts and ratings for product quality and sentiment tracking using a Shopify product reviews scraper strategy. | | Data pipelines (API) | Automate exports to JSON/CSV via the Apify API for data warehousing, BI, or enrichment workflows. | ### Why choose Shopify Store Scraper? A production-ready, reliable Shopify web scraping tool built for precision and automation. - βœ… Accurate public data: Leverages Shopify’s public JSON endpoints (/products.json, /collections.json, /meta.json) and HTML parsing for theme/app signals. - ⚑ Fast and resilient: Async/await architecture plus proxy fallback ensures successful runs at scale. - 🧠 Optional review depth: Toggle Yotpo review extraction for richer product intelligence when needed. - πŸ”Œ Easy integrations: Export JSON/CSV from Apify datasets or connect directly via API to your systems. - πŸ”’ No login required: Works with publicly accessible Shopify stores without cookies or sessions. - 🧱 Built for reliability: Live saving to dataset, robust error handling, and clear logging beat flaky browser extensions and one-off scripts. In short: a stable Shopify product data scraping engine that combines coverage, resilience, and clean outputs. ### Is it legal / ethical to use Shopify Store Scraper? Yes β€” when used responsibly. The scraper collects data from publicly available sources (Shopify’s public JSON APIs and public pages). It does not access private or password-protected content. Guidelines for compliant use: - Only collect public data and respect each site’s terms of service. - Ensure compliance with relevant regulations (e.g., GDPR, CCPA). - Use scraped data responsibly; avoid spam and misuse. - Respect rate limits and avoid abusive traffic patterns. For edge cases, verify your use with your legal team. ### Input parameters & output format Example JSON input ```json { "startUrls": [ "https://www.allbirds.com", "https://example-store.myshopify.com" ], "extractReviews": true, "proxyConfiguration": { "useApifyProxy": false } } ```` Input fields - startUrls (array, required): List one or more Shopify store URLs (e.g., https://www.allbirds.com). Default: not specified. - extractReviews (boolean, optional): Enable to extract detailed product reviews from Yotpo (slower but more comprehensive). Default: true. - proxyConfiguration (object, optional): Choose which proxies to use. If Shopify rejects the proxy, a residential proxy will be used as a fallback. Default: not specified. Example JSON output (successful store item) ```json { "store_domain": "www.allbirds.com", "store_url": "https://www.allbirds.com", "shop_name": "shop11044168.myshopify.com", "country": "US", "currency": "USD", "locale": "en-US", "theme_name": "rc-cm-2025-12-26_20-35 Update", "theme_id": 128596836432, "theme_version": "1.131.2", "apps_installed": ["Elevar Conversion Tracking", "JsEncrypt"], "apps_by_category": { "Analytics": ["Elevar Conversion Tracking"], "Other": ["JsEncrypt"] }, "apps_count": 2, "total_products": 681, "total_variants": 7276, "collections": ["Accessories", "Shoes", "Apparel"], "collections_count": 250, "categories": ["Accessories", "Apparel", "Shoes", "Socks"], "vendors": ["Allbirds"], "price_min": 2.0, "price_avg": 79.34, "price_max": 500.0, "products_on_sale": 681, "discount_pct_avg": null, "products_with_reviews": 0, "review_coverage_pct": 0, "avg_reviews_per_product": null, "avg_rating": null, "total_estimated_sales": null, "avg_sales_per_product": null, "total_variants_available": 5647, "inventory_availability_pct": 77.6, "estimated_launch_date": "2018-11-12", "oldest_product_date": "2018-11-12", "newest_product_date": "2025-11-06", "products_with_images": 680, "avg_images_per_product": 4.5, "products_with_description": 681, "top_products": [ { "title": "Allbirds Laces - Warm Red - Round", "url": "https://www.allbirds.com/products/allbirds-laces-warm-red-round", "price": 8.0, "reviews": 0, "rating": 0.0 } ], "detailed_reviews": [ { "product_url": "https://www.allbirds.com/products/allbirds-laces-warm-red-round", "product_id": "7242343809104", "reviews_count": 0, "avg_rating": 0.0, "reviews": [] } ], "scraped_at": "2025-12-29T07:34:23.715157Z", "scrape_duration_sec": 15.71, "api_blocked": false } ``` Example JSON output (error item when a store fails) ```json { "store_url": "https://unreachable-store.com", "error": "Failed to fetch homepage from https://unreachable-store.com. All request attempts failed.", "scraped_at": "2025-12-29T07:34:23.715157Z" } ``` Notes - Fields like theme\_id, price\_min/avg/max, and review metrics may be null when not available. - When no valid startUrls are provided, the dataset will contain an error item with error, input\_received, and scraped\_at for debugging. ### FAQ #### How long does it take to scrape a store? Small stores typically finish in seconds, while larger catalogs take longer. If extractReviews is enabled, additional time is spent fetching Yotpo data for the newest products. #### What happens if a store blocks my requests? The scraper implements intelligent proxy fallback. It starts without a proxy, switches to a datacenter proxy if blocked, then to a residential proxy with retries if needed. Once residential is used, it sticks with it for subsequent requests. #### Can I scrape multiple stores at once? Yes. Add multiple URLs to the startUrls array. The actor processes them sequentially and writes each result as a separate dataset item. #### Do I need to enable a proxy? No. Proxy is optional. However, enabling proxyConfiguration (e.g., useApifyProxy) can improve reliability for large-scale Shopify store crawler runs. #### Can it extract product reviews? Yes. Set extractReviews to true to fetch Yotpo review counts, average ratings, and review details for the top newest products. If review extraction fails for a product, the run continues. #### What data formats can I export? Results are stored in the Apify dataset and can be exported as JSON or CSV. You can also access data via the Apify API for downstream pipelines. #### Does it work with every Shopify store? It works with publicly accessible Shopify stores using endpoints like /products.json, /collections.json, and /meta.json. If a store heavily restricts access, proxy fallback increases the likelihood of success. #### What happens if I provide no URLs? If startUrls is empty or invalid, the actor will save an error item to the dataset describing the issue (including input\_received for troubleshooting) and end the run. ### Closing CTA / Final thoughts Shopify Store Scraper is built to extract structured Shopify store intelligence at scale. With robust proxy fallback, detailed product and collection coverage, and optional Yotpo reviews, it’s ideal for marketers, developers, analysts, and researchers who need reliable Shopify product data scraping. Export clean JSON/CSV from Apify datasets, or connect via the Apify API to automate a Shopify product CSV extractor pipeline end-to-end. Start extracting smarter insights from any Shopify store today. If you need custom features or priority support, email dev.scraperengine@gmail.com. # Actor input Schema ## `startUrls` (type: `array`): List one or more Shopify store URLs (e.g., https://www.allbirds.com). ## `extractReviews` (type: `boolean`): Enable to extract detailed product reviews from Yotpo (slower but more comprehensive). ## `proxyConfiguration` (type: `object`): Choose which proxies to use. If Shopify rejects the proxy, a residential proxy will be used as a fallback. ## Actor input object example ```json { "startUrls": [ "https://www.allbirds.com" ], "extractReviews": true, "proxyConfiguration": { "useApifyProxy": false } } ``` # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "startUrls": [ "https://www.allbirds.com" ], "proxyConfiguration": { "useApifyProxy": false } }; // Run the Actor and wait for it to finish const run = await client.actor("scrapeflow/shopify-store-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`πŸ’Ύ Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // πŸ“š Want to learn more πŸ“–? Go to β†’ https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "startUrls": ["https://www.allbirds.com"], "proxyConfiguration": { "useApifyProxy": False }, } # Run the Actor and wait for it to finish run = client.actor("scrapeflow/shopify-store-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("πŸ’Ύ Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # πŸ“š Want to learn more πŸ“–? Go to β†’ https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "startUrls": [ "https://www.allbirds.com" ], "proxyConfiguration": { "useApifyProxy": false } }' | apify call scrapeflow/shopify-store-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=scrapeflow/shopify-store-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Shopify Store Scraper", "description": "πŸ›οΈ Shopify Store Scraper extracts products, collections, prices, variants, inventory, images, reviews & SEO metadata from any Shopify store. πŸ” Ideal for competitor research, price tracking & SEO audits. πŸ“€ Export CSV/JSON for ETL, BI or CRM. ⚑ Fast, reliable, scalable.", "version": "0.1", "x-build-id": "8byGzqZTUYFZD7vhi" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/scrapeflow~shopify-store-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-scrapeflow-shopify-store-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/scrapeflow~shopify-store-scraper/runs": { "post": { "operationId": "runs-sync-scrapeflow-shopify-store-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/scrapeflow~shopify-store-scraper/run-sync": { "post": { "operationId": "run-sync-scrapeflow-shopify-store-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "required": [ "startUrls" ], "properties": { "startUrls": { "title": "Shopify Store URLs", "type": "array", "description": "List one or more Shopify store URLs (e.g., https://www.allbirds.com).", "items": { "type": "string" } }, "extractReviews": { "title": "Extract Product Reviews", "type": "boolean", "description": "Enable to extract detailed product reviews from Yotpo (slower but more comprehensive).", "default": true }, "proxyConfiguration": { "title": "Proxy Configuration", "type": "object", "description": "Choose which proxies to use. If Shopify rejects the proxy, a residential proxy will be used as a fallback." } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```