# Ecommerce Store Intelligence (`happitap/ecommerce-store-intelligence`) Actor

Extract comprehensive store and product intelligence from Shopify and WooCommerce stores with automated change tracking and monitoring

- **URL**: https://apify.com/happitap/ecommerce-store-intelligence.md
- **Developed by:** [HappiTap](https://apify.com/happitap) (community)
- **Categories:** E-commerce, Automation, AI
- **Stats:** 4 total users, 1 monthly users, 100.0% runs succeeded, 1 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.01 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## E-commerce Store Intelligence Actor

Extract comprehensive store and product intelligence from Shopify and WooCommerce stores with automated change tracking and monitoring.

### Features

#### 🏪 Store Analysis

- **Platform Detection**: Automatically detect Shopify, WooCommerce, and other platforms
- **Store Metadata**: Extract store name, description, logo, and branding
- **Contact Signals**: Find email, phone, WhatsApp, and chat widgets
- **Policies**: Extract shipping and return policy URLs
- **Promotions**: Detect promo banners, coupons, and free shipping offers

#### 📦 Product Intelligence

- **Comprehensive Extraction**: Title, price, variants, availability, images
- **Multiple Discovery Methods**: Sitemap, platform APIs, category crawl, search
- **Smart Filtering**: Filter by collection, keywords, price range
- **Variant Support**: Extract all product variants with individual pricing
- **Stock Status**: Track in-stock vs. out-of-stock products

#### 📊 Change Tracking

- **Price Monitoring**: Detect price changes and calculate percentages
- **Availability Tracking**: Monitor stock status changes
- **New Products**: Identify newly added products
- **Removed Products**: Detect products removed from catalog
- **Historical Comparison**: Compare against previous runs or specific snapshots

#### 🔔 Delivery & Alerts

- **Webhook Integration**: POST results to custom webhooks
- **Conditional Alerts**: Only notify when changes are detected
- **Multiple Output Formats**: Dataset, Key-Value Store, Webhook

### Use Cases

1. **Competitor Price Monitoring**: Track competitor pricing and promotions
2. **Product Research**: Analyze product catalogs and trends
3. **Market Intelligence**: Monitor new product launches
4. **Inventory Tracking**: Track stock availability
5. **Pricing Strategy**: Analyze pricing patterns and changes

### Input Configuration

```json
{
  "stores": [
    {
      "storeUrl": "https://example-store.myshopify.com",
      "label": "Example Store",
      "country": "IN",
      "currency": "INR"
    }
  ],
  "crawl": {
    "render": "playwright",
    "maxProducts": 200,
    "maxCollections": 50,
    "productDiscovery": ["sitemap", "platform_api", "category_crawl"],
    "includeOutOfStock": true,
    "respectRobots": true,
    "timeoutMs": 45000
  },
  "extract": {
    "storeMeta": true,
    "collections": true,
    "products": true,
    "reviewsSummary": false,
    "contactSignals": true,
    "policies": true,
    "promotions": true
  },
  "filters": {
    "onlyCollections": [],
    "onlyKeywords": [],
    "minPrice": null,
    "maxPrice": null
  },
  "changeTracking": {
    "enabled": true,
    "compareWith": "previousRun",
    "track": ["price", "availability", "new_products", "removed_products"]
  },
  "delivery": {
    "webhookUrl": "",
    "onlyWhenChanged": true
  }
}
```

### Output Format

#### Store Summary

```json
{
  "type": "store_summary",
  "storeUrl": "https://example-store.com",
  "label": "Example Store",
  "platform": {
    "name": "shopify",
    "confidence": 0.92,
    "evidence": ["window.Shopify object found", "Shopify CDN detected"]
  },
  "currency": "INR",
  "storeMeta": {
    "name": "Example Store",
    "description": "...",
    "socials": ["instagram.com/..."],
    "logo": "https://.../logo.png"
  },
  "signals": {
    "hasWhatsapp": true,
    "hasChatWidget": true,
    "emailFound": "support@example-store.com",
    "phoneFound": "+91..."
  },
  "promotions": {
    "freeShipping": true,
    "saleBannerText": "Up to 40% off",
    "couponHints": ["WELCOME10"]
  },
  "policies": {
    "shippingUrl": ".../shipping-policy",
    "returnsUrl": ".../refund-policy"
  },
  "catalogStats": {
    "collectionsCount": 18,
    "productsCollected": 200,
    "inStockCount": 164
  },
  "runMeta": {
    "runId": "xxx",
    "timestamp": "2025-12-23T10:30:00Z"
  }
}
```

#### Product Item

```json
{
  "type": "product",
  "storeUrl": "https://example-store.com",
  "productUrl": "https://example-store.com/products/red-tshirt",
  "handle": "red-tshirt",
  "title": "Red T-Shirt",
  "vendor": "Example",
  "productType": "Apparel",
  "tags": ["cotton", "new"],
  "images": ["https://..."],
  "price": {
    "min": 499,
    "max": 699,
    "currency": "INR"
  },
  "compareAtPrice": {
    "min": 799,
    "max": 999,
    "currency": "INR"
  },
  "availability": "in_stock",
  "variants": [
    {
      "title": "S",
      "price": 499,
      "available": true,
      "sku": "RTS-S"
    }
  ],
  "collectionHints": ["tshirts", "new-arrivals"],
  "extractedFrom": "platform_api"
}
```

#### Change Event

```json
{
  "type": "change_event",
  "storeUrl": "https://example-store.com",
  "productUrl": "https://example-store.com/products/red-tshirt",
  "changeType": "price",
  "before": {
    "min": 549,
    "max": 749
  },
  "after": {
    "min": 499,
    "max": 699
  },
  "changePercent": -9.1,
  "detectedAt": "2025-12-23T10:30:00Z"
}
```

### Pricing

#### Pay-Per-Event Model

**Event 1: `apify-actor-start`** (Synthetic)

- **Price**: $0.00005 per run
- **Benefit**: First 5 seconds FREE
- **Charged**: Automatically by Apify

**Event 2: `store-analyzed`**

- **Price**: $0.01 per store
- **Includes**: Platform detection, signals, catalog discovery

**Event 3: `product-processed`**

- **Price**: $0.001 per product
- **Includes**: Full product extraction and processing

#### Pricing Examples

**10 products from 1 store:**

- Actor start: $0.00005
- Store analyzed: $0.01
- Products: 10 × $0.001 = $0.01
- **Total**: $0.02005

**200 products from 5 stores:**

- Actor start: $0.00005
- Stores: 5 × $0.01 = $0.05
- Products: 200 × $0.001 = $0.20
- **Total**: $0.25005

**Daily monitoring (100 products, 24 runs):**

- Per run: ~$0.11
- Per day: $2.64
- **Per month**: ~$79

### Architecture

#### Module Structure

```
src/
├── main.js                      # Main actor logic
├── detectors/
│   └── platformDetector.js      # Platform detection
├── discovery/
│   ├── sitemapDiscovery.js      # Sitemap-based discovery
│   ├── apiDiscovery.js          # Platform API discovery
│   └── categoryDiscovery.js     # Category crawl discovery
├── extractors/
│   ├── storeExtractor.js        # Store metadata extraction
│   ├── productExtractor.js      # Product data extraction
│   └── signalsExtractor.js      # Contact/promo signals
├── tracking/
│   └── changeTracker.js         # Change detection & tracking
└── utils/
    ├── helpers.js               # Utility functions
    └── webhooks.js              # Webhook delivery
```

#### Product Discovery Flow

1. **Sitemap Discovery** (Fast, high recall)
   - Parse `/sitemap.xml`
   - Extract product URLs
   - Priority: High

2. **Platform API** (Clean, structured)
   - Shopify: `/products.json`
   - WooCommerce: `/wp-json/wc/v3/products`
   - Priority: High

3. **Category Crawl** (Fallback)
   - Crawl collection/category pages
   - Extract product links
   - Priority: Medium

4. **Search** (Last resort)
   - Use on-site search
   - Extract results
   - Priority: Low

### Best Practices

#### For Accurate Results

1. Use `playwright` render for JavaScript-heavy stores
2. Set appropriate `maxProducts` to avoid runaway crawls
3. Enable `respectRobots` to be a good citizen
4. Use filters to focus on relevant products

#### For Cost Optimization

1. Use `cheerio` render when possible (faster, cheaper)
2. Set `maxProducts` limit based on needs
3. Filter by collections or keywords to reduce scope
4. Schedule runs during off-peak hours

#### For Change Tracking

1. Enable `changeTracking` for monitoring use cases
2. Run on a schedule (hourly, daily, weekly)
3. Use `onlyWhenChanged` webhooks to reduce noise
4. Track only relevant changes (price, availability)

### Limitations

- **Rate Limiting**: Respects robots.txt and implements delays
- **JavaScript Required**: Some stores require Playwright rendering
- **API Access**: Not all stores expose public APIs
- **Authentication**: Does not handle login-required stores
- **Dynamic Pricing**: May not capture personalized pricing

### Development

#### Local Setup

```bash
cd ecommerce-store-intelligence
npm install
npm start
```

#### Testing

```bash
## Test with example input
apify run --input-file examples/shopify-store.json

## Test specific store
apify run --input '{"stores":[{"storeUrl":"https://example.myshopify.com"}]}'
```

#### Deployment

```bash
## Build and push to Apify
apify push

## Or use Apify Console to build from GitHub
```

### Support

For issues, questions, or feature requests:

- GitHub Issues: \[Link to repo]
- Apify Console: Actor page
- Email: support@example.com

### License

Apache-2.0

***

**Built with ❤️ for e-commerce intelligence**

# Actor input Schema

## `stores` (type: `array`):

List of e-commerce stores to analyze

## `crawl` (type: `object`):

Configure how stores are crawled

## `extract` (type: `object`):

What data to extract from stores

## `filters` (type: `object`):

Filter which products to extract

## `changeTracking` (type: `object`):

Track changes between runs

## `delivery` (type: `object`):

How to deliver results

## Actor input object example

```json
{
  "stores": [
    {
      "storeUrl": "https://example-store.myshopify.com",
      "label": "Example Store",
      "country": "IN",
      "currency": "INR"
    }
  ],
  "crawl": {
    "render": "playwright",
    "maxProducts": 200,
    "maxCollections": 50,
    "productDiscovery": [
      "sitemap",
      "platform_api",
      "category_crawl"
    ],
    "includeOutOfStock": true,
    "respectRobots": true,
    "timeoutMs": 45000
  },
  "extract": {
    "storeMeta": true,
    "collections": true,
    "products": true,
    "reviewsSummary": false,
    "contactSignals": true,
    "policies": true,
    "promotions": true
  },
  "filters": {
    "onlyCollections": [],
    "onlyKeywords": [],
    "minPrice": null,
    "maxPrice": null
  },
  "changeTracking": {
    "enabled": true,
    "compareWith": "previousRun",
    "track": [
      "price",
      "availability",
      "new_products",
      "removed_products"
    ]
  },
  "delivery": {
    "webhookUrl": "",
    "onlyWhenChanged": true
  }
}
```

# Actor output Schema

## `results` (type: `string`):

Link to the dataset containing store summaries, products, and change events

## `snapshots` (type: `string`):

Link to Key-Value Store containing product snapshots for change tracking

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "stores": [
        {
            "storeUrl": "https://example-store.myshopify.com",
            "label": "Example Store",
            "country": "IN",
            "currency": "INR"
        }
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("happitap/ecommerce-store-intelligence").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "stores": [{
            "storeUrl": "https://example-store.myshopify.com",
            "label": "Example Store",
            "country": "IN",
            "currency": "INR",
        }] }

# Run the Actor and wait for it to finish
run = client.actor("happitap/ecommerce-store-intelligence").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "stores": [
    {
      "storeUrl": "https://example-store.myshopify.com",
      "label": "Example Store",
      "country": "IN",
      "currency": "INR"
    }
  ]
}' |
apify call happitap/ecommerce-store-intelligence --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "command": "npx",
            "args": [
                "mcp-remote",
                "https://mcp.apify.com/?tools=happitap/ecommerce-store-intelligence",
                "--header",
                "Authorization: Bearer <YOUR_API_TOKEN>"
            ]
        }
    }
}

```

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/acts/cLoTUBTBzrztWV4Jr/builds/OeXinZWlHith8Nzq3/openapi.json