# BIG W Australia Product Scraper (`dromb/big-w-au-product-scraper`) Actor

Extract BIG W Australia products, categories, prices, availability, barcodes, images, variants, and detailed item data.

- **URL**: https://apify.com/dromb/big-w-au-product-scraper.md
- **Developed by:** [Dmitriy Gyrbu](https://apify.com/dromb) (community)
- **Categories:** E-commerce, Automation, Developer tools
- **Stats:** 5 total users, 2 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.40 / 1,000 result items

This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## BIG W Australia Product Scraper

Extract BIG W Australia product search results, categories, prices, availability, barcodes, images, variants, and detailed item data. Use the scraper for price monitoring, assortment research, competitor analysis, product research, and procurement workflows.

This is an unofficial scraper and is not affiliated with, endorsed by, or sponsored by BIG W or Woolworths Group.

### What you can collect

| Operation | Result |
| --- | --- |
| `search` | Products matching a keyword |
| `category` | Products associated with a BIG W category |
| `categories` | BIG W category tree |
| `item` | Detailed product data and national pricing |
| `search_filters` | Available category and sorting metadata |
| `probe` | Lightweight source health diagnostics |

The Actor does not currently provide store-specific stock, postcode-specific prices, reviews, or a store locator.

### Quick start

The default input returns a small, non-empty LEGO search:

```json
{
  "operation": "search",
  "query": "lego",
  "limit": 5,
  "proxyMode": "apify"
}
```

#### Search

Required: `query`. Optional: `page`, `limit`, and `max_scan_urls`.

```json
{
  "operation": "search",
  "query": "air fryer",
  "limit": 10
}
```

#### Product details

Provide one of `item_id`, `slug`, or a full BIG W product `url`.

```json
{
  "operation": "item",
  "item_id": "9900053407"
}
```

#### Category products

Provide one of `category_id`, `slug`, or a full category `url`.

```json
{
  "operation": "category",
  "slug": "baby",
  "limit": 10
}
```

#### Category tree

```json
{
  "operation": "categories",
  "enabled_only": true
}
```

### Ready-made Saved Tasks

The Actor includes bounded Saved Tasks for common buyer workflows:

- **LEGO Price & Assortment Watch** — collect a current LEGO product set for price-monitoring or catalog comparison.
- **Exact Product Price & Details** — retrieve one known product with price, availability, barcode, images, and specifications.
- **Baby Category Assortment** — collect a bounded sample from a high-value retail category.
- **Top-Level Category Map** — export the current top-level BIG W catalog structure for discovery and workflow setup.

Run a task as-is, duplicate it, and then replace the query, item ID, category, or limit with your own target.

### Output

Results are written to the default Dataset. Depending on the operation, fields can include:

- identity: `id`, `name`, `brand`, `slug`, `source_url`
- merchandising: `price`, `discount_price`, `currency`, `unit_price`
- availability: `stock_status`, `in_stock`
- catalog: `category`, `categories`, `breadcrumbs`
- content: `description`, `ingredients`, `usage_instructions`, `allergens`
- product detail: `barcode`, `size`, `color`, `specifications`, `attributes`
- media and choices: `image`, `images`, `variants`, `options`

The `OUTPUT` record contains run status, counts, errors, charge-limit status, and the effective proxy route.

### Proxy strategy

`apify` is recommended in Apify Cloud. It uses the standard datacenter pool, the cheapest cloud route verified for BIG W.

- `apify`: standard datacenter proxy by default
- `auto`: direct first with Apify datacenter fallback
- `direct`: no proxy; currently works locally but is blocked from current Apify cloud ranges
- `custom`: use proxy URLs supplied through Proxy Configuration

If the standard datacenter route becomes blocked, select an AU residential proxy through the Apify Proxy Configuration field. A browser is not required.

### Use with n8n or Make

A practical workflow is:

`BIG W Actor → Dataset → n8n/Make → Google Sheets, Telegram, email, or CRM`

1. Start the Actor through the Apify API or the Apify integration.
2. Wait until the run reaches `SUCCEEDED`.
3. Read items from the run's default Dataset.
4. Filter price or availability changes.
5. Send the final rows to Sheets, Telegram, email, or your internal system.

#### API example

```bash
curl -X POST \
  "https://api.apify.com/v2/acts/dromb~big-w-au-product-scraper/runs?token=YOUR_APIFY_TOKEN&waitForFinish=120" \
  -H "Content-Type: application/json" \
  -d '{"operation":"search","query":"lego","limit":10,"proxyMode":"apify"}'
```

Use the returned `defaultDatasetId` to download results:

```bash
curl "https://api.apify.com/v2/datasets/DATASET_ID/items?token=YOUR_APIFY_TOKEN&clean=true"
```

### Reliability and limits

- Search uses BIG W sitemap product URLs because the public native search endpoint is not consistently available.
- Category product matching is based on category metadata and product URL signals.
- Item mode uses BIG W product and national price endpoints for richer data.
- Requests and sitemap scans are bounded by the input limits.
- Standard Apify datacenter, AU residential, and the tested special proxy groups currently work. BIG W can change its protection rules.
- Free/trial users may be limited to 10 runs per UTC day.

Current charges are shown in the Actor's Pricing tab and in the Apify Console before each run.

# Actor input Schema

## `operation` (type: `string`):

Actor operation to execute.

## `query` (type: `string`):

Required for `search`. Keyword matched against product URLs in the sitemap.

## `category_id` (type: `string`):

Optional for `category`. Big W category id from the categories operation, e.g. 61.

## `slug` (type: `string`):

Optional for `category` and `item`. Category slug (e.g. baby) or product code alias.

## `url` (type: `string`):

Optional for `category` and `item`. Full Big W category or product URL.

## `item_id` (type: `string`):

Optional for `item`. Product code/SKU, e.g. 9900053407.

## `page` (type: `integer`):

Optional for `search` and `category`. 1-based page number.

## `limit` (type: `integer`):

Optional for `search` and `category`. Maximum products per page (1-50).

## `level` (type: `integer`):

Optional for `categories`. Filter by category depth (1-6).

## `enabled_only` (type: `boolean`):

Optional for `categories`. Return only enabled categories from the Big W tree.

## `max_scan_urls` (type: `integer`):

Optional for `search`. Safety cap for scanned sitemap URLs.

## `max_scan_products` (type: `integer`):

Optional for `category`. Safety cap for checked products.

## `details` (type: `boolean`):

Optional for `search`. Compatibility flag; search stays single-pass without per-item enrichment.

## `proxyMode` (type: `string`):

Access strategy. Apify uses the cheapest verified cloud route (standard datacenter proxy). Auto tries direct access first and then falls back to an Apify proxy.

## `proxyConfiguration` (type: `object`):

Optional Apify or custom proxy configuration. When supplied, it takes priority over Proxy Mode. AU residential proxy is useful only if lighter routes are blocked.

## `includeRaw` (type: `boolean`):

Optional. Include the unnormalized source payload in the OUTPUT summary.

## Actor input object example

```json
{
  "operation": "search",
  "query": "sale",
  "page": 1,
  "limit": 5,
  "enabled_only": true,
  "proxyMode": "apify",
  "includeRaw": false
}
```

# Actor output Schema

## `results` (type: `string`):

Normalized dataset items returned by the actor.

## `summary` (type: `string`):

Run summary JSON stored under the OUTPUT key.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {};

// Run the Actor and wait for it to finish
const run = await client.actor("dromb/big-w-au-product-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {}

# Run the Actor and wait for it to finish
run = client.actor("dromb/big-w-au-product-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{}' |
apify call dromb/big-w-au-product-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "command": "npx",
            "args": [
                "mcp-remote",
                "https://mcp.apify.com/?tools=dromb/big-w-au-product-scraper",
                "--header",
                "Authorization: Bearer <YOUR_API_TOKEN>"
            ]
        }
    }
}

```

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/OEDcSGrgdCsAu2eTX/builds/PgtvtXZvPhsJeS37k/openapi.json