# Dataset Delta — Added, Removed & Changed Records (`orangepinlabs/dataset-delta`) Actor

Compare two Apify datasets or track one dataset between runs. Emit only added, removed, and changed records with exact changed fields.

- **URL**: https://apify.com/orangepinlabs/dataset-delta.md
- **Developed by:** [Orange Pin Labs](https://apify.com/orangepinlabs) (community)
- **Categories:** Developer tools
- **Stats:** 1 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $2.00 / 1,000 dataset changes

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Dataset Delta

Turn full scraper outputs into a small, actionable change feed.

Dataset Delta compares any two Apify datasets—or tracks one dataset between scheduled runs—and returns only records that were added, removed, or changed. It is designed for price monitoring, job alerts, inventory tracking, lead-list updates, regulatory feeds, and any recurring Actor workflow where users care about **what changed**, not another complete export.

### What it produces

Each output row contains:

- `changeType`: `added`, `removed`, `changed`, or optionally `unchanged`
- `key`: the unique field or composite key used to match the record
- `changedFields`: exact dot paths that changed
- `before`: the old record
- `after`: the new record

The run's `OUTPUT` record contains totals and source metadata.

### Modes

#### Compare two datasets

Provide an older `baselineDatasetId` and a newer `currentDatasetId`. This mode is stateless and supports up to 50,000 records per dataset.

#### Track one dataset

Provide the current dataset and a stable `stateKey`. Dataset Delta saves a private snapshot in a named key-value store. By default, the first run only initializes the snapshot and emits no changes; enable `emitInitialSnapshot` if every existing record should be emitted as added. Later runs report the delta. Use a different `stateKey` for each scheduled source.

If a user's maximum charge is reached before every change is emitted, the snapshot is deliberately not advanced. The next run can therefore recover the unprocessed changes.

Tracking snapshots are capped at 8 MB. Use Compare mode for larger datasets.

### Example input

```json
{
  "mode": "compare",
  "baselineDatasetId": "OLDER_DATASET_ID",
  "currentDatasetId": "NEWER_DATASET_ID",
  "keyFields": ["url"],
  "ignoreFields": ["scrapedAt", "metadata.requestId"],
  "maxItems": 10000
}
```

For records that are only unique in combination, use a composite key:

```json
{
  "keyFields": ["company.id", "locationId"]
}
```

Duplicate or missing keys fail clearly instead of silently producing incorrect changes.

### Pricing

Dataset Delta uses pay-per-event pricing:

- Actor start: Apify's standard low-cost start event
- `delta-record`: **$0.002 per emitted change**

Unchanged records do not create result charges unless you explicitly enable `includeUnchanged`.

### Requirements and limits

- Both datasets must be accessible to the account running Dataset Delta.
- Every record must contain the configured unique key fields.
- Duplicate keys fail the run instead of producing an ambiguous comparison.
- Compare mode supports at most 50,000 records per dataset.
- Tracking snapshots are limited to 8 MB; use Compare mode for larger datasets.
- No external API key, proxy, browser, LLM, or third-party service is required.

### Local development

Requires Node.js 20 or newer.

```bash
npm install
npm test
npm start
```

The checked-in local storage fixture compares two tiny datasets and ignores `scrapedAt`.

# Actor input Schema

## `mode` (type: `string`):

Compare two datasets now, or compare a dataset with this Actor's saved snapshot.

## `currentDatasetId` (type: `string`):

The newer/current dataset ID or name.

## `baselineDatasetId` (type: `string`):

Required in Compare mode. The older dataset ID or name.

## `keyFields` (type: `array`):

One or more fields that uniquely identify a record, such as url, id, or company.id.

## `ignoreFields` (type: `array`):

Fields that should not trigger a change, such as scrapedAt or metadata.requestId. Dot paths are supported.

## `stateKey` (type: `string`):

Track mode only. Use a different value for each scheduled source.

## `emitInitialSnapshot` (type: `boolean`):

Off by default so initializing a tracker does not create a large, chargeable change feed.

## `maxItems` (type: `integer`):

Safety limit applied separately to the current and baseline datasets.

## `includeUnchanged` (type: `boolean`):

Normally off. Enable for debugging or complete audit exports.

## Actor input object example

```json
{
  "mode": "compare",
  "keyFields": [
    "url"
  ],
  "ignoreFields": [],
  "stateKey": "default",
  "emitInitialSnapshot": false,
  "maxItems": 10000,
  "includeUnchanged": false
}
```

# Actor output Schema

## `changes` (type: `string`):

Added, removed, changed, and optionally unchanged records. Each item includes the matching key, changed field paths, and before/after values.

## `summary` (type: `string`):

Run totals, source dataset IDs, matching key fields, ignored fields, tracking state, and charge-limit status.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {};

// Run the Actor and wait for it to finish
const run = await client.actor("orangepinlabs/dataset-delta").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {}

# Run the Actor and wait for it to finish
run = client.actor("orangepinlabs/dataset-delta").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{}' |
apify call orangepinlabs/dataset-delta --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "command": "npx",
            "args": [
                "mcp-remote",
                "https://mcp.apify.com/?tools=orangepinlabs/dataset-delta",
                "--header",
                "Authorization: Bearer <YOUR_API_TOKEN>"
            ]
        }
    }
}

```

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/acts/8eD5A514tAWfhb6Nt/builds/vpGuLTEzd0XtMOLcx/openapi.json