Dataset Egress Firewall — only the fields you name leave, forbidden names scrubbed at any depth

You're about to hand someone a dataset. This keeps only the fields you allowlist and catches forbidden field-names — password, token, api_key — at any nesting depth. Press Start on the defaults and watch it catch four planted leaks — two at the top, two buried a level down — each named by its exact path:

"leak_paths": {
  "api_key": 1,               // top level
  "internal_notes": 1,        // top level (matches the forbidden name "internal_")
  "meta.session_token": 1,    // one level down
  "user.password": 1          // one level down
}

Four caught — and you pay for two. meta.session_token and user.password sit inside meta and user, fields you're shipping, so they'd have leaked: those are the catches you're billed for. The two top-level ones (api_key, internal_notes) weren't on your allowlist — they were dropped anyway, so they're caught free. You pay only for the secret that was actually about to leave (leaks_billed in the report says how many that was).

It governs field names, not values: a secret pasted inside an allowed free-text field has no field name to catch, so this is not a value scanner — said up front, not buried. Every clean record comes out with a tamper-evident tag (a checksum, or HMAC when you set a secret). The rest of this page is that promise in engineering terms.

In plain words: you've collected some data, and you're about to hand it to someone. Is there anything inside that shouldn't leave with it — a password, a key, someone's private details buried three layers deep? This tool stands at the door. You name what's allowed out, and only that gets through; everything else is scrubbed, however deep it hides, and you get a plain receipt of exactly what was caught. If the door itself were ever broken, it refuses to open at all rather than let one record slip. That's the whole idea. The rest of this page is the same promise in engineering terms.

Point it at any records — pasted inline or straight from another Actor's dataset — name the top-level fields allowed out, and only those come through. Unknown fields are dropped, forbidden field-names (password, token, ssn, debug machinery) are scrubbed at any nesting depth, every clean record carries a tamper-evident tag, and the run hands you a leak report you can give a client or an auditor as-is. (The allowlist governs top-level fields; a nested object keeps its own sub-keys unless you also set Allowed nested keys.)

The part that is genuinely unusual: a version of this tool that could leak a forbidden-named field cannot be built. The leak checks run inside the build itself, so a broken door fails to compile and never deploys — then re-proves itself at startup before it reads a single record. You are not trusting a setting, or us; you can check it — the checks are in the repo, the build log is public. The bound, in the same breath: "leak" means a forbidden field-name escaping. A secret pasted inside an allowed free-text field has no field name to catch, so this is not a value scanner — see What it does not catch.

No AI is involved at any point, so your records are never sent to a model, and the same input always gives the same output.

---

What's inside

Four organs, ~470 lines across src/, no dependencies beyond the Apify SDK. Everything below is standard library — there is nothing here to trust that you cannot read.

The door	Projects each record onto your allowlist, then scrubs forbidden names at any nesting depth — inside lists, inside nested objects, inside objects inside lists. Fail-closed: if the door can't prove itself, nothing is emitted.
The window	A live page you can open while a run is still walking — running tallies, leak paths appearing as they're caught. GET only, by construction: it is a window, not a hand. If it can't bind a port, the run continues unlit rather than failing. Watertight is not lightproof.
The seals	Every clean record carries a tamper-evident tag, and so does the report itself. verify_provenance means whoever receives your audit can check it without you — the point is that they don't have to take your word.
The gate	The firewall's own leak checks run inside the Docker build — not a test suite you can skip. A build that could leak fails to compile and never reaches the platform.

---

What it's used for

Use it when you want to:

You want to…	Do this
Ship scraped data to a client and prove internal fields never left	scrub mode + the leak report as your receipt
Enforce which fields may leave under GDPR/CCPA egress rules	scrub mode, allowlist = the fields legal signed off on
Block a release when anything forbidden is present upstream	fail-closed mode as a CI-style shipping gate
Strip debug machinery, raw traces, and prompt internals from AI-pipeline output	scrub mode + the default forbidden vocabulary
Make any scraper in the Store safe to hand onward	chain it behind that Actor (see Chaining)
Let a consumer verify a record came from your firewall untampered	set provenanceSecret, share it with them

What it does not do:

• A secret scanner. It governs by field name, not by value. An API key sitting inside an allowed free-text field (text, description, a log line) is not caught — there's no field name to catch. Read What it does not catch before you rely on it.
• A PII redactor for prose. It removes whole fields; it does not rewrite sentences to remove names, emails, or numbers inside them.
• An encryption or access-control layer. It decides what leaves. It doesn't protect what stays.

---

Quick start (about 20 seconds)

1. Press Start with the defaults. The input ships two demo records carrying four planted problems — a top-level api_key, an internal_notes field, and two buried one level down (meta.session_token, user.password). Run it as-is and watch all four get caught.
2. Open the Dataset tab — two clean records, allowlist fields only, each stamped with a provenance tag.
3. Open the Key-value store tab and click LEAK_REPORT — the receipt: what was caught, where, and what was dropped.
4. Now point it at your own data. Either paste records into Items, or put another Actor's dataset ID into Source dataset ID.
5. Set your allowlist. Name only the fields you actually mean to ship. Everything else stops existing in the output.

Don't know your field names? Two ways: look at the upstream Actor's dataset and note the keys you want — or just run this once with a rough allowlist and read dropped_fields in the leak report. It tallies every top-level field you dropped, which is the fastest field inventory you'll get. (Top-level only — it doesn't enumerate keys inside nested objects.)

---

Why allowlist, not blocklist

A blocklist enumerates what you fear; an allowlist names what you mean to ship. Everything not named simply does not exist in the output — absence, not exposure. The forbidden-substring tripwire runs behind the allowlist as defense-in-depth, catching machinery that hides inside allowed structures (rows[3].api_key, meta.session_token, …).

---

Two modes

Two ways to stand at the door: clean whatever passes through, or refuse to open the door at all until what's behind it is fixed.

Mode	What happens
Scrub (scrub)	Every record is projected + scrubbed and emitted clean. The leak report tells you exactly what was caught and where.
Fail-closed (fail-closed)	If any forbidden field exists anywhere in the input, the run aborts and emits nothing. Use it as a shipping gate: your pipeline is broken upstream — nothing ships until it's fixed. Because the default vocabulary is over-broad on purpose, a harmless field like links_internal will also trip it — run scrub once to see what would fire, and narrow the vocabulary before you gate on it.

---

The default forbidden vocabulary

Leave Forbidden key substrings empty and these are used. Matching is case-insensitive substring at any nesting depth — so api_key also catches API_Key, apiKeyOld, and legacy_api_key_backup:

Credentials — password, passwd, secret, token, api_key, apikey, authorization, bearer, cookie, private_key

Personal & financial — ssn, social_security, credit_card, card_number, cvv, iban

Machinery — _debug, debug_, _internal, internal_, _raw, raw_, stacktrace, stack_trace

It is over-broad on purpose — a tripwire should err toward catching. Supply your own list to replace it entirely when your domain has its own names worth fearing (patient_id, account_no, auth_ref).

---

What comes back

The clean dataset — one record per input record

Your records wearing only their allowed fields, each stamped with a verifiable seal:

{
  "url": "https://example.com/pricing",
  "title": "Pricing",
  "text": "Start on the free plan…",
  "meta": { "lang": "en" },
  "provenance": "fw1:9f2c41d0a7b3…"
}

LEAK_REPORT — in the run's key-value store

{
  "mode": "scrub",
  "allowlist": ["url", "title", "text", "meta"],
  "report_tag": "fw1:9b339bac6aa7bb6f9f18b489ab789f2b",
  "failed": false,
  "records_seen": 1240,
  "records_emitted": 1240,
  "records_with_leaks": 37,
  "leaks_total": 41,
  "leaks_billed": 15,
  "leak_paths": { "rows[].api_key": 22, "meta.session_token": 15, "user.password": 4 },
  "dropped_fields": { "internal_notes": 1240, "_debug": 96 },
  "non_dict_records_skipped": 0
}

Field	What it tells you
allowlist	The door itself — the exact fields this run permitted. Without it the report cannot support its own claim: a field you allowed but that never appeared is invisible, and an allowlist widened between two runs would leave no trace. Diff it across runs to see whether the same door was used.
records_seen / records_emitted	Read vs. shipped. In fail-closed a tripped run emits 0.
records_with_leaks / leaks_total	How many records were dirty, and how many findings in total — one record can carry several.
leaks_billed	The subset of leaks_total that would actually have shipped — a forbidden field inside an allowlisted path. The only thing leak-detected charges for; a forbidden field the allowlist was already dropping is caught free. (In the example above, only the 15 meta.session_token hits are billed — rows and user weren't allowlisted.)
leak_paths	The auditor's column. Every forbidden field by its dotted path, with a count — everything caught, billed or not. rows[].api_key means it was inside a list.
dropped_fields	Top-level fields removed by the allowlist alone — not dangerous, just not named. Also your field inventory. (Keys dropped inside nested objects by allowedNestedKeys are not tallied here.)
non_dict_records_skipped	Records that weren't objects (a bare string, number, or list). See Notes.
report_tag	A seal on the receipt itself. Every clean record is tamper-evident; this makes the document describing them tamper-evident too, so the claim is no less protected than the data. Unkeyed it is a checksum; with provenanceSecret shared to whoever receives the audit, it is HMAC — they can detect an altered report themselves rather than taking your word for it.
failed	true only when fail-closed tripped.

Hand this file to a client or auditor as-is: it's the receipt for the claim that nothing else left the building.

---

The window (live view)

Watertight is not lightproof. While a run walks, the Actor serves a read-only live view — the leak report as it forms: records seen, records emitted, leaks caught so far, updating every two seconds. Open the run's live-view URL in Console and watch the receipt being written.

It is a window, not a hand: GET-only, no control endpoints, nothing that can touch the run. And it obeys the same law as the door — the page shows only allowed, aggregate counts, so no record content, no forbidden value, and no internal machinery can ever appear in it. That's not a policy; it's a build-gate invariant (a build whose window could leak fails and never deploys).

---

Chaining behind another Actor

This is the common case: a scraper produces data, and this stands between it and whoever receives it.

1. Run the upstream Actor.
2. Open its run → Dataset tab → copy the dataset ID.
3. Paste it into Source dataset ID here, set your allowlist, run.

The clean copy lands in this run's dataset — the upstream data is never modified. You can combine a source dataset with inline Items in the same run; both are read, both are governed.

It's deterministic and near-free to chain everywhere your data is clean — and where it isn't, it charges for the breach it just prevented, not for the running. Being safe is never the expensive option; only being unsafe upstream costs, and it costs exactly where the danger was.

A real example of the tripwire being over-broad

Chain the Evidence-Quoted Page Classifier into this Actor and the receipt says leak_paths: {"links_internal": 2} on a completely clean run.

Nothing leaked. links_internal is one of that Actor's published SEO fields — the count of links pointing to the same site — and it is caught because it contains _internal, which is in the default vocabulary. This is the tripwire doing exactly what this page says it does: erring toward catching, over-broadly, on purpose.

Two honest ways to handle it, and which one is right is yours:

• Leave it. The field is dropped anyway unless you allowlist it — and since it's dropped, it's caught free (only would-have-shipped catches bill). A receipt that over-reports is the direction you want a firewall to err in.
• Narrow the vocabulary. Supply your own forbiddenKeySubstrings without _internal, or with internal_ instead, which still guards internal_notes while leaving links_internal alone.

Worth knowing before it surprises you in front of a client — and worth seeing, because it is the clearest demonstration on this page that the tripwire is a blunt instrument standing behind a precise one, not a clever one pretending to judgement.

---

Input reference

{
  "datasetId": "abc123DatasetId",            // clean another actor's output…
  "items": [{ "url": "…", "…": "…" }],       // …and/or inline records
  "allowFields": ["url", "title", "text", "meta"],
  "forbiddenKeySubstrings": ["api_key", "token", "password", "_internal"],  // empty = built-in defaults
  "allowedNestedKeys": [],                   // optional strict mode for nested objects
  "mode": "scrub",                           // or "fail-closed"
  "provenanceSecret": "…",                   // optional — upgrades tags to HMAC (fw1:)
  "stampProvenance": true,                   // default true — set false to skip the provenance tag
  "maxItems": 10000
}

allowFields is the only required input. allowedNestedKeys is the strict setting: leave it empty and nested structure survives intact (minus forbidden keys); set it and keys inside nested objects must also be named to survive.

---

Verifying provenance

The seal, in plain words: every clean record carries a stamp that can be re-computed from the record itself — proof it passed this firewall and hasn't been touched since. In engineering terms:

fw0: tags are a SHA-256 over the record's clean fields (sorted-key JSON, provenance excluded) — anyone can recompute them. fw1: tags are an HMAC-SHA-256 with a key derived from your provenanceSecret (PBKDF2-HMAC-SHA256, salt equinox-egress-v1, 200k iterations, 32 bytes) — only holders of the secret can verify, and no tag reveals anything about what was removed.

import hashlib, hmac, json
body = {k: v for k, v in record.items() if k != "provenance"}
payload = json.dumps(body, sort_keys=True, ensure_ascii=False, default=str).encode()
assert record["provenance"] == "fw0:" + hashlib.sha256(payload).hexdigest()[:32]

---

Pricing

You pay only for what verifiably happened — every charge has a receipt, nothing else is ever billed. Pay-per-event, three meters that add together on every run:

Meter	When it fires	Price
run-started	Once per run — boot + firewall self-check (the fixed startup, priced at its own cost)	$1.70 / 1,000
record-cleaned	Once per record projected, scrubbed, and emitted with its verifiable seal	$0.10 / 1,000
leak-detected	Once per forbidden field that would have shipped — one inside an allowlisted field, scrubbed before it left. Each one is a credential or token that was about to leave: a prevented breach.	$5.00 / catch

Your total is run-started + (records × record-cleaned) + (leaks_billed × leak-detected). The first two meters are near-free — the engine is pure Python standard library, so a clean pass costs almost nothing. You pay for the catch that mattered — a forbidden field caught inside a field you were shipping is a breach that doesn't happen there. A forbidden field the allowlist was already dropping is caught free: nothing was going to leave, so nothing is billed. (leaks_total in the report counts everything caught; leaks_billed counts the ones you pay for.)

What a real run costs

Concrete, in scrub mode (the common case):

• A clean run — the common case: 10,000 records, zero secrets about to ship ≈ $1.00. A pipeline that isn't leaking never fires the catch meter; it pays for the plumbing and nothing else.
• One real catch: that same ~$1.00, plus $5 for the single credential that was about to leave inside a field you were shipping. Five dollars to stop one breach.
• Many catches — the alarm, not a surprise bill: 300 real would-have-shipped secrets ≈ $1.00 + 300 × $5 = $1,501. That isn't a pricing problem, it's a diagnosis: your upstream is hemorrhaging credentials — each one a ~$150+ breach, ~$45,000 of exposure — and this run is the cheapest incident report you'll ever get, plus the loudest signal to go fix it. The bill scales with the danger, exactly as it should. (Forbidden fields headed for the bin anyway are caught free and never appear here.)

A clean dataset stays cheap, and you never pay for a forbidden field you weren't shipping — those are dropped free. The one way a harmless field bills: you allowlisted it (meant to ship it) and the over-broad default vocabulary flags its name — e.g. links_internal (it contains internal_). Narrow the vocabulary to your domain and even that stops. When a billed catch is a real secret the price is a bargain: a caught credential is a breach that doesn't happen (industry average ~$150/record, IBM 2025 Cost of a Data Breach), and $5 is ~3% of that — the breach it prevents stays yours, not ours to charge for. (In fail-closed mode a trip aborts and emits nothing — you pay the startup and the would-have-shipped catches that stopped it, never for records never shipped.)

How this price was derived

Priced by derivation, not by market-positioning. The two utility meters sit at their own measured cost: from a real run (2026-07-22), startup ≈ $0.0017/run (run-started) and the marginal per-record cost is microscopic — pure stdlib — so record-cleaned sits at $0.0001/record. Their use is small (a boot; a sealed clean copy), so cost and use coincide there. The catch is different. It bills only for forbidden fields that would have shipped (inside an allowlisted field), so "catch = a prevented breach" is literal — junk the allowlist was already dropping is caught free. What should a prevented breach be priced at? Not the breach itself — that surplus is yours, given, never charged (a caught credential can cost ~$150/record and far more; you pay $5, ~3% of the floor). And not the fraction of a cent it costs to find — pricing the catch at its compute (the prior $0.0005) was the real error: a 10,000× undersell that told the reader this was a toy. $5 is set to tell the truth about worth — above the time it took to build, well under the breach it prevents, and high enough to read as the serious tool it is. A price too cheap doesn't just lose money; it lies that the work is worthless, and drags every honest tool in the category down with it. Every charged event has a receipt (the record, or its line in LEAK_REPORT); nothing is charged for what did not happen. When the meters change, the price is re-derived and this section updated.

---

Provable vs. promised — a structural difference, not a slogan

Most tools that clean or redact data work one of three ways, and each asks for a trust this one doesn't:

• Blocklists / pattern rules enumerate what to hide — so anything you didn't think to name walks straight through. An allowlist inverts it: only the fields you named survive, and everything else simply doesn't exist in the output. Absence, not omission.
• ML / heuristic redaction is probabilistic — it usually catches the secret and can't promise it always will. This engine is deterministic and rule-exact: the same input yields the same output, every time.
• Policy you configure is a promise that the config held at runtime. Here, fail-closed isn't a setting — the egress invariants run inside the Docker build, so a version that could leak fails to build and never ships. In plain words: a version of this tool that could leak cannot even be built. The firewall then re-proves itself at startup before reading a single record.

Two more things you get by construction rather than by promise: your data never leaves — the firewall engine is pure Python standard library with no AI calls and no outbound requests, and your records are processed entirely in-container. (The only network the Actor does at all is reading the source dataset you point it at, through Apify's own API — the platform handing you back your own data. Nothing is sent anywhere else.) And the leak report is a deliverable — receipts, not assurances.

None of that calls another tool careless. It's the difference between promised and provable — and it hands you the proof: read the build gate, plant a forbidden field and watch the run refuse, recompute any provenance tag yourself. You don't have to believe the door holds. You can check it.

---

What it does not catch

Stated plainly, because a security tool you misunderstand is worse than none:

• Secrets by value, inside allowed fields. The firewall governs field names. A key pasted into an allowed text or description field has no forbidden field name, so it passes. If free text might carry secrets, either don't allowlist that field, or scan it before this step.
• Names you didn't think of, if you replaced the defaults. Supplying forbiddenKeySubstrings replaces the built-in vocabulary rather than adding to it. Include the defaults you still want.
• Anything in a record that isn't an object. Bare strings, numbers, and lists give the allowlist nothing to hold; they're dropped and counted, never passed through raw.
• What stays behind. This governs egress. It doesn't secure your storage, your access control, or the upstream system that produced the secret in the first place.

---

FAQ

What's the difference between leak_paths and dropped_fields? leak_paths = forbidden things caught by the tripwire (the security finding, at any depth). dropped_fields = top-level fields removed simply because you didn't name them (routine, and the bulk of the volume).

I got charged leak-detected in fail-closed mode even though nothing shipped. You're charged only for the would-have-shipped catches (leaks_billed) that stopped the release — a forbidden field inside a field you were keeping. Forbidden junk the allowlist was already dropping is caught free, even in fail-closed. You never pay for records never emitted.

Can I use it on a dataset that's still being written? It reads what's there when it runs. Chain it after the upstream run completes.

Does it modify the upstream dataset? Never. It reads, and writes a clean copy to its own dataset.

Which provenance tag do I want? fw0: if you just need tamper-evidence anyone can check. fw1: (set provenanceSecret) if you need attributable proof that a record came from your firewall specifically.

---

Notes

• The firewall governs by field names, so a record with no named fields — a bare string, number, or list where a {...} object should be — gives the allowlist nothing to hold. Such records are never passed through raw: they are dropped and counted in the report (non_dict_records_skipped). What can't be governed doesn't get emitted.
• Matching is case-insensitive substring, over-broad on purpose: a tripwire should catch API_Key, apiKeyOld, and legacy_api_key_backup alike. Tune the vocabulary to your domain.
• The firewall runs a self-check before reading a single record and the build itself fails if any egress invariant breaks — fail-closed is not a setting here; it's the construction.

---

Pairs with: Evidence-Quoted Page Classifier — and why the pair is the point

Need the data before you govern it leaving? Evidence-Quoted Page Classifier reads pages and emits only what it can quote — no AI, same terms. Chain it into this one and both the gathering and the handing-over are receipted.

But the pairing is worth more than convenience, because each closes the other's gap.

This Actor governs by field name: it decides which fields may leave. That is complete at the structural grain and, alone, blind at the content grain — a key pasted inside an allowed description has no field name to catch, which is why that limit is stated at the top of this page.

The classifier governs the other grain. It emits only what a declared label licensed, with the sentence that earned it; anything else is simply not in its output. So text that no label licensed does not leave — not because a scanner recognised it as dangerous, but because nothing unlicensed is ever emitted in the first place.

Put them in a line and you have the same rule at both grains:

raw records ──► classifier: only licensed, quoted content ──► firewall: only named fields ──► out

A scanner enumerates what you fear, so it is always one novelty behind. An allowlist names what you mean, so what you did not name was never a candidate. Running that discipline at the field grain and the content grain is why this is a structural guarantee rather than a probabilistic one. Neither Actor claims it alone. Together, it is the whole claim.

---

Servicing of terms

We flipped the label on purpose: not terms that govern the service — a service that keeps its terms. Here they are, short enough to actually read: your data stays yours (processed in-container, never retained after the run, never sold, never trained on, never sent to any external service — this engine makes no outbound calls at all); no rights are claimed over your inputs or outputs; you pay only for receipted events; you can leave anytime with nothing held. We ask one term back: don't use the firewall as a laundry for taking — it exists to help you ship data cleanly, to people who deserve clean data.