X Ads Transparency Scraper

Export X's official historical political advertising archive (2018-05-24 through 2019-11-22) and US-only issue advertising archive (2018-08-08 through 2019-11-22) as filterable, provenance-rich dataset records.

This Actor processes fixed official archive files. It does not scrape current ads, hydrate posts, use the authenticated DSA export API, or silently substitute test data when a live source is unavailable.

Use it for historical political advertising research, US issue advertising research, advertiser and agency analysis, spend and impression analysis, targeting studies, or reproducible archive exports. Every emitted record includes the source URL, archive period, retrieval mode and timestamp, and content SHA-256.

One dataset item represents one promoted post. Results work with Apify's JSON, CSV, Excel, and API dataset exports.

Why use this Actor?

• Filter by advertiser, text, creative date, minimum spend, and minimum impressions.
• Preserve source-provided totals, campaign breakdowns, targeting, and disclosed billing data.
• Detect source totals that differ from campaign sums without rewriting either value.
• Test integrations deterministically with synthetic fixtures and the same normalized schema.
• Fail clearly when a selected official archive is unavailable; fixture data is never a fallback.

Input

Field	Default	Behavior
mode	"live"	live downloads official fixed URLs; fixture uses bundled synthetic data with no network requests
sources	["political"]	One or both of political, issue; caller order is preserved
advertisers	[]	Case-insensitive substring OR across advertiser handle and name; leading @ ignored
query	""	Case-insensitive substring across creative text, advertiser identity, agency, and reported URLs
startDate / endDate	unset	Inclusive creative creation date, YYYY-MM-DD
minSpendUsd	unset	Inclusive source-provided total spend
minImpressions	unset	Inclusive source-provided total impressions
maxItems	1000	Global emitted-record cap, 1–100000
proxyConfiguration	disabled	Accepted for Actor compatibility; current fixed archive downloader does not route through proxy

Exact deterministic fixture input:

{
  "mode": "fixture",
  "sources": ["political", "issue"],
  "advertisers": [],
  "query": "",
  "maxItems": 10,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}

This input emits the three records shown by the current bundled fixtures. Fixture mode exists for deterministic integration testing. Every fixture record sets archive.retrievalMode to fixture; fixture content is synthetic and not evidence of a real ad.

Output

Default dataset contains one normalized record per promoted post. Default key-value store record OUTPUT contains run status, selected sources, archive SHA-256 hashes, byte and record counts, warning counts, limit state, and failure details when available.

Verified fixture record (retrievedAt changes on each run):

{
  "schemaVersion": "1.0",
  "recordId": "political:1144749820298612737",
  "source": "political",
  "adType": "political-campaigning",
  "archive": {
    "sourceUrl": "https://business.x.com/content/dam/business-twitter/help/ads-transparency-archive/political.txt",
    "periodStart": "2018-05-24",
    "periodEnd": "2019-11-22",
    "retrievalMode": "fixture",
    "retrievedAt": "2026-07-23T12:39:37.233Z",
    "contentSha256": "34b9fbe7d2aeb967695ac0d729c758fe5513af61cd9254455540f12ea1209e8d"
  },
  "advertiser": {
    "handle": "CivicFuture",
    "name": "@CivicFuture - U.S. Political Campaigning",
    "agency": null,
    "bioUrl": "https://example.test/civic-future"
  },
  "creative": {
    "tweetId": "1144749820298612737",
    "tweetUrl": "https://x.com/i/status/1144749820298612737",
    "createdDate": "2019-06-28",
    "text": "Synthetic fixture about education https://example.test/education",
    "reportedUrls": ["https://example.test/education"]
  },
  "metrics": {
    "spendUsd": 80,
    "impressions": 17667
  },
  "campaigns": [
    {
      "startAt": "2019-06-28T21:28:16.000Z",
      "endAt": "2019-07-04T06:59:00.000Z",
      "spendUsd": 100,
      "impressions": 19312,
      "numberOfTweets": 1,
      "targeting": [
        {
          "type": "GEO",
          "value": "Illinois- US",
          "impressions": 1034
        }
      ]
    }
  ],
  "billing": {
    "insertionOrders": [],
    "creditCards": [
      {
        "payerName": "Synthetic Payer",
        "billingCity": "Test City",
        "billingRegion": "TS",
        "billingPostalCode": "00000",
        "spendUsd": 0.00015
      }
    ]
  },
  "qualityWarnings": [
    "SOURCE_SPEND_TOTAL_DIFFERS_FROM_CAMPAIGN_SUM",
    "SOURCE_IMPRESSION_TOTAL_DIFFERS_FROM_CAMPAIGN_SUM"
  ]
}

Exact record and summary contracts live in .actor/dataset_schema.json and .actor/output_summary_schema.json.

qualityWarnings identifies optional-date parse failures, unknown targeting types, and source totals that differ from campaign sums. Source-disclosed billing data is account-scoped and may repeat across ads. Treat payer and location fields as sensitive personal data and apply suitable access and retention controls.

Empty filter results succeed with an empty dataset and recordsEmitted: 0. Live source HTTP errors fail explicitly; live mode never falls back to fixtures. Local access to X's archive CDN may return 404 even when cloud access works.

Pricing and platform costs

Planned Store pricing: Pay-Per-Event (PPE) only. Each ad-record event costs USD 0.007 per emitted record, equal to USD 7.00 per 1,000 records. No result means no ad-record charge. This local preparation does not change Cloud pricing.

Apify platform usage costs are passed to users and are separate from PPE charges. Downloads, archive parsing, dataset storage, and optional proxy use can create platform usage. maxItems caps emitted records, but selected archives are fully validated before emission.

Scope and limitations

• Supported data is limited to the fixed 2018–2019 political and US issue archives and their source-provided fields.
• Issue data is US-only.
• A derived creative.tweetUrl does not prove that a historical post remains available.
• Source availability is controlled by X and can change.
• No current-ad discovery, commercial archive, post hydration, media recovery, engagement enrichment, browser automation, login, or authenticated DSA API access is included.

Local development

Requires Node.js 22+ and pnpm 10.30.0.

pnpm install --frozen-lockfile
pnpm test
pnpm typecheck
pnpm build
pnpm lint
apify validate-schema
apify run

For local fixture smoke, save this as storage/key_value_stores/default/INPUT.json before apify run:

{ "mode": "fixture", "sources": ["political", "issue"], "maxItems": 10 }

Archive size, parsing, and dataset writes drive runtime cost. maxItems limits output, but selected archives are fully validated before emission.

Independence and historical terminology

This project is independent and is not endorsed by X Corp. “Twitter” may appear only in historical source paths or context associated with the archive period.

License

MIT; see LICENSE.