# X Ads Transparency Scraper / Twitter Ads (`truenorth/x-ads-transparency-scraper`) Actor Converts official historical X political and US issue advertising archives into filterable, provenance-rich records. - **URL**: https://apify.com/truenorth/x-ads-transparency-scraper.md - **Developed by:** [TrueNorth](https://apify.com/truenorth) (community) - **Categories:** Social media, Lead generation, MCP servers - **Stats:** 1 total users, 0 monthly users, 0.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $3.50 / 1,000 results This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. - **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use). - **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`. - **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`. - **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`. - **Any other language** — the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## X Ads Transparency Scraper Export X's official historical political advertising archive (2018-05-24 through 2019-11-22) and US-only issue advertising archive (2018-08-08 through 2019-11-22) as filterable, provenance-rich dataset records. This Actor processes fixed official archive files. It does not scrape current ads, hydrate posts, use the authenticated DSA export API, or silently substitute test data when a live source is unavailable. Use it for historical political advertising research, US issue advertising research, advertiser and agency analysis, spend and impression analysis, targeting studies, or reproducible archive exports. Every emitted record includes the source URL, archive period, retrieval mode and timestamp, and content SHA-256. One dataset item represents one promoted post. Results work with Apify's JSON, CSV, Excel, and API dataset exports. ### Why use this Actor? - Filter by advertiser, text, creative date, minimum spend, and minimum impressions. - Preserve source-provided totals, campaign breakdowns, targeting, and disclosed billing data. - Detect source totals that differ from campaign sums without rewriting either value. - Test integrations deterministically with synthetic fixtures and the same normalized schema. - Fail clearly when a selected official archive is unavailable; fixture data is never a fallback. ### Input | Field | Default | Behavior | | ----------------------- | --------------- | ---------------------------------------------------------------------------------------------------- | | `mode` | `"live"` | `live` downloads official fixed URLs; `fixture` uses bundled synthetic data with no network requests | | `sources` | `["political"]` | One or both of `political`, `issue`; caller order is preserved | | `advertisers` | `[]` | Case-insensitive substring OR across advertiser handle and name; leading `@` ignored | | `query` | `""` | Case-insensitive substring across creative text, advertiser identity, agency, and reported URLs | | `startDate` / `endDate` | unset | Inclusive creative creation date, `YYYY-MM-DD` | | `minSpendUsd` | unset | Inclusive source-provided total spend | | `minImpressions` | unset | Inclusive source-provided total impressions | | `maxItems` | `1000` | Global emitted-record cap, 1–100000 | | `proxyConfiguration` | disabled | Accepted for Actor compatibility; current fixed archive downloader does not route through proxy | Exact deterministic fixture input: ```json { "mode": "fixture", "sources": ["political", "issue"], "advertisers": [], "query": "", "maxItems": 10, "proxyConfiguration": { "useApifyProxy": false } } ``` This input emits the three records shown by the current bundled fixtures. Fixture mode exists for deterministic integration testing. Every fixture record sets `archive.retrievalMode` to `fixture`; fixture content is synthetic and not evidence of a real ad. ### Output Default dataset contains one normalized record per promoted post. Default key-value store record `OUTPUT` contains run status, selected sources, archive SHA-256 hashes, byte and record counts, warning counts, limit state, and failure details when available. Verified fixture record (`retrievedAt` changes on each run): ```json { "schemaVersion": "1.0", "recordId": "political:1144749820298612737", "source": "political", "adType": "political-campaigning", "archive": { "sourceUrl": "https://business.x.com/content/dam/business-twitter/help/ads-transparency-archive/political.txt", "periodStart": "2018-05-24", "periodEnd": "2019-11-22", "retrievalMode": "fixture", "retrievedAt": "2026-07-23T12:39:37.233Z", "contentSha256": "34b9fbe7d2aeb967695ac0d729c758fe5513af61cd9254455540f12ea1209e8d" }, "advertiser": { "handle": "CivicFuture", "name": "@CivicFuture - U.S. Political Campaigning", "agency": null, "bioUrl": "https://example.test/civic-future" }, "creative": { "tweetId": "1144749820298612737", "tweetUrl": "https://x.com/i/status/1144749820298612737", "createdDate": "2019-06-28", "text": "Synthetic fixture about education https://example.test/education", "reportedUrls": ["https://example.test/education"] }, "metrics": { "spendUsd": 80, "impressions": 17667 }, "campaigns": [ { "startAt": "2019-06-28T21:28:16.000Z", "endAt": "2019-07-04T06:59:00.000Z", "spendUsd": 100, "impressions": 19312, "numberOfTweets": 1, "targeting": [ { "type": "GEO", "value": "Illinois- US", "impressions": 1034 } ] } ], "billing": { "insertionOrders": [], "creditCards": [ { "payerName": "Synthetic Payer", "billingCity": "Test City", "billingRegion": "TS", "billingPostalCode": "00000", "spendUsd": 0.00015 } ] }, "qualityWarnings": [ "SOURCE_SPEND_TOTAL_DIFFERS_FROM_CAMPAIGN_SUM", "SOURCE_IMPRESSION_TOTAL_DIFFERS_FROM_CAMPAIGN_SUM" ] } ``` Exact record and summary contracts live in `.actor/dataset_schema.json` and `.actor/output_summary_schema.json`. `qualityWarnings` identifies optional-date parse failures, unknown targeting types, and source totals that differ from campaign sums. Source-disclosed billing data is account-scoped and may repeat across ads. Treat payer and location fields as sensitive personal data and apply suitable access and retention controls. Empty filter results succeed with an empty dataset and `recordsEmitted: 0`. Live source HTTP errors fail explicitly; live mode never falls back to fixtures. Local access to X's archive CDN may return 404 even when cloud access works. ### Pricing and platform costs Planned Store pricing: Pay-Per-Event (PPE) only. Each `ad-record` event costs USD 0.007 per emitted record, equal to USD 7.00 per 1,000 records. No result means no `ad-record` charge. This local preparation does not change Cloud pricing. Apify platform usage costs are passed to users and are separate from PPE charges. Downloads, archive parsing, dataset storage, and optional proxy use can create platform usage. `maxItems` caps emitted records, but selected archives are fully validated before emission. ### Scope and limitations - Supported data is limited to the fixed 2018–2019 political and US issue archives and their source-provided fields. - Issue data is US-only. - A derived `creative.tweetUrl` does not prove that a historical post remains available. - Source availability is controlled by X and can change. - No current-ad discovery, commercial archive, post hydration, media recovery, engagement enrichment, browser automation, login, or authenticated DSA API access is included. ### Local development Requires Node.js 22+ and pnpm 10.30.0. ```bash pnpm install --frozen-lockfile pnpm test pnpm typecheck pnpm build pnpm lint apify validate-schema apify run ``` For local fixture smoke, save this as `storage/key_value_stores/default/INPUT.json` before `apify run`: ```json { "mode": "fixture", "sources": ["political", "issue"], "maxItems": 10 } ``` Archive size, parsing, and dataset writes drive runtime cost. `maxItems` limits output, but selected archives are fully validated before emission. ### Independence and historical terminology This project is independent and is not endorsed by X Corp. “Twitter” may appear only in historical source paths or context associated with the archive period. ### License MIT; see `LICENSE`. # Actor input Schema ## `mode` (type: `string`): Live downloads fixed official archive URLs. Fixture uses bundled synthetic sample data and makes no network requests. Live never falls back to fixture. ## `sources` (type: `array`): Political archive or US-only issue archive, processed in selected order. ## `advertisers` (type: `array`): Case-insensitive substring terms matched with OR across advertiser handle and disclosed name. Leading @ is ignored. ## `query` (type: `string`): Case-insensitive substring across creative text, advertiser handle/name, agency, and reported URLs. ## `startDate` (type: `string`): Inclusive creative creation date in YYYY-MM-DD. ## `endDate` (type: `string`): Inclusive creative creation date in YYYY-MM-DD. ## `minSpendUsd` (type: `number`): Inclusive source-provided headline spend filter. ## `minImpressions` (type: `integer`): Inclusive source-provided headline impressions filter. ## `maxItems` (type: `integer`): Global output cap. Complete archive validation happens before emission. ## `proxyConfiguration` (type: `object`): Used only for live archive requests. Proxy use may create platform cost. ## Actor input object example ```json { "mode": "fixture", "sources": [ "political", "issue" ], "advertisers": [], "query": "", "maxItems": 1000, "proxyConfiguration": { "useApifyProxy": false } } ``` # Actor output Schema ## `dataset` (type: `string`): No description ## `summary` (type: `string`): No description # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "mode": "fixture", "sources": [ "political", "issue" ], "proxyConfiguration": { "useApifyProxy": false } }; // Run the Actor and wait for it to finish const run = await client.actor("truenorth/x-ads-transparency-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "mode": "fixture", "sources": [ "political", "issue", ], "proxyConfiguration": { "useApifyProxy": False }, } # Run the Actor and wait for it to finish run = client.actor("truenorth/x-ads-transparency-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "mode": "fixture", "sources": [ "political", "issue" ], "proxyConfiguration": { "useApifyProxy": false } }' | apify call truenorth/x-ads-transparency-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=truenorth/x-ads-transparency-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification Download the OpenAPI definition: https://api.apify.com/v2/actors/K65iaHrvuUOubchaF/builds/AL3y4PZflpfXQ0jcT/openapi.json