# USA Data.gov U.S. Government's Open Data Scrape (`parseforge/data-gov-scraper`) Actor Stop wasting hours digging through thousands of government datasets. Our Data.gov scraper automatically gathers complete dataset details from the U.S. government's open data portal in minutes. Ideal for researchers, analysts, journalists, and teams needing reliable data without manual effort. - **URL**: https://apify.com/parseforge/data-gov-scraper.md - **Developed by:** [ParseForge](https://apify.com/parseforge) (community) - **Categories:** Automation, Developer tools, Other - **Stats:** 18 total users, 2 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing Pay per event This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ![ParseForge Banner](https://github.com/ParseForge/apify-assets/blob/ad35ccc13ddd068b9d6cba33f323962e39aed5b2/banner.jpg?raw=true) ## πŸ‡ΊπŸ‡Έ Data.gov Scraper > πŸš€ **Collect open dataset metadata from Data.gov in minutes.** Search by keyword, topic, or category. Export dataset titles, descriptions, download links, and agency info. No coding, no account required. > πŸ•’ **Last updated:** 2026-04-23 Β· **πŸ“Š 20+ fields** per dataset Β· **πŸ” 3 search filters** Β· **πŸ“‚ Topic + category** Β· **🚫 No auth** required
Pull structured records from USA Data.gov U.S. Open Data β€” clean fields ready as CSV, JSON, JSONL, Excel, or XML for downstream pipelines.
Related Scrapers
 USA data.gov
➸ You are here 		 UK data.gov.uk
UK open gov data 		 Korea data.go.kr
Korean open gov data
##### Copy to your AI assistant Copy this block into ChatGPT, Claude, Cursor, or any LLM to start using this actor. ```` parseforge/data-gov-scraper on Apify. Call: ApifyClient("TOKEN").actor("parseforge/data-gov-scraper").call(run\_input={...}), then client.dataset(run\["defaultDatasetId"]).list\_items().items for results. Key inputs: startUrl (string), searchQuery (string, default "climate"), topics (array), topicCategories (array), datasetType (string), tags (array). Full actor spec: fetch build via GET https://api.apify.com/v2/acts/parseforge~data-gov-scraper (Bearer TOKEN). Get token: https://console.apify.com/account/integrations ```` The **Data.gov Scraper** collects open data catalog metadata from the U.S. government's data portal, returning **20+ fields per dataset**: title, description, agency, topic, format, download URL, update frequency, and license. Data.gov hosts over 300,000 datasets from 100+ federal agencies. The Actor supports keyword search with topic and category filters, plus direct URL scraping. | 🎯 Target Audience | πŸ’‘ Primary Use Cases | |---|---| | Data scientists, policy researchers, journalists, civic tech teams, academic researchers, government analysts | Open data discovery, policy research, data journalism, civic technology, federal data monitoring | --- ### πŸ“‹ What the Data.gov Scraper does Three search filters: - πŸ” **Keyword search.** Free-text search across dataset titles and descriptions. - πŸ“‚ **Topic filter.** Subject areas (health, education, climate, finance, etc.). - πŸ“‚ **Category filter.** Detailed categorization within topics. - πŸ”— **URL mode.** Paste a direct Data.gov search URL. Each dataset record includes title, description, agency, topic, categories, data format, download URLs, update frequency, license, and portal URL. > πŸ’‘ **Why it matters:** browsing Data.gov for relevant datasets means scrolling through 300,000+ listings. This Actor exports structured catalog metadata at scale for your open data pipelines or research projects. --- ### 🎬 Full Demo _🚧 Coming soon: a 3-minute walkthrough showing how to go from sign-up to a downloaded dataset._ --- ### βš™οΈ Input
InputTypeDefaultBehavior
startUrlstring""Direct Data.gov search URL.
searchQuerystring""Keyword search.
topicsarray[]Subject area filters.
topicCategoriesarray[]Detailed categories within topics.
**Example: climate datasets.** ```json { "searchQuery": "climate change", "topics": ["Climate"], "maxItems": 50 } ```` **Example: health data from CDC.** ```json { "searchQuery": "CDC", "topics": ["Health"], "maxItems": 100 } ``` > ⚠️ **Good to Know:** Data.gov is the U.S. government's official open data portal. Datasets link to various formats (CSV, JSON, XML, API) hosted by individual agencies. *** ### πŸ“Š Output Each dataset record contains **20+ fields**. Download the catalog as CSV, Excel, JSON, or XML. #### 🧾 Schema | Field | Type | Example | |---|---|---| | πŸ“ title | string | `"Daily Climate Normals"` | | πŸ“„ description | string | `"NOAA climate normals for U.S. stations..."` | | πŸ›οΈ agency | string | `"NOAA"` | | πŸ“‚ topic | string | `"Climate"` | | πŸ“‚ categories | array | `["Weather", "Environment"]` | | πŸ“¦ format | string | `"CSV"` | | πŸ”— downloadUrl | string | `"https://catalog.data.gov/dataset/..."` | | πŸ“… lastUpdated | string | `"2026-01-15"` | | πŸ”„ updateFrequency | string | `"Annual"` | | πŸ“œ license | string | `"Public Domain"` | | πŸ”— portalUrl | string | `"https://catalog.data.gov/dataset/..."` | | πŸ•’ scrapedAt | ISO 8601 | `"2026-04-16T00:00:00.000Z"` | #### πŸ“¦ Sample records
🌑️ Climate dataset ```json { "title": "Daily Climate Normals", "description": "NOAA climate normals for U.S. weather stations, updated annually.", "agency": "NOAA", "topic": "Climate", "categories": ["Weather", "Environment"], "format": "CSV", "downloadUrl": "https://catalog.data.gov/dataset/daily-climate-normals", "lastUpdated": "2026-01-15", "updateFrequency": "Annual", "license": "Public Domain", "portalUrl": "https://catalog.data.gov/dataset/daily-climate-normals", "scrapedAt": "2026-04-16T00:00:00.000Z" } ```
πŸ₯ Health dataset ```json { "title": "COVID-19 Vaccination Data", "description": "CDC vaccination administration data by jurisdiction.", "agency": "CDC", "topic": "Health", "categories": ["Public Health", "Vaccination"], "format": "JSON", "downloadUrl": "https://data.cdc.gov/...", "lastUpdated": "2026-04-14", "updateFrequency": "Daily", "license": "Public Domain", "portalUrl": "https://catalog.data.gov/dataset/covid-vaccination", "scrapedAt": "2026-04-16T00:00:00.000Z" } ```
πŸ“‹ Dataset with sparse metadata ```json { "title": "Federal Employee Survey Results", "description": "Annual survey of federal employee satisfaction.", "agency": "OPM", "topic": "Government Operations", "categories": [], "format": "XLS", "downloadUrl": null, "lastUpdated": null, "updateFrequency": "Annual", "license": "Public Domain", "portalUrl": "https://catalog.data.gov/dataset/federal-employee-survey", "scrapedAt": "2026-04-16T00:00:00.000Z" } ```
*** ### ✨ Why choose this Actor | | Capability | |---|---| | πŸ‡ΊπŸ‡Έ | **300,000+ datasets.** Full U.S. federal open data catalog. | | πŸ” | **3 search filters.** Keyword, topic, category. | | πŸ“¦ | **Format and download links.** Direct URLs to CSV, JSON, XML, API. | | πŸ›οΈ | **Agency data.** Federal agency per dataset. | | πŸ“… | **Update frequency.** Daily, monthly, annual cadence. | | ⚑ | **Scalable.** Quick lookups to full catalog sweeps. | | 🚫 | **No authentication.** Public open data portal. | > πŸ“Š Data.gov hosts 300,000+ datasets from 100+ federal agencies. Structured access powers every open data project, policy research, and civic technology workflow. *** ### πŸ“ˆ How it compares to alternatives | Approach | Cost | Coverage | Refresh | Filters | Setup | |---|---|---|---|---|---| | **⭐ Data.gov Scraper** *(this Actor)* | $5 free credit, then pay-per-use | Full catalog | **Live per run** | keyword, topic, category | ⚑ 2 min | | Manual Data.gov browsing | Free | One at a time | Manual | UI only | πŸ•’ Hours | | CKAN API (direct) | Free | Full | Real-time | Many | ⏳ Hours | Pick this Actor when you want U.S. federal open data catalog metadata on demand, with topic and category filters. *** ### πŸš€ How to use 1. πŸ“ **Sign up.** [Create a free account with $5 credit](https://console.apify.com/sign-up?fpr=vmoqkp) (takes 2 minutes). 2. 🌐 **Open the Actor.** Go to the Data.gov Scraper page on the Apify Store. 3. 🎯 **Set input.** Enter a keyword, pick topics and categories. 4. πŸš€ **Run it.** Click **Start**. 5. πŸ“₯ **Download.** Grab results in the **Dataset** tab. > ⏱️ Total time: **3-5 minutes.** No coding required. *** ### πŸ’Ό Business use cases
#### πŸ“Š Data Science & Research - Discover federal datasets for research - Build open data catalogs - Track new publications by agency - Analyze data coverage by topic #### πŸ›οΈ Policy & Civic Tech - Monitor federal data releases - Build transparency dashboards - Track agency publishing rates - Power civic apps with open data
#### πŸ“° Data Journalism - Find datasets for investigative reporting - Track data freshness across agencies - Build story pipelines from government data - Monitor new dataset publications #### 🏒 Business Intelligence - Enrich models with federal data - Track economic indicators by agency - Build market data pipelines - Monitor regulatory data releases
*** *** ### 🌟 Beyond business use cases Data like this powers more than commercial workflows. The same structured records support research, education, civic projects, and personal initiatives.
#### πŸŽ“ Research and academia - Empirical datasets for papers, thesis work, and coursework - Longitudinal studies tracking changes across snapshots - Reproducible research with cited, versioned data pulls - Classroom exercises on data analysis and ethical scraping #### 🎨 Personal and creative - Side projects, portfolio demos, and indie app launches - Data visualizations, dashboards, and infographics - Content research for bloggers, YouTubers, and podcasters - Hobbyist collections and personal trackers
#### 🀝 Non-profit and civic - Transparency reporting and accountability projects - Advocacy campaigns backed by public-interest data - Community-run databases for local issues - Investigative journalism on public records #### πŸ§ͺ Experimentation - Prototype AI and machine-learning pipelines with real data - Validate product-market hypotheses before engineering spend - Train small domain-specific models on niche corpora - Test dashboard concepts with live input
### πŸ€– Ask an AI assistant about this scraper Open a ready-to-send prompt about this ParseForge actor in the AI of your choice: - πŸ’¬ [**ChatGPT**](https://chat.openai.com/?q=How%20do%20I%20use%20the%20USA%20Data.gov%20U.S.%20Government%27s%20Open%20Data%20Scrape%20by%20ParseForge%20on%20Apify%3F%20Show%20me%20input%20examples%2C%20output%20fields%2C%20common%20use%20cases%2C%20and%20how%20to%20integrate%20it%20into%20a%20workflow.) - 🧠 [**Claude**](https://claude.ai/new?q=How%20do%20I%20use%20the%20USA%20Data.gov%20U.S.%20Government%27s%20Open%20Data%20Scrape%20by%20ParseForge%20on%20Apify%3F%20Show%20me%20input%20examples%2C%20output%20fields%2C%20common%20use%20cases%2C%20and%20how%20to%20integrate%20it%20into%20a%20workflow.) - πŸ” [**Perplexity**](https://perplexity.ai/search?q=How%20do%20I%20use%20the%20USA%20Data.gov%20U.S.%20Government%27s%20Open%20Data%20Scrape%20by%20ParseForge%20on%20Apify%3F%20Show%20me%20input%20examples%2C%20output%20fields%2C%20common%20use%20cases%2C%20and%20how%20to%20integrate%20it%20into%20a%20workflow.) - πŸ…’ [**Copilot**](https://copilot.microsoft.com/?q=How%20do%20I%20use%20the%20USA%20Data.gov%20U.S.%20Government%27s%20Open%20Data%20Scrape%20by%20ParseForge%20on%20Apify%3F%20Show%20me%20input%20examples%2C%20output%20fields%2C%20common%20use%20cases%2C%20and%20how%20to%20integrate%20it%20into%20a%20workflow.) ### ❓ Frequently Asked Questions
πŸ’³ Do I need a paid Apify plan to run this actor? No. You can start right now on the free Apify plan, which includes **$5 in free monthly credit**. That is enough to run this actor several times and explore the output before committing to anything. Paid plans unlock higher limits, more concurrent runs, and larger datasets. [Create a free Apify account here](https://console.apify.com/sign-up?fpr=vmoqkp) to get started.
🚨 What happens if my run fails or returns no results? Failed runs are not charged. If the source site changes, proxies get rate-limited, or a specific input matches nothing, re-run the actor or open our [contact form](https://tally.so/r/BzdKgA) and we will investigate. You can also check the run log in the Apify console to see why the run stopped.
πŸ“ How many items can I scrape per run? Free users are limited to **10 items per run** so you can preview the output and confirm the actor works for your use case. Paid users can raise maxItems up to **1,000,000** per run. [Upgrade here](https://console.apify.com/sign-up?fpr=vmoqkp) if you need full scale.
πŸ•’ How fresh is the data? Every run fetches live data at the moment of execution. There is no cache or delay: the records you get reflect what the source returned at that moment. Schedule the actor to maintain a rolling snapshot of the data you need.
πŸ§‘β€πŸ’» Can I call this actor from my own code? Yes. Apify exposes every actor as a REST endpoint and ships first-class SDKs for [Node.js](https://docs.apify.com/sdk/js) and [Python](https://docs.apify.com/sdk/python). You can start a run, read the dataset, and handle webhooks from your own app in a few lines. All you need is your Apify API token.
πŸ“€ How do I export the data? Every Apify dataset can be downloaded in one click from the console as CSV, JSON, JSONL, Excel, HTML, XML, or RSS. You can also pull results programmatically via the [Apify API](https://docs.apify.com/api/v2) or stream them into BigQuery, S3, and other destinations through built-in integrations.
πŸ“… Can I schedule the actor to run automatically? Yes. Use the Apify scheduler to run the actor on any cadence, from hourly to monthly. Results are saved to your dataset and can be delivered to webhooks, email, Slack, cloud storage, or automation tools such as Zapier and Make. ***
### πŸ”Œ Automating Data.gov Scraper - 🟒 **Node.js.** Install the apify-client NPM package. - 🐍 **Python.** Use the apify-client PyPI package. - πŸ“š See the [Apify API documentation](https://docs.apify.com/api/v2) for full details. The [Apify Schedules feature](https://docs.apify.com/platform/schedules) lets you trigger this Actor on any cron interval. Weekly pulls catch new dataset publications. ### πŸ”Œ Integrate with any app - [**Make**](https://docs.apify.com/platform/integrations/make) - Automate workflows - [**Zapier**](https://docs.apify.com/platform/integrations/zapier) - Connect 5,000+ apps - [**Slack**](https://docs.apify.com/platform/integrations/slack) - Get notifications - [**Airbyte**](https://docs.apify.com/platform/integrations/airbyte) - Data pipelines - [**GitHub**](https://docs.apify.com/platform/integrations/github) - Trigger from commits - [**Google Drive**](https://docs.apify.com/platform/integrations/drive) - Export to Sheets *** ### πŸ”— Recommended Actors - [**πŸ‡¬πŸ‡§ Data.gov.uk Scraper**](https://apify.com/parseforge/data-gov-uk-scraper) - UK open data catalog - [**πŸ“Š USAspending Scraper**](https://apify.com/parseforge/usaspending-scraper) - Federal spending data - [**πŸ“Š FRED Scraper**](https://apify.com/parseforge/fred-scraper) - Economic data - [**πŸ“‹ GSA eLibrary Scraper**](https://apify.com/parseforge/gsa-elibrary-scraper) - Government contracts - [**πŸ“Š Indexmundi Scraper**](https://apify.com/parseforge/indexmundi-scraper) - Global indicators > πŸ’‘ **Pro Tip:** browse the complete [ParseForge collection](https://apify.com/parseforge) for more government and open data scrapers. *** **πŸ†˜ Need Help?** [**Open our contact form**](https://tally.so/r/BzdKgA) to request a new scraper, propose a custom data project, or report an issue. *** > **⚠️ Disclaimer:** this Actor is an independent tool and is not affiliated with, endorsed by, or sponsored by the U.S. government or Data.gov. All trademarks mentioned are the property of their respective owners. Only publicly available open data catalog metadata is collected. # Actor input Schema ## `startUrl` (type: `string`): Direct URL to scrape from Data.gov catalog. Use this OR search filters below, not both. Example: https://catalog.data.gov/dataset?q=climate ## `searchQuery` (type: `string`): Search term for datasets. Use this OR startUrl above, not both. Example: climate, healthcare ## `topics` (type: `array`): Filter by topic groups. Select one or more topics from the dropdown. Leave empty for all topics. ## `topicCategories` (type: `array`): Filter by topic categories. Select one or more categories from the dropdown. Leave empty for all categories. ## `datasetType` (type: `string`): Filter by dataset type. Currently only 'geospatial' is available. Leave empty for all types. ## `tags` (type: `array`): Filter by tags. Enter one or more tag names (e.g., 'earth science', 'noaa', 'oceans'). Tags vary by dataset type, so use the text field for flexibility. ## `formats` (type: `array`): Filter by resource formats. Select one or more formats from the dropdown. Leave empty for all formats. ## `organizationType` (type: `string`): Filter by organization type. Select a type from the dropdown or leave empty for all types. ## `organization` (type: `string`): Filter by organization. Select an organization from the dropdown or leave empty to search all organizations. ## `publisher` (type: `string`): Filter by publisher. Select a publisher from the dropdown or leave empty to search all publishers. ## `bureau` (type: `string`): Filter by bureau. Select a bureau from the dropdown or leave empty to search all bureaus. ## `location` (type: `string`): Filter by location name. Example: New York, California. When using startUrl, location parameters (ext\_location, ext\_bbox, ext\_prev\_extent) are preserved automatically. ## `sort` (type: `string`): Sort order. Default: Popular (views\_recent desc). Options: views\_recent desc (Popular), score desc, metadata\_modified desc (Relevance), title\_string asc (Name A-Z), title\_string desc (Name Z-A), metadata\_modified desc (Last Modified), metadata\_created desc (Date Added) ## `maxItems` (type: `integer`): Free users: Limited to 100. Paid users: Optional, max 1,000,000 ## Actor input object example ```json { "searchQuery": "climate", "sort": "views_recent desc", "maxItems": 10 } ``` # Actor output Schema ## `datasets` (type: `string`): Complete dataset with all fields including organization information, publisher details, topics, formats, download links, and metadata ## `overview` (type: `string`): Overview view of datasets with key fields displayed in a table format # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "searchQuery": "climate", "maxItems": 10 }; // Run the Actor and wait for it to finish const run = await client.actor("parseforge/data-gov-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`πŸ’Ύ Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // πŸ“š Want to learn more πŸ“–? Go to β†’ https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "searchQuery": "climate", "maxItems": 10, } # Run the Actor and wait for it to finish run = client.actor("parseforge/data-gov-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("πŸ’Ύ Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # πŸ“š Want to learn more πŸ“–? Go to β†’ https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "searchQuery": "climate", "maxItems": 10 }' | apify call parseforge/data-gov-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=parseforge/data-gov-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "USA Data.gov U.S. Government's Open Data Scrape", "description": "Stop wasting hours digging through thousands of government datasets. Our Data.gov scraper automatically gathers complete dataset details from the U.S. government's open data portal in minutes. Ideal for researchers, analysts, journalists, and teams needing reliable data without manual effort.", "version": "1.0", "x-build-id": "5Ul4Ns9ObukGCwIdP" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/parseforge~data-gov-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-parseforge-data-gov-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/parseforge~data-gov-scraper/runs": { "post": { "operationId": "runs-sync-parseforge-data-gov-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/parseforge~data-gov-scraper/run-sync": { "post": { "operationId": "run-sync-parseforge-data-gov-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "properties": { "startUrl": { "title": "Start URL", "type": "string", "description": "Direct URL to scrape from Data.gov catalog. Use this OR search filters below, not both. Example: https://catalog.data.gov/dataset?q=climate" }, "searchQuery": { "title": "Search Query", "type": "string", "description": "Search term for datasets. Use this OR startUrl above, not both. Example: climate, healthcare" }, "topics": { "title": "Topics", "type": "array", "description": "Filter by topic groups. Select one or more topics from the dropdown. Leave empty for all topics.", "items": { "type": "string", "enum": [ "local", "climate5434", "older-adults-health-data", "energy9485" ], "enumTitles": [ "Local Government", "Climate", "Older Adults", "Energy" ] } }, "topicCategories": { "title": "Topic Categories", "type": "array", "description": "Filter by topic categories. Select one or more categories from the dropdown. Leave empty for all categories.", "items": { "type": "string", "enum": [ "Arctic", "Water", "Ecosystem Vulnerability", "Arctic Ocean, Sea Ice and Coasts", "Human Health", "Transportation", "Energy Infrastructure", "Food Resilience", "Atmospheric, Earth and Ocean Sciences", "Permafrost and Arctic Landscapes", "Arctic Weather and Extreme Events", "Melting Glaciers, Snow and Ice", "Coastal Flooding", "Arctic Peoples and Ecosystems", "Physical Data", "Food Production", "Food Safety and Nutrition", "Arctic Development", "Transportation Nodes", "Biodiversity", "Carbon Balance", "Energy Resources", "Food Security", "Arctic-Global", "Energy Supply", "Food Distribution", "Transportation Network", "Land Cover and Land Use", "Energy Conversion", "Ecosystem Services", "Hydrology", "Transportation Flow", "Water Resources", "Changing Oceans", "Climatology and Meteorology", "Asian", "Ecosystem Change Drivers", "Education", "Environment", "Hawaii", "Pacific Islands", "Winter Storm", "Arctic Oceans, Sea Ice and Coasts", "Atmospheric Chemistry", "Biology and Habitats", "Business", "Ecosystem" ], "enumTitles": [ "Arctic", "Water", "Ecosystem Vulnerability", "Arctic Ocean, Sea Ice and Coasts", "Human Health", "Transportation", "Energy Infrastructure", "Food Resilience", "Atmospheric, Earth and Ocean Sciences", "Permafrost and Arctic Landscapes", "Arctic Weather and Extreme Events", "Melting Glaciers, Snow and Ice", "Coastal Flooding", "Arctic Peoples and Ecosystems", "Physical Data", "Food Production", "Food Safety and Nutrition", "Arctic Development", "Transportation Nodes", "Biodiversity", "Carbon Balance", "Energy Resources", "Food Security", "Arctic-Global", "Energy Supply", "Food Distribution", "Transportation Network", "Land Cover and Land Use", "Energy Conversion", "Ecosystem Services", "Hydrology", "Transportation Flow", "Water Resources", "Changing Oceans", "Climatology and Meteorology", "Asian", "Ecosystem Change Drivers", "Education", "Environment", "Hawaii", "Pacific Islands", "Winter Storm", "Arctic Oceans, Sea Ice and Coasts", "Atmospheric Chemistry", "Biology and Habitats", "Business", "Ecosystem" ] } }, "datasetType": { "title": "Dataset Type", "enum": [ "", "geospatial" ], "type": "string", "description": "Filter by dataset type. Currently only 'geospatial' is available. Leave empty for all types." }, "tags": { "title": "Tags", "type": "array", "description": "Filter by tags. Enter one or more tag names (e.g., 'earth science', 'noaa', 'oceans'). Tags vary by dataset type, so use the text field for flexibility.", "items": { "type": "string" } }, "formats": { "title": "Formats", "type": "array", "description": "Filter by resource formats. Select one or more formats from the dropdown. Leave empty for all formats.", "items": { "type": "string", "enum": [ "XML", "ZIP", "HTML", "PDF", "CSV", "Esri REST", "TEXT", "JSON", "TIFF", "RDF", "SID", "BIN", "WMS", "EXCEL", "KML", "ArcGIS GeoServices REST API", "XYZ", "GeoJSON", "JPEG", "EXE", "PNG", "application/vnd.lotus-1-2-3", "WFS", "gml", "QGIS", "API", "WCS", "OGC WMS", "application/vnd.geo+json", "DOC", "GDB", "text/vnd.ascii-art", "application/geopackage+sqlite3", "NETCDF", "sos", "GPKG", "CDF", "SHP", "GZ", "Undefined", "nc", "TAR", "chemical/x-mdl-sdfile", "00", "application/html", "xsl", "cdfnc", "POWERPOINT" ], "enumTitles": [ "XML", "ZIP", "HTML", "PDF", "CSV", "Esri REST", "TEXT", "JSON", "TIFF", "RDF", "SID", "BIN", "WMS", "EXCEL", "KML", "ArcGIS GeoServices REST API", "XYZ", "GeoJSON", "JPEG", "EXE", "PNG", "application/vnd.lotus-1-2-3", "WFS", "gml", "QGIS", "API", "WCS", "OGC WMS", "application/vnd.geo+json", "DOC", "GDB", "text/vnd.ascii-art", "application/geopackage+sqlite3", "NETCDF", "sos", "GPKG", "CDF", "SHP", "GZ", "Undefined", "nc", "TAR", "chemical/x-mdl-sdfile", "00", "application/html", "xsl", "cdfnc", "POWERPOINT" ] } }, "organizationType": { "title": "Organization Type", "enum": [ "", "Federal Government", "State Government", "City Government", "University", "County Government" ], "type": "string", "description": "Filter by organization type. Select a type from the dropdown or leave empty for all types." }, "organization": { "title": "Organization", "enum": [ "", "allegheny-county-city-of-pittsburgh-western-pa-regional-data-center", "americorps-gov", "arlington-county", "usace-army-mil", "board-of-governors-of-the-federal-reserve-system", "city-and-county-of-durham-north-carolina", "city-of-austin", "city-of-baltimore", "city-of-baton-rouge", "city-of-bloomington", "city-of-boise", "city-of-charlotte", "city-of-chesapeake", "city-of-chicago", "city-of-ferndale-michigan", "city-of-hartford", "city-of-honolulu", "city-of-jackson-mississippi", "city-of-los-angeles", "city-of-new-orleans", "city-of-new-york", "city-of-philadelphia", "city-of-providence", "city-of-san-francisco", "city-of-santa-rosa", "city-of-seattle", "city-of-sioux-falls", "city-of-somerville", "city-of-tempe", "cdatribe-nsn-gov", "commodity-futures-trading-commission", "cfpb-gov", "cook-county-of-illinois", "usda-gov", "doc-gov", "dod-gov", "ed-gov", "doe-gov", "dhs-gov", "hud-gov", "doj-gov", "dol-gov", "state-gov", "doi-gov", "treasury-gov", "dot-gov", "va-gov", "district-of-columbia", "edac-unm-edu", "eac-gov", "eop-gov", "exim-gov", "fairfax-county-virginia", "fcc-gov", "fdic-gov", "fec-gov", "fgdc-gov", "fhfa-gov", "federal-laboratory-consortium", "federal-maritime-commission", "federal-retirement-thrift-investment-board", "federal-trade-commission", "florida-department-of-environmental-protection", "gsa-gov", "government-publishing-office", "gulf-coast-ecosystem-restoration-council", "imls-gov", "ibwc-gov", "king-county-washington", "lake-county-illinois", "legal-services-corporation", "library-of-congress", "loudoun-county-virginia", "louisville-metro-government", "mspb-gov", "mcc-gov", "montgomery-county-of-maryland", "municipality-of-anchorage", "nasa-gov", "nara-gov", "national-credit-union-administration", "nea-gov", "neh-gov", "nga-mil", "national-institute-of-standards-and-technology", "noaa-gov", "nsf-gov", "national-telecommunications-and-information-administration", "ntsb-gov", "nitrd-gov", "nrc-gov", "office-of-management-and-budget", "opm-gov", "opentopography", "pennsylvania-geospatial-data-clearinghouse", "pbgc-gov", "rrb-gov", "sandbox-organization", "cdip-ucsd-edu", "sec-gov", "sba-gov", "ssa-gov", "state-of-alaska", "arkansas-gov", "ca-gov", "state-of-connecticut", "state-of-hawaii", "state-of-illinois", "state-of-iowa", "state-of-maryland", "state-mn-us", "state-of-missouri", "state-of-new-york", "nc-gov", "state-of-oklahoma", "state-of-oregon", "state-of-washington", "federal-acquisition-service", "town-of-cary-north-carolina", "town-of-chapel-hill-north-carolina", "census-gov", "hhs-gov", "u-s-department-of-labor-bureau-of-labor-statistics", "epa-gov", "usgs-gov", "uidaho-edu", "u-washington-edu", "cpsc-gov", "eeoc-gov", "usitc-gov", "vcgi-org", "wake-county", "wvu-edu" ], "type": "string", "description": "Filter by organization. Select an organization from the dropdown or leave empty to search all organizations." }, "publisher": { "title": "Publisher", "enum": [ "", "ACF", "Agricultural Research Service", "Allegheny County", "Board of Governors of the Federal Reserve System", "Bureau of Justice Statistics", "Bureau of Labor Statistics", "Bureau of Transportation Statistics", "California Department of Education", "California Department of Public Health", "California Department of Technology", "California State Water Resources Control Board", "Centers for Disease Control and Prevention", "Centers for Medicare & Medicaid Services", "City of Philadelphia", "City of Seattle ArcGIS Online", "City of Tempe", "Department of Veterans Affairs", "District of Columbia Sentencing Commission", "Economic and Risk Analysis", "Economic Research Service, Department of Agriculture", "Employment and Training Administration", "Federal Highway Administration", "Federal Highway Administration (FHWA)", "Federal Housing Finance Agency", "Federal Motor Carrier Safety Administration", "FEMA/RESILIENCE/FIMA", "Food Safety and Inspection Service", "General Services Administration", "Health Resources and Services Administration", "HIFLD", "ICE", "Institute of Museum and Library Services", "Justice Management Division", "Lake County Illinois GIS", "Metropolitan Police Department", "Mine Safety and Health Administration", "National Aeronautics and Space Administration", "National Agricultural Statistics Service, Department of Agriculture", "National Center for Education Statistics (NCES)", "National Highway Traffic Safety Administration", "National Institute of Standards and Technology", "National Institutes of Health (NIH)", "National Library of Medicine", "National Renewable Energy Laboratory", "National Science Foundation", "NCO NITRD", "Office for Civil Rights (OCR)", "Office of Analysis and Research Services", "Office of Debt Management", "Office of Federal Student Aid (FSA)", "Office of Tariff Affairs and Trade Agreements", "Office of the Chief Economist (OCE)", "Office of the Chief Technology Officer", "Office of the Secretary of Transportation", "Public Affairs", "Rural Development, Department of Agriculture", "Small Business Administration", "Social Security Administration", "State of New York", "Structured Disclosure", "Substance Abuse and Mental Health Services Administration", "U.S. Census Bureau", "U.S. Commodity Futures Trading Commission", "U.S. Department of Housing and Urban Development", "U.S. Energy Information Administration", "U.S. EPA Office of Environmental Information (OEI)", "U.S. EPA Office of Research and Development (ORD)", "U.S. Food and Drug Administration", "U.S. Geological Survey", "United States Mint, Financial Department (CFO)", "University of Maryland (UMD)", "Unspecified" ], "type": "string", "description": "Filter by publisher. Select a publisher from the dropdown or leave empty to search all publishers." }, "bureau": { "title": "Bureau", "enum": [ "", "009:70", "009:33", "005:45", "005:18", "007:57", "015:13", "029:25", "011:14", "015:20", "010:76", "012:20", "010:04", "010:06", "010:22", "006:07", "009:20", "009:38", "202:00", "007:97", "007:00", "018:00", "019:00", "009:00", "025:00", "011:00", "014:00", "015:00", "021:00", "029:00", "019:60", "029:40", "012:25", "006:05", "010:84", "015:05", "011:12", "006:06", "005:13", "006:08", "012:05", "019:20", "019:10", "020:00", "350:00", "005:49", "021:12", "011:10", "356:00", "015:11", "021:15", "021:17", "011:20", "021:27", "021:36", "015:04", "015:12", "009:10", "005:84", "005:96", "009:15", "018:50", "015:60", "015:45", "006:25", "011:05", "007:05", "026:00", "021:18", "006:55", "009:25", "019:05", "010:24", "422:00", "011:08", "006:60", "005:53", "007:17", "018:10", "018:15", "018:45", "018:12", "011:21", "018:40", "018:20", "021:04", "007:10", "200:00", "021:50", "007:15", "011:06", "005:55", "001:05", "009:30", "006:51", "010:18", "010:12", "015:25", "029:15" ], "type": "string", "description": "Filter by bureau. Select a bureau from the dropdown or leave empty to search all bureaus." }, "location": { "title": "Location", "type": "string", "description": "Filter by location name. Example: New York, California. When using startUrl, location parameters (ext_location, ext_bbox, ext_prev_extent) are preserved automatically." }, "sort": { "title": "Sort", "enum": [ "views_recent desc", "score desc, metadata_modified desc", "title_string asc", "title_string desc", "metadata_modified desc", "metadata_created desc" ], "type": "string", "description": "Sort order. Default: Popular (views_recent desc). Options: views_recent desc (Popular), score desc, metadata_modified desc (Relevance), title_string asc (Name A-Z), title_string desc (Name Z-A), metadata_modified desc (Last Modified), metadata_created desc (Date Added)", "default": "views_recent desc" }, "maxItems": { "title": "Max Items", "minimum": 1, "maximum": 1000000, "type": "integer", "description": "Free users: Limited to 100. Paid users: Optional, max 1,000,000" } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```