Bright Data Web Scraper API - structured web and company data

Bright Data Web Scraper API turns any Bright Data Web Scraper dataset into a clean, structured export. Trigger a dataset, wait for it to finish, and download the results - company data, job listings, profiles, products and more - as JSON, CSV or Excel. The trigger, poll and download steps are fully automated for you.

Bring your own Bright Data API token and pay only for the records you keep.

Why use this actor?

• Automated async flow - the actor triggers the scraper, polls the snapshot until it is ready, and downloads the records for you. No manual snapshot juggling.
• Any dataset via dataset_id - point it at any Bright Data Web Scraper dataset id (looks like gd_xxxxxxxx) and it runs, whatever the target site.
• Pass-through structured records - each record is returned as-is, so you always get the full, dataset-specific fields, plus a few common columns (url, name, id) surfaced for quick scanning.
• Discovery or direct inputs - scrape exact input URLs, or switch on discovery (keyword, category URL, location, best sellers URL) to find new records.
• Pay per result - you are charged only for the records actually returned, up to 5,000 per run.

Use cases

• Company data - pull structured company profiles from LinkedIn, Crunchbase and other supported sources.
• Job listings - collect job postings for market and hiring analysis.
• Profiles - gather people or company profiles for enrichment and research.
• Product data - scrape ecommerce product and pricing data for competitive monitoring.
• Market research - build datasets across sites for analysis, dashboards and machine learning.

Input

Configure everything from the visual editor - every option is a proper field, no code needed. Example JSON input:

{
  "datasetId": "gd_l1viktl72bvl7bjuj0",
  "inputs": [
    { "url": "https://www.linkedin.com/company/ibm" },
    { "url": "https://www.linkedin.com/company/microsoft" }
  ],
  "maxResults": 100,
  "apiKey": "YOUR_BRIGHT_DATA_TOKEN"
}

To run discovery instead of scraping exact inputs, set discoverBy:

{
  "datasetId": "gd_l1viktl72bvl7bjuj0",
  "discoverBy": "keyword",
  "inputs": [{ "keyword": "software companies" }],
  "limitPerInput": 50,
  "maxResults": 200,
  "apiKey": "YOUR_BRIGHT_DATA_TOKEN"
}

Output

Records vary per dataset, so each one is passed through exactly as Bright Data returns it. A few common top-level fields (url, name, id) are surfaced when present. An example generic record:

{
  "url": "https://www.linkedin.com/company/ibm",
  "name": "IBM",
  "id": "1009",
  "industry": "IT Services and IT Consulting",
  "company_size": "10001+ employees",
  "headquarters": "Armonk, New York",
  "founded": 1911,
  "followers": 15000000
}

The exact fields depend entirely on the dataset id you run.

Do I need an API key?

Yes. This actor calls the Bright Data Web Scraper API on your behalf, which requires your own Bright Data API token. Sign up self-serve at brightdata.com, copy the token from your Bright Data account settings, paste it into the input, and run. Your token is stored securely and never logged.

Pay per result

You pay only for the records the actor actually returns - no subscriptions, no minimums. Set Maximum results to cap each run's spend exactly. A run that returns nothing costs almost nothing.

FAQ

What is the Bright Data Web Scraper API?

It is Bright Data's async dataset API for their Web Scraper product. You trigger a dataset with a set of inputs, the scraper collects the data, and you download a structured snapshot of records. This actor automates that trigger, poll and download flow and exports the records as JSON, CSV or Excel.

What datasets can I run?

Any Bright Data Web Scraper dataset you have access to. Each dataset has an id that looks like gd_xxxxxxxx, found in your Bright Data dashboard. Datasets cover company data, job listings, social and company profiles, ecommerce products and more. The record fields differ from one dataset to the next.

How does the async flow work?

Three steps, handled for you: the actor calls the trigger endpoint with your dataset id and inputs and receives a snapshot id, polls the progress endpoint on your chosen interval until the snapshot status is ready, then downloads the full snapshot and pushes each record to the dataset.

Do I need an API key?

Yes. You bring your own Bright Data API token. It is self-serve - sign up at brightdata.com, copy the token from account settings, and paste it into the input. The actor authenticates every request with your token via a Bearer header.

How many records can I get?

Up to 5,000 records per run. Use Maximum results to cap the count and your spend. For larger jobs, split the inputs across multiple runs or use Limit per input to bound each input.

How much does it cost?

Pricing is pay-per-result: a small fixed amount per record returned, plus standard Apify platform usage, plus your own Bright Data usage against your token. You pay only for records that land in the dataset.

Can I run this on a schedule?

Yes. Use Apify Schedules to run it daily or weekly and connect the dataset to Zapier, Make, n8n, webhooks or the API to push fresh records into your CRM, warehouse or analytics stack.

Is this legal?

The actor runs Bright Data Web Scraper datasets through Bright Data's own API using your credentials, subject to Bright Data's terms and compliance controls. Use it for data you are permitted to collect, and prefer business and public data over personal data.

Keyword map

bright data api, web scraper api, brightdata dataset api, web data api, structured web data, company data api, job listings scraper, profile scraper api, product data api, ecommerce scraper api, async scraping api, dataset trigger poll download, pay per result scraper.

Support and feedback

Found an issue or missing an option? Open an issue in the Issues tab - it is monitored daily.