# Stack Overflow Scraper - Questions & Users (`fascinating_lentil/stack-overflow-scraper`) Actor

Scrape Stack Overflow questions and users via the official Stack Exchange API. Get titles, scores, answers, views, tags, bodies, and user profiles. Works across all Stack Exchange sites.

- **URL**: https://apify.com/fascinating\_lentil/stack-overflow-scraper.md
- **Developed by:** [Md Jakaria Mirza](https://apify.com/fascinating_lentil) (community)
- **Categories:** Developer tools, Other
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $2.00 / 1,000 question scrapeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md):

```bash
npm install apify-client
```

In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md):

```bash
pip install apify-client
```

In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md):

````bash
# MacOS / Linux
curl -fsSL https://apify.com/install-cli.sh | bash
# Windows
irm https://apify.com/install-cli.ps1 | iex
```bash

In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md).

If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).


# README

## Stack Overflow Scraper - Questions & Users

Scrape **Stack Overflow questions and users** via the official Stack Exchange API - no login required. Get titles, scores, answer/view counts, tags, question bodies, and user profiles (reputation, badges, location). Search by keyword, filter by tag, fetch by ID, and works across all Stack Exchange sites. Export to **JSON, CSV, Excel, or HTML**, or pull via the Apify API.

Perfect for **developer research, sentiment/trend analysis, tech support tooling, and recruiting**.

### Features

- ✅ **Official Stack Exchange API** - accurate, structured data
- ✅ **Questions and users** in one actor
- ✅ **Search, tag filter, or ID lookup**
- ✅ **Any Stack Exchange site** - Stack Overflow, Super User, Server Fault, Ask Ubuntu, etc.
- ✅ **Optional question bodies** and **user profiles** with separate dataset views
- ✅ **Optional API key** - 10,000 requests/day vs 300 without

### Input

| Parameter | Type | Description | Default |
|-----------|------|-------------|---------|
| `searchQueries` | `string[]` | Keyword searches | `["async await"]` |
| `tags` | `string[]` | Tag filters (e.g. `"javascript"`) | `[]` |
| `questionIds` | `string[]` | Specific question IDs | `[]` |
| `userIds` | `string[]` | Specific user IDs shown in the Users view | `[]` |
| `sort` | `string` | `votes`, `activity`, `creation`, `hot` | `votes` |
| `includeBody` | `boolean` | Include full question body | `false` |
| `maxResults` | `integer` | Max questions | `100` |
| `site` | `string` | Stack Exchange site | `stackoverflow` |
| `apiKey` | `string` (secret) | Optional Stack Apps key | — |
| `proxyConfiguration` | `object` | Optional proxy settings | Disabled |

#### Example input

```json
{
  "searchQueries": ["memory leak"],
  "tags": ["python"],
  "sort": "votes",
  "includeBody": true,
  "maxResults": 200
}
````

### Sample output

```json
{
  "questionId": 37576685,
  "title": "Using async/await with a forEach loop",
  "score": 3351,
  "answerCount": 35,
  "viewCount": 2476784,
  "isAnswered": true,
  "acceptedAnswerId": 37576787,
  "tags": ["javascript", "node.js", "promise", "async-await"],
  "ownerName": "Saad",
  "ownerReputation": 54989,
  "createdAt": "2016-06-01T18:55:58.000Z",
  "lastActivityAt": "2025-01-28T01:27:13.000Z",
  "link": "https://stackoverflow.com/questions/37576685/...",
  "site": "stackoverflow",
  "scrapedAt": "2026-06-11T10:00:00.000Z"
}
```

### Pricing

This Actor uses **pay-per-result** pricing:

| Event | Price |
|-------|-------|
| Per question scraped | **$0.002** ($2 / 1,000 questions) |
| Per user scraped | **$0.002** ($2 / 1,000 users) |

Each question or user is saved and charged atomically. Duplicate questions are skipped, one global question cap is enforced across inputs, and later pages or batches stop when the user's spending limit is reached. Proxy usage is disabled by default; an API key is preferred for higher quotas.

### How to Scrape Stack Overflow (Step by Step)

1. Click **Try for free** / **Run**.
2. Enter what you want: add `searchQueries` keywords, `tags`, `questionIds`, or `userIds`.
3. Choose a `sort` order and set `maxResults` (start small to test), and optionally set `site` for another Stack Exchange community.
4. Run the Actor (toggle `includeBody` if you need full question text).
5. Export the results as JSON, CSV, Excel, or HTML, or pull them via the Apify API.

### Use cases

- **Developer research** - track popular questions, tags, and trends
- **Sentiment & topic analysis** - feed titles/bodies into NLP pipelines
- **Support tooling** - surface top Q\&A for a technology
- **Recruiting** - find high-reputation users in a domain

### Tips

- Questions and users are stored atomically in the default dataset and presented through separate views.
- Add a free **Stack Apps API key** to lift the daily quota from 300 to 10,000 requests.
- Set `site` to scrape other Stack Exchange communities (e.g. `superuser`, `askubuntu`).

### Responsible Use

This Actor is intended for lawful collection of publicly available information only. Users are responsible for ensuring their use complies with the source website's terms, robots.txt, applicable privacy laws, including India's DPDP Act, and all local regulations.

Do not use this Actor to collect, store, sell, or misuse personal data without a lawful basis. The Actor author is not responsible for misuse by end users.

### License

Apache-2.0

# Actor input Schema

## `searchQueries` (type: `array`):

Keywords/phrases to search questions (e.g. "async await", "memory leak").

## `tags` (type: `array`):

Filter by tags (e.g. "javascript", "python"). Used with search, or alone to list tagged questions.

## `questionIds` (type: `array`):

Specific Stack Overflow question IDs to fetch.

## `userIds` (type: `array`):

Specific Stack Overflow user IDs to fetch (shown in the Users dataset view).

## `sort` (type: `string`):

Order questions by votes, activity, creation date, or hot.

## `includeBody` (type: `boolean`):

If enabled, the full question body text is included.

## `maxResults` (type: `integer`):

Maximum number of questions to scrape (across searches/tags).

## `site` (type: `string`):

Which Stack Exchange site (e.g. "stackoverflow", "superuser", "serverfault", "askubuntu").

## `apiKey` (type: `string`):

A Stack Apps API key raises the daily quota from 300 to 10,000 requests. Register at stackapps.com/apps/oauth/register.

## `proxyConfiguration` (type: `object`):

Proxy settings. Rotating proxies help multiply the per-IP daily quota.

## Actor input object example

```json
{
  "searchQueries": [
    "async await"
  ],
  "tags": [],
  "questionIds": [],
  "userIds": [],
  "sort": "votes",
  "includeBody": false,
  "maxResults": 100,
  "site": "stackoverflow",
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}
```

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "searchQueries": [
        "async await"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("fascinating_lentil/stack-overflow-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "searchQueries": ["async await"] }

# Run the Actor and wait for it to finish
run = client.actor("fascinating_lentil/stack-overflow-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "searchQueries": [
    "async await"
  ]
}' |
apify call fascinating_lentil/stack-overflow-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "command": "npx",
            "args": [
                "mcp-remote",
                "https://mcp.apify.com/?tools=fascinating_lentil/stack-overflow-scraper",
                "--header",
                "Authorization: Bearer <YOUR_API_TOKEN>"
            ]
        }
    }
}

```

## OpenAPI specification

```json
{
    "openapi": "3.0.1",
    "info": {
        "title": "Stack Overflow Scraper - Questions & Users",
        "description": "Scrape Stack Overflow questions and users via the official Stack Exchange API. Get titles, scores, answers, views, tags, bodies, and user profiles. Works across all Stack Exchange sites.",
        "version": "1.0",
        "x-build-id": "7wGvP6gTbg42pQ0XC"
    },
    "servers": [
        {
            "url": "https://api.apify.com/v2"
        }
    ],
    "paths": {
        "/acts/fascinating_lentil~stack-overflow-scraper/run-sync-get-dataset-items": {
            "post": {
                "operationId": "run-sync-get-dataset-items-fascinating_lentil-stack-overflow-scraper",
                "x-openai-isConsequential": false,
                "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.",
                "tags": [
                    "Run Actor"
                ],
                "requestBody": {
                    "required": true,
                    "content": {
                        "application/json": {
                            "schema": {
                                "$ref": "#/components/schemas/inputSchema"
                            }
                        }
                    }
                },
                "parameters": [
                    {
                        "name": "token",
                        "in": "query",
                        "required": true,
                        "schema": {
                            "type": "string"
                        },
                        "description": "Enter your Apify token here"
                    }
                ],
                "responses": {
                    "200": {
                        "description": "OK"
                    }
                }
            }
        },
        "/acts/fascinating_lentil~stack-overflow-scraper/runs": {
            "post": {
                "operationId": "runs-sync-fascinating_lentil-stack-overflow-scraper",
                "x-openai-isConsequential": false,
                "summary": "Executes an Actor and returns information about the initiated run in response.",
                "tags": [
                    "Run Actor"
                ],
                "requestBody": {
                    "required": true,
                    "content": {
                        "application/json": {
                            "schema": {
                                "$ref": "#/components/schemas/inputSchema"
                            }
                        }
                    }
                },
                "parameters": [
                    {
                        "name": "token",
                        "in": "query",
                        "required": true,
                        "schema": {
                            "type": "string"
                        },
                        "description": "Enter your Apify token here"
                    }
                ],
                "responses": {
                    "200": {
                        "description": "OK",
                        "content": {
                            "application/json": {
                                "schema": {
                                    "$ref": "#/components/schemas/runsResponseSchema"
                                }
                            }
                        }
                    }
                }
            }
        },
        "/acts/fascinating_lentil~stack-overflow-scraper/run-sync": {
            "post": {
                "operationId": "run-sync-fascinating_lentil-stack-overflow-scraper",
                "x-openai-isConsequential": false,
                "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.",
                "tags": [
                    "Run Actor"
                ],
                "requestBody": {
                    "required": true,
                    "content": {
                        "application/json": {
                            "schema": {
                                "$ref": "#/components/schemas/inputSchema"
                            }
                        }
                    }
                },
                "parameters": [
                    {
                        "name": "token",
                        "in": "query",
                        "required": true,
                        "schema": {
                            "type": "string"
                        },
                        "description": "Enter your Apify token here"
                    }
                ],
                "responses": {
                    "200": {
                        "description": "OK"
                    }
                }
            }
        }
    },
    "components": {
        "schemas": {
            "inputSchema": {
                "type": "object",
                "properties": {
                    "searchQueries": {
                        "title": "Search Queries",
                        "type": "array",
                        "description": "Keywords/phrases to search questions (e.g. \"async await\", \"memory leak\").",
                        "default": [
                            "async await"
                        ],
                        "items": {
                            "type": "string"
                        }
                    },
                    "tags": {
                        "title": "Tags",
                        "type": "array",
                        "description": "Filter by tags (e.g. \"javascript\", \"python\"). Used with search, or alone to list tagged questions.",
                        "default": [],
                        "items": {
                            "type": "string"
                        }
                    },
                    "questionIds": {
                        "title": "Question IDs",
                        "type": "array",
                        "description": "Specific Stack Overflow question IDs to fetch.",
                        "default": [],
                        "items": {
                            "type": "string"
                        }
                    },
                    "userIds": {
                        "title": "User IDs",
                        "type": "array",
                        "description": "Specific Stack Overflow user IDs to fetch (shown in the Users dataset view).",
                        "default": [],
                        "items": {
                            "type": "string"
                        }
                    },
                    "sort": {
                        "title": "Sort By",
                        "enum": [
                            "votes",
                            "activity",
                            "creation",
                            "hot"
                        ],
                        "type": "string",
                        "description": "Order questions by votes, activity, creation date, or hot.",
                        "default": "votes"
                    },
                    "includeBody": {
                        "title": "Include Question Body",
                        "type": "boolean",
                        "description": "If enabled, the full question body text is included.",
                        "default": false
                    },
                    "maxResults": {
                        "title": "Max Questions",
                        "minimum": 1,
                        "maximum": 10000,
                        "type": "integer",
                        "description": "Maximum number of questions to scrape (across searches/tags).",
                        "default": 100
                    },
                    "site": {
                        "title": "Stack Exchange Site",
                        "type": "string",
                        "description": "Which Stack Exchange site (e.g. \"stackoverflow\", \"superuser\", \"serverfault\", \"askubuntu\").",
                        "default": "stackoverflow"
                    },
                    "apiKey": {
                        "title": "Stack Apps API Key (optional)",
                        "type": "string",
                        "description": "A Stack Apps API key raises the daily quota from 300 to 10,000 requests. Register at stackapps.com/apps/oauth/register."
                    },
                    "proxyConfiguration": {
                        "title": "Proxy Configuration",
                        "type": "object",
                        "description": "Proxy settings. Rotating proxies help multiply the per-IP daily quota.",
                        "default": {
                            "useApifyProxy": false
                        }
                    }
                }
            },
            "runsResponseSchema": {
                "type": "object",
                "properties": {
                    "data": {
                        "type": "object",
                        "properties": {
                            "id": {
                                "type": "string"
                            },
                            "actId": {
                                "type": "string"
                            },
                            "userId": {
                                "type": "string"
                            },
                            "startedAt": {
                                "type": "string",
                                "format": "date-time",
                                "example": "2025-01-08T00:00:00.000Z"
                            },
                            "finishedAt": {
                                "type": "string",
                                "format": "date-time",
                                "example": "2025-01-08T00:00:00.000Z"
                            },
                            "status": {
                                "type": "string",
                                "example": "READY"
                            },
                            "meta": {
                                "type": "object",
                                "properties": {
                                    "origin": {
                                        "type": "string",
                                        "example": "API"
                                    },
                                    "userAgent": {
                                        "type": "string"
                                    }
                                }
                            },
                            "stats": {
                                "type": "object",
                                "properties": {
                                    "inputBodyLen": {
                                        "type": "integer",
                                        "example": 2000
                                    },
                                    "rebootCount": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "restartCount": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "resurrectCount": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "computeUnits": {
                                        "type": "integer",
                                        "example": 0
                                    }
                                }
                            },
                            "options": {
                                "type": "object",
                                "properties": {
                                    "build": {
                                        "type": "string",
                                        "example": "latest"
                                    },
                                    "timeoutSecs": {
                                        "type": "integer",
                                        "example": 300
                                    },
                                    "memoryMbytes": {
                                        "type": "integer",
                                        "example": 1024
                                    },
                                    "diskMbytes": {
                                        "type": "integer",
                                        "example": 2048
                                    }
                                }
                            },
                            "buildId": {
                                "type": "string"
                            },
                            "defaultKeyValueStoreId": {
                                "type": "string"
                            },
                            "defaultDatasetId": {
                                "type": "string"
                            },
                            "defaultRequestQueueId": {
                                "type": "string"
                            },
                            "buildNumber": {
                                "type": "string",
                                "example": "1.0.0"
                            },
                            "containerUrl": {
                                "type": "string"
                            },
                            "usage": {
                                "type": "object",
                                "properties": {
                                    "ACTOR_COMPUTE_UNITS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_WRITES": {
                                        "type": "integer",
                                        "example": 1
                                    },
                                    "KEY_VALUE_STORE_LISTS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_INTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_EXTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_RESIDENTIAL_TRANSFER_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_SERPS": {
                                        "type": "integer",
                                        "example": 0
                                    }
                                }
                            },
                            "usageTotalUsd": {
                                "type": "number",
                                "example": 0.00005
                            },
                            "usageUsd": {
                                "type": "object",
                                "properties": {
                                    "ACTOR_COMPUTE_UNITS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_WRITES": {
                                        "type": "number",
                                        "example": 0.00005
                                    },
                                    "KEY_VALUE_STORE_LISTS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_INTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_EXTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_RESIDENTIAL_TRANSFER_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_SERPS": {
                                        "type": "integer",
                                        "example": 0
                                    }
                                }
                            }
                        }
                    }
                }
            }
        }
    }
}
```