# Product Hunt Scraper — Comments, Products & User Profiles (`vulnv/producthunt-scraper`) Actor
Scrape Product Hunt comments, product details, and user profiles at scale. Extract reviews, upvotes, maker badges, product descriptions, media, launch rankings, and commenter social links (Twitter, LinkedIn, GitHub). Supports daily leaderboard URLs and individual product pages.
- **URL**: https://apify.com/vulnv/producthunt-scraper.md
- **Developed by:** [VulnV](https://apify.com/vulnv) (community)
- **Categories:** Social media, Lead generation, Other
- **Stats:** 24 total users, 7 monthly users, 100.0% runs succeeded, 1 bookmarks
- **User rating**: No ratings yet
## Pricing
from $5.00 / 1,000 products
This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event
## What's an Apify Actor?
Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".
## How to integrate an Actor?
If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.
In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md):
```bash
npm install apify-client
```
In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md):
```bash
pip install apify-client
```
In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md):
````bash
# MacOS / Linux
curl -fsSL https://apify.com/install-cli.sh | bash
# Windows
irm https://apify.com/install-cli.ps1 | iex
```bash
In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md).
If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md).
For usage examples, see the [API](#api) section below.
For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).
# README
## Product Hunt Scraper — Comments, Products & User Profiles
Scrape [Product Hunt](https://www.producthunt.com) comments, product data, and user profiles at scale. Extract reviews, upvotes, maker badges, product descriptions, screenshots, launch rankings, categories, and commenter social media links — all in structured JSON.
Works with **individual product URLs**, **post URLs**, **daily leaderboard URLs**, **category URLs**, and **topic URLs** (leaderboard, category, and topic URLs are automatically expanded into their listed products).
### Key features
- **Comments & reviews** — full text (plain + HTML), vote counts, maker/hunter badges, timestamps, sticky/pinned flags
- **Product profiles** — name, tagline, description, logo, screenshots, videos, website, social links, categories, awards, launch rank and score, hunter info
- **User profiles** — name, headline, bio, avatar, Twitter, LinkedIn, GitHub, personal website, follower counts, verified status
- **Leaderboard support** — pass a `/leaderboard/daily/YYYY/M/D` URL to scrape the top N products from any day
- **Category support** — pass a `/categories/{slug}` URL (e.g. `/categories/3d-animation`) to scrape the top N products in that category
- **Topic support** — pass a `/topics/{slug}` URL (e.g. `/topics/android`) to scrape the top N products under that topic
- **S3 cache** — optionally read previously-saved results from S3 first, skipping live fetches and Apify charges for cached products
- **Three structured datasets** with row-level cross-references between comments, products, and users
- **Toggleable outputs** — enable/disable comments, products, or user profile scraping independently
- **Concurrent scraping** — processes multiple products, comment pages, and user profiles in parallel
### Use cases
- **Market research** — analyze Product Hunt launch performance, comment sentiment, and community reception
- **Competitor analysis** — compare products by upvotes, reviews, ratings, categories, and launch rankings
- **Lead generation** — extract commenter profiles with Twitter, LinkedIn, GitHub, and website links
- **Trend tracking** — monitor daily leaderboards to identify trending products and categories
- **Content research** — study what products and launches generate the most engagement
### What data you get
| Dataset | Contents |
|---|---|
| **Comments** (default) | Comment text, HTML body, vote count, badges, timestamps, author reference, product reference |
| **Products** | Name, slug, tagline, description, logo, media, website, social URLs, categories, awards, launch details, review stats |
| **Users** | Name, username, headline, avatar, profile URL, Twitter, LinkedIn, GitHub, website, follower/following counts, verified status |
### Input parameters
| Field | Type | Default | Description |
|---|---|---|---|
| `start_urls` | array | *required* | Product Hunt URLs to scrape — accepts `/products/…`, `/posts/…`, `/leaderboard/daily/YYYY/M/D`, `/categories/…`, or `/topics/…` |
| `max_products` | integer | `1` | Maximum products to scrape (top N for leaderboards/categories/topics, cap for direct URLs) |
| `max_comments` | integer | `100` | Maximum comments to extract per product |
| `scrape_comments` | boolean | `true` | Save comments to the default dataset |
| `scrape_products` | boolean | `true` | Save product info to the products dataset |
| `scrape_users` | boolean | `true` | Fetch and save commenter profiles to the users dataset |
| `use_cache` | boolean | `false` | Read previously-saved results from S3 before scraping live. Skips Zyte/GQL fetches and charges on a cache hit. Requires S3 to be configured. |
#### Example: scrape top 5 products from the daily leaderboard
```json
{
"start_urls": [
{ "url": "https://www.producthunt.com/leaderboard/daily/2026/3/23" }
],
"max_products": 5,
"max_comments": 50,
"scrape_comments": true,
"scrape_products": true,
"scrape_users": true
}
````
#### Example: scrape a single product page
```json
{
"start_urls": [
{ "url": "https://www.producthunt.com/products/tobira-ai" }
],
"max_products": 1,
"max_comments": 200
}
```
#### Example: scrape top 10 products from a category
```json
{
"start_urls": [
{ "url": "https://www.producthunt.com/categories/3d-animation" }
],
"max_products": 10,
"max_comments": 50,
"scrape_comments": true,
"scrape_products": true,
"scrape_users": true
}
```
#### Example: scrape only product metadata (no comments or users)
```json
{
"start_urls": [
{ "url": "https://www.producthunt.com/leaderboard/daily/2026/3/23" }
],
"max_products": 10,
"scrape_comments": false,
"scrape_products": true,
"scrape_users": false
}
```
#### Example: serve results from S3 cache
```json
{
"start_urls": [
{ "url": "https://www.producthunt.com/products/tobira-ai" }
],
"max_products": 1,
"max_comments": 100,
"use_cache": true
}
```
### Output format
Results are split into three datasets. The Apify Console **Output** tab links directly to each dataset with pre-configured table views.
#### Comments dataset (default)
```json
{
"type": "comment",
"id": "5217461",
"body": "Hey PH! ...",
"body_html": "Hey PH!
...",
"vote_count": 21,
"created_at": "2026-03-17T06:24:57-07:00",
"is_sticky": true,
"is_pinned": false,
"badges": ["maker"],
"award": null,
"product_name": "Tobira.ai",
"product_tagline": "A network where AI agents find deals for their humans",
"product_url": "https://www.producthunt.com/posts/tobira-ai",
"user": { "id": "4231147", "name": "Vlad Shipilov", "username": "vlad_shipilov" },
"user_id": "4231147",
"username": "vlad_shipilov",
"product_id": "1183905",
"product_slug": "tobira-ai"
}
```
#### Products dataset
```json
{
"id": "1183905",
"name": "Tobira.ai",
"slug": "tobira-ai",
"tagline": "A network where AI agents find deals for their humans",
"description": "...",
"url": "https://www.producthunt.com/products/tobira-ai",
"logo_url": "https://ph-files.imgix.net/...",
"website_url": "https://tobira.ai",
"twitter_url": "https://twitter.com/shipilov_vlad",
"followers_count": 582,
"reviews_count": 0,
"reviews_rating": 0,
"categories": [{ "name": "AI Agents", "slug": "ai-agents", "path": "/categories/ai-agents" }],
"awards": [{ "position": 1, "period": "daily", "date": "2026-03-23" }],
"media": [{ "type": "image", "url": "https://ph-files.imgix.net/..." }],
"launch": {
"id": "1100794",
"slug": "tobira-ai",
"name": "Tobira.ai",
"daily_rank": "1",
"launch_day_score": 447,
"featured": true,
"featured_at": "2026-03-23T00:01:00-07:00",
"primary_link": "https://tobira.ai",
"hunter": { "name": "fmerian", "username": "fmerian" }
}
}
```
#### Users dataset
```json
{
"id": "4231147",
"name": "Vlad Shipilov",
"username": "vlad_shipilov",
"headline": "Founder Tobira.ai and Revoly.ai",
"profile_url": "https://www.producthunt.com/@vlad_shipilov",
"avatar_url": "https://ph-avatars.imgix.net/...",
"twitter_username": "VladShipilov",
"website_url": null,
"linkedin_url": null,
"github_username": null,
"followers_count": 134,
"is_verified": true,
"social_links": [{ "type": "twitter", "url": "https://twitter.com/VladShipilov" }]
}
```
### Cross-dataset references
Every saved record includes a `storage` object with row-level pointers for joining data across datasets:
| Dataset | `storage` fields |
|---|---|
| **Comments** | `storage.item` (this row), `storage.product` (parent product), `storage.user` (commenter profile) |
| **Products** | `storage.item` (this row) |
| **Users** | `storage.item` (this row) |
Each pointer contains `dataset_id`, `dataset_name`, `item_offset`, `api_url`, and `console_url` for direct API access.
A `DATASET_LINKS` manifest is also written to the default key-value store with dataset IDs and console URLs for all three datasets.
### How it works
1. **URL expansion** — leaderboard and category URLs are fetched via Zyte API's browser rendering and parsed for product links (Apollo SSR + DOM fallback); category pages also paginate if more products are needed
2. **Apollo SSR data extraction** — product pages are rendered in a headless browser; the scraper parses `window[Symbol.for("ApolloSSRDataTransport")]` for structured data from Product Hunt's Next.js/Apollo SSR layer
3. **GraphQL enrichment** — a parallel persisted GraphQL query (`ProductsPageLayout`) fetches additional fields like description, media, categories, awards, and external links
4. **S3 cache** — when `use_cache` is enabled, the scraper checks S3 for `products/{slug}.json` and `comments/{slug}-*.json` before hitting live pages; a full cache hit skips all Zyte/GQL fetches
5. **Comment pagination** — pages beyond the first are fetched concurrently in batches of 5
6. **User profile scraping** — commenter profiles are fetched in batches of 10 and deduplicated across all products in the run
7. **Product concurrency** — up to 5 products are scraped in parallel
### Pricing
This actor uses Apify's **pay-per-event** model. You only pay for the data you save:
| Event | Triggered when |
|---|---|
| `fetch_product` | A product record is saved |
| `fetch_comment` | A comment record is saved |
| `fetch_user` | A user profile is saved |
Disable any dataset type (e.g. `scrape_users: false`) to skip both the scraping and the charge.
### Technical details
- **Runtime**: Python 3.13, Apify SDK 3.3+
- **Dependencies**: httpx, beautifulsoup4, lxml
- **External service**: [Zyte API](https://www.zyte.com/zyte-api/) for browser-rendered HTML and HTTP proxy requests
- **Memory**: 512 MB
# Actor input Schema
## `start_urls` (type: `array`):
Product Hunt URLs to scrape. Accepts product/post URLs (/products/…, /posts/…), leaderboard URLs (/leaderboard/daily/YYYY/M/D), category URLs (/categories/…), or topic URLs (/topics/…). Leaderboard, category, and topic URLs are expanded into their top products automatically.
## `max_products` (type: `integer`):
Maximum number of products to scrape. For leaderboard, category, and topic URLs, takes the top N products. For direct product URLs, caps how many of the provided URLs are processed.
## `max_comments` (type: `integer`):
Maximum number of comments to extract per product.
## `scrape_comments` (type: `boolean`):
Save scraped comments to the comments dataset.
## `scrape_products` (type: `boolean`):
Save product information (name, description, media, launch details, social links, etc.) to the products dataset.
## `scrape_users` (type: `boolean`):
Visit each commenter's profile page and save their extended info (Twitter, LinkedIn, GitHub, website, bio, follower count) to the users dataset.
## `use_cache` (type: `boolean`):
Read previously-saved results from S3 instead of scraping live pages. On a cache hit the scraper skips all Zyte/GQL fetches and Apify charges for that product. Falls back to live scraping on a cache miss. Requires the S3 bucket to be configured.
## Actor input object example
```json
{
"start_urls": [
{
"url": "https://www.producthunt.com/products/claude"
}
],
"max_products": 1,
"max_comments": 10,
"scrape_comments": true,
"scrape_products": true,
"scrape_users": true,
"use_cache": false
}
```
# Actor output Schema
## `comments` (type: `string`):
Review comments with author, votes, and cross-references to product/user rows.
## `products` (type: `string`):
Product profiles with metadata, social links, and review stats.
## `users` (type: `string`):
User profiles including social media and contact info.
# API
You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.
## JavaScript example
```javascript
import { ApifyClient } from 'apify-client';
// Initialize the ApifyClient with your Apify API token
// Replace the '' with your token
const client = new ApifyClient({
token: '',
});
// Prepare Actor input
const input = {
"start_urls": [
{
"url": "https://www.producthunt.com/products/claude"
}
]
};
// Run the Actor and wait for it to finish
const run = await client.actor("vulnv/producthunt-scraper").call(input);
// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
console.dir(item);
});
// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs
```
## Python example
```python
from apify_client import ApifyClient
# Initialize the ApifyClient with your Apify API token
# Replace '' with your token.
client = ApifyClient("")
# Prepare the Actor input
run_input = { "start_urls": [{ "url": "https://www.producthunt.com/products/claude" }] }
# Run the Actor and wait for it to finish
run = client.actor("vulnv/producthunt-scraper").call(run_input=run_input)
# Fetch and print Actor results from the run's dataset (if there are any)
print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
print(item)
# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start
```
## CLI example
```bash
echo '{
"start_urls": [
{
"url": "https://www.producthunt.com/products/claude"
}
]
}' |
apify call vulnv/producthunt-scraper --silent --output-dataset
```
## MCP server setup
```json
{
"mcpServers": {
"apify": {
"command": "npx",
"args": [
"mcp-remote",
"https://mcp.apify.com/?tools=vulnv/producthunt-scraper",
"--header",
"Authorization: Bearer "
]
}
}
}
```
## OpenAPI specification
```json
{
"openapi": "3.0.1",
"info": {
"title": "Product Hunt Scraper — Comments, Products & User Profiles",
"description": "Scrape Product Hunt comments, product details, and user profiles at scale. Extract reviews, upvotes, maker badges, product descriptions, media, launch rankings, and commenter social links (Twitter, LinkedIn, GitHub). Supports daily leaderboard URLs and individual product pages.",
"version": "1.0",
"x-build-id": "Y6MfkkWCZKKlpK0By"
},
"servers": [
{
"url": "https://api.apify.com/v2"
}
],
"paths": {
"/acts/vulnv~producthunt-scraper/run-sync-get-dataset-items": {
"post": {
"operationId": "run-sync-get-dataset-items-vulnv-producthunt-scraper",
"x-openai-isConsequential": false,
"summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.",
"tags": [
"Run Actor"
],
"requestBody": {
"required": true,
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/inputSchema"
}
}
}
},
"parameters": [
{
"name": "token",
"in": "query",
"required": true,
"schema": {
"type": "string"
},
"description": "Enter your Apify token here"
}
],
"responses": {
"200": {
"description": "OK"
}
}
}
},
"/acts/vulnv~producthunt-scraper/runs": {
"post": {
"operationId": "runs-sync-vulnv-producthunt-scraper",
"x-openai-isConsequential": false,
"summary": "Executes an Actor and returns information about the initiated run in response.",
"tags": [
"Run Actor"
],
"requestBody": {
"required": true,
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/inputSchema"
}
}
}
},
"parameters": [
{
"name": "token",
"in": "query",
"required": true,
"schema": {
"type": "string"
},
"description": "Enter your Apify token here"
}
],
"responses": {
"200": {
"description": "OK",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/runsResponseSchema"
}
}
}
}
}
}
},
"/acts/vulnv~producthunt-scraper/run-sync": {
"post": {
"operationId": "run-sync-vulnv-producthunt-scraper",
"x-openai-isConsequential": false,
"summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.",
"tags": [
"Run Actor"
],
"requestBody": {
"required": true,
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/inputSchema"
}
}
}
},
"parameters": [
{
"name": "token",
"in": "query",
"required": true,
"schema": {
"type": "string"
},
"description": "Enter your Apify token here"
}
],
"responses": {
"200": {
"description": "OK"
}
}
}
}
},
"components": {
"schemas": {
"inputSchema": {
"type": "object",
"required": [
"start_urls"
],
"properties": {
"start_urls": {
"title": "🌐 Product Hunt URLs",
"type": "array",
"description": "Product Hunt URLs to scrape. Accepts product/post URLs (/products/…, /posts/…), leaderboard URLs (/leaderboard/daily/YYYY/M/D), category URLs (/categories/…), or topic URLs (/topics/…). Leaderboard, category, and topic URLs are expanded into their top products automatically.",
"items": {
"type": "object",
"required": [
"url"
],
"properties": {
"url": {
"type": "string",
"title": "URL of a web page",
"format": "uri"
}
}
}
},
"max_products": {
"title": "🏆 Max Products",
"minimum": 1,
"maximum": 50,
"type": "integer",
"description": "Maximum number of products to scrape. For leaderboard, category, and topic URLs, takes the top N products. For direct product URLs, caps how many of the provided URLs are processed.",
"default": 1
},
"max_comments": {
"title": "💬 Max Comments per Product",
"minimum": 1,
"maximum": 5000,
"type": "integer",
"description": "Maximum number of comments to extract per product.",
"default": 10
},
"scrape_comments": {
"title": "💬 Save Comments",
"type": "boolean",
"description": "Save scraped comments to the comments dataset.",
"default": true
},
"scrape_products": {
"title": "📦 Save Products",
"type": "boolean",
"description": "Save product information (name, description, media, launch details, social links, etc.) to the products dataset.",
"default": true
},
"scrape_users": {
"title": "👤 Save Users",
"type": "boolean",
"description": "Visit each commenter's profile page and save their extended info (Twitter, LinkedIn, GitHub, website, bio, follower count) to the users dataset.",
"default": true
},
"use_cache": {
"title": "💾 Use S3 Cache",
"type": "boolean",
"description": "Read previously-saved results from S3 instead of scraping live pages. On a cache hit the scraper skips all Zyte/GQL fetches and Apify charges for that product. Falls back to live scraping on a cache miss. Requires the S3 bucket to be configured.",
"default": false
}
}
},
"runsResponseSchema": {
"type": "object",
"properties": {
"data": {
"type": "object",
"properties": {
"id": {
"type": "string"
},
"actId": {
"type": "string"
},
"userId": {
"type": "string"
},
"startedAt": {
"type": "string",
"format": "date-time",
"example": "2025-01-08T00:00:00.000Z"
},
"finishedAt": {
"type": "string",
"format": "date-time",
"example": "2025-01-08T00:00:00.000Z"
},
"status": {
"type": "string",
"example": "READY"
},
"meta": {
"type": "object",
"properties": {
"origin": {
"type": "string",
"example": "API"
},
"userAgent": {
"type": "string"
}
}
},
"stats": {
"type": "object",
"properties": {
"inputBodyLen": {
"type": "integer",
"example": 2000
},
"rebootCount": {
"type": "integer",
"example": 0
},
"restartCount": {
"type": "integer",
"example": 0
},
"resurrectCount": {
"type": "integer",
"example": 0
},
"computeUnits": {
"type": "integer",
"example": 0
}
}
},
"options": {
"type": "object",
"properties": {
"build": {
"type": "string",
"example": "latest"
},
"timeoutSecs": {
"type": "integer",
"example": 300
},
"memoryMbytes": {
"type": "integer",
"example": 1024
},
"diskMbytes": {
"type": "integer",
"example": 2048
}
}
},
"buildId": {
"type": "string"
},
"defaultKeyValueStoreId": {
"type": "string"
},
"defaultDatasetId": {
"type": "string"
},
"defaultRequestQueueId": {
"type": "string"
},
"buildNumber": {
"type": "string",
"example": "1.0.0"
},
"containerUrl": {
"type": "string"
},
"usage": {
"type": "object",
"properties": {
"ACTOR_COMPUTE_UNITS": {
"type": "integer",
"example": 0
},
"DATASET_READS": {
"type": "integer",
"example": 0
},
"DATASET_WRITES": {
"type": "integer",
"example": 0
},
"KEY_VALUE_STORE_READS": {
"type": "integer",
"example": 0
},
"KEY_VALUE_STORE_WRITES": {
"type": "integer",
"example": 1
},
"KEY_VALUE_STORE_LISTS": {
"type": "integer",
"example": 0
},
"REQUEST_QUEUE_READS": {
"type": "integer",
"example": 0
},
"REQUEST_QUEUE_WRITES": {
"type": "integer",
"example": 0
},
"DATA_TRANSFER_INTERNAL_GBYTES": {
"type": "integer",
"example": 0
},
"DATA_TRANSFER_EXTERNAL_GBYTES": {
"type": "integer",
"example": 0
},
"PROXY_RESIDENTIAL_TRANSFER_GBYTES": {
"type": "integer",
"example": 0
},
"PROXY_SERPS": {
"type": "integer",
"example": 0
}
}
},
"usageTotalUsd": {
"type": "number",
"example": 0.00005
},
"usageUsd": {
"type": "object",
"properties": {
"ACTOR_COMPUTE_UNITS": {
"type": "integer",
"example": 0
},
"DATASET_READS": {
"type": "integer",
"example": 0
},
"DATASET_WRITES": {
"type": "integer",
"example": 0
},
"KEY_VALUE_STORE_READS": {
"type": "integer",
"example": 0
},
"KEY_VALUE_STORE_WRITES": {
"type": "number",
"example": 0.00005
},
"KEY_VALUE_STORE_LISTS": {
"type": "integer",
"example": 0
},
"REQUEST_QUEUE_READS": {
"type": "integer",
"example": 0
},
"REQUEST_QUEUE_WRITES": {
"type": "integer",
"example": 0
},
"DATA_TRANSFER_INTERNAL_GBYTES": {
"type": "integer",
"example": 0
},
"DATA_TRANSFER_EXTERNAL_GBYTES": {
"type": "integer",
"example": 0
},
"PROXY_RESIDENTIAL_TRANSFER_GBYTES": {
"type": "integer",
"example": 0
},
"PROXY_SERPS": {
"type": "integer",
"example": 0
}
}
}
}
}
}
}
}
}
}
```