# Producthunt Scraper (`runtime/producthunt-scraper`) Actor
A web scraper that extracts comprehensive product information from Product Hunt using Apify.
- **URL**: https://apify.com/runtime/producthunt-scraper.md
- **Developed by:** [scraping automation](https://apify.com/runtime) (community)
- **Categories:** Lead generation, E-commerce, News
- **Stats:** 115 total users, 2 monthly users, 89.7% runs succeeded, 7 bookmarks
- **User rating**: 5.00 out of 5 stars
## Pricing
$29.00/month + usage
To use this Actor, you pay a monthly rental fee to the developer. The rent is subtracted from your prepaid usage every month after the free trial period.You also pay for the Apify platform usage, which gets cheaper the higher Apify subscription plan you have.
Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#rental-actors
## What's an Apify Actor?
Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".
## How to integrate an Actor?
If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.
In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md):
```bash
npm install apify-client
```
In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md):
```bash
pip install apify-client
```
In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md):
````bash
# MacOS / Linux
curl -fsSL https://apify.com/install-cli.sh | bash
# Windows
irm https://apify.com/install-cli.ps1 | iex
```bash
In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md).
If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md).
For usage examples, see the [API](#api) section below.
For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).
# README
## Product Hunt Scraper
A web scraper that extracts comprehensive product information from Product Hunt using Apify and Crawlee.
### π Quick Start
1. **Input**: Configure your scraping parameters in the input field
2. **Run**: Click "Start" to begin scraping
3. **Output**: Download results from the Dataset tab
### β‘ Performance
- **Speed**: Scrapes ~10 products with full details in 4-5 minutes
- **Efficiency**: ~25-30 seconds per product (with details, comments, reviews)
- **Scale**: Can handle 100+ products per run with proper timeout settings
- **Reliability**: Built with retry logic and anti-detection measures
### π‘ Use Cases
This Actor is perfect for:
- **Market Research**: Track trending products and emerging technologies
- **Competitor Analysis**: Monitor competitor launches, upvotes, and community feedback
- **Lead Generation**: Find product makers and hunters for outreach
- **Content Creation**: Discover popular products for blog posts, newsletters, or social media
- **Trend Analysis**: Analyze product categories and built-with tools over time
- **Investment Research**: Track startup launches and community reception
- **Community Insights**: Extract comments and reviews for sentiment analysis
- **Historical Data**: Build datasets of Product Hunt launches over specific date ranges
### Input Configuration
> **Note:** If you provide a **Start Date** (and/or End Date), the **Start URLs** field will be ignored. Only one method (date range OR Start URLs) will be used per run.
#### Basic Configuration
```json
{
"startUrls": ["https://www.producthunt.com/"],
"maxRequestRetries": 3,
"maxConcurrency": 5,
"maxRequestsPerCrawl": 100,
"scrapeComingSoon": true
}
````
#### Daily Leaderboard Scraping
```json
{
"startUrls": ["https://www.producthunt.com/leaderboard/daily/2025/7/5/all"],
"scrapeDailyLeaderboard": true,
"maxRequestsPerCrawl": 50
}
```
#### Input Parameters
| Parameter | Type | Default | Description |
| ------------------------ | ------- | ---------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `startUrls` | Array | `["https://www.producthunt.com/"]` | Starting URLs for scraping |
| `maxResults` | Number | `8` | Maximum number of product detail pages to process per run. Lower values reduce timeout risk. |
| `maxRequestRetries` | Number | `3` | Maximum retry attempts for failed requests |
| `maxConcurrency` | Number | `5` | Number of concurrent requests |
| `maxRequestsPerCrawl` | Number | `100` | Maximum pages to crawl |
| `timeoutSecs` | Number | `900` | Maximum runtime (seconds) before the Actor stops automatically |
| `useApifyProxy` | Boolean | `true` | Simple checkbox to enable/disable Apify Proxy without touching advanced settings |
| `proxyConfiguration` | Object | `{}` | Advanced proxy editor (choose Apify Proxy groups, custom proxy URLs, or no proxy). When left empty it inherits the `useApifyProxy` toggle. |
| `scrapeComingSoon` | Boolean | `true` | Whether to scrape "Coming Soon" products |
| `scrapeDailyLeaderboard` | Boolean | `false` | Whether to scrape daily leaderboard |
| `sortByDate` | Boolean | `false` | Whether to sort output by date |
| `sortOrder` | String | `"desc"` | Sort order: `"asc"` (oldest first) or `"desc"` (newest first) |
| `startDate` | String | `null` | Start date for daily leaderboard range (format: `YYYY-MM-DD` or `YYYY/MM/DD`) |
| `endDate` | String | `null` | End date for daily leaderboard range (defaults to yesterday if not specified) |
| `maxCommentPages` | Number | `0` | Maximum number of comment pages to scrape per product (0-10). Comments are paginated on the main product page. The scraper will automatically stop if no more pages are available. |
### Input Method Exclusivity
- If you provide a **Start Date** (and/or End Date), the **Start URLs** field will be ignored.
- If you do **not** provide a Start Date, the scraper will use the Start URLs.
- If neither is provided, the scraper will **default to scraping the daily leaderboard for yesterday** (today - 1 day).
#### Examples
**Default (no input, scrapes yesterday's leaderboard):**
```json
{}
```
// Will scrape the daily leaderboard for yesterday
**Date range only (Start URLs ignored):**
```json
{
"startDate": "2025-07-01",
"endDate": "2025-07-05"
}
```
**Start URLs only (date fields empty):**
```json
{
"startUrls": ["https://www.producthunt.com/leaderboard/daily/2025/7/4/all"]
}
```
**Both provided (date range takes precedence):**
```json
{
"startDate": "2025-07-01",
"endDate": "2025-07-01",
"startUrls": ["https://www.producthunt.com/leaderboard/daily/2025/7/4/all"]
}
// Only 2025-07-01 will be scraped
```
### Output Format
#### Regular Products
```json
{
"name": "Product Name",
"tagline": "Product tagline",
"description": "Detailed description",
"upvotes": 1234,
"categories": ["SaaS", "Productivity"],
"launchDate": "Launch date",
"imageUrl": "https://example.com/image.jpg",
"productUrl": "https://product-website.com",
"socialLinks": [
{
"platform": "twitter",
"url": "https://twitter.com/product"
}
],
"scrapedAt": "2024-01-01T12:00:00.000Z",
"companyWebsite": "https://company.com",
"productHuntUrl": "https://www.producthunt.com/products/product-name",
"makers": [
{ "username": "maker1", "name": "Maker One", "roles": ["Maker"] },
{ "username": "maker2", "name": "Maker Two", "roles": ["Maker", "Hunter"] }
],
"hunter": { "username": "hunter1", "name": "Hunter Name" },
"builtWith": [
{
"name": "shadcn/ui",
"url": "https://www.producthunt.com/products/shadcn-ui",
"description": "Beautifully designed components.",
"imageUrl": "https://ph-files.imgix.net/..."
}
],
"launches": [
{
"postId": "1038207",
"title": "Product Name",
"url": "https://www.producthunt.com/products/product-name/launches/product-name",
"tagline": "Product tagline",
"date": "November 15th, 2025",
"rank": 1,
"upvotes": 308,
"comments": 43,
"imageUrl": "https://example.com/image.jpg",
"imageAlt": "Product Name"
}
],
"launchCount": 1,
"comments": [
{
"id": "5002031",
"username": "adrm",
"userName": "AdriΓ‘n de la Rosa",
"userUrl": "https://www.producthunt.com/@adrm",
"userAvatar": "https://ph-avatars.imgix.net/...",
"isMaker": true,
"text": "Comment text content...",
"html": "Comment HTML content...
",
"upvotes": 33,
"timestamp": "2025-11-14T09:06:07-08:00",
"timeAgo": "2d ago"
}
],
"commentCount": 43,
"reviews": [],
"reviewCount": 0
}
```
#### Daily Leaderboard Products
```json
{
"name": "Product Name",
"tagline": "Product description",
"categories": ["SaaS", "Productivity"],
"upvotes": "1234",
"launchDate": "July 5, 2025",
"imageUrl": "https://example.com/image.jpg",
"productUrl": "https://www.producthunt.com/products/product-name",
"scrapedFrom": "daily-leaderboard",
"scrapedAt": "2024-01-01T12:00:00.000Z",
"productHuntUrl": "https://www.producthunt.com/leaderboard/daily/2025/7/5/all",
"makers": [{ "username": "maker1", "name": "Maker One", "roles": ["Maker"] }],
"hunter": { "username": "hunter1", "name": "Hunter Name" }
}
```
#### Team Extraction Output
- **makers**: Array of all team members with the "Maker" role. Each object contains:
- `username`: Product Hunt username (string)
- `name`: Display name (string)
- `roles`: Array of roles (e.g. `["Maker"]` or `["Hunter", "Maker"]`)
- `title`: Job title or role description (string, optional)
- **hunter**: Object with the first team member who has the "Hunter" role, with:
- `username`: Product Hunt username (string)
- `name`: Display name (string)
- If no hunter is found, this field is `null`.
- Note: A hunter can also be a maker (they will appear in both `makers` and `hunter` fields).
#### Additional Data Fields
- **builtWith**: Array of tools/products used to build the product. Each object contains:
- `name`: Tool/product name (string)
- `url`: Product Hunt URL (string)
- `description`: Tool description (string)
- `imageUrl`: Tool thumbnail image URL (string, optional)
- **launches**: Array of all launches for the product. Each object contains:
- `postId`: Launch post ID (string)
- `title`: Launch title (string)
- `url`: Launch URL (string)
- `tagline`: Launch tagline (string)
- `date`: Launch date (string)
- `rank`: Daily rank (number)
- `upvotes`: Number of upvotes (number)
- `comments`: Number of comments (number)
- `imageUrl`: Launch image URL (string, optional)
- `imageAlt`: Image alt text (string, optional)
- **launchCount**: Total number of launches (number)
- **comments**: Array of comments from the main product page. Each object contains:
- `id`: Comment ID (string)
- `username`: Commenter's username (string)
- `userName`: Commenter's display name (string)
- `userUrl`: Commenter's profile URL (string)
- `userAvatar`: Commenter's avatar URL (string, optional)
- `isMaker`: Whether the commenter is a maker (boolean)
- `text`: Comment text content (string)
- `html`: Comment HTML content (string)
- `upvotes`: Number of upvotes (number)
- `timestamp`: ISO timestamp (string)
- `timeAgo`: Human-readable time (string, e.g. "2d ago")
- **commentCount**: Total number of comments extracted (number)
- **reviews**: Array of reviews from the `/reviews` page (same structure as comments). Often empty as reviews are less common.
- **reviewCount**: Total number of reviews extracted (number)
#### Comment Pagination
Comments are paginated on Product Hunt. The scraper supports pagination through the `maxCommentPages` parameter:
- **Default**: `0` (skip comment extraction for fastest runs)
- **Range**: `0-10` (set to `0` to disable comment extraction)
- **Behavior**:
- The scraper visits up to `maxCommentPages` pages of comments
- It automatically stops if no more pages are available (even if `maxCommentPages` is higher)
- Each page is visited sequentially: `?page=1#comments`, `?page=2#comments`, etc.
- All comments from all pages are combined into a single `comments` array
**Example with pagination:**
```json
{
"maxCommentPages": 5
}
```
This will scrape up to 5 pages of comments per product, stopping early if fewer pages are available.
### Usage Examples
#### Scrape Today's Products
```json
{
"startUrls": ["https://www.producthunt.com/"],
"maxRequestsPerCrawl": 50
}
```
#### Scrape Daily Leaderboard
```json
{
"startUrls": ["https://www.producthunt.com/leaderboard/daily/2025/7/5/all"],
"scrapeDailyLeaderboard": true
}
```
#### Scrape Coming Soon Products
```json
{
"startUrls": ["https://www.producthunt.com/coming-soon"],
"scrapeComingSoon": true
}
```
#### Scrape Specific Categories
```json
{
"startUrls": [
"https://www.producthunt.com/categories/developer-tools",
"https://www.producthunt.com/categories/productivity"
]
}
```
#### Scrape with Date Sorting
```json
{
"startUrls": ["https://www.producthunt.com/"],
"sortByDate": true,
"sortOrder": "desc"
}
```
#### Scrape with Comment Pagination
```json
{
"startUrls": ["https://www.producthunt.com/products/product-name"],
"maxCommentPages": 5
}
```
This will scrape up to 5 pages of comments per product.
#### Scrape Daily Leaderboard with Date Sorting (Oldest First)
```json
{
"startUrls": ["https://www.producthunt.com/leaderboard/daily/2025/7/5/all"],
"scrapeDailyLeaderboard": true,
"sortByDate": true,
"sortOrder": "asc"
}
```
#### Scrape Daily Leaderboard Date Range
```json
{
"startDate": "2025-07-01",
"endDate": "2025-07-05",
"scrapeDailyLeaderboard": true,
"maxRequestsPerCrawl": 200
}
```
#### Scrape Daily Leaderboard from Date to Yesterday
```json
{
"startDate": "2025-07-01",
"scrapeDailyLeaderboard": true,
"sortByDate": true,
"sortOrder": "desc"
}
```
#### Combine Custom URLs with Date Range
```json
{
"startUrls": [
"https://www.producthunt.com/leaderboard/daily/2025/7/5/all",
"https://www.producthunt.com/leaderboard/daily/2025/7/4/all"
],
"startDate": "2025-07-01",
"endDate": "2025-07-03",
"scrapeDailyLeaderboard": true
}
```
### Data Fields
| Field | Description | Available For |
| -------------- | ------------------------------------- | ---------------------------------------------------------------------- |
| `name` | Product name | All products |
| `tagline` | Short product description | All products |
| `description` | Detailed product description | Regular products |
| `upvotes` | Number of upvotes | All products |
| `categories` | Array of product categories | All products |
| `makers` | Array of makers with roles | All products (when `getDetails` is true) |
| `hunter` | Hunter information | All products (when `getDetails` is true) |
| `launchDate` | Product launch date | All products |
| `imageUrl` | Product image URL | All products |
| `productUrl` | Direct product website link | All products |
| `pricing` | Pricing information | Regular products |
| `metaKeywords` | Meta keywords | Regular products |
| `socialLinks` | Social media links | Regular products |
| `builtWith` | Tools used to build the product | Regular products (when `getDetails` is true) |
| `launches` | Array of all product launches | Regular products (when `getDetails` is true) |
| `launchCount` | Total number of launches | Regular products (when `getDetails` is true) |
| `comments` | Array of comments from main page | Regular products (when `getDetails` is true and `maxCommentPages` > 0) |
| `commentCount` | Total number of comments | Regular products (when `getDetails` is true and `maxCommentPages` > 0) |
| `reviews` | Array of reviews from `/reviews` page | Regular products (when `getDetails` is true) |
| `reviewCount` | Total number of reviews | Regular products (when `getDetails` is true) |
| `scrapedFrom` | Data source identifier | All products |
| `scrapedAt` | Timestamp of scraping | All products |
| `sourceUrl` | Original Product Hunt URL | All products |
### Performance Tips
- **Concurrency**: Increase `maxConcurrency` for faster scraping (be mindful of rate limits)
- **Retries**: Higher `maxRequestRetries` values improve reliability but slow down scraping
- **Request Limits**: Adjust `maxRequestsPerCrawl` based on your needs
- **Proxy**: Use the `useApifyProxy` checkbox for quick on/off control, or fill `proxyConfiguration` for custom pools/URLs. Camoufox-based Apify Proxy is enabled by default for anti-detection.
- **Sorting**: Enable `sortByDate` to get chronologically ordered results (adds processing time for large datasets)
- **Date Ranges**: Large date ranges will generate many URLs; increase `maxRequestsPerCrawl` accordingly
- **Comment Pagination**: Each comment page adds ~2-3 seconds per product. The default `maxCommentPages=0` skips comments entirely for maximum speed; raise it to collect discussion threads at the cost of additional time.
- **Product Details**: Set `getDetails` to `false` to skip detailed product pages and scrape only leaderboard summaries (much faster)
### β Troubleshooting & FAQs
#### Common Issues
**Q: No products found or scraping returns empty results**
- **A**: Product Hunt may have updated their HTML structure. Check the Actor logs for details. If this persists, contact support or check for Actor updates.
**Q: Actor times out before completing**
- **A**: Reduce `maxResults` (recommended: 8-10 products per run with details) or increase `timeoutSecs` to 1800-3600 seconds. Each product with full details takes ~25-30 seconds.
**Q: Missing team members (makers/hunters)**
- **A**: Some products may not have team data publicly available, or it may be behind a "Show more" button that failed to load. The Actor attempts multiple extraction strategies but some data may be unavailable.
**Q: Getting rate limited or blocked**
- **A**: Enable `useApifyProxy` (default: enabled) and reduce `maxConcurrency` to 1-3. Add more delays by reducing concurrency further.
**Q: Comments or reviews are empty**
- **A**: Set `maxCommentPages` > 0 and ensure `getDetails` is enabled. Some products may not have comments yet.
**Q: How do I scrape multiple date ranges?**
- **A**: Set `startDate` and `endDate` to define a range. The Actor will automatically generate URLs for each day. Increase `maxRequestsPerCrawl` accordingly (e.g., 200+ for multi-day scraping).
**Q: What's the difference between startUrls and date range scraping?**
- **A**: If you provide `startDate`, the Actor ignores `startUrls` and generates daily leaderboard URLs automatically. Use `startUrls` for custom scraping (specific products, categories, etc.).
**Q: Can I scrape specific product categories?**
- **A**: Yes! Use `startUrls` with category URLs like `https://www.producthunt.com/categories/developer-tools`
#### Performance Tips
- **Fast scraping**: Set `getDetails: false` to only scrape leaderboard summaries (~2-3 seconds per product)
- **Full data extraction**: Set `getDetails: true` with `maxCommentPages: 0` for detailed info without comments (~15-20 seconds per product)
- **Complete dataset**: Enable `getDetails: true` and `maxCommentPages: 1-5` for everything including discussions (~30-60 seconds per product)
#### Debug Mode
Enable debug logging by checking the actor logs in the Apify console for detailed execution information.
### Legal Notice
- Respect Product Hunt's robots.txt and terms of service
- Use reasonable request rates
- Use scraped data responsibly and in accordance with applicable laws
- This scraper is for educational and research purposes
### Support
For issues and questions:
1. Check the troubleshooting section
2. Review the actor logs
3. Contact Apify support
***
**Note**: Always respect website terms of service and use data responsibly.
### Related Actors
- [CNN Top Headlines Scraper Actor](https://apify.com/runtime/cnn-top-headlines): Scrape the latest top news headlines and full article details from CNN.
# Actor input Schema
## `startUrls` (type: `string`):
URLs to start scraping from (one per line). If Start Date is provided, this field will be ignored.
## `startDate` (type: `string`):
Start date for daily leaderboard scraping (YYYY-MM-DD format). If provided, Start URLs will be ignored. Leave empty to use Start URLs instead.
## `endDate` (type: `string`):
End date for daily leaderboard scraping (YYYY-MM-DD format). If not provided, defaults to yesterday. Only used when Start Date is provided.
## `getDetails` (type: `boolean`):
Visit each product's detail page to extract comprehensive information including makers, hunters, comments, reviews, launches, and built-with tools. If false, only leaderboard summary data will be scraped (much faster).
## `maxResults` (type: `integer`):
Maximum number of product detail pages to process. Lower values reduce timeout risk. Recommended: 8-10 products with detail scraping enabled.
## `timeoutSecs` (type: `integer`):
Maximum runtime before the Actor stops. Increase for large scraping jobs. Recommended: 900s for ~10 products with details, 1800s for ~25 products.
## `maxRequestRetries` (type: `integer`):
Maximum retries for failed requests. Higher values improve reliability but may slow down scraping.
## `maxConcurrency` (type: `integer`):
Number of pages to process simultaneously. Lower values (1-3) are more stable, higher values (5-10) are faster but may hit rate limits.
## `maxRequestsPerCrawl` (type: `integer`):
Total number of pages to crawl. Increase for large date ranges (e.g., 200+ for multi-day leaderboard scraping with details).
## `useApifyProxy` (type: `boolean`):
Enable Apify Proxy for improved reliability and to avoid rate limiting. Highly recommended for production use.
## `proxyConfiguration` (type: `object`):
Advanced proxy settings. Choose Apify Proxy groups, custom proxy URLs, or no proxy. Leave empty to use the 'Use Apify Proxy' checkbox above.
## `maxCommentPages` (type: `integer`):
Number of comment pages to scrape per product (0-10). Each page adds ~2-3 seconds per product. Set to 0 to skip comments entirely (fastest). Only applies when 'Get Product Details' is enabled.
## Actor input object example
```json
{
"startUrls": "https://www.producthunt.com/\nhttps://www.producthunt.com/leaderboard/daily/2025/1/15/all",
"startDate": "2026-01-01",
"endDate": "2026-01-21",
"getDetails": false,
"maxResults": 10,
"timeoutSecs": 900,
"maxRequestRetries": 5,
"maxConcurrency": 3,
"maxRequestsPerCrawl": 200,
"useApifyProxy": true,
"proxyConfiguration": {
"useApifyProxy": true,
"apifyProxyGroups": []
},
"maxCommentPages": 0
}
```
# Actor output Schema
## `name` (type: `string`):
Name of the product
## `tagline` (type: `string`):
Product tagline or short description
## `description` (type: `string`):
Full product description
## `upvotes` (type: `string`):
Number of upvotes on Product Hunt
## `categories` (type: `string`):
Product categories and topics
## `imageUrl` (type: `string`):
Product image or thumbnail URL
## `productUrl` (type: `string`):
External product website URL
## `productHuntUrl` (type: `string`):
Product Hunt page URL
## `socialLinks` (type: `string`):
Social media profile links
## `launchDate` (type: `string`):
Year the product was launched on Product Hunt
## `scrapedAt` (type: `string`):
ISO timestamp when this product was scraped
## `makerTwitterHandle` (type: `string`):
Twitter handle of the product maker
## `companyWebsite` (type: `string`):
Company website URL from Company Info section
## `domainUrl` (type: `string`):
Domain of the product
## `makers` (type: `string`):
Array of product makers with username, name, and roles
## `hunter` (type: `string`):
Product hunter information with username and name
## `builtWith` (type: `string`):
Tools and technologies used to build the product
## `launches` (type: `string`):
Array of all product launches on Product Hunt
## `launchCount` (type: `string`):
Total number of launches
## `comments` (type: `string`):
Array of product comments and discussions
## `commentCount` (type: `string`):
Total number of comments extracted
## `reviews` (type: `string`):
Array of product reviews
## `reviewCount` (type: `string`):
Total number of reviews extracted
# API
You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.
## JavaScript example
```javascript
import { ApifyClient } from 'apify-client';
// Initialize the ApifyClient with your Apify API token
// Replace the '' with your token
const client = new ApifyClient({
token: '',
});
// Prepare Actor input
const input = {
"useApifyProxy": true,
"proxyConfiguration": {
"useApifyProxy": true,
"apifyProxyGroups": []
}
};
// Run the Actor and wait for it to finish
const run = await client.actor("runtime/producthunt-scraper").call(input);
// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`πΎ Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
console.dir(item);
});
// π Want to learn more π? Go to β https://docs.apify.com/api/client/js/docs
```
## Python example
```python
from apify_client import ApifyClient
# Initialize the ApifyClient with your Apify API token
# Replace '' with your token.
client = ApifyClient("")
# Prepare the Actor input
run_input = {
"useApifyProxy": True,
"proxyConfiguration": {
"useApifyProxy": True,
"apifyProxyGroups": [],
},
}
# Run the Actor and wait for it to finish
run = client.actor("runtime/producthunt-scraper").call(run_input=run_input)
# Fetch and print Actor results from the run's dataset (if there are any)
print("πΎ Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
print(item)
# π Want to learn more π? Go to β https://docs.apify.com/api/client/python/docs/quick-start
```
## CLI example
```bash
echo '{
"useApifyProxy": true,
"proxyConfiguration": {
"useApifyProxy": true,
"apifyProxyGroups": []
}
}' |
apify call runtime/producthunt-scraper --silent --output-dataset
```
## MCP server setup
```json
{
"mcpServers": {
"apify": {
"command": "npx",
"args": [
"mcp-remote",
"https://mcp.apify.com/?tools=runtime/producthunt-scraper",
"--header",
"Authorization: Bearer "
]
}
}
}
```
## OpenAPI specification
```json
{
"openapi": "3.0.1",
"info": {
"title": "Producthunt Scraper",
"description": "A web scraper that extracts comprehensive product information from Product Hunt using Apify.",
"version": "0.7",
"x-build-id": "0lw2NJjXIJ3lv4LgM"
},
"servers": [
{
"url": "https://api.apify.com/v2"
}
],
"paths": {
"/acts/runtime~producthunt-scraper/run-sync-get-dataset-items": {
"post": {
"operationId": "run-sync-get-dataset-items-runtime-producthunt-scraper",
"x-openai-isConsequential": false,
"summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.",
"tags": [
"Run Actor"
],
"requestBody": {
"required": true,
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/inputSchema"
}
}
}
},
"parameters": [
{
"name": "token",
"in": "query",
"required": true,
"schema": {
"type": "string"
},
"description": "Enter your Apify token here"
}
],
"responses": {
"200": {
"description": "OK"
}
}
}
},
"/acts/runtime~producthunt-scraper/runs": {
"post": {
"operationId": "runs-sync-runtime-producthunt-scraper",
"x-openai-isConsequential": false,
"summary": "Executes an Actor and returns information about the initiated run in response.",
"tags": [
"Run Actor"
],
"requestBody": {
"required": true,
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/inputSchema"
}
}
}
},
"parameters": [
{
"name": "token",
"in": "query",
"required": true,
"schema": {
"type": "string"
},
"description": "Enter your Apify token here"
}
],
"responses": {
"200": {
"description": "OK",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/runsResponseSchema"
}
}
}
}
}
}
},
"/acts/runtime~producthunt-scraper/run-sync": {
"post": {
"operationId": "run-sync-runtime-producthunt-scraper",
"x-openai-isConsequential": false,
"summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.",
"tags": [
"Run Actor"
],
"requestBody": {
"required": true,
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/inputSchema"
}
}
}
},
"parameters": [
{
"name": "token",
"in": "query",
"required": true,
"schema": {
"type": "string"
},
"description": "Enter your Apify token here"
}
],
"responses": {
"200": {
"description": "OK"
}
}
}
}
},
"components": {
"schemas": {
"inputSchema": {
"type": "object",
"properties": {
"startUrls": {
"title": "Start URLs",
"type": "string",
"description": "URLs to start scraping from (one per line). If Start Date is provided, this field will be ignored."
},
"startDate": {
"title": "Start Date",
"pattern": "^\\d{4}-\\d{2}-\\d{2}$",
"type": "string",
"description": "Start date for daily leaderboard scraping (YYYY-MM-DD format). If provided, Start URLs will be ignored. Leave empty to use Start URLs instead."
},
"endDate": {
"title": "End Date",
"pattern": "^\\d{4}-\\d{2}-\\d{2}$",
"type": "string",
"description": "End date for daily leaderboard scraping (YYYY-MM-DD format). If not provided, defaults to yesterday. Only used when Start Date is provided."
},
"getDetails": {
"title": "Get Product Details",
"type": "boolean",
"description": "Visit each product's detail page to extract comprehensive information including makers, hunters, comments, reviews, launches, and built-with tools. If false, only leaderboard summary data will be scraped (much faster).",
"default": false
},
"maxResults": {
"title": "Max Results",
"minimum": 1,
"type": "integer",
"description": "Maximum number of product detail pages to process. Lower values reduce timeout risk. Recommended: 8-10 products with detail scraping enabled.",
"default": 8
},
"timeoutSecs": {
"title": "Timeout (seconds)",
"minimum": 300,
"maximum": 3600,
"type": "integer",
"description": "Maximum runtime before the Actor stops. Increase for large scraping jobs. Recommended: 900s for ~10 products with details, 1800s for ~25 products.",
"default": 900
},
"maxRequestRetries": {
"title": "Max Request Retries",
"minimum": 1,
"maximum": 10,
"type": "integer",
"description": "Maximum retries for failed requests. Higher values improve reliability but may slow down scraping.",
"default": 5
},
"maxConcurrency": {
"title": "Max Concurrency",
"minimum": 1,
"maximum": 10,
"type": "integer",
"description": "Number of pages to process simultaneously. Lower values (1-3) are more stable, higher values (5-10) are faster but may hit rate limits.",
"default": 3
},
"maxRequestsPerCrawl": {
"title": "Max Requests Per Crawl",
"minimum": 50,
"maximum": 1000,
"type": "integer",
"description": "Total number of pages to crawl. Increase for large date ranges (e.g., 200+ for multi-day leaderboard scraping with details).",
"default": 200
},
"useApifyProxy": {
"title": "Use Apify Proxy",
"type": "boolean",
"description": "Enable Apify Proxy for improved reliability and to avoid rate limiting. Highly recommended for production use.",
"default": true
},
"proxyConfiguration": {
"title": "Proxy Configuration",
"type": "object",
"description": "Advanced proxy settings. Choose Apify Proxy groups, custom proxy URLs, or no proxy. Leave empty to use the 'Use Apify Proxy' checkbox above.",
"default": {}
},
"maxCommentPages": {
"title": "Max Comment Pages",
"minimum": 0,
"maximum": 10,
"type": "integer",
"description": "Number of comment pages to scrape per product (0-10). Each page adds ~2-3 seconds per product. Set to 0 to skip comments entirely (fastest). Only applies when 'Get Product Details' is enabled.",
"default": 0
}
}
},
"runsResponseSchema": {
"type": "object",
"properties": {
"data": {
"type": "object",
"properties": {
"id": {
"type": "string"
},
"actId": {
"type": "string"
},
"userId": {
"type": "string"
},
"startedAt": {
"type": "string",
"format": "date-time",
"example": "2025-01-08T00:00:00.000Z"
},
"finishedAt": {
"type": "string",
"format": "date-time",
"example": "2025-01-08T00:00:00.000Z"
},
"status": {
"type": "string",
"example": "READY"
},
"meta": {
"type": "object",
"properties": {
"origin": {
"type": "string",
"example": "API"
},
"userAgent": {
"type": "string"
}
}
},
"stats": {
"type": "object",
"properties": {
"inputBodyLen": {
"type": "integer",
"example": 2000
},
"rebootCount": {
"type": "integer",
"example": 0
},
"restartCount": {
"type": "integer",
"example": 0
},
"resurrectCount": {
"type": "integer",
"example": 0
},
"computeUnits": {
"type": "integer",
"example": 0
}
}
},
"options": {
"type": "object",
"properties": {
"build": {
"type": "string",
"example": "latest"
},
"timeoutSecs": {
"type": "integer",
"example": 300
},
"memoryMbytes": {
"type": "integer",
"example": 1024
},
"diskMbytes": {
"type": "integer",
"example": 2048
}
}
},
"buildId": {
"type": "string"
},
"defaultKeyValueStoreId": {
"type": "string"
},
"defaultDatasetId": {
"type": "string"
},
"defaultRequestQueueId": {
"type": "string"
},
"buildNumber": {
"type": "string",
"example": "1.0.0"
},
"containerUrl": {
"type": "string"
},
"usage": {
"type": "object",
"properties": {
"ACTOR_COMPUTE_UNITS": {
"type": "integer",
"example": 0
},
"DATASET_READS": {
"type": "integer",
"example": 0
},
"DATASET_WRITES": {
"type": "integer",
"example": 0
},
"KEY_VALUE_STORE_READS": {
"type": "integer",
"example": 0
},
"KEY_VALUE_STORE_WRITES": {
"type": "integer",
"example": 1
},
"KEY_VALUE_STORE_LISTS": {
"type": "integer",
"example": 0
},
"REQUEST_QUEUE_READS": {
"type": "integer",
"example": 0
},
"REQUEST_QUEUE_WRITES": {
"type": "integer",
"example": 0
},
"DATA_TRANSFER_INTERNAL_GBYTES": {
"type": "integer",
"example": 0
},
"DATA_TRANSFER_EXTERNAL_GBYTES": {
"type": "integer",
"example": 0
},
"PROXY_RESIDENTIAL_TRANSFER_GBYTES": {
"type": "integer",
"example": 0
},
"PROXY_SERPS": {
"type": "integer",
"example": 0
}
}
},
"usageTotalUsd": {
"type": "number",
"example": 0.00005
},
"usageUsd": {
"type": "object",
"properties": {
"ACTOR_COMPUTE_UNITS": {
"type": "integer",
"example": 0
},
"DATASET_READS": {
"type": "integer",
"example": 0
},
"DATASET_WRITES": {
"type": "integer",
"example": 0
},
"KEY_VALUE_STORE_READS": {
"type": "integer",
"example": 0
},
"KEY_VALUE_STORE_WRITES": {
"type": "number",
"example": 0.00005
},
"KEY_VALUE_STORE_LISTS": {
"type": "integer",
"example": 0
},
"REQUEST_QUEUE_READS": {
"type": "integer",
"example": 0
},
"REQUEST_QUEUE_WRITES": {
"type": "integer",
"example": 0
},
"DATA_TRANSFER_INTERNAL_GBYTES": {
"type": "integer",
"example": 0
},
"DATA_TRANSFER_EXTERNAL_GBYTES": {
"type": "integer",
"example": 0
},
"PROXY_RESIDENTIAL_TRANSFER_GBYTES": {
"type": "integer",
"example": 0
},
"PROXY_SERPS": {
"type": "integer",
"example": 0
}
}
}
}
}
}
}
}
}
}
```