# Goodreads Scraper (`epctex/goodreads-scraper`) Actor
Scrape goodreads.com for data on millions of books. Crawl book details for images, ISBN, author, description, title, buy links, number of reviews, page number, language, and all other details. You can specify search terms, filters, and much more.
- **URL**: https://apify.com/epctex/goodreads-scraper.md
- **Developed by:** [epctex](https://apify.com/epctex) (community)
- **Categories:** Social media
- **Stats:** 598 total users, 12 monthly users, 99.9% runs succeeded, 24 bookmarks
- **User rating**: 5.00 out of 5 stars
## Pricing
$15.00/month + usage
To use this Actor, you pay a monthly rental fee to the developer. The rent is subtracted from your prepaid usage every month after the free trial period.You also pay for the Apify platform usage, which gets cheaper the higher Apify subscription plan you have.
Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#rental-actors
## What's an Apify Actor?
Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".
## How to integrate an Actor?
If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.
In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md):
```bash
npm install apify-client
```
In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md):
```bash
pip install apify-client
```
In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md):
````bash
# MacOS / Linux
curl -fsSL https://apify.com/install-cli.sh | bash
# Windows
irm https://apify.com/install-cli.ps1 | iex
```bash
In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md).
If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md).
For usage examples, see the [API](#api) section below.
For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).
# README
## Actor - Goodreads Scraper
### Goodreads scraper
Since Goodreads doesn't provide a good and free API, this actor should help you to retrieve data from it.
The Goodreads data scraper supports the following features:
- Search any keyword - You can search any keyword you would like to have and get the results
- Scrape lists - Scrape any list that you'd like to get from Goodreads
- Scrape shelf - You can check the shelves and scrape the information of the newest updates.
- Scrape genres - If you want to get the most read books on a certain category or anything related to the genres, just type the URL.
- Scrape book detail - Scrape very detailed information for each of the books that you'd like to get.
- Scrape reviews of a book - Scrape all of the reviews from a book optionally.
### Bugs, fixes, updates, and changelog
This scraper is under active development. If you have any feature requests you can create an issue from [here](https://github.com/epctex-support/goodreads-scraper/issues).
### Setup & Usage
You can see how this actor works in these videos:
#### Using Search
[](https://www.youtube.com/watch?v=7rpRBlIE--o)
You can check the output of this example [here](https://api.apify.com/v2/datasets/AVTdGvcS2iOjDgAaV/items?clean=true&format=json).
#### Using Start URLs
[](https://www.youtube.com/watch?v=ProePJ_1pwA)
You can check the output of this example [here](https://api.apify.com/v2/datasets/yiZ9wm15WMTdmdH8L/items?clean=true&format=json).
### Input Parameters
The input of this scraper should be JSON containing the list of pages on Goodreads that should be visited. Possible fields are:
- `search`: (Optional) (String) Keyword that you want to search on Goodreads.
- `startUrls`: (Optional) (Array) List of Goodreads URLs. You should only provide a news list, jobs list, or detailed URLs.
- `includeReviews`: (Optional) (Boolean) This will add all the reviews that Goodreads provides inside the book object. Please keep in mind that the time and resources the actor uses will increase proportionally by the number of reviews.
- `endPage`: (Optional) (Number) Final number of page that you want to scrape. Default is `Infinite`. This applies to all `search` requests and `startUrls` individually.
- `maxItems`: (Optional) (Number) You can limit scraped items. This should be useful when you search through the big lists or search results.
- `csvFriendlyOutput`: (Optional) (Boolean) When enabled, outputs data in a CSV-compatible format, with reviews organized on a per-page basis. Default is `false`.
- `proxy`: (Required) (Proxy Object) Proxy configuration.
- `extendOutputFunction`: (Optional) (String) Function that takes a JQuery handle ($) as an argument and returns an object with data.
- `customMapFunction`: (Optional) (String) Function that takes each object's handle as an argument and returns the object with executing the function.
This solution requires the use of **Proxy servers**, either your own proxy servers or you can use [Apify Proxy](https://www.apify.com/docs/proxy).
#### Tip
When you want to scrape over a specific listing URL, just copy and paste the link as one of the **startUrl**.
If you would like to scrape only the first page of a list then put the link for the page and have the `endPage` as 1.
With the last approach that is explained above you can also fetch any interval of pages. If you provide the 5th page of a list and define the `endPage` parameter as 6 then you'll have the 5th and 6th pages only.
#### Compute Unit Consumption
The actor is optimized to run blazing fast and scrape many listings as possible. Therefore, it forefronts all listing detail requests. If the actor doesn't block very often it'll scrape 100 listings in 2 minutes with ~0.03-0.05 compute units.
#### Goodreads Scraper Input example
```json
{
"startUrls": [
"https://www.goodreads.com/search?utf8=%E2%9C%93&query=stephen",
"https://www.goodreads.com/book/show/41865.Twilight",
"https://www.goodreads.com/list/show/99918.Cassie_s_Cover_of_the_Year_2016",
"https://www.goodreads.com/shelf/show/art",
"https://www.goodreads.com/genres/most_read/christian-fiction",
"https://www.goodreads.com/author/list/1221698.Neil_Gaiman"
],
"includeReviews": false,
"csvFriendlyOutput": false,
"proxy": {
"useApifyProxy": true
},
"endPage": 5,
"maxItems": 100
}
````
### During the Run
During the run, the actor will output messages letting you know what is going on. Each message always contains a short label specifying which page from the provided list is currently specified.
When items are loaded from the page, you should see a message about this event with a loaded item count and total item count for each page.
If you provide incorrect input to the actor, it will immediately stop with failure state and output an explanation of what is wrong.
### Goodreads Export
During the run, the actor stores results into a dataset. Each item is a separate item in the dataset.
You can manage the results in any languague (Python, PHP, Node JS/NPM). See the FAQ or our API reference to learn more about getting results from this Goodreads actor.
### Scraped Goodreads Properties
The structure of each item in Goodreads listings looks like this:
#### Book Detail
```json
{
"bookId": 58408830,
"url": "https://www.goodreads.com/book/show/58408830-the-lazarus-men",
"image": "https://images-na.ssl-images-amazon.com/images/S/compressed.photo.goodreads.com/books/1624462365i/58408830.jpg",
"title": "The Lazarus Men by Christian Warren Freed | GoodreadsLoading interface...Loading interface...",
"authorName": "Christian Warren Freed",
"authorUrl": "https://www.goodreads.com/author/show/8137590.Christian_Warren_Freed",
"authorAbout": "
Christian W. Freed was born in Buffalo, N.Y. more years ago than he would like to remember. After spending more than 20 years in the active-duty US Army he has turned his talents to writing. Since retiring, he has gone on to publish over 25 military fantasy and science fiction novels, as well as his memoirs from his time in Iraq and Afghanistan, a children's book, and a pair of how to books focused on indie authors and the decision making process for writing a book and what happens after it is published.
His first published book (Hammers in the Wind) has been the #1 free book on Kindle 4 times and he holds a fancy certificate from the L Ron Hubbard Writers of the Future Contest. Ok, so it was for 4th place in one quarter, but it's still recognition from the largest fiction writing contest in the world. And no, he's not a scientologist.
Passionate about history, he combines his knowledge of the past with modern military tactics to create an engaging, quasi-realistic world for the readers. He graduated from Campbell University with a degree in history and a Masters of Arts degree in Digital Communications from the University of North Carolina at Chapel Hill.
He currently lives outside of Raleigh, N.C. and devotes his time to writing, his family, and their two Bernese Mountain Dogs. If you drive by you might just find him on the porch with a cigar in one hand and a pen in the other. You can find out more about his work by following him on social media:
Facebook: https://www.facebook.com/ChristianFreed
Twitter:
@ChristianWFreed
Instagram: www.instagram.com/christianwarrenfreed/
Website: https://christianwfreed.com/
Join his mailing list for new releases, updates, and upcoming events: http://subscribepage.com/warfighterbooks
Books by Christian Warren Freed
The Forgotten Gods Tales
#1 Dreams of Winter
#2 The Madman on the Rocks
#3 Anguish Once Possessed
#4 Through Darkness Besieged
#5 Under Tattered Banners
#6 A Time For Tyrants
An Hour of Wolves- short story
The Northern Crusade
#1 Hammers in the Wind
#2 Tides of Blood and Steel
#3 A Whisper After Midnight
#4 Empire of Bones
#5 The Madness of Gods and Kings
#6 Evens Gods Must Fall
The Histories of Malweir (all stand alones)
#1 Armies of the Silver Mage
#2 The Dragon Hunters
#3 Beyond the Edge of Dawn
Immortality Shattered
#1 Law of the Heretic
#2 The Bitter War of Always
#3 The Land of Wicked Shadows
# Storm Upon the Dawn
The Children of Never (stand alone)
Where Have all the Elves Gone? (stand alone)
One of Our Elves is Missing
The Lazarus Men (stand alone)
Repercussions: A Lazarus Men Agenda #2
A Long Way From Home: My Time in Iraq and Afghanistan 2002-2006
Coward's Truth: A Novel of the Heart Eternal
Tomorrow's Demise: The Extinction Campaign
Tomorrow's Demise: Salvation
",
"rating": 4.23,
"numberOfRatings": 919,
"numberOfReviews": 35,
"reviews": [
{
"author": "Stjepan Cobets",
"userUrl": "https://www.goodreads.com/user/show/44823204-stjepan-cobets",
"rating": 4,
"html": "My rating 3.6
Book The Lazarus Men by Christian Warren Freed is a solid sci-fi novel set in the future where Humankind has reached the stars. In the book, the author draws us into a world full of conspiracy in which those who have everything they want even more because human greed for power is sometimes too great. In this whirlwind of events, accidentally finds Gerald LaPlante ordinary man who is a worker at the landing station on Earth. He is witness to the murder that has happened and after that, he has to save his life. As he will later learn about these events, many secret organizations are involved and they all want him dead because he has something they are looking for. The story varies from excellent to good, and sometimes stories, where should expand, is too little developed, but all in all a good story. I am convinced that all fans of sci-fi will be satisfied.",
"text": "My rating 3.6Book The Lazarus Men by Christian Warren Freed is a solid sci-fi novel set in the future where Humankind has reached the stars. In the book, the author draws us into a world full of conspiracy in which those who have everything they want even more because human greed for power is sometimes too great. In this whirlwind of events, accidentally finds Gerald LaPlante ordinary man who is a worker at the landing station on Earth. He is witness to the murder that has happened and after that, he has to save his life. As he will later learn about these events, many secret organizations are involved and they all want him dead because he has something they are looking for. The story varies from excellent to good, and sometimes stories, where should expand, is too little developed, but all in all a good story. I am convinced that all fans of sci-fi will be satisfied.",
"numOfLikes": 40,
"date": "July 20, 2018",
"shelves": [
"indie",
"science-fiction"
]
},
],
"description": "It is the 23rd century. Humankind has reached the stars, building a tentative empire across a score of worlds. Earth's central government rules weakly as several worlds continue their efforts toward independence. Shadow organizations hide in the midst of the political infighting. Their manifestations of power and influence are beholden only to the highest bidder. The most powerful/insidious/secret of these, The Lazarus Men, has existed for decades, always working outside of morality's constraints. Led by the enigmatic Mr. Shine, their agents are hand selected from the worst humanity has to offer and available for the right price. Gerald LaPlant lives an ordinary life on Old Earth. That life is thrown into turmoil on the night he stumbles upon the murder of what appears to be a street thief. Fleeing into the night, Gerald finds himself caught in a war between the Lazarus Men and Roland McMasters, an extremely powerful man dissatisfied with the current regime and with designs on ruling his own empire.",
"buyLinks": [
"https://www.amazon.com/gp/product/B09L2CPT2L/ref=x_gr_bb_kindle?caller=Goodreads&tag=x_gr_bb_kindle-20",
"http://www.amazon.com/gp/product/1365417077/ref=x_gr_bb_amazon?ie=UTF8&tag=x_gr_bb_amazon-20&linkCode=as2&camp=1789&creative=9325&creativeASIN=1365417077&SubscriptionId=1MGPYB6YW3HWK55XCGG2",
"https://www.amazon.com/s?k=The+Lazarus+Men&i=audible&ref=x_gr_w_bb_audible-20&tag=x_gr_w_bb_audible-20",
"https://www.barnesandnoble.com/w/?ean=9781365417078",
"http://affiliates.abebooks.com/c/64613/77416/2029?u=http%3A%2F%2Fwww.abebooks.com%2Fservlet%2FSearchResults%3Fisbn%3D1365417077",
"https://click.linksynergy.com/fs-bin/click?id=GwEz7vxblVU&subid=&offerid=361251.1&type=10&tmpid=9309&u1=x_gr_w_bb&RD_PARM1=https%3A%2F%2Fwww.kobo.com%2Fus%2Fen%2Fsearch%3FQuery%3D9781365417078",
"https://geo.itunes.apple.com/us/book/isbn9781365417078?at=11lvdC&mt=11&ls=1",
"https://play.google.com/store/search?q=9781365417078&c=books&PCamrefID=bookpage&PAffiliateID=10lHMS",
"http://www.bookdepository.com/search?searchTerm=1365417077&search=Find+book&a_aid=goodreads",
"http://click.linksynergy.com/fs-bin/click?id=GwEz7vxblVU&subid=&offerid=189673.1&type=10&tmpid=939&&u1=x_gr_w_bb&RD_PARM1=http%3A%2F%2Fwww.alibris.com%2Fbooksearch%3Fkeyword%3D1365417077",
"https://www.chapters.indigo.ca/en-ca/home/search/?keywords=9781365417078",
"http://www.tkqlhce.com/ob117vpyvpxCFFFKMHLCEDHLGHKECEFMIGDFFHJDDD?url=http%3A%2F%2Fwww.betterworldbooks.com%2FThe+Lazarus+Men-H0.aspx%3FSearchTerm%3D1365417077",
"https://www.indiebound.org/book/9781365417078",
"https://prf.hn/click/camref:1101ljNE7/pubref:1365417077/destination:https://www.thriftbooks.com/browse/?b.search=1365417077"
],
"bookFormat": "Paperback",
"numberOfPages": 242,
"firstPublishedDate": "January 1, 1970",
"publishedDate": "June 1, 2021",
"publishedBy": "Warfighter Books",
"Original Title": null,
"ISBN": "1365417077",
"Edition Language": null,
"Series": null,
"Characters": [],
"Literary Awards": []
}
```
### Contact
Please visit us through [epctex.com](https://epctex.com) to see all the products that are available for you. If you are looking for any custom integration or so, please reach out to us through the chat box in [epctex.com](https://epctex.com). In need of support? is at your service.
# Actor input Schema
## `search` (type: `string`):
The keyword that you can search on Goodreads
## `includeReviews` (type: `boolean`):
Including reviews on the book object. This will increase the resource consumption proportional to number of requests.
## `startUrls` (type: `array`):
URLs to start with. It should be list or detail URL
## `maxItems` (type: `integer`):
Maximum number of items that you want as output. Default is all
## `endPage` (type: `integer`):
The page number that you want to end with. By default there is no end page. This is applies to all search request and startUrls individually.
## `csvFriendlyOutput` (type: `boolean`):
When enabled, outputs data in a CSV-compatible format, with reviews organized on a per-page basis.
## `extendOutputFunction` (type: `string`):
Function that takes a JQuery handle ($) as argument and returns data that will be merged with the default output
## `customMapFunction` (type: `string`):
Function that takes each of the objects as argument and returns data that will be mapped by the function itself.
## `proxy` (type: `object`):
Select proxies to be used by your crawler.
## Actor input object example
```json
{
"search": "harry potter",
"includeReviews": false,
"startUrls": [
"https://www.goodreads.com/search?q=game+of+thrones&qid=",
"https://www.goodreads.com/book/show/59576065-acceptance",
"https://www.goodreads.com/list/show/1362.Best_History_Books_",
"https://www.goodreads.com/shelf/show/fiction",
"https://www.goodreads.com/genres/business",
"https://www.goodreads.com/author/list/1221698.Neil_Gaiman"
],
"maxItems": 1,
"endPage": 1,
"csvFriendlyOutput": false,
"extendOutputFunction": "($) => { return {} }",
"customMapFunction": "(object) => { return {...object} }",
"proxy": {
"useApifyProxy": true,
"apifyProxyGroups": [
"RESIDENTIAL"
]
}
}
```
# API
You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.
## JavaScript example
```javascript
import { ApifyClient } from 'apify-client';
// Initialize the ApifyClient with your Apify API token
// Replace the '' with your token
const client = new ApifyClient({
token: '',
});
// Prepare Actor input
const input = {
"search": "harry potter",
"startUrls": [
"https://www.goodreads.com/search?q=game+of+thrones&qid=",
"https://www.goodreads.com/book/show/59576065-acceptance",
"https://www.goodreads.com/list/show/1362.Best_History_Books_",
"https://www.goodreads.com/shelf/show/fiction",
"https://www.goodreads.com/genres/business",
"https://www.goodreads.com/author/list/1221698.Neil_Gaiman"
],
"maxItems": 1,
"endPage": 1,
"extendOutputFunction": ($) => { return {} },
"customMapFunction": (object) => { return {...object} },
"proxy": {
"useApifyProxy": true,
"apifyProxyGroups": [
"RESIDENTIAL"
]
}
};
// Run the Actor and wait for it to finish
const run = await client.actor("epctex/goodreads-scraper").call(input);
// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
console.dir(item);
});
// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs
```
## Python example
```python
from apify_client import ApifyClient
# Initialize the ApifyClient with your Apify API token
# Replace '' with your token.
client = ApifyClient("")
# Prepare the Actor input
run_input = {
"search": "harry potter",
"startUrls": [
"https://www.goodreads.com/search?q=game+of+thrones&qid=",
"https://www.goodreads.com/book/show/59576065-acceptance",
"https://www.goodreads.com/list/show/1362.Best_History_Books_",
"https://www.goodreads.com/shelf/show/fiction",
"https://www.goodreads.com/genres/business",
"https://www.goodreads.com/author/list/1221698.Neil_Gaiman",
],
"maxItems": 1,
"endPage": 1,
"extendOutputFunction": "($) => { return {} }",
"customMapFunction": "(object) => { return {...object} }",
"proxy": {
"useApifyProxy": True,
"apifyProxyGroups": ["RESIDENTIAL"],
},
}
# Run the Actor and wait for it to finish
run = client.actor("epctex/goodreads-scraper").call(run_input=run_input)
# Fetch and print Actor results from the run's dataset (if there are any)
print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
print(item)
# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start
```
## CLI example
```bash
echo '{
"search": "harry potter",
"startUrls": [
"https://www.goodreads.com/search?q=game+of+thrones&qid=",
"https://www.goodreads.com/book/show/59576065-acceptance",
"https://www.goodreads.com/list/show/1362.Best_History_Books_",
"https://www.goodreads.com/shelf/show/fiction",
"https://www.goodreads.com/genres/business",
"https://www.goodreads.com/author/list/1221698.Neil_Gaiman"
],
"maxItems": 1,
"endPage": 1,
"extendOutputFunction": "($) => { return {} }",
"customMapFunction": "(object) => { return {...object} }",
"proxy": {
"useApifyProxy": true,
"apifyProxyGroups": [
"RESIDENTIAL"
]
}
}' |
apify call epctex/goodreads-scraper --silent --output-dataset
```
## MCP server setup
```json
{
"mcpServers": {
"apify": {
"command": "npx",
"args": [
"mcp-remote",
"https://mcp.apify.com/?tools=epctex/goodreads-scraper",
"--header",
"Authorization: Bearer "
]
}
}
}
```
## OpenAPI specification
```json
{
"openapi": "3.0.1",
"info": {
"title": "Goodreads Scraper",
"description": "Scrape goodreads.com for data on millions of books. Crawl book details for images, ISBN, author, description, title, buy links, number of reviews, page number, language, and all other details. You can specify search terms, filters, and much more.",
"version": "0.0",
"x-build-id": "hGPGTHLCTI1vlaDjp"
},
"servers": [
{
"url": "https://api.apify.com/v2"
}
],
"paths": {
"/acts/epctex~goodreads-scraper/run-sync-get-dataset-items": {
"post": {
"operationId": "run-sync-get-dataset-items-epctex-goodreads-scraper",
"x-openai-isConsequential": false,
"summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.",
"tags": [
"Run Actor"
],
"requestBody": {
"required": true,
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/inputSchema"
}
}
}
},
"parameters": [
{
"name": "token",
"in": "query",
"required": true,
"schema": {
"type": "string"
},
"description": "Enter your Apify token here"
}
],
"responses": {
"200": {
"description": "OK"
}
}
}
},
"/acts/epctex~goodreads-scraper/runs": {
"post": {
"operationId": "runs-sync-epctex-goodreads-scraper",
"x-openai-isConsequential": false,
"summary": "Executes an Actor and returns information about the initiated run in response.",
"tags": [
"Run Actor"
],
"requestBody": {
"required": true,
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/inputSchema"
}
}
}
},
"parameters": [
{
"name": "token",
"in": "query",
"required": true,
"schema": {
"type": "string"
},
"description": "Enter your Apify token here"
}
],
"responses": {
"200": {
"description": "OK",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/runsResponseSchema"
}
}
}
}
}
}
},
"/acts/epctex~goodreads-scraper/run-sync": {
"post": {
"operationId": "run-sync-epctex-goodreads-scraper",
"x-openai-isConsequential": false,
"summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.",
"tags": [
"Run Actor"
],
"requestBody": {
"required": true,
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/inputSchema"
}
}
}
},
"parameters": [
{
"name": "token",
"in": "query",
"required": true,
"schema": {
"type": "string"
},
"description": "Enter your Apify token here"
}
],
"responses": {
"200": {
"description": "OK"
}
}
}
}
},
"components": {
"schemas": {
"inputSchema": {
"type": "object",
"required": [
"proxy"
],
"properties": {
"search": {
"title": "Search keyword",
"type": "string",
"description": "The keyword that you can search on Goodreads"
},
"includeReviews": {
"title": "Include Reviews",
"type": "boolean",
"description": "Including reviews on the book object. This will increase the resource consumption proportional to number of requests.",
"default": false
},
"startUrls": {
"title": "Start URLs",
"type": "array",
"description": "URLs to start with. It should be list or detail URL",
"items": {
"type": "string"
}
},
"maxItems": {
"title": "Maximum number of listing items",
"type": "integer",
"description": "Maximum number of items that you want as output. Default is all"
},
"endPage": {
"title": "List end page",
"type": "integer",
"description": "The page number that you want to end with. By default there is no end page. This is applies to all search request and startUrls individually."
},
"csvFriendlyOutput": {
"title": "CSV friendly output",
"type": "boolean",
"description": "When enabled, outputs data in a CSV-compatible format, with reviews organized on a per-page basis.",
"default": false
},
"extendOutputFunction": {
"title": "Extend output function",
"type": "string",
"description": "Function that takes a JQuery handle ($) as argument and returns data that will be merged with the default output"
},
"customMapFunction": {
"title": "Custom map function",
"type": "string",
"description": "Function that takes each of the objects as argument and returns data that will be mapped by the function itself."
},
"proxy": {
"title": "Proxy configuration",
"type": "object",
"description": "Select proxies to be used by your crawler."
}
}
},
"runsResponseSchema": {
"type": "object",
"properties": {
"data": {
"type": "object",
"properties": {
"id": {
"type": "string"
},
"actId": {
"type": "string"
},
"userId": {
"type": "string"
},
"startedAt": {
"type": "string",
"format": "date-time",
"example": "2025-01-08T00:00:00.000Z"
},
"finishedAt": {
"type": "string",
"format": "date-time",
"example": "2025-01-08T00:00:00.000Z"
},
"status": {
"type": "string",
"example": "READY"
},
"meta": {
"type": "object",
"properties": {
"origin": {
"type": "string",
"example": "API"
},
"userAgent": {
"type": "string"
}
}
},
"stats": {
"type": "object",
"properties": {
"inputBodyLen": {
"type": "integer",
"example": 2000
},
"rebootCount": {
"type": "integer",
"example": 0
},
"restartCount": {
"type": "integer",
"example": 0
},
"resurrectCount": {
"type": "integer",
"example": 0
},
"computeUnits": {
"type": "integer",
"example": 0
}
}
},
"options": {
"type": "object",
"properties": {
"build": {
"type": "string",
"example": "latest"
},
"timeoutSecs": {
"type": "integer",
"example": 300
},
"memoryMbytes": {
"type": "integer",
"example": 1024
},
"diskMbytes": {
"type": "integer",
"example": 2048
}
}
},
"buildId": {
"type": "string"
},
"defaultKeyValueStoreId": {
"type": "string"
},
"defaultDatasetId": {
"type": "string"
},
"defaultRequestQueueId": {
"type": "string"
},
"buildNumber": {
"type": "string",
"example": "1.0.0"
},
"containerUrl": {
"type": "string"
},
"usage": {
"type": "object",
"properties": {
"ACTOR_COMPUTE_UNITS": {
"type": "integer",
"example": 0
},
"DATASET_READS": {
"type": "integer",
"example": 0
},
"DATASET_WRITES": {
"type": "integer",
"example": 0
},
"KEY_VALUE_STORE_READS": {
"type": "integer",
"example": 0
},
"KEY_VALUE_STORE_WRITES": {
"type": "integer",
"example": 1
},
"KEY_VALUE_STORE_LISTS": {
"type": "integer",
"example": 0
},
"REQUEST_QUEUE_READS": {
"type": "integer",
"example": 0
},
"REQUEST_QUEUE_WRITES": {
"type": "integer",
"example": 0
},
"DATA_TRANSFER_INTERNAL_GBYTES": {
"type": "integer",
"example": 0
},
"DATA_TRANSFER_EXTERNAL_GBYTES": {
"type": "integer",
"example": 0
},
"PROXY_RESIDENTIAL_TRANSFER_GBYTES": {
"type": "integer",
"example": 0
},
"PROXY_SERPS": {
"type": "integer",
"example": 0
}
}
},
"usageTotalUsd": {
"type": "number",
"example": 0.00005
},
"usageUsd": {
"type": "object",
"properties": {
"ACTOR_COMPUTE_UNITS": {
"type": "integer",
"example": 0
},
"DATASET_READS": {
"type": "integer",
"example": 0
},
"DATASET_WRITES": {
"type": "integer",
"example": 0
},
"KEY_VALUE_STORE_READS": {
"type": "integer",
"example": 0
},
"KEY_VALUE_STORE_WRITES": {
"type": "number",
"example": 0.00005
},
"KEY_VALUE_STORE_LISTS": {
"type": "integer",
"example": 0
},
"REQUEST_QUEUE_READS": {
"type": "integer",
"example": 0
},
"REQUEST_QUEUE_WRITES": {
"type": "integer",
"example": 0
},
"DATA_TRANSFER_INTERNAL_GBYTES": {
"type": "integer",
"example": 0
},
"DATA_TRANSFER_EXTERNAL_GBYTES": {
"type": "integer",
"example": 0
},
"PROXY_RESIDENTIAL_TRANSFER_GBYTES": {
"type": "integer",
"example": 0
},
"PROXY_SERPS": {
"type": "integer",
"example": 0
}
}
}
}
}
}
}
}
}
}
```