# arXiv Papers Scraper: Search, Metadata, Authors, Full Abstracts (`reapx/arxiv-papers-scraper`) Actor

arXiv papers scraper to search and scrape academic research papers from arXiv. Extract paper titles, full abstracts, author lists, subject categories, DOIs, PDF links and publication dates by keyword, author or category. Ideal for literature reviews, RAG pipelines and LLM training datasets.

- **URL**: https://apify.com/reapx/arxiv-papers-scraper.md
- **Developed by:** [Tarek Etman](https://apify.com/reapx) (community)
- **Categories:** Other, Developer tools, Education
- **Stats:** 2 total users, 1 monthly users, 0.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $1.75 / 1,000 paper returneds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

![reapX — public sources in, addressable records out](https://reapx.dev/reapx.gif)

## arXiv Papers Scraper · Search, Metadata, Authors & Full Abstracts

An **arXiv papers scraper** and **arXiv paper scraper** to search and scrape academic research papers from arXiv. Export matching papers as structured rows: full abstract, complete author list, subject categories, submission and revision dates, DOI, journal reference and direct PDF link. It reads the official arXiv API, so there is no login, no API key, no proxy and no browser — just clean academic paper metadata for literature reviews, RAG pipelines, vector stores, and LLM training datasets.

> Maintained by **reapX**. Every row cites the Apify run that produced it — nothing is
> inferred, modelled or filled in, and a field absent from the source is absent from the row.
> The extracted archive for this source is browsable at
> [reapx.dev/data/arxiv-papers-scraper/](https://reapx.dev/data/arxiv-papers-scraper/) and mirrored as an open dataset on
> [Hugging Face](https://huggingface.co/datasets/reapxdev/arxiv-papers-scraper) and
> [Kaggle](https://www.kaggle.com/datasets/reapxdev/arxiv-papers-scraper). Questions: reapxdev@proton.me

### arXiv paper scraper features & metadata

What this arXiv scraper extracts

Every row carries 24 fields, all camelCase, all documented in the output schema:

| Field | What it holds |
|---|---|
| `arxivId`, `arxivBaseId`, `version` | Versioned ID (`2504.12342v2`), stable join key, revision number |
| `title` | Paper title, whitespace normalised |
| `abstract`, `abstractWordCount` | The **complete** abstract, never truncated |
| `authors`, `authorCount`, `firstAuthor` | Full author list in arXiv's own order |
| `authorAffiliations` | Institutions, where the submitter declared them |
| `primaryCategory`, `categories`, `isCrossListed` | Where the paper sits in the arXiv taxonomy |
| `publishedAt`, `publishedDate`, `updatedAt` | First submission and latest revision |
| `doi`, `doiUrl`, `journalRef` | Publication status once a preprint reaches a journal |
| `comment` | The submitter's note - page count, conference acceptance, code links |
| `absUrl`, `pdfUrl` | Landing page and direct PDF |
| `sourceQuery`, `scrapedAt` | The exact arXiv query and collection time, so any result set is reproducible |

Five named views ship with the dataset - **Overview**, **Full abstracts**, **Authors and affiliations**, **Publication status** and **Subject categories** - so you can export just the columns a given job needs.

### How to search arXiv by subject category

Put arXiv subject class codes into **arXiv subject categories**: `cs.AI`, `cs.LG`, `cs.CL`, `cs.CV`, `cs.CR`, `cs.DB`, `cs.SE`, `cs.RO`, `stat.ML`, `math.ST`, `q-bio.QM`, `econ.EM`, `physics.comp-ph`, `eess.SP`, `quant-ph` - any code in the arXiv taxonomy works.

- **Any of them (OR)** gives you the union - a broad feed for one field.
- **All of them (AND)** gives you only papers filed under every code at once - the fastest way to find genuinely cross-disciplinary work, for example `cs.CV` **and** `cs.CL` for vision-language research.

### How to get new arXiv papers every day

Set **Rolling window: last N days** to `1` and schedule the Actor daily. The window is recomputed on every run, so the same task keeps returning only what is new. `7` gives you a weekly reading list. Combine it with **Date to filter on → Last updated** to catch revised versions rather than first submissions.

### How to find papers by author or institution

**Authors** takes full names - `Yann LeCun`, `Yoshua Bengio` - and keeps any paper with at least one of them. **Affiliations or institutions** matches an institution name anywhere in the arXiv record. arXiv does not index affiliation as its own field, so this behaves as a strong full-text filter across the affiliation tag, comments, title and abstract, not as a registry lookup.

### How to separate cross-listed papers from a category's own output

arXiv papers are often filed under several subject classes. Two switches split them:

- **Only papers whose PRIMARY category matches** - the native output of a subject class, the way its listing page reads.
- **Only cross-listed papers** - work imported into your categories from somewhere else. Point it at `physics.comp-ph` or `q-bio.QM` to see exactly what machine learning those fields are absorbing.

### Advanced arXiv API query syntax

**Advanced: raw arXiv query string** passes your text straight to the arXiv API's `search_query`, overriding the form fields. Full boolean grammar is available:

```
cat:cs.LG AND abs:"graph neural network" ANDNOT cat:cs.CV
(cat:stat.ME OR cat:math.ST) AND ti:"causal inference"
au:"Hinton_G" AND submittedDate:[202601010000 TO 202612312359]
```

Prefixes: `ti:` title, `abs:` abstract, `au:` author, `co:` comment, `jr:` journal reference, `cat:` category, `all:` everything.

### Speed and rate limits

HTTP only, no headless browser. arXiv asks API clients to leave at least three seconds between requests, and this Actor enforces that floor - you cannot configure it lower. Papers arrive 200 per request, so 1,000 papers takes about five requests and under a minute of waiting. Requests that hit HTTP 429 or a 5xx are retried five times with exponential backoff, honouring `Retry-After`.

### Input configuration

Every field is optional. The prefilled values below are what runs if you press Start without changing anything.

#### What to search for

Combine any of these filters. Everything you fill in is combined with AND, so more filters means fewer, sharper results.

| Field | Type | Accepts | What it does |
| --- | --- | --- | --- |
| `searchQuery` | string | free text | Words or a phrase to search for. Multi-word input is matched as an exact phrase. Leave empty to browse a whole subject category by date. Prefilled with `"large language models"`. |
| `searchField` | string | one of 6: `all`, `ti`, `abs`, `au`, ... | Which part of the arXiv record the keyword above is matched against. 'Everything' searches title, abstract, authors, comments and journal reference at once. |
| `categories` | list | one value per line | arXiv subject class codes, e.g. cs.AI (artificial intelligence), cs.LG (machine learning), cs.CL (computation and language), cs.CV (computer vision), cs.CR (cryptography and security), cs.DB (databases), cs.SE (software engineering), cs.RO (robotics), stat.ML, math.ST, q-bio.QM, econ.EM, physics.comp-ph, eess.SP, quant-ph, astro-ph.CO. Any code from the arXiv taxonomy works. Leave empty to search all of arXiv. Prefilled with `["cs.AI"]`. |
| `categoryMatch` | string | `any`, `all` | 'Any' returns a paper filed under at least one of the categories. 'All' returns only papers filed under every category at once - useful for finding genuinely cross-disciplinary work. |
| `authors` | list | one value per line | Author names to match. A paper is kept if it has at least one of these authors. Full names work best. |
| `affiliations` | list | one value per line | Institution names matched anywhere in the arXiv record (author affiliation tag, comments, title or abstract). A paper is kept if it matches at least one. arXiv does not index affiliation as a separate field, so treat this as a strong text filter rather than a registry lookup. |
| `arxivIds` | list | one value per line | Fetch these exact papers instead of searching. Accepts new-style IDs (2504.12342), versioned IDs (1706.03762v7) and old-style IDs (cs/9510101). When set, all search filters above are ignored. |

#### Cross-listing filter

arXiv papers can be filed under several subject classes at once. These two switches let you isolate the native feed of a category or, the opposite, only its imported work.

| Field | Type | Accepts | What it does |
| --- | --- | --- | --- |
| `primaryCategoryOnly` | true/false | checkbox | Drop papers that were merely cross-listed into your categories. Gives you the native output of a subject class, the way its listing page reads. |
| `crossListedOnly` | true/false | checkbox | Keep only papers whose primary category is something else - work imported into your categories from another field. The fastest way to see what physics or biology is borrowing from machine learning. |

#### Date window

Use the rolling window for a feed that stays fresh on a schedule, or fixed dates for a one-off archive slice. Leave all three empty for no date limit.

| Field | Type | Accepts | What it does |
| --- | --- | --- | --- |
| `dateField` | string | `submittedDate`, `lastUpdatedDate` | Filter on the day the paper first appeared, or on the day its latest version was posted. |
| `lastDays` | integer | 1 to 3650 | Return only papers from the last N days, recomputed on every run. Use this for scheduled monitoring - 1 for a daily digest, 7 for a weekly review. Overrides the fixed dates below. |
| `dateFrom` | string | `YYYY-MM-DD` | Earliest date to include, inclusive. Ignored when a rolling window is set. |
| `dateTo` | string | `YYYY-MM-DD` | Latest date to include, inclusive. Ignored when a rolling window is set. |

#### Sorting and how many

How results are ordered and where to stop.

| Field | Type | Accepts | What it does |
| --- | --- | --- | --- |
| `sortBy` | string | `submittedDate`, `lastUpdatedDate`, `relevance` | Newest first is the right choice for monitoring. Relevance is the right choice for a one-off literature search. |
| `sortOrder` | string | `descending`, `ascending` | Descending puts the newest (or most relevant) paper first. |
| `maxPapers` | integer | 1 to 30000 | Hard stop on how many papers are returned and charged. Each paper returned costs $0.0025. 50 papers finishes in well under a minute. Prefilled with `50`. |
| `dedupeByPaper` | true/false | checkbox | Collapse v1, v2, v3 of the same paper into a single row holding the latest version seen. Turn off if you want every revision separately. |

#### Advanced

Only needed for query shapes the fields above cannot express, or to slow the scraper down further.

| Field | Type | Accepts | What it does |
| --- | --- | --- | --- |
| `rawSearchQuery` | string | free text | Full arXiv API search\_query syntax, passed straight through. Overrides every filter in sections 1 to 3. Example: cat:cs.LG AND abs:"graph neural network" ANDNOT cat:cs.CV |
| `requestDelaySecs` | integer | 3 to 60 | arXiv asks API clients to wait at least 3 seconds between requests. 3 is the minimum this Actor allows; raise it if you are running several jobs at once. |

### Pricing

Pay per event. The primary event is **Paper returned** (`paper-returned`), charged at **$0.0025 per event** on the free plan, falling to $0.00125 on the highest tier.

Charged once per arXiv paper written to the dataset, with its full abstract, complete author list, subject categories and links. Runs that are blocked, rejected by arXiv, or that match zero papers are never charged.

Other charged events: `apify-actor-start` (Actor Start).

Minimum charge cap per run: $0.05.

Tiered discounts apply automatically on every paid Apify plan. Platform usage is absorbed by the Actor, so the per-event price is the whole price.

### Usage examples

#### The prefilled run, which is what Start does with nothing changed

```json
{
  "searchQuery": "large language models",
  "searchField": "all",
  "categories": [
    "cs.AI"
  ],
  "categoryMatch": "any",
  "primaryCategoryOnly": false,
  "crossListedOnly": false,
  "dateField": "submittedDate",
  "sortBy": "submittedDate",
  "sortOrder": "descending",
  "maxPapers": 50,
  "dedupeByPaper": true,
  "requestDelaySecs": 3
}
```

#### Filter by keyword or phrase

```json
{
  "searchQuery": "retrieval augmented generation",
  "searchField": "all",
  "categories": [
    "cs.AI"
  ],
  "categoryMatch": "any",
  "primaryCategoryOnly": false,
  "crossListedOnly": false,
  "dateField": "submittedDate",
  "sortBy": "submittedDate",
  "sortOrder": "descending",
  "maxPapers": 50,
  "dedupeByPaper": true,
  "requestDelaySecs": 3
}
```

#### Filter by search this field

```json
{
  "searchQuery": "large language models",
  "searchField": "abs",
  "categories": [
    "cs.AI"
  ],
  "categoryMatch": "any",
  "primaryCategoryOnly": false,
  "crossListedOnly": false,
  "dateField": "submittedDate",
  "sortBy": "submittedDate",
  "sortOrder": "descending",
  "maxPapers": 50,
  "dedupeByPaper": true,
  "requestDelaySecs": 3
}
```

#### Filter by arxiv subject categories

```json
{
  "searchQuery": "large language models",
  "searchField": "all",
  "categories": [
    "cs.LG",
    "stat.ML"
  ],
  "categoryMatch": "any",
  "primaryCategoryOnly": false,
  "crossListedOnly": false,
  "dateField": "submittedDate",
  "sortBy": "submittedDate",
  "sortOrder": "descending",
  "maxPapers": 50,
  "dedupeByPaper": true,
  "requestDelaySecs": 3
}
```

#### How to combine categories

```json
{
  "searchQuery": "large language models",
  "searchField": "all",
  "categories": [
    "cs.AI"
  ],
  "categoryMatch": "all",
  "primaryCategoryOnly": false,
  "crossListedOnly": false,
  "dateField": "submittedDate",
  "sortBy": "submittedDate",
  "sortOrder": "descending",
  "maxPapers": 50,
  "dedupeByPaper": true,
  "requestDelaySecs": 3
}
```

### Output example

One row, exactly as the actor wrote it to the dataset:

```json
{
  "arxivId": "2607.28617v1",
  "arxivBaseId": "2607.28617",
  "version": 1,
  "title": "AISPA: User-Centric System Prompt Auditing for Large Language Model Applications",
  "abstract": "System prompts are instructions configured by developers to govern the behaviors of foundation models in AI applications. They are used throughout commercial AI products, but are rarely disclosed to the public or regu...",
  "abstractWordCount": 252,
  "authors": [
    "Xiangning Lin",
    "Shenzhe Zhu",
    "Shu Yang",
    "Zhenyu Zhang",
    "Haoqian Zhang",
    "Yipeng Zhao",
    "Chengxuan Qian",
    "Tianwei Wang",
    "Ziheng Zhang",
    "Zhenlong Yuan",
    "Dingcheng Wang",
    "Juncheng Wu",
    "Yuan Si",
    "Jiaxin Liu",
    "Baolong Bi",
    "Robert Mahari",
    "Tobin South",
    "Dazza Greenwood",
    "Zexue He",
    "Rishi Bommasani",
    "Sophia Kazinnik",
    "Andreas Haupt",
    "Samuele Marro",
    "Erik Brynjolfsson",
    "Alex Pentland",
    "Jiaxin Pei"
  ],
  "authorCount": 26,
  "firstAuthor": "Xiangning Lin",
  "authorAffiliations": [],
  "primaryCategory": "cs.AI",
  "categories": [
    "cs.AI",
    "cs.CL",
    "cs.CY",
    "cs.HC"
  ],
  "isCrossListed": true,
  "publishedAt": "2026-07-30T17:58:58Z",
  "publishedDate": "2026-07-30",
  "updatedAt": "2026-07-30T17:58:58Z",
  "doi": null,
  "doiUrl": null
}
```

That row carries 24 fields in total; 18 are shown.

#### Fields on every row

| Field | Type | What it is |
| --- | --- | --- |
| `arxivId` | string | The arXiv identifier including the version suffix, exactly as arXiv returns it. Unique per revision. |
| `arxivBaseId` | string | The arXiv identifier with the version suffix stripped. Stable across revisions - use this as the join key. |
| `version` | integer | Which revision of the paper this row describes. 1 is the original submission. |
| `title` | string | Full paper title with arXiv's line breaks and repeated spaces normalised to single spaces. |
| `abstract` | string | The complete abstract, never truncated, with line breaks normalised to single spaces. |
| `abstractWordCount` | integer | Number of whitespace-separated words in the abstract. Useful for filtering out stub records. |
| `authors` | list | Every author name, in the order arXiv lists them. |
| `authorCount` | integer | How many authors are on the paper. Large collaborations can run to hundreds. |
| `firstAuthor` | string | The first listed author, extracted for easy grouping and sorting. |
| `authorAffiliations` | list | Institutions declared in arXiv's affiliation tag. Populated on roughly one paper in forty - arXiv does not require submitters to fill it in, so an empty list means 'not declared', not 'independent'. |
| `primaryCategory` | string | The subject class the authors filed the paper under first. This is the category whose listing page the paper natively appears on. |
| `categories` | list | Every subject class the paper is filed under, primary first, including cross-lists. |
| `isCrossListed` | true/false | True when the paper carries more than one subject class, meaning it was imported into at least one field other than its own. |
| `publishedAt` | string | UTC timestamp of the original v1 submission, ISO 8601. |
| `publishedDate` | string | Calendar date of the original submission, YYYY-MM-DD, for grouping by day. |
| `updatedAt` | string | UTC timestamp of the most recent revision, ISO 8601. Equals publishedAt when the paper was never revised. |
| `doi` | string | Publisher DOI when the authors have registered one. Null while a paper is still an unpublished preprint, which is the large majority of recent submissions. |
| `doiUrl` | string | Resolvable doi.org URL for the published version. Null when there is no DOI. |
| `journalRef` | string | Free-text citation of the peer-reviewed venue, as supplied by the authors. Null for unpublished preprints. |
| `comment` | string | The submitter's own note - page count, figure count, conference acceptance, code links. Present on about half of all papers, null otherwise. |
| `absUrl` | string | Human-readable arXiv landing page for the paper. |
| `pdfUrl` | string | Direct link to the full-text PDF on arXiv. |
| `sourceQuery` | string | The exact arXiv API search\_query string that produced this row, so a result set can always be reproduced or audited. |
| `scrapedAt` | string | UTC timestamp of the moment this row was collected, ISO 8601. |

5 named dataset views ship with it: **Overview**, **Full abstracts**, **Authors and affiliations**, **Publication status**, **Subject categories**. They drive the Output tab in Console and the Output block on the Actor's `.md` page.

### FAQ

#### Do I need an arXiv API key or account?

No. The arXiv API is completely open. There is no key, no login, no rate-limit token and no proxy required.

#### Does it return the full abstract or a truncated one?

The full abstract, every time. `abstractWordCount` is included so you can spot short or malformed records at a glance.

#### Can I get papers published in a specific date range?

Yes. Use **From date** and **To date** for a fixed archive slice, or **Rolling window: last N days** for a feed that stays current. Choose whether the window applies to first submission or to the latest revision.

#### How many papers can one run return?

Up to 30,000 - the arXiv API's own paging ceiling for a single query. For a larger harvest, split the work by subject category or by date window across several runs.

#### Why is `doi` empty on most rows?

Because most arXiv papers are preprints that have not been through a journal yet. `doi`, `doiUrl` and `journalRef` fill in only once the authors come back and register the published version. On a sample of 650 recent papers taken while building this Actor, about 2% carried a DOI and 52% carried a `comment`; on older, settled papers the DOI rate is far higher. Filter on `doi != null` in the **Publication status** view to see which preprints made it into a journal.

#### Why is `authorAffiliations` usually empty?

arXiv does not require submitters to fill in the affiliation tag, and most do not - expect it on about one paper in forty. An empty list means "not declared", not "independent researcher".

#### Can I fetch specific papers I already know the ID of?

Yes. Put them in **Specific arXiv IDs**. New-style (`2504.12342`), versioned (`1706.03762v7`) and old-style (`cs/9510101`) identifiers all work, as do full `arxiv.org/abs/...` URLs. All other filters are ignored when this is set.

#### Does it deduplicate versions of the same paper?

By default yes - one row per paper, holding the latest version seen. Turn **One row per paper, not per version** off to keep every revision as its own row.

#### Can I chain this into another Actor or an agent?

Yes. The dataset schema documents the title, description, type and an example for every field, so an agent can read the output contract without a sample run. `arxivBaseId` is the stable join key and `sourceQuery` records exactly how each row was found.

***

Unofficial - not affiliated with arXiv. Collects public data only. reapx.
Contact reapxdev@proton.me.

### The full published archive

- **[arxiv papers scraper archive](https://reapx.dev/data/arxiv-papers-scraper/)** — every entity this Actor has observed
- **[All reapx datasets](https://reapx.dev/data/)** — 20 sources, tens of thousands of pages
- **[llms.txt](https://reapx.dev/llms.txt)** · **[feed.json](https://reapx.dev/feed.json)** ·
  **[feed.xml](https://reapx.dev/feed.xml)** — the machine-readable index, for agents

Each archive page carries Dataset JSON-LD, a canonical URL and the identifiers used, so an agent
can resolve an entity without running anything. Nothing on those pages is estimated or modelled.

### 🧪 Example input

A real, runnable configuration — this is an actual input this Actor has run with.

```json
{
  "searchQuery": "large language models",
  "categories": [
    "cs.AI"
  ],
  "maxPapers": 50,
  "searchField": "all",
  "categoryMatch": "any",
  "primaryCategoryOnly": false,
  "crossListedOnly": false,
  "dateField": "submittedDate",
  "sortBy": "submittedDate",
  "sortOrder": "descending",
  "dedupeByPaper": true,
  "requestDelaySecs": 3
}
```

### 📄 Sample output

One real row from a real run of this Actor, unedited.

```json
{
  "arxivId": "2607.29602v1",
  "arxivBaseId": "2607.29602",
  "version": 1,
  "title": "FriendBench: Benchmarking Dyadic Familiarity Inference in Humans and Multimodal Large Language Models",
  "abstract": "Reading a social situation often depends on behavior, not words alone. We introduce FriendBench, a benchmark for inferring whether two people are already familiar or are meeting as strangers, from a 20-second clip of a dyadic ice-breaker conversation. Every pair answers the same type of prompt, so only the manner of interaction can reveal the answer. Across text, audio, and video, we compare 26 models from seven companies against matched human panels over 96 balanced dyads. The best model and the human crowd are statistically indistinguishable on accuracy in every modality, but reach it differently: humans stay balanced across the two answers, while the strongest models lean toward \"stranger\"---a difference in effective prior, not discrimination. Richer channels help both unequally, and only humans gain from visible behavior on top of speech. We release the stimuli, human ratings, and model predictions.",
  "abstractWordCount": 140,
  "authors": [
    "Jeffrey M. Girard",
    "Jason Z. Zheng",
    "Jacqueline R. Vertino",
    "Antony D'Avirro",
    "Benjamin Peloquin"
  ],
  "authorCount": 5,
  "firstAuthor": "Jeffrey M. Girard",
  "authorAffiliations": [],
  "primaryCategory": "cs.CL",
  "categories": [
    "cs.CL",
    "cs.AI",
    "cs.CV",
    "cs.HC"
  ],
  "isCrossListed": true,
  "publishedAt": "2026-07-31T16:33:39Z",
  "publishedDate": "2026-07-31",
  "updatedAt": "2026-07-31T16:33:39Z",
  "doi": null,
  "doiUrl": null,
  "journalRef": null,
  "comment": "15 pages, 3 figures",
  "absUrl": "https://arxiv.org/abs/2607.29602v1",
  "pdfUrl": "https://arxiv.org/pdf/2607.29602v1",
  "sourceQuery": "all:\"large language models\" AND cat:cs.AI",
  "scrapedAt": "2026-08-03T16:29:31Z"
}
```

### How it works

1. You set the filters below, or none at all.
2. The Actor calls the source's own public endpoint and pages through the results.
3. Every row is pushed to the dataset **as it is built**, not buffered to the end — so a
   run that hits its time limit still returns everything it collected up to that point.
4. You are charged per row returned, so the maximum-results field is also your cost cap.

### 💬 Your feedback

Found a bug, or need a field this does not return yet? Open an issue on the Actor's **Issues**
tab, or write to **reapxdev@proton.me**. Bugs get fixed and reasonable field requests get
added.

# Actor input Schema

## `searchQuery` (type: `string`):

Words or a phrase to search for. Multi-word input is matched as an exact phrase. Leave empty to browse a whole subject category by date.

## `searchField` (type: `string`):

Which part of the arXiv record the keyword above is matched against. 'Everything' searches title, abstract, authors, comments and journal reference at once.<br><br>Options: <code>all</code>, <code>ti</code>, <code>abs</code>, <code>au</code>, <code>co</code>, <code>jr</code>.

## `categories` (type: `array`):

arXiv subject class codes, e.g. <code>cs.AI</code> (artificial intelligence), cs.LG (machine learning), cs.CL (computation and language), cs.CV (computer vision), cs.CR (cryptography and security), cs.DB (databases), cs.SE (software engineering), cs.RO (robotics), stat.ML, math.ST, q-bio.QM, econ.EM, physics.comp-ph, eess.SP, quant-ph, astro-ph.CO. Any code from the arXiv taxonomy works. Leave empty to search all of arXiv.

## `categoryMatch` (type: `string`):

'Any' returns a paper filed under at least one of the categories. 'All' returns only papers filed under every category at once - useful for finding genuinely cross-disciplinary work.<br><br>Options: <code>any</code>, <code>all</code>.

## `authors` (type: `array`):

Author names to match. A paper is kept if it has at least one of these authors. Full names work best.<br><br>Leave this field empty to include every option.

## `affiliations` (type: `array`):

Institution names matched anywhere in the arXiv record (author affiliation tag, comments, title or abstract). A paper is kept if it matches at least one. arXiv does not index affiliation as a separate field, so treat this as a strong text filter rather than a registry lookup.<br><br>Leave this field empty to include every option.

## `arxivIds` (type: `array`):

Fetch these exact papers instead of searching. Accepts new-style IDs (2504.12342), versioned IDs (1706.03762v7) and old-style IDs (cs/9510101). When set, all search filters above are ignored.<br><br>Leave this field empty to include every option.

## `primaryCategoryOnly` (type: `boolean`):

Drop papers that were merely cross-listed into your categories. Gives you the native output of a subject class, the way its listing page reads.

## `crossListedOnly` (type: `boolean`):

Keep only papers whose primary category is something else - work imported into your categories from another field. The fastest way to see what physics or biology is borrowing from machine learning.

## `dateField` (type: `string`):

Filter on the day the paper first appeared, or on the day its latest version was posted.<br><br>Options: <code>submittedDate</code>, <code>lastUpdatedDate</code>.

## `lastDays` (type: `integer`):

Return only papers from the last N days, recomputed on every run. Use this for scheduled monitoring - 1 for a daily digest, 7 for a weekly review. Overrides the fixed dates below.<br><br>The higher the number, the longer the run takes and the more you are charged, because you pay per row returned.

## `dateFrom` (type: `string`):

Earliest date to include, inclusive. Ignored when a rolling window is set.

## `dateTo` (type: `string`):

Latest date to include, inclusive. Ignored when a rolling window is set.

## `sortBy` (type: `string`):

Newest first is the right choice for monitoring. Relevance is the right choice for a one-off literature search.<br><br>Options: <code>submittedDate</code>, <code>lastUpdatedDate</code>, <code>relevance</code>.

## `sortOrder` (type: `string`):

Descending puts the newest (or most relevant) paper first.<br><br>Options: <code>descending</code>, <code>ascending</code>.

## `maxPapers` (type: `integer`):

Hard stop on how many papers are returned and charged. Each paper returned costs $0.0025. 50 papers finishes in well under a minute.<br><br>The higher the number, the longer the run takes and the more you are charged, because you pay per row returned.

## `dedupeByPaper` (type: `boolean`):

Collapse v1, v2, v3 of the same paper into a single row holding the latest version seen. Turn off if you want every revision separately.

## `rawSearchQuery` (type: `string`):

Full arXiv API search\_query syntax, passed straight through. Overrides every filter in sections 1 to 3. Example: cat:cs.LG AND abs:"graph neural network" ANDNOT cat:cs.CV<br><br>Leave this field empty to skip this filter.

## `requestDelaySecs` (type: `integer`):

arXiv asks API clients to wait at least 3 seconds between requests. 3 is the minimum this Actor allows; raise it if you are running several jobs at once.

## Actor input object example

```json
{
  "searchQuery": "retrieval augmented generation",
  "searchField": "abs",
  "categories": [
    "cs.LG",
    "stat.ML"
  ],
  "categoryMatch": "all",
  "authors": [
    "Yann LeCun",
    "Yoshua Bengio"
  ],
  "affiliations": [
    "Google DeepMind",
    "MIT CSAIL"
  ],
  "arxivIds": [
    "1706.03762",
    "1207.7214"
  ],
  "primaryCategoryOnly": true,
  "crossListedOnly": true,
  "dateField": "lastUpdatedDate",
  "lastDays": 7,
  "dateFrom": "2026-01-01",
  "dateTo": "2026-06-30",
  "sortBy": "relevance",
  "sortOrder": "ascending",
  "maxPapers": 500,
  "dedupeByPaper": true,
  "rawSearchQuery": "cat:cs.CL AND ti:\"benchmark\" ANDNOT cat:cs.CV",
  "requestDelaySecs": 5
}
```

# Actor output Schema

## `overview` (type: `string`):

One line per paper: what it is, who wrote it, where it is filed and when it landed.

## `abstracts` (type: `string`):

Every abstract in full, with the links needed to read or download the paper.

## `authors` (type: `string`):

Author lists and any declared institutions, for building co-author graphs or institution cohorts.

## `publication` (type: `string`):

Which preprints made it into a journal: DOI, venue and the author's own note.

## `categories` (type: `string`):

Where each paper sits in the arXiv taxonomy, and whether it was cross-listed in from another field.

## `results` (type: `string`):

Every record this run produced, with all fields and no view applied. Field-level titles, types, descriptions and examples are declared in the dataset schema.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "searchQuery": "large language models",
    "categories": [
        "cs.AI"
    ],
    "maxPapers": 50
};

// Run the Actor and wait for it to finish
const run = await client.actor("reapx/arxiv-papers-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "searchQuery": "large language models",
    "categories": ["cs.AI"],
    "maxPapers": 50,
}

# Run the Actor and wait for it to finish
run = client.actor("reapx/arxiv-papers-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "searchQuery": "large language models",
  "categories": [
    "cs.AI"
  ],
  "maxPapers": 50
}' |
apify call reapx/arxiv-papers-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "command": "npx",
            "args": [
                "mcp-remote",
                "https://mcp.apify.com/?tools=reapx/arxiv-papers-scraper",
                "--header",
                "Authorization: Bearer <YOUR_API_TOKEN>"
            ]
        }
    }
}

```

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/vW0faPtd11P3iEI1y/builds/RpgjvhUZKGNTaRiPC/openapi.json