# Twitter (X) Posts Search Scraper (`outspoken_strategy/twitter-post-search-scraper`) Actor

Search Twitter/X posts by keyword (advanced search). Latest/Top tabs, multiple keywords, date and media/engagement/verified filters, auto-pagination. Returns text, author, favorites, retweets, replies, quotes, views and media. Compatible with api-ninja/x-twitter-advanced-search.

- **URL**: https://apify.com/outspoken\_strategy/twitter-post-search-scraper.md
- **Developed by:** [code craker](https://apify.com/outspoken_strategy) (community)
- **Categories:** News, Social media, Other
- **Stats:** 10 total users, 7 monthly users, 99.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $5.00 / 1,000 results

This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Twitter (X) Posts Search Scraper

Scrape **Twitter/X posts by keyword** without the official API. This actor runs an X advanced search for one or many keywords, auto-paginates the results, and exports every matching tweet as structured JSON — text, author, likes, retweets, replies, quotes, views, media, hashtags and timestamps. Results can be downloaded as JSON, CSV, Excel or accessed via the Apify API.

The output is built for pipelines: flat items with `id_str`, `url`, `text`, `created_at`, engagement counts and a full `user` author object, so it slots straight into existing data workflows.

### Features

- 🔍 **Keyword search** — single keyword, exact phrases (`"like this"`), or many keywords per run, each searched separately and de-duplicated.
- 🗂 **Latest & Top tabs** — chronological or engagement-ranked results.
- 📅 **Date filters** — `since`/`until` bounds, applied as search operators and re-checked on every scraped tweet.
- 🎛 **Content filters** — links, hashtags, mentions, news, safe content, questions, verified / blue-verified authors, language.
- ♾ **Auto-pagination** — scrolls until your target tweet count is reached or the feed runs dry (up to 5,000 per run).
- 💾 **Incremental results** — tweets are pushed to the dataset as each keyword finishes, so long runs never lose collected data to a timeout.
- 🔐 **Flexible authentication** — session cookies, secret env vars, or fully automated login with session caching.
- 🛠 **Debuggable failures** — on empty runs the actor saves a screenshot and the page HTML to the run's key-value store.

### Use cases

- Brand, product and competitor monitoring
- Sentiment analysis and social listening pipelines
- Market and academic research
- Lead generation and trend discovery

### Quick start

1. Enter one or more keywords.
2. Provide a session (see **Authentication** below) — a throwaway X account's cookies or credentials.
3. Run. Tweets appear in the dataset as each keyword completes.

### Input

| Field | Type | Description |
| --- | --- | --- |
| `query` | string | One keyword/phrase. Several keywords can be pasted one per line. Advanced-search operators (`from:`, `"exact phrase"`, ...) pass through as-is. |
| `queries` | array | Multiple keywords — each searched separately, results combined and de-duplicated by tweet. `numberOfTweets` applies per keyword. |
| `search_type` | string | `Latest` (chronological) or `Top` (ranked). Default `Latest`. |
| `numberOfTweets` | integer | Target tweets per keyword (max 5000). Default 100. |
| `scrapeAll` | boolean | Auto-paginate until the target is reached. When `false` only the first page is returned. Default `true`. |
| `timeSince` / `timeUntil` | string | `yyyy-mm-dd` date bounds — applied as `since:`/`until:` operators and re-checked on the scraped tweets. |
| `language` | string | ISO 639-1 code, applied as `lang:`. |
| `contentQuestionMarks` | boolean | Questions only (`?` operator). |
| `engagementHasEngagement` | boolean | `filter:has_engagement`. |
| `mediaHasHashtags` / `mediaHasLinks` / `mediaHasMentions` | boolean | `filter:hashtags` / `filter:links` / `filter:mentions`. |
| `mediaNewsOnly` / `mediaSafeContentOnly` | boolean | `filter:news` / `filter:safe`. |
| `usersBlueVerifiedOnly` / `usersVerifiedOnly` | boolean | `filter:blue_verified` / `filter:verified`. |
| `cookies` | array | Cookies of a logged-in X session (Cookie-Editor/EditThisCookie JSON export). The `auth_token` + `ct0` cookies carry the session. |
| `twUsername` / `twPassword` / `twEmail` | string | Credentials for automated login (2FA must be disabled). The email answers X's occasional identity check. |
| `proxyConfiguration` | object | Proxy settings. Residential proxies strongly recommended. |
| `headless` | boolean | Uncheck only for local debugging. |

#### Authentication

X login-walls search for anonymous visitors, so the actor needs a session. It resolves one in this priority order:

1. the `cookies` input (per-run),
2. the `TW_COOKIES` secret env var (JSON cookie array or a `name=value; name2=value2` header string),
3. the session cached in the `twitter-session` key-value store by a previous automated login,
4. automated login with `twUsername`/`twPassword` (or the `TW_USERNAME`/`TW_PASSWORD`/`TW_EMAIL` secret env vars) — the fresh session is cached for future runs.

**Always use a throwaway account, never a personal one.** Accounts used for scraping can be restricted by X.

### Output

One dataset item per tweet:

```json
{
    "id": "1234567890123456789",
    "id_str": "1234567890123456789",
    "url": "https://x.com/user/status/1234567890123456789",
    "twitterUrl": "https://twitter.com/user/status/1234567890123456789",
    "text": "Tweet text...",
    "full_text": "Tweet text...",
    "lang": "en",
    "created_at": "2026-05-01T09:30:00.000Z",
    "createdTime": "2026-05-01T09:30:00.000Z",
    "favorites": 12,
    "retweets": 3,
    "replies": 4,
    "quotes": 1,
    "bookmarks": 2,
    "views": 4520,
    "hashtags": ["health"],
    "mentions": ["someuser"],
    "urls": ["https://example.com/article"],
    "media": [{ "type": "photo", "url": "https://pbs.twimg.com/media/...", "expanded_url": "...", "video_url": null }],
    "is_retweet": false,
    "is_quote": false,
    "is_reply": false,
    "conversation_id_str": "1234567890123456789",
    "possibly_sensitive": false,
    "user": {
        "id_str": "987654321",
        "name": "User Name",
        "screen_name": "user",
        "url": "https://x.com/user",
        "followers": 1500,
        "following": 300,
        "tweets": 8200,
        "verified": false,
        "blue_verified": true,
        "avatar": "https://pbs.twimg.com/profile_images/...",
        "created_at": "2015-03-01T00:00:00.000Z",
        "location": "Harare"
    },
    "username": "user",
    "authorName": "User Name",
    "author": { "id": "987654321", "name": "User Name", "username": "user", "profileUrl": "https://x.com/user" },
    "searchQuery": "econet",
    "search_type": "Latest",
    "scrapedAt": "2026-07-03T12:00:00.000Z"
}
```

### How it works

The actor opens `x.com/search` in a Playwright-driven Chrome with an authenticated session, intercepts the `SearchTimeline` GraphQL responses that X streams to the page, and scrolls to auto-paginate. Parsing walks the GraphQL JSON generically instead of hardcoding one payload path, so it survives X's frequent payload-shape changes.

### FAQ & troubleshooting

**Why do I get 0 results?** Almost always an expired/missing session or a blocked proxy IP. Check `DEBUG_SCREENSHOT` and `DEBUG_HTML` in the run's key-value store to see exactly what the browser saw. Also verify the keyword actually has matches on x.com/search.

**Login fails — what now?** The error message says which check X raised (identity confirmation, confirmation code/2FA, Arkose CAPTCHA, suspension) and what to do about it. Logging the account in manually once from the proxy's region usually clears security checks. When in doubt, export cookies from a logged-in browser session and pass those instead.

**Navigation timeouts?** Use Apify Proxy with the `RESIDENTIAL` group; datacenter IPs are widely blocked by X.

**Is this legal?** The actor extracts only publicly visible data. You are responsible for complying with X's Terms of Service and applicable laws (e.g. GDPR) for your use case — consult a lawyer if unsure.

# Actor input Schema

## `query` (type: `string`):

One keyword/phrase to search for. Wrap in double quotes for an exact phrase match, e.g. "econet revenue". Advanced-search operators (from:, since:, filter:, ...) are passed through as-is. You can also paste several keywords, one per line, to search them all. For a clean list use the "queries" field below instead. Leave empty if you use "queries".

## `queries` (type: `array`):

Multiple keywords/phrases to search — each is searched separately and results are combined and de-duplicated by tweet. Takes precedence over/adds to "query". numberOfTweets applies PER keyword.

## `search_type` (type: `string`):

Which search tab to scrape: "Latest" (chronological) or "Top" (ranked by relevance/engagement).

## `numberOfTweets` (type: `integer`):

Maximum number of tweets to fetch per keyword.

## `scrapeAll` (type: `boolean`):

Keep scrolling/paginating until numberOfTweets is reached or the feed runs dry. When unchecked only the first page of results is returned.

## `timeSince` (type: `string`):

Only return tweets published on or after this date (format: yyyy-mm-dd). Applied as the since: advanced-search operator and re-checked on our side.

## `timeUntil` (type: `string`):

Only return tweets published before this date (format: yyyy-mm-dd). Applied as the until: advanced-search operator and re-checked on our side.

## `language` (type: `string`):

Restrict results to a language (ISO 639-1 code, e.g. "en", "fr"). Applied as the lang: advanced-search operator.

## `contentQuestionMarks` (type: `boolean`):

Only tweets phrased as questions (adds the ? operator).

## `engagementHasEngagement` (type: `boolean`):

Only tweets with some engagement (adds filter:has\_engagement).

## `mediaHasHashtags` (type: `boolean`):

Only tweets containing hashtags (adds filter:hashtags).

## `mediaHasLinks` (type: `boolean`):

Only tweets containing links (adds filter:links).

## `mediaHasMentions` (type: `boolean`):

Only tweets mentioning other users (adds filter:mentions).

## `mediaNewsOnly` (type: `boolean`):

Only tweets linking to news (adds filter:news).

## `mediaSafeContentOnly` (type: `boolean`):

Exclude potentially sensitive content (adds filter:safe).

## `usersBlueVerifiedOnly` (type: `boolean`):

Only tweets from Twitter Blue verified accounts (adds filter:blue\_verified).

## `usersVerifiedOnly` (type: `boolean`):

Only tweets from legacy-verified accounts (adds filter:verified).

## `cookies` (type: `array`):

Cookies of a logged-in X session, exported with a browser extension such as Cookie-Editor or EditThisCookie (JSON array of {name, value, domain, ...}). REQUIRED in practice: X search is login-walled for anonymous visitors. The "auth\_token" and "ct0" cookies carry the session. Alternatively set the TW\_COOKIES secret environment variable on the actor (same JSON, or a 'name=value; name2=value2' string) to avoid passing cookies in every run. Use a throwaway account, never your personal one.

## `twUsername` (type: `string`):

Username/email/phone of a throwaway X account. If set together with twPassword (or via the TW\_USERNAME/TW\_PASSWORD secret env vars), the actor logs in itself and caches the session, so you don't need to paste cookies. The account must have 2FA disabled. Prefer setting these as secret environment variables on the actor rather than per run.

## `twPassword` (type: `string`):

Password for the account in twUsername. Stored encrypted. Prefer the TW\_PASSWORD secret environment variable instead of putting it in run input.

## `twEmail` (type: `string`):

Email of the account, used only when X asks to confirm the identity during automated login ("Enter your phone number or email"). Also settable via the TW\_EMAIL env var.

## `proxyConfiguration` (type: `object`):

Proxy to route traffic through. Residential proxies with a country set are strongly recommended for X — the random global residential pool often hands out dead exit IPs, so the actor defaults RESIDENTIAL to US exits when no country is chosen.

## `headless` (type: `boolean`):

Uncheck to run a headed browser (useful only for local debugging).

## Actor input object example

```json
{
  "query": "artificial intelligence",
  "queries": [],
  "search_type": "Latest",
  "numberOfTweets": 100,
  "scrapeAll": true,
  "contentQuestionMarks": false,
  "engagementHasEngagement": false,
  "mediaHasHashtags": false,
  "mediaHasLinks": false,
  "mediaHasMentions": false,
  "mediaNewsOnly": false,
  "mediaSafeContentOnly": false,
  "usersBlueVerifiedOnly": false,
  "usersVerifiedOnly": false,
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": [
      "RESIDENTIAL"
    ],
    "apifyProxyCountry": "US"
  },
  "headless": true
}
```

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "query": "artificial intelligence",
    "queries": [],
    "proxyConfiguration": {
        "useApifyProxy": true,
        "apifyProxyGroups": [
            "RESIDENTIAL"
        ],
        "apifyProxyCountry": "US"
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("outspoken_strategy/twitter-post-search-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "query": "artificial intelligence",
    "queries": [],
    "proxyConfiguration": {
        "useApifyProxy": True,
        "apifyProxyGroups": ["RESIDENTIAL"],
        "apifyProxyCountry": "US",
    },
}

# Run the Actor and wait for it to finish
run = client.actor("outspoken_strategy/twitter-post-search-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "query": "artificial intelligence",
  "queries": [],
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": [
      "RESIDENTIAL"
    ],
    "apifyProxyCountry": "US"
  }
}' |
apify call outspoken_strategy/twitter-post-search-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "command": "npx",
            "args": [
                "mcp-remote",
                "https://mcp.apify.com/?tools=outspoken_strategy/twitter-post-search-scraper",
                "--header",
                "Authorization: Bearer <YOUR_API_TOKEN>"
            ]
        }
    }
}

```

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/qrmKEcWmE4WGXuuvd/builds/7IhQvrTHwphnnpIaA/openapi.json