# Twitter Profile Scraper (`crawlerbros/twitter-profile-scraper`) Actor Extract comprehensive Twitter/X profile data and tweets including all engagement metrics (likes, retweets, replies, quotes, bookmarks, views), profile details, media URLs, hashtags, and mentions with anti-detection features and authenticated scraping support. - **URL**: https://apify.com/crawlerbros/twitter-profile-scraper.md - **Developed by:** [Crawler Bros](https://apify.com/crawlerbros) (community) - **Categories:** Social media, Automation, Developer tools - **Stats:** 96 total users, 2 monthly users, 100.0% runs succeeded, 3 bookmarks - **User rating**: 5.00 out of 5 stars ## Pricing from $5.00 / 1,000 results This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Twitter Profile Scraper Scrape tweets and full profile data from any public **Twitter / X** account. Extract every engagement metric (likes, retweets, replies, bookmarks, views), media URLs, hashtags, mentions, and complete author metadata — without using the official API. Twitter session cookies are **required** for reliable extraction. See the **Authentication** section below. ### What you get Each item in the dataset is one tweet, enriched with the author's profile metadata. #### Tweet fields | Field | Description | | --- | --- | | `tweet_id` | Tweet ID | | `tweet_url` | Direct tweet URL | | `text` | Full tweet text (with hashtags / mentions intact) | | `timestamp` | ISO 8601 timestamp the tweet was posted | | `likes_count` | Like count | | `retweets_count` | Retweet count | | `replies_count` | Reply count | | `bookmarks_count` | Bookmark count | | `views_count` | View count | | `is_retweet` | `true` if the tweet is a retweet | | `is_reply` | `true` if the tweet is a reply | | `reply_to` | Username being replied to (when `is_reply=true`) | | `media_urls` | Image / video URLs attached to the tweet | | `hashtags` | Hashtags used (without `#`) | | `mentions` | Usernames mentioned (without `@`) | | `urls` | External URLs in the tweet | | `scrapedProfile` | Username that was scraped | | `authorMeta` | Full profile snapshot — see below | #### `authorMeta` fields | Field | Description | | --- | --- | | `username` | Twitter/X handle | | `display_name` | Full display name | | `bio` | Profile biography | | `location` | Location text from the profile | | `website` | Website URL listed on the profile | | `joined_date` | Account creation date as shown on profile | | `followers_count` | Follower count | | `following_count` | Following count | | `tweets_count` | Total post count | | `verified` | `true` for Blue/verified accounts | | `profile_image_url` | Profile picture URL (400×400) | | `profile_banner_url` | Header / banner image URL | | `scraped_at` | ISO 8601 UTC timestamp | Empty fields are dropped from each record so the dataset stays clean. ### Input | Parameter | Type | Default | Description | | --- | --- | --- | --- | | `usernames` | Array | **required** | Twitter/X usernames to scrape (with or without `@`). | | `maxTweets` | Integer | `50` | Max tweets to collect per profile (1-500). | | `includeReplies` | Boolean | `false` | Include reply tweets. | | `includeRetweets` | Boolean | `true` | Include retweets. | | `cookies` | String | **required** | Twitter/X session cookies in JSON format. See Authentication. | | `proxyConfiguration` | Object | — | Optional proxy. Leave off unless you hit IP-based blocks. | #### Example input — minimal ```json { "usernames": ["elonmusk"], "cookies": "[{\"name\":\"auth_token\",\"value\":\"YOUR_TOKEN\",\"domain\":\".x.com\"}]" } ```` #### Example input — multiple profiles, replies excluded ```json { "usernames": ["elonmusk", "openai", "github"], "maxTweets": 200, "includeReplies": false, "includeRetweets": true, "cookies": "[...]" } ``` ### Example output ```json { "tweet_id": "1881234567890123456", "tweet_url": "https://twitter.com/elonmusk/status/1881234567890123456", "text": "Mars by 2030.", "timestamp": "2026-04-15T14:32:00.000Z", "likes_count": 480000, "retweets_count": 32000, "replies_count": 8200, "bookmarks_count": 4100, "views_count": 18400000, "is_retweet": false, "is_reply": false, "media_urls": ["https://pbs.twimg.com/media/AbCdEf123.jpg"], "hashtags": [], "mentions": [], "urls": [], "scrapedProfile": "elonmusk", "authorMeta": { "username": "elonmusk", "display_name": "Elon Musk", "bio": "Tesla, SpaceX, Neuralink, The Boring Company", "location": "Texas, USA", "website": "https://tesla.com", "joined_date": "Joined June 2009", "followers_count": 228100000, "following_count": 1219, "tweets_count": 47800, "verified": true, "profile_image_url": "https://pbs.twimg.com/profile_images/.../photo_400x400.jpg", "profile_banner_url": "https://pbs.twimg.com/profile_banners/.../banner.jpg", "scraped_at": "2026-05-05T13:42:18Z" } } ``` ### Authentication Twitter/X locks profile content behind a login wall — without valid cookies, the platform serves a curated old-tweets feed instead of the chronological timeline you see when logged in. The `cookies` input is a JSON array of your browser cookies for `x.com`. #### How to export your cookies 1. Log into [x.com](https://x.com) in Chrome or Firefox. 2. Install the [Cookie-Editor](https://cookie-editor.com) extension. 3. Click the extension icon → **Export → Export as JSON**. 4. Paste the JSON array into the `cookies` input field. `auth_token` and `ct0` are the two cookies that must be present. Cookies typically last 30-60 days before they need refreshing. #### Cookie format ```json [ { "name": "auth_token", "value": "YOUR_AUTH_TOKEN", "domain": ".x.com", "path": "/", "httpOnly": true, "secure": true }, { "name": "ct0", "value": "YOUR_CT0_VALUE", "domain": ".x.com", "path": "/", "httpOnly": false, "secure": true } ] ``` > Keep your cookies private — they grant access to your X account. ### Use cases - **Influencer analysis** — Pull complete tweet history from key accounts in your niche. - **Brand monitoring** — Track tweets and engagement on your own brand handles or competitors'. - **Lead enrichment** — Augment CRM records with bio, website, follower counts, and verified status. - **Content audits** — Export an account's posts for analytics, archiving, or compliance. - **Trend analysis** — Compare engagement curves across multiple accounts in the same vertical. - **Sentiment monitoring** — Feed tweet text into NLP / sentiment pipelines. ### FAQ **Why am I getting old tweets instead of recent ones?** Without valid cookies, Twitter shows a "Highlights" feed of popular old tweets rather than the chronological timeline. Re-export fresh cookies and retry. **Can I scrape private / protected profiles?** Only if you authenticate with cookies from an account that follows the protected profile. The actor cannot bypass Twitter's privacy controls. **How many tweets per profile?** Up to 500 (`maxTweets` maximum). For very large profiles, run multiple passes with different filter combinations or pull replies / retweets separately. **What if my cookies are expired?** The actor probes the cookie validity at startup. If the session is dead, it emits a single sentinel record `{ type: "twitter_profile_blocked", reason: "expired_cookies", ... }` and exits cleanly so you know to refresh. **Will my account get banned?** The actor uses random delays, scrolls naturally, and mimics human browsing. Risk is low for typical usage. Use a dedicated account, not your primary one, if you're running heavy workloads (500 tweets × many profiles back-to-back). **Does this work without a proxy?** Yes — proxy is off by default. Enable `proxyConfiguration` only if you encounter persistent IP-based blocks. **How current is the data?** Live — every run hits Twitter/X at request time. Schedule the actor for hourly or daily refreshes. ### Limitations - Cannot access truly private / protected accounts without being an authenticated follower. - Cookies expire — refresh every ~30-60 days for steady performance. - Very high `maxTweets` values (300+) take longer and carry a higher block risk. - Engagement counts for very old tweets may differ slightly from the official API. # Actor input Schema ## `usernames` (type: `array`): List of Twitter/X usernames to scrape (with or without @ prefix). Each profile will be scraped separately. Twitter usernames are 1-15 characters, alphanumeric and underscore only. ## `maxTweets` (type: `integer`): Maximum number of tweets to scrape for each profile (1-500). Note: Scraping many tweets takes longer. ## `includeReplies` (type: `boolean`): Include tweets that are replies to other users ## `includeRetweets` (type: `boolean`): Include retweets from the profile ## `cookies` (type: `string`): **Required.** Twitter/X authentication cookies in JSON format. Twitter heavily restricts unauthenticated browsing, so a valid session is mandatory. Format: \[{"name":"auth\_token","value":"...","domain":".x.com"}, ...]. Export from your browser via a cookies extension and paste here. Without cookies, the run returns a single placeholder record indicating cookies are needed. ## `proxyConfiguration` (type: `object`): Optional proxy to route browser traffic through. Disabled by default (adds cost). Enable if you encounter IP-based rate limiting or blocks. ## Actor input object example ```json { "usernames": [ "elonmusk", "openai", "github" ], "maxTweets": 50, "includeReplies": false, "includeRetweets": true } ``` # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "usernames": [ "elonmusk" ], "cookies": "" }; // Run the Actor and wait for it to finish const run = await client.actor("crawlerbros/twitter-profile-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "usernames": ["elonmusk"], "cookies": "", } # Run the Actor and wait for it to finish run = client.actor("crawlerbros/twitter-profile-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "usernames": [ "elonmusk" ], "cookies": "" }' | apify call crawlerbros/twitter-profile-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=crawlerbros/twitter-profile-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Twitter Profile Scraper", "description": "Extract comprehensive Twitter/X profile data and tweets including all engagement metrics (likes, retweets, replies, quotes, bookmarks, views), profile details, media URLs, hashtags, and mentions with anti-detection features and authenticated scraping support.", "version": "1.6", "x-build-id": "oGVyMD5LI889SG6e6" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/crawlerbros~twitter-profile-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-crawlerbros-twitter-profile-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/crawlerbros~twitter-profile-scraper/runs": { "post": { "operationId": "runs-sync-crawlerbros-twitter-profile-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/crawlerbros~twitter-profile-scraper/run-sync": { "post": { "operationId": "run-sync-crawlerbros-twitter-profile-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "required": [ "usernames", "cookies" ], "properties": { "usernames": { "title": "Twitter Usernames", "minItems": 1, "type": "array", "description": "List of Twitter/X usernames to scrape (with or without @ prefix). Each profile will be scraped separately. Twitter usernames are 1-15 characters, alphanumeric and underscore only.", "items": { "type": "string", "pattern": "^@?[A-Za-z0-9_]{1,15}$" } }, "maxTweets": { "title": "Maximum Tweets Per Profile", "minimum": 1, "maximum": 500, "type": "integer", "description": "Maximum number of tweets to scrape for each profile (1-500). Note: Scraping many tweets takes longer.", "default": 50 }, "includeReplies": { "title": "Include Replies", "type": "boolean", "description": "Include tweets that are replies to other users", "default": false }, "includeRetweets": { "title": "Include Retweets", "type": "boolean", "description": "Include retweets from the profile", "default": true }, "cookies": { "title": "Twitter Cookies (Required)", "type": "string", "description": "**Required.** Twitter/X authentication cookies in JSON format. Twitter heavily restricts unauthenticated browsing, so a valid session is mandatory. Format: [{\"name\":\"auth_token\",\"value\":\"...\",\"domain\":\".x.com\"}, ...]. Export from your browser via a cookies extension and paste here. Without cookies, the run returns a single placeholder record indicating cookies are needed." }, "proxyConfiguration": { "title": "Proxy Configuration", "type": "object", "description": "Optional proxy to route browser traffic through. Disabled by default (adds cost). Enable if you encounter IP-based rate limiting or blocks." } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```