# Twitter ( X ) Comment Scraper (`datapilot/twitter-x-comment-scraper`) Actor Twitter/X Comment Scraper is an Apify Actor that extracts replies from a tweet URL using . It captures comment text, user info, likes, retweets, replies, and timestamps. Uses residential proxies, auto-scrolls to load more comments, and outputs clean, structured data. - **URL**: https://apify.com/datapilot/twitter-x-comment-scraper.md - **Developed by:** [Data Pilot](https://apify.com/datapilot) (community) - **Categories:** Other, Videos - **Stats:** 50 total users, 9 monthly users, 98.0% runs succeeded, 0 bookmarks - **User rating**: 4.00 out of 5 stars ## Pricing $8.00/month + usage To use this Actor, you pay a monthly rental fee to the developer. The rent is subtracted from your prepaid usage every month after the free trial period.You also pay for the Apify platform usage, which gets cheaper the higher Apify subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#rental-actors ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Twitter/X Comment Scraper πŸš€ **Twitter/X Comment Scraper** is an advanced Apify Actor designed to scrape **Twitter/X Comment data** from tweet URLs using browser automation. This tool provides comprehensive **Twitter/X Comment information**, including comment texts, user details, timestamps, and engagement metrics like likes and retweets. Whether you're analyzing **Twitter/X Comment sentiment**, conducting social media research, or monitoring **Twitter/X Comment trends**, the Twitter/X Comment Scraper delivers detailed **Twitter/X Comment data** efficiently. With browser-based scraping using , the Twitter/X Comment Scraper ensures accurate extraction of dynamic **Twitter/X Comment content** that may not be available through simple API calls. It focuses on key **Twitter/X Comment metrics** like user profiles and engagement data, making it an essential tool for **Twitter/X Comment analysis** and social media research. ### πŸ”₯ Features - **Comprehensive Twitter/X Comment Extraction** – Scrapes detailed **Twitter/X Comment data**, including comment texts, user information, timestamps, and engagement metrics for any tweet URL. - **Browser Automation** – Uses for headless browser navigation to access fully loaded **Twitter/X Comment sections**. - **Advanced Comment Loading** – Scrolls and waits to load more **Twitter/X Comment content** for complete extraction. - **API Response Parsing** – Intercepts **Twitter/X Comment data** from GraphQL API responses for accurate **Twitter/X Comment retrieval**. - **Batch Processing** – Processes multiple tweet URLs in a single run for efficient **Twitter/X Comment data collection**. - **Residential Proxy Support** – Utilizes Apify's residential proxies to bypass restrictions and ensure high success rates for **Twitter/X Comment scraping**. - **Error Handling** – Robust logging and fallback mechanisms for failed **Twitter/X Comment scrapes**. - **Dataset Integration** – Automatically uploads **Twitter/X Comment data** to your Apify dataset for easy export and analysis. --- ### βš™οΈ How It Works The Twitter/X Comment Scraper takes a tweet URL as input and uses to launch a headless browser, navigate to the URL, and intercept **Twitter/X Comment data** from API responses. It performs scrolling actions to load more **Twitter/X Comment content**, then parses the data to extract comment texts, user profiles, and engagement metrics. The scraper returns structured **Twitter/X Comment data** on success or error details on failure, providing a reliable way to gather **Twitter/X Comment** for research and analysis. **Key Processing Steps:** 1. **URL Validation** – Parse and validate tweet URL 2. **Browser Launch** – Initialize headless browser 3. **Page Navigation** – Navigate to tweet URL 4. **API Interception** – Intercept GraphQL API responses 5. **Comment Loading** – Scroll and load more comments 6. **Data Extraction** – Parse comment text, user info, metrics 7. **Data Compilation** – Aggregate comment data 8. **Export** – Push results to dataset in JSON format **Key benefits for Twitter/X Comment analysis:** - Access **Twitter/X Comment data** including user profiles and engagement metrics. - Analyze **Twitter/X Comment sentiment** and trends. - Build **Twitter/X Comment databases** for social media research. - Track engagement patterns and user responses. - Monitor conversation trends and topics. --- ### πŸ“₯ Input The scraper accepts the following input parameters: | Field | Type | Default | Description | |----------------------|---------|-------------------|-------------| | `tweetUrl` | string | required | The tweet URL to scrape Twitter/X Comment from (e.g., `"https://x.com/user/status/123"`). | | `useApifyProxy` | boolean | `true` | Enable residential proxies for Twitter/X Comment scraping. | | `apifyProxyGroups` | array | `["RESIDENTIAL"]` | Proxy groups to use (e.g., `["RESIDENTIAL"]`). | **Example input JSON:** ```json { "tweetUrl": "https://x.com/example/status/123456789", "useApifyProxy": true, "apifyProxyGroups": ["RESIDENTIAL"] } ```` ### πŸ“€ Output The scraper outputs detailed **Twitter/X Comment data** in JSON format for each comment. Each record includes: | Field | Type | Description | |------------|--------|-------------| | `id` | string | Unique ID of the Twitter/X Comment. | | `text` | string | Text content of the Twitter/X Comment. | | `user` | object | Twitter/X Comment user details object. | | `date` | object | Twitter/X Comment date information object. | | `stats` | object | Twitter/X Comment engagement metrics object. | | `replyTo` | string | Username the comment is replying to (if applicable). | **User Object Fields:** | Field | Type | Description | |------------|---------|-------------| | `username` | string | Twitter/X username. | | `fullName` | string | Full name of the user. | | `followers`| integer | Follower count of the user. | | `verified` | boolean | Verification status of the user. | **Date Object Fields:** | Field | Type | Description | |-------------|--------|-------------| | `formatted` | string | Formatted date (YYYY-MM-DD HH:MM:SS). | | `initial` | string | Initial date string format. | **Stats Object Fields:** | Field | Type | Description | |------------|---------|-------------| | `likes` | integer | Like count for the comment. | | `retweets` | integer | Retweet count for the comment. | | `replies` | integer | Reply count for the comment. | **Example output for Twitter/X Comment:** ```json { "id": "1234567890", "text": "This is a great example comment!", "user": { "username": "example_user", "fullName": "Example User", "followers": 1500, "verified": True }, "date": { "formatted": "2025-02-14 12:00:00", "initial": "Fri Feb 14 12:00:00 +0000 2025" }, "stats": { "likes": 25, "retweets": 5, "replies": 3 }, "replyTo": null } ``` **Example error response:** ```json { "tweetUrl": "https://x.com/invalid/status/invalid", "no_comments_found": true, "error": "Tweet URL not accessible or has no comments", "scraped_at": "2025-02-14T12:00:00Z" } ``` *** ### 🧰 Technical Stack - **API Interception:** GraphQL API response parsing - **Data Parsing:** JSON parsing and data extraction - **Proxy Support:** Apify Proxy with RESIDENTIAL support for reliability - **Platform:** Apify Actor – serverless, scalable, integrated with Dataset and Key‑Value Store - **Deployment:** One‑click run on Apify Console or via REST API *** ### 🎯 Use Cases - **Sentiment Analysis** – Analyze comment sentiment and emotional tone. - **Trend Analysis** – Identify trending topics and discussions. - **Social Media Research** – Conduct comprehensive social media research. - **Engagement Analysis** – Analyze engagement patterns and user interactions. - **Competitive Analysis** – Research competitor tweets and community responses. - **Brand Monitoring** – Monitor brand mentions and sentiment in comments. - **Influencer Analysis** – Analyze influencer engagement and audience responses. - **Crisis Management** – Monitor and track crisis-related comments and sentiment. - **Market Research** – Research market reactions and consumer feedback. - **User Research** – Understand user opinions and preferences. - **Content Analysis** – Analyze what types of content generate engagement. - **Academic Research** – Collect Twitter/X comment data for research studies. - **Dataset Building** – Build comment databases for machine learning. - **Customer Feedback** – Gather customer feedback and responses. *** ### πŸš€ Quick Start 1. **Open in Apify Console** – visit the Actor page and click Try for free. 2. **Enter tweet URL** – provide a Twitter/X tweet URL with comments. 3. **Enable proxies** – enabled by default for reliable scraping. 4. **Click Start** – the Actor will scrape tweet comments using browser automation. 5. **View Results** – check the dataset for extracted comment data. 6. **Analyze Comments** – examine user details, engagement metrics, timestamps. 7. **Export Data** – download the results as JSON, CSV, or Excel. *** ### πŸ’Ž Why This Scraper? | Feature | Benefit | |---------|---------| | βœ… Browser automation | Access fully rendered tweet content and comments. | | βœ… API interception | Extract data from Twitter/X's GraphQL API. | | βœ… Comment scrolling | Load all comments through smart scrolling. | | βœ… User profiles | Get detailed user information with each comment. | | βœ… Engagement metrics | Receive likes, retweets, and reply counts. | | βœ… Residential proxies | Bypass restrictions – reliable access. | | βœ… Error handling | Robust fallback mechanisms. | | βœ… Apify ecosystem | Seamless integration with other Actors, triggers, and webhooks. | ### πŸ“¦ Changelog #### v1.0.0 (February 2025) - Initial release of Twitter/X Comment Scraper - Browser automation with for comment extraction - GraphQL API response interception for accurate data - Comment text, user profile, and engagement metric extraction - Dynamic comment loading through scrolling - Batch processing for multiple tweet URLs - Residential proxy support for reliability - Verified user status detection - Timestamp extraction and formatting - Engagement metrics (likes, retweets, replies) - Reply-to relationship tracking - Error handling with detailed error messages - Automatic dataset integration - Full Apify Actor integration *** ### πŸ§‘β€πŸ’» Support & Feedback - **Issues & Ideas:** Open a ticket on the Apify Actor issue tracker - **Documentation:** Visit Apify Docs for comprehensive platform guides - **Community:** Join the Apify community forum for discussions and support - **Bug Reports:** Submit detailed bug reports through the issue tracker - **Feature Requests:** Suggest new features to improve the scraper *** ### πŸ’° Pricing - **Free** for basic usage on Apify platform - **Paid plans** available for higher limits and priority support - **Proxy credits** consumed based on residential proxy usage *** ### πŸ“„ License & Legal **Terms of Use:** - Scrape Twitter/X comments for legitimate research and analysis - Respect Twitter/X's Terms of Service - Don't republish comments without proper attribution - Comply with applicable laws and regulations - Respect copyright and intellectual property rights **Disclaimer:** Twitter/X Comment Scraper is provided as-is for research and analysis purposes. Users are responsible for ensuring their usage complies with Twitter/X's policies and applicable laws. *** ### πŸŽ‰ Get Started Today **Begin scraping Twitter/X comments now!** Use Twitter/X Comment Scraper for: - πŸ’¬ Comment Analysis - πŸ“Š Sentiment Research - πŸ” Social Listening - πŸ“ˆ Engagement Tracking - 🎯 Trend Analysis **Perfect for:** - Social Media Researchers - Sentiment Analysts - Brand Monitors - Content Strategists - Analysts *** **Last Updated:** February 2025\ **Version:** 1.0.0\ **Status:** Active Development\ **Support:** 24/7 Customer Support Available\ **Platform:** Apify *** ### πŸ“š Related Tools For comprehensive Twitter/X analysis and research, explore our full suite of tools: - Twitter/X Tweet Scraper - Twitter/X Video Downloader - All-in-One Media Downloader - YouTube Comment Scraper - TikTok Comments Scraper # Actor input Schema ## `tweetUrl` (type: `string`): Enter the full Twitter/X tweet URL (e.g., https://x.com/username/status/1234567890). ## `useApifyProxy` (type: `boolean`): Highly recommended for Twitter/X to prevent blocks. ## `apifyProxyGroups` (type: `array`): Residential proxies are selected by default. ## Actor input object example ```json { "tweetUrl": "https://x.com/elonmusk/status/1234567890", "useApifyProxy": true, "apifyProxyGroups": [ "RESIDENTIAL" ] } ``` # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "tweetUrl": "https://x.com/elonmusk/status/1234567890" }; // Run the Actor and wait for it to finish const run = await client.actor("datapilot/twitter-x-comment-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`πŸ’Ύ Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // πŸ“š Want to learn more πŸ“–? Go to β†’ https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "tweetUrl": "https://x.com/elonmusk/status/1234567890" } # Run the Actor and wait for it to finish run = client.actor("datapilot/twitter-x-comment-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("πŸ’Ύ Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # πŸ“š Want to learn more πŸ“–? Go to β†’ https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "tweetUrl": "https://x.com/elonmusk/status/1234567890" }' | apify call datapilot/twitter-x-comment-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=datapilot/twitter-x-comment-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Twitter ( X ) Comment Scraper", "description": "Twitter/X Comment Scraper is an Apify Actor that extracts replies from a tweet URL using . It captures comment text, user info, likes, retweets, replies, and timestamps. Uses residential proxies, auto-scrolls to load more comments, and outputs clean, structured data.", "version": "0.0", "x-build-id": "cJUH3RLuabZT2kCjZ" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/datapilot~twitter-x-comment-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-datapilot-twitter-x-comment-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/datapilot~twitter-x-comment-scraper/runs": { "post": { "operationId": "runs-sync-datapilot-twitter-x-comment-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/datapilot~twitter-x-comment-scraper/run-sync": { "post": { "operationId": "run-sync-datapilot-twitter-x-comment-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "required": [ "tweetUrl" ], "properties": { "tweetUrl": { "title": "Tweet URL", "type": "string", "description": "Enter the full Twitter/X tweet URL (e.g., https://x.com/username/status/1234567890)." }, "useApifyProxy": { "title": "Use Apify Proxy", "type": "boolean", "description": "Highly recommended for Twitter/X to prevent blocks.", "default": true }, "apifyProxyGroups": { "title": "Proxy Groups", "uniqueItems": true, "type": "array", "description": "Residential proxies are selected by default.", "items": { "type": "string", "enum": [ "RESIDENTIAL", "DATACENTER", "GOOGLE" ] }, "default": [ "RESIDENTIAL" ] } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```