# GitHub Scraper - Repos, Stars, Issues & Profiles (`cryptosignals/github-scraper`) Actor Scrape GitHub repositories, profiles, and issues — extract stars, forks, contributors, README, commit history, and topics. CSV/JSON output. No login. - **URL**: https://apify.com/cryptosignals/github-scraper.md - **Developed by:** [Web Data Labs](https://apify.com/cryptosignals) (community) - **Categories:** Developer tools, AI, Lead generation - **Stats:** 27 total users, 4 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing $5.00 / 1,000 result scrapeds This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## GitHub Scraper Extract public data from GitHub — repositories, user profiles, issues, stars, contributors, and more — in a structured format ready for analysis. ### Why Use This Actor? GitHub is the world's largest code hosting platform, and it is also a rich source of signals for developer relations, recruiting, market intelligence, and open-source research. But the GitHub API has rate limits that make large-scale collection slow and painful. This actor lets you collect GitHub data at scale without hitting API throttles or writing any code. Run it on demand or on a schedule and get clean, structured output every time. **Common reasons teams use this actor:** - Identify active developers and contributors in a specific technology or niche - Track adoption of open-source tools by watching star growth over time - Build recruitment pipelines targeting developers with specific language expertise - Monitor competitor repositories for activity, issues, and community growth - Research the open-source ecosystem for a category before building or investing ### What Data You Get #### Repository Fields | Field | Description | |-------|-------------| | `name` | Repository name | | `fullName` | owner/repo format | | `description` | Repository description | | `stars` | Star count | | `forks` | Fork count | | `watchers` | Watcher count | | `language` | Primary programming language | | `topics` | Repository topic tags | | `license` | License type | | `createdAt` | Creation date | | `updatedAt` | Last update timestamp | | `openIssues` | Number of open issues | | `url` | Repository URL | #### User/Profile Fields | Field | Description | |-------|-------------| | `login` | GitHub username | | `name` | Display name | | `bio` | Profile bio | | `company` | Company listed on profile | | `location` | Self-reported location | | `publicRepos` | Number of public repositories | | `followers` | Follower count | | `following` | Following count | | `email` | Public email (if listed) | | `blog` | Website/blog URL | | `joinedAt` | Account creation date | ### Use Cases #### Developer Recruiting Search for contributors to repositories in your tech stack. Filter by location, activity level, and language expertise. Export a list of potential candidates with their public contact info. #### Open-Source Intelligence Track how fast a new library is growing. Monitor issue velocity and contributor count as signals of project health. Identify abandoned projects before depending on them. #### Competitive Research Watch when a competitor open-sources something. Track their engineering team growth via contributor activity. Benchmark your own repo metrics against the field. #### Market Research Count the number of repositories using a specific framework or library. Understand which tools are gaining traction by watching star velocity. #### Sales Prospecting Find companies actively using a technology by searching their public repos. Identify engineering leads at target accounts. ### How to Use 1. Open the actor on Apify and click **Try for free** 2. Enter a GitHub search query, username, or repository URL 3. Set your desired result limit and data type (repos, users, issues) 4. Click **Start** and wait for the run to complete 5. Download as JSON, CSV, or stream results via the Apify API ### Input Parameters | Parameter | Type | Description | |-----------|------|-------------| | `searchQuery` | string | GitHub search term (e.g., "machine learning python stars:>1000") | | `type` | string | Data type: `repositories`, `users`, or `issues` | | `maxResults` | number | Maximum number of results to return | | `username` | string | Specific GitHub username to scrape profile/repos for | | `repoUrl` | string | Specific repository URL to extract data from | | `includeContributors` | boolean | Whether to include contributor list for repos | ### Sample Output ```json { "name": "awesome-python", "fullName": "vinta/awesome-python", "description": "A curated list of awesome Python frameworks, libraries, software and resources", "stars": 218000, "forks": 24500, "language": "Python", "topics": ["python", "awesome", "awesome-list"], "license": "CC-BY-4.0", "openIssues": 523, "createdAt": "2014-06-27T21:00:06Z", "updatedAt": "2026-04-20T12:34:00Z", "url": "https://github.com/vinta/awesome-python" } ```` ### Pricing This actor uses pay-per-event billing — you are only charged for data delivered. There are no upfront fees or minimum commitments. New Apify accounts come with a free tier to test the actor at no cost. ### Frequently Asked Questions **Does this require a GitHub account or token?** No. The actor collects publicly available data. No authentication is required from you. **What are the rate limits?** The actor handles rate limiting internally. You do not need to manage API quotas yourself. **Can I scrape private repositories?** No. Only public GitHub data is accessible and collected. **Can I get contributor data for a specific repo?** Yes. Enable the `includeContributors` option when running with a specific repository URL. **Can I filter results by programming language or stars?** Yes. Use GitHub's standard search syntax in the `searchQuery` field, such as `language:rust stars:>500`. **Is scheduling supported?** Yes. Apify's built-in scheduler lets you run this actor on any cron schedule. # Actor input Schema ## `action` (type: `string`): What type of data to extract from GitHub. ## `query` (type: `string`): Search query for repos/users, username for profiles, org name for org-repos, or owner/repo for repo-details. ## `url` (type: `string`): GitHub URL for user-profile (https://github.com/username), repo-details (https://github.com/owner/repo), or org-repos. Overrides query for these actions. ## `maxItems` (type: `integer`): Maximum number of results to return. GitHub API limits apply (60 requests/hour without auth). ## `language` (type: `string`): Filter results by programming language (e.g., python, javascript, rust). Works with search-repos and search-users. ## Actor input object example ```json { "action": "search-repos", "query": "python machine learning", "maxItems": 30 } ``` # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "action": "search-repos", "query": "python machine learning", "maxItems": 30 }; // Run the Actor and wait for it to finish const run = await client.actor("cryptosignals/github-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "action": "search-repos", "query": "python machine learning", "maxItems": 30, } # Run the Actor and wait for it to finish run = client.actor("cryptosignals/github-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "action": "search-repos", "query": "python machine learning", "maxItems": 30 }' | apify call cryptosignals/github-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=cryptosignals/github-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "GitHub Scraper - Repos, Stars, Issues & Profiles", "description": "Scrape GitHub repositories, profiles, and issues — extract stars, forks, contributors, README, commit history, and topics. CSV/JSON output. No login.", "version": "1.0", "x-build-id": "bRjq3cpmwAEpWOJXD" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/cryptosignals~github-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-cryptosignals-github-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/cryptosignals~github-scraper/runs": { "post": { "operationId": "runs-sync-cryptosignals-github-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/cryptosignals~github-scraper/run-sync": { "post": { "operationId": "run-sync-cryptosignals-github-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "required": [ "action" ], "properties": { "action": { "title": "Action", "enum": [ "search-repos", "search-users", "user-profile", "repo-details", "org-repos" ], "type": "string", "description": "What type of data to extract from GitHub.", "default": "search-repos" }, "query": { "title": "Search Query / Username / Org", "type": "string", "description": "Search query for repos/users, username for profiles, org name for org-repos, or owner/repo for repo-details.", "default": "python" }, "url": { "title": "GitHub URL (optional)", "type": "string", "description": "GitHub URL for user-profile (https://github.com/username), repo-details (https://github.com/owner/repo), or org-repos. Overrides query for these actions." }, "maxItems": { "title": "Max Results", "minimum": 1, "maximum": 500, "type": "integer", "description": "Maximum number of results to return. GitHub API limits apply (60 requests/hour without auth).", "default": 30 }, "language": { "title": "Language Filter", "type": "string", "description": "Filter results by programming language (e.g., python, javascript, rust). Works with search-repos and search-users." } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```