# GitHub Repositories Search Scraper (`fetch_cat/github-repositories-search-scraper`) Actor Search public GitHub repositories by query, language, topics, stars, forks, and activity. Export repo URLs, owners, topics, licenses, timestamps, and optional owner details. - **URL**: https://apify.com/fetch\_cat/github-repositories-search-scraper.md - **Developed by:** [Hanna Nosova](https://apify.com/fetch_cat) (community) - **Categories:** Developer tools - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $0.12 / 1,000 repository results This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## GitHub Repositories Search Scraper Search and export public GitHub repositories by keyword, language, topic, stars, forks, and recent activity. Use this actor when you need a repeatable CSV, JSON, Excel, or API export of public repositories for developer relations, market research, open-source intelligence, investment research, competitive tracking, or technical lead generation. ### What does GitHub Repositories Search Scraper do? GitHub Repositories Search Scraper turns GitHub repository searches into structured datasets. It accepts one or more search queries and returns public repository records with useful discovery and scoring fields. You can search by product name, company name, technology, framework, keyword, or topic. You can optionally filter by programming language. You can sort by best match, stars, forks, or recently updated repositories. You can export results from Apify as JSON, CSV, Excel, XML, RSS, or through the Dataset API. ### Who is it for? This actor is useful for teams that need public open-source repository data in a clean table. - 🧑‍💻 Developer-relations teams can find projects that mention a platform, SDK, or integration. - 📈 Venture and market researchers can map active open-source ecosystems. - 🧲 Lead-generation teams can discover companies, maintainers, and projects by technology. - 📰 Newsletter writers can build lists of repositories around a trend or topic. - 🛡️ Open-source intelligence teams can monitor public repositories related to vendors, packages, or keywords. - 🏢 Competitive-intelligence teams can track new projects around a competitor or market category. ### Why use this actor? The actor gives you a repeatable workflow instead of copying results from a browser. It saves repository URLs, owners, stars, forks, topics, licenses, timestamps, and other fields in one dataset. It also supports multiple queries in a single run. That makes it easier to compare markets, keywords, languages, and topics. ### Typical use cases - Find public repositories mentioning your company, product, or API. - Build a list of popular repositories in a programming language. - Monitor repositories around a topic such as `web-scraping`, `agent`, or `mcp`. - Export GitHub repository URLs for enrichment in another system. - Discover high-star projects in a technical category. - Track recently updated repositories for trend research. - Create market maps for open-source ecosystems. ### GitHub repository search examples Try focused searches such as: - `language:python AI agents` to find Python AI-agent repositories. - `topic:web-scraping` to export public scraping-related repositories. - Competitor or product names to monitor open-source references. - SDK or package names with `sort:updated` to find recently active projects. ### What data can you extract? The actor returns one dataset row per repository. | Field | Description | | --- | --- | | `query` | The input query that produced the repository | | `fullName` | Owner and repository name | | `htmlUrl` | Public GitHub repository URL | | `description` | Repository description | | `ownerLogin` | Repository owner login | | `ownerType` | Owner type, such as User or Organization | | `stars` | Stargazer count | | `forks` | Fork count | | `watchers` | Watcher count | | `openIssues` | Open issue count | | `language` | Primary language | | `topics` | Public GitHub topics | | `licenseName` | Repository license name when available | | `createdAt` | Repository creation date | | `updatedAt` | Repository update date | | `pushedAt` | Last push date | | `archived` | Whether the repository is archived | | `fork` | Whether the repository is a fork | | `homepage` | Public homepage URL when set | | `cloneUrl` | HTTPS clone URL | ### Optional owner details Turn on `includeOwnerDetails` when you need more public profile data about repository owners. This can return fields such as owner name, company, blog, location, public repo count, followers, following, and public profile timestamps. Owner details use additional GitHub requests. For large enriched runs, add a GitHub token to increase API limits. ### How much does it cost to scrape GitHub repositories? The actor uses pay-per-event pricing. You pay a small start fee for each run and a per-repository fee for each saved result. Current pricing is $0.005 per run plus a BRONZE per-repository price of $0.0002, with canonical tier discounts on higher Apify plans. Exact live pricing is shown on the Apify actor page before you start a run. Use a low `maxResults` value for your first run, review the dataset, and then scale up. ### Input configuration The main input fields are: - `queries` — required list of search terms. - `language` — optional programming language filter. - `topics` — optional list of GitHub topics to require. - `sort` — best match, stars, forks, or updated. - `order` — descending or ascending. - `maxResults` — maximum repositories to save. - `includeOwnerDetails` — fetch extra owner profile fields. - `githubToken` — optional secret token for higher GitHub API limits. ### Example input ```json { "queries": ["apify", "web scraping"], "language": "javascript", "topics": ["automation"], "sort": "stars", "order": "desc", "maxResults": 50, "includeOwnerDetails": false } ```` ### Example output ```json { "query": "apify", "name": "API-mega-list", "fullName": "cporter202/API-mega-list", "htmlUrl": "https://github.com/cporter202/API-mega-list", "description": "A public API collection and developer resource list.", "ownerLogin": "cporter202", "ownerType": "User", "stars": 6733, "forks": 1294, "watchers": 6733, "openIssues": 15, "language": "JavaScript", "topics": ["api", "automation", "web-scraping"], "licenseName": null, "updatedAt": "2026-06-22T01:14:16Z", "pushedAt": "2026-01-27T18:40:57Z", "archived": false, "fork": false, "homepage": null, "cloneUrl": "https://github.com/cporter202/API-mega-list.git" } ``` ### How to run it 1. Open the actor on Apify. 2. Add one or more repository search queries. 3. Optionally choose a language and topics. 4. Set the maximum number of repositories. 5. Start the run. 6. Download the dataset or use the API. ### Search tips Use broad queries for discovery. Use language filters when you need a specific developer ecosystem. Use topic filters when you need a cleaner topical list. Sort by stars for popular projects. Sort by updated for recent activity. Use multiple queries when comparing markets or competitors. Start with 25 results before scaling to hundreds. ### Handling GitHub limits Small public searches work without a token. GitHub applies rate limits to anonymous API traffic. If you run large searches or owner enrichment, provide a GitHub token in the secret input field. The token is used only for GitHub API authentication during your run. If a rate limit is reached, the actor reports a clear error and suggests using a smaller run or token. ### Integrations You can connect the dataset to downstream tools. - 📊 Send CSV exports to spreadsheets for market analysis. - 🧩 Use Make or Zapier to trigger follow-up enrichment. - 🗃️ Load JSON results into a warehouse or database. - 📬 Feed repository lists into developer-relations workflows. - 🔎 Combine repository URLs with other Apify actors for broader research. ### API usage #### Node.js ```js import { ApifyClient } from 'apify-client'; const client = new ApifyClient({ token: process.env.APIFY_TOKEN }); const run = await client.actor('fetch_cat/github-repositories-search-scraper').call({ queries: ['apify'], language: 'javascript', maxResults: 25 }); const { items } = await client.dataset(run.defaultDatasetId).listItems(); console.log(items); ``` #### Python ```python from apify_client import ApifyClient import os client = ApifyClient(os.environ['APIFY_TOKEN']) run = client.actor('fetch_cat/github-repositories-search-scraper').call(run_input={ 'queries': ['apify'], 'language': 'javascript', 'maxResults': 25, }) items = client.dataset(run['defaultDatasetId']).list_items().items print(items) ``` #### cURL ```bash curl -X POST 'https://api.apify.com/v2/acts/fetch_cat~github-repositories-search-scraper/runs?token=YOUR_APIFY_TOKEN' \ -H 'Content-Type: application/json' \ -d '{"queries":["apify"],"language":"javascript","maxResults":25}' ``` ### MCP usage Use this actor from MCP-compatible tools through Apify MCP Server. MCP URL: ```text https://mcp.apify.com/?tools=fetch_cat/github-repositories-search-scraper ``` Claude Code setup: ```bash claude mcp add apify-github-repositories "https://mcp.apify.com/?tools=fetch_cat/github-repositories-search-scraper" ``` Claude Desktop JSON config: ```json { "mcpServers": { "apify-github-repositories": { "url": "https://mcp.apify.com/?tools=fetch_cat/github-repositories-search-scraper" } } } ``` Example prompts: - "Find 25 JavaScript repositories about Apify and summarize the top owners." - "Export popular Python repositories about web scraping to a table." - "Search GitHub repositories for MCP tools and rank them by stars." ### Data quality notes Repository counts and metadata come from public GitHub repository records. Stars, forks, issues, and timestamps can change over time. Some repositories do not have a license, homepage, topics, or recent push timestamp. Owner email is often unavailable because many GitHub users do not publish an email on their profile. ### FAQ and troubleshooting #### Why did my run return fewer repositories than requested? The source may have fewer matching public repositories for your exact query, language, and topic filters. Try removing a topic filter or using a broader query. #### Why did I hit a rate limit? Anonymous GitHub API traffic has stricter limits. Use a smaller `maxResults` value or add a GitHub token for higher limits. #### Do I need a GitHub token? No for small anonymous public searches. For large runs or owner enrichment, provide a GitHub token because GitHub API limits are stricter without authentication. #### Why are some fields empty? Some public repositories do not provide homepage, license, topics, or owner profile details. The actor keeps those fields empty instead of guessing. ### Legality and responsible use This actor extracts publicly available GitHub repository data. Use the data responsibly and follow GitHub's terms, Apify's terms, and applicable laws. Do not use exported data for spam, harassment, credential collection, or other abusive activity. If you use owner details for outreach, comply with privacy and anti-spam rules in your jurisdiction. ### Related scrapers Explore related actors from the same Apify account: - https://apify.com/fetch\_cat/website-content-crawler-lite - https://apify.com/fetch\_cat/web-page-to-markdown-extractor - https://apify.com/fetch\_cat/google-news-scraper - https://apify.com/fetch\_cat/reddit-scraper ### Changelog - `0.1` — Initial version for public GitHub repository search exports. ### Need help? If a query behaves differently than expected, start with a small `maxResults` value and inspect the dataset. Then adjust language, topic, sort, and order settings. For larger enriched runs, use a GitHub token and keep your first test small. # Actor input Schema ## `queries` (type: `array`): One or more GitHub repository search queries, such as company names, topics, frameworks, or keywords. ## `language` (type: `string`): Optional GitHub language filter, for example javascript, python, go, rust, or typescript. ## `topics` (type: `array`): Optional GitHub topics to require. Example: web-scraping, automation, ai. ## `sort` (type: `string`): Choose GitHub's ranking mode. Best match uses GitHub's default relevance ranking. ## `order` (type: `string`): Descending returns the highest or newest repositories first; ascending reverses the order. ## `maxResults` (type: `integer`): Maximum number of repository records to save across all queries. GitHub search exposes up to 1,000 results per search. ## `includeOwnerDetails` (type: `boolean`): Fetch extra public profile fields for each repository owner. This uses more GitHub API requests and can hit unauthenticated rate limits faster. ## `githubToken` (type: `string`): Optional personal access token for higher GitHub API rate limits. Leave empty for small public searches. ## Actor input object example ```json { "queries": [ "apify" ], "language": "javascript", "topics": [], "sort": "best-match", "order": "desc", "maxResults": 20, "includeOwnerDetails": false } ``` # Actor output Schema ## `overview` (type: `string`): No description # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "queries": [ "apify" ], "language": "javascript", "topics": [] }; // Run the Actor and wait for it to finish const run = await client.actor("fetch_cat/github-repositories-search-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "queries": ["apify"], "language": "javascript", "topics": [], } # Run the Actor and wait for it to finish run = client.actor("fetch_cat/github-repositories-search-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "queries": [ "apify" ], "language": "javascript", "topics": [] }' | apify call fetch_cat/github-repositories-search-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=fetch_cat/github-repositories-search-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "GitHub Repositories Search Scraper", "description": "Search public GitHub repositories by query, language, topics, stars, forks, and activity. Export repo URLs, owners, topics, licenses, timestamps, and optional owner details.", "version": "0.1", "x-build-id": "tpNK8TDJYuVaMRs6b" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/fetch_cat~github-repositories-search-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-fetch_cat-github-repositories-search-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/fetch_cat~github-repositories-search-scraper/runs": { "post": { "operationId": "runs-sync-fetch_cat-github-repositories-search-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/fetch_cat~github-repositories-search-scraper/run-sync": { "post": { "operationId": "run-sync-fetch_cat-github-repositories-search-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "required": [ "queries" ], "properties": { "queries": { "title": "Search queries", "type": "array", "description": "One or more GitHub repository search queries, such as company names, topics, frameworks, or keywords.", "items": { "type": "string" } }, "language": { "title": "Programming language", "type": "string", "description": "Optional GitHub language filter, for example javascript, python, go, rust, or typescript." }, "topics": { "title": "Topics", "type": "array", "description": "Optional GitHub topics to require. Example: web-scraping, automation, ai.", "items": { "type": "string" } }, "sort": { "title": "Sort repositories by", "enum": [ "best-match", "stars", "forks", "updated" ], "type": "string", "description": "Choose GitHub's ranking mode. Best match uses GitHub's default relevance ranking.", "default": "best-match" }, "order": { "title": "Sort order", "enum": [ "desc", "asc" ], "type": "string", "description": "Descending returns the highest or newest repositories first; ascending reverses the order.", "default": "desc" }, "maxResults": { "title": "Maximum repositories", "minimum": 1, "maximum": 1000, "type": "integer", "description": "Maximum number of repository records to save across all queries. GitHub search exposes up to 1,000 results per search.", "default": 20 }, "includeOwnerDetails": { "title": "Include owner details", "type": "boolean", "description": "Fetch extra public profile fields for each repository owner. This uses more GitHub API requests and can hit unauthenticated rate limits faster.", "default": false }, "githubToken": { "title": "GitHub token (optional)", "type": "string", "description": "Optional personal access token for higher GitHub API rate limits. Leave empty for small public searches." } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```