# GitHub Scraper — Repositories & User Profiles (`hipersoft/github-scraper`) Actor

Search GitHub and scrape repositories (stars, forks, language, topics, license, dates) or user profiles (name, company, location, bio, followers, public email, repos). Uses the GitHub REST API; optional token for higher rate limits. For developer lead generation, OSS research and recruiting.

- **URL**: https://apify.com/hipersoft/github-scraper.md
- **Developed by:** [hiper soft](https://apify.com/hipersoft) (community)
- **Categories:** Developer tools, Lead generation
- **Stats:** 3 total users, 2 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.0008 / item scraped

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## GitHub Scraper — Repositories & User Profiles, Stars, Topics & Leads

Search GitHub and scrape repositories (stars, forks, language, topics, license, dates) or user profiles (name, company, location, bio, followers, public email, repos) into clean, structured JSON. It uses the official GitHub REST API and works out of the box with no login — add an optional token for higher rate limits. Built for developer lead generation, open-source research and technical recruiting.

### Features

- 🔀 **Two modes** — search `repositories` or `users` with the same powerful GitHub query syntax.
- 🔎 **Full query power** — use qualifiers like `stars:>10000 language:typescript` or `location:berlin followers:>500`.
- ⭐ **Repo metrics** — `stars`, `forks`, `watchers`, `openIssues`, `language`, `topics`, `license` and activity dates.
- 👤 **Profile enrichment** — optionally fetch each user's full profile for `email`, `company`, `followers` and `bio`.
- ↕️ **Sort & order** — sort repos by stars/forks/updated or users by followers/repositories/joined, ascending or descending.
- 🔑 **Optional token** — supply a personal access token for ~5000 requests/hour instead of unauthenticated limits.
- ⚙️ **Reliable by design** — respects rate-limit resets and retries on 403/429 so large runs complete cleanly.

### What you get

Repository mode returns one clean JSON record per repo:

```json
{
  "type": "repository",
  "id": 20929025,
  "name": "TypeScript",
  "fullName": "microsoft/TypeScript",
  "owner": "microsoft",
  "ownerType": "Organization",
  "description": "TypeScript is a superset of JavaScript that compiles to clean JavaScript output.",
  "url": "https://github.com/microsoft/TypeScript",
  "homepage": "https://www.typescriptlang.org",
  "stars": 99123,
  "forks": 12987,
  "watchers": 99123,
  "openIssues": 5721,
  "language": "TypeScript",
  "topics": ["typescript", "javascript", "compiler"],
  "license": "Apache-2.0",
  "isFork": false,
  "isArchived": false,
  "size": 1234567,
  "defaultBranch": "main",
  "createdAt": "2014-06-17T15:28:39Z",
  "updatedAt": "2024-05-30T08:11:02Z",
  "pushedAt": "2024-05-30T07:59:41Z"
}
```

In `users` mode each record instead includes `login`, `name`, `company`, `location`, `email`, `blog`, `twitter`, `hireable`, `followers`, `following`, `publicRepos`, `publicGists` and `bio`.

### Input

```json
{
  "mode": "repositories",
  "query": "stars:>10000 language:typescript",
  "maxItems": 100,
  "sort": "stars",
  "order": "desc",
  "enrichUserProfiles": true,
  "githubToken": ""
}
```

| Field | Description |
|-------|-------------|
| `mode` | Scrape `repositories` or `users` / profiles. |
| `query` | A GitHub search query, e.g. `stars:>1000 language:python topic:ai` or `location:berlin language:go`. |
| `maxItems` | Max repositories or users to return (GitHub caps search at 1000 per query). |
| `sort` / `order` | Sort field (stars, forks, followers, joined…) and direction (`desc` or `asc`). |
| `enrichUserProfiles` | In users mode, fetch each user's full profile (email, company, followers, bio). |
| `githubToken` | Optional personal access token for higher rate limits; no scopes needed for public data. |

### Use cases

- Build developer lead lists from user profiles filtered by location, language and followers.
- Research the top repositories in a technology or topic for competitive analysis.
- Source open-source maintainers with public emails for outreach or recruiting.
- Track stars, forks and activity across a curated set of projects over time.

### Pricing

Pay-per-event: you're billed a small amount per run and per item scraped — you only pay for what you get. See the **Pricing** tab for current rates.

### FAQ

**Do I need a GitHub token or API key?**
No. The actor works out of the box against the official [GitHub](https://github.com/) REST API with no login. A personal access token is optional — supply one to raise the rate limit to about 5000 requests/hour; no scopes are needed for public data.

**How many repositories or users can I scrape per run?**
Up to 1000 per query — GitHub caps its search API at 1000 results per query. Use `maxItems` to set your own lower limit and refine the `query` to stay within that window.

**Is scraping GitHub allowed?**
This actor uses GitHub's own public REST API and reads only publicly available data, respecting rate-limit resets and retrying on 403/429. Review GitHub's Acceptable Use Policies before large-scale or commercial reuse, especially of profile emails.

**What's the output format?**
Structured JSON — in `repositories` mode, one record per repo (`stars`, `forks`, `language`, `topics`, `license`, dates); in `users` mode, one profile per user (`login`, `company`, `location`, `email`, `followers`, `bio`). Export as JSON, CSV, Excel or via the API.

**Can I filter by stars, language or location?**
Yes. Use GitHub's query qualifiers directly, e.g. `stars:>10000 language:typescript` for repos or `location:berlin followers:>500` for users, and sort by stars, forks, updated, followers or joined.

### Related Actors

Doing developer research, lead-gen or recruiting? Combine this with our other developer-data scrapers:

- [npm Package Scraper](https://apify.com/hipersoft/npm-scraper) — search and look up npm packages with metadata, dependencies and downloads.
- [PyPI Package Scraper](https://apify.com/hipersoft/pypi-scraper) — metadata, dependencies, licenses and downloads for Python packages.
- [Stack Overflow & Stack Exchange Scraper](https://apify.com/hipersoft/stackexchange-scraper) — questions by tag or search across Stack Exchange sites.
- [Hacker News Scraper](https://apify.com/hipersoft/hacker-news-scraper) — stories, comments and Ask/Show HN by keyword and filters.

### Notes

This actor uses the public GitHub REST API and reads only publicly available data. It is an independent tool and is not affiliated with, endorsed by, or sponsored by GitHub, Inc. or Microsoft; "GitHub" is a trademark of GitHub, Inc.

# Actor input Schema

## `mode` (type: `string`):

Search repositories or user profiles.

## `query` (type: `string`):

A GitHub search query. Repos: "stars:>1000 language:python topic:ai". Users: "location:berlin language:go followers:>500".

## `maxItems` (type: `integer`):

Max repositories or users to return (GitHub search caps at 1000 per query).

## `sort` (type: `string`):

Repos: stars, forks, help-wanted-issues, updated. Users: followers, repositories, joined. Empty = best match.

## `order` (type: `string`):

desc or asc.

## `enrichUserProfiles` (type: `boolean`):

For users mode: fetch each user's full public profile (email, company, followers, bio). Costs one extra request per user.

## `githubToken` (type: `string`):

Optional personal access token for higher rate limits (~30 search req/min, 5000/hr vs ~10/min unauthenticated). No scopes needed for public data.

## `proxyConfiguration` (type: `object`):

Optional. The GitHub API works without a proxy.

## Actor input object example

```json
{
  "mode": "repositories",
  "query": "stars:>10000 language:typescript",
  "maxItems": 100,
  "sort": "stars",
  "order": "desc",
  "enrichUserProfiles": true,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}
```

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "query": "stars:>10000 language:typescript"
};

// Run the Actor and wait for it to finish
const run = await client.actor("hipersoft/github-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "query": "stars:>10000 language:typescript" }

# Run the Actor and wait for it to finish
run = client.actor("hipersoft/github-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "query": "stars:>10000 language:typescript"
}' |
apify call hipersoft/github-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "command": "npx",
            "args": [
                "mcp-remote",
                "https://mcp.apify.com/?tools=hipersoft/github-scraper",
                "--header",
                "Authorization: Bearer <YOUR_API_TOKEN>"
            ]
        }
    }
}

```

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/CoqrH3fexiCXnKJTw/builds/nOozvHreuI7tOcLBh/openapi.json