# GitHub Documentation Extractor (Agentic) (`himanshi1rana/github-docs-intelligence`) Actor

An agentic AI actor that automatically extracts and analyzes documentation from GitHub repositories to help developers understand projects faster.

- **URL**: https://apify.com/himanshi1rana/github-docs-intelligence.md
- **Developed by:** [Himanshi Rana](https://apify.com/himanshi1rana) (community)
- **Categories:** AI, Developer tools, Agents
- **Stats:** 3 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

Pay per usage

This Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage, which gets cheaper the higher subscription plan you have.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-usage

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## 🤖 GitHub Documentation Intelligence

> AI-powered documentation extraction and analysis for GitHub repositories

Extract, structure, and analyze documentation from any GitHub repository in seconds. Perfect for building RAG systems, onboarding developers, and auditing documentation quality.

***

### 🎯 What It Does

Automatically extracts and structures:

- ✅ **README files** - Main project documentation
- ✅ **Documentation folders** - All markdown files from docs/, documentation/, etc.
- ✅ **Code documentation** - Docstrings from Python, JavaScript, TypeScript files
- ✅ **Metadata** - Repository info, stars, language, topics
- ✅ **Statistics** - Word counts, file counts, documentation coverage

***

### 🚀 Quick Start

#### Input Example

```json
{
  "url": "https://github.com/pallets/flask",
  "maxFiles": 20,
  "extractCodeDocs": true
}

```

#### Output Example

```json
{
  "status": "success",
  "metadata": {
    "name": "flask",
    "description": "The Python micro framework",
    "language": "Python",
    "stars": 65000,
    "url": "https://github.com/pallets/flask"
  },
  "readme": {
    "filename": "README.md",
    "content": "...",
    "sections": [...],
    "word_count": 450
  },
  "documentation_files": [...],
  "code_documentation": [...],
  "combined_markdown": "...",
  "statistics": {
    "has_readme": true,
    "documentation_files_count": 23,
    "code_files_with_docs": 15,
    "total_words": 12500,
    "total_docstrings": 87
  }
}
```

***

### ⭐ Key Features

#### 📊 Comprehensive Extraction

- Extracts README, docs folders, and code docstrings
- Supports Python, JavaScript, TypeScript
- Handles nested documentation structures
- Preserves markdown formatting and sections

#### 🎯 Structured Output

- Clean JSON format ready for processing
- Pydantic models for type safety
- Combined markdown for easy reading
- Detailed statistics and metadata

#### 🛡️ Robust & Reliable

- Proper error handling
- Rate limit management
- Partial success handling
- Detailed logging

#### ⚡ Fast & Efficient

- Async operations
- Smart file filtering
- Configurable limits
- Optimized API usage

***

### 💡 Use Cases

#### 🤖 RAG Systems

Extract clean documentation for training AI models:

```python
## Use extracted docs for embeddings
docs = result['combined_markdown']
chunks = create_embeddings(docs)
```

#### 👨‍💻 Developer Onboarding

Generate comprehensive repo overviews:

- Understand project structure
- Find key documentation
- Identify important files

#### 📈 Documentation Audits

Analyze documentation quality:

- Check completeness
- Identify gaps
- Track improvements

#### 🔍 Code Search

Enable semantic search over codebases:

- Search through docstrings
- Find relevant code examples
- Understand APIs

***

### 🔧 Configuration

#### GitHub Token (Recommended)

For private repos and higher rate limits (5,000 vs 60 requests/hour):

1. Go to https://github.com/settings/tokens
2. Generate new token (classic)
3. Select scopes: `repo` or `public_repo`
4. Add to input: `"githubToken": "ghp_your_token"`

#### Options

| Option | Type | Default | Description |
|--------|------|---------|-------------|
| `maxFiles` | integer | 100 | Maximum files to process |
| `extractCodeDocs` | boolean | true | Extract code docstrings |

***

### 📊 Statistics Provided

- **has\_readme**: Whether README exists
- **documentation\_files\_count**: Number of doc files found
- **code\_files\_with\_docs**: Number of code files with docstrings
- **total\_words**: Total documentation words
- **total\_lines**: Total documentation lines
- **total\_docstrings**: Total docstrings extracted

***

### 🛠️ Development

#### Local Testing

```bash
## Install dependencies
pip install -r requirements.txt

## Run locally
apify run
```

#### Project Structure

```
.
├── src/
│   ├── main.py           # Actor entry point
│   ├── extractor.py      # Extraction logic
│   ├── models.py         # Data models
│   └── utils.py          # Helper functions
├── .actor/
│   ├── actor.json        # Actor configuration
│   └── input_schema.json # Input schema
├── requirements.txt      # Dependencies
└── Dockerfile           # Container config
```

***

### 🤝 Contributing

Issues and pull requests welcome! This is an active project participating in the Apify $1M Challenge.

***

### 📝 License

Apache 2.0

***

### 💬 Support

- **Questions?** Join [Apify Discord](https://discord.com/invite/jyEM2PRvMU)
- **Issues?** Open a GitHub issue
- **Need help?** Check Apify documentation

***

### 🎯 Coming Soon

- 🔜 Documentation quality scoring (A-F grades)
- 🔜 MCP server for AI agents
- 🔜 Change detection and tracking
- 🔜 Multi-repo comparison
- 🔜 PDF documentation support
- 🔜 Website documentation scraping

\##FAQs
Q: Why did extraction fail?
A: Common reasons:
1.Repository doesn't exist (check URL)
2.Repository is private (add GitHub token)
3.Rate limit exceeded (add token for 5000/hour)
4.Repository is too large (reduce maxFiles)

Q: What if I hit rate limits?
A:

Without token: 60 requests/hour
With token: 5,000 requests/hour
Get token: https://github.com/settings/tokens

Q: Can I extract from private repos?
A: Yes! Add your GitHub token in the input:
json{
"source": {
"url": "...",
"githubToken": "ghp\_your\_token"
}
}
Q: What's the maximum repository size?
A:

1.Max 500 files per run
2.Max 5MB per file
3.Max 50MB total data
4.Adjust maxFiles if needed

Q: Why are some files skipped?
A: Files are skipped if they:
1.Are too large (>5MB)
2.Can't be decoded (binary files)
3.Cause encoding errors

Q: How long does extraction take?
A:
1.Small repos (<100 files): 2-5 seconds
2.Medium repos (100-500 files): 10-30 seconds
3.Large repos (500+ files): 30-60 seconds
4.Max timeout: 4 minutes

***

**Built with ❤️ for the Apify $1M Challenge**

⭐ If you find this useful, please star the Actor!

# Actor input Schema

## `source` (type: `object`):

Specify the GitHub repository to extract documentation from

## `options` (type: `object`):

Configure what to extract and how

## Actor input object example

```json
{}
```

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {};

// Run the Actor and wait for it to finish
const run = await client.actor("himanshi1rana/github-docs-intelligence").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {}

# Run the Actor and wait for it to finish
run = client.actor("himanshi1rana/github-docs-intelligence").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{}' |
apify call himanshi1rana/github-docs-intelligence --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "command": "npx",
            "args": [
                "mcp-remote",
                "https://mcp.apify.com/?tools=himanshi1rana/github-docs-intelligence",
                "--header",
                "Authorization: Bearer <YOUR_API_TOKEN>"
            ]
        }
    }
}

```

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/xIeYRrLr6QDnLmYE9/builds/J3OhCYalAv56RVMri/openapi.json