# GitLab Public Projects Scraper | Stars, Forks, Topics (`parseforge/gitlab-public-projects-scraper`) Actor
Harvest records from multiple Gitlab sources in a single run and get a unified, normalized result set. Pull names, identifiers, dates, descriptions, status flags and source links per record. Perfect for research, lead generation and intelligence pipelines.
- **URL**: https://apify.com/parseforge/gitlab-public-projects-scraper.md
- **Developed by:** [ParseForge](https://apify.com/parseforge) (community)
- **Categories:** Developer tools, Automation
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet
## Pricing
from $19.00 / 1,000 results
This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.
Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event
## What's an Apify Actor?
Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".
## How to integrate an Actor?
If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.
In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md):
```bash
npm install apify-client
```
In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md):
```bash
pip install apify-client
```
In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md):
````bash
# MacOS / Linux
curl -fsSL https://apify.com/install-cli.sh | bash
# Windows
irm https://apify.com/install-cli.ps1 | iex
```bash
In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md).
If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md).
For usage examples, see the [API](#api) section below.
For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).
# README
## ๐ฆ GitLab Public Projects Scraper
> ๐ **Pull public GitLab projects with stars, forks, topics, and owners in seconds.** Built on the official GitLab REST API.
> ๐ **Last updated:** 2026-05-27 ยท **๐ 25 fields** per record ยท **All public gitlab.com projects** ยท **Search, sort, and topic filters**
The GitLab Public Projects Scraper queries the official `gitlab.com/api/v4/projects` endpoint and returns one normalized record per public project. Useful for tracking open-source DevOps tooling, discovering self-hosted alternatives to GitHub repos, monitoring topic communities (kubernetes, gitops, AI), or building competitive intelligence on the open-source ecosystem.
Coverage: every public project on gitlab.com. Filters by search query, topic, and sort order (stars, last activity, created date). Up to 1,000,000 records per run on the paid plan.
| ๐ฏ Target Audience | ๐ก Primary Use Cases |
|---|---|
| Developer relations | Map ecosystem of related OSS projects |
| Security teams | Find packages by topic or maintainer |
| Recruiters | Spot top GitLab contributors |
| Researchers | Study OSS contribution patterns |
### ๐ What the GitLab Public Projects Scraper does
- Queries the public GitLab REST API directly
- Returns 25 normalized fields per project (name, path, stars, forks, topics, license, owner...)
- Supports search, sort, topic, and ascending/descending order
- Outputs to multiple table outputs via Apify dataset
- Auto-limits to 10 items on the free plan; up to 1,000,000 on paid
> ๐ก **Why it matters:** GitLab hosts millions of projects but no public search UI exposes them at scale. This Actor turns that catalog into a queryable dataset.
### ๐ฌ Full Demo (_๐ง Coming soon_)
### โ๏ธ Input
Field
Type
Description
search
string
Search term, e.g. react
maxItems
integer
Cap on rows (free: 10)
orderBy
enum
star_count, last_activity_at, created_at, etc.
sort
enum
asc / desc
topic
string
Filter by topic, e.g. kubernetes
```json
{ "search": "react", "maxItems": 100, "orderBy": "star_count", "sort": "desc" }
````
```json
{ "topic": "kubernetes", "orderBy": "last_activity_at", "maxItems": 200 }
```
> โ ๏ธ **Good to Know:** Without a token, GitLab's API rate-limits to ~10 requests per second per IP. The Actor paces requests automatically.
### ๐ Output
| Field | Type | Description |
|---|---|---|
| ๐ผ imageUrl | string | Project avatar URL |
| ๐ name | string | Project name |
| ๐ nameWithNamespace | string | Full namespaced name |
| ๐ url | string | gitlab.com web URL |
| ๐ id | integer | GitLab project ID |
| ๐ path | string | URL slug |
| ๐ pathWithNamespace | string | Full path with namespace |
| ๐ description | string | Project description |
| ๐ฟ defaultBranch | string | Default branch name |
| ๐ visibility | string | public / internal |
| โญ starCount | number | Stars |
| ๐ด forksCount | number | Forks |
| ๐ openIssuesCount | number | Open issues |
| ๐ท topics | array | Topics list |
| ๐ท tagList | array | Tags |
| ๐ license | string | License name |
| ๐ createdAt | string | Created ISO timestamp |
| ๐ lastActivityAt | string | Last activity ISO timestamp |
| ๐ readmeUrl | string | README URL |
| ๐ sshUrl | string | SSH clone URL |
| ๐ httpUrl | string | HTTPS clone URL |
| ๐ค owner | string | Namespace name |
| ๐ค ownerPath | string | Namespace path |
| ๐ค ownerKind | string | user / group |
| ๐ scrapedAt | string | ISO timestamp |
| โ error | string | null | Error message if extraction failed |
### โจ Why choose this Actor
- Direct GitLab REST API, no third-party caching
- Real-time data, never stale
- Pay-per-result pricing; only charged for what you keep
- Works with Make, Zapier, n8n, Airbyte, GitHub Actions, Google Sheets
### ๐ How it compares to alternatives
| Approach | Cost | Maintenance | Coverage |
|---|---|---|---|
| GitLab UI search | Free | Manual | One page at a time |
| Self-built API client | Dev cost | High | Custom |
| This actor | Pay per result | None | Full search + filters |
### ๐ How to use
1. [Create a free Apify account w/ $5 credit](https://console.apify.com/sign-up?fpr=vmoqkp)
2. Open the GitLab Public Projects Scraper actor page
3. Set `search`, `orderBy`, optional `topic`, and `maxItems`
4. Click **Start** and wait for the run to finish
5. Use the dataset as multiple table outputs
### ๐ผ Business use cases
#### Developer relations
| Need | How this Actor helps |
|---|---|
| Ecosystem mapping | Pull all projects by topic |
| Influencer tracking | Sort by stars to find top owners |
#### Security
| Need | How this Actor helps |
|---|---|
| Supply chain audit | Scan public projects in your topic |
| License compliance | Filter by license field |
#### Recruiting
| Need | How this Actor helps |
|---|---|
| Find OSS contributors | Sort by activity, drill into owner |
| Skill mapping | Filter by topic, language |
#### Research
| Need | How this Actor helps |
|---|---|
| OSS trends | Bulk pull over time |
| Comparative analysis | GitLab vs GitHub trends |
### ๐ Automating GitLab Public Projects Scraper
Run on a schedule, forward results to Make, Zapier, n8n, Slack, Airbyte, GitHub Actions, or Google Drive. Push new high-star projects into a Slack channel daily.
### ๐ Beyond business use cases
#### Research
Build a longitudinal study of OSS topic growth over months/years.
#### Personal
Discover obscure self-hosted alternatives to your favourite SaaS tools.
#### Non-profit
Track open civic-tech projects by topic (e.g. `civic-tech`, `accessibility`).
#### Experimentation
Build a "GitLab radar" Slack bot for newly active projects in your space.
### ๐ค Ask an AI assistant about this scraper
- [ChatGPT](https://chatgpt.com/)
- [Claude](https://claude.ai/)
- [Perplexity](https://www.perplexity.ai/)
- [Microsoft Copilot](https://copilot.microsoft.com/)
### โ Frequently Asked Questions
**Q: Does it need a GitLab token?** No. Public endpoints work anonymously.
**Q: Can I scrape private projects?** No - only `visibility: public` is returned.
**Q: How fresh is the data?** Live - every request hits gitlab.com.
**Q: Can I filter by language?** Use the `topic` field - many projects tag languages (e.g. `python`, `rust`).
**Q: What's the rate limit?** ~10 RPS anonymous; the Actor paces well under it.
**Q: Can I scrape self-hosted GitLab?** This Actor targets gitlab.com specifically. Self-hosted instances would need a forked Actor.
**Q: Does it include archived projects?** Yes - check `archived` field if needed.
**Q: How do I find a topic slug?** Browse `gitlab.com/explore/projects/topics`.
**Q: Can I sort by forks?** `orderBy` supports `id`, `name`, `path`, `created_at`, `updated_at`, `last_activity_at`, `star_count`. Forks not supported by upstream API.
**Q: Does it return README contents?** No - just `readmeUrl`. Fetch separately if needed.
### ๐ Integrate with any app
Slack, Discord, Sheets, Airtable, BigQuery, S3, Snowflake, and 100+ more via Apify webhooks.
### ๐ Recommended Actors
| Actor | What it does |
|---|---|
| GitHub Trending Scraper | Daily trending repos |
| Hacker News Scraper | Top tech stories |
| npm Packages Scraper | npm metadata |
| Mastodon Trends Scraper | Fediverse trends |
> ๐ก **Pro Tip:** browse the complete [ParseForge collection](https://apify.com/parseforge).
**๐ Need Help?** [Open our contact form](https://tally.so/r/BzdKgA)
> **โ ๏ธ Disclaimer:** independent tool, not affiliated with GitLab Inc. Only publicly available data is collected.
# Actor input Schema
## `search` (type: `string`):
Search term (project name / description). Leave blank to list newest public projects.
## `maxItems` (type: `integer`):
Free users: Limited to 10 items (preview). Paid users: Optional, max 1,000,000
## `orderBy` (type: `string`):
Field to sort results by.
## `sort` (type: `string`):
Sort direction (asc / desc).
## `topic` (type: `string`):
Filter projects by topic (e.g. 'kubernetes'). Optional.
## Actor input object example
```json
{
"search": "react",
"maxItems": 10,
"orderBy": "star_count",
"sort": "desc",
"topic": ""
}
```
# Actor output Schema
## `results` (type: `string`):
No description
# API
You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.
## JavaScript example
```javascript
import { ApifyClient } from 'apify-client';
// Initialize the ApifyClient with your Apify API token
// Replace the '' with your token
const client = new ApifyClient({
token: '',
});
// Prepare Actor input
const input = {
"maxItems": 10
};
// Run the Actor and wait for it to finish
const run = await client.actor("parseforge/gitlab-public-projects-scraper").call(input);
// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`๐พ Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
console.dir(item);
});
// ๐ Want to learn more ๐? Go to โ https://docs.apify.com/api/client/js/docs
```
## Python example
```python
from apify_client import ApifyClient
# Initialize the ApifyClient with your Apify API token
# Replace '' with your token.
client = ApifyClient("")
# Prepare the Actor input
run_input = { "maxItems": 10 }
# Run the Actor and wait for it to finish
run = client.actor("parseforge/gitlab-public-projects-scraper").call(run_input=run_input)
# Fetch and print Actor results from the run's dataset (if there are any)
print("๐พ Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
print(item)
# ๐ Want to learn more ๐? Go to โ https://docs.apify.com/api/client/python/docs/quick-start
```
## CLI example
```bash
echo '{
"maxItems": 10
}' |
apify call parseforge/gitlab-public-projects-scraper --silent --output-dataset
```
## MCP server setup
```json
{
"mcpServers": {
"apify": {
"command": "npx",
"args": [
"mcp-remote",
"https://mcp.apify.com/?tools=parseforge/gitlab-public-projects-scraper",
"--header",
"Authorization: Bearer "
]
}
}
}
```
## OpenAPI specification
```json
{
"openapi": "3.0.1",
"info": {
"title": "GitLab Public Projects Scraper | Stars, Forks, Topics",
"description": "Harvest records from multiple Gitlab sources in a single run and get a unified, normalized result set. Pull names, identifiers, dates, descriptions, status flags and source links per record. Perfect for research, lead generation and intelligence pipelines.",
"version": "0.1",
"x-build-id": "K6w8x6Wfw3KpIell6"
},
"servers": [
{
"url": "https://api.apify.com/v2"
}
],
"paths": {
"/acts/parseforge~gitlab-public-projects-scraper/run-sync-get-dataset-items": {
"post": {
"operationId": "run-sync-get-dataset-items-parseforge-gitlab-public-projects-scraper",
"x-openai-isConsequential": false,
"summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.",
"tags": [
"Run Actor"
],
"requestBody": {
"required": true,
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/inputSchema"
}
}
}
},
"parameters": [
{
"name": "token",
"in": "query",
"required": true,
"schema": {
"type": "string"
},
"description": "Enter your Apify token here"
}
],
"responses": {
"200": {
"description": "OK"
}
}
}
},
"/acts/parseforge~gitlab-public-projects-scraper/runs": {
"post": {
"operationId": "runs-sync-parseforge-gitlab-public-projects-scraper",
"x-openai-isConsequential": false,
"summary": "Executes an Actor and returns information about the initiated run in response.",
"tags": [
"Run Actor"
],
"requestBody": {
"required": true,
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/inputSchema"
}
}
}
},
"parameters": [
{
"name": "token",
"in": "query",
"required": true,
"schema": {
"type": "string"
},
"description": "Enter your Apify token here"
}
],
"responses": {
"200": {
"description": "OK",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/runsResponseSchema"
}
}
}
}
}
}
},
"/acts/parseforge~gitlab-public-projects-scraper/run-sync": {
"post": {
"operationId": "run-sync-parseforge-gitlab-public-projects-scraper",
"x-openai-isConsequential": false,
"summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.",
"tags": [
"Run Actor"
],
"requestBody": {
"required": true,
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/inputSchema"
}
}
}
},
"parameters": [
{
"name": "token",
"in": "query",
"required": true,
"schema": {
"type": "string"
},
"description": "Enter your Apify token here"
}
],
"responses": {
"200": {
"description": "OK"
}
}
}
}
},
"components": {
"schemas": {
"inputSchema": {
"type": "object",
"properties": {
"search": {
"title": "Search Query",
"type": "string",
"description": "Search term (project name / description). Leave blank to list newest public projects.",
"default": "react"
},
"maxItems": {
"title": "Max Items",
"minimum": 1,
"maximum": 1000000,
"type": "integer",
"description": "Free users: Limited to 10 items (preview). Paid users: Optional, max 1,000,000"
},
"orderBy": {
"title": "Order By",
"enum": [
"id",
"name",
"path",
"created_at",
"updated_at",
"last_activity_at",
"star_count"
],
"type": "string",
"description": "Field to sort results by.",
"default": "star_count"
},
"sort": {
"title": "Sort Direction",
"enum": [
"asc",
"desc"
],
"type": "string",
"description": "Sort direction (asc / desc).",
"default": "desc"
},
"topic": {
"title": "Topic Filter",
"type": "string",
"description": "Filter projects by topic (e.g. 'kubernetes'). Optional.",
"default": ""
}
}
},
"runsResponseSchema": {
"type": "object",
"properties": {
"data": {
"type": "object",
"properties": {
"id": {
"type": "string"
},
"actId": {
"type": "string"
},
"userId": {
"type": "string"
},
"startedAt": {
"type": "string",
"format": "date-time",
"example": "2025-01-08T00:00:00.000Z"
},
"finishedAt": {
"type": "string",
"format": "date-time",
"example": "2025-01-08T00:00:00.000Z"
},
"status": {
"type": "string",
"example": "READY"
},
"meta": {
"type": "object",
"properties": {
"origin": {
"type": "string",
"example": "API"
},
"userAgent": {
"type": "string"
}
}
},
"stats": {
"type": "object",
"properties": {
"inputBodyLen": {
"type": "integer",
"example": 2000
},
"rebootCount": {
"type": "integer",
"example": 0
},
"restartCount": {
"type": "integer",
"example": 0
},
"resurrectCount": {
"type": "integer",
"example": 0
},
"computeUnits": {
"type": "integer",
"example": 0
}
}
},
"options": {
"type": "object",
"properties": {
"build": {
"type": "string",
"example": "latest"
},
"timeoutSecs": {
"type": "integer",
"example": 300
},
"memoryMbytes": {
"type": "integer",
"example": 1024
},
"diskMbytes": {
"type": "integer",
"example": 2048
}
}
},
"buildId": {
"type": "string"
},
"defaultKeyValueStoreId": {
"type": "string"
},
"defaultDatasetId": {
"type": "string"
},
"defaultRequestQueueId": {
"type": "string"
},
"buildNumber": {
"type": "string",
"example": "1.0.0"
},
"containerUrl": {
"type": "string"
},
"usage": {
"type": "object",
"properties": {
"ACTOR_COMPUTE_UNITS": {
"type": "integer",
"example": 0
},
"DATASET_READS": {
"type": "integer",
"example": 0
},
"DATASET_WRITES": {
"type": "integer",
"example": 0
},
"KEY_VALUE_STORE_READS": {
"type": "integer",
"example": 0
},
"KEY_VALUE_STORE_WRITES": {
"type": "integer",
"example": 1
},
"KEY_VALUE_STORE_LISTS": {
"type": "integer",
"example": 0
},
"REQUEST_QUEUE_READS": {
"type": "integer",
"example": 0
},
"REQUEST_QUEUE_WRITES": {
"type": "integer",
"example": 0
},
"DATA_TRANSFER_INTERNAL_GBYTES": {
"type": "integer",
"example": 0
},
"DATA_TRANSFER_EXTERNAL_GBYTES": {
"type": "integer",
"example": 0
},
"PROXY_RESIDENTIAL_TRANSFER_GBYTES": {
"type": "integer",
"example": 0
},
"PROXY_SERPS": {
"type": "integer",
"example": 0
}
}
},
"usageTotalUsd": {
"type": "number",
"example": 0.00005
},
"usageUsd": {
"type": "object",
"properties": {
"ACTOR_COMPUTE_UNITS": {
"type": "integer",
"example": 0
},
"DATASET_READS": {
"type": "integer",
"example": 0
},
"DATASET_WRITES": {
"type": "integer",
"example": 0
},
"KEY_VALUE_STORE_READS": {
"type": "integer",
"example": 0
},
"KEY_VALUE_STORE_WRITES": {
"type": "number",
"example": 0.00005
},
"KEY_VALUE_STORE_LISTS": {
"type": "integer",
"example": 0
},
"REQUEST_QUEUE_READS": {
"type": "integer",
"example": 0
},
"REQUEST_QUEUE_WRITES": {
"type": "integer",
"example": 0
},
"DATA_TRANSFER_INTERNAL_GBYTES": {
"type": "integer",
"example": 0
},
"DATA_TRANSFER_EXTERNAL_GBYTES": {
"type": "integer",
"example": 0
},
"PROXY_RESIDENTIAL_TRANSFER_GBYTES": {
"type": "integer",
"example": 0
},
"PROXY_SERPS": {
"type": "integer",
"example": 0
}
}
}
}
}
}
}
}
}
}
```