reach-mcp

A controllable multi-source search MCP server for AI agents. Search Reddit, X, YouTube, Hacker News, GitHub, arXiv, Polymarket, 雪球, V2EX, B站, 小宇宙 and more -- you pick the sources, the window, and whether to synthesize. Built to replace the closed last30days server with an open, agent-driven one.

Why

last30days is a black box: the agent passes a query and gets back a finished brief. It can't choose which sources to hit, can't widen or narrow the time window (hardcoded to 30 days), can't see the raw scored rows, and can't reuse them. Every call re-searches everything.

reach-mcp keeps everything good about last30days -- parallel multi-source fetch, engagement-based scoring, cross-source dedup/clustering, an optional LLM-synthesized cited brief -- and hands the steering wheel to the agent:

• 🔌 Pick your sources. sources=["reddit","arxiv","xueqiu"] or omit for all configured ones.

• 📅 Pick your window. days=7 for this week, days=180 for the half-year -- no longer fixed at 30.

• 🪓 Decide what matters. synthesize=false returns raw scored rows for the agent to reason over itself; synthesize=true (default) also runs an LLM rerank + brief.

• 🇨🇳 Chinese sources last30days lacks -- 雪球, V2EX, B站, 小宇宙, 小红书 -- alongside nearly every source last30days ships.

• 🛡️ Polite by default -- per-host pacing, honors Retry-After, bounded timeouts. Never hammers a site.

Related MCP server: MetaSearchMCP

Sources (25)

💰 Apify gives $5 free credits EVERY MONTH (recurring, not one-time) on the Free plan -- enough for hundreds of search runs. Set APIFY_API_TOKEN to enable threads + boost tiktok/instagram/pinterest (Apify is the preferred backend; OpenCLI is a free desktop alternative; ScrapeCreators is a one-time-credit fallback).

⚠️ ScrapeCreators is 100 credits one-time, not free recurring. It's now the lowest-priority fallback for tiktok/instagram/pinterest -- prefer Apify (monthly free) or OpenCLI (desktop, free). LinkedIn uses Jina (free JINA_API_KEY = monthly rate-limit quota).

Query syntax by source

Each source has its own search-query rules. Multi-word/long queries may return fewer results on some backends — the agent should tune the query per source.

General guidance: prefer short, specific keywords (2-4 words). Very long queries or phrases that only a specific site would phrase identically will often return [] — the backend isn't broken, the query is too specific.

MCP tools

search -- the primary tool

Description:

Search up to 25 social & web sources in parallel, score by engagement, optionally synthesize a cited brief. YOU control scope.

Sources (pass any subset as sources; omit/None = all currently-configured): free -- reddit, hackernews, bluesky, github, arxiv, techmeme, polymarket, stocktwits, web, dripstack, rss; video -- youtube; chinese -- xueqiu, v2ex, bilibili, xiaoyuzhou, xiaohongshu; login-gated (off until creds set) -- x, truthsocial, linkedin, tiktok, instagram, pinterest; binary -- digg; apify -- threads.

Args: query (str, the topic/person/ticker); sources (list[str] | None, None=all available); days (int, recency window, default 30); max_per_source (int, row cap per source, default 20); synthesize (bool, default true = also LLM-rerank + write brief).

Returns: {brief: str|null, items: [Item], sources_used: [SourceReport], available_sources: [str]}. Each Item: {source, title, url, author, date, score, engagement, text}. A SourceReport tells you per-source ok/gated_off/errored so a thin result is diagnosable. Call list_sources first if unsure what's configured.

list_sources

List all registered sources with availability status, required credentials, and defaults. Call this FIRST to see which sources are active (credentials set) vs gated (off-by-default) before deciding sources. Returns [{name, description, needs_auth, available, required_env, default_days, default_limit}]. No arguments.

synthesize

Re-synthesize a cited brief from already-fetched items WITHOUT re-searching. Pass the original query and the items list from a prior search(synthesize=false). Returns {brief}. Use to re-brief cheaply with different emphasis. No source calls are made.

read_url

Read the content of any URL as clean text (via Jina Reader, free). Use this to fetch and analyze a page you found in search results -- a Reddit thread, news article, blog post, or GitHub readme -- when you need the full text beyond the snippet. Args: url (str). Returns {url, content, ok}. Keyless at 20 RPM; set JINA_API_KEY for 500 RPM.

For agents -- quick usage guide

1. list_sources()                         # see what's configured; pick targets
2. search("Peter Steinberger",
          sources=["reddit","x","github","youtube"],
          days=14, synthesize=false)      # get raw scored rows, reason yourself
   -- or --
   search("OpenAI vs Anthropic",
          days=30, synthesize=true)        # one call → cited brief + rows
3. (optional) synthesize(query, items)     # re-brief the rows you already have

• Default (sources=None, synthesize=true): searches every configured source and returns a brief + all rows. Simplest.

• Targeted (sources=[...]): only hit what you need -- faster, cheaper, less noise.

• Raw (synthesize=false): you read the rows and draw conclusions; re-brief later with synthesize.

• A gated source you name returns gated_off in sources_used -- set its credential env and retry, or drop it.

• One broken source never breaks a search; check sources_used for what failed.

Install

Option A -- uvx in your MCP host (recommended for OpenWebUI / mcpo)

// mcpo config.json
"reach": {
  "command": "uvx",
  "args": ["reach-mcp", "--transport", "stdio"]
}

Or streamable-HTTP: uvx reach-mcp --transport http --host 0.0.0.0 --port 8765.

Option B -- Docker

docker pull ghcr.io/xyonium/reach-mcp:latest
docker run -p 8765:8765 --env-file .env ghcr.io/xyonium/reach-mcp:latest

See docker-compose.yml for a full example with all available env vars (including optional xiaohongshu-mcp companion service).

Option C -- mcpo (multi-server host, recommended for OpenWebUI)

mcpo launches many MCP servers via uvx and exposes each as an OpenAPI endpoint. reach-mcp is a 1:1 drop-in replacement for the closed last30days server -- just swap the config.json entry.

1. config.json -- list reach-mcp (see deploy/mcpo-config.example.json for the full env set):

{
  "reach": {
    "command": "uvx",
    "args": ["reach-mcp", "--transport", "stdio"],
    "env": { "SEARXNG_URL": "http://searxng:8080", "OPENAI_API_KEY": "sk-..." }
  }
}

2. entrypoint.sh -- deploy/entrypoint.sh installs reach-mcp's runtime deps on first start (cached under the mounted volume): yt-dlp (youtube), bili-cli (bilibili - handles B站 wbi/412), gh (github), Go + digg/arxiv/techmeme pp-cli. Copy it into your mcpo config dir.

⚠️ mcp SDK 2.x pin: mcpo 0.0.20 (and several uvx-launched MCP servers) import 1.x mcp SDK symbols that mcp 2.0 renamed, and crash with ImportError: cannot import name 'streamablehttp_client'... / McpError. The entrypoint sets UV_CONSTRAINT=/config/uv-constraints.txt (mcp<2), which mcpo inherits into every child uvx process. Ship deploy/uv-constraints.txt next to your entrypoint.sh to customize; if absent, the entrypoint writes the same default.

3. compose -- deploy/docker-compose.mcpo.yml mirrors a production setup with UV_CACHE_DIR/UV_TOOL_DIR/npm_config_cache persistence, plus optional Searxng and xiaohongshu-mcp companion services:

mcp:
  image: ghcr.io/open-webui/mcpo:latest
  volumes:
    - ./mcpo-config:/config        # entrypoint.sh + config.json
  environment:
    UV_CACHE_DIR: /config/uv-cache
    UV_TOOL_DIR: /config/uv-tools
    UV_TOOL_BIN_DIR: /config/uv-bin
    PATH: /config/uv-bin:/config/bin:/config/go/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
    npm_config_cache: /config/npm-cache
  entrypoint: ["/bin/bash", "/config/entrypoint.sh"]
  command: ["--host", "0.0.0.0", "--port", "8000", "--config", "/config/config.json"]

OpenWebUI then connects to http://mcp:8000/reach (OpenAPI) or http://mcp:8000/reach/mcp (native MCP).

Upgrading deps: touch /config/UPGRADE in the config dir, then restart the container -- the entrypoint clears its binary caches and reinstalls fresh (yt-dlp, bili-cli, pp-cli, etc.).

OpenCLI note: the OpenCLI desktop boost (tiktok/instagram/pinterest/xueqiu) is intentionally NOT installed in this headless container -- those sources use their server-side backends (Apify / public APIs / Searxng), which is the intended default. OpenCLI only applies if you run mcpo on a desktop machine with Chrome.

Configuration

All config is environment variables (a Settings dataclass). Everything is optional -- the server degrades to free-source-only mode if you set nothing.

Quick reference

# ===== LLM (rerank + brief) =====
OPENAI_BASE_URL="https://your-gateway/v1"    # OpenAI-compatible gateway base URL
OPENAI_API_KEY="sk-..."                      # API key for rerank + brief
REACH_MCP_RERANK_MODEL="gemini-flash-lite"   # model for reranking (default: gemini-flash-lite)
REACH_MCP_BRIEF_MODEL="gemini-flash-lite"    # model for brief synthesis (default: gemini-flash-lite)

# ===== Web search (required for web source) =====
SEARXNG_URL="http://searxng:8080"            # your Searxng instance

# ===== Free sources (optional auth boosts) =====
GH_TOKEN="ghp_..."                           # GitHub personal access token (higher rate limits)
BSKY_HANDLE="you.bsky.social"                # Bluesky handle (optional)
BSKY_APP_PASSWORD="xxxx-xxxx-xxxx-xxxx"      # Bluesky app password (optional)
YTDLP_PROXY="http://proxy:8080"              # proxy for yt-dlp (optional)
JINA_API_KEY="jina_..."                      # Jina key (LinkedIn search + read_url; free monthly quota)
BRAVE_API_KEY="BSA_..."                      # Brave Search key (web boost; $5 free credits/mo recurring)
RSS_FEEDS="https://blog.example.com/feed"    # comma-separated RSS/Atom feed URLs (rss source)

# ===== Login-gated sources =====
AUTH_TOKEN="..."                             # X/Twitter auth_token cookie
CT0="..."                                    # X/Twitter ct0 (csrf) cookie
TRUTHSOCIAL_TOKEN="..."                      # Truth Social API bearer token
APIFY_API_TOKEN="apify_api_..."              # Apify token (threads + tiktok/ig/pin boost; $5 free credits/mo)
SCRAPECREATORS_API_KEY="sc_..."              # ScrapeCreators key (tiktok + instagram + pinterest fallback; 100 one-time credits)

# ===== Chinese sources =====
XHS_MCP_URL="http://xiaohongshu-mcp:18060/mcp"  # xiaohongshu-mcp companion service URL
XIAOYUZHOU_ACCESS_TOKEN="..."                # 小宇宙 access token (phone-SMS login; required for search)
WHISPER_BASE_URL="http://gpu.savorcare.com:8080/v1"  # OpenAI-compatible whisper endpoint (xiaoyuzhou transcription)
WHISPER_API_KEY=""                           # optional; LocalAI doesn't check it
WHISPER_MODEL="whisper-large"                # model name (default: whisper-large)


# ===== Server settings =====
REACH_MCP_TRANSPORT="http"                   # http or stdio (default: http)
REACH_MCP_HOST="0.0.0.0"                     # bind host (default: 0.0.0.0)
REACH_MCP_PORT="8765"                         # bind port (default: 8765)
REACH_MCP_API_KEY="..."                      # optional lock on the HTTP surface
REACH_MCP_ALLOWED_HOSTS="reach-mcp:8765,localhost:8765"  # DNS-rebinding allow-list

# ===== HTTP tuning =====
REACH_MCP_SOURCE_TIMEOUT="60"                # per-source timeout in seconds (default: 60)
REACH_MCP_REQUEST_TIMEOUT="15"               # per-request timeout in seconds (default: 15)
REACH_MCP_MIN_HOST_DELAY="0.5"               # minimum delay between requests to same host (default: 0.5)
REACH_MCP_MAX_RETRIES="3"                    # max retries on transient errors (default: 3)

Credential guide

按顺序获取每个 key 的完整步骤、入口 URL、以及推荐 RSS feed 清单,见 docs/CREDENTIALS.md。本节省略版;下面列出每个 key 的用途和快速要点。

X / Twitter (AUTH_TOKEN, CT0)

1. Log into x.com in any browser

2. Open DevTools (F12) → Application → Cookies → x.com

3. Copy the values of auth_token and ct0

4. Set AUTH_TOKEN=<auth_token value> and CT0=<ct0 value>

⚠️ Use a dedicated account; API-like usage may trigger platform anti-bot detection.

Bluesky (BSKY_HANDLE, BSKY_APP_PASSWORD)

1. Log into bsky.app → Settings → App Passwords

2. Create a new app password; copy the generated value

3. Set BSKY_HANDLE=yourhandle.bsky.social and BSKY_APP_PASSWORD=<generated password>

GitHub (GH_TOKEN)

1. GitHub → Settings → Developer settings → Personal access tokens → Fine-grained tokens

2. Create a token with Public Repositories (read-only) access

3. Set GH_TOKEN=github_pat_...

Without GH_TOKEN, GitHub search still works but hits unauthenticated rate limits (60 req/hr vs 5000 req/hr).

Truth Social (TRUTHSOCIAL_TOKEN)

1. Log into truthsocial.com in a browser

2. Open DevTools → Network tab → find any API request to truthsocial.com

3. Copy the Authorization: Bearer ... header value

4. Set TRUTHSOCIAL_TOKEN=<the bearer token>

ScrapeCreators (SCRAPECREATORS_API_KEY)

⚠️ 100 credits one-time, then paid from $10/mo. Signup gives 100 free credits (no credit card, no monthly reset). A single comprehensive search can consume 50--150+ credits. Only enable SC-backed sources (tiktok, instagram, pinterest) if you're OK with the cost.

1. Sign up at scrapecreators.com (free, no credit card)

2. Copy your API key from the dashboard

3. Set SCRAPECREATORS_API_KEY=sc_...

One key enables three SC-only sources (and an optional LinkedIn boost):

• tiktok -- TikTok search (no free alternative exists)

• instagram -- Instagram search (Meta API requires app review)

• pinterest -- Pinterest search (no free alternative exists)

• linkedin -- Optional boost (Jina is the free primary)

LinkedIn

LinkedIn uses Jina as its free primary backend (agent-reach's approach). Jina's s.jina.ai search endpoint scopes to linkedin.com/posts and linkedin.com/pulse.

1. Get a free Jina API key at jina.ai (Reader API)

2. Set JINA_API_KEY=jina_...

Jina's free key is a recurring monthly rate-limit quota (20 RPM without a key, 500 RPM with a free key, 5000 RPM paid) -- it is NOT a one-time credit. s.jina.ai search requires a key; r.jina.ai (page reader) works without one.

Optionally, setting SCRAPECREATORS_API_KEY adds ScrapeCreators results in parallel for a richer result set.

DripStack (financial newsletters)

Free and keyless -- always available, no setup. Search over premium financial newsletters (Substack/analyst write-ups). Best for ticker/company research. Complements stocktwits (retail sentiment) and polymarket (real-money odds).

RSS feeds (RSS_FEEDS)

Generic RSS/Atom source, free (uses feedparser, already a dependency). Set RSS_FEEDS to a comma-separated list of feed URLs; entries are filtered to the query within the recency window.

RSS_FEEDS="https://hnrss.org/frontpage,https://www.federalreserve.gov/feeds/press_all.xml"

📚 已验证的推荐 feed 清单——科技媒体、财经市场、政府经济数据/政策发布、政治军事、公司财报(SEC EDGAR),见 docs/CREDENTIALS.md 第 4 级。

Xiaohongshu / 小红书 (XHS_MCP_URL)

Uses the community-vetted xiaohongshu-mcp Go server (15K+ stars) as backend.

Docker (recommended): Add to your compose file (see docker-compose.yml). The ./data volume is required — it stores the login cookies (they are lost on recreate without it); ./images is only needed if you publish notes:

xiaohongshu-mcp:
  image: ghcr.io/xpzouying/xiaohongshu-mcp:latest
  ports: ["18060:18060"]
  restart: unless-stopped
  init: true
  tty: true
  volumes:
    - ./data:/app/data
    - ./images:/app/images
  environment:
    - COOKIES_PATH=/app/data/cookies.json
    - HOME=/app/data/home
    - XDG_CONFIG_HOME=/app/data/config

Then set XHS_MCP_URL=http://xiaohongshu-mcp:18060/mcp and log in (below).

Login (QR scan, once): the image runs only the MCP server — there is no separate login binary. Start it, then scan the QR via any MCP client:

npx @modelcontextprotocol/inspector      # MCP Inspector
# connect to http://localhost:18060/mcp
# call the `login` tool → QR code appears → scan with the 小红书 App
# call `check_login` to confirm (re-scan once more if prompted)

Notes: open the App before scanning (the QR expires quickly); don't log the same account in elsewhere on the web — 小红书 single-logs-in, a web login kicks the MCP account out. Cookies persist in the ./data volume.

Without Docker: Download the binary from releases. Login is the same QR flow via the MCP login tool (or go run cmd/login/main.go from source), then start the server and set XHS_MCP_URL=http://localhost:18060/mcp.

Web search (SEARXNG_URL)

Required for the web source. Point to your self-hosted Searxng instance (default http://searxng:8080). If you don't have one, Searxng runs easily in Docker.

YouTube (YTDLP_PROXY)

Optional. Set to an HTTP proxy URL (e.g. http://proxy:8080) if yt-dlp needs a proxy to reach YouTube.

Digg (digg-pp-cli)

Digg is auto-enabled when the digg-pp-cli binary is on PATH. Built from the last30days project's build steps. Without the CLI, digg stays gated off.

Apify (APIFY_API_TOKEN) -- threads, tiktok, instagram, pinterest

Apify's Free plan gives $5 in credits every month (recurring, not one-time) -- enough for hundreds of search runs. One token enables:

• threads -- via the apify/threads-scraper Actor (the only viable free server-side path; Meta's own API needs app review)

• tiktok / instagram / pinterest -- Apify is the preferred backend

1. Sign up at apify.com (Free plan, no card)

2. Copy your API token from Settings --> Integrations

3. Set APIFY_API_TOKEN=apify_api_...

OpenCLI (optional desktop free) -- tiktok, instagram, pinterest

OpenCLI (Apache--2.0, free) reuses your logged-in Chrome session via a browser bridge. If the opencli binary is on PATH, tiktok/instagram/pinterest use it automatically as a free desktop backend (no API key needed). Desktop-only -- not for headless/Docker deployments.

bilibili (bili CLI)

Prefers the community-vetted bili-cli (uv tool install bilibili-cli) when on PATH -- it handles B站's wbi signing and anti-scraping (HTTP 412) that raw API calls hit. Falls back to the public search API if bili is absent. No login needed for search.

Xiaoyuzhou / 小宇宙

Podcast search + Whisper transcription. Search requires a 小宇宙 account token (phone-SMS login):

1. Login flow: the app sends a code to your phone, which returns accessToken/refreshToken (see xiaoyuzhou-api for the exact flow)

2. Set XIAOYUZHOU_ACCESS_TOKEN=...

3. Transcription uses an OpenAI-compatible Whisper endpoint (WHISPER_BASE_URL, default http://gpu.savorcare.com:8080/v1, model whisper-large). Point it at any self-hosted LocalAI or Groq's api.groq.com/openai/v1 — the API key may be empty if your server doesn't check it.

Audio constraint (Whisper): files ≤ 25 MB, formats mp3/mp4/mpeg/mpga/m4a/wav/webm; no explicit duration cap at the API layer.

read_url tool (Jina Reader)

The read_url(url) tool fetches any URL's content as clean text via r.jina.ai (Jina Reader). Use it to read a page you found in search results -- a Reddit thread, news article, blog post, or GitHub readme -- when you need the full text beyond the snippet. Keyless at 20 RPM; set JINA_API_KEY for 500 RPM (free monthly quota). Returns {url, content, ok}.

Migration from last30days

If you're coming from the last30days MCP server (mvanhorn/last30days-skill), here's what changes:

Env var mapping

Sources: gained and lost

Behavioral differences

• days is per-call, not hardcoded to 30

• sources is per-call, not global INCLUDE_SOURCES

• synthesize=false returns raw scored rows (last30days always synthesizes)

• list_sources shows exactly what's configured and what each source needs

• No INCLUDE_SOURCES global filter -- just pass sources=[...] per search

• No --hiring-signals, --discover, --watchlist -- these were last30days CLI features; reach-mcp is server-only

License

MIT © xyonium