Skip to content

Releases: AlekseiMarchenko/central-intelligence

v1.2.2 — 75.0% LongMemEval, 52.2% LifeBench

Choose a tag to compare

@AlekseiMarchenko AlekseiMarchenko released this 20 Apr 12:51

New benchmark results and documentation parity. Central Intelligence's v2 hybrid retrieval pipeline (pgvector HNSW + BM25 + RRF fusion + bge-reranker-v2-m3 cross-encoder) scores among the top results published on both LifeBench and LongMemEval.

Benchmark Results

LongMemEval (ICLR 2025) — 75.0%

Conversational memory across 500 questions: single-session recall, multi-session reasoning, temporal reasoning, knowledge updates, preference tracking.

Overall Single-session Multi-session Temporal Preference
75.0% 91.9% 66.2% 69.9% 76.7%

Answer model: gpt-5.4-mini. Judge: gpt-4o.

LifeBench (2026) — 52.2%

Long-term multi-source memory: 2,003 questions across 10 users, 51K real-world events (messages, calendar, health records, notes, calls). The hardest published memory benchmark.

Overall Info Extraction Multi-hop Temporal Nondeclarative
52.2% 47.2% 52.9% 46.4% 64.1%

Answer model: gpt-5.4-mini. Judge: gpt-4.1-mini.

Evaluation harness: lifebench-eval.

What Got Us Here (since v1.2.1)

The journey from v1.2.1 to v1.2.2, measured on LongMemEval:

Change Delta
Reranker upgrade: MiniLM → bge-reranker-v2-m3 + top-K=10 ~+18pp
Answer model: gpt-4o-minigpt-5.4-mini ~+5pp
Answer prompt: CoT counting + factual precision + multi-hop grounding ~+3pp
Infrastructure fixes (ef_search, performance CPU, 4GB RAM) enabler

Attribution is approximate — each step was measured through the benchmark's run-to-run variance. The architecture that reached 75% is the clean v2 pipeline: pgvector HNSW + BM25 with RRF fusion, temporal decay, and a cross-encoder reranker. No fact extraction or graph traversal during recall.

Changes Since v1.2.1

Retrieval

  • Upgraded reranker to bge-reranker-v2-m3 (8K context, +14% nDCG@10 over MiniLM)
  • top-K=10 retrieval (NeurIPS 2024 finding: accuracy saturates at k=10, degrades with more docs)
  • Set hnsw.ef_search = 400 globally (was silently capped at ~40 results)
  • Reverted to v2 architecture (vector + BM25 + reranker) — removed query decomposition and fact-path that regressed multi-session scores

Benchmark infrastructure

  • Added --answer-model parameter to benchmark entrypoint
  • Performance CPU on benchmark VM (fixes extraction stalls under shared-CPU throttling)
  • DB-driven extraction queue (removed in-memory queue that caused zombie processes)

Documentation

  • README Benchmarks section now includes full category tables for both LifeBench and LongMemEval
  • Landing page at centralintelligence.online matches README numbers exactly
  • Answer/judge model attribution documented inline for reproducibility

Try It

npx central-intelligence-local signup

Full docs: README · Benchmarks: live results

v1.2.1 — Codex Review: 8 Pipeline Regressions Fixed

Choose a tag to compare

@AlekseiMarchenko AlekseiMarchenko released this 11 Apr 07:54

External code review found 8 regressions introduced by the 14 rapid-fire commits that tried to fix benchmark extraction stalls. All 8 are now fixed.

Fixed (Critical)

  • /extract endpoint now loops through ALL batches instead of processing one batch and stopping
  • Batch fallback facts use real embedding vectors instead of empty [] (was crashing on vector(1536) column)

Fixed (High)

  • Preference-only memories keep their fallback fact_unit so they stay discoverable in fact search
  • Entity graph builds automatically after extraction (no more dead graph retrieval)
  • fireAndForgetExtraction uses AbortController to cancel zombie extractions on timeout

Fixed (Medium)

  • Fact-based recall now applies scope, tags, and date filters (was ignoring all caller filters)
  • Causal relations from GPT extraction are persisted to fact_links table (graph traversal can use them)
  • Dead code cleanup: removed unused factEntityMap variable and consolidateObservations import

v1.2.0 — Benchmark Infrastructure + pgvector Fix

Choose a tag to compare

@AlekseiMarchenko AlekseiMarchenko released this 09 Apr 19:22

Benchmark runs no longer touch production. A dedicated Fly VM runs LifeBench with its own ephemeral Postgres. pgvector now survives Fly Postgres restarts. Infra costs dropped 58%.

Fixed

  • pgvector survives restarts (custom Docker image with postgresql-17-pgvector baked in)
  • Parallel migration deadlock workaround in benchmark entrypoint

Added

  • Benchmark VM: self-contained Fly machine with ephemeral Postgres + CI API + LifeBench harness
  • Custom Postgres image: db/Dockerfile extends flyio/postgres-flex:17.2 with pgvector 0.8.2

Infrastructure

  • Fly costs reduced
  • Production DB cleaned
  • New files: db/Dockerfile, benchmark/Dockerfile, benchmark/benchmark-entrypoint.sh, benchmark/fly.toml

v1.1.0 — Retrieval Reliability

Choose a tag to compare

@AlekseiMarchenko AlekseiMarchenko released this 08 Apr 21:51

retrieval reliability fixes + dual-path recall

Added

  • added v1.1.0 entry covering temporal SQL fix
  • BM25 OR-matching
  • global HNSW index
  • dual-path recall
  • query type classifier
  • topic tagging

VERSION bumped to 1.1.0. API version strings updated. README updated to
describe dual-path architecture.

v1.0.0 — 4-Way Retrieval prototype

Choose a tag to compare

@AlekseiMarchenko AlekseiMarchenko released this 06 Apr 22:42

4-Way Retrieval is currently still in development and would be available later for Enterprise customers

Recall is now 4x smarter. Every memory is decomposed into structured facts at store time, with entities extracted, resolved, and linked into a knowledge graph. Recall runs four search strategies in parallel (vector, BM25, graph traversal, temporal), fuses them with RRF, and reranks with a local cross-encoder model. Zero per-recall API cost.

Added

  • Fact decomposition. Each stored memory is broken into atomic facts with entities, temporal info, and causal relations via GPT-4o-mini. Facts are individually searchable with their own embeddings and tsvectors.
  • Entity resolution. Extracted entities are matched against existing ones using trigram similarity, co-occurrence scoring, and temporal proximity. "Alice" and "my coworker Alice" merge automatically.
  • Knowledge graph. Entities, facts, and co-occurrences form a queryable graph via junction tables. Graph traversal finds related facts through shared entities and causal links.
  • 4-way parallel retrieval. Recall runs vector search, BM25, graph traversal (dual-seed: embedding + entity name), and temporal search simultaneously. Results are fused via Reciprocal Rank Fusion.
  • Local ONNX cross-encoder reranker. ms-marco-MiniLM-L-6-v2 runs locally via @xenova/transformers. Zero per-request cost. Falls back to Cohere API, then passthrough.
  • Observation consolidation. When an entity accumulates 5+ facts, a higher-level observation is auto-synthesized. These "pre-computed answers" match directly on recall.
  • Automated Fly.io deploys via GitHub Actions on every push to main.
  • Security hardening. x402 auth context fix, non-root Docker, XSS escaping, CORS fix, OpenAI call budget, IP rate limit hardening.
  • 68 tests (up from 32). Covers fact extraction, entity scoring, 4-way RRF fusion, reranker fallback, and observation contracts.

Changed

  • Store pipeline: full fact decomposition replaces simple entity+preference enrichment (3x retry, exponential backoff, concurrency-limited queue).
  • Recall pipeline: routes to fact-based 4-way retrieval when fact_units exist, falls back to legacy 2-way (vector + BM25).
  • Reranker: 3-tier fallback chain (ONNX local, Cohere API, passthrough) replaces Cohere-only.
  • Dockerfile: non-root user, pre-downloads ONNX model during build.
  • Store cost: ~$0.0003/memory (up from ~$0.00007). Recall cost: $0 (down from ~$0.00002).

4-Way Retrieval is currently still in development and would be available later for Enterprise customers

v0.6.0 — pgvector Retrieval & LifeBench 44.4%

Choose a tag to compare

@AlekseiMarchenko AlekseiMarchenko released this 05 Apr 18:04

Headline

Central Intelligence scores 44.4% on LifeBench, the hardest published memory benchmark (2026, Nanjing University + Huawei). The key: pgvector HNSW indexing replaced the 500-row in-app vector scan, making all memories searchable by semantic similarity.

What Changed

pgvector ANN Search

The retrieval engine now uses PostgreSQL's pgvector extension with an HNSW index instead of fetching the 500 most recent memories and computing cosine similarity in JavaScript. This means all memories are searchable, not just the most recent 500.

  • migrate-pgvector.ts: enables extension, adds embedding_vec vector(1536) column, backfills from JSONB, creates HNSW index
  • Graceful fallback: if pgvector isn't installed, the old in-app cosine similarity still works
  • store() now writes both JSONB and native vector columns

Cross-Encoder Reranking (optional)

New rerank.ts service calls the Cohere Rerank API to re-score the top 100 retrieval candidates. Disabled by default — add COHERE_API_KEY as a Fly secret to enable. Falls back to passthrough scoring without the key.

Temporal Filtering

The /memories/recall endpoint now accepts date_from and date_to parameters. Both vector search and BM25 search apply the date range as a WHERE clause before scoring.

LifeBench Results on Landing Page & README

  • New benchmark section on centralintelligence.online with CI's per-category scores
  • LifeBench and AMB badges on GitHub README
  • Links to the arxiv paper and the open-source evaluation harness

LifeBench Scores

Category Accuracy
Information Extraction 50.0%
Multi-hop Reasoning 45.6%
Temporal & Knowledge Updating 28.6%
Nondeclarative 43.6%
Overall 44.4%

Tested on 1 user (14,910 memories, 207 questions). Evaluation harness: lifebench-eval

Files Changed

  • packages/api/src/db/migrate-pgvector.tsNEW pgvector migration
  • packages/api/src/services/rerank.tsNEW Cohere reranking service
  • packages/api/src/services/memories.ts — pgvector search path, reranking integration, temporal filtering
  • packages/api/src/routes/memories.tsdate_from/date_to params on recall
  • packages/api/src/index.ts — register pgvector migration on startup
  • README.md — LifeBench + AMB badges, benchmark results
  • landing/index.html — LifeBench results section

Upgrade Notes

  • pgvector migration runs automatically on startup. If the extension isn't available, the API falls back to the previous retrieval method.
  • Existing memories are backfilled from JSONB to native vector format on first startup (~30s for 15K memories).
  • No breaking API changes. date_from/date_to are optional new parameters.

v0.5.0 — Cross-Tool Memory

Choose a tag to compare

@AlekseiMarchenko AlekseiMarchenko released this 03 Apr 18:51

What's New

CI Local now reads config files from 5 AI coding platforms and merges them into the search pipeline. Memories stored via Claude Code are discoverable when using Cursor, and vice versa. This is the cross-tool memory layer: your AI memory works everywhere, not just in one tool.

The Pitch

Every AI coding tool builds its own memory silo. CLAUDE.md stays in Claude Code. .cursor/rules stays in Cursor. .windsurf/rules stays in Windsurf. Central Intelligence sits in the middle and makes them all searchable from any tool.

No platform vendor will build this (they want lock-in). CI does.

Added

  • Cross-tool file source reading. CI Local discovers and parses CLAUDE.md, .cursor/rules, .windsurf/rules, codex.md, and .github/copilot-instructions.md. Parsed sections are embedded and merged into hybrid search alongside database memories.
  • File source cache. New file_source_cache SQLite table stores content hashes, embeddings, and first-seen timestamps. Only re-embeds when content changes. First recall takes ~2-4s (embedding), subsequent calls are instant.
  • Smart recall signals. Recall and context responses now include source (db or platform name), freshness_score (0-1 exponential decay), and duplicate_group (Jaccard overlap > 0.8). Agents can prioritize fresh, unique memories.
  • Duplicate detection. Near-duplicate memories across sources are grouped so agents can deduplicate.
  • 31 new tests. File source parsers (22), DB cache (5), cross-tool E2E (4).

Supported Platforms

Platform Config file Parsing
Claude Code CLAUDE.md Section-based (## headings)
Cursor .cursor/rules Paragraph-based
Windsurf .windsurf/rules Paragraph-based
Codex codex.md Section-based
GitHub Copilot .github/copilot-instructions.md Section-based

Changed

  • MemoryWithScore extended with optional source, source_path, freshness_score, duplicate_group fields (backward compatible)
  • Hybrid search merges file-sourced candidates into vector, FTS5, and fuzzy strategies via RRF fusion

Versions

  • API: v0.5.0
  • CI Local: v1.1.0
  • MCP Server: v0.5.0
  • CLI: v0.5.0

v0.4.0 — Quality Sprint

Choose a tag to compare

@AlekseiMarchenko AlekseiMarchenko released this 31 Mar 16:51

What's New

Retrieval actually works now. BM25 full-text search was broken since launch. Trigram search ran against encrypted ciphertext. Context compression silently forwarded decrypted memories to OpenAI. All three are fixed. Benchmark score: 92/100 → 100/100.

Fixed

  • BM25 full-text search — tsvectors now populated from plaintext before encryption
  • Removed trigram search (ran against ciphertext, producing noise)
  • Removed context compression (sent decrypted memories to OpenAI, breaking encryption guarantee)
  • Batch N+1 queries in recall with WHERE id = ANY(...)
  • Specific error handling instead of bare catch {} blocks
  • forget() now logs usage events
  • Migration uses CREATE INDEX CONCURRENTLY to avoid table locks
  • Lazy tsvector backfill during recall for old memories

Added

  • Vitest test suite (32 tests)
  • GIN index on content_tsv
  • VERSION file and CHANGELOG.md

Changed

  • Hybrid retrieval: vector + BM25 (was vector + BM25 + trigram)
  • All packages unified to v0.4.0

v0.3.0 — Hybrid Retrieval Engine (94% accuracy)

Choose a tag to compare

@AlekseiMarchenko AlekseiMarchenko released this 30 Mar 14:23

What's New

Hybrid Retrieval Engine

The core memory recall system has been upgraded from single-strategy vector search to a multi-strategy hybrid pipeline:

  • Vector search — semantic similarity via OpenAI embeddings
  • BM25 full-text search — exact keyword matching via PostgreSQL tsvector
  • Trigram fuzzy search — typo tolerance via pg_trgm (handles "PostgrSQL", "Typscript", etc.)
  • Reciprocal Rank Fusion — merges results from all three strategies
  • Temporal decay — recent memories score higher (90-day half-life)
  • Context compression — large result sets compressed via GPT-4o-mini
  • Relevance threshold — irrelevant queries return 0 results instead of noise

Benchmark Results (94% overall)

Category Score
Semantic (paraphrased queries) 75%
Keyword (exact technical terms) 100%
Typo tolerance (misspelled queries) 100%
Cross-domain (business + meetings) 100%
Preference (coding style) 100%
Negative (irrelevant queries filtered) 100%
Overall 94%

New SDKs & Integrations

  • Node.js SDK on npm: central-intelligence-sdk
  • Python SDK updated on PyPI: central-intelligence
  • OpenAPI spec at /docs/openapi.json for ChatGPT Custom GPTs
  • Demo endpoint at /demo/recall — try without signup
  • "When to use" metadata in .well-known/mcp and .well-known/agent.json
  • Integration cards: Lovable, Perplexity, Antigravity, GitHub Copilot
  • ClawHub skill published

v0.2.0 — x402 Payments & Security Hardening

Choose a tag to compare

@AlekseiMarchenko AlekseiMarchenko released this 23 Mar 16:36

What's New

x402 Payment Protocol

Agents with a Base wallet can now pay per-call ($0.001 USDC) via /x402/* endpoints — no API key needed. Powered by Coinbase CDP facilitator.

USDC on Base Payments

Manual deposit flow: send USDC on Base, verify on-chain, credits added instantly. Auto-upgrades to Pro tier.

Security Hardening

  • SQL injection fix (parameterized queries)
  • Atomic double-credit prevention (INSERT ON CONFLICT)
  • TOCTOU race condition fix (pre-auth debit pattern)
  • 12-block confirmation requirement for deposits
  • Rate limiting on payment verification
  • CORS restrictions
  • 64KB request body limit

Analytics Dashboard

GitHub OAuth-protected dashboard at /dashboard with memory usage, active agents, and operation metrics.

Interactive API Docs

/docs — styled HTML docs for developers
/docs/json — machine-readable spec for agents

Distribution

  • Official MCP Registry
  • mcp.so, Glama.ai, PulseMCP
  • OpenClaw skill file
  • AstrBot plugin (China market)
  • npm: central-intelligence-mcp, central-intelligence-cli