locket

A privacy-first personal context engine: it ingests your own photo and messaging exports (WhatsApp, Instagram DMs, SMS/MMS backups, Google Photos Takeout, iPhone backups' Messages history), extracts typed, provenance-cited facts about your life with an LLM pipeline, resolves the people and places those facts mention into stable entities, stores everything in Postgres+pgvector, and serves the resulting profile to other tools over MCP (Model Context Protocol) — so you can ask Claude Code or Claude Desktop things like "when did I last see Sarah?" and get an answer that cites the exact message it came from.

Architecture

flowchart LR
    Sources["Your exports\n(WhatsApp/Instagram/SMS/Photos)"] --> Adapters["adapters/\npure parsers"]
    Adapters --> RawItems[(RawItem stream)]
    RawItems -- photos --> Vision["vision/\nSigLIP2 + RapidOCR + InsightFace\n(local, 100% of photos)"]
    Vision -- curated subset --> VisionLLM["local Ollama qwen3-vl:8b\n(curated tail only)"]
    RawItems --> Windowing["extraction/chunking.py"]
    Windowing --> Extraction["extraction/graph.py\nLangGraph: Claude API\nstructured outputs"]
    Extraction --> Resolution["resolution.py\ntiered entity resolution\n+ human confirm queue"]
    Resolution --> Store[(Postgres + pgvector)]
    Store --> Profile["profile.py\nsynthesized, cited profile"]
    Profile --> Store
    Store --> MCP["mcp_server.py\nsix tools, stdio"]
    MCP --> Client["Claude Code /\nClaude Desktop"]

Full stage-by-stage breakdown, module boundaries, and the dual-corpus design: docs/architecture.md. 90-second walkthrough with scripted questions and MCP registration commands: docs/demo.md.

Related MCP server: Memex

Quickstart (against the committed synthetic demo corpus)

docker compose up -d db          # Postgres + pgvector
uv sync                          # hand-edit pyproject.toml + `uv sync` to add deps — no `uv add`

uv run python -m locket.cli ingest demo_corpus/whatsapp/team.txt
uv run python -m locket.cli ingest demo_corpus/sms/backup.xml
uv run python -m locket.cli ingest demo_corpus/photos

# No ANTHROPIC_API_KEY needed — with no key set, extraction/resolution/
# profile all run against a local Ollama model by default (see "Running
# fully local" below). --skip-vision bypasses the local vision pre-pass +
# Ollama vision-LLM tail (~135s/image measured — see evals/BASELINE.md —
# worth skipping for a quick pass).
uv run python -m locket.cli pipeline run --skip-vision --corpus-dir demo_corpus

uv run python -m locket.cli profile build
claude mcp add --scope user locket -- uv run --directory "$(pwd)" python -m locket.mcp_server

Full command reference (every subcommand: ingest, pipeline run [--skip-vision/--cap/--corpus-dir/--retry-failed], pipeline retry-given-up, label-faces, resolve, eval extraction|rag, profile build, stats [--json], serve, serve-ui [--host/--port]) and the exact Claude Desktop registration JSON block: docs/demo.md.

To run against your own data instead of the demo corpus, set LOCKET_CORPUS_DIR in a local .env (never inside this repo — see .env.example) and point ingest / pipeline run --corpus-dir at it.

Running fully local (no API key)

Every LLM call locket makes — extraction, entity resolution, profile rendering, and the MCP server's answer_question — goes through one backend-selection seam, locket.llm.get_chat_model. It picks between two backends:

• anthropic (ChatAnthropic, real network calls to Claude, higher quality, costs money): used automatically when ANTHROPIC_API_KEY is set, or when you force it with LOCKET_LLM_BACKEND=anthropic.

• ollama (ChatOllama, a local Ollama server, free, no data leaves your machine): the default when no API key is present. locket pipeline run no longer refuses to run keylessly — it just uses this backend instead.

Requirements: an Ollama server running locally (ollama serve, or the desktop app) with the text model pulled — ollama pull gemma3:12b (the default, ~8GB) or set LOCKET_LOCAL_MODEL=qwen2.5:3b-instruct for a smaller, already-common model. OLLAMA_HOST is respected if you want to point at a different machine's Ollama (e.g. over Tailscale) instead of localhost:11434 — locket does not read or override it itself.

Honest quality/speed tradeoff, measured on this project's dev machine (CPU-only Ollama): the local backend is markedly slower and somewhat lower-quality than the Claude API backend. gemma3:12b took roughly 10-130s per extraction window (vs. sub-second-to-a-few-seconds for claude-haiku-4-5) and produced fewer, though more information-dense, facts per window than a smaller local model (qwen2.5:3b-instruct, ~10x faster but noisier — see src/locket/llm.py's module docstring for the side-by-side). Vision (qwen3-vl:8b) already ran local-only regardless of this setting, at its own separately-measured ~135s/image. Full real pipeline-run numbers (fact counts, wall time) for the local backend are in evals/BASELINE.md's "local backend (informal)" section — the official baseline stays the Claude API run, pending a real key.

Privacy posture

Stated plainly, not hand-waved:

• Storage is fully local. Postgres+pgvector runs in your own Docker container. Nothing about your facts, entities, or profile is sent anywhere except the specific API calls described below. The container's port binds to 127.0.0.1 only (see docker-compose.yml) — never reachable from another device on your LAN.

• Text extraction uses the Claude API. Message/photo-OCR text is sent to Anthropic under their no-training API terms to extract structured facts (claude-haiku-4-5, escalating to claude-sonnet-5 on repeated validation failures) and to render profile prose and answer questions. This is a real network call to a third party — disclosed honestly, not claimed as "fully private."

• Real photos are processed by local models only. EXIF/GPS, SigLIP2 zero-shot tagging, RapidOCR, and InsightFace face clustering all run locally on 100% of your photo library, for free. The one open-ended "describe this photo" step (the vision-LLM tail) runs against a small, curated subset using local Ollama qwen3-vl:8b — never a cloud vision model, for real photos.

• Gemini's free tier is explicitly forbidden for real photos. Google's free-tier terms grant Google the right to train on and have humans review submitted content — unacceptable for private photos of your life. Using Gemini as an opt-in path for generating the synthetic demo corpus (where no privacy stakes exist — the faces are AI-generated, MIT-licensed SFHQ portraits, and every conversation is invented) is planned, not yet implemented — no code path calls it today. A paid Claude-API fallback for real photos behind an explicit --cloud-ok flag, for when local Ollama is unavailable, is likewise planned and not yet implemented — still Anthropic's no-training terms, never Gemini free tier, once it exists.

• Your real exports never enter this repository. They're read from LOCKET_CORPUS_DIR, an env var pointing outside the repo, declared in a local, gitignored .env. .gitignore also blocks real_corpus/ and *.local.* (the pattern the real self-labeled eval gold set uses: evals/gold/real_gold.local.yaml). Everything under demo_corpus/ in this repo is synthetic — five invented personas, generated conversations, and staged photos of AI-generated faces — used for every test, CI run, and the public demo. No real data of any kind ships in this repository.

Eval results

locket ships two eval suites (evals/extraction_eval.py, evals/rag_eval.py), both runnable via locket eval extraction|rag --json and both gated in CI (.github/workflows/eval.yml, nightly + on-demand — kept out of the free push/PR lint+test workflow since they cost real money per run). Full methodology, every number's provenance, and the exact commands to reproduce or extend each measurement: evals/BASELINE.md.

No number above is invented — where a measurement is blocked on a still-absent API key, the table says so plainly instead of filling in a plausible-looking placeholder.

License

MIT (LICENSE). Third-party model weights and assets carry their own, narrower terms — see THIRD_PARTY_NOTICES.md before distributing or monetizing anything built on this repo (notably: InsightFace's buffalo_l face-analysis weights are non-commercial/research-personal use only, even though the InsightFace code itself is MIT).

See Claude/specs/2026-07-30-locket-design.md (private planning vault, not part of this repo) for the full design writeup.