menu() and route() were ranking the catalog on English function words, and it cost roughly a third of retrieval recall.

_idf_weights computes inverse document frequency over description prose. A pronoun appears in essentially no tool description, so idf handed it the highest weight in the query. score_action then spent that weight on a substring match against the tool name, a different corpus entirely, and a two-character token matches a large slice of snake_case names by accident.

Measured on draw me a diagram of that: check_rename_safe scored 19.3 on the me inside rena-me-_safe and beat render_diagram's 16.5 for the actual word "diagram". get_runtime_coverage and find_implementations placed above it on the same token. Even a scored 2.5 that way.

A second defect sat underneath it: the fragment branch (qt in name_l, +4.0) outranked the whole-word branch (qt in name_toks, +3.0), so a match inside a word beat an exact word hit. Since every token is trivially a substring of its own name, that whole-word branch was unreachable.

Three changes: a stopword filter on query tokens, a four-character floor before a token may match as a fragment, and the whole-word branch promoted above the fragment branch. _tokens is unchanged and still returns the raw split; the new _query_tokens is what ranking uses.

26 queries improved on menu against 7 regressed; 15 improved on route against 6. This also closes the menu gap disclosed in 1.108.212; the three transform actions now reach 100% on both paths.

Reproduce with benchmarks/route_recall/run_route_recall.py. Four regression tests pin the scorer in tests/test_counter.py.

route() mapped a plain-language task onto three actions it could never propose, no matter how the task was phrased: render_diagram, plan_refactoring, and assemble_task_context. Measured recall for that group was 0%.

These three consume another tool's output rather than querying the index, so the words a user actually types for them ("draw me a diagram", "give me a plan for renaming this", "set me up to debug this") appear nowhere in the catalog text route's fallback ranks over. No curated rule covered them either. The capability shipped; the route to it did not.

Three _INTENT_RULES entries added, plus broader task-setup phrasing for assemble_task_context. Measured on the new harness: transform-group route recall 0% -> 100%, all three at rank 1, and no other query regressed. Overall route recall@3 moved 44.1% -> 49.2%, recall@1 25.4% -> 30.5%.

Ordering is deliberate and load-bearing. A task naming both a data fetch and a render ("visualize the call graph") leads with get_call_hierarchy and offers render_diagram as an alternate, because the renderer consumes that output and has nothing to draw without it. Both directions are pinned in tests/test_counter.py.

New: benchmarks/route_recall/

The Counter cuts resident tool-schema tokens by ~98%. The cost it trades against is recall, because an action route never proposes is functionally absent. That cost had never been measured.

run_route_recall.py measures menu() and route() separately against a committed corpus, reporting recall@k, per-group breakdown, and which path answered. It refuses to report a number when a corpus target is missing from the live catalog: a typo there reads as a router failure, which is a corpus bug wearing a result's clothes.

Each query carries a measured leakage score -- how much of the target action's own vocabulary the query hands back. A corpus written by paraphrasing tool descriptions measures paraphrase matching, not retrieval. On the committed corpus that check found a real inflation, leaky queries scoring 24.5 points higher than clean ones, so the honest headline is the clean-subset figure rather than the flattering aggregate.

Follow-on to v1.108.210, raised by @rknighton in #398 while reviewing the code that shipped the same day.

What was collapsed

Removing count() was right and this does not restore it. But count() was SELECT COUNT(*) FROM symbol_embeddings over the whole repository, and the old code branched on count() > 0 before narrowing to candidates. Deriving everything from len(get_many(cand_ids)) >= 2 collapsed two states a caller acts on differently:

• this repository has no embeddings at all
• this repository is embedded, but fewer than two of these candidates are

The collapse was not only invisible, it was wrong out loud. Both states emitted one note — "Embeddings not available… Run embed_repo" — which is false advice for the second: the repository is embedded, and running embed_repo again over the same symbols changes nothing. That note pre-dates v1.108.210; the old count() guard never surfaced the difference either.

The fix

New EmbeddingStore.has_any() restores the repository-level answer for less than what was removed: SELECT 1 FROM symbol_embeddings LIMIT 1 on the same mode=ro&immutable=1 connection as get_all_readonly. No scan, no write, no WAL sidecars, and it runs only when no candidate vector came back — one fetched vector already proves the repository is embedded.

Three-state, not boolean. True / False / None. False is reserved for the two cases that actually mean nothing is embedded: the database file is absent, or the table is. A locked file, a corrupt page or a permission error answers None, because False there is a guess about a repository we could not read.

Structural responses now carry semantic_state (repo_not_embedded / candidates_not_embedded / unknown / not_applicable) beside a note written for that state. Hybrid responses are byte-identical to v1.108.210.

Two of three exits disclosed nothing at all. find_similar_symbols has three exits that can return a structural answer; the zero-edge exit returned mode: "structural" with no note whatsoever, so a zero-cluster answer from an un-embedded repository was indistinguishable from one from an embedded repository. All three now route through a single disclosure helper.

Also in this release

ROADMAP.md records the remaining #398 arc decisions: Arc 2 must never use immutable=1 (the un-checkpointed-WAL trade-off is specific to a channel that only adds candidates — a missed row is a false absence, not a weaker ranking); Arc 3's byte budget is the primary retention policy but does not subsume JCODEMUNCH_INDEX_CACHE_TTL (a byte budget is per-process, and the leak it would bound was a count of processes); Arc 4's gate is now a three-bucket breadth split with Django authoritative and thresholds named before the run.

Tests: tests/test_v1_108_211.py (16). Full suite green.

Arc 5 of #398, reported by @rknighton.

find_similar_symbols prefilters its candidate pairs on non-semantic evidence, then called EmbeddingStore.count() followed by get_all() and narrowed the result to cand_ids on the very next line. cand_ids is built three lines above the fetch, so the repository-wide read was waste by construction: every vector decoded from bytes into a Python float list, and almost all of them dropped.

New EmbeddingStore.get_many(ids) fetches only the named symbols.

Two details that carry the fix

Dropping count() is part of it, not a shortcut. It opened a read-WRITE connection, and _connect runs PRAGMA journal_mode = WAL plus a CREATE-TABLE on every connection, so the existence check bumped the database mtime before the read even started. A read-only get_many alone would not have kept the file untouched. get_many uses the same mode=ro&immutable=1 connection as get_all_readonly (v1.108.185); a test asserts the mtime does not move and no WAL sidecars appear.

Chunked at 900 ids. An IN (...) clause is bounded by SQLITE_MAX_VARIABLE_NUMBER, which is 999 on older SQLite builds. A candidate set large enough for this fix to matter is exactly the one that would overflow a naive single statement, so the bug would have surfaced only on the repositories the change targets. Tested at 2x the chunk plus a remainder, and on the exact boundary.

Measured here, not transcribed

The reporter measured a 2.63% to 7.65% fetch fraction on FastAPI and Django. On jCodeMunch's own index the figure is 22.48%: 2,838 of 12,625 vectors requested, 9,787 avoided, still hybrid mode. Both numbers are real and they describe different repository shapes. The saving held; the headline did not transfer. Re-measuring is house policy and is not a comment on the report.

Behavior is unchanged. embeddings ends up the same mapping the old code produced, since it was already restricted to cand_ids, and mode is still decided by len(embeddings) >= 2.

Also in this release

uv.lock is re-synced. v1.108.209 was pushed with the lock still pinning 1.108.208, and uv sync --locked refuses a stale lockfile, so CI died before running a single test on that commit. The local suite for .209 was green because it was started before the version bump and therefore validated the pre-bump tree. A version bump invalidates any suite run that preceded it.

The other four arcs of #398 are accepted or gated and now live in ROADMAP.md with close conditions and credit.

Tests: test_v1_108_210.py (12). Full suite 6412 passed, 7 skipped. CI green.

FreshnessProbe.classify had five exits that answered fresh for a comparison that never happened: no source root, a root that moved, a file gone from the working tree, a stat that raised, and a successful stat with no baseline to compare against (no per-file mtime and no parseable indexed_at).

repo_freshness grew a four-state split to fix exactly this in v1.108.180 (#377 item 4) and sits 40 lines above classify in the same file. Its docstring makes the argument that applies here verbatim: a Boolean has nowhere to put "I could not find out", and rendering that non-answer as fresh asserts a current-snapshot equivalence that was never established. The reasoning was never carried down to the per-file classifier.

The mislabeled population is the one that cannot self-diagnose: a .db-only starter pack, an index built on another machine, a source root that has since moved.

What changed

• classify() returns a new unknown state on all five unmeasurable exits.
• summary() carries an unknown count. This mattered as much as the classifier: it enumerated its three buckets explicitly, so a new state would have been counted internally and then dropped on the way out, leaving a block that summed to fewer rows than it described. A test asserts the buckets account for every entry.
• An entry carrying no _freshness at all now counts unknown rather than fresh.

What did not change

Measured answers are byte-for-byte identical. fresh, edited_uncommitted and stale_index all still come from real comparisons, and stale_index still outranks a per-file unknown because a stale index is something we did establish. Only the exits that previously guessed are affected.

Compatibility

Response-shape change inside 1.x, taken deliberately: _meta.freshness gains a key and _freshness gains a value. Two tests pinned the old fail-to-fresh behaviour by name and were updated with the reasoning inline rather than deleted, since it was pinned on purpose and this is a contract decision rather than a slip.

Provenance

Prompted by external recon, with the adjacent defect explicitly not ported: a same-lane project shipped a fix for serving a mis-sliced symbol body from a file that had drifted from its index. That cannot happen here. get_symbol_content seeks stored byte offsets into the content-cache copy, so the bytes and the offsets always come from one snapshot; jCodeMunch can serve stale, it cannot mis-slice. The freshness disclosure gap was the real finding.

Tests: test_v1_108_209.py (14), including a serve-path test that indexes a tree, deletes the tree, and asserts get_symbol_source still serves the cached body while reporting _freshness: "unknown". Full suite 6400 passed, 7 skipped, 0 failed.

content_hash stops riding every get_symbol_source response

get_symbol_source put the stored content_hash on every served row, next to
signature and source, and not gated on verify. It is a full 64-character
SHA-256, so it cost roughly 83 characters per symbol returned on the most-called
read tool in the server. A twenty-symbol batch spent on the order of 400 tokens
on it.

Nothing was offered a use for the raw digest. verify already answers the drift
question as a boolean (content_verified), get_changed_symbols answers it
across a diff, and an evidence receipt carries content_sha256 out of band with
only the id on the wire. It is now emitted only when verify or receipt is
requested, and both of those responses are byte-for-byte unchanged.

Why the strip lives in the dispatcher

_row_subject in the evidence producer reads the digest off the served row
and deliberately never re-reads the index, so that a receipt describes what the
caller was actually handed. Gating the field inside the tool would therefore have
kept minting valid-looking receipts whose hash_source had quietly downgraded
from index_content_hash to served_bytes: every field correct except the
provenance of the one that matters, with nothing raising.

The strip runs after the mint block instead, which makes the receipt immune by
construction rather than by remembering to thread a flag, and follows the same
ordering rule as the presentation filtering beside it: a display preference must
not decide what a scan proved. There is a comment at the emission site saying not
to gate it there, and a test at the consumer that fails under the tool-side
version.

Upgrade note

This is a response-shape change inside 1.x, taken deliberately. No test asserted
the field in a tool response. If you read content_hash off a plain
get_symbol_source result, add verify: true and you get it back along with the
comparison it implies.

Tests: 7 new, full suite 6386 passed / 7 skipped.

The first native embedding import has to happen on the main thread

Ported from jdatamunch-mcp#3, reported by @MotoMato85 against jDataMunch with the root cause and the fix both isolated in the report. Reproduced here before porting, rather than assumed from the family resemblance.

Over stdio on Windows, check_embedding_drift and embed_repo never returned. The lazy import onnxruntime / import sentence_transformers runs inside the asyncio.to_thread worker that call_tool dispatches into, so native DLLs load off the main thread while the main thread is servicing the transport. That is a Windows loader-lock deadlock: no error, no timeout, the call simply never completes.

jCodeMunch's trigger is local_onnx, the zero-config priority-0 provider, so no env var was needed to hit it — every Windows install with the [local-embed] extra was exposed on its first semantic operation.

main() now calls the new warm_up_embedding_backend() above both serve dispatch branches, so every transport is covered once. It imports whichever native backend _detect_provider selects, on the main thread, before any event loop starts. The network-backed providers load no native code and stay lazy. It never raises — a broken onnxruntime must not stop the server from starting — and JCODEMUNCH_EAGER_EMBED_IMPORT=0 opts out. The startup cost is disclosed in the README's background-behavior section.

Measured with a standalone stdio ClientSession probe: before, no return past a 90s timeout; after, 1.17s.

jDocMunch was checked and is not affected — it already imports its backend during initialize, on the main thread.

Tests: tests/test_v1_108_207.py (10), including an AST guard that the warm-up precedes every serve-transport dispatch.

init wrote a policy for a decision it had not made yet

Reported by @n-able-consulting in #397, with a throwaway-HOME reproduction and a raw tools/list probe of the same install.

On a first-ever install init wrote a 74-line CLAUDE.md naming ~25 tools by name, while the server advertised the 3-tool front door. Five of the six tools the policy actually invoked were absent from tools/list, at a measured 964 tokens of per-turn context describing a workflow the client never offered the model.

The cause sat a layer below the symptom: init never called load_config. Every config read returned a built-in default rather than the user's file, which produces both halves of the report. A tool_profile of "core" still got the full-profile policy, because the profile read never saw the config. And on a fresh install there is no config yet at all: the server creates it on first start, a genuinely-new install gets tool_surface: "counter", and init runs first. Two code paths deciding one thing, in the wrong order.

init now loads the config before it generates anything, which makes it the point where that decision is taken and observed. _get_active_tools reads tool_surface alongside tool_profile and disabled_tools, and every policy writer routes through a single active_policy() so the choice cannot be made two ways.

Under the front door the policy is a different document rather than a filtered one: 28 lines teaching order / menu / route, because filtering the direct-tool policy down to its three surviving names leaves an agent with no workflow at all.

Measured on a fresh install, the same way it was reported:

The full surface keeps its existing 74-line policy, asserted by a test rather than assumed.

⚠ Hidden tools stay callable by name. A direct tools/call on an unlisted tool returns a correct result, so nothing ever errored. The guidance was unreachable through the tool list rather than broken, which is why it survived this long.

jcodemunch_guide returned the full catalogue under the front door

Its description claimed to honor tier filtering; the snippet body honored neither that nor the surface, so an agent keeping the recommended one-line CLAUDE.md was handed direct-tool guidance on a server advertising three tools. It now returns the workflow matching the active surface, in fewer tokens than the text it replaces.

--minimal --hooks installed no hooks

--minimal assigned over a value the caller had already set, so a hardened install wanting hooks without the policy paste had to let init write the sections and delete them afterwards.

⚠ The obvious fix breaks the opposite contract: install <agent> --minimal passes hooks=True as its own hardcoded default and must still install nothing. A plain bool cannot separate a caller's default from a flag a human typed, so --hooks now parses with default=None and the CLI passes an explicit signal. Both directions are pinned by tests that pull opposite ways.

Also

run_init is wrapped in with_scoped_config, so loading the config no longer swaps process-global state under an in-process caller.

Upgrading

pip install --upgrade jcodemunch-mcp

Existing installs are unaffected: they are on the full surface with an accurate policy already. Re-run jcodemunch-mcp init to regenerate guidance matching whatever surface you are actually on.

Two changes ship in this release. The second one reached the release commit through a git add that swept a directory rather than naming files; it is complete and tested, and it is changelogged here rather than under the version its tests are named for.

Packs now say whose code they contain, and under what terms

Every packed repo is third-party open source and nine of the ten packs are sold, so a pack redistributes someone else's work commercially. It had been shipping docstrings, which are verbatim source text, with no attribution attached at all.

Packs now carry each upstream's licence, AUTHORS and NOTICE files byte-for-byte under licenses/<owner>-<name>/. The manifest records the SPDX id, the files carried, a digest over them, and the commit the pack was built from. install-pack extracts them alongside the index and prints the terms and where the full text landed; install-pack --list names the licences before a download rather than only after one. Both displays stay silent against a catalog or pack built before this field exists, so an older pack prints nothing rather than a guess.

Checking rather than transcribing changed two of the fifteen answers. nodejs/node ships a 157 KB compendium, MIT for Node itself followed by the licences of every bundled dependency including V8, ICU and OpenSSL. modelcontextprotocol/typescript-sdk is mid-relicensing from MIT to Apache-2.0, with per-contribution status varying and documentation under CC-BY-4.0. GitHub's own detector returns NOASSERTION for both, and a transcription would have recorded each as plain MIT.

The pipeline half lives in jgravelle/jcodemunch-starter-packs, where the builder refuses to package a repo with no licence at its root, or one whose attribution bytes moved since the last build. A blocked pack keeps shipping its previous build, which is the safe direction: we already held the rights that one was built under.

Known gap: the serving layer whitelists catalog fields and currently drops licenses, which is a separate deploy. Until that lands, the post-install report is the one that works and --list simply omits the line.

Call-graph traversals stop rebuilding the same map at every node

Reported and measured by @rknighton in #396, with 636 raw timing rows attached.

_callees_from_references rebuilt a whole-repository name-to-definitions map on every visited node carrying call references, making a traversal O(nodes x symbols) where O(symbols) would do. _CalleeNameIndex memoises it for the life of one traversal, built on first lookup rather than in the constructor so direction="callers", which never resolves a callee, is not charged a full symbol pass.

His measurement covers the traversal rather than the function (direction="both", depth=3, session result cache invalidated per sample, 22 pairs): django 9.25s to 1.26s over 45,561 symbols and 244 reference-carrying nodes, and fastapi 399ms to 315ms. Reproduced against jCodeMunch's own index with the baseline produced by disabling the memo's identity check so every node rebuilds exactly as 1.108.204 did: finalize_handoff and its 122 callees went 2075.6ms to 899.3ms, with result shapes identical in both arms.

Stated limit, kept from his measurement: a Q=1 express control's 99% interval on the paired saving spans zero, and our own low-fan-out control moved 710.5ms to 631.0ms with overlapping minima. The gain tracks the number of nodes that actually resolve callees. A small or narrow traversal sees none of it.

get_signal_chains gets the memo too, built once for the whole run rather than inside its BFS helper, since that path runs its own loop instead of going through bfs_callees and would otherwise rebuild per gateway.

Correctness is held against an oracle, 1.108.204's resolver copied verbatim, compared list-for-list over 3,000 randomized fixtures plus adversarial ones. The duplicate-name behaviour here is deliberately asymmetric, with same-file resolution emitting every definition and cross-file emitting the first per file, so a faster lookup returning one same-file match would read as a win and be a regression.

A body the index cannot produce is now reported instead of returned as ""

Resolving a symbol and producing its source are two different successes, and an index can manage the first without the second: the row lives in the .db while the bytes live in a separate content directory. When that directory is missing, get_symbol_source returned the row with source: "" under _meta.verdict.state: "ok", channels.index: "fresh" and the note "Confident matches returned." Every field except the one the caller asked for was correct, and nothing said so.

Measured against the published nodejs starter pack, installed into an empty store with none of its repos checked out: bodies came back for 0 of 50 symbols, each one under a confident verdict. Starter packs carry .db files only, so every pack install has this shape, but the pack is only the loudest case. A pruned cache, or a .db copied without its sibling directory, produces it too.

A resolved symbol whose body cannot be read now carries source_status: "content_cache_missing" plus a source_unavailable_reason naming the path and the fix, and the verdict degrades:

{"state": "degraded",
 "channels": {"index": "fresh", "content_cache": "missing"},
 "unavailable_source_count": 1,
 "note": "1 symbol resolved but the body could not be read... `source` is empty because it is unavailable, NOT because the symbol is empty."}

Batch mode adds _meta.unavailable_source_ids. The row itself is still returned in full, because signature, line range and docstring come from the .db and are accurate: refusing the body is not refusing the answer.

get_context_bundle gets the same label. It is what get_symbol_source's own hint points callers to, so fixing one and not the other would have moved the defect one call to the right rather than closing it.

This keys on None (could not read) rather than falsiness, so a genuinely zero-length body is untouched. A healthy store behaves exactly as before, and a test asserts that rather than only asserting the new behaviour.

install-pack --list printed a URL it was not going to use

The catalog rows showed the server's own download_url field, which the live server still composes from the host v1.108.203 moved off. The download itself was already correct, so the tool fetched from one place and printed another, and the printed line is the one a user copies. It is now composed from STARTER_PACK_API, so it is the URL install-pack will actually fetch, by construction.

Upgrading

pip install --upgrade jcodemunch-mcp

If you installed a starter pack before v1.108.203, reinstall it: jcodemunch-mcp install-pack <id> --force.