Skip to content

v1.3.1

Latest
Latest

Choose a tag to compare

View all tags
@github-actions github-actions released this 27 Jul 03:06
v1.3.1
4aed5e9

v1.3.1 — Cold-start hardening

A stability and correctness release. v1.3.0 shipped the cold-start campaign
(snapshot persistence, serve-before-warm staging, embed cache); v1.3.1 is that
campaign hardened by five independent re-audit rounds, each one auditing the
previous round's fixes. 44 commits, all bug-fix / perf / test — no new
features, no breaking changes. Every fix landed on main under full CI
(build · vet · gofmt · golangci-lint · go test -race · Windows smoke · DB
integration on Postgres/MySQL/MariaDB/SQLite), green across all five rounds.

Upgrading from v1.3.0 is a drop-in. Most fixes are in the long-lived ken-mcp
watch/staging paths, so servers that stay up across file changes and restarts
benefit most.

Highlights

  • No more staged-boot crash. A KEN_MCP_STAGED=1 server that loaded a
    snapshot on its second start could panic during startup (vectors/chunks fell
    out of alignment). Fixed, with a defensive path that degrades to BM25 instead
    of ever crashing on a misaligned corpus.
  • Search results are the source, not a synthetic line. A pre-existing
    .ken/index.bin built before v1.3.0's enrichment-label change could serve a
    fabricated # func: … first line to the agent as if it were real code. The
    on-disk format version is now bumped so those stale prebuilts are rejected and
    rebuilt.
  • Correct semantic ranking with a database configured. In the narrow
    staged-warmup window with KEN_DB_DSN set, find_related / semantic search
    could rank filesystem chunks by database vectors. Now detected and downgraded
    to BM25 until the real vectors land.
  • Snapshots actually re-persist. A staged server that reconciled a snapshot
    on boot never re-wrote it, so drift grew every restart until it forced the full
    rebuild snapshots exist to avoid. Fixed.
  • Crash-durable writes. Snapshot, prebuilt-index, rerank-cache, and
    downloaded-model writes now fsync the file before rename and the parent
    directory after — a power loss can no longer publish a torn file that passes
    its in-memory checks but fails to load.
  • Honest database errors and tool surface. reindex_db no longer appears on
    filesystem-only servers, and when a DB operation fails the full error is
    actually written to the server log the sanitized message tells you to check
    (credentials never leak to the model).

By area

Watch / incremental indexing

  • Fixed a debounce busy-spin that could peg a core during a slow flush.
  • A stray delete of a never-indexed file (an editor .swp, .orig, .DS_Store)
    no longer triggers a full rebuild + snapshot rewrite.
  • A directory renamed during an in-flight flush now correctly drops its stale
    chunks instead of orphaning them.
  • Recovery from a dropped-event (inotify overflow) storm now reconciles deletions
    against the authoritative in-memory corpus, catching files it previously missed.
  • Enrichment labels no longer double on each restart; the label now uses an
    unambiguous # ken: sentinel.

Cold start (v1.3.0 campaign)

  • Staged embedding, snapshot reconcile, and the embed cache all hardened per the
    fixes above; snapshot config keys now version and cover lazy/staged modes.

MCP server

  • reindex_db registered only when a database is actually configured.
  • Structural-index background builds are cancelled when their repo is evicted
    from the cache (no more parse goroutine walking a deleted temp clone).
  • Outline/symbols responses signal when results were truncated (overshot).

CLI / distribution

  • ken build-index output is world-readable again (0644), so a prebuilt
    .ken/index.bin baked into a container image loads under a non-root uid
    instead of silently falling back to a full cold build.

Performance

  • Index serialization holds ~2× less peak memory on large corpora.
  • Token hashing on the watch path is now zero-allocation.

Process note

The residual risk after five rounds isn't in the code — it's that the audit was
performed in a sandbox that couldn't compile the project. That gap is already
closed on main: CI builds and runs the full race-enabled suite (plus a live
database matrix) on every push, and it is green on this release.

Full commit changelog (auto-generated)

Changelog

Fixes

  • 1a3cd39: fix(db): bound startup introspection with KEN_DB_STARTUP_TIMEOUT (audit db/mcp §3) (@townsendmerino)
  • 7045bcd: fix(db): introspection determinism + cleanup (audit db/mcp §12, §15, §17, §25, §29) (@townsendmerino)
  • 1957fd0: fix(db): sampler robustness — pool churn, BLOBs, FK PKs, panics, timeouts (audit db/mcp §10,§13,§14,§26,§27,§28) (@townsendmerino)
  • 4b58a10: fix(ken-mcp): Tier-2 DB chunks survive default-repo LRU eviction (audit db/mcp §1) (@townsendmerino)
  • 9fc85c4: fix(lint): enable rowserrcheck + sqlclosecheck; fix the 6 leaks they caught (audit db/mcp §20) (@townsendmerino)
  • 1479661: fix(mcp): embedded find_related error mode, MCP rerank validation, per-clone byte cap (audit db/mcp §21, §22, §24) (@townsendmerino)
  • 656de7c: fix(mcp): single-pass outline + error survives bad output mode (audit db/mcp §16, §23) (@townsendmerino)
  • f295e4d: fix(mcp): strip Arm B enrichment label from search results (audit search §10) (@townsendmerino)
  • 034fadd: fix(mcp): structural build honors ctx, heartbeats, and retries on failure (audit db/mcp §8) (@townsendmerino)
  • f47a850: fix(mcp): tree-diff recently_changed + honor ctx; cap outline/symbols dumps (audit §5, §6) (@townsendmerino)
  • e8638e4: fix(pull-forward): db#4 allowlist, §21 vecs peak, §27 fsync, R14 comment (round-3 leftovers) (@townsendmerino)
  • f4e210c: fix(regressions): N1 staged panic, N2 orphaned dir-rename chunks, N8 debounce bypass; make the slow-flush test bite (round-3 re-audit) (@townsendmerino)
  • f9afc35: fix(regressions): N3 snapshot key/lock, N4 label sentinel (round-3 re-audit) (@townsendmerino)
  • 3bcffb9: fix(regressions): N5 zero-alloc token hash, N6 purge retry, N7 structural-build lifecycle (round-3 re-audit) (@townsendmerino)
  • 59c714b: fix(regressions): N9 overshoot signal in JSON responses (round-3 re-audit) (@townsendmerino)
  • 36f6945: fix(regressions): R1 busy-spin, R2 label doubling, R3 structural never-converges, R4 bogus row count, R5 label strip (2026-07-26 re-audit) (@townsendmerino)
  • 388d935: fix(regressions): R10 usage rotation recovery, R11 token-cache hash key (2026-07-26 re-audit) (@townsendmerino)
  • 7d35d3e: fix(regressions): R12-R15 + leftover watcher/credential holes (2026-07-26 re-audit) (@townsendmerino)
  • ca580ee: fix(regressions): R4-1 staged-boot panic, R4-3 no-op flush, R4-6 resync — round-4 re-audit (@townsendmerino)
  • 730d1d9: fix(regressions): R4-2 bump KEN1 serializeFormatVersion to 2 (round-4 re-audit) (@townsendmerino)
  • 08f3549: fix(regressions): R4-4 route LRU eviction through reapEntry (round-4 re-audit) (@townsendmerino)
  • 72384b5: fix(regressions): R4-5 reindex diagnostic logging, R4-6 remaining fsync paths (round-4 re-audit) (@townsendmerino)
  • a93168b: fix(regressions): R5-1 merged-extras vecs/chunks invariant + round-5 test guards (search) (@townsendmerino)
  • e9ea01e: fix(regressions): R5-2 guard typed-nil DBIntegration + log full reindex_db error (@townsendmerino)
  • 3e06867: fix(regressions): R5-3 gate reconcile-boot snapshot persist on wi.Warming() (@townsendmerino)
  • 4aed5e9: fix(regressions): R5-4 restore 0644 on ken build-index output (@townsendmerino)
  • d10a28d: fix(regressions): R6 staged-embed vecs misalign, R7 purge repopulates stale bundle (2026-07-26 re-audit) (@townsendmerino)
  • 4e49b7f: fix(regressions): R8 event loop no longer blocks on corpusMu, R9 gitignore-gate new dirs (2026-07-26 re-audit) (@townsendmerino)
  • ae2e3ee: fix(repo): gitignore ** respects path-component boundaries (audit search §16) (@townsendmerino)
  • 60e1370: fix(search): copy-on-write in reconcileCorpusLocked — kill the flush data race (audit §1) (@townsendmerino)
  • 9bdecab: fix(search): tombstone over-fetch, no-panic extras, synced lazy reads, misc Lows (audit §11,§23,§24,§25,§26,§29,§30) (@townsendmerino)
  • 9aca434: fix(search): unique temp name for rerank-cache atomic write (audit search §27) (@townsendmerino)
  • 17d09d2: fix(security): close three credential/topology leaks (audit db/mcp #2, #4, #7) (@townsendmerino)
  • 3aa62a2: fix(structural): case-insensitive extension lookup + correct determinism comment (audit search §18, §28) (@townsendmerino)
  • 38366ad: fix(usage): local-tz day boundary + cached append handle (audit db/mcp §9, §19) (@townsendmerino)
  • 71bd780: fix(watch): deterministic flush order, size re-check, debounce ceiling (audit search §13, §15, §17) (@townsendmerino)
  • 9cbacf3: fix(watch): index new/moved-in dirs, re-path renamed dirs (audit §2, §14) (@townsendmerino)
  • 42afd84: fix(watch): recover from fsnotify overflow; tolerate watch-registration failures (audit search §3, §4) (@townsendmerino)

Others

  • c9f46a7: perf(db): parallelize the Postgres row sampler over a pool (audit db/mcp §11) (@townsendmerino)
  • 10ac79e: perf(search): O(1) rerank min + single precompiled definition pattern (audit search §6, §7) (@townsendmerino)
  • 6be2ab9: perf(search): incremental + parallel BM25 tokenization on watch flush (audit §5) (@townsendmerino)
  • 508361e: perf(search): serialize into one buffer, pack vecs in bulk (audit search §21) (@townsendmerino)
  • 43ace00: perf(watch): run flush async with single-flight so the event loop keeps draining (audit search §12) (@townsendmerino)
  • 6fd812c: test(search): make the N2 bite test faithful cross-platform (round-4 verify) (@townsendmerino)

Contributors

townsendmerino
Assets 9
Loading