Design spec for the "versioned bundle" cycle binding issues #3 (pack the
per-file embedding store into one contiguous matrix file) and #4 (bind
{corpus snapshot, embed model, index} as one versioned artifact).
Ratified decisions (user-approved across the brainstorm):
- Approach 1: single self-describing file (magic + JSON-header manifest +
row-major f32 payload), loaded with one fs::read. No mmap (a brute-force
full-scan cosine index touches every row per query, so mmap yields no
runtime benefit and is rejected to avoid an unsafe dependency). VectorIndex
and the per-file store are untouched — the store stays the build/resume
path, the packed file is an additive serve-path artifact.
- Mismatch posture: loud but not fatal. A structurally wrong index (corpus
sha256 or embed_model disagreement) builds no VectorIndex and surfaces as a
distinct IndexStatus::Mismatch in diagnostics plus a load-time warning,
instead of silently degrading Mode Hybrid to Lexical. Mode Lexical, which
never needs the index, keeps serving.
- Two iterations, one format: iter 1 delivers the format + `index --pack` +
packed-first load; iter 2 enforces the manifest check and the observable
index_status surface.
grounding-check: PASS (7 load-bearing assumptions ratified by green tests).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Brummel
2bb1bfc534