Cycle-close tidy for the packed-versioned-index-bundle cycle (issues #3 + #4,
both shipped and closed). Architect drift review over d83b10f..HEAD against the
spec + glossary (no paths.design_ledger / project CLAUDE.md configured, so spec
+ glossary are the only references; the regression gate is empty —
commands.regression: [] — so architect was the sole cycle-close gate).
Architect verdict: drift_found, minor — no spec contract violated. Format,
header schema, on-disk layout, and the load_packed_or_store validation order
(n_rows → model → corpus hash) match the spec exactly; both #3 and #4
acceptance criteria are test-evidenced; no glossary term drift; the
IndexStatus / degraded orthogonality is honored.
Resolutions:
- [medium] Missing positional-alignment test. The spec § Testing strategy
promised a "row i == store.get(entries[i].text)" guard, and it had not
shipped — the load-bearing invariant that licenses dropping the per-row
SHA256 key (row i = corpus entry i) was unprotected. FIXED: added
tests/pack_cli_tests.rs::cli_packed_rows_are_in_corpus_order, which packs a
store with order-revealing orthogonal vectors via the CLI, reads the packed
file back, and asserts the rows follow corpus-load order (swapped order
fails the test).
- [low] Stale IndexStatus doc comments still said "structurally valid/invalid"
though the variants now also carry semantic model/corpus mismatch. FIXED:
refreshed the Ok / Mismatch comments in src/model.rs.
- [low] The `index --pack` writer logic lives inline in main rather than in the
packed module, leaving its header construction outside library test reach.
CARRY-ON: filed as backlog issue #5 (idea/debt) — behaviour is correct and
CLI-test-covered; the refactor is a self-contained testability improvement,
not forced into this tidy.
- [note] No design ledger / project CLAUDE.md configured; drift is measurable
only against spec + glossary. Recorded as an infrastructure observation.
Regression: no scripts configured (no-op). Full suite green (68 tests, +1 for
the alignment guard). cycle packed-versioned-bundle tidy (drift-clean).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The semantic index was ~90 896 per-entry files (index/embeddings/<safe-model>/
<sha256>.f32), a deployment liability: the file count — not the 356 MB — is
hostile to container layers, object storage, and Git-LFS, and Pipeline::load
opened every one individually. This packs a complete per-file store into one
self-describing matrix file loaded in a single read.
New src/packed.rs owns the format: 4-byte magic "AIPK", u32 LE format_version,
u32 LE header length, a JSON PackedHeader {n_rows, dim, embed_model,
corpus_sha256, corpus_name}, pad to a 4-byte boundary, then row-major
little-endian f32 payload in corpus order (row i = corpus entry i). write_packed
is atomic (temp+rename); read_packed rejects a wrong magic, a newer-than-
supported version, a malformed header, or a payload length inconsistent with
n_rows×dim. corpus_sha256 hashes the raw corpus file bytes.
Pipeline::load now calls a free fn load_packed_or_store(cfg, entries) that
prefers the packed file and falls back to today's all-or-nothing per-file store,
reporting which path it took via a new IndexStatus {Ok, Absent, Mismatch}
carried into Diagnostics. `alpha-id index --pack` writes the file from a
complete store (refusing an incomplete one with exit 2), recording corpus_sha256
+ embed_model in the header as the artifact-binding data.
Iteration boundary (deliberate): this is iteration 1 of the packed-versioned-
bundle cycle. load_packed_or_store validates the packed file STRUCTURALLY only
(magic/version/payload length via read_packed, plus n_rows == entries.len());
it does NOT yet compare the header's corpus_sha256/embed_model against the live
config, and IndexStatus::Mismatch fires only on structural corruption. The
load-time mismatch warning, the index= token on the diagnostics line, and the
Hybrid-degrade-names-reason wiring are iteration 2 (issue #4), which consumes
the binding data this commit writes. The per-file store is untouched — it stays
the build/resume path; the packed file is an additive serve-path artifact.
mmap was considered and rejected: the brute-force full-scan cosine search
touches every row per query, so memory-mapping yields no runtime benefit over a
single read and would add an unsafe dependency (spec § Out of scope).
Verification: cargo test green (61 tests; 9 new — 3 format round-trip/hash/bad-
magic, 4 load-path incl. packed-beats-store and structural n_rows mismatch, 2
--pack CLI incl. exit-2 refusal). An end-to-end seam test writes the packed
file via the CLI, removes the per-file store, and asserts the pipeline builds
its index from the packed file alone as IndexStatus::Ok (#3 acceptance b).
Spec: docs/specs/2026-05-31-packed-versioned-index-bundle.md
Plan: docs/plans/2026-06-01-packed-versioned-bundle-iter1.md
closes#3
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Brummel