Files
alpha-id/docs/plans/2026-06-01-packed-versioned-bundle-iter1.md
Brummel 2489d2029e plan: packed index format iteration 1 (refs #3)
Bite-sized TDD plan for iteration 1 of the packed-versioned-bundle cycle:
the on-disk packed format (src/packed.rs), the IndexStatus enum + packed-first
Pipeline::load path with per-file-store fallback, and the `index --pack`
subcommand. Structural validation only (magic/version/n_rows/payload length);
the semantic corpus+model mismatch enforcement, load warning, and index=
diagnostics token are deferred to iteration 2 (#4), as documented in the
plan's iteration-boundary note.

refs #3

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-01 00:24:15 +02:00

652 lines
23 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Packed Index Format (Iteration 1) — Implementation Plan
> **Parent spec:** `docs/specs/2026-05-31-packed-versioned-index-bundle.md`
>
> **For agentic workers:** REQUIRED SUB-SKILL: use the `implement`
> skill to run this plan. Steps use `- [ ]` checkboxes for tracking.
**Goal:** Pack a complete per-file embedding store into one
self-describing matrix file and load the semantic index from it in a
single read, with the per-file store kept as a fallback.
**Architecture:** A new `src/packed.rs` owns the on-disk format (magic +
JSON header + row-major f32 payload) and the corpus hash. `Pipeline::load`
calls a new free fn `load_packed_or_store` that prefers the packed file
(structural validation only) and falls back to today's per-file store,
reporting which path it took via a new `IndexStatus`. An `index --pack`
subcommand writes the file from a complete store.
**Tech Stack:** Rust; `serde`/`serde_json` (header), `sha2` (corpus hash) —
both already in `Cargo.toml`; `clap` (the `--pack` flag); `tempfile`
(test fixtures).
**Iteration boundary (load-bearing — do not exceed):** This plan is
Iteration 1 (issue #3, acceptance a/b/c). The packed header *carries*
`corpus_sha256` and `embed_model`, but `load_packed_or_store` validates
only the file's *structure* (magic, supported version, `n_rows ==
entries.len()`, payload length) — it does **not** compare `corpus_sha256`
or `embed_model` against the live config. The semantic mismatch check, the
load-time warning, the `index=` token on the diagnostics line, and the
Hybrid-degrade-names-reason wiring are Iteration 2 (issue #4) and are OUT
OF SCOPE here. `IndexStatus::Mismatch` is produced in this iteration *only*
for structural corruption.
---
**Files this plan creates or modifies:**
- Create: `src/packed.rs` — packed file format I/O: `PackedHeader`,
`write_packed`, `read_packed`, `packed_path`, `corpus_sha256`.
- Modify: `src/lib.rs:13` — add `pub mod packed;` to the module list.
- Modify: `src/model.rs:80-81,114-121``IndexStatus` enum;
`index_status` field on `Diagnostics`.
- Modify: `src/pipeline.rs:15-23,26-60,229-235``load_packed_or_store`
free fn replacing the `:43-50` match; `Pipeline.index_status` field;
`suggest` copies it into `Diagnostics`.
- Modify: `src/bin/alpha_id.rs:18,83-125``--pack` flag on `Index` and
its handler branch.
- Test: `tests/packed_tests.rs` — format round-trip, `corpus_sha256`,
bad-magic rejection.
- Test: `tests/packed_load_tests.rs``IndexStatus` default/serde,
packed-beats-store, structural `n_rows` mismatch.
- Test: `tests/pack_cli_tests.rs``--pack` refuses an incomplete store
(exit 2) and writes a file from a complete store.
---
### Task 1: `src/packed.rs` — packed file format + corpus hash
**Files:**
- Create: `src/packed.rs`
- Modify: `src/lib.rs:13`
- Test: `tests/packed_tests.rs`
- [ ] **Step 1: Write the failing tests**
Create `tests/packed_tests.rs`:
```rust
use alpha_id::packed::{corpus_sha256, read_packed, write_packed, PackedHeader};
use std::io::Write;
fn header(n_rows: u32, dim: u32) -> PackedHeader {
PackedHeader {
n_rows,
dim,
embed_model: "test/model".to_string(),
corpus_sha256: "deadbeef".to_string(),
corpus_name: "corpus.txt".to_string(),
}
}
#[test]
fn write_read_roundtrip_preserves_header_and_rows() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("packed.bin");
let rows = vec![vec![0.0f32, 1.5, -2.25], vec![3.0, 4.0, 5.0]];
let h = header(2, 3);
write_packed(&path, &h, &rows).unwrap();
let (got_h, got_rows) = read_packed(&path).unwrap();
assert_eq!(got_h, h);
assert_eq!(got_rows, rows);
}
#[test]
fn corpus_sha256_stable_and_sensitive() {
let dir = tempfile::tempdir().unwrap();
let p = dir.path().join("c.txt");
std::fs::write(&p, b"alpha|beta").unwrap();
let a = corpus_sha256(p.to_str().unwrap()).unwrap();
let b = corpus_sha256(p.to_str().unwrap()).unwrap();
assert_eq!(a, b, "same bytes hash identically");
std::fs::write(&p, b"alpha|betb").unwrap();
let c = corpus_sha256(p.to_str().unwrap()).unwrap();
assert_ne!(a, c, "one changed byte changes the hash");
}
#[test]
fn read_packed_rejects_bad_magic() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("bad.bin");
let mut f = std::fs::File::create(&path).unwrap();
f.write_all(b"XXXX____________").unwrap();
assert!(read_packed(&path).is_err());
}
```
- [ ] **Step 2: Run tests to verify they fail**
Run: `cargo test --test packed_tests`
Expected: FAIL — compile error, `unresolved import alpha_id::packed`
(the module does not exist yet).
- [ ] **Step 3: Create `src/packed.rs`**
```rust
use crate::model::AppError;
use serde::{Deserialize, Serialize};
use sha2::{Digest, Sha256};
use std::fs;
use std::path::{Path, PathBuf};
const MAGIC: &[u8; 4] = b"AIPK";
pub const PACKED_FORMAT_VERSION: u32 = 1;
/// Manifest written into the packed file header. `embed_model` and
/// `corpus_sha256` bind the matrix to the corpus snapshot and model it was
/// built from; iteration 1 records them, iteration 2 enforces them.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct PackedHeader {
pub n_rows: u32,
pub dim: u32,
pub embed_model: String,
pub corpus_sha256: String,
pub corpus_name: String,
}
/// Derived packed-file path `<index_dir>/packed.<safe-model>.bin`, using the
/// same `/`→`_` model sanitization `EmbeddingStore::open` applies, so `--pack`
/// and `load` compute the identical path from the same config.
pub fn packed_path(index_dir: &str, model: &str) -> PathBuf {
let safe = model.replace('/', "_");
PathBuf::from(index_dir).join(format!("packed.{safe}.bin"))
}
/// SHA-256 of the raw bytes of the corpus file at `path`, lower-case hex.
pub fn corpus_sha256(path: &str) -> Result<String, AppError> {
let bytes = fs::read(path).map_err(|e| AppError::Io(format!("{path}: {e}")))?;
let mut h = Sha256::new();
h.update(&bytes);
Ok(h.finalize().iter().map(|x| format!("{:02x}", x)).collect())
}
/// Write the packed matrix: magic + format_version + header_len + JSON header
/// + pad-to-4 + row-major little-endian f32 payload. Atomic via temp+rename.
pub fn write_packed(path: &Path, header: &PackedHeader, rows: &[Vec<f32>]) -> Result<(), AppError> {
let json = serde_json::to_vec(header)
.map_err(|e| AppError::Parse(format!("packed header: {e}")))?;
let mut out = Vec::new();
out.extend_from_slice(MAGIC);
out.extend_from_slice(&PACKED_FORMAT_VERSION.to_le_bytes());
out.extend_from_slice(&(json.len() as u32).to_le_bytes());
out.extend_from_slice(&json);
while out.len() % 4 != 0 {
out.push(0);
}
for row in rows {
for f in row {
out.extend_from_slice(&f.to_le_bytes());
}
}
let tmp = path.with_extension("tmp");
fs::write(&tmp, &out).map_err(|e| AppError::Io(format!("{}: {e}", tmp.display())))?;
fs::rename(&tmp, path).map_err(|e| AppError::Io(format!("{}: {e}", path.display())))?;
Ok(())
}
/// Read a packed matrix back into its header and row vectors. Fails on a
/// wrong magic, an unsupported (newer) format version, a malformed or
/// truncated header, or a payload length inconsistent with `n_rows × dim`.
pub fn read_packed(path: &Path) -> Result<(PackedHeader, Vec<Vec<f32>>), AppError> {
let bytes = fs::read(path).map_err(|e| AppError::Io(format!("{}: {e}", path.display())))?;
if bytes.len() < 12 || &bytes[0..4] != MAGIC {
return Err(AppError::Parse(format!("{}: bad magic", path.display())));
}
let version = u32::from_le_bytes([bytes[4], bytes[5], bytes[6], bytes[7]]);
if version > PACKED_FORMAT_VERSION {
return Err(AppError::Parse(format!(
"{}: unsupported packed format version {version}",
path.display()
)));
}
let header_len = u32::from_le_bytes([bytes[8], bytes[9], bytes[10], bytes[11]]) as usize;
let header_end = 12 + header_len;
if bytes.len() < header_end {
return Err(AppError::Parse(format!("{}: truncated header", path.display())));
}
let header: PackedHeader = serde_json::from_slice(&bytes[12..header_end])
.map_err(|e| AppError::Parse(format!("{}: header: {e}", path.display())))?;
let mut payload_start = header_end;
while payload_start % 4 != 0 {
payload_start += 1;
}
let n = header.n_rows as usize;
let dim = header.dim as usize;
let expected = n * dim * 4;
if bytes.len() - payload_start != expected {
return Err(AppError::Parse(format!(
"{}: payload length {} != n_rows×dim×4 {}",
path.display(),
bytes.len() - payload_start,
expected
)));
}
let mut rows = Vec::with_capacity(n);
let mut off = payload_start;
for _ in 0..n {
let mut row = Vec::with_capacity(dim);
for _ in 0..dim {
row.push(f32::from_le_bytes([
bytes[off],
bytes[off + 1],
bytes[off + 2],
bytes[off + 3],
]));
off += 4;
}
rows.push(row);
}
Ok((header, rows))
}
```
- [ ] **Step 4: Register the module in `src/lib.rs`**
`src/lib.rs:13` currently ends the module list with `pub mod pipeline;`
(line 13) and `pub mod eval;` (line 14). Add one line so the list reads:
```rust
pub mod packed;
pub mod pipeline;
pub mod eval;
```
- [ ] **Step 5: Run tests to verify they pass**
Run: `cargo test --test packed_tests`
Expected: PASS — 3 passed.
---
### Task 2: `IndexStatus` + packed-first load path + diagnostics plumbing
**Files:**
- Modify: `src/model.rs:80-81,114-121`
- Modify: `src/pipeline.rs:15-23,26-60,229-235`
- Test: `tests/packed_load_tests.rs`
- [ ] **Step 1: Write the failing tests**
Create `tests/packed_load_tests.rs`:
```rust
use alpha_id::model::{AlphaIdEntry, Config, IndexStatus};
use alpha_id::packed::{packed_path, write_packed, PackedHeader};
use alpha_id::pipeline::load_packed_or_store;
fn entry(text: &str) -> AlphaIdEntry {
AlphaIdEntry {
alpha_id: "X".to_string(),
valid: true,
icd_primary: "A00".to_string(),
icd_star: String::new(),
icd_addon: String::new(),
icd_primary2: String::new(),
orpha: String::new(),
text: text.to_string(),
}
}
fn cfg_in(dir: &std::path::Path) -> Config {
let mut cfg = Config::load("config/default.toml").unwrap();
cfg.index_dir = dir.to_str().unwrap().to_string();
cfg.embed_model = "test/model".to_string();
cfg
}
#[test]
fn index_status_defaults_to_absent() {
assert_eq!(IndexStatus::default(), IndexStatus::Absent);
}
#[test]
fn index_status_serializes_externally_tagged() {
let j = serde_json::to_string(&IndexStatus::Mismatch("x".to_string())).unwrap();
assert!(j.contains("Mismatch"), "got {j}");
}
#[test]
fn packed_beats_store_builds_without_store() {
let dir = tempfile::tempdir().unwrap();
let cfg = cfg_in(dir.path());
let entries = vec![entry("a"), entry("b")];
let rows = vec![vec![1.0f32, 0.0], vec![0.0f32, 1.0]];
let h = PackedHeader {
n_rows: 2,
dim: 2,
embed_model: cfg.embed_model.clone(),
corpus_sha256: "irrelevant-in-iter1".to_string(),
corpus_name: "c.txt".to_string(),
};
write_packed(&packed_path(&cfg.index_dir, &cfg.embed_model), &h, &rows).unwrap();
// No per-file store populated in `dir`; the index must come from the packed file.
let (vector, status) = load_packed_or_store(&cfg, &entries);
assert!(vector.is_some());
assert_eq!(status, IndexStatus::Ok);
}
#[test]
fn n_rows_mismatch_yields_mismatch_none() {
let dir = tempfile::tempdir().unwrap();
let cfg = cfg_in(dir.path());
let entries = vec![entry("a"), entry("b"), entry("c")]; // 3 entries
let rows = vec![vec![1.0f32, 0.0], vec![0.0f32, 1.0]]; // packed claims 2 rows
let h = PackedHeader {
n_rows: 2,
dim: 2,
embed_model: cfg.embed_model.clone(),
corpus_sha256: "x".to_string(),
corpus_name: "c.txt".to_string(),
};
write_packed(&packed_path(&cfg.index_dir, &cfg.embed_model), &h, &rows).unwrap();
let (vector, status) = load_packed_or_store(&cfg, &entries);
assert!(vector.is_none());
assert!(matches!(status, IndexStatus::Mismatch(_)));
}
```
- [ ] **Step 2: Run tests to verify they fail**
Run: `cargo test --test packed_load_tests`
Expected: FAIL — compile error: `IndexStatus` not found in `alpha_id::model`
and `load_packed_or_store` not found in `alpha_id::pipeline`.
- [ ] **Step 3: Add `IndexStatus` and the `Diagnostics` field in `src/model.rs`**
Insert the enum immediately after the `Mode` enum (`src/model.rs:80-81`):
```rust
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Default)]
pub enum IndexStatus {
Ok, // packed file present and structurally valid
#[default]
Absent, // no usable index — intentional lexical-only
Mismatch(String), // packed file present but structurally invalid; reason carried
}
```
(`Serialize` is already imported at `src/model.rs:1`.)
Add the field to `Diagnostics` (`src/model.rs:114-121`), after `millis`:
```rust
#[derive(Debug, Clone, Serialize, Default)]
pub struct Diagnostics {
pub mode: String,
pub degraded: bool,
pub segment_count: usize,
pub ionos_calls: usize,
pub millis: u128,
pub index_status: IndexStatus,
}
```
- [ ] **Step 4: Add `load_packed_or_store` and thread it through `Pipeline` in `src/pipeline.rs`**
Add `use crate::packed;` to the imports (after `use crate::model::*;` at
`src/pipeline.rs:7`; `IndexStatus` arrives via the `model::*` glob).
Add the free fn (place it above `impl Pipeline`, after the struct at
`src/pipeline.rs:23`):
```rust
/// Build the semantic index, preferring the packed matrix file and falling
/// back to the per-file store. Returns the index (`None` when none is usable)
/// and an `IndexStatus` describing the path taken.
///
/// Iteration-1 scope: the packed file is validated *structurally* only
/// (`read_packed` checks magic/version/payload length; here we additionally
/// require `n_rows == entries.len()`). The header's `corpus_sha256` and
/// `embed_model` are NOT yet compared against the config — that enforcement
/// is iteration 2.
pub fn load_packed_or_store(cfg: &Config, entries: &[AlphaIdEntry]) -> (Option<VectorIndex>, IndexStatus) {
let packed = packed::packed_path(&cfg.index_dir, &cfg.embed_model);
if packed.exists() {
match packed::read_packed(&packed) {
Ok((header, rows)) => {
if header.n_rows as usize != entries.len() {
return (
None,
IndexStatus::Mismatch(format!(
"packed n_rows {} != corpus entries {}",
header.n_rows,
entries.len()
)),
);
}
return (Some(VectorIndex::from_vectors(rows)), IndexStatus::Ok);
}
Err(e) => return (None, IndexStatus::Mismatch(format!("{e}"))),
}
}
// Fallback: per-file store, all-or-nothing (today's behaviour).
match EmbeddingStore::open(&cfg.index_dir, &cfg.embed_model) {
Ok(store) if entries.iter().all(|e| store.has(&e.text)) => {
let vecs: Option<Vec<Vec<f32>>> = entries.iter().map(|e| store.get(&e.text)).collect();
match vecs {
Some(v) => (Some(VectorIndex::from_vectors(v)), IndexStatus::Ok),
None => (None, IndexStatus::Absent),
}
}
_ => (None, IndexStatus::Absent),
}
}
```
Add the field to the `Pipeline` struct (`src/pipeline.rs:15-23`), after
`vector: Option<VectorIndex>`:
```rust
vector: Option<VectorIndex>,
index_status: IndexStatus,
}
```
Replace the `let vector = match ... { ... };` block (`src/pipeline.rs:43-50`)
with:
```rust
let (vector, index_status) = load_packed_or_store(cfg, &entries);
```
Add `index_status` to the `Self { ... }` constructor (`src/pipeline.rs:51-59`),
after `vector,`:
```rust
vector,
index_status,
})
```
In `suggest`, add the field to the `Diagnostics { ... }` literal
(`src/pipeline.rs:229-235`), after `millis: t0.elapsed().as_millis(),`:
```rust
millis: t0.elapsed().as_millis(),
index_status: self.index_status.clone(),
```
- [ ] **Step 5: Run the new tests to verify they pass**
Run: `cargo test --test packed_load_tests`
Expected: PASS — 4 passed.
- [ ] **Step 6: Run the full suite to verify no regression**
Run: `cargo test`
Expected: PASS — the whole suite is green (the new `Diagnostics` field has a
`Default`, so existing diagnostics-reading tests in
`tests/pipeline_lexical_tests.rs`, `tests/pipeline_hybrid_tests.rs`, and
`tests/cli_tests.rs` are unaffected).
---
### Task 3: `alpha-id index --pack` subcommand
**Files:**
- Modify: `src/bin/alpha_id.rs:18,83-125`
- Test: `tests/pack_cli_tests.rs`
- [ ] **Step 1: Write the failing tests**
Create `tests/pack_cli_tests.rs`:
```rust
use std::path::Path;
use std::process::Command;
// Write a minimal config TOML pointing the corpus + index dir at temp paths.
// The `--pack` branch only reads `alpha_id_path` (corpus) and `index_dir`
// (store); `claml_path` etc. are present for parse but unused on this path.
fn write_config(dir: &Path, corpus: &Path, index_dir: &Path) -> std::path::PathBuf {
let toml = format!(
r#"ionos_base_url = "http://127.0.0.1:1"
token_path = "/nonexistent"
embed_model = "test/model"
rerank_model = "test/rerank"
alpha_id_path = "{corpus}"
claml_path = "/nonexistent.xml"
index_dir = "{index}"
pool_size = 10
top_k = 10
"#,
corpus = corpus.to_str().unwrap(),
index = index_dir.to_str().unwrap(),
);
let p = dir.join("config.toml");
std::fs::write(&p, toml).unwrap();
p
}
fn corpus_line(text: &str) -> String {
// Alpha-ID format: valid|alpha_id|primary|star|addon|primary2|orpha|text
format!("1|A0001|A00|||||{text}")
}
#[test]
fn pack_refuses_incomplete_store_exit_2() {
let dir = tempfile::tempdir().unwrap();
let corpus = dir.path().join("corpus.txt");
std::fs::write(&corpus, format!("{}\n{}\n", corpus_line("alpha"), corpus_line("beta"))).unwrap();
let index_dir = dir.path().join("index");
std::fs::create_dir_all(&index_dir).unwrap();
let config = write_config(dir.path(), &corpus, &index_dir);
let out = Command::new(env!("CARGO_BIN_EXE_alpha-id"))
.args(["--config", config.to_str().unwrap(), "index", "--pack"])
.output()
.unwrap();
assert_eq!(out.status.code(), Some(2), "incomplete store must exit 2");
let stderr = String::from_utf8_lossy(&out.stderr);
assert!(stderr.contains("store incomplete"), "stderr was: {stderr}");
}
#[test]
fn pack_writes_file_from_complete_store() {
let dir = tempfile::tempdir().unwrap();
let corpus = dir.path().join("corpus.txt");
std::fs::write(&corpus, format!("{}\n{}\n", corpus_line("alpha"), corpus_line("beta"))).unwrap();
let index_dir = dir.path().join("index");
std::fs::create_dir_all(&index_dir).unwrap();
// Populate the per-file store for both entry texts so the pack succeeds.
let store = alpha_id::embed::EmbeddingStore::open(index_dir.to_str().unwrap(), "test/model").unwrap();
store.put("alpha", &[1.0f32, 0.0]).unwrap();
store.put("beta", &[0.0f32, 1.0]).unwrap();
let config = write_config(dir.path(), &corpus, &index_dir);
let out = Command::new(env!("CARGO_BIN_EXE_alpha-id"))
.args(["--config", config.to_str().unwrap(), "index", "--pack"])
.output()
.unwrap();
assert!(out.status.success(), "stderr: {}", String::from_utf8_lossy(&out.stderr));
let packed = index_dir.join("packed.test_model.bin");
assert!(packed.exists(), "packed file should be written at {}", packed.display());
}
```
- [ ] **Step 2: Run tests to verify they fail**
Run: `cargo test --test pack_cli_tests`
Expected: FAIL — both tests fail. clap rejects the unknown `--pack` flag and
prints `error: unexpected argument '--pack' found` to stderr, so
`pack_writes_file_from_complete_store` fails on `out.status.success()` (clap
exits non-zero) and `pack_refuses_incomplete_store_exit_2` fails on the
`stderr.contains("store incomplete")` assertion (the stderr is clap's usage
error, not our message). Note: clap's own usage-error exit code happens to be
2, so the exit-code assertion alone is not what reddens the first test — the
`store incomplete` substring is.
- [ ] **Step 3: Add the `--pack` flag to the `Index` subcommand**
`src/bin/alpha_id.rs:18` currently reads:
```rust
Index { #[arg(long)] sample: Option<usize>, #[arg(long)] full: bool, #[arg(long)] confirm: bool },
```
Replace it with:
```rust
Index { #[arg(long)] sample: Option<usize>, #[arg(long)] full: bool, #[arg(long)] confirm: bool, #[arg(long)] pack: bool },
```
- [ ] **Step 4: Add the `--pack` branch in the `Index` handler**
The handler arm starts at `src/bin/alpha_id.rs:83`:
`Cmd::Index { sample, full, confirm } => {`. Update the pattern to bind
`pack` and insert the pack branch right after the two `use` lines
(`src/bin/alpha_id.rs:84-85`) and before `let p = Pipeline::load(&cfg)`:
```rust
Cmd::Index { sample, full, confirm, pack } => {
use alpha_id::embed::{EmbeddingStore, build_corpus_embeddings, estimate};
use alpha_id::ionos::IonosClient;
if pack {
use alpha_id::packed::{corpus_sha256, packed_path, write_packed, PackedHeader};
let entries = alpha_id::corpus::load(&cfg.alpha_id_path).expect("corpus");
let store = EmbeddingStore::open(&cfg.index_dir, &cfg.embed_model).expect("store");
let cached = entries.iter().filter(|e| store.has(&e.text)).count();
if cached != entries.len() {
eprintln!(
"error: store incomplete: {} of {} entries embedded; \
run `index --full --confirm` first",
cached, entries.len()
);
std::process::exit(2);
}
let rows: Vec<Vec<f32>> =
entries.iter().map(|e| store.get(&e.text).expect("cached vector")).collect();
let dim = rows.first().map(|r| r.len()).unwrap_or(0) as u32;
let corpus_name = std::path::Path::new(&cfg.alpha_id_path)
.file_name().and_then(|n| n.to_str())
.unwrap_or(&cfg.alpha_id_path).to_string();
let header = PackedHeader {
n_rows: entries.len() as u32,
dim,
embed_model: cfg.embed_model.clone(),
corpus_sha256: corpus_sha256(&cfg.alpha_id_path).expect("corpus hash"),
corpus_name,
};
let path = packed_path(&cfg.index_dir, &cfg.embed_model);
write_packed(&path, &header, &rows).expect("write packed");
eprintln!("packed {} rows × {} dims → {}", header.n_rows, header.dim, path.display());
return;
}
let p = Pipeline::load(&cfg).expect("load");
```
(The remainder of the arm — the `sample` and `full` branches — is unchanged.)
- [ ] **Step 5: Run the new tests to verify they pass**
Run: `cargo test --test pack_cli_tests`
Expected: PASS — 2 passed.
- [ ] **Step 6: Run the full suite to verify no regression**
Run: `cargo test`
Expected: PASS — the whole suite is green.