Brummel/alpha-id
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

plan: packed index format iteration 1 (refs #3)

Browse Source 
Bite-sized TDD plan for iteration 1 of the packed-versioned-bundle cycle:
the on-disk packed format (src/packed.rs), the IndexStatus enum + packed-first
Pipeline::load path with per-file-store fallback, and the `index --pack`
subcommand. Structural validation only (magic/version/n_rows/payload length);
the semantic corpus+model mismatch enforcement, load warning, and index=
diagnostics token are deferred to iteration 2 (#4), as documented in the
plan's iteration-boundary note.

refs #3

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Brummel
2026-06-01 00:24:15 +02:00
parent 2bb1bfc534
commit 2489d2029e
1 changed files with 651 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/plans/2026-06-01-packed-versioned-bundle-iter1.md
+651
View File
@@ -0,0 +1,651 @@
# Packed Index Format (Iteration 1) — Implementation Plan
> **Parent spec:** `docs/specs/2026-05-31-packed-versioned-index-bundle.md`
>
> **For agentic workers:** REQUIRED SUB-SKILL: use the `implement`
> skill to run this plan. Steps use `- [ ]` checkboxes for tracking.
**Goal:** Pack a complete per-file embedding store into one
self-describing matrix file and load the semantic index from it in a
single read, with the per-file store kept as a fallback.
**Architecture:** A new `src/packed.rs` owns the on-disk format (magic +
JSON header + row-major f32 payload) and the corpus hash. `Pipeline::load`
calls a new free fn `load_packed_or_store` that prefers the packed file
(structural validation only) and falls back to today's per-file store,
reporting which path it took via a new `IndexStatus`. An `index --pack`
subcommand writes the file from a complete store.
**Tech Stack:** Rust; `serde`/`serde_json` (header), `sha2` (corpus hash) —
both already in `Cargo.toml`; `clap` (the `--pack` flag); `tempfile`
(test fixtures).
**Iteration boundary (load-bearing — do not exceed):** This plan is
Iteration 1 (issue #3, acceptance a/b/c). The packed header *carries*
`corpus_sha256` and `embed_model`, but `load_packed_or_store` validates
only the file's *structure* (magic, supported version, `n_rows ==
entries.len()`, payload length) — it does **not** compare `corpus_sha256`
or `embed_model` against the live config. The semantic mismatch check, the
load-time warning, the `index=` token on the diagnostics line, and the
Hybrid-degrade-names-reason wiring are Iteration 2 (issue #4) and are OUT
OF SCOPE here. `IndexStatus::Mismatch` is produced in this iteration *only*
for structural corruption.
---
**Files this plan creates or modifies:**
- Create: `src/packed.rs` — packed file format I/O: `PackedHeader`,
`write_packed`, `read_packed`, `packed_path`, `corpus_sha256`.
- Modify: `src/lib.rs:13` — add `pub mod packed;` to the module list.
- Modify: `src/model.rs:80-81,114-121` — `IndexStatus` enum;
`index_status` field on `Diagnostics`.
- Modify: `src/pipeline.rs:15-23,26-60,229-235` — `load_packed_or_store`
free fn replacing the `:43-50` match; `Pipeline.index_status` field;
`suggest` copies it into `Diagnostics`.
- Modify: `src/bin/alpha_id.rs:18,83-125` — `--pack` flag on `Index` and
its handler branch.
- Test: `tests/packed_tests.rs` — format round-trip, `corpus_sha256`,
bad-magic rejection.
- Test: `tests/packed_load_tests.rs` — `IndexStatus` default/serde,
packed-beats-store, structural `n_rows` mismatch.
- Test: `tests/pack_cli_tests.rs` — `--pack` refuses an incomplete store
(exit 2) and writes a file from a complete store.
---
### Task 1: `src/packed.rs` — packed file format + corpus hash
**Files:**
- Create: `src/packed.rs`
- Modify: `src/lib.rs:13`
- Test: `tests/packed_tests.rs`
- [ ] **Step 1: Write the failing tests**
Create `tests/packed_tests.rs`:
```rust
use alpha_id::packed::{corpus_sha256, read_packed, write_packed, PackedHeader};
use std::io::Write;
fn header(n_rows: u32, dim: u32) -> PackedHeader {
PackedHeader {
n_rows,
dim,
embed_model: "test/model".to_string(),
corpus_sha256: "deadbeef".to_string(),
corpus_name: "corpus.txt".to_string(),
}
}
#[test]
fn write_read_roundtrip_preserves_header_and_rows() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("packed.bin");
let rows = vec![vec![0.0f32, 1.5, -2.25], vec![3.0, 4.0, 5.0]];
let h = header(2, 3);
write_packed(&path, &h, &rows).unwrap();
let (got_h, got_rows) = read_packed(&path).unwrap();
assert_eq!(got_h, h);
assert_eq!(got_rows, rows);
}
#[test]
fn corpus_sha256_stable_and_sensitive() {
let dir = tempfile::tempdir().unwrap();
let p = dir.path().join("c.txt");
std::fs::write(&p, b"alpha|beta").unwrap();
let a = corpus_sha256(p.to_str().unwrap()).unwrap();
let b = corpus_sha256(p.to_str().unwrap()).unwrap();
assert_eq!(a, b, "same bytes hash identically");
std::fs::write(&p, b"alpha|betb").unwrap();
let c = corpus_sha256(p.to_str().unwrap()).unwrap();
assert_ne!(a, c, "one changed byte changes the hash");
}
#[test]
fn read_packed_rejects_bad_magic() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("bad.bin");
let mut f = std::fs::File::create(&path).unwrap();
f.write_all(b"XXXX____________").unwrap();
assert!(read_packed(&path).is_err());
}
```
- [ ] **Step 2: Run tests to verify they fail**
Run: `cargo test --test packed_tests`
Expected: FAIL — compile error, `unresolved import alpha_id::packed`
(the module does not exist yet).
- [ ] **Step 3: Create `src/packed.rs`**
```rust
use crate::model::AppError;
use serde::{Deserialize, Serialize};
use sha2::{Digest, Sha256};
use std::fs;
use std::path::{Path, PathBuf};
const MAGIC: &[u8; 4] = b"AIPK";
pub const PACKED_FORMAT_VERSION: u32 = 1;
/// Manifest written into the packed file header. `embed_model` and
/// `corpus_sha256` bind the matrix to the corpus snapshot and model it was
/// built from; iteration 1 records them, iteration 2 enforces them.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct PackedHeader {
pub n_rows: u32,
pub dim: u32,
pub embed_model: String,
pub corpus_sha256: String,
pub corpus_name: String,
}
/// Derived packed-file path `<index_dir>/packed.<safe-model>.bin`, using the
/// same `/`→`_` model sanitization `EmbeddingStore::open` applies, so `--pack`
/// and `load` compute the identical path from the same config.
pub fn packed_path(index_dir: &str, model: &str) -> PathBuf {
let safe = model.replace('/', "_");
PathBuf::from(index_dir).join(format!("packed.{safe}.bin"))
}
/// SHA-256 of the raw bytes of the corpus file at `path`, lower-case hex.
pub fn corpus_sha256(path: &str) -> Result<String, AppError> {
let bytes = fs::read(path).map_err(|e| AppError::Io(format!("{path}: {e}")))?;
let mut h = Sha256::new();
h.update(&bytes);
Ok(h.finalize().iter().map(|x| format!("{:02x}", x)).collect())
}
/// Write the packed matrix: magic + format_version + header_len + JSON header
/// + pad-to-4 + row-major little-endian f32 payload. Atomic via temp+rename.
pub fn write_packed(path: &Path, header: &PackedHeader, rows: &[Vec<f32>]) -> Result<(), AppError> {
let json = serde_json::to_vec(header)
.map_err(|e| AppError::Parse(format!("packed header: {e}")))?;
let mut out = Vec::new();
out.extend_from_slice(MAGIC);
out.extend_from_slice(&PACKED_FORMAT_VERSION.to_le_bytes());
out.extend_from_slice(&(json.len() as u32).to_le_bytes());
out.extend_from_slice(&json);
while out.len() % 4 != 0 {
out.push(0);
}
for row in rows {
for f in row {
out.extend_from_slice(&f.to_le_bytes());
}
}
let tmp = path.with_extension("tmp");
fs::write(&tmp, &out).map_err(|e| AppError::Io(format!("{}: {e}", tmp.display())))?;
fs::rename(&tmp, path).map_err(|e| AppError::Io(format!("{}: {e}", path.display())))?;
Ok(())
}
/// Read a packed matrix back into its header and row vectors. Fails on a
/// wrong magic, an unsupported (newer) format version, a malformed or
/// truncated header, or a payload length inconsistent with `n_rows × dim`.
pub fn read_packed(path: &Path) -> Result<(PackedHeader, Vec<Vec<f32>>), AppError> {
let bytes = fs::read(path).map_err(|e| AppError::Io(format!("{}: {e}", path.display())))?;
if bytes.len() < 12 || &bytes[0..4] != MAGIC {
return Err(AppError::Parse(format!("{}: bad magic", path.display())));
}
let version = u32::from_le_bytes([bytes[4], bytes[5], bytes[6], bytes[7]]);
if version > PACKED_FORMAT_VERSION {
return Err(AppError::Parse(format!(
"{}: unsupported packed format version {version}",
path.display()
)));
}
let header_len = u32::from_le_bytes([bytes[8], bytes[9], bytes[10], bytes[11]]) as usize;
let header_end = 12 + header_len;
if bytes.len() < header_end {
return Err(AppError::Parse(format!("{}: truncated header", path.display())));
}
let header: PackedHeader = serde_json::from_slice(&bytes[12..header_end])
.map_err(|e| AppError::Parse(format!("{}: header: {e}", path.display())))?;
let mut payload_start = header_end;
while payload_start % 4 != 0 {
payload_start += 1;
}
let n = header.n_rows as usize;
let dim = header.dim as usize;
let expected = n * dim * 4;
if bytes.len() - payload_start != expected {
return Err(AppError::Parse(format!(
"{}: payload length {} != n_rows×dim×4 {}",
path.display(),
bytes.len() - payload_start,
expected
)));
}
let mut rows = Vec::with_capacity(n);
let mut off = payload_start;
for _ in 0..n {
let mut row = Vec::with_capacity(dim);
for _ in 0..dim {
row.push(f32::from_le_bytes([
bytes[off],
bytes[off + 1],
bytes[off + 2],
bytes[off + 3],
]));
off += 4;
}
rows.push(row);
}
Ok((header, rows))
}
```
- [ ] **Step 4: Register the module in `src/lib.rs`**
`src/lib.rs:13` currently ends the module list with `pub mod pipeline;`
(line 13) and `pub mod eval;` (line 14). Add one line so the list reads:
```rust
pub mod packed;
pub mod pipeline;
pub mod eval;
```
- [ ] **Step 5: Run tests to verify they pass**
Run: `cargo test --test packed_tests`
Expected: PASS — 3 passed.
---
### Task 2: `IndexStatus` + packed-first load path + diagnostics plumbing
**Files:**
- Modify: `src/model.rs:80-81,114-121`
- Modify: `src/pipeline.rs:15-23,26-60,229-235`
- Test: `tests/packed_load_tests.rs`
- [ ] **Step 1: Write the failing tests**
Create `tests/packed_load_tests.rs`:
```rust
use alpha_id::model::{AlphaIdEntry, Config, IndexStatus};
use alpha_id::packed::{packed_path, write_packed, PackedHeader};
use alpha_id::pipeline::load_packed_or_store;
fn entry(text: &str) -> AlphaIdEntry {
AlphaIdEntry {
alpha_id: "X".to_string(),
valid: true,
icd_primary: "A00".to_string(),
icd_star: String::new(),
icd_addon: String::new(),
icd_primary2: String::new(),
orpha: String::new(),
text: text.to_string(),
}
}
fn cfg_in(dir: &std::path::Path) -> Config {
let mut cfg = Config::load("config/default.toml").unwrap();
cfg.index_dir = dir.to_str().unwrap().to_string();
cfg.embed_model = "test/model".to_string();
cfg
}
#[test]
fn index_status_defaults_to_absent() {
assert_eq!(IndexStatus::default(), IndexStatus::Absent);
}
#[test]
fn index_status_serializes_externally_tagged() {
let j = serde_json::to_string(&IndexStatus::Mismatch("x".to_string())).unwrap();
assert!(j.contains("Mismatch"), "got {j}");
}
#[test]
fn packed_beats_store_builds_without_store() {
let dir = tempfile::tempdir().unwrap();
let cfg = cfg_in(dir.path());
let entries = vec![entry("a"), entry("b")];
let rows = vec![vec![1.0f32, 0.0], vec![0.0f32, 1.0]];
let h = PackedHeader {
n_rows: 2,
dim: 2,
embed_model: cfg.embed_model.clone(),
corpus_sha256: "irrelevant-in-iter1".to_string(),
corpus_name: "c.txt".to_string(),
};
write_packed(&packed_path(&cfg.index_dir, &cfg.embed_model), &h, &rows).unwrap();
// No per-file store populated in `dir`; the index must come from the packed file.
let (vector, status) = load_packed_or_store(&cfg, &entries);
assert!(vector.is_some());
assert_eq!(status, IndexStatus::Ok);
}
#[test]
fn n_rows_mismatch_yields_mismatch_none() {
let dir = tempfile::tempdir().unwrap();
let cfg = cfg_in(dir.path());
let entries = vec![entry("a"), entry("b"), entry("c")]; // 3 entries
let rows = vec![vec![1.0f32, 0.0], vec![0.0f32, 1.0]]; // packed claims 2 rows
let h = PackedHeader {
n_rows: 2,
dim: 2,
embed_model: cfg.embed_model.clone(),
corpus_sha256: "x".to_string(),
corpus_name: "c.txt".to_string(),
};
write_packed(&packed_path(&cfg.index_dir, &cfg.embed_model), &h, &rows).unwrap();
let (vector, status) = load_packed_or_store(&cfg, &entries);
assert!(vector.is_none());
assert!(matches!(status, IndexStatus::Mismatch(_)));
}
```
- [ ] **Step 2: Run tests to verify they fail**
Run: `cargo test --test packed_load_tests`
Expected: FAIL — compile error: `IndexStatus` not found in `alpha_id::model`
and `load_packed_or_store` not found in `alpha_id::pipeline`.
- [ ] **Step 3: Add `IndexStatus` and the `Diagnostics` field in `src/model.rs`**
Insert the enum immediately after the `Mode` enum (`src/model.rs:80-81`):
```rust
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Default)]
pub enum IndexStatus {
Ok, // packed file present and structurally valid
#[default]
Absent, // no usable index — intentional lexical-only
Mismatch(String), // packed file present but structurally invalid; reason carried
}
```
(`Serialize` is already imported at `src/model.rs:1`.)
Add the field to `Diagnostics` (`src/model.rs:114-121`), after `millis`:
```rust
#[derive(Debug, Clone, Serialize, Default)]
pub struct Diagnostics {
pub mode: String,
pub degraded: bool,
pub segment_count: usize,
pub ionos_calls: usize,
pub millis: u128,
pub index_status: IndexStatus,
}
```
- [ ] **Step 4: Add `load_packed_or_store` and thread it through `Pipeline` in `src/pipeline.rs`**
Add `use crate::packed;` to the imports (after `use crate::model::*;` at
`src/pipeline.rs:7`; `IndexStatus` arrives via the `model::*` glob).
Add the free fn (place it above `impl Pipeline`, after the struct at
`src/pipeline.rs:23`):
```rust
/// Build the semantic index, preferring the packed matrix file and falling
/// back to the per-file store. Returns the index (`None` when none is usable)
/// and an `IndexStatus` describing the path taken.
///
/// Iteration-1 scope: the packed file is validated *structurally* only
/// (`read_packed` checks magic/version/payload length; here we additionally
/// require `n_rows == entries.len()`). The header's `corpus_sha256` and
/// `embed_model` are NOT yet compared against the config — that enforcement
/// is iteration 2.
pub fn load_packed_or_store(cfg: &Config, entries: &[AlphaIdEntry]) -> (Option<VectorIndex>, IndexStatus) {
let packed = packed::packed_path(&cfg.index_dir, &cfg.embed_model);
if packed.exists() {
match packed::read_packed(&packed) {
Ok((header, rows)) => {
if header.n_rows as usize != entries.len() {
return (
None,
IndexStatus::Mismatch(format!(
"packed n_rows {} != corpus entries {}",
header.n_rows,
entries.len()
)),
);
}
return (Some(VectorIndex::from_vectors(rows)), IndexStatus::Ok);
}
Err(e) => return (None, IndexStatus::Mismatch(format!("{e}"))),
}
}
// Fallback: per-file store, all-or-nothing (today's behaviour).
match EmbeddingStore::open(&cfg.index_dir, &cfg.embed_model) {
Ok(store) if entries.iter().all(|e| store.has(&e.text)) => {
let vecs: Option<Vec<Vec<f32>>> = entries.iter().map(|e| store.get(&e.text)).collect();
match vecs {
Some(v) => (Some(VectorIndex::from_vectors(v)), IndexStatus::Ok),
None => (None, IndexStatus::Absent),
}
}
_ => (None, IndexStatus::Absent),
}
}
```
Add the field to the `Pipeline` struct (`src/pipeline.rs:15-23`), after
`vector: Option<VectorIndex>`:
```rust
vector: Option<VectorIndex>,
index_status: IndexStatus,
}
```
Replace the `let vector = match ... { ... };` block (`src/pipeline.rs:43-50`)
with:
```rust
let (vector, index_status) = load_packed_or_store(cfg, &entries);
```
Add `index_status` to the `Self { ... }` constructor (`src/pipeline.rs:51-59`),
after `vector,`:
```rust
vector,
index_status,
})
```
In `suggest`, add the field to the `Diagnostics { ... }` literal
(`src/pipeline.rs:229-235`), after `millis: t0.elapsed().as_millis(),`:
```rust
millis: t0.elapsed().as_millis(),
index_status: self.index_status.clone(),
```
- [ ] **Step 5: Run the new tests to verify they pass**
Run: `cargo test --test packed_load_tests`
Expected: PASS — 4 passed.
- [ ] **Step 6: Run the full suite to verify no regression**
Run: `cargo test`
Expected: PASS — the whole suite is green (the new `Diagnostics` field has a
`Default`, so existing diagnostics-reading tests in
`tests/pipeline_lexical_tests.rs`, `tests/pipeline_hybrid_tests.rs`, and
`tests/cli_tests.rs` are unaffected).
---
### Task 3: `alpha-id index --pack` subcommand
**Files:**
- Modify: `src/bin/alpha_id.rs:18,83-125`
- Test: `tests/pack_cli_tests.rs`
- [ ] **Step 1: Write the failing tests**
Create `tests/pack_cli_tests.rs`:
```rust
use std::path::Path;
use std::process::Command;
// Write a minimal config TOML pointing the corpus + index dir at temp paths.
// The `--pack` branch only reads `alpha_id_path` (corpus) and `index_dir`
// (store); `claml_path` etc. are present for parse but unused on this path.
fn write_config(dir: &Path, corpus: &Path, index_dir: &Path) -> std::path::PathBuf {
let toml = format!(
r#"ionos_base_url = "http://127.0.0.1:1"
token_path = "/nonexistent"
embed_model = "test/model"
rerank_model = "test/rerank"
alpha_id_path = "{corpus}"
claml_path = "/nonexistent.xml"
index_dir = "{index}"
pool_size = 10
top_k = 10
"#,
corpus = corpus.to_str().unwrap(),
index = index_dir.to_str().unwrap(),
);
let p = dir.join("config.toml");
std::fs::write(&p, toml).unwrap();
p
}
fn corpus_line(text: &str) -> String {
// Alpha-ID format: valid|alpha_id|primary|star|addon|primary2|orpha|text
format!("1|A0001|A00|||||{text}")
}
#[test]
fn pack_refuses_incomplete_store_exit_2() {
let dir = tempfile::tempdir().unwrap();
let corpus = dir.path().join("corpus.txt");
std::fs::write(&corpus, format!("{}\n{}\n", corpus_line("alpha"), corpus_line("beta"))).unwrap();
let index_dir = dir.path().join("index");
std::fs::create_dir_all(&index_dir).unwrap();
let config = write_config(dir.path(), &corpus, &index_dir);
let out = Command::new(env!("CARGO_BIN_EXE_alpha-id"))
.args(["--config", config.to_str().unwrap(), "index", "--pack"])
.output()
.unwrap();
assert_eq!(out.status.code(), Some(2), "incomplete store must exit 2");
let stderr = String::from_utf8_lossy(&out.stderr);
assert!(stderr.contains("store incomplete"), "stderr was: {stderr}");
}
#[test]
fn pack_writes_file_from_complete_store() {
let dir = tempfile::tempdir().unwrap();
let corpus = dir.path().join("corpus.txt");
std::fs::write(&corpus, format!("{}\n{}\n", corpus_line("alpha"), corpus_line("beta"))).unwrap();
let index_dir = dir.path().join("index");
std::fs::create_dir_all(&index_dir).unwrap();
// Populate the per-file store for both entry texts so the pack succeeds.
let store = alpha_id::embed::EmbeddingStore::open(index_dir.to_str().unwrap(), "test/model").unwrap();
store.put("alpha", &[1.0f32, 0.0]).unwrap();
store.put("beta", &[0.0f32, 1.0]).unwrap();
let config = write_config(dir.path(), &corpus, &index_dir);
let out = Command::new(env!("CARGO_BIN_EXE_alpha-id"))
.args(["--config", config.to_str().unwrap(), "index", "--pack"])
.output()
.unwrap();
assert!(out.status.success(), "stderr: {}", String::from_utf8_lossy(&out.stderr));
let packed = index_dir.join("packed.test_model.bin");
assert!(packed.exists(), "packed file should be written at {}", packed.display());
}
```
- [ ] **Step 2: Run tests to verify they fail**
Run: `cargo test --test pack_cli_tests`
Expected: FAIL — both tests fail. clap rejects the unknown `--pack` flag and
prints `error: unexpected argument '--pack' found` to stderr, so
`pack_writes_file_from_complete_store` fails on `out.status.success()` (clap
exits non-zero) and `pack_refuses_incomplete_store_exit_2` fails on the
`stderr.contains("store incomplete")` assertion (the stderr is clap's usage
error, not our message). Note: clap's own usage-error exit code happens to be
2, so the exit-code assertion alone is not what reddens the first test — the
`store incomplete` substring is.
- [ ] **Step 3: Add the `--pack` flag to the `Index` subcommand**
`src/bin/alpha_id.rs:18` currently reads:
```rust
Index { #[arg(long)] sample: Option<usize>, #[arg(long)] full: bool, #[arg(long)] confirm: bool },
```
Replace it with:
```rust
Index { #[arg(long)] sample: Option<usize>, #[arg(long)] full: bool, #[arg(long)] confirm: bool, #[arg(long)] pack: bool },
```
- [ ] **Step 4: Add the `--pack` branch in the `Index` handler**
The handler arm starts at `src/bin/alpha_id.rs:83`:
`Cmd::Index { sample, full, confirm } => {`. Update the pattern to bind
`pack` and insert the pack branch right after the two `use` lines
(`src/bin/alpha_id.rs:84-85`) and before `let p = Pipeline::load(&cfg)`:
```rust
Cmd::Index { sample, full, confirm, pack } => {
use alpha_id::embed::{EmbeddingStore, build_corpus_embeddings, estimate};
use alpha_id::ionos::IonosClient;
if pack {
use alpha_id::packed::{corpus_sha256, packed_path, write_packed, PackedHeader};
let entries = alpha_id::corpus::load(&cfg.alpha_id_path).expect("corpus");
let store = EmbeddingStore::open(&cfg.index_dir, &cfg.embed_model).expect("store");
let cached = entries.iter().filter(|e| store.has(&e.text)).count();
if cached != entries.len() {
eprintln!(
"error: store incomplete: {} of {} entries embedded; \
run `index --full --confirm` first",
cached, entries.len()
);
std::process::exit(2);
}
let rows: Vec<Vec<f32>> =
entries.iter().map(|e| store.get(&e.text).expect("cached vector")).collect();
let dim = rows.first().map(|r| r.len()).unwrap_or(0) as u32;
let corpus_name = std::path::Path::new(&cfg.alpha_id_path)
.file_name().and_then(|n| n.to_str())
.unwrap_or(&cfg.alpha_id_path).to_string();
let header = PackedHeader {
n_rows: entries.len() as u32,
dim,
embed_model: cfg.embed_model.clone(),
corpus_sha256: corpus_sha256(&cfg.alpha_id_path).expect("corpus hash"),
corpus_name,
};
let path = packed_path(&cfg.index_dir, &cfg.embed_model);
write_packed(&path, &header, &rows).expect("write packed");
eprintln!("packed {} rows × {} dims → {}", header.n_rows, header.dim, path.display());
return;
}
let p = Pipeline::load(&cfg).expect("load");
```
(The remainder of the arm — the `sample` and `full` branches — is unchanged.)
- [ ] **Step 5: Run the new tests to verify they pass**
Run: `cargo test --test pack_cli_tests`
Expected: PASS — 2 passed.
- [ ] **Step 6: Run the full suite to verify no regression**
Run: `cargo test`
Expected: PASS — the whole suite is green.