Brummel
d83b10f939
The retrieval-mode enum used opaque single letters `Mode { C, A }`;
neither the identifier nor the operator-facing `--mode a` token
conveyed what the mode does. Rename to self-describing names that map
directly onto the glossary definitions:
Mode::C -> Mode::Lexical (BM25 + tag filter, offline, no IONOS)
Mode::A -> Mode::Hybrid (lexical u semantic -> RRF -> rerank)
Decisions taken (the issue's open questions, resolved):
- CLI canonical tokens are now `lexical` / `hybrid`; the default is
`lexical`. The legacy `c` / `a` tokens stay accepted as
case-insensitive input aliases (zero cost; protects hand-typed
invocations). A unit test pins the alias contract.
- The diagnostics `mode` string (Debug-derived) now emits
"Lexical"/"Hybrid". Verified safe: no downstream consumer parses it.
doctate is the future integrator and embeds the library (the `Mode`
enum), not the JSON string (design spec §103).
- The reserved, out-of-scope generative tier (spec's "Mode B") keeps
conceptual room: a future `Mode::Generative` does not collide with
the retrieval-strategy names Lexical/Hybrid.
Surface: enum + FromStr, CLI defaults and `eval --mode both`
expansion, the private `collect_mode_c`/`collect_mode_a` methods (now
`collect_lexical`/`collect_hybrid`) and their comments, the two
pipeline mode test files (renamed), the eval CLI test, the glossary
(old letter forms demoted to Avoid lists), and the spec CLI synopsis.
closes #2
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
30 lines
1.1 KiB
Rust
30 lines
1.1 KiB
Rust
use std::process::Command;
|
|
|
|
#[test]
|
|
fn eval_lexical_runs_and_reports_billable_recall() {
|
|
let out = Command::new(env!("CARGO_BIN_EXE_alpha-id"))
|
|
.args(["eval", "--mode", "lexical", "--cases", "30"])
|
|
.arg("--config").arg("config/default.toml")
|
|
.output().unwrap();
|
|
assert!(out.status.success(), "stderr: {}", String::from_utf8_lossy(&out.stderr));
|
|
let s = String::from_utf8_lossy(&out.stdout);
|
|
assert!(s.contains("BillableRecall@10="));
|
|
}
|
|
|
|
#[test]
|
|
fn eval_is_deterministic_same_seed() {
|
|
let run = || {
|
|
let out = Command::new(env!("CARGO_BIN_EXE_alpha-id"))
|
|
.args(["eval", "--mode", "lexical", "--cases", "40", "--seed", "42"])
|
|
.arg("--config").arg("config/default.toml")
|
|
.output().unwrap();
|
|
assert!(out.status.success(), "stderr: {}", String::from_utf8_lossy(&out.stderr));
|
|
String::from_utf8(out.stdout).unwrap()
|
|
};
|
|
let first = run();
|
|
let second = run();
|
|
assert!(!first.is_empty(), "eval produced no output");
|
|
assert_eq!(first, second,
|
|
"eval output differed between runs with identical args (seed=42):\n run1: {first:?}\n run2: {second:?}");
|
|
}
|