Brummel
8348e33264
Smoke-testing invented German dictations exposed that `ae`/`oe`/`ue` spellings collapsed lexical recall (e.g. "Hyperlipidaemie" → garbage, "Hyperlipidämie" → correct). normalize() now folds ä→ae, ö→oe, ü→ue, ß→ss; it is applied only in lexical.rs (both corpus index and query), so the path is symmetric under either spelling. The embedding path (raw text for bge-m3) is intentionally untouched. eval.rs inject_errors case 2 changed from umlaut-deletion (ä→a) to the realistic ASCII-digraph variant (ä→ae) so the eval harness exercises — and guards against regression of — exactly this failure class. TDD: failing tests first; normalize_tests 5/5, full suite 44/0. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
40 lines
1.5 KiB
Rust
40 lines
1.5 KiB
Rust
use alpha_id::normalize::normalize;
|
|
|
|
#[test]
|
|
fn lowercases_and_collapses_whitespace() {
|
|
assert_eq!(normalize(" Diabetes Mellitus\tTyp 2 "), "diabetes mellitus typ 2");
|
|
}
|
|
|
|
#[test]
|
|
fn expands_common_medical_abbreviations() {
|
|
assert_eq!(normalize("V.a. art. Hypertonie"), "verdacht auf arterielle hypertonie");
|
|
assert_eq!(normalize("Z.n. OP"), "zustand nach operation");
|
|
}
|
|
|
|
// The system must match regardless of whether the dictation uses real
|
|
// umlauts (ä, ö, ü, ß) or their ASCII digraphs (ae, oe, ue, ss). Both the
|
|
// indexed corpus text and the query pass through normalize(), so folding
|
|
// both forms to one canonical (ASCII-digraph) representation makes the
|
|
// lexical path symmetric — "Hyperlipidämie" and "Hyperlipidaemie" match.
|
|
#[test]
|
|
fn folds_umlauts_and_eszett_to_ascii_digraphs() {
|
|
assert_eq!(normalize("Ödem"), "oedem");
|
|
assert_eq!(normalize("Fußödem"), "fussoedem");
|
|
assert_eq!(normalize("über"), "ueber");
|
|
}
|
|
|
|
#[test]
|
|
fn umlaut_and_digraph_forms_normalize_identically() {
|
|
assert_eq!(normalize("Hyperlipidämie"), normalize("Hyperlipidaemie"));
|
|
assert_eq!(normalize("Ödem"), normalize("Oedem"));
|
|
assert_eq!(normalize("Fußödem"), normalize("Fussoedem"));
|
|
assert_eq!(normalize("über"), normalize("ueber"));
|
|
}
|
|
|
|
#[test]
|
|
fn folding_is_idempotent_on_already_ascii_text() {
|
|
// ASCII-only medical text must be unchanged by the fold.
|
|
assert_eq!(normalize("hyperlipidaemie"), "hyperlipidaemie");
|
|
assert_eq!(normalize("Diabetes Mellitus"), "diabetes mellitus");
|
|
}
|