Refactor gazetteer annotation format
The gazetteer now annotates text with `Canonical [?original]`. This prioritizes the corrected term for the LLM while keeping the original transcription as a fallback. This change aligns the gazetteer's annotation strategy with the LLM's prompt, which expects corrections to be marked for review. Previously, the format was `original [?Canonical]`, which could lead to the LLM using the potentially incorrect original term.
This commit is contained in:
@@ -361,9 +361,9 @@ async fn analyze_worker_writes_document_via_wiremock() {
|
||||
}
|
||||
|
||||
/// The analyze worker must route transcripts through the Gazetteer before
|
||||
/// sending them to the LLM. "Zerebrum" shares a Kölner-Phonetik code with
|
||||
/// the gazetteer entry "Cerebrum" (both encode to `87176`) and differs by
|
||||
/// exactly one character — so the annotation `Zerebrum [?Cerebrum]` must
|
||||
/// sending them to the LLM. "Zerebrum" differs from the gazetteer entry
|
||||
/// "Cerebrum" by exactly one character — so the annotation
|
||||
/// `Cerebrum [?Zerebrum]` (canonical anchor, original in bracket) must
|
||||
/// end up in the LLM request body.
|
||||
#[tokio::test]
|
||||
async fn analyze_worker_annotates_with_gazetteer() {
|
||||
@@ -432,7 +432,7 @@ async fn analyze_worker_annotates_with_gazetteer() {
|
||||
assert_eq!(requests.len(), 1, "expected exactly one LLM request");
|
||||
let body = String::from_utf8(requests[0].body.clone()).unwrap();
|
||||
assert!(
|
||||
body.contains("Zerebrum [?Cerebrum]"),
|
||||
body.contains("Cerebrum [?Zerebrum]"),
|
||||
"expected gazetteer annotation in LLM request body, got:\n{body}",
|
||||
);
|
||||
|
||||
|
||||
Reference in New Issue
Block a user