Add Fachrichtung to prompt and refine medical terms.
Added Fachrichtung to the system prompt, instructing the LLM to consider
the medical specialty for precise terminology.
Also, added "Abdomensonographie" to the `medical_terms.txt` file.
Introduce a new field `reasoning_effort` to the `ChatRequest` struct.
This field, set to "medium" by default, allows tuning LLM reasoning
depth to balance hallucination risk against latency and token cost. This
setting is specifically for Ionos' GPT-OSS reasoning models and is
ignored by other OpenAI-compatible models.
The system prompt for the medical assistant has been refactored to
improve clarity and precision. Key changes include:
- Enhanced instructions on handling transcribed errors, emphasizing
verbatim transcription and the use of `==...==` for uncertain parts.
- More explicit guidance on the formatting of dosage schemes, including
the conversion of numerical representations to hyphenated formats
(e.g., "1-0-2").
- Introduction of specific examples for common medical abbreviations and
terms, such as "CDAI > 450", "per os", and "iv.".
- A new critical rule to enclose any output not part of the transcript
in `[...]`.
- The addition of "Ileus" to `medical_terms.txt` and "Adalimumab",
"Infliximab" to `vocabulary.txt`.
The `Gazetteer` struct has been refactored to store `Entry` structs,
where each entry contains a canonical form and a set of bypass aliases.
This change enables the bypassing of Hunspell's dictionary veto for
specific, explicitly defined aliases.
The `parse_line` function has been introduced to handle the new line
format, which allows for `-`-prefixed aliases. Malformed lines with
secondary tokens not prefixed by `-` now result in a hard error, failing
server startup with `io::ErrorKind::InvalidData`.
The `replace` method has been updated to check bypass aliases before
consulting the dictionary, ensuring that explicitly allowed aliases are
used for rewriting. This addresses issues where Hunspell incorrectly
identifies compounds, blocking necessary corrections.
The `README.md` in the `vocab` directory has been updated to document
the new bypass-alias functionality.
This commit removes the `whisper_variant` and `whisper_hotwords_variant`
fields from the `run_id` generation and the `print_meta_summary`
function.
These variants are no longer used as the project is shifting focus to
LLM-based generation. The `run_full_case.rs` example has also been
updated to reflect this change.
Sandbox-validated against case c414cf52 (3 runs each, fair pre+post-LLM
gazetteer pipeline): the new prompt eliminates two hallucination classes
the prior version produced — unmarked "Hypotonie" when the dictation said
"Hypertonie" (0/3 vs 2/3) and inventing units like "35 ng/l" for values
without unit (0/3 vs 2/3) — while keeping Latin terms (Punctum Maximum
etc.) intact in 3/3 runs vs 2/3.
Block layout reorganized so each rule appears exactly once: AUFGABE /
QUELLE / KORREKTUR-POLITIK / TREUE / CHRONOLOGIE / DOSIERUNGSSCHEMA /
MARKIERUNGEN. The removed "AKTIVE KORREKTUR (nicht durchreichen)" hammer
block is no longer load-bearing because the gazetteer's pre-LLM pass now
catches the typical drug-name typos before the LLM sees them.
Vocabulary additions (single-token, alphabetical, ≥5 chars):
- vocabulary.txt: Enoxaparin (was missing — Inoxaparin→Enoxaparin is a
recurrent ASR error and the prior pipeline relied on the LLM alone)
- medical_terms.txt (new file, picked up by the dir-glob loader):
Holosystolikum, Klappenvitien, Koronarbaum, Pumpfunktion
The build scripts for generating `anatomy.txt` and related files have
been removed as they are no longer used or maintained. The Kölner
Phonetik module was also removed as it was not being utilized.
The `filter-hunspell-de.sh` script now uses a `MIN_LEN` variable that is
synchronized with the `MIN_TOKEN_LEN` constant in
`src/gazetteer/mod.rs`.
This ensures that short tokens, which are prone to false positives due
to
collisions with common German words, are filtered out by both the
gazetteer
loading and the Hunspell dictionary processing.
Additionally, the script's dependency instructions have been updated to
be
more comprehensive, listing package manager commands for Arch/CachyOS,
Debian/Ubuntu, and Fedora. The `awk` command has been introduced to
filter
out tokens shorter than `MIN_LEN` before sorting and deduplicating.
The `Gazetteer::insert` method has been updated to skip entries shorter
than `MIN_TOKEN_LEN` to prevent noisy phonetic collisions. The
`best_candidate`
helper function has been extracted to improve readability and structure.
The test cases have been updated to reflect the change in
`MIN_TOKEN_LEN`
from 4 to 5 and to use more relevant examples for the updated
functionality,
such as "Cerebrum" and "Zerebrum". The `anatomy.txt` file has been
updated
to reflect that anatomical terms are currently disabled. The
`medications.txt` file has been significantly expanded with a curated
list
of German medication brand and active ingredient names.
Adds three new vocabulary files to the server's vocabulary directory:
`anatomy.txt`, `medications.txt`, and `substances.txt`. A README.md is
also added to explain their purpose, format, and population strategy.
The `server/vocab/raw/` directory is added to `.gitignore`.
Brummel