Add gazetteer configuration and dependency

This commit introduces the `VOCAB_DIR` configuration option, which
specifies the directory for gazetteer files. It also adds the `strsim`
crate, which is used for calculating string similarity and is essential
for the gazetteer's functionality.

The gazetteer module has been significantly expanded to include the
`Gazetteer` struct, its loading mechanism from `.txt` files, and an
annotation function. This functionality allows for pre-LLM correction of
proper names and technical vocabulary by identifying potential
misspellings based on phonetic similarity and edit distance.
This commit is contained in:
2026-04-16 17:23:52 +02:00
parent 412bf9355e
commit 1fd252f363
5 changed files with 326 additions and 5 deletions
+1
View File
@@ -26,6 +26,7 @@ time = { version = "0.3.47", features = ["local-offset", "parsing", "formatting"
toml_edit = "0.25.11"
rpassword = "7.4.0"
pulldown-cmark = { version = "0.13", default-features = false, features = ["html"] }
strsim = "0.11"
[dev-dependencies]
tower = { version = "0.5", features = ["util"] }