Add gazetteer configuration and dependency

This commit introduces the `VOCAB_DIR` configuration option, which
specifies the directory for gazetteer files. It also adds the `strsim`
crate, which is used for calculating string similarity and is essential
for the gazetteer's functionality.

The gazetteer module has been significantly expanded to include the
`Gazetteer` struct, its loading mechanism from `.txt` files, and an
annotation function. This functionality allows for pre-LLM correction of
proper names and technical vocabulary by identifying potential
misspellings based on phonetic similarity and edit distance.
This commit is contained in:
2026-04-16 17:23:52 +02:00
parent 412bf9355e
commit 1fd252f363
5 changed files with 326 additions and 5 deletions
+7
View File
@@ -345,6 +345,7 @@ dependencies = [
"rpassword",
"serde",
"serde_json",
"strsim",
"tempfile",
"time",
"tokio",
@@ -1642,6 +1643,12 @@ version = "1.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6ce2be8dc25455e1f91df71bfa12ad37d7af1092ae736f3a6cd0e37bc7810596"
[[package]]
name = "strsim"
version = "0.11.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7da8b5736845d9f2fcb837ea5d9e2628564b3b043a70948a3f0b778838c5fb4f"
[[package]]
name = "subtle"
version = "2.6.1"