feat: Add hunspell dictionary support to gazetteer
Integrates the `spellbook` crate to allow for a Hunspell dictionary to be used as a veto mechanism within the gazetteer. This prevents common German words from being incorrectly rewritten to vocabulary entries if they fall within the edit distance threshold. For example, "Kaktus" will no longer be rewritten to "Lantus" if a Hunspell dictionary is provided and contains "Kaktus". The `hunspell_dict_path` configuration option is added, defaulting to `/usr/share/hunspell/de_DE`. This path should point to the stem of the Hunspell dictionary files (e.g., `/usr/share/hunspell/de_DE.aff` and `/usr/share/hunspell/de_DE.dic`). The gazetteer now supports an optional `DictChecker` trait, allowing for pluggable dictionary implementations. `SpellbookDict` is the initial implementation using the `spellbook` crate. Includes new tests to verify the dict veto functionality, including handling of case sensitivity and exact matches. An ignored test is added for live verification against system-installed Hunspell dictionaries.
This commit is contained in:
Generated
+11
@@ -345,6 +345,7 @@ dependencies = [
|
||||
"rpassword",
|
||||
"serde",
|
||||
"serde_json",
|
||||
"spellbook",
|
||||
"strsim",
|
||||
"tempfile",
|
||||
"time",
|
||||
@@ -1631,6 +1632,16 @@ dependencies = [
|
||||
"windows-sys 0.61.2",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "spellbook"
|
||||
version = "0.4.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "28332bb44a9e7e7f1aa43ce4d97b6d1b27323a1fc160d4cfe0089245e1750845"
|
||||
dependencies = [
|
||||
"foldhash",
|
||||
"hashbrown 0.17.0",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "spin"
|
||||
version = "0.9.8"
|
||||
|
||||
Reference in New Issue
Block a user