feat: multi-backend LLM layer with per-request choice on case page

Replace the single [llm] block in settings.toml with a curated catalog
of LLM "backends" (provider + model + sampling params + system prompt)
in server/src/analyze/backend.rs. Two backends ship initially:
gpt_oss_120b (default, reasoning_effort=medium) and llama_3_1_405b
(uses response_format: json_schema to sidestep the <|eot_id|> leak).

The case page now renders one submit button per available backend; the
clicked button's name=backend value rides through AnalysisInput.backend_id
to the worker, which looks it up via find_backend() with default fallback.
A submitted unknown backend id is rejected as 400. .analysis_failed.json
records the failed backend and the failure banner surfaces its label.

The API key now comes from IONOS_API_KEY (env), not settings.toml; the
[llm] section is read into a discarded field so old configs still load.
Backends without a satisfied requires_api_key are filtered from the UI
(any_backend_available()). Three end-to-end tests that mocked the LLM
endpoint via settings.llm.url are #[ignore]'d with a clear note — they
need per-test backend injection (Arc<Vec<LlmBackend>> through the worker)
before they can be re-enabled.
This commit is contained in:
2026-05-03 14:57:21 +02:00
parent cbb072d0cc
commit 23ef84d9d8
18 changed files with 596 additions and 307 deletions
+20 -9
View File
@@ -47,10 +47,27 @@ async fn main() {
.init();
// Surface configuration gaps that silently disable features.
if !settings.llm_configured() {
let backends = doctate_server::analyze::backend::backends();
let available = doctate_server::analyze::backend::available_backends();
info!(
total = backends.len(),
available = available.len(),
"LLM backends loaded"
);
for b in backends {
info!(
id = %b.id,
label = %b.label,
model = %b.model_id,
available = b.is_available(),
requires_api_key = b.requires_api_key,
"LLM backend"
);
}
if available.is_empty() {
warn!(
"LLM not configured — 'Fall abschließen' is disabled. \
Set [llm].url and [llm].model in settings.toml to enable."
"No LLM backend available — analyze button will be hidden. \
Set IONOS_API_KEY (or another provider's env var) to enable."
);
}
@@ -63,11 +80,6 @@ async fn main() {
);
}
info!(
prompt_chars = settings.llm.system_prompt.chars().count(),
"system prompt loaded"
);
// Shared HTTP client for both downstream pipelines.
let http_client = reqwest::Client::builder()
.build()
@@ -178,7 +190,6 @@ async fn main() {
std::sync::Arc::new(std::sync::atomic::AtomicBool::new(false));
tokio::spawn(analyze::worker::run(
analyze_rx,
settings.clone(),
http_client.clone(),
analyze_busy.clone(),
vocab.clone(),