feat: multi-backend LLM layer with per-request choice on case page

Replace the single [llm] block in settings.toml with a curated catalog
of LLM "backends" (provider + model + sampling params + system prompt)
in server/src/analyze/backend.rs. Two backends ship initially:
gpt_oss_120b (default, reasoning_effort=medium) and llama_3_1_405b
(uses response_format: json_schema to sidestep the <|eot_id|> leak).

The case page now renders one submit button per available backend; the
clicked button's name=backend value rides through AnalysisInput.backend_id
to the worker, which looks it up via find_backend() with default fallback.
A submitted unknown backend id is rejected as 400. .analysis_failed.json
records the failed backend and the failure banner surfaces its label.

The API key now comes from IONOS_API_KEY (env), not settings.toml; the
[llm] section is read into a discarded field so old configs still load.
Backends without a satisfied requires_api_key are filtered from the UI
(any_backend_available()). Three end-to-end tests that mocked the LLM
endpoint via settings.llm.url are #[ignore]'d with a clear note — they
need per-test backend injection (Arc<Vec<LlmBackend>> through the worker)
before they can be re-enabled.
This commit is contained in:
2026-05-03 14:57:21 +02:00
parent cbb072d0cc
commit 23ef84d9d8
18 changed files with 596 additions and 307 deletions
+5 -28
View File
@@ -23,31 +23,8 @@ url = "http://localhost:11434"
model = "gemma3:4b"
keep_alive = 0
# LLM provider: swap url/api_key/model between blocks to switch providers.
# Only one [llm] block may be active at a time — comment the others.
# Example: Ionos hosted
[llm]
url = "https://openai.inference.de-txl.ionos.com"
api_key = ""
model = ""
temperature = 0
timeout_seconds = 180
# Single line or TOML triple-string for multi-line prompts.
system_prompt = """
Du bist ein medizinischer Assistent. Fasse die folgenden diktierten Abschnitte
zu einem zusammenhängenden Text zusammen. Bereinigen und strukturieren, nichts
hinzufügen, nichts interpretieren, keine Diagnose, keine Arztbrief-Struktur.
Entferne Redundanzen. Spätere Aufnahmen haben Vorrang — Korrekturen, Nachträge
und Widersprüche zugunsten der chronologisch letzten Aussage auflösen.
Antworte auf Deutsch. Nur der zusammengefasste Text, keine Einleitung, kein
Abschlusssatz.
"""
# Example: Ollama local
# [llm]
# url = "http://localhost:11434"
# api_key = ""
# model = "SimonPu/gpt-oss:20b_Q4_K_M"
# temperature = 0
# timeout_seconds = 300
# [llm] is no longer read.
# LLM backends are defined in code (server/src/analyze/backend.rs); the API
# key comes from the IONOS_API_KEY env var (or the equivalent for other
# providers). An old [llm] block may stay in this file — it is parsed and
# discarded for backwards compatibility, but values inside have no effect.