Brummel/doctate
1
0
Fork 0
Code Issues 9 Pull Requests Actions Packages Projects Releases Wiki Activity

feat: multi-backend LLM layer with per-request choice on case page

Browse Source 
Replace the single [llm] block in settings.toml with a curated catalog
of LLM "backends" (provider + model + sampling params + system prompt)
in server/src/analyze/backend.rs. Two backends ship initially:
gpt_oss_120b (default, reasoning_effort=medium) and llama_3_1_405b
(uses response_format: json_schema to sidestep the <|eot_id|> leak).

The case page now renders one submit button per available backend; the
clicked button's name=backend value rides through AnalysisInput.backend_id
to the worker, which looks it up via find_backend() with default fallback.
A submitted unknown backend id is rejected as 400. .analysis_failed.json
records the failed backend and the failure banner surfaces its label.

The API key now comes from IONOS_API_KEY (env), not settings.toml; the
[llm] section is read into a discarded field so old configs still load.
Backends without a satisfied requires_api_key are filtered from the UI
(any_backend_available()). Three end-to-end tests that mocked the LLM
endpoint via settings.llm.url are #[ignore]'d with a clear note — they
need per-test backend injection (Arc<Vec<LlmBackend>> through the worker)
before they can be re-enabled.
This commit is contained in:
Brummel
2026-05-03 14:57:21 +02:00
parent cbb072d0cc
commit 23ef84d9d8
18 changed files with 596 additions and 307 deletions
Download Patch File Download Diff File Expand all files Collapse all files
server/templates/case_page.html
+20 -27
View File
@@ -438,10 +438,13 @@
<form method="post" action="/web/cases/{{ case_id }}/analyze">
{% call csrf::field(csrf_token) %}
<input type="hidden" name="return_to" value="/web/cases/{{ case_id }}">
<button type="submit">Analysieren</button>
<span class="analyze-prompt">Analysieren mit:</span>
{% for b in backends %}
<button type="submit" name="backend" value="{{ b.id }}">{{ b.label }}</button>
{% endfor %}
</form>
{% else if llm_missing && !has_document %}
<span class="llm-missing">LLM-Analyse nicht konfiguriert</span>
<span class="llm-missing">Kein LLM-Backend verfügbar — IONOS_API_KEY setzen.</span>
{% endif %} {% if is_admin %}
<form
class="admin-only"
@@ -471,31 +474,15 @@
<form method="post" action="/web/cases/{{ case_id }}/analyze">
{% call csrf::field(csrf_token) %}
<input type="hidden" name="return_to" value="/web/cases/{{ case_id }}">
<span class="analyze-prompt">Neu analysieren mit:</span>
{% for b in backends %}
<button
type="submit"
class="copy-btn"
title="Neu analysieren"
aria-label="Neu analysieren"
>
<svg
xmlns="http://www.w3.org/2000/svg"
viewBox="0 0 24 24"
width="18"
height="18"
fill="none"
stroke="currentColor"
stroke-width="2"
stroke-linecap="round"
stroke-linejoin="round"
>
<polyline points="23 4 23 10 17 10"></polyline>
<polyline points="1 20 1 14 7 14"></polyline>
<path d="M3.51 9a9 9 0 0 1 14.85-3.36L23 10"></path>
<path
d="M20.49 15a9 9 0 0 1-14.85 3.36L1 14"
></path>
</svg>
</button>
name="backend"
value="{{ b.id }}"
title="Neu analysieren mit {{ b.label }}"
>{{ b.label }}</button>
{% endfor %}
</form>
</div>
{% endif %}
@@ -587,7 +574,9 @@
</script>
{% when None %} {% match analysis_failed %} {% when Some with (failure) %}
<div class="failure-banner" role="alert">
<strong>Analyse fehlgeschlagen.</strong>
<strong>
Analyse{% if !failure.backend_label.is_empty() %} mit {{ failure.backend_label }}{% endif %} fehlgeschlagen.
</strong>
Bitte erneut versuchen.
<span class="failure-time"
>Zuletzt versucht:
@@ -609,7 +598,11 @@
name="return_to"
value="/web/cases/{{ case_id }}"
/>
<button type="submit">Erneut analysieren</button>
{% for b in backends %}
<button type="submit" name="backend" value="{{ b.id }}">
Erneut mit {{ b.label }}
</button>
{% endfor %}
</form>
</div>
{% when None %} {% if analyzing %}