This commit makes the `write_input_overwrite` function public. This is
necessary because the new `handle_analyze_required` function in
`bulk.rs` needs to call it. Previously, it was only used internally by
`auto_trigger.rs`.
Introduces a visual indicator for stale analysis on the case page and a
new section on the "My Cases" page to highlight cases requiring
analysis. This improves user awareness of data currency and pending
actions.
The `evaluate_case` function has been replaced with `evaluate_state` in
the `auto_trigger` module. This change introduces a new enum,
`CaseAnalysisState`, which provides a more granular representation of a
case's analysis status.
The `try_enqueue` and `try_enqueue_all_for_user` functions now utilize
`evaluate_state` to determine if analysis should proceed. This ensures
that analysis is only triggered for cases in the `Idle` or `Required`
states.
Additionally, the `CaseFlags` struct has been updated to include
`analysis_stale_with_doc`, and corresponding logic has been added in
`compute_flags` and `compute_case_view` to accurately reflect the
analysis status for UI display. The `UserCaseView` and `MyCasesTemplate`
structs have also been updated to incorporate these new fields.
This commit introduces a significant refactor to the auto-trigger and
recovery mechanisms for the analysis pipeline. The core change is the
introduction of a more granular `CaseAnalysisState` enum, replacing the
simpler `AutoDecision`.
The `evaluate_state` function now computes the full lifecycle state of a
case, considering factors like the presence of recordings, pending
transcripts, queued jobs, failure markers, document staleness, and an
idle threshold. This provides a richer understanding of the case's
status.
The `evaluate_case` function is retained as a backwards-compatible
adapter for existing callsites and tests. It maps the new
`CaseAnalysisState` to the old `AutoDecision` enum, simplifying the
decision to "enqueue" or "skip."
The `recovery` module is also refactored. The `scan_and_enqueue`
function is replaced by `boot_scan_state`, which is now purely
observational. It aggregates statistics about cases in different states
at boot time and logs them, but it no longer attempts to re-enqueue
jobs. The responsibility for triggering analysis now lies with the
per-render `try_enqueue_all_for_user` logic, which utilizes the new
typed state machine.
This refactoring aims to provide more clarity and control over the
analysis pipeline's state management, particularly in handling cases
that have previously failed or are waiting for further input.
The `document.md` file has been replaced by `document.json`. This new
format acts as a JSON envelope, containing the original markdown content
along with metadata such as `covered_mtime`, `written_at`, and
`backend_id`.
This change addresses a bug where new recordings arriving during an LLM
analysis call could be ignored. The `covered_mtime` field in the new
JSON envelope ensures that the document's metadata accurately reflects
the state of recordings at the time of analysis, preventing incorrect
"analysis current" states.
Additionally, the `analysis_input.json` file is now removed upon
successful analysis, and a failure marker is used to indicate analysis
failures. This ensures that a case is re-analyzed if necessary.
The `analysis_idle_threshold_secs` configuration option has been
introduced to control the delay before a case in the `Required` state is
automatically promoted to `Idle` and enqueued for LLM analysis.
Failure markers now carry finish_reason and usage tokens (prompt /
completion / reasoning) pulled from the chat-completions response, so
the case-page "Technische Details" block can show why an analysis
truncated — e.g. gpt-oss-120b hitting max_completion_tokens with the
chain-of-thought before producing any visible answer.
The worker also emits a new CaseEventKind::AnalysisFailed after writing
the marker, so subscribed browsers reload immediately instead of leaving
the page on the "Wird analysiert …" placeholder.
Mirror of the 2026-05-05 Wear-OS fix: port the canonical client sync
spec (project_client_sync_semantics.md) into the Rust desktop client.
Before this change, server_sync::run_loop had a `UploadOutcome::Terminal`
branch that deleted the m4a + sidecar pair on any non-2xx + non-5xx
response (401, 413, 415, parse drift, audio read failures) and never
called mark_synced — so the marker stayed `synced_to_server=false` as a
non-removable ghost in the desktop UI. Same shape as the Wear-OS bug.
Changes:
- `UploadOutcome` reduced to two variants (`Succeeded` | `Transient`).
The Terminal arm in `run_loop` is gone; `delete_pair` only fires from
the success path.
- `post_upload` now classifies every failure mode as `Transient`:
4xx incl. 401, parse-ack drift on 200, file-read errors, multipart
build errors, and network errors. Backoff retries forever.
- New `CaseStore::cleanup_orphaned_unsynced(pending_case_ids, cutoff)`:
removes markers that are unsynced AND have no waiting upload pair AND
are older than a small grace window. Self-heals pre-spec ghosts.
- `run_startup_cleanup` gains a third sweep that calls
`cleanup_orphaned_unsynced` with `DEFAULT_ORPHAN_MARKER_GRACE = 5 min`,
matching the Wear-OS StartupCleanup contract.
- Tests inverted/added: `post_upload_transient_on_401`,
`post_upload_transient_on_2xx_unparseable_body`,
`former_terminal_401_keeps_pair_and_retries`, four new
`cleanup_orphaned_unsynced_*` cases, plus `sweeps_orphan_unsynced_marker`
and `keeps_unsynced_marker_with_pending_pair` integration tests.
Followup (out of scope): `UploadEvent::Failed.will_retry` can now only
be `true` and is vestigial. Removing it touches every Failed-event
consumer in app.rs/UI; left for a separate cleanup commit.
Treat every non-Success upload response as Transient so audio survives
expired sessions, 4xx replies, and unparseable 2xx bodies — the watch
keeps retrying instead of silently dropping the recording. Drop the
UploadResult.Terminal variant entirely and the SyncWorkerLoop branch
that consumed it.
Also reap orphaned unsynced markers on app start: a syncedToServer=false
marker without a matching pending pair (audio gone) and older than a
5-minute grace window is inconsistent — the user couldn't discard it,
and no other path could clean it up. Eliminates the 2026-05-03 21:08
ghost-marker class.
Test fixture (InMemoryCaseStore), SyncWorkerLoopTest, UploadClientTest,
and StartupCleanupTest updated to reflect the new contract.
The /web/ prefix predated the /api/ split; today it just clutters every URL
without disambiguating anything. All 16 browser routes move to the apex
(/cases, /login, /magic, /events, /audio/...). The 6 /api/* routes are
unchanged. A new /->>/cases redirect closes the apex 404.
The open-redirect guard in magic.rs and case_actions.rs flips from a
positive whitelist (starts_with("/web/")) to a deny-list: same-origin path,
not protocol-relative, not under /api/, no \. The /api/ exclusion is now
load-bearing and covered by tests.
Pre-production: no transition redirects.
Introduces a `--force` or `-f` flag to the `deploy-server.sh` script,
allowing deployments even when the working tree is dirty. The script now
also checks for a dirty working tree and exits with an error unless
`--force` is provided.
Introduce `display_name` field to `User` struct for human-readable names
in the UI.
This falls back to the user's `slug` if not provided.
Update `AuthenticatedWebUser` and `MyCasesTemplate` to use this new
field for rendering.
Add tests for parsing `display_name` and ensure fallback logic works
correctly.
Update `users.toml.example` to document the new field.
One-line wrapper around docker compose ... restart plus a /api/health
poll, useful after editing users.toml/settings.toml/.env on minerva
(boot-time read). docs/deployment.md updated (user-mgmt, troubleshooting,
TLS-proxy sections) to reference the wrapper instead of the inline
ssh+docker command.
Multi-stage Dockerfile, host-network compose, idempotent deploy script.
Configs and persistent data bind-mounted from /opt/stacks/doctate-server.
deploy-server.sh handles bootstrap + re-deploy in one command with
auto-prune (keep 5 last SHA-tagged images).
See docs/deployment.md for bootstrap, rollback, and TLS-proxy notes.
This script automates checks for formatting, linting, building, and
testing across
all Cargo workspaces and the Wear OS Gradle build. It provides a summary
of all
stages, indicating PASS, WARN, or FAIL. The script's exit code reflects
the number
of failed stages.
Introduces a new `LlmBackend` struct to centralize LLM configuration.
This change refactors the `call_llm` function in both `run_full_case.rs`
and `run_llm_only.rs` to accept an `LlmBackend` instance instead of
individual settings.
The `doctate_server::analyze::backend` module is now used to obtain the
default backend, simplifying configuration and improving consistency.
The `clone_backend_with_system_prompt` helper function is introduced to
allow temporary modification of LLM backends for experimentation without
altering the production catalog.
This change improves code organization and makes it easier to manage LLM
configurations across different parts of the application.
Add Fachrichtung to prompt and refine medical terms.
Added Fachrichtung to the system prompt, instructing the LLM to consider
the medical specialty for precise terminology.
Also, added "Abdomensonographie" to the `medical_terms.txt` file.
The tile layout has been refactored to display a list of recent cases
instead of the currently active case. This change includes:
- Removing the "current case" view and its associated logic.
- Implementing a new `caseListContent` function to render a list of up
to 8 recent cases.
- Modifying the main `buildTileLayout` to integrate the new case list
and a prominent "Neu" (New) button.
- Updating the styling and click handling to reflect the new layout and
functionality.
Use the current case's oneliner text as the title for the remote input
intent. This ensures consistency with the CaseListScreen's display when
no oneliner is present, showing a "…" placeholder. This improves the
user experience by making the system input picker's title match what the
user just selected.
Increase `max_completion_tokens` for `gpt_oss_120b` from 8192 to 16384
to prevent incomplete responses.
Navigate to the detail screen before the recording screen when starting
a new case, ensuring correct back navigation.
When a previous analysis attempt fails and leaves a failure marker, the
retry mechanism now correctly identifies this state. It removes the
stale
input file, allowing the retry to proceed without a conflict error. This
ensures users can recover from analysis failures more gracefully.
An empty `backend` form field now correctly falls back to the default. A
non-empty field submitted by a non-admin user is rejected with a 403
error, preventing potential tampering or issues arising from stale
sessions after role changes.
This commit adjusts the configuration for the `gpt-oss-120b` model to
run in plain-text mode. This is necessary because when
`reasoning_effort` is set to "medium", the reasoning tokens combined
with schema-constrained decoding can exhaust the `max_completion_tokens`
budget, leading to an empty content response.
The change restores the previous behavior where `gpt-oss-120b` was
configured without `response_format` or `top_p`, ensuring the request
body sent to Ionos remains byte-identical to the pre-multi-backend
state. This prevents regressions and maintains stability for existing
production workflows.
Configure Llama 3.1 405B with specific temperature, top_p, and a format
instruction. This addresses issues with infinite repetition and empty
document generation observed with this model. The format instruction is
added as a separate system message to avoid modifying the core medical
prompt.
`recovery::enqueue_pending_for_user` previously enqueued any case with
`analysis_input.json` and no `document.md`, ignoring the failure marker
that `auto_trigger::evaluate_case` already honours. A timed-out LLM
call (e.g. Llama 3.1 405B at ~20 tok/s with default 4096 max_tokens)
left the input in place, the next case-list reload re-enqueued it, and
the worker hung another full timeout window — starving fresh user
clicks behind a failed retry loop.
Recovery now mirrors the same skip: if a marker exists whose
`last_recording_mtime` matches the input's, treat it as "already
tried, do not retry until input changes". Fresh recordings bump the
mtime and recovery picks the case back up.
Replace the single [llm] block in settings.toml with a curated catalog
of LLM "backends" (provider + model + sampling params + system prompt)
in server/src/analyze/backend.rs. Two backends ship initially:
gpt_oss_120b (default, reasoning_effort=medium) and llama_3_1_405b
(uses response_format: json_schema to sidestep the <|eot_id|> leak).
The case page now renders one submit button per available backend; the
clicked button's name=backend value rides through AnalysisInput.backend_id
to the worker, which looks it up via find_backend() with default fallback.
A submitted unknown backend id is rejected as 400. .analysis_failed.json
records the failed backend and the failure banner surfaces its label.
The API key now comes from IONOS_API_KEY (env), not settings.toml; the
[llm] section is read into a discarded field so old configs still load.
Backends without a satisfied requires_api_key are filtered from the UI
(any_backend_available()). Three end-to-end tests that mocked the LLM
endpoint via settings.llm.url are #[ignore]'d with a clear note — they
need per-test backend injection (Arc<Vec<LlmBackend>> through the worker)
before they can be re-enabled.
When an LLM analysis fails, a `.analysis_failed.json` marker is created.
If no document exists yet and the auto-trigger is blocked by this
marker,
the case page must display a banner. This banner provides the failure
reason and a button to retry the analysis, ensuring users are not left
in a dead-end state.
The `read_failure_marker` function is made public to allow the web layer
to access this failure information. The `CasePageTemplate` is updated to
include `analysis_failed` data, which conditionally renders the new
`.failure-banner` HTML.
This change prevents cases from becoming unrecoverable due to transient
LLM errors.
Includes a button for administrators to copy all transcripts and the
document markdown to the clipboard.
The data is embedded as JSON in a script tag and pre-sanitized to
prevent
`</script>` tag injection.
The JavaScript handles copying to the clipboard using the modern
Clipboard API,
with a fallback for older browsers.
Mirrors scripts/.gitignore from feature/mesh-vocab-pipeline so the
regenerable pipeline outputs under scripts/data/processed/ stop
showing up as untracked on main. Identical content avoids a merge
conflict when the feature branch lands.
The `client-desktop` crate is now located within the `clients/desktop/`
directory and renamed to `doctate-desktop`. This change improves
organization and clarity within the project structure.
Replace stale references in 17 Kotlin files:
- doctate-client-core::* -> doctate-desktop::*
- doctate-client-core/src/* -> clients/desktop/src/*
- doctate-common/src/* -> common/src/*
The doctate-common:: crate path stays valid (crate name unchanged).
PendingStoreTest.kt is in src/test/, so the test mirror is also
updated.
Verification:
- grep for "doctate-client-core" / "doctate-common/src/" in
clients/wearos/ returns 0 hits.
- gradlew :app:testDebugUnitTest BUILD SUCCESSFUL (23s) -- also
retroactively confirms stage-2's directory move did not break
any Gradle relative-path assumptions.
- cargo clippy --all-targets -- -D warnings clean in both
server/ and clients/desktop/ workspaces.
This concludes the four-stage server-vs-clients build-world split:
- stage 1 (39cc666): dissolve doctate-client-core
- stage 2 (0c66fc6): rename to common/, clients/desktop, clients/wearos
- stage 3 (7b4fe87): split into standalone workspaces
- stage 4 (this): update wearos anchors
docs/projektplan.md to be updated separately.
Add [workspace] / [workspace.package] / [workspace.dependencies]
blocks to server/Cargo.toml and clients/desktop/Cargo.toml so each
becomes a self-contained workspace with its own Cargo.lock.
Remove the root Cargo.toml and Cargo.lock; the repo now hosts
three independent build worlds: server, clients/desktop, experiments.
Convert common/Cargo.toml to concrete versions (was workspace-
inherited from the root). common/ is consumed via path-dep from
all three worlds, but is not itself a workspace -- consumers
unify versions via their own lockfiles. Add common/Cargo.lock to
.gitignore per Cargo lib-only-crate convention.
Verification:
- server/Cargo.lock has 0 eframe/egui/winit entries
- cargo build --release --locked passes in server/
- test split: server 385 + desktop 89 + common 34 = 508 total
- bin output: clients/desktop/target/debug/doctate (named "doctate")
Move three directories into their target topology:
- doctate-common/ -> common/
- client-desktop/ -> clients/desktop/
- watch/wearos/ -> clients/wearos/
Update doctate-common path-dependency in 3 Cargo.toml files
(server, clients/desktop, experiments) and the workspace
members list in the root Cargo.toml. Crate names unchanged
(doctate-server, doctate-common, doctate-desktop).
Workspace still in single-Cargo.lock form; isolation into
standalone workspaces follows in stage 3. All 508 tests pass,
experiments standalone-workspace also resolves the new path.
Move the 9 client-core modules (case_store, case_update, config,
footer_status, pending_cleanup, server_sync, snapshot_cache,
startup, upload) into client-desktop/src/. Resolve the config.rs
name collision by keeping the TOML schema in config.rs (was core)
and renaming the platform-path wrapper to config_path.rs.
Introduce client-desktop/src/lib.rs so the modules are reachable
under the doctate-desktop:: crate path -- needed for Wear-OS doc
anchors that mirror Rust symbols. Bin target stays as `doctate`
via [[bin]] name in Cargo.toml.
This is the first of four stages preparing the server-vs-clients
build-world split. Workspace still has 3 members; directory layout
unchanged. All 508 workspace tests pass.
Introduce a new field `reasoning_effort` to the `ChatRequest` struct.
This field, set to "medium" by default, allows tuning LLM reasoning
depth to balance hallucination risk against latency and token cost. This
setting is specifically for Ionos' GPT-OSS reasoning models and is
ignored by other OpenAI-compatible models.
The system prompt for the medical assistant has been refactored to
improve clarity and precision. Key changes include:
- Enhanced instructions on handling transcribed errors, emphasizing
verbatim transcription and the use of `==...==` for uncertain parts.
- More explicit guidance on the formatting of dosage schemes, including
the conversion of numerical representations to hyphenated formats
(e.g., "1-0-2").
- Introduction of specific examples for common medical abbreviations and
terms, such as "CDAI > 450", "per os", and "iv.".
- A new critical rule to enclose any output not part of the transcript
in `[...]`.
- The addition of "Ileus" to `medical_terms.txt` and "Adalimumab",
"Infliximab" to `vocabulary.txt`.
The `Gazetteer` struct has been refactored to store `Entry` structs,
where each entry contains a canonical form and a set of bypass aliases.
This change enables the bypassing of Hunspell's dictionary veto for
specific, explicitly defined aliases.
The `parse_line` function has been introduced to handle the new line
format, which allows for `-`-prefixed aliases. Malformed lines with
secondary tokens not prefixed by `-` now result in a hard error, failing
server startup with `io::ErrorKind::InvalidData`.
The `replace` method has been updated to check bypass aliases before
consulting the dictionary, ensuring that explicitly allowed aliases are
used for rewriting. This addresses issues where Hunspell incorrectly
identifies compounds, blocking necessary corrections.
The `README.md` in the `vocab` directory has been updated to document
the new bypass-alias functionality.
This commit removes the `whisper_variant` and `whisper_hotwords_variant`
fields from the `run_id` generation and the `print_meta_summary`
function.
These variants are no longer used as the project is shifting focus to
LLM-based generation. The `run_full_case.rs` example has also been
updated to reflect this change.
This commit introduces the `doctate-canary` service and associated sweep
experiments.
The service provides access to the `nvidia/canary-1b-v2` ASR model via a
native FastAPI API. It includes deployment scripts, Docker
configurations, and detailed documentation on installation, API usage,
and environment variables.
The sweep experiments aim to thoroughly evaluate the Canary model's
performance under various configurations. This includes testing
different inference pipelines (single-shot vs. buffered), decoding
strategies (greedy vs. beam search), and parameter tuning (chunk length,
overlap, batch size, precision). The goal is to reproduce previous
findings and identify optimal settings.
The commit also includes:
- Utility scripts for audio transcoding and manipulation.
- Comprehensive logging and result collection mechanisms for the sweep
runs.
- Detailed analysis of `dur=0` occurrences and word confidence,
concluding they are not reliable indicators of hallucination.
- Documentation on the interaction between decoding parameters,
especially `return_hypotheses`, and the availability of confidence
scores.
Pre-pulling the base Docker image in the deploy script prevents
potential TLS certificate errors during the `docker build` process. This
ensures a smoother deployment by leveraging the cached image layer.
This commit introduces a new standalone FastAPI service for the NVIDIA
Canary ASR model.
Key changes include:
- A new `canary/` directory containing the service code.
- `Dockerfile`: Defines the Docker image for the Canary service.
- `docker-compose.yml`: Configures the Docker Compose setup for running
the service.
- `main.py`: Implements the FastAPI application, model loading, and
inference logic.
- `requirements.txt`: Lists the Python dependencies for the service.
- `CHUNKING.md`: Documents the long-form audio handling strategy for
Canary.
- `README.md`: Provides an overview of the service, API, and deployment
instructions.
- `deploy.sh`: A script for deploying the service to a remote host.
This service allows for independent evaluation and deployment of the
Canary ASR model, separate from the existing `doctate-whisper` service.
It utilizes a buffered inference approach for handling long audio files,
as detailed in `CHUNKING.md`.
The previous run_llm_only only applied the gazetteer post-LLM, which
silently underestimated the production pipeline: tokens like Inoxaparin
or Klappenvizien are caught pre-LLM in production, but the test tool
left them in the LLM input. Adding `gazetteer.replace()` to each
recording before render_prompt mirrors transcribe/worker.rs:138 exactly,
making sandbox results faithful to production behavior.
New prompt variants kept for the iteration record:
- v3_treue_konsolidiert.txt — the winning variant now in
server/src/analyze/prompt.rs; redundancies eliminated where they were
not load-bearing
- v4_treue_kompakt.txt — rejected variant (3/3 unmarked Hypotonie
hallucinations); kept so future iterations don't repeat the
experiment
baseline.txt now mirrors the current server prompt (was a stale
pre-v2 snapshot), so future sandbox runs against `baseline.txt`
compare against what is actually shipping.
Sandbox-validated against case c414cf52 (3 runs each, fair pre+post-LLM
gazetteer pipeline): the new prompt eliminates two hallucination classes
the prior version produced — unmarked "Hypotonie" when the dictation said
"Hypertonie" (0/3 vs 2/3) and inventing units like "35 ng/l" for values
without unit (0/3 vs 2/3) — while keeping Latin terms (Punctum Maximum
etc.) intact in 3/3 runs vs 2/3.
Block layout reorganized so each rule appears exactly once: AUFGABE /
QUELLE / KORREKTUR-POLITIK / TREUE / CHRONOLOGIE / DOSIERUNGSSCHEMA /
MARKIERUNGEN. The removed "AKTIVE KORREKTUR (nicht durchreichen)" hammer
block is no longer load-bearing because the gazetteer's pre-LLM pass now
catches the typical drug-name typos before the LLM sees them.
Vocabulary additions (single-token, alphabetical, ≥5 chars):
- vocabulary.txt: Enoxaparin (was missing — Inoxaparin→Enoxaparin is a
recurrent ASR error and the prior pipeline relied on the LLM alone)
- medical_terms.txt (new file, picked up by the dir-glob loader):
Holosystolikum, Klappenvitien, Koronarbaum, Pumpfunktion
Datei enthielt Bisoprolol/Ramipril mit den exakten Schemata aus dem
Test-Case c414cf52 — klassisches Curve-Fitting, das beim Test gegen
genau diesen Fall keinen Erkenntnisgewinn bringt und beim Roll-out
auf andere Fälle aktiv halluziniert (Whisper bias'd auf gelistete
Termini, auch bei Stille oder Gemurmel).
Whisper-Initial-Prompts werden vorerst nicht weiter iteriert: Das
Hauptproblem 1-0-0/1-0-1 ist durch den LLM-DOSIERUNGSSCHEMA-Block
bereits gelöst, und der Halluzinations-Risikoaufschlag bei Whisper-
Prompts ist höher als bei der LLM-Stufe. baseline.txt (leer) bleibt
als Slot-Referenz erhalten.
Sandbox-iteration unter experiments/case_c414cf52 (3 Runs je Variante,
gpt-oss-120b @ Ionos, temperature=0). v2_treue gewinnt gegen Baseline
und v1_treue auf folgenden Achsen ohne Curve-Fitting im Prompt:
- Active drug-name correction (Inoxaparin→Enoxaparin, Thorazemit→
Torasemid, AFREF→HFrEF) durchgesetzt: 3/3 statt 0/3
- Schema 1-0-0 / 1-0-1 zuverlässiger: 3/3 statt 2/3
- Troponin-T fehlende Einheit als ==35== markiert (vorher unkommuniziert)
- Hypo/Hyper-Verlaufsgeschichte aus widersprüchlichen Aufnahmen
unterdrückt: 0/3 (war 1/3 in v1_treue)
- Anatomische Lokalisationen, "adipöser Ernährungszustand" und
Diktat-Reihenfolge bleiben treu (vorher selten aber gelegentlich
weggelassen oder umsortiert)
- Format als Fließtext explizit, Pseudo-Headings reduziert
Strukturell offen (kein Prompt-Fix möglich): "Holosystolikum" aus
Whispers Halbwortruine "Tolikum", "Koronarbund"-Halluzination aus
"Corona-Baum" — beides braucht Gazetteer-Erweiterung oder
Tokenizer-Refactor, separate PRs.
Umstellung auf Rust-Raw-String macht die Prompt-Datei deutlich
lesbarer (keine \n-Escapes, keine \"-Escapes mehr).
Brummel