Replace default_system_prompt.md with the llama prompt content. The default
backend (gpt_oss_120b) carries all real traffic, but used the permissive
prompt that lets the model SUBSTITUTE dictated terms with its own guess;
the llama prompt keeps the dictated word and only flags it for review.
Validated in the experiments sandbox against 16 real alpha-test cases
(full pipeline: pre-gazetteer -> gpt_oss_120b -> post-gazetteer, 3 runs
each, independently judged): dangerous-error rate per run drops from
44% (20/45) to 10% (5/49), and the 3 runs that produced no output at all
(reasoning length-exhaustion) drop to 0.
The two prompt files are now content-identical, as backend.rs already
anticipated. Follow-up: the DEFAULT_SYSTEM_PROMPT doc-comment in
analyze/backend.rs still describes the old prompt and should be refreshed.
Cycle-close audit for the case-page copy+close cycle (spec 0002, plan
0003, feat 0308245). Architect drift review + scripts/check.sh.
Regression gate (scripts/check.sh): exit 0 — 0 failures across 19
stages (common/server/clients-desktop/experiments fmt+clippy+build+test,
wearos assembleDebug+lintDebug+testDebugUnitTest), 74s. The single
`wearos :: lintDebug` WARN is pre-existing and untouched by this
frontend-only cycle; not introduced here, carried forward.
Architect findings and resolution:
- [high] template emits two separate close forms, not the spec's single
#copy-close-form with a has_document label switch. Resolved as a SPEC
defect, not a code defect: the spec's single-form snippet was
infeasible — #doc-body exists only in the has-document match arm, so a
form anchored after it can never serve the documentless case, and it
must precede the copy <script> to be bound. The shipped two-form split
is canonical; spec 0002 §Concrete code shapes + §Data flow reconciled
to it (this commit).
- [high] the spec's `if (!getElementById("doc-body")) return` guard is
absent from the shipped handler. Not a defect: the no-document form
carries no id, so the handler (which binds #copy-close-form) never
attaches to it — the guard is structurally unnecessary. Spec script
block + prose updated to state this mechanism.
- [medium] the documentless form's distinctness was unpinned. Added an
assertion to case_page_empty_case_shows_placeholder that the empty
case renders no #copy-close-form (it is the bare close form).
- [low] plan counter 0003 vs spec counter 0002: not drift — the naming
policy is per-directory counters (stable_per_directory_4digit over
[docs/specs, docs/plans]); independent sequences are expected.
- [note] deferred case-list half of #14 leaves a coherent state
(refs #14, my_cases.html untouched). No drift.
Recommendation: carry-on. No fix iteration needed; the high items were
spec-vs-code reconciliation (docs) and a test guard, both done here.
refs #14
A finished case is handled in one gesture: a text button below the
document copies the document text to the clipboard and then closes the
case, landing back on /cases. Replaces the icon-only trash-can close
that sat in the H1 title row.
Behaviour:
- Open case with a document: "Kopieren + Schließen" below the document.
The submit handler awaits the clipboard write inside the user gesture,
then submits the close form; a FAILED copy aborts the close and reveals
an inline error, so a finished case never disappears from the default
list while the clipboard is empty.
- Open case without a document: a bare "Schließen" button (nothing to
copy) keeps the close affordance the removed trash-can used to provide.
- Closed case: neither button renders; the reopen affordance is
untouched.
Frontend-only. No Rust route/handler change: POST /cases/{id}/close
already strips the case-page Referer to /cases, so "back to list" needs
no new server work. The two-path clipboard routine (async Clipboard API
+ textarea/execCommand fallback) is extracted into a shared copyDocText()
used by both the standalone icon copy button (kept, copy-without-close)
and the new combined control — one source, no drift.
Tests (RED-first, server/tests/case_page_test.rs): open+document renders
the copy-close form with the close action and CSRF token; closed case
hides it; the H1 trash-can is gone on an open case; the documentless
open case keeps a bare Schließen. Full server suite green, clippy clean,
fmt no-op. The JS submit-handler ordering (copy-before-navigate,
abort-on-fail) has no Rust render harness — same inherent layer gap as
the pre-existing #copy-btn — and is verified manually.
Covers the case-page half of #14; the per-row case-list copy+close is
deferred.
refs #14
Reverse proxies (OpenResty in front of minerva) buffer text/event-stream
by default, holding events until the response ends. An SSE stream never
ends, so the browser stays silent and WebUI pages never live-reload when
reached via app.doctate.de — while a direct minerva.lan:3000 client
(bypassing the proxy) works, explaining the desktop/Chromebook split.
Emit X-Accel-Buffering: no on the /events response so the proxy streams
it unbuffered; a direct client ignores the header. Return type widens
from Sse<..> to impl IntoResponse to carry the header tuple.
Regression guard: sse_integration asserts the header is present.
refs #12
Follow-up to 6d1ca71: that commit made the recordings sub-page render for a
closed case by threading ?show_closed=1, but the page's own links and the
audio player still 404'd — playback was dead and the back link led nowhere.
Root cause: the `.closed` marker was implemented as an ACCESS gate (every
case-scoped read 404s a closed case unless the opt-in flag is threaded
through), while it is documented as a DISCOVERY gate ("hidden from the
default listing"). Threading the flag through every link/redirect is
whack-a-mole; the next new link forgets it.
Resolved in favour of discovery-gate semantics: the `.closed` marker only
filters the default case LIST (scan_user_cases). Direct/deep-link access to
a known, owned case and all its sub-resources is closed-agnostic. IDOR is
unchanged — it comes from the per-user root + existence check, orthogonal
to closed-ness.
Scope 1 — closed cases are fully viewable:
- handle_audio: drop the is_closed -> 404 gate (web.rs). Audio is a
capability URL behind the same per-user path + slug/UUID/filename
validation as an open case.
- handle_case_page / handle_case_recordings: always resolve via
locate_closed_case_or_404; show_closed is no longer an access gate, so
back links (recordings -> case, case -> list) reach live pages without
threading the flag.
- the "back to list" links carry ?show_closed=1 when the case is closed
(computed server-side from the marker) so the user returns to the
closed-inclusive list, not the default list that hides the case.
Scope 2 — closed means read-only:
- case_page.html withholds the analyze / reset / re-analyze / retry forms
on closed cases; only the reopen affordance remains.
- case_recordings.html withholds the per-recording delete form on closed
cases (CaseRecordingsTemplate gains is_closed).
- the mutating handlers (analyze/reset/delete/oneliner) keep rejecting
closed cases via locate_case_or_404 as defense-in-depth behind the
now-hidden buttons.
Tests (RED-first):
- web_test: audio serves a closed case's file (200, not 404).
- case_page_test: closed case page + recordings render without
?show_closed=1; closed case hides analyze/reset; open case still shows
delete; closed case hides delete.
- analyze_test: closed_case_returns_404_on_detail rewritten to
closed_case_detail_renders_read_only — the old test pinned the
now-superseded access-gate contract.
cargo test (440 passed), clippy, and fmt all clean.
closes#17
Following "Aufnahmen anzeigen" on a closed case returned HTTP 404
{"error":"Case not found"} — an apparently empty page. handle_case_recordings
unconditionally resolved the case via locate_case_or_404, which rejects any
case carrying the .closed marker, so the recordings (which stay on disk after
a close — only a marker is written) became unreachable.
Fix mirrors handle_case_page: the handler now takes Query<CaseListQuery> and
branches to locate_closed_case_or_404 when show_closed=1 is requested, falling
back to the IDOR-guarded locate_case_or_404 otherwise. The in-app link in
case_page.html now carries ?show_closed=1 for closed cases (mirroring
my_cases.html), so navigation from a closed case's detail page no longer drops
the flag and 404s.
Reuses the existing CaseListQuery/include_closed() extractor; no parallel type.
Minimal fix, no surrounding refactor.
RED+GREEN combined: regression test case_recordings_honours_show_closed_for_closed_case
in server/tests/case_page_test.rs seeds a closed case with a recording and
asserts the route returns 200 with the recording listed, not 404. Full
case_page_test suite green (18 passed); cargo clippy clean; cargo fmt clean.
closes#17
German practice software (INDAMED MEDICAL OFFICE) stores free text in
ISO 8859-15, an 8-bit code. Pasting our cleaned document there rendered
typographic Unicode the LLM emits — en-/em-dashes, curly quotes,
ellipses, fraction slashes — as replacement boxes, while umlauts, the
micro sign, guillemets and the euro sign (all within 8859-15) pasted
fine. The KBV xDT data standard confirms ISO 8859-15 as the canonical
charset for German practice systems.
Add `analyze::charset::fold_to_iso8859_15`, applied in the worker right
after the gazetteer and before `content_md` is persisted: representable
chars pass through verbatim, the residual is transliterated via the
`deunicode` table (en-dash -> '-', ellipsis -> '...', '1/2' -> "1/2"),
and unmapped chars are left untouched rather than dropped. Because
representable chars never reach deunicode, its ASCII-only nature does
not flatten umlauts or the micro sign.
refs #13
The server-rendered pages worked on desktop but were near-unusable on a
phone: no viewport meta tag (mobile browsers laid out in a ~980px canvas
and zoomed out, rendering everything tiny) and a fixed `max-width: 900px`
desktop body with zero responsive breakpoints.
Fix is CSS-only and purely additive:
- Add `<meta name="viewport" content="width=device-width, initial-scale=1">`
to all four page templates (each owns its own <head> — askama uses
import/include here, no shared base layout, so the tag lands in 4 places).
- Append a single `@media (max-width: 640px)` block to the three
desktop-width pages (my_cases, case_recordings, case_page): case rows
and action bars reflow to a single column, the audio player goes
full-width, and primary touch targets (delete/reopen/play) get a 44px
minimum. login.html already has a 360px column layout, so it needs the
viewport tag only.
Desktop layout is preserved by construction: a `max-width: 640px` query
never matches at >=900px, so the desktop render is byte-identical. The
diff adds lines only — no existing rule is touched.
Breakpoint chosen at 640px (well below the 900px body width) so all
portrait phones, including the ~390px target, fall under it while
tablets keep the desktop layout.
New integration test server/tests/responsive_test.rs pins the two
textual acceptance criteria that survive without a headless browser:
every page ships the viewport meta, and the three desktop-width pages
ship an @media breakpoint. The visual criteria (operable at 390px, 44px
touch targets) are CSS-driven and verified by eye — deliberately not
covered by a browser-automation dependency.
Verified: cargo test -p doctate-server --test responsive_test (4/4
green); web/login/case_page regression suites green; diff is additive.
closes#10
render_prompt prefixed each recording with a '## <recorded_at>' heading.
No system prompt consumes that timestamp, and the LLM occasionally echoed
it into the final document as a heading. Join recordings with the bare
'\n\n---\n\n' separator instead; chronology is carried by recording order.
Adds a regression test pinning that no RFC3339 timestamp or '##' reaches
the prompt.
closes#8
This commit replaces the disk-based `analysis_input.json` marker for
in-progress analysis jobs with an in-memory
`Arc<RwLock<HashSet<PathBuf>>>`.
Previously, the existence of `analysis_input.json` was used as a signal
that a case was queued or being analyzed. This led to stale "Queued"
states after server restarts, as the disk file would persist while the
server process tracking it had vanished.
The new `InFlight` struct, held in `AppState`, provides a process-local
marker. This ensures that:
- Server restarts correctly reset the state, as the `HashSet` is
cleared.
- Race conditions for triggering analysis are handled by the `try_claim`
method, replacing the previous reliance on file system `O_EXCL` flags.
- The `AnalyzeJob` payload now travels with the job over the channel,
eliminating the need for workers to re-read `analysis_input.json` from
disk.
This change simplifies state management and improves the reliability of
analysis job tracking.
Introduce a `Badge` enum and a `badge_for` function to consolidate
case status logic. This replaces multiple boolean flags (`analyzing`,
`has_failed_recording`, `analysis_required`) with a single, more
expressive `badge` field.
The HTML template has been updated to use this new badge system,
removing old status classes and logic. The `scan_user_cases` function
no longer needs the `worker_busy` argument as the analysis state is
now derived directly from the `auto_trigger` state.
This change improves code clarity and maintainability by centralizing
status determination and reducing redundant boolean flags.
The `boot_time` parameter is introduced to `evaluate_state` and its
callers. This parameter acts as a lower bound for the "seen at"
timestamp used in determining the `Idle` state for cases.
Previously, the `elapsed` duration was calculated as
`now.duration_since(latest_mtime)`. This meant that a server restart
could immediately cause old recordings to be classified as `Idle` if
their modification times were significantly in the past, potentially
triggering a large number of LLM calls upon the first render of the
`/cases` page.
By using `seen_at = std::cmp::max(latest_mtime, boot_time)`, the idle
calculation now ensures that a fresh server start respects a grace
period. Old recordings will not satisfy the idle threshold until the
elapsed time since the server's boot (or the recording's modification
time, whichever is later) exceeds the `idle_threshold`.
`SystemTime::UNIX_EPOCH` is used as the default `boot_time` in tests and
in `create_router_and_session_store_with_settings`. This preserves
existing test behavior where the idle grace period is effectively
disabled. For production, `main.rs` now captures the server's actual
boot time and passes it down.
This change also refactors `evaluate_case` and `try_enqueue` to accept
the new `boot_time` parameter. `try_enqueue_all_for_user` is updated to
pass `boot_time` along. `boot_scan_state` and `compute_state_for_user`
in `recovery.rs` also receive `boot_time`.
Additionally, a new test case
`analyze_required_endpoint_enqueues_only_required_cases` is added to
verify the behavior of the `/cases/analyze-required` endpoint. This
endpoint now correctly enqueues only `Required`
This commit makes the `write_input_overwrite` function public. This is
necessary because the new `handle_analyze_required` function in
`bulk.rs` needs to call it. Previously, it was only used internally by
`auto_trigger.rs`.
Introduces a visual indicator for stale analysis on the case page and a
new section on the "My Cases" page to highlight cases requiring
analysis. This improves user awareness of data currency and pending
actions.
The `evaluate_case` function has been replaced with `evaluate_state` in
the `auto_trigger` module. This change introduces a new enum,
`CaseAnalysisState`, which provides a more granular representation of a
case's analysis status.
The `try_enqueue` and `try_enqueue_all_for_user` functions now utilize
`evaluate_state` to determine if analysis should proceed. This ensures
that analysis is only triggered for cases in the `Idle` or `Required`
states.
Additionally, the `CaseFlags` struct has been updated to include
`analysis_stale_with_doc`, and corresponding logic has been added in
`compute_flags` and `compute_case_view` to accurately reflect the
analysis status for UI display. The `UserCaseView` and `MyCasesTemplate`
structs have also been updated to incorporate these new fields.
This commit introduces a significant refactor to the auto-trigger and
recovery mechanisms for the analysis pipeline. The core change is the
introduction of a more granular `CaseAnalysisState` enum, replacing the
simpler `AutoDecision`.
The `evaluate_state` function now computes the full lifecycle state of a
case, considering factors like the presence of recordings, pending
transcripts, queued jobs, failure markers, document staleness, and an
idle threshold. This provides a richer understanding of the case's
status.
The `evaluate_case` function is retained as a backwards-compatible
adapter for existing callsites and tests. It maps the new
`CaseAnalysisState` to the old `AutoDecision` enum, simplifying the
decision to "enqueue" or "skip."
The `recovery` module is also refactored. The `scan_and_enqueue`
function is replaced by `boot_scan_state`, which is now purely
observational. It aggregates statistics about cases in different states
at boot time and logs them, but it no longer attempts to re-enqueue
jobs. The responsibility for triggering analysis now lies with the
per-render `try_enqueue_all_for_user` logic, which utilizes the new
typed state machine.
This refactoring aims to provide more clarity and control over the
analysis pipeline's state management, particularly in handling cases
that have previously failed or are waiting for further input.
The `document.md` file has been replaced by `document.json`. This new
format acts as a JSON envelope, containing the original markdown content
along with metadata such as `covered_mtime`, `written_at`, and
`backend_id`.
This change addresses a bug where new recordings arriving during an LLM
analysis call could be ignored. The `covered_mtime` field in the new
JSON envelope ensures that the document's metadata accurately reflects
the state of recordings at the time of analysis, preventing incorrect
"analysis current" states.
Additionally, the `analysis_input.json` file is now removed upon
successful analysis, and a failure marker is used to indicate analysis
failures. This ensures that a case is re-analyzed if necessary.
The `analysis_idle_threshold_secs` configuration option has been
introduced to control the delay before a case in the `Required` state is
automatically promoted to `Idle` and enqueued for LLM analysis.
Failure markers now carry finish_reason and usage tokens (prompt /
completion / reasoning) pulled from the chat-completions response, so
the case-page "Technische Details" block can show why an analysis
truncated — e.g. gpt-oss-120b hitting max_completion_tokens with the
chain-of-thought before producing any visible answer.
The worker also emits a new CaseEventKind::AnalysisFailed after writing
the marker, so subscribed browsers reload immediately instead of leaving
the page on the "Wird analysiert …" placeholder.
The /web/ prefix predated the /api/ split; today it just clutters every URL
without disambiguating anything. All 16 browser routes move to the apex
(/cases, /login, /magic, /events, /audio/...). The 6 /api/* routes are
unchanged. A new /->>/cases redirect closes the apex 404.
The open-redirect guard in magic.rs and case_actions.rs flips from a
positive whitelist (starts_with("/web/")) to a deny-list: same-origin path,
not protocol-relative, not under /api/, no \. The /api/ exclusion is now
load-bearing and covered by tests.
Pre-production: no transition redirects.
Introduce `display_name` field to `User` struct for human-readable names
in the UI.
This falls back to the user's `slug` if not provided.
Update `AuthenticatedWebUser` and `MyCasesTemplate` to use this new
field for rendering.
Add tests for parsing `display_name` and ensure fallback logic works
correctly.
Update `users.toml.example` to document the new field.
Multi-stage Dockerfile, host-network compose, idempotent deploy script.
Configs and persistent data bind-mounted from /opt/stacks/doctate-server.
deploy-server.sh handles bootstrap + re-deploy in one command with
auto-prune (keep 5 last SHA-tagged images).
See docs/deployment.md for bootstrap, rollback, and TLS-proxy notes.
Add Fachrichtung to prompt and refine medical terms.
Added Fachrichtung to the system prompt, instructing the LLM to consider
the medical specialty for precise terminology.
Also, added "Abdomensonographie" to the `medical_terms.txt` file.
Increase `max_completion_tokens` for `gpt_oss_120b` from 8192 to 16384
to prevent incomplete responses.
Navigate to the detail screen before the recording screen when starting
a new case, ensuring correct back navigation.
When a previous analysis attempt fails and leaves a failure marker, the
retry mechanism now correctly identifies this state. It removes the
stale
input file, allowing the retry to proceed without a conflict error. This
ensures users can recover from analysis failures more gracefully.
An empty `backend` form field now correctly falls back to the default. A
non-empty field submitted by a non-admin user is rejected with a 403
error, preventing potential tampering or issues arising from stale
sessions after role changes.
This commit adjusts the configuration for the `gpt-oss-120b` model to
run in plain-text mode. This is necessary because when
`reasoning_effort` is set to "medium", the reasoning tokens combined
with schema-constrained decoding can exhaust the `max_completion_tokens`
budget, leading to an empty content response.
The change restores the previous behavior where `gpt-oss-120b` was
configured without `response_format` or `top_p`, ensuring the request
body sent to Ionos remains byte-identical to the pre-multi-backend
state. This prevents regressions and maintains stability for existing
production workflows.
Configure Llama 3.1 405B with specific temperature, top_p, and a format
instruction. This addresses issues with infinite repetition and empty
document generation observed with this model. The format instruction is
added as a separate system message to avoid modifying the core medical
prompt.
`recovery::enqueue_pending_for_user` previously enqueued any case with
`analysis_input.json` and no `document.md`, ignoring the failure marker
that `auto_trigger::evaluate_case` already honours. A timed-out LLM
call (e.g. Llama 3.1 405B at ~20 tok/s with default 4096 max_tokens)
left the input in place, the next case-list reload re-enqueued it, and
the worker hung another full timeout window — starving fresh user
clicks behind a failed retry loop.
Recovery now mirrors the same skip: if a marker exists whose
`last_recording_mtime` matches the input's, treat it as "already
tried, do not retry until input changes". Fresh recordings bump the
mtime and recovery picks the case back up.
Replace the single [llm] block in settings.toml with a curated catalog
of LLM "backends" (provider + model + sampling params + system prompt)
in server/src/analyze/backend.rs. Two backends ship initially:
gpt_oss_120b (default, reasoning_effort=medium) and llama_3_1_405b
(uses response_format: json_schema to sidestep the <|eot_id|> leak).
The case page now renders one submit button per available backend; the
clicked button's name=backend value rides through AnalysisInput.backend_id
to the worker, which looks it up via find_backend() with default fallback.
A submitted unknown backend id is rejected as 400. .analysis_failed.json
records the failed backend and the failure banner surfaces its label.
The API key now comes from IONOS_API_KEY (env), not settings.toml; the
[llm] section is read into a discarded field so old configs still load.
Backends without a satisfied requires_api_key are filtered from the UI
(any_backend_available()). Three end-to-end tests that mocked the LLM
endpoint via settings.llm.url are #[ignore]'d with a clear note — they
need per-test backend injection (Arc<Vec<LlmBackend>> through the worker)
before they can be re-enabled.
When an LLM analysis fails, a `.analysis_failed.json` marker is created.
If no document exists yet and the auto-trigger is blocked by this
marker,
the case page must display a banner. This banner provides the failure
reason and a button to retry the analysis, ensuring users are not left
in a dead-end state.
The `read_failure_marker` function is made public to allow the web layer
to access this failure information. The `CasePageTemplate` is updated to
include `analysis_failed` data, which conditionally renders the new
`.failure-banner` HTML.
This change prevents cases from becoming unrecoverable due to transient
LLM errors.
Includes a button for administrators to copy all transcripts and the
document markdown to the clipboard.
The data is embedded as JSON in a script tag and pre-sanitized to
prevent
`</script>` tag injection.
The JavaScript handles copying to the clipboard using the modern
Clipboard API,
with a fallback for older browsers.
Add [workspace] / [workspace.package] / [workspace.dependencies]
blocks to server/Cargo.toml and clients/desktop/Cargo.toml so each
becomes a self-contained workspace with its own Cargo.lock.
Remove the root Cargo.toml and Cargo.lock; the repo now hosts
three independent build worlds: server, clients/desktop, experiments.
Convert common/Cargo.toml to concrete versions (was workspace-
inherited from the root). common/ is consumed via path-dep from
all three worlds, but is not itself a workspace -- consumers
unify versions via their own lockfiles. Add common/Cargo.lock to
.gitignore per Cargo lib-only-crate convention.
Verification:
- server/Cargo.lock has 0 eframe/egui/winit entries
- cargo build --release --locked passes in server/
- test split: server 385 + desktop 89 + common 34 = 508 total
- bin output: clients/desktop/target/debug/doctate (named "doctate")
Move three directories into their target topology:
- doctate-common/ -> common/
- client-desktop/ -> clients/desktop/
- watch/wearos/ -> clients/wearos/
Update doctate-common path-dependency in 3 Cargo.toml files
(server, clients/desktop, experiments) and the workspace
members list in the root Cargo.toml. Crate names unchanged
(doctate-server, doctate-common, doctate-desktop).
Workspace still in single-Cargo.lock form; isolation into
standalone workspaces follows in stage 3. All 508 tests pass,
experiments standalone-workspace also resolves the new path.
Introduce a new field `reasoning_effort` to the `ChatRequest` struct.
This field, set to "medium" by default, allows tuning LLM reasoning
depth to balance hallucination risk against latency and token cost. This
setting is specifically for Ionos' GPT-OSS reasoning models and is
ignored by other OpenAI-compatible models.
The system prompt for the medical assistant has been refactored to
improve clarity and precision. Key changes include:
- Enhanced instructions on handling transcribed errors, emphasizing
verbatim transcription and the use of `==...==` for uncertain parts.
- More explicit guidance on the formatting of dosage schemes, including
the conversion of numerical representations to hyphenated formats
(e.g., "1-0-2").
- Introduction of specific examples for common medical abbreviations and
terms, such as "CDAI > 450", "per os", and "iv.".
- A new critical rule to enclose any output not part of the transcript
in `[...]`.
- The addition of "Ileus" to `medical_terms.txt` and "Adalimumab",
"Infliximab" to `vocabulary.txt`.
The `Gazetteer` struct has been refactored to store `Entry` structs,
where each entry contains a canonical form and a set of bypass aliases.
This change enables the bypassing of Hunspell's dictionary veto for
specific, explicitly defined aliases.
The `parse_line` function has been introduced to handle the new line
format, which allows for `-`-prefixed aliases. Malformed lines with
secondary tokens not prefixed by `-` now result in a hard error, failing
server startup with `io::ErrorKind::InvalidData`.
The `replace` method has been updated to check bypass aliases before
consulting the dictionary, ensuring that explicitly allowed aliases are
used for rewriting. This addresses issues where Hunspell incorrectly
identifies compounds, blocking necessary corrections.
The `README.md` in the `vocab` directory has been updated to document
the new bypass-alias functionality.
This commit removes the `whisper_variant` and `whisper_hotwords_variant`
fields from the `run_id` generation and the `print_meta_summary`
function.
These variants are no longer used as the project is shifting focus to
LLM-based generation. The `run_full_case.rs` example has also been
updated to reflect this change.
Sandbox-validated against case c414cf52 (3 runs each, fair pre+post-LLM
gazetteer pipeline): the new prompt eliminates two hallucination classes
the prior version produced — unmarked "Hypotonie" when the dictation said
"Hypertonie" (0/3 vs 2/3) and inventing units like "35 ng/l" for values
without unit (0/3 vs 2/3) — while keeping Latin terms (Punctum Maximum
etc.) intact in 3/3 runs vs 2/3.
Block layout reorganized so each rule appears exactly once: AUFGABE /
QUELLE / KORREKTUR-POLITIK / TREUE / CHRONOLOGIE / DOSIERUNGSSCHEMA /
MARKIERUNGEN. The removed "AKTIVE KORREKTUR (nicht durchreichen)" hammer
block is no longer load-bearing because the gazetteer's pre-LLM pass now
catches the typical drug-name typos before the LLM sees them.
Vocabulary additions (single-token, alphabetical, ≥5 chars):
- vocabulary.txt: Enoxaparin (was missing — Inoxaparin→Enoxaparin is a
recurrent ASR error and the prior pipeline relied on the LLM alone)
- medical_terms.txt (new file, picked up by the dir-glob loader):
Holosystolikum, Klappenvitien, Koronarbaum, Pumpfunktion
Sandbox-iteration unter experiments/case_c414cf52 (3 Runs je Variante,
gpt-oss-120b @ Ionos, temperature=0). v2_treue gewinnt gegen Baseline
und v1_treue auf folgenden Achsen ohne Curve-Fitting im Prompt:
- Active drug-name correction (Inoxaparin→Enoxaparin, Thorazemit→
Torasemid, AFREF→HFrEF) durchgesetzt: 3/3 statt 0/3
- Schema 1-0-0 / 1-0-1 zuverlässiger: 3/3 statt 2/3
- Troponin-T fehlende Einheit als ==35== markiert (vorher unkommuniziert)
- Hypo/Hyper-Verlaufsgeschichte aus widersprüchlichen Aufnahmen
unterdrückt: 0/3 (war 1/3 in v1_treue)
- Anatomische Lokalisationen, "adipöser Ernährungszustand" und
Diktat-Reihenfolge bleiben treu (vorher selten aber gelegentlich
weggelassen oder umsortiert)
- Format als Fließtext explizit, Pseudo-Headings reduziert
Strukturell offen (kein Prompt-Fix möglich): "Holosystolikum" aus
Whispers Halbwortruine "Tolikum", "Koronarbund"-Halluzination aus
"Corona-Baum" — beides braucht Gazetteer-Erweiterung oder
Tokenizer-Refactor, separate PRs.
Umstellung auf Rust-Raw-String macht die Prompt-Datei deutlich
lesbarer (keine \n-Escapes, keine \"-Escapes mehr).
The `RecordingView` struct now includes `gain_db` to represent
replay-gain. This value is read from the recording's metadata (`.json`
sidecar) and passed to the frontend.
The JavaScript in `case_recordings.html` uses this `data-gain-db`
attribute to apply replay-gain using the Web Audio API. This ensures
consistent playback loudness across different recordings. The
implementation uses a shared `AudioContext` to manage resources
efficiently and handles cases where the browser might not support
`AudioContext`.
Additionally, the audio recording on Wear OS now uses
`MediaRecorder.AudioSource.VOICE_RECOGNITION` instead of `MIC`. This
leverages platform noise and echo cancellation, aiming for a cleaner
signal before server-side gain adjustment.
This commit extracts the logic for mapping audio analysis results into
the `RecordingMeta` fields into a new, pure function
`analysis_to_meta_fields`. This improves testability by separating the
core mapping logic from logging and other side effects.
The previous implementation directly handled logging and conditional
logic within the `run` function. The new function encapsulates these
mapping rules, including handling finite values, silent audio, and error
propagation. The caller, `run`, is now responsible for logging any
analysis failures before calling the helper. Unit tests have been added
to verify the behavior of `analysis_to_meta_fields` in various
scenarios.
This commit introduces the `Loudness` struct to store replay-gain data
(mean, max, and calculated gain in dB) derived from `ffmpeg`'s
`volumedetect`.
The `RecordingMeta` struct is updated to include an optional `loudness`
field. This allows for consistent playback volume across recordings with
varying microphone levels by applying gain in the browser.
Additionally, new `server/src/loudness.rs` and
`server/src/transcribe/ffmpeg.rs` modules are added. The former defines
constants for replay-gain targets and logic to compute gain, while the
latter handles audio analysis using `ffmpeg` to extract loudness
metrics.
The `#[serde(default)]` attribute on the `loudness` field in
`RecordingMeta` ensures backward compatibility with older metadata files
that do not contain this field. Tests are included to verify round-trip
serialization and parsing of older formats.
Introduces a new `validate` module to centralize input validation for
web
requests. This module contains functions to check the shape and format
of
various user-supplied strings before they are processed by the main
application logic.
Specifically, this commit adds validation for:
- Login slugs: Ensures slugs conform to a strict pattern (lowercase
alphanumerics, `_`, `-`) to prevent path traversal and other attacks.
- Magic link tokens: Validates token length and character set to ensure
they are URL-safe and within expected bounds.
- Recorded at timestamps: Enforces a strict RFC3339 format with second
granularity (`YYYY-MM-DDTHH:MM:SSZ`) to prevent filename parsing
issues
and ensure consistent data grouping.
These validators are integrated into the `handle_login_submit`,
`handle_consume` (magic link), and `handle_upload` routes. This change
enhances security by preventing malformed inputs from reaching sensitive
parts of the application and improves robustness by catching data format
errors early.
Unit and integration tests have been added to verify the functionality
of
these validators and their integration into the respective endpoints.
This commit changes the way transcriptions are stored and accessed.
Instead of using plain text files (`.transcript.txt`), transcriptions
will now be part of a JSON metadata file (`<stem>.json`). This allows
for richer metadata to be stored alongside the transcript, such as
duration, and provides a more robust mechanism for tracking
transcription states.
The changes include:
- Updating documentation and code to reflect the new `.json` file
extension.
- Modifying file handling logic to read and write JSON metadata.
- Adjusting tests to accommodate the new file format.
Replaces the `.transcript.txt` sidecar with a structured `.json` file
for recording metadata. This change consolidates transcript text,
duration, and other potential metadata into a single, extensible JSON
object.
This also refactors the `TranscriptState` enum to better represent the
on-disk state (absence of file means pending) and the in-memory
representation. The `Transcript` enum now specifically models the
terminal outcomes of the transcriber (`Silent` or `Content`).
The commit includes updates to documentation, data structures, path
handling, and various tests to align with the new metadata format.
Config sheds its 14 Bucket-B fields (retention, whisper, ollama, llm) and
gains a sibling Arc<Settings> in AppState, loaded from settings.toml at
startup via Settings::load_or_default. Workers (transcribe, analyze,
recovery, auto_trigger) and routes (health, bulk, case_actions,
user_web) take settings as a separate parameter or State<> extractor;
Config::llm_configured moves to Settings::llm_configured.
TestConfig::build_pair() returns (Arc<Config>, Arc<Settings>); the new
create_router_with_settings / create_router_and_session_store_with_settings
helpers wire both into integration tests that exercise LLM/Whisper/Ollama.
The legacy single-Arc create_router falls back to Settings::default()
so the 17 non-LLM tests stay untouched.
Verified: cargo build clean, clippy --all-targets clean, 318 tests pass.
claude
Brummel