The /web/ prefix predated the /api/ split; today it just clutters every URL
without disambiguating anything. All 16 browser routes move to the apex
(/cases, /login, /magic, /events, /audio/...). The 6 /api/* routes are
unchanged. A new /->>/cases redirect closes the apex 404.
The open-redirect guard in magic.rs and case_actions.rs flips from a
positive whitelist (starts_with("/web/")) to a deny-list: same-origin path,
not protocol-relative, not under /api/, no \. The /api/ exclusion is now
load-bearing and covered by tests.
Pre-production: no transition redirects.
When a previous analysis attempt fails and leaves a failure marker, the
retry mechanism now correctly identifies this state. It removes the
stale
input file, allowing the retry to proceed without a conflict error. This
ensures users can recover from analysis failures more gracefully.
Replace the single [llm] block in settings.toml with a curated catalog
of LLM "backends" (provider + model + sampling params + system prompt)
in server/src/analyze/backend.rs. Two backends ship initially:
gpt_oss_120b (default, reasoning_effort=medium) and llama_3_1_405b
(uses response_format: json_schema to sidestep the <|eot_id|> leak).
The case page now renders one submit button per available backend; the
clicked button's name=backend value rides through AnalysisInput.backend_id
to the worker, which looks it up via find_backend() with default fallback.
A submitted unknown backend id is rejected as 400. .analysis_failed.json
records the failed backend and the failure banner surfaces its label.
The API key now comes from IONOS_API_KEY (env), not settings.toml; the
[llm] section is read into a discarded field so old configs still load.
Backends without a satisfied requires_api_key are filtered from the UI
(any_backend_available()). Three end-to-end tests that mocked the LLM
endpoint via settings.llm.url are #[ignore]'d with a clear note — they
need per-test backend injection (Arc<Vec<LlmBackend>> through the worker)
before they can be re-enabled.
This commit changes the way transcriptions are stored and accessed.
Instead of using plain text files (`.transcript.txt`), transcriptions
will now be part of a JSON metadata file (`<stem>.json`). This allows
for richer metadata to be stored alongside the transcript, such as
duration, and provides a more robust mechanism for tracking
transcription states.
The changes include:
- Updating documentation and code to reflect the new `.json` file
extension.
- Modifying file handling logic to read and write JSON metadata.
- Adjusting tests to accommodate the new file format.
Replaces the `.transcript.txt` sidecar with a structured `.json` file
for recording metadata. This change consolidates transcript text,
duration, and other potential metadata into a single, extensible JSON
object.
This also refactors the `TranscriptState` enum to better represent the
on-disk state (absence of file means pending) and the in-memory
representation. The `Transcript` enum now specifically models the
terminal outcomes of the transcriber (`Silent` or `Content`).
The commit includes updates to documentation, data structures, path
handling, and various tests to align with the new metadata format.
Config sheds its 14 Bucket-B fields (retention, whisper, ollama, llm) and
gains a sibling Arc<Settings> in AppState, loaded from settings.toml at
startup via Settings::load_or_default. Workers (transcribe, analyze,
recovery, auto_trigger) and routes (health, bulk, case_actions,
user_web) take settings as a separate parameter or State<> extractor;
Config::llm_configured moves to Settings::llm_configured.
TestConfig::build_pair() returns (Arc<Config>, Arc<Settings>); the new
create_router_with_settings / create_router_and_session_store_with_settings
helpers wire both into integration tests that exercise LLM/Whisper/Ollama.
The legacy single-Arc create_router falls back to Settings::default()
so the 17 non-LLM tests stay untouched.
Verified: cargo build clean, clippy --all-targets clean, 318 tests pass.
- Replace native confirm() in the recording list with an inline
two-step confirm row; hide the player via visibility:hidden
while open so the row height stays stable.
- Hard-redirect recording delete back to /web/cases/{id}/recordings
instead of relying on the Referer header (which silently fell
back to /web/cases when stripped).
- Switch analyze/reset to a hidden return_to form field so the
source page (case detail vs. case list) is preserved reliably,
with same-origin /web/ prefix validation.
- Tighten delete_recording_test on the concrete Location header;
adjust analyze/reset redirect expectations to the new
case-detail fallback.
Pulls three pattern groups into shared helpers so a rename or wire-format
tweak edits one file instead of 10+:
- BulkAction enum in doctate-common replaces the "close"/"analyze"/"reset"
string literals that were duplicated between the bulk handler and 3
attack/CSRF test files. FromStr preserves the exact "Unbekannte Aktion"
error shape; the handler match is now exhaustive over the enum.
- join_url helper in doctate-common absorbs 7 identical
trim_end_matches+format! call sites across client-core, client-desktop,
server/transcribe (ollama, whisper), and server/analyze (llm).
- server/tests/common/artefacts.rs re-exports ONELINER_FILENAME
(doctate-common), DOCUMENT_FILE + ANALYSIS_INPUT_FILE
(doctate-server::analyze), and CLOSE_MARKER (doctate-server::paths) so
test files reference the canonical name instead of inlining literals.
Also lifts client-desktop local duplication:
- paths.rs: project_path helper collapses 4 identical ProjectDirs chains
- main.rs: or_die helper replaces 4 eprintln!+exit(1) blocks
- app.rs: named RecordingContext struct replaces (Uuid, String) tuple
at 3 sites around the ffmpeg-flush finalization path
Verification: 397 tests pass (baseline was 389; +8 for new unit tests on
BulkAction and join_url), 0 failed, 4 ignored (unchanged). clippy clean.
Finishes the lift of shared helpers into `tests/common/`. Covered:
- health, web, oneliners_api, upload (31 tests; upload exercises the
new `multipart_upload_body` helper driven by doctate_common field
constants)
- sse_integration, sse_cleanup, transcribe, oneliner_heal_decoupled,
silent_case_empty, failed_only_case_empty_oneliner,
transient_failure_retries (24 tests)
Also applied cargo fmt across the test tree and fixed one clippy
needless_borrows_for_generic_args warning in analyze_test.
All 387 tests pass; 3 ignored (as before).
Side effect: health_test previously used a hardcoded `/tmp/doctate-test`
data path, which parallel `cargo test` runs could collide on. The
migration replaces it with the common unique-tmpdir pattern, removing
a latent flake.
Lift duplicated test-harness code (User factories, TestConfig builder,
login/CSRF flow, form/json helpers, case-directory seeding, URL path
builders) into `server/tests/common/` so future API changes touch one
file instead of every integration test.
Migrated batches: csrf_attack, auth, login, magic_link, security_headers
(28 tests); analyze, case_page, delete_recording, close_watermark,
retention_sweep (67 tests). All 95 tests still green.
Net line delta across these 10 files: +1113 / -1896 (~780 lines removed),
plus ~500 lines of new shared infrastructure under tests/common/.
Introduces CsrfForm<T>: a FromRequest extractor that looks up the
session's csrf_token and constant-time-compares it to a csrf_token
field in the form body. Drop-in replacement for Form<T> on every
state-changing /web/ POST handler. Missing or expired session →
redirect to /web/login; malformed body → 400; token mismatch → 403.
WebSession carries csrf_token, minted alongside the session token at
login and magic-link consume. Not rotated per request — rotation
would break multi-tab use and buys little over SameSite=Strict
cookies.
Handlers: bulk, purge-closed, reset, close, reopen, analyze,
delete-recording, logout all now require CsrfForm<_>. Login stays
unprotected (SameSite=Strict alone is sufficient — forced-login CSRF
has no impact on this codebase). /api/... is header-auth, exempt.
Constant-time compare via subtle::ConstantTimeEq avoids timing
oracles on the token.
Template work is NOT in this commit — browser forms still post
without csrf_token, so the live web UI will 403 until Schritt 6
(templates render the hidden field).
Tests: all 12 red specs in csrf_attack_test.rs now green, #[ignore]
removed. Integration tests that POSTed to protected endpoints
switched to a new create_router_and_session_store entrypoint which
returns a handle to the store so tests can read csrf_token for the
active session.
Move admin checks from individual bulk actions to the main handler for
`bulk.rs` and `case_actions.rs`. This consolidates the authorization
logic for these sensitive operations.
Additionally, refine the HTML template to conditionally render bulk
action UI elements and the purge form only when the user is an admin.
This ensures that non-admin users do not see or have access to these
administrative functions.
Update tests to reflect these changes, ensuring that non-admin users are
correctly rejected for these actions and that the UI is properly hidden.
Introduce a `preview_lines` setting in `users.toml` to control the
number of visible lines for the analysis preview in the case list. This
feature extracts the first paragraph of the `document.md` file, strips
markdown formatting, and displays it, capped by the configured
`preview_lines`. The actual line clamping is handled client-side via CSS
`line-clamp`.
The /api/oneliners time window is now read per-user from users.toml
(window_hours, default 72h). Clients no longer carry a window:
client.toml oneliner_window_hours, SyncConfig.window_hours, the
?hours=N query param, and the render_case_list cutoff filter are
gone. ETag suffix keeps the effective hours so an admin edit to
users.toml invalidates client caches on the next request.
OnelinersResponse.window_hours stays in the wire format, but now
exists solely to anchor client reconciliation.
Three UX polish items for the show-closed listing, grouped because
they are the same root problem in three surfaces:
* handle_reopen_case now redirects via resolve_return_path(&headers)
instead of hard-coding /web/cases, so a reopen from the
?show_closed=1 listing stays in that view. Matches the pattern
already used by analyze/reset/delete-recording.
* handle_close_case now redirects via the new helper
resolve_list_return_path(&headers), which wraps resolve_return_path
and additionally collapses a /web/cases/{uuid} detail path to
/web/cases while keeping the query string. A close from the detail
page previously would have tried to redirect back to the now-404
detail, or when triggered from ?show_closed=1 would have dropped
the query.
* Closed cases in my_cases.html now render the checkbox,
Analysieren/Neu analysieren and Reset controls in disabled state
instead of being hidden. Layout stays consistent with mixed-state
listings, and the user can see what actions would be available
after reopen. The handlers remain the authoritative guard via
locate_case_or_404 -> 404 for closed cases, so the HTML disabled
is a hint only.
Adds four unit tests for resolve_list_return_path and two integration
tests for the new redirect behaviour (close from listing with query,
close from detail with query).
GET /web/cases now honours ?show_closed=1: the toggle link flips the
listing between open-only (default) and open+closed (muted styling,
grey "geschlossen" badge, countdown "wird in N Tagen entfernt" when
auto_delete_days > 0). Each closed row shows a Lucide rotate-ccw
reopen button in the same slot where the close button sits on open
cases; selection checkboxes are suppressed so bulk-close cannot touch
closed cases.
GET /web/cases/:id honours ?show_closed=1 too: without the query a
closed case still 404s (keeps the default listing clean), with it the
detail page renders the reopen form in the header. The case_page
template swaps the icon based on is_closed.
The purge-closed bulk button from the earlier commit finally gets a
UI: it appears below the listing when show_closed=1 is active AND at
least one closed case is visible, with a JS confirm() + the
server-side confirm=yes guard. Two integration tests (listing +
detail) cover the happy path.
New [user.retention] TOML sub-block with auto_close_days and
auto_delete_days. Both default to 0, which disables the respective
sweep — a conservative default that preserves current behaviour for
existing deployments and lets admins test auto-close in isolation.
Only parsing + the test User literals are touched here; the actual
lazy sweep consuming these values lands in the next commit.
Iterates the user's data dir and removes every case directory that
carries a .closed marker. Requires form field confirm=yes as a server-
side guard against accidental browser-history re-POSTs. Emits CasePurged
per successfully removed case; per-case failures are logged but do not
abort the sweep.
UI button is deferred to the show-closed step so the endpoint is only
reachable after an explicit navigation. Three integration tests cover
the happy path, missing-confirm, and coexistence with open cases.
- POST /web/cases/:id/delete renamed to /close; emits CaseClosed.
- New POST /web/cases/:id/reopen removes the marker; emits CaseReopened.
- Bulk action "delete" renamed to "close"; the shared closed_at timestamp
still groups the selection for future UI features.
- POST /web/cases/undo-delete removed along with summarize_latest_close_group,
latest_close_timestamp and restore_close_group. Per-case reopen makes
the group-undo banner obsolete.
- CaseEventKind::CaseDeleted -> CaseClosed, CaseRestored -> CaseReopened.
- Templates updated to Lucide trash-2 icon, German label "Fall schliessen".
- Tests migrated; delete_watermark_test.rs moved to close_watermark_test.rs.
Internal rename: DELETE_MARKER -> CLOSE_MARKER, DeleteMarker ->
CloseMarker, is_deleted -> is_closed. The batch UUID is dropped;
undo now groups cases by their exact closed_at timestamp, which
bulk-close writes identically across its selection while per-case
close writes a unique stamp.
Behavior unchanged. No migration (project is pre-production);
legacy .deleted files in tmpdata were removed manually.
The oneliner is now stored as `OnelinerState` enum which can represent
three states: `Ready`, `Empty` or `Error`. This allows the client to
differentiate between a case with no medical content and a case where
the oneliner generation failed.
The `OnelinerState` enum is serialized to JSON and stored in
`oneliner.json` file. The `OnelinerEntry` struct has been updated to
reflect this change.
The client-desktop application has been updated to handle the new
`OnelinerState` enum and display appropriate UI elements for each state.
The server-side code has also been updated to read and write the
`OnelinerState` enum, and to handle the new file format.
The `reset_case_artefacts` function in
`server/src/routes/case_actions.rs` has been updated to remove
`oneliner.txt` and create `oneliner.json` instead.
The tests have been updated to reflect these changes.
Introduces a system for broadcasting real-time events to the web UI.
This enables features like automatic page reloads when case data changes
in the background, improving the user experience by keeping the UI
synchronized with the backend.
Key components:
- `events` module: Contains the `CaseEventKind` enum, `CaseEvent`
struct, and `EventSender` type for managing the broadcast channel.
- SSE endpoint (`/web/events`): Streams events to connected browsers.
- Client-side JavaScript: Listens for events and triggers debounced page
reloads.
- Integration points: Workers and route handlers now emit events when
relevant state changes occur.
This commit separates the case detail page into two distinct routes:
`/web/cases/{case_id}` for the main case information and document, and
`/web/cases/{case_id}/recordings` for a dedicated view of audio files
and their transcripts.
This change improves the organization and clarity of the case viewing
experience by segmenting the related but distinct information into their
own UI sections.
This commit consolidates the upload and polling functionalities into a
single `ServerSync` worker. The `oneliner_poller` and `uploader` modules
have been removed, and their responsibilities are now handled by
`server_sync`.
The `ServerSync` worker manages both uploading pending recordings and
polling the `/api/oneliners` endpoint. Uploads take precedence, and
polling only occurs when the pending upload directory is empty. The
worker now uses a unified HTTP client and a single `tokio::select!` loop
for managing these tasks.
Key changes include:
- Removal of `oneliner_poller.rs` and `uploader.rs`.
- Introduction of `server_sync.rs` and `upload.rs` in
`doctate-client-core`.
- `ServerSync` now handles upload events (`UploadEvent`) and provides a
`WorkerSnapshot` for UI feedback.
- Upload logic has been refactored to handle transient and terminal
errors more robustly, including file deletion for terminal failures.
- Polling logic is integrated into the `ServerSync` loop, utilizing the
existing `SnapshotCache` and `CaseStore`.
- The `App` component in `client-desktop` has been updated to use
`ServerSync` instead of the separate poller and uploader.
- Documentation comments have been updated to reflect the new structure.
The gazetteer has been refactored to act as a post-AI normalization
filter. Previously, it was used to annotate LLM input with potential
corrections from a vocabulary. This approach was ineffective because
LLMs often override such hints.
The new approach applies the gazetteer *after* the LLM has generated its
output. This allows for deterministic correction of known terminology,
including fixing LLM output drift (e.g., anglicized drug names).
Key changes:
- `annotate` function renamed to `replace`.
- The output format changes from `Canonical [?original]` to simply the
`Canonical` form.
- The gazetteer now operates on the final LLM output before persistence,
ensuring consistency.
- Tests have been updated to reflect this new behavior, focusing on the
final output rather than the LLM request payload.
The gazetteer now annotates text with `Canonical [?original]`. This
prioritizes the corrected term for the LLM while keeping the original
transcription as a fallback.
This change aligns the gazetteer's annotation strategy with the LLM's
prompt, which expects corrections to be marked for review. Previously,
the format was `original [?Canonical]`, which could lead to the LLM
using the potentially incorrect original term.
The `filter-hunspell-de.sh` script now uses a `MIN_LEN` variable that is
synchronized with the `MIN_TOKEN_LEN` constant in
`src/gazetteer/mod.rs`.
This ensures that short tokens, which are prone to false positives due
to
collisions with common German words, are filtered out by both the
gazetteer
loading and the Hunspell dictionary processing.
Additionally, the script's dependency instructions have been updated to
be
more comprehensive, listing package manager commands for Arch/CachyOS,
Debian/Ubuntu, and Fedora. The `awk` command has been introduced to
filter
out tokens shorter than `MIN_LEN` before sorting and deduplicating.
The `Gazetteer::insert` method has been updated to skip entries shorter
than `MIN_TOKEN_LEN` to prevent noisy phonetic collisions. The
`best_candidate`
helper function has been extracted to improve readability and structure.
The test cases have been updated to reflect the change in
`MIN_TOKEN_LEN`
from 4 to 5 and to use more relevant examples for the updated
functionality,
such as "Cerebrum" and "Zerebrum". The `anatomy.txt` file has been
updated
to reflect that anatomical terms are currently disabled. The
`medications.txt` file has been significantly expanded with a curated
list
of German medication brand and active ingredient names.
Pass a `Gazetteer` to the analyze worker to enable proper-name
correction. The gazetteer is loaded from disk at application startup. If
the directory is missing or unreadable, the worker will start without
proper-name correction capabilities.
The LLM client now conditionally adds the `Authorization` header only
when an API key is provided. The `llm_configured` check is updated to
reflect that an API key is not strictly required for Ollama-style
endpoints. A new integration test verifies the functionality against an
Ollama-compatible endpoint without an API key.
Introduce a new "reset" action for bulk operations and a dedicated
endpoint for individual case resets. These features allow administrators
to revert a case to its raw audio state by deleting derived artifacts
like transcripts, analysis input, and documents. The functionality also
includes renaming `.m4a.failed` files back to `.m4a` to re-trigger
transcription.
This commit also:
- Adds a `reset_case_artefacts` helper function to `case_actions.rs`.
- Implements the `handle_reset_case` endpoint.
- Adds the "Reset" button to the UI for administrators.
- Includes unit and integration tests to verify the new functionality
and access control.
The versioning of analysis input and document files
(`analysis_input_v{N}.json`, `document_v{N}.md`) has been removed. All
analysis inputs will now use `analysis_input.json` and generated
documents will use `document.md`.
This simplifies file management, as there's no longer a need to track
and manage multiple versions of these files within a case directory. The
analysis worker will now overwrite the existing `document.md` if it
exists, ensuring that the latest analysis result is always present. The
`version` field has also been removed from `AnalyzeJob` and
`AnalysisInput`.
The analyze recovery scan has been refactored to iterate through user
directories and enqueue pending analysis jobs more efficiently. The
`WorkerBusy` status has been introduced as an `Arc<AtomicBool>` to track
whether the analyze worker is currently processing a job.
This `WorkerBusy` flag is used in the `analyze::worker::run` function
and managed by a `BusyGuard` RAII struct, ensuring the flag is correctly
set and unset even in case of panics.
The `handle_my_cases` and `handle_case_detail` handlers in `user_web.rs`
now utilize the `WorkerBusy` flag to accurately display the "analyzing"
status and to trigger a self-heal of orphaned analysis inputs when the
worker is idle.
Additionally, the `WorkerBusy` type is now exported from
`server/src/lib.rs` to be accessible by other modules. The test cases
have been updated to include the `WorkerBusy` parameter when spawning
the analyze worker.
Introduces a new `/web/cases/bulk` endpoint to handle multiple case
actions simultaneously.
This change adds the following:
- A new `bulk` module for handling bulk operations.
- The `handle_bulk_action` function in `bulk.rs` to dispatch to specific
actions.
- `bulk_analyze` and `bulk_delete` functions to process the actions.
- Updates `Cargo.toml` and `Cargo.lock` to include necessary
dependencies: `form_urlencoded`, `serde_core`, `serde_html_form`, and
`serde_path_to_error`.
- Modified `axum-extra` features to include `form`.
- Adds checks for deleted cases in `collect_pending` for both analysis
and transcription recovery.
- Renames and modifies `handle_close_case` to `handle_analyze_case` in
`case_actions.rs` to better reflect its functionality.
- Adds a new `handle_delete_case` and `handle_undo_delete` to
`case_actions.rs`.
- Updates `my_cases.html` and `case_detail.html` to support bulk
actions, including checkboxes, a bulk action bar, and an "Undo last
delete" feature.
- Modifies `handle_audio` and `scan_cases` in `web.rs` to respect delete
markers.
- Updates `analyze_test.rs` with new request builders for analyze and
delete actions.
Brummel