This commit introduces the `Loudness` struct to store replay-gain data
(mean, max, and calculated gain in dB) derived from `ffmpeg`'s
`volumedetect`.
The `RecordingMeta` struct is updated to include an optional `loudness`
field. This allows for consistent playback volume across recordings with
varying microphone levels by applying gain in the browser.
Additionally, new `server/src/loudness.rs` and
`server/src/transcribe/ffmpeg.rs` modules are added. The former defines
constants for replay-gain targets and logic to compute gain, while the
latter handles audio analysis using `ffmpeg` to extract loudness
metrics.
The `#[serde(default)]` attribute on the `loudness` field in
`RecordingMeta` ensures backward compatibility with older metadata files
that do not contain this field. Tests are included to verify round-trip
serialization and parsing of older formats.
Replaces the `.transcript.txt` sidecar with a structured `.json` file
for recording metadata. This change consolidates transcript text,
duration, and other potential metadata into a single, extensible JSON
object.
This also refactors the `TranscriptState` enum to better represent the
on-disk state (absence of file means pending) and the in-memory
representation. The `Transcript` enum now specifically models the
terminal outcomes of the transcriber (`Silent` or `Content`).
The commit includes updates to documentation, data structures, path
handling, and various tests to align with the new metadata format.
Config sheds its 14 Bucket-B fields (retention, whisper, ollama, llm) and
gains a sibling Arc<Settings> in AppState, loaded from settings.toml at
startup via Settings::load_or_default. Workers (transcribe, analyze,
recovery, auto_trigger) and routes (health, bulk, case_actions,
user_web) take settings as a separate parameter or State<> extractor;
Config::llm_configured moves to Settings::llm_configured.
TestConfig::build_pair() returns (Arc<Config>, Arc<Settings>); the new
create_router_with_settings / create_router_and_session_store_with_settings
helpers wire both into integration tests that exercise LLM/Whisper/Ollama.
The legacy single-Arc create_router falls back to Settings::default()
so the 17 non-LLM tests stay untouched.
Verified: cargo build clean, clippy --all-targets clean, 318 tests pass.
Introduces `OnelinerLocks` to serialize read-modify-write operations on
`oneliner.json`. This prevents race conditions between the manual
override API (`PUT /api/cases/{case_id}/oneliner`) and the transcription
worker's auto-regeneration process.
The manual override now acquires a lock specific to the `case_dir`
before writing. The transcription worker's `update_oneliner` function
also acquires the same lock around its post-LLM re-read-and-write
sequence. This ensures that a doctor's manual edit always takes
precedence, even if it arrives while an auto-regeneration is in
progress.
This change also includes:
- A new `routes::oneliner_override` module for the PUT handler.
- Validation for the oneliner text length and emptiness.
- Integration tests for the override functionality, covering happy path,
error conditions, early latching, race conditions, and parallel PUTs.
This commit introduces a new `Manual` variant to the `OnelinerState`
enum, allowing for manual overrides of the oneliner text. This change
affects the client and server components to handle and display this new
state.
A `OnelinerOverrideRequest` struct is also added for API requests to set
manual oneliners. New tests are included to ensure the `Manual` variant
serializes and deserializes correctly.
The `compute_oneliner_display` function in
`server/src/routes/user_web.rs` is updated to treat `Manual` states the
same as `Ready` states for display purposes. The
`cases_needing_oneliner_in` function in
`server/src/transcribe/recovery.rs` is updated to not retry cases that
are in a `Manual` state.
Before, every Whisper error (5xx, timeout, Minerva down, corrupt audio,
4xx) renamed `<ts>.m4a` to `<ts>.m4a.failed` uniformly, turning transient
outages into permanent sackgassen that only a manual admin reset could
undo. Cases whose every recording was `.m4a.failed` stuck without a
persisted oneliner state; the UI masked them via a fallback in
`compute_oneliner_display`.
The core insight: a recoverable error is not an error. Transient Whisper
failures now leave the audio as plain `.m4a` so the existing page-load
heal (`enqueue_pending_for_user`) re-enqueues it on the next refresh —
which is exactly what happens when Minerva comes back. No new sidecar,
no retry count, no scheduler.
Changes:
- `WhisperError::is_transient` classifies `Http`, 5xx, 408, 429 as
transient; `Io` and other 4xx as permanent.
- Transcribe worker: transient → info log + continue (audio stays .m4a);
permanent → `mark_failed` as before.
- `has_any_transcript` counts `.m4a.failed` as terminal, so
`update_oneliner` runs for failed-only cases and settles
`OnelinerState::Empty` (analogous to the silent-only fix in 4531f85).
- New verdrängender `fehler`-Badge (#c00) in case list and detail when
at least one `.m4a.failed` exists; recording-level message shortened
to plain "Transkription fehlgeschlagen".
Tests:
- New integration: transient 503 leaves `.m4a` intact, heal recovers.
- New integration: `.m4a.failed`-only case settles to
`OnelinerState::Empty` without calling Ollama.
- New unit: `is_transient` table test across relevant status codes.
- New unit: `has_any_transcript` returns true for `.m4a.failed`-only
case and false for pending-only case.
- Existing worker test retargeted from 500 to 400 and renamed; added
companion `worker_leaves_m4a_intact_on_transient_whisper_error`.
Introduce `TranscriptState { Pending, Silent, Content }` in
doctate-common as the canonical three-way state of a recording's
`.transcript.txt` sidecar. Previously each call-site projected the
raw `Option<String>` / `.exists()` onto its own 2-state view and the
projections disagreed: the UI treated a 0-byte silent transcript like
`Content` while the oneliner worker treated it like `Pending`,
leaving silent-only cases stuck on "generiere Titel …" forever with
no persisted oneliner.json.
`update_oneliner` now settles a silent-only case to
`OnelinerState::Empty` when no `Content` transcript exists and no
recording is still `Pending`, so the UI resolves to "unbenannt" and
recovery treats it as terminal.
Single reader: `paths::read_transcript_state`. All call-sites
(scan_recordings, compute_oneliner_display, has_pending_recordings,
all_transcripts_joined, read_recordings, enqueue_pending_for_user,
scan_m4as, cases_needing_oneliner_in, case_recordings.html) go
through the same typed abstraction and must handle `Silent` via
exhaustive match.
Regression tests:
- silent_case_empty_test: heal path settles silent-only case to
Empty without calling Ollama
- case_page_silent_only_shows_empty_not_generating: UI renders
"unbenannt", not "generiere Titel …"
The oneliner is now stored as `OnelinerState` enum which can represent
three states: `Ready`, `Empty` or `Error`. This allows the client to
differentiate between a case with no medical content and a case where
the oneliner generation failed.
The `OnelinerState` enum is serialized to JSON and stored in
`oneliner.json` file. The `OnelinerEntry` struct has been updated to
reflect this change.
The client-desktop application has been updated to handle the new
`OnelinerState` enum and display appropriate UI elements for each state.
The server-side code has also been updated to read and write the
`OnelinerState` enum, and to handle the new file format.
The `reset_case_artefacts` function in
`server/src/routes/case_actions.rs` has been updated to remove
`oneliner.txt` and create `oneliner.json` instead.
The tests have been updated to reflect these changes.
Introduce `OllamaError::EmptyResponse` to represent cases where the
Ollama
model returns an empty result due to silence rules or lack of keywords.
This change prevents treating an empty response as a parse error and
logs
it at a lower severity level. A new integration test verifies this
behavior.
Introduces a system for broadcasting real-time events to the web UI.
This enables features like automatic page reloads when case data changes
in the background, improving the user experience by keeping the UI
synchronized with the backend.
Key components:
- `events` module: Contains the `CaseEventKind` enum, `CaseEvent`
struct, and `EventSender` type for managing the broadcast channel.
- SSE endpoint (`/web/events`): Streams events to connected browsers.
- Client-side JavaScript: Listens for events and triggers debounced page
reloads.
- Integration points: Workers and route handlers now emit events when
relevant state changes occur.
This commit enables serving audio files via HTTP Range requests. This is
crucial for allowing HTML5 audio players to seek to specific positions
within an audio file without re-downloading the entire file.
The changes include:
- Modifying `handle_audio` in `server/src/routes/web.rs` to parse
`Range` headers.
- Implementing `serve_range` to handle partial content responses.
- Adding a `parse_range` helper function.
- Updating tests to verify range request functionality.
- Adding `ACCEPT_RANGES: bytes` header to indicate support for range
requests.
- Storing recording duration in a sidecar file for faster UI rendering.
- Enhancing the HTML template to support a custom audio player with
seeking.
Remove the per-user oneliner watermark, which was previously used to
generate ETags for the `/api/oneliners` endpoint. The watermark was
intended to track the last successful oneliner write for each user.
The ETag generation has been refactored to use a deterministic FNV-1a
hash. This hash is computed over a sorted list of tuples containing
`(case_id, last_recording_at_ns, oneliner_mtime_ns)` for all visible
cases. This approach ensures that the ETag changes whenever any
listen-relevant file system modification occurs, such as a new
recording, oneliner regeneration, soft-delete, undo, or
upload-auto-reopen. This new method is more robust and avoids the
complexity of managing per-user state.
The `OnelinerWatermark` type alias and its associated logic have been
removed from `AppState` and the relevant modules (`transcribe::worker`,
`transcribe::recovery`, `routes::oneliners`, `main`). New integration
tests have been added to verify that various delete operations (single
delete, undo delete, bulk delete) correctly trigger an ETag update,
ensuring cache invalidation.
This commit introduces a new API endpoint `/api/oneliners` that allows
authenticated users to retrieve a list of their recent oneliners.
The endpoint supports conditional GET requests using the `ETag` header
for efficient caching. It also allows specifying a time window for the
oneliners via the `hours` query parameter, with sensible defaults and
clamping.
The implementation includes:
- A new route handler `handle_oneliners`.
- A new type `OnelinerWatermark` for tracking user-specific timestamps.
- Logic for scanning user data directories, filtering by time, and
handling deleted cases.
- Test cases to cover various scenarios, including caching, time
windowing, and edge cases.
Avoid redundant LLM calls by waiting until all recordings in a batch
have been processed before regenerating the oneliner. This change
introduces a helper function `has_pending_recordings` to check for any
`.m4a` files without a corresponding transcript, and the
`update_oneliner` logic now only proceeds if no pending recordings are
found.
This commit integrates the Gazetteer into the transcription pipeline to
normalize terminology.
The Gazetteer, loaded at startup, is now passed to the transcription
worker. Before persisting the transcription text, it is processed
through the Gazetteer's `replace` method, ensuring standardized
terminology. This normalization is also applied to the generated
"oneliner" text.
Additionally, the `regenerate_missing_oneliners` function in the
recovery module has been updated to accept and utilize the Gazetteer,
ensuring that regenerated oneliners also benefit from terminology
normalization.
A new integration test, `transcribe_worker_normalizes_whisper_output`,
has been added to verify that the Whisper output is correctly normalized
by the Gazetteer before being saved to the transcript file.
This commit introduces a recovery mechanism to regenerate `oneliner.txt`
files for cases where transcription was successful but the oneliner
generation failed due to a crash between writing the transcript and the
oneliner.
The `transcribe::recovery::regenerate_missing_oneliners` function is
added and called at startup after `scan_and_enqueue`. This function
identifies cases with non-empty transcripts but missing oneliners and
triggers their regeneration.
The `transcribe::worker::update_oneliner` function is refactored to
always regenerate the oneliner from all available transcripts in a case,
ensuring that later recordings can correct earlier ones. This replaces
the previous `ensure_oneliner` logic which only generated the oneliner
if it didn't exist.
New helper functions `cases_needing_oneliner` and
`all_transcripts_joined` are introduced to support these changes.
Comprehensive unit tests are included for the new recovery and file
processing logic.
The case listing on the "My Cases" page is now grouped by local date,
with labels for "Heute", "Gestern", or the ISO date. This improves
readability and organization.
Additionally, the handling of silent recordings has been refined.
Previously, the absence of usable recordings would result in an error.
Now, the analysis worker gracefully handles this by writing a stub
document and skipping the LLM call. This prevents unnecessary errors and
provides a clearer status for silent cases.
The `dictate.sh` script has been updated to simplify the case directory
lookup logic. Instead of iterating through `open` and `done`
subdirectories, it now directly checks for the case ID and ensures it's
not marked as deleted. This simplifies the script and improves
efficiency.
The `server/Cargo.toml` and `server/Cargo.lock` have been updated to
include the `num_threads` dependency and enable additional features for
the `time` crate, which are necessary for proper local time zone
handling.
The `WorkerBusy` type was previously a single `Arc<AtomicBool>` shared
between the analyze and transcribe workers. This commit refactors this
to:
- Introduce `AnalyzeBusy` and `TranscribeBusy` newtype wrappers around
`WorkerBusy` to distinguish between the two flags. This allows
`axum::extract::State` to target them individually.
- Move the `BusyGuard` RAII guard into `src/lib.rs` and make it generic
to work with any `WorkerBusy` instance.
- Update the `main.rs`, `worker.rs`, and `routes/user_web.rs` files to
use the new types and guards, ensuring correct state management for
both pipelines.
- Enhance the transcription recovery scan
(`transcribe::recovery::scan_and_enqueue`) to iterate over user
directories and call `enqueue_pending_for_user` for each, making it
more robust and aligned with the per-user self-healing mechanism.
- Add a check for `transcribe_busy` in the `case_detail.html` template
to correctly display "Transkription läuft…" only when the transcribe
worker is actually active.
Add a new `failed` field to `RecordingView` and corresponding logic in
the worker.
When FFmpeg remuxing or Whisper transcription fails, the worker now
renames the
original `.m4a` file to `.m4a.failed`. This prevents the recovery scan
from
reprocessing these files and signals to the UI that transcription
failed.
The UI displays a message indicating failure and suggests renaming the
file
back to `.m4a` to retry. A new integration test
`worker_renames_audio_to_failed_on_whisper_error`
is included to verify this behavior.
Include per-user hotwords and initial prompts in the multipart form data
sent to the Whisper API. The language parameter is now dynamically set
based on user configuration, defaulting to "de" if not specified.
This commit introduces functionality to automatically generate a
one-liner summary from a completed transcription.
The `ensure_oneliner` function checks if `oneliner.txt` already exists
in the case directory. If not, it uses the Ollama API to generate a
summary from the full transcript. Any errors during generation or
writing are logged but do not halt the transcription process.
Introduce an `AppState` struct to hold shared application state,
including configuration and the transcription sender.
Modify `create_router` and `create_router_with_state` to use `AppState`.
Update `main.rs` to initialize the transcription worker and pass the
`AppState` to the router.
Implement the transcription worker logic in
`server/src/transcribe/worker.rs`, handling job queues, FFmpeg remuxing,
and Whisper transcription.
Update `server/src/auth.rs` to use `Arc<Config>` from the shared state.
Modify `server/src/routes/upload.rs` to enqueue transcription jobs and
handle potential errors.
Add `TranscribeJob` and channel types in `server/src/transcribe/mod.rs`.
Brummel