This commit removes the `whisper_variant` and `whisper_hotwords_variant`
fields from the `run_id` generation and the `print_meta_summary`
function.
These variants are no longer used as the project is shifting focus to
LLM-based generation. The `run_full_case.rs` example has also been
updated to reflect this change.
Config sheds its 14 Bucket-B fields (retention, whisper, ollama, llm) and
gains a sibling Arc<Settings> in AppState, loaded from settings.toml at
startup via Settings::load_or_default. Workers (transcribe, analyze,
recovery, auto_trigger) and routes (health, bulk, case_actions,
user_web) take settings as a separate parameter or State<> extractor;
Config::llm_configured moves to Settings::llm_configured.
TestConfig::build_pair() returns (Arc<Config>, Arc<Settings>); the new
create_router_with_settings / create_router_and_session_store_with_settings
helpers wire both into integration tests that exercise LLM/Whisper/Ollama.
The legacy single-Arc create_router falls back to Settings::default()
so the 17 non-LLM tests stay untouched.
Verified: cargo build clean, clippy --all-targets clean, 318 tests pass.
Introduces `OnelinerLocks` to serialize read-modify-write operations on
`oneliner.json`. This prevents race conditions between the manual
override API (`PUT /api/cases/{case_id}/oneliner`) and the transcription
worker's auto-regeneration process.
The manual override now acquires a lock specific to the `case_dir`
before writing. The transcription worker's `update_oneliner` function
also acquires the same lock around its post-LLM re-read-and-write
sequence. This ensures that a doctor's manual edit always takes
precedence, even if it arrives while an auto-regeneration is in
progress.
This change also includes:
- A new `routes::oneliner_override` module for the PUT handler.
- Validation for the oneliner text length and emptiness.
- Integration tests for the override functionality, covering happy path,
error conditions, early latching, race conditions, and parallel PUTs.
heal_orphans_if_idle called regenerate_missing_oneliners_for_user
inline on the request task, blocking the browser for up to N×60s
when orphans existed (observed with 16 cases after manual cleanup).
Now: CAS on new OnelinerHealBusy flag + tokio::spawn + BusyGuard
reset-on-drop. Handler returns immediately; the existing
OnelinerUpdated SSE event delivers results via the usual reload.
Startup recovery shares the flag so a concurrent page-load during
boot cannot fire a parallel regen loop against Ollama.
Regression test uses wiremock + timeout(150ms) + .expect(5) to
verify both non-blocking return and dedup of parallel invocations.
This commit introduces a new API endpoint for deleting recordings. The
endpoint
handles the removal of the audio file and its associated sidecar files
(transcript,
duration). It also invalidates derived artifacts like oneliner.json,
document.md,
and analysis_input.json. Additionally, it includes checks for path
traversal,
correct file extensions, and case ownership to ensure security and data
integrity.
A new `RecordingDeleted` event is also added to the `CaseEventKind`
enum.
Introduces a system for broadcasting real-time events to the web UI.
This enables features like automatic page reloads when case data changes
in the background, improving the user experience by keeping the UI
synchronized with the backend.
Key components:
- `events` module: Contains the `CaseEventKind` enum, `CaseEvent`
struct, and `EventSender` type for managing the broadcast channel.
- SSE endpoint (`/web/events`): Streams events to connected browsers.
- Client-side JavaScript: Listens for events and triggers debounced page
reloads.
- Integration points: Workers and route handlers now emit events when
relevant state changes occur.
This commit introduces a new magic link authentication flow.
The desktop client can now request a temporary, one-time-use token from
the server.
This token is then used to open a URL in the system browser, which
redirects to the server.
The server consumes the token, installs a regular web session, and
redirects the user
Remove the per-user oneliner watermark, which was previously used to
generate ETags for the `/api/oneliners` endpoint. The watermark was
intended to track the last successful oneliner write for each user.
The ETag generation has been refactored to use a deterministic FNV-1a
hash. This hash is computed over a sorted list of tuples containing
`(case_id, last_recording_at_ns, oneliner_mtime_ns)` for all visible
cases. This approach ensures that the ETag changes whenever any
listen-relevant file system modification occurs, such as a new
recording, oneliner regeneration, soft-delete, undo, or
upload-auto-reopen. This new method is more robust and avoids the
complexity of managing per-user state.
The `OnelinerWatermark` type alias and its associated logic have been
removed from `AppState` and the relevant modules (`transcribe::worker`,
`transcribe::recovery`, `routes::oneliners`, `main`). New integration
tests have been added to verify that various delete operations (single
delete, undo delete, bulk delete) correctly trigger an ETag update,
ensuring cache invalidation.
Introduce `last_recording_at` to `OnelinerEntry` and use it as the
primary sort key for cases. This ensures that cases with recent
dictation activity are prioritized, even if their initial creation date
is older.
The logic for determining a case's activity has been updated to consider
the most recent `.m4a` file's modification time (`last_recording_at`),
falling back to `updated_at` or `created_at` if necessary.
This change also refactors the timestamp formatting and handling within
the web interface to correctly display and sort cases based on their
actual last activity time, improving user experience and data relevance.
The `now_rfc3339` function is updated to strip sub-second precision for
consistent filename generation.
This commit introduces a new API endpoint `/api/oneliners` that allows
authenticated users to retrieve a list of their recent oneliners.
The endpoint supports conditional GET requests using the `ETag` header
for efficient caching. It also allows specifying a time window for the
oneliners via the `hours` query parameter, with sensible defaults and
clamping.
The implementation includes:
- A new route handler `handle_oneliners`.
- A new type `OnelinerWatermark` for tracking user-specific timestamps.
- Logic for scanning user data directories, filtering by time, and
handling deleted cases.
- Test cases to cover various scenarios, including caching, time
windowing, and edge cases.
Integrates the `spellbook` crate to allow for a Hunspell dictionary to
be used as a veto mechanism within the gazetteer.
This prevents common German words from being incorrectly rewritten to
vocabulary entries if they fall within the edit distance threshold. For
example, "Kaktus" will no longer be rewritten to "Lantus" if a Hunspell
dictionary is provided and contains "Kaktus".
The `hunspell_dict_path` configuration option is added, defaulting to
`/usr/share/hunspell/de_DE`. This path should point to the stem of the
Hunspell dictionary files (e.g., `/usr/share/hunspell/de_DE.aff` and
`/usr/share/hunspell/de_DE.dic`).
The gazetteer now supports an optional `DictChecker` trait, allowing for
pluggable dictionary implementations. `SpellbookDict` is the initial
implementation using the `spellbook` crate.
Includes new tests to verify the dict veto functionality, including
handling of case sensitivity and exact matches. An ignored test is added
for live verification against system-installed Hunspell dictionaries.
This commit integrates the Gazetteer into the transcription pipeline to
normalize terminology.
The Gazetteer, loaded at startup, is now passed to the transcription
worker. Before persisting the transcription text, it is processed
through the Gazetteer's `replace` method, ensuring standardized
terminology. This normalization is also applied to the generated
"oneliner" text.
Additionally, the `regenerate_missing_oneliners` function in the
recovery module has been updated to accept and utilize the Gazetteer,
ensuring that regenerated oneliners also benefit from terminology
normalization.
A new integration test, `transcribe_worker_normalizes_whisper_output`,
has been added to verify that the Whisper output is correctly normalized
by the Gazetteer before being saved to the transcript file.
Pass a `Gazetteer` to the analyze worker to enable proper-name
correction. The gazetteer is loaded from disk at application startup. If
the directory is missing or unreadable, the worker will start without
proper-name correction capabilities.
This commit introduces a recovery mechanism to regenerate `oneliner.txt`
files for cases where transcription was successful but the oneliner
generation failed due to a crash between writing the transcript and the
oneliner.
The `transcribe::recovery::regenerate_missing_oneliners` function is
added and called at startup after `scan_and_enqueue`. This function
identifies cases with non-empty transcripts but missing oneliners and
triggers their regeneration.
The `transcribe::worker::update_oneliner` function is refactored to
always regenerate the oneliner from all available transcripts in a case,
ensuring that later recordings can correct earlier ones. This replaces
the previous `ensure_oneliner` logic which only generated the oneliner
if it didn't exist.
New helper functions `cases_needing_oneliner` and
`all_transcripts_joined` are introduced to support these changes.
Comprehensive unit tests are included for the new recovery and file
processing logic.
The `WorkerBusy` type was previously a single `Arc<AtomicBool>` shared
between the analyze and transcribe workers. This commit refactors this
to:
- Introduce `AnalyzeBusy` and `TranscribeBusy` newtype wrappers around
`WorkerBusy` to distinguish between the two flags. This allows
`axum::extract::State` to target them individually.
- Move the `BusyGuard` RAII guard into `src/lib.rs` and make it generic
to work with any `WorkerBusy` instance.
- Update the `main.rs`, `worker.rs`, and `routes/user_web.rs` files to
use the new types and guards, ensuring correct state management for
both pipelines.
- Enhance the transcription recovery scan
(`transcribe::recovery::scan_and_enqueue`) to iterate over user
directories and call `enqueue_pending_for_user` for each, making it
more robust and aligned with the per-user self-healing mechanism.
- Add a check for `transcribe_busy` in the `case_detail.html` template
to correctly display "Transkription läuft…" only when the transcribe
worker is actually active.
The analyze recovery scan has been refactored to iterate through user
directories and enqueue pending analysis jobs more efficiently. The
`WorkerBusy` status has been introduced as an `Arc<AtomicBool>` to track
whether the analyze worker is currently processing a job.
This `WorkerBusy` flag is used in the `analyze::worker::run` function
and managed by a `BusyGuard` RAII struct, ensuring the flag is correctly
set and unset even in case of panics.
The `handle_my_cases` and `handle_case_detail` handlers in `user_web.rs`
now utilize the `WorkerBusy` flag to accurately display the "analyzing"
status and to trigger a self-heal of orphaned analysis inputs when the
worker is idle.
Additionally, the `WorkerBusy` type is now exported from
`server/src/lib.rs` to be accessible by other modules. The test cases
have been updated to include the `WorkerBusy` parameter when spawning
the analyze worker.
Allow overriding the LLM system prompt via the `LLM_SYSTEM_PROMPT`
environment variable. This provides flexibility for customizing LLM
behavior without code changes. A default prompt is used if the
environment variable is unset or empty.
Updates the `LLM_URL` in the project plan and `.env.example` to the
correct value.
Also, adds a warning in `main.rs` to inform the user if the LLM is not
configured, which effectively disables the 'Fall abschließen' feature.
Introduces a new channel for the analyze pipeline, including its sender
in the `AppState`.
Also updates the test router to create both transcribe and analyze
channels.
This commit introduces the foundation for web-based authentication and
user interface. It includes:
- **Session Management**: Securely handling user sessions using
cryptographically generated tokens, cookies with appropriate security
flags, and server-side storage.
- **Login/Logout Functionality**: Endpoints for users to log in with
their credentials and log out, clearing their session.
- **User Interface**: Basic templates for the login page, a list of
cases, and a detailed view of a single case, allowing users to
navigate and view their data.
- **Authorization**: An `AuthenticatedWebUser` extractor to ensure only
logged-in users can access protected web routes.
- **Configuration**: Added a `cookie_secure` option to the configuration
to control the `Secure` flag on session cookies, useful for local
development.
- **Dependencies**: Added necessary dependencies for password hashing,
time handling, and TOML file manipulation.
- **Helper Binary**: Included a `hash-password` binary to simplify
generating bcrypt hashes for user passwords.
This commit introduces a new module `transcribe::recovery` and a
function `scan_and_enqueue`.
This function is called at startup to find audio files that have not yet
been transcribed
and re-enqueues them for processing. This ensures that recordings are
not lost if the
server restarts before transcription is complete.
A new integration test `recovery_enqueues_only_pending_recordings` has
been added to
verify the functionality.
Introduce an `AppState` struct to hold shared application state,
including configuration and the transcription sender.
Modify `create_router` and `create_router_with_state` to use `AppState`.
Update `main.rs` to initialize the transcription worker and pass the
`AppState` to the router.
Implement the transcription worker logic in
`server/src/transcribe/worker.rs`, handling job queues, FFmpeg remuxing,
and Whisper transcription.
Update `server/src/auth.rs` to use `Arc<Config>` from the shared state.
Modify `server/src/routes/upload.rs` to enqueue transcription jobs and
handle potential errors.
Add `TranscribeJob` and channel types in `server/src/transcribe/mod.rs`.
Introduce a new health check endpoint at `/api/health`. This endpoint
returns a JSON response indicating the server's status and configuration
details, such as the port, data path, and configured URLs for external
services.
This change also refactors the application setup by:
- Creating a `lib.rs` file to house the `create_router` function, making
it
reusable for both the main application and integration tests.
- Moving configuration loading logic into `main.rs` and passing it as a
shared state (`Arc<Config>`) to the router.
- Adding a unit test (`health_test.rs`) to verify the functionality of
the
health check endpoint.
- Updating `Cargo.toml` and `Cargo.lock` to include the `tower`
dependency,
which is used for testing.
Introduces a new `config` module for loading application settings from
environment variables and a `users.toml` file. An `error` module is
added to handle application-specific errors gracefully. An
`.env.example` file is created to show available configuration options,
and `users.toml.example` provides a template for user data. The main
function now initializes the configuration and prints the server port.
Brummel