Introduced a `test_default` method to `Config` to provide sane defaults
for integration tests. This refactors several test files to use this new
method, reducing boilerplate and improving maintainability.
Added `LLM_TIMEOUT_SECONDS` to the environment variables and `Config`
struct.
This commit introduces several improvements to the `hash-password`
binary:
- **New command-line interface:** Supports printing hashes directly,
hashing via prompts, or patching `web_password` in `users.toml` in
place.
- **Atomic file patching:** Utilizes `toml_edit` to preserve comments,
ordering, and formatting, and writes changes atomically via a
temporary file and rename operation.
- **Robust argument parsing:** Handles various command-line scenarios
and provides informative usage messages.
- **Error handling:** Improves error reporting for failed operations.
- **Added unit tests:** Includes tests for patching behavior and error
conditions.
This commit introduces the foundation for web-based authentication and
user interface. It includes:
- **Session Management**: Securely handling user sessions using
cryptographically generated tokens, cookies with appropriate security
flags, and server-side storage.
- **Login/Logout Functionality**: Endpoints for users to log in with
their credentials and log out, clearing their session.
- **User Interface**: Basic templates for the login page, a list of
cases, and a detailed view of a single case, allowing users to
navigate and view their data.
- **Authorization**: An `AuthenticatedWebUser` extractor to ensure only
logged-in users can access protected web routes.
- **Configuration**: Added a `cookie_secure` option to the configuration
to control the `Secure` flag on session cookies, useful for local
development.
- **Dependencies**: Added necessary dependencies for password hashing,
time handling, and TOML file manipulation.
- **Helper Binary**: Included a `hash-password` binary to simplify
generating bcrypt hashes for user passwords.
Add a new `failed` field to `RecordingView` and corresponding logic in
the worker.
When FFmpeg remuxing or Whisper transcription fails, the worker now
renames the
original `.m4a` file to `.m4a.failed`. This prevents the recovery scan
from
reprocessing these files and signals to the UI that transcription
failed.
The UI displays a message indicating failure and suggests renaming the
file
back to `.m4a` to retry. A new integration test
`worker_renames_audio_to_failed_on_whisper_error`
is included to verify this behavior.
This commit introduces the capability to configure user-specific Whisper
settings, including language, hotwords, and initial prompts. These
settings are stored in `users.toml` and are passed to the Whisper
service for more tailored transcription results.
New unit tests have been added to verify that the `transcribe` function
correctly forwards these optional settings to the Whisper client and
omits them when they are not provided.
Additionally, a new directory `tests/fixtures/dictations` has been
created to store M4A audio files, their expected transcripts, and
associated hotword files. This serves as a regression corpus for the
Whisper pipeline. A README file explains the structure and conventions
for adding new fixtures. Several new fixtures have been added, covering
various medical domains and transcription scenarios.
Include per-user hotwords and initial prompts in the multipart form data
sent to the Whisper API. The language parameter is now dynamically set
based on user configuration, defaulting to "de" if not specified.
The `TranscribeJob` struct now includes a `user_slug` field. This is
necessary for the transcription worker to look up per-user settings such
as hotwords and initial prompts. The `collect_pending` function has been
updated to return user slugs along with audio paths, and the recovery
scan now enqueues jobs with the correct user slug.
Introduce a new struct `WhisperUserSettings` to hold per-user
transcription settings. This allows users to customize Whisper
transcription for their specific needs directly in the `users.toml`
configuration file.
The changes include:
- Defining `WhisperUserSettings` with optional fields for `language`,
`hotwords`, and `initial_prompt`.
- Integrating `WhisperUserSettings` into the `User` struct, with
`#[serde(default)]` to ensure it defaults to
`WhisperUserSettings::default()` if not present in the TOML.
- Adding new unit tests to verify that users can be parsed correctly
with and without a `[user.whisper]` block.
- Updating existing tests in `auth_test.rs`, `upload_test.rs`, and
`web_test.rs` to include the new `whisper` field, ensuring backward
compatibility.
Update task status to 'done' for several items in the project plan. This
includes setting up the Axum server, filesystem structure,
configuration, logging, and upload endpoints. It also covers the Docker
setup for Axum and faster-whisper, as well as Ollama configuration.
Additionally, the plan now reflects a dedicated `whisper/` service for
faster-whisper, detailing its endpoints, model configuration, hotword
support, and deployment. Changes to the Ollama client configuration and
the oneliner generation logic are also documented.
The transcription queue and recovery mechanisms are marked as done,
along with the faster-whisper and Ollama health checks and retries.
Several aspects of the case management and UI development, including
askama templates and the detail view, have been updated.
Finally, the project plan includes a decision to use `large-v3-turbo`
for the Whisper model and details the implementation of a dedicated
`whisper/` service due to hallucination issues with previous wrappers.
The Ollama model is updated to `gemma4:latest`, and the GPU phase worker
logic is adjusted to leverage the smaller VRAM footprint of the
`large-v3-turbo` model. A new script `scripts/dictate.sh` is introduced
as a stand-in for testing the watch-flow without actual hardware.
This script provides a command-line interface for recording dictations,
uploading them to a server, and retrieving transcripts. It allows for
continuing an existing case, starting a new one, or refreshing the
status of the current case.
The script uses `ffmpeg` for audio recording and `curl` for server
communication. It manages the current case state using a temporary file
`/tmp/doctate-current-case`. Configuration details like `DATA_PATH` and
`API_KEY` are read from the server's `.env` and `users.toml` files.
Hotwords give per-request logit boosts to specified terms throughout
decoding (not just as prefix context like initial_prompt). Measured on
a cardiology dictation: 8/8 correct fachbegriffe vs. 3/8 without.
Off-the-shelf Whisper wrappers hardcode faster-whisper defaults that
cause cascading hallucinations on real dictations (2/13 runs against
ahmetoner v1.9.1). This thin FastAPI service mirrors the /asr interface
but fixes condition_on_previous_text=False, temperature=0.0, and
vad_filter=True so the Axum server can switch by changing WHISPER_URL.
Introduce an `oneliner` field to the `CaseView` struct and update the
`scan_cases` function to read it from `oneliner.txt`. The `cases.html`
template is updated to display the oneliner if present. A new
integration test is added to verify the functionality.
This commit introduces functionality to automatically generate a
one-liner summary from a completed transcription.
The `ensure_oneliner` function checks if `oneliner.txt` already exists
in the case directory. If not, it uses the Ollama API to generate a
summary from the full transcript. Any errors during generation or
writing are logged but do not halt the transcription process.
This commit adds several unit tests for the `generate_oneliner`
function, which interacts with the Ollama API. These tests cover:
- Successful parsing of a chat response.
- Normalization of the response content to remove extra whitespace and
quotes.
- Handling of HTTP 500 errors from the server.
- Verification of sent request parameters like `keep_alive` and `model`.
- Parsing errors when the `content` field is missing in the response.
Introduces a new module `ollama` to handle communication with an Ollama
LLM for generating concise, one-line summaries of medical transcripts.
This module includes:
- `generate_oneliner`: The main function to interact with the Ollama
API.
- `normalize`: A pure function for cleaning and formatting the LLM's
output according to specific rules.
- Error handling for API communication and response parsing.
- Unit tests for the normalization logic.
Introduces a dedicated function `validate_and_index_users` to
encapsulate the logic for validating user configurations, checking for
duplicate slugs and API keys. This function returns a `Result` to handle
errors gracefully, and its usage in `load_users` is updated to panic
with a more informative error message.
Additionally, this commit modifies the web routing to fetch and display
audio transcripts. When scanning recordings, it now attempts to read a
corresponding `.transcript.txt` file. If found, the transcript content
is included in the `RecordingView` and rendered in the `cases.html`
template. If the transcript file is not found, a "Transcription pending"
message is displayed.
The `transcribe` function in `whisper.rs` is updated to explicitly set
the language to German (`language=de`), which can improve transcription
accuracy by skipping language detection. The `encode=false` query
parameter has been removed, as the service is designed to handle encoded
audio.
Finally, tests have been added for the new `validate_and_index_users`
function to ensure duplicate slugs and API keys are correctly rejected.
Integration tests in `web_test.rs` have been updated to verify the
rendering of transcripts and the pending status.
This commit introduces a new module `transcribe::recovery` and a
function `scan_and_enqueue`.
This function is called at startup to find audio files that have not yet
been transcribed
and re-enqueues them for processing. This ensures that recordings are
not lost if the
server restarts before transcription is complete.
A new integration test `recovery_enqueues_only_pending_recordings` has
been added to
verify the functionality.
Introduce an `AppState` struct to hold shared application state,
including configuration and the transcription sender.
Modify `create_router` and `create_router_with_state` to use `AppState`.
Update `main.rs` to initialize the transcription worker and pass the
`AppState` to the router.
Implement the transcription worker logic in
`server/src/transcribe/worker.rs`, handling job queues, FFmpeg remuxing,
and Whisper transcription.
Update `server/src/auth.rs` to use `Arc<Config>` from the shared state.
Modify `server/src/routes/upload.rs` to enqueue transcription jobs and
handle potential errors.
Add `TranscribeJob` and channel types in `server/src/transcribe/mod.rs`.
Introduces a new module for interacting with the Whisper ASR service.
This includes
an async function to POST audio files and retrieve transcripts. Error
handling
for network and HTTP status codes is also implemented.
New tests are added to cover successful transcription, HTTP error
responses,
and request timeouts.
Introduces a new web interface to list cases and play back audio
recordings. This includes:
- A new `web` module in `routes` to handle web requests.
- `handle_case_list` to scan and display available cases and their
recordings.
- `handle_audio` to serve audio files, with validation to prevent path
traversal and ensure correct file types.
- `AppError::NotFound` variant to handle missing resources gracefully.
- Updates to `projektplan.md` to document the change in STT container
and m4a preprocessing.
- New tests in `server/tests/web_test.rs` to ensure the web interface
functions correctly.
This commit introduces a new API endpoint `/api/upload` for handling
audio file uploads. It supports multipart form data, extracts `case_id`,
`recorded_at`, and `audio` fields. The audio data is saved to a
filesystem path determined by the user and case ID, within either the
`open/` or `done/` directories.
New features include:
- Integration with `axum`'s `Multipart` extractor for parsing form data.
- Validation of `case_id` as a UUID.
- Creation of case directories if they do not exist.
- Handling of late uploads to already closed cases by placing files in
the `done/` directory and removing any `.remove` marker.
- Logging of received recordings.
- Definition of `AckResponse` and `AckStatus` models for API responses.
- Addition of comprehensive unit tests for various upload scenarios,
including new case creation, invalid inputs, authentication, and late
uploads.
Refactor configuration loading to separate user credentials from server
settings.
Update dependencies to their latest compatible versions to leverage new
features
and security patches.
Key changes include:
- Moving user credentials (`api_key`, `web_password`, `role`) from
`.env`
to a dedicated `users.toml` file for better manageability and
security.
- Introducing an `AuthenticatedUser` struct and extractor for API
key-based
authentication.
- Updating `axum` and related crates to leverage Rust edition 2024
improvements.
- Adjusting configuration values and tests to reflect these changes.
Introduce a new health check endpoint at `/api/health`. This endpoint
returns a JSON response indicating the server's status and configuration
details, such as the port, data path, and configured URLs for external
services.
This change also refactors the application setup by:
- Creating a `lib.rs` file to house the `create_router` function, making
it
reusable for both the main application and integration tests.
- Moving configuration loading logic into `main.rs` and passing it as a
shared state (`Arc<Config>`) to the router.
- Adding a unit test (`health_test.rs`) to verify the functionality of
the
health check endpoint.
- Updating `Cargo.toml` and `Cargo.lock` to include the `tower`
dependency,
which is used for testing.
Introduces a new `config` module for loading application settings from
environment variables and a `users.toml` file. An `error` module is
added to handle application-specific errors gracefully. An
`.env.example` file is created to show available configuration options,
and `users.toml.example` provides a template for user data. The main
function now initializes the configuration and prints the server port.
This commit introduces a .gitignore file to exclude build artifacts and
temporary files from version control.
Specifically, it ignores the 'target' directory (where Rust builds
output), new snapshot files from tests (`*.snap.new`), and the
`repomix-output.xml` file.
Brummel