doctate/whisper/README.md

doctate-whisper

Thin FastAPI wrapper around faster-whisper.

Why a custom service?

Existing Whisper HTTP wrappers (ahmetoner/whisper-asr-webservice, speaches, linuxserver/faster-whisper, hwdsl2/docker-whisper) hardcode the faster-whisper defaults. Those defaults include:

• condition_on_previous_text=True — classic cascading-hallucination source
• temperature fallback cascade [0.0, 0.2, …, 1.0] — when confidence drops, Whisper gets "creative"

On real cardiology dictations we saw 2/13 runs with catastrophic hallucinations (Russian/Chinese fragments, repetition loops) using ahmetoner's defaults. None of the available wrappers expose these parameters as request fields or env vars.

This service fixes them to safe values that never change.

Anti-hallucination params (fixed)

Param	Value	Reason
temperature	0.0	No fallback cascade; deterministic output
condition_on_previous_text	False	Prevents cascading hallucinations
vad_filter	True	Silence → no transcription (not garbage)
vad_parameters.min_silence_duration_ms	500	VAD sensitivity
no_speech_threshold	0.6	Drop silent segments
log_prob_threshold	-1.0	Drop low-confidence segments
compression_ratio_threshold	2.4	Anti-repetition
beam_size / best_of	5 / 5	Default quality beam search

Endpoints

• POST /asr — multipart audio_file, query output=txt|json, language=de, form initial_prompt
• GET /health — liveness: {"status":"ok","model":...,"device":...}
• GET /info — debug: model, device, compute_type, offline flag

Compatible with ahmetoner's /asr interface so the Axum server needs no change.

Env vars

Var	Default	Purpose
WHISPER_MODEL	large-v3-turbo	Any faster-whisper model name
WHISPER_COMPUTE_TYPE	float16	float16, int8_float16, int8 (CPU)
WHISPER_DEVICE	cuda	cuda or cpu
WHISPER_MODELS_DIR	/models	Cache location
WHISPER_OFFLINE	0	1 → no HuggingFace download

Model choice

Default is large-v3-turbo: same 32-layer encoder as large-v3, only decoder distilled to 4 layers. OpenAI benchmarks: equal quality on European languages. ~1.6 GB VRAM vs. ~3 GB — leaves room for Ollama (gemma4, 9 GB) in a 12 GB card.

Switch to large-v3 at any time via WHISPER_MODEL=large-v3 + container restart.

Build

docker build -t doctate-whisper ./whisper

Takes 2–5 minutes. First run of the container pulls ~1.6 GB from HuggingFace into /models (volume-persisted).

Deploy to minerva via Dockge

Transport the image without a registry:

docker save doctate-whisper | ssh minerva 'docker load'

Then add the compose file in Dockge and start.

Integration with Axum server

In server/.env:

WHISPER_URL=http://minerva.lan:9001

No other change. The Axum server speaks ahmetoner's interface; we mirror it.

Offline mode

# Pre-pull on a host with internet:
docker run --rm -v /opt/stacks/doctate-whisper/models:/models doctate-whisper \
  python3 -c "from faster_whisper import WhisperModel; WhisperModel('large-v3-turbo', download_root='/models')"

# Then set in docker-compose.yml:
environment:
  - WHISPER_OFFLINE=1