# doctate-whisper

Thin FastAPI wrapper around [faster-whisper](https://github.com/SYSTRAN/faster-whisper).

## Why a custom service?

Existing Whisper HTTP wrappers (`ahmetoner/whisper-asr-webservice`,
`speaches`, `linuxserver/faster-whisper`, `hwdsl2/docker-whisper`) hardcode
the faster-whisper defaults. Those defaults include:

- `condition_on_previous_text=True` — classic cascading-hallucination source
- temperature fallback cascade `[0.0, 0.2, …, 1.0]` — when confidence drops,
  Whisper gets "creative"

On real cardiology dictations we saw **2/13 runs with catastrophic
hallucinations** (Russian/Chinese fragments, repetition loops) using
ahmetoner's defaults. None of the available wrappers expose these
parameters as request fields or env vars.

This service fixes them to safe values that never change.

## Anti-hallucination params (fixed)

| Param | Value | Reason |
|---|---|---|
| `temperature` | `0.0` | No fallback cascade; deterministic output |
| `condition_on_previous_text` | `False` | Prevents cascading hallucinations |
| `vad_filter` | `True` | Silence → no transcription (not garbage) |
| `vad_parameters.min_silence_duration_ms` | `500` | VAD sensitivity |
| `no_speech_threshold` | `0.6` | Drop silent segments |
| `log_prob_threshold` | `-1.0` | Drop low-confidence segments |
| `compression_ratio_threshold` | `2.4` | Anti-repetition |
| `beam_size` / `best_of` | `5` / `5` | Default quality beam search |

## Endpoints

- `POST /asr` — multipart `audio_file`, query `output=txt|json`, `language=de`, form `initial_prompt`
- `GET /health` — liveness: `{"status":"ok","model":...,"device":...}`
- `GET /info` — debug: model, device, compute_type, offline flag

Compatible with ahmetoner's `/asr` interface so the Axum server needs no change.

## Env vars

| Var | Default | Purpose |
|---|---|---|
| `WHISPER_MODEL` | `large-v3-turbo` | Any faster-whisper model name |
| `WHISPER_COMPUTE_TYPE` | `float16` | `float16`, `int8_float16`, `int8` (CPU) |
| `WHISPER_DEVICE` | `cuda` | `cuda` or `cpu` |
| `WHISPER_MODELS_DIR` | `/models` | Cache location |
| `WHISPER_OFFLINE` | `0` | `1` → no HuggingFace download |

## Model choice

**Default is `large-v3-turbo`**: same 32-layer encoder as `large-v3`,
only decoder distilled to 4 layers. OpenAI benchmarks: equal quality on
European languages. ~1.6 GB VRAM vs. ~3 GB — leaves room for Ollama
(gemma4, 9 GB) in a 12 GB card.

Switch to `large-v3` at any time via `WHISPER_MODEL=large-v3` + container restart.

## Build

```bash
docker build -t doctate-whisper ./whisper
```

Takes 2–5 minutes. First run of the container pulls ~1.6 GB from
HuggingFace into `/models` (volume-persisted).

## Deploy to minerva via Dockge

Transport the image without a registry:

```bash
docker save doctate-whisper | ssh minerva 'docker load'
```

Then add the compose file in Dockge and start.

## Integration with Axum server

In `server/.env`:

```
WHISPER_URL=http://minerva.lan:9001
```

No other change. The Axum server speaks ahmetoner's interface; we mirror it.

## Offline mode

```bash
# Pre-pull on a host with internet:
docker run --rm -v /opt/stacks/doctate-whisper/models:/models doctate-whisper \
  python3 -c "from faster_whisper import WhisperModel; WhisperModel('large-v3-turbo', download_root='/models')"

# Then set in docker-compose.yml:
environment:
  - WHISPER_OFFLINE=1
```