Brummel
1de5cf3891
Pre-pulling the base Docker image in the deploy script prevents potential TLS certificate errors during the `docker build` process. This ensures a smoother deployment by leveraging the cached image layer.
doctate-canary
Standalone FastAPI wrapper around NVIDIA Canary ASR models, served via the NeMo toolkit.
Why a separate service?
This is not a drop-in replacement for doctate-whisper. The Axum server
keeps speaking the ahmetoner /asr interface to Whisper. This container exists
to evaluate Canary 1B v2 on real medical-German dictations side-by-side with
Whisper, before any decision about pipeline integration.
Inspired by EvilFreelancer/docker-canary-serve — written from scratch, MIT-friendly.
API
GET /health → liveness JSON.
GET /info → model name, device, precision, models dir.
POST /inference (multipart/form-data):
| Field | Type | Default | Notes |
|---|---|---|---|
file |
upload | — | any audio format (mp3, m4a, opus, wav). ffmpeg transcodes to 16 kHz mono WAV internally. |
language |
text | de |
one of en, de, fr, es |
pnc |
text | yes |
punctuation/capitalization (yes/no) |
timestamps |
text | no |
segment-level timestamps in json mode (yes/no) |
response_format |
text | text |
text → plain transcript; json → {text, language, duration, segments?} |
Env vars
| Var | Default | Purpose |
|---|---|---|
CANARY_MODEL |
nvidia/canary-1b-v2 |
any NeMo ASR model name |
CANARY_DEVICE |
cuda |
cuda or cpu |
CANARY_PRECISION |
bf16 |
bf16 / fp16 / fp32 |
CANARY_MODELS_DIR |
/models |
HF cache (volume-persistent) |
HF_HOME |
(auto) | mirrored from CANARY_MODELS_DIR |
VRAM
Canary 1B v2 in bf16 fits comfortably in <5 GB. On a 12 GB GPU it can coexist
with doctate-whisper (~3 GB) but not also with Ollama running a 9 GB
model. For evaluation runs, stop one of the others if VRAM gets tight.
Build & deploy
cd doctate/canary
./deploy.sh minerva.lan
The script rsyncs the directory to /opt/stacks/doctate-canary/, rebuilds
the image on the remote, and polls /health for up to 10 min (first start
downloads ~3 GB and warms up the model).
Smoke test
curl http://minerva.lan:9002/health
curl -F file=@dictation.m4a -F language=de -F response_format=text \
http://minerva.lan:9002/inference
curl -F file=@dictation.m4a -F language=de -F timestamps=yes \
-F response_format=json http://minerva.lan:9002/inference | jq .
License
The wrapper code is project-internal. The Canary 1B v2 model weights are distributed by NVIDIA under CC-BY-4.0 (commercial use permitted, attribution required).