# doctate-canary

Standalone FastAPI wrapper around [NVIDIA Canary](https://huggingface.co/nvidia/canary-1b-v2)
ASR models, served via the NeMo toolkit.

## Why a separate service?

This is **not** a drop-in replacement for `doctate-whisper`. The Axum server
keeps speaking the ahmetoner `/asr` interface to Whisper. This container exists
to evaluate Canary 1B v2 on real medical-German dictations side-by-side with
Whisper, before any decision about pipeline integration.

Inspired by `EvilFreelancer/docker-canary-serve` — written from scratch, MIT-friendly.

## API

`GET /health` → liveness JSON.

`GET /info` → model name, device, precision, models dir.

`POST /inference` (multipart/form-data):

| Field | Type | Default | Notes |
|---|---|---|---|
| `file` | upload | — | any audio format (mp3, m4a, opus, wav). ffmpeg transcodes to 16 kHz mono WAV internally. |
| `language` | text | `de` | one of `en`, `de`, `fr`, `es` |
| `pnc` | text | `yes` | punctuation/capitalization (`yes`/`no`) |
| `timestamps` | text | `no` | segment-level timestamps in `json` mode (`yes`/`no`) |
| `response_format` | text | `text` | `text` → plain transcript; `json` → `{text, language, duration, segments?}` |

## Env vars

| Var | Default | Purpose |
|---|---|---|
| `CANARY_MODEL` | `nvidia/canary-1b-v2` | any NeMo ASR model name |
| `CANARY_DEVICE` | `cuda` | `cuda` or `cpu` |
| `CANARY_PRECISION` | `bf16` | `bf16` / `fp16` / `fp32` |
| `CANARY_MODELS_DIR` | `/models` | HF cache (volume-persistent) |
| `HF_HOME` | (auto) | mirrored from `CANARY_MODELS_DIR` |

## VRAM

Canary 1B v2 in bf16 fits comfortably in <5 GB. On a 12 GB GPU it can coexist
with `doctate-whisper` (~3 GB) but **not** also with Ollama running a 9 GB
model. For evaluation runs, stop one of the others if VRAM gets tight.

## Build & deploy

```bash
cd doctate/canary
./deploy.sh minerva.lan
```

The script rsyncs the directory to `/opt/stacks/doctate-canary/`, rebuilds
the image on the remote, and polls `/health` for up to 10 min (first start
downloads ~3 GB and warms up the model).

## Smoke test

```bash
curl http://minerva.lan:9002/health

curl -F file=@dictation.m4a -F language=de -F response_format=text \
  http://minerva.lan:9002/inference

curl -F file=@dictation.m4a -F language=de -F timestamps=yes \
  -F response_format=json http://minerva.lan:9002/inference | jq .
```

## License

The wrapper code is project-internal. The Canary 1B v2 model weights are
distributed by NVIDIA under CC-BY-4.0 (commercial use permitted, attribution
required).