This commit introduces the `doctate-canary` service and associated sweep
experiments.
The service provides access to the `nvidia/canary-1b-v2` ASR model via a
native FastAPI API. It includes deployment scripts, Docker
configurations, and detailed documentation on installation, API usage,
and environment variables.
The sweep experiments aim to thoroughly evaluate the Canary model's
performance under various configurations. This includes testing
different inference pipelines (single-shot vs. buffered), decoding
strategies (greedy vs. beam search), and parameter tuning (chunk length,
overlap, batch size, precision). The goal is to reproduce previous
findings and identify optimal settings.
The commit also includes:
- Utility scripts for audio transcoding and manipulation.
- Comprehensive logging and result collection mechanisms for the sweep
runs.
- Detailed analysis of `dur=0` occurrences and word confidence,
concluding they are not reliable indicators of hallucination.
- Documentation on the interaction between decoding parameters,
especially `return_hypotheses`, and the availability of confidence
scores.
Pre-pulling the base Docker image in the deploy script prevents
potential TLS certificate errors during the `docker build` process. This
ensures a smoother deployment by leveraging the cached image layer.
This commit introduces a new standalone FastAPI service for the NVIDIA
Canary ASR model.
Key changes include:
- A new `canary/` directory containing the service code.
- `Dockerfile`: Defines the Docker image for the Canary service.
- `docker-compose.yml`: Configures the Docker Compose setup for running
the service.
- `main.py`: Implements the FastAPI application, model loading, and
inference logic.
- `requirements.txt`: Lists the Python dependencies for the service.
- `CHUNKING.md`: Documents the long-form audio handling strategy for
Canary.
- `README.md`: Provides an overview of the service, API, and deployment
instructions.
- `deploy.sh`: A script for deploying the service to a remote host.
This service allows for independent evaluation and deployment of the
Canary ASR model, separate from the existing `doctate-whisper` service.
It utilizes a buffered inference approach for handling long audio files,
as detailed in `CHUNKING.md`.
Brummel