feat: containerize doctate-server for minerva deployment

Multi-stage Dockerfile, host-network compose, idempotent deploy script. Configs and persistent data bind-mounted from /opt/stacks/doctate-server. deploy-server.sh handles bootstrap + re-deploy in one command with auto-prune (keep 5 last SHA-tagged images). See docs/deployment.md for bootstrap, rollback, and TLS-proxy notes.
2026-05-04 11:14:31 +02:00
parent 88e469d0a1
commit cf80bc7e97
5 changed files with 407 additions and 0 deletions
@@ -0,0 +1,18 @@
+# Whitelist build-context: only `server/` and `common/` go into the image.
+# Cargo's path dep `server/Cargo.toml` -> `path = "../common"` requires
+# both directories side by side, so we must include `common/` explicitly.
+*
+!server/
+!common/
+
+# Re-exclude artifacts inside the whitelisted dirs.
+server/target/
+common/target/
+**/*.log
+**/*.bk
+
+# Secrets and operator-owned configs MUST NEVER end up in the image —
+# they are bind-mounted at runtime from /opt/stacks/doctate-server/config/.
+server/.env
+server/users.toml
+server/settings.toml
@@ -0,0 +1,47 @@
+# Compose file for the doctate-server container on minerva.
+# Operator config + persistent data live under /opt/stacks/doctate-server/
+# (Dockge convention). The image itself is self-contained except at the
+# bind-mount points listed under `volumes:`.
+#
+# `network_mode: host` means the container binds directly on minerva's
+# network namespace. Settings.toml URLs like http://minerva.lan:9001
+# resolve identically inside and outside the container — no Docker DNS
+# magic, no port mapping.
+
+services:
+  doctate-server:
+    image: doctate-server:latest
+    container_name: doctate-server
+    network_mode: host
+    restart: unless-stopped
+    working_dir: /app
+    environment:
+      # Required — server panics on boot if missing (see config.rs).
+      USERS_FILE: /app/users.toml
+      SERVER_PORT: "3000"
+      DATA_PATH: /data
+      # Optional — explicit values to avoid silent defaults.
+      SETTINGS_FILE: /app/settings.toml
+      LOG_PATH: /var/log/recorder
+      LOG_LEVEL: info
+      LOG_MAX_DAYS: "90"
+      VOCAB_DIR: /app/vocab
+      HUNSPELL_DICT: /usr/share/hunspell/de_DE.dic
+      SESSION_TIMEOUT_HOURS: "8"
+      # false because no TLS reverse-proxy is in front of this service yet.
+      # Browsers refuse Secure cookies on plain HTTP — set to true once a
+      # TLS proxy (e.g. nginx on the Unraid box) terminates in front of
+      # this container. Override via /opt/stacks/doctate-server/config/.env.
+      COOKIE_SECURE: "false"
+      ASR_BACKEND: whisper
+    volumes:
+      # Operator-owned configs — read-only, atomically edited on host.
+      - /opt/stacks/doctate-server/config/.env:/app/.env:ro
+      - /opt/stacks/doctate-server/config/settings.toml:/app/settings.toml:ro
+      - /opt/stacks/doctate-server/config/users.toml:/app/users.toml:ro
+      # Persistent data — container writes here.
+      - /opt/stacks/doctate-server/data:/data
+      - /opt/stacks/doctate-server/logs:/var/log/recorder
+      # Optional vocab override — uncomment if a curated list is maintained
+      # on minerva that should win over the bundled /app/vocab/*.txt.
+      # - /opt/stacks/doctate-server/config/vocab:/app/vocab:ro
@@ -0,0 +1,155 @@
+# Deployment
+
+The doctate-server runs in two flavors that can coexist:
+
+- **Local dev** — `cargo run -p doctate-server` (or `scripts/run-server.sh`),
+  reads `server/.env` + `server/settings.toml` + `server/users.toml` from
+  the repo working tree, writes to `tmpdata/`.
+- **Production on minerva** — Docker container, reads operator-owned
+  configs from `/opt/stacks/doctate-server/config/`, writes to
+  `/opt/stacks/doctate-server/data/`.
+
+Both have independent configs, independent data directories, and
+listen on different hosts — no interference.
+
+## Architecture
+
+```mermaid
+graph LR
+    Dev[Dev laptop<br/>scripts/deploy-server.sh] -->|ssh + scp| Host
+
+    subgraph Host[minerva.lan]
+        Stack["/opt/stacks/doctate-server/"]
+        Container["doctate-server container<br/>network_mode: host<br/>:3000"]
+
+        Stack -->|bind-mount :ro| Configs[("config/.env<br/>config/settings.toml<br/>config/users.toml")]
+        Stack -->|bind-mount :rw| Data[("data/<br/>logs/")]
+
+        Configs -.->|read on boot| Container
+        Container -.->|writes recordings + logs| Data
+    end
+
+    Host -->|http :3000| nginx[nginx on Unraid<br/>TLS termination]
+    nginx -->|https| Browser
+```
+
+## First deploy
+
+The script handles bootstrap idempotently. The first run will fail at
+phase 3 (placeholder guard) so you can fill in the configs.
+
+1. **Run the script** from the dev laptop:
+
+   ```bash
+   ./scripts/deploy-server.sh
+   ```
+
+   Phases 1–2 create `/opt/stacks/doctate-server/{config,data,logs}` and
+   seed the three config files from `server/*.example`. Phase 3 then
+   aborts with placeholder errors.
+
+2. **Edit configs on minerva** (via `nano` over SSH or through Dockge).
+   Required fields:
+
+   | File | Edit |
+   |---|---|
+   | `config/users.toml` | At least one `[[user]]` block with a real `api_key` and a real bcrypt hash for `web_password` |
+   | `config/settings.toml` | `[whisper] url`, `[ollama] url` for the production hosts |
+   | `config/.env` | Optionally `IONOS_API_KEY=…` if any LLM backend uses Ionos |
+
+   Generate the bcrypt hash inside the container — the image is already
+   loaded on minerva after phase 5 of run 1:
+
+   ```bash
+   ssh minerva.lan 'docker run --rm -i doctate-server:latest hash-password'
+   ```
+
+   Copy the resulting `$2b$12$…` line into `users.toml`.
+
+3. **Re-run the script**:
+
+   ```bash
+   ./scripts/deploy-server.sh
+   ```
+
+   This time it runs all phases, ending with a green health-check and a
+   rollback hint.
+
+## Re-deploy
+
+Once the configs are in place, every subsequent deploy is one command:
+
+```bash
+./scripts/deploy-server.sh
+```
+
+The script is idempotent. Phase 1 (`mkdir -p`) and phase 2 (template
+seeding) are no-ops on a populated host. Phase 3 (placeholder guard)
+passes. Phases 4–9 build, transfer, restart, and verify health.
+
+The image is tagged twice on minerva: as `doctate-server:latest` and
+as `doctate-server:<git-short-sha>`. The script keeps the **last 5
+SHA-tagged builds** automatically (phase 9) and prunes older ones plus
+their dangling layers, so disk usage stays bounded even after many
+deploys. Adjust `KEEP=5` near the bottom of `scripts/deploy-server.sh`
+if you want a different rollback window.
+
+## User management
+
+Adding or rotating a user means editing `users.toml` on minerva and
+restarting the container so the boot-time parser picks up the change:
+
+```bash
+ssh minerva.lan
+docker run --rm -i doctate-server:latest hash-password   # silent prompt, prints hash
+nano /opt/stacks/doctate-server/config/users.toml        # paste the hash
+docker compose -f /opt/stacks/doctate-server/compose.yaml restart doctate-server
+```
+
+The `users.toml` mount is read-only by design — the container is not
+allowed to modify the auth file. Edits happen on the host.
+
+## Rollback
+
+Each deploy leaves a `doctate-server:<sha>` tag behind on minerva, so
+rolling back to a previous build is one retag plus a restart:
+
+```bash
+ssh minerva.lan
+docker images doctate-server                              # list tags + dates
+docker tag doctate-server:<previous-sha> doctate-server:latest
+cd /opt/stacks/doctate-server && docker compose up -d --force-recreate
+```
+
+Persistent data (`/opt/stacks/doctate-server/data/`) is unaffected by
+rollback — only the binary swaps.
+
+## Troubleshooting
+
+| Symptom | First check |
+|---|---|
+| Container restarts in a loop | `ssh minerva.lan 'docker logs --tail 80 doctate-server'` — most boot panics name the missing env-var |
+| `/api/health` doesn't respond | `ssh minerva.lan 'ss -tlnp \| grep :3000'` — is the server actually bound? |
+| Login refused on HTTPS | `COOKIE_SECURE` and TLS-proxy mismatch — see "TLS" section below |
+| File missing inside container | `docker compose -f /opt/stacks/doctate-server/compose.yaml exec doctate-server ls /app` |
+| Health-check status | `ssh minerva.lan 'docker inspect --format "{{.State.Health.Status}}" doctate-server'` |
+| Config edit not picked up | `docker compose ... restart doctate-server` — most config is read on boot, not per request |
+| Stale image tag on minerva | `docker images --filter dangling=true` — retag or `docker image prune` |
+
+## TLS / nginx reverse proxy
+
+The container ships with `COOKIE_SECURE=false` because the default
+deployment is plain HTTP between minerva:3000 and whatever sits in
+front of it. When nginx on the Unraid box terminates TLS and forwards
+to `http://minerva.lan:3000`, set the cookie flag accordingly so
+browsers actually accept the session cookie:
+
+```bash
+ssh minerva.lan
+echo 'COOKIE_SECURE=true' >> /opt/stacks/doctate-server/config/.env
+docker compose -f /opt/stacks/doctate-server/compose.yaml restart doctate-server
+```
+
+Without this override under a TLS proxy, the login flow looks like it
+works (`POST /api/login` returns 200) but the next request lacks the
+cookie and bounces back to the login page.
@@ -0,0 +1,124 @@
+#!/usr/bin/env bash
+# Deploy doctate-server to a remote Docker host (default: minerva.lan).
+# Idempotent: re-runs preserve operator-owned config, data, logs.
+#
+# Usage: ./scripts/deploy-server.sh [host]
+#   host  default: minerva.lan
+#
+# See docs/deployment.md for first-time bootstrap.
+
+set -euo pipefail
+
+HOST="${1:-minerva.lan}"
+REMOTE="/opt/stacks/doctate-server"
+REPO_ROOT="$(cd "$(dirname "$0")/.." && pwd)"
+GIT_SHA="$(cd "$REPO_ROOT" && git rev-parse --short HEAD)"
+IMAGE_LATEST="doctate-server:latest"
+IMAGE_TAGGED="doctate-server:$GIT_SHA"
+
+cd "$REPO_ROOT"
+
+# ─── Phase 0 ─── local pre-flight
+echo ">>> [0/10] cargo fmt --check (server)"
+( cd server && cargo fmt --check )
+
+if ! git -C "$REPO_ROOT" diff --quiet || ! git -C "$REPO_ROOT" diff --cached --quiet; then
+    echo "    WARNING: working tree is dirty — building from on-disk state, not from $GIT_SHA" >&2
+fi
+
+# ─── Phase 1 ─── remote bootstrap (idempotent)
+echo ">>> [1/10] Ensuring remote layout at $HOST:$REMOTE"
+ssh "$HOST" "mkdir -p $REMOTE/config $REMOTE/data $REMOTE/logs"
+
+# ─── Phase 2 ─── seed templates only if missing
+echo ">>> [2/10] Seeding config templates (only if missing)"
+for pair in \
+    ".env.example:.env" \
+    "settings.toml.example:settings.toml" \
+    "users.toml.example:users.toml"
+do
+    src="server/${pair%%:*}"
+    dst="${pair##*:}"
+    remote_path="$REMOTE/config/$dst"
+    if ssh "$HOST" "test -f $remote_path"; then
+        echo "    KEEP    $remote_path"
+    else
+        echo "    SEED    $remote_path  <- $src"
+        scp -q "$src" "$HOST:$remote_path"
+    fi
+done
+
+# ─── Phase 3 ─── placeholder guard
+echo ">>> [3/10] Verifying configs have been edited from defaults"
+if ssh "$HOST" "grep -qE '\\\$2b\\\$12\\\$\\.\\.\\.|change-me-to-a-secure-key' $REMOTE/config/users.toml" 2>/dev/null; then
+    cat >&2 <<EOF
+
+ERROR: $REMOTE/config/users.toml still has placeholder values.
+
+Edit it on $HOST (e.g. via nano or Dockge) and replace:
+  - '\$2b\$12\$...'             with a real bcrypt hash
+  - 'change-me-to-a-secure-key' with a real api_key
+
+For the bcrypt hash, the image is already loaded on $HOST (after the
+first run reaches phase 5); generate one with:
+  docker run --rm -i $IMAGE_LATEST hash-password
+
+Then re-run this script. See docs/deployment.md "First deploy" section.
+EOF
+    exit 1
+fi
+
+# ─── Phase 4 ─── build
+echo ">>> [4/10] Building $IMAGE_TAGGED + $IMAGE_LATEST"
+DOCKER_BUILDKIT=1 docker build \
+    -f server/Dockerfile \
+    -t "$IMAGE_TAGGED" \
+    -t "$IMAGE_LATEST" \
+    .
+
+# ─── Phase 5 ─── stream image to host
+echo ">>> [5/10] Transferring image to $HOST (gzip-streamed via ssh)"
+docker save "$IMAGE_TAGGED" "$IMAGE_LATEST" | gzip | ssh "$HOST" 'gunzip | docker load'
+
+# ─── Phase 6 ─── compose file sync
+echo ">>> [6/10] Syncing compose file"
+scp -q deploy/doctate-server.compose.yml "$HOST:$REMOTE/compose.yaml"
+
+# ─── Phase 7 ─── restart
+echo ">>> [7/10] (Re)starting container"
+ssh "$HOST" "cd $REMOTE && docker compose up -d --force-recreate"
+
+# ─── Phase 8 ─── health poll
+echo ">>> [8/10] Waiting for /api/health"
+for i in $(seq 1 20); do
+    if curl -fsS "http://$HOST:3000/api/health" >/dev/null 2>&1; then
+        echo "    OK after ${i} attempt(s)"
+        # ─── Phase 9 ─── prune old image tags (keep last KEEP + :latest)
+        # Containers hold images via internal IDs, not tags, so removing
+        # an old <sha>-tag never affects the running container as long as
+        # :latest still points to it.
+        KEEP=5
+        echo ">>> [9/10] Pruning old image tags on $HOST (keep last $KEEP)"
+        ssh "$HOST" "
+            set -e
+            docker images doctate-server --format '{{.CreatedAt}}\t{{.Tag}}' | \
+                grep -v -P '\\tlatest\$' | \
+                sort -r | \
+                awk 'NR > $KEEP { print \$NF }' | \
+                xargs -I{} -r docker rmi doctate-server:{} 2>/dev/null || true
+            docker image prune -f >/dev/null 2>&1 || true
+        "
+        # ─── Phase 10 ─── final echo
+        echo ">>> [10/10] Deployed $GIT_SHA"
+        echo
+        echo "Rollback:"
+        echo "  ssh $HOST 'docker tag doctate-server:<prev-sha> $IMAGE_LATEST && cd $REMOTE && docker compose up -d --force-recreate'"
+        exit 0
+    fi
+    sleep 3
+done
+
+# health timeout
+echo ">>> TIMEOUT (60s) — last 50 lines of container logs:" >&2
+ssh "$HOST" "docker logs --tail 50 doctate-server" >&2 || true
+exit 1
@@ -0,0 +1,63 @@
+# syntax=docker/dockerfile:1.7
+# Multi-stage build for doctate-server. Stage 1 compiles the Rust binary;
+# stage 2 ships only the binary + runtime tools (ffmpeg, hunspell dict).
+
+# -----------------------------------------------------------------------
+# Stage 1: builder
+# -----------------------------------------------------------------------
+FROM rust:1.94-slim-bookworm AS builder
+
+# pkg-config + ca-certificates suffice — reqwest uses rustls-tls
+# (no libssl-dev needed), and our other crates don't probe native libs.
+RUN apt-get update && apt-get install -y --no-install-recommends \
+        pkg-config ca-certificates && \
+    rm -rf /var/lib/apt/lists/*
+
+WORKDIR /build
+# Copy `common/` AND `server/` so the path dependency resolves
+# (server/Cargo.toml references `../common`).
+COPY common/ ./common/
+COPY server/ ./server/
+
+WORKDIR /build/server
+# BuildKit cache mounts: registry/ and target/ persist between builds
+# but are NOT part of the resulting image layers. Output binaries
+# therefore have to be copied to /out (outside the cache mount) so
+# stage 2 can pick them up via COPY --from=builder.
+RUN --mount=type=cache,target=/usr/local/cargo/registry \
+    --mount=type=cache,target=/build/server/target \
+    cargo build --release --bin doctate-server --bin hash-password && \
+    mkdir -p /out && \
+    cp target/release/doctate-server /out/ && \
+    cp target/release/hash-password /out/
+
+# -----------------------------------------------------------------------
+# Stage 2: runtime
+# -----------------------------------------------------------------------
+FROM debian:bookworm-slim AS runtime
+
+# ffmpeg/ffprobe are invoked as subprocesses by server/src/transcribe/ffmpeg.rs.
+# hunspell-de-de installs the dict files /usr/share/hunspell/de_DE.{dic,aff}
+# which the `spellbook` Rust library reads via the HUNSPELL_DICT env var.
+# tzdata gives us proper local-time formatting in logs.
+# curl is required by HEALTHCHECK below.
+RUN apt-get update && apt-get install -y --no-install-recommends \
+        ffmpeg hunspell-de-de ca-certificates curl tzdata && \
+    rm -rf /var/lib/apt/lists/*
+
+WORKDIR /app
+
+COPY --from=builder /out/doctate-server /usr/local/bin/
+COPY --from=builder /out/hash-password  /usr/local/bin/
+
+# Default vocab ships with the image; operator can override via bind-mount.
+COPY server/vocab/ /app/vocab/
+
+EXPOSE 3000
+
+# Container-level health probe. SERVER_PORT defaults to 3000 if unset
+# (matches the value in server/.env.example).
+HEALTHCHECK --interval=30s --timeout=5s --start-period=10s --retries=3 \
+    CMD curl -fsS "http://127.0.0.1:${SERVER_PORT:-3000}/api/health" || exit 1
+
+CMD ["doctate-server"]