Files
DocSite/docs/plans/0001-walking-skeleton.md
T
Brummel 27e09a395b plan: 0001 walking-skeleton
The bite-sized implementation plan for the walking skeleton (parent spec
docs/specs/0001-walking-skeleton.md): nine TDD-structured tasks — scaffold,
config, gitea client, render, theme, cache, check, server, and main wiring —
building the live-render server for aura + data-server docs from Gitea main.
src/lib.rs registers modules incrementally so the lib compiles after every
task; main stays a stub until the final wiring task.

refs #1
2026-06-28 14:48:38 +02:00

1118 lines
37 KiB
Markdown

# Walking Skeleton — Implementation Plan
> **Parent spec:** `docs/specs/0001-walking-skeleton.md`
>
> **For agentic workers:** REQUIRED SUB-SKILL: use the `implement` skill to
> run this plan. Steps use `- [ ]` checkboxes for tracking.
**Goal:** A `docsite serve` HTTP server plus a `docsite check` subcommand
that render aura + data-server docs live from Gitea `main`, dark-themed,
navigated from a DocSite-owned whitelist registry, with `code` + `table`
directives.
**Architecture:** Single crate with a lib (`src/lib.rs`, modules below) and
a thin bin (`src/main.rs`, subcommand dispatch). Source is fetched read-only
through a `Fetcher` trait — `GiteaClient` (blocking reqwest) in production,
`MockFetcher` in tests — so tests run offline. Render is comrak (GFM) +
syntect; the themed shell wraps it; an in-memory `(repo,path,sha)` cache
keys on the source revision. `docsite check` and a boot-time pass validate
every nav reference against the live sources.
**Tech Stack:** Rust 2021, axum + tokio (server), reqwest blocking (Gitea),
comrak + syntect (render), serde + toml (config). lib+bin split; theme.css
embedded via `include_str!`.
**Derived layout decisions (plan-recon open questions):** (1) lib+bin split
— required for `tests/` integration tests; (2) theme.css embedded via
`include_str!` — self-contained binary; (3) dependency versions via
`cargo add` — current compatible versions, not pinned by guess.
**Module registration (ordering contract):** `src/lib.rs` starts with NO
module declarations. Each component task below FIRST appends its own
`pub mod <name>;` line to `src/lib.rs`, so the library compiles after every
task — a per-task `cargo test --lib <name>::` gate would otherwise fail on a
lib that declares a not-yet-created module. `src/main.rs` stays a standalone
dispatch stub until Task 9 wires it to `server::run` / `check::run_cli`. Task
order respects the dependency graph: config, gitea, render, theme, cache
(independent or config-only), then check (config+gitea), then server (all).
---
### Task 1: Scaffold — dependencies + lib/bin split
**Files:**
- Modify: `Cargo.toml:7-9` (populate `[dependencies]`)
- Create: `src/lib.rs`
- Modify: `src/main.rs:9-11` (subcommand dispatch stub)
- [ ] **Step 1: Add dependencies**
Run:
```
cargo add axum tokio --features tokio/rt-multi-thread,tokio/macros,tokio/net
cargo add reqwest --no-default-features --features blocking,rustls-tls
cargo add comrak syntect serde --features serde/derive
cargo add toml
```
Expected: each prints `Adding <crate> v<x> to dependencies`; `cargo` resolves
without error. (LAN Gitea is plain HTTP, but reqwest still needs a TLS
backend to build; rustls avoids a system-openssl dependency.)
- [ ] **Step 2: Create the lib crate root**
Create `src/lib.rs` (no module declarations yet — each component task appends
its own `pub mod` line, per the ordering contract above):
```rust
//! DocSite library crate — the render/serve/check machinery behind the
//! `docsite` binary. Split out as a lib so the integration tests under
//! `tests/` can drive it directly with an injected `Fetcher`. Module
//! declarations are appended by the tasks that create each module, so the
//! lib compiles after every task.
```
- [ ] **Step 3: Replace the main stub with subcommand dispatch**
Replace `src/main.rs` entirely:
```rust
//! `docsite` — a single-source-of-truth LAN documentation server.
//! Subcommands: `serve` (run the HTTP server) and `check` (validate every
//! configured nav reference against the live Gitea sources, exit-coded).
use std::process::ExitCode;
fn main() -> ExitCode {
let args: Vec<String> = std::env::args().collect();
match args.get(1).map(String::as_str) {
// server::run / check::run_cli are wired in Task 9, once those modules
// exist; until then the dispatch shape compiles as a standalone stub.
Some("serve") => { eprintln!("docsite: serve not yet wired"); ExitCode::from(70) }
Some("check") => { eprintln!("docsite: check not yet wired"); ExitCode::from(70) }
_ => {
eprintln!("usage: docsite <serve|check>");
ExitCode::from(2)
}
}
}
```
- [ ] **Step 4: Build gate**
Run: `cargo build`
Expected: PASS — the scaffold compiles: the lib has no modules yet and main's
dispatch is a standalone stub referencing nothing unbuilt. The real
serve/check wiring lands in Task 9.
---
### Task 2: config — registry model, parse, lookup, whitelist
**Files:**
- Create: `src/config/mod.rs`
- Create: `src/config/registry.rs`
- Test: in-module `#[cfg(test)]` in `src/config/mod.rs`
- [ ] **Step 1: Write the serde model**
Create `src/config/registry.rs`:
```rust
//! The DocSite-owned `registry.toml` model: the publish whitelist + nav.
//! NO `deny_unknown_fields` — unknown keys are tolerated so the schema can
//! grow (forward-compatible).
use serde::Deserialize;
#[derive(Debug, Deserialize)]
pub struct Registry {
pub bind: String,
pub gitea: String,
#[serde(default)]
pub project: Vec<Project>,
}
#[derive(Debug, Deserialize)]
pub struct Project {
pub slug: String,
pub repo: String,
pub branch: String,
pub docs: String,
#[serde(default)]
pub section: Vec<Section>,
}
#[derive(Debug, Deserialize)]
pub struct Section {
pub title: String,
pub pages: Vec<String>,
}
/// A nav page resolved to its source location: the project plus the page
/// path relative to the project's `docs` root.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct PageRef {
pub slug: String,
pub rel: String,
}
```
- [ ] **Step 2: Write the failing test for load + lookup + whitelist**
Create `src/config/mod.rs`:
```rust
pub mod registry;
pub use registry::{PageRef, Project, Registry, Section};
use std::path::Path;
/// Parse a `registry.toml` from disk.
pub fn load(path: impl AsRef<Path>) -> Result<Registry, String> {
let text = std::fs::read_to_string(path).map_err(|e| format!("registry: {e}"))?;
toml::from_str(&text).map_err(|e| format!("registry parse: {e}"))
}
impl Registry {
pub fn project(&self, slug: &str) -> Option<&Project> {
self.project.iter().find(|p| p.slug == slug)
}
/// Resolve a served url path (`<slug>/<page-no-ext>`) to a `PageRef`,
/// IFF that page is listed in some section of the project (whitelist).
/// `url_page` is the path after the slug, without a trailing `.md`
/// (e.g. `design/INDEX`). A `..` component is refused (traversal).
pub fn resolve(&self, slug: &str, url_page: &str) -> Option<PageRef> {
if url_page.split('/').any(|c| c == ".." || c.is_empty()) {
return None;
}
let proj = self.project(slug)?;
let want = format!("{url_page}.md");
let listed = proj
.section
.iter()
.flat_map(|s| &s.pages)
.any(|p| p == &want);
listed.then(|| PageRef { slug: slug.to_string(), rel: want })
}
}
#[cfg(test)]
mod tests {
use super::*;
fn sample() -> Registry {
toml::from_str(
r#"
bind = "0.0.0.0:8080"
gitea = "http://gitea.test"
[[project]]
slug = "aura"
repo = "Brummel/aura"
branch = "main"
docs = "docs"
[[project.section]]
title = "Design"
pages = ["design/INDEX.md"]
"#,
)
.expect("sample registry parses")
}
#[test]
fn resolves_a_listed_page() {
let r = sample();
assert_eq!(
r.resolve("aura", "design/INDEX"),
Some(PageRef { slug: "aura".into(), rel: "design/INDEX.md".into() })
);
}
#[test]
fn refuses_an_unlisted_page() {
// glossary is not in any section -> not reachable (whitelist)
assert_eq!(sample().resolve("aura", "glossary"), None);
}
#[test]
fn refuses_traversal() {
assert_eq!(sample().resolve("aura", "../secret"), None);
}
#[test]
fn refuses_unknown_slug() {
assert_eq!(sample().resolve("nope", "design/INDEX"), None);
}
#[test]
fn tolerates_unknown_keys() {
// a future key must not break parsing (forward-compatible)
let r: Result<Registry, _> =
toml::from_str("bind=\"x\"\ngitea=\"y\"\nfuture_key=\"z\"\n");
assert!(r.is_ok(), "unknown top-level key must be tolerated");
}
}
```
- [ ] **Step 3: Run the tests to verify they pass**
Run: `cargo test --lib config::`
Expected: PASS — 5 tests in `config::tests`.
---
### Task 3: gitea — Fetcher trait, GiteaError, client + mock
**Files:**
- Create: `src/gitea/mod.rs`
- Create: `src/gitea/client.rs`
- Test: in-module `#[cfg(test)]` in `src/gitea/mod.rs`
- [ ] **Step 1: Write the trait, error, and mock**
Create `src/gitea/mod.rs`:
```rust
pub mod client;
pub use client::GiteaClient;
use std::collections::HashMap;
/// A read-only source fetcher. Injected so tests run offline. SYNC by design
/// — the production impl uses blocking reqwest; axum handlers call it inside
/// `spawn_blocking`.
pub trait Fetcher: Send + Sync {
/// The current commit SHA of `branch` in `repo` (the cache key source).
fn branch_sha(&self, repo: &str, branch: &str) -> Result<String, GiteaError>;
/// The raw bytes of `path` at `branch` in `repo`.
fn fetch_raw(&self, repo: &str, branch: &str, path: &str) -> Result<String, GiteaError>;
}
/// `NotFound` is the broken-reference signal — distinct from a transport
/// failure, so a renamed source and an unreachable Gitea render differently.
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum GiteaError {
NotFound { repo: String, path: String },
Transport(String),
}
/// In-memory fetcher for tests: `(repo, path)` -> body, plus a branch SHA.
pub struct MockFetcher {
pub sha: String,
pub files: HashMap<(String, String), String>,
}
impl MockFetcher {
pub fn new(sha: &str) -> Self {
Self { sha: sha.to_string(), files: HashMap::new() }
}
pub fn with(mut self, repo: &str, path: &str, body: &str) -> Self {
self.files.insert((repo.to_string(), path.to_string()), body.to_string());
self
}
}
impl Fetcher for MockFetcher {
fn branch_sha(&self, _repo: &str, _branch: &str) -> Result<String, GiteaError> {
Ok(self.sha.clone())
}
fn fetch_raw(&self, repo: &str, _branch: &str, path: &str) -> Result<String, GiteaError> {
self.files
.get(&(repo.to_string(), path.to_string()))
.cloned()
.ok_or_else(|| GiteaError::NotFound { repo: repo.to_string(), path: path.to_string() })
}
}
#[cfg(test)]
mod tests {
use super::*;
fn mock() -> MockFetcher {
MockFetcher::new("a517899").with("Brummel/aura", "docs/glossary.md", "# Glossary\n")
}
#[test]
fn fetch_hit_returns_body() {
assert_eq!(mock().fetch_raw("Brummel/aura", "main", "docs/glossary.md").unwrap(), "# Glossary\n");
}
#[test]
fn fetch_miss_is_notfound() {
let err = mock().fetch_raw("Brummel/aura", "main", "docs/gone.md").unwrap_err();
assert!(matches!(err, GiteaError::NotFound { .. }), "miss must be NotFound, got {err:?}");
}
#[test]
fn branch_sha_returns_pinned() {
assert_eq!(mock().branch_sha("Brummel/aura", "main").unwrap(), "a517899");
}
}
```
- [ ] **Step 2: Run mock tests**
Run: `cargo test --lib gitea::`
Expected: PASS — 3 tests.
- [ ] **Step 3: Write the production GiteaClient**
Create `src/gitea/client.rs`:
```rust
//! Read-only Gitea source client (blocking reqwest). Resolves a branch SHA
//! via the API and fetches raw file bytes via the raw endpoint. Token from
//! `$DOCSITE_GITEA_TOKEN` for private repos.
use super::{Fetcher, GiteaError};
pub struct GiteaClient {
base: String,
token: Option<String>,
http: reqwest::blocking::Client,
}
impl GiteaClient {
pub fn new(base: &str) -> Self {
Self {
base: base.trim_end_matches('/').to_string(),
token: std::env::var("DOCSITE_GITEA_TOKEN").ok(),
http: reqwest::blocking::Client::new(),
}
}
fn get(&self, url: &str) -> Result<reqwest::blocking::Response, GiteaError> {
let mut req = self.http.get(url);
if let Some(t) = &self.token {
req = req.header("Authorization", format!("token {t}"));
}
req.send().map_err(|e| GiteaError::Transport(e.to_string()))
}
}
impl Fetcher for GiteaClient {
fn branch_sha(&self, repo: &str, branch: &str) -> Result<String, GiteaError> {
// GET {base}/api/v1/repos/{repo}/branches/{branch} -> { "commit": { "id": "<sha>" } }
let url = format!("{}/api/v1/repos/{repo}/branches/{branch}", self.base);
let resp = self.get(&url)?;
if resp.status() == reqwest::StatusCode::NOT_FOUND {
return Err(GiteaError::NotFound { repo: repo.into(), path: format!("branch:{branch}") });
}
let body = resp.text().map_err(|e| GiteaError::Transport(e.to_string()))?;
// minimal extraction without a json dep on the model: the API returns
// {"name":...,"commit":{"id":"<sha>",...}}
let key = "\"id\":\"";
let start = body.find(key).ok_or_else(|| GiteaError::Transport("no commit id".into()))? + key.len();
let end = body[start..].find('"').ok_or_else(|| GiteaError::Transport("unterminated id".into()))?;
Ok(body[start..start + end].to_string())
}
fn fetch_raw(&self, repo: &str, branch: &str, path: &str) -> Result<String, GiteaError> {
// GET {base}/{repo}/raw/branch/{branch}/{path}
let url = format!("{}/{repo}/raw/branch/{branch}/{path}", self.base);
let resp = self.get(&url)?;
if resp.status() == reqwest::StatusCode::NOT_FOUND {
return Err(GiteaError::NotFound { repo: repo.into(), path: path.into() });
}
if !resp.status().is_success() {
return Err(GiteaError::Transport(format!("status {}", resp.status())));
}
resp.text().map_err(|e| GiteaError::Transport(e.to_string()))
}
}
```
- [ ] **Step 4: Build gate**
Run: `cargo build`
Expected: PASS — the client compiles (no test asserts the live endpoints;
those are exercised by the gated live smoke in Task 9).
---
### Task 4: render — comrak (GFM) + syntect
**Files:**
- Create: `src/render/mod.rs`
- Test: in-module `#[cfg(test)]`
- [ ] **Step 1: Write the failing render test**
Create `src/render/mod.rs`:
```rust
//! Markdown -> HTML render: comrak (GitHub-flavored, tables on) with syntect
//! highlighting of code fences (dark theme). `render_doc` is pure. A no-op
//! directive-expansion seam is reserved here (invariant 4) for `code-include`
//! in a later iteration; `code` and `table` are native Markdown.
use comrak::plugins::syntect::SyntectAdapter;
use comrak::{markdown_to_html_with_plugins, Options, Plugins};
/// Render GFM markdown to an HTML fragment (no <html>/<body> wrapper).
pub fn render_doc(markdown: &str) -> String {
let mut options = Options::default();
options.extension.table = true;
options.extension.strikethrough = true;
options.extension.autolink = true;
let adapter = SyntectAdapter::new(Some("base16-ocean.dark"));
let mut plugins = Plugins::default();
plugins.render.codefence_syntax_highlighter = Some(&adapter);
markdown_to_html_with_plugins(markdown, &options, &plugins)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn renders_a_table() {
let html = render_doc("| a | b |\n|---|---|\n| 1 | 2 |\n");
assert!(html.contains("<table>"), "GFM table not rendered: {html}");
assert!(html.contains("<td>1</td>"), "table cell missing: {html}");
}
#[test]
fn highlights_a_code_fence() {
let html = render_doc("```rust\nfn main() {}\n```\n");
// syntect emits inline-styled spans inside a <pre>
assert!(html.contains("<pre"), "no <pre> for the fence: {html}");
assert!(html.contains("style=\"color"), "syntect did not style the fence: {html}");
}
#[test]
fn unknown_language_falls_back_without_panic() {
let html = render_doc("```nosuchlang\nplain text\n```\n");
assert!(html.contains("<pre"), "unknown-language fence still needs a <pre>: {html}");
}
}
```
- [ ] **Step 2: Run render tests**
Run: `cargo test --lib render::`
Expected: PASS — 3 tests. (If the syntect adapter constructor signature
differs in the resolved comrak version, adjust the `SyntectAdapter::new`
call to the resolved API; the assertion contract — `<table>`, styled `<pre>`,
no panic on unknown lang — is fixed.)
---
### Task 5: theme — shell + embedded stylesheet + page_html
**Files:**
- Create: `src/theme/mod.rs`
- Create: `assets/theme.css`
- Test: in-module `#[cfg(test)]`
- [ ] **Step 1: Write the dark stylesheet**
Create `assets/theme.css`:
```css
:root {
--bg: #16161a; --fg: #cdd6f4; --dim: #6c7086; --link: #89b4fa;
--panel: #1e1e2e; --border: #313244; --accent: #f5e0dc;
}
html, body { margin: 0; background: var(--bg); color: var(--fg);
font-family: system-ui, sans-serif; line-height: 1.6; }
header.crumb { padding: 10px 16px; border-bottom: 1px solid var(--border);
font-size: 13px; color: var(--dim); }
header.crumb a { color: var(--link); text-decoration: none; }
nav { padding: 10px 16px; border-bottom: 1px solid var(--border); font-size: 14px; }
nav .sec { color: var(--dim); margin: 8px 0 2px; text-transform: uppercase; font-size: 12px; }
nav a { display: block; color: var(--link); text-decoration: none; padding: 1px 0; }
main.doc { max-width: 860px; margin: 0 auto; padding: 24px 16px 64px; }
main.doc code, main.doc pre { font-family: ui-monospace, monospace; }
main.doc pre { background: var(--panel); border: 1px solid var(--border);
border-radius: 6px; padding: 12px; overflow-x: auto; }
main.doc table { border-collapse: collapse; margin: 12px 0; }
main.doc th, main.doc td { border: 1px solid var(--border); padding: 6px 10px; }
main.doc a { color: var(--link); }
.docsite-error { background: #3a1e26; border: 1px solid #f38ba8; color: #f38ba8;
border-radius: 6px; padding: 12px; margin: 12px 0; font-family: ui-monospace, monospace; }
```
- [ ] **Step 2: Write the shell + nav builder with a failing test**
Create `src/theme/mod.rs`:
```rust
//! The themed HTML shell: header breadcrumb, nav built from the registry,
//! and the rendered doc body. The stylesheet is embedded (served at
//! /assets/theme.css by the server module) so the binary is self-contained.
use crate::config::{Project, Registry};
pub const THEME_CSS: &str = include_str!("../../assets/theme.css");
/// Build the nav `<nav>` block for a project from its registry sections.
fn nav_html(proj: &Project) -> String {
let mut s = String::from("<nav>");
for sec in &proj.section {
s.push_str(&format!("<div class=\"sec\">{}</div>", html_escape(&sec.title)));
for page in &sec.pages {
let url = page.strip_suffix(".md").unwrap_or(page);
s.push_str(&format!(
"<a href=\"/{}/{}\">{}</a>",
proj.slug, url, html_escape(url)
));
}
}
s.push_str("</nav>");
s
}
/// Wrap a rendered doc body in the full themed page for `slug`/`title`.
pub fn page_html(reg: &Registry, slug: &str, title: &str, body: &str) -> String {
let nav = reg.project(slug).map(nav_html).unwrap_or_default();
format!(
"<!doctype html>\n<html lang=\"en\">\n<head><meta charset=\"utf-8\">\
<link rel=\"stylesheet\" href=\"/assets/theme.css\"><title>{t}</title></head>\n<body>\n\
<header class=\"crumb\"><a href=\"/\">docsite</a> / <a href=\"/{s}/\">{s}</a> / {t}</header>\n\
{nav}\n<main class=\"doc\">{body}</main>\n</body>\n</html>\n",
t = html_escape(title), s = html_escape(slug),
)
}
fn html_escape(s: &str) -> String {
s.replace('&', "&amp;").replace('<', "&lt;").replace('>', "&gt;")
}
#[cfg(test)]
mod tests {
use super::*;
fn reg() -> Registry {
toml::from_str(
r#"
bind="x"
gitea="y"
[[project]]
slug="aura"
repo="Brummel/aura"
branch="main"
docs="docs"
[[project.section]]
title="Design"
pages=["design/INDEX.md"]
"#,
).unwrap()
}
#[test]
fn page_has_shell_nav_and_body() {
let html = page_html(&reg(), "aura", "Design Ledger", "<p>hi</p>");
assert!(html.starts_with("<!doctype html>"), "no doctype");
assert!(html.contains("href=\"/assets/theme.css\""), "stylesheet link missing");
assert!(html.contains("class=\"crumb\""), "breadcrumb missing");
assert!(html.contains("<a href=\"/aura/design/INDEX\">"), "nav link from registry missing");
assert!(html.contains("<main class=\"doc\"><p>hi</p></main>"), "body not wrapped");
}
#[test]
fn theme_css_is_embedded() {
assert!(THEME_CSS.contains("--bg: #16161a"), "embedded css missing");
}
}
```
- [ ] **Step 3: Run theme tests**
Run: `cargo test --lib theme::`
Expected: PASS — 2 tests.
---
### Task 6: cache — revision-keyed render cache
**Files:**
- Create: `src/cache.rs`
- Test: in-module `#[cfg(test)]`
- [ ] **Step 1: Write the cache with a failing test**
Create `src/cache.rs`:
```rust
//! Revision-keyed render cache: (repo, path, sha) -> rendered HTML. A new
//! sha is a new key, so a changed source re-renders; an unchanged sha serves
//! byte-identical output (invariant 2). Thread-safe for the axum handlers.
use std::collections::HashMap;
use std::sync::Mutex;
#[derive(Default)]
pub struct Cache {
map: Mutex<HashMap<(String, String, String), String>>,
}
impl Cache {
pub fn new() -> Self {
Self::default()
}
/// Return the cached render, or compute it with `f`, store, and return it.
pub fn get_or_render<F: FnOnce() -> String>(
&self, repo: &str, path: &str, sha: &str, f: F,
) -> String {
let key = (repo.to_string(), path.to_string(), sha.to_string());
if let Some(hit) = self.map.lock().unwrap().get(&key) {
return hit.clone();
}
let html = f();
self.map.lock().unwrap().insert(key, html.clone());
html
}
}
#[cfg(test)]
mod tests {
use super::*;
use std::cell::Cell;
#[test]
fn same_sha_renders_once_and_is_byte_identical() {
let cache = Cache::new();
let calls = Cell::new(0);
let render = || { calls.set(calls.get() + 1); "<p>x</p>".to_string() };
let a = cache.get_or_render("r", "p", "sha1", render);
let b = cache.get_or_render("r", "p", "sha1", render);
assert_eq!(a, b, "same sha must be byte-identical");
assert_eq!(calls.get(), 1, "same sha must render exactly once");
}
#[test]
fn changed_sha_re_renders() {
let cache = Cache::new();
let calls = Cell::new(0);
let render = || { calls.set(calls.get() + 1); "x".to_string() };
cache.get_or_render("r", "p", "sha1", render);
cache.get_or_render("r", "p", "sha2", render);
assert_eq!(calls.get(), 2, "a new sha must re-render");
}
}
```
- [ ] **Step 2: Run cache tests**
Run: `cargo test --lib cache::`
Expected: PASS — 2 tests.
---
### Task 7: check — reference validator + exit code
**Files:**
- Create: `src/check.rs`
- Test: in-module `#[cfg(test)]`
- [ ] **Step 1: Write check_all + report with a failing test**
Create `src/check.rs`:
```rust
//! Reference-integrity validator (invariant 3): for every nav page in the
//! registry, confirm the source exists at the project's branch. Drives
//! `docsite check` (exit-coded) and the boot-time validation pass. Detects
//! and reports; never writes to a source.
use crate::config::Registry;
use crate::gitea::{Fetcher, GiteaError};
use std::process::ExitCode;
pub struct RefResult {
pub slug: String,
pub path: String,
pub ok: bool,
pub detail: String,
}
pub struct CheckReport {
pub results: Vec<RefResult>,
}
impl CheckReport {
pub fn broken(&self) -> usize {
self.results.iter().filter(|r| !r.ok).count()
}
}
/// Validate every nav reference in `reg` against `fetcher`.
pub fn check_all(reg: &Registry, fetcher: &dyn Fetcher) -> CheckReport {
let mut results = Vec::new();
for proj in &reg.project {
let sha = fetcher.branch_sha(&proj.repo, &proj.branch);
for page in proj.section.iter().flat_map(|s| &s.pages) {
let full = if proj.docs == "." {
page.clone()
} else {
format!("{}/{page}", proj.docs.trim_end_matches('/'))
};
let (ok, detail) = match &sha {
Err(e) => (false, format!("branch error: {e:?}")),
Ok(sha) => match fetcher.fetch_raw(&proj.repo, &proj.branch, &full) {
Ok(_) => (true, format!("ok (main @ {})", &sha[..sha.len().min(7)])),
Err(GiteaError::NotFound { .. }) => {
(false, format!("BROKEN — not found at {} {}:{}", proj.repo, proj.branch, full))
}
Err(GiteaError::Transport(t)) => (false, format!("transport error: {t}")),
},
};
results.push(RefResult { slug: proj.slug.clone(), path: page.clone(), ok, detail });
}
}
CheckReport { results }
}
/// `docsite check`: load the registry, validate against live Gitea, print a
/// report, exit 0 iff nothing is broken.
pub fn run_cli(registry_path: &str) -> ExitCode {
let reg = match crate::config::load(registry_path) {
Ok(r) => r,
Err(e) => { eprintln!("docsite: {e}"); return ExitCode::from(2); }
};
let fetcher = crate::gitea::GiteaClient::new(&reg.gitea);
let report = check_all(&reg, &fetcher);
for r in &report.results {
println!("{}/{:<28} {}", r.slug, r.path, r.detail);
}
let broken = report.broken();
println!("{} references ok, {} broken", report.results.len() - broken, broken);
if broken == 0 { ExitCode::SUCCESS } else { ExitCode::from(1) }
}
#[cfg(test)]
mod tests {
use super::*;
use crate::gitea::MockFetcher;
fn reg() -> Registry {
toml::from_str(
r#"
bind="x"
gitea="y"
[[project]]
slug="aura"
repo="Brummel/aura"
branch="main"
docs="docs"
[[project.section]]
title="Design"
pages=["design/INDEX.md","glossary.md"]
"#,
).unwrap()
}
#[test]
fn all_ok_reports_zero_broken() {
let f = MockFetcher::new("a517899")
.with("Brummel/aura", "docs/design/INDEX.md", "x")
.with("Brummel/aura", "docs/glossary.md", "y");
assert_eq!(check_all(&reg(), &f).broken(), 0);
}
#[test]
fn missing_source_is_broken() {
// only one of the two pages exists -> one broken
let f = MockFetcher::new("a517899").with("Brummel/aura", "docs/design/INDEX.md", "x");
let report = check_all(&reg(), &f);
assert_eq!(report.broken(), 1, "the missing glossary must be broken");
assert!(report.results.iter().any(|r| !r.ok && r.detail.contains("BROKEN")));
}
}
```
- [ ] **Step 2: Run check tests**
Run: `cargo test --lib check::`
Expected: PASS — 2 tests.
---
### Task 8: server — axum router, handlers, whitelist, error block
**Files:**
- Create: `src/server/mod.rs`
- Test: `tests/e2e.rs` (integration, injected fetcher)
- [ ] **Step 1: Write the server with a testable assembly seam**
Create `src/server/mod.rs`:
```rust
//! axum router. Routes: `GET /` (project index), `GET /:slug/*page`
//! (rendered doc), `GET /assets/theme.css`. The fetcher and registry live in
//! shared state; the page handler enforces the whitelist (config::resolve)
//! and renders a visible error block on a NotFound (fail-loud, invariant 3).
use crate::cache::Cache;
use crate::config::Registry;
use crate::gitea::{Fetcher, GiteaError};
use axum::{
extract::{Path, State},
http::StatusCode,
response::{Html, IntoResponse, Response},
routing::get,
Router,
};
use std::sync::Arc;
#[derive(Clone)]
pub struct AppState {
pub reg: Arc<Registry>,
pub fetcher: Arc<dyn Fetcher>,
pub cache: Arc<Cache>,
}
/// Build the router for a given state — the test seam (no network, no bind).
pub fn router(state: AppState) -> Router {
Router::new()
.route("/", get(index))
.route("/assets/theme.css", get(theme_css))
.route("/:slug/*page", get(page))
.with_state(state)
}
async fn theme_css() -> impl IntoResponse {
([(axum::http::header::CONTENT_TYPE, "text/css")], crate::theme::THEME_CSS)
}
async fn index(State(st): State<AppState>) -> Html<String> {
let mut s = String::from("<!doctype html><link rel=\"stylesheet\" href=\"/assets/theme.css\">\
<header class=\"crumb\">docsite</header><main class=\"doc\"><ul>");
for p in &st.reg.project {
s.push_str(&format!("<li><a href=\"/{0}/\">{0}</a></li>", p.slug));
}
s.push_str("</ul></main>");
Html(s)
}
async fn page(State(st): State<AppState>, Path((slug, page)): Path<(String, String)>) -> Response {
let url_page = page.trim_end_matches('/');
let Some(pref) = st.reg.resolve(&slug, url_page) else {
return (StatusCode::NOT_FOUND, "404 — not a published page").into_response();
};
let proj = st.reg.project(&pref.slug).expect("resolved slug exists");
let full = if proj.docs == "." {
pref.rel.clone()
} else {
format!("{}/{}", proj.docs.trim_end_matches('/'), pref.rel)
};
let title = url_page.rsplit('/').next().unwrap_or(url_page).to_string();
// fetch SHA + raw inside spawn_blocking (the Fetcher is sync)
let f = st.fetcher.clone();
let (repo, branch) = (proj.repo.clone(), proj.branch.clone());
let full2 = full.clone();
let fetched = tokio::task::spawn_blocking(move || {
let sha = f.branch_sha(&repo, &branch)?;
let body = f.fetch_raw(&repo, &branch, &full2)?;
Ok::<_, GiteaError>((sha, body))
})
.await
.expect("join");
match fetched {
Ok((sha, md)) => {
let html = st.cache.get_or_render(&proj.repo, &full, &sha, || {
let body = crate::render::render_doc(&md);
crate::theme::page_html(&st.reg, &pref.slug, &title, &body)
});
Html(html).into_response()
}
Err(GiteaError::NotFound { repo, path }) => {
// fail-loud: a visible error block, 200 (the broken ref is content)
let block = format!(
"<div class=\"docsite-error\">broken reference: {repo} {}:{path} not found</div>",
proj.branch
);
let html = crate::theme::page_html(&st.reg, &pref.slug, &title, &block);
Html(html).into_response()
}
Err(GiteaError::Transport(t)) => {
(StatusCode::BAD_GATEWAY, format!("502 — source unreachable: {t}")).into_response()
}
}
}
/// Bind and serve (the production entry from `main`).
pub fn run(registry_path: &str) -> std::process::ExitCode {
let reg = match crate::config::load(registry_path) {
Ok(r) => r,
Err(e) => { eprintln!("docsite: {e}"); return std::process::ExitCode::from(2); }
};
let fetcher = Arc::new(crate::gitea::GiteaClient::new(&reg.gitea));
let report = check_all_boot(&reg, fetcher.as_ref());
eprintln!("docsite: serving {} projects on {}", reg.project.len(), reg.bind);
eprintln!("docsite: {} nav references, {} broken at boot", report.results.len(), report.broken());
let bind = reg.bind.clone();
let state = AppState { reg: Arc::new(reg), fetcher, cache: Arc::new(Cache::new()) };
let rt = tokio::runtime::Runtime::new().expect("tokio runtime");
rt.block_on(async move {
let listener = tokio::net::TcpListener::bind(&bind).await.expect("bind");
axum::serve(listener, router(state)).await.expect("serve");
});
std::process::ExitCode::SUCCESS
}
fn check_all_boot(reg: &Registry, f: &dyn Fetcher) -> crate::check::CheckReport {
crate::check::check_all(reg, f)
}
```
- [ ] **Step 2: Write the e2e integration test (injected fetcher)**
Create `tests/e2e.rs`:
```rust
//! End-to-end: drive the router with a MockFetcher (offline). Asserts the
//! themed render, the whitelist 404, the traversal refusal, and the
//! fail-loud broken-reference block.
use axum::body::to_bytes;
use axum::http::{Request, StatusCode};
use docsite::cache::Cache;
use docsite::config::Registry;
use docsite::gitea::MockFetcher;
use docsite::server::{router, AppState};
use std::sync::Arc;
use tower::ServiceExt; // oneshot
fn reg() -> Registry {
toml::from_str(
r#"
bind="0.0.0.0:0"
gitea="http://gitea.test"
[[project]]
slug="aura"
repo="Brummel/aura"
branch="main"
docs="docs"
[[project.section]]
title="Design"
pages=["design/INDEX.md"]
"#,
).unwrap()
}
fn state() -> AppState {
let fetcher = MockFetcher::new("a517899").with(
"Brummel/aura", "docs/design/INDEX.md",
"# Ledger\n\n| a | b |\n|---|---|\n| 1 | 2 |\n\n```rust\nfn x() {}\n```\n",
);
AppState {
reg: Arc::new(reg()),
fetcher: Arc::new(fetcher),
cache: Arc::new(Cache::new()),
}
}
async fn get(app: axum::Router, uri: &str) -> (StatusCode, String) {
let resp = app.oneshot(Request::builder().uri(uri).body(axum::body::Body::empty()).unwrap())
.await.unwrap();
let status = resp.status();
let bytes = to_bytes(resp.into_body(), usize::MAX).await.unwrap();
(status, String::from_utf8_lossy(&bytes).to_string())
}
#[tokio::test]
async fn renders_listed_page_with_theme_and_nav() {
let (status, body) = get(router(state()), "/aura/design/INDEX").await;
assert_eq!(status, StatusCode::OK);
assert!(body.starts_with("<!doctype html>"), "no themed shell");
assert!(body.contains("href=\"/assets/theme.css\""), "no stylesheet");
assert!(body.contains("<a href=\"/aura/design/INDEX\">"), "no nav from registry");
assert!(body.contains("<table>"), "table not rendered");
assert!(body.contains("<pre"), "code fence not rendered");
}
#[tokio::test]
async fn unlisted_page_is_404() {
// glossary is not whitelisted in any section
let (status, _) = get(router(state()), "/aura/glossary").await;
assert_eq!(status, StatusCode::NOT_FOUND);
}
#[tokio::test]
async fn traversal_is_refused() {
let (status, _) = get(router(state()), "/aura/../secret").await;
assert_eq!(status, StatusCode::NOT_FOUND);
}
#[tokio::test]
async fn broken_reference_shows_failloud_block() {
// a registry listing a page the fetcher does not have -> visible error block
let mut s = state();
let r: Registry = toml::from_str(
"bind=\"x\"\ngitea=\"y\"\n[[project]]\nslug=\"aura\"\nrepo=\"Brummel/aura\"\nbranch=\"main\"\ndocs=\"docs\"\n[[project.section]]\ntitle=\"D\"\npages=[\"gone.md\"]\n",
).unwrap();
s.reg = Arc::new(r);
let (status, body) = get(router(s), "/aura/gone").await;
assert_eq!(status, StatusCode::OK, "broken ref is content, not a server fault");
assert!(body.contains("docsite-error"), "no fail-loud error block: {body}");
assert!(body.contains("broken reference"), "error block text missing");
}
```
- [ ] **Step 3: Add the test-only dev-dependency**
Run: `cargo add --dev tower --features util`
Expected: `tower` dev-dep added (for `tower::ServiceExt::oneshot`). The
`#[tokio::test]` macro is already covered by the regular `tokio` dependency
(it carries `macros` + `rt-multi-thread`).
- [ ] **Step 4: Run the e2e suite**
Run: `cargo test --test e2e`
Expected: PASS — 4 tests (render, 404, traversal, fail-loud block).
---
### Task 9: main wiring + gated live smoke
**Files:**
- Modify: `src/main.rs` (replace the stub arms with the real entries)
- Test: `tests/live_smoke.rs`
- [ ] **Step 1: Wire main.rs to the real entries**
Replace the two stub arms in `src/main.rs`'s match with:
```rust
Some("serve") => docsite::server::run("registry.toml"),
Some("check") => docsite::check::run_cli("registry.toml"),
```
Both modules now exist (Task 8 created `server`, Task 7 created `check`), and
both return `std::process::ExitCode`, matching `main`'s return type.
- [ ] **Step 2: Full build + whole suite gate**
Run: `cargo build && cargo test`
Expected: PASS — `docsite serve` / `docsite check` dispatch compiles; all lib
+ e2e tests green.
- [ ] **Step 3: Write the gated live smoke test**
Create `tests/live_smoke.rs`:
```rust
//! Gated smoke test against the real Gitea. Skips cleanly when Gitea is
//! unreachable (the project's skip-on-no-dependency convention) so it never
//! fails on an offline machine.
use docsite::gitea::{Fetcher, GiteaClient};
#[test]
fn aura_glossary_is_fetchable_from_live_gitea() {
let client = GiteaClient::new("http://192.168.178.103:3000");
match client.fetch_raw("Brummel/aura", "main", "docs/glossary.md") {
Ok(body) => assert!(!body.is_empty(), "live glossary should be non-empty"),
Err(e) => {
eprintln!("skip: live Gitea unreachable or auth-gated: {e:?}");
}
}
}
```
- [ ] **Step 4: Run the live smoke (non-fatal)**
Run: `cargo test --test live_smoke -- --nocapture`
Expected: PASS — either the assertion holds (Gitea reachable) or it prints
`skip:` and passes (unreachable/auth-gated). Never fails the suite.
- [ ] **Step 5: Manual serve sanity (optional, recorded not gated)**
Run: `DOCSITE_GITEA_TOKEN=<ro-token> cargo run -- serve` then
`curl -s localhost:8080/aura/design/INDEX | head -3`
Expected: the themed HTML shell for the aura design ledger. (A manual check
the orchestrator records; not part of `cargo test`.)