Brummel/Aura
1
0
Fork 0
Code Issues 31 Pull Requests Actions Packages Projects Releases Wiki Activity

feat: trace charts — tap graph streams to disk, serve as web charts (visual face, first cut) #101

New Issue
Closed
opened 2026-06-18 21:43:39 +02:00 by Brummel · 3 comments
Brummel commented 2026-06-18 21:43:39 +02:00
Owner
Copy Link
Copy Source

Design record (in-context entry, visual face — first cut)

This issue seeds aura's visual face: a downstream seam that taps arbitrary
recorder streams from a run, persists them to disk, and serves them as web
charts in the browser. It supersedes the ledger's prior C22 "egui-native"
direction. The design was settled in an in-context discussion; this records it
with provenance so the forward queue stays honest.

Goal

Tap data from a running graph and serve it as a chart in the browser.

Resolved load-bearing forks (with provenance)

  • Fork: visual-face medium -> a web frontend served from disk-persisted
    recorder traces; egui is out for now.
    Provenance (user, 2026-06-18): "egui bleibt bis auf weiteres aussen vor. Ich
    stelle mir das eher so vor: Recorder (die im Prinzip ALLES abgreifen koennen,
    nicht nur Equity) speichern die Run-Daten auf die Platte und die werden dann
    auf einem Web-Frontend als Chart serviert."

  • Fork A: trace-store shape -> raw per-tap (each recorder tap persisted
    with its own timestamps); join_on_ts runs at serve time, not record
    time. Keeps every feed at its native timestamps so the view layer can choose
    overlay vs. timestamp-aligned panels per request.
    Provenance (user, 2026-06-18): approved "roh-pro-Tap auf Platte" with "ja,
    passt".

  • Fork B: serving model -> a static self-contained HTML page first (the
    aura-cli::render::render_html precedent); a local HTTP server is deferred to
    when families-comparison (C21) / live-during need it.
    Provenance (user, 2026-06-18): approved "static-HTML-Chart zuerst, Server
    spaeter fuer Familien" with "ja, passt".

  • Defaults taken (user delegated): on-disk format = columnar JSON per
    tap     (the SoA form preserved to disk, C7; one column = one chart series);
    charting lib = uPlot, vendored like the existing Graphviz-WASM blob.
    Provenance (user, 2026-06-18): "Format/Lib nehme ich als Default mit, falls du
    nichts anderes sagst" -> "ja, passt".

Scope (first cut)

  1. Persist drained recorder traces to disk: raw per-tap, columnar JSON, keyed to
    the run and indexed alongside the existing run registry (C18, runs/).
  2. A CLI verb that reads one persisted run's traces and emits a self-contained
    HTML chart page (uPlot vendored), with feeds overlaid in one chart OR
    timestamp-aligned in stacked panels.
  3. Serve-time multi-feed alignment via the existing join_on_ts primitive
    (today only used in-memory in example code).

Out of scope (explicit follow-on)

  • Families / comparison views across multiple runs (C21).
  • A local HTTP server (aura serve).
  • Level-of-detail / "mipmapping" downsampling; an Arrow/Parquet columnar store
    (the scale path).
  • Live streaming during a run.

Ledger impact

Amends C22 (egui-native -> web-from-disk seam). The engine stays headless
(C14): the web frontend is a downstream consumer, the engine a serializer; no
UI/pixel knowledge enters the engine. This is arguably a cleaner fit to C14 than
egui's in-process zero-copy from the SoA columns was.

context: settled in-context design discussion; first cut of the visual face.

## Design record (in-context entry, visual face — first cut) This issue seeds aura's **visual face**: a downstream seam that taps arbitrary recorder streams from a run, persists them to disk, and serves them as web charts in the browser. It supersedes the ledger's prior C22 "egui-native" direction. The design was settled in an in-context discussion; this records it with provenance so the forward queue stays honest. ### Goal Tap data from a running graph and serve it as a chart in the browser. ### Resolved load-bearing forks (with provenance) - **Fork: visual-face medium** -> a web frontend served from disk-persisted recorder traces; egui is out for now. Provenance (user, 2026-06-18): "egui bleibt bis auf weiteres aussen vor. Ich stelle mir das eher so vor: Recorder (die im Prinzip ALLES abgreifen koennen, nicht nur Equity) speichern die Run-Daten auf die Platte und die werden dann auf einem Web-Frontend als Chart serviert." - **Fork A: trace-store shape** -> raw **per-tap** (each recorder tap persisted with its own timestamps); `join_on_ts` runs at **serve time**, not record time. Keeps every feed at its native timestamps so the view layer can choose overlay vs. timestamp-aligned panels per request. Provenance (user, 2026-06-18): approved "roh-pro-Tap auf Platte" with "ja, passt". - **Fork B: serving model** -> a **static self-contained HTML** page first (the `aura-cli::render::render_html` precedent); a local HTTP server is deferred to when families-comparison (C21) / live-during need it. Provenance (user, 2026-06-18): approved "static-HTML-Chart zuerst, Server spaeter fuer Familien" with "ja, passt". - **Defaults taken (user delegated):** on-disk format = **columnar JSON per tap** (the SoA form preserved to disk, C7; one column = one chart series); charting lib = **uPlot**, vendored like the existing Graphviz-WASM blob. Provenance (user, 2026-06-18): "Format/Lib nehme ich als Default mit, falls du nichts anderes sagst" -> "ja, passt". ### Scope (first cut) 1. Persist drained recorder traces to disk: raw per-tap, columnar JSON, keyed to the run and indexed alongside the existing run registry (C18, `runs/`). 2. A CLI verb that reads one persisted run's traces and emits a self-contained HTML chart page (uPlot vendored), with feeds **overlaid** in one chart OR **timestamp-aligned** in stacked panels. 3. Serve-time multi-feed alignment via the existing `join_on_ts` primitive (today only used in-memory in example code). ### Out of scope (explicit follow-on) - Families / comparison views across multiple runs (C21). - A local HTTP server (`aura serve`). - Level-of-detail / "mipmapping" downsampling; an Arrow/Parquet columnar store (the scale path). - Live streaming during a run. ### Ledger impact Amends **C22** (egui-native -> web-from-disk seam). The engine stays headless (C14): the web frontend is a downstream consumer, the engine a serializer; no UI/pixel knowledge enters the engine. This is arguably a cleaner fit to C14 than egui's in-process zero-copy from the SoA columns was. context: settled in-context design discussion; first cut of the visual face.
Brummel added the feature label 2026-06-18 21:43:39 +02:00
Brummel commented 2026-06-18 23:03:25 +02:00
Author
Owner
Copy Link
Copy Source

Design decisions recorded (spec 0056 §6, autonomous-build mandate)

Spec 0056-trace-charts-persist-and-serve §6 settles two overlay-chart
sub-decisions as naive-viewer details under the user's autonomous-build
mandate ("Mach da ein issue draus und fang sofort damit an ... du musst voellig
autonom arbeiten ... Falls Entscheidungen getroffen werden muessen, dokumentiere,
was du getan hast", 2026-06-18). These are documented orchestrator decisions
within that delegated authority — not user-picked design forks:

  • Overlay y-axis -> each series on its own auto-ranged y-scale (uPlot
    multi-scale), sharing only the x-axis. Rationale: pip-equity (~+/-0.004) and
    exposure (+/-1.0) on one shared y-scale would flatten the equity curve to a
    line; per-series scales keep every feed legible. A "naive viewer" detail the
    mandate explicitly covers ("ein naiver Viewer ohne Mipmapping reicht erst mal").

  • Overlay spine -> x = the sorted union of all taps' recorded timestamps,
    each series null-filled where its tap did not fire at a timestamp (via the
    existing join_on_ts). Rationale: faithful (drops no points); for the
    first-cut co-cadenced equity/exposure taps it equals their shared
    timestamps, so it is moot for what ships now and only bites different-cadence
    taps later.

Provenance for the delegation: the user mandate quoted above. Both are
naive-viewer defaults; if either proves wrong it is a forward correction, not a
contract.

## Design decisions recorded (spec 0056 §6, autonomous-build mandate) Spec `0056-trace-charts-persist-and-serve` §6 settles two overlay-chart sub-decisions as **naive-viewer details under the user's autonomous-build mandate** ("Mach da ein issue draus und fang sofort damit an ... du musst voellig autonom arbeiten ... Falls Entscheidungen getroffen werden muessen, dokumentiere, was du getan hast", 2026-06-18). These are **documented orchestrator decisions within that delegated authority** — not user-picked design forks: - **Overlay y-axis** -> each series on its own auto-ranged y-scale (uPlot multi-scale), sharing only the x-axis. Rationale: pip-equity (~+/-0.004) and exposure (+/-1.0) on one shared y-scale would flatten the equity curve to a line; per-series scales keep every feed legible. A "naive viewer" detail the mandate explicitly covers ("ein naiver Viewer ohne Mipmapping reicht erst mal"). - **Overlay spine** -> x = the sorted union of all taps' recorded timestamps, each series null-filled where its tap did not fire at a timestamp (via the existing `join_on_ts`). Rationale: faithful (drops no points); for the first-cut co-cadenced `equity`/`exposure` taps it equals their shared timestamps, so it is moot for what ships now and only bites different-cadence taps later. Provenance for the delegation: the user mandate quoted above. Both are naive-viewer defaults; if either proves wrong it is a forward correction, not a contract.
Brummel commented 2026-06-19 01:05:12 +02:00
Author
Owner
Copy Link
Copy Source

Implemented + verified + audited (on local main, pending push)

The visual face's first cut shipped across two iterations:

  • persist (8bb5256): aura run [--macd] [--real <SYM> …] --trace <name> writes
    each drained recorder tap to runs/traces/<name>/{index,equity,exposure}.json as
    columnar (SoA, C7) JSON. Default (no --trace) run byte-for-byte unchanged.
  • serve (15ee6d5): aura chart <name> [--panels] reads them back, aligns all
    taps on a synthetic-union timestamp spine via the post-run join_on_ts, and emits
    a static self-contained uPlot page — feeds overlaid (each on its own y-scale), or
    in timestamp-aligned stacked panels. Missing run -> stderr + exit 2.
  • uPlot 1.6.32 vendored (d534f10); C22/C14 ledger amended (3b56efb) + realization
    note added at close (5bff7e0).

Verified end-to-end (orchestrator): full cargo test --workspace green
(ColumnarTrace + TraceStore round-trips, node DOM guards over buildCharts, the
chart e2e), cargo clippy --workspace --all-targets -D warnings exit 0, and a real
aura run --trace demo && aura chart demo emitting a 57 KB self-contained page (+
--panels, + aura chart nope exit 2). Cycle audited drift-clean.

One ratified scope deviation tracked as #102 (the served page renders the series but
not yet a run-context header — a naive-viewer call, manifest is on disk).

Committed on local main; not pushed (left for review). Closing is deferred to your
push/review.

## Implemented + verified + audited (on local main, pending push) The visual face's first cut shipped across two iterations: - **persist** (8bb5256): `aura run [--macd] [--real <SYM> …] --trace <name>` writes each drained recorder tap to `runs/traces/<name>/{index,equity,exposure}.json` as columnar (SoA, C7) JSON. Default (no `--trace`) run byte-for-byte unchanged. - **serve** (15ee6d5): `aura chart <name> [--panels]` reads them back, aligns all taps on a synthetic-union timestamp spine via the post-run `join_on_ts`, and emits a static self-contained uPlot page — feeds overlaid (each on its own y-scale), or in timestamp-aligned stacked panels. Missing run -> stderr + exit 2. - uPlot 1.6.32 vendored (d534f10); C22/C14 ledger amended (3b56efb) + realization note added at close (5bff7e0). **Verified end-to-end** (orchestrator): full `cargo test --workspace` green (ColumnarTrace + TraceStore round-trips, node DOM guards over `buildCharts`, the chart e2e), `cargo clippy --workspace --all-targets -D warnings` exit 0, and a real `aura run --trace demo && aura chart demo` emitting a 57 KB self-contained page (+ `--panels`, + `aura chart nope` exit 2). Cycle audited **drift-clean**. One ratified scope deviation tracked as #102 (the served page renders the series but not yet a run-context header — a naive-viewer call, manifest is on disk). Committed on local `main`; not pushed (left for review). Closing is deferred to your push/review.
Brummel commented 2026-06-21 16:27:01 +02:00
Author
Owner
Copy Link
Copy Source

Closing as done. The visual-face first cut is shipped on main and pushed: per-tap columnar (SoA, C7) trace persistence (aura run … --trace <name>), the aura chart <name> [--panels] static self-contained uPlot page, and serve-time multi-feed alignment via join_on_ts. cargo test --workspace green; C22 amended in the ledger (egui-native → web-from-disk seam).

The one ratified scope deviation (run-context header on the served page) is tracked in #102; the families-comparison meta-views (cross-member overlay / small-multiples) remain the open follow-on. Grounded against current main during the 2026-06-21 open-issue review.

Closing as done. The visual-face first cut is shipped on main and pushed: per-tap columnar (SoA, C7) trace persistence (`aura run … --trace <name>`), the `aura chart <name> [--panels]` static self-contained uPlot page, and serve-time multi-feed alignment via `join_on_ts`. `cargo test --workspace` green; C22 amended in the ledger (egui-native → web-from-disk seam). The one ratified scope deviation (run-context header on the served page) is tracked in #102; the families-comparison meta-views (cross-member overlay / small-multiples) remain the open follow-on. Grounded against current `main` during the 2026-06-21 open-issue review.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
BLOCKER
Core functionality blocked; preempts other selection
 bug
Observable misbehaviour
 feature
Substantial extension; no full spec required
 idea
Droppable; no commitment
No Label feature
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
claude (Claude) Brummel
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Brummel/Aura#101
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?