Brummel/Aura
1
0
Fork 0
Code Issues 31 Pull Requests Actions Packages Projects Releases Wiki Activity

Registry lineage for orchestration families: store walk-forward/MC runs as related records (C18) #70

New Issue
Closed
opened 2026-06-14 23:08:30 +02:00 by Brummel · 3 comments
Brummel commented 2026-06-14 23:08:30 +02:00
Owner
Copy Link
Copy Source

C18's lineage clause is currently design-only: aura-registry (lib.rs) stores flat RunReports with no links between them. A family (sweep / MC / walk-forward) is a set of runs with a common origin — they should be stored as related records (run←inputs lineage: which blueprint, which base point, which window/seed produced each), so a family can be re-listed, compared, and re-derived as a unit rather than as loose runs.

Build

  • A lineage link in the registry (a family id / parent reference) so MC realizations and walk-forward windows are queryable as a family.
  • CLI surface to list/compare a family (aura mc, aura walkforward, and a family-aware aura runs).

Depends on

  • The family types (McFamily, WalkForwardResult, SweepFamily) exist. This lands LAST — it persists what the other issues produce.

Acceptance

A family run writes related records; the family can be listed and ranked as a unit; lineage round-trips (the stored links re-derive the family).

Contracts

C18 (run registry + lineage), C21.

C18's lineage clause is currently design-only: aura-registry (lib.rs) stores flat RunReports with no links between them. A family (sweep / MC / walk-forward) is a set of runs with a common origin — they should be stored as *related* records (run←inputs lineage: which blueprint, which base point, which window/seed produced each), so a family can be re-listed, compared, and re-derived as a unit rather than as loose runs. ## Build - A lineage link in the registry (a family id / parent reference) so MC realizations and walk-forward windows are queryable as a family. - CLI surface to list/compare a family (`aura mc`, `aura walkforward`, and a family-aware `aura runs`). ## Depends on - The family types (McFamily, WalkForwardResult, SweepFamily) exist. This lands LAST — it persists what the other issues produce. ## Acceptance A family run writes related records; the family can be listed and ranked as a unit; lineage round-trips (the stored links re-derive the family). ## Contracts C18 (run registry + lineage), C21.
Brummel added this to the The World, part II — orchestration families milestone 2026-06-14 23:08:30 +02:00
Brummel added the feature label 2026-06-14 23:08:30 +02:00
Brummel commented 2026-06-15 09:27:04 +02:00
Author
Owner
Copy Link
Copy Source

Design constraint (World-II eager-agnostic firewall — refs #71)

Keep lineage producer-agnostic. Re-derivation of a family run must mean RE-RUNNING from (blueprint commit + base point + window + seed), never replaying a stored input artifact: do NOT add an input-stream field (recorded blob, file path, byte payload) to RunManifest or the lineage record. The window: (Timestamp, Timestamp) must be a producer-supplied INPUT, not derived by scanning a materialized Vec (today's prices.first()/.last() at the run_one call-site). A lazy producer knows its bounds without materializing, so the stored lineage stays byte-identical whether the source is eager or streamed. (C18 + C6.)

## Design constraint (World-II eager-agnostic firewall — refs #71) Keep lineage producer-agnostic. Re-derivation of a family run must mean RE-RUNNING from (blueprint commit + base point + window + seed), never replaying a stored input artifact: do NOT add an input-stream field (recorded blob, file path, byte payload) to `RunManifest` or the lineage record. The `window: (Timestamp, Timestamp)` must be a producer-supplied INPUT, not derived by scanning a materialized Vec (today's `prices.first()/.last()` at the run_one call-site). A lazy producer knows its bounds without materializing, so the stored lineage stays byte-identical whether the source is eager or streamed. (C18 + C6.)
Brummel commented 2026-06-15 17:19:53 +02:00
Author
Owner
Copy Link
Copy Source

Design reconciliation (specify, in-context entry)

Spec: 0045-registry-lineage-families. The fork below is still listed open on this issue (the body says only "a family id / parent reference"); it was resolved in the in-context design discussion.

  • Fork: family_id form -> name + counter: family_id = "{name}-{counter}", where name is author-supplied (a builder-function argument, surfaced on the CLI as --name with a per-kind default such as mc / sweep / walkforward) and counter is a per-name run-index assigned at persistence (the smallest k for which "{name}-{k}" is not already in the family store). NOT a reproducible content-hash, NOT a UUID.

    Rationale (from the discussion): the id is used only as (1) the grouping key for the round-trip (group_families) and (2) the user-facing CLI lookup handle (aura runs family <id>). Neither use needs reproducibility, so the content-hash's only extra property is unused this cycle (and, against an append-only store, would silently merge two identical-input runs). A counter avoids both a new dependency and nondeterministic entropy (which a UUID would introduce, against the engine's seeded-determinism discipline) while staying human-typable; a user-supplied name makes the registry human-navigable, matching the "World = named experiments" framing (C21).

    Provenance: user decision in-session 2026-06-15 — verbatim "ja, name + counter, stell die Spec um", following the user's own narrowing "reicht nicht ein einfacher Counter?" and "Koennte ein Praefix vom User kommen?".

## Design reconciliation (specify, in-context entry) Spec: 0045-registry-lineage-families. The fork below is still listed open on this issue (the body says only "a family id / parent reference"); it was resolved in the in-context design discussion. - **Fork: family_id form** -> **name + counter**: `family_id = "{name}-{counter}"`, where `name` is author-supplied (a builder-function argument, surfaced on the CLI as `--name` with a per-kind default such as `mc` / `sweep` / `walkforward`) and `counter` is a per-name run-index assigned at persistence (the smallest `k` for which `"{name}-{k}"` is not already in the family store). NOT a reproducible content-hash, NOT a UUID. Rationale (from the discussion): the id is used only as (1) the grouping key for the round-trip (`group_families`) and (2) the user-facing CLI lookup handle (`aura runs family <id>`). Neither use needs reproducibility, so the content-hash's only extra property is unused this cycle (and, against an append-only store, would silently merge two identical-input runs). A counter avoids both a new dependency and nondeterministic entropy (which a UUID would introduce, against the engine's seeded-determinism discipline) while staying human-typable; a user-supplied name makes the registry human-navigable, matching the "World = named experiments" framing (C21). Provenance: user decision in-session 2026-06-15 — verbatim "ja, name + counter, stell die Spec um", following the user's own narrowing "reicht nicht ein einfacher Counter?" and "Koennte ein Praefix vom User kommen?".
Brummel commented 2026-06-15 18:36:09 +02:00
Author
Owner
Copy Link
Copy Source

Done — shipped in cycle 0045 and audited clean.

A sweep / Monte-Carlo / walk-forward run now persists as a family: related FamilyRunRecords sharing a family_id = "{name}-{counter}" in a sibling families.jsonl, leaving the flat runs.jsonl API byte-for-byte unchanged. group_families re-derives a family from the stored links (the round-trip), so a family is re-listable and rankable as a unit (C18/C21).

CLI: aura mc (new), aura runs families, aura runs family <id> [rank <metric>]; aura sweep / walkforward / mc take an optional --name.

Producer-agnostic per the #71 firewall: a member's window is supplied by Source::bounds() / window_of (eager or streamed → byte-identical lineage), never a materialized-Vec scan; no input-stream artifact enters a manifest or lineage record — the manifest is the sole re-derivation recipe.

Commits: 1d9555c (engine + registry core), bcd1072 (CLI surface), a168f07 (audit). Ledger: docs/design/INDEX.md C18 realization note (cycle 0045).

Follow-up filed as #73 (the now-CLI-unwritten flat-store path: give aura run a persisting path, or retire runs list / rank).

Done — shipped in cycle 0045 and audited clean. A sweep / Monte-Carlo / walk-forward run now persists as a **family**: related `FamilyRunRecord`s sharing a `family_id = "{name}-{counter}"` in a sibling `families.jsonl`, leaving the flat `runs.jsonl` API byte-for-byte unchanged. `group_families` re-derives a family from the stored links (the round-trip), so a family is re-listable and rankable as a unit (C18/C21). CLI: `aura mc` (new), `aura runs families`, `aura runs family <id> [rank <metric>]`; `aura sweep` / `walkforward` / `mc` take an optional `--name`. Producer-agnostic per the #71 firewall: a member's window is supplied by `Source::bounds()` / `window_of` (eager or streamed → byte-identical lineage), never a materialized-`Vec` scan; no input-stream artifact enters a manifest or lineage record — the manifest is the sole re-derivation recipe. Commits: 1d9555c (engine + registry core), bcd1072 (CLI surface), a168f07 (audit). Ledger: `docs/design/INDEX.md` C18 realization note (cycle 0045). Follow-up filed as #73 (the now-CLI-unwritten flat-store path: give `aura run` a persisting path, or retire `runs list` / `rank`).
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
BLOCKER
Core functionality blocked; preempts other selection
 bug
Observable misbehaviour
 feature
Substantial extension; no full spec required
 idea
Droppable; no commitment
No Label feature
Milestone
No items
No Milestone The World, part II — orchestration families
Projects
Clear projects
No project
Assignees
Clear assignees
claude (Claude) Brummel
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Brummel/Aura#70
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?