Adopt self-description as a ledger design principle — every surface explains itself #316

Closed
opened 2026-07-23 12:35:20 +02:00 by claude · 6 comments
Collaborator

Decision (user, 2026-07-23): self-description becomes a general design goal, to be recorded in the design ledger — every surface, not only the CLI, must be able to explain itself. The 2026-07-22 field test (triage in #314; CLI-help/introspection subset in #315) showed the cost of its absence: an agent holding only the release binary had to reverse-engineer execution semantics and document schemas forensically.

Surfaces in scope beyond #315's help and introspection audit: error messages (already at the target bar — the --trace refusals name #224 and the working verbs), scaffolded project docs (aura new / aura nodes new), the generated bootstrap card (#267), data-plane documents (blueprints, process/campaign documents) and their introspection, the registered vocabulary of native node crates, run artifacts (manifests), HTML surfaces.

Open design fork — mandatory doc-strings for the whole vocabulary, including the data plane:

  • Engine-shipped vocabulary (std nodes, blocks, metrics, folds, taps): mandatory is cheap and enforceable at registration/compile time; the block vocabulary already models the target register (process introspect --vocabulary).
  • Extension vocabulary (native node crates): mandatory at load/registration, with a refusal naming the missing field — an undescribed vocabulary entry is an incomplete contract.
  • Data-plane documents (blueprints, composite nodes): a plausible shape is mandatory-at-register — what enters the content-addressed store describes itself; loose working files stay unconstrained, avoiding placeholder pressure during authoring. Composites surface in the vocabulary by name, so they need a meaning entry like any other member.
  • Guardrail: doc-strings are non-load-bearing metadata — never evaluated by the engine, no freetext logic hole; the closed-vocabulary doctrine (domain invariant 10) is untouched. Open detail: whether a description participates in the content id or is stripped for the identity id, mirroring the existing debug-name treatment (graph introspect --content-id vs --identity-id).
  • Known risk of "mandatory": placeholder alibi-docs. A minimal shape gate at the register boundary (non-empty, not merely restating the name) is the counterweight; its strictness is part of the fork.

Status: direction decided (self-description as a ledger-recorded principle); the mandatory-doc-string scope and the register-gate shape are the open forks for the design work.

Decision (user, 2026-07-23): self-description becomes a general design goal, to be recorded in the design ledger — every surface, not only the CLI, must be able to explain itself. The 2026-07-22 field test (triage in #314; CLI-help/introspection subset in #315) showed the cost of its absence: an agent holding only the release binary had to reverse-engineer execution semantics and document schemas forensically. Surfaces in scope beyond #315's help and introspection audit: error messages (already at the target bar — the `--trace` refusals name #224 and the working verbs), scaffolded project docs (`aura new` / `aura nodes new`), the generated bootstrap card (#267), data-plane documents (blueprints, process/campaign documents) and their introspection, the registered vocabulary of native node crates, run artifacts (manifests), HTML surfaces. Open design fork — mandatory doc-strings for the whole vocabulary, including the data plane: - **Engine-shipped vocabulary** (std nodes, blocks, metrics, folds, taps): mandatory is cheap and enforceable at registration/compile time; the block vocabulary already models the target register (`process introspect --vocabulary`). - **Extension vocabulary** (native node crates): mandatory at load/registration, with a refusal naming the missing field — an undescribed vocabulary entry is an incomplete contract. - **Data-plane documents** (blueprints, composite nodes): a plausible shape is mandatory-at-`register` — what enters the content-addressed store describes itself; loose working files stay unconstrained, avoiding placeholder pressure during authoring. Composites surface in the vocabulary by name, so they need a meaning entry like any other member. - **Guardrail:** doc-strings are non-load-bearing metadata — never evaluated by the engine, no freetext logic hole; the closed-vocabulary doctrine (domain invariant 10) is untouched. Open detail: whether a description participates in the content id or is stripped for the identity id, mirroring the existing debug-name treatment (`graph introspect --content-id` vs `--identity-id`). - **Known risk of "mandatory":** placeholder alibi-docs. A minimal shape gate at the register boundary (non-empty, not merely restating the name) is the counterweight; its strictness is part of the fork. Status: direction decided (self-description as a ledger-recorded principle); the mandatory-doc-string scope and the register-gate shape are the open forks for the design work.
claude added the feature label 2026-07-23 12:35:20 +02:00
claude added this to the Self-description — every surface explains itself milestone 2026-07-23 13:42:24 +02:00
claude self-assigned this 2026-07-23 13:55:27 +02:00
Author
Collaborator

Design reconciliation (specify)

Spec: self-description. The forks below are still listed open on this issue; this records their resolution.

  • Fork: mandatory doc-string scope → mandatory for every engine-shipped closed vocabulary, enforced at its natural seam; load-time refusal for extension vocabulary; register-time gate for blueprints/composites; optional (not mandatory) description field for process/campaign documents.
    • Engine-shipped: blocks already carry doc: &'static str (BlockSchema, crates/aura-research/src/lib.rs:178-182) — the model. Nodes gain a doc field on NodeSchema (crates/aura-core/src/node.rs:344), compile-time-required and not serialized — blueprints reference nodes by type identity and carry no node logic (C24), so node docs never move a content id. Metrics: metric_vocabulary() (crates/aura-research/src/lib.rs:749) promotes from a bare name array to a schema carrier with a doc line — one carrier, no fourth roster site beside the #190-guarded triple. Tap slots (tap_vocabulary(), lib.rs:780) likewise. Tap folds: the fold registry already carries a doc line per entry (#283); no new fold mechanics this cycle (surface pending #310).
    • Extension vocabulary: at native-crate load (load_crate, crates/aura-runner/src/project.rs:532-570) every resolved schema must pass the gate; a miss refuses with the type and field named. Compile-time enforcement cannot cross the cdylib ABI boundary, so load time is the earliest honest seam.
    • Data plane: aura graph register requires a gate-passing doc on the root composite and every named nested composite — composites surface by name, so each needs a meaning entry. Loose working files stay unconstrained; the gate binds where the content-addressed store is entered. Process/campaign documents gain an additive-optional description (gated only when present): the mandatory fork in the issue body names blueprints/composites only, and a mandatory document field would shift every generated document's content id and break the generating verbs for no ratified gain.
    • Basis: derived — the issue body's own sketch, the existing carriers (BlockSchema, Composite.doc), and blast-radius containment (the additive-optional pattern is byte-stable for absent fields, pinned by unknown_optional_field_is_tolerated_byte_identically, crates/aura-engine/src/blueprint_serde.rs:447).
    • Also in force: user decision 2026-07-23 on #319 — the research-verb quintet retires, so its 48 flag fields are excluded from the doc-string duty (recorded on #319).
  • Fork: register-gate shape → deterministic minimal gate, one shared predicate at all three seams (std vocabulary unit test, native load, register): trimmed non-empty AND not a restatement of the entry's own name under case/separator-insensitive normalization. No length minimums, no content heuristics. The refusal names the entry, the field, and the failed rule.
    Basis: derived — the issue body names exactly this counterweight ("non-empty, not merely restating the name"); anything stronger has the engine judging prose, which brushes the non-load-bearing guardrail (doc-strings are never evaluated, domain invariant 10 untouched).
  • Fork: id treatment of descriptions → descriptions participate in the content id and are blanked for the identity id, exactly as the existing composite doc behaves (strip_debug_symbols blanks b.doc, crates/aura-engine/src/blueprint_serde.rs:156; pinned by graph_doc_field_moves_content_id_but_not_identity_id, crates/aura-cli/tests/graph_construct.rs:1033). Process/campaign documents have only a content id, which their optional description participates in.
    Basis: derived — the mechanism already exists and is test-pinned; a new field mirrors it rather than inventing a second treatment.
  • Carrier detail: node doc rides NodeSchema, not the cdylib descriptor — the descriptor stays a resolver table; the doc travels in the schema its vocabulary fn resolves, so std and extension nodes share one carrier and one gate.
    Basis: derived — C8 places the static signature on the value-empty recipe; a second descriptor-level doc channel would drift from the schema.

The ledger records the principle as a new contract (self-description — every surface explains itself; doc-strings are non-load-bearing metadata, never evaluated). Scope boundary within the milestone: this issue lands storage, enforcement, and the authored std meaning texts; the help/introspection surfacing lands with #315; the bootstrap card with #267.

## Design reconciliation (specify) Spec: self-description. The forks below are still listed open on this issue; this records their resolution. - **Fork: mandatory doc-string scope** → mandatory for every engine-shipped closed vocabulary, enforced at its natural seam; load-time refusal for extension vocabulary; register-time gate for blueprints/composites; **optional** (not mandatory) `description` field for process/campaign documents. - Engine-shipped: blocks already carry `doc: &'static str` (`BlockSchema`, crates/aura-research/src/lib.rs:178-182) — the model. Nodes gain a `doc` field on `NodeSchema` (crates/aura-core/src/node.rs:344), compile-time-required and **not serialized** — blueprints reference nodes by type identity and carry no node logic (C24), so node docs never move a content id. Metrics: `metric_vocabulary()` (crates/aura-research/src/lib.rs:749) promotes from a bare name array to a schema carrier with a doc line — one carrier, no fourth roster site beside the #190-guarded triple. Tap slots (`tap_vocabulary()`, lib.rs:780) likewise. Tap folds: the fold registry already carries a doc line per entry (#283); no new fold mechanics this cycle (surface pending #310). - Extension vocabulary: at native-crate load (`load_crate`, crates/aura-runner/src/project.rs:532-570) every resolved schema must pass the gate; a miss refuses with the type and field named. Compile-time enforcement cannot cross the cdylib ABI boundary, so load time is the earliest honest seam. - Data plane: `aura graph register` requires a gate-passing `doc` on the root composite and every named nested composite — composites surface by name, so each needs a meaning entry. Loose working files stay unconstrained; the gate binds where the content-addressed store is entered. Process/campaign documents gain an additive-optional `description` (gated only when present): the mandatory fork in the issue body names blueprints/composites only, and a mandatory document field would shift every generated document's content id and break the generating verbs for no ratified gain. - Basis: derived — the issue body's own sketch, the existing carriers (BlockSchema, `Composite.doc`), and blast-radius containment (the additive-optional pattern is byte-stable for absent fields, pinned by `unknown_optional_field_is_tolerated_byte_identically`, crates/aura-engine/src/blueprint_serde.rs:447). - Also in force: user decision 2026-07-23 on #319 — the research-verb quintet retires, so its 48 flag fields are excluded from the doc-string duty (recorded on #319). - **Fork: register-gate shape** → deterministic minimal gate, one shared predicate at all three seams (std vocabulary unit test, native load, register): trimmed non-empty AND not a restatement of the entry's own name under case/separator-insensitive normalization. No length minimums, no content heuristics. The refusal names the entry, the field, and the failed rule. Basis: derived — the issue body names exactly this counterweight ("non-empty, not merely restating the name"); anything stronger has the engine judging prose, which brushes the non-load-bearing guardrail (doc-strings are never evaluated, domain invariant 10 untouched). - **Fork: id treatment of descriptions** → descriptions participate in the content id and are blanked for the identity id, exactly as the existing composite `doc` behaves (`strip_debug_symbols` blanks `b.doc`, crates/aura-engine/src/blueprint_serde.rs:156; pinned by `graph_doc_field_moves_content_id_but_not_identity_id`, crates/aura-cli/tests/graph_construct.rs:1033). Process/campaign documents have only a content id, which their optional `description` participates in. Basis: derived — the mechanism already exists and is test-pinned; a new field mirrors it rather than inventing a second treatment. - **Carrier detail: node doc rides `NodeSchema`, not the cdylib descriptor** — the descriptor stays a resolver table; the doc travels in the schema its vocabulary fn resolves, so std and extension nodes share one carrier and one gate. Basis: derived — C8 places the static signature on the value-empty recipe; a second descriptor-level doc channel would drift from the schema. The ledger records the principle as a new contract (self-description — every surface explains itself; doc-strings are non-load-bearing metadata, never evaluated). Scope boundary within the milestone: this issue lands storage, enforcement, and the authored std meaning texts; the help/introspection surfacing lands with #315; the bootstrap card with #267.
Author
Collaborator

Autonomous sign-off (specify)

The self-description design spec — description carriers for every closed vocabulary, one shared deterministic shape gate, three enforcement seams (compile/unit for engine-shipped entries, load for native node crates, register for the content-addressed store), the authored std meaning texts, an additive-optional document description, and the ledger contract recording the principle — was signed autonomously. The signature is the independent grounding verification's PASS: every load-bearing assumption the spec makes about current codebase behaviour is ratified by a currently-green test (12/12 on the second dispatch; the first dispatch returned BLOCK on a mis-stated node-construction shape, which was corrected to the real struct-literal surface before fresh re-verification). No human signed.

Status: spec signed — ready for implementation planning.

## Autonomous sign-off (specify) The self-description design spec — description carriers for every closed vocabulary, one shared deterministic shape gate, three enforcement seams (compile/unit for engine-shipped entries, load for native node crates, register for the content-addressed store), the authored std meaning texts, an additive-optional document `description`, and the ledger contract recording the principle — was signed autonomously. The signature is the independent grounding verification's PASS: every load-bearing assumption the spec makes about current codebase behaviour is ratified by a currently-green test (12/12 on the second dispatch; the first dispatch returned BLOCK on a mis-stated node-construction shape, which was corrected to the real struct-literal surface before fresh re-verification). No human signed. Status: spec signed — ready for implementation planning.
Author
Collaborator

Load-seam refinements (iteration 2)

Three small forks derived while turning the spec's load-seam section into the iteration plan; recorded per the decision-log discipline.

  • The refusal variant carries the failed rule. The spec sketches ProjectError::UndescribedVocabularyEntry { type_id: String }, while its error-handling section requires refusal prose naming entry + field + rule. With only the type id the prose could not distinguish the empty-doc alibi from a name restatement, so the variant is { type_id: String, fault: DocGateFault } and the prose arm maps the fault to "has an empty doc" / "has a doc that merely restates its name". The normative error-handling text wins over the sketch.
  • The refusal e2e drives a dedicated gate-failing fixture crate (undescribed-project: one resolvable und::Opaque node with doc: ""), modelled on the existing charter-violation fixture — the empty doc is compile-legal, so only a real load through the dylib path proves the seam. The passing path needs no new fixture: both existing extension fixtures carry gate-clean docs and are already pinned by the load e2e suite.
  • No new unit test inside the runner for the gate loop: the gate predicate is unit-tested in core since iteration 1, and the spec's testing strategy names the end-to-end pattern as the load-seam test; a unit copy would duplicate the e2e without adding a caught failure mode.

Status: iteration 1 (compile/unit seam) complete and green on the run branch; iteration 2 (load seam) planned — ready for implementation.

## Load-seam refinements (iteration 2) Three small forks derived while turning the spec's load-seam section into the iteration plan; recorded per the decision-log discipline. - **The refusal variant carries the failed rule.** The spec sketches `ProjectError::UndescribedVocabularyEntry { type_id: String }`, while its error-handling section requires refusal prose naming entry + field + rule. With only the type id the prose could not distinguish the empty-doc alibi from a name restatement, so the variant is `{ type_id: String, fault: DocGateFault }` and the prose arm maps the fault to "has an empty doc" / "has a doc that merely restates its name". The normative error-handling text wins over the sketch. - **The refusal e2e drives a dedicated gate-failing fixture crate** (`undescribed-project`: one resolvable `und::Opaque` node with `doc: ""`), modelled on the existing charter-violation fixture — the empty doc is compile-legal, so only a real load through the dylib path proves the seam. The passing path needs no new fixture: both existing extension fixtures carry gate-clean docs and are already pinned by the load e2e suite. - **No new unit test inside the runner for the gate loop**: the gate predicate is unit-tested in core since iteration 1, and the spec's testing strategy names the end-to-end pattern as the load-seam test; a unit copy would duplicate the e2e without adding a caught failure mode. Status: iteration 1 (compile/unit seam) complete and green on the run branch; iteration 2 (load seam) planned — ready for implementation.
Author
Collaborator

Register-seam refinements (iteration 3)

Four forks derived while planning the register seam; recorded per the decision-log discipline.

  • The gate lives inside the store's own public write API. The spec's normative sentence — "the gate guards the store boundary, not a verb" — is read structurally: the registry's public put_blueprint now parses the canonical bytes and walks root + nested composite docs before writing, so the verb, every register-then-run sugar path, and any future caller are gated by construction. The raw write survives as a private in-crate put_blueprint_unchecked for store-mechanics tests — which doubles as the witness that pre-C29 doc-less store entries stay readable (never-retroactive). A CLI-side gate at the N call sites was rejected: it is bypassable by the next caller, and admission policy belongs to the boundary it protects. Canonical bytes that do not parse as a blueprint refuse too — the gate cannot certify what it cannot read.
  • Sugar paths are gated, and the shipped example blueprints gain authored root docs. The four examples and their four open-fixture twins were doc-less; under the store-boundary reading they must describe themselves. Their content ids shift — the spec's testing strategy explicitly anticipates this deliberate pin movement, and no literal id pins exist in the tests (comparisons are relational).
  • An absent composite doc refuses exactly like an empty one. doc: None and doc: "" are the same fault ("carries no doc") — otherwise omission would be the trivial bypass of the shape gate.
  • No document-side identity twin pin. Documents have no identity projection (only a content id), so identity-blindness of description holds by construction; the meaningful document pair is: absent is byte-identical to today (already pinned), present moves the content id (new test).

Status: iteration 2 (load seam) complete and green on the run branch; iteration 3 (register seam + document description) planned — ready for implementation.

## Register-seam refinements (iteration 3) Four forks derived while planning the register seam; recorded per the decision-log discipline. - **The gate lives inside the store's own public write API.** The spec's normative sentence — "the gate guards the store boundary, not a verb" — is read structurally: the registry's public `put_blueprint` now parses the canonical bytes and walks root + nested composite docs before writing, so the verb, every register-then-run sugar path, and any future caller are gated by construction. The raw write survives as a private in-crate `put_blueprint_unchecked` for store-mechanics tests — which doubles as the witness that pre-C29 doc-less store entries stay readable (never-retroactive). A CLI-side gate at the N call sites was rejected: it is bypassable by the next caller, and admission policy belongs to the boundary it protects. Canonical bytes that do not parse as a blueprint refuse too — the gate cannot certify what it cannot read. - **Sugar paths are gated, and the shipped example blueprints gain authored root docs.** The four examples and their four open-fixture twins were doc-less; under the store-boundary reading they must describe themselves. Their content ids shift — the spec's testing strategy explicitly anticipates this deliberate pin movement, and no literal id pins exist in the tests (comparisons are relational). - **An absent composite doc refuses exactly like an empty one.** `doc: None` and `doc: ""` are the same fault ("carries no doc") — otherwise omission would be the trivial bypass of the shape gate. - **No document-side identity twin pin.** Documents have no identity projection (only a content id), so identity-blindness of `description` holds by construction; the meaningful document pair is: absent is byte-identical to today (already pinned), present moves the content id (new test). Status: iteration 2 (load seam) complete and green on the run branch; iteration 3 (register seam + document description) planned — ready for implementation.
Author
Collaborator

Op-script doc slot (iteration 3, mid-iteration fork)

The store-boundary gate exposed an anticipated hole: op-script-built composites (GraphSession::finish) deliberately carry no doc surface — the construction vocabulary's own comment marks it as "until that vocabulary grows a slot" (a #125 scope cut). Under C29 a doc-less op-script product can no longer register, which broke the previously-green op-script register path.

Decision: the op vocabulary grows the anticipated slot. A new doc op — {"op":"doc","text":"..."}, the op-script twin of the Rust builder's .doc(...) — a closed, typed metadata construct with no logic content (squarely inside the C25 closed-vocabulary/P1 pattern). At most one per script; a second is a fault, matching the vocabulary's existing duplicate-fault family. Rejected alternatives: auto-deriving a doc from the script (an alibi doc — the principle forbids machine-invented meaning lines), and exempting op-script products from the store gate (the trivial bypass that would hollow the seam).

Also recorded: the implement run's third task discarded the first two tasks' edits via a file-level checkout; the loop's snapshot guard caught it and the content was restored from the recovery snapshot and re-verified — the run continues on a checkpoint commit.

Status: iteration 3 tasks 1–3 checkpointed on the run branch; register e2e, op-script doc slot, and full gates remain — ready to continue implementation.

## Op-script doc slot (iteration 3, mid-iteration fork) The store-boundary gate exposed an anticipated hole: op-script-built composites (`GraphSession::finish`) deliberately carry no doc surface — the construction vocabulary's own comment marks it as "until that vocabulary grows a slot" (a #125 scope cut). Under C29 a doc-less op-script product can no longer register, which broke the previously-green op-script register path. **Decision: the op vocabulary grows the anticipated slot.** A new `doc` op — `{"op":"doc","text":"..."}`, the op-script twin of the Rust builder's `.doc(...)` — a closed, typed metadata construct with no logic content (squarely inside the C25 closed-vocabulary/P1 pattern). At most one per script; a second is a fault, matching the vocabulary's existing duplicate-fault family. Rejected alternatives: auto-deriving a doc from the script (an alibi doc — the principle forbids machine-invented meaning lines), and exempting op-script products from the store gate (the trivial bypass that would hollow the seam). Also recorded: the implement run's third task discarded the first two tasks' edits via a file-level checkout; the loop's snapshot guard caught it and the content was restored from the recovery snapshot and re-verified — the run continues on a checkpoint commit. Status: iteration 3 tasks 1–3 checkpointed on the run branch; register e2e, op-script doc slot, and full gates remain — ready to continue implementation.
Author
Collaborator

Fieldtest fork: RestatesName vs the namespaced short name

The cycle's fieldtest surfaced an underspecified reading in the C29 gate: at the load seam the entry name is the full namespaced type id, so an extension node ns::Scale with doc: "Scale" — its own display name, the most natural alibi — passes the gate, while the equivalent composite alibi at the register seam is caught (composite names are bare).

Decision: tighten, not ratify. The gate exists to refuse docs that add nothing beyond the name, and the bare display segment restates the name in every sense that matters to a reader. doc_gate additionally refuses a doc that normalizes to the name's final ::-segment; for unnamespaced entries the two checks coincide, so no shipped entry changes behaviour. The C29 contract sentence gains the precision (full id or its bare display segment). Rejected alternative: ratifying the full-id-only reading — it would keep the most natural alibi legal at exactly the seam built for extension authors.

## Fieldtest fork: RestatesName vs the namespaced short name The cycle's fieldtest surfaced an underspecified reading in the C29 gate: at the load seam the entry name is the full namespaced type id, so an extension node `ns::Scale` with `doc: "Scale"` — its own display name, the most natural alibi — passes the gate, while the equivalent composite alibi at the register seam is caught (composite names are bare). **Decision: tighten, not ratify.** The gate exists to refuse docs that add nothing beyond the name, and the bare display segment restates the name in every sense that matters to a reader. `doc_gate` additionally refuses a doc that normalizes to the name's final `::`-segment; for unnamespaced entries the two checks coincide, so no shipped entry changes behaviour. The C29 contract sentence gains the precision (full id or its bare display segment). Rejected alternative: ratifying the full-id-only reading — it would keep the most natural alibi legal at exactly the seam built for extension authors.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Brummel/Aura#316