Aura/docs/specs/0090-blueprint-forward-compat.md

Blueprint format forward-compatibility: the two-tier discipline — Design Spec

Date: 2026-06-30 Status: Draft — awaiting user spec review Authors: orchestrator + Claude

Goal

Close #156: guarantee the blueprint data format (C24) extends additively — later additions land without breaking existing serializations — under a single, codified two-tier discipline:

• Tier-1 (additive-optional, ignore / default-safe). A new optional field or section is tolerated by an older reader: serde ignores unknown fields (no deny_unknown_fields anywhere — verified rg = 0), skip_serializing_if omits absent options, and no format_version bump occurs. Constraint (C1): a new optional field's default equals prior behaviour, so an old blueprint loaded by a new engine reproduces the same graph.
• Tier-2 (load-bearing, must-understand). A new node type, edge semantics, or structural-axis kind must bump format_version, so an old reader refuses cleanly (LoadError::UnsupportedVersion) rather than silently building a different graph. An out-of-vocabulary node likewise refuses (LoadError::UnknownNodeType).

Most of #156 already shipped with #155 (cycle 0087). This cycle is a small verification-plus-codification iteration: it adds the one missing proof — forward-tolerance of an unknown optional field on the current loader — and writes the discipline durably into the LoadError doc and the C24 ledger entry. It is not a new mechanism.

What #155 already delivers (verified, green — the grounding evidence)

• The format_version envelope: BLUEPRINT_FORMAT_VERSION = 1, BlueprintDoc { format_version, blueprint } (blueprint_serde.rs:17-24).
• Canonical omit-defaults form: #[serde(default, skip_serializing_if = ...)] on CompositeData::{edges,input_roles,output} and PrimitiveData::{name,bound}. Pinned by signal_serializes_to_canonical_golden.
• Both Tier-2 refusals at the load path, each with a green test:
  • LoadError::UnsupportedVersion { found, supported } — blueprint_serde.rs:162-167, test unsupported_version_fails_named (format_version: 2 → refused).
  • LoadError::UnknownNodeType(String) — blueprint_serde.rs:128, test unknown_node_type_fails_named ("NoSuchNode" → refused).

There is no CLI load-a-blueprint-JSON command today; the version-refusal has no CLI surface, so the engine-level typed LoadError is the Tier-2 contract. The CLI build path (aura graph build) already exit-codes an unknown node type via OpError (cycle 0088). This cycle adds no CLI surface.

Architecture

The forward-compat guarantee is a property of the existing serde projection, not a new component. Tier-1 tolerance is already true (serde's default unknown-field handling + the omit-defaults discipline); it is simply unproven. The deliverable is therefore:

1. one executable proof (the Tier-1 forward-tolerance test), and
2. a documentation codification turning the currently-deferred "must-understand horizon … is #156" note into the settled two-tier rule.

No struct, signature, or runtime behaviour changes. The derived design fork (the per-section required-flag / PNG critical-vs-ancillary mechanism) is not built — recorded on #156 with its rationale: the version envelope already gives the must-understand refusal at version granularity, there is no current Tier-2 section to validate a finer scheme against, and C1 holds either way.

Concrete code shapes

The deliverable: the Tier-1 forward-tolerance test (new — the additive property as executable evidence)

Added to the blueprint_serde.rs test module (it already has run_recording, sink_free_sma_cross_signal, and std_vocabulary in scope):

#[test]
fn unknown_optional_field_is_tolerated_byte_identically() {
    // Forward-compat (C24 / #156, Tier-1): a future writer that has additively
    // extended the format emits optional keys the CURRENT loader does not know.
    // The loader must silently ignore them and reconstruct the IDENTICAL graph —
    // never refuse, never let an unknown key leak into the built blueprint. serde
    // ignores unknown fields by default (no deny_unknown_fields). This pins the
    // additive property the canonical-golden + omit-defaults discipline claims but
    // no existing test exercises.
    let canonical =
        blueprint_to_json(&crate::test_fixtures::sink_free_sma_cross_signal()).expect("serializes");

    // Inject unknown optional keys at two additive shapes a new writer might emit:
    // a doc-level section ("metadata") and a per-primitive key ("annotations").
    let with_unknown = canonical
        .replacen(
            r#"{"format_version":1,"#,
            r#"{"format_version":1,"metadata":{"author":"future"},"#,
            1,
        )
        .replacen(r#"{"type":"Sub"}"#, r#"{"type":"Sub","annotations":["future"]}"#, 1);
    assert_ne!(with_unknown, canonical, "probe must actually inject unknown keys");

    // tolerated, not refused
    let loaded = blueprint_from_json(&with_unknown, &|t| aura_std::std_vocabulary(t))
        .expect("an unknown optional field is tolerated, not refused");

    // the unknown keys did not leak into the graph: re-serialization equals the
    // canonical form WITHOUT them (byte-identical reconstruction).
    assert_eq!(blueprint_to_json(&loaded).expect("re-serializes"), canonical);

    // and the loaded graph runs bit-identically to the plain canonical one (C1).
    let plain = blueprint_from_json(&canonical, &|t| aura_std::std_vocabulary(t)).expect("loads");
    assert_eq!(
        run_recording(loaded),
        run_recording(plain),
        "run diverged under an unknown optional field",
    );
}

The codification: LoadError::UnsupportedVersion doc (before → after)

Before (blueprint_serde.rs:110-113):

    /// `format_version` the loader does not understand. The full must-understand
    /// horizon (multiple versions, unknown-section policy) is #156; #155 supports
    /// exactly one version and refuses any other, cleanly.
    UnsupportedVersion { found: u32, supported: u32 },

After (the discipline is now settled, not deferred):

    /// `format_version` the loader does not understand — the **Tier-2
    /// (must-understand)** refusal. The format extends under a two-tier discipline
    /// (#156): a **Tier-1** additive-optional change (a new optional field /
    /// section) does NOT bump the version — an older reader tolerates it (serde
    /// ignores unknown fields; see `unknown_optional_field_is_tolerated_byte_identically`)
    /// and, by C1, a new optional field defaults to prior behaviour. A **Tier-2**
    /// load-bearing change (a new node type, edge semantics, or structural-axis
    /// kind) MUST bump `BLUEPRINT_FORMAT_VERSION`, so an old reader refuses here
    /// rather than silently building a different graph (C1 / C18). The
    /// per-section required-flag scheme is deferred until a real sub-version Tier-2
    /// addition needs finer granularity than a version bump.
    UnsupportedVersion { found: u32, supported: u32 },

(The BLUEPRINT_FORMAT_VERSION doc at :15-17 already states "Bumped only by a load-bearing (Tier-2) change; additive optional fields do not bump it (#156)" and needs no change — it is consistent with the settled discipline.)

The codification: C24 ledger Status block (before → after, shape only)

The C24 entry's **Remaining** list (INDEX.md:1823-1827) names "the forward-compat must-understand gate (#156)". This cycle moves #156 out of Remaining into a shipped note stating the two-tier discipline and the proof test. Exact prose is the planner's; the shape: drop #156 from Remaining, add one sentence to Status that the forward-compat discipline is codified (Tier-1 serde-default tolerance proven; Tier-2 = version/vocabulary refusal, already green) — leaving #158 and #159 in Remaining.

Components

• crates/aura-engine/src/blueprint_serde.rs — add one test (unknown_optional_field_is_tolerated_byte_identically); reword the LoadError::UnsupportedVersion doc comment. No code/behaviour change.
• docs/design/INDEX.md — C24 Status/Remaining update (codification).

Data flow

Unchanged. blueprint_from_json → serde_json::from_str::<BlueprintDoc> (ignores unknown fields) → version check → reconstruct (resolves each type_id through the injected vocabulary). The test exercises exactly this path with an unknown-key-bearing input.

Error handling

Unchanged and already complete: LoadError::{Json, UnsupportedVersion, UnknownNodeType} — typed, named, never a panic, never a silent wrong graph. The two Tier-2 arms are the must-understand refusals; the new test asserts the Tier-1 path does not error.

Testing strategy

• New (Tier-1, the gap): unknown_optional_field_is_tolerated_byte_identically — an unknown optional key at doc and primitive level is ignored; the reconstructed graph re-serializes byte-identically to the canonical form and runs bit-identically (C1).
• Existing (Tier-2, cited as grounding evidence, unchanged): unsupported_version_fails_named, unknown_node_type_fails_named.
• Full workspace gate: cargo test --workspace; lint cargo clippy --workspace --all-targets -- -D warnings.

Acceptance criteria

• (a) Tier-1 additive forward-tolerance — a blueprint JSON carrying an unknown optional field/section loads on the current loader, the unknown key does not leak into the graph (byte-identical re-serialization), and the run is bit-identical. Pinned by the new test.
• (b) Tier-2 must-understand refusal — a too-new format_version and an out-of-vocabulary node each yield a clean, named LoadError (never a panic, never a silent graph change). Already green (unsupported_version_fails_named, unknown_node_type_fails_named); unchanged by this cycle.
• The two-tier discipline is codified in the LoadError doc and the C24 ledger entry; the deferred per-section required-flag mechanism is recorded as out-of-scope on #156.
• cargo test --workspace green; cargo clippy --workspace --all-targets -- -D warnings clean.