Brummel/Skills
1
0
Fork 0
Code Issues 3 Pull Requests Actions Packages Projects Releases Wiki Activity

Compound-hallucination failure mode: spec bytes treated as authoritative instead of hypothesis #1

New Issue
Closed
opened 2026-05-29 14:02:11 +02:00 by Brummel · 0 comments
Brummel commented 2026-05-29 14:02:11 +02:00
Owner
Copy Link
Copy Source

What surfaced

During AILang's raw-buf.2 iteration (the first real polymorphic
kernel-tier module after the kernel-extension-mechanics milestone),
the spec at docs/specs/0054-raw-buf.md shipped through brainstorm

  • grounding-check + planner + plan-recon all green, then BLOCKED at
    implement with a chain of three surface-parse defects + one
    structural defect that none of the upstream skills had caught. Full
    diagnosis in the AILang BLOCKED.md from the iteration (commit
    647121c.. working tree).

Root cause is not "the spec was wrong" in isolation — it is that
every defensive line in the skill pipeline treated the spec's
Form-A code blocks as authoritative truth instead of as hypothesis
to verify.

The spec inherited aspirational Form-A from
design/models/0007-kernel-extensions.md (a models/ whitepaper,
per the AILang design/INDEX.md convention "evolves / onboarding"
not "test-pinned / stable"). No skill in the pipeline questioned
whether the Form-A snippets in the model actually parsed against
the current surface. Brainstorm copied them into the spec. Grounding-
check PASSed. Planner inlined them verbatim into a plan task body.
Plan-recon validated paths but not surface-parseability. Only the
implementer hit them at execution and surfaced the panic.

This is the compound-hallucination pattern: an earlier model
writes an aspirational construct into a checked-in document, a
later model reads that document and treats the construct as
canonical because of its design/ location, and no validation step
between the two enforces the model/contract distinction.

The irony: AILang exists to harden LLM authoring against
hallucination. The first real spec extension after the kernel-
extension-mechanics milestone demonstrated that the hardening
mechanisms (content-addressing, local-reasoning, test-pinned
contracts) only work if the skill pipeline consults them. None of
the four defense lines did.

The four defense lines and where each failed

  1. brainstorm Step 4 (feature-acceptance + concrete code).
    Already requires concrete code as the criterion's empirical
    evidence. Does NOT require that code to have been parsed
    against the live tool. A trivial ail parse <code> (or whatever
    the project's surface tool is) on each block before PASS would
    have caught (storage-tag), the implicit forall, and the
    (RawBuf a) / (con RawBuf a) shorthand confusion.

  2. grounding-check. Searches for green tests ratifying the
    spec's assumptions about codebase behaviour. Does NOT extract
    the spec's own code blocks and validate them against the
    corresponding parser. Complementary defense missing.

  3. planner Step 2 (plan-recon). Found one workspace-pin
    site (workspace_pin.rs) but missed the second pin in
    e2e.rs::workspace_lists_imported_modules — same invariant,
    different test file. The "enumerate hash-pin blast radius
    across ALL crates/*/tests/" discipline is a project-memory
    item but is not enforced in plan-recon's mandatory checks.

  4. planner Step 5 self-review. Procedural checks
    (placeholder scan, type consistency, compile-gate vs deferred-
    caller, filter-string resolution). Does NOT include: "verbatim
    bytes the plan inlines for the implementer — have they been
    parsed against the live tool?"

The implementer behaved correctly (respected the spec, surfaced
BLOCKED instead of patching). That is the only line that worked —
and only because feedback_dont_adapt_tests_to_bugs discipline is
sharp.

Proposed fixes

All five for the next time a substantive spec extension is on the
roadmap. None of these is AILang-specific; the pattern recurs
anywhere a models/ (or RFCs/ or proposals/) directory carries
aspirational code that a later spec is expected to lift.

Fix 1 — brainstorm Step 4: parse-every-block gate

Tighten the feature-acceptance step to:

Every code block in the spec that has a parser-or-tool behind it
(the project's surface, .json against a schema, IR against
clang, configuration against its loader, etc.) must have been
run through that parser and produced a clean parse before PASS.
The parse-trace goes into the brainstorm chat for the record.

Verifiable: if a Run block appears in the brainstorm chat showing
the parse output, the gate fired. If not, the gate was skipped.

Fix 2 — grounding-check: extract-and-validate code blocks

Add a third pass to the agent: extract every fenced code block
from the spec, infer its language (the fence label or convention),
dispatch to the appropriate parser, report any parse failures as
BLOCK. Complementary to the existing test-ratification search; no
overlap.

This needs a per-project mapping of "fence label → parser command"
to be configurable (AILang has .ail, .ail.json, .ll; another
project has .toml, .yaml, .proto; etc.). The mapping lives
in the project profile.

Fix 3 — plan-recon: hash-pin blast radius enumeration

Promote the existing project-memory item
("feedback_hash_pin_blast_radius_audit" in AILang's case) to a
plan-recon mandatory check. The agent enumerates EVERY *.rs (or
language-appropriate equivalent) file under crates/*/tests/
that grep-matches the spec's load-bearing constant, fixture name,
or assertion shape. The dispatch prompt template should embed the
memory directly so the discipline does not depend on the
orchestrator remembering to reinforce it.

Fix 4 — planner Step 5: parse-the-bytes-i-inline gate

Add a ninth self-review check:

For every verbatim code body the plan inlines into a task step,
the planner has run that body through the corresponding parser
on the planner's own machine before commit. The parse-trace
belongs in the plan's self-review section.

This is the last defensive line before implementer dispatch. If
it fails, the plan is wrong, regardless of what brainstorm or
grounding-check said.

Fix 5 — design/models/ aspirational marker

Adopt a convention (recommendation to projects, not enforced by
the plugin) where files under a "models" / "RFCs" / "proposals"
directory carry an explicit frontmatter marker:

---
status: aspirational
validated-against: <commit-sha or "no validation">
---

When a brainstorm session lifts content from such a file, the
markers feed forward as a flag: "this paragraph was lifted from
an aspirational source; have you validated it against current
code at lift-time?"

design/INDEX.md (or the equivalent design ledger) should
distinguish "model" rows from "contract" rows in a way the
skill agents can mechanically read.

Scope

The five fixes are independent; each can ship in its own
iteration. The skill plugin is shared across projects, so a fix
here benefits every consumer.

Priority order (substantive impact, not effort):

  1. Fix 1 (brainstorm parse gate) — biggest single defense line.
  2. Fix 4 (planner self-review parse gate) — last-mile catch.
  3. Fix 2 (grounding-check code-block validation) — complementary.
  4. Fix 3 (plan-recon hash-pin enumeration) — recurring defect family.
  5. Fix 5 (models/ aspirational marker) — convention-level, not
    mechanism-level.

Out of scope

Mechanically auto-running every code block in every skill is not
the goal. The goal is that the orchestrator and the dispatched
agents both fail closed when they have not validated a load-
bearing assumption against the live tool. The validation can be a
shell-out, a static lookup, or an explicit "no parser exists for
this fence label, skip"-and-document. What is unacceptable is the
current state: silent assumption-as-truth.

Why this is a feature not an idea

The diagnosis is concrete. The five fixes are named with specific
skill files and step numbers. The patterns are recurring (the same
shape will reappear on the next substantive spec extension of any
project that uses the plugin). This earns its place above the
idea threshold.

References

  • AILang BLOCKED.md from iteration raw-buf.2 (uncommitted; see
    the conversation transcript at file rotation if needed).
  • AILang feedback_specs_need_concrete_code.md — the existing
    memory that points at the same pattern from a different angle.
  • AILang feedback_hash_pin_blast_radius_audit.md — the existing
    memory that captures Fix 3's content from a recent recurrence.
  • AILang feedback_grounding_check_misses_insource_tests.md — a
    prior known-gap in grounding-check, same family of "agent does
    not consult the full surface".

Triage (2026-05-29)

Diagnosis re-verified against the current plugin tree. Verdict per
fix:

Fix Verdict Anchor in current tree
1 — brainstorm parse-gate accurate brainstorm/SKILL.md Step 4 (:112-147) requires concrete code but never a parse; Step 7 self-review check 5 (:224) checks code presence, not parseability.
2 — grounding-check code-block validation accurate brainstorm/agents/grounding-check.md ratifies assumptions about codebase behaviour via green tests; no pass extracts/parses the spec's own fenced blocks. Genuinely complementary.
3 — plan-recon hash-pin blast radius partially stale planner/agents/plan-recon.md:120-127 already grew a "Non-compile-checked sites" sweep covering drift-pins/fixtures — but :211-215 gate it behind signature/variant/removal scopes. The raw-buf.2 failure was a module addition that shifted a workspace-listing hash (no signature change), so the sweep was legitimately omittable. Fix 3 narrows to: make the hash-pin enumeration mandatory for content-pin-perturbing scopes too.
4 — planner self-review parse-gate accurate planner/SKILL.md Step 5 (:162-225) has 8 checks today (the issue's "ninth" is consistent); all are textual/structural, none runs inlined bytes through a parser.
5 — design/models aspirational marker accurate design_models slot exists (templates/project-profile.yml:11, schema :16) but no status/validated-against frontmatter convention, and brainstorm does not flag lifts from it.

Cross-cutting prerequisite. Fix 1 and Fix 2 both need the same
new profile slot — a fence-label → parser-command mapping — which
does not exist in docs/profile-schema.md today. Landing it once as
a prerequisite avoids Fix 1 and Fix 2 each defining the slot.

Per ~/dev/CLAUDE.md ("each fix is independent; each can ship in its
own iteration"), kept as a single tracking issue with a checklist
rather than split into a milestone. Labelled feature.

Checklist (author's priority order)

  • P0 — profile slot: add fence-label → parser-command
    mapping to docs/profile-schema.md + templates/project-profile.yml
    (prerequisite for Fix 1 and Fix 2). Done on branch
    feat/spec-validation-slot (4a2c70b).
  • Fix 1 — brainstorm parse gate: every spec code block with a
    parser behind it must parse clean before PASS; parse-trace into
    the brainstorm chat. Landed as Step-7 self-review check #6 (not
    Step 4 — the spec file exists only after Step 6). Branch
    feat/spec-validation-slot (e1d3336).
  • Fix 4 — planner self-review parse gate: 9th Step-5 check —
    every verbatim surface-language body the plan inlines is parsed
    against the live tool before hand-off; trace into the planner
    session. Branch feat/spec-validation-slot (305d797).
  • Fix 2 — grounding-check code-block validation: third pass
    extracts fenced blocks, runs the mapped parser, reports parse
    failures as BLOCK. Complementary to test-ratification and
    independent of Fix 1 (fresh-context second line). Branch
    feat/spec-validation-slot (fc36647).
  • Fix 3 — plan-recon hash-pin blast radius: non-compile-checked
    sweep now mandatory for content-pin-perturbing scopes (not only
    signature/variant/removal); enumerate EVERY matching test file.
    Embedded in plan-recon.md itself (no new slot; leans on CLAUDE.md
    lockstep pairs). Branch feat/spec-validation-slot (e019d04).
  • Fix 5 — design/models aspirational marker: recommended
    status: aspirational / validated-against: frontmatter
    convention, consumed by brainstorm Step 1 + Step 4 as a
    lift-time flag (upstream signal to the Fix-1 gate). Branch
    feat/spec-validation-slot (56ef25c).
## What surfaced During AILang's `raw-buf.2` iteration (the first real polymorphic kernel-tier module after the kernel-extension-mechanics milestone), the spec at `docs/specs/0054-raw-buf.md` shipped through brainstorm + grounding-check + planner + plan-recon all green, then BLOCKED at implement with a chain of three surface-parse defects + one structural defect that none of the upstream skills had caught. Full diagnosis in the AILang `BLOCKED.md` from the iteration (commit `647121c..` working tree). Root cause is not "the spec was wrong" in isolation — it is that **every defensive line in the skill pipeline treated the spec's Form-A code blocks as authoritative truth instead of as hypothesis to verify**. The spec inherited aspirational Form-A from `design/models/0007-kernel-extensions.md` (a `models/` whitepaper, per the AILang `design/INDEX.md` convention "evolves / onboarding" not "test-pinned / stable"). No skill in the pipeline questioned whether the Form-A snippets in the model actually parsed against the current surface. Brainstorm copied them into the spec. Grounding- check PASSed. Planner inlined them verbatim into a plan task body. Plan-recon validated paths but not surface-parseability. Only the implementer hit them at execution and surfaced the panic. This is the **compound-hallucination pattern**: an earlier model writes an aspirational construct into a checked-in document, a later model reads that document and treats the construct as canonical because of its `design/` location, and no validation step between the two enforces the model/contract distinction. The irony: AILang exists to harden LLM authoring against hallucination. The first real spec extension after the kernel- extension-mechanics milestone demonstrated that the hardening mechanisms (content-addressing, local-reasoning, test-pinned contracts) only work *if the skill pipeline consults them*. None of the four defense lines did. ## The four defense lines and where each failed 1. **`brainstorm` Step 4 (feature-acceptance + concrete code).** Already requires concrete code as the criterion's empirical evidence. Does NOT require that code to have been parsed against the live tool. A trivial `ail parse <code>` (or whatever the project's surface tool is) on each block before PASS would have caught `(storage-tag)`, the implicit `forall`, and the `(RawBuf a)` / `(con RawBuf a)` shorthand confusion. 2. **`grounding-check`.** Searches for green tests ratifying the spec's *assumptions about codebase behaviour*. Does NOT extract the spec's own code blocks and validate them against the corresponding parser. Complementary defense missing. 3. **`planner` Step 2 (`plan-recon`).** Found one workspace-pin site (`workspace_pin.rs`) but missed the second pin in `e2e.rs::workspace_lists_imported_modules` — same invariant, different test file. The "enumerate hash-pin blast radius across ALL `crates/*/tests/`" discipline is a project-memory item but is not enforced in plan-recon's mandatory checks. 4. **`planner` Step 5 self-review.** Procedural checks (placeholder scan, type consistency, compile-gate vs deferred- caller, filter-string resolution). Does NOT include: "verbatim bytes the plan inlines for the implementer — have they been parsed against the live tool?" The implementer behaved correctly (respected the spec, surfaced BLOCKED instead of patching). That is the only line that worked — and only because `feedback_dont_adapt_tests_to_bugs` discipline is sharp. ## Proposed fixes All five for the next time a substantive spec extension is on the roadmap. None of these is AILang-specific; the pattern recurs anywhere a `models/` (or `RFCs/` or `proposals/`) directory carries aspirational code that a later spec is expected to lift. ### Fix 1 — brainstorm Step 4: parse-every-block gate Tighten the feature-acceptance step to: > Every code block in the spec that has a parser-or-tool behind it > (the project's surface, `.json` against a schema, IR against > `clang`, configuration against its loader, etc.) must have been > run through that parser and produced a clean parse before PASS. > The parse-trace goes into the brainstorm chat for the record. Verifiable: if a Run block appears in the brainstorm chat showing the parse output, the gate fired. If not, the gate was skipped. ### Fix 2 — grounding-check: extract-and-validate code blocks Add a third pass to the agent: extract every fenced code block from the spec, infer its language (the fence label or convention), dispatch to the appropriate parser, report any parse failures as BLOCK. Complementary to the existing test-ratification search; no overlap. This needs a per-project mapping of "fence label → parser command" to be configurable (AILang has `.ail`, `.ail.json`, `.ll`; another project has `.toml`, `.yaml`, `.proto`; etc.). The mapping lives in the project profile. ### Fix 3 — plan-recon: hash-pin blast radius enumeration Promote the existing project-memory item ("`feedback_hash_pin_blast_radius_audit`" in AILang's case) to a plan-recon mandatory check. The agent enumerates EVERY `*.rs` (or language-appropriate equivalent) file under `crates/*/tests/` that grep-matches the spec's load-bearing constant, fixture name, or assertion shape. The dispatch prompt template should embed the memory directly so the discipline does not depend on the orchestrator remembering to reinforce it. ### Fix 4 — planner Step 5: parse-the-bytes-i-inline gate Add a ninth self-review check: > For every verbatim code body the plan inlines into a task step, > the planner has run that body through the corresponding parser > on the planner's own machine before commit. The parse-trace > belongs in the plan's self-review section. This is the last defensive line before implementer dispatch. If it fails, the plan is wrong, regardless of what brainstorm or grounding-check said. ### Fix 5 — `design/models/` aspirational marker Adopt a convention (recommendation to projects, not enforced by the plugin) where files under a "models" / "RFCs" / "proposals" directory carry an explicit frontmatter marker: ```yaml --- status: aspirational validated-against: <commit-sha or "no validation"> --- ``` When a brainstorm session lifts content from such a file, the markers feed forward as a flag: "this paragraph was lifted from an aspirational source; have you validated it against current code at lift-time?" `design/INDEX.md` (or the equivalent design ledger) should distinguish "model" rows from "contract" rows in a way the skill agents can mechanically read. ## Scope The five fixes are independent; each can ship in its own iteration. The skill plugin is shared across projects, so a fix here benefits every consumer. Priority order (substantive impact, not effort): 1. Fix 1 (brainstorm parse gate) — biggest single defense line. 2. Fix 4 (planner self-review parse gate) — last-mile catch. 3. Fix 2 (grounding-check code-block validation) — complementary. 4. Fix 3 (plan-recon hash-pin enumeration) — recurring defect family. 5. Fix 5 (models/ aspirational marker) — convention-level, not mechanism-level. ## Out of scope Mechanically auto-running every code block in every skill is not the goal. The goal is that *the orchestrator and the dispatched agents both fail closed* when they have not validated a load- bearing assumption against the live tool. The validation can be a shell-out, a static lookup, or an explicit "no parser exists for this fence label, skip"-and-document. What is unacceptable is the current state: silent assumption-as-truth. ## Why this is a `feature` not an `idea` The diagnosis is concrete. The five fixes are named with specific skill files and step numbers. The patterns are recurring (the same shape will reappear on the next substantive spec extension of any project that uses the plugin). This earns its place above the `idea` threshold. ## References - AILang `BLOCKED.md` from iteration raw-buf.2 (uncommitted; see the conversation transcript at file rotation if needed). - AILang `feedback_specs_need_concrete_code.md` — the existing memory that points at the same pattern from a different angle. - AILang `feedback_hash_pin_blast_radius_audit.md` — the existing memory that captures Fix 3's content from a recent recurrence. - AILang `feedback_grounding_check_misses_insource_tests.md` — a prior known-gap in grounding-check, same family of "agent does not consult the full surface". --- ## Triage (2026-05-29) Diagnosis re-verified against the current plugin tree. Verdict per fix: | Fix | Verdict | Anchor in current tree | |-----|---------|------------------------| | 1 — brainstorm parse-gate | **accurate** | `brainstorm/SKILL.md` Step 4 (`:112-147`) requires concrete code but never a parse; Step 7 self-review check 5 (`:224`) checks code *presence*, not parseability. | | 2 — grounding-check code-block validation | **accurate** | `brainstorm/agents/grounding-check.md` ratifies *assumptions about codebase behaviour* via green tests; no pass extracts/parses the spec's own fenced blocks. Genuinely complementary. | | 3 — plan-recon hash-pin blast radius | **partially stale** | `planner/agents/plan-recon.md:120-127` already grew a "Non-compile-checked sites" sweep covering drift-pins/fixtures — but `:211-215` gate it behind signature/variant/removal scopes. The `raw-buf.2` failure was a *module addition that shifted a workspace-listing hash* (no signature change), so the sweep was legitimately omittable. Fix 3 narrows to: make the hash-pin enumeration mandatory for content-pin-perturbing scopes too. | | 4 — planner self-review parse-gate | **accurate** | `planner/SKILL.md` Step 5 (`:162-225`) has 8 checks today (the issue's "ninth" is consistent); all are textual/structural, none runs inlined bytes through a parser. | | 5 — design/models aspirational marker | **accurate** | `design_models` slot exists (`templates/project-profile.yml:11`, schema `:16`) but no `status`/`validated-against` frontmatter convention, and brainstorm does not flag lifts from it. | **Cross-cutting prerequisite.** Fix 1 and Fix 2 both need the same new profile slot — a `fence-label → parser-command` mapping — which does not exist in `docs/profile-schema.md` today. Landing it once as a prerequisite avoids Fix 1 and Fix 2 each defining the slot. Per ~/dev/CLAUDE.md ("each fix is independent; each can ship in its own iteration"), kept as a single tracking issue with a checklist rather than split into a milestone. Labelled `feature`. ### Checklist (author's priority order) - [x] **P0 — profile slot:** add `fence-label → parser-command` mapping to `docs/profile-schema.md` + `templates/project-profile.yml` (prerequisite for Fix 1 and Fix 2). Done on branch `feat/spec-validation-slot` (`4a2c70b`). - [x] **Fix 1 — brainstorm parse gate:** every spec code block with a parser behind it must parse clean before PASS; parse-trace into the brainstorm chat. Landed as Step-7 self-review check #6 (not Step 4 — the spec file exists only after Step 6). Branch `feat/spec-validation-slot` (`e1d3336`). - [x] **Fix 4 — planner self-review parse gate:** 9th Step-5 check — every verbatim surface-language body the plan inlines is parsed against the live tool before hand-off; trace into the planner session. Branch `feat/spec-validation-slot` (`305d797`). - [x] **Fix 2 — grounding-check code-block validation:** third pass extracts fenced blocks, runs the mapped parser, reports parse failures as BLOCK. Complementary to test-ratification and independent of Fix 1 (fresh-context second line). Branch `feat/spec-validation-slot` (`fc36647`). - [x] **Fix 3 — plan-recon hash-pin blast radius:** non-compile-checked sweep now mandatory for content-pin-perturbing scopes (not only signature/variant/removal); enumerate EVERY matching test file. Embedded in plan-recon.md itself (no new slot; leans on CLAUDE.md lockstep pairs). Branch `feat/spec-validation-slot` (`e019d04`). - [x] **Fix 5 — design/models aspirational marker:** recommended `status: aspirational` / `validated-against:` frontmatter convention, consumed by brainstorm Step 1 + Step 4 as a lift-time flag (upstream signal to the Fix-1 gate). Branch `feat/spec-validation-slot` (`56ef25c`).
Brummel added the feature label 2026-05-29 14:12:35 +02:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
BLOCKER
Kernfunktionalität kaputt; preemptet jede andere Auswahl
 bug
beobachtbares Fehlverhalten
 feature
substantielle Erweiterung, kein voller Spec nötig
 idea
kann gestrichen werden, kein Commitment
No Label feature
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
claude (Claude) Brummel
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Brummel/Skills#1
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?