From d96d8c5cc21354f1ec0f8d29bfd6a96810207593 Mon Sep 17 00:00:00 2001 From: Brummel Date: Mon, 11 May 2026 19:12:09 +0200 Subject: [PATCH] =?UTF-8?q?draft:=20brainstorm=20grounding-check=20agent?= =?UTF-8?q?=20=E2=80=94=20discussion=20input=20for=20fresh=20brainstorm?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Discussion-draft, not an approved spec. Captures today's findings: - Repeated BLOCKEDs on iter 23.4 family were a spec-defect signal, not a plan-defect signal. - Today's spec-verification (brainstorm Step 7 + 8) is purely linguistic — no code-grounded check of load-bearing assumptions. - Spec-23's 'work the same way as 22b.3' half-sentence was the exact load-bearing assumption that broke. Three preps spent patching what should have been caught pre-approval. Proposes a new agent that, with fresh context between brainstorm Step 6 and Step 7, extracts load-bearing assumptions from the spec draft and searches for an existing green fixture ratifying each. Default on miss: spec discarded, blocked idea moved to roadmap with 'depends on: '. Eight open design questions left for the next session's brainstorm to answer. --- ...-05-11-brainstorm-grounding-check-draft.md | 225 ++++++++++++++++++ 1 file changed, 225 insertions(+) create mode 100644 docs/specs/2026-05-11-brainstorm-grounding-check-draft.md diff --git a/docs/specs/2026-05-11-brainstorm-grounding-check-draft.md b/docs/specs/2026-05-11-brainstorm-grounding-check-draft.md new file mode 100644 index 0000000..14a5626 --- /dev/null +++ b/docs/specs/2026-05-11-brainstorm-grounding-check-draft.md @@ -0,0 +1,225 @@ +# Brainstorm Grounding Check — Discussion Draft + +**Date:** 2026-05-11 +**Status:** Discussion draft — input for a fresh brainstorm in a future session. NOT an approved spec. +**Authors:** Brummel (orchestrator) + Claude + +## Why this draft exists + +On 2026-05-11 the iter 23.4 family (Eq/Ord prelude free fns) BLOCKED +four times in a row on architectural prerequisites the parent spec +had not anticipated. Three prep iters were merged on main (prep, +prep2, prep3) before a fourth gap surfaced and the user stopped the +chain: "Wäre es nicht sinnvoll, es einmal richtig zu machen, anstatt +auf drei Pflaster noch ein viertes zu kleben?" + +The repository was reset to the pre-session HEAD (`1625264`) and the +day's commits discarded. This draft captures the diagnosis and the +proposed pipeline change so a fresh brainstorm can take it as input. + +## What we learned + +### 1. Repeated BLOCKEDs are a spec-defect signal, not a plan-defect signal + +When the same iter family blocks twice or more on related +architectural prerequisites, the spec under-estimated the +implementation surface. Patching with another prep iter accumulates +band-aids on what is really one missing architectural commitment. +The right response is to amend the spec; the wrong response is to +continue patching. + +Captured as user memory (`feedback_spec_over_plan_patches.md`). + +### 2. Spec verification today is purely linguistic + +The `brainstorm` skill's Step 7 (self-review) and Step 8 +(user-approval gate) are both linguistic: + +- **Step 7 (self-review)** is a four-point checklist: placeholder + scan, internal consistency, scope check, ambiguity check. None of + these touches code. +- **Step 8 (user-approval)** asks the user to read and approve. In + practice the user does not have time for forensic code-reading. + The spec gets approved on linguistic merits. + +Nobody opens the codebase to check whether spec assertions about +compiler behaviour are actually true. The architect agent +(`ailang-architect`) does exactly this kind of code-vs-doc check — +but only at milestone close, far too late. + +### 3. Spec-23 illustrates the failure mode + +Forensic reading of `docs/specs/2026-05-10-23-eq-ord-prelude.md` +(created 2026-05-10 20:35 in commit `03b6d15`, approved 21:04 in +commit `9c3d290`, 29 minutes apart): + +The Data-flow section contains verbatim: + +> "Polymorphic helpers (`fn cmp_max : forall a. Ord a => (a, a) -> a`) +> work the same way: the typeclass-elaboration in 22b.3 inserts the +> right monomorphised `compare__T` symbol per use site." + +This was the load-bearing assumption. It is wrong: the 22b.3 path +specialises class-method calls, not polymorphic free-fn calls. The +assumption sat as a half-sentence in the spec, never explicitly +flagged as an assumption, never verified against any existing green +test. Three iter 23.4 BLOCKEDs on 2026-05-11 surfaced it the hard way. + +The linguistic self-review caught nothing (text is internally +consistent and unambiguous). The user-review caught nothing (the +half-sentence reads plausible). Only a code-grounded prerequisite +check would have caught this. + +## Proposed addition to the pipeline + +A new agent, dispatched between the spec draft and the user-approval +gate. + +### Role + +Read the spec draft with fresh context (no sunk-cost from the +brainstorm session). Extract its load-bearing assumptions — both +explicit and implicit. For each assumption, search the existing +codebase for a green fixture that ratifies it. Report status per +assumption. + +### Why fresh context + +The brainstorm-driven orchestrator has spent 30+ minutes building +the spec and is biased toward shipping it. A fresh agent has no such +bias. The asymmetry mirrors how `ailang-architect` reviews +code-vs-DESIGN.md drift today: without loyalty to the implementation +that produced the drift. + +### What counts as ratification + +A committed, currently-green test. Not "code that does X exists" +(open to interpretation), not "I recall this pattern works" (memory +bias), not "another spec said it works" (transitive uncertainty). A +green `cargo test` is concrete: it ran today, the symbol it pins +exists, the assertion it makes holds. + +Invariant in the planning phase: the workspace has no red fixtures. +If a red fixture exists, the planning phase is not the right phase — +something else needs fixing first. + +### Pipeline placement + +``` +brainstorm Step 5/6 (spec drafted) + → NEW: grounding-check agent (fresh context, read-only) + → all assumptions ratified + → Step 7 self-review, Step 8 user-approval-gate + → at least one assumption unratified + → spec draft discarded; Boss adds roadmap entry with the + missing prerequisite as `depends on:`. The blocked idea + waits until the prerequisite ships. + → planner ... +``` + +The discard-and-add-to-roadmap default (Boss-overridable) rests on +the observation that prerequisites can take days or weeks to ship. +By then, the original spec's framing is likely stale; re-brainstorming +on the shipped prerequisite is more honest than dusting off the old +draft. + +## Open design questions (to brainstorm in the next session) + +The following decisions are intentionally NOT made in this draft. +They are the Q&A material for the next session's brainstorm. + +1. **How does the agent define an "assumption"?** + Strictness scale: + - (a) every spec assertion about compiler behaviour; + - (b) only assertions that extend an existing mechanism beyond its + current scope; + - (c) only assertions the agent's own extraction step flags. + Each has different false-positive / false-negative tradeoffs. + +2. **What format does the report take?** + Per-assumption: claim, search query / test names checked, status + (ratified / unratified). Aggregate status at the top. Probably + mirrors the existing spec/quality reviewer output formats. + +3. **Where does the agent live in the skill repository?** + Frontmatter, reading list, iron-law section, output format. + Naming convention candidates: `ailang-grounding-check`, + `ailang-spec-premise-auditor`, `ailang-prerequisite-check`. + +4. **What changes in `skills/brainstorm/SKILL.md`?** + New step between current 6 and 7. Hard-gate language. Failure-mode + procedure (the spec-discard / roadmap-entry default). + +5. **What changes in `docs/roadmap.md`?** + How are deferred ideas annotated. The existing `depends on:` line + type may already suffice; or a new "blocked-on-prerequisite" + status label is needed. + +6. **What's the test-naming convention that makes ratification + searches feasible?** + Today tests are scattered. If the agent has to grep for "is there + a green test ratifying mono-specialisation-of-polymorphic-free-fns?", + the answer needs to come out in a finite query. Either we + standardise test-naming so the search is mechanical, or we accept + that the agent makes a judgement call and document its heuristic. + +7. **What happens to the discarded brainstorm artefacts?** + Brainstorm conversation, partial spec draft. Do they go to a + `docs/scratchpad/` ledger, or are they lost? The cost of losing + them is "the same idea gets re-brainstormed cold in 3 months"; the + cost of keeping them is "stale drafts linger and bias future + thinking". The user has already indicated (a) is the preferred + default — Roadmap-only, drafts discarded — but the question stays + open for the next session to ratify formally. + +8. **Does this agent run on existing specs too?** + Spec-23 still exists at `docs/specs/2026-05-10-23-eq-ord-prelude.md`, + approved but mid-flight (iters 23.1–23.3 shipped). Should the new + agent be run against it retroactively? Or do existing specs + grandfather past the new check? + +## Adjacent observations (not in scope for this draft) + +Recorded for future skill-system work, not part of the grounding-check +brainstorm itself: + +- **Branch-hygiene papercut.** The `ailang-implement-orchestrator` + agent ends each run with the worktree on `iter/`. The Boss + merge step needs `git switch main` before any further commit; + otherwise commits land on the iter branch by mistake. This happened + three times on 2026-05-11. Worth a one-line addition to + `skills/implement/SKILL.md` Step 3 prelude. + +- **Plan-recon dispatches only after spec-approval.** Today + `ailang-plan-recon` maps files for the planner but never sees the + spec before it is approved. The grounding-check agent could + conceivably re-use plan-recon's read-only file-survey discipline. + Worth considering during the next brainstorm — same recon agent, + earlier dispatch with a different brief. + +## Memory entries already captured + +These should NOT be re-derived in the next session — they are +persistent and live outside the project tree at +`~/.claude/projects/-home-brummel-dev-ailang/memory/`: + +- `feedback_spec_over_plan_patches.md` — Repeated BLOCKEDs = spec + defect, not plan defect. +- `feedback_plain_german_no_jargon.md` — Chat in plain German, no + self-coined Anglo-Deutsch terms when writing for the user. + +## How to start the next session + +1. Read this draft as the input. +2. Invoke the `brainstorm` skill with the topic "brainstorm + grounding-check agent". +3. The skill does its Q&A; this draft pre-answers the WHY and + pre-states the open questions, so the Q&A can focus on the + eight open design questions above. +4. The output is a proper spec at + `docs/specs/YYYY-MM-DD-brainstorm-grounding-check.md`, then a + plan, then implementation — standard pipeline. +5. Note the bootstrap irony: the new agent will not yet exist when + the brainstorm produces a spec for the new agent itself. The + first run of the new check happens on the second spec written + after it ships, not on its own spec.