grounding-check: a vacuous PASS (zero assumptions extracted) auto-signs the spec under /boss #13

Closed
opened 2026-06-27 13:21:58 +02:00 by Brummel · 1 comment
Owner

Found by the sibling scan while fixing #11. The same "done-signal
vacuously satisfiable by doing nothing" shape appears in the
grounding-check, where it is load-bearing because under /boss a PASS
auto-signs the spec.

Problem

Under /boss, the grounding-check PASS is the autonomous spec
signature: it commits the spec and continues to planner
(docs/pipeline.md:136-148, boss/SKILL.md:464-466).

PASS is emitted when "all assumptions ratified"
(specify/agents/grounding-check.md ~:170), and that includes the
empty case
: :226-230 explicitly emit PASS with an empty ratified table
when assumption-extraction yields an empty list. That is legitimate for a
genuinely trivial spec — but the same code path fires if the agent
under-extracts on a real spec: a no-op extraction → zero assumptions →
PASS → auto-sign. There is no floor relating the extracted-assumption
count to spec size; the only guard against a no-op extraction is the
agent's own diligence red flags ("< 2 minutes searching", "aim for
completeness").

This is the #11 shape: a success signal whose condition only checks
"nothing was found unratified," never "extraction actually happened" — so
extracting nothing reads as PASS. It is more dangerous than #11's original
form because the PASS auto-commits under /boss with no human in the loop.

Direction (not prescribing the implementation)

A structural floor so an empty extraction on a non-trivial spec cannot
silently PASS-and-sign — e.g. relate a minimum extracted-assumption count
to spec length/section count, or downgrade an empty extraction over a
non-trivial spec to BLOCK (kick it back for re-extraction) rather than
treating "extracted nothing" as "nothing to ratify." Interactive (non-boss)
runs are less exposed because the user still signs, but the same floor
helps there too.

Refs #11.

Found by the sibling scan while fixing #11. The same "done-signal vacuously satisfiable by doing nothing" shape appears in the grounding-check, where it is **load-bearing** because under `/boss` a PASS auto-signs the spec. ## Problem Under `/boss`, the grounding-check `PASS` **is** the autonomous spec signature: it commits the spec and continues to planner (`docs/pipeline.md:136-148`, `boss/SKILL.md:464-466`). `PASS` is emitted when "all assumptions ratified" (`specify/agents/grounding-check.md` ~`:170`), and that **includes the empty case**: `:226-230` explicitly emit PASS with an empty ratified table when assumption-extraction yields an empty list. That is legitimate for a genuinely trivial spec — but the *same* code path fires if the agent **under-extracts** on a real spec: a no-op extraction → zero assumptions → PASS → auto-sign. There is no floor relating the extracted-assumption count to spec size; the only guard against a no-op extraction is the agent's own diligence red flags ("< 2 minutes searching", "aim for completeness"). This is the #11 shape: a success signal whose condition only checks "nothing was found unratified," never "extraction actually happened" — so extracting nothing reads as PASS. It is more dangerous than #11's original form because the PASS auto-commits under `/boss` with no human in the loop. ## Direction (not prescribing the implementation) A structural floor so an empty extraction on a non-trivial spec cannot silently PASS-and-sign — e.g. relate a minimum extracted-assumption count to spec length/section count, or downgrade an empty extraction over a non-trivial spec to BLOCK (kick it back for re-extraction) rather than treating "extracted nothing" as "nothing to ratify." Interactive (non-boss) runs are less exposed because the user still signs, but the same floor helps there too. Refs #11.
Brummel added the bug label 2026-06-27 13:21:58 +02:00
Author
Owner

Fixed in f3eba8a (closes on push)

Step 5 emitted PASS on "all assumptions ratified", which is vacuously true on an empty extraction. Under /boss a grounding PASS is the autonomous spec signature, so an under-extraction on a real spec (zero assumptions → PASS) would auto-sign an unread spec.

An empty assumption list is now a PASS only on a demonstrably trivial spec (pure rename / doc-only / cosmetic, with the <kind> named as positive evidence). An empty extraction over a spec that adds or changes behaviour is an under-extractionBLOCK, which routes it to the human sign-off pause it should have hit anyway.

Design note: the fix lives at the source (the grounding-check agent), not the consumer. The /boss auto-sign trusts the PASS directly (boss/SKILL.md § Spec auto-sign), and a BLOCK already routes to the human pause — so tightening the agent's verdict closes the auto-sign hole with no consumer change. Tightened consistently across Step 5, the Iron Law, the trivial-spec clause, the Status protocol table, and the Common Rationalisations so a future editor cannot reopen it from one unsynced spot.

Adversarially reviewed SOUND (closes the hole at the load-bearing /boss point; genuine trivial specs still PASS — no false positive).

## Fixed in f3eba8a (closes on push) Step 5 emitted `PASS` on "all assumptions ratified", which is **vacuously true on an empty extraction**. Under `/boss` a grounding `PASS` *is* the autonomous spec signature, so an under-extraction on a real spec (zero assumptions → PASS) would auto-sign an unread spec. An empty assumption list is now a `PASS` **only on a demonstrably trivial spec** (pure rename / doc-only / cosmetic, with the `<kind>` named as positive evidence). An empty extraction over a spec that adds or changes behaviour is an *under-extraction* → `BLOCK`, which routes it to the human sign-off pause it should have hit anyway. **Design note:** the fix lives at the *source* (the grounding-check agent), not the consumer. The `/boss` auto-sign trusts the `PASS` directly (`boss/SKILL.md` § Spec auto-sign), and a `BLOCK` already routes to the human pause — so tightening the agent's verdict closes the auto-sign hole with no consumer change. Tightened consistently across Step 5, the Iron Law, the trivial-spec clause, the Status protocol table, and the Common Rationalisations so a future editor cannot reopen it from one unsynced spot. Adversarially reviewed **SOUND** (closes the hole at the load-bearing /boss point; genuine trivial specs still PASS — no false positive).
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Brummel/Skills#13