feat(specify): bound the auto-sign panel with an editorial self-correction loop
The spec-skeptic auto-sign panel was one-shot: under /boss with spec_auto_sign on, any panel BLOCK fell straight back to the human sign-off pause. Because the panel is adversarial-by-design and rarely returns a unanimous SOUND, /boss escalated to the user even for defects the orchestrator could mechanically repair. Partition the five lenses by what a BLOCK means and run Step 6 as a bounded loop: - Editorial (criterion, ambiguity, plan-readiness) — the spec is under-articulated; the orchestrator self-corrects in place, re-runs Step 4 + Step 5, and re-dispatches all five lenses. - Design (scope-fork, grounding) — the ground is not settled; escalate, never self-correct. Re-loop limit: <= 2 rounds, the 3rd unresolved editorial BLOCK escalates (hard-coded N, matching implement-orchestrator's idiom). INFRA_ERROR and exhausted budget escalate too. Auto-sign still requires a fresh unanimous SOUND on a green objective gate. Re-dispatching all five lenses every round is the backstop: an editorial repair that launders an unsourced design decision is caught by the re-run scope-fork / grounding jurors and escalates. A criterion repair is bounded to supplying the worked evidence the criterion already demands — not re-judging acceptance (architect drift item). specify/SKILL.md owns the loop; spec-skeptic.md keeps the juror lens-agnostic; boss/SKILL.md, README.md, docs/profile-schema.md and docs/pipeline.md are synced to the new semantics. closes #6
This commit is contained in:
@@ -149,14 +149,21 @@ never on its own confidence. Signing requires two stages to clear:
|
||||
`grounding`, `scope-fork`, `ambiguity`, `plan-readiness`), each
|
||||
trying to *refute* the spec. All five must return `SOUND`.
|
||||
|
||||
A single juror `BLOCK`, or any objective gate not green, falls back to
|
||||
the human sign-off pause the spec would have hit anyway. On a clean
|
||||
sign the orchestrator commits the spec (`(boss-signed)` in the
|
||||
subject), sends a mandatory informational notify that names the signed
|
||||
capability and invites a veto, and continues to `planner` without
|
||||
stopping. A later veto is a forward correction, never a history
|
||||
rewind. See `specify/SKILL.md` Step 6, `specify/agents/spec-skeptic.md`,
|
||||
and `boss/SKILL.md` §"Spec auto-sign".
|
||||
A `BLOCK` is never signed over. The five lenses split by what a `BLOCK`
|
||||
means: an *editorial* one (`criterion`, `ambiguity`, `plan-readiness` —
|
||||
the spec is under-articulated) the orchestrator repairs in a bounded
|
||||
loop (edit, re-run the objective gates, re-dispatch all five lenses, up
|
||||
to 2 rounds); a *design* one (`scope-fork`, `grounding` — the ground is
|
||||
not settled), an `INFRA_ERROR`, any objective gate not green, or an
|
||||
exhausted budget falls back to the human sign-off pause the spec would
|
||||
have hit anyway. Re-dispatching all five lenses every round is the
|
||||
backstop that stops an editorial repair from quietly settling a design
|
||||
question. On a clean sign the orchestrator commits the spec
|
||||
(`(boss-signed)` in the subject), sends a mandatory informational notify
|
||||
that names the signed capability and invites a veto, and continues to
|
||||
`planner` without stopping. A later veto is a forward correction, never
|
||||
a history rewind. See `specify/SKILL.md` Step 6,
|
||||
`specify/agents/spec-skeptic.md`, and `boss/SKILL.md` §"Spec auto-sign".
|
||||
|
||||
## Install
|
||||
|
||||
|
||||
+17
-7
@@ -354,11 +354,21 @@ and defined there. The boss-side contract is just this:
|
||||
- **The gate is not the orchestrator's judgement.** Signing requires
|
||||
every objective gate green (precondition, parse, a `grounding-check`
|
||||
`PASS` that needed no human override) AND a unanimous five-lens
|
||||
`spec-skeptic` panel returning `SOUND`. A single `BLOCK` — or any
|
||||
objective gate not green — routes the spec to the human sign-off
|
||||
pause it would have hit anyway. "The spec looks good to me" is never
|
||||
a signature; that confidence is the exact signal the gate refuses to
|
||||
trust.
|
||||
`spec-skeptic` panel returning `SOUND`. "The spec looks good to me" is
|
||||
never a signature; that confidence is the exact signal the gate
|
||||
refuses to trust.
|
||||
- **A `BLOCK` is repaired or escalated, never signed over.** The gate
|
||||
splits the five lenses by what a `BLOCK` means. An *editorial* `BLOCK`
|
||||
(`criterion` / `ambiguity` / `plan-readiness` — the spec is
|
||||
under-articulated) the orchestrator may repair in a bounded loop:
|
||||
edit, re-run the objective gates, re-dispatch all five lenses, up to
|
||||
2 rounds. A *design* `BLOCK` (`scope-fork` / `grounding` — the ground
|
||||
is not settled), an `INFRA_ERROR`, any objective gate not green, or an
|
||||
exhausted 2-round budget routes the spec to the human sign-off pause
|
||||
it would have hit anyway. The orchestrator never settles a design
|
||||
question in the user's place; the full re-panel each round is what
|
||||
stops an editorial repair from laundering one in. The gate owns this
|
||||
loop (`../specify/SKILL.md` Step 6) — boss only enforces the notify.
|
||||
- **On a clean sign, do not stop.** Commit the spec (the auto-sign
|
||||
commit subject carries `(boss-signed)` so the act is auditable in
|
||||
history), fire the auto-sign notify, and continue to `planner` in
|
||||
@@ -395,7 +405,7 @@ and defined there. The boss-side contract is just this:
|
||||
| "The issue is exhaustive but it's a new cycle, so bounce to the user before `specify`" | `specify` direct-entry is bounded and autonomously dispatchable — it is NOT the high-context `brainstorm` cycle that trigger 4 reserves for the user. Dispatch it; it will pause at its own user-review gate for sign-off. The pre-dispatch bounce is for an *open* design that needs discovery, not for a settled one that needs only production. |
|
||||
| "`tdd` is opt-in / profile-gated, so it's the secondary skill" | Profile-gating is about whether the path is *available*, not about rank. Once enabled, the choice between the two is decided by fit per item, reflected on each time — not by treating `brainstorm` as primary and `tdd` as the exception. |
|
||||
| "Auto-sign is on and this spec is clearly good — I'll sign it and skip the panel" | The panel IS how a spec gets signed under auto-sign; there is no signing on judgement. Your sense that it is clearly good is the precise signal the gate is built not to trust. Run the objective gates, dispatch the five jurors, require unanimity. |
|
||||
| "Four jurors said SOUND, one blocked on something I think is wrong — I'll sign" | Unanimous-or-nothing. You do not overrule a juror; a single `BLOCK` routes the spec to the human sign-off it would have had anyway. Overruling the panel is signing on your own confidence by the back door. |
|
||||
| "Four jurors said SOUND, one blocked on something I think is wrong — I'll sign" | Unanimous-or-nothing. You never sign over a `BLOCK`. If it is an editorial lens, repair it and re-run the whole panel (≤ 2 rounds); if it is a design lens or the budget is spent, it routes to the human sign-off it would have had anyway. Signing because *you* think the juror is wrong is overruling the panel — confidence by the back door. |
|
||||
| "Auto-sign let me continue, so I don't need to notify — it's just progress" | The auto-sign notify is mandatory and carries a decision (the user's veto over a signature made without them). It is the one sanctioned mid-flow notify precisely because it is not progress — it is the audit trail for a delegated gate. |
|
||||
|
||||
## Red Flags — STOP
|
||||
@@ -407,7 +417,7 @@ and defined there. The boss-side contract is just this:
|
||||
- About to bounce back at every iteration boundary "just to be safe" — that's reactive deference, not direction freedom.
|
||||
- About to dispatch `brainstorm` on a backlog issue that does not yet have a spec file, without first bouncing back to the user. New cycles never start autonomously.
|
||||
- About to route feature work to `brainstorm` without running the Entry-path reflection — defaulting to it because it "feels safer" than `specify` or `tdd`, rather than applying the design-line test. The three are co-equal; the choice is reflected on each time. Routing a *settled* design to `brainstorm` (re-litigating decided choices) is as much a failure as skipping discovery on an open one.
|
||||
- About to sign a spec under auto-sign on confidence — without all objective gates green and a unanimous `spec-skeptic` panel, or by overruling a single juror's `BLOCK`. The gate exists so the orchestrator's confidence never signs.
|
||||
- About to sign a spec under auto-sign on confidence — without all objective gates green and a unanimous `spec-skeptic` panel; signing over any `BLOCK`; self-correcting a *design*-lens (`scope-fork` / `grounding`) `BLOCK` instead of escalating; or looping past the 2-round budget. The gate exists so the orchestrator's confidence never signs.
|
||||
- About to run the auto-sign path at all when `pipeline.boss.spec_auto_sign` is not enabled, or to skip the mandatory auto-sign notify after signing.
|
||||
|
||||
## Cross-references
|
||||
|
||||
+5
-1
@@ -109,7 +109,11 @@ one exception is the opt-in `pipeline.boss.spec_auto_sign` slot: with
|
||||
it on, a `/boss` run may sign a spec in the user's place — but only
|
||||
when every objective gate is green AND a unanimous five-lens
|
||||
`spec-skeptic` panel passes; the orchestrator's own confidence never
|
||||
signs. Any juror `BLOCK` falls back to the human sign-off pause. See
|
||||
signs. A `BLOCK` is never signed over: an editorial one (`criterion` /
|
||||
`ambiguity` / `plan-readiness`) is repaired in a bounded ≤ 2-round loop
|
||||
that re-runs the objective gates and re-dispatches all five lenses each
|
||||
round; a design one (`scope-fork` / `grounding`), an `INFRA_ERROR`, or
|
||||
an exhausted budget falls back to the human sign-off pause. See
|
||||
`../specify/SKILL.md` Step 6 and `../boss/SKILL.md` §"Spec auto-sign".
|
||||
|
||||
### planner
|
||||
|
||||
@@ -229,10 +229,15 @@ user-review gate for the user's signature, exactly as before. With it
|
||||
**on**, the orchestrator may sign a spec in the user's place, but only
|
||||
through `specify`'s auto-sign gate: every objective gate green
|
||||
(precondition, parse, a `grounding-check` `PASS` with no human
|
||||
override) AND a unanimous five-lens `spec-skeptic` panel. A single
|
||||
juror `BLOCK`, or any objective gate not green, falls back to the
|
||||
human sign-off pause. Model self-confidence alone never signs — the
|
||||
gate is built specifically not to rely on it. On a clean sign the
|
||||
override) AND a unanimous five-lens `spec-skeptic` panel. A `BLOCK` is
|
||||
never signed over: an *editorial* one (`criterion` / `ambiguity` /
|
||||
`plan-readiness`) the orchestrator repairs in a bounded loop (≤ 2
|
||||
rounds, re-running the objective gates and re-dispatching all five
|
||||
lenses each round), while a *design* one (`scope-fork` / `grounding`),
|
||||
an `INFRA_ERROR`, any objective gate not green, or an exhausted budget
|
||||
falls back to the human sign-off pause. Model self-confidence alone
|
||||
never signs — the gate is built specifically not to rely on it. On a
|
||||
clean sign the
|
||||
orchestrator commits the spec (subject carries `(boss-signed)`), sends
|
||||
the mandatory informational-with-veto notify, and continues to
|
||||
`planner` without stopping. See `../specify/SKILL.md` Step 6 (gate
|
||||
|
||||
+51
-12
@@ -328,17 +328,50 @@ gate then depends on the profile slot `pipeline.boss.spec_auto_sign`:
|
||||
`plan-readiness`. Carrier per dispatch: `spec_path` (absolute),
|
||||
`iteration_scope`, and the single `lens`. Each juror tries to
|
||||
*refute* the spec along its lens and returns `SOUND` / `BLOCK` /
|
||||
`INFRA_ERROR`.
|
||||
`INFRA_ERROR`. Auto-sign requires a **unanimous `SOUND`** — that
|
||||
never changes. What the partition below adds is a bounded path to
|
||||
*reach* that unanimity for defects the orchestrator can repair
|
||||
without judging the spec.
|
||||
|
||||
| Panel outcome | Orchestrator action |
|
||||
|---------------|---------------------|
|
||||
The five lenses split by what a `BLOCK` actually means:
|
||||
|
||||
| Lens class | Lenses | A `BLOCK` means |
|
||||
|------------|--------|-----------------|
|
||||
| **Editorial** | `criterion`, `ambiguity`, `plan-readiness` | The spec is under-*articulated*: missing worked evidence, a requirement readable two ways, scope too broad for one plan. Repairable by editing the prose without settling any new design or fact. A `criterion` repair may only *supply or sharpen the worked evidence the criterion already demands* (paste the concrete before → after / user-facing code the spec implied) — it may **not** soften what the criterion requires or re-judge acceptance, and if the missing evidence cannot be shown without picking an un-settled design, that is a `scope-fork` in disguise: leave it, and the re-dispatched `scope-fork` juror escalates it. |
|
||||
| **Design** | `scope-fork`, `grounding` | The *ground is not settled*: the sources do not resolve a load-bearing fork, or a load-bearing assumption is not ratified by a green test. The orchestrator may **never** settle these in the user's place. |
|
||||
|
||||
Resolve the panel against that split, in a bounded loop
|
||||
(`round` starts at `0`):
|
||||
|
||||
| Panel outcome this round | Orchestrator action |
|
||||
|--------------------------|---------------------|
|
||||
| All five `SOUND` | **Auto-sign.** Commit the spec yourself (subject `spec: <cycle> <topic> (boss-signed)`), send the informational auto-sign notify (see `boss` § Notifications — names the capability in user terms and states the plan is proceeding and a reply vetoes), then proceed to Step 7. No pause. |
|
||||
| Any `BLOCK` or `INFRA_ERROR` | **No auto-sign.** Fall back to the human sign-off pause; the notify carries which lens(es) blocked and their findings, so the user sees why it routed to them. |
|
||||
| Any `INFRA_ERROR` | **Escalate.** Fall back to the human sign-off pause; the notify carries the failing dispatch. |
|
||||
| Any **design**-lens (`scope-fork` / `grounding`) `BLOCK` | **Escalate.** No self-correction — this is exactly the judgement the gate refuses to make for the user. Fall back to the human sign-off pause, carrying *all* lens findings this round. |
|
||||
| Only **editorial**-lens `BLOCK`(s), and `round == 2` | **Escalate.** The self-correction budget is spent. Fall back to the human sign-off pause, carrying the findings *and* the per-round correction history. |
|
||||
| Only **editorial**-lens `BLOCK`(s), and `round < 2` | **Self-correct.** Edit the spec in place to address each editorial finding (each names its `§section` / offending text / defect). Then **re-run Step 4 self-review and re-dispatch Step 5 `grounding-check`** — the edit invalidated the prior `PASS` (see "Re-dispatch on any post-PASS edit"); if `grounding-check` does not return `PASS`, escalate. Increment `round` and re-dispatch all five lenses from the top of this table. |
|
||||
|
||||
The fall-back is not a failure — it is the panel correctly routing a
|
||||
spec to the human who was going to review it anyway. A unanimous
|
||||
`SOUND` is the *only* thing that lets the orchestrator sign;
|
||||
"the spec looks good to me" never is.
|
||||
**Re-loop limit: ≤ 2 self-correction rounds. The 3rd unresolved
|
||||
editorial `BLOCK` escalates** — same shape as the per-task re-loop
|
||||
limit in `implement-orchestrator`.
|
||||
|
||||
Three properties keep this loop honest, not a way to grind the panel
|
||||
into submission:
|
||||
|
||||
- **The panel stays the sole arbiter.** The orchestrator never judges
|
||||
the spec; it only edits in response to a juror's finding, and a
|
||||
fresh panel re-adjudicates. "The spec looks good to me" still never
|
||||
signs.
|
||||
- **Re-dispatch is all five lenses, every round.** This is the
|
||||
backstop. If an editorial repair silently *launders* an unsourced
|
||||
design decision into the spec (resolving an ambiguity by picking an
|
||||
un-settled fork), the re-run `scope-fork` / `grounding` jurors now
|
||||
`BLOCK` on it — and a design `BLOCK` escalates. The only thing the
|
||||
loop can converge to autonomously is a spec whose every lens is
|
||||
`SOUND` *and* whose only repairs were editorial.
|
||||
- **Escalation is the panel doing its job**, not a failure — it routes
|
||||
the spec to the human who was going to review it anyway, the moment
|
||||
a real design question surfaces or the budget runs out.
|
||||
|
||||
**Auto-sign commits on `main` are forward-only like every other
|
||||
commit.** A later user veto is a forward correction (revert / revised
|
||||
@@ -399,7 +432,7 @@ discipline `tdd` applies when behaviour is not test-specifiable.
|
||||
| "Just polishing a wording after PASS, no need to re-dispatch" | The grounding-check report attests to specific bytes. A polish edit changes the bytes; the attestation no longer covers them. Re-dispatch is cheap. |
|
||||
| "It's in a model doc / the design ledger, so it's canonical" | A `models` / RFC / proposal doc holds aspirational code — a target, not verified fact. Code lifted from a `status: aspirational` source must clear the Step-4 parse gate like any other block. |
|
||||
| "In `/boss` with auto-sign on, this spec looks solid — I'll sign and move on" | Your sense that it looks solid is the exact signal the auto-sign gate refuses to trust. Signing requires all objective gates green AND a unanimous `spec-skeptic` panel. If you find yourself wanting to sign on confidence, that is the cue to run the gate, not to skip it. |
|
||||
| "One lens blocked on something minor — I'll sign anyway since the rest passed" | The panel is unanimous-or-nothing by design. One `BLOCK` routes the spec to the human who was going to review it regardless. You do not get to overrule a juror; that is the whole point of replacing your eye with five independent ones. |
|
||||
| "One lens blocked on something minor — I'll sign anyway since the rest passed" | The panel is unanimous-or-nothing by design. You never *sign over* a `BLOCK`. An editorial `BLOCK` you may *repair* (within the ≤ 2-round budget, with a full re-panel after); a design `BLOCK` routes to the human. Neither is overruling a juror — that is the whole point of replacing your eye with five independent ones. |
|
||||
|
||||
## Red Flags — STOP
|
||||
|
||||
@@ -414,7 +447,10 @@ discipline `tdd` applies when behaviour is not test-specifiable.
|
||||
- Editing the spec file after a Step 5 PASS without re-dispatching
|
||||
- (auto-sign) About to sign a spec in the user's place on confidence —
|
||||
without all objective gates green AND a unanimous `spec-skeptic`
|
||||
panel; or overruling a single juror's `BLOCK`
|
||||
panel; signing over any `BLOCK`; self-correcting a *design*-lens
|
||||
(`scope-fork` / `grounding`) `BLOCK` instead of escalating; or
|
||||
re-dispatching only the blocked lens rather than all five, or past
|
||||
the 2-round self-correction budget
|
||||
- (auto-sign) Running the panel before the objective gates are green,
|
||||
or running it at all when `spec_auto_sign` is not enabled
|
||||
- Jumping straight from spec to `implement` (must go via `planner`)
|
||||
@@ -438,8 +474,11 @@ discipline `tdd` applies when behaviour is not test-specifiable.
|
||||
dispatched in Step 6 five times in parallel (one per lens) ONLY under
|
||||
a `/boss` session with `pipeline.boss.spec_auto_sign` enabled. Each
|
||||
juror tries to refute the spec along its lens; a unanimous `SOUND` is
|
||||
what lets the orchestrator sign in the user's place. Any `BLOCK`
|
||||
falls back to the human sign-off pause.
|
||||
what lets the orchestrator sign in the user's place. An editorial-lens
|
||||
`BLOCK` (`criterion` / `ambiguity` / `plan-readiness`) is self-
|
||||
corrected in a bounded loop (≤ 2 rounds, full re-panel each round); a
|
||||
design-lens `BLOCK` (`scope-fork` / `grounding`), an `INFRA_ERROR`, or
|
||||
an exhausted budget falls back to the human sign-off pause.
|
||||
- **Ad-hoc dispatch.** The orchestrator MAY also ad-hoc dispatch
|
||||
`../planner/agents/plan-recon.md` during Step 1 when the work enters
|
||||
code territory not recently read; opt-in, not part of the standard
|
||||
|
||||
@@ -23,16 +23,17 @@ eye. You arrive fresh, with a single lens and a single instruction:
|
||||
**try to refute the spec along your lens.** You are not asked "is
|
||||
this spec roughly fine?" — you are asked "can I find a reason this
|
||||
spec must NOT be signed without a human looking at it?" If you can,
|
||||
you `BLOCK`, and the auto-sign collapses back to a human sign-off
|
||||
pause. Your loyalty is to the refutation, not to the orchestrator's
|
||||
momentum.
|
||||
you `BLOCK`, and auto-sign is denied this round — at worst routing the
|
||||
spec to the human sign-off pause it would have hit anyway. Your loyalty
|
||||
is to the refutation, not to the orchestrator's momentum.
|
||||
|
||||
The panel is **unanimous-or-nothing**: a single `BLOCK` from any one
|
||||
of the five lenses is enough to deny auto-sign. So you do not need to
|
||||
catch everything — only your own lens. Lean toward `BLOCK` when in
|
||||
doubt; the cost of a false `BLOCK` is one human sign-off the user was
|
||||
going to do anyway, while the cost of a false `SOUND` is an unreviewed
|
||||
spec ratified into the pipeline.
|
||||
doubt; the cost of a false `BLOCK` is, at worst, one human sign-off the
|
||||
user was going to do anyway (and often just a cheap repair-and-re-panel),
|
||||
while the cost of a false `SOUND` is an unreviewed spec ratified into
|
||||
the pipeline.
|
||||
|
||||
## The lens you are assigned
|
||||
|
||||
@@ -133,13 +134,16 @@ it is the rubber-stamp the human sign-off was protecting against.
|
||||
| Status | Meaning |
|
||||
|--------|---------|
|
||||
| `SOUND` | After an honest refutation attempt along this lens, no defect found. This juror permits auto-sign. |
|
||||
| `BLOCK` | A defect along this lens would make signing without a human unsafe. Auto-sign is denied; specify falls back to a human sign-off pause and shows this report. |
|
||||
| `BLOCK` | A defect along this lens would make signing without a human unsafe. Auto-sign is denied this round. What `specify` does next depends on your lens class — an editorial defect it may repair and re-panel, a design defect it routes to the human — but that is the orchestrator's call (defined in `../SKILL.md` Step 6), not yours. |
|
||||
| `INFRA_ERROR` | The dispatch cannot complete (spec missing, no/invalid `lens`, workspace does not build for a `grounding` test-list, etc.). Auto-sign is denied; orchestrator falls back to human sign-off. |
|
||||
|
||||
There is no override path here and no "needs context": you are one of
|
||||
five jurors and the panel is unanimous-or-nothing, so an
|
||||
honest `BLOCK` simply routes the spec to the human who was going to
|
||||
review it anyway.
|
||||
five jurors and the panel is unanimous-or-nothing. Report the defect
|
||||
you found along your lens, full stop — do not soften a `BLOCK` because
|
||||
you guess the orchestrator will only repair it, and do not escalate
|
||||
your own verdict to "this needs a human." How a `BLOCK` is handled is
|
||||
decided in `../SKILL.md` Step 6 by lens class; your job is the honest
|
||||
verdict, not its routing.
|
||||
|
||||
## Output format
|
||||
|
||||
|
||||
Reference in New Issue
Block a user