Single-source duplicated carrier/handoff contracts across skills (drift risk) #2

New Issue

2026-05-30T10:57:24+02:00

Brummel commented

2026-05-30 10:57:24 +02:00

Problem

Interface contracts between a SKILL.md and its agent file(s) are
written out field-by-field in both places — the carrier (what the
orchestrator hands the agent), the handoff/output contract (fields the
agent returns), dispatch parameter tables. Because the two files load
into separate, non-overlapping contexts (SKILL.md → orchestrator
context; agents/*.md → the subagent's system prompt) and there is no
transclusion, the two copies drift apart.

This was just found and fixed in the debug skill (commits
6410732..de42974): the agent file became the single source for each
field's semantics, and SKILL.md was reduced to naming the fields plus
a pointer. An audit of the remaining skills shows the same pattern is
widespread.

DRIFT = the two copies already diverge in wording (sometimes the
agent carries load-bearing semantics SKILL.md omits).
LATENT = still consistent, but duplicated and drift-prone.

Findings

Skill	Contract	Status
`audit`	—	clean — already uses pointers (reference implementation)
`brainstorm`	carrier `spec_path`/`iteration_scope` → grounding-check	DRIFT
`docwriter`	carrier (`priority_items`) + output (`status`/`files_touched`/`warnings_cleared`/`findings`)	DRIFT (both)
`fieldtest`	carrier `cycle_id`/`cycle_scope`/`axis_hints`/`commit_range`	LATENT
`implement`	SKILL↔orchestrator carrier (`iter_id`) + `task_text_path` (impl↔spec-reviewer) + sub-status vocab	DRIFT + 2× LATENT
`planner`	carrier `spec_path`/`iteration_scope`/`focus_hint` → plan-recon	DRIFT

Detail (file:line)

brainstorm — SKILL.md:277-281 (Step 7.5 Dispatch carrier) ↔ agents/grounding-check.md:57-60. DRIFT: spec_path — SKILL says "path to the spec just written in Step 6", agent says "absolute path…". The agent's absolute requirement is absent from SKILL.md, so an orchestrator following only the skill could hand a relative path.
docwriter —
- Carrier: SKILL.md:62-69 ↔ agents/docwriter.md:60-65. DRIFT on priority_items (agent adds "e.g. recently added public APIs").
- Output/produces: SKILL.md:70-80 ↔ agents/docwriter.md:150-161 (+ status protocol 136-148). DRIFT: same four return fields defined twice; status enum inline in SKILL, findings wording diverges.
fieldtest — SKILL.md:104-112 (consumes table) ↔ agents/fieldtester.md:58-65. LATENT: four fields duplicated; axis_hints differs cosmetically. Agent already owns the empty-axis_hints fallback (67-69).
implement —
- Finding 1 (SKILL↔agent, primary): SKILL.md:96-114 (dispatch code-blocks) ↔ agents/implement-orchestrator.md:54-62 (Carrier contract). DRIFT on iter_id (agent encodes scratch-dir/stats-filename/not-a-branch semantics the SKILL omits).
- Finding 2 (agent↔agent): task_text_path in agents/implementer.md:54 ↔ agents/spec-reviewer.md:64. LATENT.
- Finding 3 (SKILL↔agent, by design but duplicated): per-task sub-status table SKILL.md:71-87 ↔ agents/implement-orchestrator.md:160-216 (Phase 2 prose re-states each sub-status + the ≤2-retry rule). LATENT.
planner — SKILL.md:75-82 (Dispatch carrier) ↔ agents/plan-recon.md:53-58. DRIFT on all three fields; agent carries (mandatory)/optional markers and the BLOCKED-on-missing rule (60-61) that SKILL.md omits.

Recommended fix (per skill, mirroring the `debug` fix)

Make the agent file the single source for each field's semantics
(the subagent must carry the contract inline regardless — it runs in
a fresh context and cannot follow a pointer at runtime).
Fold any richer description that currently lives only in SKILL.md
into the agent file so nothing is lost.
Reduce the SKILL.md copy to field names + a pointer to the
agent's contract section; mark the agent's table authoritative.
For the one agent↔agent case (implement Finding 2), pick the
producer/owner agent as source and have the other cross-reference.

audit is the reference: its SKILL.md already points to the agents'
contracts instead of restating them.

Acceptance

Every interface field's semantics defined in exactly one file; the other
file names the fields and points. No field description appears twice.

## Problem Interface contracts between a `SKILL.md` and its agent file(s) are written out **field-by-field in both places** — the carrier (what the orchestrator hands the agent), the handoff/output contract (fields the agent returns), dispatch parameter tables. Because the two files load into separate, non-overlapping contexts (`SKILL.md` → orchestrator context; `agents/*.md` → the subagent's system prompt) and there is no transclusion, the two copies drift apart. This was just found and fixed in the `debug` skill (commits `6410732`..`de42974`): the agent file became the single source for each field's semantics, and `SKILL.md` was reduced to naming the fields plus a pointer. An audit of the remaining skills shows the same pattern is widespread. `DRIFT` = the two copies already diverge in wording (sometimes the agent carries load-bearing semantics `SKILL.md` omits). `LATENT` = still consistent, but duplicated and drift-prone. ## Findings | Skill | Contract | Status | |---|---|---| | `audit` | — | clean — already uses pointers (reference implementation) | | `brainstorm` | carrier `spec_path`/`iteration_scope` → grounding-check | **DRIFT** | | `docwriter` | carrier (`priority_items`) + output (`status`/`files_touched`/`warnings_cleared`/`findings`) | **DRIFT** (both) | | `fieldtest` | carrier `cycle_id`/`cycle_scope`/`axis_hints`/`commit_range` | LATENT | | `implement` | SKILL↔orchestrator carrier (`iter_id`) + `task_text_path` (impl↔spec-reviewer) + sub-status vocab | **DRIFT** + 2× LATENT | | `planner` | carrier `spec_path`/`iteration_scope`/`focus_hint` → plan-recon | **DRIFT** | ### Detail (file:line) - **brainstorm** — `SKILL.md:277-281` (Step 7.5 Dispatch carrier) ↔ `agents/grounding-check.md:57-60`. DRIFT: `spec_path` — SKILL says "path to the spec just written in Step 6", agent says "absolute path…". The agent's **absolute** requirement is absent from `SKILL.md`, so an orchestrator following only the skill could hand a relative path. - **docwriter** — - Carrier: `SKILL.md:62-69` ↔ `agents/docwriter.md:60-65`. DRIFT on `priority_items` (agent adds "e.g. recently added public APIs"). - Output/produces: `SKILL.md:70-80` ↔ `agents/docwriter.md:150-161` (+ status protocol `136-148`). DRIFT: same four return fields defined twice; `status` enum inline in SKILL, `findings` wording diverges. - **fieldtest** — `SKILL.md:104-112` (consumes table) ↔ `agents/fieldtester.md:58-65`. LATENT: four fields duplicated; `axis_hints` differs cosmetically. Agent already owns the empty-`axis_hints` fallback (`67-69`). - **implement** — - Finding 1 (SKILL↔agent, primary): `SKILL.md:96-114` (dispatch code-blocks) ↔ `agents/implement-orchestrator.md:54-62` (Carrier contract). DRIFT on `iter_id` (agent encodes scratch-dir/stats-filename/not-a-branch semantics the SKILL omits). - Finding 2 (agent↔agent): `task_text_path` in `agents/implementer.md:54` ↔ `agents/spec-reviewer.md:64`. LATENT. - Finding 3 (SKILL↔agent, by design but duplicated): per-task sub-status table `SKILL.md:71-87` ↔ `agents/implement-orchestrator.md:160-216` (Phase 2 prose re-states each sub-status + the ≤2-retry rule). LATENT. - **planner** — `SKILL.md:75-82` (Dispatch carrier) ↔ `agents/plan-recon.md:53-58`. DRIFT on all three fields; agent carries `(mandatory)`/optional markers and the BLOCKED-on-missing rule (`60-61`) that `SKILL.md` omits. ## Recommended fix (per skill, mirroring the `debug` fix) 1. Make the **agent file** the single source for each field's semantics (the subagent must carry the contract inline regardless — it runs in a fresh context and cannot follow a pointer at runtime). 2. Fold any richer description that currently lives only in `SKILL.md` into the agent file so nothing is lost. 3. Reduce the `SKILL.md` copy to **field names + a pointer** to the agent's contract section; mark the agent's table authoritative. 4. For the one agent↔agent case (`implement` Finding 2), pick the producer/owner agent as source and have the other cross-reference. `audit` is the reference: its `SKILL.md` already points to the agents' contracts instead of restating them. ## Acceptance Every interface field's semantics defined in exactly one file; the other file names the fields and points. No field description appears twice.

Brummel referenced this issue from a commit

2026-05-30 11:22:08 +02:00

refactor: single-source carrier/handoff contracts across skills

Brummel closed this issue

2026-05-30 11:22:08 +02:00

claude referenced this issue

2026-07-24 20:11:16 +02:00

Experiment: full pipeline with implement-skills disabled — quality vs token tracking #35

claude referenced this issue

2026-07-25 05:17:59 +02:00

Experiment: full pipeline with implement-skills disabled — quality vs token tracking #35

Sign in to join this conversation.