fix(postmortem): anchor terminal_status on the structured Status line
The subagent terminal-status classifier scanned the whole final report text for status keywords, so `\bBLOCKED\b` fired on the implement end-report's own template lines (`BLOCKED file: BLOCKED.md`, `Blocked detail:`) and on prose narrating a surmounted blocker. Shipped implement runs whose end-report reads `Status: DONE` were flipped to BLOCKED — making the agent-effectiveness axis actively misleading. Anchor on the structured `Status:` line of the end-report instead (the implement-orchestrator emits a fixed `Status: DONE|PARTIAL|BLOCKED|...` header, optionally markdown-bold). Fall back to a status token standing alone on its own line for agents that emit a bare terminal token; prose mentions and `BLOCKED.md`-style substrings no longer match. Agents with no structured status now read `unknown` rather than a fabricated BLOCKED. Add an executable spec (postmortem/tests/test_terminal_status.py, plain asserts, no pytest dep) pinning the issue-#3 case plus the genuine BLOCKED / bare-token / no-signal guards. RED before, GREEN after. Verified on the real AILang-style session 2811d227: 3 BLOCKED (one a false positive) -> 2 genuine BLOCKED + 13 honest unknown. closes #3
This commit is contained in:
@@ -45,7 +45,22 @@ DEFAULT_PRICING = {
|
||||
}
|
||||
PRICING_AS_OF = "2026-01 list prices (estimate, override via POSTMORTEM_PRICING)"
|
||||
|
||||
STATUS_TOKENS = ["DONE_WITH_CONCERNS", "NEEDS_CONTEXT", "BLOCKED", "PARTIAL", "DONE"]
|
||||
_TOK_ALT = "DONE_WITH_CONCERNS|NEEDS_CONTEXT|BLOCKED|PARTIAL|DONE" # longest-first
|
||||
# Authoritative signal: the implement end-report leads with a structured
|
||||
# "Status: DONE" line (optionally wrapped in markdown bold or a list
|
||||
# marker). Anchoring here is what stops the scan from flipping a shipped
|
||||
# DONE run to BLOCKED on the strength of the report's own template lines
|
||||
# (`BLOCKED file: BLOCKED.md`, `Blocked detail:`) or prose narrating a
|
||||
# surmounted blocker. See issue #3.
|
||||
STATUS_LINE_RE = re.compile(
|
||||
r"^[\s>*#\-]*\**\s*status\s*\**\s*[:\-]\s*\**\s*(" + _TOK_ALT + r")\b",
|
||||
re.IGNORECASE | re.MULTILINE)
|
||||
# Fallback for agents that emit a bare status token as their final line
|
||||
# (no "Status:" prefix). Requires the token to BE the line — excludes
|
||||
# substrings like "BLOCKED.md" or "BLOCKED file:" buried in prose.
|
||||
STATUS_STANDALONE_RE = re.compile(
|
||||
r"^[\s>*#\-]*\**\s*(" + _TOK_ALT + r")\**\s*$",
|
||||
re.IGNORECASE | re.MULTILINE)
|
||||
|
||||
|
||||
def load_pricing(override_json):
|
||||
@@ -286,9 +301,15 @@ def subagent_terminal_status(path):
|
||||
for block in content:
|
||||
if block.get("type") == "text" and block.get("text", "").strip():
|
||||
last_text = block["text"]
|
||||
for tok in STATUS_TOKENS:
|
||||
if re.search(r"\b" + tok + r"\b", last_text):
|
||||
return tok
|
||||
# Primary: the structured "Status:" line. Take the LAST one so a
|
||||
# restated final status wins over any earlier mention.
|
||||
structured = STATUS_LINE_RE.findall(last_text)
|
||||
if structured:
|
||||
return structured[-1].upper()
|
||||
# Fallback: a status token standing alone on its own line.
|
||||
standalone = STATUS_STANDALONE_RE.findall(last_text)
|
||||
if standalone:
|
||||
return standalone[-1].upper()
|
||||
return "unknown"
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user