fix(postmortem): anchor terminal_status on the structured Status line

The subagent terminal-status classifier scanned the whole final report
text for status keywords, so `\bBLOCKED\b` fired on the implement
end-report's own template lines (`BLOCKED file:  BLOCKED.md`,
`Blocked detail:`) and on prose narrating a surmounted blocker. Shipped
implement runs whose end-report reads `Status: DONE` were flipped to
BLOCKED — making the agent-effectiveness axis actively misleading.

Anchor on the structured `Status:` line of the end-report instead (the
implement-orchestrator emits a fixed `Status:  DONE|PARTIAL|BLOCKED|...`
header, optionally markdown-bold). Fall back to a status token standing
alone on its own line for agents that emit a bare terminal token; prose
mentions and `BLOCKED.md`-style substrings no longer match. Agents with
no structured status now read `unknown` rather than a fabricated BLOCKED.

Add an executable spec (postmortem/tests/test_terminal_status.py, plain
asserts, no pytest dep) pinning the issue-#3 case plus the genuine
BLOCKED / bare-token / no-signal guards. RED before, GREEN after.

Verified on the real AILang-style session 2811d227: 3 BLOCKED (one a
false positive) -> 2 genuine BLOCKED + 13 honest unknown.

closes #3
This commit is contained in:
2026-06-02 16:22:12 +02:00
parent 6ec1c28111
commit 69e4b63c6c
2 changed files with 150 additions and 4 deletions
+25 -4
View File
@@ -45,7 +45,22 @@ DEFAULT_PRICING = {
}
PRICING_AS_OF = "2026-01 list prices (estimate, override via POSTMORTEM_PRICING)"
STATUS_TOKENS = ["DONE_WITH_CONCERNS", "NEEDS_CONTEXT", "BLOCKED", "PARTIAL", "DONE"]
_TOK_ALT = "DONE_WITH_CONCERNS|NEEDS_CONTEXT|BLOCKED|PARTIAL|DONE" # longest-first
# Authoritative signal: the implement end-report leads with a structured
# "Status: DONE" line (optionally wrapped in markdown bold or a list
# marker). Anchoring here is what stops the scan from flipping a shipped
# DONE run to BLOCKED on the strength of the report's own template lines
# (`BLOCKED file: BLOCKED.md`, `Blocked detail:`) or prose narrating a
# surmounted blocker. See issue #3.
STATUS_LINE_RE = re.compile(
r"^[\s>*#\-]*\**\s*status\s*\**\s*[:\-]\s*\**\s*(" + _TOK_ALT + r")\b",
re.IGNORECASE | re.MULTILINE)
# Fallback for agents that emit a bare status token as their final line
# (no "Status:" prefix). Requires the token to BE the line — excludes
# substrings like "BLOCKED.md" or "BLOCKED file:" buried in prose.
STATUS_STANDALONE_RE = re.compile(
r"^[\s>*#\-]*\**\s*(" + _TOK_ALT + r")\**\s*$",
re.IGNORECASE | re.MULTILINE)
def load_pricing(override_json):
@@ -286,9 +301,15 @@ def subagent_terminal_status(path):
for block in content:
if block.get("type") == "text" and block.get("text", "").strip():
last_text = block["text"]
for tok in STATUS_TOKENS:
if re.search(r"\b" + tok + r"\b", last_text):
return tok
# Primary: the structured "Status:" line. Take the LAST one so a
# restated final status wins over any earlier mention.
structured = STATUS_LINE_RE.findall(last_text)
if structured:
return structured[-1].upper()
# Fallback: a status token standing alone on its own line.
standalone = STATUS_STANDALONE_RE.findall(last_text)
if standalone:
return standalone[-1].upper()
return "unknown"