Commit Graph

3 Commits

Author SHA1 Message Date
Brummel 97ba871d56 refactor(postmortem): report raw token counts, drop dollar estimates
Remove the price-table cost derivation entirely. The transcript carries
no metered cost, and list prices vary by plan — so converting tokens to
a dollar figure dressed an estimate up as a number. Report raw token
counts instead, kept split by class (input / output / cache-creation /
cache-read) with a `total_tokens` sum per scope.

Script: drop DEFAULT_PRICING / load_pricing / rate_for / cost_of and the
--pricing flag; aggregate_transcript and analyze_subagents no longer take
a pricing arg; every `cost_usd` field becomes `total_tokens`; the pricing
warning is gone. SKILL.md: the first graded axis is now "Token spend &
efficiency", the scorecard and output format cite token counts, and the
Iron Law / Red Flags now forbid converting tokens to a cost.

cache_hit_ratio and the active-session warning are unchanged.
2026-06-02 16:34:53 +02:00
Brummel 69e4b63c6c fix(postmortem): anchor terminal_status on the structured Status line
The subagent terminal-status classifier scanned the whole final report
text for status keywords, so `\bBLOCKED\b` fired on the implement
end-report's own template lines (`BLOCKED file:  BLOCKED.md`,
`Blocked detail:`) and on prose narrating a surmounted blocker. Shipped
implement runs whose end-report reads `Status: DONE` were flipped to
BLOCKED — making the agent-effectiveness axis actively misleading.

Anchor on the structured `Status:` line of the end-report instead (the
implement-orchestrator emits a fixed `Status:  DONE|PARTIAL|BLOCKED|...`
header, optionally markdown-bold). Fall back to a status token standing
alone on its own line for agents that emit a bare terminal token; prose
mentions and `BLOCKED.md`-style substrings no longer match. Agents with
no structured status now read `unknown` rather than a fabricated BLOCKED.

Add an executable spec (postmortem/tests/test_terminal_status.py, plain
asserts, no pytest dep) pinning the issue-#3 case plus the genuine
BLOCKED / bare-token / no-signal guards. RED before, GREEN after.

Verified on the real AILang-style session 2811d227: 3 BLOCKED (one a
false positive) -> 2 genuine BLOCKED + 13 honest unknown.

closes #3
2026-06-02 16:22:12 +02:00
Brummel daa7b49f8f feat(postmortem): add session-retrospective skill + aggregator
A new utility skill that grades a finished session on three axes —
cost & efficiency, toolchain health, agent effectiveness — from the
session's own Claude Code flight recorder (the JSONL transcript plus
one sidechain log per dispatched subagent).

The heavy lifting lives in scripts/postmortem.py, a stdlib-only,
read-only aggregator. It handles the two accounting traps the raw log
sets: usage is repeated per streamed assistant line under a shared
requestId (deduped max-per-field, then summed across requests), and
the transcript carries no costUSD/durationMs (both null) — so cost is
derived from tokens x an overridable list-price table and labelled an
estimate, wall-clock from timestamp deltas. Subagent spend is billed
separately from its own usage objects and graded by terminal status.

Single-session by design: defaults to the newest transcript in the
project's log dir (flagged active if still live), or --session <id>
for a finished run. Secret files are out of bounds in both the Iron
Law and the Red Flags.

Verified end-to-end against a real 15-subagent session.
2026-06-02 16:11:33 +02:00