Aggregate Workflow run logs in postmortem.py — workflow sessions read as zero subagent spend #28

New Issue

2026-07-17T15:20:05+02:00

claude commented

2026-07-17 15:20:05 +02:00

Problem

postmortem/scripts/postmortem.py aggregates subagent spend from <session>/subagents/agent-*.jsonl only (single-level glob in analyze_subagents(), ~line 272). Runs on the Workflow substrate write their per-agent transcripts elsewhere: raw usage under <session>/subagents/workflows/wf_*/agent-*.jsonl plus a per-agent summary (label, model, token counter) in <session>/workflows/wf_*.json. Neither path is read.

The implement execution loop moved onto the Workflow substrate on 2026-06-17 (7a58a53). Since then a session that ran implement-loop grades as near-zero subagent spend — the tool built to grade token spend cannot see the pipeline's own dispatches. Measured over local workflow-run telemetry, 2026-06-17..2026-07-17: 206 implement-loop runs, 2,883 agent instances, ~164.5M non-cache-read tokens, all invisible to postmortem.py.

Fix scope

Second aggregation path: walk workflows/wf_*.json for per-agent stage labels/models, and the matching subagents/workflows/wf_*/agent-*.jsonl for raw usage, deduplicated by requestId with max-per-field within a request — the same method aggregate_transcript() already applies.
Report per stage label (e.g. impl, spec, qual, mini-verify), so per-stage spend becomes visible next to the existing per-subagent table.
The two counters differ by design: the wf_*.json tokens field under-tracks vs. the deduplicated raw usage (~11% in the corpus above); the raw-transcript number is authoritative, the summary supplies labels/models.

Relation to #27

#27 proposes removing the postmortem skill on a no-consumers premise. This change gives the skill a structural consumer: before/after verification for pipeline token-cost changes. The removal decision is unaffected by this issue's scope but its premise changes once this lands.

## Problem `postmortem/scripts/postmortem.py` aggregates subagent spend from `<session>/subagents/agent-*.jsonl` only (single-level glob in `analyze_subagents()`, ~line 272). Runs on the Workflow substrate write their per-agent transcripts elsewhere: raw usage under `<session>/subagents/workflows/wf_*/agent-*.jsonl` plus a per-agent summary (label, model, token counter) in `<session>/workflows/wf_*.json`. Neither path is read. The implement execution loop moved onto the Workflow substrate on 2026-06-17 (`7a58a53`). Since then a session that ran `implement-loop` grades as near-zero subagent spend — the tool built to grade token spend cannot see the pipeline's own dispatches. Measured over local workflow-run telemetry, 2026-06-17..2026-07-17: 206 implement-loop runs, 2,883 agent instances, ~164.5M non-cache-read tokens, all invisible to `postmortem.py`. ## Fix scope - Second aggregation path: walk `workflows/wf_*.json` for per-agent stage labels/models, and the matching `subagents/workflows/wf_*/agent-*.jsonl` for raw usage, deduplicated by `requestId` with max-per-field within a request — the same method `aggregate_transcript()` already applies. - Report per stage label (e.g. `impl`, `spec`, `qual`, `mini-verify`), so per-stage spend becomes visible next to the existing per-subagent table. - The two counters differ by design: the `wf_*.json` `tokens` field under-tracks vs. the deduplicated raw usage (~11% in the corpus above); the raw-transcript number is authoritative, the summary supplies labels/models. ## Relation to #27 #27 proposes removing the postmortem skill on a no-consumers premise. This change gives the skill a structural consumer: before/after verification for pipeline token-cost changes. The removal decision is unaffected by this issue's scope but its premise changes once this lands.

claude added the bug label 2026-07-17 15:20:05 +02:00

claude self-assigned this 2026-07-17 15:20:05 +02:00

claude referenced this issue

2026-07-17 15:20:21 +02:00

Remove the postmortem skill #27

Brummel referenced this issue from a commit

2026-07-17 16:01:20 +02:00

feat(postmortem): aggregate Workflow run logs per stage

Brummel closed this issue

2026-07-17 16:01:20 +02:00

Sign in to join this conversation.