iter cma.2: harness binary + 4 tasks + reference solutions + 4 integration tests
Sibling standalone Cargo crate `harness/` under experiments/2026-05-12-cross-model-authoring/ (out-of-workspace idiom carried forward from cma.1 verbatim). Six modules: strip_locations (regex pass for form-asymmetric location info, calibrated against five real `ail check`/`ail parse` stderr captures), pipeline (subprocess wrapper for parse|check|build + 5s-timeout exec; preflight on ail+clang), ionos (blocking reqwest client + retry policy per spec), mock (canned-response loader), scoring (CSV + summary.md), tasks (definition struct + loader). main.rs ties them into the per-(cohort,task) loop with budget accounting and per-turn artefact recording. Four MVP tasks land with reference solutions that compile, build, and execute green through the actual ail+clang pipeline: t1_add_three (chained `+` + io/print_int), t2_length (polymorphic List + recursion), t3_main_prints (minimal IO module), t4_count_zeros (prelude Eq Int + branched if). Reference solutions stay in canonical AILang form — param_modes is omitted when every parameter is the Implicit default, consistent with the existing examples/ corpus. 13/13 tests green: 5 lib unit (strip_locations) + 5 integration (strip_locations against verbatim captured fixtures) + 1 verify_references (drives every reference through the real ail+clang pipeline) + 1 mock_full_run (full eight-row sweep with mixed green-on-turn-2 + turn-limit cycles) + 1 budget_abort (synthetic budget exhaustion with budget_abort rows + run_status). Two implementer-phase repairs beyond the plan, both small and surfaced in the iter journal Concerns: 1. pipeline.rs renames the program file to `<module-name>.ail.json` between parse and check because `ail check` enforces filename stem == module name (compiler contract the plan did not anticipate). 2. main.rs fills synthetic budget_abort rows for tasks the outer loop never reached so scores.csv preserves the expected eight-row shape on budget exhaustion. One plan/text mismatch carried over: README "Total 8 passed" reflects the plan's four-suite count; actual sweep produces 13. Doc-fix candidate for cma.3. cma.3 (live IONOS run + DESIGN.md addendum + roadmap edits) remains out of scope.
This commit is contained in:
@@ -0,0 +1,31 @@
|
||||
{
|
||||
"iter_id": "cma.2",
|
||||
"date": "2026-05-12",
|
||||
"mode": "standard",
|
||||
"outcome": "DONE",
|
||||
"tasks_total": 13,
|
||||
"tasks_completed": 13,
|
||||
"reloops_per_task": {
|
||||
"1": 0,
|
||||
"2": 0,
|
||||
"3": 0,
|
||||
"4": 0,
|
||||
"5": 0,
|
||||
"6": 0,
|
||||
"7": 0,
|
||||
"8": 0,
|
||||
"9": 0,
|
||||
"10": 0,
|
||||
"11": 0,
|
||||
"12": 1,
|
||||
"13": 0
|
||||
},
|
||||
"review_loops_spec": 0,
|
||||
"review_loops_quality": 0,
|
||||
"blocked_reason": null,
|
||||
"notes": [
|
||||
"Task 3 + Task 12 each landed one implementer-phase repair (pipeline filename-matching; budget-abort row-filling) — both flagged as Concerns in the journal, not re-loops against the spec/quality checks. reloops_per_task['12'] = 1 records the budget_abort test going RED-then-GREEN after the main.rs repair.",
|
||||
"All 13 tests pass: 5 lib unit (strip_locations) + 5 strip_locations integration + 1 verify_references + 1 mock_full_run + 1 budget_abort.",
|
||||
"AIL_BIN env required for verify_references and mock_full_run tests when ail is not on PATH."
|
||||
]
|
||||
}
|
||||
Reference in New Issue
Block a user