Brummel/AILang
1
0
Fork 0
Code Issues 29 Pull Requests Actions Packages Projects Releases Wiki Activity

iter ms.2: Qwen retroactive re-run + first CodeLlama-13b-Instruct run + DESIGN.md §Decision-6 addendum extended to two subjects

Browse Source 
Two live IONOS harness invocations with the ms.1 fix in place:

  - Qwen/Qwen3-Coder-Next   → runs/2026-05-12-080864/  (RUN_STATUS=ok)
  - meta-llama/CodeLlama-13b-Instruct-hf
                            → runs/2026-05-12-9197fd/  (RUN_STATUS=ok;
                              2/8 JSON cells terminated as api_failure
                              on turn 1 zero-token — transient IONOS
                              terminal error, disclosed in addendum)

Per-cell results (reached/4, prompt tokens, completion tokens):

  Qwen      JSON     1/4   182,378   14,972
  Qwen      AILX     1/4   110,575    3,474
  CodeLlama JSON     0/4   116,015    2,711   [2 cells api_failure]
  CodeLlama AILX     2/4    93,017    2,234

Both subjects agree on direction: AILX cohort cheaper on prompt
tokens (Qwen 61%, CodeLlama 80% of JSON-cohort spend) and on
completion tokens (Qwen 23%, CodeLlama 82%); AILX reached-green
≥ JSON reached-green for each subject. Three of four
first-attempt-green cells across the two subjects are AILX.
Failure classes split symmetric to form: JSON cohorts fail at
typecheck, AILX cohorts at parse — each form's
front-of-pipeline check.

DESIGN.md §"Decision 6 / Empirical addendum (2026-05-12)"
replaced: 2-column single-subject view → 4-column two-subject
view, framing paragraph reports direction agreement and the
api_failure / Qwen-rerun-noise anomalies, scope paragraph
updated to 'two subjects, n=1 each, deterministic; not a
verdict — a universal claim would need ≥3 subjects with
statistical robustness'.

Roadmap P3 'Multi-subject expansion' entry removed; scoped
goal met for two subjects.

Milestone token spend: prompt 501,985 + completion 23,391
= 525,376.
This commit is contained in:
Brummel
2026-05-12 13:29:15 +02:00
parent 5858377c99
commit 65a4f0aa16
281 changed files with 4662 additions and 43 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/journals/INDEX.md
+1
View File
@@ -26,3 +26,4 @@
- 2026-05-12 — iter cma.3: live Qwen3-Coder-Next run + DESIGN.md §Decision-6 empirical addendum + roadmap close (AILX 2/4 green vs JSON 1/4; ~98k vs ~207k tokens; single-subject scope explicit, multi-subject expansion queued as P3) → 2026-05-12-iter-cma.3.md
- 2026-05-12 — audit-cma: milestone close (Cross-model authoring-form test) — architect drift fixed in 2 record-keeping items (INDEX backfill for brainstorm Step 7.5, README test-count correction); bench all-green (5 unexplained improvements on latency.explicit_at_rc not coupled to milestone, baseline left pristine) → 2026-05-12-audit-cma.md
- 2026-05-12 — iter ms.1: pipeline anyhow-chain preservation — `{e}` → `{e:#}` at `pipeline.rs:118` restores `serde_json` parse-error leaf in JSON-cohort feedback; pinning unit test added (14/14 green) → 2026-05-12-iter-ms.1.md
- 2026-05-12 — iter ms.2: Qwen3-Coder-Next retroactive re-run + first CodeLlama-13b-Instruct run via IONOS; DESIGN.md §Decision-6 addendum extended from 2-col single-subject to 4-col two-subject; both subjects agree on direction (AILX cheaper + ≥ green); roadmap P3 multi-subject entry removed → 2026-05-12-iter-ms.2.md