Brummel 8b7e683cc9 experiment(cma): SMA probe run 3 — full grammar+spec+lib makes Qwen degenerate (refs #68)
Third run of the SMA authoring probe, this time with Qwen fully equipped: a
distilled complete Form-A construct grammar (with explicit arity/paren rules),
the mini-spec, and the verbatim Series library source. Hypothesis: the formal
grammar clears the bracket failures of runs 1-2. It did the opposite.

With ~80k prompt tokens the model degenerated into an identical pathological
(seq (seq (seq ...))) tower every turn (321 open parens vs 10 close,
byte-identical across all 6 turns, indentation exploding). Doubling max_tokens
to confirm it was not mere truncation produced HTTP 413 (request too large) —
more budget yields an even larger degenerate output, confirming degeneration.

Verdict (n=1 model): Qwen3-Coder-Next wrote no working SMA in any of three
configurations; more reference material made it WORSE, not better (the lean
run 2 got the algorithm right, the fully-equipped run 3 looped); it never
converged from ail-check feedback in any run; all failures are Form-A
bracket-shape errors, never semantics. Driver + raw logs updated.
2026-06-02 17:49:23 +02:00
S
Description
No description provided
26 MiB
Languages
Rust 91.6%
Python 4.6%
C 1.8%
LLVM 1.6%
Shell 0.4%