Brummel
9c833f65e4
Every surface run in format-findings.md hit the same L7 (SMA) wall, read so far as a "genuine single-shot complexity ceiling". Three controls on the plain Form-A baseline (identical L7 prompt + ail oracle) pin down what that ceiling actually is. Control 1 — blind frontier model. Ran the full L0–L7 ladder through eight fresh, clueless Claude Opus 4.8 agents, each given verbatim the same prompt the IONOS models got, with no repo access, no compiler, no oracle (all returned tool_uses: 0). Oracled afterward exactly as the harness oracles Qwen/Llama: 8/8 green, including L7. So L7 is not a language ceiling — Form-A is authorable, SMA included, by a strong enough model. The eight modules are kept under clueless-agents/ (one m.ail per dir to satisfy module-name=stem) and re-check green; L7 also runs to the expected output. Claude generalised operators never shown in the few-shot (float +, a / absent from every example, float print). Control 2 — agentic Qwen (qwen_agentic.py). The fairer question: does the compiler-as-a-tool close the gap? Runs the real Claude-Code loop — model proposes a module, harness runs ail, the raw unedited compiler output goes back into the dialogue, up to 8 turns. Result: never solved. From turn 1 Qwen collapses into a repetition loop (~274× "(seq" until max_tokens, completion=1500 every turn); the module is truncated mid-cascade, the oracle returns the correct parse error every turn, and Qwen repeats. Tool access is not an equaliser — it lifts a model only when its reasoning already operates in the right neighbourhood. Also settles the context question: turn 1 collapses at prompt=729 (no accumulated history), so the failure is "can't", not "drowns in context". Raw doc qwen-agentic-sma.md (repetition cascades collapsed to keep the repo small; the collapse marker records the original run length). Control 3 — same SMA in Python (qwen_python_sma.py). Is the AILang failure about coding ability or the unfamiliar surface? Given the same task zero-shot in Python, Qwen returns a clean, idiomatic, correct sliding-window in 113 tokens (verified by inspection + hand-trace; the harness deliberately does not auto-execute LLM code). 113 tokens of correct Python vs a 1500-token collapse in AILang, same model and temperature. Synthesis: the L7 wall is neither an algorithm ceiling nor a "model is weak" ceiling. Qwen owns the algorithm (Python) and a frontier model owns the surface (Claude, blind, first try). Qwen's AILang collapse is specifically the unfamiliar, fully-parenthesised surface it never saw in training, and a compiler-in-the-loop does not rescue it. For an LLM-authored language: making Form-A compiler-driven does not lower the model bar — AILang either targets frontier-class authors, or its surface must drop the structure-tracking load enough for a small model to keep its place. Live IONOS limited to the consented Qwen agentic + Python runs.
6 lines
156 B
Plaintext
6 lines
156 B
Plaintext
(module m
|
|
(fn main
|
|
(type (fn-type (params) (ret (own (con Unit))) (effects IO)))
|
|
(params)
|
|
(body (seq (app print 7) (do io/print_str "\n")))))
|