Brummel
577bb6be43
Probe whether a foreign model can write the Series SMA worked example in
AILang Form-A from the mini-spec + Series API alone. Two live runs, neither
reached green, but a clean progression:
- Run 1 (stock API doc): 6 byte-identical turns, blocked on (app new ...) —
Qwen treats the term head as an ordinary function.
- Run 2 (new clarified as a term head): clears that; ownership threading
comes out idiomatically right (nested let s = Series.push s v), but
S-expression bracketing breaks (push sees 4 args, not 2).
Cross-cutting: Qwen clears a hurdle only with per-quirk guidance and never
converges from the ail check diagnostic (same failure repeated all 6 turns
in both runs); the failures are Form-A bracket-shape errors, not semantic.
n=1 model, single session. Driver: sma_probe.py; raw logs sma-probe-run{1,2}.md.
Live IONOS run, with consent.
532 KiB
532 KiB
SMA authoring probe — Qwen3-Coder-Next via IONOS
Model: Qwen/Qwen3-Coder-Next
Max turns: 6
Task: write the Series SMA worked example in AILang Form-A, given the Form-A mini-spec + Series API only (not the reference).
Expected stdout: '3.0\n5.33333\n5.66667\n5.33333\n'
Turn 1 — check ❌
Tokens this turn: prompt=6833, completion=2000
(module sma
(fn main
(doc "Compute SMA with window 3 over the stream 1.0,5.0,3.0,8.0,6.0,2.0 and print averages after the 3rd value onward.")
(type (fn-type (params) (ret (own (con Unit))) (effects IO)))
(params)
(body
(let s (app new (con Float) 3)
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(
ail check feedback:
error: [surface-parse-error] parse error in term: unknown term head `con`; expected one of `app`, `tail-app`, `lam`, `let`, `let-rec`, `if`, `match`, `do`, `tail-do`, `seq`, `term-ctor`, `clone`, `reuse-as`, `loop`, `recur`, `new`, `lit-unit` at byte 259
Turn 2 — check ❌
Tokens this turn: prompt=8952, completion=2000
(module sma
(fn main
(doc "Compute SMA with window 3 over the stream 1.0,5.0,3.0,8.0,6.0,2.0 and print averages after the 3rd value onward.")
(type (fn-type (params) (ret (own (con Unit))) (effects IO)))
(params)
(body
(let s (app new (con Float) 3)
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
ail check feedback:
error: [surface-parse-error] parse error in term: unknown term head `con`; expected one of `app`, `tail-app`, `lam`, `let`, `let-rec`, `if`, `match`, `do`, `tail-do`, `seq`, `term-ctor`, `clone`, `reuse-as`, `loop`, `recur`, `new`, `lit-unit` at byte 259
Turn 3 — check ❌
Tokens this turn: prompt=11071, completion=2000
(module sma
(fn main
(doc "Compute SMA with window 3 over the stream 1.0,5.0,3.0,8.0,6.0,2.0 and print averages after the 3rd value onward.")
(type (fn-type (params) (ret (own (con Unit))) (effects IO)))
(params)
(body
(let s (app new (con Float) 3)
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(
ail check feedback:
error: [surface-parse-error] parse error in term: unknown term head `con`; expected one of `app`, `tail-app`, `lam`, `let`, `let-rec`, `if`, `match`, `do`, `tail-do`, `seq`, `term-ctor`, `clone`, `reuse-as`, `loop`, `recur`, `new`, `lit-unit` at byte 259
Turn 4 — check ❌
Tokens this turn: prompt=13190, completion=2000
(module sma
(fn main
(doc "Compute SMA with window 3 over the stream 1.0,5.0,3.0,8.0,6.0,2.0 and print averages after the 3rd value onward.")
(type (fn-type (params) (ret (own (con Unit))) (effects IO)))
(params)
(body
(let s (app new (con Float) 3)
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
ail check feedback:
error: [surface-parse-error] parse error in term: unknown term head `con`; expected one of `app`, `tail-app`, `lam`, `let`, `let-rec`, `if`, `match`, `do`, `tail-do`, `seq`, `term-ctor`, `clone`, `reuse-as`, `loop`, `recur`, `new`, `lit-unit` at byte 259
Turn 5 — check ❌
Tokens this turn: prompt=15309, completion=2000
(module sma
(fn main
(doc "Compute SMA with window 3 over the stream 1.0,5.0,3.0,8.0,6.0,2.0 and print averages after the 3rd value onward.")
(type (fn-type (params) (ret (own (con Unit))) (effects IO)))
(params)
(body
(let s (app new (con Float) 3)
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
ail check feedback:
error: [surface-parse-error] parse error in term: unknown term head `con`; expected one of `app`, `tail-app`, `lam`, `let`, `let-rec`, `if`, `match`, `do`, `tail-do`, `seq`, `term-ctor`, `clone`, `reuse-as`, `loop`, `recur`, `new`, `lit-unit` at byte 259
Turn 6 — check ❌
Tokens this turn: prompt=17428, completion=2000
(module sma
(fn main
(doc "Compute SMA with window 3 over the stream 1.0,5.0,3.0,8.0,6.0,2.0 and print averages after the 3rd value onward.")
(type (fn-type (params) (ret (own (con Unit))) (effects IO)))
(params)
(body
(let s (app new (con Float) 3)
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
(seq
ail check feedback:
error: [surface-parse-error] parse error in term: unknown term head `con`; expected one of `app`, `tail-app`, `lam`, `let`, `let-rec`, `if`, `match`, `do`, `tail-do`, `seq`, `term-ctor`, `clone`, `reuse-as`, `loop`, `recur`, `new`, `lit-unit` at byte 259
Outcome: did NOT reach green after 6 turn(s).
Total tokens: prompt=72783, completion=12000.