Brummel/AILang
1
0
Fork 0
Code Issues 29 Pull Requests Actions Packages Projects Releases Wiki Activity
Files
682b935ba325a2c17070d689a8c88a02b9f44a50
AILang/experiments/2026-05-12-cross-model-authoring/qwen-fmt4.md
T
Brummel 682b935ba3 experiment(cma): annotated-paren format helps Qwen — cracks the bracket wall (refs #68) 
Tests the user's hypothesis: if no human reads the surface, lift the paren-
counting burden. Format 4 = annotated parens, every paren carries its nesting
depth. Discipline: the format has no ail parser, so the converter is round-trip
verified (parse-identity on known-good demos) before judging any model output.

Result: it helps. Plain Form-A 6/8 -> format 4 7/8 on the same ablation ladder.
The depth annotation cracks the bracket-balance wall that broke L4. At the
hardest level (L7 SMA) Qwen produced perfectly balanced brackets through the
deepest nesting AND correct SMA logic; the only remaining issue was a construct
quirk (it treats seq as variadic, but AILang seq is binary). Proof the blocks
were right: Qwen's exact logic with a manual binary-seq fix runs and emits the
exact expected SMA output.

Pushing further (format 4 + explicit seq-binary hint) fixed the seq nesting but
L7 still failed on two MODEL-behaviour limits, not encoding: it simplified the
logic, and fell into a closing-token repetition tail (hundreds of #0) ). The
tail is paren-driven, so the natural next test is a bracket-free format
(indent / keyword-delimited) with no closing token to loop on — a deliberate
next step needing its own verified converter. Synthesis in format-findings.md.
2026-06-02 18:30:24 +02:00

10 KiB
Raw Blame History

Qwen format probe 4 — annotated parens (#N ... #N)

Model: Qwen/Qwen3-Coder-Next; single-shot per level; annotated-paren dialect; output stripped via round-trip-verified converter then ail-checked.

L0_const — ✅ green

tokens: prompt=675 completion=74

annotated output:

(#0 module m
  (#1 fn answer
    (#2 type (#3 fn-type (#4 params #4) (#4 ret (#5 own (#6 con Int #6) #5) #4) #3) #2)
    (#2 params #2)
    (#2 body 42 #2) #1) #0)

stripped to Form-A:

(module m
  (fn answer
    (type (fn-type (params) (ret (own (con Int)))))
    (params)
    (body 42)))

L1_arith — ✅ green

tokens: prompt=677 completion=95

annotated output:

(#0 module m
  (#1 fn triple
    (#2 type (#3 fn-type (#4 params (#5 own (#6 con Int #6) #5) #4) (#4 ret (#5 own (#6 con Int #6) #5) #4) #3) #2)
    (#2 params x #2)
    (#2 body (#3 app * x 3 #3) #2) #1) #0)

stripped to Form-A:

(module m
  (fn triple
    (type (fn-type (params (own (con Int))) (ret (own (con Int)))))
    (params x)
    (body (app * x 3))))

L2_print — ✅ green

tokens: prompt=672 completion=105

annotated output:

(#0 module m
  (#1 fn main
    (#2 type (#3 fn-type (#4 params #4) (#4 ret (#5 own (#6 con Unit #6) #5) #4) (#4 effects IO #4) #3) #2)
    (#2 params #2)
    (#2 body (#3 seq (#4 app print 7 #4) (#4 do io/print_str "\n" #4) #3) #2) #1) #0)

stripped to Form-A:

(module m
  (fn main
    (type (fn-type (params) (ret (own (con Unit))) (effects IO)))
    (params)
    (body (seq (app print 7) (do io/print_str "\n")))))

L3_adt — ✅ green

tokens: prompt=691 completion=151

annotated output:

(#0 module m
  (#1 data Light (#2 ctor Red #2) (#2 ctor Green #2) #1)
  (#1 fn code
    (#2 type (#3 fn-type (#4 params (#5 own (#6 con Light #6) #5) #4) (#4 ret (#5 own (#6 con Int #6) #5) #4) #3) #2)
    (#2 params x #2)
    (#2 body (#3 match x
      (#4 case (#5 pat-ctor Red #5) 0 #4)
      (#4 case (#5 pat-ctor Green #5) 1 #4) #3) #2) #1) #0)

stripped to Form-A:

(module m
  (data Light (ctor Red) (ctor Green))
  (fn code
    (type (fn-type (params (own (con Light))) (ret (own (con Int)))))
    (params x)
    (body (match x
      (case (pat-ctor Red) 0)
      (case (pat-ctor Green) 1)))))

L4_rec — ✅ green

tokens: prompt=703 completion=349

annotated output:

(#0 module m
  (#1 data IntList (#2 ctor Nil #2) (#2 ctor Cons (#3 con Int #3) (#3 con IntList #3) #2) #1)
  (#1 fn length
    (#2 type (#3 fn-type (#4 params (#5 own (#6 con IntList #6) #5) #4) (#4 ret (#5 own (#6 con Int #6) #5) #4) #3) #2)
    (#2 params xs #2)
    (#2 body
      (#3 match xs
        (#4 case (#5 pat-ctor Nil #5) 0 #4)
        (#4 case (#5 pat-ctor Cons h t #5) (#5 app + 1 (#6 app length t #6) #5) #4) #3) #2) #1)
  (#1 fn main
    (#2 type (#3 fn-type (#4 params #4) (#4 ret (#5 own (#6 con Unit #6) #5) #4) (#4 effects IO #4) #3) #2)
    (#2 params #2)
    (#2 body
      (#3 let xs (#4 term-ctor IntList Cons 1 (#5 term-ctor IntList Cons 2 (#6 term-ctor IntList Cons 3 (#7 term-ctor IntList Nil #7) #6) #5) #4)
        (#4 seq (#5 app print (#6 app length xs #6) #5) (#5 do io/print_str "\n" #5) #4) #3) #2) #1) #0)

stripped to Form-A:

(module m
  (data IntList (ctor Nil) (ctor Cons (con Int) (con IntList)))
  (fn length
    (type (fn-type (params (own (con IntList))) (ret (own (con Int)))))
    (params xs)
    (body
      (match xs
        (case (pat-ctor Nil) 0)
        (case (pat-ctor Cons h t) (app + 1 (app length t))))))
  (fn main
    (type (fn-type (params) (ret (own (con Unit))) (effects IO)))
    (params)
    (body
      (let xs (term-ctor IntList Cons 1 (term-ctor IntList Cons 2 (term-ctor IntList Cons 3 (term-ctor IntList Nil))))
        (seq (app print (app length xs)) (do io/print_str "\n"))))))

L5_series_len — ✅ green

tokens: prompt=964 completion=201

annotated output:

(#0 module m
  (#1 fn main
    (#2 type (#3 fn-type (#4 params #4) (#4 ret (#5 own (#6 con Unit #6) #5) #4) (#4 effects IO #4) #3) #2)
    (#2 params #2)
    (#2 body
      (#3 let s (#4 new Series (#5 con Float #5) 3 #4)
        (#4 let s (#5 app Series.push s 1.0 #5)
          (#5 let s (#6 app Series.push s 2.0 #6)
            (#6 let s (#7 app Series.push s 3.0 #7)
              (#7 seq (#8 app print (#9 app Series.len s #9) #8) (#8 do io/print_str "\n" #8) #7) #6) #5) #4) #3) #2) #1) #0)

stripped to Form-A:

(module m
  (fn main
    (type (fn-type (params) (ret (own (con Unit))) (effects IO)))
    (params)
    (body
      (let s (new Series (con Float) 3)
        (let s (app Series.push s 1.0)
          (let s (app Series.push s 2.0)
            (let s (app Series.push s 3.0)
              (seq (app print (app Series.len s)) (do io/print_str "\n")))))))))

L6_series_at — ✅ green

tokens: prompt=972 completion=203

annotated output:

(#0 module m
  (#1 fn main
    (#2 type (#3 fn-type (#4 params #4) (#4 ret (#5 own (#6 con Unit #6) #5) #4) (#4 effects IO #4) #3) #2)
    (#2 params #2)
    (#2 body
      (#3 let s (#4 new Series (#5 con Float #5) 3 #4)
        (#4 let s (#5 app Series.push s 1.0 #5)
          (#5 let s (#6 app Series.push s 2.0 #6)
            (#6 let s (#7 app Series.push s 3.0 #7)
              (#7 seq (#8 app print (#9 app Series.at s 0 #9) #8) (#8 do io/print_str "\n" #8) #7) #6) #5) #4) #3) #2) #1) #0)

stripped to Form-A:

(module m
  (fn main
    (type (fn-type (params) (ret (own (con Unit))) (effects IO)))
    (params)
    (body
      (let s (new Series (con Float) 3)
        (let s (app Series.push s 1.0)
          (let s (app Series.push s 2.0)
            (let s (app Series.push s 3.0)
              (seq (app print (app Series.at s 0)) (do io/print_str "\n")))))))))

L7_sma — ❌ check

tokens: prompt=1054 completion=1800

annotated output:

(#0 module m
  (#1 fn main
    (#2 type (#3 fn-type (#4 params #4) (#4 ret (#5 own (#6 con Unit #6) #5) #4) (#4 effects IO #4) #3) #2)
    (#2 params #2)
    (#2 body
      (#3 let s (#4 new Series (#5 con Float #5) 3 #4)
        (#4 let s (#5 app Series.push s 1.0 #5)
          (#5 let s (#6 app Series.push s 5.0 #6)
            (#6 let s (#7 app Series.push s 3.0 #7)
              (#7 seq (#8 app print (#9 app / (#10 app + (#11 app Series.at s 0 #11) (#12 app + (#13 app Series.at s 1 #13) (#14 app Series.at s 2 #14) #12) #11) 3.0 #9) #8) (#8 do io/print_str "\n" #8) (#8 let s (#9 app Series.push s 8.0 #9)
                (#9 seq (#10 app print (#11 app / (#12 app + (#13 app Series.at s 0 #13) (#14 app + (#15 app Series.at s 1 #15) (#16 app Series.at s 2 #16) #14) #12) 3.0 #11) #10) (#10 do io/print_str "\n" #10) (#10 let s (#11 app Series.push s 6.0 #11)
                  (#11 seq (#12 app print (#13 app / (#14 app + (#15 app Series.at s 0 #15) (#16 app + (#17 app Series.at s 1 #17) (#18 app Series.at s 2 #18) #16) #14) 3.0 #13) #12) (#12 do io/print_str "\n" #12) (#12 let s (#13 app Series.push s 2.0 #13)
                    (#13 seq (#14 app print (#15 app / (#16 app + (#17 app Series.at s 0 #17) (#18 app + (#19 app Series.at s 1 #19) (#20 app Series.at s 2 #20) #18) #16) 3.0 #15) #14) (#14 do io/print_str "\n" #14) (#14 do io/print_str "\n" #14) #13) #12) #11) #10) #9) #8) #7) #6) #5) #4) #3) #2) #1) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0) #0)

stripped to Form-A:

(module m
  (fn main
    (type (fn-type (params) (ret (own (con Unit))) (effects IO)))
    (params)
    (body
      (let s (new Series (con Float) 3)
        (let s (app Series.push s 1.0)
          (let s (app Series.push s 5.0)
            (let s (app Series.push s 3.0)
              (seq (app print (app / (app + (app Series.at s 0) (app + (app Series.at s 1) (app Series.at s 2))) 3.0)) (do io/print_str "\n") (let s (app Series.push s 8.0)
                (seq (app print (app / (app + (app Series.at s 0) (app + (app Series.at s 1) (app Series.at s 2))) 3.0)) (do io/print_str "\n") (let s (app Series.push s 6.0)
                  (seq (app print (app / (app + (app Series.at s 0) (app + (app Series.at s 1) (app Series.at s 2))) 3.0)) (do io/print_str "\n") (let s (app Series.push s 2.0)
                    (seq (app print (app / (app + (app Series.at s 0) (app + (app Series.at s 1) (app Series.at s 2))) 3.0)) (do io/print_str "\n") (do io/print_str "\n"))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))

ail check:

error: [surface-parse-error] parse error: expected `)` (end of seq-term), got `(` at byte 415

Ladder

  • L0_const: ✅ green
  • L1_arith: ✅ green
  • L2_print: ✅ green
  • L3_adt: ✅ green
  • L4_rec: ✅ green
  • L5_series_len: ✅ green
  • L6_series_at: ✅ green
  • L7_sma: ❌ check
Reference in New Issue View Git Blame Copy Permalink