Brummel/AILang
1
0
Fork 0
Code Issues 29 Pull Requests Actions Packages Projects Releases Wiki Activity
Files
main
AILang/experiments/2026-05-12-cross-model-authoring/qwen-yamlish.md
T
Brummel 525112e32e experiment(cma): bracket-free formats tested — they make Qwen WORSE, not better (refs #68) 
Tested the user's bracket-free ideas (indent, YAML) against the annotated-paren
result. Hypothesis was that removing the closing token removes the repetition
tail. The opposite held.

Instrument note (the watch-out paid off twice): a real YAML library auto-quotes
and gives *,-,true special meaning, which collides with the AILang operators —
Qwen wrote `- *` (multiply), a YAML alias marker, and the library crashed. That
is a measurement bug, not a model failure. Rebuilt with a hand-written literal
indent parser (atoms verbatim, no quoting); `- *` then round-trips fine.

With the instrument fixed, the result is monotone across four formats:
  format 4 annotated parens 7/8 > plain Form-A parens 6/8 > yamlish indent 3/8
  > YAML library 2/8.
More explicit structure marking helps this model; less hurts. Qwen's weakness
is structure-tracking at depth, and indentation makes it track the nesting
itself — as error-prone as matching parens but without the redundant cue. The
bracket-free surfaces fail EARLIER (L3/L4), before the L7 repetition ceiling is
even reached.

Lever for an LLM-authored language: redundant explicit structure (depth-
annotated brackets), not a lighter surface. Synthesis in format-findings.md;
raw ladders qwen-yaml.md (PyYAML, discarded) and qwen-yamlish.md (own parser).
2026-06-02 18:49:35 +02:00

545 lines
11 KiB
Markdown
Raw Permalink Blame History

# Qwen format probe — yamlish (indented tree, own literal parser)
**Model:** Qwen/Qwen3-Coder-Next; single-shot; atoms verbatim (no YAML lib, no quoting).
---
## L0_const — ✅ green
tokens: prompt=683 completion=127
model output:
```
module:
- demo_c
- fn:
- answer
- type:
- fn-type:
- params:
- ret:
- own:
- con:
- Int
- params:
- body:
- 42
- fn:
- m
- type:
- fn-type:
- params:
- ret:
- own:
- con:
- Unit
- effects:
- IO
- params:
- body:
- app:
- print
- app:
- answer
```
converted Form-A:
```
(module demo_c (fn answer (type (fn-type (params) (ret (own (con Int))))) (params) (body 42)) (fn m (type (fn-type (params) (ret (own (con Unit))) (effects IO))) (params) (body (app print (app answer)))))
```
## L1_arith — ✅ green
tokens: prompt=685 completion=84
model output:
```
module:
- demo_m
- fn:
- triple
- type:
- fn-type:
- params:
- own:
- con:
- Int
- ret:
- own:
- con:
- Int
- params:
- x
- body:
- app:
- *
- x
- 3
```
converted Form-A:
```
(module demo_m (fn triple (type (fn-type (params (own (con Int))) (ret (own (con Int))))) (params x) (body (app * x 3))))
```
## L2_print — ✅ green
tokens: prompt=680 completion=146
model output:
```
module:
- demo_c
- fn:
- m
- type:
- fn-type:
- params:
- ret:
- own:
- con:
- Int
- params:
- body:
- 7
- fn:
- main
- type:
- fn-type:
- params:
- ret:
- own:
- con:
- Unit
- effects:
- IO
- params:
- body:
- seq:
- app:
- print
- app:
- m
- do:
- io/print_str
- "\n"
```
converted Form-A:
```
(module demo_c (fn m (type (fn-type (params) (ret (own (con Int))))) (params) (body 7)) (fn main (type (fn-type (params) (ret (own (con Unit))) (effects IO))) (params) (body (seq (app print (app m)) (do io/print_str "\n")))))
```
## L3_adt — ❌ check
tokens: prompt=699 completion=241
model output:
```
module:
- demo_c
- data:
- Light
- ctor:
- Red
- ctor:
- Green
- fn:
- code
- type:
- fn-type:
- params:
- own:
- con:
- Light
- ret:
- own:
- con:
- Int
- params:
- l
- body:
- match:
- l
- case:
- pat-ctor:
- Red
- 0
- case:
- pat-ctor:
- Green
- 1
- fn:
- main
- type:
- fn-type:
- params:
- ret:
- own:
- con:
- Unit
- effects:
- IO
- params:
- body:
- seq:
- app:
- print
- app:
- code
- term-ctor:
- Light
- Red
- do:
- io/print_str
- "\n"
```
converted Form-A:
```
(module demo_c (data Light (ctor Red) (ctor Green)) (fn code (type (fn-type (params (own (con Light))) (ret (own (con Int))))) (params l) (body (match l (case (pat-ctor Red) 0) (case (pat-ctor Green) 1)))) (fn main (type (fn-type (params) (ret (own (con Unit))) (effects IO))) (params) (body (seq (app print (app code) (term-ctor Light Red)) (do io/print_str "\n")))))
```
**check:**
```
error: [arity-mismatch] main: arity mismatch for `print`: expected 1 args, got 2
```
## L4_rec — ❌ check
tokens: prompt=711 completion=360
model output:
```
module:
- demo_m
- data:
- IntList
- ctor:
- Nil
- ctor:
- Cons
- con:
- Int
- con:
- IntList
- fn:
- length
- type:
- fn-type:
- params:
- own:
- con:
- IntList
- ret:
- own:
- con:
- Int
- params:
- xs
- body:
- match:
- xs
- case:
- pat-ctor:
- Nil
- 0
- case:
- pat-ctor:
- Cons
- h
- t
- app:
- +
- 1
- app:
- length
- t
- fn:
- main
- type:
- fn-type:
- params:
- ret:
- own:
- con:
- Unit
- effects:
- IO
- params:
- body:
- let:
- xs
- term-ctor:
- IntList
- Cons
- 1
- term-ctor:
- IntList
- Cons
- 2
- term-ctor:
- IntList
- Cons
- 3
- term-ctor:
- IntList
- Nil
- seq:
- app:
- print
- app:
- length
- xs
- do:
- io/print_str
- "\n"
```
converted Form-A:
```
(module demo_m (data IntList (ctor Nil) (ctor Cons) (con Int) (con IntList)) (fn length (type (fn-type (params (own (con IntList))) (ret (own (con Int))))) (params xs) (body (match xs (case (pat-ctor Nil) 0) (case (pat-ctor Cons h t) (app + 1 (app length t)))))) (fn main (type (fn-type (params) (ret (own (con Unit))) (effects IO))) (params) (body (let xs (term-ctor IntList Cons 1 (term-ctor IntList Cons) 2 (term-ctor IntList Cons) 3 (term-ctor IntList) Nil) (seq (app print) (app length) xs) (do io/print_str "\n")))))
```
**check:**
```
error: [surface-parse-error] parse error in data-def: unknown data attribute `con`; expected `doc`, `ctor`, `drop-iterative`, or `param-in` at byte 52
```
## L5_series_len — ❌ check
tokens: prompt=975 completion=214
model output:
```
module:
- demo_m
- fn:
- main
- type:
- fn-type:
- params:
- ret:
- own:
- con:
- Unit
- effects:
- IO
- params:
- body:
- let:
- s
- new:
- Series
- con:
- Float
- 3
- let:
- s
- app:
- Series.push
- s
- 1.0
- let:
- s
- app:
- Series.push
- s
- 2.0
- let:
- s
- app:
- Series.push
- s
- 3.0
- seq:
- app:
- print
- app:
- Series.len
- s
- do:
- io/print_str
- "\n"
```
converted Form-A:
```
(module demo_m (fn main (type (fn-type (params) (ret (own (con Unit))) (effects IO))) (params) (body (let s (new Series (con Float) 3)) (let s (app Series.push s 1.0)) (let s (app Series.push s 2.0)) (let s (app Series.push s 3.0)) (seq (app print (app Series.len) s)) (do io/print_str "\n"))))
```
**check:**
```
error: [surface-parse-error] parse error: expected term, got `)` at byte 134
```
## L6_series_at — ❌ check
tokens: prompt=983 completion=225
model output:
```
module:
- demo_m
- fn:
- main
- type:
- fn-type:
- params:
- ret:
- own:
- con:
- Unit
- effects:
- IO
- params:
- body:
- let:
- s
- new:
- Series
- con:
- Float
- 3
- let:
- s1
- app:
- Series.push
- s
- 1.0
- let:
- s2
- app:
- Series.push
- s1
- 2.0
- let:
- s3
- app:
- Series.push
- s2
- 3.0
- seq:
- app:
- print
- app:
- Series.at
- s3
- 0
- do:
- io/print_str
- "\n"
```
converted Form-A:
```
(module demo_m (fn main (type (fn-type (params) (ret (own (con Unit))) (effects IO))) (params) (body (let s (new Series (con Float) 3) (let) s1 (app Series.push s 1.0) (let) s2 (app Series.push s1 2.0) (let) s3 (app Series.push s2 3.0) (seq (app print (app Series.at s3 0)) (do io/print_str "\n"))))))
```
**check:**
```
error: [surface-parse-error] parse error: expected ident (let-bound name), got `)` at byte 139
```
## L7_sma — ❌ check
tokens: prompt=1066 completion=811
model output:
```
module:
- demo_m
- fn:
- main
- type:
- fn-type:
- params:
- ret:
- own:
- con:
- Unit
- effects:
- IO
- params:
- body:
- let:
- s
- new:
- Series
- con:
- Float
- 3
- seq:
- let:
- s
- app:
- Series.push
- s
- 1.0
- seq:
- let:
- s
- app:
- Series.push
- s
- 5.0
- seq:
- app:
- print
- app:
- /
- app:
- +
- app:
- Series.at
- s
- 0
- app:
- +
- app:
- Series.at
- s
- 1
- app:
- Series.at
- s
- 2
- 3.0
- do:
- io/print_str
- "\n"
- let:
- s
- app:
- Series.push
- s
- 3.0
- seq:
-
```
converted Form-A:
```
(module demo_m (fn main (type (fn-type (params) (ret (own (con Unit))) (effects IO))) (params) (body (let s (new Series (con Float) 3) (seq (let s (app Series.push s 1.0) (seq (let s (app Series.push s 5.0)) (seq (app print (app /) (app +) (app Series.at s 0) (app +) (app Series.at s 1) (app Series.at s 2) 3.0)) (do io/print_str "\n"))) (let s (app Series.push s 3.0)) (seq (app print) (app /) (app +) (app Series.at) s 0 (app +) (app Series.at) s 1 (app Series.at) s 2 3.0) (do io/print_str "\n") (let s (app) Series.push s 8.0) (seq (app) print (app) / (app) + (app) Series.at s 0 (app) + (app) Series.at s 1 (app) Series.at s 2 3.0) (do io/print_str "\n") (let s (app) Series.push s 6.0) (seq (app) print (app) / (app) + (app) Series.at s 0 (app) + (app) Series.at s 1 (app) Series.at s 2 3.0) (do io/print_str "\n") (let s (app) Series.push s 2.0) (seq (app) print (app) / (app) + (app) Series.at s 0 (app) + (app) Series.at s 1 (app) Series.at s 2 3.0) (do io/print_str "\n"))))))
```
**check:**
```
error: [surface-parse-error] parse error: expected term, got `)` at byte 206
```
---
## Ladder
- L0_const: ✅ green
- L1_arith: ✅ green
- L2_print: ✅ green
- L3_adt: ❌ check
- L4_rec: ❌ check
- L5_series_len: ❌ check
- L6_series_at: ❌ check
- L7_sma: ❌ check
Reference in New Issue View Git Blame Copy Permalink