# Why does Qwen3-Coder-Next fail at AILang Form-A? — ablation finding

**Date:** 2026-06-02  **Model:** Qwen/Qwen3-Coder-Next (IONOS, temp 0.2)
**Method:** try-and-error from a minimum upward. A ladder L0 (trivial) → L7
(full SMA), each level run independently with minimal few-shot context, single
shot. Two ladders: ladder 1 with a base few-shot that only *destructures*
(`match`); ladder 2 after adding one example that *constructs* (`term-ctor`).
Raw logs: `qwen-ablation-min.md` (ladder 1), `qwen-ablation-ctor.md` (ladder 2).
Driver: `qwen_ablation.py`.

## The two ladders

| Level | task | ladder 1 (min) | ladder 2 (+term-ctor example) |
|---|---|---|---|
| L0 | fn returns 42 | ✅ | ✅ |
| L1 | fn ×3 | ✅ | ✅ |
| L2 | main prints 7 | ✅ | ✅ |
| L3 | ADT + match | ❌ check | ✅ |
| L4 | build list + recursive length | ❌ `Cons` unbound | ❌ paren misplace |
| L5 | Series, print len | ❌ missing newline | ✅ |
| L6 | Series.at | ✅ | ✅ |
| L7 | full SMA | ❌ check | ❌ paren imbalance |

One example of the missing construct moved Qwen from 4/8 to **6/8**.

## The answer: it is NOT general incapacity — there are two distinct walls

**Qwen is not "stupid" at Form-A.** With minimal, *correct* few-shot context
it writes 6 of 8 tasks green — including ADTs, `match`, recursive functions,
and Series library use (`new Series`, `push`, `at`, `len`). The two failures
are specific and different in kind:

### Wall 1 — unfamiliar constructs (SOLVABLE by one example)

Qwen does not know constructs it hasn't seen, and guesses plausibly wrong:
- L4 ladder 1: it built a list with `(app Cons 1 (app Cons 2 …))` and bare
  `Nil` — calling the constructor like a function → `unbound-var: Cons`. It
  had only seen `match`/`pat-ctor` (destructuring), never `term-ctor`
  (construction).
- Adding ONE few-shot that uses `term-ctor` flipped L3 and L5 to green, and
  removed the `Cons`-as-function error in L4. The model generalises from a
  single example of a construct.

This is the cheap wall: feed the construct once and it clears.

### Wall 2 — paren/nesting discipline at depth (NOT solved by examples)

The two that stayed red (L4, L7) both fail on **S-expression bracketing**,
not on any missing knowledge:
- L4 ladder 2: the `length` function is flawless. `main` builds
  `(let xs (term-ctor … (term-ctor … (term-ctor … (term-ctor IntList Nil)))`
  — closing the nested `term-ctor` value one paren too early, so the `let`
  body slides inside the value. **Total paren count is balanced (43 open / 43
  close)** — the parens are merely *misplaced*. The model is not under-closing
  globally; it loses track of the nesting structure at depth.
- L7 (SMA): 49 open / 47 close — under-closed.

The grin/red split tracks **nesting depth**, not feature: every green task is
shallow (one or two levels), every red one is deeply nested (4× nested
`term-ctor` in a `let`; the SMA's stream-loop composition). This is a
mechanical limit on tracking deep paren nesting, and examples do not fix it —
consistent with the earlier full-SMA probes (run 2: correct ownership
threading, died on brackets; run 3: ~80k-token context tipped the same
weakness into a non-terminating `(seq (seq …))` repetition loop).

## Takeaway

The bottleneck for this model on Form-A is the **fully-parenthesised surface
itself** at depth, not the language's semantics or its vocabulary. Two
levers follow, both deliberate next steps, not run here:
1. Few-shot every non-obvious construct (cheap, clears Wall 1).
2. For Wall 2, the surface is the problem — the natural test is the
   **`.ail.json` authoring form**, which removes human-style paren-counting
   (structure is carried by JSON nesting the model is far more trained on).
   The cma harness already renders both cohorts; this is the obvious A/B.