Files
AILang/docs/roadmap.md
T
Brummel e91e31aadb iter cma.3: live Qwen3-Coder-Next run + DESIGN.md §Decision-6 empirical addendum + milestone close
Milestone "Cross-model authoring-form test" closed. Live IONOS run
against Qwen/Qwen3-Coder-Next executed at temperature=0, max-turns=5,
token-budget=500000. Raw dataset under runs/2026-05-12-df7531/:
RUN_STATUS=ok, scores.csv with 8 rows (4 tasks × 2 cohorts),
summary.md, plus 156 per-turn raw artefacts (request/response/program/
stderr/stdout) preserved for any future analysis.

Headline numbers:
- AILX cohort: 2/4 green, 1/4 first-attempt, ~95k prompt + ~2.6k completion
- JSON cohort: 1/4 green, 0/4 first-attempt, ~192k prompt + ~15k completion
- Only first-attempt success was AILX on t3_main_prints (~5 lines of
  tagged s-expr vs ~23 lines of structured JSON for the JSON cohort's
  turn-1 attempt that needed correction).
- Per spec, single subject + single deterministic run = data point,
  not verdict.

DESIGN.md §"Decision 6: authoring surface" gains an "Empirical
addendum (2026-05-12)" subsection: 6-row metric table, illustrative
t3 contrast, explicit single-subject scope note pointing at the
roadmap follow-up entry for multi-subject expansion.

Roadmap: P2 entry "Cross-model authoring-form test" removed (this
journal is the convention's one-line mirror); P3 entry
"Multi-subject expansion (cross-model authoring-form follow-up)"
added with the run dir named as baseline.

Milestone artefacts span three iters (cma.1 master mini-spec +
renderer; cma.2 harness + tasks + reference solutions; cma.3 this
live run + DESIGN.md addendum + close).
2026-05-12 12:16:32 +02:00

15 KiB
Raw Blame History

AILang Roadmap

Priority-ordered list of upcoming work — milestones, features, todos, and ideas. The orchestrator maintains this file. The user can request additions; the orchestrator chooses what to remove or reprioritise as work progresses.

Conventions

  • One checkbox per entry. - [ ] is open; - [~] is in progress (work has started — a plan exists, a branch is open, or commits are landing); - [x] is done. A finished entry may stay briefly for context, then is removed (with a one-line mirror in docs/journals/).
  • Each entry is tagged by kind and lives under a priority bucket:
    • [milestone] — big chunk that will get a docs/specs/<milestone>.md.
    • [feature] — smaller addition inside a milestone, no full spec.
    • [todo] — concrete task that can run without a brainstorm (cleanup, doc fix, mechanical refactor, test backfill).
    • [idea] — not yet decision-ready, no commitment.
  • Optional depends on: line names another entry that has to land first.
  • Optional context: line points to the journal entry (per-iter file under docs/journals/ or, for pre-2026-05-11 entries, the archived docs/journal-archive.md) where the rationale lives. The roadmap is intentionally terse; rationale stays in the journals.
  • Priority buckets:
    • P0 — in flight. Spec or plan already exists.
    • P1 — next up. Decision made; not yet started.
    • P2 — medium-term. Decided in principle, scheduled later.
    • P3 — ideas. No commitment; may be cut.

P0 — In flight

(empty — canonical-type-names milestone closed 2026-05-11; Floats closed 2026-05-10 and was removed once it stopped being load-bearing context. Pick the next milestone from P1.)

P1 — Next

  • [milestone] Heap-Str ABI — runtime infrastructure for malloc-backed, refcounted Str values alongside the existing static @.str_* globals. Today the Str path is static-only (DESIGN.md §"Float semantics" notes this explicitly), so any primitive that needs to produce a Str at runtime — int_to_str, float_to_str (currently type-installed but CodegenError::Internal on call), eventual Show.show, future ++ on strings — cannot ship. Scope: pick the representation (likely a {rc_header, len, bytes…} slab consistent with the rest of the RC runtime), teach codegen to accept both static and heap Str at the same ABI slot, wire RC inc/drop/clone/compare/eq on the heap form, and ship int_to_str + float_to_str as the first two callers so the ABI gets exercised end-to-end. Unblocks Show + print rewire below.

    • context: DESIGN.md §"Float semantics" (float_to_str is type- installed, codegen-deferred); spec 2026-05-11-23-eq-ord-prelude §"Show. Defers behind a heap-Str-ABI milestone"; spec 2026-05-10-fieldtest-floats §F4 ("dynamic Str allocation in the runtime"); spec 2026-05-09-22-typeclasses §22b.4b ("Show#Int needs an int_to_str primitive returning heap-allocated Str").
  • [milestone] Post-22 Prelude — Show + print rewire — ship Show typeclass with Show Int/Bool/Str/Float instances; rewire print through Show.show. Originally queued as 22b.4 in the typeclass milestone, kept on hold pending demand and the heap-Str prerequisite.

    • depends on: Heap-Str ABI (above). Without it, Show Int cannot allocate a Str to return.
    • context: brainstorm 2026-05-12 (iter 23.5 wrap).

P2 — Medium-term

  • [feature] Operator routing through Eq / Ord==, < etc. resolved via the typeclass instead of the built-in primitive comparators. No commitment; gated on bench re-baselining to make sure the indirection doesn't tank latency.

    • context: JOURNAL 2026-05-09
    • depends on: Post-22 Prelude — Eq/Ord (shipped 23.5)
  • [todo] types / ctor_index overlay shape question — decide whether the env's two parallel ctor maps should collapse into one overlay, or stay split. Surfaced during the env-construction unify audit.

    • context: JOURNAL 2026-05-10 ("Audit close: env-construction unify")
  • [todo] CLI human-mode diagnostic surface for WorkspaceLoadError — non-JSON ail check routes ct.1 errors via anyhow/thiserror Display (Error: <message>) instead of going through workspace_error_to_diagnostic, so the bracketed [code] prefix the JSON path carries is missing. Plausibly applies to every CLI subcommand that calls load_workspace(&path)? directly.

    • context: JOURNAL 2026-05-11 ("Iteration ct.1") — surfaced by ct.1.8 tester.
  • [todo] Retire dead KindMismatch arm — validate_classdefs's walk_kind_mismatch path is structurally unreachable through well-formed schema post-ct.1 (the canonical-form validator catches the malformed Type::Con { name: param } shape earlier). The enum variant + walk_kind_mismatch helper stay as dead-but-defensive code; a future tidy can delete both.

    • context: JOURNAL 2026-05-11 ("Iteration ct.1").
  • [todo] Re-key Registry.type_def_module to handle bare-name-collision-across-modules — BTreeMap<String, String> keyed by bare type name silently overwrites when two modules each define type Foo; normalize_type_for_registry would then collapse M.Foo and N.Foo to whichever insert won. Acceptable for current corpus (distinct bare type names across modules), but the proper fix is to key by (owning_module, bare_name) → defining_module.

    • context: JOURNAL 2026-05-11 ("Iteration ct.1") — flagged by ct.1.5a quality reviewer.
  • [feature] 22c — typeclass corpus expansion. User-defined classes beyond the prelude four; multi-parameter classes; superclass chains; richer instance bodies. Deferred from milestone 22.

    • context: JOURNAL 2026-05-09
  • [milestone] Module-qualified class names + type-driven method dispatch — retire the MethodNameCollision workaround (crates/ailang-core/src/workspace.rs:472) that today keeps bare class names viable by enforcing workspace-global method-name uniqueness. Replaces name-driven method resolution (ModuleGlobals::class_methods: IndexMap<String, ClassMethodEntry>, crates/ailang-check/src/lib.rs:983; consumed by mono::rewrite_class_method_calls, crates/ailang-check/src/mono.rs:657) with type-driven dispatch — look up eq candidates by argument type, pick the unique class instance, fail closed on ambiguity. Once shipped, two libraries can each declare class Eq with their own eq. Carries its own DESIGN spec: inference rules, ambiguity diagnostics, and the canonical-form extension for class-reference fields (InstanceDef.class, SuperclassRef.class, Constraint.class) that the canonical-type-names milestone explicitly out-of-scoped.

    • context: docs/specs/2026-05-10-canonical-type-names.md "Out of scope: Class names" — the workaround is named there so it stays visible until this milestone retires it.
  • [todo] Boehm full retirement — remove the transitional Boehm GC path now that RC + uniqueness is the canonical memory story.

    • context: JOURNAL pre-22, "Boehm transitional"
  • [feature] Closure-pair slab / pool — codegen tweak to pool the env+code closure pairs instead of one-shot heap allocs. Bench-gated.

  • [todo] FnDef::synthetic flag — formalise the monomorphiser-emitted FnDefs so downstream passes can tell user-authored from synthesised at a glance. Currently inferred from symbol naming.

  • [todo] 21'h iteration — final 21' carry-over (latency methodology pass). Numbering kept for continuity with the 21' arc.

  • [todo] io/print_float always-emit-.0 — surface printer always emits . or e/E so re-lex routes to Float; the runtime printer (printf("%g\n", v)) doesn't, so 2.0 prints as 2 (Int-shaped). Asymmetric. Either switch the runtime path to a .0-fallback printer (matching surface) or document the %g contract in DESIGN.md §"Float semantics" so the LLM-author knows io/print_float's output is for-humans not round-trip.

    • context: docs/specs/2026-05-10-fieldtest-floats.md finding F1.
  • [todo] Rustdoc warning sweep — cargo doc --no-deps reports 16 pre-existing warnings (15 in ailang-check, 1 in ailang-core: private-item links from public doc, unresolved intra-crate links). All predate the design-md-consolidation milestone; treat as a one-off sweep.

    • context: JOURNAL 2026-05-10 ("Audit close: design-md-consolidation").
  • [todo] design_schema_drift.rs fidelity widening — current test checks anchor presence anywhere in DESIGN.md; the audit found that [high] schema gaps in §"Data model" are invisible because anchors live in Decision 11 instead. Constrain the test to scan only §"Data model" + ParamMode block, or extract JSON-schema blocks into a machine-readable file the test consumes.

    • context: JOURNAL 2026-05-10 ("Audit close").
  • [todo] Split BadCrossModuleTypeRef into two diagnostics — the current single shape collapses unknown-owner and known-owner / unknown-type-in-owner into one message. The two cases suggest different fixes (add (import <owner>) vs. fix the type name). In the known-owner branch, list the owner's available type defs as candidates the way bare-cross-module-type-ref lists candidates from imports.

    • context: fieldtest 2026-05-11 — examples/ct_3*.ailx exhibits both branches with identical-shape diagnostics.
  • [todo] Workspace search beyond entry-module's directory — load_workspace only finds sibling .ail.json files in the same directory as the entry module, so any consumer of prelude/std in a subdirectory has no way to resolve cross-module imports. Add either a --workspace-root flag on ail check / ail build / ail run, or upward-search from the entry module's directory. Alternatively ratify the flat-workspace assumption in DESIGN.md if intentional.

    • context: fieldtest 2026-05-11 — fieldtest fixtures could not be placed under examples/fieldtest/ because of this; predates the canonical-type-names milestone but surfaces every time.
  • [todo] ail check/build/run accept .ailx extension — today ail check foo.ailx produces a misleading JSON-parse error (json: expected value at line 1 column 1) because the loader only accepts .ail.json. Either teach the CLI subcommands to auto-parse .ailx internally, or detect the extension and emit a "did you mean ail parse foo.ailx?" hint. The LLM-author's natural first command should not be the one that produces a misleading diagnostic.

    • context: fieldtest 2026-05-11 — orthogonal to canonical-type-names but every fieldtest invocation hits this on its first command.
  • [todo] check_in_workspace per-module overlay narrowing — crates/ailang-check/src/lib.rs:1234 still clears+rebuilds env.ctor_index per-module; ct.3.2 narrowed the analogous mono overlay to types-only. The typecheck-side overlay's env.ctor_index half serves the duplicate-detection diagnostic at workspace-build time, not the runtime ctor lookup (which is type-driven post-ct.2.2). Narrowing is mechanical but needs a careful read to confirm no other consumer survives.

    • context: JOURNAL 2026-05-11 ("Iteration ct.4") — milestone close follow-up.

P3 — Ideas

  • [todo] compare_primitives_smoke.ailx counterpart — the canonical happy-path exhibit for canonical-type-names ships only as JSON. Per Decision 6, Surface is the LLM-author surface; the milestone should have a .ailx counterpart. Two paths: author examples/compare_primitives_smoke.ailx to round-trip into the existing JSON, OR retire the JSON fixture in favour of examples/ct_1_ordering_signum.{ailx,ail.json} as the new canonical happy-path exhibit.

    • context: fieldtest 2026-05-11 (spec_gap).
  • [todo] Codegen lookup_ctor_in_pattern type-anchoring — crates/ailang-codegen/src/lib.rs:1790 still walks every module's ctor_index by bare ctor name. Plumbing the scrutinee's qualified Type::Con through pattern lowering would type-anchor it symmetric to the ct.2.2 typecheck-side fix. Not load-bearing (uniqueness is enforced at typecheck), but cleaner.

    • context: JOURNAL 2026-05-11 ("Iteration ct.3" + ct.4 close).
  • [todo] compare_primitives_smoke IR-shape assertion — observe compare__Int / compare__Bool / compare__Str symbols in the emitted IR. Blocked on emit-ir CLI not running mono; resolution paths include extending the CLI to run mono, using library APIs directly in e2e.rs, or adding --dump-ir to ail build. The E2E stdout assertion already covers correctness; the IR-shape test would catch refactor regressions that rename mono symbols.

    • context: JOURNAL 2026-05-11 ("Iteration ct.4") — dropped from ct.4.4 when the BLOCKED condition surfaced.
  • [idea] Latency methodology rework — switch from per-run timing to a histogram-based approach so tail-latency regressions are visible without re-running. Queued from 21'g.

  • [idea] Parser-test backfill for 22b.4a-era duplicate-clause sites (class × 3, class method × 2, instance × 3, instance method × 1). No regression pin today; only worth it if a 22b.4a-era diagnostic needs to change.

    • context: JOURNAL 2026-05-10 ("Iteration 22-tidy.7")
  • [idea] write_type Type::Forall arm in crates/ailang-prose/src/lib.rs silently drops constraints in inline-type rendering. Dormant — surface forms today only carry forall at fn-signature top level. Fix only if a future feature carries forall + constraints inline.

    • context: JOURNAL 2026-05-10 ("Iteration 22-tidy.6")
  • [idea] Richer integration paths between RC and uniqueness — deferred from the 21' arc; revisit once the uniqueness inference covers more program shapes.

  • [milestone] Multi-subject expansion (cross-model authoring-form follow-up) — extend the cross-model authoring-form test to ≥2 additional foreign LLMs and ≥4 more tasks (broader surface coverage; current MVP run hit only 3/8 green so the failure clusters dominate the signal). Optional: repetitions for statistical robustness; calibrate strip_locations to also strip at byte N from AILX-cohort feedback so the asymmetry observed in the cma.3 baseline is removed before re-measuring. Goal: turn the single-subject data point recorded in DESIGN.md §"Decision 6 / Empirical addendum (2026-05-12)" into either a ratification or retirement of Decision 6's universal claim. Baseline run: experiments/2026-05-12-cross-model-authoring/runs/2026-05-12-df7531/.

    • context: JOURNAL 2026-05-12 ("iter cma.3"). Other IONOS catalogue models named in the brainstorm record (Meta-Llama-3.1-405B, Llama-3.3-70B, gpt-oss-120b, Mistral-Small-24B, CodeLlama-13b) are reasonable next subjects.