Brummel
5bd148a607
Three-cohort harness against IONOS-hosted Qwen3-Coder-Next, testing
whether application-head tag (`app` vs `apply` vs `call`) affects
LLM authoring success. Cohorts share lambda/constructor/typecon tag
renamings (the non-discriminator changes); only the head differs.
Result: 100 % cohort-treue — Qwen writes whatever the spec teaches,
no measurable natural preference. Pipeline pass-rate (classic 1/8,
apply 0/8, call 1/8) is cohort-independent. The four task failures
are general Form-A friction, not naming-related. Refactor not
justified per the feature-acceptance gate.
Tracked harness:
- 2026-05-12-cross-model-authoring/rename-spec.py — regex tag
rewriter for the two non-classic spec variants
- 2026-05-21-naming-ab/run.py, reprocess.py — three-cohort runner
+ post-processing with disjoint-discriminator counting and
module-name-matched temp dirs
Generated artefacts (runs/, rendered/ail-{renamed,call}.md) are
gitignored — regeneratable from rename-spec.py + run.py.
Side-effects filed against current spec/schema bugs surfaced during
the run:
- refs #28 spec teaches (ctor X) for term position; parser requires
(term-ctor X) — 100 % t4 failure across cohorts
- refs #29 io/print_str appends newline (de facto println);
spec now documents the behavior, code question still open
- refs #30 schema camelCase outlier: paramTypes/retType in Term::Lam
Spec patch in rendered/ail.md:
- replaces stale io/print_int references with io/print_str (bitrot
from the print-builtin consolidation)
- documents io/print_str's trailing-newline behavior
- corrects the prelude claim about int_to_str (IS in prelude)
63 lines
1.8 KiB
Python
63 lines
1.8 KiB
Python
#!/usr/bin/env python3
|
|
"""Produce a renamed-tags variant of the rendered Form-A spec.
|
|
|
|
Two modes:
|
|
- apply: full-form resolution of abbreviations
|
|
app→apply, lam→lambda, ctor→constructor, con→typecon, tail-app→tail-apply
|
|
- call: like apply but with call as the application head (semantic shift)
|
|
app→call, tail-app→tail-call, rest as in apply
|
|
|
|
Each pattern is matched in two contexts:
|
|
- S-expression head: `(NAME ` and `(NAME)`
|
|
- Backtick mention: `` `NAME` ``
|
|
|
|
Plain-text occurrences ("application", "construction", etc.) are
|
|
preserved by anchoring on the contexts above.
|
|
"""
|
|
import re
|
|
import sys
|
|
from pathlib import Path
|
|
|
|
PAIRS_APPLY = [
|
|
("tail-app", "tail-apply"),
|
|
("term-ctor", "term-constructor"),
|
|
("pat-ctor", "pat-constructor"),
|
|
("ctor", "constructor"),
|
|
("app", "apply"),
|
|
("lam", "lambda"),
|
|
("con", "typecon"),
|
|
]
|
|
|
|
PAIRS_CALL = [
|
|
("tail-app", "tail-call"),
|
|
("term-ctor", "term-constructor"),
|
|
("pat-ctor", "pat-constructor"),
|
|
("ctor", "constructor"),
|
|
("app", "call"),
|
|
("lam", "lambda"),
|
|
("con", "typecon"),
|
|
]
|
|
|
|
MODES = {"apply": PAIRS_APPLY, "call": PAIRS_CALL}
|
|
|
|
def transform(src: str, pairs) -> str:
|
|
out = src
|
|
for old, new in pairs:
|
|
out = re.sub(rf"\({re.escape(old)}(?=[\s)])", f"({new}", out)
|
|
out = re.sub(rf"`{re.escape(old)}`", f"`{new}`", out)
|
|
return out
|
|
|
|
def main() -> int:
|
|
if len(sys.argv) != 4 or sys.argv[1] not in MODES:
|
|
print(f"usage: {sys.argv[0]} <apply|call> <input.md> <output.md>", file=sys.stderr)
|
|
return 2
|
|
mode = sys.argv[1]
|
|
inp = Path(sys.argv[2])
|
|
out = Path(sys.argv[3])
|
|
out.write_text(transform(inp.read_text(), MODES[mode]))
|
|
print(f"wrote {out} ({out.stat().st_size} bytes, mode={mode})")
|
|
return 0
|
|
|
|
if __name__ == "__main__":
|
|
sys.exit(main())
|