Brummel
a8c29d130b
New top-level experiments/2026-05-12-cross-model-authoring/ (out of
root Cargo workspace; nested-crate standalone via empty [workspace]
table per Cargo idiom). The renderer projects one canonical
master/spec.md into two form-specific files (rendered/json.md and
rendered/ailx.md) by walking three directive forms ({form-only:
json}, {form-only: ailx}, {example: <id>}) and emitting per-example
fenced blocks sourced from 13 curated .ail.json fixtures.
Four test gates green (10/10 total):
- splitter_unit (7) — directive parser, malformed inputs panic
- spec_completeness (1) — AST-variant visitor lifted verbatim from
schema_coverage.rs; 34/34 variants covered by the curated subset
- example_roundtrip (1) — every fixture roundtrips through
ailang_surface::{print, parse} to identical canonical bytes
- token_balance (1) — form-only block totals balanced to 3.22% under
tiktoken-rs::cl100k_base (gate is ±5%)
Six of the 34 variants did not have a dedicated fixture topic in the
plan and were absorbed into the closest existing fixture per the
plan's Step 4.14 escape hatch (Let/Clone → fn_calls_prelude; LetRec
→ data_with_match; If → match_literal_pattern; ReuseAs →
data_simple; Const → floats). Surfaced as a journal note, not a
plan revision.
Two ancillary additions vs. plan, both structural Cargo necessities,
spec-compatible: render/Cargo.toml carries an empty [workspace] table
so the nested crate refuses parent-workspace discovery, and
render/.gitignore drops /target + Cargo.lock. Both surfaced inline
in the per-iter journal's Concerns section.
cma.2 (harness) and cma.3 (live IONOS run + DESIGN.md addendum +
journal + roadmap edits) remain out of scope per the spec.
47 lines
1.7 KiB
Rust
47 lines
1.7 KiB
Rust
//! Token-balance gate for master/spec.md form-only blocks.
|
|
//!
|
|
//! The mini-spec is the experimental treatment; if the JSON-cohort's
|
|
//! form-only scaffolding is meaningfully longer than the AILX-cohort's
|
|
//! (or vice versa), the experiment is biased before the model ever
|
|
//! sees the prompts. This test catches gross imbalance.
|
|
|
|
use std::path::PathBuf;
|
|
use xmodel_render::splitter::{self, Form, Segment};
|
|
|
|
fn master_dir() -> PathBuf {
|
|
let manifest = PathBuf::from(env!("CARGO_MANIFEST_DIR"));
|
|
manifest.parent().unwrap().join("master")
|
|
}
|
|
|
|
#[test]
|
|
fn form_only_block_token_totals_within_five_percent() {
|
|
let spec_text = std::fs::read_to_string(master_dir().join("spec.md"))
|
|
.expect("master/spec.md must exist");
|
|
let segments = splitter::split(&spec_text);
|
|
|
|
let mut json_buf = String::new();
|
|
let mut ailx_buf = String::new();
|
|
for seg in &segments {
|
|
if let Segment::FormOnly { form, body } = seg {
|
|
match form {
|
|
Form::Json => json_buf.push_str(body),
|
|
Form::Ailx => ailx_buf.push_str(body),
|
|
}
|
|
}
|
|
}
|
|
|
|
let bpe = tiktoken_rs::cl100k_base().expect("cl100k_base tokenizer should load");
|
|
let json_count = bpe.encode_with_special_tokens(&json_buf).len();
|
|
let ailx_count = bpe.encode_with_special_tokens(&ailx_buf).len();
|
|
assert!(json_count > 0, "JSON-form blocks produced zero tokens");
|
|
assert!(ailx_count > 0, "AILX-form blocks produced zero tokens");
|
|
|
|
let max = json_count.max(ailx_count) as f64;
|
|
let diff = (json_count as i64 - ailx_count as i64).unsigned_abs() as f64;
|
|
let ratio = diff / max;
|
|
assert!(
|
|
ratio <= 0.05,
|
|
"form-only block token imbalance: json={json_count}, ailx={ailx_count}, ratio={ratio:.4} > 0.05",
|
|
);
|
|
}
|