From 2d0de46feab2b3e55d1c195b859cd979638304bb Mon Sep 17 00:00:00 2001 From: Brummel Date: Sat, 30 May 2026 18:29:11 +0200 Subject: [PATCH] fieldtest: build the artefact from the current tree before running MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A field test that runs a stale pre-built binary inverts its own purpose — it reports the previously-shipped state as current, producing false positives on already-fixed bugs and masking newly-introduced ones. This bit an AILang raw-buf field test: the agent ran a target/release binary built before a fix landed, reported a fixed bug as still-broken, and masked a real check-clean/build-crash defect until orchestrator re-verification against a fresh build. Phase 2 now opens with a mandatory build-from-HEAD step (run commands.build, or invoke the tool through the build system rather than a path to a pre-existing artefact; build the release profile if the examples invoke a release binary), and the agent records which build the run exercised. --- fieldtest/agents/fieldtester.md | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/fieldtest/agents/fieldtester.md b/fieldtest/agents/fieldtester.md index e68d31b..50ac50c 100644 --- a/fieldtest/agents/fieldtester.md +++ b/fieldtest/agents/fieldtester.md @@ -120,6 +120,18 @@ Each phase completes before the next starts. ### Phase 2 — Implement each example as a downstream consumer +**Before running any example, build the project's artefact from the +current working tree.** A field test that runs a stale pre-built binary +silently inverts its own purpose: it reports the *previously shipped* +state as current, producing false positives on already-fixed bugs and +masking newly-introduced ones. Run the build command from +`commands.build` (or build the specific consumer binary the examples +invoke) and confirm it succeeds before Phase 2 step 1; if the examples +invoke a release binary, build the release profile, not just debug. When +in doubt, invoke the tool through the build system (e.g. `cargo run`) +rather than a path to a pre-existing artefact, so HEAD is always what +runs. Record which binary/build the run exercised. + For each example, in this order: 1. Draft the program / task in the project's canonical