fieldtest: build the artefact from the current tree before running
A field test that runs a stale pre-built binary inverts its own purpose — it reports the previously-shipped state as current, producing false positives on already-fixed bugs and masking newly-introduced ones. This bit an AILang raw-buf field test: the agent ran a target/release binary built before a fix landed, reported a fixed bug as still-broken, and masked a real check-clean/build-crash defect until orchestrator re-verification against a fresh build. Phase 2 now opens with a mandatory build-from-HEAD step (run commands.build, or invoke the tool through the build system rather than a path to a pre-existing artefact; build the release profile if the examples invoke a release binary), and the agent records which build the run exercised.
This commit is contained in:
@@ -120,6 +120,18 @@ Each phase completes before the next starts.
|
||||
|
||||
### Phase 2 — Implement each example as a downstream consumer
|
||||
|
||||
**Before running any example, build the project's artefact from the
|
||||
current working tree.** A field test that runs a stale pre-built binary
|
||||
silently inverts its own purpose: it reports the *previously shipped*
|
||||
state as current, producing false positives on already-fixed bugs and
|
||||
masking newly-introduced ones. Run the build command from
|
||||
`commands.build` (or build the specific consumer binary the examples
|
||||
invoke) and confirm it succeeds before Phase 2 step 1; if the examples
|
||||
invoke a release binary, build the release profile, not just debug. When
|
||||
in doubt, invoke the tool through the build system (e.g. `cargo run`)
|
||||
rather than a path to a pre-existing artefact, so HEAD is always what
|
||||
runs. Record which binary/build the run exercised.
|
||||
|
||||
For each example, in this order:
|
||||
|
||||
1. Draft the program / task in the project's canonical
|
||||
|
||||
Reference in New Issue
Block a user