fieldtest: build the artefact from the current tree before running
A field test that runs a stale pre-built binary inverts its own purpose — it reports the previously-shipped state as current, producing false positives on already-fixed bugs and masking newly-introduced ones. This bit an AILang raw-buf field test: the agent ran a target/release binary built before a fix landed, reported a fixed bug as still-broken, and masked a real check-clean/build-crash defect until orchestrator re-verification against a fresh build. Phase 2 now opens with a mandatory build-from-HEAD step (run commands.build, or invoke the tool through the build system rather than a path to a pre-existing artefact; build the release profile if the examples invoke a release binary), and the agent records which build the run exercised.
This commit is contained in:
@@ -120,6 +120,18 @@ Each phase completes before the next starts.
|
|||||||
|
|
||||||
### Phase 2 — Implement each example as a downstream consumer
|
### Phase 2 — Implement each example as a downstream consumer
|
||||||
|
|
||||||
|
**Before running any example, build the project's artefact from the
|
||||||
|
current working tree.** A field test that runs a stale pre-built binary
|
||||||
|
silently inverts its own purpose: it reports the *previously shipped*
|
||||||
|
state as current, producing false positives on already-fixed bugs and
|
||||||
|
masking newly-introduced ones. Run the build command from
|
||||||
|
`commands.build` (or build the specific consumer binary the examples
|
||||||
|
invoke) and confirm it succeeds before Phase 2 step 1; if the examples
|
||||||
|
invoke a release binary, build the release profile, not just debug. When
|
||||||
|
in doubt, invoke the tool through the build system (e.g. `cargo run`)
|
||||||
|
rather than a path to a pre-existing artefact, so HEAD is always what
|
||||||
|
runs. Record which binary/build the run exercised.
|
||||||
|
|
||||||
For each example, in this order:
|
For each example, in this order:
|
||||||
|
|
||||||
1. Draft the program / task in the project's canonical
|
1. Draft the program / task in the project's canonical
|
||||||
|
|||||||
Reference in New Issue
Block a user