Reshape the aura data namespace: JSON info, retire coverage
#273
Reference in New Issue
Block a user
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
No
aura dataverb surfaces a symbol's price geometry — decimals, pip, tick, lot, quote currency — so an agent operating through the CLI alone cannot obtain it headlessly. Two further frictions sit in the same namespace: its output formats diverge (listprints bare text lines, a geometry verb would print JSON), andaura data coverage(#264) has lost its purpose since per-cell fault isolation (#272) landed.This reshapes the
aura datanamespace around one principle — every verb emits JSON, no format flag — and resolves all three.What
aura data info <symbol>— new. Prints a single flat JSON object. It always carries the queriedsymbolas an identity anchor; every other field is optional geometry read from the<SYMBOL>.meta.jsonsidecar —description,digits,pipSize,tickSize,lotSize,baseAsset,quoteAsset— each omitted when the sidecar is absent, so a sidecar-less known symbol yields just{"symbol":"…"}rather than a half-success:A consumer tests
"pipSize" in obj. No provider-specific field escapes —generator,schemaVersion,generatedAtUtcstay internal (parity with the data-server reader's neutrality guarantee).aura data list— now NDJSON. One JSON value per line instead of a bare name per line, so the whole namespace is JSON while line-oriented composition (| while read,| grep) survives:aura data coverage— retired; the rationale is in § Whycoverageretires, not merges.Format: JSON across the namespace, no flag
aura dataexists for the headless agent that builds campaigns through the CLI alone (the deployment posture #264 named). For that consumer a format mode is a hazard: the output must not depend on whether a--jsonflag was set, soinfoandlistemit JSON unconditionally. JSON also makes partiality first-class — the reasoninfois JSON at all — which a human-text default would forfeit; and one renderer plus its tests is less to carry than two.infois one object;listis NDJSON, because the two verbs share a category (inspection) but not a cardinality (one rich object vs. many flat names) — "both JSON" is not "both the same schema". A terminal reader loses little: compact single-line JSON reads fine, and| jqpretty-prints on demand.Why
coverageretires, not mergesaura data coverage <symbol>printed a symbol's span and interior month gaps — a prophylactic check: know before a run whether the target window is gapless, so the run can be trusted. Per-cell fault isolation (#272) took that job over, and does it better: a run reportsCellCoverage{effective_from_ms, effective_to_ms, gap_months[]}for the exact window that matters, at the point it matters, instead of a manual pre-run reconciliation. No information is lost — the span/gap facts live on in the fault report where they bear on a decision; only a redundant verb goes. (Arc-5 in the aura-quadriga research project demonstrated the redundancy:coveragewas run to confirm five instruments gapless, then the campaign confirmed the same at exit 0 with no fault blocks — the pre-check was duplicated work.) Thespan/gapsfields therefore do not migrate intoinfo:infois geometry only.Error mode
no archive files found for symbolrefusal)Cross-repo relationships
Brummel/data-server#3(closed): addedsymbol_meta(symbol) -> Option<InstrumentGeometry>, which loads<SYMBOL>.meta.jsonand returns neutral geometry (nothing provider-specific escapes the API), paired with the existingsymbols/has_symbolsurface.infois the thin CLI surfacing of that reader — the library side is built.#232(closed): the sidecar's pip reaches the real-data path asgeo.pip_size(crates/aura-cli/src/campaign_run.rs:395); the gap here is purely the read-only inspect surface.#124(open): the instrument-reference & cost resolution hierarchy that pulls recorded dataset-metadata in as the middle tier of InstrumentSpec/cost resolution, under the always-winning authored override. That issue resolves geometry into a spec/cost model; this one inspects the raw recorded metadata read-only. #124's premise — "the Pepperstone NFS files today are bars-only … no instrument-structural metadata is carried" (2026-06-22) — was overtaken three days later by the sidecar export andBrummel/data-server#3(2026-06-25).#264(closed): addedaura data list/aura data coverage. This changeslist's output shape to NDJSON and retirescoverage.coverageredundant —#272(closed): per-cell fault isolation, whoseCellCoverage{effective_from_ms, effective_to_ms, gap_months[]}on the run path subsumes the prophylactic coverage check.Brummel/cTrader: theMS_SimpleExportcBot (src/MS_SimpleExport/MS_SimpleExport.cs, generator stringMS_SimpleExport (cTrader.Automate),schemaVersion 2) writes the sidecars at curate time. (NotBrummel/cTrader-DataExport— despite the apt name, that is an unrelated legacy Delphi export project with no sidecar generation.)Deliberately out of scope
No
providerorresolutionfield, and no stub for either, though a second data provider and tick-resolution data beside M1 are both foreseen. Those are not additive-field questions — they are symbol-identity questions — and a constant stub ("provider":"Pepperstone") would feign a selectability that does not exist and leak provider identity this verb otherwise withholds. Tracked separately as #274.Acceptance
aura data info <symbol>prints one flat JSON object carryingsymbolplus whatever sidecar geometry exists; geometry fields are omitted when the sidecar is absentgenerator,schemaVersion,generatedAtUtc) reaches the outputaura data listemits one JSON value per line (NDJSON)aura data coverageis removed; nospan/gapsfield appears ininfoExpose per-symbol instrument geometry through an `aura data info` verbto Reshape the `aura data` namespace: JSON `info`, retire `coverage`Re-anchoring note after cycle #295 (merged
4ed6455, 2026-07-21), which extracted the member-run recipe out of aura-cli into the new aura-runner and aura-measurement crates and shrank aura-cli to argv/dispatch, argv-to-document translation, and presentation (see crates/aura-vocabulary/tests/c28_layering.rs for the enforced shell allow-list).This issue's ask is otherwise fully undelivered and unaffected in substance: crates/aura-cli/src/main.rs:1137-1149 still declares
enum DataCommand { Coverage(DataCoverageCmd), List }as before,dispatch_data_list(main.rs:2156) still prints one bare symbol name per line (not NDJSON),dispatch_data_coverage(main.rs:2181) is unchanged in behavior, and noaura data infoverb exists.One citation in the issue body is now stale. The '#232 (closed)' bullet points to
geo.pip_sizeatcrates/aura-cli/src/campaign_run.rs:395as evidence the sidecar's pip already reaches the real-data path. That call site moved during #295 part 1 (commit5c2ac98) and now lives atcrates/aura-runner/src/runner.rs:180(andcrates/aura-runner/src/member.rs:154) — campaign_run.rs no longer referencesgeo.pip_sizeat all. The underlying claim (sidecar geometry already resolves into the real-data path) still holds at the new location; only the file:line needs updating for anyone implementing against it.When this is implemented, the new
aura data infodispatch and any report-shaping helper (analogous to the existingdata_list_report/aura_runner::coverage::data_coverage_reportsplit) should follow the #295 layering: thin CLI dispatch in aura-cli, with the reusable report body sourced fromaura_ingest::instrument_geometry/DataServer::symbol_meta(crates/aura-ingest/src/lib.rs:411-412) directly, or via an aura-runner helper if a second consumer emerges — consistent with howaura_runner::coverage::interior_gap_monthsis already the one gap walk shared between the CLI verb and a campaign cell's own coverage annotation.Design minutes (2026-07-25)
Three derived decisions taken while implementing this issue, recorded for audit and veto:
descriptionstays out ofinfofor now. The body's example output carriesdescription, but the neutral reader this verb surfaces (data-server'sInstrumentGeometryviaDataServer::symbol_meta) does not expose it: the schemaVersion-2 sidecar carries the field, the parser drops it. Extending the reader is a change in thedata-serverrepo (tracked there as Brummel/data-server#4), out of this cycle's scope — the body itself names the reader as the contract ("the library side is built").infotherefore shipssymbolplus the six reader-exposed fields (digits,pipSize,tickSize,lotSize,baseAsset,quoteAsset); the acceptance criterion is field-agnostic ("symbol plus whatever sidecar geometry exists") and remains satisfied.descriptionjoins when the reader exposes it.Unknown-symbol refusal prose. The body asks for a refusal "matching the current no archive files found for symbol refusal" — matched in shape (prose on stderr, exit 1), not in bytes:
info's unknown condition is broader than coverage's was (no bars AND no sidecar), so the prose names both absences:no recorded data for symbol "X" (no archive files, no geometry sidecar).Empty archive under NDJSON
list. The retired bare-textlistprinted ano symbolsprose line on stdout; under the all-JSON namespace stdout must stay parseable, so an empty archive emits zero stdout lines (the honest NDJSON of an empty set) and the human affordance moves to a stderr notice in the stderr-class vocabulary (#278).