Add per-symbol price-metadata sidecars (<symbol>.meta.json)
The price files carry no facts for interpreting the bars (pip size, digit count, contract size, quote currency); those are known only to the broker via cTrader's Symbol object. Produce a JSON sidecar per symbol next to the price files so a downstream reader does not have to hardcode (and drift on) its own per-symbol table. Scope is the price-interpretation geometry only — digits, pipSize, tickSize, lotSize, baseAsset, quoteAsset — facts intrinsic to the price series and stable across its whole history. Costs, swaps, volume limits, trading status and the deposit-currency pip/tick values are intentionally left out: the cBot only ever sees the current contract, so those are a point-in-time snapshot of today's trading conditions and would misrepresent a multi-year series (the deposit-currency values are account-dependent too). - cBot: a MetaOnly mode (--MetaOnly --MetaFile) serializes the geometry to a flat JSON. Numbers use InvariantCulture; a non-finite value becomes null; keys mirror the cTrader property names. - runner: write_meta runs once per symbol (mode-independent), validates the JSON, and atomically promotes the sidecar. Best-effort — a metadata failure is logged and never aborts the export, and a pre-existing sidecar is kept on failure. A freshness guard (META_MAX_AGE_DAYS, default 7) keeps scheduled runs short; EXPORT_META=0 disables it. - The file is a raw serialization of what cTrader delivers; a reader derives its own shape (contract_size, per-lot pip value) from the raw fields. See docs/symbol-metadata.md. - Add a source-level regression test for the freshness guard. Verified end-to-end against the live broker for all 30 configured symbols (FX, indices, metals, crypto, commodities, equities); the price corpus was untouched.
This commit is contained in:
@@ -71,6 +71,20 @@ i=$(midx 2026 5)
|
||||
check "midx/idx_year round-trip" "2026" "$(idx_year "$i")"
|
||||
check "midx/idx_month round-trip" "5" "$(idx_month "$i")"
|
||||
|
||||
# --- write_meta freshness guard (no Docker needed) -------------------------------------
|
||||
# A recent sidecar must be skipped without spawning a container, so scheduled runs stay
|
||||
# short. Point HOST_DATA_PATH at a temp dir, drop a just-created meta file, and assert
|
||||
# write_meta classifies it as a fresh-skip. A docker stub fails loudly if it wrongly runs.
|
||||
TMPDATA="$(mktemp -d)"; HOST_DATA_PATH="$TMPDATA"
|
||||
: > "$TMPDATA/EURUSD.meta.json" # mtime = now -> inside any positive window
|
||||
META_OK=0; META_FAIL=0; META_SKIP=0; META_MAX_AGE_DAYS=7
|
||||
docker() { echo "UNEXPECTED docker call" >&2; return 1; } # must NOT be reached on a fresh file
|
||||
write_meta EURUSD >/dev/null 2>&1
|
||||
unset -f docker
|
||||
check "write_meta: fresh sidecar -> skipped" "1" "$META_SKIP"
|
||||
check "write_meta: fresh sidecar -> not produced" "0" "$META_OK"
|
||||
rm -rf "$TMPDATA"
|
||||
|
||||
echo "-----"
|
||||
echo "$PASS passed, $FAIL failed"
|
||||
[[ $FAIL -eq 0 ]]
|
||||
|
||||
Reference in New Issue
Block a user