The price files carry no facts for interpreting the bars (pip size, digit count, contract size, quote currency); those are known only to the broker via cTrader's Symbol object. Produce a JSON sidecar per symbol next to the price files so a downstream reader does not have to hardcode (and drift on) its own per-symbol table. Scope is the price-interpretation geometry only — digits, pipSize, tickSize, lotSize, baseAsset, quoteAsset — facts intrinsic to the price series and stable across its whole history. Costs, swaps, volume limits, trading status and the deposit-currency pip/tick values are intentionally left out: the cBot only ever sees the current contract, so those are a point-in-time snapshot of today's trading conditions and would misrepresent a multi-year series (the deposit-currency values are account-dependent too). - cBot: a MetaOnly mode (--MetaOnly --MetaFile) serializes the geometry to a flat JSON. Numbers use InvariantCulture; a non-finite value becomes null; keys mirror the cTrader property names. - runner: write_meta runs once per symbol (mode-independent), validates the JSON, and atomically promotes the sidecar. Best-effort — a metadata failure is logged and never aborts the export, and a pre-existing sidecar is kept on failure. A freshness guard (META_MAX_AGE_DAYS, default 7) keeps scheduled runs short; EXPORT_META=0 disables it. - The file is a raw serialization of what cTrader delivers; a reader derives its own shape (contract_size, per-lot pip value) from the raw fields. See docs/symbol-metadata.md. - Add a source-level regression test for the freshness guard. Verified end-to-end against the live broker for all 30 configured symbols (FX, indices, metals, crypto, commodities, equities); the price corpus was untouched.
5.7 KiB
Per-symbol price metadata (<SYMBOL>.meta.json)
Alongside the price files (<SYMBOL>_YYYY_MM.m1 / .tick) the export driver writes one
small JSON sidecar per symbol, <SYMBOL>.meta.json. Its sole purpose is to let a reader
interpret the historical price data — the price files contain only raw OHLC / spread /
volume numbers, with nothing that says how to read them (how many decimals a price has, what
a "pip" is, which currency the price is quoted in, what one lot is worth). Those facts are
known only to the broker, exposed through cTrader's Symbol object inside a running cBot.
This sidecar captures them so a downstream reader does not have to hardcode (and drift on) a
per-symbol table. The per-instrument differences are real and easy to get wrong by hand: a
JPY pair's pip is 0.01 (not 0.0001); gold's lot size is 100 (not the FX 100000).
Scope: price-interpretation geometry only
The file carries only the geometry needed to interpret the price series — facts that are intrinsic to the data and effectively constant over its whole multi-year span:
| Field | cTrader source | Meaning |
|---|---|---|
digits |
Symbol.Digits |
price decimal places |
pipSize |
Symbol.PipSize |
price units per pip (EURUSD 0.0001, USDJPY 0.01, XAUUSD 0.1) |
tickSize |
Symbol.TickSize |
smallest price increment |
lotSize |
Symbol.LotSize |
base-currency units per 1.0 lot — turns a price move into money (FX 100000, XAUUSD 100) |
baseAsset |
Symbol.BaseAsset.Name |
base asset / currency ISO label |
quoteAsset |
Symbol.QuoteAsset.Name |
quote asset / currency the price is denominated in |
description |
Symbol.Description |
human-readable name |
Plus provenance: schemaVersion, symbol, generatedAtUtc, generator.
What is deliberately NOT here — and why
cTrader's Symbol also exposes costs (Commission*, Swap*), volume limits
(VolumeInUnits*), trading status (TradingMode, IsTradingEnabled) and deposit-currency
pip/tick values (PipValue, TickValue). These are intentionally excluded, because they
do not describe the historical price series — they are a point-in-time snapshot of today's
trading conditions:
- The cBot only ever sees the current contract. A backtest runs against the symbol as it is defined now, never as it stood in a historical month. So the bot physically cannot emit a historically-accurate swap rate or commission for 2015 — only today's. Swap rates in particular change weekly; stamping a single 2026 value onto a 15-year series would misrepresent it.
PipValue/TickValueare additionally account-dependent. cTrader reports them already converted into the exporting account's deposit currency at the export-time FX rate, so they are a property of (account × moment), not of the instrument. The stable, account-free pip value is pure arithmetic:lotSize × pipSize, in the quote currency.
If a consumer needs current trading conditions (for a friction model, position sizing, etc.), those are a separate, time-varying concern and belong in their own time-indexed source — not baked into a per-symbol price-geometry file as if they held across history.
Mechanism
MS_SimpleExportcBot,MetaOnlymode (src/MS_SimpleExport/MS_SimpleExport.cs). Run with--MetaOnly=true --MetaFile=<path>, the bot does no price export: inOnStartit readsSymbol, serializes the geometry fields to JSON atMetaFile, printsMETA_SUCCESS, and stops.export-runner.sh,write_meta. Once per symbol per run (mode-independent), the driver spawns a shortMetaOnlybacktest, writes<SYMBOL>.meta.json.tmp, validates it is parseable JSON, then renames it to<SYMBOL>.meta.json.- Best-effort by design: a metadata failure is logged (
⚠️) and never aborts the export run; a pre-existing sidecar is left untouched (overwrite only on a validated success). - Cadence: refreshed at most once per
META_MAX_AGE_DAYS(default7) so scheduled runs stay short.EXPORT_META=0disables;META_MAX_AGE_DAYS=0forces a refresh every run.
- Best-effort by design: a metadata failure is logged (
Format
A flat JSON object. Conventions:
- Numbers use invariant formatting — a pip size is
0.0001, never a locale0,0001. Round-trip formatting may emit scientific notation (1E-05); valid JSON, parses as a number. - A non-finite broker value (NaN / Infinity) is emitted as JSON
null. schemaVersionis bumped on any incompatible change to field names or units (currently2).
Example
{
"schemaVersion": 2,
"symbol": "EURUSD",
"generatedAtUtc": "2026-06-25T11:06:01Z",
"generator": "MS_SimpleExport (cTrader.Automate)",
"description": "Euro vs US Dollar",
"digits": 5,
"pipSize": 0.0001,
"tickSize": 1E-05,
"lotSize": 100000,
"baseAsset": "EUR",
"quoteAsset": "USD"
}
Consumer-side derivations
A reader composes whatever shape it wants from these raw fields, e.g.:
contract_size = lotSize
pip_value_per_lot = lotSize * pipSize # in quoteAsset (EURUSD: 100000*0.0001 = 10 USD)
point_size = 10 ** -digits
Trading constraints a consumer's own instrument spec might want — minimum lot, lot step, a stable instrument id — are not price-interpretation geometry and are not in this file; the consumer authors those itself (an id is a registry choice; min-lot / lot-step are current order limits, not facts about the historical bars).
Worked geometry, verified against the live broker:
| symbol | digits | pipSize | tickSize | lotSize | base/quote | → pip_value_per_lot |
|---|---|---|---|---|---|---|
| EURUSD | 5 | 0.0001 | 1E-05 | 100000 | EUR/USD | 10.0 USD |
| USDJPY | 3 | 0.01 | 0.001 | 100000 | USD/JPY | 1000 JPY |
| XAUUSD | 2 | 0.1 | 0.01 | 100 | XAU/USD | 10.0 USD |