The price files carry no facts for interpreting the bars (pip size, digit count,
contract size, quote currency); those are known only to the broker via cTrader's
Symbol object. Produce a JSON sidecar per symbol next to the price files so a
downstream reader does not have to hardcode (and drift on) its own per-symbol table.
Scope is the price-interpretation geometry only — digits, pipSize, tickSize,
lotSize, baseAsset, quoteAsset — facts intrinsic to the price series and stable
across its whole history. Costs, swaps, volume limits, trading status and the
deposit-currency pip/tick values are intentionally left out: the cBot only ever
sees the current contract, so those are a point-in-time snapshot of today's trading
conditions and would misrepresent a multi-year series (the deposit-currency values
are account-dependent too).
- cBot: a MetaOnly mode (--MetaOnly --MetaFile) serializes the geometry to a flat
JSON. Numbers use InvariantCulture; a non-finite value becomes null; keys mirror
the cTrader property names.
- runner: write_meta runs once per symbol (mode-independent), validates the JSON,
and atomically promotes the sidecar. Best-effort — a metadata failure is logged
and never aborts the export, and a pre-existing sidecar is kept on failure. A
freshness guard (META_MAX_AGE_DAYS, default 7) keeps scheduled runs short;
EXPORT_META=0 disables it.
- The file is a raw serialization of what cTrader delivers; a reader derives its
own shape (contract_size, per-lot pip value) from the raw fields. See
docs/symbol-metadata.md.
- Add a source-level regression test for the freshness guard.
Verified end-to-end against the live broker for all 30 configured symbols (FX,
indices, metals, crypto, commodities, equities); the price corpus was untouched.
MS_SimpleExport is a market-data exporter (M1/Tick -> zipped binary), driven
headless via the official cTrader-console Docker image. This adds the bot and a
hardened bash driver that runs the scrape on a Linux host, spawning one ephemeral
console container per backtest window.
- src/MS_SimpleExport: the cBot; csproj aligned to the repo's Linux template
(AlgoPublish=false, cTrader.Automate 1.*-*).
- scripts/export-runner.sh: hardened port of the Unraid driver -- retry-then-abort
on transient failure, gap-aware resume, current-month end-date clamp, 3-way run
classification with a per-run timeout, and ZIP validation before rename.
- scripts/export.env.example: deployment config template (real values live in the
gitignored runtime/export.env).
- docs/export-runner-quirks.md: the multi-lens audit findings and hardening decisions.
- .gitignore: ignore runtime/ (secrets, the placed .algo, the symbol list).
Build cBots in C# and compile them to .algo entirely on Linux via the
official cTrader.Automate NuGet package + 'dotnet build' — no Windows and
no cTrader desktop app. Run/backtest headless through Spotware's official
cTrader console Docker image.
Includes:
- src/SampleCBot: example SMA-crossover cBot (SDK-style net6.0 class library)
- scripts/install-dotnet.sh: root-free local .NET SDK install (~/.dotnet)
- scripts/build.sh: dotnet build -> dist/<ProjectName>.algo, with a workaround
for cTrader.Automate naming the .algo after the project's parent folder
- scripts/run-console.sh: wrapper over ghcr.io/spotware/ctrader-console
- docs/research-findings.md: sourced research; .algo verified as a proprietary
'algo'-magic container (not a ZIP); build verified end to end on Linux
Brummel