Brummel/cTrader
1
0
Fork 0
Code Issues 1 Pull Requests Actions Packages Projects Releases Wiki Activity

Add per-symbol price-metadata sidecars (<symbol>.meta.json)

Browse Source 
The price files carry no facts for interpreting the bars (pip size, digit count,
contract size, quote currency); those are known only to the broker via cTrader's
Symbol object. Produce a JSON sidecar per symbol next to the price files so a
downstream reader does not have to hardcode (and drift on) its own per-symbol table.

Scope is the price-interpretation geometry only — digits, pipSize, tickSize,
lotSize, baseAsset, quoteAsset — facts intrinsic to the price series and stable
across its whole history. Costs, swaps, volume limits, trading status and the
deposit-currency pip/tick values are intentionally left out: the cBot only ever
sees the current contract, so those are a point-in-time snapshot of today's trading
conditions and would misrepresent a multi-year series (the deposit-currency values
are account-dependent too).

- cBot: a MetaOnly mode (--MetaOnly --MetaFile) serializes the geometry to a flat
  JSON. Numbers use InvariantCulture; a non-finite value becomes null; keys mirror
  the cTrader property names.
- runner: write_meta runs once per symbol (mode-independent), validates the JSON,
  and atomically promotes the sidecar. Best-effort — a metadata failure is logged
  and never aborts the export, and a pre-existing sidecar is kept on failure. A
  freshness guard (META_MAX_AGE_DAYS, default 7) keeps scheduled runs short;
  EXPORT_META=0 disables it.
- The file is a raw serialization of what cTrader delivers; a reader derives its
  own shape (contract_size, per-lot pip value) from the raw fields. See
  docs/symbol-metadata.md.
- Add a source-level regression test for the freshness guard.

Verified end-to-end against the live broker for all 30 configured symbols (FX,
indices, metals, crypto, commodities, equities); the price corpus was untouched.
This commit is contained in:
Brummel
2026-06-25 13:12:54 +02:00
parent ddd8274fad
commit cd6874b43f
4 changed files with 272 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/MS_SimpleExport/MS_SimpleExport.cs
+67 -3
View File
@@ -2,6 +2,7 @@ using System;
using System.IO;
using System.IO.Compression;
using System.Collections.Generic;
using System.Globalization;
using System.Runtime.InteropServices;
using cAlgo.API;
@@ -35,16 +36,34 @@ namespace cAlgo.Robots
[Parameter("Is Probing", DefaultValue = false)]
public bool IsProbing { get; set; }
// Metadata sidecar: when MetaOnly is set, the bot does NOT export price data. It
// dumps the broker's authoritative per-symbol contract specs (pip size, lot size,
// currencies, costs, …) as a flat JSON file to MetaFile, then stops. This is the
// ONLY authoritative source for these values — the host-side driver cannot know
// them, it can only place the file the bot produces. See docs/symbol-metadata.md.
[Parameter("Metadata Only", DefaultValue = false)]
public bool MetaOnly { get; set; }
[Parameter("Metadata File", DefaultValue = "")]
public string MetaFile { get; set; }
private List<M1Record> _m1Buffer = new List<M1Record>();
private List<TickRecord> _tickBuffer = new List<TickRecord>();
protected override void OnStart() {
// Metadata run: emit the contract specs and stop — no price export.
if (MetaOnly) {
WriteMetadata();
Stop();
return;
}
// No price factor needed for double/float export
}
protected override void OnTick() {
if (MetaOnly) return;
if (Mode != ExportMode.Tick) return;
if (IsProbing) {
Print("DATA_FOUND");
Stop();
@@ -59,8 +78,9 @@ namespace cAlgo.Robots
}
protected override void OnBar() {
if (MetaOnly) return;
if (Mode != ExportMode.M1) return;
if (IsProbing) {
Print("DATA_FOUND");
Stop();
@@ -80,7 +100,7 @@ namespace cAlgo.Robots
}
protected override void OnStop() {
if (IsProbing || string.IsNullOrEmpty(OutputFile)) return;
if (MetaOnly || IsProbing || string.IsNullOrEmpty(OutputFile)) return;
try {
using var fileStream = new FileStream(OutputFile, FileMode.Create);
@@ -104,5 +124,49 @@ namespace cAlgo.Robots
Print("ERROR: {0}", ex.Message);
}
}
// Serialize the symbol's PRICE-INTERPRETATION GEOMETRY — the facts needed to read the
// historical price files: digit count, pip/tick size, contract size, and the currencies
// the price is quoted in. These are intrinsic to the price series and effectively
// time-stable, so they can be applied across the whole multi-year history.
//
// Deliberately EXCLUDED: costs, swaps, volume limits, trading status, and the
// deposit-currency pip/tick values. Those are a point-in-time snapshot of *today's*
// trading conditions — the cBot only ever sees the current contract, never the contract
// as it stood in a historical month — so stamping them onto a years-long series would
// misrepresent history (and the deposit-currency values further depend on the exporting
// account). See docs/symbol-metadata.md for the rationale.
//
// Keys mirror the cTrader property names; numbers use InvariantCulture (a pip size is
// "0.0001", never a locale "0,0001"); a non-finite value becomes JSON null.
private void WriteMetadata() {
try {
var s = Symbol;
var inv = CultureInfo.InvariantCulture;
string Num(double d) => (double.IsNaN(d) || double.IsInfinity(d)) ? "null" : d.ToString("R", inv);
string Str(string v) => v == null ? "null" : "\"" + v.Replace("\\", "\\\\").Replace("\"", "\\\"") + "\"";
var f = new List<string> {
// provenance of this sidecar
"\"schemaVersion\": 2",
"\"symbol\": " + Str(s.Name),
"\"generatedAtUtc\": " + Str(DateTime.UtcNow.ToString("yyyy-MM-ddTHH:mm:ssZ", inv)),
"\"generator\": " + Str("MS_SimpleExport (cTrader.Automate)"),
// --- price-interpretation geometry (intrinsic, time-stable) ---
"\"description\": " + Str(s.Description),
"\"digits\": " + s.Digits.ToString(inv),
"\"pipSize\": " + Num(s.PipSize),
"\"tickSize\": " + Num(s.TickSize),
"\"lotSize\": " + Num(s.LotSize),
"\"baseAsset\": " + Str(s.BaseAsset?.Name),
"\"quoteAsset\": " + Str(s.QuoteAsset?.Name),
};
System.IO.File.WriteAllText(MetaFile, "{\n " + string.Join(",\n ", f) + "\n}\n");
Print("META_SUCCESS");
} catch (Exception ex) {
Print("META_ERROR: {0}", ex.Message);
}
}
}
}