export-runner: fix new-symbol discovery aborting with a year-0000 backtest
discover_start's stdout is its result channel (read via command substitution in process_symbol), but the probes inside it stream the cBot's "[BOT] …" progress on stdout too. That noise leaked into the captured value, so start_idx parsed to 0 and the resume walk began at year 0 -> an unparseable "--start=01/01/0" backtest that retried out and hard-aborted the whole run. Only a brand-new symbol (no files on disk yet) hits this path, so it stayed invisible until AUDUSD was added. Redirect every probe call in discover_start to stderr (1>&2) so its stdout carries only the "YEAR MONTH" result, and document the contract. Add scripts/test-export-runner.sh: source-level tests that stub probe_run to emit the same stdout noise and assert a clean result (RED without the redirect). Make SYMBOL_FILE overridable for scoped runs/tests.
This commit is contained in:
@@ -31,7 +31,7 @@ BOT_FILE="MS_SimpleExport.algo"
|
||||
PWD_FILE="/mnt/Robots/password.txt" # container-internal path (console reads it)
|
||||
IMAGE="ghcr.io/spotware/ctrader-console:latest"
|
||||
HOST_ROBOTS_PATH="$REPO_ROOT/runtime/Robots" # holds MS_SimpleExport.algo + password.txt
|
||||
SYMBOL_FILE="$REPO_ROOT/runtime/ScrapeSymbols.txt"
|
||||
SYMBOL_FILE="${SYMBOL_FILE:-$REPO_ROOT/runtime/ScrapeSymbols.txt}" # overridable for a scoped run/test
|
||||
RETRIES="${RETRIES:-3}" # attempts before a transient failure aborts
|
||||
RUN_TIMEOUT="${RUN_TIMEOUT:-600}" # seconds per backtest before it counts as a hang
|
||||
DISCOVER_MIN_YEAR="${DISCOVER_MIN_YEAR:-2005}" # earliest year the first-data probe scans
|
||||
@@ -211,12 +211,18 @@ probe_run() {
|
||||
# nothing if the symbol has no data at all. Yearly probe uses December (catches data
|
||||
# that started any time that year); monthly probe uses the FULL month (catches data
|
||||
# starting after the 10th — the old probe only tested days 01–10).
|
||||
#
|
||||
# CONTRACT: this function's STDOUT carries ONLY the result line; the caller reads it via
|
||||
# command substitution. probe_run -> run_one stream the cBot's "[BOT] …" lines on stdout,
|
||||
# so every probe call here MUST be redirected to stderr (`1>&2`) — otherwise that progress
|
||||
# noise pollutes the captured value and start_idx is parsed from garbage (see the AUDUSD
|
||||
# year-0000 abort, docs/export-runner-quirks.md). All diagnostics here go to stderr too.
|
||||
discover_start() {
|
||||
local S=$1 MODE=$2 y mm MM edate found_year=""
|
||||
echo "🌑 Discovery: locating first data month for $S ($MODE)..." >&2
|
||||
for ((y=DISCOVER_MIN_YEAR; y<CURRENT_YEAR; y++)); do
|
||||
echo " 🧪 year $y" >&2
|
||||
if probe_run "$S" "$MODE" "01/12/$y" "31/12/$y"; then found_year=$y; break; fi
|
||||
if probe_run "$S" "$MODE" "01/12/$y" "31/12/$y" 1>&2; then found_year=$y; break; fi
|
||||
done
|
||||
if [[ -z "$found_year" ]]; then found_year=$CURRENT_YEAR; fi # brand-new this year → check months below
|
||||
for ((mm=1; mm<=12; mm++)); do
|
||||
@@ -225,7 +231,7 @@ discover_start() {
|
||||
edate="$(month_end_or_clamp "$found_year" "$mm")"
|
||||
[[ -z "$edate" ]] && break
|
||||
echo " 🧪 month $found_year-$MM" >&2
|
||||
if probe_run "$S" "$MODE" "01/$MM/$found_year" "$edate"; then echo "$found_year $mm"; return 0; fi
|
||||
if probe_run "$S" "$MODE" "01/$MM/$found_year" "$edate" 1>&2; then echo "$found_year $mm"; return 0; fi
|
||||
done
|
||||
return 1 # no data anywhere
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user