Bound the campaign executor fan-out to a worker pool #268

Closed
opened 2026-07-13 18:22:12 +02:00 by claude · 6 comments
Collaborator

During a 42-cell campaign run (21 instruments x 2 regimes, 9-member axes, 90/30 walk-forward + 1000-resample bootstrap, engine 84e1075 release) on a 24-core box, the executor process averages ~1410% CPU (~14 cores, ~58% of the box) while its thread count oscillates hard. Twenty 0.5-s samples of nlwp:

209 98 228 159 139 208 166 1 10 10 10 10 10 11 10 10 6 1 239 222

Three phases alternate: bursts of 150-240 threads (10x oversubscription of the cores), multi-second plateaus of exactly 10-11 threads (suspiciously close to 9 sweep members + main), and 1-thread valleys where 23 cores idle. RSS stays flat at ~2.9 GB.

Claim (unverified): the bursts are unbounded thread-per-sim fan-out (e.g. all walk-forward windows x members spawned at once) and the valleys are serial seams (ingest/merge, bootstrap, record writing). A bounded worker pool sized to the cores, plus a look at what the serial seams spend their time on, is worth roughly 40% of campaign wall time on this box if the valleys can overlap with sim execution. Per-cell wall time is otherwise healthy (~11-13 s per cell over 8.5 y of m1 with costs; the Arc-1-era baseline in the external research project recorded 9.5 s/cell without cost nodes).

During a 42-cell campaign run (21 instruments x 2 regimes, 9-member axes, 90/30 walk-forward + 1000-resample bootstrap, engine 84e1075 release) on a 24-core box, the executor process averages ~1410% CPU (~14 cores, ~58% of the box) while its thread count oscillates hard. Twenty 0.5-s samples of `nlwp`: ``` 209 98 228 159 139 208 166 1 10 10 10 10 10 11 10 10 6 1 239 222 ``` Three phases alternate: bursts of 150-240 threads (10x oversubscription of the cores), multi-second plateaus of exactly 10-11 threads (suspiciously close to 9 sweep members + main), and 1-thread valleys where 23 cores idle. RSS stays flat at ~2.9 GB. Claim (unverified): the bursts are unbounded thread-per-sim fan-out (e.g. all walk-forward windows x members spawned at once) and the valleys are serial seams (ingest/merge, bootstrap, record writing). A bounded worker pool sized to the cores, plus a look at what the serial seams spend their time on, is worth roughly 40% of campaign wall time on this box if the valleys can overlap with sim execution. Per-cell wall time is otherwise healthy (~11-13 s per cell over 8.5 y of m1 with costs; the Arc-1-era baseline in the external research project recorded 9.5 s/cell without cost nodes).
claude added the idea label 2026-07-13 18:22:12 +02:00
Author
Collaborator

Ruled-out alternative: the 1-thread valleys are not network/data-paging stalls. The archive is an NFS mount (nfs4, rsize 1M), but the m1 working set is 1.4 GB against ~12 GB of page cache. Measured over 10 s mid-run: the process reads ~15 MB/s via syscalls (rchar +152 MB) while the NIC receives ~73 KB/s — i.e. reads are served from the local page cache, nothing is re-paged over the wire, and local block reads are zero. The valleys are therefore CPU-bound single-threaded work; the ingest parse/transpose of the next instrument (AoS -> SoA before sim fan-out) is the prime suspect. Claim (unverified): a profile of the valley phases would confirm the seam.

Ruled-out alternative: the 1-thread valleys are not network/data-paging stalls. The archive is an NFS mount (nfs4, rsize 1M), but the m1 working set is 1.4 GB against ~12 GB of page cache. Measured over 10 s mid-run: the process reads ~15 MB/s via syscalls (`rchar` +152 MB) while the NIC receives ~73 KB/s — i.e. reads are served from the local page cache, nothing is re-paged over the wire, and local block reads are zero. The valleys are therefore CPU-bound single-threaded work; the ingest parse/transpose of the next instrument (AoS -> SoA before sim fan-out) is the prime suspect. Claim (unverified): a profile of the valley phases would confirm the seam.
claude self-assigned this 2026-07-15 18:37:59 +02:00
Author
Collaborator

Root cause (verified from the code, not the profile)

The thread oscillation traces to nested run_indexed, the shared
disjoint-parallel core behind both sweep and walk_forward. Each entry
independently sizes its pool to available_parallelism() (sweep.rs:381,
mc.rs:127, walkforward.rs:189), and the campaign nests two of them:

  • run_walk_forward_stage calls walk_forward (exec.rs:988), which fans the
    windows across available_parallelism() workers (walkforward.rs:210);
  • each window worker calls sweep for the in-sample optimize (exec.rs:994),
    fanning the survivor points across available_parallelism() again
    (sweep.rs:394).

The two pools multiply: on a 24-core box, ~24 windows x ~9 members ~= 216
concurrent threads — matching the observed 150-240 bursts. The 10-11 plateaus
are the flat sweep stage with no walk-forward above it (exec.rs:653, members +
main). The 1-thread valleys are genuinely serial seams — the cell loop runs the
42 cells sequentially (exec.rs:132, C1) and r_bootstrap is single-threaded
(mc.rs:193) — not the fan-out. The process averaged ~1410% CPU (~14 of 24
cores): ~42% of the box was idle, and that idle is the valleys, not the bursts.

Direction

Set by the maintainer, 2026-07-15: maximize throughput regardless of
dependency footprint
— rayon accepted, the std-only-kernel constraint lifted
for this path. That moves the objective from "bound the fan-out" to "saturate
the cores", which makes the idle valleys — not the oversubscription bursts — the
primary target, since the idle is the larger loss.

Fix: one shared work-stealing pool over all disjoint work

  • Replace the per-call std::thread::scope fan-out in run_indexed with a
    single process-wide rayon pool.
    rayon's pool is persistent and global:
    nested par_iter enqueues into the same N workers instead of spawning fresh
    OS threads, so the multiplicative oversubscription disappears and work-stealing
    keeps every worker busy while any work remains. (This is the deadlock-free
    nesting the earlier "shared pool risks deadlock" note misjudged — rayon steals
    inner work rather than blocking on a join.)
  • Parallelize the cell loop (exec.rs:132), which is C1-disjoint. This is
    the main lever: with 42 cells' work in one pool, the per-cell serial seams
    (the next instrument's ingest transpose, the single-threaded bootstrap) fill
    with parallel work from other cells, so the cores stay saturated across the
    seams that currently idle 23 of them.

Rationale: a single shared pool over the coarsest-to-finest disjoint work
(cells x members x windows x seeds) is the structure that saturates the cores
regardless of nesting shape — work-stealing balances flat across levels and
across the serial seams, which a per-level bounded pool cannot.

Constraints held: C1 determinism is non-negotiable — jobs stay pure and
disjoint, results are collected in input order, so runs remain bit-identical
across worker counts (the existing _with_threads(1) == _with_threads(8) tests
stay green, extended to the parallel cell loop). Parallel cell execution
multiplies the resident working set, so the count of concurrently-active cells
is bounded to a RAM budget — swap thrashing would be slower, i.e. the opposite
of the goal (this box: ~1.4 GB working set per instrument, 32 GB total).

Alternative held back

  • A thread-local recursion-depth guard that serializes the inner
    run_indexed level (std-only, no dependency, minimal). It removes the bursts
    but leaves the valleys idle and under-utilizes when the outer level has few
    jobs — insufficient for the saturate-the-cores objective. It was the right fix
    for the narrower "bound the fan-out" framing, superseded by the maintainer's
    maximize-throughput direction.

Scope

The shared pool, the parallel cell loop, and a reproducible local benchmark that
measures core utilization / wall time before and after (the real 42-cell
workload's data is external and not reproducible here). RAM-aware bounding of
concurrently-active cells is part of the pool work, not a separate track.

## Root cause (verified from the code, not the profile) The thread oscillation traces to **nested `run_indexed`**, the shared disjoint-parallel core behind both `sweep` and `walk_forward`. Each entry independently sizes its pool to `available_parallelism()` (`sweep.rs:381`, `mc.rs:127`, `walkforward.rs:189`), and the campaign nests two of them: - `run_walk_forward_stage` calls `walk_forward` (`exec.rs:988`), which fans the windows across `available_parallelism()` workers (`walkforward.rs:210`); - each window worker calls `sweep` for the in-sample optimize (`exec.rs:994`), fanning the survivor points across `available_parallelism()` *again* (`sweep.rs:394`). The two pools multiply: on a 24-core box, ~24 windows x ~9 members ~= 216 concurrent threads — matching the observed 150-240 bursts. The 10-11 plateaus are the flat sweep stage with no walk-forward above it (`exec.rs:653`, members + main). The 1-thread valleys are genuinely serial seams — the cell loop runs the 42 cells sequentially (`exec.rs:132`, C1) and `r_bootstrap` is single-threaded (`mc.rs:193`) — not the fan-out. The process averaged ~1410% CPU (~14 of 24 cores): ~42% of the box was idle, and that idle is the valleys, not the bursts. ## Direction Set by the maintainer, 2026-07-15: **maximize throughput regardless of dependency footprint** — rayon accepted, the std-only-kernel constraint lifted for this path. That moves the objective from "bound the fan-out" to "saturate the cores", which makes the idle valleys — not the oversubscription bursts — the primary target, since the idle is the larger loss. ## Fix: one shared work-stealing pool over all disjoint work - **Replace the per-call `std::thread::scope` fan-out in `run_indexed` with a single process-wide rayon pool.** rayon's pool is persistent and global: nested `par_iter` enqueues into the same N workers instead of spawning fresh OS threads, so the multiplicative oversubscription disappears and work-stealing keeps every worker busy while any work remains. (This is the deadlock-free nesting the earlier "shared pool risks deadlock" note misjudged — rayon steals inner work rather than blocking on a join.) - **Parallelize the cell loop (`exec.rs:132`), which is C1-disjoint.** This is the main lever: with 42 cells' work in one pool, the per-cell serial seams (the next instrument's ingest transpose, the single-threaded bootstrap) fill with parallel work from other cells, so the cores stay saturated across the seams that currently idle 23 of them. Rationale: a single shared pool over the coarsest-to-finest disjoint work (cells x members x windows x seeds) is the structure that saturates the cores regardless of nesting shape — work-stealing balances flat across levels and across the serial seams, which a per-level bounded pool cannot. Constraints held: **C1 determinism is non-negotiable** — jobs stay pure and disjoint, results are collected in input order, so runs remain bit-identical across worker counts (the existing `_with_threads(1) == _with_threads(8)` tests stay green, extended to the parallel cell loop). Parallel cell execution multiplies the resident working set, so the count of concurrently-active cells is bounded to a RAM budget — swap thrashing would be slower, i.e. the opposite of the goal (this box: ~1.4 GB working set per instrument, 32 GB total). ### Alternative held back - **A thread-local recursion-depth guard** that serializes the inner `run_indexed` level (std-only, no dependency, minimal). It removes the bursts but leaves the valleys idle and under-utilizes when the outer level has few jobs — insufficient for the saturate-the-cores objective. It was the right fix for the narrower "bound the fan-out" framing, superseded by the maintainer's maximize-throughput direction. ## Scope The shared pool, the parallel cell loop, and a reproducible local benchmark that measures core utilization / wall time before and after (the real 42-cell workload's data is external and not reproducible here). RAM-aware bounding of concurrently-active cells is part of the pool work, not a separate track.
Author
Collaborator

Recon: the parallel cell loop is a correctness rework — scope split

A working-set / thread-safety recon of the executor surfaced two facts that
reshape the scope of the maximize-throughput direction.

1. The FileCache resident set scales with concurrently-active distinct
symbols, not pool workers.
The external data-server FileCache (pinned
694f96f) has no capacity/LRU eviction — a symbol's parsed month-files stay
resident until its refcount hits 0 (cache.rs release_symbol/evict_symbol).
Today the cell loop is sequential, so ~1 instrument's files are resident at a
time (each drops before the next opens). Parallel cells over distinct
instruments would hold K instruments' parsed files at once (~1.4 GB each here),
so a parallel cell loop needs a RAM budget bounding concurrently-active
instruments, not just a worker cap. The per-member SoA buffer itself is local
to one run_member and drops on return (bounded by worker count) — it is the
FileCache retention, not the ring buffer, that drives RAM.

2. The Registry write path is not concurrency-safe. Registry::append_family
is called once per cell per stage from inside the hot loop (exec.rs:310-312)
and does an unsynchronized OpenOptions::append().open() + writeln! to a
shared families.jsonl (lineage.rs:263-288); its own doc states "single-process
CLI invocations do not race". Parallel cells would interleave concurrent appends
to the same file — a correctness bug, not a perf issue. A registry-write fault
also escapes #272's per-cell fault isolation (it aborts the whole run via the
top-level ?, exec.rs:148-156).

Confirmed harmless: SilencedPanic (exec.rs:594-627) is already
multi-thread-correct (refcounted hook swap, no recursive lock, no deadlock) — a
parallel cell loop adds contention on its one global mutex but no correctness
hazard. No thread_local!/RefCell anywhere in the hot path. The cell-loop
accumulators (cells_out/cells_rec/nominees, exec.rs:118-123) are
sequential-loop artifacts a parallel loop must rebuild with the established
Mutex<Vec> + index-resort pattern (run_members already uses it for faults).

Revised sequencing (derived)

The oversubscription fix and the valley fix separate cleanly, and the recon shows
the valley fix carries its own correctness work — so the direction lands as a
short sequence rather than one par_iter:

  • This issue (#268) — the shared worker pool. Its title. Replace the per-call
    std::thread::scope fan-out in run_indexed with one process-wide rayon pool;
    nested par_iter shares the same N workers, so the multiplicative nesting
    (walk_forward windows x inner-sweep members => ~216 threads) collapses to
    <= available_parallelism(). Self-contained in aura-engine, C1-preserving,
    RAM-neutral (one cell = one symbol; the shared pool only reduces concurrent
    threads vs today). This is also the prerequisite for any cell-loop parallelism —
    without a shared pool, parallel cells would spawn worker-count x cell-count
    threads.
  • Follow-up: Registry write-path concurrency-safety. A correctness fix pinned
    by a failing test first (concurrent appends corrupt families.jsonl). The
    prerequisite for a parallel cell loop.
  • Follow-up: the parallel cell loop. On the shared pool, with the accumulator
    rebuild and a RAM budget bounding concurrently-active instruments. The valley
    lever (the ~42% idle).

Basis: derived. The maximize-throughput direction (maintainer, 2026-07-15) is
unchanged; the recon just shows it is a sequence whose later steps carry
correctness work, and that the shared pool both stands alone (it fixes the
nesting oversubscription regardless) and is the prerequisite for the rest. The
benchmark accompanies each step. Follow-up issues will be filed as this pool
work lands.

## Recon: the parallel cell loop is a correctness rework — scope split A working-set / thread-safety recon of the executor surfaced two facts that reshape the scope of the maximize-throughput direction. **1. The FileCache resident set scales with concurrently-active distinct symbols, not pool workers.** The external `data-server` FileCache (pinned `694f96f`) has no capacity/LRU eviction — a symbol's parsed month-files stay resident until its refcount hits 0 (`cache.rs` release_symbol/evict_symbol). Today the cell loop is sequential, so ~1 instrument's files are resident at a time (each drops before the next opens). Parallel cells over *distinct* instruments would hold K instruments' parsed files at once (~1.4 GB each here), so a parallel cell loop needs a RAM budget bounding concurrently-active *instruments*, not just a worker cap. The per-member SoA buffer itself is local to one `run_member` and drops on return (bounded by worker count) — it is the FileCache retention, not the ring buffer, that drives RAM. **2. The Registry write path is not concurrency-safe.** `Registry::append_family` is called once per cell per stage from inside the hot loop (`exec.rs:310-312`) and does an unsynchronized `OpenOptions::append().open()` + `writeln!` to a shared `families.jsonl` (`lineage.rs:263-288`); its own doc states "single-process CLI invocations do not race". Parallel cells would interleave concurrent appends to the same file — a correctness bug, not a perf issue. A registry-write fault also escapes #272's per-cell fault isolation (it aborts the whole run via the top-level `?`, `exec.rs:148-156`). Confirmed harmless: `SilencedPanic` (`exec.rs:594-627`) is already multi-thread-correct (refcounted hook swap, no recursive lock, no deadlock) — a parallel cell loop adds contention on its one global mutex but no correctness hazard. No `thread_local!`/`RefCell` anywhere in the hot path. The cell-loop accumulators (`cells_out`/`cells_rec`/`nominees`, `exec.rs:118-123`) are sequential-loop artifacts a parallel loop must rebuild with the established `Mutex<Vec>` + index-resort pattern (`run_members` already uses it for faults). ### Revised sequencing (derived) The oversubscription fix and the valley fix separate cleanly, and the recon shows the valley fix carries its own correctness work — so the direction lands as a short sequence rather than one par_iter: - **This issue (#268) — the shared worker pool.** Its title. Replace the per-call `std::thread::scope` fan-out in `run_indexed` with one process-wide rayon pool; nested `par_iter` shares the same N workers, so the multiplicative nesting (walk_forward windows x inner-sweep members => ~216 threads) collapses to <= available_parallelism(). Self-contained in aura-engine, C1-preserving, RAM-neutral (one cell = one symbol; the shared pool only *reduces* concurrent threads vs today). This is also the prerequisite for any cell-loop parallelism — without a shared pool, parallel cells would spawn worker-count x cell-count threads. - **Follow-up: Registry write-path concurrency-safety.** A correctness fix pinned by a failing test first (concurrent appends corrupt `families.jsonl`). The prerequisite for a parallel cell loop. - **Follow-up: the parallel cell loop.** On the shared pool, with the accumulator rebuild and a RAM budget bounding concurrently-active instruments. The valley lever (the ~42% idle). Basis: derived. The maximize-throughput direction (maintainer, 2026-07-15) is unchanged; the recon just shows it is a sequence whose later steps carry correctness work, and that the shared pool both stands alone (it fixes the nesting oversubscription regardless) and is the prerequisite for the rest. The benchmark accompanies each step. Follow-up issues will be filed as this pool work lands.
Author
Collaborator

Spec auto-signed (grounding-check PASS)

The design spec for the shared worker pool — replacing the per-call
std::thread::scope fan-out in the engine's disjoint-parallel core with one
process-wide rayon pool — was auto-signed under /boss on a grounding-check
PASS, with no human in the loop. The PASS ratified every load-bearing assumption
about current behaviour against named, currently-green tests: the run_indexed
enumeration-order contract, the _with_threads(1) == _with_threads(8) == public
determinism tests across sweep / monte_carlo / walk_forward, the n == 0
empty-vec contract, and the campaign catch_unwind wrapping that keeps a member
panic out of the pool. The rayon-library properties the change relies on
(order-preserving indexed collect, deadlock-free nested work-stealing, 1-worker
inline progress) are the new dependency's, out of scope for ratification — but
the retained _with_threads(1) test goes red on a 1-worker nested-join deadlock,
so even that risk sits under an existing green test kept green.

Design settled — ready for planning.

## Spec auto-signed (grounding-check PASS) The design spec for the shared worker pool — replacing the per-call `std::thread::scope` fan-out in the engine's disjoint-parallel core with one process-wide rayon pool — was auto-signed under `/boss` on a `grounding-check` PASS, with no human in the loop. The PASS ratified every load-bearing assumption about current behaviour against named, currently-green tests: the `run_indexed` enumeration-order contract, the `_with_threads(1) == _with_threads(8) == public` determinism tests across sweep / monte_carlo / walk_forward, the `n == 0` empty-vec contract, and the campaign `catch_unwind` wrapping that keeps a member panic out of the pool. The rayon-library properties the change relies on (order-preserving indexed collect, deadlock-free nested work-stealing, 1-worker inline progress) are the new dependency's, out of scope for ratification — but the retained `_with_threads(1)` test goes red on a 1-worker nested-join deadlock, so even that risk sits under an existing green test kept green. Design settled — ready for planning.
Author
Collaborator

Delivered: the shared pool (commit b19eba6, branch worktree-issue-268-executor-pool)

run_indexed now drives one process-wide rayon pool instead of a per-call
std::thread::scope batch. The public sweep/monte_carlo/walk_forward call
it on the ambient pool through a shared assemble_* helper; the _with_threads
variants are now thin, test-only (#[cfg(test)]) wrappers running the same
assembly inside a local pool. The nested walk_forward -> sweep no longer
multiplies OS-thread batches — nested par_iter work-steals within the pool, so
live workers stay <= available_parallelism().

C1 held bit-for-bit: the _with_threads(1) == _with_threads(8) == public
determinism tests stay green (now driving local rayon pools of 1 and N workers).
Added a nested-oversubscription regression test (peak concurrency <= pool size,
where the pre-rayon shape ran POOL^2) and an ignored wall-time benchmark.
Verified: aura-engine 276 passed, clippy -D warnings clean, workspace tests no
failures.

The two follow-ups the recon surfaced are filed:

  • #276 — Registry write-path concurrency-safety (the prerequisite; the
    unsynchronized append_family races on a shared file). RED-first.
  • #277 — parallelize the C1-disjoint cell loop with a RAM budget bounding
    concurrently-active instruments (the valley lever, the ~42% idle). Needs a
    spec for the budget mechanic.

The pool is the first of three steps; the valleys — the main throughput win —
land with #276 then #277, both building on this branch once merged. Awaiting
review + merge.

## Delivered: the shared pool (commit b19eba6, branch worktree-issue-268-executor-pool) `run_indexed` now drives one process-wide rayon pool instead of a per-call `std::thread::scope` batch. The public `sweep`/`monte_carlo`/`walk_forward` call it on the ambient pool through a shared `assemble_*` helper; the `_with_threads` variants are now thin, test-only (`#[cfg(test)]`) wrappers running the same assembly inside a local pool. The nested `walk_forward` -> `sweep` no longer multiplies OS-thread batches — nested `par_iter` work-steals within the pool, so live workers stay <= `available_parallelism()`. C1 held bit-for-bit: the `_with_threads(1) == _with_threads(8) == public` determinism tests stay green (now driving local rayon pools of 1 and N workers). Added a nested-oversubscription regression test (peak concurrency <= pool size, where the pre-rayon shape ran POOL^2) and an ignored wall-time benchmark. Verified: aura-engine 276 passed, clippy -D warnings clean, workspace tests no failures. The two follow-ups the recon surfaced are filed: - **#276** — Registry write-path concurrency-safety (the prerequisite; the unsynchronized `append_family` races on a shared file). RED-first. - **#277** — parallelize the C1-disjoint cell loop with a RAM budget bounding concurrently-active instruments (the valley lever, the ~42% idle). Needs a spec for the budget mechanic. The pool is the first of three steps; the valleys — the main throughput win — land with #276 then #277, both building on this branch once merged. Awaiting review + merge.
Author
Collaborator

Rebased onto the new main after #275 landed (main advanced ad4249f88667a6, the FF #275 merge). The branch was based on ad4249f; it now sits linearly on 88667a6 as commit b048923 (was b19eba6).

The integration is clean: source files are disjoint from #275 (#268 touches only aura-engine/src/{sweep,mc,walkforward}.rs + Cargo.toml; #275 touched harness.rs/lib.rs/blueprint.rs/report.rs + the CLI), the only shared file is the auto-regenerated Cargo.lock, and git merge-tree reported no conflicts.

Re-verified green on the combined tree: cargo build --workspace clean, aura-engine 283 passed / 0 failed, the 4 C1 deterministic_across_thread_counts tests green (the executor's _with_threads(1)==_with_threads(8)==public invariant holds over #275's reworked harness), the nested-oversubscription regression test green, full workspace suite no failures, clippy --workspace --all-targets -D warnings clean. The rayon F: Sync bound also type-proves that #275's bind_sources/SourceSpec.role path introduced nothing non-Sync into what run_one closes over.

The branch stays FF-mergeable onto current main. Still awaiting review + merge; #276#277 build on it once merged.

Rebased onto the new main after #275 landed (main advanced ad4249f → 88667a6, the FF #275 merge). The branch was based on ad4249f; it now sits linearly on 88667a6 as commit `b048923` (was `b19eba6`). The integration is clean: source files are disjoint from #275 (#268 touches only `aura-engine/src/{sweep,mc,walkforward}.rs` + `Cargo.toml`; #275 touched `harness.rs`/`lib.rs`/`blueprint.rs`/`report.rs` + the CLI), the only shared file is the auto-regenerated `Cargo.lock`, and `git merge-tree` reported no conflicts. Re-verified green on the combined tree: `cargo build --workspace` clean, `aura-engine` 283 passed / 0 failed, the 4 C1 `deterministic_across_thread_counts` tests green (the executor's `_with_threads(1)==_with_threads(8)==public` invariant holds over #275's reworked harness), the nested-oversubscription regression test green, full workspace suite no failures, `clippy --workspace --all-targets -D warnings` clean. The rayon `F: Sync` bound also type-proves that #275's `bind_sources`/`SourceSpec.role` path introduced nothing non-`Sync` into what `run_one` closes over. The branch stays FF-mergeable onto current main. Still awaiting review + merge; #276 → #277 build on it once merged.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Brummel/Aura#268