Table of Contents
Columnar Data Layout (Structure-of-Arrays)
A high-throughput numeric streaming hot path scans the same field across many records, computes over it, and moves on — it almost never wants one whole record at a time. The data-layout decision that follows from this access pattern is columnar: store each field as its own contiguous column, the arrangement known in systems programming as Structure-of-Arrays (SoA) and in databases as column-oriented storage. The two names describe the same physical idea — same-type values of one field laid out adjacent in memory — and the same payoff: better cache-line utilization on scans and amenability to SIMD vectorization. This page defines the layout, explains why it suits streaming numerics, relates it to columnar analytics and the Apache Arrow in-memory format, and states the trade-offs that decide when not to use it.
Markers: [L] law/exact, [C] convention, [CORR] corrected.
See also: Reactive Streaming Dataflow, Backtesting Engine Architecture, Determinism and Reproducibility, Metric Catalogue, Metric Verification Log.
AoS vs SoA: two ways to lay out a sequence of records
Given a sequence of records, each with several fields, there are two contrasting ways to arrange them in linear memory, differing in whether fields are interleaved.
Array-of-Structures (AoS) is the conventional layout "in which data for different fields is interleaved" [L] (AoS and SoA). All fields of record 0 sit adjacent, then all fields of record 1, and so on. For a record of three coordinates (x, y, z), memory reads:
x0 y0 z0 x1 y1 z1 x2 y2 z2 ...
This is the layout most programming languages give by default when an array holds a struct type — it is "more intuitive, and supported directly by most programming languages" [L] (AoS and SoA).
Structure-of-Arrays (SoA) is "a layout separating elements of a record … into one parallel array per field" [L] (AoS and SoA). Each field becomes its own contiguous column; the record at index i is reconstructed by reading index i of every column. The same three-coordinate data reads:
x0 x1 x2 ... (the x column)
y0 y1 y2 ... (the y column)
z0 z1 z2 ... (the z column)
A record is now a logical tuple — index i across the columns — not a contiguous physical object. The columnar / column-oriented database community describes precisely this layout from the storage side: a column store places "each column of the table … one after the other. In this orientation, values on the same column are close in space," whereas a row store places "each row … one after the other" so that "values in the same row are close in space" [L] (Column-oriented DBMS). SoA (the in-memory term) and column-oriented storage (the database term) are the same physical decision viewed from two communities.
A third, hybrid layout, Array-of-Structures-of-Arrays (AoSoA), interleaves fields "using tiles or blocks with size equal to the SIMD vector size" [L] (AoS and SoA). It "can achieve the memory throughput of the SoA approach, while being more friendly to the cache locality and load port architectures of modern processors" [C] (AoS and SoA). AoSoA is an advanced tuning option; the rest of this page treats the AoS/SoA dichotomy, which is the decision that matters for a streaming hot path.
Why SoA for streaming numerics
The case for columnar layout in a numeric streaming hot path rests on two hardware facts — cache-line transfer and SIMD vector registers — and one workload fact: the hot path scans one field across many records.
Cache lines and spatial locality
Memory does not move between RAM and the CPU one byte at a time. "Data is transferred between memory and cache in blocks of fixed size, called cache lines or cache blocks" [L] (CPU cache). On widely deployed hardware a cache line is 64 bytes [C] — e.g. the original Pentium 4 had "64-byte cache blocks" in its L1 data cache [L] (CPU cache); the exact size is microarchitecture-dependent (32, 64, and 128-byte lines all appear historically), so 64 bytes is a convention, not a universal law.
Because a whole line is fetched on every miss, the layout that fills each line with bytes the loop will actually use wins. This is spatial locality: "if a particular storage location is referenced at a particular time, then it is likely that nearby memory locations will be referenced in the near future" [L] (Locality of reference), and "data elements are brought into cache one cache line at a time … if one element is referenced, a few neighboring elements will also be brought into cache" [L] (Locality of reference).
Now apply this to a scan of one field across many records:
- SoA. The field's values are contiguous, so every byte of every fetched cache line is a value the loop consumes. Reading an 8-byte field, a 64-byte line delivers 8 consecutive values — eight usable values per memory transaction. "If only a specific part of the record is needed, only those parts need to be iterated over, allowing more data to fit onto a single cache line" [L] (AoS and SoA).
- AoS. The field's values are separated by the other fields of each record. A fetched line is mostly the other fields, which the single-field scan discards. The concrete contrast: a column-scan over an 8-byte field "loads 8 … entries at once (8 * 8 byte = 64 byte), whereas the AoS storage can only move 2 records (2 * 32 byte = 64 byte) per cache line" — so on such a scan the AoS layout "wastes 75% of the memory bandwidth, achieving a goodput of only 16 bytes per 64 bytes loaded" [C] (CedarDB — Optimizing Data Layouts).
The cache-line transfer mechanism and what a given layout puts on a line are facts of the hardware and the layout [L]. The resulting magnitude of any speed-up ("4× faster", "wastes 75%") is workload-, field-width-, and machine-dependent [C] — quoted figures illustrate the mechanism, they are not portable constants.
SIMD and auto-vectorization
The second reason is SIMD (Single Instruction, Multiple Data): hardware with "multiple processing elements that perform the same operation on multiple data points simultaneously," exploiting "data level parallelism" [L] (SIMD). A SIMD vector register holds a fixed number of equal-width elements, called lanes; an instruction operates on every lane at once. Intel AVX-512 "SIMD instructions process 512 bits of data at once" [L] (SIMD) — sixteen 32-bit floats, or eight 64-bit values, per instruction.
To fill a vector register in a single load, the elements must be contiguous and same-type — exactly what a SoA column is. Automatic vectorization is "a special case of automatic parallelization, where a computer program is converted from a scalar implementation, which processes a single pair of operands at a time, to a vector implementation, which processes one operation on multiple pairs of operands at once" [L] (Automatic vectorization). The canonical vectorizable kernel is a tight loop over contiguous arrays:
for (i = 0; i < n; i++)
c[i] = a[i] + b[i]; // each of a, b, c is a contiguous SoA column
The compiler turns this into vector adds over batches of lanes. SoA "can greatly benefit of auto vectorization within the compiler" [C] (CedarDB — Optimizing Data Layouts). AoS works against the vectorizer: a single-field scan over an interleaved layout needs gathers (strided/scattered loads) to assemble a vector register, which is slower than a contiguous load and often blocks vectorization outright. The relationship is structural [L]: a contiguous same-type column can be loaded straight into a vector register; an interleaved field generally cannot. (Vectorization additionally requires the compiler to prove the loop carries no disqualifying dependence — a separate condition from layout — (Automatic vectorization).)
For a streaming engine, this matters because the per-tick work over a window is typically a tight reduction or map over one column — moving averages, sums, comparisons, exposure integration. Columnar layout is what lets that inner loop vectorize. See Reactive Streaming Dataflow for how such per-field work is scheduled across nodes, and Backtesting Engine Architecture for the overall hot-path shape.
Columnar analytics: column stores and Apache Arrow
Columnar layout is the defining choice of analytical (OLAP) data systems, and the reasoning there mirrors the streaming case exactly.
Selective column I/O. An analytical query usually reads a few fields of a wide table. With columns stored separately, "a query that aggregates prices touches only the price column and skips everything else. On a table with 50 columns, a query that reads 3 of them does roughly 6% of the I/O that a row store would need" [C] (ClickHouse — What is a columnar database). The same logic that keeps unused fields off a cache line at the CPU level keeps unused columns off the I/O path at the storage level.
Vectorized execution. Column stores "make analytical operations like filtering, grouping, aggregations and others more efficient thanks to memory locality," and enable "vectorization of the computations" because "most modern CPUs have SIMD instructions" [L] (Apache Arrow — Introduction). The columnar execution model loads a batch of one column's values into a vector register and applies an operation to all lanes together.
Compression. Adjacency of same-type values is what makes columns compress well. "Because adjacent values in a column share the same data type — and often similar magnitudes — encodings like dictionary encoding, run-length encoding, and delta encoding achieve compression ratios of 5–10× on real-world columnar data" [C] (ClickHouse — What is a columnar database). The classic illustration: a 128-row boolean column "requires 128 bytes in a row-oriented format … but 128 bits (16 bytes) in a column-oriented format (via a bitmap)" [L] (Column-oriented DBMS). The compression ratios are data-dependent [C]; the reason compression is possible — homogeneous adjacent values — is structural [L].
Apache Arrow as a reference columnar format
Apache Arrow is a standardized, language-independent in-memory columnar format, and a useful reference for how a columnar layout is specified in practice. Its stated design goals read as a checklist of the benefits above: "Data adjacency for sequential access (scans)", "O(1) (constant-time) random access", "SIMD and vectorization-friendly", and arrays "Relocatable without 'pointer swizzling', allowing for true zero-copy access in shared memory" [L] (Apache Arrow — Columnar Format).
Arrow also documents the alignment convention that makes columns vectorize cleanly: a recommended 64-byte alignment and padding, because "the recommendation for 64 byte alignment comes from the Intel performance guide that recommends alignment of memory to match SIMD register width," and "the recommended padding of 64 bytes allows for using SIMD instructions consistently in loops without additional conditional checks" [C] (Apache Arrow — Columnar Format). 64 bytes matches AVX-512's 512-bit register; it is a convention pinned to a specific (widely deployed) SIMD width, not a universal law.
Crucially, Arrow shows how composite / nested data is represented as a bundle of plain columns plus structure metadata rather than as interleaved records. Each column is an Array made of "one or more buffers"; a struct type "has one child array for each field," and those "child arrays are independent and need not be adjacent to each other in memory" [L] (Apache Arrow — Introduction). A nested list keeps "all the values being stored consecutively in a values child array" with a separate offsets buffer [L] (Apache Arrow — Introduction). This is the same principle a streaming engine applies to a composite stream such as OHLCV: it is not one interleaved record stream but a bundle of base-typed columns travelling together (see the closed-set section below).
Trade-offs: choose by access pattern
Columnar layout is not universally faster — it is faster for the access pattern it is built for, and slower for the opposite one. The decision is governed by how the data is accessed, not by which layout is "modern".
SoA / columnar wins when the operation scans or transforms one (or a few) fields across many records: aggregations, filters, vectorized math, streaming reductions over a window. With a field's values laid out as a contiguous column, "if only a specific part of the record is needed, only those parts need to be iterated over, allowing more data to fit onto a single cache line," and the homogeneous adjacency enables "easier manipulation with packed SIMD instructions … since a single SIMD register can load homogeneous data" [C] (AoS and SoA). The cache-line and SIMD arguments above all apply.
AoS / row layout wins when the operation touches most fields of one record at a time — random whole-record access, per-record mutation, and transactional insert/update/delete. "If you usually touch most of the data of your structure when looping over it, it is better to do AoS" [C] (CedarDB — Optimizing Data Layouts), because all of a record's fields then share a cache line and one fetch serves the whole record. From the database side, columnar "provides analytical performance and data locality guarantees in exchange for comparatively more expensive mutation operations" [L] (Apache Arrow — Columnar Format), while row stores offer "fast insertion of a new row" and suit OLTP workloads where complete records are written and read [C] (Column-oriented DBMS). The same split appears system-wide: "most traditional database systems focus on transactional processing … and implement AoS, whereas modern analytical database systems … usually prefer the SoA data orientation" [C] (CedarDB — Optimizing Data Layouts).
The practical rule [C]: profile the real access pattern before restructuring, because the wrong layout for a given pattern can cost a large multiple in cache misses, and the best layout depends on both hardware and access pattern. A deterministic, replayable streaming backtest is squarely in the column-scan regime — it appends and replays, it does not randomly mutate individual past records — which is exactly why columnar is the natural fit. The reproducibility consequences of an append-only, replay-oriented column stream are discussed in Determinism and Reproducibility.
Tie-in: a closed set of scalar machine types, streamed as columns
Columnar layout pays off most when the columns are uniform and machine-vectorizable, which argues for streaming only a small, fixed set of scalar machine types as columns:
- integers (a fixed-width signed integer),
- floating-point (a fixed-width float),
- booleans (naturally a packed bitmap column — the 128-bit-for-128-rows case above), and
- timestamps (a fixed-width integer column under the hood).
Each is fixed-width and same-type down the column, so each fills cache lines fully and loads straight into vector lanes — the two properties this whole page rests on [L]. Keeping the set closed keeps every stream uniform, so a single vectorized scan kernel applies to any column without per-field special-casing.
Composite streams are bundles of base columns. A structured stream such as OHLCV is not an interleaved record stream; it is several base-typed columns (open, high, low, close as float columns; volume as an integer column) carried together — exactly the Arrow model where a struct is "one child array for each field" with independent child buffers [L] (Apache Arrow — Introduction). Any node consuming one field reads one contiguous column; the composite is an authoring-level convenience, not a runtime interleaving.
Non-scalar data lives beside the hot path, not in it. Strings, lookup tables, calendars, and other variable-width or structured values do not belong in a fixed-width, vectorizable column stream — they would break uniformity, defeat SIMD, and bloat cache lines. They are kept as metadata alongside the stream and referenced from it (e.g. by an integer id or dictionary index, the same indirection a column store uses for dictionary encoding), so the hot path stays a set of clean numeric columns. This is the in-memory analogue of Arrow keeping variable-length data in a separate child buffer addressed by offsets [L] (Apache Arrow — Introduction). The metrics computed downstream over these columns are catalogued in Metric Catalogue, and the practice of verifying those numeric results against independent references is covered in Metric Verification Log.
References
AoS vs SoA: definitions and layout
- AoS and SoA — Wikipedia — definitions of Array-of-Structures, Structure-of-Arrays, and the AoSoA hybrid; SIMD and cache-line rationale for SoA; access-pattern trade-offs.
Why SoA for streaming numerics
- CPU cache — Wikipedia — cache lines / blocks as the fixed-size unit of memory transfer; 64-byte (and 32/128-byte) line examples.
- Locality of reference — Wikipedia — spatial vs temporal locality; data brought into cache one line at a time.
- Single instruction, multiple data — Wikipedia — SIMD definition, vector registers and lanes, AVX-512 = 512 bits.
- Automatic vectorization — Wikipedia — converting a scalar loop into vector operations; the contiguous-array loop kernel; dependence conditions.
- Optimizing Data Layouts — CedarDB — cache-line utilization of SoA vs AoS field scans; auto-vectorization of SoA; OLAP-vs-OLTP layout split.
Columnar analytics and Apache Arrow
- Column-oriented DBMS — Wikipedia — column- vs row-oriented storage; analytical-scan and compression benefits; boolean bitmap example; OLTP strengths of row stores.
- Apache Arrow — Columnar Format — design goals (sequential-scan adjacency, O(1) random access, SIMD-friendly, zero-copy); 64-byte alignment/padding rationale; analytics-vs-mutation trade-off.
- Apache Arrow — Introduction — columnar vs row layout and its benefits; nested/struct types as bundles of independent child buffers.
- What is a columnar database — ClickHouse — selective per-column I/O; columnar compression ratios via dictionary/RLE/delta encoding; vectorized execution lineage.
Backtesting Engine Architecture
- Architecture overview
- Event-Driven Backtesting
- Look-Ahead Bias & Causality
- Determinism & Reproducibility
- Reactive Streaming Dataflow
- Columnar Data Layout
- Signal, Exposure & Execution
- Graph Compilation & Optimization
- Authoring & Deployment Lifecycle
Strategy Analysis & Validation