Inside DuckDB 2.0: Morsel-Driven Parallelism, SIMD Filters, and the Micro-Architecture of In-Process Analytics

The Shift from Distributed Bloat to In-Process Analytical Engines

For nearly a decade, data engineering was dominated by a single design dogma: if you need analytical throughput, deploy a distributed cluster. Systems like Apache Spark, Presto, and cloud data warehouses became the default architecture even for workloads involving tens or hundreds of gigabytes—well within the addressable memory limits of a modern developer workstation. The overhead of this approach was immense: serialization costs, network partition latency, distributed consensus coordination, and brittle container orchestration.

DuckDB radically disrupted this paradigm by proving that micro-architectural sympathetic design—optimizing for CPU cache hierarchies, memory bandwidth, and SIMD instruction sets—often outperforms distributed architectures by orders of magnitude for small-to-medium scale OLAP queries.

With DuckDB 2.0 on the horizon, the database engine undergoes fundamental changes in its query execution pipeline, storage layer, and vector scheduling mechanics. Understanding these engineering choices offers critical insights for systems developers building high-performance data processing pipelines and edge applications.

---

Morsel-Driven Parallelism: Dynamic Work Stealing Across Cores

Traditional relational query engines frequently execute parallel pipelines using static partition splits. If a query scans eight Parquet files, an engine might assign one thread per file. If one file contains high-entropy string columns requiring heavy dictionary decoding while others contain easily compressible integers, worker threads suffer from severe thread skew. Cores sit idle while a single lagging thread bottlenecks the pipeline.

DuckDB 2.0 builds upon the seminal Morsel-Driven Parallelism model originally pioneered by Leis et al. at TU Munich, adapting it for modern multi-core heterogeneous architectures (such as Apple Silicon and AMD Zen 4 NUMA domains).

+--------------------------------------------------------------+
|                        Global Pipeline                       |
|  [Scan Parquet] -> [Filter & Project] -> [Local Hash Table]  |
+--------------------------------------------------------------+
                               | (Pulls 2048-tuple Morsels)
       +-----------------------+-----------------------+
       |                                               |
+------v-------+                               +-------v------+
|   Worker 1   |                               |   Worker N   |
| - Core 0     |                               | - Core M     |
| - Local Sink |                               | - Local Sink |
+--------------+                               +--------------+
       |                                               |
       +-----------------------+-----------------------+
                               v (Lock-free Merge)
+--------------------------------------------------------------+
|                      Global Hash Join Sink                   |
+--------------------------------------------------------------+

How DuckDB Schedules Execution Morsels

1. Fixed-Size Chunk Allocation: The query planner decomposes intermediate relations into standard DataChunks—vectors containing up to 2,048 tuples.
2. Pull-Based Task Graphs: Rather than pushing data across thread boundaries via inter-thread channels (which introduce cache line bouncing and lock contention), workers actively pull fixed-size morsels from a concurrent task queue.
3. Lock-Free State Aggregations: During hash joins or grouped aggregations, each worker constructs a thread-local hash table. Only upon completing the pipeline break does the engine execute a parallel, lock-free radix merge into the final global state.

This architecture guarantees that cores automatically pick up remaining work slices regardless of column skew, ensuring near-linear CPU saturation across 32+ cores without OS-level thread swapping overhead.

---

The Vectorized Execution Model: Defeating the Memory Wall

Relational engines historically relied on the classic Volcano Iterator Model, where an execution operator exposes a next() method returning a single tuple. While conceptually elegant, this model incurs catastrophic penalties on modern hardware:

• Branch Mispredictions: Virtual function calls for every single tuple disrupt the CPU pipeline.
• Poor Instruction Cache Locality: The CPU constantly jumps between scan, filter, and projection instructions.
• Zero Auto-Vectorization: Compilers cannot emit SIMD (AVX2, AVX-512, NEON) instructions when operations process individual scalars.

DuckDB eschews both pure tuple-at-a-time execution and pure whole-stage JIT code generation (used by systems like Hyper) in favor of Vectorized Block Execution.

// Conceptual DuckDB Vector Filter Primitive
void VectorizedGreaterThanConstant(const int32_t* __restrict src,
                                   const int32_t target_val,
                                   uint16_t* __restrict sel_out,
                                   idx_t& count) {
    idx_t match_count = 0;
    #pragma clang loop vectorize(enable)
    for (idx_t i = 0; i < count; ++i) {
        // Compiler emits SIMD compare: vpcmpeqd / vpcmpgtd
        if (src[i] > target_val) {
            sel_out[match_count++] = static_cast<uint16_t>(i);
        }
    }
    count = match_count;
}

Selection Vectors: Zero-Copy Filtering

In DuckDB 2.0, when a filter predicate executes, the engine does not allocate memory and physically copy qualifying rows into a new vector buffer. Instead, it computes a Selection Vector—a 16-bit array of indices referencing the original memory segment.

Subsequent operators (projections, arithmetic evaluations) operate over this indirect selection vector. If a pipeline applies multiple consecutive filters (e.g., WHERE age > 30 AND status = 'ACTIVE'), DuckDB updates the selection vector in-place. Physical copying is deferred until absolutely necessary, such as when flushing tuples to an external network socket or writing unaligned blocks to disk.

---

Micro-Architectural Optimizations: Fast Paths in DuckDB 2.0

The architectural jump in DuckDB 2.0 focuses on granular hardware-level efficiency:

1. Unified Out-of-Core Execution and Buffer Manager Paging

Earlier versions of embedded engines could choke when intermediate hash tables exceeded physical RAM, requiring the user to configure external temp storage manually. DuckDB 2.0 introduces an adaptive, unified buffer manager. When memory consumption approaches the configured limit, the engine automatically pivots from purely in-memory hash joins to Partitioned Grace Hash Joins.

Intermediate partitions are streamed to a lightweight, block-compressed temporary disk file using direct I/O (O_DIRECT), bypassing the OS page cache entirely to eliminate double-buffering penalties.

2. SIMD String Lookups and Compressed Null Masks

String handling has seen a radical overhaul. Strings up to 12 bytes are directly inlined into the vector pointer itself (4 bytes length prefix, 4 bytes prefix cache, and 4 bytes inline payload).

When evaluating string equivalence (WHERE country = 'UNITED_KINGDOM'), DuckDB performs a 4-byte prefix comparison first. Over 90% of non-matching comparisons are rejected with a single scalar register comparison before the engine ever dereferences memory pointers.

Null tracking utilizes a dense bitmask layout. Mask evaluation leverages bit-manipulation primitives (__builtin_popcount / _mm_popcnt_u64), allowing the engine to check 64 records for nullability in a single CPU cycle.

---

Practical Example: Zero-Copy Interop with Apache Arrow

One of the most potent production patterns enabled by DuckDB is querying in-memory columnar streams directly without copying data across application boundaries.

Below is an end-to-end Python pipeline utilizing DuckDB and PyArrow to run analytical aggregations directly over an Arrow stream:

import duckdb
import pyarrow as pa
import pyarrow.compute as pc
import numpy as np

# 1. Generate 50 Million records in a contiguous memory block
num_records = 50_000_000
categories = np.random.choice(["ALPHA", "BETA", "GAMMA", "DELTA"], size=num_records)
metrics = np.random.normal(loc=100.0, scale=15.0, size=num_records)

arrow_table = pa.Table.from_arrays([
    pa.array(categories),
    pa.array(metrics)
], names=["category", "metric_val"])

# 2. Spin up an in-process DuckDB session
con = duckdb.connect(database=":memory:")

# 3. Register the Arrow Table (Zero-Copy Pointer Registration)
con.register("arrow_source", arrow_table)

# 4. Execute complex OLAP aggregation entirely in DuckDB's vectorized core
query = """
SELECT 
    category,
    COUNT(*) AS total_samples,
    AVG(metric_val) AS mean_metric,
    APPROX_QUANTILE(metric_val, 0.95) AS p95_metric
FROM arrow_source
WHERE metric_val > 90.0
GROUP BY category
ORDER BY mean_metric DESC;
"""

# Execute and materialize directly back into Arrow
result_arrow = con.execute(query).fetch_arrow_table()
print(result_arrow.to_pandas())

Because DuckDB implements the Arrow C Data Interface, registration involves passing memory addresses rather than serializing data. DuckDB's scan operator iterates over Arrow's record batches by casting them directly to internal Vector references, avoiding gigabytes of memory duplication.

---

Key Architectural Takeaways for Software Engineers

DuckDB 2.0 shows how thoughtful systems programming outperforms indiscriminate infrastructure scaling. As you design data-intensive client applications or high-throughput backends, keep these principles in mind:

1. Hardware Locality Trumps Raw Parallelism: An un-vectorized algorithm running on 64 distributed nodes will frequently lose to a single-core implementation optimized for CPU instruction pipelines and memory bandwidth.
2. Avoid Tuple Iteration: If your internal engine handles streaming data or analytics, refactor APIs from NextTuple() to ProcessVector(Batch). The gains in cache residency and SIMD auto-vectorization make it an unmatched performance lever.
3. Adopt Standard Columnar Formats: Leveraging zero-copy standards like Apache Arrow eliminates the boundary penalties between programming runtimes (Rust, Python, Go, C++) and the storage layer.