It's the Memory Latency, Stupid
November 1, 2026
When your database CPU shows 100%, you probably think it's working hard. The truth is more uncomfortable: your CPU is mostly waiting. Waiting for memory.
A modern CPU can execute 4 to 8 instructions per nanosecond. A single access to main memory? 50 to 100 nanoseconds. That means for every RAM access, your CPU could have done 400 to 800 instructions. Instead, it sits idle, staring at the memory controller like someone waiting for a slow elevator.
This is the core argument of a recent piece by Maor Kern at Pivot Lake [1]. The title echoes the famous "It's the memory latency, stupid" line that has haunted systems engineers for decades, and the article backs it up with hard numbers from a real GROUP BY benchmark on 99 million rows.
The hierarchy looks like this. L1 cache, 64 KB, about 1 nanosecond per access. L2, around 1 MB, 3 to 4 nanoseconds. L3, a few hundred MB on high-end chips, 10 to 15 nanoseconds. DRAM, gigabytes, 50 to 100 nanoseconds. Each layer down is an order of magnitude slower [1].
The benchmark is simple. Take ClickBench's "hits" dataset, run a count grouped by UserID across 99 million rows with roughly 17 million unique users. The naive approach, a HashMap that grows with every new group key, takes 3.6 seconds on an AMD EPYC 9124 pinned to a single core. Perf counters reveal that 50.1% of CPU slots are stalling on memory. The CPU is not computing. It is waiting [1].
The insight here is not new. It is the same lesson that every database engine designer eventually learns the hard way. ClickHouse, DuckDB, and every serious analytical engine has spent years optimizing for cache locality. The trick is not to make the CPU faster. It is to keep data close enough that the CPU never has to wait.
What makes this piece worth reading is the clarity of the demonstration. It strips away all the clever engineering and shows you the naive baseline, then shows you the perf counters proving that the bottleneck is not computation but memory access patterns. The CPU is at 100% and doing almost nothing useful [1].
For anyone building data-intensive systems, this is required reading. Not because the lesson is new, but because it is the lesson we keep forgetting. Your CPU is faster than your memory. Everything else follows from that.
Source: [1] It's the memory latency, stupid, Pivot Lake Blog, October 28, 2026