Memory Hierarchy¶

Memory systems are organized in layers of decreasing speed and increasing capacity because physics constrains fast memory to be small and close to the processor - understanding this hierarchy explains most performance mysteries.

Tension Addressed¶

Speed vs. Capacity

Recognizable As¶

Any tiered storage system; CPU caches, buffer pools, CDNs, read replicas - all implement the same principle of keeping frequently-accessed data closer to computation.

Canonical Sources¶

Patterson & Hennessy, Computer Organization and Design (1994+); Drepper, "What Every Programmer Should Know About Memory" (2007)

Transfers To¶

Database buffer pools, web caching layers, CDN architecture, distributed systems data locality, storage tiering

The memory hierarchy is not a design choice but a physical necessity. Registers (nanoseconds) -> L1 cache -> L2 cache -> L3 cache -> RAM (100ns) -> SSD (100us) -> HDD (10ms). Each level is roughly 10-100x slower and 10-100x larger than the previous. Software performance often depends on which level serves the data.