How Storage Works, Part 1: The Memory and Storage Hierarchy

Every computer keeps its data in a hierarchy of memories. A register answers in one processor cycle, and a core’s integer registers hold about 2KB. A tapeMagnetic tape stores archive data on cartridges kept in tape libraries, with a very low total cost of ownership and very low power use. cartridge holds 40TB and takes tens of seconds to reach its first byte. Between them sit SRAM caches, DRAM, HBM, CXL memory, persistent memory, SSDs and hard drives, each one a different trade of speed, size, cost and whether the data survives a power cut. This guide puts sourced numbers on all nine, explains why each one exists, and follows data as it moves between them.

Why the hierarchy exists

No single memory technology wins on every axis. Inside the processor sit the fastest cells, which take up the most silicon per bit. Spinning plattersA platter is a spinning disk inside a hard drive that holds the magnetic recording media. One or more actuators give the drive physical access to the data on it. and tape hold the cheapest bits, far from the processor and slow to reach. Every memory in between is a compromise, and a system stacks several of them so that most accesses hit a fast one while most bytes sit on a cheap one.

Programs reuse data, which is why a hierarchy works. A value read once is likely to be read again soon, and its neighbors are likely to be read next. Each tier keeps a copy of what the tier below served most recently, so a hit near the top saves a trip to the bottom. One line of the IEEE’s 2026 storage roadmap sums up the trade: memory performance is correlated with cost per bit.

Why the hierarchy exists

Each step down holds more and answers later

One real part per memory, log scales on both axes

Each dot is a single real part, with one exception. The tape dot pairs a 40TB LTO-10 cartridge with the 65s access time of a 2017 LTO-7 library, which is the only tape access time in the sources. Zen 5 cache latencies are AMD’s cycle counts, converted at an assumed 5GHz clock. HBM2 on an NVIDIA V100 gives the HBM dot. The DRAM and Optane dots were measured on one Intel server, and the SSD and hard drive are my own CrystalDiskMark runs on my test system, four runs each. From the lower left to the upper right, the dots climb in one rough line, because every step down buys capacity with time. One wide break interrupts that line, between Optane persistent memory at 305ns and the NAND SSD at 52µs, and that gap is where most of the newer memories try to fit. CXL memory has no dot, because no source gives one device’s latency and capacity together, and Figure 4 places it.

Below, a table puts the same memories on the axes the chart leaves out, bandwidth and the capacity of one part. Latency repeats so each row reads across.

Memory	Example part	Access latency	Bandwidth	Capacity of one part
Registers	AMD Zen 5 core	1 cycle	n/a	240 integer registers, 1.9KB
SRAM caches	AMD Zen 5	4–46 cycles (L1 to L3)	2TB/s to a stacked L3 die (Zen 3)	48KB L1 to 96MB L3
HBM	NVIDIA V100 (HBM2); Rubin (HBM4)	375 GPU cycles, about 272ns (HBM2)	900GB/s (V100); 19.2TB/s (Rubin)	32GB (V100); 288GB (Rubin)
DRAM	DDR4 and DDR5 server memory	80–81ns	1.6TB/s per socket (MRDIMM)	32GB module
CXL memory	DRAM in a CXL pool (8–16 sockets)	151–171ns (estimate)	64GB/s each way, x16	DRAM modules on the card
Persistent memory	Intel Optane DC PMem	305ns random, 169ns sequential	6.6GB/s read per module	128–512GB per module
NVMe SSD	Kioxia XG8 2TB, my test system	52µs (4KiB, QD1)	7.1GB/s read	245.76TB in the largest drives
Hard drive	WD Red Pro 20TB, my test system	4.2ms (4KiB, QD1)	270MB/s read	up to 44TB, in volume to two cloud providers
Tape	Linear Tape-Open (LTO-10) cartridge	tens of seconds	400MB/s native per drive	40TB native

Rows mix kinds of number on purpose, and each section below says where its numbers come from. Treat the table as a scale of orders of magnitude.

Registers and SRAM caches

Registers and caches sit on the processor die and are built from static RAM (SRAMSRAM is fast memory of latching cells with several transistors each. It needs no refresh but takes far more area per bit than DRAM, so it serves as on-chip memory.). In the IEEE’s 2022 More Moore roadmap, the six-transistor cell is the conventional SRAM cell. Six transistors per bit is fast and needs no refresh, but it is large, and an MIT lecture puts a DRAM cell at about a twentieth of an SRAM cell’s area. That area cost is why the caches stay small. Working Memory, Part 1 covers the SRAM cell and why it has stopped shrinking with logic.

AMD’s Zen 5 optimization guide gives every level its size and latency. Zen 5’s integer physical register file holds 240 registers of 64 bits, about 1.9KB, of which up to 40 per thread hold architectural state and the rest serve out-of-order renames. A simple integer operation on two registers completes in one cycle. An integer load from the 48KB L1 data cache takes 4 or 5 cycles. A hit in the 1MB L2 takes at least 14 cycles. The L3 (up to 96MB, shared by the cores in a complex) averages 46 cycles.

Registers and SRAM caches

Each step away from the core holds more and costs more cycles

AMD Zen 5: capacity as bar length on a log scale, load-to-use latency in cycles

Each step away from the core multiplies capacity and adds cycles. L1 to L2 is about 21 times the capacity for three to four times the latency. L2 to L3 is 96 times the capacity for about three times the latency. A load that misses all three caches goes to DRAM, about 80ns or roughly 400 cycles at the same assumed 5GHz clock, so a miss costs about 100 times an L1 hit. Data moves between all of these levels in 64-byte cache linesA cache line is the block of neighboring bytes a cache stores and moves as one unit. When the processor needs any of them, the whole line is loaded., never single bytes, and the L2 to L1 path is one line wide.

AMD also stacks extra L3 on top of the core die. On Zen 3, the hybrid-bonded V-Cache die in Wuu and colleagues’ ISSCC 2022 paper raised the L3 from 32MB to 96MB, with up to 2TB/s between the two dies. Stacking adds capacity without moving the cache further from the cores, which is the same move HBM makes for DRAM.

DRAM and HBM: main memory, and memory built for bandwidth

DRAMDRAM stores each bit as charge on a tiny capacitor behind one transistor. The charge leaks away, so every cell must be refreshed many times a second, and the data is gone when power is removed. is main memory because its cell is small: one transistor and one capacitor. Its capacitor leaks, so every row has to be read and written back on a schedule. Micron’s DDR4 datasheet requires 8,192 refresh commands every 64ms. DRAM sits off the processor die, on modules, so a load that reaches it costs about 80ns. About 80ns is the system access time in the IEEE’s beyond-CMOS roadmap, and Izraelevitz and colleagues measured 81ns for a random load on an Intel server. Working Memory, Part 2 covers the cell and the bank.

How wide the path to DRAM is sets its bandwidth. AMD’s Venice server processor has 16 memory channels and, with MRDIMMs at 12,800MT/s, reaches about 1.6TB/s per socket, by StorageReview’s account. Sun and colleagues give a plain DDR5-4800 channel as 38.4GB/s, and that figure comes from a different platform.

High Bandwidth MemoryHigh Bandwidth Memory stacks DRAM dies on a base die and links them with thousands of vertical wires, giving a very wide, short path to a processor on the same package. uses the same kind of cell and changes the path. Stacks of DRAM dies sit beside the processor on one package, each with a very wide interface, and Photolithography Part 5 shows how that package is built. Samsung’s ISSCC 2026 HBM4 paper describes a 36GB stack at 3.3TB/s, and Samsung’s HBM4 announcement gives it 2,048 I/Os. NVIDIA’s Rubin GPU carries up to 288GB of HBM4, at 19.2TB/s in NVIDIA’s spec table.

DRAM and HBM: main memory, and memory built for bandwidth

HBM buys bandwidth, not speed

How many times better each memory is, on a log scale

Against one DDR5 socket, one Rubin GPU’s HBM moves 12 times as much data per second and holds up to 288GB per GPU, yet it is no faster to reach. Jia and colleagues measured an HBM2 access on a V100 at 375 GPU cycles when it was an L2 miss with a translation lookaside buffer (TLB)The translation lookaside buffer is a small, fast cache of recent translations from virtual to physical memory addresses, so the processor does not have to work each one out again. hit, about 272ns at the card’s 1,380MHz maximum graphics clock and about 3.4 times a CPU’s DRAM load. The two numbers come from a GPU and a CPU of different generations and never shared one system, but the order is clear. HBM is the memory for work that streams huge amounts of data, and its gain comes in bandwidth inside a fixed power budget and nothing in latency. Working Memory, Part 4 explains why the path is so wide.

Filling the gap between DRAM and flash

Between DRAM and NAND flash lies the widest gap in the hierarchy, at 81ns against 52µs, about 640 times. Two kinds of memory have tried to fill it. One adds more DRAM further away. Memory of the other kind uses a different cell that keeps its data with the power off.

Compute Express Link (CXLCXL runs over the same physical layer as PCIe but adds cache-coherent memory access, so it moves cache lines rather than blocks of storage.) attaches memory over the PCIe physical layer, with load and store access instead of block reads. Per the IEEE’s 2026 roadmap, CXL 1.0 to 2.0 sits on PCIe 5.0, at up to 64GB/s each way on a x16 link, and CXL 3.0 on PCIe 6.0, which doubles the lane rate to 64GT/s. Distance is the cost, and Microsoft’s Pond paper estimates that CXL adds 70 to 90ns over same-NUMA-node DRAM for a pool shared by 8 to 16 sockets, and more than 180ns for a pool shared across a rack. Sun and colleagues measured three real CXL memory devices and found load latency 35% to three times longer than DDR5 on the other socket, depending on the CXL controller.

Intel’s Optane persistent memory took the other route: a non-volatile memory on a module with the same physical and electrical interface as DDR4. Izraelevitz and colleagues measured it in 2019: 305ns for a random load against 81ns for DRAM on the same server, and 169ns for sequential loads. One module read at up to 6.6GB/s and wrote at 2.3GB/s, and modules came in 128GB, 256GB and 512GB.

Filling the gap between DRAM and flash

The memories in the gap sit next to DRAM, not halfway to flash

Access latency and bandwidth per device, log scales

Both sit close to DRAM. Adding Pond’s estimate, made on a different platform, to the 81ns DRAM load puts CXL memory at about 151 to 171ns, and Optane sits under four times DRAM’s latency. Next to Optane, the NAND SSD is still 170 times further out. On bandwidth the order changes, and a CXL x16 link moves more than one DDR5 channel, while one Optane module reads at about the speed of the XG8 SSD.

Intel’s 10-Q for the quarter ended July 2, 2022 says it began winding down its Optane memory business that quarter, so Optane did not last. CXL is the one still growing, and it is mostly more DRAM reached over a different link. Beyond NAND, Part 2 covers phase-change memory and Optane, and Flash Foundations Part 1 covers CXL on the same bays as SSDs.

Flash, disk and tape: capacity priced by the gigabyte

All three bottom tiers keep data with the power off, and they are where most bytes live. They answer in microseconds, milliseconds and tens of seconds.

A NAND SSD reads a page of cells through a controller and a PCIe link. In CrystalDiskMark 8.0.4 on my test system, a Kioxia XG8 2TB answered a random 4KiB read at a queue depth (QDIOPS counts commands finished per second. Queue depth is how many commands are outstanding at once, and a drive needs a deeper queue to reach its top IOPS.) of 1 in a median of 51.95µs over four runs and read sequentially at a median of 7.1GB/s over two runs. As of October 2026, the largest SSDs reach 245.76TB, in Kioxia’s LC9. A hard drive has to move its head and wait for the platter to turn. A WD Red Pro 20TB on the same system took a median of 4,215µs over four runs, about 81 times the SSD, and read at a median of 270MB/s over two runs. Seagate was sampling 36TB drives in 2025, and the IEEE’s 2026 roadmap reports a 40TB drive unveiled by Western Digital. By 2026 Seagate was delivering Mozaic 4+ drives of up to 44TB in volume to two hyperscale cloud providers, and Part 2 explains how those drives record.

Tape is the slowest to reach, and once a cartridge is loaded and positioned, it streams faster than a hard drive. Per the IEEE roadmap, Linear Tape-Open (LTO-10) cartridges are 30TB or 40TB native, drives stream at up to 400MB/s native, and access latencies run in the range of tens of seconds, because a robot has to load the cartridge and the drive has to wind to the data. Appuswamy and colleagues list a 65s access time for a small 2017 library.

Flash, disk and tape: capacity priced by the gigabyte

DRAM cost 200 times as much as a hard drive per gigabyte

US dollars per GB, log scale; one dated price set per strip

On one set of 2017 prices, from Appuswamy, Borovica-Gajic, Graefe and Ailamaki, DRAM cost 5.00 USD per gigabyte and a 2TB hard drive 0.025 USD, about 200 times less. NAND SSDs sat in between at 0.63 to 0.70 USD, and an Optane SSD at 3.96 USD, close to DRAM. A small 2017 tape library with four drives and 10 cartridges came to 0.073 USD per gigabyte compressed and 0.147 native, more than the hard drive either way, because four drives and a robot were spread over only 10 cartridges. Past a small library, the IEEE’s 2026 roadmap puts tape media under 5 USD per terabyte, against over 10 USD for hard drive media, and notes that a library spreads one drive over 50 to 100 cartridges.

Prices move. As of October 2026, DRAMeXchange’s spot average for a 16Gb DDR5 chip was 59.43 USD on October 12, about 29.7 USD per gigabyte, and a 16GB DDR5 module averaged 239 USD on October 5, about 14.9 USD per gigabyte. Those are 2026 shortage prices, three to six times the 2017 figure. In that roadmap, SSD and hard drive cost per gigabyte are not expected to cross, because hard drive areal densityAreal density is how much data a hard drive platter stores in a given area of its surface. Raising it is how makers fit more data on each platter. keeps rising too.

What the latency chart leaves out

Latency and capacity are only two of the axes. Each memory also has a bandwidth and an energy cost for every bit it moves, and those do not follow the same order. HBM is DRAM, but one GPU’s stacks move more than ten times the data of a server socket’s DDR5. A hard drive is slow to reach but still holds a byte for less than any chip.

What the latency chart leaves out

Each memory wins on a different axis

Example figures for each memory, log scales

Read the figure one strip at a time. On bandwidth, HBM leads and the hard drive trails by almost five orders of magnitude. On capacity the order flips, and the largest SSDs hold more than a million times what the cache holds. Cost per gigabyte (Figure 5) follows capacity and ignores speed.

Moving a bit costs energy, and the cost depends on how far the bit travels. O’Connor and colleagues at NVIDIA put GDDR5 at about 14.0pJ per bit and HBM2 at about 3.97pJ per bit, mostly data movement from the row buffer to the I/O pins plus activation energy. Sun and colleagues give PCIe 4.0 at about 6pJ per bit against 22pJ for DDR4, at the price of about 40ns of link latency against under 1ns. A serial link spends less energy per bit and more time, which is the trade CXL makes.

An SSD pays far more per bit, because the NAND array, the controller and the link all draw power. On my test system, the Kioxia XG8 took 36.6s at an average 3.99W in a 50GB copy test from and to the same drive, about 146J, or roughly 340pJ for every bit copied, each bit read once and written once. That figure is whole-drive power in a whole-drive test, the 1.6W idle draw included, so it is not an access energy and not a like-for-like match for HBM2’s. Still, 340pJ is about 86 times HBM2’s 3.97pJ and about 57 times PCIe 4.0’s 6pJ, and it buys data that stays put with the power off.

The five-minute rule and where data lives

If DRAM costs hundreds of times as much as a hard drive per gigabyte, data should sit in DRAM only when it is read often enough to pay for the space. Jim Gray and Franco Putzolu turned that into a rule in 1987. Their five-minute rule compares the cost of holding a page in memory against the cost of the disk reads it would take to fetch that page again.

It gives a break-even interval. A page read again within the interval is cheaper to keep in the faster memory. Appuswamy’s team writes it as two ratios multiplied together. One ratio is the pages per megabyte of DRAM divided by the reads per second one disk can serve, and the other is the price of a disk divided by the price of a megabyte of DRAM. Hardware sets the first ratio, and the market sets the second. With Tandem prices in 1987 the interval came to 400s, which Gray and Putzolu rounded down to five minutes.

The five-minute rule and where data lives

Disks pushed the DRAM interval up, and flash pulled it down

Break-even interval for caching a page one tier up, by year of the prices used

Goetz Graefe redid the sum with 2007 prices and 4KB pages. A megabyte of RAM held 256 pages and cost about 0.047 USD. A hard disk served 83 reads per second for 80 USD, so the interval was 256 / 83 × 80 / 0.047, about 5,248s or 90 minutes. A flash disk served 6,200 reads per second for 999 USD, and the same sum gave 876s, about 15 minutes.

With 2017 prices the DRAM to hard drive interval had grown to four hours, because DRAM got much cheaper per megabyte while disk reads per second improved only from 83 to 200 a second in a decade. From DRAM to SSD, the interval went the other way, at 7 minutes for a SATA SSD and 41s for a PCIe NAND SSD, because those drives serve so many reads per second. The interval between an SSD and a hard drive grew to a full day. Tape sits beyond all of them, and Appuswamy’s DRAM to tape interval is over 300 years for a 4KB page, so tape holds data that is scanned in bulk and never read a page at a time.

On 2017 prices, those intervals draw the modern hierarchy. For 4KB pages, data read more often than about once a minute earns a place in DRAM in front of a fast SSD. Data read less often than that, but more than once a day, belongs on flash. Data read less than once a day is cheapest on a hard drive, and data kept for years and read in bulk belongs on tape. Dearer DRAM in the 2026 shortage shortens the DRAM intervals, because the formula divides by the price of DRAM. I haven’t recomputed the rule with 2026 prices, because that needs a drive’s rated random readsRandom I/O reads or writes small requests at scattered addresses across a drive, so each request lands somewhere new. and its price from the same date. Part 7 of the series works the sum through for flash that answers in microseconds, taking one input at a time.

How data moves toward the processor

A program that reads a file does not reach into the SSD. Data moves up one tier at a time, and each tier keeps a copy. Along the way, the unit of transfer changes: blocks and pages from the drive, 4KB pages in memory, 64-byte lines in the caches, and single registers in the core.

First the application asks the operating system for the file. If the data is not already in memory, the NVMe driver places a read command in a submission queueNVMe uses pairs of queues in host memory: the host places commands in a submission queue and the drive reports results in a completion queue. in host DRAM and rings the drive’s doorbell. Next, the drive reads the flash and copies the data into host DRAM by direct memory accessDirect memory access lets a device such as an SSD read or write the host’s memory on its own, without the processor copying each byte. (DMA) over PCIe, without the CPU moving each byte. Flash Foundations Part 1 covers the NVMe queues and the link.

How data moves toward the processor

A file read stops in host DRAM, and a GPU read can skip it

The same read through the kernel, and through GPUDirect Storage

On Linux the data goes into the page cache, the part of DRAM where the kernel keeps file data. While it stays cached, the next read does not touch the drive. Writes go into the page cache too and reach the drive later. When programs need the memory the kernel reclaims page cache, so the cache grows into whatever DRAM is free. The CPU pulls the 64-byte lines it needs from DRAM into L2 and L1, and the core loads them into registers. On Zen 5 the L3 fills with lines the L2 evicts, so a line reaches it only on the way out.

Some paths skip a tier. NVIDIA’s GPUDirect StorageGPUDirect Storage is NVIDIA’s path that moves data straight between an NVMe drive and GPU memory, skipping a copy through the CPU’s memory. sets up a direct DMA path between the drive and GPU memory, so data for a GPU no longer stops in a bounce buffer on the CPU side. A drive without its own DRAM can borrow a slice of host memory for its mapping table through the host memory bufferHost memory buffer lets a DRAMless NVMe drive borrow a small region of host RAM to cache part of its map. covered in Flash Foundations Part 6. CXL memory takes loads and stores like local DRAM, moving a cache line at a time, with no block read at all. Every copy on the way up is a cache. A read that hits a higher tier never reaches a lower one, and that is why a slow memory at the bottom can sit under a fast system.

Tiers inside an SSD: cache, fold, then TLC

An SSD is a small hierarchy of its own. Its controller holds the mapping table in DRAM, or in the host’s memory on a drive without DRAM. Below that, a triple-level cell (TLC) drive keeps part of its flash in single-level cellA cell storing n bits needs 2 to the power n voltage levels: two for SLC, four for MLC, eight for TLC and sixteen for QLC. (SLC) mode with one bit per cell instead of three, as a fast landing zone for writes. An SLC cell needs only two charge levels, so it programs in a few coarse steps. A TLC cell needs eight, each placed with many small program and verify steps. Flash Foundations Part 3 shows the timing numbers, and Part 9 covers the cache.

Micron’s Dynamic Write Acceleration white paper describes how such a cache behaves on a long write. On a freshly erased 128GB M600, one sequential fill ran in three regions: fast while writes landed in SLC, slower while the drive wrote directly to the dense mode, and slowest while it converted SLC data into the dense mode with the host still writing. Flash Foundations Part 6 covers that folding step.

Tiers inside an SSD: cache, fold, then TLC

A long write steps through three speeds on an empty drive

Kioxia XG8 2TB, 1MiB sequential writes for 75 minutes, averaged per 20 gigabytes written

With Iometer I wrote 1MiB blocks to an empty Kioxia XG8 2TB without stopping for 4,529s, about 75 minutes, in one run, and logged the speed every second. For the first 27s it wrote at a median of 6,035MB/s, about 163GB into the SLC cacheAn SLC cache writes incoming data first in a fast one-bit-per-cell mode, then folds it into denser TLC or QLC cells when the drive has time.. Then it fell to a median of 1,738MB/s until about 1,700GB had been written, and after that rose to a median of 2,204MB/s to the end, 9,622GB in total. Micron’s third region matches the middle step. My reading is that the drive keeps borrowing free blocks as SLC and folding them into TLC while the host writes, until the free blocks run out at about 1,700GB, short of the drive’s 2,048GB. After that, writes go straight to TLC and the speed rises. I have no Kioxia document on the XG8’s cache policy, so this is my reading of the log and makes no claim about the firmware.

Even the steady phase isn’t perfectly flat. Once the writes wrap and start overwriting the speed dips about 7% for 23s every 275GB written, a pattern that resets at each wrap and so points at housekeeping the drive schedules by data written. The run’s saved log has no temperature readings. The XG8 log shows the three steps of SLC, folding and TLC, and Micron’s M600 note describes them on another drive, which is why one SSD can show three different write speeds in one test.

Why SI stretches the hierarchy

SI models made the gap between compute and memory bandwidthMemory bandwidth is how fast a processor can read and write its memory, in bytes per second. For SI inference it often sets the speed more than raw compute does. hard to ignore. Gholami and colleagues measured it over 20 years. Peak server compute grew about 60,000 times, or 3.0 times every two years. DRAM bandwidth grew about 100 times, 1.6 times every two years, and interconnect bandwidth about 30 times, 1.4 times every two years. They call the result the memory wall: a processor that can do more math than its memory can feed.

Why SI stretches the hierarchy

Compute grew far faster than the memory that feeds it

Growth from a common start, at the rates per two years measured over 20 years

From one starting point the curves split within a few years. After 20 years, compute has grown about 600 times as much as DRAM bandwidth, so every byte the memory delivers has to feed far more math. Across the industry, the answer is the memories this guide has walked through: HBM beside the processor for bandwidth, CXL for more DRAM behind it, and SSDs for everything that does not fit. Memory and Storage for SI, Part 2 follows flash moving up beside the GPU as High Bandwidth Flash, and The Memory Cycle shows what that demand does to memory prices.

As prices and parts change the memories keep their order, but the break-even lines between them move. Memory and Storage for SI, Part 8 places each memory family by the job it does, The memory map is the hierarchy with a first look at each family, and Flash Foundations follows flash from the cell to the drive.