The Memory Map: What Comes After Flash

NAND flash won the storage market on cost per bit, and every memory since has had to answer that. Flash Foundations teaches how NAND itself is built, and The Memory and Storage Hierarchy scores each memory on five axes. The map below asks a narrower question. Where does NAND sit between DRAM and the hard drive, what did Optane prove and why did it end, and what could phase-change, magnetic, resistive and ferroelectric memory add? Each family gets a first look here and a full treatment in Working Memory or Beyond NAND.

Two drives that never slow down

Two of the drives in my test results never slow down. Intel’s Optane P5800X wrote about 6.1GB/s for the whole 15-minute run, and the older Optane 905P held about 2.3GB/s. A Samsung 980 started at 3.0GB/s, then fell to between 0.4GB/s and 0.5GB/s after its first minute or so. All three ran the same kind of 15-minute Iometer sequential writeSequential I/O reads or writes one address after the next, usually in large requests. test, and I describe it in how I made SSD validation repeatable.

Small reads show the same gap

On small reads the same drives look just as different. In CrystalDiskMark, with one request in flight at a time, the two Optane drives answered a 4KiB random read in about a fifth of the time the NAND drive needed, taking the median of four runs.

Why this guide exists

That gap is the reason most of the memories in Beyond NAND exist. Those memories are attempts to get speed without giving up too much of what makes NAND flashNAND flash is the memory that keeps data with the power off. Writing it is much slower than writing DRAM, and it must be erased before it is rewritten. cheap. Optane was the attempt that reached customers, and it ended anyway.

This guide is the map. It places NAND in the memory hierarchy, follows the Optane story, and gives the non-volatile families a first look, with three series taking every family in depth.

Working Memory covers the memory a processor works from. Its parts are SRAM and the processor cache, the DRAM cell from the inside, the cells that could follow DRAM and High Bandwidth Memory. That last part puts a DRAM stack beside the processor. Beyond NAND covers the non-volatile memory beside flash. Its parts are NOR and memory on the processor die, phase-change memory and Optane, MRAM, ferroelectric memory, and resistive memory and computing in the crossbar. Memory and Storage for SI starts with memories used as weights and High Bandwidth Flash, which moves flash beside the GPU, continues with the 512-byte read, and ends with where memory goes next, which places every family by the job it could do.

Where NAND sits in the hierarchy

Every computer has a hierarchy of memories. Fast ones are small and expensive, and big ones are slow and cheap. NAND sits between DRAM and the hard drive. Cache has a part of its own, Working Memory, Part 1, so this guide touches the cache only in passing. DRAM’s cell is in Working Memory, Part 2, and Working Memory, Part 3 asks what could follow it.

The gap storage-class memory targets

Because the rows measure different things, treat the table as a scale and not a ranking of products. The spacing is what counts. Between DRAM at about 16ns and NAND at 25µs there is a gap of more than three orders of magnitude. Burr and colleagues described a similar gap in their 2010 review of phase-change memory. They put it at roughly 100ns to 1ms of access time that nothing filled. As a term, storage-class memoryStorage class memory keeps data with the power off like flash but answers far faster, aimed at the performance gap between DRAM and ordinary flash. comes from Freitas and Wilcke at IBM in 2008. Burr’s review described two flavors of it. An S-class memory behaves like an SSD and accepts 1µs to 3µs access. An M-class memory sits on the memory bus and needs 300ns or less.

IEEE’s roadmap puts numbers on the same two flavors. Its 2020 Beyond CMOS chapter lists target specifications for S-class and M-class storage-class memory and widens the S-class latency window to 5µs. Its 2022 edition puts off-chip DRAM, measured as a system access rather than a die timing, at about 80ns and NAND flash at 20µs.

These numbers are roadmap targets, and product specifications are a separate matter. In the roadmap itself, the M-class endurance row carries a caveat. Per the 2020 chapter, some studies show 10⁷ cycles is enough for a lifetime of 3 to 10 years, and the roadmap asks for 10⁹ to leave engineering margin. For M-class retention it says perhaps one to three weeks of data survival would be enough to recover server data after a power outage.

Every memory in the rest of this guide is a candidate for that gap, or for the memories on either side of it. The Memory and Storage Hierarchy scores each of these memories on five axes and follows a read from the drive to the core.

NOR and NAND are cousins with different jobs

NOR and NAND are both flash, and they split the work. NOR can read any byte directly, which lets a processor run code straight from the chip, and NAND gives that up for capacity at a lower cost per bit. Beyond NAND, Part 1 traces the two at the pins, in reads, writes and wear, and Flash Foundations Part 3 teaches why a NOR array pays for a contact per cell.

Why NAND is cheap

NAND went vertical to keep scaling, and the IRDS 2022 report explains why that worked. Stacking complete device layers one at a time stops saving money after a few layers. Building the NAND stringA NAND string is a column of cells connected in series between a bitline and the source line, with a select transistor at each end. vertically makes many layers in few steps, and the report says the cost benefit of that approach does not saturate until quite high layer countsThe layer count is the number of cell layers in a vertical stack. More layers means more bits per unit of die area.. When it was written, 176 layers were in volume production and 4 bits per cellA cell storing n bits needs 2 to the power n voltage levels: two for SLC, four for MLC, eight for TLC and sixteen for QLC. (QLC) was shipping. More bits per cell slow programming and reading and cost reliability, which is acceptable for read-heavy uses where cost matters most.

Layer counts kept climbing. Micron announced volume production of 232-layer TLC NAND in July 2022, at 14.6Gb/mm².

No dollar-per-gigabyte figure traces to a primary source, so none appears here. Structure explains it well enough. NAND has one transistor per cell, strings that share contacts, stacks that are built in bulk and several bits per cell. Any memory that wants to replace it has to beat that on cost per bit, or find a job where cost per bit matters less than speed.

Cell area in units of F²

This argument can be made in silicon area instead of dollars. A cell’s footprint is some multiple of F², where F is the smallest feature the process can draw. IRDS 2022 says 6F² is the most common DRAM cell and 4F² is the practical limit of the 1T-1C cell. In its Beyond CMOS chapter, a storage element with a planar select transistor takes 6F² to 8F², and the densest two-dimensional layout, 4F², needs a vertical transistor or a two-terminal selector in a crossbar. A NAND cell is 4F² as well. Burr’s review made the consequence plain in 2010: even a phase-change cell at 4F² holding one bit would fall short of NAND, because multi-level NAND was already twice as dense per bit.

Stacking is the lever. In the ideal case, each crosspoint deck adds another independent layer of 4F² cells, so n decks give 4F²/n per bit. TechInsights counted two decks in Optane’s first generation and four in its second. This plot puts those ideal numbers beside the cells that exist.

Four decks of ideal crosspoint cells would only tie a 4F² NAND cell holding four bits. Burr’s review saw NAND moving toward that density. This plot leaves out NAND’s move to vertical strings, and that move is the gap a replacement has to close.

Optane: the attempt that shipped

Intel and Micron’s 3D XPoint3D XPoint is the memory in Intel’s Optane products. Each cell stacks a phase change memory element on an ovonic threshold switch selector., sold as Optane, was the attempt at a memory between DRAM and NAND that reached customers, and it ended. Beyond NAND, Part 2 holds the whole account. It covers what the companies announced, what a teardown found in the cell, the products and how they measured. This section keeps a preview of three parts of it.

Products and memory modes

Optane came as data-center drives and then as modules that fit DDR4 slots and ran as large memory or as persistent memory. Beyond NAND, Part 2 lists the products and the two modes.

The measured gap

That gap held up, as the two drives at the top of this guide show. Beyond NAND, Part 2 sets the ratings, the drive runs and the module measurements side by side.

Why it ended

Micron stopped developing 3D XPoint in 2021, and Intel began winding down its Optane business in 2022. Beyond NAND, Part 2 gives both statements and what they pointed at. My reading of the record is a memory that was faster and tougher than NAND, in a market that didn’t pay the difference at the volumes needed to keep the fab running. That fits the last section’s reading. Speed alone did not beat cost per bit.

Phase-change memory

PCMPhase-change memory stores data in a material that switches between a crystalline, low-resistance state and an amorphous, high-resistance state when heated. stores a bit in the structure of a chalcogenide, a glassy alloy that is high resistance when amorphous and low resistance when crystalline. A current pulse melts it or anneals it. Beyond NAND, Part 2 opens the cell, and its section on heat, drift and wear gives the endurance, the drift and the retention arithmetic.

PCM that shipped

PCM reached products as a parallel part from Micron and then in a phone memory package, and ST still builds it into car microcontrollers. Beyond NAND, Part 2 says who built it and what was sold.

MRAM

A magnetic tunnel junction has two magnetic layers separated by a thin magnesium-oxide barrier. One layer is fixed, and the other is free to flip. When the two point the same way, the junction has low resistance. When they oppose, it has high resistance. That difference is called tunnel magnetoresistance, and it is the bit. Beyond NAND, Part 3 opens the junction up.

Two main write methods differ in where the current flows. In spin-transfer torque (STT) MRAM the write current goes through the junction, the same path that reads it. In spin-orbit torque (SOT) MRAM the write current runs along a channel beside the junction, so reading and writing use separate paths. Less stress on the barrier is the argument for SOT, and the cost is a bigger cell and more write current. Beyond NAND, Part 3 works through both.

Products make different trades. Everspin’s EMD4E001G is a 1Gb STT-MRAM with a DDR4 interface. Its endurance is 10¹⁰ cycles per page, its retention only three months at 70°C, and its activate-to-read delay 135ns where ordinary DDR4 needs 12.5ns. It works as a persistent buffer. Samsung began commercial shipment of embedded MRAM on 28nm FD-SOI in March 2019 and said it writes about 1,000 times faster than embedded flash. That figure is a vendor claim. Borders and colleagues built probabilistic bits from spintronic devices and factored integers up to 945 with eight of them in a 2019 Nature paper. That was a proof of concept and not a memory product. Beyond NAND, Part 3 covers what is sold.

Resistive memory and the crossbar

How the cell switches

A resistive memory cell is two electrodes around a switching layer. High resistance reads as 0 and low resistance reads as 1. Going from high to low is a SET, and the reverse is a RESET. Because the cell is just a stack between two wires, it fits at the crossings of a grid of wires, and that grid is a crossbar.

In most of these cells the switching is a filament, grown once by a high-voltage forming step and then broken and rebuilt by each RESET and SET. Interface-type cells instead change resistance gradually through many levels. A quick test tells them apart. When the current is carried by a single filament it does not depend on the cell’s area, while in non-filamentary cells such as the tunnel-oxide type the set, reset and read currents scale with area, according to the roadmap. Beyond NAND, Part 5 covers forming, the two kinds of filament and why two identical cells disagree.

This idea is older than either name. Hickmott reported negative resistance in metal-oxide-metal sandwiches in 1962, with peak voltages between 1.7V and 3.1V. Chua proposed the memristor as a missing circuit element in 1971, and Strukov and colleagues reported a device matching it in 2008.

The hard trade is a dilemma. Schroeder and colleagues argued in 2010 that a purely electronic switching mechanism cannot give 10-year retention and 100ns read and write pulses at once at densities and voltages competitive with flash, below 1V in cells under 100nm by 100nm. IEEE’s roadmap calls the general problem the voltage-time dilemma and treats long retention, low read voltage and fast switching at moderate write voltage as goals that fight each other.

Resistive memory as a chip

Resistive memory has been built at memory-chip scale as well as in single cells. A 16Gb part in 27nm technology was reported at ISSCC 2014 with 200MB/s writes and 1GB/s reads, and a year earlier a 32Gb two-layer device in 24nm with an area of 130.7mm². Smaller macros went faster. A 4Mb embedded macro reached a 7.2ns random read-write access time in 2011. None of these papers is evidence of a product that competed with NAND. IEEE’s roadmap says why it is hard. The switching currents of the most mature oxides are still too high for large arrays, and filament formation and rupture are stochastic, which smears the resistance levels and hurts multi-level cells.

The sneak-current problem

A crossbar has one serious problem. Current can sneak through neighboring cells in a passive grid and corrupt a read. Real arrays add a selector at each crossing or a transistor per cell, or they use cells built to rectify on their own.

The selector has a tight leakage budget, which Beyond NAND, Part 5 works through along with the kinds of selector. Crosspoint Lab shows how sneak currents arise. Pi and colleagues demonstrated crossbar arrays with a 2nm feature size and 6nm half-pitch, at a density of 4.5 terabits per square inch, in Nature Nanotechnology in 2019.

Computing inside the crossbar

A crossbar can also compute. Set each cell’s conductance to a weight, apply voltages along the rows, and Ohm’s law multiplies each voltage by its conductance. Kirchhoff’s current law then sums the currents down each column. One read gives a full vector-matrix multiply. This idea underlies compute in memory, though only as an ideal-limit statement. Wire resistance and sneak currents erode it in real arrays. You can see the mechanism, and where it breaks down, in the Crosspoint Lab. Beyond NAND, Part 5 works one example through and covers what has been built. Memory and Storage for SI, Part 1 takes the cell’s side of it. It asks what a memory has to do to act as an adjustable weight, and how well the ones tried so far manage.

Randomness that makes these cells hard to use as memory has uses of its own: random numbers, chip fingerprints and computations that need noise. Beyond NAND, Part 5 covers them.

Other candidates: ferroelectric memory

A ferroelectric FET stores its bit in the polarization of a ferroelectric layer in the gate, which shifts the threshold voltage of the transistor and stays put without power. Doped hafnium oxide has been shown as the ferroelectric in standard foundry processes. Böscke and colleagues showed it working as the ferroelectric gate insulator of a FET. In 2016, Trentzsch and colleagues reported a JTAG-controlled 64kbit memory on a 28nm process, and in 2017 Dünkel and colleagues wrote patterns into 32Mbit arrays on a 22nm process. In a FeFET the memory window is set mainly by the coercive voltage of the ferroelectric rather than its remnant polarization, according to the IRDS roadmap. Because it stores a bit in a transistor’s threshold, a FeFET can serve as a logic gate. Breyer and colleagues showed a reconfigurable NAND/NOR gate concept on a single FeFET in 28nm and 22nm simulations.

Ferroelectric memory is not all research. Ferroelectric RAM stores charge on a ferroelectric capacitor, has a destructive read that forces it to rewrite the bit, and is sold in small parts. Fujitsu rates its parts for 10¹⁰ to 10¹⁴ cycles, according to a 2024 industry survey, and Texas Instruments rates the FRAM in its MSP430FR5994 microcontroller for 10¹⁵. Lead-zirconate-titanate ceramic in those capacitors stopped shrinking at the 130nm generation, according to Mikolajick and colleagues, which is part of why doped hafnium oxide matters.

As a drive medium it is still research: the 3D ferroelectric NAND strings built so far are papers, not products. Beyond NAND, Part 4 opens the film, then the capacitor, the transistor and what is sold.

Memory on the processor die

A microcontroller keeps its firmware on the same piece of silicon as the processor, in a block of embedded nonvolatile memory. Beyond NAND, Part 1 tells how that memory began, why flash stopped fitting and what a code memory has to survive. The memories it turned to are in Part 2, Part 3 and Part 5.

Where embedded memory came from

Beyond NAND, Part 1 puts EPROM, EEPROM and NOR on one line, with the dates the sources give.

Why embedded flash stopped fitting

Embedded flash is added to a logic process with extra steps, and near 28nm the sources say it stopped scaling. Beyond NAND, Part 1 covers what they said.

Phase-change memory in a car

An engine bay is hot, and phase-change memory has a heat problem built in: the amorphous state slowly crystallizes, and the hotter the part, the faster it goes. ST rates its embedded PCM for automotive temperatures. Beyond NAND, Part 2 gives the ratings and what ST announced.

MRAM on the die

MRAM went onto logic processes too, and Beyond NAND, Part 3 covers the embedded parts.

Resistive memory on the die

Resistive memory was built into logic chips as well. The products and the qualified processes are in Part 5 of that series, along with the trade-off between write current and retention.

The soldering test

Any of these memories that holds firmware has to come through the solder oven with its code intact. Beyond NAND, Part 1 gives the roadmap and standards numbers. For a memory sitting beside the logic, surviving the solder oven looks to me as important as speed.

What NAND’s cost per bit makes possible

Cost per bit decides

Put the candidates side by side. Optane, PCM and MRAM each beat NAND on speed or endurance, and in my reading none of them came near NAND’s cost per bit. NAND keeps that edge with vertical strings, shared contacts and several bits per cell. Memories that last tend to find a job where cost per bit matters less. MRAM finds one as an embedded flash and SRAMSRAM is fast memory of latching cells with several transistors each. It needs no refresh but takes far more area per bit than DRAM, so it serves as on-chip memory. replacement and as a persistent buffer. PCM finds one in phone memory packages and small parallel parts, with car microcontrollers as the embedded target its papers aim at, covered in the section on embedded memory. When Micron and Intel stepped back from Optane, both pointed to CXLCXL runs over the same physical layer as PCIe but adds cache-coherent memory access, so it moves cache lines rather than blocks of storage..

Why none of them replaced DRAM

DRAM is the other end of the gap, and the evidence in this guide says nothing has matched it. Optane modules loaded random data in 305ns against 81ns for DRAM, in measurements by Izraelevitz and colleagues. PCM arrays have reported 10⁸ to 10¹⁰ cycles, which Burr’s review calls short of what DRAM replacement would need without wear levelingWear leveling spreads erases evenly over all blocks. Dynamic leveling picks young blocks for new writes, while static leveling also moves cold data off them.. For STT-MRAM, the IEEE roadmap calls DRAM-density versions an open research area. Ferroelectric RAM rewrites a bit on every read, which spends endurance, and its ceramic capacitors stopped shrinking at the 130nm generation. Resistive memory has been held back by switching currents too high for large arrays. Each memory beat DRAM on persistence and lost on something else. So the memories in this guide fit the gap between DRAM and NAND, or the chip next to a processor, better than they fit either end. Working Memory, Part 2 covers the DRAM cell from the inside, and Working Memory, Part 3 asks what could follow it.

Where that leaves NAND

That leaves NAND doing the job it does best: holding a very large amount of data at a price people will pay. Cache and DRAM keep hot data close, and NAND holds the rest. For the layer underneath, see how NAND is built from cell to die, and for how drives behave when the fast write area runs out, see how I made SSD validation repeatable. For NAND moved up beside the GPU as a capacity tier, see High Bandwidth Flash, Part 2 of the SI series. The last part of that series comes back to this hierarchy and places every family by the job the evidence shows it filling, in Part 8.