Eyes on Flash/ Driven by Insight
Contact

Search Eyes on Flash

Tip: press / or Ctrl+K on any page to open search.

Type to search.

    Technical guide

    Memory and Storage for SI

    The memory and storage being built for SI, from a memory cell used as a neural-network weight to the SSDs designed for inference.

    A series on the memory and storage SI is changing, from the device out to the system. Part 1 asks whether a memory cell can be a neural-network weight. Part 2 sets stacked NAND beside an accelerator as high bandwidth flash, and Part 3 follows Samsung’s zNAND-O concept, flash inside the processor’s own package. Part 4 covers NVIDIA’s Storage-Next, why inference reads storage 512 bytes at a time, and the drives made for it. Parts 5 to 7 move outward through the KV cache, the capacity drives and the five-minute rule, and the last part places every memory family by the job it could do. Every number is attributed to the maker that rated it or the outlet that reported it. The memory map and Working Memory cover the memories this series builds on.

    The series

    Read in order

    Each part builds on the one before, so start with Part 1.

    Memory and Storage for SI, Part 1: a weight as two conductances, a staircase of update steps and a spread around a targetA three-panel schematic. Left: two vertical bars of different heights with a copper bracket between their tops, a weight stored as the difference of two conductances. Center: a conductance staircase that rises in equal steps and falls in equal steps, with the pulses drawn as ticks beneath. Right: a bell-shaped spread of conductances around a dashed target line.GUIDE / MEMORY AND STORAGE FOR SI, PART 1A weight, a step, a spreadWEIGHT · UPDATE · SPREAD01 WEIGHTG+G−Two cells,one weight02 UPDATESmall steps,up and down03 SPREADtargetMisses land arounda target

    Part 1

    Memory as a Synapse

    How a memory cell becomes an adjustable weight, how that weight is scored, why updates miss, what noise and drift cost, and what has shipped.

    Read Part 1
    An HBM stack and an HBF stack beside a GPU on a silicon interposerA schematic package in side view. A GPU die sits in the middle of a silicon interposer on a package substrate. On its left stands an HBM stack of DRAM dies on a base die. On its right stands an HBF stack of NAND dies on a base die. Thin vertical lines through each stack stand for the through-silicon vias that connect the dies, and bands of copper wires through the interposer join each base die to the GPU.GUIDE / MEMORY AND STORAGE FOR SI, PART 2Flash beside the GPUHBM · GPU · HIGH BANDWIDTH FLASHPACKAGE SUBSTRATEGPUBASE DIEHBM · DRAM DIESBASE DIEHBF · NAND DIES

    Part 2

    High Bandwidth Flash

    What High Bandwidth Flash is, what the August 2026 OCP draft spec says, where its bandwidth comes from, and why writes and retention are its weak side.

    Read Part 2
    A processor die and NAND dies joined by a UCIe link inside one packageA schematic, not a Samsung drawing. A dashed package outline holds a processor die on the left and a stack of NAND dies on the right, joined by a short bundle of UCIe lines. The number of dies and how they are joined are illustrative; Samsung has not published them.GUIDE / MEMORY AND STORAGE FOR SI, PART 3Flash in the processor’s packagePROCESSOR · UCIe LINK · NAND · NOT TO SCALEONE PACKAGEPROCESSORUCIeNAND

    Part 3

    zNAND-O

    What Samsung has said about zNAND-O, NAND packaged with a processor over UCIe for on-device SI: what is claimed, what is simulated, and what is unpublished.

    Read Part 3
    Memory and Storage for SI, Part 4: a GPU reading an SSD through the CPU, then directly, and a 512-byte slice of a 4KB unitA three-panel schematic. Left: a read travels from an SSD board up to a CPU and across to a GPU along a dashed path. Center: the GPU reads the SSD along one solid arrow with no CPU in the way. Right: one 4KB unit drawn as eight equal slices, with the first slice, 512 bytes, picked out as the part wanted.GUIDE / MEMORY AND STORAGE FOR SI, PART 4Read 512 bytes, skip the CPUOLD PATH · DIRECT · ONE SLICE01 OLD PATHGPUSSD02 DIRECTGPUSSDCPUNo CPU hop03 ONE SLICE4KB unit512B wanted

    Part 4

    Storage-Next and the 512-Byte Read

    What NVIDIA’s Storage-Next initiative is, why SI inference reads storage 512 bytes at a time, and what Kioxia, Micron, and SK hynix have shown or planned.

    Read Part 4
    Memory and Storage for SI, Part 5: the KV cache stepping down from GPU memory through host memory and an SSD to shared storage, and returning to the GPUOne large staircase of four steps that descends from left to right: GPU, host, SSD and shared storage. The GPU and host steps are neutral; the SSD and shared steps are teal because they are flash. Small teal cache blocks sit on each step, and a copper arrow carries them down from one step to the next. A copper return path runs from the last step underneath the staircase and back up to the GPU.GUIDE / MEMORY AND STORAGE FOR SI, PART 5Context that outgrows the GPUGPU · HOST · SSD · SHAREDGPUHOSTSSDSHAREDblocks return to the GPU

    Part 5

    Context Memory

    How big a model’s KV cache gets, why inference keeps it, and the GPU, CPU memory, SSD, and shared storage tiers that hold it once HBM fills.

    Read Part 5
    Memory and Storage for SI, Part 6: a capacity SSD with a stack of QLC dies, a large unit with a small write landing in it, and a short row of SSD racks beside a longer row of hard drive racksA three-panel schematic. Left: a stack of thin QLC dies sits above an SSD board with flash packages and gold connector fingers. Center: one large unit drawn as four slots, with a small write landing in the first slot along a copper arrow. Right: a short row of SSD racks beside a longer row of hard drive racks, all the same size.GUIDE / MEMORY AND STORAGE FOR SI, PART 6Built for capacityDIES · UNITS · RACKS01 DIES02 UNITSunit rewritten03 RACKSSSD racksHard drive racks

    Part 6

    The Capacity Drives

    What 122TB to 300TB QLC drives do in an SI cluster, why bigger drives write in bigger pieces, and how to read their endurance and hard drive comparisons.

    Read Part 6
    Memory and Storage for SI, Part 7: a balance with a DRAM module on one pan and a flash SSD on the other, and a clock dial whose sweep shrinksA level balance beam on a central post. On the left pan sits a DRAM module, on the right pan a flash SSD. The pivot of the beam is a clock dial: a long dashed sweep stands for the old interval, and a short bright sweep inside it for the new one.GUIDE / MEMORY AND STORAGE FOR SI, PART 7Five minutes to secondsRENT · READ · BALANCE

    Part 7

    The Five-Minute Rule in Microseconds

    Gray and Putzolu’s five-minute rule redone for flash that answers in microseconds, 512-byte reads, and GPU hosts, with every input sourced or labeled.

    Read Part 7
    Memory and Storage for SI, Part 8: four jobs, four evidence lanes, and a stack of memories with a dashed gapA three-panel schematic. Left: four rows labeled die, memory, gap and capacity, with the memory row outlined. Center: four rows labeled order, qualify, ended and research, with bars sized by how many items each holds. Right: five stacked bars, the middle one dashed and highlighted, with no large resident.GUIDE / MEMORY AND STORAGE FOR SI, PART 8Jobs, evidence, and the gapJOBS · EVIDENCE · GAP01 JOBSDieMemoryGapCapacity02 EVIDENCEOrderQualifyEndedResearchResearch isthe long lane03 THE GAPNo large resident

    Part 8

    Where Memory Goes Next

    Places each memory family by the job it fills, sorts shipping from research, and says why DRAM is still unmatched.

    Read Part 8

    All 8 parts

    1. Part 1Memory as a SynapsePublished
    2. Part 2High Bandwidth FlashPublished
    3. Part 3zNAND-OPublished
    4. Part 4Storage-Next and the 512-Byte ReadPublished
    5. Part 5Context MemoryPublished
    6. Part 6The Capacity DrivesPublished
    7. Part 7The Five-Minute Rule in MicrosecondsPublished
    8. Part 8Where Memory Goes NextPublished

    On this site

    Where it connects

    The guides link these pages where they lean on them.

    Continue on this site

    Flash Foundations Working Memory Timelines