How Storage Works, Part 4: PCIe Gen6 and Gen7 for Storage

PCIe Gen6 doubles the lane rate to 64.0GT/s, but for storage the changes that matter sit behind that number. Every Gen6 link sends data in fixed 256-byte units called flitsA flit is the fixed-size unit a newer PCIe link sends in place of packets of varying size, so error correction can work on blocks of one size. Several packets can share a flit, and one packet can span several flits., protected by forward error correction (FEC)Forward error correction adds extra bits to data as it is sent, so the receiver can fix some bit errors itself without asking for the data again. and a cyclic redundancy check (CRC), and the same generation adds a link state that saves power while traffic keeps moving. Flash Foundations Part 1 gives the generation ladder, and Part 10 explains the PAM4 signal and the PCIe 7.0 release.

This part of How Storage Works takes up the storage layer those two leave out, from the flit to the power and heat limits a drive is built to and the first Gen6 drives. Where the SSD sits among the other tiers is the job of the memory and storage hierarchy, which opens the series.

Why Gen6 changed the signal

Through Gen5, PCIe sent one bit per symbol on two signal levels, called NRZ (non-return-to-zero), and Gen6 doubles the rate to 64.0GT/s with PAM4 (four-level pulse amplitude modulation), four levels and two bits per symbol, which leaves less room between levels for noise. Part 10 follows that signal in detail.

For a drive, the smaller margin shows up first as link error correction that earlier generations did without, which the next two sections cover. PCI-SIG’s design targets hold channel reach similar to PCIe 5.0 with at most two retimersA retimer is a chip partway along a high-speed link that recovers the signal and sends it on afresh, so the link can reach farther across a board or cable than it could alone., chips that re-drive the signal partway along the board, so the margin had to come from the link’s design and not from shorter traces. Part 10 gives the tighter channel-loss budget PCI-SIG set alongside that reach.

The flit and what it costs an x4 link

PCI-SIG settled on a 256-byte flit with 236 bytes of transaction layer packet (TLP) payload and a TLP efficiency of 92%. A TLP is the packet PCIe uses for a memory read, a memory write or a completion, which is how an SSD fetches commands and moves data. PCI-SIG’s HOT Interconnects deck splits the remaining bytes into 6 of data link layer payload, 8 of CRC and 6 of FEC. One TLP can span several flits, and one flit can hold several TLPs.

Framing sets what an x4 link delivers. A Gen6 x16 link reaches up to 256.0GB/s counting both directions, which works out to 32GB/s each way on an x4 link before framing. At 92% that leaves about 30GB/s, an estimate from those two figures, and Part 1 sets that limit beside drive speeds.

PCI-SIG also tested a 740-byte flit with 720 bytes of payload and about 97% efficiency. It dropped the larger flit because waiting for it to fill would have tripled the latency, adding 16ns on an x4 link.

Errors at a millionth: FEC, CRC and replay

PAM4’s smaller eyes raise the raw bit error rate (BER), and flit mode is built for the BER expected with PAM4, 10⁻⁶, against 10⁻¹² for NRZ in earlier generations. At 10⁻⁶ a link would see a raw bit error about every million bits, so correction has to be part of the link.

Each flit carries a CRC and a 3-way interleaved FEC. Receivers decode the FEC first, which corrects any correctable error within each FEC group, and then check the CRC. Any flit that still fails the CRC draws a NAK (negative acknowledgment) back to the transmitter, which replays it. Every flit has a dedicated slot to acknowledge or NAK the flit received.

PCI-SIG’s latency target for all of this is under 10ns added by the transmitter and receiver over 32.0GT/s, FEC included. Replay sits inside the link, below NVMe, so for a drive a retried flit should show up as added latency on that transfer and not as a failed command.

L0p, power that follows traffic

PCI-SIG’s deck lists L0s, L1, dynamic link width and speed change as the low-power states PCIe already had, and says demand has grown for power that scales with bandwidth use without interrupting traffic. L0pL0p is a PCIe low-power link state that turns off some of a link’s lanes while at least one keeps carrying traffic, so power falls with use without a pause in data., new in PCIe 6.0, keeps at least one lane active, and the link keeps carrying traffic while its width changes.

OCP’s Datacenter NVMe SSD Specification 2.7 makes it a drive requirement. Requirement PCI-42 reads, “For PCIe Gen6 devices, L0p shall be supported.” Gen6 drives built to that text can narrow their link while they keep serving reads.

The power and heat a Gen6 drive is held to

The same OCP specification sets power and heat limits for every drive built to it, whatever its PCIe generation; of the rules here, only PCI-42 is written for Gen6 alone. Max average power sets a drive’s burst allowance, and peak power is measured over a 100µs window. Drives at 25W or less may peak at no more than 1.5 times their average. From above 25W up to 29W the peak may be no more than 37.5W, which is 1.5 times 25W held flat, and above 29W the peak may be no more than 1.3 times the average. At idle, requirement PWR-6 asks for 5W or less whatever the form factor or capacity, reached automatically without hurting latency or performance.

Heat has three marks. A composite temperature of 77°C starts throttling, and the drive leaves throttling once it falls below 75°C. Once any component reaches a critical temperature, the drive reports 85°C and sends the host an event before it shuts off to protect itself. Those are requirements TTHROTTLE-5, -7 and -8, so the 85°C report is the drive’s last message before it shuts off.

The first Gen6 drives and controllers

Quarch judges it quite likely that the SFF-8639 connector behind U.2 and U.3 will not go to Gen6, the case Part 10 makes in full. The Gen6 drives below whose makers state a form factor use EDSFF (Enterprise and Data Center Standard Form Factor): Micron’s 9650 in E1.S and E3.S, and Kioxia’s GP1 in E3.S and E1.S 9.5mm. OCP 2.7 writes its Gen6 form factor rule into its E3 section, requirement FFE3-6, which has 64GT/s devices follow PCI Express Base Specification revision 6.2 or later. Part 1 covers the bays themselves, and its section on what is on offer places these drives among the rest.

Micron’s 9650 is rated up to 28GB/s sequential read and more than 5.5 million random read IOPS, and Micron’s blog calls it the first Gen6 SSD in mass production. Micron’s own 1,120MB/s per watt figure divides the 28GB/s rating by 25W, so it assumes a 25W draw. The brief also lists sequential read power of up to 18W for the 9650 PRO and MAX, and supports liquid cooling in E1.S 9.5mm.

Samsung announced mass production of the PM1763 on July 8, 2026. Its 16TB model is rated up to 28,400MB/s sequential read, Samsung puts the power efficiency at more than 1.8 times its predecessor’s, and the drive is optimized for direct-to-chip liquid cooling. FADU unveiled its Sierra (FC6161) controller in August 2025 with up to 28.5GB/s sequential read and write and a power envelope below 9W. Kioxia’s GP1, announced August 3, 2026, is a PCIe 6.0 NVMe 2.2 SSD rated up to 10 million random read IOPS at 512 bytes, with cold-plate liquid cooling in E3.S and E1.S 9.5mm. Liquid cooling is not a Gen6 feature alone: in October 2026 Kioxia added E3.S direct liquid cooling models to its PCIe 5.0 CM9 series. Part 10 covers the other Gen6 controllers announced so far, Marvell’s Bravera SC6 and Phison’s X3.

Sequential ratings of 28 to 28.5GB/s sit just under the roughly 30GB/s an x4 link leaves after framing. They come from makers under different conditions, and FADU’s is a controller rating, so they show that the link is now the limit for sequential reads, not which drive is faster. The two IOPS ratings on this page use different block sizes, so they do not rank the drives either.

PCI-SIG keeps PCIe 6.0 backward compatible with every earlier generation, so a Gen6 drive also works in an older slot, but it needs a Gen6 host to run at Gen6. On the host side, AMD announced on July 23, 2026 that its 6th Gen EPYC 9006 server CPUs, code-named Venice, bring PCIe Gen6, and StorageReview reports that Venice platforms come in waves, with SP7 expected in the fourth quarter of 2026 and SP8 in the first half of 2027.

Gen7, and the Gen8 draft

PCIe 7.0 runs at 128.0GT/s, double the 64.0GT/s of PCIe 6.x, and PCI-SIG released the specification to members on June 11, 2025, rated at up to 512GB/s counting both directions on an x16 link. Its draft 0.5 of PCIe 8.0 targets 256.0GT/s and up to 1.0TB/s bidirectionally on an x16 link, with full release on track for 2028. As of November 2026 these are specifications and drafts, and every drive above runs Gen6.

Gen7’s target for SSDs comes from the GPU side. StorageReview reports that NVIDIA’s Storage-Next roadmap calls for PCIe Gen7 SSDs sustaining 100 million IOPS each, a target that controller vendors including Marvell are designing toward. Kioxia’s GP1 rates 10 million at a 512-byte block size, and the report’s sentence gives no block size for the 100 million, so the two are not yet a like-for-like gap.

Two things to look for next are a controller datasheet that lists a 128.0GT/s interface, and an OCP revision that adds Gen7 rules the way version 2.7 adds PCI-42 for Gen6. Part 5 of the series moves from the link to the host’s side of the drive, with zoned namespaces (ZNS) and flexible data placement (FDP), and the series page lists every part.