Working Memory, Part 7: CXL Memory Explained

A Compute Express Link (CXL)CXL runs over the same physical layer as PCIe but adds cache-coherent memory access, so it moves cache lines rather than blocks of storage. memory module is DRAM that the processor reaches through a PCIe link instead of a DIMM socket. Its reads and writes use the same load and store instructions as the processor’s own memory, and the operating system sees it as one more memory node with no processor cores of its own. This lets a server add capacity past what its DIMM slots hold, and every access to that memory takes a longer trip.

Part 6 of this series covered the modules that put DRAM beside the processor, and this part moves that DRAM one link further out. It answers what a spec sheet means by CXL. That comes down to the kind of device, the generation of the standard, what the processor has to support, and how much slower the memory is than a DIMM. Flash Foundations Part 1 explains why a CXL module in an SSD bay is not an SSD, and Networking for SI, Part 5 follows CXL memory onto the storage path of an SI cluster.

Three protocols, three device types

CXL builds on the physical and electrical layers of PCIe and adds its own protocols on top. According to the consortium’s CXL 3.0 white paper, three of them share one link. CXL.io is the one based on PCIe. Devices use CXL.cache to cache the host’s memory. CXL.memory, usually written CXL.mem, lets the host load and store to memory on the device. Lenovo’s CXL 2.0 primer adds that the CXL protocols switch on only when both ends of the link support CXL, and otherwise the link runs as plain PCIe.

Devices are sorted by which protocols they carry beyond CXL.io, into Type 1, Type 2 and Type 3.

Source: SMART Modular, Introduction to CXL Memory Modules white paper, device types, and Lenovo Press LP2146, Type 1 to Type 3 devices.

SMART Modular’s white paper reasons from the protocols. A Type 1 device has no CXL.mem, so SMART infers that it holds no memory the host can use. A Type 2 device such as a GPU works in both directions, and in Lenovo’s description it can make host memory available to the accelerator and its own memory available to the host CPU. Type 3 is the kind this part is about. It carries only CXL.io and CXL.mem, and the same white paper says it targets memory capacity and bandwidth expansion.

Inside, a Type 3 device is a memory controller with a CXL port on one side and DRAM on the other. Such a controller can be built for DDR4 or DDR5, and SMART Modular’s CXA-8F2W-D4 card shows how loosely the two sides are tied. It is a CXL 2.0 Type 3 card on a PCIe Gen5 x16 link that turns CXL.mem requests into DDR4 commands, so reused DDR4 modules show up as extra host memory. The CXL Consortium’s blog says Penguin Solutions will show it at SC26 as a mechanical sample beside its target server, not as a running system.

What each generation added

CXL 1.0 came out in March 2019, and 1.1 followed that September. Both use the PCIe 5.0 physical layer at up to 32GT/s, which Lenovo’s primer puts at up to 64GB/s in each direction over a 16-lane link. A 1.0 or 1.1 memory device is one logical device that one host reaches at a time.

CXL 2.0 arrived in November 2020 and added switching, so one host can fan out to several devices and a device’s memory can be pooled among hosts. In Lenovo’s primer, a 2.0 device can also be split into several logical devices, and up to 16 hosts can each reach a different part of it at once. Only a 2.0 host can use a switch, while the memory devices behind it can mix 1.0, 1.1 and 2.0 hardware.

CXL 3.0 followed in August 2022 at 64GT/s on the PCIe 6.0 physical layer, and the consortium says the faster rate adds no latency over 2.0. Data crosses the link in 256-byte flits (the fixed-size units the link moves). Fabrics came with that release as well. Rambus explains that 2.0 allowed one level of switching while 3.0 allows switches behind switches, and the consortium’s 3.0 white paper gives port based routing a reach of up to 4,096 nodes. Pooling across such a fabric differs from sharing, where several hosts reach the same memory at once, and the CXL and SDXI guide sets the two apart.

CXL 3.1 (November 2023) worked on those fabrics. It added direct peer-to-peer CXL.mem through port based routing switches, host-to-host communication through what it calls Global Integrated Memory, and the Trusted Execution Environment Security Protocol (TSP). CXL 3.2 (December 2024) turned to the memory devices themselves, with a Hot-Page Monitoring Unit (CHMU) for memory tiering, Post Package Repair enhancements and wider TSP protection. CXL 4.0 (November 18, 2025) doubled the rate again, from 64 to 128GT/s, and keeps full backward compatibility with 3.x, 2.0, 1.1 and 1.0.

Seven releases from 2019 to 2025 sit side by side in the table.

Source: CXL Consortium releases for 2.0, 3.0, 3.1, 3.2 and 4.0, the consortium’s SNIA SDC 2023 deck for 1.0 and 1.1, and Lenovo Press LP2146 for the 1.x, 2.0 and 3.1 rates. The 3.2 release states no link rate, so that cell is blank.

A memory node with no cores

The processor reaches CXL memory through its home agent. That part of the CPU handles CXL.mem, and Sun and colleagues describe how it exposes the device’s memory as memory in a remote non-uniform memory access (NUMA) node. Software reaches it with ordinary loads and stores, the way it reaches memory on the other socket of a two-socket server.

Microsoft’s Pond pooling design gives that node memory and no cores. It hands each virtual machine its share of pooled CXL memory as a zero-core virtual NUMA node. Linux already supports CPU-less NUMA nodes of the same shape, and Pond’s authors found that the guest OS steers its allocations away from such a node. Deciding which pages belong on the slower node is memory tiering, and CXL 3.2 added the Hot-Page Monitoring Unit for it.

Software reaches CXL memory with loads and stores rather than block commands. Flash Foundations Part 1 covers what that means in an E3.S bay, where a CXL module and an NVMe SSD can share the same slot.

Bandwidth, latency and energy

A CXL link is a PCIe link, so its bandwidth follows the lane count and the PCIe generation. On PCIe 5.0, an x16 link carries up to 64GB/s each way. On PCIe 6.0, the lane rate doubles to 64GT/s and an x16 link carries about 128GB/s each way. The International Roadmap for Devices and Systems gives up to 256GB/s for CXL 3.0 on x16, which equals both directions added together.

Latency is the larger cost. For CXL.cache and CXL.mem, the consortium’s SNIA deck sets a target of under 200ns load to use. Pond’s authors estimate that CXL adds 70 to 90ns over same-node DRAM for pools of 8 to 16 sockets, and Sun and colleagues measured three real devices with load latency from 35% longer to 3x longer than DDR5 in a remote NUMA node. Those figures sit on the latency chart in the memory and storage hierarchy guide, and Networking for SI, Part 5 sets them against an SSD read.

Energy per bit is a separate comparison. As an example, Sun and colleagues cite about 6pJ to move a bit over PCIe 4.0 against 22pJ for DDR4.

As this guide reads it, the trade is capacity and bandwidth for latency. A workload whose busy data fits in local DRAM gains room from CXL at little cost, while one that reads at random across all of its memory pays the extra latency on most accesses.

What is on sale, as of November 2026

As of November 2026, Type 3 memory expansion is the part of CXL with products behind it. Samsung announced what it called the industry’s first CXL memory module on May 11, 2021, and that DDR5 module was validated on Intel server platforms. Modules, controllers and switches from the sources here are listed below, each with the status its maker or a report gives.

Source: Samsung CMM-D white paper, StorageReview’s report of Micron’s release, the SK hynix newsroom, Astera Labs releases of November 18, 2025 and September 15, 2026, Marvell’s release of October 12, 2026, and XConn’s release of March 26, 2024. A blank cell is not published in the source.

The Samsung and Micron modules use E3.S. Data center SSDs use that form factor too, and Flash Foundations Part 1 covers it. Controllers let server and card makers build their own expanders, and Structera X takes up to 9TB of DDR4 or 6TB of DDR5 per controller. Pooling needs a switch or a pooling controller, and in this list that means XConn’s Apollo and the Leo 2 P-Series. Which processors support which version is a separate question, and Flash Foundations Part 10 keeps that table.

What to watch

AMD’s 6th Gen EPYC (Venice) is the next processor to watch. StorageReview reports that it carries CXL 3.1 on its PCIe 6.0 lanes for memory expansion, and that its SP7 platforms arrive in Q4 2026. CXL 3.2 expanders come after it, with Marvell expecting Structera X samples in Q1 2027 and SK hynix already showing its 3.2 module. Until those reach servers, a CXL 3.x feature on a module data sheet describes what the module can do with a host that supports 3.x.

Both the 3.2 and 4.0 releases state full backward compatibility with earlier versions, so a newer module in an older server should still work, limited to the older version’s features.

That ends this series. Part 1 starts again with the processor’s SRAM, which is the fastest memory.