DRAM-Less SSDs Are Slow

A DRAM-less SSDA DRAM-less SSD has no DRAM of its own inside the drive. It can use the NVMe host memory buffer feature to borrow some of the computer’s memory instead. is not slow by default. In my test results, the four DRAM-less drives I can prove wrote small random data about 24% slower than the three drives with DRAM, yet their PC workload scores overlapped the DRAM drives’ range, and the two highest scores belonged to DRAM-less drives. The set is small and mixes controllers and NAND, so it shows that DRAM doesn’t decide the result. It can’t say how much DRAM is worth.

This page is part of Flash Myths, Tested. The sequential speed myth set up the PCMark 10 Storage score, which replays traces of real application use, and the QLC myth looked at the cell type that also shows up in the tables below. This one asks whether a drive without its own DRAM has to be the slow one.

Where the myth comes from

An SSD keeps a map from the addresses the computer uses to the places on the flash where the data actually sits. A page-level map takes about 0.1% of the drive’s capacity, according to Kim and Kim, which is about 1GB for each 1TB. A drive with DRAM keeps a large part of that map in DRAM, so finding where a piece of data lives is a quick lookup. The Samsung 990 PRO data sheet shows the pattern, with 1GB of LPDDR4 on the 1TB model and 2GB on the 2TB model.

A DRAM-less drive has no such chip. I rely on Kim and Kim’s 2020 paper in PLOS ONE for the mechanism, and it plainly states that the DRAM-less drives it tested had worse I/O performance than similar drives with DRAM. That explains the myth, and it isn’t wrong about those drives.

What the host memory buffer does

NVMe lets a drive borrow some of the computer’s own memory, called the host memory bufferHost memory buffer lets a DRAMless NVMe drive borrow a small region of host RAM to cache part of its map., or HMB. Per section 8.9 of the NVMe Base Specification, revision 1.4c, it is optional. How the drive uses the memory is vendor specific. A drive that can use it still has to work without it, and the memory is not kept across a reset. I checked revision 1.4c only, not later ones. A drive that depends on an HMB instead of its own DRAM is a DRAM-less drive. The Memory and Storage Hierarchy shows where the buffer sits on the path a read takes. Flash Foundations Part 6 covers the map and the buffer from the controller’s side.

Kim and Kim tested three DRAM-less drives that support an HMB against three drives with DRAM, all on PCIe 3.0, using 4KB reads and writes. In a light test of one thread and 16GB, two of the three DRAM-less drives were far behind a similar DRAM drive with the HMB off, and the third matched its DRAM counterpart. In a heavy test of 72 threads over 100% of each drive, the gap in reads was substantial. With the HMB switched on, random reads on all three DRAM-less drives came close to their sequential reads in the heavy test. Their probing found that the drives used the buffer mainly as a cache for the address map. They didn’t use it as a read cache or a write buffer.

Its results come as charts with no overall percentage, so I quote none. It also tested PCIe 3.0 drives, where my set A results use PCIe 4.0 drives.

Figure 1 shows why a buffer can help without being large. Both Kingston DRAM-less drives in my results report an enabled HMB of 64MiB in their flash-ID dumps. Against a map of about 1,000MB on a 1TB drive that is about 6.7%, and against about 2,000MB on a 2TB drive it is about 3.4%. This is arithmetic on the 0.1% rule. It measures nothing about what the drive caches. A cache that holds the parts of the map in use doesn’t need to hold all of it. I have no number for how well these drives do that.

What my results show

I call my archive of test-suite results bench set A below, and it has no column for DRAM. I can prove it only for seven drives whose flash-ID dump states it, and three have 2,048MB of DRAM and four are DRAM-less, two of them marked as having no buffer and two with the 64MiB HMB. Another 79 drives in the archive are unknown, so I left them out rather than guess.

Those 4KiB columns are CrystalDiskMarkCrystalDiskMark is a simple disk benchmark program for Windows that measures sequential and random read and write performance. random tests at queue depthIOPS counts commands finished per second. Queue depth is how many commands are outstanding at once, and a drive needs a deeper queue to reach its top IOPS. 1 on one thread, and I report the median of the repeat runs. PCMark 10 Storage is the median of 3 runs. The archive does not name the tool versions, how many times CrystalDiskMark repeated each test, or its test size and loop settings. All of these numbers come from my archive of results.

Figure 2 is the clean result. Every DRAM-less drive wrote 4KiB at queue depth 1 at or below 302.1MB/s, and every DRAM drive wrote at or above 379.7MB/s. Median DRAM-less writes ran 24% lower. Small random writes are the kind of access the map lookup sits on, so the gap fits the mechanism, but with three drives against four I can’t separate DRAM from the controller, the firmware and the capacity.

Random reads at the same depth do not split the groups. DRAM drives ran from 77.9MB/s to 92.6MB/s and the DRAM-less drives from 70.4MB/s to 87.8MB/s, and two DRAM-less drives read faster than one of the DRAM drives.

Figure 3 is the result that matters for the myth. PCMark 10 Storage replays traces of real application use, and the DRAM scores ran from 2,912 to 3,372 while the DRAM-less scores ran from 2,479 to 3,631. Two DRAM-less Adata drives post the highest scores and two DRAM-less Kingston drives the lowest, so the group label does not predict the score. This set has two DRAM-less drives that beat every DRAM drive, which answers a claim that all DRAM-less drives are slow.

Sustained write runs point the same way. After its cache ran out, one DRAM-less drive averaged 3,299MB/s, above the three DRAM drives at 1,881MB/s to 2,402MB/s, but the other two with a run averaged 512MB/s and 250MB/s, and the 250MB/s drive is the QLC one. A sustained sequential write is not mostly a map lookup problem, and this set cannot say why those two are slower.

A second set of drives from my bench

Those seven drives are bench set A. I have a second archive from my own bench, which I call bench set B, with 39 physical drives from 250GB to 8TB that I looked up by spec sheet instead of by flash-ID dump. I don’t pool the two sets. Set B has 20 drives with DRAM, 7 without and 12 I cannot classify. Only 2 of the 39 have a physical flash-ID dump, and both of those have DRAM. Not one of the 7 DRAM-less drives is proven by a dump. Four are DRAM-less because the controller spec sheet says it supports a host memory buffer, and three because a spec table says so. That is a weaker level of proof than set A, so read set B as strong hints about groups. It proves nothing about any single drive. Two of the 7 are early samples rather than retail products, and I left them out of the medians below.

Each cell is the median across the drives in the group that have the measurement. Random tests are CrystalDiskMark RND 4KiB at queue depth 1, and the last column is the sustained write average after the cache ran out, as in set A. Group sizes are small. The DRAM, TLC group is one controller family on PCIe 4.0, the DRAM-less TLC group is two families on PCIe 4.0, and the DRAM, QLC group is one family on PCIe 3.0, so its low numbers mix cell type, controller and interface.

The first row and the third make the closest comparison. At queue depth 1, the DRAM-less TLC drives read 7% below the DRAM TLC drives and wrote 14% below them. The direction matches set A, with a smaller gap in writes, and it still mixes in a different controller. Median sustained write is much lower for the DRAM-less TLC group, but those drives are 250GB to 1TB and the DRAM drives go up to 8TB, and the DRAM drives’ own results vary from 876MB/s to 3,683MB/s with capacity. I can’t separate DRAM from drive size there.

Cell type moved the result more than DRAM did. The DRAM QLC group read 50% below the DRAM TLC group, and its sustained write after the cache was 153.0MB/s. That group is also on an older interface, so it is not a clean cell-type test either.

Set B’s one same-controller pair is a 1TB TLC drive and a 2TB QLC drive with no DRAM in either. The QLC drive read 16% below the TLC drive at queue depth 1 (17,556.4 against 21,012.7 IOPS) and wrote after its cache at 127MB/s against 1,545MB/s. Its cache was larger, 491.8GB against 76.2GB. That QLC drive was faster at small writes and scored higher in PCMark 10 Storage, 2,860 against 2,631, so it was not slower everywhere. Capacities differ, so this is one pair and a poor stand-in for a controlled test.

PCMark 10 Storage points the other way from set A. At 1TB, the DRAM-less TLC drive scored 20% lower than the DRAM TLC drive, 2,631 against 3,295. At 2TB, the DRAM-less QLC drive scored 13% lower than the DRAM TLC drive, 2,860 against 3,296. Those are single drives, and the DRAM and DRAM-less drives in each pair are on different controllers, so I take them as a caution and not as a new rule. They do mean set A’s result, that DRAM-less scores overlapped the DRAM scores, doesn’t carry over to every DRAM-less drive.

What this does and does not show

It shows that DRAM status does not predict the score in this set, and that a DRAM-less drive can post the best score in it. It shows a gap in small random writes at queue depth 1 in set A (24%) and between the TLC groups in set B (14%), and a smaller gap in small random reads in set B. The write gap does not hold for every group. The DRAM-less QLC drive in set B wrote 398.1MB/s, above the DRAM TLC median of 380.5MB/s, and the DRAM QLC group wrote 109.8MB/s, below every DRAM-less group.

It doesn’t show how a DRAM-less drive behaves on a nearly full drive, under many threads over the whole drive, or on a different PC. Kim and Kim’s heavy test is that kind of load, and it is where their read gap was widest. I don’t have a number from my sources for how often a desktop looks like their heavy test, so I would not claim it never happens. Kingston’s NV2 pair had the lowest scores here, and the data can’t say whether DRAM, the controller or the NAND is the reason.

What to do with this

Do not rule a drive out because the spec sheet says DRAM-less. Look at the drive’s own measured results for the kind of work you do, and treat the missing DRAM as one reason a drive might be weaker at small random writes. If the work is many threads hitting the whole drive, the heavy test in Kim and Kim is the case where the gap was widest, and a drive with DRAM is the safer choice on that evidence. That rests on Kim and Kim’s 2020 PCIe 3.0 test, since I have no heavy-load number of my own.