DDR4 memory organization and how it affects memory bandwidth

DDR4 Rank and Organization: What They Mean for Bandwidth

When you buy a DDR4 module, the important specs are usually capacity and speed—a 32GB DDR4-2666 module, for example, operates at 2666 MT/s. But if you look closer at the available options, you'll notice additional parameters like rank and device organization, often written as 1Rx8 or 2Rx4. These describe the internal layout of the module's DRAM chips, and they can influence performance.

Understanding Memory Rank

A memory rank is a set of DRAM devices that together make up a 64-bit data bus (72-bit with ECC). All chips within a single rank share address, command, and control signals. This is analogous to a memory bank, except that a rank exists at the module level (the printed circuit board), while a bank is a structural unit inside each individual DRAM chip.

A single-rank (1R) module has one set of DRAM chips. A dual-rank (2R) module has two sets, effectively doubling capacity per module. A quad-rank (4R) module has four sets, similar to two dual-rank modules. In all multi-rank configurations, only one rank is active at a time via chip select signals; the other ranks cannot be accessed simultaneously.

However, idle ranks are not dormant: multi-rank modules use rank interleaving, allowing non-accessed ranks to complete refresh cycles in parallel. This pipelining can reduce response time because the next rank can start transmitting as soon as the current rank finishes. There is, however, an I/O latency penalty since the memory controller needs extra clock cycles to switch between ranks. Which effect dominates depends heavily on the application workload. Additionally, because single-rank modules have fewer chips, they generate less heat and may be more reliable.

Memory Depth and Width

Each DRAM chip's capacity is expressed as memory depth × width. Width refers to the number of data I/O (DQ) lines per chip, which is standardized at x4, x8, or x16. To fill a 64-bit rank, you need 16 x4 chips or 8 x8 chips (18 or 9 with ECC, respectively).

The two high-level configurations are illustrated below. A 1Gbx8 device and a 2Gbx4 device both offer a total capacity of 8Gb, but through different depth and width combinations.

Within each DRAM chip, rows are activated through a row address MUX, and data is loaded into sense amplifiers before the column decoder selects the output. The density calculation is straightforward:

Memory Depth = Number of Rows × Number of Columns × Number of Banks

Total Memory Capacity = Memory Depth × Memory Width

In the 1Gbx8 example, there are 16 row address bits (65,536 rows) and 10 column address bits (1,024 columns) across 16 banks:

Memory Depth = 65536 × 1024 × 16 = 1Gb

Total Capacity = 1Gb × 8 = 8Gb

For the 2Gbx4 device, adding one more row address bit doubles the row count:

Memory Depth = 131072 × 1024 × 16 = 2Gb

Total Capacity = 2Gb × 4 = 8Gb

Module Capacity and Device Width

Rank and device width together determine how many DRAM chips are needed on a module. An ECC-capable 64-bit module has a 72-bit data bus, so a single-rank module using x4 devices needs 18 chips (16 data + 2 ECC). A dual-rank version needs 36. With 8Gb x4 devices, a single-rank ECC module yields 16GB:

8Gb × 16 = 128Gb = 16GB

A dual-rank ECC module with 36 devices reaches 32GB. By contrast, a single-rank module using 8Gb x8 devices yields only 8GB—a single-rank x8 ECC module requires just 9 chips:

8Gb × 8 = 64Gb = 8GB

The key takeaway: with the same underlying DRAM technology (8Gb in these examples), higher module capacity is achieved by using x4 devices, dual-rank, or quad-rank configurations.

Timing Impact of Device Width: tFAW

Device width also influences timing parameters such as tFAW (Four Active Window). tFAW defines the minimum window in which four ACTIVATE commands can be issued to open rows. Each bank has its own sense amplifiers, so one row can remain open per bank, and a controller can issue four back-to-back ACTIVATEs—but a fifth must wait until the tFAW window has expired.

Within the same DDR4 speed grade, page size affects tFAW: a 1/2KB page at DDR4-1600 allows a 20ns (16 CK) four-active window, while larger pages require longer windows. Page size is directly tied to device width:

Page Size = Number of Columns × Memory Device Width = 1024 × Device Width

Since x16 devices have larger pages than x4 devices, they also have the longest tFAW limits. This is a negative timing impact for wider devices, which can limit achievable bandwidth in certain access patterns.

Testing 2Rx4 vs. 2Rx8: Benchmarks on AMD EPYC

To quantify the performance effect, tests were run on a Gen11 server with an AMD EPYC 7713 (Milan) CPU. That processor has 64 cores and 8 memory channels. A typical production configuration uses one DIMM per channel, combining four 32GB 2Rx8 and four 64GB 2Rx4 modules to reach the 6GB/core ratio (384GB total).

For the comparison, two isolated test cases were built using identical servers: one populated entirely with 32GB 2Rx4 RDIMMs and one entirely with 32GB 2Rx8 RDIMMs, each with one DIMM per channel.

Intel Memory Latency Checker (MLC) results showed only a 0.21% difference in all-read workloads between the two configurations.

The gap widens as writes increase: at 3:1 read-to-write (25% writes), the 2Rx4 configuration showed 1.13% higher bandwidth; at 1:1 (50% writes), the difference grew to 2.93%.

LMBench results followed a similar trend.

The largest gap was found in the write workload, where 2Rx4 modules delivered 3.88% higher write bandwidth than 2Rx8 modules.

Summary and Power Considerations

Memory organization and device width do have a measurable but modest effect on bandwidth. The difference favors x4 devices and is most visible in write-heavy workloads, but even there it stays below 4% in these benchmarks.

There is a trade-off beyond raw bandwidth: x4 modules require twice as many DRAM chips as x8 modules of the same capacity. More chips consume more power. Micron's measurement data shows that 32GB 2Rx8 modules built with 16Gb devices use 31% less power than 32GB 2Rx4 modules using 8Gb devices. For memory-heavy servers, that level of power savings could be more valuable than the minor bandwidth advantage of x4 modules, making wider-device modules a logical choice for future system designs where energy efficiency is a priority.