This paper presents a study of DDR4 DRAM faults in a large fleet of commodity servers, covering several billion memory device-hours of data. The goal of this study is to understand faults in DDR4 DRAM devices to measure the efficacy of existing hardware resilience techniques and aid in designing more resilient systems for future large-scale systems.The study has several key findings about the fault characteristics of DDR4 DRAMs and adds several novel insights about system reliability to the existing literature. Specifically, the data show sixteen unique fault modes in the DDR4 DRAM under study, including several that have not been previously reported. Over 45% of the faults that occurred affected multiple DRAM bits. The time-to-failure characteristics of faults internal to the DRAM die differ from those external to the DRAM die. We also examine faults from multiple DRAM vendors, finding that fault rates vary by more than 1.34x among vendors.Finally, we use the data to compare chipkill ECC and an ECC that covers a DDR5 "bounded fault." Given the fault rates in this data, a bounded fault ECC increases the rate of faults that cause uncorrectable errors by up to 5.71 FIT per DRAM device compared to chipkill ECC.

We have discovered that processors can experience a super-linear increase in detected unrecoverable errors (DUE) when the write-back L2 cache is doubled in size. This paper explains how an increase in the cache tag's Architectural Vulnerability Factor or AVF caused such a super-linear increase in the DUE rate. AVF expresses the fraction of faults that become user-visible errors. Our hypothesis is that this increase in AVF is caused by a super-linear increase in ¿dirty¿ data residence times in the L2 cache. Using proton beam irradiation, we measured the DUE rates from the write-back cache tags and analyzed the data to show that our hypothesis holds. We utilized a combination of simulation and measurements to help develop and prove this hypothesis. Our investigation reveals two methods by which dirty line residency causes super-linear increases in the L2 cache tag's AVF. One is a reduction in the miss rates as we move to the larger cache part, resulting in fewer evictions of data required for architecturally correct execution. The second is the occurrence of strided cache access patterns, which cause a significant increase in the ¿dirty¿ residency times of cache lines without increasing the cache miss rate.