Despite significant improvements in peak bandwidth, the HBM industry has neglected random-access (irregular) bandwidth, limiting performance in many real-world applications.Improving effective HBM bandwidth is challenging due to power-constrained activations and coarse-grained, long-distance data movement.Rather than addressing these issues directly, hardware vendors have opted for incremental changes, achieving a 6.4× increase in sequential access bandwidth over two generations while leaving the irregular bandwidth challenges unresolved.To remedy this, we introduce Folded Banks (FB-HBM), a novel 3D bank design that redistributes bank subarrays ("folds") across multiple dies and relocates command, control, and global sense amplifiers to an additional base layer.By implementing this logic in a new base layer, we eliminate the DRAM die overheads inherent in previous designs.This architecture enables vertical routing of intra-bank wires-column select lines (CSLs) and master data lines (MDLs)-through thin-pitch through-silicon vias (TSVs) and hybrid bonds, significantly reducing RC power losses.By employing self-timed sense amplifiers, we eliminate costly dummy subarrays * Work done while Wantong Li was an intern at AMD RAD.

The pace of advancement of the top-end supercomputers historically followed an exponential curve similar to (and driven in part by) Moore's Law. Shortly after hitting the petaflop mark, the community started looking ahead to the next milestone: Exascale. However, many obstacles were already looming on the horizon, such as the slowing of Moore's Law, and others like the end of Dennard Scaling had already arrived. Anticipating significant challenges for the overall high-performance computing (HPC) community to achieve the next 1000x improvement, the U.S. Department of Energy (DOE) launched the Exascale Computing Program to enable and accelerate fundamental research across the many technologies needed to achieve exascale computing.

Cache-block compression is a highly effective technique for both reducing accesses to lower levels in the memory hierarchy (cache compression) and minimizing data transfers (link compression). While many effective cache-block compression algorithms have been proposed, the design of these algorithms is largely ad hoc and manual and relies on human recognition of patterns. In this article, we take an entirely different approach. We introduce a class of “byte-select” compression algorithms, as well as an automated methodology for generating compression algorithms in this class. We argue that, based on upper bounds within the class, the study of this class of byte-select algorithms has potential to yield algorithms with better performance than existing cache-block compression algorithms. The upper bound we establish on the compression ratio is 2X that of any existing algorithm. We then offer a generalized representation of a subset of byte-select compression algorithms and search through the resulting space guided by a set of training data traces. Using this automated process, we find efficient and effective algorithms for various hardware applications. We find that the resulting algorithms exploit novel patterns that can inform future algorithm designs. The generated byte-select algorithms are evaluated against a separate set of traces and evaluations show that Byte-Select has a 23% higher compression ratio on average. While no previous algorithm performs best for all our data sets which include CPU and GPU applications, our generated algorithms do. Using an automated hardware generator for these algorithms, we show that their decompression and compression latency is one and two cycles respectively, much lower than any existing algorithm with a competitive compression ratio.