In modern multi-processor systems-on-chips (MPSoCs), writebacks from the private caches to the shared cache can introduce significant performance bottlenecks, especially because multiple threads from different co-executing programs contend for the shared cache resources. Intelligent cache bypass decisions for writebacks help mitigate such contention and enhance the utilization of the shared cache. Most prior cache bypass strategies account for contention for shared cache capacity by focusing primarily on data reuse, with only recent research beginning to consider bandwidth contention also in dynamic bypass decisions. However, data sharing, a crucial characteristic of modern multithreaded workloads, remains largely overlooked by state-of-the-art cache bypass decisions. Bypassing highly shared cache lines can increase the volume of main memory accesses, potentially resulting in performance bottlenecks. We introduce SHARP, a novel cache bypass policy that incorporates three key factors: data sharing, contention, and data reuse, into its dynamic bypass decisions for cache writebacks. In addition to prioritizing the caching of data with high reuse, we prioritize the caching of data shared across multiple threads to enhance cache utilization. We dynamically modulate our bypass decisions, employing aggressive bypass for writebacks when shared cache contention is high, while employing conservative bypass when contention is low. Experiments across a diverse set of PARSEC workloads demonstrate that SHARP improves overall system throughput by 12% and 8% compared to the no-bypass baseline and the state-of-the-art bypass baseline, respectively. SHARP also reduces the overall cache energy consumption by 14% over the no-bypass baseline.

Contemporary multiprocessor systems-on-chips (MPSoCs) continue to confront energy-related challenges, primarily originating from off-chip data movements. Nonvolatile memories (NVMs) emerge as a promising solution with their high-storage density and low leakage, yet they suffer from slow and expensive write operations. Writebacks from higher-level caches and responses from off-chip memory create significant contention at the shared nonvolatile last-level cache (LLC), affecting system performance with increased queuing for critical reads. Previous research primarily addresses the performance issues by trying to mitigate contention through the bypassing of NVM writes. Nevertheless, off-chip memory energy, one of the most critical components of system energy, remains unaddressed by state-of-the-art bypass policies. While certain energy components, such as leakage and refresh, depend on system performance, performance-optimizing bypass policies may not ensure energy efficiency. Aggressive bypass decisions aimed only at performance enhancement could degrade cache reuse, potentially outweighing reductions in leakage and refresh energies with the increase in off-chip dynamic energy. While both performance and off-chip memory energy are influenced by both cache contention and reuse, the tradeoffs for achieving optimal performance versus optimal energy are different. We introduce nonvolatile last-level cache bypass for optimizing off-chip memory energy (NOVELLA), a novel bypass policy for the nonvolatile LLC, to optimize off-chip memory energy by exploiting tradeoffs between cache contention and reuse, achieving a balance across different components of the energy. Compared to a naïve no-bypass baseline, while state-of-the-art reuse-aware bypass solutions reduce off-chip memory energy consumption by up to 8%, and a contention- and reuse-aware bypass baseline by 12%, NOVELLA achieves significant energy savings of 21% across diverse SPEC workloads.

In modern multi-processor systems-on-chip (MPSoCs), requests from different processor cores, accelerators, and their responses from the lower-level memory contend for the shared cache bandwidth, making it a critical performance bottleneck. Prior research on shared cache management has considered requests from cores but has ignored crucial contributions from their responses. Prior cache bypass techniques focused on data reuse and neglected the system-level implications of shared cache contention. We propose COBRRA, a novel shared cache controller policy that mitigates the contention by aggressively bypassing selected responses from the lower-level memory and scheduling the remaining requests and responses to the cache efficiently. COBRRA is able to improve the average performance of a set of 15 SPEC workloads by 49% and 33% compared to the no-bypass baseline and the best-performing state-of-the-art bypass solution, respectively. Furthermore, COBRRA reduces the overall cache energy consumption by 38% and 31% compared to the no-bypass baseline and the most energy-efficient state-of-the-art bypass solution, respectively.

Non-volatile memories (NVMs), with their high storage density and ultra-low leakage power, offer promising potential for redesigning the memory hierarchy in next-generation Multi-Processor Systems-on-Chip (MPSoCs). However, the adoption of NVMs in cache designs introduces challenges such as NVM write overheads and limited NVM endurance. The shared NVM cache in an MPSoC experiences requests from different processor cores and responses from the off-chip memory when the requested data is not present in the cache. Besides, upon evictions of dirty data from higher-level caches, the shared NVM cache experiences another source of write operations, known as writebacks . These sources of write operations—writebacks and responses—further exacerbate the contention for the shared bandwidth of the NVM cache and create significant performance bottlenecks. Uncontrolled write operations can also affect the endurance of the NVM cache, posing a threat to cache lifetime and system reliability. Existing strategies often address either performance or cache endurance individually, leaving a gap for a holistic solution. This study introduces the Performance Optimization and Endurance Management (POEM) methodology, a novel approach that aggressively bypasses cache writebacks and responses to alleviate the NVM cache contention. Contrary to the existing bypass policies that do not pay adequate attention to the shared NVM cache contention and focus too much on cache data reuse, POEM’s aggressive bypass significantly improves the overall system performance, even at the expense of data reuse. POEM also employs effective wear leveling to enhance the NVM cache endurance by careful redistribution of write operations across different cache lines. Across diverse workloads, POEM yields an average speedup of 34% over a naïve baseline and 28.8% over a state-of-the-art NVM cache bypass technique while enhancing the cache endurance by 15% over the baseline. POEM also explores diverse design choices by exploiting a key policy parameter that assigns varying priorities to the two system-level objectives.

Modern multi-processor systems-on-chip (MPSoCs) are characterized by caches shared by multiple cores. These shared caches receive requests issued by the processor cores. Requests that are subject to cache misses may result in the generation of responses . These responses are received from the lower level of the memory hierarchy and written to the cache. The outstanding requests and responses contend for the shared cache bandwidth. To mitigate the impact of the cache bandwidth contention on the overall system performance, an efficient request and response arbitration policy is needed. Research on shared cache management has neglected the additional cache contention caused by responses, which are written to the cache. We propose CABARRE , a novel request and response arbitration policy at shared caches, so as to improve the overall system performance. CABARRE shows a performance improvement of 23% on average across a set of SPEC workloads compared to straightforward adaptations of state-of-the-art solutions.

Modern high-performance and high-bandwidth three-dimensional (3D) memories are characterized by frequent heating. Prior art suggests turning off hot channels and migrating data to the background DDR memory, incurring significant performance and energy overheads. We propose three Dynamic Thermal Management (DTM) approaches for 3D memories, reducing these overheads. The first approach, Rotating-channel Low-power-state-based DTM (RL-DTM) , minimizes the energy overheads by avoiding data migration. RL-DTM places 3D memory channels into low power states instead of turning them off. Since data accesses are disallowed during low power state, RL-DTM balances each channel’s low-power-state duration. The second approach, Masked rotating-channel Low-power-state-based DTM (ML-DTM) , is a fine-grained policy that minimizes the energy-delay product (EDP) and improves the performance of RL-DTM by considering the channel access rate. The third strategy, Partial channel closure and ML-DTM , minimizes performance overheads of existing channel-level turn-off-based policies by closing a channel only partially and integrating ML-DTM, reducing the number of channels being turned off. We evaluate the proposed DTM policies using various mixes of SPEC benchmarks and multi-threaded workloads and observe them to significantly improve performance, energy, and EDP over state-of-the-art approaches for different 3D memory architectures.