Graphics Processing Unit (GPU) performance has relied heavily on our ability to scale of number of transistors on chip, in order to satisfy the ever-increasing demands for more computation. However, transistor scaling has become extremely challenging, limiting the number of transistors that can be crammed onto a single die. Manufacturing large, fast and energy-efficient monolithic GPUs, while growing the number of stream processing units on-chip, is no longer a viable solution to scale performance. GPU vendors are aiming to exploit multi-GPU solutions, interconnecting multiple GPUs in the single node with a high bandwidth network (such as NVLink), or exploiting Multi-Chip-Module (MCM) packaging, where multiple GPU modules are integrated in a single package. The inter-GPU bandwidth is an expensive and critical resource for designing multi-GPU systems. The design of the inter-GPU network can impact performance significantly. To address this challenge, in this paper we explore the potential of hardware-based memory compression algorithms to save bandwidth and improve energy efficiency in multi-GPU systems. Specifically, we propose an adaptive inter-GPU data compression scheme to efficiently improve both performance and energy efficiency. Our evaluation shows that the proposed optimization on multi-GPU architectures can reduce the interGPU traffic up to 62%, improve system performance by up to 33%, and save energy spent powering the communication fabric by 45%, on average.

Three-dimensional DRAMs (3D-DRAMs) are emerging as a promising solution to address the memory wall problem in computer systems. However, high fabrication cost per bit and thermal issues are the main reasons that prevent architects from using 3D-DRAM alone as the main memory building block. In this article, we address this issue by proposing a heterogeneous memory system that combines a double data rate (DDRx) DRAM with an emerging 3D hybrid memory cube (HMC) technology. Bandwidth and temperature management are the challenging issues for this heterogeneous memory architecture. To address these challenges, first we introduce a memory page allocation policy for the heterogeneous memory system to maximize performance. Then, using the proposed policy, we introduce a temperature-aware algorithm that dynamically distributes the requested bandwidth between HMC and DDRx DRAM to reduce the thermal hotspot while maintaining high performance. We take into account the impact of both core count and HMC channel count on performance while using the proposed policies. The results show that the proposed memory page allocation policy can utilize the memory bandwidth close to 99% of the ideal bandwidth utilization. Moreover, our temperate-aware bandwidth adaptation reduces the average steady-state temperature of the HMC hotspot across various workloads by 4.5 K while incurring 2.5% performance overhead.

Block-level cooperation is an endurance management technique that operates on top of error correction mechanisms to extend memory lifetimes. Once an error recovery scheme fails to recover from faults in a data block, the entire physical page associated with that block is disabled and becomes unavailable to the physical address space. To reduce the page waste caused by early block failures, other blocks can be used to support the failed block, working cooperatively to keep it alive and extend the faulty page’s lifetime. We combine the proposed technique with existing error recovery schemes, such as Error Correction Pointers (ECP) and Aegis, to increase memory lifetimes. Block cooperation is realized through metadata sharing in ECP, where one data block shares its unused metadata with another data block. When combined with Aegis, block cooperation is realized through reorganizing data layout, where blocks possessing few faults come to the aid of failed blocks, bringing them back from the dead. Our evaluation using Monte Carlo simulation shows that block cooperation at a single level (or multiple levels) on top of ECP and Aegis, boosts memory lifetimes by 28% (37%) and 8% (14%) on average, respectively. Furthermore, using trace-driven benchmark evaluation shows that lifetime boost can reach to 68% (30%) exploiting metadata sharing (or data layout reorganization).

Heterogeneous architectures have emerged as a promising solution to address the dark silicon challenge by providing customized cores for each running application. To harness the power of heterogeneity, a critical challenge is simultaneously fine-tuning several parameters at the application, architecture, system, as well as circuit levels for heterogeneous architectures that improve the energy-efficiency envelope. To address this challenge, an ElasticCore platform is described where core resources along with the operating voltage and frequency settings are scaled to match the application behavior at run-time. A quantile linear regression model for power and performance prediction is used to guide the adaptation of the core resources, along with the operating voltage and frequency, to improve the energy efficiency. In addition, the dynamically scalable partitions of the ElasticCore are powered with multiple on-chip voltage regulators with high-power conversion efficiency that are able to realize fast dynamic voltage/frequency scaling. The results indicate that ElasticCore predicts application power and performance behavior with a small error at run-time across all studied benchmarks and achieves, on average close to 93% energy efficiency, as compared to an architecture with the Oracle power and performance predictor.

Block-level cooperation is an endurance management technique that operates on top of error correction mechanisms to extend memory lifetimes. Once an error recovery scheme fails to recover from faults in a data block, the entire physical page associated with that block is disabled and becomes unavailable to the physical address space. To reduce the page waste caused by early block failures, other blocks can be used to support the failed block, working cooperatively to keep it alive and extend the faulty page's lifetime. We combine the proposed technique with existing error recovery schemes, such as Error Correction Pointers (ECP) and Aegis, to increase memory lifetimes. Block cooperation is realized through metadata sharing in ECP, where one data block shares its unused metadata with another data block. When combined with Aegis, block cooperation is realized through reorganizing data layout, where blocks possessing few faults come to the aid of failed blocks, bringing them back from the dead. Employing block cooperation at a single level (or multiple levels) on top of ECP and Aegis, we can increase memory lifetimes by 28% (37%), and 8% (14%) on average, respectively.

Limited write endurance is the main obstacle standing in the way of using phase change memory (PCM) in future computing systems. While several wear-leveling and hard-error tolerant techniques have been proposed for improving PCM lifetime, most of these approaches assume that the underlying memory uses a very simple write traffic reduction scheme (e.g., buffering, differential writes). In particular, most PCM prototypes/chips are equipped with an embedded circuit to support differential writes (DW) - on a write, only the bits that differ between the old and new data are updated. With DW, the bit-pattern of updates in a memory block is usually random, which limits the opportunity to exploit the resulting bit pattern for lifetime enhancement at an architecture level (e.g., using techniques such as wear-leveling and hard-error tolerance). This paper focuses on this inefficiency and proposes a solution based on data compression. Employing compression can improve the lifetime of the PCM memory. Using state-of-the-art compression schemes, the size of the compressed data is usually much smaller than the original data written back to memory from the last-level cache on an eviction. By storing data in a compressed format in the target memory block, first, we limit the number of bit flips to fewer memory cells, enabling more efficient intra-line wear-leveling and error recovery, and second, the unused bits in the memory block can be reused as replacements for faulty bits given the reduced size of the (compressed) data. It can also happen that for a portion of the memory blocks, the resulting compressed data is not very small. This can be due to increased data entropy introduced by compression, where the total number of bit flips will be increased over the baseline system. In this paper, we present an approach that provides collaborative operation of data compression, differential writes, wear-leveling and hard-error tolerant techniques targeting PCM memories. We propose approaches that reap the maximum benefits from compression, while also enjoying the benefits of techniques that reduce the number of high-entropy writes. Using an approach that combines different solutions, our mechanism tolerates 2.9× more cell failures per memory line and achieves a 4.3× increase in PCM memory lifetime, relative to our baseline state-of-the-art PCM DIMM memory.

Rapid technology scaling has enabled the integration of many cores into a single chip. Given this level of core integration, the requirements for a large and scalable main memory system will only grow. Current DRAM-based main memory systems face power and scalability issues when working at sub-micron scales. Phase Change Memory (PCM) has been proposed as one of the most promising technology candidates to replace or complement DRAM. However, scaling down cell sizes introduces significant thermal-based write disturbance challenges in PCM. Due to the heat generated for programming cells, neighboring cells may be disturbed, experiencing changes in their values. A naive solution is to increase inter-cell space, attempting to isolate cell programming and eliminating write disturbance, but this approach significantly reduces PCM density. In this paper, we propose two cost-effective solutions to reduce the probability of write disturbance. Our solutions come with few side-effects on other memory system metrics. The first technique is based on data encoding, and tries to reduce the number of vulnerable data patterns when writing data to main memory. The second technique detects vulnerable cells, and overwrites them if their occurrence is below a set threshold. The proposed techniques are general and can avoid much of the performance overhead introduced by write disturbance. Our proposed solutions can reduce the average number of writes by 49% over traditional schemes, while incurring minimal impact on PCM lifetime and energy consumption.

A dynamic power management system for homogeneous chip multi-processors (CMP) is proposed. Each core of the CMP includes on chip DC-DC switching buck converters that are interconnected through a switch network. The peak current rating of the buck converter is selected to meet only the average current demand of the load circuit. A real-time load balancing algorithm is developed which reconfigures the power delivery network by combining the output of multiple buck converters when the workload demand exceeds the peak current rating. Simulation results for the proposed power delivery method indicate up to a 44% reduction in the energy consumption of the CMP system. In addition, the on-chip footprint of the power delivery network, including the on-chip voltage regulators and the switching network, is reduced by at least 23%.

In this paper, a power delivery system for homogeneous chip multi-processor (CMP) systems is proposed. The power delivery system is modified at run time by clustering multiple on-chip voltage regulators (OCVR) depending on the power demand of the workload. The OCVRs are designed to deliver up to the average current requirement of the typical workloads executed on the CMP platform. When the current demand of a core cluster exceeds the average value, the output of multiple OCVRs is combined through a high-speed s witch network to provide the necessary current. Two OCVR topologies (Buck and LDO) are analyzed to characterize the impact on the characteristics of the voltage regulator as the peak load current is reduced. Simulation results for run-time OCVR clustering indicate a 36% reduction in the energy consumption of the system at an average load current with improvement in the load regulation. In addition, the area occupied by the OCVRs is reduced by at least 70%.

Checkpointing with rollback recovery is a well-established technique to tolerate transient faults. However, it incurs significant time and energy overheads, which go wasted in fault-free execution states and may not even be feasible in hard real-time systems. This paper presents a low-overhead two-state checkpointing (TsCp) scheme for fault-tolerant hard real-time systems. It differentiates between the fault-free and faulty execution states and leverages two types of checkpoint intervals for these two different states. The first type is nonuniform intervals that are used while no fault has occurred. These intervals are determined based on postponing checkpoint insertions in fault-free states, with the aim of decreasing the number of checkpoint insertions. The second type is uniform intervals that are used from the time when the first fault occurs. They are determined so as to minimize execution time for faulty states, leaving more time available for energy management in fault-free states. Experimental evaluation on an embedded processor (LEON3) and an emerging nonvolatile memory technology (ReRAM) illustrates that TsCp significantly reduces the number of checkpoints (62% on average) compared with previous works, while preserving fault tolerance. This results in 14% and 13% reduced execution time and energy consumption, respectively. Furthermore, we combine TsCp with dynamic voltage scaling (DVS) and achieve up to 26% (21% on average) energy saving compared with the state-of-the-art techniques.

Heterogeneous architectures have emerged as a promising solution to enhance energy-efficiency by allowing each application to run on a core that matches resource needs more closely than a one-size-fits-all core. In this paper, an ElasticCore platform is described where core resources along with the operating voltage and frequency settings are scaled to match the application behavior at run-time. Furthermore, a linear regression model for power and performance prediction is used to guide the scaling of the core size and the operating voltage and frequency to maximize efficiency. Circuit considerations that further optimize the power efficiency of ElasticCore are also considered. Specifically, the efficiency of both off-chip and on-chip voltage regulators is analyzed for the heterogeneous architecture where the required load current changes dynamically at run-time. A distributed on-chip voltage regulator topology is proposed to accommodate the heterogeneous nature of the ElasticCore. The results indicate that ElasticCore on average achieves close to a 96% efficiency as compared to an architecture implementing the Oracle predictor where the application behavior is perfectly matched at run-time. Moreover, the proposed architecture is 30% more energy-efficient as compared to the BigLitte architecture.

High fabrication cost per bit and thermal issues are the main reasons that prevent architects from using 3D-DRAM alone as the main memory. In this paper we address this issue by proposing a heterogeneous memory system that combines a DDRx DRAM with an emerging 3D hybrid memory cube (HMC) technology. Bandwidth and temperature management are the challenging issues for such heterogeneous memory architecture. To address these challenges, first we introduce a memory page allocation policy for the heterogeneous memory system to maximize performance. Then, using the proposed memory page allocation policy, we propose a temperature-aware algorithm that adaptively distributes the requested bandwidth between HMC and DDRx DRAM to reduce the thermal hotspot while maintaining high performance. The results show that the proposed memory page allocation policy can utilize the memory bandwidth close to 99% of the ideal bandwidth utilization. Moreover our temperate-aware bandwidth adaptation reduces the average steady-state temperature of the HMC hotspot across various workloads by 4.5oK while incurring 2.5% performance overhead.

Wide I/O, the recent JEDEC DRAM standard, has created an opportunity for architects to overcome the "memory wall" challenge. 2.5D/3D integration enables Wide-IO to deliver high memory bandwidth and low latency for mobile applications. On the other hand, LPDDR3 was introduced to mainly address the power budget challenge in embedded MPSoCs. Employing either Wide I/O or LPDDR3 leads to different power, performance and thermal behavior of the system that necessitates a thorough analysis of both technologies. In this paper, we present a comprehensive analysis of the latency, performance, power and thermal behavior of these two emerging DRAM technologies in embedded MPSoCs. We conduct a comprehensive study to understand the impact of core count, core micro-architecture, and core-memory integration technology on the trade-offs that these two memory technologies offer. We show that while stacked Wide I/O outperforms LPDDR3 by as much as 7%, it increases the power consumption by 14%. To improve the power efficiency, we evaluate stacked LPDDR3, a DRAM design that is as high performance as Wide I/O - yet it is as power efficient as LPDDR3.

In the recent years, many-core platforms have emerged to boost performance while meeting tight power constraints. Per-core Dynamic Voltage and Frequency Scaling (DVFS) maximizes energy savings and meets the performance requirements of a given workload. Given a limited number of I/O pins and the need for finer control of voltage and frequency settings per core, there is a substantial cost in using off-chip voltage regulators. Consequently, there has been increased attention on the use of on-chip voltage regulators (OCVR) in many-core systems. However, integrating OCVRs comes at a cost of reduced power conversion efficiency (PCE) and increased complexity in the power delivery network and management of the OCVRs. In this paper, the effect of PCE on the thread-to-core mapping algorithm is investigated and the importance of the PCE-aware mapping scheme to optimize energy-efficiency is highlighted. Based on the results, up to 38% more energy savings is achieved as compared to PCE-agnostic algorithms. Moreover, the impact of core clustering granularity and process variation on the total efficiency of the system is explored. When relaxing the energy constraints by just 10%, an effective mapping reduces the complexity of the power delivery system by allowing the use of a significantly smaller number of voltage regulators, as compared to per-core OCVR. The results provided in the paper indicate an important opportunity for system and circuit co-design to implement energy-efficient and complexity-effective platforms for a target workload.

Many-core processors facilitate coarse-grained reliability by exploiting available cores for redundant multithreading. However, ensuring high reliability with reduced power consumption necessitates joint considerations of variations in vulnerability, performance and power properties of software as well as the underlying hardware. In this paper, we propose a power-efficient reliability management system for many-core processors. It exploits various basic redundancy techniques (like, dual and triple modular redundancy) operating in different voltage-frequency levels, each offering distinct reliability, performance and power properties. Our system performs Dynamic Redundancy and Voltage Scaling (DRVS) considering process variations in hardware, and diversities in software vulnerability and execution time properties. Experiments show that DRVS system provides significant reliability improvements while providing up to 60% reduced power consumption compared to state-of-the-art techniques.

Future computing platforms will need to be flexible, scalable, and power-conservative, while saving size, weight, energy, etc. Heterogeneous architecture can address these challenges by allowing each application to run on a core that matches resource needs more closely than a one-size-fits-all core. Dynamic heterogeneous architectures can extend these benefits further, allowing the system to construct the right core at run-time for each application, borrowing or freeing resources only as needed by the particular application that is running. The key insight in the described design is that 3D stacking of cores eliminates the fundamental barrier to dynamic heterogeneity, allowing various resources belonging to different cores to be shared at run-time with minimal overhead.

The variability of deep-submicron technologies creates systems with asymmetric cores from a frequency and leakage power viewpoint, which makes an opportunity for performance-power optimization. In particular, process variation can transform a homogeneous many-core platform into a heterogeneous system where the task mapping is NP-hard problem. In this paper, we propose a mapping algorithm that selects the appropriate task mapping along with voltage and frequency assignment for a cluster of cores. The mapping algorithm, which is based on simulated annealing, determines cluster voltages and core frequencies to minimize energy consumption and EDP under process variation. We examine the effectiveness of our proposed algorithm on a fully placed and routed 128-core biomedical accelerator in 45 nm when running various applications including compressive sensing, seizure detection and ultrasound spectral Doppler and linear regression. The results indicate that exposing frequency and power variation to the mapping algorithm results in up to 22% (on average 11%) energy saving and 31% (on average19%) EDP improvement.

Standby-sparing is one of the common techniques in order to design fault-tolerant safety-critical systems where the high level of reliability is needed. Recently, the minimization of energy consumption in embedded systems has attracted a lot of concerns. Simultaneous considering of high reliability and low energy consumption by DVS is a challenging problem in designing such a system, since using DVS has been shown to reduce the reliability profoundly. In this article, we have studied different schemes of standby-sparing systems from the energy consumption and reliability point of view. Moreover, we propose a new standby-sparing scheme which addresses both reliability and energy consumption jointly together. This scheme uses a simple energy management coupled with an online task scheduler which tries to dispatch those ready tasks which are expected to lead to high reliability and low energy consumption in the system. The effectiveness of the proposed scheme has been shown on TGFF under stochastic workloads. The results show 52% improvement on energy saving compared to the conventional hot standby-sparing system. Moreover, two orders of magnitude higher reliability is obtained on average, while preserving the same level of energy saving as compared to the state-of-the-art low-energy standby-sparing system (LESS).

This paper proposes a nanophotonic Network-on-Chip architecture based on the traditional Cube-Connected Cycles topology (CCC), which is named as ONC3. We also suggest a contention-free quasi-Dimension-Order-Routing algorithm for the proposed structure. Compared to the previous 2D layouts, our novel scheme lessens the crosstalk parameter of the insertion loss and consequently, the power consumption. Besides, the router structure is area-efficient. On the other hand, optical destination checking supersedes electrical resource reservation, with utilizing passive wavelength routing method and Wavelength Division Multiplexing scheme, simultaneously. The efficiency of the proposed architecture, in terms of the essential NoC parameters such as delay and power consumption, is investigated through simulation-based experiments and is compared with electrical CCC and Torus topologies, as well as the existing ONoCs such as the optical crossbar and ?-router in various synthetic traffic patterns.

The interaction between fault tolerance and energy consumption is an interesting avenue in the realm of designing embedded systems. In this paper, a scheme for reducing energy consumption in conventional standby-sparing systems is introduced. In the proposed method, the primary unit exploits dynamic voltage scaling (DVS) and dynamic power management (DPM) is employed for the spare unit. The framework which is used in the primary unit is composed of a feedback system to follow up workload along with a three-layer yet light-weight energy manager which guarantees hard real-time constraints of the system. Moreover, an optimal approach (but not practical) as a margin for the minimum energy consumption of this system is presented and the capability of other methods in reducing energy consumption is compared. Simulation results show an improvement in energy saving as compared with previous works and also show that the proposed method is near optimal for task sets with different dynamic workloads.