Supply voltage underscaling has been an effective approach to improve the energy-efficiency of modern high-performance processors, such as GPUs. However, energy efficiency and reliability are two sides of a trade-off. Undervolting will inevitably undermine reliability, since it reduces chip manufacturers’ voltage guardbands that is designed to ensure correct operations under worst-case scenarios. To achieve optimal energy efficiency while maintaining enough reliability, it is necessary to deeply understand the error characteristics caused by undervolting. Unlike previous works which focus mostly on program level, we perform the first comprehensive instruction-level voltage margin and error characteristics evaluation for GPU architectures. We systematically measure the error probability and patterns of GPU instructions during undervolting. Then, we also analyze the impact of locations (SMs, threads, and bits) and operand data values on the error characteristics. Based on our observations, we reduce the voltage to the minimum safe limit for different instructions which achieves 18.37% energy saving, and we further propose an error detection strategy which reduces the performance and energy overhead by 14.8% with negligible 0.01% degradation for error detection rate.

GPUs are widely used hardware acceleration platforms in many areas due to their great computational throughput. In the meanwhile, GPUs are vulnerable to transient hardware faults in the post-Moore era. Analyzing the error resilience of GPUs are critical for both hardware and software. Statistical fault injection approaches are commonly used for error resilience analysis, which are highly accurate but very time consuming. In this work, we propose GEREM, a first framework to speed up fault injection process so as to estimate the error resilience of GPU microarchitectures swiftly and precisely. We find early fault behaviors can be used to accurately predict the final outcomes of program execution. Based on this observation, we categorize the early behaviors of hardware faults into GPU Early Fault Manifestation models (EFMs). For data structures, EFMs are early propagation characteristics of faults, while for pipeline instructions, EFMs are heuristic properties of several instruction contexts. We further observe that EFMs are determined by static microarchitecture states, so we can capture them without actually simulating the program execution process under fault injections. Leveraging these observations, our GEREM framework first profiles the microarchitectural states related for EFMs at one time. It then injects faults into the profiled traces to immediately generate EFMs. For data storage structures, EFMs are directly used to predict final fault outcomes, while for pipeline instructions, machine learning is used for prediction. Evaluation results show GEREM precisely assesses the error resilience of GPU microarchitecture structures with$237\times$speedup on average comparing with traditional fault injections.

Convolutional neural network (CNN) accelerators are widely used in artificial intelligence applications, such as image recognitions, due to their superior computing performance. However, as manufacturing technology scales down, the shrinking of chip size and the increasing of integration density make CNN accelerators vulnerable to soft errors, which are key factors affecting the reliability of integrated circuits. For emerging CNN applications where reliability is critical (such as self-driving cars), visible faults in applications’ outputs caused by soft errors may lead to catastrophic consequences. Therefore, it is important to consider reliability into CNN accelerators’ architecture design. Recent modeling and fault injection approaches analyze the reliability features of CNN models, but none evaluate the soft error reliability of CNN accelerators from architecture level. The Architectural Vulnerability Factor (AVF) indicates the probability that a soft error results in faulty computation outcomes. The Architecturally Correct Execution (ACE) method analyzes AVF of structures by identifying ACE bits and calculating their residency time. Based on the traditional ACE-based AVF analysis model, we propose a quantitative approach saca-AVF to analyze the AVF of systolic array based CNN accelerators that perform homogeneous matrix arithmetic operations instead of various kinds of instructions. We explore the reliability features of CNN accelerators under different design choices at different levels leveraging saca-AVF. We also observe several reliability characteristics of CNN accelerator architecture. We believe our proposed saca-AVF and the observations we made are able to guide designers to discover the reliability hotspots of CNN accelerators and build highly reliable CNN accelerators.

Due to manufacturing process and yield constraints, scaling GPU performance via increasing chip area becomes difficult. In the meanwhile, the demand for high computational throughput is increasing for high performance computing applications. As an alternative, multichip module GPU (MCM-GPU) achieves performance scalability via integrating multiple GPU chip modules (GPMs) on the same package. However, large MCM-GPU systems are susceptible to voltage noise effects, which cause voltage instability during program execution and result in energy inefficiency. In this work, we first model and analyze the voltage noise of MCM-GPUs at architecture level in detail. We characterize the voltage noise distributions of MCM-GPUs at different levels and under various design parameters. We further propose two architecture level voltage noise mitigation approaches, including GPM aware mitigation (GAM) and droop magnitude aware smoothing (DMAS), that leverage the voltage noise characteristics of MCM-GPUs. Evaluation shows both techniques are effective in reducing the voltage droop magnitudes and achieve good energy savings with negligible performance degradation.

Convolutional neural network (CNN) accelerators are popular specialized platforms for efficient CNN processing. As semiconductor manufacturing technology scales down to nano scale, process variation dramatically affects the chip’s quality. Process variation causes delay variation within the chip due to transistor parameter differences. CNN accelerators adopt a large number of processing elements (PEs) for parallel computing, which are highly susceptible to process variation effects. Fast CNN processing desires consistent performance among PEs; otherwise the processing speed is limited by the slowest PE within the chip. In this work, we first quantitatively model and analyze the impact of process variation on CNN accelerators’ operating frequency. We further analyze the utilization of CNN accelerators and the characteristics of CNN models. We then leverage the PE underutilization to propose a sub-matrix reformation mechanism and leverage the pixel similarity of images to propose a weight transfer technique. Both techniques are able to tolerate the low-frequency PEs and achieve performance improvement at chip level. Furthermore, a novel resilience-aware mapping technique that exploits the diversity in the importance of weights is also proposed to improve the performance. Evaluation results show that our techniques are able to achieve significant processing speed improvement with negligible accuracy loss.

Multi-chip-module (MCM) based package-level integration, a technology of forming large logical processing systems with small chiplets, is promising in the post-Moore era. It achieves great performance scalability and yield improvement for GPUs, which are highly desired by emerging high performance computing applications. However, critical problems, such as energy overhead and voltage instability, introduced by MCM-GPU designs, must be carefully resolved. Modern processors adopt large voltage guardbands to tolerate worst-case variations, such as voltage noises, which is pessimistic for most of the execution time. Exploiting supply voltage guardbands are effective to improve the processor’s energy efficiency. Even though voltage noise is studied for traditional monolithic GPUs, its effect on MCM-GPUs is unexplored.In this work, we characterize voltage noise effects on MCM-GPU architectures in detail. We analyze the maximum voltage droops of different applications, the temporal and spatial voltage droop distributions, and location-aware power variations. Based on the observations of staggered voltage droops across GPU modules (GPMs), we propose a GPM-aware voltage noise mitigation technique. Evaluation shows our technique is able to reduce the worst-case voltage droop by 29.4% on average, and up to 38.1% with negligible performance degradation.

Dense SLAM is an important application on an embedded environment. However, embedded platforms usually fail to provide enough computation resources for high-accuracy real-time dense SLAM, even with high-parallelism architecture such as GPUs. To tackle this problem, one solution is to design proper approximation techniques for dense SLAM on embedded GPUs. In this work, we propose two novel approximation techniques, critical data identification and redundant branch elimination. We also analyze the error characteristics of the other two techniques—loop skipping and thread approximation. Then, we propose SLaPP, an online adaptive approximation controller, which aims to control the error to be under an acceptable threshold. The evaluation shows SLaPP can achieve 2.0× performance speedup and 30% energy saving on average compared to the case without approximation.

Micro aerial vehicles (MAVs) are popular in many intelligent robot areas nowadays. As an battery-powered vehicle, the energy-efficiency of MAVs becomes the bottleneck for its wide adoption. This issue exacerbates for autonomous flight MAVs, since inefficient action executions may results in flight mission failure. In this work, we address the energy-inefficiency of deep Q-network (DQN) based autonomous flight of MAVs via approximate execution. We first investigate data sensing during flight process and observe two consecutive steps tend to perceive consistent data. We further analyze the decision making characteristics of DQN algorithm, and find the margins between maximum and the second largest Q-value in different steps are usually symmetrically distributed between action changes. Leveraging these two characteristics, we propose to Approximately Execute the autonomous flight pROcessing of MAVs for energy-efficiency improvement (AERO). AERO is composed of two techniques of AERO-S and AERO-AP. AERO-S uses the sensed data from the previous step to approximately infer the current sensing data. AERO-AP uses the symmetry of margins between the maximum and the second largest Q-value to skip the decision makings of some steps. Our evaluations show in environments with no obstacle, AERO achieves relative improvement of time (RIT) of 24.28% and relative improvement of energy (RIE) of 18.75% with no reduction of success rate. For environments with obstacles, AERO achieves 19.11% RIT and 12.72% RIE with no reduction of success rate.

Convolutional Neural Networks (CNNs) are popular in artificial intelligence areas due to their high accuracy. Meanwhile, as manufacturing process technology scales, the probability of soft errors occurrence in computer systems increases, which causes reliability challenges for CNNs. For emerging safety and reliability critical CNN applications, such as autonomous vehicles, soft errors may cause catastrophic consequences. Thus it is important to analyze the vulnerability characteristics of CNNs. A common approach to analyze CNNs' reliability is fault injection, which requires a lot of computation resources and is time consuming. Thus a more efficient method is desired. In this work, we propose an analytical model named SERN to analyze the soft error reliability of CNNs, which requires only a small number of parameters in CNN models. Validation on several common CNN models shows SERN can efficiently and accurately perform soft error reliability analysis for CNNs. We also observe that the reliability of CNNs depend on data types, values, the sign of data and types of layers. Take advantage of these observations, we propose to protect vulnerable bits through ECC and protect error-prone layers through redundancy.

Rapidly evolving technologies and applications of mobile devices inevitably increase the power demands on the battery. However, the development of batteries can hardly keep pace with the fast-growing demands, leading to short battery life, which becomes the top complaints from customers. In this article, we investigate a novel energy supply technology, fuel cell (FC), and leverage its advantages of providing long-term energy storage to build a hybrid FC-battery power system. Therefore, mobile device operation time is dramatically extended, and users are no longer bothered by battery recharging. We examine real-world smartphone usage data and find that a naive hybrid power system cannot meet many users’ highly diversified power demands. We thus propose an OS-level power management policy that reduces the device power consumption for each power peak to solve this mismatch. This technique trades the quality-of-service (QoS) for a larger FC ratio in the system and thus much longer device operation time. We further observe that the user’s personality largely determines his/her satisfaction with the QoS degradation and the operation time extension. Thus, applying a hybrid system with fixed configuration (i.e., peak throttling level coupled with corresponding FC/battery ratio) fails to satisfy every user. We then explore customized hybrid system configuration based on each individual user’s personality to deliver the optimal satisfaction for him/her. The experimental results show that our personality-aware hybrid FC-battery solution can achieve 4× longer operation time and 25% higher satisfaction score compared to the common setting for state-of-the-art mobile devices.

This article presents a novel energy-efficient cache design for massively parallel, throughput-oriented architectures like GPUs. Unlike L1 data cache on modern GPUs, L2 cache shared by all of the streaming multiprocessors is not the primary performance bottleneck, but it does consume a large amount of chip energy. We observe that L2 cache is significantly underutilized by spending 95.6% of the time storing useless data. If such “dead time” on L2 is identified and reduced, L2’s energy efficiency can be drastically improved. Fortunately, we discover that the SIMT programming model of GPUs provides a unique feature among threads: instruction-level data locality similarity, which can be used to accurately predict the data re-reference counts at L2 cache block level. We propose a simple design that leverages this Lo cality S imilarity to build an energy-efficient GPU L2 Cache , named LoSCache . Specifically, LoSCache uses the data locality information from a small group of cooperative thread arrays to dynamically predict the L2-level data re-reference counts of the remaining cooperative thread arrays. After that, specific L2 cache lines can be powered off if they are predicted to be “dead” after certain accesses. Experimental results on a wide range of applications demonstrate that our proposed design can significantly reduce the L2 cache energy by an average of 64% with only 0.5% performance loss. In addition, LoSCache is cost effective, independent of the scheduling policies, and compatible with the state-of-the-art L1 cache designs for additional energy savings.

This paper presents a novel energy-efficient cache design for massively parallel, throughput-oriented architectures like GPUs. Unlike L1 data cache on modern GPUs, L2 cache shared by all the streaming multiprocessors is not the primary performance bottleneck but it does consume a large amount of chip energy. We observe that L2 cache is significantly under-utilized by spending 95.6% of the time storing useless data. If such "dead time" on L2 is identified and reduced, L2's energy efficiency can be drastically improved. Fortunately, we discover that the SIMT programming model of GPUs provides a unique feature among threads: instruction-level data locality similarity, which can be used to accurately predict the data re-reference counts at L2 cache block level. We propose a simple design that leverages this Locality Similarity to build an energy-efficient GPU L2 Cache, named LoSCache. Specifically, LoSCache uses the data locality information from a small group of CTAs to dynamically predict the L2-level data re-reference counts of the remaining CTAs. After that, specific L2 cache lines can be powered off if they are predicted to be "dead" after certain accesses. Experimental results on a wide range of applications demonstrate that our proposed design can significantly reduce the L2 cache energy by an average of 64% with only 0.5% performance loss.

Convolutional Neural Network (CNN) accelerators are popular specialized platforms for efficient CNN processing. As semiconductor manufacturing technology scales down to nano scale, process variation dramatically affects the chip's quality. Process variation causes delay variation within the chip due to transistor parameter differences. CNN accelerators adopt a large number of Processing Elements (PEs) for parallel computing, which are highly susceptible to process variation effects. Fast CNN processing desires consistent performance among PEs, otherwise the processing speed is limited by the slowest PE within the chip. In this work, we first quantitatively model and analyze the impact of process variation on CNN accelerator's operating frequency. We further analyze the utilization of CNN accelerator and the characteristics of CNN models. We then leverage the PE underutilization to propose a sub-matrix reformation mechanism and leverage the pixel similarity of images to propose a weight transfer technique. Both techniques are able to tolerate the low-frequency PEs, and achieve performance improvement at chip level. Evaluation results show our techniques are able to achieve significant processing speed improvement with negligible accuracy loss.

Graphics Processing Units (GPUs) are widely used in general-purpose high-performance computing fields due to their highly parallel architecture. In recent years, a new era with the nanometer scale integrated circuit manufacture process has come. As a consequence, GPUs’ computation capability gets even stronger. However, as process technology scales down, hardware variability, e.g., process variations (PVs) and negative bias temperature instability (NBTI), has a higher impact on the chip quality. The parallelism of GPU desires high consistency of hardware units on chip; otherwise, the worst unit will inevitably become the bottleneck. So the hardware variability becomes a pressing concern to further improve GPUs’ performance and lifetime, not only in integrated circuit fabrication, but more in GPU architecture design. Streaming Processors (SPs) are the key units in GPUs, which perform most of parallel computing operations. Therefore, in this work, we focus on mitigating the impact of hardware variability in GPU SPs. We first model and analyze SPs’ performance variations under hardware variability. Then, we observe that both PV and NBTI have a large impact on SPs’ performance. We further observe unbalanced SP utilization, e.g., some SPs are idle when others are active, during program execution. Leveraging this observation, we propose a Hardware Variability-aware SPs’ Management policy (HVSM), which dynamically dispatches computation in appropriate SPs to balance the utilizations. In addition, we find that a large portion of compute operations are duplicate. We also propose an Operation Compression (OC) technique to minimize the unnecessary computations to further mitigate the hardware variability effects. Our experimental results show the combined HVSM and OC technique effectively reduces the impact of hardware variability, which can translate to 37% performance improvement or 18.3% lifetime extension for a GPU chip.

GPUs are widely used in general-purpose high performance computing field due to their highly parallel architecture. In recent years, a new era with nanometer scale integrated circuit manufacture process has come, as a consequence, GPUs' computation capability gets even stronger. However, as process technology scales down, hardware variability, e.g., process variations (PVs) and negative bias temperature instability (NBTI), has a higher impact on the chip quality. The parallelism of GPU desires high consistency of hardware units on chip, otherwise, the worst unit will inevitably become the bottleneck. So the hardware variability becomes a pressing concern to further improve GPUs' performance and lifetime, not only in integrated circuit fabrication, but more in GPU architecture design. Streaming Processors (SPs) are the key units in GPUs, which perform most of parallel computing operations. Therefore, in this work, we focus on mitigating the impact of hardware variability in GPU SPs. We first model and analyze SPs' performance variations under hardware variability. Then, we observe that both PV and NBTI have large impact on SP's performance. We further observe unbalanced SP utilization, e.g., some SPs are idle when others are active, during program execution. Leveraging both observations, we propose a Hardware Variability-aware SPs' Management policy (HVSM), which dynamically prioritizes the fast SPs, regroups SPs in a two-level granularity and dispatches computation in appropriate SPs. Our experimental results show HVSM effectively reduces the impact of hardware variability, which can translate to 28% performance improvement or 14.4% lifetime extension for a GPU chip.

Mobile devices are quickly becoming the most widely used processors in consumer devices. Since their major power supply is battery, energy-efficient computing is highly desired. In this article, we focus on energy-efficient cache design in emerging mobile platforms. We observe that more than 40% of L2 cache accesses are OS kernel accesses in interactive smartphone applications. Such frequent kernel accesses cause serious interferences between the user and kernel blocks in the L2 cache, leading to unnecessary block replacements and high L2 cache miss rate. We first propose to statically partition the L2 cache into two separate segments, which can be accessed only by the user code and kernel code, respectively. Meanwhile, the overall size of the two segments is shrunk, which reduces the energy consumption while still maintaining the similar cache miss rate. We then find completely different access behaviors between the two separated kernel and user segments and explore the multi-retention STT-RAM-based user and kernel segments to obtain higher energy savings in this static partition-based cache design. Finally, we propose to dynamically partition the L2 cache into the user and kernel segments to minimize overall cache size. We also integrate the short-retention STT-RAM into this dynamic partition-based cache design for maximal energy savings. The experimental results show that our static technique reduces cache energy consumption by 75% with 2% performance loss, and our dynamic technique further shows strong capability to reduce cache energy consumption by 85% with only 3% performance loss.

Supply voltage reduction is an effective approach to significantly reduce GPU energy consumption. As the largest on-chip storage structure, the GPU register file becomes the reliability hotspot that prevents further supply voltage reduction below the safe limit ($V_{min}$) due to process variation effects. This work addresses the reliability challenge of the GPU register file at low supply voltages, which is an essential first step for aggressive supply voltage reduction of the entire GPU chip. To better understand the reliability issues posed by undervolting and its energy-saving potential, we first rigorously model and analyze the process variation impact on the GPU register file at different voltages. By further analyzing the GPU architecture, we make a key observation that the time GPU registers contain useless data (i.e., dead time) is long, providing a unique opportunity to enhance register reliability. We then propose GR-Guard, an architectural solution that leverages long register dead time to enable reliable operations from unreliable register file at low voltages. GR-Guard is both effective and low-cost, and does not affect normal (i.e., non-faulty) register accesses. Experimental results show that for a 28nm baseline GPU under aggressive voltage reduction, GR-Guard can maintain the register file reliability with less than 2\% overall performance degradation, while achieving an average of 31% energy reduction across various applications.

Delivering an excellent use experience to the customers is the top challenge faced by today's mobile device designers and producers. There have been multiple studies on achieving the good trade-offs between QoS and energy to enhance the user experience, however, they generally lack a comprehensive and accurate understanding of QoS, and ignore the fact that each individual user has his/her own preference between QoS and energy. In this study, we overcome these two drawbacks and propose a customized power management policy that dynamically configures the mobile platform to achieve the user-specific optimal QoS and energy trade-offs and hence, satisfy each individual mobile user. We first introduce a novel and comprehensive definition of QoS, and propose the accurate QoS measurement and management methodologies. We then observe that user's personality greatly determines his/her preferences between QoS and energy, and propose an online personality-guided user satisfaction prediction model based on the QoS and energy, guided by the user personality inferred from his/her device usage history. Our validation proves our model can achieve very high prediction accuracy. Finally, we propose our customized power management policy based on the prediction model for individual users. The experiment results show that our technique can improve the user experience by around 36% compared with the state-of-the-art power management policies.

Mobile users always require an excellent user experience which is the top challenge faced by today's mobile device designers and producers. Mobile devices are battery constrained, thus developing energy-saving techniques to extend the battery life is critical in terms of the user experience. Since the discrepancy between the device energy and battery energy consumption is becoming large when the battery is approaching to depleted, the battery energy-savings mechanisms (instead of the previously explored device energy-saving mechanisms) that target at the low battery level are highly desirable. Besides the battery life, mobile users demand a good system responsiveness, which is also an essential component in the user experience.