Tiny devices have become ubiquitous in people's daily lives. Their applications dictate tight energy budgets, but also require reasonable performance to meet user expectations. To this end, the hardware of tiny devices has been highly optimized, making further optimizations difficult. In this work, we identify a missed opportunity: the bitwidth selection of program variables. Today's compilers directly translate the bitwidth specified in the source code to the binary. However, we observe that most variables do not utilize the full bitwidth specified in the source code for the majority of execution. To leverage this opportunity, we propose BitSpec : a system that performs fine-grained speculation on the bitwidth of program variables. BitSpec is implemented as a compiler-architecture co-design, where the compiler transparently reduces the bitwidth of program variables to their expected needs and the hardware monitors speculative variables, reporting misspeculation to the software, which re-executes at the original bitwidth, ensuring correctness. BitSpec reduces energy consumption by 9.9% on average, up to 28.2% .

In deep pipelines, cache accesses must be split over many cycles to accommodate high-clock rates. Given the variable memory access times, the volume of in-flight cache operations, and potential for memory data hazards, implementing the control logic necessary to provide correct operation and maximize performance emerges as a major design challenge. This paper introduces Forecast, a tool for synthesizing deep microprocessor pipelines with support for high-performance cache organizations. Given a functional description of a processor datapath and directives on how to split logic and state accesses into stages, Forecast automatically generates the complete pipeline datapath and fully functional control logic to correctly sequence operations across pipeline stages while maintaining correct operational semantics. Forecast produces logic to enable speculatively scheduled pipelined cache accesses, support buffered stores, and detect and resolve dependencies in the memory system. This tool allows designers to construct deeper pipelined processors with larger capacity and higher throughput caches, improving the performance of these automatically generated processors.

Machine learning inference has become an essential task for embedded edge devices requiring the deployment of costly deep neural network accelerators onto extremely resource-constrained hardware. Although many optimization strategies have been proposed to improve the efficiency of standalone accelerators, the optimization for end-to-end performance of a computing device with heterogeneous cores is still challenging and often overlooked, especially for low power devices. In this paper, we propose a unified reconfigurable architecture, referred as Neural CPU (NCPU), for low-cost embedded systems. The proposed architecture is built on a binary neural network accelerator with the capability to emulate an in-order RISC-V CPU pipeline. The NCPU supports flexible programmability of RISC-V and maintains data locally to avoid costly core-to-core data transfer. A two-core NCPU SoC is designed and fabricated in a 65nm CMOS process. Compared with the conventional heterogeneous architecture, a single NCPU achieves 35% area reduction and 12% energy saving at 0.4V, which is suitable for low power and low-cost embedded edge devices. The NCPU design also features the capability of smooth switching between general-purpose CPU operation and a binary neural network inference to realize full utilization of the cores. The implemented two-core NCPU SoC achieves an end-to-end performance speed-up of 43% or an equivalent 74% energy saving based on use cases of real-time image classification and motion detection.

Dynamic timing slack has emerged as a compelling opportunity for eliminating inefficiency in ultra-low power embedded systems. This slack arises when all the signals have propagated through logic paths well in advance of the clock signal. When it is properly identified, the system can exploit this unused cycle time for energy savings. In this paper, we describe compiler and architecture co-design that opens new opportunities for timing slack that are otherwise impossible. Through cross-layer optimization, we introduce novel mechanisms in the hardware and in the compiler that work together to improve the benefit of circuit-level timing speculation by effectively squeezing time during execution. This approach is particularly well-suited to tiny embedded devices. Our evaluation on a gate-level model of a complete processor shows that our co-design saves (on average) 40.5% of the original energy consumption (additional 16.5% compared to the existing clock scheduling technique) across 13 workloads while retaining transparency to developers.

Despite the significant promise that circuit-level timing speculation has for enabling operation in marginal conditions, overheads associated with recovery prove to be a serious drawback. We show that fine-grained clock adjustment guided by the compiler can be used to stretch and shrink the clock to maximize benefits of timing speculation and reduce the overheads associated with recovery. We present a formulation for compiler-driven clock scheduling and explore the benefits in several scenarios. Our results show that there are significant opportunities to exploit timing slack when there are appropriate channels for the compiler to select clock period at cycle-level.

In this paper, a novel Greybox design methodology is proposed to establish a design and co-optimization flow across the boundary of conventional software and hardware design. The dynamic timing of each software instruction is simulated and associated with processor hardware design, which provides the basis of ultra-dynamic clock management. The proposed scheme effectively implements the instruction-based clock management and achieves 21.71% frequency speedup. Besides, a novel program-driven hardware optimization flow is proposed, in which software operations are mapped with hardware gate netlist and sorted by the usage frequency. The experiments on an ARM based pipeline design in commercial 65nm CMOS process show an extra 10% frequency speedup is obtained with high optimization efficiency. Overall, the proposed Greybox design method achieves frequency speedup by 31.56%, comparing with conventional design method.

This paper explores a novel associative low power operation where instructions govern the operation of on-chip regulators in real time. Based on explicit association between long delay instruction patterns and hardware performance, an instruction based power management scheme is developed with energy models formulated for deriving the energy efficiency of the associative operation. The proposed system scheme is demonstrated using a low power microprocessor design with an integrated switched capacitor regulator in 45nm CMOS technology. Simulations on benchmark programs show a power saving of around 14% from the proposed scheme. A novel compiler optimization strategy is also proposed to further improve the energy efficiency.

Voltage scaling is one of the most effective techniques for providing power savings on a chip-wide basis. However, reducing supply voltage in the presence of process variation introduces significant reliability challenges for large SRAM arrays. In this work, we demonstrate that the emergence of SRAM failures in delay sensitive L1 caches presents significant impediments to voltage scaling. We show that increases in the L1 cache latency would have a detrimental impact on a processor's performance and power consumption at aggressively scaled voltages. We propose techniques for L1 instruction/data caches to enable deep voltage scaling without compromising the L1 cache latency. For the data cache, we employ fault-free windows to adaptively hold the likely accessed data using the fault-free words within each cache line. For the instruction cache, we avoid the addresses that map to defective words by relocating basic blocks. During high voltage operation, both L1 caches have full capability to support high-performance. During low voltage operation, our schemes reduce Vccmin below 400mV. Compared to a conventional cache with a Vccmin of 760mV, we reduce the energy per instruction by 64%.

Voltage regulation circuity can be removed from embedded systems, commonly saving 30% of the printed circuit board area. Most unregulated systems suffer from performance degradation or premature failure as battery voltage decreases. Researchers have described a technique called power deregulation, in which performance is preserved by activating processor cores as battery voltage decreases. Unfortunately, the operating lifespans of such systems may be limited due to mismatches between the battery discharge profiles and the voltage-performance characteristics of processors. This paper provides a technique for codesign of power deregulated systems in which battery, processor, and workload characteristics are jointly considered. This technique extends lifespans of deregulated embedded systems. We also describe new battery and system performance and power models, some of which are based on a recently commercialized silver-zinc battery technology that is well suited to power deregulated systems. These models support lifetime simulation of regulated and deregulated embedded systems. If it is possible to find an appropriate match between embedded system and battery characteristics, the proposed design technique results in lifetimes similar to those of regulated systems and eliminates the need for bulky power regulation components.

While transistor performance and energy efficiency have dramatically improved in recent years, electrical interconnect improvements has failed to keep pace. Recent advances in nanophotonic fabrication have made on-chip optics an attractive alternative. However, system integration challenges remain. In particular, the parameters of on-chip nanophotonic structures are sensitive to fabrication-induced process variation and run-time spatial thermal variation across the die. This work addresses the performance and reliability challenges that arise from this sensitivity to variation. The paper first presents a model predicting the system-level effects of thermal and process variation in nanophotonic networks. It then shows how to optimize many-core system performance and reliability by using run-time techniques to compensate for the thermal and process variation effects.

By adjusting the design of the ISA and enabling circuit timing-sensitive optimizations in a compiler, we can more effectively exploit timing speculation. While there has been growing interest in systems that leverage circuit-level timing speculation to improve the performance and power-efficiency of processors, most of the innovation has been at the microarchitectural level. We make the observation that some code sequences place greater demand on circuit timing deadlines than others. Furthermore, by selectively replacing these codes with instruction sequences which are semantically equivalent but reduce activity on timing critical circuit paths, we can trigger fewer timing errors and hence reduce recovery costs.

This paper addresses the problem of efficient and effective parameter variation modeling and sampling in computer architecture simulations. While there has been substantial progress in accelerating simulation time for circuit designs subject to manufacturing variations, these approaches are not generally suitable for architectural studies. Toward this we investigated two complementary avenues: (1) adapting low-discrepancy sampling methods for use in Monte Carlo architectural simulations. We apply techniques previously developed for gate-level circuit models to higher level component models and in so doing drastically reduce the number of samples needed for detailed simulation; (2) applying multi-resolution analysis to appropriately decompose geometric regions of a chip, and achieve more effective description of parameter variations without increasing computational complexity. Our experimental results demonstrate that the combined techniques can reduce the number of Monte Carlo trials by a factor of 3.3, maintaining the same accuracy while significantly reducing the overall simulation run-time.

Circuit-level timing speculation has been proposed as a technique to reduce dependence on design margins and eliminating power/performance overheads. Recent work has proposed microarchitectural methods to dynamically detect and recover from timing errors in processor logic. To a large extent existing work has relied on statistical error models and has not evaluated potential disparity of error rates at the level of static instructions. In this paper, we analyze gate-level hardware models for an execution pipeline and demonstrate pronounced locality in instruction-level error rates due to value locality and data dependences. We propose timing error prediction to dynamically anticipate timing errors at the instruction-level and error padding techniques to avoid the full recovery cost of timing errors. We show that with simple prediction strategies our mechanism can reduce 80% of the performance penalty incurred by error recovery on average. This allows us to alleviate some limitations of timing speculation and improves energy-efficiency by 21% when compared to baseline timing speculation techniques using the same dynamic adaptive tuning mechanism.

Within-die variation in leakage power consumption is substantial and increasing for chip-level multiprocessors (CMPs) and multiprocessor systems-on-chip. Dealing with this problem via conservative assumptions is sub-optimal. Instead, operating systems may adapt task assignment and power management decisions to the variable characteristics of cores, improving system-wide power consumption and performance. Researchers have proposed such adaptation techniques. However, they rely on knowledge of CMP process variation (PV) maps. These maps are not provided by processor vendors, providing them would impose additional cost during the testing process, and static maps would not permit adaptation to aging effects. Further progress on developing and validating PV aware control techniques for CMPs requires access to PV maps for real processors. We present an online technique to extract the PV maps of CMPs. Potentially automatic temperature measurements with built-in on-die sensors during the execution of characterization workloads are used to determine variation in leakage power consumption. The proposed technique is applied to real CMPs, and the resulting PV maps are used within a PV aware task assignment and scheduling algorithm.

The emergence of power as a first-class design constraint has fueled the proposal of a growing number of run-time power optimizations. Many of these optimizations trade-off power saving opportunity for a variable performance loss which depends on application characteristics and program phase. Furthermore, the potential benefits of these optimizations are sometimes non-additive, and it can be difficult to identify which combinations of these optimizations to apply. Trial-and-error approaches have been proposed to adaptively tune a processor. However, in a chip multiprocessor, the cost of individually configuring each core under a wide range of optimizations might be prohibitive under simple trial-and-error approaches.

Three-dimensional integration has the potential to improve the communication latency and integration density of chip-level multiprocessors (CMPs). However, the stacked high-power density layers of 3D CMPs increase the importance and difficulty of thermal management. In this paper, we investigate the 3D CMP run-time thermal management problem and describe efficient management techniques. This paper makes the following main contributions: 1) It identifies and describes the critical concepts required for optimal thermal management, namely the methods by which heterogeneity in both workload power characteristics and processor core thermal characteristics should be exploited; and 2) it proposes an efficient proactive continuously engaged hardware and operating system thermal management technique governed by optimal thermal management polices. The proposed technique is evaluated using multiprogrammed and multithreaded benchmarks in an integrated power, performance, and temperature full-system simulation environment. We find that proactive power-thermal budgeting allows a 30% improvement in instruction throughput compared to a proactive thermal management approach that bases decisions only upon local information. The software components of the proposed thermal management technique have been implemented in the Linux 2.6.8 kernel. This source code will be publicly released. The analysis and technique developed in this paper provide a general solution for future 3D and 2D CMPs.

Recent efforts to address microprocessor power dissipation through aggressive supply voltage scaling and power management require that designers be increasingly cognizant of power supply variations. These variations, primarily due to fast changes in supply current, can be attributed to architectural gating events that reduce power dissipation. In order to study this problem, the authors propose a fine-grain, parameterizable model for power-delivery networks that allows system designers to study localized, on-chip supply fluctuations in high-performance microprocessors. Using this model, the authors analyze voltage variations in the context of next-generation chip-multiprocessor (CMP) architectures using both real applications and synthetic current traces. They find that the activity of distinct cores in CMPs present several new design challenges when considering power supply noise, and they describe potentially problematic activity sequences that are unique to CMP architectures

Parameter variation due to manufacturing error is an unavoidable consequence of technology scaling in future generations. The impact of random variation in physical factors such as gate length and interconnect spacing have a profound impact on not only performance of chips, but also their power behavior. While circuit-level techniques such as adaptive body-biasing can help to mitigate mal-fabricated chips, they cannot completely alleviate severe within die variations forecasted for near future designs. Despite the large impact that power variability have on future designs, there is a lack of published work that examines architectural implications of this phenomenon. In this work, we develop architecture level models that model power variability due to manufacturing error and examine its influence on multicore designs. We introduce VariPower, a tool for modeling power variability based on an microarchitectural description and floorplan of a chip. In particular, our models are based on layout level SPICE simulations and project power variability for different microarchitectural blocks using statistical analysis. Using VariPower: (1) we characterize power variability for multicore processors, (2) explore application sensitivity to power variability, and (3) examine clustering techniques that can appropriately classify groups of processors and chips that have similar variability characteristics

In a few technology generations, limitations of fabrication processes will make accurate design time power estimates a daunting challenge. Static leakage current which comprises a significant fraction of total power due to large on-chip caches, is exponentially dependent on widely varying physical parameters such as gate length, gate oxide thickness, and dopant ion concentration. In large structures like on-chip caches, this may mean that one portion of a cache may consume an order of magnitude larger static power than equivalently sized regions.Under this climate, egalitarian management of physical resources is clearly untenable. In this paper, we analyze the effects of within-die and die-to-die leakage variation for on-chip caches. We then propose way prioritization, a manufacturing variation aware scheme that minimizes cache leakage energy. Our results show that significant average power reductions are possible without undue hardware complexity or performance compromise.

As microprocessors become increasingly complex, the techniques used to analyze and predict their behavior must become increasingly rigorous. We apply wavelet analysis techniques to the problem of dl/dt estimation and control in modern microprocessors. While prior work has considered Bayesian phase analysis, Markov analysis, and other techniques to characterize hardware and software behavior, we know of no prior work using wavelets for characterizing computer systems. The dl/dt problem has been increasingly vexing in recent years, because of aggressive drops in supply voltage and increasingly large relative fluctuations in CPU current dissipation. Because the dl/dt problem has natural frequency dependence (it is worst in the mid-frequency range of roughly 50-200 MHz) it is natural to apply frequency-oriented techniques like wavelets to understand it. Our work proposes (i) an offline wavelet-based estimation technique that can accurately predict a benchmark's likelihood of causing voltage emergencies, and (ii) an online wavelet-based control technique that uses key wavelet coefficients to predict and avert impending voltage emergencies. The offline estimation technique works with roughly 0.94% error. The online control technique reduces false positives in dl/dt prediction, allowing, voltage control to occur with less than 2.5% performance overhead on the SPEC benchmark suite.

Performance and power analysis in modern processors requires managing a large amount of complex information across many time-scales. For a example, thermal control issues are a power subproblem with relevant time constants of millions of cycles or more, while the so-called dI/dT problem is also a power subproblem but occurs because of current variability on a much finer granularity: tens to hundreds of cycles. Likewise, for performance issues, program phase analysis for selecting simulation regions requires looking for periodicity on the order of millions of cycles or more, while some aspects of cache performance optimization requires understanding repetitive patterns on much finer granularities. Fourier analysis allows one to transform waveform into a sum of component (usually sinusoidal) waveforms in the frequency domain; in this way, the waveform's fundamental frequencies (periodicities of repetition) can be clearly identified. This paper shows how one can use Fourier analysis to produce frequency spectra for some of the time waveforms seen in processor execution. By working in the frequency domain, one can easily identify key application tendencies. For example, we demonstrate how to use spectral analysis to characterize the power behavior of real programs. As we show, this is useful for understanding both the temperature profile of a program and its voltage stability. These are particularly relevant issues for architects since thermal concerns and the dI/dT problem have significant influence on processor design. Frequency analysis can also be used to examine program performance. In particular, it can also identify periodic occurrences of important microarchitectural events like cache misses. Overall, the paper demonstrates the value of using frequency analysis in different research efforts related to characterizing and optimizing application performance and power.

Increasing focus on power dissipation issues in current microprocessors has led to a host of proposals for clock gating and other power-saving techniques. While generally effective at reducing average power, many of these techniques have the undesired side-effect of increasing both the variability of power dissipation and the variability of current drawn by the processor This increase in current variability, often referred to as the dI/dt problem, can cause supply voltage fluctuations. Such voltage fluctuations lead to unreliable circuits if not addressed, and increasingly expensive chip packaging techniques are needed to mitigate them. This paper proposes and evaluates a methodology for augmenting packaging techniques for dI/dt with microarchitectural control mechanisms. We discuss the resonant frequencies most relevant to current microprocessor packages, produce and evaluate a "dI/dt stressmark" that exercises the system at its resonant frequency, and characterize the behavior of more mainstream applications. Based on these results plus evaluations of the impact of controller error and delay, our microarchitectural control proposals offer bounds on supply voltage fluctuations, with nearly negligible impact on performance and energy. With the ITRS roadmap predicting aggressive drops in supply voltage and power supply impedances in coming chip generations, novel voltage control techniques will be required to stay on track. Our microarchitectural dI/dt controllers represent a step in this direction.

ABSTRACT Power concerns are becoming increasingly pressing in highperformance processors. Building power-aware and even power-adaptive computer architectures requires being able to track power consumption and attribute energy consumption to the portions of the chip that are responsible for it. This paper presents the Castle project which aims to deduce the actual runtime power dissipated by different processor units on the CPU chip by leveraging existing hardware. Namely, we examine the use of hardware performance counters as proxies for power meters. We discuss which performance counters count power-relevant events, and how to estimate event counts for power-relevant events not well supported by current, commonly available performance counters. We also discuss sampling-based approaches for estimating signal transition activity within the processor. Overall, we find that these performance counters can be quite useful in providing good power apportionment estimates for programs as they run. 1.