While selecting the hyper-parameters of Neural Networks (NNs) has been so far treated as an art, the emergence of more complex, deeper architectures poses increasingly more challenges to designers and Machine Learning (ML) practitioners, especially when power and memory constraints need to be considered. In this work, we propose HyperPower, a framework that enables efficient Bayesian optimization and random search in the context of power- and memory-constrained hyperparameter optimization for NNs running on a given hardware platform. HyperPower is the first work (i) to show that power consumption can be used as a low-cost, a priori known constraint, and (ii) to propose predictive models for the power and memory of NNs executing on GPUs. Thanks to HyperPower, the number of function evaluations and the best test error achieved by a constraint-unaware method are reached up to 112.99× and 30.12× faster, respectively, while never considering invalid configurations. HyperPower significantly speeds up the hyper-parameter optimization, achieving up to 57.20× more function evaluations compared to constraint-unaware methods for a given time interval, effectively yielding significant accuracy improvements by up to 67.6%.

Recent breakthroughs in Machine Learning (ML) applications, and especially in Deep Learning (DL), have made DL models a key component in almost every modern computing system. The increased popularity of DL applications deployed on a wide-spectrum of platforms (from mobile devices to datacenters) have resulted in a plethora of design challenges related to the constraints introduced by the hardware itself. “What is the latency or energy cost for an inference made by a Deep Neural Network (DNN)?” “Is it possible to predict this latency or energy consumption before a model is even trained?” “If yes, how can machine learners take advantage of these models to design the hardware-optimal DNN for deployment?” From lengthening battery life of mobile devices to reducing the runtime requirements of DL models executing in the cloud, the answers to these questions have drawn significant attention. One cannot optimize what isn't properly modeled. Therefore, it is important to understand the hardware efficiency of DL models during serving for making an inference, before even training the model. This key observation has motivated the use of predictive models to capture the hardware performance or energy efficiency of ML applications. Furthermore, ML practitioners are currently challenged with the task of designing the DNN model, i.e., of tuning the hyper-parameters of the DNN architecture, while optimizing for both accuracy of the DL model and its hardware efficiency. Therefore, state-of-the-art methodologies have proposed hardware-aware hyper-parameter optimization techniques. In this paper, we provide a comprehensive assessment of state-of-the-art work and selected results on the hardware-aware modeling and optimization for ML applications. We also highlight several open questions that are poised to give rise to novel hardware-aware designs in the next few years, as DL applications continue to significantly impact associated hardware systems and platforms.

Energy and temperature are the main constraints for modern high-performance multicore systems. To save power or increase performance, dynamic voltage and frequency scaling (DVFS) is widely applied in literally all computing systems. As CMOS technology continues scaling, FinFET has recently become the common choice for multicore systems. In contrast with planar CMOS, FinFET is characterized by lower delay under higher temperatures in super-threshold voltage region, an effect called temperature effect inversion (TEI). This paper explores TEI-aware performance improvement and energy savings for multicore systems. Our experimental results show that on average 15.70% throughput improvement or 31.26% energy savings can be achieved in steady state by a TEI-aware DVFS policy over a TEI-agnostic one. By further investigation, multiple sweet spots (SSs) resulting from TEI effects are observed. Based on these SS operation regimes, this paper introduces fast algorithms which provide iso-power maximum performance or iso-performance minimum energy consumption. Experimental results confirm the effectiveness of the proposed approach by exhibiting a 45.9× -55.3× speedup when compared to state-of-the-art algorithms while losing only 0.22% or 0.68% in achieved performance or energy, respectively.

Power and thermal issues are the main constraints for high-performance multi-core systems. As the current technology of choice, FinFET is observed to have lower delay under higher temperature in super-threshold voltage region, an effect called temperature effect inversion (TEI). While it has been shown that system performance can be improved under power constraints, as technology aggressively scales down to sub-20nm nodes, thermal issues also emerge as important reliability concerns throughout the system lifetime. To the best of our knowledge, we are the first to provide a comprehensive evaluation of both TEI and aging effects on the performance and power of FinFET-based multi-core systems with multiple voltage/frequency levels. Our experimental results show that aging effects can be reduced by up to 53.59% by exploiting the TEI effect. Based on a combined multivariate objective for power and aging, this work proposes an aging-aware algorithm, dubbed AgingMin, to select the optimal TEI-aware voltage/frequency operation points for decelerating the aging effect. Experimental results show that AgingMin improves the classic 10-year system lifetime by an average of 1.61 years while introducing less than 1% power overhead when compared to existing state-of-the-art techniques.

Near-threshold computing has emerged as a promising solution to significantly increase the energy efficiency of next-generation multicore systems. This paper evaluates and analyzes the behavior of dynamic voltage and frequency scaling for multicore systems operating under extended range: including near-threshold, nominal, and turbo modes. We adapt the model selection technique from machine learning to determine the relationship between performance and power. The theoretical results show that the resulting models satisfy convexity, which efficiently determines the optimal voltage/frequency operating points for: 1) minimizing energy consumption under throughput constraints or 2) maximizing throughput under a given power budget. We validate our models on FinFET-based chip-multiprocessors. Considering process variations (PVs), experimental results show that at 30% PV levels, our proposed method: 1) reduces energy consumption by 31.09% at iso-performance condition and 2) increases throughput by 11.46% at iso-power when compared with variation-agnostic nominal case.

Energy and temperature are the main constraints for modern high-performance multi-core systems. To save power or increase performance, Dynamic Voltage and Frequency Scaling (DVFS) is widely applied in industry. As CMOS technology continues scaling, FinFET has recently become the common choice for multi-core systems. In contrast with planar CMOS, FinFET is observed to have lower delay under higher temperature in super-threshold voltage region, an effect called temperature effect inversion (TEI). Due to this effect, performance can be further improved under power constraints. This work explores TEI-aware performance improvement for power limited multi-core systems. Our experimental results show that on average 15.70% throughput improvement can be achieved in steady state by a TEI-aware DVFS policy over a TEI-agnostic one. In further investigations, we observe multiple sweet spots in the operating voltage/frequency regions resulting from TEI effects. Based on these sweet spot operation regimes, this work introduces a fast algorithm which determines the maximum performance under power constraints. Experimental results confirm its effectiveness by exhibiting a speedup of an average of 45.9X in runtime while keeping resulting performance only 0.22% away from existing state-of-the-art algorithms.