Deep Neural Networks (DNNs) impose great challenges on resource-constraint embedded devices since they employ billions of computational operations. To satisfy these computational demands such devices utilize hardware accelerators, which can lead to increased power consumption whatsoever. To that end, the compression of DNNs to lower precision has been proposed, in order to achieve savings in energy consumption at the cost of some accuracy loss during inference. However, DNNs do not behave similarly under lower-precision execution and the accuracy degradation can be severe. In this work, we utilize the notion of image difficulty and explore how we can change DNN precision during inference to achieve gains in energy consumption without big drops in accuracy. We evaluate our work on the ImageNet dataset and show how the proposed framework achieves energy savings at run-time.

Embedded devices struggle with the heavy computational demands of extensive neural network models, a problem partially addressed by integrating accelerators with numerous multiply-accumulate units. However, this solution increases energy consumption. While using approximate circuits in accelerators can lower energy usage, it compromises accuracy and raises concerns about maintaining fair inference across diverse populations, particularly in the medical field. This work leverages approximate multipliers in deep neural networks to retain fairness and reduce energy consumption while keeping the inference accuracy within strict thresholds.

Deep Neural Networks (DNNs) are utilized in a variety of domains, and their computation intensity is stressing embedded devices that comprise limited power budgets. DNN compression has been employed to achieve gains in energy consumption on embedded devices at the cost of accuracy loss. Compression-induced accuracy degradation is addressed through fine-tuning or retraining, which can not always be feasible. Additionally, state-of-art approaches compress DNNs with respect to the average accuracy achieved during inference, which can be a misleading evaluation metric. In this work, we explore more fine-grain properties of DNN inference accuracy, and generate energy-efficient DNNs using signal temporal logic and falsification jointly through pruning and quantization. We offer the ability to control at run-time the quality of the DNN inference, and propose an automated framework that can generate compressed DNNs that satisfy tight fine-grain accuracy requirements. The conducted evaluation on the ImageNet dataset has shown over 30% in energy consumption gains when compared to baseline DNNs.

Reconfigurable approximate multipliers have been proposed as a way to improve the energy efficiency of neural network inference. However, selecting the optimal combination of approximate modes is a challenging problem due to the tradeoff between energy savings and accuracy loss. In this paper, we propose a methodology for selecting the best triad of approximate multipliers to form a reconfigurable approximate multiplier that can satisfy a maximum accuracy drop threshold and achieve the highest possible energy savings. We use formal methods to produce a Pareto-front of solutions that satisfy the accuracy constraint and maximize energy savings. Experimental results show that our methodology can achieve significant gains in energy with negligible drops in accuracy when compared to the baseline.

Modern applications are being increasingly demanding in terms of computing capabilities, and high performance is required at all times. Chip multiprocessors (CMPs) comprise multiple cores and have been widely employed to address this demand. However, the cores of a CMP share several components of the memory hierarchy for which concurrent executing applications compete to access at run-time. This contention can lead to severe performance loss and has a different impact on each application, resulting in potential starvation for selected applications. Thus, there is a need for a scheduling policy to efficiently address this contention-induced unfairness. In this work, we utilize matrix reconstruction techniques to enhance scheduling decisions at run-time, ensuring the fair and efficient execution of any given application workload. Our evaluation shows that our proposed scheduling policy can achieve up to 25.8% gains in fairness when compared to the Linux completely fair scheduler, and up to 6.1% when compared to another state-of-the-art approach, without inflicting performance degradation.

Recent breakthroughs in Neural Networks (NNs) have made DNN accelerators ubiquitous and led to an ever-increasing quest on adopting them from Cloud to edge computing. However, state-of-the-art DNN accelerators pack immense computational power in a relatively confined area, inducing significant on-chip power densities that lead to intolerable thermal bottlenecks. Existing state of the art focuses on using approximate multipliers only to trade-off efficiency with inference accuracy. In this work, we present a thermal-aware approximate DNN accelerator design in which we additionally trade-off approximation with temperature effects towards designing DNN accelerators that satisfy tight temperature constraints. Using commercial multi-physics tool flows for heat simulations, we demonstrate how our thermal-aware approximate design reduces the temperature from 139$^{\circ }$C, in an accurate circuit, down to 79$^{\circ }$C. This enables DNN accelerators to fulfill tight thermal constraints, while still maximizing the performance and reducing the energy by around 75% with a negligible accuracy loss of merely 0.44% on average for a wide range of NN models. Furthermore, using physics-based transistor aging models, we demonstrate how reductions in voltage and temperature obtained by our approximate design considerably improve the circuit’s reliability. Our approximate design exhibits around 40% less aging-induced degradation compared to the baseline design.

Deep neural networks (DNNs) are being heavily utilized in modern applications, putting energy-constraint devices to the test. To bypass high energy consumption issues, approximate computing has been employed in DNN accelerators to balance out the accuracy-energy reduction trade-off. However, the approximation-induced accuracy loss can be very high and drastically degrade the performance of the DNN. Therefore, there is a need for a fine-grain mechanism that would assign specific DNN operations to approximation to maintain acceptable DNN accuracy, while achieving low energy consumption. We present an automated framework for weight-to-approximation mapping through formal property exploration for approximate DNN accelerators. At the MAC unit level, our experimental evaluation surpassed already energy-efficient mappings by more than$\times 2$in terms of energy gains, while supporting a fine-grain control over the introduced approximation.

In this work, we introduce a control variate approximation technique for low error approximate Deep Neural Network (DNN) accelerators. The control variate technique is used in Monte Carlo methods to achieve variance reduction. Our approach significantly decreases the induced error due to approximate multiplications in DNN inference, without requiring time-exhaustive retraining compared to state-of-the-art. Leveraging our control variate method, we use highly approximated multipliers to generate power-optimized DNN accelerators. Our experimental evaluation on six DNNs, for Cifar-10 and Cifar100 datasets, demonstrates that, compared to the accurate design, our control variate approximation achieves same performance and 24% power reduction for a merely 0.16% accuracy loss.

Modern embedded systems have adopted the clustered Chip Multi-Processor (CMP) paradigm in conjunction with dynamic frequency scaling techniques to improve application performance and power consumption. Nonetheless, modern applications are becoming more aggressive in terms of computational power. At the same time, the integration of multiple cores in the same cluster has resulted in significant increase of power consumption creating thermal hotspots. Conventional design approaches consider fixed power and temperature constraints, which are mostly extracted experimentally leading many times to pessimistic run-time decisions and performance losses. In this paper, we present a unified framework for efficient resource management of clustered CMPs by enabling formal property exploration and integrating robustness analysis. Specifically, we bridge the gap between run-time decisions and design-time exploration by using Parametric Signal Temporal Logic (PSTL) for mining the values of system constraints. Then, we utilize the extracted values to enhance the decisions of the run-time resource manager. Results on the Odroid-XU3 show that the proposed methodology offers more coarse- and fine-grain optimizations.

Recent Deep Neural Networks (DNNs) manage to deliver superhuman accuracy levels on many AI tasks. DNN accelerators are becoming integral components of modern systems-on-chips. DNNs perform millions of arithmetic operations per inference and DNN accelerators integrate thousands of multiply-accumulate units leading to increased energy requirements. To lower the energy consumption of DNN accelerators, approximate computing principles are employed. However, complex DNNs can be increasingly sensitive to approximation. In this work, we present a dynamically configurable approximate multiplier that supports three operation modes, i.e., exact, positive error, and negative error. In addition, we propose a filter-oriented approximation method to map the weights to the appropriate modes of the approximate multiplier. Our mapping algorithm balances the positive with the negative errors due to the approximate multiplications, aiming at maximizing the energy reduction while minimizing the overall convolution error. We evaluate our approach on multiple DNNs and datasets against state-of-the-art approaches, where our method achieves 18.33% energy gains on average across 7 NNs on 4 different datasets for a maximum accuracy drop of only 1%.