With shrinking device geometries, the device-to-device (D2D) variability of emerging memory devices, affect the performance of deep neural networks (DNN) and restrict the use of standalone devices to be used as Analog in-Memory Computing (AiMC) accelerators. For FeFET based AiMC designs, such effects can be controlled by using resistors or capacitors connected in series with FeFETs. However, in both cases, the effect of variability will again start dominating when the programming window shrinks further due to scaling towards smaller nanometer dimensions. Additionally, while resistance based solutions will add to the existing IR drop issues within the macro, capacitor based approach avoids such a case. In this paper, we evaluate a FeFET and capacitor based AiMC accelerator and study how finetuning cell operating parameters along with design modifications of the AiMC macro, can help in achieving a variability aware AiMC design. Using a 22-nm CMOS PDK augmented with FeFET model based on measured data, evaluation on a DNN with CIFAR-10 benchmark, demonstrated that following our approach, the implemented DNN achieves an accuracy of ∼85% which is an improvement of ∼13% over an untuned AiMC macro.

This paper presents a tiny charge injection-Successive Approximation (ci-SAR) A/D converter (ADC) to be integrated at the periphery of analog Matrix Vector Multiplication (MVM) accelerators for Deep Neural Network (DNN) inference. Derived from the ci-SAR ADC, this converter exploits a single charge injecting cell to minimize area and energy consumption. The ADC exhibits a signal-to-noise and distortion ratio of 30.5 dB, at 5 bits of nominal resolution. The energy per conversion is 86 fJ, running at 34 MS/s, with a silicon area of $75 \mu m ^{2}$, in 22 nm technology node. From the results of our analytical framework, an SRAM-based Analog in-Memory Compute (AiMC) array, including the proposed ADC at 5 bits of resolution, can achieve an energy efficiency of 1650 TOPs/W.

Large weight variations cause significant degradation in Deep Neural Networks (DNNs) accuracy in machine learning (ML) context. Indium-Gallium-Zinc-Oxide (IGZO) DRAM compute cell is a promising option for Analog in-Memory Computing (AiMC) accelerators, but its applicability requires the compensation of large variations affecting the voltage threshold of the readout device. This paper proposes a write-verify scheme for IGZO-based AiMC accelerator, designed in 22-nm technology, based on a compensation loop operating on the analog weight value stored in the IGZO DRAM cell. After the compensation routine, the IGZO IONnormalized variation drops from $\pm \mathbf{2 7} \%$ to $\pm 3 \%$ in simulation. With sufficiently large weight reuse, the additional energy spent for the compensation of the entire arrays becomes negligible, recovering the 2000 TOPS/W baseline performance of an ideal IGZO array without the write-verify.

Analog in Memory Computing (AiMC) based neural network acceleration is a promising solution to increase the energy efficiency of deep neural networks deployment. However, the quantization requirements of these analog systems are not compatible with state-of-the-art neural network quantization techniques. Indeed, while the quantization of the weights and activations is considered by modern deep neural network quantization techniques, AiMC accelerators also impose the quantization of each Matrix Vector Multiplication (MVM) result. In most demonstrated AiMC implementations, the quantization range of MVM results is considered a fixed parameter of the accelerator. This work demonstrates that dynamic control over this quantization range is possible but also desirable for analog neural networks acceleration. An AiMC compatible quantization flow coupled with a hardware aware quantization range driving technique is introduced to fully exploit these dynamic ranges. Using CIFAR-10 and ImageNet as benchmarks, the proposed solution results in networks that are both more accurate and more robust to the inherent vulnerability of analog circuits than fixed quantization range based approaches.

Analog in memory computing (AiMC) is a promising hardware solution to efficiently perform inference with deep neural networks (DNNs). Similar to digital DNN accelerators, AiMC systems benefit from aggressively quantized DNNs. In addition, AiMC systems also suffer from noise on activations and weights. Training strategies to condition DNNs against weight noise can increase the efficiency of AiMC systems by enabling the use of more compact but more noisy weight memory devices. In this work, we utilize noise-aware training and introduce gradual noise training and network width scaling to increase the tolerance of DNNs against weight noise. Our results show that noise-aware training and gradual noise training drastically lowers the impact of weight noise without changing the network size. By utilizing network width scaling, the weight noise tolerance is increased even more with the penalty of more network parameters.

Extremely energy-efficient convolutional neural network inference (CNN) is recently enabled by analog in-memory compute (AiMC). The integration of AiMC in a primarily digital inference system brings new challenges ranging from device specifications to defining novel system architectures. A novel framework to evaluate the impact of AiMC array at the system level is presented. The framework is used to model a SRAM-based 1024 × 512 prototype AiMC array, capable of energy efficiency of upto 675 TMACs/W. The proposed framework allows modelling different compute cells, array dimensions, operating voltage, and activation buffer energy models which can be used to determine the overall energy efficiency for various CNN workloads.

This paper proposes TEASE (Technology Exploration and Analysis for SoC-level Evaluation), a framework to systematically analyze and evaluate system design in finFET-based technology node. The proposed framework combines both lithography and electrical constraints of a particular technology node to optimize the standard cell library performance. Growing complexity of logic design at nodes below 20nm causes to adopt a design style that can embrace the simplicity required to enable manufacturing, along with a process technology that can be finely tuned to the desired performance constraints. Additionally, the introduction of finFET based devices poses a new challenge for the designers to come up with an efficient standard cell template. The proposed framework can be used to detect the technology constraints that act as the bottleneck for the enablement of design at these advanced nodes. Results presented in this paper show by optimizing these bottlenecks we can improve the performance of a standard cell library significantly. Furthermore, adapting to such an analysis framework at an early stage of technology development helps to take the design constraints into the decision loop for realization of technology research into real products.

The complexity and performance requirements of embedded software are continuously increasing, making Multiprocessor System-on-Chip (MPSoC) architectures more and more important in the domain of embedded and cyber-physical systems. Using multiple cores in a single system reduces problems concerning energy consumption, heat dissipation, and increases performance. Nevertheless, these benefits do not come for free. Porting existing, mostly sequential, applications to MPSoCs requires extracting efficient parallelism to utilize all available cores. Many embedded applications, like network services and multimedia tasks for voice-, image- and video processing, are operating on data streams and thus have a streaming-based structure. Despite the abundance of parallelism in streaming applications, it is a non-trivial task to split and efficiently map sequential applications to MPSoCs. Therefore, we present an algorithm which automatically extracts pipeline parallelism from sequential ANSI-C applications. The presented tool employs an integer linear programming (ILP) based approach enriched with an adequate cost model to automatically control the granularity of the parallelization. By applying our tool to real-life applications, it can be shown that our approach is able to speed up applications by a factor of up to 3.9x on a four-core MPSoC architecture, compared to a sequential execution.

As embedded systems are becoming the center of our digital life, system design becomes progressively harder. The integration of multiple features on devices with limited resources requires careful and exhaustive exploration of the design search space in order to efficiently map modern applications to an embedded multi-processor platform. The MNEMEE project [1] addresses this challenge by offering a unique integrated tool flow that performs source-to-source transformations to automatically optimize the original source code and map it on the target platform. The optimizations aim at reducing the number of memory accesses and the required memory storage of both dynamically and statically allocated data. Furthermore, the MNEMEE tool flow performs optimal assignment of all data on the memory hierarchy of the target platform. Overall, the MNEMEE techniques embedded in it will lead to more cost efficient systems that offer a better performance and lower energy consumption. This tutorial gives an overview of the MNEMEE tool flow. The objective of the tutorial is to familiarize the audience with the tool framework and the optimizations used in the individual tools. The tutorial also features a demonstration of the tool flow. This demonstration shows that the tools developed in the MNEMEE project provide a user-friendly and efficient framework for MPSoC programming and memory management.

The key characteristic of next generation embedded applications will be the intensive data transfer and storage and the need for efficient memory management. The embedded system designer community needs optimization methodologies and techniques, which do not change the input-output functionality of the software applications or the design of the underlying hardware platform. In this paper, the key focus is the efficient data access and memory storage of both dynamically and statically allocated data and their assignment on the data memory hierarchy of an MPSoC platform. We propose a design tool framework to efficiently automate the time-consuming optimizations for parallelization and memory mapping of static and dynamic data for MPSoCs.

We describe and evaluate two new, independently-applicable power reduction techniques for power management on processors that support dynamic voltage and frequency scaling (DVFS): user-driven frequency scaling (UDFS) and process-driven voltage scaling (PDVS). In PDVS, a CPU-customized profile is derived offline that encodes the minimum voltage needed to achieve stability at each combination of CPU frequency and temperature. On a typical processor, PDVS reduces the voltage below the worst-case minimum operating voltages given in datasheets. UDFS, on the other hand, dynamically adapts CPU frequency to the individual user and the workload through direct user feedback. Our UDFS algorithms dramatically reduce typical operating frequencies and voltages while maintaining performance at a satisfactory level for each user. We evaluate our techniques independently and together through user studies conducted on a Pentium M laptop running Windows applications. We measure the overall system power and temperature reduction achieved by our methods. Combining PDVS and the best UDFS scheme reduces measured system power by 49.9% (27.8% PDVS, 22.1% UDFS), averaged across all our users and applications, compared to Windows XP DVFS. The average temperature of the CPU is decreased by 13.2degC. User trace-driven simulation to evaluate the CPU only indicates average CPU dynamic power savings of 57.3% (32.4% PDVS, 24.9% UDFS), with a maximum reduction of 83.4%. In a multitasking environment, the same UDFS+PDVS technique reduces the CPU dynamic power by 75.7% on average.

The ultimate goal of a computer system is to satisfy its users. The success of architectural or system-level optimizations depends largely on having accurate metrics for user satisfaction. We propose to derive such metrics from information that is close to flesh and apparent to the user rather than from information that is close to metal and hidden from the user. We describe and evaluate PICSEL, a dynamic voltage and frequency scaling (DVFS) technique that uses measurements of variations in the rate of change of a computer's video output to estimate user-perceived performance. Our adaptive algorithms, one conservative and one aggressive, use these estimates to dramatically reduce operating frequencies and voltages for graphically-intensive applications while maintaining performance at a satisfactory level for the user. We evaluate PICSEL through user studies conducted on a Pentium M laptop running Windows XP. Experiments performed with 20 users executing three applications indicate that the measured laptop power can be reduced by up to 12.1%, averaged across all of our users and applications, compared to the default Windows XP DVFS policy. User studies revealed that the difference in overall user satisfaction between the more aggressive version of PICSEL and Windows DVFS were statistically insignificant, whereas the conservative version of PICSEL actually improved user satisfaction when compared to Windows DVFS.

The ultimate goal of computer design is to satisfy the end-user. In particular computing domains, such as interactive applications, there exists a variation in user expectations and user satisfaction relative to the performance of existing computer systems. In this work, we leverage this variation to develop more efficient architectures that are customized to end-users. We first investigate the relationship between microarchitectural parameters and user satisfaction. Specifically, we analyze the relationship between hardware performance counter (HPC) readings and individual satisfaction levels reported by users for representative applications. Our results show that the satisfaction of the user is strongly correlated to the performance of the underlying hardware. More importantly, the results show that user satisfaction is highly user-dependent. To take advantage of these observations, we develop a framework called Individualized Dynamic Voltage and Frequency Scaling (iDVFS). We study a group of users to characterize the relationship between the HPCs and individual user satisfaction levels. Based on this analysis, we use artificial neural networks to model the function from HPCs to user satisfaction for individual users. This model is then used online to predict user satisfaction and set the frequency level accordingly. A second set of user studies demonstrates that iDVFS reduces the CPU power consumption by over 25% in representative applications as compared to the Windows XP DVFS algorithm.

Chip multiprocessor designs are the most common types of architectures seen in Network Processors. As the Network Processors are used to implement increasingly complicated applications, task distribution among the cores is becoming an important problem. In this paper, we propose a new task allocation scheme for such architectures. This scheme relies on the inherent modular nature of the networking applications and intelligently distributes modules among different execution cores. Additionally, we selectively replicate modules to parallelize execution of tasks having longer processing time. We have developed a technique that uses the probability distribution of the execution times of different modules in the networking applications. The proposed schemes result in resource utilization of up to 95%, 89%, and 84% on average for the processors with 2, 4, and 8 cores, respectively. The schemes are highly scalable and can improve the throughput by 6.72 times for 8 core processors, aggregated over four representative applications. The combination of selective replication of modules and variation-aware task allocation result in up to 12.5% (9.9% on average) performance improvement as compared to a scheme based on just mean processing time.

Variability is one of the important issues in nanoscale processors. Due to increasing importance of interconnect structures in submicron technologies, the physical location and phenomena such as coupling have an increasing impact on the latency of operations. Therefore, traditional view of rigid access latencies to components wil result in suboptimal architectures. In this paper, we devise a cache architecture with variable access latency. Particularly, we a) develop a non-uniform access level 1 data-cache, b) study the impact of coupling and physical location on level 1 data cache access latencies, and c) develop and study an architecture where the variable latency cache can be accessed while the rest of the pipeline remains synchronous. To find the access latency with different input address transitions and environmental conditions, we first build a SPICE model at a 45nm technology for a cache similar to that of the level 1 data cache of the Intel Prescott architecture. Motivated by the large difference between the worst and best case latencies and the shape of the distribution curve, we change the cache architecture to allow variable latency accesses. Since the latency of the cache is not known at the time of instruction scheduling, we also modify the functional units with the addition of special queues that will temporarily store the dependent instructions and allow the data to be forwarded from the cache to the functional units correctly. Simulations based on SPEC2000 benchmarks show that our variable access latency cache structure can reduce the execution time by as much as 19.4% and 10.7% on average compared to a conventional cache architecture.

Dynamic Voltage and Frequency Scaling (DVFS) is one of the most commonly used power reduction techniques in high-performance processors. DVFS varies the frequency and voltage of a microprocessor in real-time according to processing needs. Although there are different versions of DVFS, at its core DVFS adapts power consumption and performance to the current workload of the CPU. Specifically, existing DVFS techniques in high-performance processors select an operating point (CPU frequency and voltage) based on the utilization of the processor. This approach integrates OS-level control, but such control is pessimistic. Existing DVFS techniques are pessimistic about the user. Indeed, they ignore the user, assuming that CPU utilization or the OS events prompting it are sufficient proxies. A high CPU utilization simply leads to a high frequency and high voltage, regardless of the user’s satisfaction or expectation of performance. Existing DVFS techniques are pessimistic about the CPU. They assume worst-case manufacturing process variation and operating temperature by basing their policies on loose worstcase bounds given by the processor manufacturer. A voltage level for a given frequency is set such that even the worst shipped processor of a given generation will be stable at the highest specified temperature. In response to these observations, we have developed, implemented, and evaluated the following two new power management techniques that can be readily employed independently or together. We elaborate on these techniques in detail elsewhere [4].

The modern era of embedded system design is geared toward the design of low-power systems. One way to reduce power in an application-specified integrated circuit (ASIC) implementation is to reduce the bit-width precision of its computation units. This paper describes algorithms to optimize the bit widths of fixed-point variables for low power in a SystemC-based ASIC design environment. We propose an optimal bit-width allocation algorithm for two variables and a greedy heuristic that works for any number of variables. The algorithms are used in the automation of converting floating-point SystemC programs into ASIC synthesizable SystemC programs. Expected inputs are profiled to estimate errors in the finite precision conversions. Experimental results for the tradeoffs between quantization error, power consumption, and hardware resources used are reported on a set of four SystemC benchmarks that are mapped onto a 0.18-mum ASIC cell library from Artisan Components. We demonstrate that it is possible to reduce the power consumption by 50% on the average by allowing roundoff errors to increase from 0.5% to 1%

The modern era of embedded system design is geared towards design of low-power systems. One way to reduce power in an ASIC implementation is to reduce the bit-width precision of its computation units. This paper describes algorithms to optimize the bit-widths of fixed point variables for low power in a SystemC design environment. We propose an algorithm for optimal bit width precision for two variables and a greedy heuristic which works for any number of variables. The algorithms are used in the automation of converting floating point SystemC programs into ASIC synthesizable SystemC programs. Expected inputs are profiled to estimate errors in the finite precision conversions. Experimental results on the trade-offs between quantization error, power consumption and hardware resources used are reported on a set of four SystemC benchmarks that are mapped onto 0.18 micron ASIC cell library from Artisan Components. We demonstrate that it is possible to reduce the power consumption by 50% on average by allowing round-off errors to increase from 0.5% to 1%. 1

Register files are in the critical path of most high-performance processors and their latency is one of the most important factors that limit their size. Our goal is to develop error correction mechanisms at the architecture level. Utilizing this increased robustness, the clock frequencies of the circuits are pushed beyond the point of allowing full voltage swing. This increases the errors observed due to noise and other external factors. The resulting errors are then corrected through the error correction mechanisms. We first develop a realistic model for error probability in register files for a given clock frequency. Then, we present the overall architecture, which allows the error detection computation to be overlapped with other computation in the pipeline. We develop novel techniques that utilize the fact that at a given instance many physical registers are not used in superscalar processors. These underutilized registers are used to store the values of active registers. Our simulation results show that for a fixed architecture the access times to the registers can be reduced by as much as 80% while increasing the number of execution cycles by 0.12%. On the other hand, by reducing the register file access pipeline stages by 75%, the average number of execution cycles of SPEC applications can be reduced by 11.5%.

The performance requirements of emerging embedded applications are rapidly increasing. One attractive approach to increase the performance of processors, while keeping their energy consumption low, is to utilize instruction-level parallelism. Hence, we are witnessing a significant increase in the number of superscalar embedded processors. In this paper, we present a method to reduce the energy consumption in such processors. Particularly, we will show that a) the load instructions in representative applications exhibit a large address locality, i.e., two consecutive executions of the same load instruction is very likely to access the same data, and b) the register file utilization of these applications are usually low. To take advantage of these observations, we devise a load elimination scheme, which tries to store the data values of load instructions in the register file. Our results with 11 MediaBench applications reveal that this method eliminates 20.5% of all cache accesses, resulting in 11.5% reduction in the energy consumption.

We introduce a new cache architecture that can be used to increase performance and reduce energy consumption in Network Processors. This new architecture is based on the observation that there is a strong correlation between different memory accesses. In other words, if load X and load Y are two consecutively executed load instructions, the offset between the source addresses of these instructions remain usually constant between different iterations. We utilize this information by building a correlating cache architecture. This architecture consists of a Dynamic Correlation Extractor, a Correlation History Table, and a Correlation Buffer. We first show simulation results investigating the frequency of correlating loads. Then, we evaluate our architecture using SimpleScalar/ARM. For a set of representative applications, the correlating cache architecture is able to reduce the average data access time by as much as 52.7% and 36.1% on average, while reducing the energy consumption of the caches by as much as 49.2% and 25.7% on average.

Hardware faults can occur in any computer system. Although faults cannot be tolerated for most systems (e.g., servers or desktop processors), many applications (e.g., networking applications) provide robustness in software. However, processors do not utilize this resiliency, i.e., regardless of the application at hand, a processor is expected to operate completely fault-free. In this paper, we will question this traditional approach of complete correctness and investigate possible performance and energy optimizations when this correctness constraint is released. We first develop a realistic model that estimates the change in the fault rates according to the clock frequency of the cache. Then, we present a scheme that dynamically adjusts the clock frequency of the data caches to achieve the desired optimization goal, e.g., reduced energy or reduced access latency. Finally, we present simulation results investigating the optimal operation frequency of the data caches, where reliability is compromised in exchange of reduced energy and increased performance. Our simulation results indicate that the clock frequency of the data caches can be increased as much as 4 times without incurring a major penalty on the reliability. This also results in 41% reduction in the energy consumed in the data caches and a 24% reduction in the energy-delay-fallibility product.