The processing in-memory (PIM) approach that combines memory and processor appears to solve the memory wall problem. NAND flash memory, which is widely adopted in edge devices, is one of the promising platforms for PIM with its high-density property and the intrinsic ability for analog vector-matrix multiplication. Despite its potential, the domain conversion process, which converts an analog current to a digital value, accounts for most energy consumption on the NAND flash-based accelerator. It restricts the NAND flash memory usage for PIM compared to the other platforms. In this paper, we propose a NAND flash-based DNN accelerator to achieve both large memory density and energy efficiency among various platforms. As the NAND flash memory already shows higher memory density than other memory platforms, we aim to enhance energy efficiency by reducing the domain conversion process burden. Firstly, we optimize the bit width of partial multiplication by considering the analog-to-digital converter (ADC) resource. For further optimization, we propose a methodology to exploit many zero partial multiplication results for enhancing both energy efficiency and throughput. The proposed work successfully exploits the bit-level sparsity of DNN, which results in achieving up to 8.6/8.2 larger energy efficiency/throughput over the provisioned baseline.

We propose a deep convolutional neural network (CNN) inference processor based on a novel enhanced output stationary (EOS) dataflow. Based on the observation that some activations are commonly used in two successive convolutions, the EOS dataflow employs dedicated register files (RFs) for storing such reused activation data to eliminate redundant memory accesses for highly energy-consuming SRAM banks. In addition, processing elements (PEs) are split into multiple small groups such that each group covers a tile of input activation map to increase the usability of activation RFs (ARFs). The processor has two different voltage/frequency domains. The computation domain with 512 PEs operates at near-threshold voltage (NTV) (0.4 V) and 60-MHz frequency to increase energy efficiency, while the rest of the processors including 848-KB SRAMs run at 0.7 V and 120-MHz frequency to increase both on-chip and off-chip memory bandwidths. The measurement results show that our processor is capable of running AlexNet at 831 GOPS/W, VGG-16 at 1151 GOPS/W, ResNet-18 at 1004 GOPS/W, and MobileNet at 948 GOPS/W energy efficiency.

Popular deep learning technologies suffer from memory bottlenecks, which significantly degrade the energy-efficiency, especially in mobile environments. In-memory processing for binary neural networks (BNNs) has emerged as a promising solution to mitigate such bottlenecks, and various relevant works have been presented accordingly. However, their performances are severely limited by the overheads induced by the modification of the conventional memory architectures. To alleviate the performance degradation, we propose NAND-Net, an efficient architecture to minimize the computational complexity of in-memory processing for BNNs. Based on the observation that BNNs contain many redundancies, we decomposed each convolution into sub-convolutions and eliminated the unnecessary operations. In the remaining convolution, each binary multiplication (bitwise XNOR) is replaced by a bitwise NAND operation, which can be implemented without any bit cell modifications. This NAND operation further brings an opportunity to simplify the subsequent binary accumulations (popcounts). We reduced the operation cost of those popcounts by exploiting the data patterns of the NAND outputs. Compared to the prior state-of-the-art designs, NAND-Net achieves 1.04-2.4x speedup and 34-59% energy saving, thus making it a suitable solution to implement efficient in-memory processing for BNNs.

CNN-based Super-Resolution (SR), the most representative of low-level vision task, is a promising solution to improve users' QoS on IoT devices that suffer from limited network bandwidth and storage capacity by effectively enhancing image/video resolution. Although prior accelerators to embed CNN show tremendous performance and energy efficiency, they are not suitable for SR tasks regarding off-chip memory accesses. In this work, we present eSRCNN, a framework that enables performing energy-efficient SR tasks on diverse embedded CNN accelerators by decreasing off-chip memory accesses. To reduce off-chip memory accesses, our framework consists of three steps: a network reformation using a cross-layer weight scaling, a precision minimization with priority-based quantization, and an activation map compression exploiting a data locality. As a result, the energy consumption of off-chip memory accesses is reduced up to 71.89% with less than 3.52% area overhead.

Many recent studies have focused on Processing-in-memory (PIM) architectures for neural networks to resolve the memory bottleneck problem. Especially, an increased interest in Spin Orbit Torque (SOT)-MRAMs has emerged due to its low latency, high energy efficiency, and non-volatility. However, the previous work added extra computing circuits to support complicated computations, which results in large energy overheads. In this work, we propose a new PIM architecture with relatively small peripheral circuit, which produces the highest energy efficiency for processing a Long Short Term Memory (LSTM) among the PIM architectures. We improve the efficiency with a new computing method for logical operations, which exploits characteristics of SOT-MRAMs. We reduce the number of word lines (WLs) activated concurrently to one from two in the previous works. As a result, the energy for driving WLs is saved, and the sensing current for computation is reduced. Moreover, we propose efficient methods for additions, multiplications and non-linear activation functions in memory to process an LSTM. Accordingly, we achieve 1.26x energy efficiency with the proposed computing method for logical operations compared to the previous study based on SOT-MRAMs and up to 5.54x energy efficiency over the previous PIM architectures based on other memories.

Deep neural networks (DNNs) are widely used for real-world applications. However, large amount of kernel and intermediate data incur a memory wall problem in resource-limited edge devices. The recent advances of a binary deep neural network (BNN) and a computing in-memory (CIM) have effectively alleviated this bottleneck especially when they are combined together. However, previous CIM-based accelerators for BNN are highly vulnerable to process/supply voltage/temperature (PVT) variation, resulting in severe accuracy degradation which makes them impractical to be employed in real-world edge devices. To address this vulnerability, we propose a PVT-robust accelerator architecture for BNN with a computable 4T embedded DRAM (eDRAM) cell array. First, we implement the XNOR operation of BNN in a time-multiplexed manner by utilizing the fundamental read operation of the conventional eDRAM cell. Next, a PVT-robust bit-count based on charge sharing is proposed with a computable 4T eDRAM cell array. In result, the proposed architecture achieves 6.9× less variation in PVT-variant environments which guarantees a stable accuracy and 2.03-49.4× improvement of energy efficiency over previous CIM-based accelerators.

Recent large-scale CNNs suffer from a severe memory wall problem as their number of weights range from tens to hundreds of millions. Processing in-memory (PIM) and binary CNN have been proposed to alleviate the number of memory accesses and footprints, respectively. By combining the two separate concepts, we propose a novel processing in-DRAM framework for binary CNN, called NID, where dominant convolution operations are processed using in-DRAM bulk bitwise operations. We first identify the problem that the bitcount operations with only bulk bitwise AND/OR/NOT incur significant overhead in terms of delay when the size of kernels gets larger. Then, we not only optimize the performance by efficiently allocating inputs and kernels to DRAM banks for both convolutional and fully-connected layers through design space explorations, but also mitigate the overhead of bitcount operations by splitting kernels into multiple parts. Partial sum accumulations and tasks of the other layers such as max-pooling and normalization layers are processed in the peripheral area of DRAM with negligible overheads. In results, our NID framework achieves 19×-36× performance and 9×-14× EDP improvements for convolutional layers, and 9×-17× performance and 1.4×-4.5× EDP improvements for fully-connected layers over previous PIM technique in four large-scale CNN models.

Training convolutional neural network on device has become essential where it allows applications to consider user's individual environment. Meanwhile, the weight update operation from the training process is the primary factor of high energy consumption due to its substantial memory accesses. We propose a dedicated weight update architecture with two key features: (1) a specialized local buffer for the DRAM access deduction (2) a novel dataflow and its suitable processing element array structure for weight gradient computation to optimize the energy consumed by internal memories. Our scheme achieves 14.3%-30.2% total energy reduction by drastically eliminating the memory accesses.

The binary-weight CNN is one of the most efficient solutions for mobile CNNs. However, a large number of operations are required to process each image. To reduce such a huge operation count, we propose an energy-efficient kernel decomposition architecture, based on the observation that a large number of operations are redundant. In this scheme, all kernels are decomposed into sub-kernels to expose the common parts. By skipping the redundant computations, the operation count for each image was consequently reduced by 47.7%. Furthermore, a low cost bit-width quantization technique was implemented by exploiting the relative scales of the feature data. Experimental results showed that the proposed architecture achieves a 22% energy reduction.

Applying highly accurate neural networks to mobile devices encounters energy problems in battery-limited mobile environments. To resolve these problems, neuromorphic hardware solutions that enable event-driven operation have been proposed. In this work, we present a novel sparse neuromorphic system that implements an E-I Net algorithm to further improve energy efficiency. We introduce a neuron clock-gating technique that significantly reduces energy consumption by predicting future neuron spike activity without any loss of accuracy. We also propose synaptic pruning to save additional energy with minimal impact on classification accuracy. For fast adaptation to a changing environment, a learning algorithm is implemented in the proposed system. Compared to prior studies, our experimental results illustrate that the proposed system achieves 5.3×-11.4× energy efficiency improvement with comparable accuracy.

A digital clock and data recovery (CDR) employing a time-dithered delta–sigma modulator (TDDSM) is presented. By enabling hybrid dithering of a sampling period as well as an output bit of the TDDSM, the proposed CDR enhances the resolution of digitally controlled oscillator, removes a low-pass filter in the integral path, and reduces jitter generation. Fabricated in a 65-nm CMOS process, the proposed CDR operates at 5-Gb/s data rate with$\textrm {BER}<10^{-12}$for PRBS 31. The CDR consumes 13.32 mW at 5 Gb/s and achieves 2.14 and 29.7 ps of a long-term rms and peak-to-peak jitter, respectively.

A significant amount of energy is consumed by a voltage guardband to ensure error-free operations under the worsening PVT variations in modern processors. Circuit-level timing speculation has become a popular approach that increases energy efficiency by removing such guardband and tolerating occasional timing errors. However, SIMD processors suffer from a large throughput and energy efficiency loss induced by a conventional error correction mechanism which requires several extra cycles for each timing error. In this paper, we present an error masking scheme to eliminate the chances of performing the error correction. The error masking is done by allowing potential erroneous addition instructions to reuse the partial result of previous operations. We show that reuse can be applied to a large number of addition instructions by exploiting the observations that SIMD applications exhibit high levels of temporal operand value locality and operand value locality across SIMD lanes. Our implementation of the proposed masking scheme is augmented with the conventional pipeline logics. Simulation results verify that our scheme achieves up to 5.1% improvement in energy efficiency and 30% improvement in EDP (Energy-Delay-Product) over the baseline design.

Temperature-to-power technique is useful for post-silicon power model validation. However, the previous works were applicable only to the steady-state analysis. In this paper, we propose a new temperature-to-power technique, named PowerField, supporting both transient and steady-state analysis based on a probabilistic approach. Unlike the previous works, PowerField uses two consecutive thermal images to find the most feasible power distribution that causes the change between the two input images. To obtain the power map with the highest probability, we adopted maximum a posteriori Markov random field (MAP-MRF). For MAP-MRF framework, we modeled the spatial thermal system as a set of thermal nodes and derived an approximated transient heat transfer equation that requires only the local information of each thermal node. Experimental results with a thermal simulator show that PowerField outperforms the previous method in transient analysis reducing the error by half on average. We also show that our framework works well for steady-state analysis by using two identical steady-state thermal maps as inputs. Lastly, an application to determining the binary power patterns of an FPGA device is presented achieving 90.7% average accuracy.

Transient temperature-to-power conversion is as important as steady-state analysis since power distributions tend to change dynamically. In this work, we propose PowerField framework to find the most probable power distribution from consecutive thermal images. Since the transient analysis is vulnerable to spatio-temporal thermal noise, we adopted a maximum-a-posteriori Markov random field framework to enhance the noise immunity. The most probable power map is obtained by minimizing the energy function which is calculated using an approximated transient thermal equation. Experimental results with a thermal simulator shows that PowerField outperforms the previous method in transient analysis reducing the error by half on average. We also applied our method to a real silicon achieving 90.7% accuracy.