Optical deep learning (DL) accelerators have attracted significant interests due to their latency and power advantages. In this article, we focus on incoherent optical designs. A significant challenge is that there is no known solution to perform single-wavelength accumulation (a key operation required for DL workloads) using incoherent optical signals efficiently. Therefore, we devise a hybrid approach, where accumulation is done in the electrical domain, and multiplication is performed in the optical domain. The key technology enabler of our design is the transistor laser, which performs electrical-to-optical and optical-to-electrical conversions efficiently. Through detailed design and evaluation of our design, along with a comprehensive benchmarking study against state-of-the-art RRAM-based designs, we derive the following key results: (1) For a four-layer multilayer perceptron network, our design achieves 115× and 17.11× improvements in latency and energy, respectively, compared to the RRAM-based design. We can take full advantage of the speed and energy benefits of the optical technology because the inference task can be entirely mapped onto our design. (2) For a complex workload (Resnet50), weight reprogramming is needed, and intermediate results need to be stored/re-fetched to/from memories. In this case, for the same area, our design still outperforms the RRAM-based design by 15.92× in inference latency, and 8.99× in energy.

We present a hybrid optical-electrical analog deep learning (DL) accelerator, the first work to use incoherent optical signals for DL workloads. Incoherent optical designs are more attractive than coherent ones as the former can be more easily realized in practice. However, a significant challenge in analog DL accelerators, where multiply-accumulate operations are dominant, is that there is no known solution to perform accumulation using incoherent optical signals. We overcome this challenge by devising a hybrid approach: accumulation is done in the electrical domain, while multiplication is performed in the optical domain. The key technology enabler of our design is the transistor laser, which performs electrical-to-optical and optical-to-electrical conversions efficiently to tightly integrate electrical and optical devices into compact circuits. As such, our design fully realizes the ultra high-speed and high-energy-efficiency advantages of analog and optical computing. Our evaluation results using the MNIST benchmark show that our design achieves 2214× and 65× improvements in latency and energy, respectively, compared to a state-of-the-art memristor-based analog design.

The memory wall bottleneck is a key challenge across many data-intensive applications. Multi-level FeFET-based embedded non-volatile memories are a promising solution for denser and more energy-efficient on-chip memory. However, reliable multi-level cell storage requires careful optimizations to minimize the design overhead costs. In this work, we investigate the interplay between FeFET device characteristics, programming schemes, and memory array architecture, and explore different design choices to optimize performance, energy, area, and accuracy metrics for critical data-intensive workloads. From our cross-stack design exploration, we find that we can store DNN weights and social network graphs at a density of over 8MB/mm2and sub-2ns read access latency without loss in application accuracy.

Co-exploration of neural architectures and hardware design is promising due to its capability to simultaneously optimize network accuracy and hardware efficiency. However, state-of-the-art neural architecture search algorithms for the co-exploration are dedicated for the conventional von-Neumann computing architecture, whose performance is heavily limited by the well-known memory wall. In this article, we are the first to bring the computing-in-memory architecture, which can easily transcend the memory wall, to interplay with the neural architecture search, aiming to find the most efficient neural architectures with high network accuracy and maximized hardware efficiency. Such a novel combination makes opportunities to boost performance, but also brings a bunch of challenges: The optimization space spans across multiple design layers from device type and circuit topology to neural architecture; and the presence of device variation may drastically degrade the neural network performance. To address these challenges, we propose a cross-layer exploration framework, namely NACIM, which jointly explores device, circuit and architecture design space and takes device variation into consideration to find the most robust neural architectures, coupled with the most efficient hardware design. Experimental results demonstrate that NACIM can find the robust neural network with 0.45 percent accuracy loss in the presence of device variation, compared with a 76.44 percent loss from the state-of-the-art NAS without consideration of variation; in addition, NACIM achieves an energy efficiency up to 16.3 TOPs/W, 3.17x higher than the state-of-the-art NAS.

Emerging memory devices are an attractive choice for implementing very energy-efficient in-situ matrix-vector multiplication (MVM) for use in intelligent edge platforms. Despite their great potential, device-level non-idealities have a large impact on the application-level accuracy of deep neural network (DNN) inference. We introduce a low-density parity-check code (LDPC) based approach to correct non-ideality induced errors encountered during in-situ MVM. We first encode the weights using error correcting codes (ECC), perform MVM on the encoded weights, and then decode the result after in-situ MVM. We show that partial encoding of weights can maintain DNN inference accuracy while minimizing the overhead of LDPC decoding. Within two iterations, our ECC method recovers 60% of the accuracy in MVM computations when 5% of underlying computations are error-prone. Compared to an alternative ECC method which uses arithmetic codes, using LDPC improves AlexNet classification accuracy by 0.8% at iso-energy. Similarly, at iso-energy, we demonstrate an improvement in CIFAR-10 classification accuracy of 54% with VGG-11 when compared to a strategy that uses 2× redundancy in weights. Further design space explorations demonstrate that we can leverage the resilience endowed by ECC to improve energy efficiency (by reducing operating voltage). A 3.3× energy efficiency improvement in DNN inference on CIFAR-10 dataset with VGG-11 is achieved at iso-accuracy.

Urban land usage classification is a critical task in big data based smart city applications that aim to understand the social-economic land functions and physical land attributes in urban environments. This paper focuses on a migratable urban land usage classification problem using remote sensing data (i.e., satellite images). Our goal is to accurately classify the land usage of locations in a target city where the ground truth land usage data is not available by leveraging a classification model from a source city where such data is available. This problem is motivated by the limitation of current solutions that primarily rely on a rich set of ground-truth data for accurate model training, which encounters high annotation costs. Two important challenges exist in solving our problem: i) the target and source cities often have different urban characteristics that prevent the direct application of a model learned from the source city to the target city; ii) the complex visual features in satellite images make it non-trivial to “translate” the images from the target city to the source city for an accurate classification. To address the above challenges, we develop TransLand, an adversarial transfer learning framework to translate the satellite images from the target city to the source city for accurate land usage classification. We evaluate our scheme on the real-world satellite imagery and land usage datasets collected from live different cities in Europe. The results show that TransLand significantly outperforms the state-of-the-art land usage classification baselines in classifying the land usage of locations in a city.

Deep Neural Networks (DNNs) have achieved tremendous success in many application domains. Inspired by its success, specialized accelerators have been and continue to be developed to process DNN workloads in an energy-efficient manner. The design space for DNN accelerators can be extremely large since they can employ different datapaths, data mapping strategies, circuits, and device technologies. To explore the design space for developing DNN accelerators, it is important to quickly estimate the energy cost associated with an accelerator. This paper introduces a uniform modeling framework, Eva-DNN, to estimate the dynamic energy (a major component of total energy) consumed by a DNN accelerator. Specifically, we model the number of accesses and associated energy cost at different levels of memory and functional units. We derive a uniform expression that estimates the number of accesses as a function of the number of basic operations normalized by data reuse and activity factor of corresponding units. To model the energy cost of an individual functional unit operation, we employ a device-level benchmarking approach. Eva-DNN can accurately model energy contributions from device technology, circuits, architecture, data mapping strategy, and network. We applied our model on three accelerator architectures from the literature, namely: Eyeriss, ShiDianNao, and TrueNorth. Results suggest that Eva-DNN can accurately estimate energy contributions from different architectural units, achieving 4.5% to 8.0% of deviation from energy costs obtained from hardware measurements for different DNN workloads.

Deep neural network (DNN) accelerators with improved energy and delay are desirable for meeting the requirements of hardware targeted for IoT and edge computing systems. Convolutional neural networks (CoNNs) belong to one of the most popular types of DNN architectures. This article presents the design and evaluation of an accelerator for CoNNs. The system-level architecture is based on mixed-signal, cellular neural networks (CeNNs). Specifically, we present (i) the implementation of different layers, including convolution, ReLU, and pooling, in a CoNN using CeNN, (ii) modified CoNN structures with CeNN-friendly layers to reduce computational overheads typically associated with a CoNN, (iii) a mixed-signal CeNN architecture that performs CoNN computations in the analog and mixed signal domain, and (iv) design space exploration that identifies what CeNN-based algorithm and architectural features fare best compared to existing algorithms and architectures when evaluated over common datasets—MNIST and CIFAR-10. Notably, the proposed approach can lead to 8.7× improvements in energy-delay product (EDP) per digit classification for the MNIST dataset at iso-accuracy when compared with the state-of-the-art DNN engine, while our approach could offer 4.3× improvements in EDP when compared to other network implementations for the CIFAR-10 dataset.

Neural network accelerators with low latency and low energy consumption are desirable for edge computing. To create such accelerators, we propose a design flow for accelerating the extremely low bit-width neural network (ELB-NN) in embedded FPGAs with hybrid quantization schemes. This flow covers both network training and FPGA-based network deployment, which facilitates the design space exploration and simplifies the tradeoff between network accuracy and computation efficiency. Using this flow helps hardware designers to deliver a network accelerator in edge devices under strict resource and power constraints. We present the proposed flow by supporting hybrid ELB settings within a neural network. Results show that our design can deliver very high performance peaking at 10.3 TOPS and classify up to 325.3 image/s/watt while running large-scale neural networks for less than 5W using embedded FPGA. To the best of our knowledge, it is the most energy efficient solution in comparison to GPU or other FPGA implementations reported so far in the literature.

This paper discusses the development and evaluation of a Cellular Neural Network (CeNN) friendly deep learning network for solving the MNIST digit recognition problem. Prior work has shown that CeNNs leveraging emerging technologies such as tunnel transistors can improve energy or EDP of CeNNs, while simultaneously offering richer/more complex functionality. Important questions to address are what applications can benefit from CeNNs, and whether CeNNs can eventually outperform other alternatives at the application-level in terms of energy, performance, and accuracy. This paper begins to address these questions by using the MNIST problem as a case study.

A Cellular Neural Network (CNN) is a powerful processor that can significantly improve the performance of spatio-temporal applications such as pattern recognition, image processing, motion detection, when compared to the more traditional von Neumann architecture. In this paper, we show how tunneling field effect transistors (TFETs) can be utilized to enhance the performance of CNNs. Specifically, power consumption of TFET-based CNNs can be significantly lower when compared to MOSFET-based CNNs due to improved voltage controlled current sources (VCCSs) - an important component in CNN systems. We demonstrate that CNNs can benefit from low power conventional linear VCCSs implemented via TFETs. We also show that TFETs can be useful to realize non-linear VCCSs, which are either not possible or exhibit degraded performance when implemented via CMOS. Such non-linear VCCSs help to improve the performance of certain CNN operations (e.g., global maximum/minimum). We provide two case studies - image contrast enhancement and maximum row selection - that illustrate the benefits of non-linear VCCSs (e.g., reduced computation time, energy dissipation, etc.) when compared to CMOS-based approaches.

Cellular neural networks (CNNs) are a powerful analog architecture that can outperform traditional von Neumann architecture for spatio-temporal information processing applications, e.g., image processing and speech recognition. Much existing work reports energy dissipation for CNNs at the chip level, which includes dissipation of sensors, actuators, and other components. As such, the impacts of various system variables, e.g., application templates, characteristics of the resistive element, etc., on the energy profile of a CNN cannot be easily determined. In this work, we propose analytical models to estimate CNN power and performance (measured by settling time). Power dissipations, and settling times obtained via the models for different linear, and non-linear characteristics are verified through circuit simulation. Simulation results show that the proposed models predict power dissipation and settling time with less than 1% and 3% errors, respectively. By using these models, we have also performed case studies for a tactile sensing problem, and a pattern recognition problem to compare power and performance between tunneling field effect transistor (TFET) based non-linear CNN and conventional linear resistor based CNN.

Traditional CMOS based von Neumann architectures face daunting challenges in performing complex computational tasks at high speed and with low power on spatio-temporal data, e.g., image processing, pattern recognition, etc. In this study, we discuss the utilities of various steep slope, beyond-CMOS emerging devices for image processing applications within the non-von Neumann computing paradigm of cellular neural networks (CNNs). In general, the steep subthreshold swing of the devices obviates the output transfer hardware used in a conventional CNN cell. For image processing with binary stable outputs, Tunnelling FETs (TFETs) can facilitate low power operation. For multi-valued problems, devices like graphene transistors, Symmetric tunnelling FETs (SymFETs) might be leveraged to solve a problem with fewer computational steps. The potential for additional hardware reduction when compared to functional equivalents via conventional CNNs is also possible. Emerging devices can also lead to lower power implementations of the voltage controlled current sources (VCCSs) that are an integral component of any CNN cell. Furthermore, non-linear implementations of the VCCSs via emerging devices could enable simpler computational paths for many image processing tasks.