The long short-term memory (LSTM) is a widely used neural network model for dealing with time-varying data. To reduce the memory requirement, pruning is often applied to the weight matrix of the LSTM, which makes the matrix sparse. In this paper, we present a new sparse matrix format, named rearranged compressed sparse column (RCSC), to maximize the inference speed of the LSTM hardware accelerator. The RCSC format speeds up the inference by: 1) evenly distributing the computation loads to processing elements (PEs) and 2) reducing the input vector load miss within the local buffer. We also propose a hardware architecture adopting hierarchical input buffer to further reduce the pipeline stalls which cannot be handled by the RCSC format alone. The simulation results for various datasets show that combined use of the RSCS format and the proposed hardware requires 2× smaller inference runtime on average compared to the previous work.

The probabilistic Bayesian inference of real-time input data is becoming more popular, and the importance of semisupervised learning is growing. We present a classification restricted Boltzmann machine (ClassRBM)-based hardware accelerator with on-chip semisupervised learning and Bayesian inference capability. ClassRBM is a specific type of Markov network that can perform classification tasks and reconstruct its input data. ClassRBM has several advantages in terms of hardware implementation compared to other backpropagation-based neural networks. However, its accuracy is relatively low compared to backpropagation-based learning. To improve the accuracy of ClassRBM, we propose the multi-neuron-per-class (multi-NPC) voting scheme. We also reveal that the contrastive divergence (CD) algorithm, which is commonly used to train RBM, shows poor performance in this multi-NPC ClassRBM. As an alternative, we propose an asymmetric contrastive divergence (ACD) training algorithm that improves the accuracy of multi-NPC ClassRBM. With the ACD learning algorithm, ClassRBM operates in the form of a combination of Markov Chain training and Bayesian inference. The experimental results on a field-programmable gate array (FPGA) board for a Modified National Institute of Standards and Technology data set confirm that the inference accuracy of the proposed ACD algorithm is 5.82% higher for a supervised learning case and 12.78% higher for a 1% labeled semisupervised learning case than the conventional CD algorithm. Also, the GeCo ver.2 hardware implemented on a Xilinx ZCU102 FPGA board was 349.04 times faster than the C simulation on CPU.

Deep Neural Networks (DNNs) have various performance requirements and power constraints depending on applications. To maximize the energy-efficiency of hardware accelerators for different applications, the accelerators need to support various bit-width configurations. When designing bit-reconfigurable accelerators, each PE must have variable shift-addition logic, which takes a large amount of area and power. This paper introduces an area and energy efficient precision-scalable neural network accelerator (BitBlade), which reduces the control overhead for variable shift-addition using bitwise summation method. The proposed BitBlade, when synthesized in a 28nm CMOS technology, showed reduction in area by 41% and in energy by 36-46% compared to the state-of-the-art precision-scalable architecture [14].

In memristor crossbar array (MCA)-based neural network hardware, it is generally assumed that entire word-lines (WLs) are simultaneously enabled for parallel matrix-vector multiplication (MxV) operation. However, the error probability of MxV in a memristor crossbar array (MCA) increases as the resistance ratio (R-ratio) of a memristor decreases and the resistance variation and the number of simultaneously activated WLs increase. In this paper, we analyze the effect of R-ratio and variation of memristor devices on read sense margin and inference accuracy of MCA-based Binary Neural Network (BNN) hardware. We first show that only a limited number of WLs should be enabled to ensure correct MxV output when the R-ratio is small. On the other hand, we also show that, if the resistance variation becomes higher than a certain level, simultaneous activation of large number of WLs produces the higher accuracy even when R-ratio is small. Based on the analysis, we propose the Accuracy Estimation (AE) factor to find the optimal number of word lines that are simultaneously activated.

The LSTM is a popular neural network model for modeling or analyzing the time-varying data. The main operation of LSTM is a matrix-vector multiplication and it becomes sparse (spMxV) due to the widely-accepted weight pruning in deep learning. This paper presents a new sparse matrix format, named CBSR, to maximize the inference speed of the LSTM accelerator. In the CBSR format, speed-up is achieved by balancing out the computation loads over PEs. Along with the new format, we present a simple network transformation to completely remove the hardware overhead incurred when using the CBSR format. Also, the detailed analysis on the impact of network size or the number of PEs is performed, which lacks in the prior work. The simulation results show 16~38% improvement in the system performance compared to the well-known CSC/CSR format. The power analysis is also performed in 65nm CMOS technology to show 9~22% energy savings.

We present a Classification Restricted Boltzmann Machine (Class-RBM) hardware for embedded machines with on-chip learning capability. The RBM is a kind of the generative model, and has been used as one of the most popular feature extractors and image pre-processors. The ClassRBM is a variant of the RBM that is adapted to classification tasks. We propose the multi-Neuron-Per-Class (multi-NPC) voting scheme for improving accuracy of ClassRBM. We also show that the Contrastive Divergence (CD), which is one of the most popular algorithms to train RBM, has limitations in multi-NPC ClassRBM learning and propose a modified CD algorithm to overcome the limitation. Experimental results on FPGA flatform for MNIST datasets confirm that classification accuracy of the proposed algorithm is ∼ 2.12% higher than the conventional CD.