Artificial intelligence (AI) using neural networks is now widely employed, and developing hardware for AI training computations is vital. In a previous study, we proposed a hardware-oriented optimizer (Holmes), which enables faster training with a smaller memory footprint compared to conventional optimizers. As a further step toward futuristic edge AI online training, in this study, we developed an architecture for hardware that incorporates Holmes and benefits from parallelization and pipelining to achieve significant throughput improvement. To evaluate this architecture, we investigated the scaling of the throughput and memory size over the degree of parallelism and the neural network model size through software simulation. The result proved the linear scalability of the memory and computational resources over the model size.

Out-of-distribution (OOD) data detection is critical when using machine learning models for practical applications. Many methods have been proposed to estimate the uncertainty of machine learning models for input data. Therefore, the purpose of this paper is to propose and verify toward elucidation of the nature and the practical use. We proposes data preprocessing to improve OOD image data detection performance using Bayesian neural networks (BNNs) to estimate two types of uncertainties. The aim of the proposed preprocessing is to increase the usefulness of the two uncertainties by providing more information to the BNN. The proposed preprocessing extracts high-frequency components of the input image by wavelet transform and adds the channel direction to create new input data. The contributions of this paper are as follows. First, we show that in training FashionMNIST, the BNN has a high performance in detecting OOD data by epistemic uncertainty that can be estimated. Second, we infer the relationship between the degree of BNN learning and OOD data detection performance. Finally, the proposed pre-processing is demonstrated to improve the OOD data detection performance. The validation results show that the average area under the receiver operating characteristic (AUROC) increases from 0.936 to 0.970 with the proposed preprocessing for BNNs trained in 10 epochs. These contributions reveal some aspects of the nature of uncertainty and provide clues to approaches for practical applications.

Hidden Networks (Ramanujan et al., 2020) showed the possibility of finding accurate subnetworks within a randomly weighted neural network by training a connectivity mask, referred to as supermask. We show that the supermask stops improving even though gradients are not zero, thus underutilizing backpropagated information. To address this we propose a method that extends Hidden Networks by training an overlay of multiple hierarchical supermasks{—}a multicoated supermask. This method shows that using multiple supermasks for a single task achieves higher accuracy without additional training cost. Experiments on CIFAR-10 and ImageNet show that Multicoated Supermasks enhance the tradeoff between accuracy and model size. A ResNet-101 using a 7-coated supermask outperforms its Hidden Networks counterpart by 4%, matching the accuracy of a dense ResNet-50 while being an order of magnitude smaller.

A 4b-quantized convolutional neural network (CNN) inference engine for edge-AI is presented featuring a Cartesian-product MAC array and pipelined activation aligners targeting deep-/random-pruned models. A 40nm prototype with 32x32 MACs and 5Mb SRAM runs at 534 MHz, 1.07 TOPS, 352 mW at 1.1V, and attains 5.30 dense TOPS/W, 234 MHz at 0.8V. Sparse TOPS/W reaches 26.5 when running a randomly pruned model (after 88% pruning). Training algorithms for obtaining highly efficient sparse/quantized models are also proposed.

A 28.8-GB/s 96-MB 3D-stacked SRAM is presented. A total of eight SRAM dies, designed in a 40-nm CMOS process, are vertically stacked and connected using an inductive coupling wireless link with a low-voltage NMOS push-pull transmitter that reduces the power of the link by 45% with a 0.4-V power supply. The SRAM utilizes an inverted bit insertion scheme that compensates the degradation of the first signal, a coil termination scheme that aims to eliminate the noise of 3D inductive coupling bus, and a 12:1 SerDes. The data density of the SRAM should reach 12.3-MB/mm3, which extends beyond that of state-of-the-art stacked DRAMs.

Energy-constrained neural network processing is in high demanded for various mobile applications. Binarized neural network (BNN) aggressively enhances the computational efficiency, and in contrast, it suffers from degradation of accuracy due to its extreme approximation. We propose a neural network model using a new activation function "Delta" based on binarization of differences between weighted-sums. The "Delta" retains the magnitude relation between numerical values, and conveys richer information than ordinary binarization. We can design the hardware architecture for the proposed model with almost the same elements as BNN. The evaluation shows that it achieves higher recognition accuracy than a conventional BNN with almost the same hardware configuration.

Energy-constrained neural network processing is in high demanded for various mobile applications. Binary neural network aggressively enhances the computational efficiency, and in contrast, it suffers from degradation of accuracy due to its extreme approximation. We propose a novel accurate neural network model based on binarization and "dithering" that distributes the quantization error to neighboring pixels. The quantization errors in the binarization are distributed in the plane, so that a pixel in the multi-level source expression more accurately represented in the resulting binarized plane by multiple pixels. We designed a low-overhead binary-based hardware architecture for the proposed model. The evaluation results show that this method can be realized with a few additional lightweight hardware components.

The convolutional neural network (CNN) is a state-of-the-art model that can achieve significantly high accuracy in many machine-learning tasks. Recently, for further developing the practical applications of CNNs, efficient hardware platforms for accelerating CNN have been throughly studied. A binarized neural network has been reported to minimize the multipliers, which consume a large amount of resources, with a minimal decrease in accuracy. In this study, we analyzed the optimal performance of CNN implemented on an field programmable gate array (FPGA) considering its logic resources and a memory bandwidth, using multiple types of parallelisms such as kernels, pixels, and channels both in conventional and binarized CNNs. As a result, it became clear that all the parallelisms are required for the binarized neural network to obtain the best performance of 8.38 TOPS.