General-purpose optical accelerators (GOAs) have emerged as a promising platform to accelerate deep neural networks (DNNs) due to their low latency and energy consumption. Such an accelerator is usually composed of a given number of interleaving Mach-Zehnder-Interferometers (MZIs). This interleaving architecture, however, has a low efficiency when accelerating neural networks of various sizes due to the mismatch between weight matrices and the GOA architecture. In this work, a hybrid GOA architecture is proposed to enhance the mapping efficiency of neural networks onto the GOA. In this architecture, independent MZI modules are connected with microring resonators (MRRs), so that they can be combined to process large neural networks efficiently. Each of these modules implements a unitary matrix with inputs adjusted by tunable coefficients. The parameters of the proposed architecture are searched using genetic algorithm. To enhance the accuracy of neural networks, selected weight matrices are expanded to multiple unitary matrices applying singular value decomposition (SVD). The kernels in neural networks are also adjusted to use up the on-chip computational resources. Experimental results show that with a given number of MZIs, the mapping efficiency of neural networks on the proposed architecture can be enhanced by 21.87%, 21.20%, 24.69%, and 25.52% for VGG16 and Resnet18 on datasets Cifar10 and Cifar100, respectively. The energy consumption and computation latency can also be reduced by over 67% and 21%, respectively.

Deep neural networks (DNNs) have demonstrated remarkable success in various fields. However, the large number of floating-point operations (FLOPs) in DNNs poses challenges for their deployment in resource-constrained applications, e.g., edge devices. To address the problem, pruning has been introduced to reduce the computational cost in executing DNNs. Previous pruning strategies are based on weight values, gradient values and activation outputs. Different from previous pruning solutions, in this paper, we propose a class-aware pruning technique to compress DNNs, which provides a novel perspective to reduce the computational cost of DNNs. In each iteration, the neural network training is modified to facilitate the class-aware pruning. Afterwards, the importance of filters with respect to the number of classes is evaluated. The filters that are only important for a few number of classes are removed. The neural network is then retrained to compensate for the incurred accuracy loss. The pruning iterations end until no filter can be removed anymore, indicating that the remaining filters are very important for many classes. This pruning technique outperforms previous pruning solutions in terms of accuracy, pruning ratio and the reduction of FLOPs. Experimental results confirm that this class-aware pruning technique can significantly reduce the number of weights and FLOPs, while maintaining a high inference accuracy. Our code is available at https://github.com/HWAI-TUDa/Class-Aware-Pruning

Having the potential for high speed, high throughput, and low energy cost, optical neural networks (ONN s) have emerged as a promising candidate for accelerating deep learning tasks. In conventional ONNs, light amplitudes are modulated at the input and detected at the output. However, the light phases are still ignored in conventional structures, although they can also carry information for computing. To address this issue, in this paper, we propose a framework called OplixNet to compress the areas of ONNs by modulating input image data into the amplitudes and phase parts of light signals. The input and output parts of the ONN s are redesigned to make full use of both amplitude and phase information. Moreover, mutual learning across different ONN structures is introduced to maintain the accuracy. Experimental results demonstrate that the proposed framework significantly reduces the areas of ONNs with the accuracy within an acceptable range. For instance, 75.03 % area is reduced with a 0.33% accuracy decrease on fully connected neural network (FCNN) and 74.88% area is reduced with a 2.38% accuracy decrease on ResNet-32.

Deep neural networks (DNNs) have made breakthroughs in various fields including image recognition and language processing. DNNs execute hundreds of millions of multiply-and-accumulate (MAC) operations. To efficiently accelerate such computations, analog in-memory-computing platforms have emerged leveraging emerging devices such as resistive RAM (RRAM). However, such accelerators face the hurdle of being required to have sufficient on-chip crossbars to hold all the weights of a DNN. Otherwise, RRAM cells in the crossbars need to be reprogramed to process further layers, which causes huge time/energy overhead due to the extremely slow writing and verification of the RRAM cells. As a result, it is still not possible to deploy such accelerators to process large-scale DNNs in industry. To address this problem, we propose the BasisN framework to accelerate DNNs on any number of available crossbars without reprogramming. BasisN introduces a novel representation of the kernels in DNN layers as combinations of global basis vectors shared between all layers with quantized coefficients. These basis vectors are written to crossbars only once and used for the computations of all layers with marginal hardware modification. BasisN also provides a novel training approach to enhance computation parallelization with the global basis vectors and optimize the coefficients to construct the kernels. Experimental results demonstrate that cycles per inference and energy-delay product were reduced to below 1% compared with applying reprogramming on crossbars in processing large-scale DNNs such as DenseNet and ResNet on ImageNet and CIFAR100 datasets, while the training and hardware costs are negligible.

The last decade has witnessed the breakthrough of deep neural networks (DNNs) in many fields. With the increasing depth of DNNs, hundreds of millions of multiply-and-accumulate (MAC) operations need to be executed. To accelerate such operations efficiently, analog in-memory computing platforms based on emerging devices, e.g., resistive RAM (RRAM), have been introduced. These acceleration platforms rely on analog properties of the devices and thus suffer from process variations. Consequently, weights in neural networks configured into these platforms can deviate from the nominal trained values, which may lead to feature errors and a significant degradation of the inference accuracy. Besides, additional HW aspects represent key controlling factors for such computing platforms, namely, the limited RRAM cell programmable conductance levels, which limits the number of bits stored in one RRAM cell, the ADC noise converting analog values to digital domain and the ADC power scaling with the number of bits of its output. To address these points, in this article, we propose a framework to enhance the robustness of neural networks under variations. First, an enhanced Lipschitz constant regularization is adopted during neural network training to suppress the amplification of errors propagated through network layers. Additionally, the quantization setting of a NN model is optimized considering robustness against weight variations and total ADC power consumption. Afterward, error compensation is introduced at necessary locations determined by reinforcement learning (RL) to rescue the feature maps with remaining errors. Experimental results demonstrate that inference accuracy of neural networks can be recovered from as low as 1.69% under variations back to more than 95% of their original accuracy at the highest level of variations and reducing total ADC power consumption by 55% while the training and hardware cost are negligible.

In-memory-computing (IMC) has become an efficient solution for implementing neural networks on hardware. However, IMC platforms request weights in neural networks to be programmed to exact values. This is a very demanding task due to programming complexity, process variations, noise, as well as thermal effects. Accordingly, new methods should be introduced to counter such uncertainties. In this paper, we first discuss a method to train neural networks statistically with process variations modeled as correlated random variables. The statistical effect is incorporated in the cost function during training. Consequently, a neural network after statistical training becomes robust to uncertainties. To deal with variations and noise further, we also introduce a compensation method with extra layers for neural networks. These extra layers are trained offline again after the weights in the original neural network are determined to enhance the inference accuracy. Finally, we will discuss a method for testing the effect of process variations in an optical acceleration platform for neural networks. This optical platform uses Mach-Zehnder Interferometers (MZIs) to implement the multiply-accumulate operations. However, trigonometric functions in the transformation matrix of an MZI make it very sensitive to process variations. To address this problem, we apply a recursive test procedure to determine the properties of MZIs inside an optical acceleration module, so that process variations can be compensated accordingly to maintain the inference accuracy of neural networks.

Optical neural networks with Mach-Zender interferometers (MZIs) have demonstrated advantages over their electronic counterparts in computing efficiency and power consumption. However, implementing the computation with a weight matrix in DNNs using this technique requires the decomposition of the weight matrix into two unitary matrices, because an optical network can only realize a single unitary matrix due to its structural property. Accordingly, a direct implementation of DNNs onto optical networks suffer from a low area efficiency. To address this challenge, in this paper, a near-unitary training framework is proposed. In this framework, a weight matrix in DNNs is first partitioned into square submatrices to reduce the number of MZIs in the optical networks. Afterwards, training is adjusted to make the partitioned submatrices as close to unitary as possible. Such a matrix is then represented further by the sum of a unitary matrix and a sparse matrix. The latter implements the difference between the unitary matrix and the near-unitary matrix after training. In this way, only one optical network is needed to implement this unitary matrix and the low computation load in the sparse matrix can be implemented with area-efficient microring resonators (MRRs). Experimental results show that the area footprint can be reduced by 81.81%, 85.51%, 48.6% for ResNet34, VGG16, and fully connected neural networks, respectively, while the inference accuracy is still maintained on CIFAR100 and MNIST datasets.