Convolutional neural networks (CNNs) still remain crucial in the field of computer vision , especially in industrial-embedded scenarios. Although modern artificial intelligence chips such as embedded graphics processing units (GPUs) and neural process units (NPUs) are equipped with sufficient computability, making CNNs more lightweight always has non-negligible significance. Until now, many researchers have made multiple corresponding achievements, in which a series of tensor decomposition methods have represented their unique advantages such as concision, flexibility, and low-rank approximation theory. However, balancing the compression, acceleration, and precision, is still an open issue, because the traditional tensor decompositions are hard to deal with the trade-off between approximation and compression ability, while the so-called fine-grained tensor decompositions such as Kronecker canonical polyadic (KCP) have not created a way to merge the factors for efficient inference. In this paper, we first review related works on convolutional neural network (CNN) compression and the necessary prior knowledge. We then propose a novel matrix decomposition method, termed hierarchical singular value (HSV) decomposition, and validate its effectiveness. Subsequently, we introduce a fast contraction strategy based on the merged factors of HSV and explain how our method addresses the inefficiencies in inference associated with traditional contraction processes. Additionally, we validate the advantages of HSV by comparing its complexity with that of other classical tensor decomposition methods. Thereafter, we apply HSV to CNN compression and acceleration by transforming convolution operations into matrix multiplication. We also propose a self-adaptive rank selection algorithm tailored to standard CNN architecture and conduct a theoretical analysis of the convergence of our method. Multiple experiments on CIFAR-10, ImageNet, COCO, and Cityscapes benchmark datasets show that the proposed HSV-Conv can simultaneously gain considerable compression ratio and acceleration ratio, while the precision loss is almost non-existent. We also make a comprehensive comparison with the other related works, and the superiority of our method is further validated. Besides, we give a deep discussion about the rank selection issue of HSV in the aspects of practice and theory, which explains the strategy of the proposed self-adaptive rank selection and the reason for choosing fine-tuning rather than training from scratch.

Recurrent neural networks (RNNs) are powerful in the tasks oriented to sequential data, such as natural language processing and video recognition. However, because the modern RNNs have complex topologies and expensive space/computation complexity, compressing them becomes a hot and promising topic in recent years. Among plenty of compression methods, tensor decomposition, e.g., tensor train (TT), block term (BT), tensor ring (TR), and hierarchical Tucker (HT), appears to be the most amazing approach because a very high compression ratio might be obtained. Nevertheless, none of these tensor decomposition formats can provide both space and computation efficiency. In this article, we consider to compress RNNs based on a novel Kronecker CANDECOMP/PARAFAC (KCP) decomposition, which is derived from Kronecker tensor (KT) decomposition, by proposing two fast algorithms of multiplication between the input and the tensor-decomposed weight. According to our experiments based on UCF11, Youtube Celebrities Face, UCF50, TIMIT, TED-LIUM, and Spiking Heidelberg digits datasets, it can be verified that the proposed KCP-RNNs have a comparable performance of accuracy with those in other tensor-decomposed formats, and even 278 219× compression ratio could be obtained by the low-rank KCP. More importantly, KCP-RNNs are efficient in both space and computation complexity compared with other tensor-decomposed ones. Besides, we find KCP has the best potential of parallel computing to accelerate the calculations in neural networks.

How to effectively and efficiently deal with spatio-temporal event streams, where the events are generally sparse and non-uniform and have the μs temporal resolution, is of great value and has various real-life applications. Spiking neural network (SNN), as one of the brain-inspired event-triggered computing models, has the potential to extract effective spatio-temporal features from the event streams. However, when aggregating individual events into frames with a new higher temporal resolution, existing SNN models do not attach importance to that the serial frames have different signal-to-noise ratios since event streams are sparse and non-uniform. This situation interferes with the performance of existing SNNs. In this work, we propose a temporal-wise attention SNN (TA-SNN) model to learn frame-based representation for processing event streams. Concretely, we extend the attention concept to temporal-wise input to judge the significance of frames for the final decision at the training stage, and discard the irrelevant frames at the inference stage. We demonstrate that TA-SNN models improve the accuracy of event streams classification tasks. We also study the impact of multiple-scale temporal resolutions for frame-based representation. Our approach is tested on three different classification tasks: gesture recognition, image classification, and spoken digit recognition. We report the state-of-the-art results on these tasks, and get the essential improvement of accuracy (almost 19%) for gesture recognition with only 60 ms.