To address the challenges of wireless video transmission over multipath fading channels, we propose a robust deep joint source-channel coding (DeepJSCC) framework by effectively exploiting temporal redundancy and incorporating robust innovations at the modulation, coding, and decoding stages. At the modulation stage, tailored orthogonal frequency division multiplexing (OFDM) for robust video transmission is employed, decomposing wideband signals into orthogonal frequency-flat sub-channels to effectively mitigate frequency-selective fading. At the coding stage, conditional contextual coding with multi-scale Gaussian warped features is introduced to efficiently model temporal redundancy, significantly improving reconstruction quality under strict bandwidth constraints. At the decoding stage, a lightweight denoising module is integrated to robustly simplify signal restoration and accelerate convergence, addressing the suboptimality and slow convergence typically associated with simultaneously performing channel estimation, equalization, and semantic reconstruction. Experimental results demonstrate that the proposed robust framework significantly outperforms state-of-the-art video DeepJSCC methods, which achieves an average reconstruction quality gain of 5.13 dB under challenging multipath fading channel conditions1.

In recent years, low-rank representation (LRR) has received increasing attention on subspace clustering. Due to inevitable matrix inversion and singular value decomposition in each iteration, however, most of existing LRR algorithms may suffer from high computational complexity, and hence can not cope with the large-scale sample data commendably. To overcome this problem, in this paper, we propose a bilateral fast low-rank representation (BFLRR), which has a linear time complexity with respect to the number of samples. Specifically, we introduce the equivalent transformation method to remove the null spaces of both the columns and rows of the coefficient matrix so that a hypercompact coefficient matrix can be learned. Furthermore, the proposed BFLRR is embedded into a distributed framework as DFC-BFLRR to make it more efficient, which utilizes a combination of the global and local projection matrices. Extensive experiments are carried out on real datasets, and the results testify that the proposed methods not only perform faster-computing speed but also obtain favorable clustering accuracy in comparison with the competing methods among large-scale sample data.

Recently, Learning-based image compression has reached comparable performance with traditional image codecs(such as JPEG, BPG, WebP). However, computational complexity and rate flexibility are still two major challenges for its practical deployment. To tackle these problems, this paper proposes two universal modules named Energy-based Channel Gating(ECG) and Bit-rate Modulator(BM), which can be directly embedded into existing end-to-end image compression models. ECG uses dynamic pruning to reduce FLOPs for more than 50% in convolution layers, and a BM pair can modulate the latent representation to control the bit-rate in a channel-wise manner. By implementing these two modules, existing learning-based image codecs can obtain ability to output arbitrary bit-rate with a single model and reduced computation.

The performance of neural image compression have reached or suppressed traditional methods (such as JPEG, BPG, WebP). However, their sophisticated network structures with cascaded convolution layers bring heavy computational burden for practical deployment. In this paper, we explore structural sparsity in neural image compression network to obtain real-time acceleration without any specialized hardware design or algorithm. We propose a simple plug-in adaptive binary channel masking(ABCM) to judge the importance of each convolution channel and introduce sparsity during training. During inference, the unimportant channels are pruned to obtain slimmer network and less computation. We implement our method into three neural image compression networks with different entropy models to verify its effectiveness and generalization, the experiment results show that up to 7× computation reduction and 3× acceleration can be achieved with negligible performance drop.

Inspired by the mean calculation of RPCA_OM and inductiveness of IRPCA, we first propose an inductive robust principal component analysis method with removing the optimal mean automatically, which is shorted as IRPCA_OM. Furthermore, IRPCA_OM is extended to Schatten-$p$norm and a more general framework (i.e., EIRPCA_OM) is presented. The objective function of EIRPCA_OM includes two terms, the first term is a robust reconstruction error term constrained by an$\ell _{2,1}$-norm and the second term is a regularization term constrained by a Schatten-$p$norm. The proposed EIRPCA_OM method is robust, inductive and accurate. However, on the high-dimensional data, it would spend a large computation cost in training stage. To this end, a fast version of EIRPCA_OM called as FEIRPCA_OM is proposed, and its basic idea is to eliminate the zero eigenvalues of data matrix. More importantly, an effective theoretical proof is presented to ensure that FEIRPCA_OM has faster processing speed than EIRPCA_OM when processing high-dimensional data, but without any performance loss. Based on it, we also can exchange the less performance loss for the higher computation efficiency by removing the small eigenvalues of data matrix. Experimental results on the public datasets demonstrate that FEIRPCA_OM works efficiently on the high-dimensional data.

Multi-scale methodology can enhance the performance of the model in deep learning. The current multi-scale methodology focuses on changing the formation, which will increase the parameters and calculations of the network. This paper offers a multi-scale method for data based on bit plane slicing(MBB). This expands the receptive field of valid information in image data. It is done by multi-level fusing image with high bit planes. Our experimentation shows that by adding MBB in front of the backbone network, one can achieve a significant performance improvement. The MBB approach is widely applicable because it does not require changes to the structure of the backbone network.

Convolutional neural networks (CNNs) have been widely used in a range of tasks because of its robust convolutional feature transformation ability. In this paper, we propose a novel type of convolution called Boosting Attention Convolution (BAC) to improve the basic convolutional feature transformation process of CNNs. The proposed method is designed based on two principles, boosting and attention mechanism. Specifically, we design a set of simple yet effective Boosting Attention Modules (BAM) within grouped convolution, which progressively recalibrate distribution of feature map and enable the future filters nested in a convolution layer to focus more on the feature regions that are unactivated by previous filters. Thus, it can help CNNs generate more discriminative representations by explicitly incorporating richer information. The experimental results on various datasets verify that BAC outperforms state-of-the-art methods. More importantly, the proposed BAC is a general convolution that can be deployed to various modern networks without introducing much parameters and computational complexity.

Existing deep Thermal InfraRed (TIR) trackers usually use the feature models of RGB trackers for representation. However, these feature models learned on RGB images are neither effective in representing TIR objects nor taking fine-grained TIR information into consideration. To this end, we develop a multi-task framework to learn the TIR-specific discriminative features and fine-grained correlation features for TIR tracking. Specifically, we first use an auxiliary classification network to guide the generation of TIR-specific discriminative features for distinguishing the TIR objects belonging to different classes. Second, we design a fine-grained aware module to capture more subtle information for distinguishing the TIR objects belonging to the same class. These two kinds of features complement each other and recognize TIR objects in the levels of inter-class and intra-class respectively. These two feature models are learned using a multi-task matching framework and are jointly optimized on the TIR tracking task. In addition, we develop a large-scale TIR training dataset to train the network for adapting the model to the TIR domain. Extensive experimental results on three benchmarks show that the proposed algorithm achieves a relative gain of 10% over the baseline and performs favorably against the state-of-the-art methods. Codes and the proposed TIR dataset are available at https://github.com/QiaoLiuHit/MMNet.

Human action recognition using skeleton data has lots of potential applications in content-based action retrieval and intelligent surveillance, with wide usage of depth sensors and robust skeleton estimation algorithms. Previous methods describe spatial temporal skeleton joints as a compact color image and then use Convolutional Neural Network (CNN) to extract more discriminative deep features. However, these methods ignore the effect of speed variation, which is a common phenomenon and can bring severe intra-varieties to same types of actions. To solve this problem, this paper presents a novel hierarchical dropped CNN architecture, which is constructed in two stages. Dropped CNN (d-CNN) is firstly developed to extract deep features from a probabilistic speed insensitive color image. This image expresses both spatial distributions and temporal evolutions of skeleton joints meanwhile avoids the effect of speed variations. To enhance the temporal discriminative power, we extend d-CNN to a hierarchical structure (h-CNN), where multiple scales of temporal information are encoded. Extensive experiments on benchmark MSRC-12 dataset and the largest NTU RGB+D dataset verify the effectiveness and robustness of the proposed method.

In this paper, we propose an unsupervised feature selection method from the perspective of optimal reconstruction. The features selected by the proposed method can well represent the original data, and the effectiveness of the selected features is demonstrated by robust reconstruction and clustering. The proposed method emphasizes the joint l2, 1-norms minimization on both reconstruction term and regularization term to make them be column-sparse. Relying on the column-sparse property of reconstruction term and regularization term, the proposed method is able to improve the robustness to outliers and select the effective features. The proposed objective function is nonconvex. Fortunately, it can be equivalently reformulated as a convex form (with change of variables) to capture a global optimization solution. In fact, the proposed method is related to the optimal mean robust principal component analysis (OMRPCA) because the proposed method is a sparse self-contained regression type of OMRPCA. Since OMRPCA essentially adds the adaptive weights for data samples, we call the proposed method adaptive weighted sparse principal component analysis (AW-SPCA). Experimental results demonstrate the effectiveness of AW-SPCA.