The effectiveness of deep learning models is greatly dependent on the availability of a vast amount of labeled data. However, in the realm of surface defect classification, acquiring and annotating defect samples proves to be quite challenging. Consequently, accurately predicting defect types with only a limited number of labeled samples has emerged as a prominent research focus in recent years. Few-shot learning, which leverages a restricted sample set in the support set, can effectively predict the categories of unlabeled samples in the query set. This approach is particularly well-suited for defect classification scenarios. In this article, we propose a transductive few-shot surface defect classification method, which using both the instance-level relations and distribution-level relations in each few-shot learning task. Furthermore, we calculate class center features in transductive manner and incorporate them into the feature aggregation operation to rectify the positioning of edge samples in the mapping space. This adjustment aims to minimize the distance between samples of the same category, thereby mitigating the influence of unlabeled samples at category boundary on classification accuracy . Experimental results on the public dataset show the outstanding performance of our proposed approach compared to the state-of-the-art methods in the few-shot learning settings. Our code is available at https://github.com/Harry10459/CIDnet .

At present, in the research of multimodal human action recognition, the weighted fusion method with fixed weight is widely applied in the decision level fusion of most models. In this way, the weight is usually obtained from the original experience or traversal search, which is inaccurate or has a large amount of calculation, and ignores the different representation ability of various modal data for various classes of action information. With the help of the powerful decision-making ability of deep reinforcement learning, we propose a multimodal decision-making fusion weight allocation network based on deep reinforcement learning. This letter mainly discusses the design of the model, which involves the modeling of reinforcement learning problem in action recognition, the design of neural network and the selection of problem-solving scheme. Experimental results on NTU RGB + D and HMDB51 datasets show the effectiveness of the proposed method.

The objective of multimodal information fusion is to mathematically analyze information carried in different sources and create a new representation that will be more effectively utilized in pattern recognition and other multimedia information processing tasks. In this paper, we introduce a new method for multimodal information fusion and representation based on the Labeled Multiple Canonical Correlation Analysis (LMCCA). By incorporating class label information of the training samples, the proposed LMCCA ensures that the fused features carry discriminative characteristics of the multimodal information representations and are capable of providing superior recognition performance. We implement a prototype of LMCCA to demonstrate its effectiveness on handwritten digit recognition, face recognition, and object recognition utilizing multiple features, bimodal human emotion recognition involving information from both audio and visual domains. The generic nature of LMCCA allows it to take as input features extracted by any means, including those by deep learning (DL) methods. Experimental results show that the proposed method enhanced the performance of both statistical machine learning methods, and methods based on DL.

In this paper, we propose the discriminative multiple canonical correlation analysis (DMCCA) for multimodal information analysis and fusion. DMCCA is capable of extracting more discriminative characteristics from multimodal information representations. Specifically, it finds the projected directions, which simultaneously maximize the within-class correlation and minimize the between-class correlation, leading to better utilization of the multimodal information. In the process, we analytically demonstrate that the optimally projected dimension by DMCCA can be quite accurately predicted, leading to both superior performance and substantial reduction in computational cost. We further verify that canonical correlation analysis (CCA), multiple canonical correlation analysis (MCCA) and discriminative canonical correlation analysis (DCCA) are special cases of DMCCA, thus establishing a unified framework for canonical correlation analysis. We implement a prototype of DMCCA to demonstrate its performance in handwritten digit recognition and human emotion recognition. Extensive experiments show that DMCCA outperforms the traditional methods of serial fusion, CCA, MCCA, and DCCA.

Human action recognition of depth sensors has drawn wide attentions in computer vision and multimedia processing areas. In contrast to simple periodic actions, irrelevant actions or sharing sub-actions between different classes of two-person non-periodic interactions make this task challenging. This paper presents heterogeneous features fusion with Collaborative Representation (CR) to address this challenge. Two effective high dimensional low-level features are developed from depth image sequence and skeleton pose sequence respectively. In the Canonical Correlations Analysis (CCA) feature space of these two features, Collaborative Representation (CR) is learned and adopted as the final high-level discriminative representation. Experiments on two depth action datasets (SBU Kinect-Interaction and MSR Action 3D) show that the proposed method is superior to the state-of-the-art methods compared, including some recent deep learning based methods.

The effective interpretation and integration of multiple information content are important for the efficacious utilisation of multimedia in a wide variety of application context. The major challenge in information fusion lies in the difficulty of identifying the complementary and discriminatory representations from individual channels or data sources. In this paper, we propose a novel framework integrating kernel entropy-estimation and discriminative multiple canonical correlation (DMCC) to address this challenge. Not only the distribution and complementary representations of input data are revealed by entropy estimation, but also the discriminative representations are considered by DMCC, achieving improved recognition accuracy. The effectiveness of the proposed method is demonstrated on two audio emotion databases. Experimental results show that it outperforms the existing methods based on similar principles.

We propose a 3D action recognition algorithm which uses depth-based Gradient Local Auto-Correlations (GLAC) feature and Locality-constrained Affine Subspace Coding (LASC) to improve the discriminative ability of human actions in spatio-temporal subsequences of 3D depth videos. First, each entire depth video sequence is divided automatically into a set of subsequences (i.e., multi-scale sub-actions) by the normalized motion energy vector. Next Depth Motion Maps (DMMs) based GLAC features are employed to capture the shape information and motion cues of each sub-action. In order to obtain a more compact and discriminative representation, LASC is then proposed to encode the features extracted from the depth video. We show that the use of LASC exhibits better performance compared to existing methods such as Locality-constrained Linear Coding (LLC). On all three datasets we obtain competitive results compared to fifteen methods, while using fewer features and less complex models.

In this paper, we propose a posture selection method based on two-layer Affinity propagation (AP) for human action recognition using Hidden markov models (HMMs). A two-layer AP is used as the clustering algorithm instead of K-means in order to avoid the problem of random initialization. After two-layer AP, each cluster center of the skeleton features represents the pose of this activity. Each frame sequence of an action can be labeled by a sequence of these poses, meanwhile, the initial parameters of HMM can be calculated from these sequences. The effectiveness of the proposed method is implemented through MSR Action3D and UTKinect databases.

Based on depth information, this letter introduces a new local depth map feature describing local spatiotemporal details of human motion and a collaborative representation for classification with regularized least squares. By extracting a multilayered depth motion feature and then applying a multiscale Histograms of Oriented Gradient (HOG) descriptor to it, the proposed feature characterizes the local temporal change of human motion and the local spatial structure (appearance) of an action. Instead of class-specific dictionary, the test action sample is represented collaboratively by the common shared dictionary. Moreover, we present an analytical solution of collaborative representation, which is independent of the query and can be precalculated as a projection matrix, leading to low computational cost in recognition. The evaluations on MSRAction3D and MSRGesture3D datasets demonstrate its effectiveness.

In this paper, we propose a new spatio-temporal feature based method for human action recognition using depth image sequence. Fist, Layered Depth Motion maps (LDM) are utilized to capture the temporal motion feature. Next, multi-scale HOG descriptors are computed on LDM to characterize the structural information of actions. Then sparse coding is applied for feature representation. Extending Sparse fisher Discriminative Dictionary Learning (SDDL) model and its corresponding classification scheme are also introduced. In SDDL model, the sub-dictionary is updated class by class, leading to class-specific compact discriminative dictionaries. The proposed method is evaluated on public MSR Action3D datasets and demonstrates great performance, especially in cross subject test.

This paper presents a novel human action recognition method by using depth maps. Each depth frame in a depth video sequence is projected onto three orthogonal Cartesian planes. Under each projection view, we divide the entire depth maps into several sub-actions. The absolute difference between two consecutive projected maps is accumulated through a depth video (several sub-actions) sequence to form a Depth Motion Map (DMM) to describe the dynamic feature of an action. Also the difference within the threshold between two consecutive projected maps is calculated through the entire depth video to form another kind of Depth Static Map (DSM) to describe the static feature. Collectively, we call them Temporal Pyramid of Depth Model (TPDM). Then Spatial Pyramid Histograms of Oriented Gradient (SPHOG) is computed from the TPDM for the representation of an action. For classification, we apply support vector machine (SVM) to classify the proposed descriptorsbased on MSR Action3D dataset. Experimental results demonstrates the effectiveness of our proposed method.

MIMO-OFDM technique combines the advantages of MIMO and OFDM and is widely used in high data rate system. Conventional linear channel estimators are considered optimal under the assumption of rich multipath, while the practical physical multipath channels tend to exhibit sparse structure. By exploiting the coherent sparsity of the multipath channels, channel estimation based on compressive sensing (CS) method can greatly decrease the pilot overhead burden. In this paper, we present CS-based channel estimation method by exploiting the time-domain sparsity and one of the CS algorithms-Subspace Pursuit (SP) in order to estimate the channel impulse response in MIMO-OFDM system. Simulation results demonstrate a significant reduction of the number of pilots compared with least squares (LS) channel estimation.

This paper presents a novel approach for multi-feature information fusion. The proposed method is based on the Discriminative Multiple Canonical Correlation Analysis (DMCCA), which can extract more discriminative characteristics for recognition from multi-feature information representation. It represents the different patterns among multiple subsets of features identified by minimizing the Frobenius norm. We will demonstrate that the Canonical Correlation Analysis (CCA), the Multiple Canonical Correlation Analysis (MCCA), and the Discriminative Canonical Correlation Analysis (DCCA) are special cases of the DMCCA. The effectiveness of the DMCCA is demonstrated through experimentation in speaker recognition and speech-based emotion recognition. Experimental results show that the proposed approach outperforms the traditional methods of serial fusion, CCA, MCCA and DCCA.