Few-shot learning can find the latent structure information between the support and query data by the similarity metric of meta-learning to construct the discriminative model for recognizing the new categories with the less labeled samples. Most existing methods attempt to model the similarity relationships among samples within meta-tasks for achieving this goal. However, the relationship assessment among samples from distinct meta-tasks is challenging due to the differing metric relationships inherent to each respective meta-task. To address this issue, high-order structure-preserving graph neural network (HOSP-GNN) can deeply explore the rich samples structure of the different meta-tasks to predict the label of the queried data based on the graph. HOSP-GNN can mine high-order structures to enhance their relevance by triple samples. In addition, it can also generate the updating rule of the different-order structures for node and edge representation optimization under manifold constraints. Notably, HOSP-GNN eliminates the need for retraining the learning model to recognize new classes, thanks to its high generalization of high-order structure that ensures model adaptability. The experiments demonstrate that HOSP-GNN outperforms state-of-the-art methods in four benchmark datasets, as well as on a self-built dataset focusing on endangered animals. The available code ishttps://github.com/yangfeifei02/HOSP.

Abstract Temporal action detection aims to judge whether there existing a certain number of action instances in a long untrimmed videos and to locate the start and end time of each action. Even though the existing action detection methods have shown promising results in recent years with the widespread application of Convolutional Neural Network (CNN), it is still a challenging problem to accurately locate each action segment while ensuring real-time performance. In order to achieve a good tradeoff between detection efficiency and accuracy, we present a coarse-to-fine hierarchical temporal action detection method by using multi-scale sliding window mechanism. Since the complexity of the convolution operator is proportional to the number and the size of the input video clips, the idea of our proposed method is to first determine candidate action proposals and then perform the detection task on these candidate action proposals only with a view to reducing the overall complexity of the detection method. By making full use of the spatio-temporal information of video clips, a lightweight 3D-CNN classifier is first used to quickly determine whether the video clip is a candidate action proposal, avoiding the re-detection of a large number of non-action video clips by the heavyweight deep network. A heavyweight detector is designed to further improve the accuracy of action positioning by considering both boundary regression loss and category loss in the target loss function. In addition, the Non-Maximum Suppression (NMS) is performed to eliminate redundant detection results among the overlapping proposals. The mean Average Precision (mAP) is 40.6%, 51.7% and 20.4% on THUMOS14, ActivityNet and MPII Cooking dataset when the Intersection-over-Union (tIoU) threshold is set to 0.5, respectively. Experimental results show the superior performance of the proposed method on three challenging temporal activity detection datasets while achieving real-time speed. At the same time, our method can generate proposals for unseen action classes with high recalls.

Structure fusion (SF) has been presented for multiple feature fusion via mining the discriminative and complementary information from different feature sets. As the typical methods, SF based on locality preserving projections (SFLPP) and SF based on tensor subspace analysis (SFTSA) have been developed for classification by capturing the complete structure from different features. However, the jointed optimisation function of SFLPP or SFTSA does not clearly explain the modelling mechanism of SF, and its solving process is complex because of iterative eigenvalue decomposition. In this study, structure modelling based on maximisation posterior probability (SMMPP) is proposed for solving these issues. It jointly considers both the certain prior structure (the mutual structure of multiple feature structure described by Ising model) and the uncertain likelihood structure (the possible fusion structure of multiple feature structure represented by Markov random field model) into the framework of Bayes’ rule. The proposed computational solution is faster‐converging speed than SFLPP or SFTSA with the guarantee of convergence. Extensive experiments conducted on shape analysis and human action recognition demonstrate the superiority of SMMPP over the state of art methods.