Abstract With the goal of solving the problem of feature extractors lacking strong supervision training and insufficient time information concerning single‐sequence model learning, a hierarchical sequence memory network with a multi‐level iterative optimisation strategy is proposed for continuous sign language recognition. This method uses the spatial‐temporal fusion convolution network (STFC‐Net) to extract the spatial‐temporal information of RGB and Optical flow video frames to obtain the multi‐modal visual features of a sign language video. Then, in order to enhance the temporal relationships of visual feature maps, the hierarchical memory sequence network is used to capture local utterance features and global context dependencies across time dimensions to obtain sequence features. Finally, the decoder decodes the final sentence sequence. In order to enhance the feature extractor, the authors adopted a multi‐level iterative optimisation strategy to fine‐tune STFC‐Net and the utterance feature extractor. The experimental results on the RWTH‐Phoenix‐Weather multi‐signer 2014 dataset and the Chinese sign language dataset show the effectiveness and superiority of this method.

Recently, skeleton-based human action recognition has attracted a lot of research attention in the field of computer vision. Graph convolutional networks (GCNs), which model the human body skeletons as spatial-temporal graphs, have shown excellent results. However, the existing methods only focus on the local physical connection between the joints, and ignore the non-physical dependencies among joints. To address this issue, we propose a hypergraph neural network (Hyper-GNN) to capture both spatial-temporal information and high-order dependencies for skeleton-based action recognition. In particular, to overcome the influence of noise caused by unrelated joints, we design the Hyper-GNN to extract the local and global structure information via the hyperedge (i.e., non-physical connection) constructions. In addition, the hypergraph attention mechanism and improved residual module are induced to further obtain the discriminative feature representations. Finally, a three-stream Hyper-GNN fusion architecture is adopted in the whole framework for action recognition. The experimental results performed on two benchmark datasets demonstrate that our proposed method can achieve the best performance when compared with the state-of-the-art skeleton-based methods.

In this work, we propose a combinative strategy based on regression and clustering for solving point set matching problems under a Bayesian framework, in which the regression estimates the transformation from the model to the sceneand the clustering establishes the correspondence between two point sets. The point set matching model is illustrated by a hierarchical directed graph, and the matching uncertainties are approximated by a coarse-to-fine variational inference algorithm. Furthermore, two Gaussian mixtures are proposed for the estimation of heteroscedastic noise and spurious outliers, and an isotropic or anisotropic covariance can be imposed on each mixture in terms of the transformed model points. The experimental results show that the proposed approach achieves comparable performance to state-of-the-art matching or registration algorithms in terms of both robustness and accuracy.

In this paper, we present two hashing based techniques for fast palmprint identification. We first propose three properties required by a hash function and then introduce our first fast identification method based on orientation pattern (OP) hashing. We give the definition of OP and demonstrate that it meets all the requirements of the hash function, and thus, would be appropriate for hashing based fast palmprint identification. We then introduce the second fast identification method based on principal orientation pattern (POP) hashing. Because the POPs are constructed using more stable orientation features, POP hashing can find the target template more quickly thus causing earlier termination of the identification process. We evaluate our methods on the Hong Kong PolyU large-scale database (9667 palms) and the CASIA palmprint database (600 palms) plus a synthetic database (100 000 palms). Experimental results show that, on the Hong Kong PolyU large-scale database, the speedups of OP hashing and POP hashing over brute-force search are 16.93 and 19.91, respectively, and the identification accuracy is slightly higher. While on the CASIA database plus the synthetic database, the speedups of OP hashing and POP hashing are 8.03 and 15.67, respectively, and the identification accuracy almost remains the same. Results also show that, in terms of accuracy, our methods are comparable to several state-of-the-art palmprint identification approaches, while in terms of speed, our methods are much faster.