As a periodic movement, gait contains informative biometric traits formed by individual body structures, motion patterns, and behavioral habits. Previous gait recognition methods mainly focus on mining the appearance cues from gait sequences, while neglecting the dynamic motion characteristics. Motion cues are important complementary information for generating high-quality gait representations that can help models accurately recognize individuals. In this article, we propose a novel gait recognition framework named GaitDS to model dynamic motion information and construct salient gait representations. Specifically, we develop a motion information perception module that can directly represent dynamic regions during walking and extract fine-grained motion features based on the appearance of body parts over time. In addition, since some frames in gait sequences share partial similarities, we present saliency identity representation learning to focus on key frames along the temporal dimension, and integrate salient identity features to enhance sequence-level representations. Furthermore, a channel enhanced module is designed to generate more discriminative gait representations, where motion and temporal salient features can be complemented with global representations. Compared with existing state-of-the-art methods, our model achieves superior average rank-1 recognition accuracy on three benchmark datasets, i.e., 93.7% on CASIA-B, 92.4% on OU-MVLP, and 50.7% on Gait3D.

Research on transferring the learned person re-identification (ReID) model in the source domain to other domains is of great importance since deploying a ReID model to a new scenario is common in practical applications. Most of existing unsupervised domain adaptation methods for person ReID employ the framework of pre-training in the source domain, and clustering and fine-tuning in the target domain. However, how to reduce the intra-domain variations and narrow the inter-domain gaps is far from solved and remains a challenging problem under this framework. In this paper, we address these issues from two aspects. Firstly, a voted-mask guided image channel shuffling strategy for data augmentation is proposed to enhance visual diversity, where image channel shuffling is used as an efficient tool to bridge the inter-domain gap, and voted masks are employed to extract the foregrounds of pedestrian images to relief the negative effects of various backgrounds for reducing the intra-domain variations. Secondly, a novel plug-and-play metric named adaptive camera margin is proposed to fully exploit the low-cost camera tags for producing high-quality pseudo labels, which can significantly reduce the intra-domain variations without extra training cost. Specifically, the proposed network consists of a sensitive branch and an adaptive branch accompanied with our strategy of data augmentation, which are embedded into a joint learning framework to decouple visual representations for better capturing transferable features across different domains in both two stages. Adaptive camera margin is employed to pull samples with different camera IDs closer in the procedure of DBSCAN clustering, which can reduce the influence of intra-domain variations caused by camera shift to a large extent in an effective and efficient manner. Comprehensive experiments show that the proposed method achieves competitive performance compared with state-of-the-art methods on the benchmark datasets. Source code will be released at: https://github.com/ahuwangrui/MACM.

Crowd counting remains a challenging vision task due to the presence of several problems such as severe occlusions, perspective distortions and scale variations in the target scene. How to design an accurate and robust crowd counting estimator has attracted intensive research interest in the past few decades. It is well-known that learning rich features representation is crucial for crowd counting. However, the existing neural-networks-based methods only employ CNN features extracted from the last convolutional layer, and the useful hierarchical information contained in the CNN features is overlooked. To address this problem, we propose a CNN architecture based on the fully convolutional network, which is used to build an end-to-end density map estimation system by combining some of the meaningful convolutional features. Such a combination is exploited to effectively capture both the multi-scale and the multi-level information in complex scenes. Extensive experiments on most existing crowd counting dataset- s including ShanghaiTech Part A, ShanghaiTech Part B and UCF CC 50 demonstrate the effectiveness and the reliability of our approach.

Hashing based approximate nearest neighbor search has received considerable attention due to the demand of fast query for big multimedia data. Cross-modal hashing focuses on retrieval tasks across different modalities, which is more useful in practical applications. In this paper, we propose a novel two-stage cross-modal hashing method, referred to as Multi-Kernel Supervised Hashing with Graph Regularization (MKSRH). To better capture the essential attribute of original data, MKSRH first maps original data to a kernel space constructed by a linear combination of multiple kernel functions. Then the preliminary hash functions are learned using the Adaboost framework in the kernel space. To produce more accurate hash codes, the obtained hash function are then refined using a graph regularization based strategy. Experimental results on two canonical datasets show that MKSRH significantly outperforms than some typical cross-modal hashing methods, demonstrating the effectiveness and the superiority of the proposed approach.

As a promising alternative to traditional search techniques, hashing‐based approximate nearest neighbour search provides an applicable solution for big data. Most existing efforts are devoted to finding better projections to preserve the neighbouring structure of original data points in Hamming space, but ignore the quantisation procedure which may lead to the breakdown of the neighbouring structure maintained in the projection stage. To address this issue, the authors propose a novel multi‐bit quantisation (MBQ) method using a Matthews correlation coefficient (MCC) term and a regularisation term. The authors' method utilises the neighbouring relationship and the distribution information of original data points instead of the projection dimension usually used in the previous MBQ methods to adaptively learn optimal quantisation thresholds, and allocates multiple bits per projection dimension in terms of the learned thresholds. Experiments on two typical image data sets demonstrate that the proposed method effectively preserves the similarity between data points in the original feature space and outperforms state‐of‐the‐art quantisation methods.

Band selection is an important preprocessing technique for hyperspectral imagery, through which a subset of critical and representative spectral bands can be selected from a raw image cube for compact yet effect representation. Among the valid selection strategies, performing band selection in an unsupervised manner is usually considered more general due to its application-independent characteristic. This letter proposed a novel unsupervised hyperspectral band selector that can capture the interband redundancy nature of hyperspectral images through low-rank modeling. Experiments on three real-world hyperspectral data sets demonstrated that the proposed band selector can generate band subsets suitable in the context of hyperspectral pixel classification.

This paper investigates graph spectral approaches to the problem of point pattern matching. Specifically, we concentrate on the issue of how to effectively use graph spectral properties to characterize point patterns in the presence of positional jitter and outliers. A novel local spectral descriptor is proposed to represent the attribute domain of feature points. For a point in a given point-set, weight graphs are constructed on its neighboring points and then their normalized Laplacian matrices are computed. According to the known spectral radius of the normalized Laplacian matrix, the distribution of the eigenvalues of these normalized Laplacian matrices is summarized as a histogram to form a descriptor. The proposed spectral descriptor is finally combined with the approximate distance order for recovering correspondences between point-sets. Extensive experiments demonstrate the effectiveness of the proposed approach and its superiority to the existing methods.

The target of cross-modal hashing is to embed heterogeneous multimedia data into a common low-dimensional Hamming space, which plays a pivotal part in multimedia retrieval due to the emergence of big multimodal data. Recently, matrix factorization has achieved great success in cross-modal hashing. However, how to effectively use label information and local geometric structure is still a challenging problem for these approaches. To address this issue, we propose a cross-modal hashing method based on collective matrix factorization, which considers both the label consistency across different modalities and the local geometric consistency in each modality. These two elements are formulated as a graph Laplacian term in the objective function, leading to a substantial improvement on the discriminative power of latent semantic features obtained by collective matrix factorization. Moreover, the proposed method learns unified hash codes for different modalities of an instance to facilitate cross-modal search, and the objective function is solved using an iterative strategy. The experimental results on two benchmark data sets show the effectiveness of the proposed method and its superiority over state-of-the-art cross-modal hashing methods.

This study presents a local image descriptor based on spectral embedding. Specifically, the spectra of line graph are used to represent image edges, corners and edge points with big curvature. The authors theoretically analyse and experimentally verify that the spectra of line graph are robust to noise and are invariant to rotation and linear intensity changes. Based on such a fact, some local image descriptors are constructed using the spectra of line graph. Comparative experiments demonstrate the effectiveness of the proposed descriptor and its superiority to some state‐of‐the‐art descriptors under image rotation, image blur, viewpoint change, illumination change, JPEG compression and noise.

Spectral methods have been extensively studied for point pattern matching. In this work, we aim to render the spectral matching algorithm more robust for positional jitter and outliers. We concentrate on the issue of spectral representation for point patterns. A local structural descriptor, called the line graph spectral context, is proposed to characterize the attribute of point patterns, making it fundamentally different from the available representation approaches at the global level. For any given point, we first construct a line graph using its neighboring points. Then the eigenvalues of various matrix representations associated with the obtained line graph are used as the point descriptor. Furthermore, the similarities between the descriptors are evaluated by comparing their low dimensional embedding via the technique of multiview spectral embedding. The proposed descriptor is finally integrated with a graph-matching framework for establishing the correspondences. Comparative experiments conducted on both synthetic data and real-world images show the effectiveness of the proposed method, especially in the presence of positional jitter and outliers.

As a large proportion of the available video media concerns humans, human action retrieval is posed as a new topic in the domain of content-based video retrieval. For retrieving complex human actions, measuring the similarity between two videos represented by local features is a critical issue. In this paper, a fast and explicit feature correspondence approach is presented to compute the match cost serving as the similarity metric. Then the proposed similarity metric is embedded into the framework of manifold ranking for action retrieval. In contrast to the Bag-of-Words model and its variants, our method yields an encouraging improvement of accuracy on the KTH and the UCF YouTube datasets with reasonably efficient computation.