Visible-infrared person re-identification is a challenging task in video surveillance. Most existing works achieve performance gains by aligning feature distributions or image styles across modalities, whereas the multi-granularity information and domain knowledge are usually neglected. Motivated by these issues, we propose a novel modality-aware domain alignment network (MDANet) for visible-infrared person re-identification (VI-ReID), which utilizes global-local context cues and the generalized domain alignment strategy to solve modal differences and poor generalization. Firstly, modality-aware global-local context attention (MGLCA) is proposed to obtain multi-granularity context features and identity-aware patterns. Secondly, we present a generalized domain alignment learning head (GDALH) to relieve the modality discrepancy and enhance the generalization of MDANet, whose core idea is to enrich feature diversity in the domain alignment procedure. Finally, the entire network model is trained by proposing cross-modality circle, classification, and domain alignment losses in an end-to-end fashion. We conduct comprehensive experiments on two standards and their corrupted VI-ReID datasets to validate the robustness and generalization of our approach. MDANet is obviously superior to the most state-of-the-art methods. Specifically, the proposed method can gain 8.86% and 2.50% in Rank-1 accuracy on SYSU-MM01 (all-search and single-shot mode) and RegDB (infrared to visible mode) datasets, respectively. The source code will be made available soon.

Privacy with the use of face images is becoming a major concern in civilians’ applications. Recent studies have exploited privacy protection methods by means of facial attributes editing or de-identifying face images. Altering attributes causes loss of information for facial analysis while most de-identification studies did not quantitatively evaluate how well facial attributes are preserved. Moreover, state-of-the-art face analysis utilized 3D information for better performance. Existing face privacy studies only focusing in 2D domain is a key limitation towards the compatibility of more advanced 3D face analysis. This paper presents the first study on the possibility of 3D face de-identification with preserving facial attributes. We systematically evaluate the performance of 2D/3D face/facial attribute recognition and develop 2D/3D de-identification methods with preserving facial attributes using Auto Encoder and Generative Adversarial Networks approaches. We present comprehensive and reproducible experimental results using a publicly available 3D face database with facial attribute annotations for benchmarking and further research. https://github.com/kevinhmcheng/3d-face-de-id

Contactless 3D finger knuckle pattern is a new biometric identifier which offers highly discriminative features for the finger knuckle based personal identification. State-of-the-art methods for object recognition, a more generic problem, employ deep neural network based approaches and demonstrate superior effectiveness. However, any direct applications from those methods do not outperform specialized hand-crafted feature description approaches for the problem addressed in this paper. In addition, such deep neural network based methods have to address challenges associated with emerging biometrics, e.g. availability of very limited training data, large intra-class or train-test sample variations as observed for the real applications, etc. This paper attempts to address the above challenges and introduces a new deep neural network based approach for the contactless 3D finger knuckle identification. Our approach simultaneously encodes and incorporates deep features from multiple scales to form a more robust deep feature representation. Such collaborative feature representations are robustly matched using an efficient alignment scheme with a fully convolutional architecture to accommodate involuntary finger variations during the contactless imaging. Comparative experimental results in the two-session 3D finger knuckle images database, acquired from over 200 subjects and is publicly introduced from this paper, illustrate superior performance over the state-of-the-art methods, e.g. offering ~22% GAR improvement at extremely low FAR under challenging comparison scenarios. Additional experiments in other publicly available databases including 3D palmprint, 3D fingerprint, and 2D finger knuckle further validate the effectiveness and demonstrate the generalizability of the proposed approach.

Contactless 3D hand biometrics offers hygienic and convenient approaches for biometric recognition. This paper investigates a distinctive feature representation using 3D surface normal information for more accurate 3D hand biometric identification. Prior research on contactless 3D hand biometric identification largely incorporates 3D depth and surface curvature information to recover discriminative features. Our investigation presented in this paper indicates that extracting distinctive features from surface normal information, which can also be directly obtained from low-cost photometric stereo based imaging systems, can offer a computationally simpler alternative and is therefore highly desirable. The directions of neighbouring surface normal vectors can encode frequently observed irregular ridge and valley regions, which can enable more accurate surface feature description. Comparative experimental results presented in this paper validates the effectiveness of the proposed approach.

Study on finger knuckle patterns has attracted increasing attention for the automated biometric identification. However, finger knuckle pattern is essentially a 3D biometric identifier and the usage or availability of only 2D finger knuckle databases in the literature is the key limitation to avail full potential from this biometric identifier. This paper therefore introduces (first) contactless 3D finger knuckle database in public domain, which is acquired from 130 different subjects in two-session imaging using photometric stereo approach. This paper investigates on the 3D information from the finger knuckle patterns and introduces a new feature descriptor to extract discriminative 3D features for more accurate 3D finger knuckle matching. An individuality model for the proposed feature descriptor is also presented. Comparative experimental results using the state-of-the-art feature extraction methods on this challenging 3D finger knuckle database validate the effectiveness of our approach. Although our feature descriptor is designed for 3D finger knuckle patterns, it is also attractive for other hand-based biometric identifiers with similar patterns such as the palmprint and fingerprint. This observation is validated from the outperforming results, using the state-of-the-art pixel-wise 3D palmprint and 3D fingerprint feature descriptors, on other publicly available datasets.

Contactless 3D finger knuckle is a new biometric identifier with a lot of potentials, which can provide an accurate, efficient and convenient alternative for the personal identification. The current 3D finger knuckle recognition methods are limited by computationally complex or inefficient matching algorithms, which attempt to compute the matching scores from all possible translational and rotational parameters for matching a pair of templates. The strength of such approach lies in its simplicity and reliability for accurately matching intra-class samples, but expensive computational time is required. Furthermore, attempting on excessive numbers of translational and rotational parameters can also degrade the overall recognition accuracy because the imposter matches can be increased. In fact, this conventional matching approach is commonly adopted in many biometric studies, but its drawbacks have not received adequate attention. This paper addresses such 3D finger knuckle recognition problem by developing a more efficient matching approach using surface key points extracted from 3D finger knuckle surfaces. Our comparative experimental results with the state-of-the art method on a publicly available 3D finger knuckle database indicates that our approach can offer over 23 times faster with performance improvement on the accuracy. Although the focus of our work is on 3D finger knuckle recognition, we also present the performance of our method on other publicly available databases with similar 3D biometric patterns including 3D palmprint and 3D fingerprint, to validate the effectiveness of the proposed approach.

Photometric stereo offers a single camera based approach to recover 3D information and has attracted wide range of applications in computer vision. Presence of non-Lambertian reflections in almost all the real-world objects limits the usage of the Lambertian model for surface normal vector estimation. Previous methods proposed to address such non-Lambertian phenomena employ an outlier rejection approach while more recent methods introduce BRDF models which can generate more accurate results. However, results with comparable accuracy can also be achieved by simply filtering the observed intensity values. This paper presents two novel outlier rejection techniques which attempt to identify the data which are more reliable and likely to be Lambertian. In the first technique, observed intensity values with less reliability are automatically eliminated. This reliability is determined by the responses from a newly introduced inter-relationship function. In the second technique, those photometric ratio equations which are less likely to be Lambertian are identified by observing the residue of the equations. By eliminating the data which is unreliable and likely to be non-Lambertian, surface normal vectors are more accurately estimated. Our comparative and reproducible experimental results using both real and synthetic datasets illustrate superior performance over the state-of-the-art methods, which validates our theoretical arguments presented in this paper.

Accurate and efficient feature descriptors are crucial for the success of many pattern recognition tasks including human identification. Existing studies have shown that features extracted from 3D depth images are more reliable than those from 2D intensity images because intensity images are generally noisy and sensitive to illumination variation, which is challenging for many real-world applications like biometrics. Recently introduced 3D feature descriptors like Binary Shape and Surface Code have been shown improved effectiveness for 3D palm recognition. However, both methods lack theoretical support for the construction of the feature templates, which limits their matching accuracy and efficiency. In this paper, we further advance the Surface Code method and introduce the Efficient Surface Code, which describes whether a point tends to be concave or convex using only one bit per pixel. Our investigation also reveals that the discriminative abilities of the convex and concave regions are not necessarily equal. For example, line patterns on human palms and finger knuckles are expected to reveal more discriminative information than non-line regions. Therefore, we also propose a weighted similarity method in conjunction with the Efficient Surface Code instead of the traditional Hamming distance adopted in both Binary Shape and Surface Code. Comparative experimental results on both 3D palmprint and 3D finger knuckle databases illustrate superior performance to the aforementioned state-of-the-art methods, which validates our theoretical arguments.