A straightforward method for multimodal palm-based authentication is to integrate palm shape into the system, which enhances reliability, security, and accuracy compared to unimodal methods. However, most existing methods rely on handcrafted feature extraction, which fails to fully exploit palm shape information. Moreover, there have been limited attempts to apply deep learning-based methods in this field. This paper explores a deep multimodal fusion method of palm vein (PV) and palm shape (PS) for authentication called multimodal contrastive learning channel-exchanging networks (MCCENet) to better utilize palm shape contour information. Specifically, we observe that the discriminative palm shape contour information is primarily captured in the shallow layers of the model, while the deeper layers tend to focus on irrelevant local high-level semantics. Based on this, we design hierarchical feature fusion (HFF), a module that enables inter-modal channel exchange at shallow layers. Further, we introduce a multimodal contrastive learning loss to align features across modalities, enhancing their representational embeddings. Extensive experiments across eight widely-used public datasets demonstrate that MCCENet achieves state-of-the-art performance in all cases.

In the conventional two-stage palm vein (PV) recognition the feature extraction is followed by region of interest (ROI) extraction, thus the recognition performance heavily depends on the robustness of the ROI extraction method, and the feature extractor lacks angular modeling capabilities. To address this, the relationship between oriented object detection and palm vein recognition is extensively studied in this paper. We propose a rotation-aware, one-stage PV recognition algorithm, where the rotated bounding box (RBB) of ROI is introduced as supervision information to guide the feature extractor and we annotate the ROI RBB of five public PV datasets. To the best of our knowledge, this is the first work that applies oriented object detection to one-stage PV recognition. Extensive experiments demonstrate that our method can achieve one-stage, high-accuracy PV recognition system and get remarkable results on five annotated datasets.

In hand-based biometrics, fingerprint, finger vein, finger knuckle print, palm print, palm vein, dorsal hand vein, and hand shape are the traits that are getting much attention. However, finger shape (FS), a forgettable trait, has not been studied specifically for identification purposes. In this work, we explore this content as a complement to the hand-based biometrics. Firstly, we annotate the FS on a publicly available finger vein dataset as the ground truth for finger semantic segmentation. Then we explore the finger semantic segmentation task on the annotated data and propose a lightweight network, namely FinSeg-Net (finger segmentation network). Finally, we conduct the FS authentication experiment based on four matching methods; experimental results show that the FS traits can achieve identity authentication. This work is the first study for FS biometrics specifically, and built the first FS dataset, which will be accessed via: https://github.com/SCUT-BIP-Lab/FinSeg.

Hand-based multimodal biometrics have attracted significant attention due to their high security and performance. However, existing methods fail to adequately decouple various hand biometric traits, limiting the extraction of unique features. Moreover, effective feature extraction for multiple hand traits remains a challenge. To address these issues, we propose a novel method for the precise decoupling of hand multimodal features called 'Normalized-Full-Palmar-Hand' and construct an authentication system based on this method. First, we propose HSANet, which accurately segments various hand regions with diverse backgrounds based on low-level details and high-level semantic information. Next, we establish two hand multimodal biometric databases with HSANet: SCUT Normalized-Full-Palmar-Hand Database Version 1 (SCUT_NFPH_v1) and Version 2 (SCUT_NFPH_v2). These databases include full hand images, semantic masks, and images of various hand biometric traits obtained from the same individual at the same scale, totaling 157,500 images. Third, we propose the Full Palmar Hand Authentication Network framework (FPHandNet) to extract unique features of multiple hand biometric traits. Finally, extensive experimental results, performed via the publicly available CASIA, IITD, COEP databases, and our proposed databases, validate the effectiveness of our methods.

More palm features, such as veins and shapes obtained from an enlarged contactless palm vein region of interest (ROI), have been shown to improve recognition performance. However, a few efforts have been made to adequately utilize these features for mining identity information. To address this issue, we propose a Region-Specific Network (RSNet) for contactless palm vein authentication. Our RSNet is a dual-branch structure for global and local feature extraction. Firstly, a Region-based Local feature Enhancement Block (RLEB) is proposed at the local branch to extract region-specific features. In the RLEB, the intermediate feature maps are divided into three asymmetrical patches based on the physiological characteristics of palm vein and palm shape for extracting diversified features, enhancing the local feature representation. Then, a Multi-scale Aggregation Block (MAB) is proposed that efficiently aggregates multi-scale features at a more granular level. Furthermore, to guide the global and local branches in learning complementary feature aspects, a difference loss is introduced to apply a soft subspace orthogonality constraint between the global and local vectors during training. The global branch is designed to assist the learning process of local features, without being adopted for inference. Extensive experiments have demonstrated the effectiveness and superiority of our method, and the RSNet achieves new State-Of-The-Art (SOTA) authentication performance on seven public contactless palm vein databases in the open-set scenario.

Contactless palm vein has attracted significant attention for its high security, stability, and user-friendliness. However, current contactless palm vein recognition predominantly relies on databases collected from platforms with spatial and temporal constrained design, which inadequately reflect relaxed palm vein imaging circumstances. This paper proposes a novel manner called on-the-fly palm vein that frees the user’s palm from spatial and temporal constraints, enabling palm vein recognition under unconstrained and weak-cooperative conditions. Firstly, Designing efficient and user-friendly palm vein imaging and authentication via two dynamic palm motions is proposed, resulting in an on-the-fly palm vein recognition platform. Next, a large-scale and challenging palm vein database, SCUT Palm Vein Database Version 1 (SCUT_PV_v1), is constructed. It is the first palm vein database with images collected under unconstrained and weak-cooperative conditions, encompassing a wider range of palm pose variations, grayscale variations, and lower-quality images. Finally, a lightweight and efficient Adaptive Margin Palm Vein Authentication Network (AMPVNet) is proposed as a baseline for the SCUT_PV_v1, where a vein pattern-specific convolutional neural network (CNN) is designed to extract features and a tailored online data augmentation method, combining Random Perspective Transformation (RPT) with Random Grayscale Adjustment (RGA), is proposed to enrich the diversify of out-of-plane palm pose and grayscale variations. Extensive experimental results demonstrate the effectiveness of our proposed methods. As the first work for palm vein recognition under unconstrained and weak-cooperation conditions, the AMPVNet achieves a promising accuracy and computation result while maintaining robustness to palm pose and grayscale variations. The SCUT_PV_ v1 database will be public at https://github.com/SCUT-BIP-Lab/SCUT_PV_v1.