Currently, multimodal biometric recognition technology is receiving increasing attention. Most existing multimodal biometric recognition techniques require complete multimodal data during both the training and testing phases. However, due to various limitations, it is challenging to obtain complete and high-quality multimodal biometric data. To address this problem, we proposed a Mutual Distillation Framework (MDF) for palmprint and palmvein based multimodal biometric recognition with uncertain missing modality. Specifically, we firstly design an intra-inter modal data missing augmentation mechanism to generate heterogeneous missing samples. Moreover, a mutual knowledge distillation module is introduced to establish correlation of beneficial semantic information across different missing scenarios. In particular, we incorporate a Sample Semantic Discrepancy Guided mechanism within the mutual distillation framework, which calculates the semantic discrepancy between samples under corresponding heterogeneous missing patterns to encourage the model to focus on samples containing more complementary information. Experimental results demonstrate that the proposed model outperforms state-of-the-art incomplete multimodal learning models across three multimodal biometric benchmark datasets.

Multimodal biometric recognition has shown great potential in identity authentication tasks and has attracted increasing interest recently. Currently, most existing multimodal biometric recognition algorithms require test samples with complete multimodal data. However, it often encounters the problem of missing modality data and thus suffers severe performance degradation in practical scenarios. To this end, we proposed a hierarchical cross-modal image generation for palmprint and palmvein based multimodal biometric recognition with missing modality. First, a hierarchical cross-modal image generation model is designed to achieve the pixel alignment of different modalities and reconstruct the image information of missing modality. Specifically, a cross-modal texture transfer network is utilized to implement the texture style transformation between different modalities, and then a cross-modal structure generation network is proposed to establish the correlation mapping of structural information between different modalities. Second, multimodal dynamic sparse feature fusion model is presented to obtain more discriminative and reliable representations, which can also enhance the robustness of our proposed model to dynamic changes in image quality of different modalities. The proposed model is evaluated on three multimodal biometric benchmark datasets, and experimental results demonstrate that our proposed model outperforms recent mainstream incomplete multimodal learning models.