This talk surveys my research journey toward building reliable machine learning systems that behave safely and predictably in the open world. While modern machine learning models—including foundation models (FMs)—have demonstrated unprecedented capabilities, they often suffer from reliability failures under distribution shift, leading to overconfident mispredictions, hallucinated generations, or susceptibility to adversarial prompts. My research rethinks reliability not as an afterthought, but as a first-class algorithmic principle, to be optimized alongside accuracy with minimal human supervision. The talk is organized around three key threads. To respect the allotted 20-30 minutes, the first and second parts will be briefly discussed. 1. Unknown-Aware Learning via Outlier Synthesis. I introduce a class of learning algorithms that synthesize “virtual outliers” in representation or pixel space to explicitly teach models what they don’t know. This includes the VOS, NPOS, and Dream-OOD frameworks, which shape the energy landscape around in-distribution data to avoid overconfidence on OOD. 2. Learning in the Wild with Unlabeled Data. I present theoretical insights and practical algorithms for leveraging unlabeled in-the-wild data to improve reliability. This includes SAL framework, which uses a gradient-based spectral method to separate potential outliers, and SCONE, which handles semantic and covariate shifts via constrained optimization. These results turn unlabeled data contamination into a learning signal. 3. Reliable Foundation Models. I explore reliability failures in LLMs and multimodal systems. I introduce HaloScope for hallucination detection via subspace separation on LLM representations, and TSV that performs LLM latent steering for improved hallucination detection. I will also briefly cover the LLM security and alignment, which includes VLMGuard for detecting malicious prompts in vision-language models and a data-centric paradigm for AI alignment through source-aware feedback cleaning. Throughout the talk, I highlight how representation learning, data generation, and theoretical guarantees intersect to produce scalable, label-efficient reliability methods. I will also reflect on my broader vision: designing proactive and collaborative AI systems that anticipate uncertainty and support rich human-AI interaction—especially for underrepresented communities and emerging scientific domains. This talk will be accessible to a broad AAAI audience, combining foundational algorithmic insights with real-world applications and forward-looking perspectives on the future of responsible AI.

Vision-Language Models (VLMs) have revolutionized multimedia applications by enabling open-vocabulary search and generation. While significant strides have been made in developing powerful VLMs, the field remains in the early stages of rigorously evaluating and understanding their real-world vulnerabilities, both empirically and theoretically. In response to this important but under-explored gap, the workshop aims to provide a forum for researchers to share advanced in theories, algorithms and applications of trustworthy multi-modal learning, fostering diverse viewpoints on core principles and emerging techniques for developing trustworthy VLMs in the wild. The workshop provides a platform for researchers to showcase their work, exchange ideas, and foster potential collaborations. Additionally, it serves as a valuable opportunity for practitioners to stay abreast of the latest developments in developing trustworthy vision-language models.

Secure and efficient service authentication is vital for trustable interactions between users and service nodes in decentralized metaverse environments. However, conventional PKI-based authentication methods face limitations such as high query latency, privacy leakage risks, and centralized trust dependencies, making them unsuitable for large-scale, real-time metaverse services. To address these challenges, we propose ESAChain, a novel blockchain-based authentication framework that e nsures lightweight, decentralized, and privacy-preserving identity verification. Specifically, we propose a Mutually Exclusive Cuckoo Filter (MECF) integrated with a Filter Hash Chain (FHC), which provides lightweight data structures and efficient querying capabilities for certificate status. Furthermore, we design a trust-decay-based Delegated Proof-of-Stake (TD-DPoS) consensus mechanism to maintain the integrity and reliability of certificate status data by dynamically adjusting node trust values and decaying votes to prevent single-node dominance. We also incorporate a blind-signature-based authentication mechanism to enhance privacy-preserving identity authentication by preventing tracking of certificate verification requests. Extensive simulation experiments and security analyses demonstrate that ESAChain significantly reduces query latency and data transmission overhead, enhances consensus robustness, and provides an efficient, trustworthy, and privacy-preserving authentication solution for secure metaverse services.

Multimodal large language models (MLLMs) have shown promising capabilities but struggle under distribution shifts, where evaluation data differ from instruction tuning distributions. Although previous works have provided empirical evaluations, we argue that establishing a formal framework that can characterize and quantify the risk of MLLMs is necessary to ensure the safe and reliable application of MLLMs in the real world. By taking an information-theoretic perspective, we propose the first theoretical framework that enables the quantification of the maximum risk of MLLMs under distribution shifts. Central to our framework is the introduction of Effective Mutual Information (EMI), a principled metric that quantifies the relevance between input queries and model responses. We derive an upper bound for the EMI difference between in-distribution (ID) and out-of-distribution (OOD) data, connecting it to visual and textual distributional discrepancies. Extensive experiments on real benchmark datasets, spanning 61 shift scenarios, empirically validate our theoretical insights.

Hallucinations in LLMs pose a significant concern to their safe deployment in real-world applications. Recent approaches have leveraged the latent space of LLMs for hallucination detection, but their embeddings, optimized for linguistic coherence rather than factual accuracy, often fail to clearly separate truthful and hallucinated content. To this end, we propose the **T**ruthfulness **S**eparator **V**ector (**TSV**), a lightweight and flexible steering vector that reshapes the LLM’s representation space during inference to enhance the separation between truthful and hallucinated outputs, without altering model parameters. Our two-stage framework first trains TSV on a small set of labeled exemplars to form compact and well-separated clusters. It then augments the exemplar set with unlabeled LLM generations, employing an optimal transport-based algorithm for pseudo-labeling combined with a confidence-based filtering process. Extensive experiments demonstrate that TSV achieves state-of-the-art performance with minimal labeled data, exhibiting strong generalization across datasets and providing a practical solution for real-world LLM applications.

Drones are a viable solution for searching and rescuing people in distress at sea. However, traditional methods of unmanned aerial vehicle (UAV) search and rescue have slow emergency response speed, limited task environment perception and decision-making ability, high search and rescue task cost, and low efficiency. Therefore, this letter proposes a distributed drone dynamic ocean search and rescue framework that integrates deep reinforcement learning (DRL) and predicts the drift probability range based on ocean data. The existence probability of each area is calculated through a mixture of Gaussian distributions, and the possibility changes dynamically with ocean currents. The predicted probability will be input into the learning network in a matrix with the same dimension as the environment and trained using DDQN. In addition, we introduce a distance penalty in the reward function and establish a dynamic greedy strategy to accelerate learning and improve training accuracy and stability. This framework can dynamically identify the location of potential victims and guide drone groups to coordinate search and rescue operations. In particular, to adapt to the decision-making quality and synergy of DRL under distributed drone swarms, we introduce a distributed priority experience replay mechanism to share the best experiences of drones in different scenarios. By comparing various methods, this study shows excellent comprehensive performance and effectively improves search and rescue efficiency.

Reward modeling, crucial for aligning large language models (LLMs) with human preferences, is often bottlenecked by the high cost of preference data. Existing textual data synthesis methods are computationally expensive. We propose a novel framework LENS for synthesizing preference data directly in the LLM's latent embedding space. Our method employs a Variational Autoencoder (VAE) to learn a structured latent representation of response embeddings. By performing controlled perturbations in this latent space and decoding back to the embedding space, we efficiently generate diverse, semantically consistent synthetic preference pairs, bypassing costly text generation and annotation. We provide theoretical guarantees that our synthesized pairs approximately preserve original preference ordering and improve reward model generalization. Empirically, our latent-space synthesis significantly outperforms text-based augmentation on standard benchmarks, achieving superior results while being 18× faster in generation and using a 16,000× smaller model. Our work offers a scalable and effective alternative for enhancing reward modeling through efficient data augmentation. Code is publicly available at https://github.com/deeplearning-wisc/lens.

Deep learning-based palmprint recognition methods take performance to the next level. However, most current methods rely on samples with clean labels. Noisy labels are difficult to avoid in practical applications and may affect the reliability of models, which poses a big challenge. In this paper, we propose a novel Multi-stage Noisy Label Selection and Correction (MNLSC) framework to address this issue. Three stages are proposed to improve the robustness of palmprint recognition. Clean simple samples are firstly selected based on self-supervised learning. A Fourier-based module is constructed to select clean hard samples. A pototype-based module is further introduced for selecting noisy labels from the remaining samples and correcting them. Finally, the model is trained by using clean and corrected labels to improve the performance. Experiments are conducted on several constrained and unconstrained palmprint databases. The results demonstrate the superiority of our method over other methods in dealing with different noise rates. Compared with the baseline method, the accuracy can be improved by up to 33.45% when there are 60% noisy labels.

Using unlabeled data to regularize the machine learning models has demonstrated promise for improving safety and reliability in detecting out-of-distribution (OOD) data. Harnessing the power of unlabeled in-the-wild data is non-trivial due to the heterogeneity of both in-distribution (ID) and OOD data. This lack of a clean set of OOD samples poses significant challenges in learning an optimal OOD classifier. Currently, there is a lack of research on formally understanding how unlabeled data helps OOD detection. This paper bridges the gap by introducing a new learning framework SAL (Separate And Learn) that offers both strong theoretical guarantees and empirical effectiveness. The framework separates candidate outliers from the unlabeled data and then trains an OOD classifier using the candidate outliers and the labeled ID data. Theoretically, we provide rigorous error bounds from the lens of separability and learnability, formally justifying the two components in our algorithm. Our theory shows that SAL can separate the candidate outliers with small error rates, which leads to a generalization guarantee for the learned OOD classifier. Empirically, SAL achieves state-of-the-art performance on common benchmarks, reinforcing our theoretical insights. Code is publicly available at https://github.com/deeplearning-wisc/sal.

Detecting data points deviating from the training distribution is pivotal for ensuring reliable machine learning. Extensive research has been dedicated to the challenge, spanning classical anomaly detection techniques to contemporary out-of-distribution (OOD) detection approaches. While OOD detection commonly relies on supervised learning from a labeled in-distribution (ID) dataset, anomaly detection may treat the entire ID data as a single class and disregard ID labels. This fundamental distinction raises a significant question that has yet to be rigorously explored: when and how does ID label help OOD detection? This paper bridges this gap by offering a formal understanding to theoretically delineate the impact of ID labels on OOD detection. We employ a graph-theoretic approach, rigorously analyzing the separability of ID data from OOD data in a closed-form manner. Key to our approach is the characterization of data representations through spectral decomposition on the graph. Leveraging these representations, we establish a provable error bound that compares the OOD detection performance with and without ID labels, unveiling conditions for achieving enhanced OOD detection. Lastly, we present empirical results on both simulated and real datasets, validating theoretical guarantees and reinforcing our insights.

The surge in applications of large language models (LLMs) has prompted concerns about the generation of misleading or fabricated information, known as hallucinations. Therefore, detecting hallucinations has become critical to maintaining trust in LLM-generated content. A primary challenge in learning a truthfulness classifier is the lack of a large amount of labeled truthful and hallucinated data. To address the challenge, we introduce HaloScope, a novel learning framework that leverages the unlabeled LLM generations in the wild for hallucination detection. Such unlabeled data arises freely upon deploying LLMs in the open world, and consists of both truthful and hallucinated information. To harness the unlabeled data, we present an automated scoring function for distinguishing between truthful and untruthful generations within unlabeled mixture data, thereby enabling the training of a binary classifier on top. Importantly, our framework does not require extra data collection and human annotations, offering strong flexibility and practicality for real-world applications. Extensive experiments show that HaloScope can achieve superior hallucination detection performance, outperforming the competitive rivals by a significant margin.

Modern machine learning models deployed in the wild can encounter both covariate and semantic shifts, giving rise to the problems of out-of-distribution (OOD) generalization and OOD detection respectively. While both problems have received significant research attention lately, they have been pursued independently. This may not be surprising, since the two tasks have seemingly conflicting goals. This paper provides a new unified approach that is capable of simultaneously generalizing to covariate shifts while robustly detecting semantic shifts. We propose a margin-based learning framework that exploits freely available unlabeled data in the wild that captures the environmental test-time OOD distributions under both covariate and semantic shifts. We show both empirically and theoretically that the proposed margin constraint is the key to achieving both OOD generalization and detection. Extensive experiments show the superiority of our framework, outperforming competitive baselines that specialize in either OOD generalization or OOD detection. Code is publicly available at https://github.com/deeplearning-wisc/scone.

Utilizing auxiliary outlier datasets to regularize the machine learning model has demonstrated promise for out-of-distribution (OOD) detection and safe prediction. Due to the labor intensity in data collection and cleaning, automating outlier data generation has been a long-desired alternative. Despite the appeal, generating photo-realistic outliers in the high dimensional pixel space has been an open challenge for the field. To tackle the problem, this paper proposes a new framework Dream-OOD, which enables imagining photo-realistic outliers by way of diffusion models, provided with only the in-distribution (ID) data and classes. Specifically, Dream-OOD learns a text-conditioned latent space based on ID data, and then samples outliers in the low-likelihood region via the latent, which can be decoded into images by the diffusion model. Different from prior works [16, 95], Dream-OOD enables visualizing and understanding the imagined outliers, directly in the pixel space. We conduct comprehensive quantitative and qualitative studies to understand the efficacy of Dream-OOD, and show that training with the samples generated by Dream-OOD can significantly benefit OOD detection performance.

Although combining the Internet of Things (IoT) and industrial scenarios has brought about a technological revolution, it has also caused equipment security issues. Due to the characteristics of Industrial Internet of Things (IIoT) devices with a wide distribution, complex application scenarios, considerable differences in node performance, and device heterogeneity, spoofing attacks and third-party attacks are common. Identity authentication for IIoT devices can solve this dilemma. However, most existing authentication technologies involve a tradeoff between traditional centralized certificate issuance and sacrificing device storage resources, resulting in lower efficiency of IIoT device authentication, and the process is complicated. Therefore, ensuring the security and trustworthiness of device identities in the IIoT is imminent. In this article, we propose an IIoT device authentication scheme based on an editable blockchain, that can solve the problem of the device’s low energy while satisfying the usage needs of large-scale scenarios. In particular, we created a suite of secure, efficient, and innovative technical solutions for this protocol. First, to solve the problem of the authentication difficulty between industrial devices, we propose a lightweight identity authentication protocol called BLMA. Moreover, we propose the validate-practical Byzantine fault tolerance algorithm and introduce the online and offline signature algorithm to reduce communication overhead and resource consumption between devices. Finally, considering the top security and dynamics of the IIoT environment, we use the chameleon hash function to build a hash chain of authentication results. Extensive simulation and experimental results demonstrate the reliability of our protocol.

Building reliable object detectors that can detect out-of-distribution (OOD) objects is critical yet underexplored. One of the key challenges is that models lack supervision signals from unknown data, producing over-confident predictions on OOD objects. We propose a new unknown-aware object detection framework through Spatial-Temporal Unknown Distillation (STUD), which dis-tills unknown objects from videos in the wild and meaningfully regularizes the model's decision boundary. STUD first identifies the unknown candidate object proposals in the spatial dimension, and then aggregates the candidates across multiple video frames to form a diverse set of unknown objects near the decision boundary. Along-side, we employ an energy-based uncertainty regularization loss, which contrastively shapes the uncertainty space between the in-distribution and distilled unknown objects. STUD establishes the state-of-the-art performance on OOD detection tasks for object detection, reducing the FPR95 score by over 10% compared to the previous best method. Code is available at https://github.com/deep/earning-wisc/stud.

Knowledge distillation has shown great effectiveness for improving neural architecture search (NAS). Mutual knowledge distillation (MKD), where a group of models mutually generate knowledge to train each other, has achieved promising results in many applications. In existing MKD methods, mutual knowledge distillation is performed between models without scrutiny: a worse-performing model is allowed to generate knowledge to train a better-performing model, which may lead to collective failures. To address this problem, we propose a performance-aware MKD (PAMKD) approach for NAS, where knowledge generated by model$A$is allowed to train model$B$only if the performance of$A$is better than B. We propose a three-level optimization framework to formulate PAMKD, where three learning stages are performed end-to-end: 1) each model trains an initial model independently; 2) the initial models are evaluated on a validation set and better-performing models generate knowledge to train worse-performing models; 3) architectures are updated by minimizing a validation loss. Experimental results on a variety of datasets demonstrate that our method is effective.

Detecting out-of-distribution (OOD) objects is indispensable for safely deploying object detectors in the wild. Although distance-based OOD detection methods have demonstrated promise in image classification, they remain largely unexplored in object-level OOD detection. This paper bridges the gap by proposing a distance-based framework for detecting OOD objects, which relies on the model-agnostic representation space and provides strong generality across different neural architectures. Our proposed framework SIREN contributes two novel components: (1) a representation learning component that uses a trainable loss function to shape the representations into a mixture of von Mises-Fisher (vMF) distributions on the unit hypersphere, and (2) a test-time OOD detection score leveraging the learned vMF distributions in a parametric or non-parametric way. SIREN achieves competitive performance on both the recent detection transformers and CNN-based models, improving the AUROC by a large margin compared to the previous best method. Code is publicly available at https://github.com/deeplearning-wisc/siren.

Out-of-distribution (OOD) detection is vital to safety-critical machine learning applications and has thus been extensively studied, with a plethora of methods developed in the literature. However, the field currently lacks a unified, strictly formulated, and comprehensive benchmark, which often results in unfair comparisons and inconclusive results. From the problem setting perspective, OOD detection is closely related to neighboring fields including anomaly detection (AD), open set recognition (OSR), and model uncertainty, since methods developed for one domain are often applicable to each other. To help the community to improve the evaluation and advance, we build a unified, well-structured codebase called OpenOOD, which implements over 30 methods developed in relevant fields and provides a comprehensive benchmark under the recently proposed generalized OOD detection framework. With a comprehensive comparison of these methods, we are gratified that the field has progressed significantly over the past few years, where both preprocessing methods and the orthogonal post-hoc methods show strong potential.

In adversarial training (AT), the main focus has been the objective and optimizer while the model has been less studied, so that the models being used are still those classic ones in standard training (ST). Classic network architectures (NAs) are generally worse than searched NA in ST, which should be the same in AT. In this paper, we argue that NA and AT cannot be handled independently, since given a dataset, the optimal NA in ST would be no longer optimal in AT. That being said, AT is time-consuming itself; if we directly search NAs in AT over large search spaces, the computation will be practically infeasible. Thus, we propose diverse-structured network (DS-Net), to significantly reduce the size of the search space: instead of low-level operations, we only consider predefined atomic blocks, where an atomic block is a time-tested building block like the residual block. There are only a few atomic blocks and thus we can weight all atomic blocks rather than find the best one in a searched block of DS-Net, which is an essential tradeoff between exploring diverse structures and exploiting the best structures. Empirical results demonstrate the advantages of DS-Net, i.e., weighting the atomic blocks.

MOTIVATION: Cryo-Electron Tomography (cryo-ET) is a 3D bioimaging tool that visualizes the structural and spatial organization of macromolecules at a near-native state in single cells, which has broad applications in life science. However, the systematic structural recognition and recovery of macromolecules captured by cryo-ET are difficult due to high structural complexity and imaging limits. Deep learning-based subtomogram classification has played critical roles for such tasks. As supervised approaches, however, their performance relies on sufficient and laborious annotation on a large training dataset. RESULTS: To alleviate this major labeling burden, we proposed a Hybrid Active Learning (HAL) framework for querying subtomograms for labeling from a large unlabeled subtomogram pool. Firstly, HAL adopts uncertainty sampling to select the subtomograms that have the most uncertain predictions. This strategy enforces the model to be aware of the inductive bias during classification and subtomogram selection, which satisfies the discriminativeness principle in AL literature. Moreover, to mitigate the sampling bias caused by such strategy, a discriminator is introduced to judge if a certain subtomogram is labeled or unlabeled and subsequently the model queries the subtomogram that have higher probabilities to be unlabeled. Such query strategy encourages to match the data distribution between the labeled and unlabeled subtomogram samples, which essentially encodes the representativeness criterion into the subtomogram selection process. Additionally, HAL introduces a subset sampling strategy to improve the diversity of the query set, so that the information overlap is decreased between the queried batches and the algorithmic efficiency is improved. Our experiments on subtomogram classification tasks using both simulated and real data demonstrate that we can achieve comparable testing performance (on average only 3% accuracy drop) by using less than 30% of the labeled subtomograms, which shows a very promising result for subtomogram classification task with limited labeling resources. AVAILABILITY AND IMPLEMENTATION: https://github.com/xulabs/aitom. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

As an effective method of biometrics, palmprint recognition allows the safe identity recognition of humans without spatial and temporal limitations. To build a more robust palmprint recognition system, recent promising Convolutional Neural Networks (CNN) has been incorporated for better palmprint feature extraction and representation. However, the increasing number of palmprint datasets presents us with a cross-domain recognition problem where the upcoming images may come from different imaging conditions compared to the registered palmprints, which will undermine the recognition accuracy significantly. As a supervised approach, the performance of CNN-based model depends on the availability of data and labels from the same domain, which is hard for transferring recognition. To keep the outperforming recognition result of CNN-based models, we propose a novel Regularized Adversarial Domain Adaptative Hashing method (R-ADAH) for cross-domain palmprint recognition based on Deep Hashing Network (DHN). During training, the Maximum Mean Discrepancy (MMD) is incorporated for better adaptive performance. In this scenario, we only train a DHN on the source domain. With the adversarial training, the target network is becoming adaptive to the unlabeled palmprint images with more stable training, unbiased sample gradient and less sensitivity to the hyper-parameter tuning when only domain-specific label is provided. Extensive validation experiments are conducted on benchmark datasets and our self-collected palmprint datasets by mobile phones to test the performance of our model. The results show a promising increase of the recognition performance.

We study the problem of node classification on graphs with few-shot novel labels, which has two distinctive properties: (1) There are novel labels to emerge in the graph; (2) The novel labels have only a few representative nodes for training a classifier. The study of this problem is instructive and corresponds to many applications such as recommendations for newly formed groups with only a few users in online social networks. To cope with this problem, we propose a novel Meta Transformed Network Embedding framework (MetaTNE), which consists of three modules: (1) A \emph{structural module} provides each node a latent representation according to the graph structure. (2) A \emph{meta-learning module} captures the relationships between the graph structure and the node labels as prior knowledge in a meta-learning manner. Additionally, we introduce an \emph{embedding transformation function} that remedies the deficiency of the straightforward use of meta-learning. Inherently, the meta-learned prior knowledge can be used to facilitate the learning of few-shot novel labels. (3) An \emph{optimization module} employs a simple yet effective scheduling strategy to train the above two modules with a balance between graph structure learning and meta-learning. Experiments on four real-world datasets show that MetaTNE brings a huge improvement over the state-of-the-art methods.

As a promising topic of biometrics, palmprint recognition helps to effectively verify a person's identity, which is suitable for building a security system. Recent progress has achieved high recognition accuracy in different benchmark datasets due to deep learning. However, these applications are almost implemented in one dataset with iterative training epochs to help neural network generalize. When applied practically where many new users' palmprints registered in sequence, deep learning-based recognition systems cannot avoid the problem of catastrophic forgetting. In this paper, we propose a continual learning framework based on reinforcement learning to dynamically expand the neural network when facing newly registered palmprints without costly retraining or fine-tuning. Experiments on different datasets demonstrate the high adaptability of our model that is promising for solving the forgetting attack of every biometric system.

Palmprint recognition allows accurate identity verification to build a security system. Recently, researchers introduce deep learning to this area that largely improves the recognition accuracy. However, as a supervised approach, its performance relies on availability of data and labels for every registered identity. For large-scale security systems, after image acquisition, we need to check the whole dataset and manually assign labels through comparison, which is a time-consuming task. Besides, labelling some redundant training samples contributes little to the recognition result. In this paper, we introduce an active learning framework to select the best sample set for label assignment. We regard the active learning as a binary classification task and attempt to make the labeled and unlabeled set indistinguishable. Experiments on different datasets demonstrate our model can reduce the annotation cost while achieving comparable recognition performance.

Recently, excellent palmprint recognition algorithms have emerged and achieved satisfactory performance. However, cross-domain palmprint recognition is rarely considered. In this paper, we proposed transfer autoencoder for crossdomain palmprint recognition. Convolutional autoencoders were firstly used to extract low-dimensional features. A discriminator was then introduced to reduce the gap of two domains. The autoencoders and discriminator were alternately trained, and finally the features with the same distribution were extracted. The databases collected from different environments are defined as source and target domains, respectively. Based on the labels in source domain, unsupervised identification of target domain can be achieved. The experiments were performed on 24 cross-domain pairs composed by multispectral database and our self-built uncontrolled databases. The results show that transfer autoencoder can greatly improve cross-domain recognition accuracy, up to 23.26%. At the same time, the accuracy in a single domain can reach over 99% in controlled database.

Palmprint is one of the discriminant biometrical features of humans. Recognizing palmprints in complex environments is a significant multimedia task, which is highly suitable for applications in information security and forensics. Recently, deep learning-based recognition methods have improved the accuracy and robustness of recognition results to a new level. However, obtaining the required large amount of training data and labels is impracticable in practical scenarios. Therefore, in this paper, we exploit few-shot learning for palmprint recognition. We propose Meta-Siamese network based on Siamese network. Specifically, we train this network episodically with a more flexible framework to learn both the feature embedding and the deep similarity metric function. Moreover, we extend our model to zero-shot recognition tasks based on deep hashing network. Experiment result shows competitive improvements compared to baseline methods in eight different datasets.

At present, the fusion of different unimodal biometrics has attracted increasing attention from researchers, who are dedicated to the practical application of biometrics. In this paper, we explored a multi-biometric algorithm that integrates palmprints and dorsal hand veins (DHV). Palmprint recognition has a rather high accuracy and reliability, and the most significant advantage of DHV recognition is the biopsy (Liveness detection). In order to combine the advantages of both and implement the fusion method, deep learning and graph matching were, respectively, introduced to identify palmprint and DHV. Upon using the deep hashing network (DHN), biometric images can be encoded as 128-bit codes. Then, the Hamming distances were used to represent the similarity of two codes. Biometric graph matching (BGM) can obtain three discriminative features for classification. In order to improve the accuracy of open-set recognition, in multi-modal fusion, the score-level fusion of DHN and BGM was performed and authentication was provided by support vector machine (SVM). Furthermore, based on DHN, all four levels of fusion strategies were used for multi-modal recognition of palmprint and DHV. Evaluation experiments and comprehensive comparisons were conducted on various commonly used datasets, and the promising results were obtained in this case where the equal error rates (EERs) of both palmprint recognition and multi-biometrics equal 0, demonstrating the great superiority of DHN in biometric verification.