Periocular and face are two common biometric modalities for identity management. Recently, the emergence of conditional biometrics has enabled the exploitation of the correlation between face and periocular to enhance each modality’s performance, in which we coin intra-modal matching in this paper. However, limitations arise in each modality, particularly when wearing sunglasses or helmets, causing the absence of periocular or facial occlusion. A biometric system empowered with inter-modal matching capability between periocular and face is essential to mitigate these challenges. This paper presents a novel reciprocal learning model that utilizes periocular and face conditioning to facilitate flexible intra-modal and inter-modal matching. To address the intra-modal matching challenge, we devise a lightweight Gated Convolutional Channel-wise Self-Attention Network that enables selective attention to shared salient periocular and face features. On the other hand, to bridge the modality gap without sacrificing the intra-modal matching performance, we propose a modality and augmentation-aware contrastive loss that incorporates semi-supervised positive sampling and alignment-specific logit rescaling. Extensive identification and verification experiments on five face-periocular datasets under the open-set protocol attest to the efficacy of our proposed methods.

Very low-resolution face recognition (VLRFR) poses unique challenges, such as tiny regions of interest and poor resolution due to extreme standoff distance or wide viewing angle of the acquisition devices. In this paper, we study principled approaches to elevate the recognizability of a face in the embedding space instead of the visual quality. We first formulate a robust learning-based face recognizability measure, namely recognizability index (RI), based on two criteria: (i) proximity of each face embedding against the unrecognizable faces cluster center and (ii) closeness of each face embedding against its positive and negative class prototypes. We then devise an index diversion loss to push the hard-to-recognize face embedding with low RI away from unrecognizable faces cluster to boost the RI, which reflects better recognizability. Additionally, a perceptibility attention mechanism is introduced to attend to the most recognizable face regions, which offers better explanatory and discriminative traits for embedding learning. Our proposed model is trained end-to-end and simultaneously serves recognizability-aware embedding learning and face quality estimation. To address VLRFR, our extensive evaluations on three challenging low-resolution datasets and face quality assessment demonstrate the superiority of the proposed model over the state-of-the-art methods.

A neural network trained on a classification dataset often exhibits a higher vector norm of hidden layer features for in-distribution (ID) samples, while producing relatively lower norm values on unseen instances from out-of-distribution (OOD). Despite this intriguing phenomenon being utilized in many applications, the underlying cause has not been thoroughly investigated. In this study, we demystify this very phenomenon by scrutinizing the discriminative structures concealed in the intermediate layers of a neural network. Our analysis leads to the following discoveries: (1) The feature norm is a confidence value of a classifier hidden in the network layer, specifically its maximum logit. Hence, the feature norm distinguishes OOD from ID in the same manner that a classifier confidence does. (2) The feature norm is class-agnostic, thus it can detect OOD samples across diverse discriminative models. (3) The conventional feature norm fails to capture the deactivation tendency of hidden layer neurons, which may lead to misidentification of ID samples as OOD instances. To resolve this drawback, we propose a novel negative-aware norm (NAN) that can capture both the activation and deactivation tendencies of hidden layer neurons. We conduct extensive experiments on NAN, demonstrating its efficacy and compatibility with existing OOD detectors, as well as its capability in label-free environments.

Multimodal biometrics has been attributed to achieving better performance compared to unimodal biometrics, despite there being some limitations on its utilization e.g. availability, deployment cost, templates management, etc. In this paper, we revolve around a generalized multimodal biometrics notion, which we coin as Conditional Multimodal Biometrics (CMB). The CMB is substantiated by a learning model which is trained with N multimodal biometrics. During enrollment and query, the trained CMB model is utilized as a feature encoder to transform any x biometric raw input(s) yielding x reference and query instances, respectively, where 1≤x≤ N. Depending on application needs, multimodal biometrics system enjoys better performance by deploying either a single biometrics, a subset, or all N modalities. As a means of realization, we consider face and periocular biometrics and propose a deep CMB network, known as CMB-Net. The CMB-Net is composed of two predictors corresponding to face and periocular with a shared-parameter convolutional backbone. Apart from classification losses for each face and periocular, a CMB loss with regularization is devised to attract periocular-face intra-subject feature embeddings and repel periocular-face inter-subject feature embeddings, whilst each face and periocular regulates one another throughout CMB-Net training. We scrutinize three CMB configurations, namely periocular conditioned by face, face conditioned by periocular and periocular-face, under the CMB regimen. Our experimental results on five periocular-face in the wild datasets demonstrate that all three CMB configurations outperform their respective baselines under both identification and verification modes.