While advances in deep learning have enabled novel applications in various fields, face recognition in open-set scenarios remains a complex task, owing to the challenges posed by the extensive volume of low-quality face images. We introduce a new approach for recognizing faces in unconstrained open-set settings by leveraging uncertainty-aware embeddings through contrastive learning. Our model, called UCFace, effectively regulates the contribution of each face image based on the face uncertainty derived from image quality as an inverse proxy. Face embeddings are reinterpreted as a probabilistic distribution within the embedding space, where the degree of sharpness (i.e., distribution concentration) reflects the underlying uncertainty and probability density is used as a similarity metric to facilitate contrastive learning. Experiments on a wide range of face datasets, including those with high, mixed, and real-world low-resolution face images, demonstrate that UCFace enhances open-set face recognition performance by integrating the aspect of uncertainty.

Periocular and face are two common biometric modalities for identity management. Recently, the emergence of conditional biometrics has enabled the exploitation of the correlation between face and periocular to enhance each modality’s performance, in which we coin intra-modal matching in this paper. However, limitations arise in each modality, particularly when wearing sunglasses or helmets, causing the absence of periocular or facial occlusion. A biometric system empowered with inter-modal matching capability between periocular and face is essential to mitigate these challenges. This paper presents a novel reciprocal learning model that utilizes periocular and face conditioning to facilitate flexible intra-modal and inter-modal matching. To address the intra-modal matching challenge, we devise a lightweight Gated Convolutional Channel-wise Self-Attention Network that enables selective attention to shared salient periocular and face features. On the other hand, to bridge the modality gap without sacrificing the intra-modal matching performance, we propose a modality and augmentation-aware contrastive loss that incorporates semi-supervised positive sampling and alignment-specific logit rescaling. Extensive identification and verification experiments on five face-periocular datasets under the open-set protocol attest to the efficacy of our proposed methods.

Very low-resolution face recognition (VLRFR) poses unique challenges, such as tiny regions of interest and poor resolution due to extreme standoff distance or wide viewing angle of the acquisition devices. In this paper, we study principled approaches to elevate the recognizability of a face in the embedding space instead of the visual quality. We first formulate a robust learning-based face recognizability measure, namely recognizability index (RI), based on two criteria: (i) proximity of each face embedding against the unrecognizable faces cluster center and (ii) closeness of each face embedding against its positive and negative class prototypes. We then devise an index diversion loss to push the hard-to-recognize face embedding with low RI away from unrecognizable faces cluster to boost the RI, which reflects better recognizability. Additionally, a perceptibility attention mechanism is introduced to attend to the most recognizable face regions, which offers better explanatory and discriminative traits for embedding learning. Our proposed model is trained end-to-end and simultaneously serves recognizability-aware embedding learning and face quality estimation. To address VLRFR, our extensive evaluations on three challenging low-resolution datasets and face quality assessment demonstrate the superiority of the proposed model over the state-of-the-art methods.

Multimodal biometrics has been attributed to achieving better performance compared to unimodal biometrics, despite there being some limitations on its utilization e.g. availability, deployment cost, templates management, etc. In this paper, we revolve around a generalized multimodal biometrics notion, which we coin as Conditional Multimodal Biometrics (CMB). The CMB is substantiated by a learning model which is trained with N multimodal biometrics. During enrollment and query, the trained CMB model is utilized as a feature encoder to transform any x biometric raw input(s) yielding x reference and query instances, respectively, where 1≤x≤ N. Depending on application needs, multimodal biometrics system enjoys better performance by deploying either a single biometrics, a subset, or all N modalities. As a means of realization, we consider face and periocular biometrics and propose a deep CMB network, known as CMB-Net. The CMB-Net is composed of two predictors corresponding to face and periocular with a shared-parameter convolutional backbone. Apart from classification losses for each face and periocular, a CMB loss with regularization is devised to attract periocular-face intra-subject feature embeddings and repel periocular-face inter-subject feature embeddings, whilst each face and periocular regulates one another throughout CMB-Net training. We scrutinize three CMB configurations, namely periocular conditioned by face, face conditioned by periocular and periocular-face, under the CMB regimen. Our experimental results on five periocular-face in the wild datasets demonstrate that all three CMB configurations outperform their respective baselines under both identification and verification modes.

Low-resolution (LR) face recognition (LRFR) tacklestiny face imagesdetected from real-world surveillance camera footage, which are unconstrained and generally poor in quality. Owing to the absence of a million-scale labeled LR face dataset,identity-invariantdata augmentation (DA) transformations such as flipping, rotation, rescaling, etc., are applied to inflate the effective training examples with respect to the source identities for representation learning. Unfortunately, the identity-invariant property incurs additional intra-class disparity that impairs generalization performance. In this paper, we put forward a new means of DA strategy, termedidentity-extendedDA, that satisfies bothaffinityanddiversityrequirements essential to DA. We instantiate an implicit identity-extended augmentation network, or simply IDEA-Net, to realize the proposed identity-extended DA for LRFR. More specifically, training an IDEA-Net instance augments the small-scale LR (query) face dataset with identity-extended (auxiliary) face examples implicitly in the representation space. We also introduce a calibrator to regulate the disordered representation space by refining the intra-class compactness and the inter-class separation. This diminishes the distribution shift between the original and the augmented examples (affinity) and increases the learning complexity (diversity). We substantiate that IDEA-Net renders a high affinity and diversity representation space. On the other hand, our experimental results on three real-world LR face datasets demonstrate that IDEA-Nets outperform the baselines and other counterparts trained without leveraging the identity-extended examples for LRFR.

Open set recognition (OSR) models need not only discriminate between known classes but also detect unknown class samples unavailable during training. One promising approach is to learn discriminative representations over known classes with strong intra-class similarity and inter-class discrepancy. Then, the powerful class discrimination learned from the known classes can be extended to known and unknown classes. Without appropriate regularization, however, the model may learn representations trivially, collapsing unknown class representations to the known class ones. To resolve this problem, we propose Divergent Angular Representation (DivAR) based on two approaches. Firstly, DivAR maximizes its representational discrimination between known classes via a highly discriminative loss. Secondly, to ensure separation between known and unknown classes in the representation space, DivAR boosts the directional variation of representations over global samples. In addition, self-supervision is leveraged to improve the representation's robustness and extend DivAR to one-class classification. Moreover, unlike other OSR methods that require an extra machinery for inference, DivAR learns and infers in a single module. Extensive experiments on generic image datasets demonstrate the plausibility and effectiveness of DivAR for both OSR and One-Class Classification (OCC) problems.

Realistic low-resolution (LR) face images refer to those captured by the real-world surveillance cameras at extreme standoff distances, thereby LR and poor in quality essentially. Owing to severe scarcity of labeled data, a high-capacity deep convolution neural networks (CNN) is hardly trained to confront the realistic LR face recognition (LRFR) challenge. We introduce in this letter a dual-stream mutual information distillation network (MIND-Net), whereby the non-identity specific mutual information (MI) characterized by generic face features coexistent on realistic and synthetic LR face images are distilled to render a resolution-invariant embedding space for LRFR. For a thorough analysis, we quantify the degree of MI distillation in terms normalized MI index. Our experimental results on the realistic LR face datasets substantiate that the MIND-Net instances assembled from the pre-learned CNNs stand out from the baselines and other state of the arts by a notable margin.

In this paper, we propose a data-driven cancellable biometrics scheme, referred to as SoftmaxOut Transformation-Permutation Network (SOTPN). The SOTPN is a neural version of Random Permutation Maxout (RPM) transform, which was introduced for facial template protection. We present a specialized SoftmaxOut layer integrated with the permutable MaxOut units and the parameterized softmax function to approximate the nondifferentiable permutation and the winner-takes-all operations in the RPM transform. On top of that, a novel pairwise ArcFace loss and a code balancing loss are also formulated to ensure that the SOTPN-transformed facial template is cancellable, discriminative, high entropy and free from quantization errors when coupled with the SoftmaxOut layer. The proposed SOTPN is evaluated on three face datasets, namely LFW, YouTube Face and Facescrub, and our experimental results disclosed that the SOTPN outperforms the RPM transform significantly.

Periocular biometric covering the immediate vicinity of human eye is a synergistic alternative to face particularly when the face is masked or occluded. Most present work for periocular recognition in the wild are mainly convolutional neural networks learned based on cross-entropy loss. However, periocular images only capture the least salient face features, and thus suffering from severe intra-class compactness and inter-class dispersion issues for discriminative deep feature learning. Recently, label smoothing regularization (LSR) is discerned capable of diminishing the intra-class variation by minimizing the Kullback-Liebler divergence of a uniform distribution and a network prediction distribution. In this letter, we extend LSR to that of Generalized LSR (GLSR) by learning a pre-task network prediction, in place of the predefined uniform distribution. Extensive experiments on four periocular in the wild datasets disclose that the GSLR-trained networks prevail over the LSR-based counterpart and other most recent the state of the arts. This is supported by our empirical analyses that the embedding periocular features rendered by GLSR results in better class-wise cluster separation than the conventional LSR.

Stacking-based deep neural network (S-DNN) is aggregated with pluralities of basic learning modules, one after another, to synthesize a deep neural network (DNN) alternative for pattern classification. Contrary to the DNNs trained from end to end by backpropagation (BP), each S-DNN layer, that is, a self-learnable module, is to be trained decisively and independently without BP intervention. In this paper, a ridge regression-based S-DNN, dubbed deep analytic network (DAN), along with its kernelization (K-DAN), are devised for multilayer feature relearning from the pre-extracted baseline features and the structured features. Our theoretical formulation demonstrates that DAN/K-DAN relearn by perturbing the intra/interclass variations, apart from diminishing the prediction errors. We scrutinize the DAN/K-DAN performance for pattern classification on datasets of varying domains-faces, handwritten digits, generic objects, to name a few. Unlike the typical BP-optimized DNNs to be trained from gigantic datasets by GPU, we reveal that DAN/K-DAN are trainable using only CPU even for small-scale training sets. Our experimental results show that DAN/K-DAN outperform the present S-DNNs and also the BP-trained DNNs, including multiplayer perceptron, deep belief network, etc., without data augmentation applied.

This paper devises a new means of filter diversification, dubbed multi-fold filter convolution (M-FFC), for face recognition. On the assumption that M-FFC receives single-scale Gabor filters of varying orientations as input, these filters are selfcross convolved by M-fold to instantiate a filter offspring set. The M-FFC flexibility also permits cross convolution amongst Gabor filters and other filter banks of profoundly dissimilar traits, e.g., principal component analysis (PCA) filters and independent component analysis (ICA) filters. The 2-FFC of Gabor, PCA, and ICA filters thus yields three offspring sets: 1) Gabor filters solely; 2) Gabor-PCA filters; and 3) Gabor-ICA filters, to render the learning-free and the learning-based 2-FFC descriptors. To facilitate a sensible Gabor filter selection for M-FFC, the 40 multi-scale, multi-orientation Gabor filters are condensed into eight elementary filters. Aside from that, an average histogram pooling operator is employed to leverage the 2-FFC histogram features, prior to the final whitening PCA compression. The empirical results substantiate that the 2-FFC descriptors prevail over, or on par with, other face descriptors on both identification and verification tasks.

Touch-stroke dynamics is an emerging behavioral biometrics justified feasible for mobile identity management. A touch-stroke dynamics authentication system is composed of a hand-engineered feature extractor and a classifier separately. In this letter, we propose a stacking-based deep learning network that performs feature extraction and classification, collectively dubbed Kernel Deep Regression Network (KDRN). The KDRN is built on multiple kernel ridge regressions (KRR) hierarchically, where each is trained analytically and independently. In principal, KDRN does not mean to learn directly from the raw touch-stroke data like other deep learning models, but it relearns from the pre-extracted features to yield a richer and a relatively more discriminative feature set. Subsequent to that, the authentication is carried out by KRR. Overall, KDRN achieves an equal error rate of 0.013% for intrasession authentication, 0.023% for intersession authentication, and 0.121% for interweek authentication on the Touchlaytics dataset.

Stacking-based deep neural network (S-DNN), in general, denotes a deep neural network (DNN) resemblance in terms of its very deep, feedforward network architecture. The typical S-DNN aggregates a variable number of individually learnable modules in series to assemble a DNN-alike alternative to the targeted object recognition tasks. This work likewise devises an S-DNN instantiation, dubbed deep analytic network (DAN), on top of the spectral histogram (SH) features. The DAN learning principle relies on ridge regression, and some key DNN constituents, specifically, rectified linear unit, fine-tuning, and normalization. The DAN aptitude is scrutinized on three repositories of varying domains, including FERET (faces), MNIST (handwritten digits), and CIFAR10 (natural objects). The empirical results unveil that DAN escalates the SH baseline performance over a sufficiently deep layer.

The principal component analysis network (PCANet) is asserted as a parsimonious stacking-based convolutional neural networks (CNNs) instance for generic object recognition including face. However, to be regarded a CNN resemblance, PCANet lacks a nonlinearity in between two successive convolutional layers. The multilayer PCANet (by neglecting the nonlinearity pre-requisite) is also deemed far-fetched for the network depth beyond two, due to feature dimensionality explosion. We thus devise a PCANet alternative, dubbed PCANet+ in this letter, to untangle these constraints. To be more precise, conforming to the CNN essentials, PCANet+ conveys a mean-pooling unit manipulating each feature map. On top of that, we streamline the PCANet topology to permit a deep construction with an expanded PCA filter ensemble. We scrutinize the PCANet+ performance using face recognition technology and other two faces in the wild datasets, namely, labeled faces in the wild and YouTube faces. The experimental results reveal that the PCANet+ descriptor prevails over its predecessor and other stacking-based descriptors in face identification and verification, serving a baseline for ConvNets.

The standard multi-scale, multi-orientation Gabor filter ensemble (SGFE) in the face recognition task reposits 40 filters localized in 8 orientations and 5 scales, with a real and an imaginary constituent. This paper devises a simple means of filter diversification, dubbed as multi-fold Gabor filter convolution (-FGFC), where a set of pre-selected filters, e.g., single-scale Gabor filters across varying orientations, are self-cross convolved by folds to instantiate the offspring filters. To facilitate filter selection for-FGFC, this paper summarizes SGFE into the condensed Gabor filter ensemble (CGFE) of only 8 filters. In addition, an average histogram pooling operator is proposed to downsample and regulate the demodulated Gabor phase features prior to the final compression stage. The performance of a specific M-FGFC instance, i.e., the 2-FGFC descriptor, is investigated on FERET I (frontal), FERET II (nonfrontal) and AR datasets. The experimental results on FERET I substantiates that the 2-FGFC descriptor outperforms the leading state of the art face descriptors.

Gabor-based region covariance matrix (GRCM) has been demonstrated as a promising descriptor for face recognition. However, GRCM requires large number of filters to achieve satisfactory performance. Furthermore, complex-valued Gabor filters require double convolution operations for each filter that makes the computation more expensive. To alleviate the problem, we propose to adopt real-valued discrete cosine transform (DCT) as filter bank in place of complex-valued Gabor filter. DCT as an orthogonal transform however decorrelates the signal, leads to most energies fall into the diagonal entries of the constructed covariance matrix, which is ill-formed for RCM. We demonstrate that applying non-linear operation on the DCT filter responses ameliorates the decorrelated filter responses effects. Apart from that, while RCM offers spatial information that is useful for recognition tasks, overly small RCM region renders poor covariance estimation, which can affect the recognition performance drastically. In this paper we also propose Log-TiedRank to mitigate the potential undersampling effect suffered by covariance matrix estimation. From the experiments Log-TiedRank shows surprising performance boost over AIRM and Log-Euclidean metric especially when both gallery set and probe set have very different distributions.