One common critique of biometric authentication is that if an individual’s biometric is compromised, then the individual has no recourse. The concept of revocable biometrics was developed to address this concern. A biometric scheme is revocable if an individual can have their current enrollment in the scheme revoked, so that the compromised biometric template becomes worthless, and the individual can re-enroll with a new template that has similar recognition power. We show that modern deep CNN face matchers inherently allow for a robust revocable biometric scheme. For a given state-of-the-art deep CNN backbone and training set, it is possible to generate an unlimited number of distinct face matcher models that have both (1) equivalent recognition power, and (2) strongly incompatible biometric templates. The equivalent recognition power extends to the point of generating impostor and genuine distributions that have the same shape and placement on the similarity dimension, meaning that the models can share a similarity threshold for a 1-in-10,000 false match rate. The biometric templates from different model instances are so strongly incompatible that the cross-instance similarity score for images of the same person is typically lower than the sameinstance similarity score for images of different persons. That is, a stolen biometric template that is revoked is of less value in attempting to match the re-enrolled identity than the average impostor template. We also explore the feasibility of using a Vision Transformer (ViT) backbone-based face matcher in the revocable biometric system proposed in this work and demonstrate that it is less suitable compared to typical ResNet-based deep CNN backbones.

The first layer of a deep CNN backbone applies filters to an image to extract the basic features available to later layers. During training, some filters may go inactive, meaning all weights in the filter are near zero. An inactive filter in the final model represents a missed opportunity to extract a useful feature. This phenomenon is especially prevalent in specialized CNNs such as for face recognition (as opposed to, e.g., ImageNet). For example, in one of the most widely-used face recognition models (ArcFace), about half of the filters in the first layer are inactive. We propose a novel approach designed and tested specifically for face recognition networks, known as “CRAFT: Contextual Re-Activation of Filters for Face Recognition Training”. Additionally, CRAFT achieves statistically significant improvements in accuracy over standard training on face recognition benchmarks such as AgeDB-30, CPLFW, LFW, CALFW, CFP-FP, IJBB, and IJBC where accuracy has largely saturated. Notable improvements are observed, with significant gains on the highly challenging Hadrian and Eclipse datasets.

A person’s facial hairstyle, such as presence and size of beard, can significantly impact face recognition accuracy. While previous research has examined the facial hair effect using binary attributes, no work utilizes a segmentation model to capture the full extent of the facial hair. To investigate the effect of facial hair size in a rigorous manner, we first created a set of fine-grained facial hair annotations to train a segmentation model. Cross-dataset evaluation is performed and accuracy across African-American and Caucasian face images is reported. We then use our facial hair predictions to categorize image pairs according to the degree of difference or similarity in the facial hairstyle. We find that the False Match Rates for image pairs with different categories of facial hairstyle varies by a factor of over 10 for African-American males and over 25 for Caucasian males on MORPH dataset. Also, False Non-Match Rates of 4 race categories on BA-Test dataset are analyzed to measure the accuracy bias in unconstrained settings. Our findings suggest that, while facial hair can cause a shift in similarity score distributions, this effect can be mitigated by employing an adaptive threshold based on facial hair predictions. Facial hair annotations: https://github.com/kaganozturk/Effects-of-Facial-Hair-on-Face-Recognition.

The prevailing approach in face recognition is to specialize a deep network for general computer vision to the face recognition task. ResNet being specialized for use as the backbone in SoTA face recognition systems is a prime example of this. One significant architectural deviation in the ResNet backbone adapted for face recognition is the linearization of the output spatial map from the last convolution layer to feed the linear layer, rather than utilizing Global Average Pooling (GAP). The utilization of GAP treats all pixel values in the output spatial map as equally significant and averages them naively, thereby compromising the performance of the face recognition model. However, linearization of the spatial map inflates the total parameters in the model by up to 58% (R34) in the lighter version of the ResNet backbone that is typically used for face recognition. Leveraging the prior knowledge that face images during training and testing are pre-aligned, we introduce a novel Gaussian Weighted Pooling (GWP) layer, integrating a pre-computed Gaussian Attention Kernel with the Average Pooling Layer that weighs the importance of the pixel based on the spatial position. Our findings show that utilizing GWP consistently outperforms GAP and achieves results comparable to those of parameter-inflated baseline models.

Face attribute research has so far used only simple binary attributes for facial hair; e.g., beard / no beard. We have created a new, more descriptive facial hair annotation scheme and applied it to create a new facial hair attribute dataset, FH37K. Face attribute research also so far has not dealt with logical consistency and completeness. For example, in prior research, an image might be classified as both having no beard and also having a goatee (a type of beard). We show that the test accuracy of previous classification methods on facial hair attribute classification drops significantly if logical consistency of classifications is enforced. We propose a logically consistent prediction loss, LCPLoss, to aid learning of logical consistency across attributes, and also a label compensation training strategy to eliminate the problem of no positive prediction across a set of related attributes. Using an attribute classifier trained on FH37K, we investigate how facial hair affects face recognition accuracy, including variation across demographics. Results show that similarity and difference in facial hairstyle have important effects on the impostor and genuine score distributions in face recognition. The code is at https://github.com/HaiyuWu/LogicalConsistency.

Thorough evaluation is critical for developing models that are fair and robust. In this work, we describe the Conditional Attribute Subsampling Toolkit (CAST) for selecting data subsets for fine-grained scientific evaluations. Our toolkit efficiently filters data given an arbitrary number of conditions for metadata attributes. The purpose of the toolkit is to allow researchers to easily to evaluate models on targeted test distributions. The functionality of CAST is demonstrated on the WebFace42M face Recognition dataset. We calculate over 50 attributes for this dataset including race, image quality, facial features, and accessories. Using our toolkit, we create over a hundred test sets conditioned on one or multiple attributes. Results are presented for subsets of various demographics and image quality ranges. Using eleven different subsets, we build a face recognition 1:1 verification benchmark called C11 that exclusively contains pairs that are near the decision threshold. Evaluation on C11 with state-of-the-art methods demonstrates the suitability of the proposed benchmark. The toolkit is publicly available at https://github.com/WesRobbins/CAST.