A person’s facial hairstyle, such as presence and size of beard, can significantly impact face recognition accuracy. While previous research has examined the facial hair effect using binary attributes, no work utilizes a segmentation model to capture the full extent of the facial hair. To investigate the effect of facial hair size in a rigorous manner, we first created a set of fine-grained facial hair annotations to train a segmentation model. Cross-dataset evaluation is performed and accuracy across African-American and Caucasian face images is reported. We then use our facial hair predictions to categorize image pairs according to the degree of difference or similarity in the facial hairstyle. We find that the False Match Rates for image pairs with different categories of facial hairstyle varies by a factor of over 10 for African-American males and over 25 for Caucasian males on MORPH dataset. Also, False Non-Match Rates of 4 race categories on BA-Test dataset are analyzed to measure the accuracy bias in unconstrained settings. Our findings suggest that, while facial hair can cause a shift in similarity score distributions, this effect can be mitigated by employing an adaptive threshold based on facial hair predictions. Facial hair annotations: https://github.com/kaganozturk/Effects-of-Facial-Hair-on-Face-Recognition.

Facial filters are now commonplace for social media users around the world. Previous work has demonstrated that facial filters can negatively impact automated face recognition performance. However, these studies focus on small numbers of hand-picked filters in particular styles. In order to more effectively incorporate the wide ranges of filters present on various social media applications, we introduce a framework that allows for larger-scale study of the impact of facial filters on automated recognition. This framework includes a controlled dataset of face images, a principled filter selection process that selects a representative range of filters for experimentation, and a set of experiments to evaluate the filters’ impact on recognition. We demonstrate our framework with a case study of filters from the American applications Instagram and Snapchat and the Chinese applications Meitu and Pitu to uncover cross-cultural differences. Finally, we show how the filtering effect in a face embedding space can easily be detected and restored to improve face recognition performance.

The prevailing approach in face recognition is to specialize a deep network for general computer vision to the face recognition task. ResNet being specialized for use as the backbone in SoTA face recognition systems is a prime example of this. One significant architectural deviation in the ResNet backbone adapted for face recognition is the linearization of the output spatial map from the last convolution layer to feed the linear layer, rather than utilizing Global Average Pooling (GAP). The utilization of GAP treats all pixel values in the output spatial map as equally significant and averages them naively, thereby compromising the performance of the face recognition model. However, linearization of the spatial map inflates the total parameters in the model by up to 58% (R34) in the lighter version of the ResNet backbone that is typically used for face recognition. Leveraging the prior knowledge that face images during training and testing are pre-aligned, we introduce a novel Gaussian Weighted Pooling (GWP) layer, integrating a pre-computed Gaussian Attention Kernel with the Average Pooling Layer that weighs the importance of the pixel based on the spatial position. Our findings show that utilizing GWP consistently outperforms GAP and achieves results comparable to those of parameter-inflated baseline models.