In humanitarian and emergency scenarios, the use of biometrics can dramatically improve the efficiency of operations, but it poses risks for the data subjects, which are exacerbated in contexts of vulnerability. To address this, we present a mobile biometric system implementing a biometric template protection (BTP) scheme suitable for these scenarios. After rigorously formulating the functional, operational, and security and privacy requirements of these contexts, we perform a broad comparative analysis of the BTP landscape. PolyProtect, a method designed to operate on neural network face embeddings, is identified as the most suitable method due to its effectiveness, modularity, and lightweight computational burden. We evaluate PolyProtect in terms of verification and identification accuracy, irreversibility, and unlinkability, when this BTP method is applied to face embeddings extracted using EdgeFace, a novel state-of-the-art efficient feature extractor, on a real-world face dataset from a humanitarian field project in Ethiopia. Moreover, as PolyProtect promises to be modality-independent, we extend its evaluation to fingerprints. To the best of our knowledge, this is the first time that PolyProtect has been evaluated for the identification scenario and for fingerprint biometrics. Our experimental results are promising, and we plan to release our code2.

Presentation Attack Detection (PAD) is essential for ensuring the security of face recognition (FR) systems, particularly in the context of mobile authentication in various sectors, such as online banking and government services. However, current PAD methods are often sensitive to the data domain, partly due to the limitations of training PAD datasets. In this paper, we introduce the SO-TERIA dataset, which provides captures of bona-fide and diverse Presentation Attacks (PAs) recorded using smart-phones. The dataset was collected responsibly from 70 consenting individuals, as opposed to web scraping. It includes face videos, motion data, and depth information (when available) as well as a novel projector-based replay attack. To demonstrate the utility of the SOTERIA dataset, we evaluate the vulnerability of a SOTA FR model (IRes-Net100) to the PAs in the dataset. We also analyze the PAD capabilities of a SOTA PAD model (DeepPixBis) through cross-dataset experiments as well as on real attacks observed in an industrial application. Our findings show the effectiveness and versatility of the SOTERIA dataset in advancing PAD research, in particular toward generalization.

Face recognition systems use the templates (extracted from users’ face images) stored in the system’s database for recognition. In a template inversion attack, the adversary gains access to the stored templates and tries to enter the system using images reconstructed from those templates. In this paper, we propose a framework to evaluate the vulnerability of face recognition systems to template inversion attacks. We build our framework upon a real-world scenario and measure the vulnerability of the system in terms of the adversary’s success attack rate in entering the system using the reconstructed face images. We propose a face reconstruction network based on a new block called “enhanced deconvolution using cascaded convolution and skip connections" (shortly,DSCasConv), and train it with a multi-term loss function. We use our framework to evaluate the vulnerability of state-of-the-art face recognition models, with different network structures and loss functions (in total 31 models), on the MOBIO, LFW, and AgeDB face datasets. Our experiments show that the reconstructed face images can be used to enter the system, which threatens the system’s security. Additionally, the reconstructed face images may reveal important information about each user’s identity, such as race, gender, and age, and hence jeopardize the users’ privacy.

The proliferation of automated face recognition (FR) necessitates increasingly accurate person identification. The COVID-19 pandemic has exposed the limitations of FR systems when presented with faces occluded by hygienic masks. However, the security risks of personalised hygienic mask attacks, whereby an attacker wears the mask on which the bottom part of an enrolled user’s face is printed, have not yet been studied. To address this research gap, we introduce a novel face dataset consisting of smartphone-recorded videos of real (bona-fide) faces and personalised hygienic mask attacks. We also analyse the vulnerability of two state-of-the-art FR systems to this type of attack, using our dataset. Our results indicate that personalised hygienic mask attacks have the potential to compromise system security, particularly for FR systems that are tuned towards optimising user convenience. These findings underscore the importance of developing suitable Presentation Attack Detection (PAD) algorithms. Our dataset will help researchers and practitioners work towards this goal, thereby enhancing the security and reliability of FR systems.

As automated face recognition applications tend towards ubiquity, there is a growing need to secure the sensitive face data used within these systems. This paper presents a survey of biometric template protection (BTP) methods proposed for securing face “templates” (images/features) in neural-network-based face recognition systems. The BTP methods are categorised into two types: Non-NN and NN-learned. Non-NN methods use a neural network (NN) as a feature extractor, but the BTP part is based on a non-NN algorithm applied at either image-level or feature-level. In contrast, NN-learned methods specifically employ a NN to learn a protected template from the unprotected face image/features. We present examples of Non-NN and NN-learned face BTP methods from the literature, along with a discussion of the two categories’ comparative strengths and weaknesses. We also investigate the techniques used to evaluate these BTP methods, in terms of the three most common BTP criteria: “recognition accuracy”, “irreversibility”, and “renewability/unlinkability”. As expected, the recognition accuracy of protected face recognition systems is generally evaluated using the same (empirical) techniques employed for evaluating standard (unprotected) biometric systems. On the contrary, most irreversibility and renewability/unlinkability evaluations are found to be based on theoretical assumptions/estimates or verbal implications, with a lack of empirical validation in a practical face recognition context. We recommend, therefore, a greater focus on empirical evaluation strategies, to provide more concrete insights into the irreversibility and renewability/unlinkability of face BTP methods in practice. Additionally, an exploration of the reproducibility of the studied BTP works, in terms of the public availability of their implementation code and evaluation datasets/procedures, suggests that it would currently be difficult for the BTP community to faithfully replicate (and thus validate) most of the reported findings. So, we advocate for a push towards reproducibility, in the hope of furthering our understanding of the face BTP research field.

Homomorphic Encryption (HE) has become a well-known tool for privacy-preserving recognition in biometric systems. Despite some important advantages of HE (such as preservation of recognition accuracy), there are two main drawbacks in the application of HE to biometric recognition systems: first, the security of the system solely depends on the secrecy of the private (decryption) key; second, the computational costs of the operations on the ciphertexts are expensive. To address these challenges, in this paper we propose a hybrid scheme for the protection of biometric templates, which combines cancelable biometrics (CB) methods and HE. Applying CB prior to HE enhances both the security and privacy of the overall system, since the protected templates remain irreversible even if the secret keys are leaked (commonly referred to as the full disclosure scenario). In addition, we can reduce the dimensionality of templates using CB before applying HE, which speeds up the computation over the ciphertexts. We use BioHashing, Multi-Layer Perceptron (MLP) hashing, and Index-of-Maximum (IoM) hashing as different CB methods, and for each of these schemes, we propose a method for computing scores between hybrid-protected templates in the encrypted domain. We evaluate our proposed hybrid scheme using different state-of-the-art face recognition models (Ar-cFace, ElasticFace, and FaceNet) on the MOBIO and LFW datasets. The source code of our experiments is publicly available, so our work can be fully reproduced.

State-of-the-art (SOTA) face recognition systems generally use deep convolutional neural networks (CNNs) to extract deep features, called embeddings, from face images. The face embeddings are stored in the system’s database and are used for recognition of the enrolled system users. Hence, these features convey important information about the user’s identity, and therefore any attack using the face embeddings jeopardizes the user’s security and privacy. In this paper, we propose a CNN-based structure to reconstruct face images from face embeddings and we train our network with a multi-term loss function. In our experiments, our network is trained to reconstruct face images from SOTA face recognition models (ArcFace and ElasticFace) and we evaluate our face reconstruction network on the MOBIO and LFW datasets. The source code of all the experiments presented in this paper is publicly available so our work can be fully reproduced.

This paper investigates the probability of a reference minutia repeating in another sample of the same person’s fingerprint, when that probability depends only on the consistency with which a user places their finger onto a fingerprint scanner. The investigation targets cooperative users in a civilian fingerprint recognition application. A database of 800 fingerprint samples from 100 participants was collected for the purpose of simulating such a scenario. Analysis of this database showed that such users are typically sufficiently consistent in the placement of their fingers onto the scanner to ensure that there is a 0.95 probability of a reference minutia repeating in another sample of the same fingerprint. Combining multiple samples of the same fingerprint during enrolment to filter out only the most reliable reference minutiae was shown to improve this probability even further. Additional analysis showed that, as the number of reference fingerprints used increases, the number of reference minutiae remaining for recognition purposes decreases. While this trend is expected, our results indicate that this loss in the number of reference minutiae is not very significant among cooperative users.