When storing face biometric samples in accordance with ISO/IEC 19794 as JPEG2000 encoded images, it is necessary to encrypt them for the sake of users’ privacy. Literature suggests selective encryption of JPEG2000 images as fast and efficient method for encryption, the trade-off is that some information is left in plaintext. This could be used by an attacker, in case the encrypted biometric samples are leaked. In this work, we will attempt to utilize a convolutional neural network to perform cryptanalysis of the encryption scheme. That is, we want to assess if there is any information left in plaintext in the selectively encrypted face images which can be used to identify the person. The chosen approach is to train CNNs for biometric face recognition not only with plaintext face samples but additionally conduct a refinement training with partially encrypted data. If this system can successfully utilize encrypted face samples for biometric matching, we can show that the information left in encrypted biometric face samples is information actually usable for biometric recognition.The method works and we can show that a supposedly secure biometric sample still contains identifying information on average over the whole database.

When biometric databases grow larger, a security breach or leak can affect millions. In order to protect against such a threat, the use of encryption is a natural choice. However, a biometric identification attempt then requires the decryption of a potential huge database, making a traditional approach potentially unfeasible. The use of selective JPEG2000 encryption can reduce the encryption’s computational load and enable a secure storage of biometric sample data. In this paper we will show that selective encryption of face biometric samples is secure. We analyze various encoding settings of JPEG2000, selective encryption parameters on the "Labeled Faces in the Wild" database and apply several traditional and deep learning based face recognition methods.

We conduct an ISO/IEC Standards 24745 and 30136 compliant assessment of block-based warping sample transformation techniques aiming for template protection. Particular focus is laid on the results' evaluation considering the evolution of face recognition technology ranging from more “historic” hand-crafted features to state-of-the-art deep-learning (DL) based schemes. It turns out that the high robustness of todays face recognition technology can handle geometrical distortions introduced by warping as another form of variability like pose, illumination, and expression variations, thereby disabling the intended protection functionality of warping. Therefore, block-based warping sample transformation must not be used as template protection technique for todays state-of-the-art face recognition schemes, while some settings could be identified providing template protection to some extent for less recent face recognition technology.

Most existing works regarding facial demographic estimation are focused on still image datasets, although nowadays the need to analyze video content in real applications is increasing. We propose to tackle gender, age and ethnicity estimation in the context of video scenarios. Our main contribution is to use an attribute-specific quality assessment procedure to select most relevant frames from a video sequence for each of the three demographic modalities. Selected frames are classified with fine-tuned MobileNet models and a final video prediction is obtained with a majority voting strategy. Our validation on three different datasets and our comparison with state-of-the-art models, show the effectiveness of the proposed demographic classifiers and the quality pipeline, which allows to reduce both: the number of frames to be classified and the processing time in practical applications; and improves the soft biometrics prediction accuracy.

Significant advances have been achieved in face recognition in the last decade thanks to the development of deep learning methods. However, recognizing faces captured in uncontrolled environments is still a challenging problem for the scientific community. In these scenarios, the performance of most of existing deep learning based methods abruptly falls, due to the bad quality of the face images. In this work, we propose to use an activation map to represent the quality information in a face image. Different face regions are analyzed to determine their quality and then only those regions with good quality are used to perform the recognition using a given deep face model. For experimental evaluation, in order to simulate unconstrained environments, three challenging databases, with different variations in appearance, were selected: the Labeled Faces in the Wild Database, the Celebrities in Frontal-Profile in the Wild Database, and the AR Database. Three deep face models were used to evaluate the proposal on these databases and in all cases, the use of the proposed activation map allows the improvement of the recognition rates obtained by the original models in a range from 0.3 up to 31%. The obtained results experimentally demonstrated that the proposal is able to select those face areas with higher discriminative power and enough identifying information, while ignores the ones with spurious information.

Typically, real-world requirements to deploy face recognition models in unconstrained surveillance scenarios demand to identify low-resolution faces with extremely low computational cost. In the last years, several methods based on complex deep learning models have been proposed with promising recognition results but at a high computational cost. Inspired by the compactness and computation efficiency of lightweight deep face networks and their high accuracy on general face recognition tasks, in this work we propose to benchmark two recently introduced lightweight face models on low-resolution surveillance imagery to enable efficient system deployment. In this way, we conduct a comprehensive evaluation on the two typical settings: LR-to-HR and LR-to-LR matching. In addition, we investigate the effect of using trained models with downsampled synthetic data from high-resolution images, as well as the combination of different models, for face recognition on real low-resolution images. Experimental results show that the used lightweight face models achieve state-of-the-art results on low-resolution benchmarks with low memory footprint and computational complexity. Moreover, we observed that combining models trained with different degradations improves the recognition accuracy on low-resolution surveillance imagery, which is feasible due to their low computational cost.

Face analysis and recognition systems have shown to be a valuable tool for forensic examiners. Particularly, the automatic estimation of age and gender from face images, can be useful in a wide range of forensic applications. In this work we propose to use a local appearance descriptor in a component-based way, to classify age and gender from face images. We subdivide a face image into regions of interest based on automatically detected landmarks, and represent them by using Histograms of Oriented Gradient (HOG). The representations obtained from different face regions are feeded to Support Vector Machine (SVM) classifiers to estimate the age and gender of the person in the image. Experimental analysis show the good results of this component-based approach, and its additional benefits when face images are affected by occlusions.

One of the main problems of recognizing faces in videos is to achieve accurate algorithms which can be used in real-time applications. Recently, Fisher Vector representation of local descriptors (e.g., SIFT) has gained widespread popularity, achieving good recognition rates. In this work, we propose to use Fisher Vector encoding of binary features for video face recognition, in order to speed up the computation time of the representation. The experimental evaluation was conducted on the challenging YouTube Faces database, showing that the proposed method is very efficient, and has an accuracy comparable with state-of-the-art methods.

Tracking-by-detection methods have become increasingly popular recently. This work presents a new multi-face tracking algorithm based on the association of detection responses given by a spatio-temporal face detector; which are considered as initial small trajectories or tracklets. An appearance mo del based on the spatio-termporal information is used to guide the tracker. Besides, a new adaptive Kalman filter that dynamically adjusts its parameters on the basis of the quality of the detector output is proposed. The introduced approach is evaluated on several challenging video sequences from the YouTube Faces database, achieving a very good performance.

The ability of a biometric system to reliably recognize registered individuals significantly depends on the kind and amount of variation that the exploited biometric trait may undergo throughout acquisitions. Those variations may be due both to acquisition devices, or to different environment settings, or to modification of the trait appearance. One of the strategies to address changes in biometric features is to store more templates for the same person, in order to increase the chances to identify her. The problem arises to choose the templates to store in a way which actually achieves better performance, while avoiding flooding the system with an excessively huge gallery. This work proposes an approach for the selection of the best templates for face recognition, which is based on a notion of gallery entropy, and can be also used for large datasets. Though relying on a clustering process, its main achievement is to automatically derive the best number of clusters/prototypes per subject without requiring to fixing it in advance. Comparative tests with existing approaches show that it is a very promising solution.

Face detection has been a very active research topic in recent years. However, when applied to uncontrolled environments, some systems exhibit poor generalization ability. Even though few of existing methods can achieve promising results in some challenging situations, they usually have the requirement of high computational cost. This will definitely limit the use of those methods in some mobile platforms which have limited computational resources and strict power-consumption control. In this paper, a novel facial representation method for multi-view face detection in uncontrolled environment is presented. The proposed method, named Flexible Block based Local Binary Patterns (FBLBP), has low storage requirements and it is fast to compute; while its performance is comparable with the state of the art methods, demonstrated on the challenging Face Detection Data set and Benchmark (FDDB).

Great progress has been achieved in face recognition in the last three decades. However, it is still challenging to characterize the identity related features in face images. This paper proposes a novel facial feature extraction method named Gabor ordinal measures (GOM), which integrates the distinctiveness of Gabor features and the robustness of ordinal measures as a promising solution to jointly handle inter-person similarity and intra-person variations in face images. In the proposal, different kinds of ordinal measures are derived from magnitude, phase, real, and imaginary components of Gabor images, respectively, and then are jointly encoded as visual primitives in local regions. The statistical distributions of these visual primitives in face image blocks are concatenated into a feature vector and linear discriminant analysis is further used to obtain a compact and discriminative feature representation. Finally, a two-stage cascade learning method and a greedy block selection method are used to train a strong classifier for face recognition. Extensive experiments on publicly available face image databases, such as FERET, AR, and large scale FRGC v2.0, demonstrate state-of-the-art face recognition performance of GOM.

The performance of face recognition systems tends to decrease when images are affected by illumination. Feature extraction is one of the main steps of a face recognition process, where it is possible to alleviate the illumination effects on face images. In order to increase the accuracy of recognition tasks, different methods for obtaining illumination invariant features have been developed. The aim of this work is to compare two different ways to represent face image descriptions in terms of their illumination invariant properties for face recognition. The first representation is constructed following the structure of complex numbers and the second one is based on quaternion numbers. Using four different face description approaches both representations are constructed, transformed into frequency domain and expressed in polar coordinates. The most illumination invariant component of each frequency domain representation is determined and used as the representative information of the face image. Verification and identification experiments are then performed in order to compare the discriminative power of the selected components. Representative component of the quaternion representation overcame the complex one.

Variations in illumination is one of major limiting factors of face recognition system performance. The effect of changes in the incident light on face images is analyzed, as well as its influence on the low frequency components of the image. Starting from this analysis, a new photometric normalization method for illumination invariant face recognition is presented. Low-frequency Discrete Cosine Transform coefficients in the logarithmic domain are used in a local way to reconstruct a slowly varying component of the face image which is caused by illumination. After smoothing, this component is subtracted from the original logarithmic image to compensate for illumination variations. Compared to other preprocessing algorithms, our method achieved a very good performance with a total error rate very similar to that produced by the best performing state-of-the-art algorithm. An in-depth analysis of the two preprocessing methods revealed notable differences in their behavior, which is exploited in a multiple classifier fusion framework to achieve further performance improvement. The superiority of the proposal is demonstrated in both face verification and identification experiments.

A method to determine whether face images are affected or not by lighting problems is proposed. The method is the result of combining the analysis of lighting effect on face regions with the analysis of special areas which have a weight on the decision. Good results were obtained classifying well and badly illuminated images. The proposed method was inserted on a face recognition framework in order to apply the preprocessing step only to those images affected by illumination variations. The good performance achieved on verification and identification experiments, confirm that it is better to apply the proposed methodology than to preprocess all images when the lighting conditions are variable.

Local Binary Patterns (LBP) is one of the most used methods in face recognition. This paper presents a different way of obtaining the regions that are used to construct the LBP histograms, in order to improve its performance in front of illumination problems. The proposed method takes into account the shape of the face to build a triangular mesh in which a better description of the face image through LBP is achieved. Experimental results conducted on Yale B database show that under varying lighting conditions, the proposal improves the performance of the traditional rectangular division of the LBP method.