The presence of bias in deep models leads to unfair outcomes for certain demographic subgroups. Research in bias focuses primarily on facial recognition and attribute prediction with scarce emphasis on face detection. Existing studies consider face detection as binary classification into ‘face’ and ‘non-face’ classes. In this work, we investigate possible bias in the domain of face detection through facial region localization which is currently unexplored. Since facial region localization is an essential task for all face recognition pipelines, it is imperative to analyze the presence of such bias in popular deep models. Most existing face detection datasets lack suitable annotation for such analysis. Therefore, we web-curate the Fair Face Localization with Attributes (F2LA) dataset and manually annotate more than 10 attributes per face, including facial localization information. Utilizing the extensive annotations from F2LA, an experimental setup is designed to study the performance of four pre-trained face detectors. We observe (i) a high disparity in detection accuracies across gender and skin-tone, and (ii) interplay of confounding factors beyond demography. The F2LA data and associated annotations can be accessed at http://iab-rubric.org/index.php/F2LA.

Deep learning algorithms have achieved tremendous success over the past few years. However, the biased behavior of deep models, where the models favor/disfavor certain demographic subgroups, is a major concern in the deep learning community. Several adverse consequences of biased predictions have been observed in the past. One solution to alleviate the problem is to train deep models for fair outcomes. Therefore, in this research, we propose a novel loss function, termed as Uniform Misclassification Loss (UML) to train deep models for unbiased outcomes. The proposed UML function penalizes the model for the worst-performing subgroup for mitigating bias and enhancing the overall model performance. The proposed loss function is also effective while training with imbalanced data as well. Further, a metric, Joint Performance Disparity Measure (JPD) is introduced to jointly measure the overall model performance and the bias in model prediction. Multiple experiments have been performed on four publicly available datasets for facial attribute prediction and comparisons are performed with existing bias mitigation algorithms. Experimental results are reported using performance and bias evaluation metrics . The proposed loss function outperforms existing bias mitigation algorithms that showcase its effectiveness in obtaining unbiased outcomes and improved performance.

Existing facial analysis systems have been shown to yield biased results against certain demographic subgroups. Due to its impact on society, it has become imperative to ensure that these systems do not discriminate based on gender, identity, or skin tone of individuals. This has led to research in the identification and mitigation of bias in AI systems. In this paper, we encapsulate bias detection/estimation and mitigation algorithms for facial analysis. Our main contributions include a systematic review of algorithms proposed for understanding bias, along with a taxonomy and extensive overview of existing bias mitigation algorithms. We also discuss open challenges in the field of biased facial analysis.

The sharing of fake content on social media platforms has become a major concern. In many cases, the same content with small variations is shared multiple times on different social media platforms. This leads to the circulation of manipulated content on the web. With the rapid advancement in deep learning algorithms, the generation of manipulated images with small variations in original images has become an easy task. These contents raise serious concerns when used for malicious activities. Therefore, detection of manipulated contents is of paramount importance. However, no large-scale dataset having manipulated images generated using both handcrafted and deep learning algorithms is available. Therefore, in this research, we have proposed a large dataset with more than 5.5 million images, termed as ManNet dataset. Additionally, we have benchmarked the performance of existing algorithms for manipulated image detection. The experimental results highlight that inter-set (disjoint training testing) evaluations are the major challenge of manipulated image detection.

The availability of large training data has led to the development of sophisticated deep learning algorithms to achieve state-of-the-art performance on various tasks and several applications have been benefited immensely. Despite the unparalleled success, the performance of deep learning algorithms depends significantly on the training data distribution. An imbalance in training data distribution affects the performance of deep models. Our research focuses on designing and developing solutions for different real-world problems, specifically related to facial analytic tasks, with imbalanced data distribution. These problems include injured face recognition, fake image detection, and estimation and mitigation of bias in model prediction.

With the advancements in deep learning technologies, real-world applications like face detection, gender prediction, and face recognition have achieved human-level performance. However, the emergence of the COVID-19 pandemic brought new challenges to existing deep learning algorithms. People are forced to wear a mask to limit the spread of COVID-19. These face masks occlude a significant portion of the face, thereby posing multiple challenges to existing algorithms. Images captured using surveillance cameras have a low resolution which hinders the model performance. Along with this, skin tone, ethnicity and attire also play a significant role in detection and recognition performance. India is a large country with huge diversity in skin tone and attire of the people. To address the challenges due to masks in the Indian context, we propose a novel Dual Sensor Indian Masked Face (DS- IMF) dataset, which contains images captured in constrained environmental settings with a variety of masks and degrees of occlusion. Multiple experiments are performed on the DS- IMF dataset at different resolutions. Experimental results demonstrate the limitations of existing algorithms on low-resolution masked face images. The proposed dataset can be found at http://www.iab-rubric.org/resources/dsimf.html.

Due to the COVID-19 pandemic, wearing face masks has become a mandate in public places worldwide. Face masks occlude a significant portion of the facial region. Additionally, people wear different types of masks, from simple ones to ones with graphics and prints. These pose new challenges to face recognition algorithms. Researchers have recently proposed a few masked face datasets for designing algorithms to overcome the challenges of masked face recognition. However, existing datasets lack the cultural diversity and collection in the unrestricted settings. Country like India with attire diversity, people are not limited to wearing traditional masks but also clothing like a thin cotton printed towel (locally called as “gamcha”), “stoles”, and “handkerchiefs” to cover their faces. In this paper, we present a novel Indian Masked Faces in the Wild (IMFW) dataset which contains images with variations in pose, illumination, resolution, and the variety of masks worn by the subjects. We have also benchmarked the performance of existing face recognition models on the proposed IMFW dataset. Experimental results demonstrate the limitations of existing algorithms in presence of diverse conditions.

Projection algorithms learn a transformation function to project the data from input space to the feature space, with the objective of increasing the inter-class distance. However, increasing the inter-class distance can affect the intra-class distance. Maintaining an optimal inter-class separation among the classes without affecting the intra-class distance of the data distribution is a challenging task. In this paper, inspired by the Coulomb's law of Electrostatics, we propose a new algorithm to compute the equilibrium space of any data distribution where the separation among the classes is optimal. The algorithm further learns the transformation between the input space and equilibrium space to perform classification in the equilibrium space. The performance of the proposed algorithm is evaluated on four publicly available datasets at three different resolutions. It is observed that the proposed algorithm performs well for lowresolution images.

In real-world applications, commercial off-the-shelf systems are utilized for performing automated facial analysis including face recognition, emotion recognition, and attribute prediction. However, a majority of these commercial systems act as black boxes due to the inaccessibility of the model parameters which makes it challenging to fine-tune the models for specific applications. Stimulated by the advances in adversarial perturbations, this research proposes the concept of Data Fine-tuning to improve the classification accuracy of a given model without changing the parameters of the model. This is accomplished by modeling it as data (image) perturbation problem. A small amount of “noise” is added to the input with the objective of minimizing the classification loss without affecting the (visual) appearance. Experiments performed on three publicly available datasets LFW, CelebA, and MUCT, demonstrate the effectiveness of the proposed concept.