Feature selection is a critical aspect of eXplainable Artificial Intelligence (XAI), and it has implications for model interpretability and predictive performance. CoFE (Consistency-driven Feature Elimination) framework was introduced recently using a frequentist approach. CoFE eliminates features with inconsistent coefficient signs in Linear Regression models by estimating the Sign Entropy (variability of the sign) of the coefficients using Bootstrapping. However, the uncertainty associated with estimating Sign Entropy using bootstrapping leads to slower convergence and inconsistency in feature subset selection in CoFE. In this paper, we present Bay-CoFE, 1 a Bayesian reformulation of CoFE, to solve the slow convergence and inconsistency issues of CoFE while retaining the benefits of selecting features with lower Sign Entropy in a Bayesian framework. We provide theoretical justifications and empirical evidence to prove Bay-CoFE’s superior convergence properties. Across all datasets, Bay-CoFE achieves significantly superior sign stability compared to traditional feature selection methods (Mann-Whitney U test, p-value < = 1.63e-03 and mean Cliff’s delta ≈ 0.83), with minimal predictive performance differences (Mann-Whitney U test, p-value > 0.1 and mean Cliff’s delta ≈ 0.32), demonstrating a highly favorable trade-off for interpretable modeling.

Saliency maps are widely used as a post-hoc approach to explain the decision-making process of Deep Learning (DL) based image classification models, but evaluating their fidelity remains a complex problem. While saliency metrics have been introduced to evaluate the fidelity of saliency maps, existing saliency metrics, such as perturbation-based saliency metrics, have been previously reported to demonstrate statistical inconsistency. Although inconsistencies have been noted in different works, there exists no mechanism for estimating the same, i.e., Inconsistency Estimation (IE). Our primary objective is to address this limitation, and therefore, we propose a framework to estimate the inconsistency of saliency metrics for any given DL model. The framework enables building IE models for estimating the inconsistency by employing a set of perturbation types and schemes. The framework’s modular architecture provides flexibility across (i) perturbation types (Inpainting, Uniform, and Gaussian blur), (ii) perturbation schemes (pixel-wise and patch-wise), (iii) learning mechanisms (Convolutional Neural Networks and Vision Transformers) and (iv) IE modeling techniques (bagging and boosting). Extensive experimental results are shown on three well-known DL architectures (Inception-V3, Xception, and ResNet-50) on three different public datasets, including the Imagenette, Oxford-IIIT Pets Dataset, and PASCAL VOC 2007, along with results on ViTs for Oxford-IIIT Pets Dataset, and PASCAL VOC 2007. With a comprehensive evaluation of seven different perturbation types that include two inpainting, two Gaussian blur (with kernel widths of 0.9 and 1.5), and three uniform perturbations, our work shows the effectiveness of the proposed approach in estimating inconsistency. Statistically founded tests such as repeated cross-validation and the Permutation Test further validate the idea of the proposed framework for estimating the inconsistency of saliency metrics across unseen perturbations, making it useful in real-world scenarios.

Face morphing attacks pose a substantial risk to the reliability of face recognition systems used in passport issuance, border control, and digital identity verification. Detecting morphing attacks from a single facial image remains challenging owing to the lack of a trusted reference and the diversity of attack generation methods. This paper presents a new Single-Image Face Morphing Attack Detection (S-MAD) framework that integrates high-frequency Laplacian residual statistics with representations from a frozen, foundation-scale vision transformer. The approach employs residual-statistic-gated low-rank adapters (R-FLoRA) and feature-wise residual fusion (Res-FiLM) to enhance sensitivity to local morphing artefacts while preserving the semantic context of the backbone. A novel residual-contrastive alignment loss further regularises the fused token space, improving discrimination under unseen morphing conditions. Comprehensive experiments on four ICAO-compliant datasets, encompassing seven morph generation techniques, demonstrate that the proposed method consistently surpasses nine recent state-of-the-art S-MAD algorithms in detection accuracy and cross-domain (or dataset) generalisation. With a frozen backbone and minimal trainable parameters, the model achieves real-time efficiency and interpretability, making it suitable for real-life scenarios in biometric verification systems.

Leveraging the power of multimodal large language models (LLMs) offers a promising approach to enhancing the accuracy and interpretability of morphing attack detection (MAD), especially in real-world biometric applications. This work introduces the use of LLMs for differential morphing attack detection (D-MAD). To the best of our knowledge, this is the first study to employ multimodal LLMs to D-MAD using real biometric data. To effectively utilize these models, we design Chain-of-Thought (CoT)-based prompts to reduce failure-to-answer rates and enhance the reasoning behind decisions. Our contributions include: (1) the first application of multimodal LLMs for D-MAD using real data subjects, (2) CoT-based prompt engineering to improve response reliability and explainability, (3) comprehensive qualitative and quantitative benchmarking of LLM performance using data from 54 individuals captured in passport enrollment scenarios, and (4) comparative analysis of two multimodal LLMs: ChatGPT-4o and Gemini providing insights into their morphing attack detection accuracy and decision transparency. Experimental results show that ChatGPT-4o outperforms Gemini in detection accuracy, especially against GAN-based morphs, though both models struggle under challenging conditions. While Gemini offers more consistent explanations, ChatGPT-4o is more resilient but prone to a higher failure-to-answer rate.

Deepfake technology employs sophisticated machine-learning techniques to create highly convincing video and audio recordings of individuals doing or saying things that they never actually did or said. These falsified media pieces have the potential to deceive and manipulate viewers, posing significant risks to their privacy, security, and trust in digital media. In this paper, we present a novel method DualStreamNet for reliable Audio-Video (AV) fake media detection which exploits the complementary information. The proposed DualStreamNet includes independent detector for video and audio modality. We introduced a novel video fake detection framework using a SlowFast encoder as the backbone, and a novel architecture based on a 3D CNN with skip connections. We also introduced novel features to reliably detect audio fakes using a Continuous Wavelet Transform (CWT) Filter Bank that was further processed using the ResNet50 architecture. Finally, the decisions from the video and audio detectors are combined using the logical OR rule to make the final decision. Extensive experiments were performed on two publicly available audio-video fake datasets: FakeAVCeleb and SWAN-DF. The obtained results indicate the improved detection accuracy of the proposed method compared to existing methods.

Face morphing attacks threaten the integrity of biometric identity systems by enabling multiple individuals to share a single identity. To develop and evaluate effective morphing attack detection (MAD) systems, we need access to high-quality, realistic morphed images that reflect the challenges posed in real-world scenarios. However, existing morph generation methods often produce images that are blurry, riddled with artifacts, or poorly constructed—making them easy to detect and not representative of the most dangerous attacks. In this work, we introduce StableMorph, a novel approach that generates highly realistic, artifact-free morphed face images using modern diffusion-based image synthesis. Unlike prior methods, StableMorph produces full-head images with sharp details, avoids common visual flaws, and offers unmatched control over visual attributes. Through extensive evaluation, we show that StableMorph images not only rival or exceed the quality of genuine face images, but also maintain a strong ability to fool face recognition systems—posing a greater challenge to existing MAD solutions and setting a new standard for morph quality in research and operational testing. StableMorph improves the evaluation of biometric security by creating more realistic and effective attacks and supports the development of more robust detection systems.

Identity details within an ECG is jointly situated within local and global features. The current methods for ECG recognition emphasize only on local or global details. They have also paid limited attention to unseen and cross-domain scenarios. Furthermore, there exists a lack of consensus on evaluation strategies. Thus, this paper introduces LGTraNet, a generalized architecture designed to establish baselines for securing personal identity using ECG biometrics in cross-domain scenarios. Our proposed model firstly extracts identity details at local temporal levels. The extracted features are then calibrated with globally details using a Self-Calibrated Normalizing Residual Network (SCNRNet). Finally, the refined local details are aggregated using a transformer model to formulate robust global identity representations. We evaluate LGTraNet over challenging cross-domain scenarios, such as cross-session and cross-database. To mitigate challenges in domain-shift, we also introduce an transfer learning based training strategy. Experimental study conducted on three benchmark datasets, ECG1D, MIT-BIH, and PTB, shows that the LGTraNet achieves significant performance in cross-domain settings, and outperforms state-of-the-art. Our code is available at: https://github.com/AmanVerma2307/LGTraNet.

Presentation attack detection (PAD) remains a key challenge in contactless fingerprint recognition, especially with the rise of fingerphoto based authentication using smartphones. This work introduces a novel PAD approach using multimodal Large Language Models (LLMs) such as GPT-4o and Gemini 2.0 for interpretable and data-efficient spoof detection. We assess these models under zero-shot, few-shot, and chain-of-thought (CoT) prompting to evaluate their reasoning ability, generalization to unseen attacks, and multi-class classification of presentation attack instruments (PAIs). Experiments on a newly collected dataset (300 bona fide and 1200 spoof samples across four PAIs) show that GPT-4o, especially with CoT and few-shot prompting, outperforms Gemini 2.0 in both detection accuracy and interpretability. These results highlight the potential of LLMs as generalizable and explainable PAD solutions, minimizing the need for large annotated datasets and domain-specific models.

This paper presents a summary of the 2nd Latent in the Wild Fingerprint Recognition Competition held at the 2025 International Joint Conference on Biometrics. The competition has two tracks: latent fingerprint 1) recognition, and 2) quality assessment. It attracted a total of 12 participating teams from academia and industry for both tracks, representing 10 countries. In total, 8 valid submissions were evaluated by the organizers. The competition aimed to advance the state-of-the-art in latent fingerprint recognition and quality assessment by providing a challenging dataset of latent fingerprints collected in natural, non-ideal conditions. This paper summarizes the dataset, evaluation protocols, submitted methods, and the competition results.

Morphing attack detection is a critical component of face recognition systems, ensuring reliable applications in border control. In this work, we propose a novel Differential Morphing Attack Detection (D-MAD) method capable of effectively capturing variations in differential features extracted using a Face Recognition System (FRS). Our proposed model, TimFusNet, combines time-frequency and temporal variations from the difference in facial embeddings computed from passport and trusted face images. The TimFusNet architecture comprises two branches: the first branch captures time-frequency variations from the input differential embeddings, while the second branch focuses on extracting temporal variations. The features from both branches are concatenated and passed through a self-attention layer to reliably detect morphing attacks on FRS. We conducted extensive experiments using a morphing dataset constructed from the FRGC dataset, employing five different morphing generation techniques. The detection performance of TimFusNet was benchmarked against four existing D-MAD techniques under two evaluation protocols. The results demonstrate that TimFusNet delivers outstanding detection performance across both protocols.

LivDet-Iris 2025 is the sixth edition of the iris liveness detection competition. Held every two to three years, the competition aims to foster the development of robust algorithms capable of detecting a wide range of physically-and digitally-presented attacks in iris biometrics. The 2025 edition obtained the largest number of submissions in the history of the competition: ten algorithms from five institutions, and one commercial iris recognition system. LivDet-Iris 2025 also introduced new tasks compared to previous editions: (Task 1) a benchmark offered by an industry partner, (Task 2) morphed iris images, in which two different-identity samples were blended into one image, and (Task 3) evaluation of presentation attack detection robustness against advanced manufacturing techniques for textured contact lenses. This edition, for the first time in the series, offers a systematic testing of a commercial iris recognition system (software and hardware) using physical artifacts presented to the sensor. Dermalog-Iris team submitted algorithms that won all tasks, achieving the area under the ROC curve of 90.57%, 68.23% and 99.99% in tasks 1, 2, and 3, respectively. Additionally, we include results for baseline algorithms, based on modern deep convolutional neural networks and trained with all available public datasets of iris images representing bona fide samples and anomalies (physical attacks, eye diseases, post-mortem cases, and synthetically-generated iris images). Test samples created for tasks 2 and 3, and baseline models are made available to offer the state-of-the-art benchmark for iris liveness detection.

Advancements in virtual and head-mounted devices have introduced new challenges for iris biometrics, such as varying gaze directions, partial occlusions, and inconsistent lighting conditions. To address these obstacles, we present VREyeSAM, a robust iris segmentation framework specifically designed for images captured under both steady and dynamic gaze scenarios. Our pipeline includes a quality-aware pre-processing module that filters out partially or fully closed eyes, ensuring that only high-quality, fully open iris images are used for training and inference. In addition, we introduce an uncertainty weighted hybrid loss function that adaptively balances multiple learning objectives, enhancing the robustness of the model under diverse visual conditions. Using this approach, we evaluate VREyeSAM on the VRBiom dataset, where it achieves state-of-the-art performance with a Precision of 0.751, Recall of 0.870, F1-Score of 0.806, and a mean IoU of 0.647, significantly outperforming existing segmentation methods.

This work summarises and reports the results of the second Presentation Attack Detection competition on ID cards. This new version includes new elements compared to the previous one. (1) An automatic evaluation platform was enabled for automatic benchmarking; (2) Two tracks were proposed in order to evaluate algorithms and datasets respectively; and (3) A new ID card dataset was shared with Track 1 teams to serve as the baseline dataset for the training and optimisation. The Hochschule Darmstadt, Fraunhofer-IGD, and Facephi company jointly organised this challenge. 20 teams were registered, and 74 submitted models were evaluated. For Track 1, the "Dragons" team reached first place with an Average Ranking and Equal Error rate (EER) of (AVRank) of 40.48% and 11.44% EER, respectively. For the more challenging approach in Track 2, the "Incode" team reached the best results with an AVRank of 14.76% and 6.36% EER, improving on the results of the first edition of 74.30% and 21.87% EER, respectively. These results suggest that PAD on ID cards is improving, but it is still a challenging problem related to the number of images, especially of bona fide images.

This paper presents a summary of the 2025 Sclera Segmentation Benchmarking Competition (SSBC), which focused on the development of privacy-preserving sclera-segmentation models trained using synthetically generated ocular images. The goal of the competition was to evaluate how well models trained on synthetic data perform in comparison to those trained on real-world datasets. The competition featured two tracks: (i) one relying solely on synthetic data for model development, and (ii) one combining/mixing synthetic with (a limited amount of) real-world data. A total of nine research groups submitted diverse segmentation models, employing a variety of architectural designs, including transformer-based solutions, lightweight models, and segmentation networks guided by generative frameworks. Experiments were conducted across three evaluation datasets containing both synthetic and real-world images, collected under diverse conditions. Results show that models trained entirely on synthetic data can achieve competitive performance, particularly when dedicated training strategies are employed, as evidenced by the top performing models that achieved F1scores of over 0.8 in the synthetic data track. Moreover, performance gains in the mixed track were often driven more by methodological choices rather than by the inclusion of real data, highlighting the promise of synthetic data for privacy-aware biometric development. The code and data for the competition is available at: https://github.com/dariant/SSBC_2025.

Explanations of Local Interpretable Model-agnostic Explanations (LIME) are often inconsistent across different runs making them unreliable for eXplainable AI (XAI). The inconsistency stems from sign flips and variability in ranks of the segments for each different run. We propose a Bayesian Regularization approach to reduce sign flips, which in turn stabilizes feature rankings and ensures significantly higher consistency in explanations. The proposed approach enforces sparsity by incorporating a Sign Entropy prior on the coefficient distribution and dynamically eliminates features during optimization. Our results demonstrate that the explanations from the proposed method exhibit significantly better consistency and fidelity than LIME (and its earlier variants). Further, our approach exhibits comparable consistency and fidelity with a significantly lower execution time than the latest LIME variant, i.e., SLICE (CVPR 2024).

Morphing attacks can successfully deceive face recognition systems, resulting in unreliable access control, especially in the border control scenario. Consequently, the development of Morphing Attack Detection (MAD) algorithms is crucial for detecting morphing attacks based on either a single facial image (S-MAD) or two facial images (Differential-MAD or D-MAD). In this work, we proposed a novel D-MAD approach, PoolAtnRes, to reliably detect morphing attacks. The proposed PoolAtnRes architecture is constructed using three main functional blocks, namely convolution pooling, Hybrid Attention and Residual blocks, which are serially connected to detect morphing attacks. Extensive experiments were performed on the newly constructed morphing dataset using nine morphing-generation techniques. The detection performance of the proposed PoolAtnRes model was compared with three state-of-the-art (SOTA) D-MAD techniques with different performance evaluation protocols to benchmark its generalizabllity to unseen morphing generation. The results obtained indicated the best performance of the proposed PoolAtnRes D-MAD.

Finger photo Presentation Attack Detection (PAD) can significantly strengthen smartphone device security. However, these algorithms are trained to detect certain types of attacks. Furthermore, they are designed to operate on images acquired by specific capture devices, leading to poor generalization and a lack of robustness in handling the evolving nature of mobile hardware. The proposed investigation is the first to systematically analyze the performance degradation of existing deep learning PAD systems, convolutional and transformers, in cross-capture device settings. In this paper, we introduce the ColFigPhotoAttnNet11The implementation of ColFigPhotoAttnNet can be accessed at https://github.com/avurity/ColFigPhotoAttnNet architecture designed based on window attention on color channels, followed by the nested residual network as the predictor to achieve a reliable PAD. Extensive experiments using various capture devices, including iPhone13 Pro, GooglePixel 3, Nokia C5, and OnePlusOne, were carried out to evaluate the performance of proposed and existing methods on three publicly available databases. The findings underscore the effectiveness of our approach.

Person re-identification (ReID) has gained significant interest due to growing public safety concerns that require advanced surveillance and identification mechanisms. While most existing ReID research relies on static surveillance cameras, the use of Unmanned Aerial Vehicles (UAVs) for surveillance has recently gained popularity. Noting the promising application of UAVs in ReID, this paper presents a comprehensive overview of UAV-based ReID, highlighting publicly available datasets, key challenges, and methodologies. We summarize and consolidate evaluations conducted across multiple studies, providing a unified perspective on the state of UAV-based ReID research. Despite their limited size and diversity, We underscore current datasets’ importance in advancing UAV-based ReID research. The survey also presents a list of all available approaches for UAV-based ReID. The survey presents challenges associated with UAV-based ReID, including environmental conditions, image quality issues, and privacy concerns. We discuss dynamic adaptation techniques, multi-model fusion, and lightweight algorithms to leverage ground-based person ReID datasets for UAV applications. Finally, we explore potential research directions, highlighting the need for diverse datasets, lightweight algorithms, and innovative approaches to tackle the unique challenges of UAV-based person ReID. • Comprehensive survey of all publicly available datasets for UAV-based person ReID. • Detailed discussion on challenges in UAV-based person ReID. • Detailed analysis of UAV ReID methodologies and current state-of-the-art. • Potential future directions for UAV-based person ReID.

Local Interpretable Model-agnostic Explanations (LIME) - a widely used post-ad-hoc model agnostic ex-plainable AI (XAI) technique. It works by training a simple transparent (surrogate) model using random samples drawn around the neighborhood of the instance (image) to be explained (IE). Explanations are then extracted for a black-box model and a given IE, using the surrogate model. However, the explanations of LIME suffer from inconsistency across different runs for the same model and the same IE. We identify two main types of inconsistencies: variance in the sign and importance ranks of the segments (superpixels). These factors hinder LIME from obtaining consistent explanations. We analyze these inconsistencies and propose a new method, Stabilized LIME for Consistent Explanations (SLICE). The proposed method handles the stabilization problem in two aspects: using a novel feature selection technique to eliminate spurious superpixels and an adaptive perturbation technique to generate perturbed images in the neighborhood of IE. Our results demonstrate that the explanations from SLICE exhibit significantly better consistency and fidelity than LIME (and its variant BayLime).

Numerous studies have shown that existing Face Recognition Systems (FRS), including commercial ones, often exhibit biases toward certain ethnicities due to under-represented data. In this work, we explore ethnicity alteration and skin tone modification using synthetic face image generation methods to increase the diversity of datasets. We conduct a detailed analysis by first constructing a balanced face image dataset representing three ethnicities: Asian, Black, and Indian. We then make use of existing Generative Adversarial Network-based (GAN) image-to-image translation and manifold learning models to alter the ethnicity from one to another. A systematic analysis is further conducted to assess the suitability of such datasets for FRS by studying the realistic skin-tone representation using Individual Typology Angle (ITA). Further, we also analyze the quality characteristics using existing Face image quality assessment (FIQA) approaches. We then provide a holistic FRS performance analysis using four different systems. Our findings pave the way for future research works in (i) developing both specific ethnicity and general (any to any) ethnicity alteration models, (ii) expanding such approaches to create databases with diverse skin tones, (iii) creating datasets representing various ethnicities which further can help in mitigating bias while addressing privacy concerns.

Facial biometrics are an essential components of smartphones to ensure reliable and trustworthy authentication. However, face biometric systems are vulnerable to Presentation Attacks (PAs), and the availability of more sophisticated presentation attack instruments such as 3D silicone face masks will allow attackers to deceive face recognition systems easily. In this work, we propose a novel Presentation Attack Detection (PAD) algorithm based on 3D point clouds captured using the frontal camera of a smartphone to detect presentation attacks. The proposed PAD algorithm, VoxAtnNet, processes 3D point clouds to obtain voxelization to preserve the spatial structure. Then, the voxelized 3D samples were trained using the novel convolutional attention network to detect PAs on the smartphone. Extensive experiments were carried out on the newly constructed 3D face point cloud dataset comprising bona fide and two different 3D PAIs (3D silicone face mask and wrap photo mask), resulting in 3480 samples. The performance of the proposed method was compared with existing methods to benchmark the detection performance using three different evaluation protocols. The experimental results demonstrate the improved performance of the proposed method in detecting both known and unknown face presentation attacks.

Face Recognition Systems (FRS) are widely used in commercial environments, such as e-commerce and e-banking, owing to their high accuracy in real-world conditions. However, these systems are vulnerable to facial morphing attacks, which are generated by blending face color images of different subjects. This paper presents a new method for generating 3D face morphs from two bona fide point clouds. The proposed method first selects bona fide point clouds with neutral expressions. The two input point clouds were then registered using a Bayesian Coherent Point Drift (BCPD) without optimization, and the geometry and color of the registered point clouds were averaged to generate a face morphing point cloud. The proposed method generates 388 face-morphing point clouds from 200 bona fide subjects. The effectiveness of the method was demonstrated through extensive vulnerability experiments, achieving a Generalized Morphing Attack Potential (G-MAP) of 97.93%, which is superior to the existing state-of-the-art (SOTA) with a G-MAP of 81.61%.

The automatic prediction of gender from the face has been studied extensively because of its potential relevance in numerous applications related to security. Although the problem of gender classification based on the face is substantial, it remains far from being solved under difficult environmental exposure, especially for different illuminations. In this work, we demonstrate the merits and demerits of classifying gender under cross-illumination variants. We present our approach by employing multi-spectral imaging in nine narrow-spectrum bands stemming from the visible to near-infrared range. The experimental evaluation results were obtained on 78300 sample face images of 145 subjects captured under six different illumination conditions. Further, we present quantitative and qualitative experimental evaluations to determine the average classification accuracy for setting the benchmark results. To demonstrate the goal of this work, we present the results based on three image fusion techniques independently processed using five feature extraction methods for cross-illumination scenarios. This work obtained the highest classification accuracy of $96.32 \%$ for cross-illumination conditions, demonstrating the reliability of employing an image fusion approach to combine complementary information from spectral bands in difficult environmental exposure.

Morphing attacks blend face images of multiple distinct identities into a single photo combining their facial characteristics. Since the morphed image can be verified to multiple identities, these attacks greatly threaten face recognition systems. Morphing Attack Detection (MAD), addresses this problem by determining if an image is a bona fide or morphed image. Previous works on MAD focused on the models’ decision to improve their performance and generalizability. However, it has not been investigated whether the estimated probabilities of such decisions accurately reflect the true confidence with which predictions are made. Since these models struggle with unseen attacks, it is important to determine the models’ decision confidence to decide to trust or distrust the decisions. In this work, we (a) demonstrate that state-of-the-art MAD models struggle with providing reliable confidence estimations, (b) propose two new metrics to categorize their confidence prediction behavior, and (c) demonstrate that a simple calibration method can make the model more trustworthy without changing general model performance. The experiments were conducted in cross-dataset evaluation settings across two different MAD models using three different preprocessing and five morphing generation techniques. We found that both models are not calibrated and therefore not trustworthy. Moreover, it is shown that a simple and effective adjustment of the prediction confidence is possible. This will make future presentation attack detection, and especially MAD, models more trustworthy.

Acquiring face images of sufficiently high quality is important for online ID and travel document issuance applications using face recognition systems (FRS). Low-quality, manipulated (intentionally or unintentionally), or distorted images degrade the FRS performance and facilitate documents’ misuse. Securing quality for enrolment images, especially in the unsupervised self-enrolment scenario via a smartphone, becomes important to assure FRS performance. In this work, we focus on the less studied area of radial distortion (a.k.a., the fish-eye effect) in face images and its impact on FRS performance. We introduce an effective radial distortion detection model that can detect and flag radial distortion in the enrolment scenario. We formalize the detection model as a face image quality assessment (FIQA) algorithm and provide a careful inspection of the effect of radial distortion on FRS performance. Evaluation results show excellent detection results for the proposed models, and the study on the impact on FRS uncovers valuable insights into how to best use these models in operational systems.

Smartphone-based contactless fingerphoto authentication has become a reliable alternative to traditional contact-based fingerprint biometric systems owing to rapid advances in smartphone camera technology. Despite its convenience, fingerprint authentication through fingerphotos is more vulnerable to presentation attacks, which has motivated recent research efforts towards developing fingerphoto Presentation Attack Detection (PAD) techniques. However, prior PAD approaches utilized supervised learning methods that require labeled training data for both bona fide and attack samples. This can suffer from two key issues, namely (i) generalization—the detection of novel presentation attack instruments (PAIs) unseen in the training data, and (ii) scalability—the collection of a large dataset of attack samples using different PAIs. To address these challenges, we propose a novel unsupervised approach based on a state-of-the-art deep-learning-based diffusion model, the Denoising Diffusion Probabilistic Model (DDPM), which is trained solely on bona fide samples. The proposed approach detects Presentation Attacks (PA) by calculating the reconstruction similarity between the input and output pairs of the DDPM. We present extensive experiments across three PAI datasets to test the accuracy and generalization capability of our approach. The results show that the proposed DDPM-based PAD method achieves significantly better detection error rates on several PAI classes compared to other baseline unsupervised approaches.

Deepfakes are a major security risk for biometric authentication. This technology creates realistic fake videos that can impersonate real people, fooling systems that rely on facial features and voice patterns for identification. Existing multimodal deepfake detectors rely on conventional fusion methods, such as majority rule and ensemble voting, which often struggle to adapt to changing data characteristics and complex patterns. In this paper, we introduce the Straight-through Gumbel-Softmax (STGS) framework, offering a comprehensive approach to search multimodal fusion model architectures. Using a two-level search approach, the framework optimizes the network architecture, parameters, and performance. Initially, crucial features were efficiently identified from backbone networks, whereas within the cell structure, a weighted fusion operation integrated information from various sources. An architecture that maximizes the classification performance is derived by varying parameters such as temperature and sampling time. The experimental results on the FakeAVCeleb and SWAN-DF datasets demonstrated an impressive AUC value 94.4% achieved with minimal model parameters. https://github.com/Aravinda27/STGS-BMNAS

Recent studies have emphasized the potential of forehead-crease patterns as an alternative for face, iris, and periocular recognition, presenting contactless and convenient solutions, particularly in situations where faces are covered by surgical masks. However, collecting forehead data presents challenges, including cost and time constraints, as developing and optimizing forehead verification methods requires a substantial number of high-quality images. To tackle these challenges, the generation of synthetic biometric data has gained traction due to its ability to protect privacy while enabling effective training of deep learning-based biometric verification methods. In this paper, we present a new framework to synthesize forehead-crease image data while maintaining important features, such as uniqueness and realism. The proposed framework consists of two main modules: a Subject-Specific Generation Module (SSGM), based on an image-to-image Brownian Bridge Diffusion Model (BBDM), which learns a one-to-many mapping between image pairs to generate identity-aware synthetic forehead creases corresponding to real subjects, and a Subject-Agnostic Generation Module (SAGM), which samples new synthetic identities with assistance from the SSGM. We evaluate the diversity and realism of the generated forehead-crease images primarily using the Fréchet Inception Distance (FID) and the Structural Similarity Index Measure (SSIM). In addition, we assess the utility of synthetically generated forehead-crease images using a forehead-crease verification system (FHCVS). The results indicate an improvement in the verification accuracy of the FHCVS by utilizing synthetic data.

This paper summarises the Competition on Presentation Attack Detection on ID Cards (PAD-IDCard) held at the 2024 International Joint Conference on Biometrics (IJCB 2024). The competition attracted a total of ten registered teams, both from academia and industry. In the end, the participating teams submitted five valid submissions, with eight models to be evaluated by the organisers. The competition presented an independent assessment of current state-of-the-art algorithms. Today, no independent evaluation on cross-dataset is available; therefore, this work determined the state-of-the-art on ID cards. To reach this goal, a sequestered test set and baseline algorithms were used to evaluate and compare all the proposals. The sequestered test dataset contains ID cards from four different countries. In summary, a team that chose to be "Anonymous" reached the best average ranking results of 74.80%, followed very closely by the "IDVC" team with 77.65%.

Finger photo verification has emerged as a viable alternative to traditional biometric authentication methods in smartphones, offering improved hygiene and user experience while utilizing standard RGB cameras to capture images of human fingers. However, the vulnerability of finger photo technology to Presentation Attacks (PAs) necessitates the integration of a robust detection mechanism. The current study evaluates the effectiveness of deep-learning-based finger photo Presentation Attack detectors for various types of Presentation Attack Instruments (PAIs). This paper compares and interprets the performance of fine-tuned Convolutional Neural Networks (CNNs) and transformer models for Presentation Attack Detection (PAD). Experiments were conducted on three datasets, including MFPAD-i-22 with 112 subjects, MFPAD-g-23 with 100 subjects, and IIIT-D with 64 subjects, encompassing 19 attack scenarios involving devices such as iPhones, iPads, and HP printers. To interpret the PAD algorithm decisions against a range of PAs, eXplainable Artificial Intelligence (XAI) methodologies were employed to gain insight into the importance of the features used by PAD. The results indicated that the swine transformer outperformed CNNs in detecting various types of PAs. Furthermore, quantifying the interpretable results using the signal-to-noise ratio indicated the importance of features used by the PAD to detect the PAs.

Biometric verification systems are deployed in various security-based access-control applications that require user-friendly and reliable person verification. Among the different biometric characteristics, fingervein biometrics have been extensively studied owing to their reliable verification performance. Furthermore, fingervein patterns reside inside the skin and are not visible outside; therefore, they possess inherent resistance to presentation attacks and degradation due to external factors. In this paper, we introduce a novel fingervein verification technique using a convolutional multihead attention network called VeinAtnNet. The proposed VeinAtnNet is designed to achieve light weight with a smaller number of learnable parameters while extracting discriminant information from both normal and enhanced fingervein images. The proposed VeinAtnNet was trained on the newly constructed fingervein dataset with 300 unique fingervein patterns that were captured in multiple sessions to obtain 92 samples per unique fingervein. Extensive experiments were performed on the newly collected dataset FV-300 and the publicly available FV-USM and FV-PolyU fingervein dataset. The performance of the proposed method was compared with five state-of-the-art fingervein verification systems, indicating the efficacy of the proposed VeinAtnNet.

Face morphing attack detection is emerging as an increasingly challenging problem owing to advancements in high-quality and realistic morphing attack generation. Reliable detection of morphing attacks is essential because these attacks are targeted for border control applications. This paper presents a multispectral framework for differential morphing-attack detection (D-MAD). The D-MAD methods are based on using two facial images that are captured from the ePassport (also called the reference image) and the trusted device (for example, Automatic Border Control (ABC) gates) to detect whether the face image presented in ePassport is morphed. The proposed multi-spectral D-MAD framework introduce a multispectral image captured as a trusted capture to acquire seven different spectral bands to detect morphing attacks. Extensive experiments were conducted on the newly created Multispectral Morphed Datasets (MSMD) with 143 unique data subjects that were captured using both visible and multispectral cameras in multiple sessions. The results indicate the superior performance of the proposed multispectral framework compared to visible images.

Latent fingerprints are among the most important and widely used evidence in crime scenes, digital forensics and law enforcement worldwide. Despite the number of advancements reported in recent works, we note that significant open issues such as independent benchmarking and lack of large-scale evaluation databases for improving the algorithms are inadequately addressed. The available databases are mostly of semi-public nature, lack of acquisition in the wild environment, and post-processing pipelines. Moreover, they do not represent a realistic capture scenario similar to real crime scenes, to benchmark the robustness of the algorithms. Further, existing databases for latent fingerprint recognition do not have a large number of unique subjects/fingerprint instances or do not provide ground truth/reference fingerprint images to conduct a cross-comparison against the latent. In this paper, we introduce a new wild large-scale latent fingerprint database that includes five different acquisition scenarios: reference fingerprints from (1) optical and (2) capacitive sensors, (3) smartphone fingerprints, latent fingerprints captured from (4) wall surface, (5) Ipad surface, and (6) aluminium foil surface. The new database consists of 1,318 unique fingerprint instances captured in all above mentioned settings. A total of 2,636 reference fingerprints from optical and capacitive sensors, 1,318 fingerphotos from smartphones, and 9,224 latent fingerprints from each of the 132 subjects were provided in this work. The dataset is constructed considering various age groups, equal representations of genders and backgrounds. In addition, we provide an extensive set of analysis of various subset evaluations to highlight open challenges for future directions in latent fingerprint recognition research.

Biometric systems are vulnerable to presentation attacks (PAs) performed using various PA instruments (PAIs). Even though there are numerous PA detection (PAD) techniques based on both deep learning and hand-crafted features, the generalization of PAD for unknown PAI is still a challenging problem. In this work, we empirically prove that the initialization of the PAD model is a crucial factor for generalization, which is rarely discussed in the community. Based on such observation, we proposed a self-supervised learning-based method, denoted as DF-DM. Specifically, DF-DM is based on a global-local view coupled with de-folding and de-mixing to derive the task-specific representation for PAD. During de-folding, the proposed technique will learn region-specific features to represent samples in a local pattern by explicitly minimizing the generative loss. While de-mixing drives detectors to obtain the instance-specific features with global information for more comprehensive representation by minimizing the interpolation-based consistency. Extensive experimental results show that the proposed method can achieve significant improvements in terms of both face and fingerprint PAD in more complicated and hybrid datasets when compared with the state-of-the-art methods. When training in CASIA-FASD and Idiap Replay-Attack, the proposed method can achieve an 18.60% equal error rate (EER) in OULU-NPU and MSU-MFSD, exceeding the baseline performance by 9.54%. The source code of the proposed technique is available at https://github.com/kongzhecn/dfdm.

Detecting forged handwriting is important in a wide variety of machine learning applications, and it is challenging when the input images are degraded with noise and blur. This article presents a new model based on conformable moments (CMs) and deep ensemble neural networks (DENNs) for forged handwriting detection in noisy and blurry environments. Since CMs involve fractional calculus with the ability to model nonlinearities and geometrical moments as well as preserving spatial relationships between pixels, fine details in images are preserved. This motivates us to introduce a DENN classifier, which integrates stenographic kernels and spatial features to classify input images as normal (original, clean images), altered (handwriting changed through copy-paste and insertion operations), noisy (added noise to original image), blurred (added blur to original image), altered-noise (noise is added to the altered image), and altered-blurred (blur is added to the altered image). To evaluate our model, we use a newly introduced dataset, which comprises handwritten words altered at the character level, as well as several standard datasets, namely ACPR 2019, ICPR 2018-FDC, and the IMEI dataset. The first two of these datasets include handwriting samples that are altered at the character and word levels, and the third dataset comprises forged International Mobile Equipment Identity (IMEI) numbers. Experimental results demonstrate that the proposed method outperforms the existing methods in terms of classification rate.

Temporal video segmentation is the get-to- go automatic video analysis, which decomposes a long-form video into smaller components for the following-up understanding tasks. Recent works have studied several levels of granularity to segment a video, such as shot, event, and scene. Those segmentations can help compare the semantics in the corresponding scales, but lack a wider view of larger temporal spans, especially when the video is complex and structured. Therefore, we present two abstractive levels of temporal segmentations and study their hierarchy to the existing fine-grained levels. Accordingly, we collect NewsNet, the largest news video dataset consisting of 1,000 videos in over 900 hours, associated with several tasks for hierarchical temporal video segmentation. Each news video is a collection of stories on different topics, represented as aligned audio, visual, and textual data, along with extensive frame-wise annotations in four granularities. We assert that the study on NewsNet can advance the understanding of complex structured video and benefit more areas such as short-video creation, personalized advertisement, digital instruction, and education. Our dataset and code is publicly available at https://github.com/NewsNet-Benchmark/NewsNet.

The face morphing process will combine two or more facial images to generate a single morphed facial image demonstrating Face Recognition Systems (FRS) vulnerability. The attack potential of the morphing image directly depends on the perceptual image quality, and when generated with no visible artefacts, it can deceive both human observers and automatic FRS. The current softwares for face morphing generates a morphing image with ghosting artefacts, especially in the eye region, nose and mouth area, which may serve as a potential cue to detect morphing attacks. Hence in this work, we introduce a new dataset comprising 10710 facial images before and after manual post-processing to reduce the visual artefacts and to generate high-quality attacks. Further, we propose a novel single image-based Morph Attack Detection (S-MAD) technique based on the ensemble of features and classifiers using the scale-space domain. The novel concept in the proposed method is the multilevel fusion that combines the comparison scores from different features and classifiers. Extensive experiments are carried out on the newly generated high-quality face images with (i) Morphs before post-processing and (ii) Morphs after post-processing. Further, the experiments are also carried out on two different mediums such as (i) Digital and (ii) Print-scan (or re-digitized) with and without compression. Extensive experimental results are performed to benchmark the detection performance with the existing S-MAD techniques. Obtained results indicate the best performance of the proposed method over existing methods.

This paper presents a summary of the Competition on Face Presentation Attack Detection Based on Privacy-aware Synthetic Training Data (SynFacePAD 2023) held at the 2023 International Joint Conference on Biometrics (IJCB 2023). The competition attracted a total of 8 participating teams with valid submissions from academia and industry. The competition aimed to motivate and attract solutions that target detecting face presentation attacks while considering synthetic-based training data motivated by privacy, legal and ethical concerns associated with personal data. To achieve that, the training data used by the participants was limited to synthetic data provided by the organizers. The submitted solutions presented innovations and novel approaches that led to outperforming the considered baseline in the investigated benchmarks.

This study introduces a new technique for generating synthetic faces using eyes-to-face inpainting methods. The proposed method can synthesize a face image using a combination of the eyes of two different individuals and use it as an input for inpainting, demonstrating its vast potential for various applications in biometrics. Despite minor biases in age and gender, our method proved effective in training reliable age- and gender-detection models using the generated datasets. We also addressed the challenge of training face recognition models using synthetic datasets, and the results demonstrated satisfactory accuracy across four benchmark face recognition datasets. This method could be particularly beneficial for underrepresented groups, for whom there is a scarcity of face samples in biometric datasets.

Facial biometrics are widely deployed in smartphone-based applications because of their usability and increased verification accuracy in unconstrained scenarios. The evolving applications of smartphone-based facial recognition have also increased Presentation Attacks (PAs), where an attacker can present a Presentation Attack Instrument (PAI) to maliciously gain access to the application. Because the materials used to generate PAI are not deterministic, the detection of unknown presentation attacks is challenging. In this paper, we present an acoustic echo-based face Presentation Attack Detection (PAD) on a smartphone in which the PAs are detected based on the reflection profiles of the transmitted signal. We propose a novel transmission signal based on the wide pulse that allows us to model the background noise before transmitting the signal and increase the Signal-to-Noise Ratio (SNR). The received signal reflections were processed to remove background noise and accurately represent reflection characteristics. The reflection profiles of the bona fide and PAs are different owing to the different reflection characteristics of the human skin and artefact materials. Extensive experiments are presented using the newly collected Acoustic Sound Echo Dataset (ASED) with 4807 samples captured from bona fide and four different types of PAIs, including print (two types), display, and silicone face-mask attacks. The obtained results indicate the robustness of the proposed method for detecting unknown face presentation attacks.

Face Recognition Systems (FRS) are vulnerable to morphing attacks that are targeted towards highly secured applications, including Automatic Border Control (ABC) gates. In this paper, we investigate a 3D-face custom silicone mask as the source for generating face morphing attacks for the first time. We present a systematic study to benchmark the attack potential of mask morphing (digital) attacks on both commercial and academic FRS. To this extent, a new dataset is constructed using eight custom 3D silicone face masks and corresponding bona fide face images captured using three different smartphones. The mask morphing is carried out using a landmark-based method, and the newly constructed dataset comprises 635 bona fide, 1034 face masks and 613 mask morphing face images. Extensive experiments are carried out to benchmark the attack potential and detection of mask morphing attacks on FRS.

This paper presents a summary of the Competition on Face Morphing Attack Detection Based on Privacy-aware Synthetic Training Data (SYN-MAD) held at the 2022 In-ternational Joint Conference on Biometrics (IJCB 2022). The competition attracted a total of 12 participating teams, both from academia and industry and present in 11 differ-ent countries. In the end, seven valid submissions were submitted by the participating teams and evaluated by the organizers. The competition was held to present and at-tract solutions that deal with detecting face morphing at-tacks while protecting people's privacy for ethical and le-gal reasons. To ensure this, the training data was limited to synthetic data provided by the organizers. The submitted solutions presented innovations that led to out-performing the considered baseline in many experimental settings. The evaluation benchmark is now available at: https://github.com/marcohuber/SYN-MAD-2022.

Face Recognition Systems (FRS) are vulnerable to various attacks performed directly and indirectly. Among these attacks, face morphing attacks are highly potential in deceiving automatic FRS and human observers and indicate a severe security threat, especially in the border control scenario. This work presents a face morphing attack detection, especially in the On-The-Fly (OTF) Automatic Border Control (ABC) scenario. We present a novel Differential-MAD (D-MAD) algorithm based on the spherical interpolation and hierarchical fusion of deep features computed from six different pre-trained deep Convolutional Neural Networks (CNNs). Extensive experiments are carried out on the newly generated face morphing dataset (SCFace-Morph) based on the publicly available SCFace dataset by considering the real-life scenario of Automatic Border Control (ABC) gates. Experimental protocols are designed to benchmark the proposed and state-of-the-art (SOTA) D-MAD techniques for different camera resolutions and capture distances. Obtained results have indicated the superior performance of the proposed D-MAD method compared to the existing methods.

Classification of photos captured by different photographers is an important and challenging problem in knowledge-based and image processing. Monitoring and authenticating images uploaded on social media are essential, and verifying the source is one key piece of evidence. We present a novel framework for classifying photos of different photographers based on the combination of local features and deep learning models. The proposed work uses focused and defocused information in the input images to extract contextual information. The model estimates the weighted gradient and calculates entropy to strengthen context features. The focused and defocused information is fused to estimate cross-covariance and define a linear relationship between them. This relationship results in a feature matrix fed to Knowledge Enforcement Network (KEN) for obtaining representative features. Due to the strong discriminative ability of deep learning models, we employ the lightweight and accurate MobileNetV2. The output of KEN and MobileNetV2 is sent to a classifier for photographer classification. Experimental results of the proposed model on our dataset of 46 photographer classes (46234 images) and publicly available datasets of 41 photographer classes (218303 images) show that the method outperforms the existing techniques by 5%–10% on average. The dataset created for the experimental purpose will be made available upon publication.

Face recognition systems (FRS) are vulnerable to different kinds of attacks. Morphing attack combines multiple face images to obtain a single face image that can verify equally against all contributing subjects. Various Morphing Attack Detection (MAD) algorithms have been proposed in recent years albeit limited generalizability. We present a new approach for MAD in this work with better generalization than state-of-the-art (SOTA) algorithms. We propose an end-to-end multi-stage encoder-decoder network for learning the residuals of morphing process to detect attacks. Leveraging the residuals, we learn an efficient classifier using cross-entropy loss and asymmetric loss. The use of asymmetric loss in our approach is motivated by imbalanced distribution of morphs and bona fides. An extensive set of experiments are conducted on five different datasets consisting of two landmark based and three Generative Adversarial Network (GAN) based morphs in various settings such as digital, print-scan and print-scan-compression. We first demonstrate a near-ideal performance of the proposed MAD with Detection Equal Error Rate (D-EER) of 0% in the best case and 2.58% in the worst case in the digital domain in closed-set protocol, i.e., known attacks. Further, we demonstrate the applicability of the proposed approach on 60 different combinations where the testing set contains unknown morphing attacks in open-set protocol to illustrate the generalization ability of our proposed approach. Through training the proposed approach on landmark-based morph generation data alone, we obtain an EER of 3.59% in the best case and 12.89% in the worst case for morphed images in the digital domain, reducing the error rates from 45.67% and 30.23% respectively, in open-set protocol. We further present an extensive analysis of the proposed approach through Class Activation Maps (CAM) to explain the decisions using by making use of three complementary CAM analysis.

Tattoo text detection provides a vital clue for person and crime identification. Due to the freestyle and unconstrained nature of handwritten tattoo text over skin regions, accurate tattoo text detection is very challenging. This paper proposes a comprehensive scheme for tattoo text detection which comprises (a) adaptive Deformable Convolutional Neural Network (DCNN) for skin region detection to reduce text detection complexity (b) a Decoupled Gradient Text Detector (DGTD) for tattoo text detection from skin region (c) a Deep Q-Network (DQN) to refine the bounding boxes detected by DGTD, and (d) a Term-Frequency-Inverse-Document-Frequency (TF-IDF) model to group the words into text lines based on semantic information to fix the bounding box for the line. To test the effectiveness, the proposed method is evaluated on different datasets, namely, (i) a newly developed tattoo text dataset, (ii) benchmark bib number dataset of the marathon, and (iii) person re-identification dataset. The proposed method achieves 91.2, 87.5, and 88.8 F-scores from these three respective datasets. To demonstrate its superior performance, the text detection module (without skin detection) is also compared with state-of-the-art scene text detection methods on benchmark datasets, namely, ICDAR 2019 ArT, Total-Text, and DAST1500 and the proposed method achieves 90.3, 88.5 and 89.8 F-score from these respective datasets.

Due to the proliferation of sports-related multimedia content on the WWW, effective visual search and retrieval present interesting research challenges. These are caused by poor image quality, a wide range of possible camera points of view, pose variations on the part of athletes engaged in playing a sport, deformations of text appearing on sports person’s clothing and uniforms in motion, occlusions caused by other objects, etc. To address these challenges, this paper presents a new method for detecting text on human bodies in sports images. Unlike most existing methods, which attempt to exploit locations of a player’s torso, face, and skin, we propose an end-to-end episodic learning approach that employs inductive learning criteria for detecting clothing regions in an image, which are, in turn, then used for text detection. Our method integrates a Residual Network (ResNet) and Pyramidal Pooling Module (PPM) for generating a spatial attention map. The Progressive Scalable Expansion Algorithm (PSE) is adapted for text detection from these regions. Experimental results on our own dataset as well as several benchmarks (like RBNR and MMM which contain images of runners in marathons, and Re-ID which is a person re-identification dataset) demonstrate that the proposed method outperforms existing methods in terms of precision and F1-score. We also present results for sports images chosen from natural scene text detection datasets such as CTW1500 and MS-COCO to show the proposed method is effective and reliable across a range of inputs.

With the evolution of electronic devices, such as 3D cameras, addressing the challenges of text localization in 3D video (e.g., for indexing) is increasingly drawing the attention of the multimedia and video processing community. Existing methods focus on 2D video and their performance in the presence of the challenges in 3D video, such as shadow areas associated with text and irregularly sized and shaped text, degrades. This paper proposes the first approach that successfully addresses the challenges of 3D video in addition to those of 2D. It employs a number of innovations, among which, the first is the Generalized Gradient Vector Flow (GGVF) for dominant points detection. The second is the Wavefront concept for text candidate point detection from those dominant points. In addition, an Adaptive B-Spline Polygon Curve Network (ABS-Net) is proposed for accurate text localization in 3D videos by constructing tight fitting bounding polygons using text candidate points. Extensive experiments on custom (3D video) and standard datasets (2D video and scene text) show that the proposed method is practical and useful, and overall outperforms existing state-of-the-art methods.

As face presentation attacks (PAs) are realistic threats for unattended face verification systems, face presentation attack detection (PAD) has been intensively investigated in past years, and the recent advances in face PAD have significantly reduced the success rate of such attacks. In this article, an empirical study on a novel and effective face impostor PA is made. In the proposed PA, a facial artifact is created by using the most vulnerable facial components, which are optimally selected based on the vulnerability analysis of different facial components to impostor PAs. An attacker can launch a face PA by presenting a facial artifact on his or her own real face. With a collected PA database containing various types of artifacts and presentation attack instruments (PAIs), the experimental results and analysis show that the proposed PA poses a more serious threat to face verification and PAD systems compared with the print, replay, and mask PAs. Moreover, the generalization ability of the proposed PA and the vulnerability analysis with regard to commercial systems are also investigated by evaluating unknown face verification and real-world PAD systems. It provides a new paradigm for the study of face PAs.

Recently, deep convolutional neural networks have achieved remarkable performance for the task of Single Image Super-Resolution (SISR); however these models set huge amount of computational complexity to achieve such performance. Hence, computationally efficient and low memory models are needed for SISR if such models are deployed on resources with low-computational devices (for instance, Mobile). We propose a novel approach referred to as Channel Split Convolutional Neural Network for Single Image Super-Resolution (CSISR). The proposed work aims to create a light-weight but efficient network to enhance SR performance. It employs a unique channel splitting alongside channel reduction blocks, leading to more effectiveness with a less computational burden to obtain state-of-the-art accuracy. Further, we suggest a new strategy for Channel Attention (CA) using a combination of global average and standard deviation pooling accompanied by conventional non-linear mapping layers to improve the learning. The performance of the proposed network is validated on various benchmark testing datasets for the SISR task, which shows its superiority over other existing methods.

This paper presents a summary of the Masked Face Recognition Competitions (MFR) held within the 2021 International Joint Conference on Biometrics (IJCB 2021). The competition attracted a total of 10 participating teams with valid submissions. The affiliations of these teams are diverse and associated with academia and industry in nine different countries. These teams successfully submitted 18 valid solutions. The competition is designed to motivate solutions aiming at enhancing the face recognition accuracy of masked faces. Moreover, the competition considered the deployability of the proposed solutions by taking the compactness of the face recognition models into account. A private dataset representing a collaborative, multisession, real masked, capture scenario is used to evaluate the submitted solutions. In comparison to one of the topperforming academic face recognition solutions, 10 out of the 18 submitted solutions did score higher masked face verification accuracy.

Liveness Detection (LivDet)-Face is an international competition series open to academia and industry. The competition’s objective is to assess and report state-of-the-art in liveness / Presentation Attack Detection (PAD) for face recognition. Impersonation and presentation of false samples to the sensors can be classified as presentation attacks and the ability for the sensors to detect such attempts is known as PAD. LivDet-Face 2021 * will be the first edition of the face liveness competition. This competition serves as an important benchmark in face presentation attack detection, offering (a) an independent assessment of the current state of the art in face PAD, and (b) a common evaluation protocol, availability of Presentation Attack Instruments (PAI) and live face image dataset through the Biometric Evaluation and Testing (BEAT) platform. The competition can be easily followed by researchers after it is closed, in a platform in which participants can compare their solutions against the LivDet-Face winners.

Face morphing attacks are evolving as a significant threat to the Face Recognition Systems (FRS) operating in border control and passport issuance. As newborn face has very limited discriminative facial characteristics, it is challenging for both human and machines to verify the newborns based on the facial biometrics accurately. Further, the introduction of face morphing elevates the problem of baby trafficking as it can challenge both human and machine-based facial verification. In this paper, we pose a question if the morphed images of newborns can threaten FRS and present first systematic study on the vulnerability analysis of FRS towards morphed faces of newborns. To effectively benchmark threat of newborns’ facial morphing attacks, we introduce a new face morphing dataset constructed based on 42 unique newborns with 852 bona fide and 2451 morphing images. Extensive experiments are carried out on the newly constructed dataset to benchmark the vulnerability against both Commercial-Off-The-Shelf (COTS) FRS (Cognitec FaceVACS-SDK Version 9.4.2) and deep learning based FRS (Arcface) for three different morphing factors. Further, we also evaluate the performance of Morphing Attack Detection (MAD) in detecting such morphing attacks of newborn faces. We conduct experiments on four different Off-The-Shelf MAD techniques to benchmark the detection performance on newborn morph attacks.

Morphing attacks have posed a severe threat to Face Recognition System (FRS). Despite the number of advancements reported in recent works, we note serious open issues such as independent benchmarking, generalizability challenges and considerations to age, gender, ethnicity that are inadequately addressed. Morphing Attack Detection (MAD) algorithms often are prone to generalization challenges as they are database dependent. The existing databases, mostly of semi-public nature, lack in diversity in terms of ethnicity, various morphing process and post-processing pipelines. Further, they do not reflect a realistic operational scenario for Automated Border Control (ABC) and do not provide a basis to test MAD on unseen data, in order to benchmark the robustness of algorithms. In this work, we present a new sequestered dataset for facilitating the advancements of MAD where the algorithms can be tested on unseen data in an effort to better generalize. The newly constructed dataset consists of facial images from 150 subjects from various ethnicities, age-groups and both genders. In order to challenge the existing MAD algorithms, the morphed images are with careful subject pre-selection created from the contributing images, and further post-processed to remove morphing artifacts. The images are also printed and scanned to remove all digital cues and to simulate a realistic challenge for MAD algorithms. Further, we present a new online evaluation platform to test algorithms on sequestered data. With the platform we can benchmark the morph detection performance and study the generalization ability. This work also presents a detailed analysis on various subsets of sequestered data and outlines open challenges for future directions in MAD research.

The deep learning models for the Single Image Super-Resolution (SISR) task have found success in recent years. However, one of the prime limitations of existing deep learning-based SISR approaches is that they need supervised training. Specifically, the Low-Resolution (LR) images are obtained through known degradation (for instance, bicubic downsampling) from the High-Resolution (HR) images to provide supervised data as an LR-HR pair. Such training results in a domain shift of learnt models when real-world data is provided with multiple degradation factors not present in the training set. To address this challenge, we propose an unsupervised approach for the SISR task using Generative Adversarial Network (GAN), which we refer to hereafter as DUS-GAN. The novel design of the proposed method accomplishes the SR task without degradation estimation of real-world LR data. In addition, a new human perception-based quality assessment loss, i.e., Mean Opinion Score (MOS), has also been introduced to boost the perceptual quality of SR results. The pertinence of the proposed method is validated with numerous experiments on different reference-based (i.e., NTIRE Real-world SR Challenge validation dataset) and no-reference based (i.e., NTIRE Real-world SR Challenge Track-1 and Track-2) testing datasets. The experimental analysis demonstrates committed improvement from the proposed method over the other state-of-the-art unsupervised SR approaches, both in terms of subjective and quantitative evaluations on different reference metrics (i.e., LPIPS, PI-RMSE graph) and no-reference quality measures such as NIQE, BRISQUE and PIQE. We also provide the implementation of the proposed approach (https://github.com/kalpeshjp89/DUSGAN) to support reproducible research.

Due to various applications, research on personal traits using information on social media has become an important area. In this paper, a new method for the classification of behavior-oriented social images uploaded on various social media platforms is presented. The proposed method introduces a multimodality concept using skin of different parts of human body and background information, such as indoor and outdoor environments. For each image, the proposed method detects skin candidate components based on R, G, B color spaces and entropy features. The iterative mutual nearest neighbor approach is proposed to detect accurate skin candidate components, which result in foreground components. Next, the proposed method detects the remaining part (other than skin components) as background components based on structure tensor of R, G, B color spaces, and Maximally Stable Extremal Regions (MSER ) concept in the wavelet domain. We then explore Hanman Transform for extracting context features from foreground and background components through clustering and fusion operation. These features are then fed to an SVM classifier for the classification of behavior-oriented images. Comprehensive experiments on 10-class datasets of Normal Behavior-Oriented Social media Image (NBSI) and Abnormal Behavior-Oriented Social media Image (ABSI) show that the proposed method is effective and outperforms the existing methods in terms of average classification rate. Also, the results on the benchmark dataset of five classes of personality traits and two classes of emotions of different facial expressions (FERPlus dataset) demonstrated the robustness of the proposed method over the existing methods.

The growing popularity of Virtual Reality and Augmented Reality (VR/AR) devices in many applications also demands authentication of users. As the devices inherently capture the eye image while capturing the user interaction, the authentication can be devised using the iris and periocular recognition. While both iris and periocular data being non-ideal unlike the data captured from standard biometric sensors, the authentication performance is expected to be lower. In this work, we present and evaluate a fusion framework for improving the biometric authentication performance. Specifically, we employ score-level fusion for two independent biometric systems of iris and periocular region to avoid expensive feature-level fusion. With a detailed evaluation of three different score-level fusion after the score normalization on a dataset of 12579 images, we report the performance gain in authentication using score-level fusion for iris and periocular recognition.

Face morphing attacks have demonstrated a severe threat in the passport issuance protocol that weakens the border control operations. A morphed face images if used after printing and scanning (re-digitizing) to obtain a passport is very challenging to be detected as attack. In this paper, we present a novel method to detect such morphing attacks using an ensemble of features computed on the scale-space representation derived from the color space for a given image. Given the limited availability of datasets representing realistic morphing attacks, we introduce and present a new print-scan image dataset of morphed face images. Experiments are carried out on the two different datasets and compared with sixteen existing state-of-art Morphing Attack Detection (MAD) mechanism based on single image MAD (S-MAD). The proposed approach indicates a superior MAD performance on both datasets suggesting the applicability in operational scenarios.

Augmented and virtual reality is being deployed in different fields of applications. Such applications might involve accessing or processing critical and sensitive information, which requires strict and continuous access control. Given that Head-Mounted Displays (HMD) developed for such applications commonly contains internal cameras for gaze tracking purposes, we evaluate the suitability of such setup for verifying the users through iris recognition. In this work, we first evaluate a set of iris recognition algorithms suitable for HMD devices by investigating three well-established handcrafted feature extraction approaches, and to complement it, we also present the analysis using four deep learning models. While taking into consideration the minimalistic hardware requirements of stand-alone HMD, we employ and adapt a recently developed miniature segmentation model (EyeMMS) for segmenting the iris. Further, to account for non-ideal and non-collaborative capture of iris, we define a new iris quality metric that we termed as Iris Mask Ratio (IMR) to quantify the iris recognition performance. Motivated by the performance of iris recognition, we also propose the continuous authentication of users in a non-collaborative capture setting in HMD. Through the experiments on a publicly available OpenEDS dataset, we show that performance with EER = 5% can be achieved using deep learning methods in a general setting, along with high accuracy for continuous user authentication.

Face morphing attacks have raised critical concerns as they demonstrate a new vulnerability of Face Recognition Systems (FRS), which are widely deployed in border control applications. The face morphing process uses the images from multiple data subjects and performs an image blending operation to generate a morphed image of high quality. The generated morphed image exhibits similar visual characteristics corresponding to the biometric characteristics of the data subjects that contributed to the composite image and thus making it difficult for both humans and FRS, to detect such attacks. In this paper, we report a systematic investigation on the vulnerability of the Commercial-Off- The-Shelf (COTS) FRS when morphed images under the influence of ageing are presented. To this extent, we have introduced a new morphed face dataset with ageing derived from the publicly available MORPH II face dataset, which we refer to as MorphAge dataset. The dataset has two bins based on age intervals, the first bin - MorphAge-I dataset has 1002 unique data subjects with the age variation of 1 year to 2 years while the MorphAge-II dataset consists of 516 data subjects whose age intervals are from 2 years to 5 years. To effectively evaluate the vulnerability for morphing attacks, we also introduce a new evaluation metric, namely the Fully Mated Morphed Presentation Match Rate (FMMPMR), to quantify the vulnerability effectively in a realistic scenario. Extensive experiments are carried out using two different COTS FRS (COTS I Cognitec FaceVACS-SDK Version 9.4.2 and COTS II - Neurotechnology version 10.0) to quantify the vulnerability with ageing. Further, we also evaluate five different Morph Attack Detection (MAD) techniques to benchmark their detection performance with respect to ageing.

Methods developed for normal 2D text detection do not work well for text that is rendered using decorative, 3D effects, etc. This paper proposes a new method for classification of 2D and 3D natural scene text images so that an appropriate recognition method can be chosen accordingly based on the classification results for better performance. The proposed method explores local gradient differences for obtaining candidate pixels, which represent a stroke. To study the spatial distribution of candidate pixels, we propose a measure, called COLD, which is denser for pixels toward the center of strokes and scattered for non-stroke pixels. This observation leads us to introduce mass features for extracting the regular spatial pattern of COLD, which indicates a 2D text image. The extracted features are fed into a Neural Network (NN) for classification. The proposed method is tested on (i) a new dataset introduced in this work (ii) a second dataset assembled from standard natural scene datasets (iii) Non-Text Image datasets which does not contain text, rather it contains objects. Experimental results of the proposed method on images with text and non-text show that the proposed method is independent of text. The proposed approach improves text detection and recognition performance significantly after classification.

Wrist-wearable devices such as smartwatch hardware have gained popularity as they provide quick access to various information and easy access to multiple applications. Among the numerous smartwatch applications, user verification based on the handwriting is gaining momentum by considering its reliability and user-friendliness. In this paper, we present a novel technique for user verification using a smartwatch based writing pattern or style. The proposed approach leverages accelerometer data captured from the smartwatch that is further represented using 2D Continuous Wavelet Transform (CWT) and deep features extracted using the pre-trained ResNet50. These features are classified using an ensemble of classifiers to make the final decision on user verification. Extensive experiments are carried out on a newly captured dataset using two different smartwatches with three different writing scenarios (or activities). Experimental results provide critical insights and analysis of the results in such a verification scenario.

Along with the deployment of the Face Recognition Systems (FRS), concerns were raised related to the vulnerability of those systems towards various attacks including morphed attacks. The morphed face attack involves two different face images in order to obtain via a morphing process a resulting attack image, which is sufficiently similar to both contributing data subjects. The obtained morphed image can successfully be verified against both subjects visually (by a human expert) and by a commercial FRS. The face morphing attack poses a severe security risk to the e-passport issuance process and to applications like border control, unless such attacks are detected and mitigated. In this work, we propose a new method to reliably detect a morphed face attack using a newly designed demising framework. To this end, we design and introduce a new deep Multi-scale Context Aggregation Network (MS-CAN) to obtain denoised images, which is subsequently used to determine if an image is morphed or not. Extensive experiments are carried out on three different morphed face image datasets. The Morphing Attack Detection (MAD) performance of the proposed method is also benchmarked against 14 different state-of-the-art techniques using the ISO-IEC 30107-3 evaluation metrics. Based on the obtained quantitative results, the proposed method has indicated the best performance on all three datasets and also on cross-dataset experiments.

The periocular region is used for authentication in the recent days under unconstrained acquisition in biometrics. This work presents two new feature extraction techniques to achieve robust and blur invariant biometric verification using periocular images captured using smartphones - (1) Deep Sparse Features (DSF) and (2) Deep Sparse Time Frequency Features (DeSTiFF). Both the approaches are based on extracting features via convolution of periocular images with a set of filters also referred as Deep Sparse Filters. The filters are learnt using natural image patches and sparse filtering approach. The DSF is obtained through convolution via Deep Sparse Filters. Further, convoluted responses are analyzed using Short Term Fourier Transform (STFT) to obtain time and frequency features of the images referred as DeSTIFF. The features obtained from the newly proposed feature extraction techniques are further represented in a collaborative subspace to achieve better verification performance. Both of the proposed feature extraction schemes are evaluated on two publicly available smartphone periocular databases and a new database (Visible Spectrum Periocular Image (VISPI) database) released with this article. The robustness of the proposed feature extraction is exemplified by comparing it with state-of-art approaches along with multiple deep networks where the improvement is evidently seen on large scale database with an average verification accuracy of Genuine Match Rate ≈ 98% at False Match Rate = 0.01%. We further support reproducible research by making the code and the database available for the academic research.

The primary challenge in tracing the participants in sports and marathon video or images is to detect and localize the jersey/Bib number that may present in different regions of their outfit captured in cluttered environment conditions. In this work, we proposed a new framework based on detecting the human body parts such that both Jersey Bib number and text is localized reliably. To achieve this, the proposed method first detects and localize the human in a given image using Single Shot Multibox Detector (SSD). In the next step, different human body parts namely, Torso, Left Thigh, Right Thigh, that generally contain a Bib number or text region is automatically extracted. These detected individual parts are processed individually to detect the Jersey Bib number/text using a deep CNN network based on the 2-channel architecture based on the novel adaptive weighting loss function. Finally, the detected text is cropped out and fed to a CNN-RNN based deep model abbreviated as CRNN for recognizing jersey/Bib/text. Extensive experiments are carried out on the four different datasets including both bench-marking dataset and a new dataset. The performance of the proposed method is compared with the state-of-the-art methods on all four datasets that indicates the improved performance of the proposed method on all four datasets.

Facilitation of fake face generation in recent years, thanks to advancements in computer graphics and artificial intelligence, raises concerns about malicious use of these techniques for personal or political gains. Media consumers are exposed to hours of audiovisual content daily, while their vulnerability to fake audiovisual content is not yet fully studied and understood. In contrast, many recent automated fake content generation techniques are readily accessible to the public. A first step to address this vulnerability is to study the effectiveness of existing methods in passing human judgment. To this end, we examined the performance of 30 participants in the detection of 48 real and fake videos. The fake videos were sourced from six different methods of generation and were collected from a public video sharing website1, ranging from prosthetic makeup to Deepfakes. Our results show that the participants failed to detect two different types of fake videos. However, participants' detection performance improves when they know of the displayed individual or when a biometric reference video (introducing the individual and its behavior) is available to them during the test.

Face recognition systems are widely known for their vulnerability against presentation attacks or spoofing attacks. The exponential deployment of face recognition systems has been further challenged even by the simple and low-cost face artefacts generated using conventional printers. In this paper, we present a novel scheme to detect face presentation attacks posed by high-quality print attacks which are relatively difficult to detect. The proposed scheme leverages the phase information extracted from the spatial-frequency representation of the given image. We also present a new face presentation attack database collected using the iPhone 6S. The new database is comprised of 100 subjects collected in two different sessions that have resulted in a total of 31228 samples (or images). Extensive experiments are carried out on the newly constructed database and the obtained results show the improved performance of the proposed scheme when compared aaainst six different state-of-the-art methods.

Multi-spectral imaging has been explored to handle a set of deficiencies found in traditional imaging that capture the images only in visible spectrum (VIS) or Near-Infra Red (NIR) spectrum. The promising performance obtained in the experimental works indicates the use-case in real-life biometric systems. As biometric systems should also consider protecting biometric templates, it is required to have an efficient template protection scheme for multi-spectral biometric systems to avoid the leakage of biometric data and subsequent linkability issues. In this work, we propose a new template protection scheme for multi-spectral biometric systems through the use of biometric information across different spectra to provide protected templates. Through the proposed approach of kernalized hashing, we can reach a fully unlinkable template protection scheme that works across all spectra in a multi-spectral system with a comparable performance to unprotected system. Further, we propose a template level fusion across all the spectral bands to improve the performance of the multi-spectral biometric system with integrated template protection. Through the use of a relatively large sized multispectral face biometric database of 168 subjects captured in 9 narrow spectral bands in VIS and NIR bands (530nm to 1000nm), we illustrate the effectiveness of the proposed approach in achieving a robust and secure template protection while accounting for irreversibility, unlinkability and renewability. Through the experiments we establish the performance of the proposed template protection approach and demonstrate a high Genuine Match Rate (GMR ≈ 100% at False Accept Rate of FMR=0.01%) and low Equal Error Rate EER= ≈ 0%, while satisfying other requirements of biometric template protection. Further, we present a security analysis of the proposed approach to demonstrate the unlinkability of the biometric templates.

Biometric systems are prone to random and systematic errors which are typically attributed to the variations in terms of inter-session data capture and intra-session variability. Furthermore, these errors cannot be defined and modeled mathematically in many cases, but we can associate them with uncertainty based on certain conditions. In such cases, one of the possible approach to improve biometric system performance is to employ multi-biometric fusion by incorporating the uncertainties. In the literature, researchers have proposed many fusion techniques, but most of these techniques do not take uncertainty into account while performing fusion. Since the decision made by uni-modal biometric comparators do not consider the uncertainty involved in such decisions, it is essential first to model the uncertainty before combining the decision from multiple uni-modal biometric systems efficiently. To this end, we propose a score level multi-biometric fusion scheme using Subjective Logic which incorporates the uncertainty of the system's information channels while fusing the scores. Extensive experiments are carried out on the multi-biometric NIST BSSR1, and the proposed scheme has indicated a superior performance with a genuine match rate of 99.02 % at a false match rate fixed to 0.01 %.

Presentation attacks against Face Recognition System (FRS) have incrementally posed challenges to create new detection methods. Among the various presentation attacks, disguise attacks allow concealing the identity of the attacker thereby increasing the vulnerability of the FRS. In this paper, we present a new approach for attack detection in multi-spectral systems, where face disguise attacks are carried out. The approach is based on using spectral signatures obtained from a spectral camera operating in eight narrow spectral bands across the Visible (VIS) and Near Infra-Red (NIR) (530nm to 1000nm) spectrum and learning deeply coupled auto-encoders. The robustness of the proposed approach is validated using a newly collected spectral face database of subjects conducting both bona fide (i.e. real) presentations and disguise attack presentations. The database is designed to capture 2 different kinds of attacks from 54 subjects, amounting to a total number of 6480 samples. Extensive experiments carried on the multi-spectral face database indicate the robust performance of proposed scheme when benchmarked with three different state-of-the-art methods.

Multi-spectral imaging has recently acquired significant attention in biometrics based authentication due to it's potential ability to capture spatio-spectral images across the electromagnetic spectrum. Especially, in the case of facial biometrics, multi-spectral imaging has shown significant promising results under unknown/varying illumination environment. However, the challenge arises when surveillance cameras provide the visible images while the enrollment are spectral band images. In order to address the backward/cross compatibility of probing visible images from regular surveillance cameras against the high quality spectral band images in enrollment, development of robust algorithms are required. In this paper, we present a new approach of selecting optimal band based on highest correlation coefficients of individual feature vectors from bands in comparison with feature vectors from visible images of respective individual classes for robust recognition performance. The proposed approach of band selection is validated on a newly collected face database of 168 subjects whose face images are collected in 9 different spectral bands and correspondingly their visible images from a regular camera operating in visible spectrum. The extensive set of experiments conducted on the new database with selected single band and multiple spectral bands in enrollment data versus the visible probe image has indicated the significance of the band selection. The new approach of spectral to visible matching with the proposed band selection method shows significant Rank- 1 recognition rate of 94.04% supporting the applicability of proposed method

Multispectral face recognition systems are widely used in various access control applications. The vulnerability of multispectral face recognition sensors towards low-cost Presentation Attack Instrument (PAI) such as printed photos used in attacks has emerged as a serious security threat. In this paper, we present a novel framework to detect presentation attacks against an extended multispectral face sensor. The proposed framework stems from the idea of exploring the complementary information available from different bands of an extended multispectral face sensor. To this extent, two different frameworks are proposed where the first framework is based on image fusion and the second builds on the Presentation Attack Detection (PAD) score level fusion. Extensive experiments are carried out on the extended multispectral face sensor database comprising of 50 subjects with two different presentation attacks generated using the printed photo artefacts. The obtained results indicate the superior performance of the PAD score level fusion on detecting both known and unknown attacks.

Periocular characteristics has gained substantial importance in recent times to supplement the performance of facial biometrics or as a stand-alone characteristics. While most of the current biometric systems for authentication or surveillance operate either in NIR spectrum or visible spectrum, the ocular information can be well utilized if a comparison of images from different spectra has to be conducted. In this work, we present a novel approach employing the features obtained from steerable pyramids to compare the ocular images captured from NIR versus the images captured from visible spectrum. The set of features obtained using the proposed cross-spectral approach are then used to learn a multi-class SVM classifier such that the probe image originating from another spectrum can be classified. Further, a fusion frame-work for combining the scores from different orientations of the steerable pyramid is proposed for a particular scale to strengthen the biometric performance of the algorithm. An extensive set of experiments conducted on a large database consisting of ocular images captured from 120 subjects (240 unique ocular instances) indicates the robustness of the proposed approach with a GMR of 100% at the FMR of 0.01% in a benchmark against other state-of-the-art techniques suggesting the applicability of proposed approach to greater extent.

With the availability of sensor technology across the broad electromagnetic spectrum, multi-spectral imaging is increasingly used in biometric systems. Especially for face recognition, multi-spectral imaging has gained a lot of attention due to it's invariant property against variation caused by unknown illumination. However, obtaining best performance using multi-spectral imaging is still a challenge due to presence of a modality gap between the spectral imaging data and redundant band information. In this paper, we propose a fused band representation with a set of selected bands represented in Quaternion space for spectral band images to efficiently maintain the inter band relationship in spatial domain. The selection is based on measuring the information content in bands using entropy and fusion is carried out in Quaternion space for three best bands. The features from newly obtained image is collaboratively represented to achieve robust performance. The proposed approach is experimentally validated on the extended multi-spectral face database of 168 subjects, whose spectral band images are captured in 9 narrow spectral bands in visible and near infrared range (530nm to 1000nm). The quantitative performance analysis, obtained using the proposed method indicates 96.13% recognition rate at Rank-1, outperforming other state-of-the-art methods.

The Face Recognition System (FRS) is known to be vulnerable to the attacks using the morphed face. As the use of face characteristics are mandatory in the electronic passport (ePass), morphing attacks have raised the potential concerns in the border security. In this paper, we analyze the vulnerability of the FRS to the new attack performed using the averaged face. The averaged face is generated by simple pixel level averaging of two face images corresponding to two different subjects. We benchmark the vulnerability of the commercial FRS to both conventional morphing and averaging based face attacks. We further propose a novel algorithm based on the collaborative representation of the micro-texture features that are extracted from the colour space to reliably detect both morphed and averaged face attacks on the FRS. Extensive experiments are carried out on the newly constructed morphed and averaged face image database with 163 subjects. The database is built by considering the real-life scenario of the passport issuance that typically accepts the printed passport photo from the applicant that is further scanned and stored in the ePass. Thus, the newly constructed database is built to have the print-scanned bonafide, morphed and averaged face samples. The obtained results have demonstrated the improved performance of the proposed scheme on print-scanned morphed and averaged face database.

Smartphone based facial biometric systems have been well used in many of the security applications starting from simple phone unlocking to secure banking applications. This work presents a new approach of exploring the intrinsic characteristics of the smartphone camera to capture a number of stack images in the depth-of-field. With the set of stack images obtained, we present a new feature-free and classifier-free approach to provide the presentation attack resistant face biometric system. With the entire system implemented on the smartphone, we demonstrate the applicability of the proposed scheme in obtaining a stack of images with varying focus to effectively determine the presentation attacks. We create a new database of 13250 images at different focal length to present a detailed analysis of vulnerability together with the evaluation of proposed scheme. An extensive evaluation of the newly created database comprising of 5 different Presentation Attack Instruments (PAI) has demonstrated an outstanding performance on all 5 PAI through proposed approach. With the set ofcomplementary benefits of proposed approach illustrated in this work, we deduce the robustness towards unseen 2D attacks.

This work presents the 2ndCross-Spectrum Iris/Periocular Recognition Competition (Cross-Eyed2017). The main goal of the competition is to promote and evaluate advances in cross-spectrum iris and periocular recognition. This second edition registered an increase in the participation numbers ranging from academia to industry: five teams submitted twelve methods for the periocular task and five for the iris task. The benchmark dataset is an enlarged version of the dual-spectrum database containing both iris and periocular images synchronously captured from a distance and within a realistic indoor environment. The evaluation was performed on an undisclosed test-set. Methodology, tested algorithms, and obtained results are reported in this paper identifying the remaining challenges in path forward.

Spectral face recognition is gaining importance as the information from different bands can lead to robust face representation that are presentation (a.k.a, spoofing) attack resistant. However, the key challenge here is to process the high dimensional spatio-spectral data to extract and represent the reliable data while discarding redundant data information for processing. In this work, we present a new approach to represent the spectral images in a holistic manner by employing the well-known Projection Metric Learning (PML) in Grassmann manifold such that the redundant information from the spectral data is discarded while retaining significant information. The approach is adopted to learn discriminative information from an high dimensional spectral dataset. Further, we propose an extension using collaborative representation of learnt projection metrics for improving the classification accuracy of spectral data. With the extensive set of experiments conducted on a relatively large scale extended-spectral face image database (6048 images) of 168 subjects, we demonstrate the applicability of the proposed framework. The obtained results indicates highest accuracy by achieving a Rank-1 recognition rate of 98.21% in classifying the spectral face images.

Gender prediction based on facial features has received significant attention in computer vision and biometric community. Most of the gender classification studies mainly focused there attention on approaches that operate in the visible spectrum. In this paper we present gender classification using extended multi-spectral face data captured in nine narrow spectral bands across the visible near infrared spectrum (530nm to 1000nm). Further, we present the proposed method, that learns for this image set the discriminative spectral band features in the affine space and then classifies the features with a Support Vector Machine (SVM) in a robust manner. The extensive experimental results are presented on the reasonable sample size of 78300 spectral band images using our proposed method. The obtained results shows 90.49±3.56% average classification accuracy, indicating the applicability of our proposed method for gender classification.

Contact lens detection in the eye is a significant task to improve the reliability of iris recognition systems. A contact lens overlays the iris region and prevents the iris sensor from capturing the normal iris region. In this paper, we present a novel scheme for detection to detecting a contact lens using Deep Convolutional Neural Network (CNN). The proposed CNN architecture ContlensNet is structured to have fifteen layers and configured for the three-class detection problem with the following classes: images with textured (or colored) contact lens, soft (or transparent) contact lens, and no contact lens. The proposed ContlensNet is trained using numerous iris image patches and the problem of overfitting the network is addressed by using the dropout regularization method. Extensive experiments are carried out on two publicly available large-scale databases, namely: IIIT-Delhi Contact lens iris database (IIITD) and Notre Dame cosmetic contact lens database 2013 (ND) that are comprised of contact lens iris samples captured using four different sensors. The obtained results have demonstrated the improved performance of the proposed scheme with an average performance improvement of more than 10% in Correct Classification Rate (CCR%) when compared with eight different state-of-the-art contact lens detection systems.

Smartphone based periocular recognition has received substantial attention from the biometric research community. In this work, we propose a new scheme for the smartphone based periocular recognition. The proposed scheme is based on the texture features extracted from the periocular images using Maximum Response (MR) filters. These texture features are then classified using a deep neural network based on deeply coupled autoencoders. Extensive experiments are carried out on the large-scale VISOB database with 550 subjects captured using three different smartphones. The obtained results demonstrate the average performance of the proposed scheme with GMR of over 92% at FMR = 10-3.

Ocular recognition on smartphone authentication applications are gaining popularity in academic research and in the commercial sector where operators are requesting reliable and robust biometric authentication. The wide acceptance of such ocular based authentication systems also depends on the verification performance on large scale testing with different data subject ethnic groups and platforms. In this work, we evaluate such a large database of ocular images collected using three different phones. Further, we benchmark the verification performance and propose a new framework to improve it. The proposed framework is based on collaboratively represented features from deep sparse filtering. We obtain a verification performance of Genuine Match Rate (GMR) of 97.56% at a False Match Rate (FMR) of 0.001% for periocular images obtained from a Samsung device. The overall performance of around 95% GMR at a FMR of 0.001% not only indicates the robust nature of the proposed framework, but also illustrates the efficacy in applying them for real-life authentication scenarios.

Drug abuse leads to high degree of the change in the facial structure of a person due to multiple factors which include loss of fat in face, change of facial structure due to changes in facial muscles and change of skin texture due to appearance of acnes. The combination of such effects lead to completely deformed face which presents a complex challenge to face based authentication systems. To study the impact of the drug-abuse on face recognition systems, in this work, we create a new face database collected before and after the drug-abuse. The newly collected database which is referred as Drug Abuse Database (DAD) consists of images obtained from 101 subjects. Further, the collected database is analysed using various state-of-art face recognition algorithms along with a widely used commercial-off-the-shelf (COTS) system. The obtained performance of GMR = 12.90% at FMR = 0.01% indicates the challenge presented by such face images and underlines the importance of newer algorithms to handle such challenge.

The challenges of presentation attacks (spoofing attacks) at sensor level is increasing for biometric systems due to the evolving method of artefact presentation. The sophisticated attacks now employ high quality printed artefacts and electronic screens to present the biometric samples which make it difficult to separate the real presentations and artefact presentations. In this work, we propose a new scheme to detect the artefacts in both NIR and visible spectrum biometric sensors for ocular biometric systems using a new set of feature descriptor. The scheme employs adaptive and quantized texture patters obtained from local microfeatures and global spatial features for different color channels in an image. Further, the texture descriptors are used to learn a spectrally regressed discriminant classifier to classify the normal ocular images against the artefact ocular images. The proposed scheme is used to perform extensive experiments on 5 publicly available ocular datasets including 2 datasets acquired in NIR domain and two datasets acquired using smartphones along with a dataset acquired using high quality camera. The experiments conducted on all the datasets have consistently indicated the robust performance against attacks by showing a classification error of 0%.

As biometric authentication methods become more and more common in daily life we have seen an increased interest and developments of more convenient and secure authentication methods for online services. With biometrics enabled smartphones the cost associated with deploying biometric systems is removed and the opportunity for use in new applications is opened. However, with biometrics there are several challenges in terms of security and privacy that must be addressed. In this work we look at two emerging authentication protocols, FIDO Universal Authentication Framework and Biometric Open Protocol Standard, and analyze their security and privacy features from a biometrics perspective.

A light field sensor can provide useful information in terms of multiple depth (or focus) images, holding additional information that is quite useful for biometric applications. In this paper, we examine the applicability of a light field camera for biometric applications by considering two prominently used biometric characteristics: 1) face and 2) iris. To this extent, we employed a Lytro light field camera to construct two new and relatively large scale databases, for both face and iris biometrics. We then explore the additional information available from different depth images, which are rendered by light field camera, in two different manners: 1) by selecting the best focus image from the set of depth images and 2) combining all the depth images using super-resolution schemes to exploit the supplementary information available within the set elements. Extensive evaluations are carried out on our newly constructed database, demonstrating the significance of using additional information rendered by a light field camera to improve the overall performance of the biometric system.

Automatic Border Control (ABC) systems employ biometrics for the identity verification of the travelers. Among the various biometric modalities, the face biometric is predominately used by considering its many advantages including contactless capture, non-intrusive and user-friendly interaction. Since the inflow of passengers at the airport is growing with time, the ABC systems are expected to work with high throughput. One way of addressing this is by performing the on-the-fly face recognition with ABC systems. However, due to various factors, including the change in illumination conditions, it is often challenging to capture a good quality face sample to facilitate a speedy biometric recognition with the ABC systems. One possible way to address this is by employing multiple cameras to capture different views of the face and then combine these views in the face recognition system. In this paper, we propose a framework for combining the information from multiple cameras to improve the face recognition accuracy. To this extent, we investigate five different fusion schemes to present an empirical study on a prototype version of MorphoWay™ ABC system. Extensive experiments are carried out on a database of 61 subjects that are recorded using a prototype version of MorphoWay™ ABC system in three different lighting conditions. The experimental results indicate the improvement in face recognition performance by combing information from multiple cameras in the prototype version of MorphoWay™ ABC system.

Palmprint is a widely used biometric trait deployed in various access-control applications due to its convenience in use, reliability, and low cost. In this paper, we propose a novel scheme for palmprint recognition using a sparse representation of features obtained from Bank of Binarized Statistical Image Features (B-BSIF). The palmprint image is characterized by a rich set of features including principal lines, ridges, and wrinkles. Thus, the use of an appropriate texture descriptor scheme is expected to capture this information accurately. To this extent, we explore the idea of B-BSIF that comprises of 56 different BSIF filters whose responses on the given palmprint image is processed independently and classified using sparse representation classifier (SRC). Extensive experiments are carried out on three different large-scale publicly available palmprint databases. We then present an extensive analysis by comparing the proposed scheme with seven different contemporary state-of-the-art schemes that reveals the efficacy of the proposed scheme for robust palmprint recognition.

Vulnerability of iris recognition systems remains a challenge due to diverse presentation attacks that fail to assure the reliability when adopting these systems in real-life scenarios. In this paper, we present an in-depth analysis of presentation attacks on iris recognition systems especially focusing on the photo print attacks and the electronic display (or screen) attack. To this extent, we introduce a new relatively large scale visible spectrum iris artefact database comprised of 3300 iris normal and artefact samples that are captured by simulating five different attacks on iris recognition system. We also propose a novel presentation attack detection (PAD) scheme based on multiscale binarized statistical image features and linear support vector machines. Extensive experiments are carried out on four different publicly available iris artefact databases that have revealed the outstanding performance of the proposed PAD scheme when benchmarked with various well-established state-of-the-art schemes.

The gaining popularity of the visible spectrum iris recognition has sparked the interest in adopting it for various access control applications. Along with the popularity of visible spectrum iris recognition comes the threat of identity spoofing, presentation, or direct attack. This paper presents a novel scheme for detecting video presentation attacks in visible spectrum iris recognition system by magnifying the phase information in the eye region of the subject. The proposed scheme employs modified Eulerian video magnification (EVM) to enhance the subtle phase information in eye region and novel decision module to classify it as artefact(spoof attack) or normal presentation. The proposed decision module is based on estimating the change of phase information obtained from EVM, specially tailored to detect presentation attacks on video-based iris recognition systems in visible spectrum. The proposed scheme is extensively evaluated on the newly constructed database consisting of 62 unique iris video acquired using two smartphones-iPhone 5S and Nokia Lumia 1020. We also construct the artefact database with 62 iris acquired by replaying normal presentation iris video on iPad with retina display. Extensive evaluation of proposed presentation attack detection (PAD) scheme on the newly constructed database has shown an outstanding performance of average classification error rate = 0% supporting the robustness of the proposed PAD scheme.

The vulnerability of face recognition systems isa growing concern that has drawn the interest from both academic and research communities. Despite the availability of a broad range of face presentation attack detection (PAD)(or countermeasure or antispoofing) schemes, there exists no superior PAD technique due to evolution of sophisticated presentation attacks (or spoof attacks). In this paper, we present a new perspective for face presentation attack detection by introducing light field camera (LFC). Since the use of a LFC can record the direction of each incoming ray in addition to the intensity, it exhibits an unique characteristic of rendering multiple depth(or focus) images in a single capture. Thus, we present a novel approach that involves exploring the variation of the focus between multiple depth (or focus) images rendered by the LFC that in turn can be used to reveal the presentation attacks. To this extent, we first collect a new face artefact database using LFC that comprises of 80 subjects. Face artefacts are generated by simulating two widely used attacks, such as photo print and electronic screen attack. Extensive experiments carried out on the light field face artefact database have revealed the outstanding performance of the proposed PAD scheme when benchmarked with various well established state-of-the-art schemes.

Presentation (or spoof) attacks on biometric system is a growing concern that received substantial attention from both academics and industry. In this paper, we present a novel way of addressing a Presentation Attack Detection (PAD) (or spoof detection) by exploiting the inherent characteristics of the Light Field Camera (LFC) for visible spectrum iris biometric system. The proposed PAD algorithm will capture the variation in the depth (or focus) between multiple depth images rendered by the LFC that in turn can be used to reveal the presentation attacks. To this extent, we introduce a new presentation attack database comprised of 52 subjects with 104 unique eye samples. The database is collected using LFC by simulating the attacks through visible spectrum iris biometric artefacts like printed photo and electronic display (using both Apple iPad (4thgeneration) and Samsung Galaxy Note 10.1 tablet). Extensive experiments carried out on this database reveal the efficacy of the proposed PAD algorithm with a lowest Average Classification Error Rate = 0.5% when confronted with diverse set of attacks on visible spectrum iris biometric system.

Multimodal biometric systems based on fingerprint and finger vein modality provide promising features useful for robust and reliable identity verification. In this paper, we present a robust imaging device that can capture both fingerprint and finger vein simultaneously. The presented low-cost sensor employs a single camera followed by both near infrared and visible light sources organized along with the physical structures to capture good quality finger vein and fingerprint samples. We further present a novel finger vein recognition algorithm that explores both the maximum curvature method and Spectral Minutiae Representation (SMR). Extensive experiments are carried out on our newly collected database that comprises of 1500 samples of fingerprint and finger vein from 150 unique fingers corresponding to 41 subjects. Our results demonstrate the efficacy of the proposed sensor with a lowest Equal Error Rate of 0.78%.

The face biometric systems are highly vulnerable for the presentation attack that can be carried out by presenting a photo or video or even a 3D mask. In this paper, we present a novel Presentation Attack Detection (PAD) algorithm that can accurately detect and mitigate the 3D mask attacks on a face recognition system. The proposed scheme extracts both local and global features from the captured face image. The local features employed in this work corresponds to the eye (periocular) and nose region that are expected to provide clue on the presence of the mask. In addition, we also capture the micro-texture variation as a global feature using Binarized Statistical Image Features (BSIF). We then train a linear Support Vector Machine (SVM) independently on these two features whose scores are fused using the weighted sum rule before making the decision about a real face or an artefact. Extensive experiments are carried out on the public 3D mask database 3DMAD that shows the superiority of the proposed scheme with an outstanding performance of HTER = 0.03%.

The face quality assessment from video must quantitatively measure the applicability of the face images that are typically captured over multiple frames with various degradations. In this work, we address the face quality assessment from the video captured using Automatic Border Control (ABC) system. To this extent, we employed MorphoWayTM ABC system as a data capture device to construct a new database by simulating real-life scenario. We then propose a new scheme for face quality estimation that can be viewed in three steps: (1) Pose estimation by detecting face parts (eyes and nose) to separate frontal from non-frontal faces. (2) We then consider the frontal face and evaluate its corresponding image quality by analyzing its texture components using Grey Level Co-occurrence Matrix (GLCM). (3) Finally, we quantify the quality of the given face image using likelihood values obtained using Gaussian Mixture Model (GMM). Extensive experiments are carried out on our new database that exhibits various quality degradations due to head pose variations, change in illumination, expression, motion blur, etc. The experimental results have indicated that the proposed face quality assessment algorithm can effectively classify the input image into relevant quality bins that in turn can be employed for the improved face verification.

This paper proposes a new scheme for robust palmprint verification using sparse representation of Binarized Statistical Image Features (BSIF). Since palmprint comprises of rich set of features including principal lines, ridges and wrinkles, the use of appropriate texture descriptor is expected to accurately capture these information. To this extent, we explore the BSIF texture descriptor which codes each pixel of the given palmprint image in terms of binary strings based on the filter response. The BSIF learns the filter basis from the natural images by exploring statistical independence. We then use the Sparse Representation Classifier (SRC) on these BSIF features to perform the subject verification. Extensive experiments are carried out on three different large scale publically available palmprint databases. We then present an extensive analysis by comparing the proposed scheme with five different contemporary state-of-the-art schemes that reveals the outstanding performance.

The advanced technologies and sensors in smartphones has led to showcase their potential as a biometric sensor. In this work, we present the feasibility study and challenges in the path forward for using smartphone as a biometric sensor for iris recognition in visible spectrum. Especially, with a limited shelf-life of smartphones, it is anticipated to have enrolment and verification using different camera. In this work, we propose an improvement to segmentation scheme for contactless iris acquisition by approximating the radius range. The proposed method has resulted in a segmentation accuracy of 81%. We also propose various protocols for real-life verification scenarios using smartphones for visible spectrum iris recognition. Finally, results from an extensive set of experiments are presented to validate the anticipated challenges in using smartphone based iris recognition. Being the first of its kind, this work provides the benchmarking results for the smartphone iris database. The best EER is obtained for iPhone in indoor scenario with an impressive EER of 0.48%.

In this paper, we present an empirical study on exploring the Light Field Camera (LFC) for identifying multiple faces present at different distance. Since LFC can render multiple focus images in single exposure, one can combine these multiple images to obtain single all-in-focus image. Thus the constructed all-in-focus image will have all regions in focus and hence allows one to capture more information about the subject present even at a far distance. At the same time one can also construct the super resolution image to further improve the face recognition at a distance. Thus, in this work, we explore both all-in-focus and super resolution schemes to evaluate the multiple face recognition at a distance using LFC. We carry out extensive experiments on light field face dataset and present both qualitative and quantitative results.