Deepfake technology employs sophisticated machine-learning techniques to create highly convincing video and audio recordings of individuals doing or saying things that they never actually did or said. These falsified media pieces have the potential to deceive and manipulate viewers, posing significant risks to their privacy, security, and trust in digital media. In this paper, we present a novel method DualStreamNet for reliable Audio-Video (AV) fake media detection which exploits the complementary information. The proposed DualStreamNet includes independent detector for video and audio modality. We introduced a novel video fake detection framework using a SlowFast encoder as the backbone, and a novel architecture based on a 3D CNN with skip connections. We also introduced novel features to reliably detect audio fakes using a Continuous Wavelet Transform (CWT) Filter Bank that was further processed using the ResNet50 architecture. Finally, the decisions from the video and audio detectors are combined using the logical OR rule to make the final decision. Extensive experiments were performed on two publicly available audio-video fake datasets: FakeAVCeleb and SWAN-DF. The obtained results indicate the improved detection accuracy of the proposed method compared to existing methods.

The automatic prediction of gender from the face has been studied extensively because of its potential relevance in numerous applications related to security. Although the problem of gender classification based on the face is substantial, it remains far from being solved under difficult environmental exposure, especially for different illuminations. In this work, we demonstrate the merits and demerits of classifying gender under cross-illumination variants. We present our approach by employing multi-spectral imaging in nine narrow-spectrum bands stemming from the visible to near-infrared range. The experimental evaluation results were obtained on 78300 sample face images of 145 subjects captured under six different illumination conditions. Further, we present quantitative and qualitative experimental evaluations to determine the average classification accuracy for setting the benchmark results. To demonstrate the goal of this work, we present the results based on three image fusion techniques independently processed using five feature extraction methods for cross-illumination scenarios. This work obtained the highest classification accuracy of $96.32 \%$ for cross-illumination conditions, demonstrating the reliability of employing an image fusion approach to combine complementary information from spectral bands in difficult environmental exposure.

The face morphing process will combine two or more facial images to generate a single morphed facial image demonstrating Face Recognition Systems (FRS) vulnerability. The attack potential of the morphing image directly depends on the perceptual image quality, and when generated with no visible artefacts, it can deceive both human observers and automatic FRS. The current softwares for face morphing generates a morphing image with ghosting artefacts, especially in the eye region, nose and mouth area, which may serve as a potential cue to detect morphing attacks. Hence in this work, we introduce a new dataset comprising 10710 facial images before and after manual post-processing to reduce the visual artefacts and to generate high-quality attacks. Further, we propose a novel single image-based Morph Attack Detection (S-MAD) technique based on the ensemble of features and classifiers using the scale-space domain. The novel concept in the proposed method is the multilevel fusion that combines the comparison scores from different features and classifiers. Extensive experiments are carried out on the newly generated high-quality face images with (i) Morphs before post-processing and (ii) Morphs after post-processing. Further, the experiments are also carried out on two different mediums such as (i) Digital and (ii) Print-scan (or re-digitized) with and without compression. Extensive experimental results are performed to benchmark the detection performance with the existing S-MAD techniques. Obtained results indicate the best performance of the proposed method over existing methods.

The growing popularity of Virtual Reality and Augmented Reality (VR/AR) devices in many applications also demands authentication of users. As the devices inherently capture the eye image while capturing the user interaction, the authentication can be devised using the iris and periocular recognition. While both iris and periocular data being non-ideal unlike the data captured from standard biometric sensors, the authentication performance is expected to be lower. In this work, we present and evaluate a fusion framework for improving the biometric authentication performance. Specifically, we employ score-level fusion for two independent biometric systems of iris and periocular region to avoid expensive feature-level fusion. With a detailed evaluation of three different score-level fusion after the score normalization on a dataset of 12579 images, we report the performance gain in authentication using score-level fusion for iris and periocular recognition.

Face morphing attacks have demonstrated a severe threat in the passport issuance protocol that weakens the border control operations. A morphed face images if used after printing and scanning (re-digitizing) to obtain a passport is very challenging to be detected as attack. In this paper, we present a novel method to detect such morphing attacks using an ensemble of features computed on the scale-space representation derived from the color space for a given image. Given the limited availability of datasets representing realistic morphing attacks, we introduce and present a new print-scan image dataset of morphed face images. Experiments are carried out on the two different datasets and compared with sixteen existing state-of-art Morphing Attack Detection (MAD) mechanism based on single image MAD (S-MAD). The proposed approach indicates a superior MAD performance on both datasets suggesting the applicability in operational scenarios.

The primary challenge in tracing the participants in sports and marathon video or images is to detect and localize the jersey/Bib number that may present in different regions of their outfit captured in cluttered environment conditions. In this work, we proposed a new framework based on detecting the human body parts such that both Jersey Bib number and text is localized reliably. To achieve this, the proposed method first detects and localize the human in a given image using Single Shot Multibox Detector (SSD). In the next step, different human body parts namely, Torso, Left Thigh, Right Thigh, that generally contain a Bib number or text region is automatically extracted. These detected individual parts are processed individually to detect the Jersey Bib number/text using a deep CNN network based on the 2-channel architecture based on the novel adaptive weighting loss function. Finally, the detected text is cropped out and fed to a CNN-RNN based deep model abbreviated as CRNN for recognizing jersey/Bib/text. Extensive experiments are carried out on the four different datasets including both bench-marking dataset and a new dataset. The performance of the proposed method is compared with the state-of-the-art methods on all four datasets that indicates the improved performance of the proposed method on all four datasets.

Face recognition systems are widely known for their vulnerability against presentation attacks or spoofing attacks. The exponential deployment of face recognition systems has been further challenged even by the simple and low-cost face artefacts generated using conventional printers. In this paper, we present a novel scheme to detect face presentation attacks posed by high-quality print attacks which are relatively difficult to detect. The proposed scheme leverages the phase information extracted from the spatial-frequency representation of the given image. We also present a new face presentation attack database collected using the iPhone 6S. The new database is comprised of 100 subjects collected in two different sessions that have resulted in a total of 31228 samples (or images). Extensive experiments are carried out on the newly constructed database and the obtained results show the improved performance of the proposed scheme when compared aaainst six different state-of-the-art methods.

Multi-spectral imaging has been explored to handle a set of deficiencies found in traditional imaging that capture the images only in visible spectrum (VIS) or Near-Infra Red (NIR) spectrum. The promising performance obtained in the experimental works indicates the use-case in real-life biometric systems. As biometric systems should also consider protecting biometric templates, it is required to have an efficient template protection scheme for multi-spectral biometric systems to avoid the leakage of biometric data and subsequent linkability issues. In this work, we propose a new template protection scheme for multi-spectral biometric systems through the use of biometric information across different spectra to provide protected templates. Through the proposed approach of kernalized hashing, we can reach a fully unlinkable template protection scheme that works across all spectra in a multi-spectral system with a comparable performance to unprotected system. Further, we propose a template level fusion across all the spectral bands to improve the performance of the multi-spectral biometric system with integrated template protection. Through the use of a relatively large sized multispectral face biometric database of 168 subjects captured in 9 narrow spectral bands in VIS and NIR bands (530nm to 1000nm), we illustrate the effectiveness of the proposed approach in achieving a robust and secure template protection while accounting for irreversibility, unlinkability and renewability. Through the experiments we establish the performance of the proposed template protection approach and demonstrate a high Genuine Match Rate (GMR ≈ 100% at False Accept Rate of FMR=0.01%) and low Equal Error Rate EER= ≈ 0%, while satisfying other requirements of biometric template protection. Further, we present a security analysis of the proposed approach to demonstrate the unlinkability of the biometric templates.

Biometric systems are prone to random and systematic errors which are typically attributed to the variations in terms of inter-session data capture and intra-session variability. Furthermore, these errors cannot be defined and modeled mathematically in many cases, but we can associate them with uncertainty based on certain conditions. In such cases, one of the possible approach to improve biometric system performance is to employ multi-biometric fusion by incorporating the uncertainties. In the literature, researchers have proposed many fusion techniques, but most of these techniques do not take uncertainty into account while performing fusion. Since the decision made by uni-modal biometric comparators do not consider the uncertainty involved in such decisions, it is essential first to model the uncertainty before combining the decision from multiple uni-modal biometric systems efficiently. To this end, we propose a score level multi-biometric fusion scheme using Subjective Logic which incorporates the uncertainty of the system's information channels while fusing the scores. Extensive experiments are carried out on the multi-biometric NIST BSSR1, and the proposed scheme has indicated a superior performance with a genuine match rate of 99.02 % at a false match rate fixed to 0.01 %.

Multispectral face recognition systems are widely used in various access control applications. The vulnerability of multispectral face recognition sensors towards low-cost Presentation Attack Instrument (PAI) such as printed photos used in attacks has emerged as a serious security threat. In this paper, we present a novel framework to detect presentation attacks against an extended multispectral face sensor. The proposed framework stems from the idea of exploring the complementary information available from different bands of an extended multispectral face sensor. To this extent, two different frameworks are proposed where the first framework is based on image fusion and the second builds on the Presentation Attack Detection (PAD) score level fusion. Extensive experiments are carried out on the extended multispectral face sensor database comprising of 50 subjects with two different presentation attacks generated using the printed photo artefacts. The obtained results indicate the superior performance of the PAD score level fusion on detecting both known and unknown attacks.

Periocular characteristics has gained substantial importance in recent times to supplement the performance of facial biometrics or as a stand-alone characteristics. While most of the current biometric systems for authentication or surveillance operate either in NIR spectrum or visible spectrum, the ocular information can be well utilized if a comparison of images from different spectra has to be conducted. In this work, we present a novel approach employing the features obtained from steerable pyramids to compare the ocular images captured from NIR versus the images captured from visible spectrum. The set of features obtained using the proposed cross-spectral approach are then used to learn a multi-class SVM classifier such that the probe image originating from another spectrum can be classified. Further, a fusion frame-work for combining the scores from different orientations of the steerable pyramid is proposed for a particular scale to strengthen the biometric performance of the algorithm. An extensive set of experiments conducted on a large database consisting of ocular images captured from 120 subjects (240 unique ocular instances) indicates the robustness of the proposed approach with a GMR of 100% at the FMR of 0.01% in a benchmark against other state-of-the-art techniques suggesting the applicability of proposed approach to greater extent.

With the availability of sensor technology across the broad electromagnetic spectrum, multi-spectral imaging is increasingly used in biometric systems. Especially for face recognition, multi-spectral imaging has gained a lot of attention due to it's invariant property against variation caused by unknown illumination. However, obtaining best performance using multi-spectral imaging is still a challenge due to presence of a modality gap between the spectral imaging data and redundant band information. In this paper, we propose a fused band representation with a set of selected bands represented in Quaternion space for spectral band images to efficiently maintain the inter band relationship in spatial domain. The selection is based on measuring the information content in bands using entropy and fusion is carried out in Quaternion space for three best bands. The features from newly obtained image is collaboratively represented to achieve robust performance. The proposed approach is experimentally validated on the extended multi-spectral face database of 168 subjects, whose spectral band images are captured in 9 narrow spectral bands in visible and near infrared range (530nm to 1000nm). The quantitative performance analysis, obtained using the proposed method indicates 96.13% recognition rate at Rank-1, outperforming other state-of-the-art methods.