We present the Membership Inference Test (MINT) Demo 2, a framework designed to improve transparency in machine learning training processes. MINT is a technique for experimentally determining whether specific data were used during machine learning model training. We establish the theoretical framework and propose multiple architectures for MINT depending on the amount of information known about the models that are being audited. Experimental results using a popular face recognition model, 4 state-of-the-art LLMs, and multiple, diverse, and large-scale public image and text databases achieve promising accuracy levels in the detection of training data of up to 90%. Building on these results, we introduce a comprehensive web platform1 that expands these capabilities to image and text modalities. The platform integrates a diverse technological stack, including MINT, aMINT, and gMINT, allowing users to audit a wide range of models. This demonstrator aims to promote AI transparency and provides a practical tool to foster compliance with emerging AI regulations.

Fatigue, sleep, or disturbances in daily activities are common symptoms among patients with neurodegenerative disorders (NDD) and immune-mediated inflammatory diseases (IMID). The current assessment of such symptoms is usually conducted using patient reported outcomes (PROs) based on standardized questionnaires that patients usually complete every few months. This assessment protocol has raised some concerns, due to its propensity to exhibit biases derived from its subjectivity nature, or the low sensibility to changes, which may lead to a failure when trying to capture variability over time. In this work, we explore the use of smartphone data, which can serve as a proxy for how patients interact with their devices, to provide an effective, reliable, and objective assessment of the symptoms mentioned above. Our study comprises data from 137 participants belonging to 6 different disease groups, plus a healthy control group. We conducted statistical analysis based on repeated measures correlation, in which we analyze the correlation between screen-time and app-usage features with scores obtained from the PROs collected from the participants using a smartphone application.

We present LoRA-MINT, a new methodology for Membership Inference Test (MINT) applied to recent Large Language Models (LLMs) fine-tuned for specific Natural Language Processing (NLP) tasks through Low-Rank Adaptation (LoRA). The primary goal is to assess whether individual samples were part of the training data of these adapted models, providing a useful auditing tool for the management of intellectual property and sensitive data. Our analysis explores the relationship between model perplexity and membership status, providing a systematic framework for estimating data exposure in fine-tuned LLMs. We conducted experiments on four models and three benchmark datasets, obtaining precision values in determining if given data were used for training ranging from 0.77 to 0.92, which outperform state-of-the-art baselines and demonstrate the robustness and generality of the proposed method. In general, our findings underscore the potential of LoRA-MINT as an effective and scalable framework for auditing LLMs, improving transparency, and fostering the ethical and responsible deployment of AI and NLP technologies. For the sake of concreteness and current relevance, our discussion and experiments are centered on LoRAadjusted LLMs, but note that most of the presented methodology is easily applicable for auditing training data given any other technique for adapting LLMs or, more generally, any other domain-adapted AI models.

• A new metric is proposed to compare statistical distributions, with a focus on detecting disparities in the distribution tails. • The method separately analyzes genuine and impostor score distributions to provide a view of bias in high-performance biometric systems. • The proposal was tested extensively on real-world, and large-scale operational scenarios. Demographic bias in high-performance face recognition (FR) systems often eludes detection by existing metrics, especially with respect to subtle disparities in the tails of the score distribution. We introduce the Comprehensive Equity Index (CEI), a novel metric designed to address this limitation. CEI uniquely analyzes genuine and impostor score distributions separately, enabling a configurable focus on tail probabilities while also considering overall distribution shapes. Our extensive experiments (evaluating state-of-the-art FR systems, intentionally biased models, and diverse datasets) confirm CEI’s superior ability to detect nuanced biases where previous methods fall short. Furthermore, we present CEI A , an automated version of the metric that enhances objectivity and simplifies practical application. CEI provides a robust and sensitive tool for operational FR fairness assessment. The proposed methods have been developed particularly for bias evaluation in face biometrics but, in general, they are applicable for comparing statistical distributions in any problem where one is interested in analyzing the distribution tails.

Human pose estimation has witnessed significant advancements in recent years, mainly due to the integration of deep learning models, the availability of a vast amount of data, and large computational resources. These developments have led to highly accurate body tracking systems, which have direct applications in sports analysis and performance evaluation. This work analyzes the performance of six trackers: two point trackers and four joint trackers for biomechanical analysis in sprints. The proposed framework compares the results obtained from these pose trackers with the manual annotations of biomechanical experts for more than 5870 frames. The experimental framework employs forty sprints from five professional runners, focusing on three key angles in sprint biomechanics: trunk inclination, hip flex extension, and knee flex extension. We propose a post-processing module for outlier detection and fusion prediction in the joint angles. The experimental results demonstrate that using joint-based models yields root mean squared errors ranging from 11.41° to 4.37°. When integrated with the post-processing modules, these errors can be reduced to 6.99° and 3.88°, respectively. The experimental findings suggest that human pose tracking approaches can be valuable resources for the biomechanical analysis of running. However, there is still room for improvement in applications where high accuracy is required.

Verifying the authenticity of identity documents (IDs) has become a critical challenge for real-life applications such as digital banking, crypto-exchanges, renting, etc. This study focuses on the topic of fake ID detection, covering several limitations in the field. In particular, there are no publicly available data from real IDs for proper research in this area, and most published studies rely on proprietary internal databases that are not available for privacy reasons. In order to advance this critical challenge of real data scarcity that makes it so difficult to advance the technology of machine learning-based fake ID detection, we introduce a new patch-based methodology that trades off privacy and performance, and propose a novel patch-wise approach for privacy-aware fake ID detection: FakeIDet. In our experiments, we explore: i) two levels of anonymization for an ID (i.e., fully- and pseudo-anonymized), and ii) different patch size configurations, varying the amount of sensitive data visible in the patch image. State-of-the-art methods, such as vision transformers and foundation models, are considered as backbones. Our results show that, on an unseen database (DLC-2021), our proposal for fake ID detection achieves 13.91% and 0% EERs at the patch and the whole ID level, showing a good generalization to other databases. In addition to the path-based methodology introduced and the new FakeIDet method based on it, another key contribution of our article is the release of the first publicly available database that contains 48,400 patches from real and fake IDs, called FakeIDet-db, together with the experimental framework1.

Person re-identification (ReID) across aerial and ground vantage points has become crucial for large-scale surveillance and public safety applications. Although significant progress has been made in ground-only scenarios, bridging the aerial-ground domain gap remains a formidable challenge due to extreme viewpoint differences, scale variations, and occlusions. Building upon the achievements of the AG-ReID 2023 Challenge, this paper introduces the AG-VPReID 2025 Challenge—the first large-scale video-based competition focused on high-altitude (80–120 m) aerial-ground person ReID. Constructed on the new AG-VPReID dataset with 3,027 identities, over 13,500 tracklets, and approximately 3.7 million frames captured from UAVs, CCTV, and wearable cameras, the challenge featured four international teams. These teams developed solutions ranging from multi-stream architectures to transformer-based temporal reasoning and physics-informed modeling. The leading approach, X-TFCLIP from UAM, attained 72.28% Rank-1 accuracy in the aerial-to-ground ReID setting and 70.77% in the ground-to-aerial ReID setting, surpassing existing baselines while highlighting the dataset’s complexity. For additional details, please refer to the official website at https://agvpreid25.github.io.

Photorealistic talking-head avatars are becoming increasingly common in virtual meetings, gaming, and social platforms. These avatars allow for more immersive communication, but they also introduce serious security risks. One emerging threat is impersonation: an attacker can steal a user’s avatar, preserving his appearance and voice,making it nearly impossible to detect its fraudulent usage by sight or sound alone. In this paper, we explore the challenge of biometric verification in such avatar-mediated scenarios. Our main question is whether an individual’s facial motion patterns can serve as reliable behavioral biometrics to verify their identity when the avatar’s visual appearance is a facsimile of its owner. To answer this question, we introduce a new dataset of realistic avatar videos created using a stateof-the-art one-shot avatar generation model, GAGAvatar, with genuine and impostor avatar videos. We also propose a lightweight, explainable spatio-temporal Graph Convolutional Network architecture with temporal attention pooling, that uses only facial landmarks to model dynamic facial gestures. Experimental results demonstrate that facial motion cues enable meaningful identity verification with AUC values approaching 80%. The proposed benchmark and biometric system are available<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> for the research community in order to bring attention to the urgent need for more advanced behavioral biometric defenses in avatar-based communication systems.

Behavioral biometrics based on smartphone motion sensors are growing in popularity for authentication purposes. In this study, AirSignatureDB is presented: a new publicly accessible dataset of in-air signatures collected from 108 participants under real-world conditions, using 83 different smartphone models across four sessions. This dataset includes genuine samples and skilled forgeries, enabling a comprehensive evaluation of system robustness against realistic attack scenarios. Traditional and deep learningbased methods for in-air signature verification are benchmarked, while analyzing the influence of sensor modality and enrollment strategies. Beyond verification, a first approach to reconstructing the three-dimensional trajectory of in-air signatures from inertial sensor data alone is introduced. Using on-line handwritten signatures as a reference, we demonstrate that the recovery of accurate trajectories is feasible, challenging the long-held assumption that in-air gestures are inherently traceless. Although this approach enables forensic traceability, it also raises critical questions about the privacy boundaries of behavioral biometrics. Our findings underscore the need for a reevaluation of the privacy assumptions surrounding inertial sensor data, as they can reveal user-specific information that had not previously been considered in the design of in-air signature systems.

Active Membership Inference Test (aMINT) is a method designed to detect whether given data were used during the training of machine learning models. In Active MINT, we propose a novel multitask learning process that involves training simultaneously two models: the original or Audited Model, and a secondary model, referred to as the MINT Model, responsible for identifying the data used for training the Audited Model. This novel multi-task learning approach has been designed to incorporate the auditability of the model as an optimization objective during the training process of neural networks. The proposed approach incorporates intermediate activation maps as inputs to the MINT layers, which are trained to enhance the detection of training data. We present results using a wide range of neural networks, from lighter architectures such as MobileNet to more complex ones such as Vision Transformers, evaluated in 5 public benchmarks. Our proposed Active MINT achieves over 80% accuracy in detecting if given data was used for training, significantly outperforming previous approaches in the literature. Our aMINT and related methodological developments contribute to increasing transparency in AI models, facilitating stronger safeguards in AI deployments to achieve proper security, privacy, and copyright protection.

Abstract Maintaining a healthy lifestyle has become increasingly challenging in today’s sedentary society marked by poor eating habits. To address this issue, both national and international organisations have made numerous efforts to promote healthier diets and increased physical activity. However, implementing these recommendations in daily life can be difficult, as they are often generic and not tailored to individuals. This study presents the AI4Food-NutritionDB database, the first nutrition database that incorporates food images and a nutrition taxonomy based on recommendations by national and international health authorities. The database offers a multi-level categorisation, comprising 6 nutritional levels, 19 main categories (e.g., “Meat”), 73 subcategories (e.g., “White Meat”), and 893 specific food products (e.g., “Chicken”). The AI4Food-NutritionDB opens the doors to new food computing approaches in terms of food intake frequency, quality, and categorisation. Also, we present a standardised experimental protocol and benchmark including three tasks based on the nutrition taxonomy (i.e., category, subcategory, and final product recognition). These resources are available to the research community, including our deep learning models trained on AI4Food-NutritionDB, which can serve as pre-trained models, achieving accurate recognition results for challenging food image databases. All these resources are available in GitHub ( https://github.com/BiDAlab/AI4Food-NutritionDB ).

This article presents the Keystroke Verification Challenge - onGoing (KVC-onGoing) 1 1 https://sites.google.com/view/bida-kvc/ . , on which researchers can easily benchmark their systems in a common platform using large-scale public databases, the Aalto University Keystroke databases, and a standard experimental protocol. The keystroke data consist of tweet-long sequences of variable transcript text from over 185,000 subjects, acquired through desktop and mobile keyboards simulating real-life conditions. The results on the evaluation set of KVC-onGoing have proved the high discriminative power of keystroke dynamics, reaching values as low as 3.33% of Equal Error Rate (EER) and 11.96% of False Non-Match Rate (FNMR) @1% False Match Rate (FMR) in the desktop scenario, and 3.61% of EER and 17.44% of FNMR @1% at FMR in the mobile scenario, significantly improving previous state-of-the-art results. Concerning demographic fairness, the analyzed scores reflect the subjects’ age and gender to various extents, not negligible in a few cases. The framework runs on CodaLab 2 2 https://codalab.lisn.upsaclay.fr/competitions/14063 . . • We set up a novel framework for developing and evaluating keystroke biometrics. • We designed a unified experimental protocol with desktop and mobile scenarios. • We employ the biggest databases of keystroke dynamics, with over 185,000 subjects. • We provide a competitive performance baseline based on a limited-time challenge. • We provide a first exploration of the biometric fairness of keystroke dynamics.

Large-scale face recognition datasets are collected by crawling the Internet and without individuals' consent, raising legal, ethical, and privacy concerns. With the recent advances in generative models, recently several works proposed generating synthetic face recognition datasets to mitigate concerns in web-crawled face recognition datasets. This paper presents the summary of the Synthetic Data for Face Recognition (SDFR) Competition held in conjunction with the 18th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2024) and established to investigate the use of synthetic data for training face recognition models. The SDFR competition was split into two tasks, allowing participants to train face recognition systems using new synthetic datasets and/or existing ones. In the first task, the face recognition backbone was fixed and the dataset size was limited, while the second task provided almost complete freedom on the model backbone, the dataset, and the training pipeline. The submitted models were trained on existing and also new synthetic datasets and used clever methods to improve training with synthetic data. The submissions were evaluated and ranked on a diverse set of seven benchmarking datasets. The paper gives an overview of the submitted face recognition models and reports achieved performance compared to baseline models trained on real and synthetic datasets. Furthermore, the evaluation of submissions is extended to bias assessment across different demography groups. Lastly, an outlook on the current state of the research in training face recognition models using synthetic data is presented, and existing problems as well as potential future directions are also discussed.

The growing number of mobile devices over the past few years brings a large amount of personal information, which needs to be properly protected. As a result, several mobile authentication methods have been developed. In particular, behavioural biometrics has become one of the most relevant methods due to its ability to extract the uniqueness of each subject in a secure, non-intrusive, and continuous way. This article presents SwipeFormer, a novel Transformer-based system for mobile subject authentication by means of swipe gestures in an unconstrained scenario (i.e., subjects could use their personal devices freely, without restrictions on the direction of swipe gestures or the position of the device). Our proposed system contains two modules: (i) a Transformer-based feature extractor, and (ii) a similarity computation module. Mobile data from the touchscreen and different background sensors (accelerometer and gyroscope) have been studied, including in the analysis both Android and iOS operating systems. A complete analysis of SwipeFormer is carried out using an in-house large-scale database acquired in unconstrained scenarios. In these operational conditions, SwipeFormer achieves Equal Error Rate (EER) values of 6.6% and 3.6% on Android and iOS respectively, outperforming the state of the art. In addition, we evaluate SwipeFormer on the popular publicly available databases Frank DB and HuMIdb, achieving EER values of 11.0% and 5.0% respectively, outperforming previous approaches under the same experimental setup.

Abstract This article proposes a novel children–computer interaction (CCI) approach for the task of age group detection. This approach focuses on the automatic analysis of the time series generated from the interaction of the children with mobile devices. In particular, we extract a set of 25 time series related to spatial, pressure, and kinematic information of the children interaction while colouring a tree through a pen stylus tablet, a specific test from the large-scale public ChildCIdb database. A complete analysis of the proposed approach is carried out using different time series selection techniques to choose the most discriminative ones for the age group detection task: (i) a statistical analysis and (ii) an automatic algorithm called sequential forward search (SFS). In addition, different classification algorithms such as dynamic time warping barycenter averaging (DBA) and hidden Markov models (HMM) are studied. Accuracy results over 85% are achieved, outperforming previous approaches in the literature and in more challenging age group conditions. Finally, the approach presented in this study can benefit many children-related applications, for example, towards an age-appropriate environment with the technology.

Abstract International Organizations urge the protection of our oceans and their ecosystems due to their immeasurable importance to humankind. Since illegal fishing activities, commonly known as IUU fishing, cause irreparable damage to these ecosystems, concerned organisms are pushing to detect and combat IUU fishing practices. The automatic identification system allows to locate the position and trajectory of fishing vessels. In this study we address the task of detecting vessels’ fishing gears based on the trajectory behavior defined by GPS position data, a useful task to prevent the proliferation of IUU fishing practices. We present a new database including trajectories that span 7 different fishing gears and analyze these as in a time sequence analysis problem. We leverage from feature extraction techniques from the online signature verification domain to model vessel trajectories, and extract relevant information in the form of both local and global feature sets. We show how, based on these sets of features, the kinematics of vessels according to different fishing gears can be effectively classified using common supervised learning algorithms with accuracies up to $$90\%$$ 90 % . Furthermore, motivated by the concerns raised by several organizations on the adverse impact of bottom trawling on marine biodiversity, we present a binary classification experiment in which we were able to distinguish this kind of fishing gear with an accuracy of $$99\%$$ 99 % . We also illustrate in an ablation study the relevance of factors such as data availability and the sampling period to perform fishing gear classification. Compared to existing works, we highlight these factors, especially the importance of using sampling periods in the order of minutes instead of hours.

Face inpainting, the technique of restoring missing or damaged regions in facial images, is pivotal for applications like face recognition in occluded scenarios and image analysis with poor-quality captures. This process not only needs to produce realistic visuals but also preserve individual identity characteristics. The aim of this paper is to inpaint a face given periocular region (eyes-to-face) through a proposed new Generative Adversarial Network (GAN)-based model called Eyes-to-Face Network (E2F-Net). The proposed approach extracts identity and non-identity features from the periocular region using two dedicated encoders have been used. The extracted features are then mapped to the latent space of a pre-trained StyleGAN generator to benefit from its state-of-the-art performance and its rich, diverse and expressive latent space without any additional training. We further improve the StyleGAN's output to find the optimal code in the latent space using a new optimization for GAN inversion technique. Our E2F-Net requires a minimum training process reducing the computational complexity as a secondary benefit. Through extensive experiments, we show that our method successfully reconstructs the whole face with high quality, surpassing current techniques, despite significantly less training and supervision efforts. We have generated seven eyes-to-face datasets based on well-known public face datasets for training and verifying our proposed methods. The code and datasets are publicly available1.

This work introduces a new multispectral database and framework to train and evaluate eyeblink detection in RGB and Near-Infrared (NIR). Our contributed dataset (mEBAL2, multimodal EyeBlink and Attention Level estimation, Version 2) is the largest existing eyeblink database, representing a great opportunity to improve data-driven multispectral approaches for blink detection and related applications (e.g., attention level estimation). mEBAL2 includes 21,100 image sequences from 180 different students (more than 2 million labeled images in total) while conducting a number of e-learning tasks of varying difficulty or taking a real course on HTML initiation through the edX MOOC platform. mEBAL2 uses multiple sensors, including two Near-Infrared (NIR) and one RGB camera to capture facial gestures during the execution of the tasks, as well as an Electroencephalogram (EEG) band to get the cognitive activity of the user and blinking events. Furthermore, this work proposes 3 data-driven approaches as benchmarks for blink detection on mEBAL2, where the architecture based on Convolutional Long Short-Term Memory (ConvLSTM) achieved performances of up to 99%. The experiments explored whether combining RGB and NIR spectrum data improves blink detection in training and architectures that merge both types of data. Experiments showed that the NIR spectrum enhances results, even when only RGB images are available during inference. Finally, the generalization capacity of the proposed eyeblink detectors, along with state-of-the-art eyeblink detection implementations, is validated in wilder and more challenging environments like the HUST-LEBW dataset to show the usefulness of mEBAL2 to train a new generation of data-driven approaches for eyeblink detection.

We present edBB-Demo, a demonstrator of an AI-powered research platform for student monitoring in remote education. The edBB platform aims to study the challenges associated to user recognition and behavior understanding in digital platforms. This platform has been developed for data collection, acquiring signals from a variety of sensors including keyboard, mouse, webcam, microphone, smartwatch, and an Electroencephalography band. The information captured from the sensors during the student sessions is modelled in a multimodal learning framework. The demonstrator includes: i) Biometric user authentication in an unsupervised environment; ii) Human action recognition based on remote video analysis; iii) Heart rate estimation from webcam video; and iv) Attention level estimation from facial expression analysis.

Institute, Warsaw, Poland This paper describes the results of the IEEE BigData 2023 Keystroke Verification Challenge1(KVC), that considers the biometric verification performance of Keystroke Dynamics (KD), captured as tweet-long sequences of variable transcript text from over 185,000 subjects. The data are obtained from two of the largest public databases of KD up to date, the Aalto Desktop and Mobile Keystroke Databases, guaranteeing a minimum amount of data per subject, age and gender annotations, absence of corrupted data, and avoiding excessively unbalanced subject distributions with respect to the considered demographic attributes. Several neural architectures were proposed by the participants, leading to global Equal Error Rates (EERs) as low as 3.33% and 3.61% achieved by the best team respectively in the desktop and mobile scenario, outperforming the current state of the art biometric verification performance for KD. Hosted on CodaLab2, the KVC will be made ongoing to represent a useful tool for the research community to compare different approaches under the same experimental conditions and to deepen the knowledge of the field.

In this article, we present a Web-based System called M2LADS, which supports the integration and visualization of multimodal data recorded in learning sessions in a MOOC in the form of Web-based Dashboards. Based on the edBB platform, the multimodal data gathered contains biometric and behavioral signals including electroencephalogram data to measure learners’ cognitive attention, heart rate for affective measures, visual attention from the video recordings. Additionally, learners’ static background data and their learning performance measures are tracked using LOGCE and MOOC tracking logs respectively, and both are included in the Web-based System. M2LADS provides opportunities to capture learners’ holistic experience during their interactions with the MOOC, which can in turn be used to improve their learning outcomes through feedback visualizations and interventions, as well as to enhance learning analytics models and improve the open content of the MOOC.

The new regulatory framework proposal on Artificial Intelligence (AI) published by the European Commission establishes a new risk-based legal approach. The proposal highlights the need to develop adequate risk assessments for the different uses of AI. This risk assessment should address, among others, the detection and mitigation of bias in AI. In this work we analyze statistical approaches to measure biases in automatic decision-making systems. We focus our experiments in face recognition technologies. We propose a novel way to measure the biases in machine learning models using a statistical approach based on the N-Sigma method. N-Sigma is a popular statistical approach used to validate hypotheses in general science such as physics and social areas and its application to machine learning is yet unexplored. In this work we study how to apply this methodology to develop new risk assessment frameworks based on bias analysis and we discuss the main advantages and drawbacks with respect to other popular statistical tests.

The remarkable success of face recognition (FR) has endangered the privacy of internet users particularly in social media. Recently, researchers turned to use adversarial examples as a countermeasure to privacy attacks. In this paper, we assess the effectiveness of using two widely known adversarial methods (BIM and ILLC) for de-identifying personal images. We discovered, unlike previous claims in the literature, that it is not easy to get a high protection success rate (suppressing identification rate) with imperceptible adversarial perturbation to the human visual system. Finally, we found out that the transferability of adversarial examples is highly affected by the training parameters of the network with which they are generated

Presentation Attack Detection (PAD) is a crucial stage in facial recognition systems to avoid leakage of personal information or spoofing of identity to entities. Recently, pulse detection based on remote photoplethysmography (rPPG) has been shown to be effective in face presentation attack detection. This work presents three different approaches to the presentation attack detection based on rPPG: (i) The physiological domain, a domain using rPPG-based models, (ii) the Deepfakes domain, a domain where models were retrained from the physiological domain to specific Deepfakes detection tasks; and (iii) a new Presentation Attack domain was trained by applying transfer learning from the two previous domains to improve the capability to differentiate between bona-fides and attacks. The results show the efficiency of the rPPG-based models for presentation attack detection, evidencing a 21.70% decrease in average classification error rate (ACER) (from 41.03% to 19.32%) when the presentation attack domain is compared to the physiological and Deepfakes domains. Our experiments highlight the efficiency of transfer learning in rPPG-based models and perform well in presentation attack detection in instruments that do not allow copying of this physiological feature.

This paper presents a summary of the Competition on Face Presentation Attack Detection Based on Privacy-aware Synthetic Training Data (SynFacePAD 2023) held at the 2023 International Joint Conference on Biometrics (IJCB 2023). The competition attracted a total of 8 participating teams with valid submissions from academia and industry. The competition aimed to motivate and attract solutions that target detecting face presentation attacks while considering synthetic-based training data motivated by privacy, legal and ethical concerns associated with personal data. To achieve that, the training data used by the participants was limited to synthetic data provided by the organizers. The submitted solutions presented innovations and novel approaches that led to outperforming the considered baseline in the investigated benchmarks.

The International Fingerprint Liveness Detection Competition (LivDet) is a biennial event that invites academic and industry participants to prove their advancements in Fingerprint Presentation Attack Detection (PAD). This edition, LivDet2023, proposed two challenges, “Liveness Detection in Action” and “Fingerprint Representation”, to evaluate the efficacy of PAD embedded in verification systems and the effectiveness and compactness of feature sets. A third, “hidden” challenge is the inclusion of two subsets in the training set whose sensor information is unknown, testing participants’ ability to generalize their models. Only bona fide fingerprint samples were provided to participants, and the competition reports and assesses the performance of their algorithms suffering from this limitation in data availability.

Mobile devices such as smartphones and smartwatches are part of our everyday life, acquiring large amount of personal information that needs to be properly secured. Among the different authentication techniques, behavioural biometrics has become a very popular method as it allows authentication in a non-intrusive and continuous way. This study proposes M-GaitFormer, a novel mobile biometric gait verification system based on Transformer architectures. This biometric system only considers the accelerometer and gyroscope data acquired by the mobile device. A complete analysis of the proposed M-GaitFormer is carried out using the popular available databases whuGAIT and OU-ISIR. M-GaitFormer achieves Equal Error Rate (EER) values of 3.42% and 2.90% on whuGAIT and OU-ISIR, respectively, outperforming other state-of-the-art approaches based on popular Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs).

This paper proposes a method to generate the synthetic kinematic of signatures in 3D. The analysis of 3D signatures is becoming a hot topic due to the irruption of commercial off-the-shelf devices for easy acquisition of 3D movements. However, the novelty of this technology reveals the scarce publicly available signatures in 3D, which hinder their de-velopment. A solution is the synthesis of Signatures in 3D. As a first step, this paper synthesizes the kinematics of 3D signatures based on the Kinematic Theory of Rapid Movements and its associated Sigma-Lognormal model in 3D. To evaluate the method, we regenerate signature databases with synthetic speed profiles in all genuine and forgeries found in two 3D signature databases. Then, we analyze the similarities in the performance of a signature verifier when real and synthetic signatures are used in random and skilled forgeries experiments.

This paper describes the experimental framework and results of the IJCB 2022 Mobile Behavioral Biometrics Competition (MobileB2C). The aim of MobileB2C is bench-marking mobile user authentication systems based on behavioral biometric traits transparently acquired by mobile devices during ordinary Human-Computer Interaction (HCI), using a novel public database, BehavePassDB11https://github.com/BiDAlab/MobileB2C_BehavePassDE, and a standard experimental protocol. The competition is divided into four tasks corresponding to typical user activities: keystroke, text reading, gallery swiping, and tapping. The data are composed of touchscreen data and several background sensor data simultaneously acquired. “Random” (different users with different devices) and “skilled” (different user on the same device attempting to imitate the legitimate one) impostor scenarios are considered. The results achieved by the participants show the feasibility of user authentication through behavioral biometrics, although this proves to be a non-trivial challenge. MobileB2C will be established as an on-going competition22https://sites.google.com/view/mobileb2c/.

We propose a discrimination-aware learning method to improve both the accuracy and fairness of biased face recognition algorithms. The most popular face recognition benchmarks assume a distribution of subjects without paying much attention to their demographic attributes. In this work, we perform a comprehensive discrimination-aware experimentation of deep learning-based face recognition. We also propose a notational framework for algorithmic discrimination with application to face biometrics. The experiments include three popular face recognition models and three public databases composed of 64,000 identities from different demographic groups characterized by sex and ethnicity. We experimentally show that learning processes based on the most used face databases have led to popular pre-trained deep face models that present evidence of strong algorithmic discrimination. Finally, we propose a discrimination-aware learning method, Sensitive Loss, based on the popular triplet loss function and a sensitive triplet generator. Our approach works as an add-on to pre-trained networks and is used to improve their performance in terms of average accuracy and fairness. The method shows results comparable to state-of-the-art de-biasing networks and represents a step forward to prevent discriminatory automatic systems.

Media forensics has attracted a tremendous attention in the last years in part due to the increasing concerns around DeepFakes. Since the release of the initial DeepFakes databases of the 1st generation such as UADFV and FaceForensics++ up to the latest databases of the 2nd generation such as Celeb-DF and DFDC, many visual improvements have been carried out, making fake videos almost indistinguishable to the human eye. This study provides an in-depth analysis of both 1st and 2nd DeepFakes generations in terms of fake detection performance. Two different methods are considered in our experimental framework: (i) the traditional one followed in the literature based on selecting the entire face as input to the fake detection system, and (ii) a novel approach based on the selection of specific facial regions as input to the fake detection system. Fusion techniques are applied both to the facial regions and also to three different state-of-the-art fake detection systems (Xception, Capsule Network, and DSP-FWA) in order to further increase the robustness of the detectors considered. Finally, experiments regarding intra- and inter-database scenarios are performed. Among all the findings resulting from our experiments, we highlight: (i) the very good results achieved using facial regions and fusion techniques with fake detection results above 99% Area Under the Curve (AUC) for UADFV, FaceForensics++, and Celeb-DF v2 databases, and (ii) the necessity to put more efforts on the analysis of inter-database scenarios to improve the ability of the fake detectors against attacks unseen during learning.

This work presents a new deep learning approach for keystroke biometrics based on a novel Distance Metric Learning method (DML). DML maps input data into a learned representation space that reveals a “semantic” structure based on distances. In this work, we propose a novel DML method specifically designed to address the challenges associated to free-text keystroke identification where the classes used in learning and inference are disjoint. The proposed SetMargin Loss (SM-L) extends traditional DML approaches with a learning process guided by pairs of sets instead of pairs of samples, as done traditionally. The proposed learning strategy allows to enlarge inter-class distances while maintaining the intra-class structure of keystroke dynamics. We analyze the resulting representation space using the mathematical problem known as Circle Packing, which provides neighbourhood structures with a theoretical maximum inter-class distance. We finally prove experimentally the effectiveness of the proposed approach on a challenging task: keystroke biometric identification over a large set of 78,000 subjects. Our method achieves state-of-the-art accuracy on a comparison performed with the best existing approaches.

This article presents SVC-onGoing1, an on-going competition for on-line signature verification where researchers can easily benchmark their systems against the state of the art in an open common platform using large-scale public databases, such as DeepSignDB2 and SVC2021_EvalDB3, and standard experimental protocols. SVC-onGoing is based on the ICDAR 2021 Competition on On-Line Signature Verification (SVC 2021), which has been extended to allow participants anytime. The goal of SVC-onGoing is to evaluate the limits of on-line signature verification systems on popular scenarios (office/mobile) and writing inputs (stylus/finger) through large-scale public databases. Three different tasks are considered in the competition, simulating realistic scenarios as both random and skilled forgeries are simultaneously considered on each task. The results obtained in SVC-onGoing prove the high potential of deep learning methods in comparison with traditional methods. In particular, the best signature verification system has obtained Equal Error Rate (EER) values of 3.33% (Task 1), 7.41% (Task 2), and 6.04% (Task 3). Future studies in the field should be oriented to improve the performance of signature verification systems on the challenging mobile scenarios of SVC-onGoing in which several mobile devices and the finger are used during the signature acquisition.

This study proposes DeepWriteSYN, a novel on-line handwriting synthesis approach via deep short-term representations. It comprises two modules: i) an optional and interchangeable temporal segmentation, which divides the handwriting into short-time segments consisting of individual or multiple concatenated strokes; and ii) the on-line synthesis of those short-time handwriting segments, which is based on a sequence-to-sequence Variational Autoencoder (VAE). The main advantages of the proposed approach are that the synthesis is carried out in short-time segments (that can run from a character fraction to full characters) and that the VAE can be trained on a configurable handwriting dataset. These two properties give a lot of flexibility to our synthesiser, e.g., as shown in our experiments, DeepWriteSYN can generate realistic handwriting variations of a given handwritten structure corresponding to the natural variation within a given population or a given subject. These two cases are developed experimentally for individual digits and handwriting signatures, respectively, achieving in both cases remarkable results. Also, we provide experimental results for the task of on-line signature verification showing the high potential of DeepWriteSYN to improve significantly one-shot learning scenarios. To the best of our knowledge, this is the first synthesis approach capable of generating realistic on-line handwriting in the short term (including handwritten signatures) via deep learning. This can be very useful as a module toward long-term realistic handwriting generation either completely synthetic or as natural variation of given handwriting samples.

In this paper we develop FaceQgen11Publicly available in: https://github.com/uam-biometrics/FaceQgen, a No-Reference Quality Assessment approach for face images based on a Generative Adversarial Network that generates a scalar quality measure related with the face recognition accuracy. FaceQgen does not require labelled quality measures for training. It is trained from scratch using the SCface database. FaceQgen applies image restoration to a face image of unknown quality, transforming it into a canonical high quality image, i.e., frontal pose, homogeneous background, etc. The quality estimation is built as the similarity between the original and the restored images, since low quality images experience bigger changes due to restoration. We compare three different numerical quality measures: a) the MSE between the original and the restored images, b) their SSIM, and c) the output score of the Discriminator of the GAN. The results demonstrate that FaceQgen's quality measures are good estimators of face recognition accuracy. Our experiments include a comparison with other quality assessment methods designed for faces and for general images, in order to position FaceQgen in the state of the art. This comparison shows that, even though FaceQgen does not surpass the best existing face quality assessment methods in terms of face recognition accuracy prediction, it achieves good enough results to demonstrate the potential of semi-supervised learning approaches for quality estimation (in particular, data -driven learning based on a single high quality image per subject), having the capacity to improve its performance in the future with adequate refinement of the model and the significant advantage over competing methods of not needing quality labels for its development. This makes FaceQgen flexible and scalable without expensive data curation.

This work explores facial expression bias as a security vulnerability of face recognition systems. Despite the great performance achieved by state-of-the-art face recognition systems, the algorithms are still sensitive to a large range of covariates. We present a comprehensive analysis of how facial expression bias impacts the performance of face recognition technologies. Our study analyzes: i) facial expression biases in the most popular face recognition databases; and ii) the impact of facial expression in face recognition performances. Our experimental framework includes two face detectors, three face recognition models, and three different databases. Our results demonstrate a huge facial expression bias in the most widely used databases, as well as a related impact of face expression in the performance of state-of-the-art algorithms. This work opens the door to new research lines focused on mitigating the observed vulnerability.

This work proposes a novel privacy-preserving neural network feature representation to suppress the sensitive information of a learned space while maintaining the utility of the data. The new international regulation for personal data protection forces data controllers to guarantee privacy and avoid discriminative hazards while managing sensitive data of users. In our approach, privacy and discrimination are related to each other. Instead of existing approaches aimed directly at fairness improvement, the proposed feature representation enforces the privacy of selected attributes. This way fairness is not the objective, but the result of a privacy-preserving learning method. This approach guarantees that sensitive information cannot be exploited by any agent who process the output of the model, ensuring both privacy and equality of opportunity. Our method is based on an adversarial regularizer that introduces a sensitive information removal function in the learning objective. The method is evaluated on three different primary tasks (identity, attractiveness, and smiling) and three publicly available benchmarks. In addition, we present a new face annotation dataset with balanced distribution between genders and ethnic origins. The experiments demonstrate that it is possible to improve the privacy and equality of opportunity while retaining competitive performance independently of the task.

In this paper we list the sensors commonly available in modern smartphones and provide a general outlook of the different ways these sensors can be used for modeling the interaction between human and smartphones. We then provide a taxonomy of applications that can exploit the signals originated by these sensors in three different dimensions, depending on the main information content embedded in the signals exploited in the application: neuromotor skills, cognitive functions, and behaviors/routines. We then summarize a representative selection of existing research datasets in this area, with special focus on applications related to user authentication, including key features and a selection of the main research results obtained on them so far. Then, we perform the experimental work using the HuMIdb database (Human Mobile Interaction database), a novel multimodal mobile database that includes 14 mobile sensors captured from 600 participants. We evaluate a biometric authentication system based on simple linear touch gestures using a Siamese Neural Network architecture. Very promising results are achieved with accuracies up to 87% for person authentication based on a simple and fast touch gesture.

The world is becoming more interconnected every day. With the high technological evolution and the increasing deployment of it in our society, scenarios based on the Internet of Things (IoT) can be considered a reality nowadays. However, and before some predictions become true (75 billion devices are expected to be interconnected in the next few years), many efforts must be carried out in terms of scalability and security. In this study we propose and evaluate a new approach based on the incorporation of Blockchain into current IoT scenarios. The main contributions of this study are as follows: i) an in-depth analysis of the different possibilities for the integration of Blockchain into IoT scenarios, focusing on the limited processing capabilities and storage space of most IoT devices, and the economic cost and performance of current Blockchain technologies; ii) a new method based on a novel module named BIoT Gateway that allows both unidirectional and bidirectional communications with IoT devices on real scenarios, allowing to exchange any kind of data; and iii) the proposed method has been fully implemented and validated on two different real-life IoT scenarios, extracting very interesting findings in terms of economic cost and execution time. The source code of our implementation is publicly available in the Ethereum testnet.

In this study we estimate the heart rate from face videos for student assessment. This information could be very valuable to track their status along time and also to estimate other data such as their attention level or the presence of stress that may be caused by cheating attempts. The recent edBBplat, a platform for student behavior modelling in remote education, is considered in this study1. This platform permits to capture several signals from a set of sensors that capture biometric and behavioral data: RGB and near infrared cameras, microphone, EEG band, mouse, smartwatch, and keyboard, among others. In the experimental framework of this study, we focus on the RGB and near-infrared video sequences for performing heart rate estimation applying remote photoplethysmography techniques. The experiments include behavioral and physiological data from 25 different students completing a collection of tasks related to e-learning. Our proposed face heart rate estimation approach is compared with the heart rate provided by the smartwatch, achieving very promising results for its future deployment in e-learning applications.

This paper presents a comparative evaluation of methods for remote heart rate estimation using face videos, i.e., given a video sequence of the face as input, methods to process it to obtain a robust estimation of the subject's heart rate at each moment. Four alternatives from the literature are tested, three based in hand-crafted approaches and one based on deep learning. The methods are compared using RGB videos from the COHFACE database. Experiments show that the learning-based method achieves much better accuracy than the hand-crafted ones. The low error rate achieved by the learning-based model makes possible its application in real scenarios, e.g. in medical or sports environments.

This work proposes and analyzes the use of keystroke biometrics for content de-anonymization. Fake news have become a powerful tool to manipulate public opinion, especially during major events. In particular, the massive spread of fake news during the COVID-19 pandemic has forced governments and companies to fight against missinformation. In this context, the ability to link multiple accounts or profiles that spread such malicious content on the Internet while hiding in anonymity would enable proactive identification and blacklisting. Behavioral biometrics can be powerful tools in this fight. In this work, we have analyzed how the latest advances in keystroke biometric recognition can help to link behavioral typing patterns in experiments involving 100,000 users and more than 1 million typed sequences. Our proposed system is based on Recurrent Neural Networks adapted to the context of content de-anonymization. Assuming the challenge to link the typed content of a target user in a pool of candidate profiles, our results show that keystroke recognition can be used to reduce the list of candidate profiles by more than 90%. In addition, when keystroke is combined with auxiliary data (such as location), our system achieves a Rank-1 identification performance equal to 52.6% and 10.9% for a background candidate list composed of 1K and 100K profiles, respectively.

We study the suitability of keystroke dynamics to authenticate 100 K users typing free-text. For this, we first analyze to what extent our method based on a Siamese Recurrent Neural Network (RNN) is able to authenticate users when the amount of data per user is scarce, a common scenario in free-text keystroke authentication. With 1 K users for testing the network, a population size comparable to previous works, TypeNet obtains an equal error rate of 4.8% using only 5 enrollment sequences and 1 test sequence per user with 50 keystrokes per sequence. Using the same amount of data per user, as the number of test users is scaled up to 100K, the performance in comparison to 1 K decays relatively by less than 5%, demonstrating the potential of Type-Net to scale well at large scale number of users. Our experiments are conducted with the Aalto University keystroke database. To the best of our knowledge, this is the largest free-text keystroke database captured with more than 136M keystrokes from 168K users.

In this paper, we investigate how to detect intruders with low latency for Active Authentication (AA) systems with multiple-users. We extend the Quickest Change Detection (QCD) framework to the multiple-user case and formulate the Multiple-user Quickest Intruder Detection (MQID) algorithm. Furthermore, we extend the algorithm to the data-efficient scenario where intruder detection is carried out with fewer observation samples. We evaluate the effectiveness of the proposed method on two publicly available AA datasets on the face modality.

With the aim of studying how current multimodal AI algorithms based on heterogeneous sources of information are affected by sensitive elements and inner biases in the data, this demonstrator experiments over an automated recruitment testbed based on Curriculum Vitae: FairCVtest. The presence of decision-making algorithms in society is rapidly increasing nowadays, while concerns about their transparency and the possibility of these algorithms becoming new sources of discrimination are arising. This demo shows the capacity of the Artificial Intelligence (AI) behind a recruitment tool to extract sensitive information from unstructured data, and exploit it in combination to data biases in undesirable (unfair) ways. Aditionally, the demo includes a new algorithm (SensitiveNets) for discrimination-aware learning which eliminates sensitive information in our multimodal AI framework.

We propose two face representations that are blind to facial expressions associated to emotional responses. This work is in part motivated by new international regulations for personal data protection, which enforce data controllers to protect any kind of sensitive information involved in automatic processes. The advances in Affective Computing have contributed to improve human-machine interfaces but, at the same time, the capacity to monitorize emotional responses triggers potential risks for humans, both in terms of fairness and privacy. We propose two different methods to learn these expression-blinded facial features. We show that it is possible to eliminate information related to emotion recognition tasks, while the performance of subject verification, gender recognition, and ethnicity classification are just slightly affected. We also present an application to train fairer classifiers in a case study of attractiveness classification with respect to a protected facial expression attribute. The results demonstrate that it is possible to reduce emotional information in the face representation while retaining competitive performance in other face-based artificial intelligence tasks.

This work explores the biases in learning processes based on deep neural network architectures. We analyze how bias affects deep learning processes through a toy example using the MNIST database and a case study in gender detection from face images. We employ two gender detection models based on popular deep neural networks. We present a comprehensive analysis of bias effects when using an unbalanced training dataset on the features learned by the models. We show how bias impacts in the activations of gender detection models based on face images. We finally propose InsideBias, a novel method to detect biased models. InsideBias is based on how the models represent the information instead of how they perform, which is the normal practice in other existing methods for bias detection. Our strategy with InsideBias allows to detect biased models with very few samples (only 15 images in our case study). Our experiments include 72K face images from 24K identities and 3 ethnic groups.

In the last five years, deep learning methods, in particular CNN, have attracted considerable attention in the field of face-based recognition, achieving impressive results. Despite this progress, it is not yet clear precisely to what extent deep features are able to follow all the intra-class variations that the face can present over time. In this paper we investigate the performance the performance improvement of face recognition systems by adopting self updating strategies of the face templates. For that purpose, we evaluate the performance of a well-known deep-learning face representation, namely, FaceNet, on a dataset that we generated explicitly conceived to embed intra-class variations of users on a large time span of captures: the APhotoEveryday (APE) dataset11https://github.com/PRALabBiometrics/APhotoEverydayDB. Moreover, we compare these deep features with handcrafted features extracted using the BSIF algorithm. In both cases, we evaluate various template update strategies, in order to detect the most useful for such kind of features. Experimental results show the effectiveness of “optimized” self-update methods with respect to systems without update or random selection of templates.

Passwords are still used on a daily basis for all kind of applications. However, they are not secure enough by themselves in many cases. This work enhances password scenarios through two-factor authentication asking the users to draw each character of the password instead of typing them as usual. The main contributions of this study are as follows: i) We present the novel MobileTouchDB public database, acquired in an unsupervised mobile scenario with no restrictions in terms of position, posture, and devices. This database contains more than 64K on-line character samples performed by 217 users, with 94 different smartphone models, and up to 6 acquisition sessions. ii) We perform a complete analysis of the proposed approach considering both traditional authentication systems such as Dynamic Time Warping (DTW) and novel approaches based on Recurrent Neural Networks (RNNs). In addition, we present a novel approach named Time-Aligned Recurrent Neural Networks (TA-RNNs). This approach combines the potential of DTW and RNNs to train more robust systems against attacks. A complete analysis of the proposed approach is carried out using both MobileTouchDB and e-BioDigitDB databases. Our proposed TA-RNN system outperforms the state of the art, achieving a final 2.38% Equal Error Rate, using just a 4-digit password and one training sample per character. These results encourage the deployment of our proposed approach in comparison with traditional typed-based password systems where the attack would have 100% success rate under the same impostor scenario.

This work enhances traditional authentication systems based on Personal Identification Numbers (PIN) and One-Time Passwords (OTP) through the incorporation of biometric information as a second level of user authentication. In our proposed approach, users draw each digit of the password on the touchscreen of the device instead of typing them as usual. A complete analysis of our proposed biometric system is carried out regarding the discriminative power of each handwritten digit and the robustness when increasing the length of the password and the number of enrolment samples. The new e-BioDigit database, which comprises on-line handwritten digits from 0 to 9, has been acquired using the finger as input on a mobile device. This database is used in the experiments reported in this work and it is available together with benchmark results in GitHub.11.https://github.com/BiDAlab/eBioDigitDB. Finally, we discuss specific details for the deployment of our proposed approach on current PIN and OTP systems, achieving results with Equal Error Rates (EERs) ca. 4.0 percent when the attacker knows the password. These results encourage the deployment of our proposed approach in comparison to traditional PIN and OTP systems where the attack would have 100 percent success rate under the same impostor scenario.

In this paper we evaluate the suitability of handwriting patterns as potential biomarkers to model Parkinson's disease (PD). Although the study of PD is attracting the interest of many researchers around the world, databases to evaluate handwriting patterns are scarce and knowledge about patterns associated to PD is limited and biased to the existing datasets. This paper introduces a database with a total of 935 handwriting tasks collected from 55 PD patients and 94 healthy controls (45 young and 49 old). Three feature sets are extracted from the signals: neuromotor, kinematic, and nonlinear dynamic. Different classifiers are used to discriminate between PD and healthy subjects: support vector machines, k-nearest neighbors, and a multilayer perceptron. The proposed features and classifiers enable to detect PD with accuracies between 81% and 97%. Additionally, new insights are presented on the utility of the studied features for monitoring and detecting PD.

Data have become one of the most valuable things in this new era where deep learning technology seems to overcome traditional approaches. However, in some tasks, such as the verification of handwritten signatures, the amount of publicly available data is scarce, what makes difficult to test the real limits of deep learning. In addition to the lack of public data, it is not easy to evaluate the improvements of novel approaches compared with the state of the art as different experimental protocols and conditions are usually considered for different signature databases. To tackle all these mentioned problems, the main contribution of this study is twofold: i) we present and describe the new DeepSignDB on-line handwritten signature biometric public database, and ii) we propose a standard experimental protocol and benchmark to be used for the research community in order to perform a fair comparison of novel approaches with the state of the art. The DeepSignDB database is obtained through the combination of some of the most popular on-line signature databases, and a novel dataset not presented yet. It comprises more than 70K signatures acquired using both stylus and finger inputs from a total of 1526 users. Two acquisition scenarios are considered, office and mobile, with a total of 8 different devices. Additionally, different types of impostors and number of acquisition sessions are considered along the database. The DeepSignDB and benchmark results are available in GitHub.

This paper proposes a novel approach for on-line signature complexity detection based on Recurrent Neural Networks (RNNs). Complexity of handwritten signatures can vary from very simple ones (just a simple flourish) to very complex signatures (including the handwritten full name and complex flourish). Three different complexity levels are proposed: low, medium, and high. Time functions are extracted from the on-line signatures and a system based on RNNs (BLSTM in particular) is trained to classify the three levels of complexity over a ground truth manually labelled database (BiosecurID with 400 subjects). This initial model is used to automatically label a very large database (DeepSignDB) containing over 1500 subjects, which is then used to train the proposed RNN for signature complexity detection. Promising results ca. 85% of accuracy are achieved. This complexity detector could be used as a first stage in a signature verification system in order to train a specific biometric system per signature complexity level and improve the overall system performance.

BioTouchPass enhances traditional authentication systems based on Personal Identification Numbers (PIN) and One-Time Passwords (OTP) through the incorporation of biometric information from handwriting as a second level of user authentication. In our proposed approach, users draw each digit of the password on the touchscreen of the device instead of typing them as usual. This way the security of the authentication system increases as impostors need more than the traditional password to get access to the system. BioTouchPass achieves results with Equal Error Rates (EERs) ca. 4.0% when the attacker knows the password, outperforming other authentication schemes based on touch biometrics, and providing a user-friendly interface easily adaptable to a variety of mobile devices and application scenarios.

In this paper we evaluate body static information to improve the performance of face recognition at a distance. To this aim, we assess one state-of-the-art face recognition system based on deep features and three body-based person recognition systems, namely: i) row profiles with correlation coefficient, ii) row and column profiles with Support Vector Machines, and iii) contour coordinates with Dynamic Time Warping. Results are reported using the Multi-Biometric Tunnel Database, emphasizing on three distance settings: far, medium, and close, ranging from full body exposure to head and shoulders exposure. Several conclusions can be drawn from this work: a) row and column profiles are more robust than contour coordinates, b) face-based systems perform poorly at far distances, being body-based information more reliable at that distances, c) in general face-based systems perform better than body-based approaches at medium and close distances, and d) the multimodal fusion approach manages to outperform face-only recognition at distance in all distance-settings considered.

We study user interaction with touchscreens based on swipe gestures for personal authentication. This approach has been analyzed only recently in the last few years in a series of disconnected and limited works. We summarize those recent efforts and then compare them to three new systems (based on support vector machine and Gaussian mixture model using selected features from the literature) exploiting independent processing of the swipes according to their orientation. For the analysis, four public databases consisting of touch data obtained from gestures sliding one finger on the screen are used. We first analyze the contents of the databases, observing various behavioral patterns, e.g., horizontal swipes are faster than vertical independently of the device orientation. We then explore an intra-session scenario, where users are enrolled and authenticated within the same day, and an inter-session one, where enrollment and test are performed on different days. The resulting benchmarks and processed data are made public, allowing the reproducibility of the key results obtained based on the provided score files and scripts. In addition to the remarkable performance, thanks to the proposed orientation-based conditional processing, the results show various new insights into the distinctiveness of swipe interaction, e.g., some gestures hold more user-discriminant information, data from landscape orientation is more stable, and horizontal gestures are more discriminative in general than vertical ones.

The role of soft biometrics to enhance person recognition systems in unconstrained scenarios has not been extensively studied. Here, we explore the utility of the following modalities: gender, ethnicity, age, glasses, beard, and moustache. We consider two assumptions: 1) manual estimation of soft biometrics and 2) automatic estimation from two commercial off-the-shelf systems (COTS). All experiments are reported using the labeled faces in the wild (LFW) database. First, we study the discrimination capabilities of soft biometrics standalone. Then, experiments are carried out fusing soft biometrics with two state-of-the-art face recognition systems based on deep learning. We observe that soft biometrics is a valuable complement to the face modality in unconstrained scenarios, with relative improvements up to 40%/15% in the verification performance when using manual/automatic soft biometrics estimation. Results are reproducible as we make public our manual annotations and COTS outputs of soft biometrics over LFW, as well as the face recognition scores.

Architectures based on Recurrent Neural Networks (RNNs) have been successfully applied to many different tasks such as speech or handwriting recognition with state-of-the art results. The main contribution of this work is to analyse the feasibility of RNNs for on-line signature verification in real practical scenarios. We have considered a system based on Long Short-Term Memory (LSTM) with a Siamese architecture whose goal is to learn a similarity metric from pairs of signatures. For the experimental work, the BiosecurID database comprised of 400 users and 4 separated acquisition sessions are considered. Our proposed LSTM RNN system has outperformed the results of recent published works on the BiosecurID benchmark in figures ranging from 17.76% to 28.00% relative verification performance improvement for skilled forgeries.

On-line signature verification systems are mainly based on two approaches: feature- or time functions-based systems (a.k.a. global and local systems). However, new sources of information can be also considered in order to complement these traditional approaches, reduce the intra-class variability and achieve more robust signature verification systems against forgers. In this paper we focus on the use of the concept of complexity in on-line signature verification systems. The main contributions of the present work are: 1) classification of users according to the complexity level of their signatures using features extracted from the Sigma LogNormal writing generation model, and 2) a new architecture for signature verification exploiting signature complexity that results in highly improved performance. Our proposed approach is tested considering the BiosecurID on-line signature database with a total of 400 users. Results of 5.8% FRR for a FAR = 5.0% have been achieved against skilled forgeries outperforming recent related works. In addition, an analysis of the optimal time functions for each complexity level is performed providing practical insights for the application of signature verification in real scenarios.

Due to the ability of millimeter waves (mmWs) to penetrate dielectric materials, such as plastic, polymer, and clothes, the mmW imaging technology has been widely used for the detection of concealed weapons and objects. The use of mmW images has also recently been proposed for biometric person recognition to overcome certain limitations in image acquisition at visible frequencies. This paper proposes a biometric person recognition system based on the shape information extracted from real mmW images. To this aim, we report experimental results using the mmW images with different body shape-based feature approaches, such as contour coordinates, shape contexts, Fourier descriptors, and row and column profiles. We also study various distance-based and classifier-based matching schemes. Experimental results suggest the potential of performing person recognition through mmW imaging using only shape information, a functionality that could be integrated in the security scanners deployed in airports.

Gender estimation has received increased attention due to its use in a number of pertinent security and commercial applications. Automated gender estimation algorithms are mainly based on extracting representative features from face images. In this work we study gender estimation based on information deduced jointly from face and body, extracted from single-shot images. The approach addresses challenging settings such as low-resolution-images, as well as settings when faces are occluded. Specifically the face-based features include local binary patterns (LBP) and scale-invariant feature transform (SIFT) features, projected into a PCA space. The features of the novel body-based algorithm proposed in this work include continuous shape information extracted from body silhouettes and texture information retained by HOG descriptors. Support Vector Machines (SVMs) are used for classification for body and face features. We conduct experiments on images extracted from video-sequences of the Multi-Biometric Tunnel database, emphasizing on three distance-settings: close, medium and far, ranging from full body exposure (far setting) to head and shoulders exposure (close setting). The experiments suggest that while face-based gender estimation performs best in the close-distance-setting, body-based gender estimation performs best when a large part of the body is visible. Finally we present two score-level-fusion schemes of face and body-based features, outperforming the two individual modalities in most cases.

User authentication using simple gestures is now common in portable devices. In this work, authentication with free-form sketches is studied. Verification systems using dynamic time warping and Gaussian mixture models are proposed, based on dynamic signature verification approaches. The most discriminant features are studied using the sequential forward floating selection algorithm. The effects of the time lapse between capture sessions and the impact of the training set size are also studied. Development and validation experiments are performed using the DooDB database, which contains passwords from 100 users captured on a smartphone touchscreen. Equal error rates between 3% and 8% are obtained against random forgeries and between 21% and 22% against skilled forgeries. High variability between capture sessions increases the error rates.

This work presents a case study on the application of biometric systems for student authentication services. We analyze the accuracy of a keystroke dynamics algorithm used to authenticate students during a real online exam of an introductory computer science course.

On-line signature verification still remains a challenging task within biometrics . Due to their behavioural nature (opposed to anatomic biometric traits), signatures present a notable variability even between successive realizations. This leads to higher error rates than other largely used modalities such as iris or fingerprints and is one of the main reasons for the relatively slow deployment of this technology. As a step towards the improvement of signature recognition accuracy , the present paper explores and evaluates a novel approach that takes advantage of the performance boost that can be reached through the fusion of on-line and off-line signatures. In order to exploit the complementarity of the two modalities, we propose a method for the generation of enhanced synthetic static samples from on-line data. Such synthetic off-line signatures are used on a new on-line signature recognition architecture based on the combination of both types of data: real on-line samples and artificial off-line signatures synthesized from the real data. The new on-line recognition approach is evaluated on a public benchmark containing both real versions (on-line and off-line) of the exactly same signatures. Different findings and conclusions are drawn regarding the discriminative power of on-line and off-line signatures and of their potential combination both in the random and skilled impostors scenarios.

Biometric data are considered sensitive personal data and any privacy leakage poses severe security risks. Biometric templates should hence be protected, obscuring the biometric signal in a non-reversible manner, while preserving the unprotected system's performance. In the present work, irreversible face templates based on adaptive Bloom filters are proposed. Experiments are carried out on the publicly available Bio Secure DB utilizing the free Bob image processing toolbox, so that research is fully reproducible. The performance and security evaluations proof the irreversibility of the protected templates, while preserving the verification performance. Furthermore, template size is considerably reduced.

The use of Millimetre wave images has been proposed recently in the biometric field to overcome certain limitations when using images acquired at visible frequencies. In this paper, several body shape-based techniques were applied to model the silhouette of images of people acquired at 94 GHz. We put forward several methods for the parameterization and classification stage with the objective of finding the best configuration in terms of biometric recognition performance. Contour coordinates, shape contexts, Fourier descriptors and silhouette landmarks were used as feature approaches and for classification we utilized Euclidean distance and a dynamic programming method. Results showed that the dynamic programming algorithm improved the performance of the system with respect to the baseline Euclidean distance and the necessity of a minimum resolution of the contour to achieve promising equal error rates. The use of the contour coordinates is the most suitable feature to use in the system regarding the performance and the computational cost involved when having at least 3 images for model training. Besides, Fourier descriptors are more robust against rotations, which may be of interest when dealing with few training images.

Comparing a latent fingerprint minutiae set against a ten print fingerprint minutiae set using an automated fingerprint identification system is a challenging problem. This is mainly because latent fingerprints obtained from crime scenes are mostly partial fingerprints, and most automated systems expect approximately the same number of minutiae between query and the reference fingerprint under comparison for good performance. In this work, we propose a methodology to reduce the minutiae set of ten print with respect to that of query latent minutiae set by registering the orientation field of latent fingerprint with the ten print orientation field. By reducing the search space of minutiae from the ten print, we can improve the performance of automated identification systems for latent fingerprints. We report the performance of our registration algorithm on the NIST-SD27 database as well as the improvement in the Rank Identification accuracy of a standard minutiae-based automated system.

Soft biometric information extracted from a human body (e.g., height, gender, skin color, hair color, and so on) is ancillary information easily distinguished at a distance but it is not fully distinctive by itself in recognition tasks. However, this soft information can be explicitly fused with biometric recognition systems to improve the overall recognition when confronting high variability conditions. One significant example is visual surveillance, where face images are usually captured in poor quality conditions with high variability and automatic face recognition systems do not work properly. In this scenario, the soft biometric information can provide very valuable information for person recognition. This paper presents an experimental study of the benefits of soft biometric labels as ancillary information based on the description of human physical features to improve challenging person recognition scenarios at a distance. In addition, we analyze the available soft biometric information in scenarios of varying distance between camera and subject. Experimental results based on the Southampton multibiometric tunnel database show that the use of soft biometric traits is able to improve the performance of face recognition based on sparse representation on real and ideal scenarios by adaptive fusion rules.

To ensure the actual presence of a real legitimate trait in contrast to a fake self-manufactured synthetic or reconstructed sample is a significant problem in biometric authentication, which requires the development of new and efficient protection measures. In this paper, we present a novel software-based fake detection method that can be used in multiple biometric systems to detect different types of fraudulent access attempts. The objective of the proposed system is to enhance the security of biometric recognition frameworks, by adding liveness assessment in a fast, user-friendly, and non-intrusive manner, through the use of image quality assessment. The proposed approach presents a very low degree of complexity, which makes it suitable for real-time applications, using 25 general image quality features extracted from one image (i.e., the same acquired for authentication purposes) to distinguish between legitimate and impostor samples. The experimental results, obtained on publicly available data sets of fingerprint, iris, and 2D face, show that the proposed method is highly competitive compared with other state-of-the-art approaches and that the analysis of the general image quality of real biometric samples reveals highly valuable information that may be very efficiently used to discriminate them from fake traits.

Footstep recognition is a relatively new biometric which aims to discriminate people using walking characteristics extracted from floor-based sensors. This paper reports for the first time a comparative assessment of the spatiotemporal information contained in the footstep signals for person recognition. Experiments are carried out on the largest footstep database collected to date, with almost 20,000 valid footstep signals and more than 120 people. Results show very similar performance for both spatial and temporal approaches (5 to 15 percent EER depending on the experimental setup), and a significant improvement is achieved for their fusion (2.5 to 10 percent EER). The assessment protocol is focused on the influence of the quantity of data used in the reference models, which serves to simulate conditions of different potential applications such as smart homes or security access scenarios.

In forensic applications the evidential value of palmprints is obvious according to surveys of law enforcement agencies which indicate that 30 percent of the latents recovered from crime scenes are from palms. Consequently, developing forensic automatic palmprint identification technology is an urgent and challenging task which deals with latent (i.e., partial) and full palmprints captured or recovered at 500 ppi at least (the current standard in forensic applications) for minutiae-based offline recognition. Moreover, a rigorous quantification of the evidential value of biometrics, such as fingerprints and palmprints, is essential in modern forensic science. Recently, radial triangulation has been proposed as a step towards this objective in fingerprints, using minutiae manually extracted by experts. In this work we help in automatizing such comparison strategy, and generalize it to palmprints. Firstly, palmprint segmentation and enhancement are implemented for full prints feature extraction by a commercial biometric SDK in an automatic way, while features of latent prints are manually extracted by forensic experts. Then a latent-to-full palmprint comparison algorithm based on radial triangulation is proposed, in which radial triangulation is utilized for minutiae modeling. Finally, 22 latent palmprints from real forensic cases and 8680 full palmprints from criminal investigation field are used for performance evaluation. Experimental results proof the usability and efficiency of the proposed system, i.e, rank-1 identification rate of 62% is achieved despite the inherent difficulty of latent-to-full

An analysis of the quality of on-line handwritten signatures is carried out based on the Sigma-Lognormal model. In the study, two main issues are addressed from a kinematic perspective of humanly-produced movements. On the one hand, what makes some signatures perform better than others in automatic signature verification systems, and on the other hand if that information may be used as a quality measure in order to predict the expected performance of a given sample. Experiments were carried out on the MCYT database and show the high potential of certain kinematic features for signature quality assessment.

Questioned document examination is extensively used by forensic specialists for criminal identification. This paper presents a writer recognition system based on allographic features operating in identification mode (one-to-many). It works at the level of isolated characters, considering that each writer uses a reduced number of shapes for each one. Individual characters of a writer are manually segmented and labeled by an expert as pertaining to one of 62 alphanumeric classes (10 numbers and 52 letters, including lowercase and uppercase letters), being the particular setup used by the forensic laboratory participating in this work. A codebook of shapes is then generated by clustering and the probability distribution function of allograph usage is the discriminative feature used for recognition. Results obtained on a database of 30 writers from real forensic documents show that the character class information given by the manual analysis provides a valuable source of improvement, justifying the proposed approach. We also evaluate the selection of different alphanumeric channels, showing a dependence between the size of the hit list and the number of channels needed for optimal performance.

The kinematical information present in synthetically generated signatures is analyzed using the Sigma-Lognormal model and compared to the kinematical properties of real samples. Experiments are carried out on totally independent development and test sets and show a high degree of similarity between humanly produced and artificial signatures. One particular flaw is found in the velocity profile of synthetic signatures. Two possible solutions are proposed to improve the synthetic generation method using the Kinematic Theory of rapid human movements.

Touch screen-enabled devices are proliferating in the communications and entertainment markets. In this scenario, the use of graphical passwords for user validation is receiving an increasing interest in the last years. Unlike in other fields of research on automatic user authentication, such as biometrics, there are no public databases of graphical passwords usable for research purposes (to the extent of our knowledge). In the present work, the recently captured DooDB database is introduced. This database comprises two sub corpora: doodles and simplified signatures (pseudo-signatures). These data were produced by 100 users, who were asked to draw with their fingertips over a mobile device touch screen. Forgeries are also included in the database. A quantitative analysis of both datasets is first performed. Preliminary verification experiments using the two kinds of graphical passwords are reported.

Doodle-based graphical passwords represent a challenging scenario due to their high variability and the tendency to be graphically simple. Despite this, doodle-based authentication using touch screens is a promising lightweight user verification method. Several works have been published in this field, although they report in general experimental verification results over small and private databases. In this paper we analyze the performance of several state-of-the-art systems for doodle verification, using the recently acquired DooDB database, which is publicly available. Several algorithms are tested, from the fields of gesture recognition and doodle and signature verification. A comparative study of their performance is done, and future research directions are pointed out.

This work studies the performance of a state-of-the-art fingerprint recognition technology, in several practical scenarios of interest in forensic casework. First, the differences in performance between manual and automatic minutiae extraction for latent fingerprints are presented. Then, automatic minutiae extraction is analyzed using three different types of fingerprints: latent, rolled and plain. The experiments are carried out using a database of latent finger marks and fingerprint impressions from real forensic cases. The results show high performance degradation in automatic minutiae extraction compared to manual extraction by human experts. Moreover, high degradation in performance on latent finger marks can be observed in comparison to fingerprint impressions.

A new multimodal biometric database designed and acquired within the framework of the European BioSecure Network of Excellence is presented. It is comprised of more than 600 individuals acquired simultaneously in three scenarios: 1) over the Internet, 2) in an office environment with desktop PC, and 3) in indoor/outdoor environments with mobile portable hardware. The three scenarios include a common part of audio/video data. Also, signature and fingerprint data have been acquired both with desktop PC and mobile portable hardware. Additionally, hand and iris data were acquired in the second scenario using desktop PC. Acquisition has been conducted by 11 European institutions. Additional features of the BioSecure Multimodal Database (BMDB) are: two acquisition sessions, several sensors in certain modalities, balanced gender and age distributions, multimodal realistic scenarios with simple and quick tasks per modality, cross-European diversity, availability of demographic data, and compatibility with other multimodal databases. The novel acquisition conditions of the BMDB allow us to perform new challenging research and evaluation of either monomodal or multimodal biometric systems, as in the recent BioSecure Multimodal Evaluation campaign. A description of this campaign including baseline results of individual modalities from the new database is also given. The database is expected to be available for research purposes through the BioSecure Association during 2008.

As biometric technology is increasingly deployed, it will be common to replace parts of operational systems with newer designs. The cost and inconvenience of reacquiring enrolled users when a new vendor solution is incorporated makes this approach difficult and many applications will require to deal with information from different sources regularly. These interoperability problems can dramatically affect the performance of biometric systems and thus, they need to be overcome. Here, we describe and evaluate the ATVS-UAM fusion approach submitted to the quality-based evaluation of the 2007 BioSecure Multimodal Evaluation Campaign, whose aim was to compare fusion algorithms when biometric signals were generated using several biometric devices in mismatched conditions. Quality measures from the raw biometric data are available to allow system adjustment to changing quality conditions due to device changes. This system adjustment is referred to as quality-based conditional processing. The proposed fusion approach is based on linear logistic regression, in which fused scores tend to be log-likelihood-ratios. This allows the easy and efficient combination of matching scores from different devices assuming low dependence among modalities. In our system, quality information is used to switch between different system modules depending on the data source (the sensor in our case) and to reject channels with low quality data during the fusion. We compare our fusion approach to a set of rule-based fusion schemes over normalized scores. Results show that the proposed approach outperforms all the rule-based fusion schemes. We also show that with the quality-based channel rejection scheme, an overall improvement of 25% in the equal error rate is obtained.

Recent years have seen the rapid spread of biometric technologies for automatic people recognition. However, security and privacy issues still represent the main obstacles for the deployment of biometric-based authentication systems. In this paper, we propose an approach, which we refer to as BioConvolving, that is able to guarantee security and renewability to biometric templates. Specifically, we introduce a set of noninvertible transformations, which can be applied to any biometrics whose template can be represented by a set of sequences, in order to generate multiple transformed versions of the template. Once the transformation is performed, retrieving the original data from the transformed template is computationally as hard as random guessing. As a proof of concept, the proposed approach is applied to an on-line signature recognition system, where a hidden Markov model-based matching strategy is employed. The performance of a protected on-line signature recognition system employing the proposed BioConvolving approach is evaluated, both in terms of authentication rates and renewability capacity, using the MCYT signature database. The reported extensive set of experiments shows that protected and renewable biometric templates can be properly generated and used for recognition, at the expense of a slight degradation in authentication performance.

In this paper, we study the impact of an incremental level of skill in the forgeries against signature verification systems. Experiments are carried out using both off-line systems, involving the discrimination of signatures written on a piece of paper, and on-line systems, in which dynamic information of the signing process (such as velocity and acceleration) is also available. We use for our experiments the BiosecurID database, which contains both on-line and off-line versions of signatures, acquired in four sessions across a 4 month time span with incremental level of skill in the forgeries for different sessions. We compare several scenarios with different size and variability of the enrolment set, showing that the problem of skilled forgeries can be alleviated as we consider more signatures for enrolment.

A brute force attack using synthetically generated handwritten signatures is performed against a HMM-based signature recognition system. The generation algorithm of synthetic signatures is based on the spectral analysis of the trajectory functions and has proven to produce very realistic results. The experiments are carried out by attacking real signature models from the MCYT database (which comprises 8,250 signature samples from 330 users). Results show that such an attack is feasible, thus arising the necessity of introducing countermeasures against this type of vulnerability in real applications.

A novel scheme to generate multiple synthetic samples from a real on-line handwritten signature is proposed. The algorithm models a transmission channel which introduces a certain distortion into the real signature to produce the different synthetic samples. The method is used to increase the amount of data of the clients enrolling on a state-of-the-art HMM-based signature verification system. The enhanced enrollment results in performance improve up to70% between the case in which only one real sample of the user was available for the training, and the case where the proposed algorithm was used to generate additional synthetic training data.

Automatically verifying the identity of a person by means of biometrics (e.g., face and fingerprint) is an important application in our day-to-day activities such as accessing banking services and security control in airports. To increase the system reliability, several biometric devices are often used. Such a combined system is known as a multimodal biometric system. This paper reports a benchmarking study carried out within the framework of the BioSecure DS2 (Access Control) evaluation campaign organized by the University of Surrey, involving face, fingerprint, and iris biometrics for person authentication, targeting the application of physical access control in a medium-size establishment with some 500 persons. While multimodal biometrics is a well-investigated subject in the literature, there exists no benchmark for a fusion algorithm comparison. Working towards this goal, we designed two sets of experiments: quality-dependent and cost-sensitive evaluation. The quality-dependent evaluation aims at assessing how well fusion algorithms can perform under changing quality of raw biometric images principally due to change of devices. The cost-sensitive evaluation, on the other hand, investigates how well a fusion algorithm can perform given restricted computation and in the presence of software and hardware failures, resulting in errors such as failure-to-acquire and failure-to-match. Since multiple capturing devices are available, a fusion algorithm should be able to handle this nonideal but nevertheless realistic scenario. In both evaluations, each fusion algorithm is provided with scores from each biometric comparison subsystem as well as the quality measures of both the template and the query data. The response to the call of the evaluation campaign proved very encouraging, with the submission of 22 fusion systems. To the best of our knowledge, this campaign is the first attempt to benchmark quality-based multimodal fusion algorithms. In the presence of changing image quality which may be due to a change of acquisition devices and/or device capturing configurations, we observe that the top performing fusion algorithms are those that exploit automatically derived quality measurements. Our evaluation also suggests that while using all the available biometric sensors can definitely increase the fusion performance, this comes at the expense of increased cost in terms of acquisition time, computation time, the physical cost of hardware, and its maintenance cost. As demonstrated in our experiments, a promising solution which minimizes the composite cost is sequential fusion, where a fusion algorithm sequentially uses match scores until a desired confidence is reached, or until all the match scores are exhausted, before outputting the final combined score.

A biometric template protection system for dynamic signature verification is presented. The approach uses auxiliary (helper) data that allows the matching with secure templates but do not provide information to a potential attacker. The performance of the proposed system is evaluated using the MCYT signature database comprising 330 users, with 25 genuine signatures and 25 skilled forgeries per user. The results show similar performance compared to the baseline unprotected system. However, the security of the proposed system against attacks to the template database is significantly higher.

A performance and robustness study for on-line signature verification is presented. Experiments are carried out on the MCYT database comprising 16,500 signatures from 330 subjects, which are parameterized by means of a 100-feature set which can be divided into four different groups according to the signature information they contain, namely: (i) time, (ii) speed and acceleration, (iii) direction, and (iv) geometry. The SFFS feature selection algorithm is used to search for the best performing feature subsets under the skilled and random forgeries scenarios, and to find the most robust subsets against a hill-climbing attack. Comparative experiments are given, where it is shown that the most discriminant parameters are those regarding geometry information, while the most robust are the time related features.

This work reports a preliminary study on the vulnerability evaluation of fingerprint verification systems to direct attacks carried out with fake fingertips created from minutiae templates. The attack is performed by presenting to the acquisition sensor a fake fingertip generated from an image reconstructed from a compromised minutiae-based template. Experiments carried out against a state-of-the-art fingerprint recognition algorithm show that the proposed attack scheme is definitely feasible and highlight a potential security threat in the use of non-encrypted minutiae-based templates.

The proliferation of handheld devices such as PDAs and smart phones represents a new scenario for automatic signature verification. Traditionally, research on signature verification has been carried out employing signatures acquired using digitizing tablets or Tablet-PCs. In this paper we study the effects of the mobile acquisition conditions and we analyze the considerations that must be taken in the new handheld scenario. A signature verification system adapted to handheld devices via feature selection is proposed and a systematic comparison with a traditional pen tablet-based system is performed. The system is combined with another based on hidden Markov models using score fusion. Results confirm an increased signature variability in the case of handheld devices.

Signal-quality awareness has been found to increase recognition rates and to support decisions in multisensor environments significantly. Nevertheless, automatic quality assessment is still an open issue. Here, we study the orientation tensor of fingerprint images to quantify signal impairments, such as noise, lack of structure, blur, with the help of symmetry descriptors. A strongly reduced reference is especially favorable in biometrics, but less information is not sufficient for the approach. This is also supported by numerous experiments involving a simpler quality estimator, a trained method (NFIQ), as well as the human perception of fingerprint quality on several public databases. Furthermore, quality measurements are extensively reused to adapt fusion parameters in a monomodal multialgorithm fingerprint recognition environment. In this study, several trained and nontrained score-level fusion schemes are investigated. A Bayes-based strategy for incorporating experts' past performances and current quality conditions, a novel cascaded scheme for computational efficiency, besides simple fusion rules, is presented. The quantitative results favor quality awareness under all aspects, boosting recognition rates and fusing differently skilled experts efficiently as well as effectively (by training).

In the present contribution, the applicability of off-line handwritten signatures to the fuzzy vault construction is studied. Feature extraction is based on quantized maxima and minima from upper and lower envelopes of the signature. Baseline results are reported for skilled and random forgeries of the MCYT off-line signature database, showing that the proposed scheme is suitable for signers with good separability between genuine signatures and skilled forgeries.

In this work, resampling techniques and interpolation are applied to on-line signatures. Their effect on the performance of an on-line signature system using hidden Markov models is studied. The presented techniques are based on linear interpolation and Catmull-Rom cubic splines. Experimental results are provided on the MCYT database comprising 16,500 signatures from 330 subjects and as many skilled forgeries. Our approach allows to reduce the sampling rate of the on-line signature capture system, leading to the reduction of storage resource requirements and increased simplicity without compromising and even enhancing the system performance. This also leads to the discussion of which is the minimum sampling rate for HMM-based dynamic signature verification.

Performance in terms of accuracy is one of the most important goal of a biometric system. Hence, having a measure which is able to predict the performance with respect to a particular sample of interest is specially useful, and can be exploited in a number of ways. In this paper, we present two automatic measures for predicting the performance in off-line signature verification. Results obtained on a sub-corpus of the MCYT signature database confirms a relationship between the proposed measures and system error rates measured in terms of equal error rate (EER), false acceptance rate (FAR) and false rejection rate (FRR).

One of the open issues in fingerprint verification is the lack of robustness against image-quality degradation. Poor-quality images result in spurious and missing features, thus degrading the performance of the overall system. Therefore, it is important for a fingerprint recognition system to estimate the quality and validity of the captured fingerprint images. In this work, we review existing approaches for fingerprint image-quality estimation, including the rationale behind the published measures and visual examples showing their behavior under different quality conditions. We have also tested a selection of fingerprint image-quality estimation algorithms. For the experiments, we employ the BioSec multimodal baseline corpus, which includes 19 200 fingerprint images from 200 individuals acquired in two sessions with three different sensors. The behavior of the selected quality measures is compared, showing high correlation between them in most cases. The effect of low-quality samples in the verification performance is also studied for a widely available minutiae-based fingerprint matching system.

Information fusion in fingerprint recognition has been studied in several papers. However, only a few papers have been focused on sensor interoperability and sensor fusion. In this paper, these two topics are studied using a multisensor database acquired with three different fingerprint sensors. Authentication experiments using minutiae and ridge-based matchers are reported. Results show that the performance drops dramatically when matching images from different sensors. We have also observed that fusing scores from different sensors results in better performance than fusing different instances from the same sensor

Score normalization methods in biometric verification, which encompass the more traditional user-dependent decision thresholding techniques, are reviewed from a test hypotheses point of view. These are classified into test dependent and target dependent methods. The focus of the paper is on target dependent score normalization techniques, which are further classified into impostor-centric, target-centric, and target-impostor methods. These are applied to an on-line signature verification system on signature data from the First International Signature Verification Competition (SVC 2004). In particular, a target-centric technique based on the cross-validation procedure provides the best relative performance improvement testing both with skilled (19%) and random forgeries (53%) as compared to the raw verification performance without score normalization (7.14% and 1.06% Equal Error Rate for skilled and random forgeries, respectively).

A novel strategy for combining general and user-dependent knowledge in a multimodal biometric verification system is presented. It is based on SVM classifiers and trade-off coefficients introduced in the standard SVM training problem. Experiments are reported on a bimodal biometric system based on fingerprint and on-line signature traits. A comparison between three fusion strategies, namely user-independent, user-dependent and the proposed adapted user-dependent, is carried out. As a result, the suggested approach outperforms the former ones. In particular, a highly remarkable relative improvement of 68% in the EER with respect to the user-independent approach is achieved. The severe and very common problem of training data scarcity in the user-dependent strategy is also relaxed by the proposed scheme, resulting in a relative improvement of 40% in the EER compared to the raw user-dependent strategy.

This paper proposes a support vector machine (SVM) based combining scheme that incorporates ideolectal and acoustic characteristics for speaker recognition. Two statistical model paradigms, namely GMM for acoustic modeling and bigrams for language modeling, provide multilevel speaker information that affords a better classification performance when SVM-based fusion is accomplished. This combining approach is useful for all speaker recognition tasks where a considerable amount of data is available. Motivated by the absence of Spanish databases that made feasible our research experiments, more than nine hours of Spanish conversational speech was collected and manually transcribed from broadcasted radio talk shows.

We show how any speaker recognition system can be adapted to provide its results according to the Bayesian approach for evidence analysis and forensic reporting. This approach, firmly established in other forensic areas as fingerprint, DNA or fiber analysis, suits the needs of both the court and the forensic scientist. We show the inadequacy of the classical approach to forensic reporting because of the use of thresholds and the suppression of the prior probabilities related to the case. We also show how to assess the performance of those forensic systems through Tippet plots. Finally, an example is shown using NIST-Ahumada eval'2001 data, where the speaker recognition abilities of our system are assessed through DET plots, using then these raw scores as evidences into the forensic system, where relative to populations we will obtain the corresponding likelihood ratios values, which are assessed through Tippet (1968) plots.

This paper is focused on algorithmic issues for biometric face verification (i.e., given an image of the face and an identity claim, decide whether they correspond to each other or not). Several alternatives for geometric normalization of images, photometric normalization, dimensionality reduction and similarity measures are proposed and compared using the XM2VTS database and the associated Lausanne protocol [K. Messer et al., 1999], [J. Luettin et al., 1998]. Experiments under this particular framework show that best verification results are obtained when holistic approaches for face recognition (such as eigenfaces or fisherfaces) are combined with techniques traditionally associated to local feature-based approaches, such as Gabor decompositions.

The aim of this paper, regarding multimodal biometric verification, is twofold: on one hand, to compare experimentally a selection of them using as monomodal baseline systems as our template-based face, minutiae-based fingerprint and HMM-based on-line signature verification systems on the MCYT multimodal database. A new strategy is proposed and discussed in order to compute a multimodal combined score by means of support vector machine (SVM) classifiers.

This paper proposed a support vector machine (SVM) based combining scheme that incorporates idiolectal and acoustic characteristics for speaker recognition. Two statistical model paradigms, namely GMM for acoustic modeling and bigrams for language modeling, provide multilevel speaker information that affords a better classification performance when SVM-based fusion is accomplished. This combining approach is useful for all speaker recognition tasks where a considerable amount of data is available. Motivated by the absence of Spanish databases that made feasible our research experiments, more than nine hours of Spanish conversational speech was collected and manually transcribed from broadcasted radio talk shows.

An efficient segmentation technique of regular shaped sudden temporal intensity changes in image sequences is presented. It is based on morphological analysis of binarized difference images, and its parameters are dynamically modified according to an efficient estimation of the global motion of the scene. Motion analysis is carried out with the combination of a multiresolution flow estimation technique and a closed formula over the resulting motion vectors based on a simplified motion model. Specially suited for applications demanding a temporal reference, the new segmentation technique will be described in the context of a time stamp detection application.