Urban waste management remains a critical challenge for the development of smart cities. Despite the growing number of litter detection datasets, the problem of monitoring overflowing waste containers — particularly from images captured by garbage trucks — has received little attention. While existing datasets are valuable, they often lack annotations for specific container tracking or are captured in static, decontextualized environments, limiting their utility for real-world logistics. To address this gap, we present StreetView-Waste, a comprehensive dataset of urban scenes featuring litter and waste containers. The dataset supports three key evaluation tasks: (1) waste container detection, (2) waste container tracking, and (3) waste overflow segmentation. Alongside the dataset, we provide baselines for each task by benchmarking state-of-the-art models in object detection, tracking, and segmentation. Additionally, we enhance baseline performance by proposing two complementary strategies: a heuristic-based method for improved waste container tracking and a model-agnostic framework that leverages geometric priors to refine litter segmentation. Our experimental results show that while fine-tuned object detectors achieve reasonable performance in detecting waste containers, baseline tracking methods struggle to accurately estimate their number; however, our proposed heuristics reduce the mean absolute counting error by 79.6%. Similarly, while segmenting amorphous litter is challenging, our geometry-aware strategy improves segmentation [email protected] by 27% on lightweight models, demonstrating the value of multimodal inputs for this task. Ultimately, StreetView-Waste provides a challenging benchmark to encourage research into real-world perception systems for urban waste management.

Understanding and justifying the decisions generated by automated recognizers has been a focal point of biometrics research, driven by the need for systems that are not only effective but transparent and accountable. Clear explanations improve user trust and system credibility, while addressing the growing concerns regarding the role of AI in our daily lives. In this work, we propose a framework that integrates biometric recognition with visual interpretations of the features that contribute the most to a match/non-match decision. Our method leverages adversarial generative models to create a set composed exclusively of ``genuine" image pairs. From these, the most similar candidates to a given query are identified and, assuming enough similarity in phase between the query and the retrieved pairs, the pixel-wise differences highlight the image regions that supported the decision. Comparative evaluations against established interpretability techniques (SHAP, LIME, and Saliency Maps) as well as other state-of-the-art fine-grained visual recognizers demonstrate that our framework delivers intuitive visual explanations without compromising recognition performance. These findings underscore our method's potential to enhance the transparency and credibility of biometric systems while maintaining high accuracy.

Bias analysis for synthetic face detection is bound to become a critical topic in the coming years. Although many detection models have been developed and several datasets have been released to reliably identify synthetic content, one crucial aspect has been largely overlooked: these models and training datasets can be biased, leading to failures in detection for certain demographic groups and raising significant social, legal, and ethical issues. In this work, we introduce an evaluation framework to contribute to the analysis of bias of synthetic face detectors with respect to several facial attributes. This framework exploits synthetic data generation, with evenly distributed attribute labels, for mitigating any skew in the data that could otherwise influence the outcomes of bias analysis. We build on the proposed framework to provide an extensive case study of the bias level of five state-of-the-art detectors in synthetic datasets with 25 controlled facial attributes. While the results confirm that, in general, synthetic face detectors are biased towards the presence/absence of specific facial attributes, our study also sheds light on the origins of the observed bias through the analysis of the correlations with the balancing of facial attributes in the training sets of the detectors, and the analysis of detectors activation maps in image pairs with controlled attribute modifications.

Person re-identification (ReID) across aerial and ground vantage points has become crucial for large-scale surveillance and public safety applications. Although significant progress has been made in ground-only scenarios, bridging the aerial-ground domain gap remains a formidable challenge due to extreme viewpoint differences, scale variations, and occlusions. Building upon the achievements of the AG-ReID 2023 Challenge, this paper introduces the AG-VPReID 2025 Challenge—the first large-scale video-based competition focused on high-altitude (80–120 m) aerial-ground person ReID. Constructed on the new AG-VPReID dataset with 3,027 identities, over 13,500 tracklets, and approximately 3.7 million frames captured from UAVs, CCTV, and wearable cameras, the challenge featured four international teams. These teams developed solutions ranging from multi-stream architectures to transformer-based temporal reasoning and physics-informed modeling. The leading approach, X-TFCLIP from UAM, attained 72.28% Rank-1 accuracy in the aerial-to-ground ReID setting and 70.77% in the ground-to-aerial ReID setting, surpassing existing baselines while highlighting the dataset’s complexity. For additional details, please refer to the official website at https://agvpreid25.github.io.

Reliable aerial-ground video-based person re-identification (ReID) remains a challenge due to severe changes in data quality and features, such as viewpoint disparities, resolution drops, and cross-camera appearance inconsistency. This paper presents VM-TAPS, a lightweight and modular extension to the well-known TF-CLIP framework, designed to increase the robustness of ReID, without requiring end-to-end backbone retraining. When compared to its ancestor, VM-TAPS' novelties are five-fold: 1) View-Specific Processing Layers to normalize camera-dependent biases; 2) Scale-Aware Feature Adaptation for resolution-invariant feature fusion; 3) a View-Aware Memory Bank enabling long-range identity context; 4) a Motion Pattern Analyzer capturing temporal dynamics; and (5) Cross-View Interaction Modules that harmonize multi-view feature spaces. Despite adding fewer than two million parameters, VM-TAPS achieves +4.97% Rank-1 and +3.08% mAP gains over TF-CLIP on the challenging AG-VPReID2025 benchmark. At 80m and 120m altitudes, it sets a new performance baseline of 73.68%/75.73% and 69.45%/71.63% (Rank-1/mAP), respectively. All components are trained with frozen CLIP visual encoders in the early stages, enabling efficient and stable convergence. Our results support that the carefully disentanglement of viewpoint, scale, motion and memory factors substantially increases the robustness of cross-view ReID under real-world conditions.

State-of-the-art Active Speaker Detection (ASD) approaches mainly use audio and facial features as input. However, the main hypothesis in this paper is that body dynamics is also highly correlated to "speaking" (and "listening") actions and should be particularly useful in wild conditions (e.g., surveillance settings), where face cannot be reliably accessed. We propose ASDnB, a model that singularly integrates face with body information by merging the inputs at different steps of feature extraction. Our approach splits 3D convolution into 2D and 1D to reduce computation cost without loss of performance, and is trained with adaptive weight feature importance for improved complement of face with body data. Our experiments show that ASDnB achieves state-of-the-art results in the benchmark dataset (AVA-ActiveSpeaker), in the challenging data of WASD, and in cross-domain settings using Columbia. This way, ASDnB can perform in multiple settings, which is positively regarded as a strong baseline for robust ASD models (code available at https://github.com/Tiago-Roxo/ASDnB).

Sketch understanding poses unique challenges for general-purpose vision algorithms due to the sparse and semantically ambiguous nature of sketches. This paper introduces a novel approach to biometric recognition that leverages sketch-based representations of ears, a largely unexplored but promising area in biometric research. Specifically, we address the “ sketch-2-image ” matching problem by synthesizing ear sketches at multiple abstraction levels, achieved through a triplet-loss function adapted to integrate these levels. The abstraction level is determined by the number of strokes used, with fewer strokes reflecting higher abstraction. Our methodology combines sketch representations across abstraction levels to improve robustness and generalizability in matching. Extensive evaluations were conducted on four ear datasets (AMI, AWE, IITDII, and BIPLab) using various pre-trained neural network backbones, showing consistently superior performance over state-of-the-art methods. These results highlight the potential of ear sketch-based recognition, with cross-dataset tests confirming its adaptability to real-world conditions and suggesting applicability beyond ear biometrics. • Sketch-Based Datasets Expansion: Leveraging CLIPasso, we transformed ear images into sketches at various abstraction levels, preserving key features and introducing a novel data representation for biometric analysis. • Triplet-Loss Function Enhancement: Adapting the triplet-loss function to incorporate multiple abstraction levels significantly improves recognition performance over traditional methods. • Comparative Backbone Analysis: An exhaustive evaluation of different backbones highlights their effectiveness in sketch-based ear recognition, guiding advancements in biometric technologies. • Cross-Dataset Generalizability Tests: Training on combined datasets and testing on distinct ones validate our approach’s robustness and effectiveness against unseen data distributions.

Large vision models based in deep learning architectures have been consistently advancing the state-of-the-art in biometric recognition. However, three weaknesses are commonly reported for such kind of approaches: 1) their extreme demands in terms of learning data; 2) the difficulties in generalising between different domains; and 3) the lack of interpretability/explainability, with biometrics being of particular interest, as it is important to provide evidence able to be used for forensics/legal purposes (e.g., in courts).To the best of our knowledge, this paper describes the first recognition framework/strategy that aims at addressing the three weaknesses simultaneously. At first, it relies exclusively in synthetic samples for learning purposes. Instead of requiring a large amount and variety of samples for each subject, the idea is to exclusively enroll a 3D point cloud per identity. Then, using generative strategies, we synthesize a very large (potentially infinite) number of samples, containing all the desired covariates (poses, clothing, distances, perspectives, lighting, occlusions,…). Upon the synthesizing method used, it is possible to adapt precisely to different kind of domains, which accounts for generalization purposes. Such data are then used to learn a model that performs local registration between image pairs, establishing positive correspondences between body parts that are the key, not only to recognition (according to cardinality and distribution), but also to provide an interpretable description of the response (e.g.: "both samples are from the same person, as they have similar facial shape, hair color and legs thickness").

Video anomaly detection (VAD) has been densely explored in the last few years, mostly in single-camera scenarios. Despite significant advancements in this field, effectiveness is still seriously compromised in challenging environments (e.g., varying lighting conditions, under partial occlusions, and in crowded environments). For the sake of affordable data annotation, the most relevant methods assume the weakly supervised paradigm, where the label is available only at the video level (WS-VAD). Also, these methods are conventionally designed for single-camera mode and do not consider the multi-view information yielded from overlapping surveillance cameras, which is very common in practical scenarios. In this work, we started by systematically evaluating the WS-VAD performance that can be attained when different camera combinations are used as data sources. Interestingly, we observed that the rule "the more cameras, the better" should not be assumed, as there were always particular subsets of cameras that consistently outperformed the remaining configurations. Upon these conclusions, we present a semi-automated procedure to identify the optimal camera sources based on the image features/characteristics (distance, pose, and lighting) each one is capturing. Extensive experiments were carried out in three overlapping multi-camera datasets, which suggest that 1) multi-camera schemes consistently outperform single-camera methods and - most interestingly - 2) the correlation between the data acquired by the different cameras severely impacts performance, turning the selection of cameras a crucial step in VAD. Our findings open an intriguing research topic about methods/algorithms that filter out/select the camera sources we should use in overlapping camera scenarios.

Dimensionality reduction is a crucial step in data preprocessing, particularly for high-dimensional datasets, where the excessive number of features increases the risk of overfitting in machine learning models. Traditional dimensionality reduction methods rely on statistical associations or the relative position of the feature embeddings in the hyper-space to map original features to a compact subspace that preserves the most relevant information of the data. However, these methods fail to capture the causal relationships among variables during the transfor-mation process, leading to a loss of structural coherence of the data in low-dimensional spaces. By employing causal discovery and causal inference, it is possible to simplify these problems, effectively merging critical features while reducing both complex-ity and dimensionality. Our paper introduces a novel approach, Causal Feature Construction via Average Treatment Effect (CFC-ATE), which leverages causal discovery and inference to create more interpretable and reliable features for predictive modeling. Our methodology consists of the following phases: i) leveraging the causal structure of data through the inference of the causal graph. ii) transforming features through the use of the average treatment effect conditioned on the causal structure of the data. The experiments on diverse real-world datasets and synthetic datasets demonstrate the effectiveness of CFC-ATE in improving model performance by comparing it with three methods of feature selection and three benchmark dimensionality reduction techniques.

Human re-identification (re-ID) is nowadays among the most popular topics in computer vision, due to the increasing importance given to safety/security in modern societies. Being expected to sun in totally uncontrolled data acquisition settings (e.g., visual surveillance) automated re-ID not only depends on various factors that may occur in non-controlled data acquisition settings, but - most importantly - performance varies with respect to different subject features (e.g., gender, height, ethnicity, clothing, and action being performed), which may result in highly biased and undesirable automata. While many efforts have been putted in increase the robustness of identification to uncontrolled settings, a systematic assessment of the actual variations in performance with respect to each subject feature remains to be done. Accordingly, the contributions of this paper are threefold: 1) we report the correlation between the performance of three state-of-the-art re-ID models and different subject features; 2) we discuss the most concerning features and report valuable insights about the roles of the various features in re-ID performance, which can be used to develop more effective and unbiased re-ID systems; and 3) we leverage the concept of biometric menagerie, in order to identify the groups of individuals that typically fall into the most common menagerie families (e.g., goats, lambs, and wolves). Our findings not only contribute to a better understanding of the factors affecting re-ID performance, but also may offer practical guidance for researchers and practitioners concerned on human re-identification development.

The decisions behind the mechanics of a biometric verification system based on Machine Learning (ML) are difficult to comprehend. Although there is now well-established research in various fields of application, such as health or justice, the use of ML-based methods is accompanied by a lack of confidence that results in their limited use. The explainability of a ML system and the comprehension of what lies behind its prediction is one of the numerous characteristics that define “trust” in these systems. Over the years, face-based biometric authentication has been the subject of extensive research in both academia and industry. However, existing biometric authentication systems still have problems regarding accuracy, robustness and, explainability. Still lacking in the literature is a comprehensive examination of the use of post-hoc explainability techniques for such systems. Cognitive neuroscience has always been interested in the method by which people perceive faces; local elements such as the nose, eyes, and mouth are critical to the perception and recognition of a face. In this work, starting from this assumption, we propose a framework of visual and textual explainability based on the parts of a face by analyzing them with respect to the facial attributes reported in the CelebA dataset. The primary objective is to be able to explain why two pictures of different subjects are distinct. This is done by sinthesizing pairs of images that illustrate how dissimilar the various parts of the face under investigation are and incisive and direct textual explanations of the distinguishing features are generated. A further study analyzes an interpretable mapping between the semantic space of the text and the space of the image.

Self-supervised learning (SSL) is a promising method for gaining perception and common sense from unlabelled data. Existing approaches to analyzing human body skeletons address the problem similar to SSL models for image and video understanding, but pixel data is far more challenging than coordinates. This paper presents ATOM, an SSL model designed for skeleton-based data analysis. Unlike video-based SSL approaches, ATOM leverages atomic movements within skeleton actions to achieve a more fine-grained representation. The proposed architecture predicts the action order at the frame level, leading to improved perceptions and representations of each action. ATOM outperforms state-of-the-art approaches in two well-known datasets (NTU RGB + D and NTU-120 RGB + D), and its weight transferability enables performance improvements on supervised and semi-supervised tasks, up to 4.4% (3.3% p.p.) and 14.1% (6.3% p.p.), respectively, in Top-1 Accuracy.

Synthesising the spatial and temporal dynamics of the human body skeleton remains a challenging task, not only in terms of the quality of the generated shapes, but also of their diversity, particularly to synthesise realistic body movements of a specific action (action conditioning). In this paper, we propose Kinetic-GAN, a novel architecture that leverages the benefits of Generative Adversarial Networks and Graph Convolutional Networks to synthesise the kinetics of the human body. The proposed adversarial architecture can condition up to 120 different actions over local and global body movements while improving sample quality and diversity through latent space disentanglement and stochastic variations. Our experiments were carried out in three well-known datasets, where Kinetic-GAN notably surpasses the state-of-the-art methods in terms of distribution quality metrics while having the ability to synthesise more than one order of magnitude regarding the number of different actions. Our code and models are publicly available at https://github.com/DegardinBruno/Kinetic-GAN.

We propose a reversible face de-identification method for video surveillance data, where landmark-based techniques cannot be reliably used. Our solution generates a photorealistic de-identified stream that meets the data protection regulations and can be publicly released under minimal privacy concerns. Notably, such stream still encapsulates the information required to later reconstruct the original scene, which is useful for scenarios, such as crime investigation, where subjects identification is of most importance. Our learning process jointly optimizes two main components: 1) apublicmodule, that receives the raw data and generates the de-identified stream; and 2) aprivatemodule, designed for security authorities, that receives the public stream and reconstructs the original data, disclosing the actual IDs of the subjects in a scene. The proposed solution is landmarks-free and uses a conditional generative adversarial network to obtain synthetic faces that preserve pose, lighting, background information and even facial expressions. Also, we keep full control over the set of soft facial attributes to be preserved/changed between the raw/de-identified data, which extends the range of applications for the proposed solution. Our experiments were conducted in three visual surveillance datasets (BIODI, MARS and P-DESTRE) plus one video face data set (YouTube Faces), showing highly encouraging results. The source code is available athttps://github.com/hugomcp/uu-net.

For over three decades, the Gabor-basedIrisCodeapproach has been acknowledged as thegold standardfor iris recognition, mainly due to the high entropy and binary nature of its signatures. This method is highly effective in large scale environments (e.g., national ID applications), where millions of comparisons per second are required. However, it is known that non-linear deformations in the iris texture, with fibers vanishing/appearing in response to pupil dilation/contraction, often flip the signature coefficients, being the main cause for the increase of false rejections. This paper addresses this problem, describing a customised Deep Learning (DL) framework that: 1) virtually emulates theIrisCodefeature encoding phase; while also 2) detects the deformations in the iris texture that may lead to bit flipping, and autonomously adapts the filter configurations for such cases. The proposed DL architecture seamlessly integrates the Gabor kernels that extract theIrisCodeand a multi-scale texture analyzer, from where the biometric signatures yield. In this sense, it can be seen as anadaptive encoderthat is fully compatible to theIrisCodeapproach, while increasing the permanence of the signatures. The experiments were conducted in two well known datasets (CASIA-Iris-Lamp and CASIA-Iris-Thousand) and showed a notorious decrease of the mean/standard deviation values of thegenuinesdistribution, at expenses of only a marginal deterioration in theimpostorsscores. The resulting decision environments consistently reduce the levels of false rejections with respect to the baseline for most operating levels (e.g., over 50% at 1e-3FAR values). The source code of theDeepGaborencoder is available at: https://github.com/hugomcp/DeepGabor.

Over the years, unmanned aerial vehicles (UAVs) have been regarded as a potential solution to surveil public spaces, providing a cheap way for data collection, while covering large and difficult-to-reach areas. This kind of solutions can be particularly useful to detect, track and identify subjects of interest in crowds, for security/safety purposes. In this context, various datasets are publicly available, yet most of them are only suitable for evaluating detection, tracking and short-term re-identification techniques. This paper announces the free availability of the P-DESTRE dataset, the first of its kind to provide video/UAV-based data for pedestrian long-term re-identification research, with ID annotations consistent across data collected in different days. As a secondary contribution, we provide the results attained by the state-of-the-art pedestrian detection, tracking, short/long term re-identification techniques in well-known surveillance datasets, used as baselines for the corresponding effectiveness observed in the P-DESTRE data. This comparison highlights the discriminating characteristics of P-DESTRE with respect to similar sets. Finally, we identify the most problematic data degradation factors and co-variates for UAV-based automated data analysis, which should be considered in subsequent technologic/conceptual advances in this field. The dataset and the full specification of the empirical evaluation carried out are freely available at http://p-destre.di.ubi.pt/.

This article describes one objective function for learning semantically coherent feature embeddings in multi-output classification problems, i.e., when the response variables have dimension higher than one. Such coherent embeddings can be used simultaneously for different tasks, such as identity retrieval and soft biometrics labelling. We propose a generalization of the triplet loss that: 1) defines a metric that considers the number of agreeing labels between pairs of elements; 2) introduces the concept of similar classes, according to the values provided by the metric; and 3) disregards the notion of anchor, sampling four arbitrary elements at each time, from where two pairs are defined. The distances between elements in each pair are imposed according to their semantic similarity (i.e., the number of agreeing labels). Likewise the triplet loss, our proposal also privileges small distances between positive pairs. However, the key novelty is to additionally enforce that the distance between elements of any other pair corresponds inversely to their semantic similarity. The proposed loss yields embeddings with a strong correspondence between the classes centroids and their semantic descriptions. In practice, it is a natural choice to jointly infer coarse (soft biometrics) + fine (ID) labels, using simple rules such as k-neighbours. Also, in opposition to its triplet counterpart, the proposed loss appears to be agnostic with regard to demanding criteria for mining learning instances (such as the semi-hard pairs). Our experiments were carried out in five different datasets (BIODI, LFW, IJB-A, Megaface and PETA) and validate our assumptions, showing results that are comparable to the state-of-the-art in both the identity retrieval and soft biometrics labelling tasks.

The hair appearance is among the most valuable soft biometric traits when performing human recognition at-a-distance. Even in degraded data, the hair's appearance is instinctively used by humans to distinguish between individuals. In this paper we propose a multi-task deep neural model capable of segmenting the hair region, while also inferring the hair color, shape and style, all from in-the-wild images. Our main contributions are two-fold: 1) the design of an all-in-one neural network, based on depthwise separable convolutions to extract the features; and 2) the use convolutional feature masking layer as an attention mechanism that enforces the analysis only within the `hair' regions. In a conceptual perspective, the strength of our model is that the segmentation mask is used by the other tasks to perceive - at feature-map level - only the regions relevant to the attribute characterization task. This paradigm allows the network to analyze features from nonrectangular areas of the input data, which is particularly important, considering the irregularity of hair regions. Our experiments showed that the proposed approach reaches a hair segmentation performance comparable to the state-of-the-art, having as main advantage the fact of performing multiple levels of analysis in a single-shot paradigm.

Having observed the unsatisfactory state-of-the-art performance in detecting abnormal events, this paper describes an iterative self-supervised learning method for such purpose. The proposed solution is composed of two experts that - at each step - find the most confidently classified instances to augment the amount of data available for the next iteration. Our contributions are four-fold: 1) we describe the iterative learning framework composed of experts working in the weak/self-supervised paradigms and providing learning data to each other, with the novel instances being filtered by a Bayesian framework; 2) upon Sultani et al. [14]'s work, we suggest a novel term the loss function that spreads the scores in the unit interval and is important for the performance of the iterative framework; 3) we propose a late decision fusion scheme, in which an ensemble of Decision Trees learned from bootstrap samples fuses the scores of the top-3 methods, reducing the EER values about 20% over the state-of-the-art; and 4) we announce the “Fights” dataset, fully annotated at the frame level, that can be freely used by the research community. The code, details of the experimental protocols and the dataset are publicly available at http://github.com/DegardinBruno/.

Ocular biometric systems working in unconstrained environments usually face the problem of small within-class compactness caused by the multiple factors that jointly degrade the quality of the obtained data. In this work, we propose an attribute normalization strategy based on deep learning generative frameworks, that reduces the variability of the samples used in pairwise comparisons, without reducing their discriminability. The proposed method can be seen as a preprocessing step that contributes for data regularization and improves the recognition accuracy, being fully agnostic to the recognition strategy used. As proof of concept, we consider the “eyeglasses” and “gaze” factors, comparing the levels of performance of five different recognition methods with/without using the proposed normalization strategy. Also, we introduce a new dataset for unconstrained periocular recognition, composed of images acquired by mobile devices, particularly suited to perceive the impact of “wearing eyeglasses” in recognition effectiveness. Our experiments were performed in two different datasets, and support the usefulness of our attribute normalization scheme to improve the recognition performance.

Caricatures refer to a representation of a person, in which the distinctive features are deliberately exaggerated, with several studies showing that humans perform better at recognizing people from caricatures than using original images. Inspired by this observation, this paper introduces the first fully automated caricature-based face recognition approach capable of working with data acquired in the wild. Our approach leverages the 3D face structure from a single 2D image and compares it with a reference model for obtaining a compact representation of face features deviations. This descriptor is subsequently deformed using a “measure locally, weight globally” strategy to resemble the caricature drawing process. The deformed deviations are incorporated in the 3D model using the Laplacian mesh deformation algorithm, and the 2D face caricature image is obtained by projecting the deformed model in the original camera view. To demonstrate the advantages of caricature-based face recognition, we train the VGG-face network from scratch using either original face images (baseline) or caricatured images and use these models for extracting face descriptors from the LFW, IJB-A, and MegaFace data sets. The experiments show an increase in the recognition accuracy when using caricatures rather than original images. Moreover, our approach achieves competitive results with the state-of-the-art face recognition methods, even without explicitly tuning the network for any of the evaluation sets.

Convolutional neural networks (CNNs) have emerged as the most popular classification models in biometrics research. Under the discriminative paradigm of pattern recognition, CNNs are used typically in one of two ways: (1) verification mode (“ are samples from the same person? ”), where pairs of images are provided to the network to distinguish between genuine and impostor instances and (2) identification mode (“ whom is this sample from? ”), where appropriate feature representations that map images to identities are found. This paper postulates a novel mode for using CNNs in biometric identification, by learning models that answer the question “ is the query's identity among this set? ”. The insight is a reminiscence of the classical Mastermind game: by iteratively analyzing the network responses when multiple random samples of k gallery elements are compared to the query, we obtain weakly correlated matching scores that, altogether, provide solid cues to infer the most likely identity. In this setting, identification is regarded as a variable selection and regularization problem, with sparse linear regression techniques being used to infer the matching probability with respect to each gallery identity. As main strength, this strategy is highly robust to outlier matching scores, which are known to be a primary error source in biometric recognition. Our experiments were carried out in full versions of two well-known irises near-infrared (CASIA-IrisV4-Thousand) and periocular visible wavelength (UBIRIS.v2) datasets, and confirm that recognition performance can be solidly boosted-up by the proposed algorithm, when compared with the traditional working modes of CNNs in biometrics.

This paper is based on a disruptive hypothesis for periocular biometrics-in visible-light data, the recognition performance is optimized when the components inside the ocular globe (the iris and the sclera) are simply discarded, and the recognizer's response is exclusively based on the information from the surroundings of the eye. As a major novelty, we describe a processing chain based on convolution neural networks (CNNs) that defines the regions-of-interest in the input data that should be privileged in an implicit way, i.e., without masking out any areas in the learning/test samples. By using an ocular segmentation algorithm exclusively in the learning data, we separate the ocular from the periocular parts. Then, we produce a large set of “multi-class” artificial samples, by interchanging the periocular and ocular parts from different subjects. These samples are used for data augmentation purposes and feed the learning phase of the CNN, always considering as label the ID of the periocular part. This way, for every periocular region, the CNN receives multiple samples of different ocular classes, forcing it to conclude that such regions should not be considered in its response. During the test phase, samples are provided without any segmentation mask and the network naturally disregards the ocular components, which contributes for improvements in performance. Our experiments were carried out in full versions of two widely known data sets (UBIRIS.v2 and FRGC) and show that the proposed method consistently advances the state-of-the-art performance in the closed-world setting, reducing the EERs in about 82% (UBIRIS.v2) and 85% (FRGC) and improving the Rank-1 over 41% (UBIRIS.v2) and 12% (FRGC).

The effectiveness of current iris recognition systems depends on the accurate segmentation and parameterisation of the iris boundaries, as failures at this point misalign the coefficients of the biometric signatures. This paper describes IRINA, an algorithm for Iris Recognition that is robust against INAccurately segmented samples, which makes it a good candidate to work in poor-quality data. The process is based in the concept of corresponding patch between pairs of images, that is used to estimate the posterior probabilities that patches regard the same biological region, even in case of segmentation errors and non-linear texture deformations. Such information enables to infer a free-form deformation field (2D registration vectors) between images, whose first and second-order statistics provide effective biometric discriminating power. Extensive experiments were carried out in four datasets (CASIA-IrisV3-Lamp, CASIA-IrisV4-Lamp, CASIA-IrisV4-Thousand and WVU) and show that IRINA not only achieves state-of-the-art performance in good quality data, but also handles effectively severe segmentation errors and large differences in pupillary dilation/constriction.

An error-correcting code (ECC) is a process of adding redundant data to a message, such that it can be recovered by a receiver even if a number of errors are introduced in transmission. Inspired by the principles of ECC, we introduce a method capable of detecting degraded features in biometric signatures by exploiting feature correlation. The main novelty is that, unlike existing biometric cryptosystems, the proposed method works directly on the biometric signature. Our approach performs a redundancy analysis of non-degraded data to build an undirected graphical model (Markov Random Field), whose energy minimization determines the sequence of degraded components of the biometric sample. Experiments carried out in different biometric traits ascertain the improvements attained when disregarding degraded features during the matching phase. Also, we stress that the proposed method is general enough to work in different classification methods, such as CNNs.

Markov Random Fields (MRFs) are a popular tool in many computer vision problems and faithfully model a broad range of local dependencies. However, rooted in the Hammersley-Clifford theorem, they face serious difficulties in enforcing the global coherence of the solutions without using too high order cliques that reduce the computational effectiveness of the inference phase. Having this problem in mind, we describe a multi-layered (hierarchical) architecture for MRFs that is based exclusively in pairwise connections and typically produces globally coherent solutions, with 1) one layer working at the local (pixel) level, modeling the interactions between adjacent image patches; and 2) a complementary layer working at the object (hypothesis) level pushing toward globally consistent solutions. During optimization, both layers interact into an equilibrium state that not only segments the data, but also classifies it. The proposed MRF architecture is particularly suitable for problems that deal with biological data (e.g., biometrics), where the reasonability of the solutions can be objectively measured. As test case, we considered the problem of hair / facial hair segmentation and labeling, which are soft biometric labels useful for human recognition in-the-wild. We observed performance levels close to the state-of-the-art at a much lower computational cost, both in the segmentation and classification (labeling) tasks.

The growing interest for mobile biometrics stems from the increasing need to secure personal data and services, which are often stored or accessed from there. Modern user mobile devices, with acquisition and computation resources to support related operations, are nowadays widely available. This makes this research topic very attracting and promising. Iris recognition plays a major role in this scenario. However, mobile biometrics still suffer from some hindering factors. The resolution of captured images and the computational power are not comparable to desktop systems yet. Furthermore, the acquisition setting is generally uncontrolled, with users who are not that expert to autonomously generate biometric samples of sufficient quality. Mobile Iris CHallenge Evaluation aims at providing a testbed to assess the progress of mobile iris recognition, and to evaluate the extent of its present limitations. This paper presents the results of the competition launched at the 2016 edition of the International Conference on Pattern Recognition (ICPR).

Soft biometrics have been emerging to complement other traits and are particularly useful for poor quality data. In this paper, we propose an efficient algorithm to estimate human head poses and to infer soft biometric labels based on the 3D morphology of the human head. Starting by considering a set of pose hypotheses, we use a learning set of head shapes synthesized from anthropometric surveys to derive a set of 3D head centroids that constitutes a metric space. Next, representing queries by sets of 2D head landmarks, we use projective geometry techniques to rank efficiently the joint 3D head centroids/pose hypotheses according to their likelihood of matching each query. The rationale is that the most likely hypotheses are sufficiently close to the query, so a good solution can be found by convex energy minimization techniques. Once a solution has been found, the 3D head centroid and the query are assumed to have similar morphology, yielding the soft label. Our experiments point toward the usefulness of the proposed solution, which can improve the effectiveness of face recognizers and can also be used as a privacy-preserving solution for biometric recognition in public environments.

The use of pan-tilt-zoom (PTZ) cameras for capturing high-resolution data of human-beings is an emerging trend in surveillance systems. However, this new paradigm entails additional challenges, such as camera scheduling, that can dramatically affect the performance of the system. In this paper, we present a camera scheduling approach capable of determining - in real-time - the sequence of acquisitions that maximizes the number of different targets obtained, while minimizing the cumulative transition time. Our approach models the problem as an undirected graphical model (Markov random field, MRF), which energy minimization can approximate the shortest tour to visit the maximum number of targets. A comparative analysis with the state-of-the-art camera scheduling methods evidences that our approach is able to improve the observation rate while maintaining a competitive tour time.

Sparse representations for classification (SRC) are considered a relevant advance to the biometrics field, but are particularly sensitive to data misalignments. In previous studies, such misalignments were compensated for by finding appropriate geometric transforms between the elements in the dictionary and the query image, which is costly in terms of computational burden. This study describes an algorithm that compensates for data misalignments in SRC in an implicit way, that is, without finding/applying any geometric transform at every recognition attempt. The authors' study is based on three concepts: (i) sparse representations; (ii) projections on orthogonal subspaces; and (iii) discriminant locality preserving with maximum margin projections. When compared with the classical SRC algorithm, apart from providing slightly better performance, the proposed method is much more robust against global/local data misalignments. In addition, it attains performance close to the state‐of‐the‐art algorithms at a much lower computational cost, offering a potential solution for real‐time scenarios and large‐scale applications.

The concept of fragility of some bits in the iris codes regards exclusively their within-class variation, i.e., the probability that they take different values in templates computed from different images of the same iris. This paper extends that concept, by noticing that a similar phenomenon occurs for the between-classes comparisons, i.e., some bits have higher probability than others of assuming a predominant value, which was observed for near-infrared and (in a more evident way) for visible wavelength data. Accordingly, we propose a new measure (bit discriminability) that considers both the within-class and between-classes variabilities, and has roots in the Fisher discriminant. Based on the bit discriminability, we compare the usefulness of the different regions of the iris for biometric recognition, with respect to multispectral data and to different filters parameterizations. Finally, we measure the amount of information lost in codes quantization, which gives insight to further research on iris matching strategies that consider both phase and magnitude. Albeit augmenting the computational burden of recognition, such kind of strategies will consistently improve performance, particularly in poor-quality data.

Using the periocular region for biometric recognition is an interesting possibility: this area of the human body is highly discriminative among subjects and relatively stable in appearance. In this paper, the main idea is that improved solutions for defining the periocular region-of-interest and better pose / gaze estimates can be obtained by segmenting (labelling) all the components in the periocular vicinity. Accordingly, we describe an integrated algorithm for labelling the periocular region, that uses a unique model to discriminate between seven components in a single-shot: iris, sclera, eyelashes, eyebrows, hair, skin and glasses. Our solution fuses texture / shape descriptors and geometrical constraints to feed a two-layered graphical model (Markov Random Field), which energy minimization provides a robust solution against uncontrolled lighting conditions and variations in subjects pose and gaze.

The concept of periocular biometrics emerged to improve the robustness of iris recognition to degraded data. Being a relatively recent topic, most of the periocular recognition algorithms work in a holistic way and apply a feature encoding/matching strategy without considering each biological component in the periocular area. This not only augments the correlation between the components in the resulting biometric signature, but also increases the sensitivity to particular data covariates. The main novelty in this paper is to propose a periocular recognition ensemble made of two disparate components: 1) one expert analyses the iris texture and exhaustively exploits the multispectral information in visible-light data and 2) another expert parameterizes the shape of eyelids and defines a surrounding dimensionless region-of-interest, from where statistics of the eyelids, eyelashes, and skin wrinkles/furrows are encoded. Both experts work on disjoint regions of the periocular area and meet three important properties. First, they produce practically independent responses, which is behind the better performance of the ensemble when compared to the best individual recognizer. Second, they do not share particularly sensitivity to any image covariate, which accounts for augmenting the robustness against degraded data. Finally, it should be stressed that we disregard information in the periocular region that can be easily forged (e.g., shape of eyebrows), which constitutes an active anticounterfeit measure. An empirical evaluation was conducted on two public data sets (FRGC and UBIRIS.v2), and points for consistent improvements in performance of the proposed ensemble over the state-of-the-art periocular recognition algorithms.

Sparse representations have been advocated as a relevant advance in biometrics research. In this paper we propose a new algorithm for fusion at the data level of sparse representations, each one obtained from image patches. The main novelties are two-fold: 1) a dictionary fusion scheme is formalised, using the l1--- minimization with the gradient projection method; 2) the proposed representation and classification method does not require the non-overlapping condition of image patches from where individual dictionaries are obtained.

Iris recognition is a popular technique for recognizing humans. However, as is the case with most biometric traits, it is difficult to collect data that are suitable for use in experiments due to three factors: 1) the substantial amount of data that is required; 2) the time that is spent in the acquisition process; and 3) the security and privacy concerns of potential volunteers. This paper describes a stochastic method for synthesizing ocular data to support experiments on iris recognition. Specifically, synthetic data are intended for use in the most important phases of those experiments: segmentation and signature encoding/matching. The resulting data have an important characteristic: they simulate image acquisition under uncontrolled conditions. We have experimentally confirmed that the proposed strategy can mimic the data degradation factors that usually result from such conditions. Finally, we announce the availability of an online platform for generating degraded synthetic ocular data. This platform is freely accessible worldwide.

Most of the methods to index iris biometric signatures were designed for decision environments with a clear separation between genuine and impostor matching scores. However, in case of less controlled data acquisition, images will be degraded and the decision environments poorly separated. This paper proposes an indexing/retrieval method for degraded images and operates at the code level, making it compatible with different feature encoding strategies. Gallery codes are decomposed at multiple scales, and according to their most reliable components at each scale, the position in an n-ary tree determined. In retrieval, the probe is decomposed similarly, and the distances to multiscale centroids are used to penalize paths in the tree. At the end, only a subset of the branches is traversed up to the last level. When compared with related strategies, the proposed method outperforms them on degraded data, particularly in the performance range most important for biometrics . Finally, according to the computational cost of the retrieval phase, the number of enrolled identities above which indexing is computationally cheaper than an exhaustive search is determined.

This paper announces and discusses the experimental results from the Noisy Iris Challenge Evaluation (NICE), an iris biometric evaluation initiative that received worldwide participation and whose main innovation is the use of heavily degraded data acquired in the visible wavelength and uncontrolled setups, with subjects moving and at widely varying distances. The NICE contest included two separate phases: 1) the NICE.I evaluated iris segmentation and noise detection techniques and 2) the NICE:II evaluated encoding and matching strategies for biometric signatures. Further, we give the performance values observed when fusing recognition methods at the score level, which was observed to outperform any isolated recognition strategy. These results provide an objective estimate of the potential of such recognition systems and should be regarded as reference values for further improvements of this technology, which-if successful-may significantly broaden the applicability of iris biometric systems to domains where the subjects cannot be expected to cooperate.

Corner detection has motivated a great deal of research and is particularly important in a variety of tasks related to computer vision, acting as a basis for further stages. In particular, the detection of eye-corners in facial images is important in applications in biometric systems and assisted- driving systems. We empirically evaluated the state-of-the-art of eye-corner detection proposals and found that they achieve satisfactory results only when dealing with high-quality data. Hence, in this paper, we describe an eye-corner detection method that emphasizes robustness, i.e., its ability to deal with degraded data, and applicability to real-world conditions. Our experiments show that the proposed method outperforms others in both noise-free and degraded data (blurred and rotated images and images with significant variations in scale), which is a major achievement.

Data quality assessment is a key issue, in order to broaden the applicability of iris biometrics to unconstrained imaging conditions. Previous research efforts sought to use visible wavelength (VW) light imagery to acquire data at significantly larger distances than usual and on moving subjects, which makes this real-world data notoriously different from the acquired in the near-infrared setup. Having empirically observed that published strategies to assess iris image quality do not handle the specificity of such data, this paper proposes a method to assess the quality of VW iris samples captured in unconstrained conditions, according to the factors that are known to determine the quality of iris biometric data: focus, motion, angle, occlusions, area, pupillary dilation, and levels of iris pigmentation. The key insight is to use the output of the segmentation phase in each assessment, which permits us to handle severely degraded samples that are likely to result of such imaging setup. Also, our experiments point that the given method improves the effectiveness of VW iris recognition, by avoiding that poor quality samples are considered in the recognition process.

The human iris supports contact less data acquisition and can be imaged covertly. These factors give raise to the possibility of performing biometric recognition procedure with-out subjects' knowledge and in uncontrolled data acquisition scenarios. The feasibility of this type of recognition has been receiving increasing attention, as is of particular interest in visual surveillance, computer forensics, threat assessment, and other security areas. In this paper we stress the role played by the spectrum of the visible light used in the acquisition process and assess the discriminating iris patterns that are likely to be acquired according to three factors: type of illuminant, it's luminance, and levels of iris pigmentation. Our goal is to perceive and quantify the conditions that appear to enable the biometric recognition process with enough confidence.

Abstract: Continuous efforts have been made to improve the robustness of iris coding methods since Daugman's pioneering work on iris recognition was published. Iris recognition is at present used in several scenarios (airport check‐in, refugee control etc.) with very satisfactory results. However, in order to achieve acceptable error rates several imaging constraints are enforced, which reduce the fluidity of the iris recognition systems. The majority of the published iris recognition methods follow a statistical pattern recognition paradigm and encode the iris texture information through phase, zero‐crossing or texture‐analysis based methods. In this paper we propose a method that follows the structural (syntactic) pattern recognition paradigm. In addition to the intrinsic advantages of this type of approach (intuitive description and human perception of the system functioning), our experiments show that the proposed method behaves comparably to the statistical approach that constitutes the basis of nearly all deployed systems.

Iris recognition imaging constraints are receiving increasing attention. There are several proposals to develop systems that operate in the visible wavelength and in less constrained environments. These imaging conditions engender acquired noisy artifacts that lead to severely degraded images, making iris segmentation a major issue. Having observed that existing iris segmentation methods tend to fail in these challenging conditions, we present a segmentation method that can handle degraded images acquired in less constrained conditions. We offer the following contributions: 1) to consider the sclera the most easily distinguishable part of the eye in degraded images, 2) to propose a new type of feature that measures the proportion of sclera in each direction and is fundamental in segmenting the iris, and 3) to run the entire procedure in deterministically linear time in respect to the size of the image, making the procedure suitable for real-time applications.

The iris is regarded as one of the most useful traits for biometric recognition and the dissemination of nationwide iris-based recognition systems is imminent. However, currently deployed systems rely on heavy imaging constraints to capture near infrared images with enough quality. Also, all of the publicly available iris image databases contain data correspondent to such imaging constraints and therefore are exclusively suitable to evaluate methods thought to operate on these type of environments. The main purpose of this paper is to announce the availability of the UBIRIS.v2 database, a multisession iris images database which singularly contains data captured in the visible wavelength, at-a-distance (between four and eight meters) and on on-the-move. This database is freely available for researchers concerned about visible wavelength iris recognition and will be useful in accessing the feasibility and specifying the constraints of this type of biometric recognition.

Road accidents have a dramatic social and economic impact on the society, a situation that fostered the research of mechanisms for increasing road safety. Many of these mechanisms require the ability of the vehicles to communicate among each other and/or with fixed road-side stations. Due to the inherent mobility constraints, wireless technologies play a central role in this type of applications. Despite having been originally developed to delivery last mile wireless broadband access, as an alternative to cable and DSL, WiMAX presents some attractive attributes, such as quality-of-service management and traffic differentiation that are well suited for use in traffic applications. Nevertheless, to the best of our knowledge, no research in this area has considered the use of this technology. Thus, this WIP presents a preliminary study and assessment of the WiMAX technology for vehicular communications usage.

This paper focuses on noncooperative iris recognition, i.e., the capture of iris images at large distances, under less controlled lighting conditions, and without active participation of the subjects. This increases the probability of capturing very heterogeneous images (regarding focus, contrast, or brightness) and with several noise factors (iris obstructions and reflections). Current iris recognition systems are unable to deal with noisy data and substantially increase their error rates, especially the false rejections, in these conditions. We propose an iris classification method that divides the segmented and normalized iris image into six regions, makes an independent feature extraction and comparison for each region, and combines each of the dissimilarity values through a classification rule. Experiments show a substantial decrease, higher than 40 percent, of the false rejection rates in the recognition of noisy iris images.

In this paper we analyze the relationship between the accuracy of the segmentation algorithm and the error rates of typical iris recognition systems. We selected 1000 images from the UBIRIS database that the segmentation algorithm can accurately segment and artificially introduced segmentation inaccuracies. We repeated the recognition tests and concluded about the strong relationship between the errors in the pupil segmentation and the overall false reject rate. Based on this fact, we propose a method to identify these inaccuracies.