Abstract Image masking is essential in the field of 3D reconstruction for cultural heritage objects. It is used to accelerate the reconstruction process by removing background noise and accurately reconstructing the object only. The autonomous iterative Multi-View Stereo 3D scanner from the Fraunhofer Institute for Computer Graphics Research, within the Cultural Heritage Digitization department, requires binary masks to scan objects efficiently, regardless of the surrounding environment, geometry or color of the object, background color, and stabilizing mount. However, conventional masking methods can produce incorrect masks, leading to an inefficient or even abortive scan. Until now, these cases have been solved by parameter optimizations of the conventional masking method or changes in the scanning environment. This does not align with the principles of automation, since non-technical users in museums, archives, etc. should be able to use the autonomous iterative scanning workflow without additional effort. In addition to the real scan data used for training the presented networks, an automated Blender pipeline is also introduced, which generates additional synthetic data for training. Therefore, we evaluate if the latest state-of-the-art artificial intelligence segmentation methods can be used for these challenging cases without compromising their performance in simpler scenarios. This paper shows that with the proper network and datasets, masks of difficult objects or scenarios can be generated that can be used within the autonomous iterative scanning workflow. Thus, parameter and environment optimizations are no longer necessary.

Desk-based VR interactions carry safety risks when users unintentionally collide with unseen physical objects. Existing solutions often compromise immersion or lack situational precision. We present Reach-to-Reveal, a proximity-based system that dynamically visualizes real-world desk objects using headsetintegrated depth sensing, passthrough, and hand tracking. Objects are revealed only when the user reaches into their vicinity, enhancing safety while preserving immersion. Implemented on Meta Quest 3, Reach-to-Reveal was evaluated in a user study with 19 participants. Results show improved interaction confidence in cluttered desk setups, while highlighting challenges in visual realism and usability for future work.

Virtual Reality (VR) offers highly immersive experiences but often isolates users from their physical surroundings, leading to safety concerns. Conventional boundary systems such as Meta's Guardian or Valve's Chaperone provide static perimeters that fail to account for dynamically appearing obstacles during VR sessions. This paper explores the feasibility of dynamically integrating real-world obstacles directly into the virtual scene using the built-in RGB and depth sensors of the Meta Quest 3 headset. We present an implementation that dynamically integrates real-world obstacles in the play area to improve situational awareness without relying on external sensors or hardware modifications. A user study evaluates how this integration affects users' sense of presence, involvement, and perceived safety compared to the Guardian system. Results suggest that while involvement may slightly decrease due to visual inconsistencies, spatial presence and general presence remain statistically equivalent. These findings highlight the potential and current limitations of leveraging consumer-grade VR hardware for dynamic real-world integration in virtual environments.

Standard virtual reality (VR) controllers often provide limited tactile feedback, which can detract from player immersion and the seamlessness of interactions diminishing the overall experience. This paper evaluates MUHI - a Multi-Use Haptic Interface, which is a novel passive haptic tool designed to enhance player experience by convincingly simulating multiple virtual tools through its distinct physical handles. These tools are central to a series of playable mini-games embedded within a narrative-driven, redirected walking (RDW) based VR experience. The experience incorporates the RDW related drift, such that the temporarily stationary physical prop MUHI can represent various virtual tools in different parts of the virtual environment. The core investigation focuses on how integrating tangible, tool-based interactions via MUHI into these minigames enhances the game experience and immersive qualities, potentially concealing the RDW resets incidentally. A user study (N=17) compared the experience of playing the VR game with MUHI against a standard VR controller. Results demonstrate that MUHI significantly enhanced user enjoyment and increased reported presence scores. These findings suggest that specialized passive haptic interfaces like MUHI can substantially improve player engagement and the overall experience in VR applications, particularly by making core game mechanics, including those designed to mask RDW resets, more interactive and compelling.

657

6D object pose estimation suffers from reduced accuracy when applied to metallic objects. We set out to improve the state-of-the-art by addressing challenges such as reflections and specular highlights in industrial applications. Our novel BOP-compatible dataset [1], [2], featuring a diverse set of metallic objects (cans, household, and industrial items) under various lighting and background conditions, provides additional geometric and visual cues. We demonstrate that these cues can be effectively leveraged to enhance overall performance. To illustrate the usefulness of the additional features, we improve upon the GDRNPP [3] algorithm by introducing an additional keypoint prediction and material estimator head in order to improve spatial scene understanding. Evaluations on the new dataset show improved accuracy for metallic objects, supporting the hypothesis that additional geometric and visual cues can improve learning.

The development of a Digital Product Passport (DPP) is an important step in the process of sustainable production and an increasingly mandatory tool for the manufacturing industry, particularly in the EU. However, DPPs have so far been understood in the industry as a purely technical implementation for the exchange of relevant data toward product manufacturing, which is not suitable for reading and understanding by humans. This paper, therefore, describes an approach for a visual analytics system that enables human decision making through the visualization and analysis of DPPs, with a particular focus on optimizing CO2efficiency. The system uses intuitive analytical visualizations to enable quick understanding and actionable insights. The main contribution is the concrete data processing and visualization of DPP information for human access, and is so far one of the first available approaches for a visual representation of DPPs in general.

In previous work, we have presented an approach to index 3D LiDAR point clouds in real time, i.e. while they are being recorded. We have further introduced a novel data structure called M 3 NO, which allows arbitrary attributes to be indexed directly during data acquisition. Based on this, we now present an integrated approach that supports not only real-time indexing but also visualization with attribute filtering. We specifically focus on large datasets from airborne and land-based mobile mapping systems. Compared to traditional indexing approaches running offline, the M 3 NO is created incrementally. This enables dynamic queries based on spatial extent and value ranges of arbitrary attributes. The points in the data structure are assigned to levels of detail (LOD), which can be used to create interactive visualizations. This is in contrast to other approaches, which focus on either spatial or attribute indexing, only support a limited set of attributes, or do not support real-time visualization. Using several publicly available large data sets, we evaluate the approach, assess quality and query performance, and compare it with existing state-of-the-art indexing solutions. The results show that our data structure is able to index 5.24 million points per second. This is more than most commercially available laser scanners can record and proves that low-latency visualization during the capturing process is possible. • Integrated system for indexing, querying, and visualizing 3D point clouds. • Works in real-time, i.e. while the data is recorded, and supports any attributes. • Out-of-core processing allows arbitrarily large datasets to be indexed. • Large point clouds can be visualized even on devices with limited capacity.

Abstract Image-based 3D reconstruction is a powerful method for accurately reconstructing an object’s geometry and texture from images. A crucial factor for the accuracy and completeness of the resulting reconstructed model is the choice of poses for capturing images, which is called view planning. One possible view planning strategy uses an iterative feedback loop that switches between planning poses and an incremental reconstruction to autonomously digitize an object without prior knowledge. However, this approach requires identifying which parts of an object are “poorly reconstructed” and thus would benefit from being part of additional images. This work explores the use of point cloud quality metrics to provide this feedback by comprehensively comparing a set of existing and newly introduced metrics in terms of their time-dependent behavior, similarity, and their applicability to view planning. Among the newly proposed metrics this work introduces the Reconstruction Quality Feedback (RQF), which shows a significantly improved performance in simulations when being used for view planning. The effectiveness of RQF is also demonstrated for real objects on an autonomous robotic 3D digitization system.

Synthetic data is emerging as a substitute for authentic data to solve ethical and legal challenges in handling authentic face data. The current models can create real-looking face images of people who do not exist. However, it is a known and sensitive problem that face recognition systems are susceptible to bias, i.e. performance differences between different demographic and non-demographics attributes, which can lead to unfair decisions. In this work, we investigate how the diversity of synthetic face recognition datasets compares to authentic datasets, and how the distribution of the training data of the generative models affects the distribution of the synthetic data. To do this, we looked at the distribution of gender, ethnicity, age, and head position. Furthermore, we investigated the concrete bias of three recent synthetic-based face recognition models on the studied attributes in comparison to a baseline model trained on authentic data. Our results show that the generator generate a similar distribution as the used training data in terms of the different attributes. With regard to bias, it can be seen that the synthetic-based models share a similar bias behavior with the authentic-based models. However, with the uncovered lower intra-identity attribute consistency seems to be beneficial in reducing bias.

Abstract Recent advancements in ubiquitous computing have emphasized the need for privacy-preserving occupancy detection in smart environments to enhance security. This work presents a novel occupancy detection solution utilizing privacy-aware sensing technologies. The solution analyzes time-series data to detect not only occupancy as a binary problem, but also determines whether one or multiple individuals are present in an indoor environment. On three real-world datasets, our models outperformed various state-of-the-art algorithms, achieving F1-scores up to 94.91% in single-occupancy detection and a macro F1-score of 91.55% in multi-occupancy detection. This makes our approach a promising solution for improving security in smart environments.

Summary Function as a service (FaaS) promises low operating costs, reduced complexity, and good application performance. However, it is still an open question how to migrate monolithic applications to FaaS. In this paper, we present a guideline for software designers to split monolithic applications into smaller functions that can be executed in a FaaS environment. This enables independent scaling of individual parts of the application. Our approach consists of three steps: We first identify the main tasks (and their subtasks) of the application to split. Then, we define the program flow to be able to tell which application tasks can be converted to functions and how they interact with each other. In the final step, we specify actual functions and possibly merge those that are too small and which would produce too much communication overhead or maintenance effort. Compared to existing work, our approach applies to applications of any size and results in functions that are small enough—but not too small—for efficient execution in a FaaS environment. We evaluate the usefulness of our approach by applying it to a real‐world application for the storage of geospatial data. We describe the experiences made and finish the paper with a discussion, conclusions, and ideas for future work.

Abstract Neural radiance fields (NeRFs) have revolutionized novel view synthesis, leading to an unprecedented level of realism in rendered images. However, the reconstruction quality of NeRFs suffers significantly from out-of-focus regions in the input images. We propose NeRF-FF, a plug-in strategy that estimates image masks based on Focus Frustums (FFs), i.e., the visible volume in the scene space that is in-focus. NeRF-FF enables a subsequently trained NeRF model to omit out-of-focus image regions during the training process. Existing methods to mitigate the effects of defocus blurred input images often leverage dynamic ray generation. This makes them incompatible with the static ray assumptions employed by runtime-performance-optimized NeRF variants, such as Instant-NGP, leading to high training times. Our experiments show that NeRF-FF outperforms state-of-the-art approaches regarding training time by two orders of magnitude—reducing it to under 1 min on end-consumer hardware—while maintaining comparable visual quality.

Over the past years, the main research innovations in face recognition focused on training deep neural networks on large-scale identity-labeled datasets using variations of multi-class classification losses. However, many of these datasets are retreated by their creators due to increased privacy and ethical concerns. Very recently, privacy-friendly synthetic data has been proposed as an alternative to privacy-sensitive authentic data to comply with privacy regulations and to ensure the continuity of face recognition research. In this paper, we propose an unsupervised face recognition model based on unlabeled synthetic data (USynthFace). Our proposed USynthFace learns to maximize the similarity between two augmented images of the same synthetic instance. We enable this by a large set of geometric and color transformations in addition to GAN-based augmentation that contributes to the USynthFace model training. We also conduct numerous empirical studies on different components of our USynthFace. With the proposed set of augmentation operations, we proved the effectiveness of our USynthFace in achieving relatively high recognition accuracies using unlabeled synthetic data. The training code and pretrained model are publicly available under https://github.com/fdbtrs/Unsupervised-Face-Recognition-using-Unlabeled-Synthetic-Data.

Biometric face recognition models are widely used in many different real-world applications. The output of these models can be used to make decisions that may strongly impact people. However, an explanation of how and why such outputs are derived is usually not given to humans. The lack of explainability of face recognition models leads to distrust in their decisions and does not encourage their use. The performance of face recognition models is influenced by the quality of the input image. In case the quality of a face image is too low, the face recognition system will reject it to avoid compromising its performance. The quality is evaluated by Face Image Quality (FIQ) approaches, which assigned quality scores to the input images. Pixel-level face image quality (PLFIQ) increases the explainability of quality scores by explaining face image quality at the pixel level. This allows the users of face recognition systems to spot low-quality areas and allows them to make guided corrections. Previous works introduced the concept of PLFIQ and proposed evaluation procedures. This work proposes a new way of computing PLFIQ values depending on given FIQ methods using Shapley Values. They score the contribution of each pixel to the overall image quality evaluation. Therefore, Integrating Shapley Values increases the explainability of the FIQ models. Results show that using these methods leads to significantly better and more robust PLFIQ values estimates and thus provide better explainability.

Deep generative models have recently presented impressive results in generating realistic face images of random synthetic identities. To generate multiple samples of a certain synthetic identity, previous works proposed to disentangle the latent space of GANs by incorporating additional supervision or regularization, enabling the manipulation of certain attributes. Others proposed to disentangle specific factors in unconditional pretrained GANs latent spaces to control their output, which also requires supervision by attribute classifiers. Moreover, these attributes are entangled in GAN’s latent space, making it difficult to manipulate them without affecting the identity information. We propose in this work a framework, ExFaceGAN, to disentangle identity information in pretrained GANs latent spaces, enabling the generation of multiple samples of any synthetic identity. Given a reference latent code of any synthetic image and latent space of pretrained GAN, our ExFaceGAN learns an identity directional boundary that disentangles the latent space into two sub-spaces, with latent codes of samples that are either identity similar or dissimilar to a reference image. By sampling from each side of the boundary, our ExFaceGAN can generate multiple samples of synthetic identity without the need for designing a dedicated architecture or supervision from attribute classifiers. We demonstrate the generalizability and effectiveness of ExFaceGAN by integrating it into learned latent spaces of three SOTA GAN approaches. As an example of the practical benefit of our ExFaceGAN, we empirically prove that data generated by ExFaceGAN can be successfully used to train face recognition models (https://github.com/fdbtrs/ExFaceGAN).

The availability of large-scale authentic face databases has been crucial to the significant advances made in face recognition research over the past decade. However, legal and ethical concerns led to the recent retraction of many of these databases by their creators, raising questions about the continuity of future face recognition research without one of its key resources. Synthetic datasets have emerged as a promising alternative to privacy-sensitive authentic data for face recognition development. However, recent synthetic datasets that are used to train face recognition models suffer either from limitations in intra-class diversity or cross-class (identity) discrimination, leading to less optimal accuracies, far away from the accuracies achieved by models trained on authentic data. This paper targets this issue by proposing IDiff-Face, a novel approach based on conditional latent diffusion models for synthetic identity generation with realistic identity variations for face recognition training. Through extensive evaluations, our proposed synthetic-based face recognition approach pushed the limits of state-of-the-art performances, achieving, for example, 98.00% accuracy on the Labeled Faces in the Wild (LFW) benchmark, far ahead from the recent synthetic-based face recognition solutions with 95.40% and bridging the gap to authentic-based face recognition with 99.82% accuracy*.

In this work, we propose QMagFace, a simple and effective face recognition solution (QMagFace) that combines a quality-aware comparison score with a recognition model based on a magnitude-aware angular margin loss. The proposed approach includes model-specific face image qualities in the comparison process to enhance the recognition performance under unconstrained circumstances. Exploiting the linearity between the qualities and their comparison scores induced by the utilized loss, our quality-aware comparison function is simple and highly generalizable. The experiments conducted on several face recognition databases and benchmarks demonstrate that the introduced quality-awareness leads to consistent improvements in the recognition performance. Moreover, the proposed QMagFace approach performs especially well under challenging circumstances, such as cross-pose, cross-age, or cross-quality. Consequently, it leads to state-of-the-art performances on several face recognition benchmarks, such as 98.50% on AgeDB, 83.95% on XQLFQ, and 98.74% on CFP-FP. The code for QMagFace is publicly available1.

Bias and fairness of biometric algorithms have been key topics of research in recent years, mainly due to the societal, legal and ethical implications of potentially unfair decisions made by automated decision-making models. A considerable amount of work has been done on this topic across different biometric modalities, aiming at better understanding the main sources of algorithmic bias or devising mitigation measures. In this work, we contribute to these efforts and present the first study investigating bias and fairness of sclera segmentation models. Although sclera segmentation techniques represent a key component of sclera-based biometric systems with a considerable impact on the overall recognition performance, the presence of different types of biases in sclera segmentation methods is still underexplored. To address this limitation, we describe the results of a group evaluation effort (involving seven research groups), organized to explore the performance of recent sclera segmentation models within a common experimental framework and study performance differences (and bias), originating from various demographic as well as environmental factors. Using five diverse datasets, we analyze seven independently developed sclera segmentation models in different experimental configurations. The results of our experiments suggest that there are significant differences in the overall segmentation performance across the seven models and that among the considered factors, ethnicity appears to be the biggest cause of bias. Additionally, we observe that training with representative and balanced data does not necessarily lead to less biased results. Finally, we find that in general there appears to be a negative correlation between the amount of bias observed (due to eye color, ethnicity and acquisition device) and the overall segmentation performance, suggesting that advances in the field of semantic segmentation may also help with mitigating bias.

Face presentation attack detection (PAD) is essential in mitigating spoofing attack vulnerabilities in face recognition systems. Despite the relatively good detection performance of PADs on known attacks, they tend to be challenged by unknown samples. To address this issue, we present our PatchSwap approach that aims at creating more challenging and complex bona fide, attack, and partial attack samples despite limited training resources. The PatchSwap operates by swapping intra-identity patches between training samples and correspondingly updates their pixel-wise mask label, all under a controlled strategy. The PatchSwap is deployed as an augmentation technique and can be effortlessly integrated into any model training process. The different choices towards our PatchSwap design are exhaustively investigated and proven in detailed studies. We conduct extensive experiments under intra-dataset and cross-dataset scenarios and on three different network backbones. The experimental results showed that the PatchSwap successfully induces significant gains in the PAD performance under different evaluation settings.

A MasterFace is a face image that can successfully match against a large portion of the population. Since their generation does not require access to the information of the enrolled subjects, MasterFace attacks represent a potential security risk for widely-used face recognition systems. Previous works proposed methods for generating such images and demonstrated that these attacks can strongly compromise face recognition. However, previous works followed evaluation settings consisting of older recognition models, limited cross-dataset and cross-model evaluations, and the use of low-scale testing data. This makes it hard to state the generalizability of these attacks. In this work, we comprehensively analyse the generalizability of MasterFace attacks in empirical and theoretical investigations. The empirical investigations include the use of six state-of-the-art face recognition models, cross-dataset and cross-model evaluation protocols, and utilizing testing datasets of significantly higher size and variance. The results indicate a low generalizability when MasterFaces are training on a different face recognition model than the one used for testing. In these cases, the attack performance is similar to zero-effort imposter attacks. In the theoretical investigations, we define and estimate the face capacity and the maximum MasterFace coverage under the assumption that identities in the face space are well separated. The current trend of increasing the fairness and generalizability in face recognition indicates that the vulnerability of future systems might further decrease. Future works might analyse the utility of MasterFaces for understanding and enhancing the robustness of face recognition models.

Structure from Motion (SfM) plays a crucial role in unstructured capturing. While images are usually taken by perspective cameras, orthographic camera projections do not suffer from the foreshortening effect, that leads to varying capturing quality in image regions. Most contributions to orthographic image SfM assume a perspective setup with nearly infinite focal length. These assumptions lead to potentially sub-optimal camera pose estimation. Therefore, we propose a SfM pipeline that is optimized for orthographically projected images. For this, we estimate initial camera poses using the factorization method by Tomasi and Kanade. These poses are further refined by a specialized bundle adjustment implementation for orthographic projections. The proposed pipeline surpasses the precision of state-of-the-art work by an order of magnitude, while consuming considerably less computational resources.

Deep learning-based face recognition models follow the common trend in deep neural networks by utilizing full-precision floating-point networks with high computational costs. Deploying such networks in use-cases constrained by computational requirements is often infeasible due to the large memory required by the full-precision model. Previous compact face recognition approaches proposed to design special compact architectures and train them from scratch using real training data, which may not be available in a real-world scenario due to privacy concerns. We present in this work the QuantFace solution based on low-bit precision format model quantization. QuantFace reduces the required computational cost of the existing face recognition models without the need for designing a particular architecture or accessing real training data. QuantFace introduces privacy-friendly synthetic face data to the quantization process to mitigate potential privacy concerns and issues related to the accessibility to real training data. Through extensive evaluation experiments on seven benchmarks and four network architectures, we demonstrate that QuantFace can successfully reduce the model size up to 5x while maintaining, to a large degree, the verification performance of the full-precision model without accessing real training datasets. All training codes are publicly available1.

With the increased deployment of face recognition systems in our daily lives, face presentation attack detection (PAD) is attracting much attention and playing a key role in securing face recognition systems. Despite the great performance achieved by the hand-crafted and deep-learning-based methods in intra-dataset evaluations, the performance drops when dealing with unseen scenarios. In this work, we propose a dual-stream convolution neural networks (CNNs) framework. One stream adapts four learnable frequency filters to learn features in the frequency domain, which are less influenced by variations in sensors/illuminations. The other stream leverages the RGB images to complement the features of the frequency domain. Moreover, we propose a hierarchical attention module integration to join the information from the two streams at different stages by considering the nature of deep features in different layers of the CNN. The proposed method is evaluated in the intra-dataset and cross-dataset setups, and the results demonstrate that our proposed approach enhances the generalizability in most experimental setups in comparison to state-of-the-art, including the methods designed explicitly for domain adaption/shift problems. We successfully prove the design of our proposed PAD solution in a stepwise ablation study that involves our proposed learnable frequency decomposition, our hierarchical attention module design, and the used loss function. Training codes and pre-trained models are publicly released1.

Abstract Iris Presentation Attack Detection (PAD) algorithms address the vulnerability of iris recognition systems to presentation attacks. With the great success of deep learning methods in various computer vision fields, neural network-based iris PAD algorithms emerged. However, most PAD networks suffer from overfitting due to insufficient iris data variability. Therefore, we explore the impact of various data augmentation techniques on performance and the generalizability of iris PAD. We apply several data augmentation methods to generate variability, such as shift, rotation, and brightness. We provide in-depth analyses of the overlapping effect of these methods on performance. In addition to these widely used augmentation techniques, we also propose an augmentation selection protocol based on the assumption that various augmentation techniques contribute differently to the PAD performance. Moreover, two fusion methods are performed for more comparisons: the strategy-level and the score-level combination. We demonstrate experiments on two fine-tuned models and one trained from the scratch network and perform on the datasets in the Iris-LivDet-2017 competition designed for generalizability evaluation. Our experimental results show that augmentation methods improve iris PAD performance in many cases. Our least overlap-based augmentation selection protocol achieves the lower error rates for two networks. Besides, the shift augmentation strategy also exceeds state-of-the-art (SoTA) algorithms on the Clarkson and IIITD-WVU datasets.

Wearing a mask has proven to be one of the most effective ways to prevent the transmission of SARS-Co V-2 coronavirus. However, wearing a mask poses challenges for different face recognition tasks and raises concerns about the performance of masked face presentation detection (PAD). The main issues facing the mask face PAD are the wrongly classified bona fide masked faces and the wrongly classified partial attacks (covered by real masks). This work addresses these issues by proposing a method that considers partial attack labels to supervise the PAD model training, as well as regional weighted inference to further improve the PAD performance by varying the focus on different facial areas. Our proposed method is not directly linked to specific network architecture and thus can be directly incorporated into any common or custom-designed network. In our work, two neural networks (DeepPixBis [21] and MixFaceNet [4]) are selected as backbones. The experiments are demonstrated on the collaborative real mask attack (CRMA) database [17]. Our proposed method outperforms established PAD methods in the CRMA database by reducing the mentioned shortcomings when facing masked faces. Moreover, we present a detailed step-wise ablation study pointing out the individual and joint benefits of the proposed concepts on the overall PAD performance.

In this paper, we present a set of extremely efficient and high throughput models for accurate face verification, Mix-FaceNets which are inspired by Mixed Depthwise Convolutional Kernels. Extensive experiment evaluations on Label Face in the Wild (LFW), Age-DB, MegaFace, and IARPA Janus Benchmarks IJB-B and IJB-C datasets have shown the effectiveness of our MixFaceNets for applications requiring extremely low computational complexity. Under the same level of computation complexity (≤ 500M FLOPs), our MixFaceNets outperform MobileFaceNets on all the evaluated datasets, achieving 99.60% accuracy on LFW, 97.05% accuracy on AgeDB-30, 93.60 TAR (at FAR1e-6) on MegaFace, 90.94 TAR (at FAR1e-4) on IJB-B and 93.08 TAR (at FAR1e-4) on IJB-C. With computational complexity between 500M and 1G FLOPs, our MixFaceNets achieved results comparable to the top-ranked models, while using significantly fewer FLOPs and less computation over-head, which proves the practical value of our proposed Mix-FaceNets. All training codes, pre-trained models, and training logs have been made available https://github.com/fdbtrs/mixfacenets.

This paper presents a summary of the Masked Face Recognition Competitions (MFR) held within the 2021 International Joint Conference on Biometrics (IJCB 2021). The competition attracted a total of 10 participating teams with valid submissions. The affiliations of these teams are diverse and associated with academia and industry in nine different countries. These teams successfully submitted 18 valid solutions. The competition is designed to motivate solutions aiming at enhancing the face recognition accuracy of masked faces. Moreover, the competition considered the deployability of the proposed solutions by taking the compactness of the face recognition models into account. A private dataset representing a collaborative, multisession, real masked, capture scenario is used to evaluate the submitted solutions. In comparison to one of the topperforming academic face recognition solutions, 10 out of the 18 submitted solutions did score higher masked face verification accuracy.

Iris presentation attack detection (PAD) plays a vital role in iris recognition systems. Most existing CNN-based iris PAD solutions 1) perform only binary label supervision during the training of CNNs, serving global information learning but weakening the capture of local discriminative features, 2) prefer the stacked deeper convolutions or expert-designed networks, raising the risk of overfitting, 3) fuse multiple PAD systems or various types of features, increasing difficulty for deployment on mobile devices. Hence, we propose a novel attention-based deep pixel-wise bi-nary supervision (A-PBS) method. Pixel-wise supervision is first able to capture the fine-grained pixel/patch-level cues. Then, the attention mechanism guides the network to automatically find regions that most contribute to an accurate PAD decision. Extensive experiments are performed on LivDet-Iris 2017 and three other publicly available databases to show the effectiveness and robustness of proposed A-PBS methods. For instance, the A-PBS model achieves an HTER of 6.50% on the IIITD-WVU database outperforming state-of-the-art methods.

Liveness Detection (LivDet)-Face is an international competition series open to academia and industry. The competition’s objective is to assess and report state-of-the-art in liveness / Presentation Attack Detection (PAD) for face recognition. Impersonation and presentation of false samples to the sensors can be classified as presentation attacks and the ability for the sensors to detect such attempts is known as PAD. LivDet-Face 2021 * will be the first edition of the face liveness competition. This competition serves as an important benchmark in face presentation attack detection, offering (a) an independent assessment of the current state of the art in face PAD, and (b) a common evaluation protocol, availability of Presentation Attack Instruments (PAI) and live face image dataset through the Biometric Evaluation and Testing (BEAT) platform. The competition can be easily followed by researchers after it is closed, in a platform in which participants can compare their solutions against the LivDet-Face winners.

An essential factor to achieve high accuracies in finger-print recognition systems is the quality of its samples. Previous works mainly proposed supervised solutions based on image properties that neglects the minutiae extraction process, despite that most fingerprint recognition techniques are based on detected minutiae. Consequently, a fingerprint image might be assigned a high quality even if the utilized minutia extractor produces unreliable information. In this work, we propose a novel concept of assessing minutia and fingerprint quality based on minutia detection confidence (MiDeCon). MiDeCon can be applied to an arbitrary deep learning based minutia extractor and does not require quality labels for learning. We propose using the detection reliability of the extracted minutia as its quality indicator. By combining the highest minutia qualities, MiDeCon also accurately determines the quality of a full fingerprint. Experiments are conducted on the publicly available databases of the FVC 2006 and compared against several baselines, such as NIST’s widely-used fingerprint image quality software NFIQ1 and NFIQ2. The results demonstrate a significantly stronger quality assessment performance of the proposed MiDeCon-qualities as related works on both, minutia- and fingerprint-level. The implementation is publicly available.

For iris recognition in non-cooperative environments, iris segmentation has been regarded as the first most important challenge still open to the biometric community, affecting all downstream tasks from normalization to recognition. In recent years, deep learning technologies have gained significant popularity among various computer vision tasks and also been introduced in iris biometrics, especially iris segmentation. To investigate recent developments and attract more interest of researchers in the iris segmentation method, we organized the 2021 NIR Iris Challenge Evaluation in Non-cooperative Environments: Segmentation and Localization (NIR-ISL 2021) at the 2021 International Joint Conference on Biometrics (IJCB 2021). The challenge was used as a public platform to assess the performance of iris segmentation and localization methods on Asian and African NIR iris images captured in non-cooperative environments. The three best-performing entries achieved solid and satisfactory iris segmentation and localization results in most cases, and their code and models have been made publicly available for reproducibility research.

Due to the deforming and dynamically changing textile fibers, the quality assurance of cleaned industrial textiles is still a mostly manual task. Usually, textiles need to be spread flat, in order to detect defects using computer vision inspection methods. Already known methods for detecting defects on such inhomogeneous, voluminous surfaces use mainly supervised methods based on deep neural networks and require lots of labeled training data. In contrast, we present a novel unsupervised method, based on SURF keypoints, that does not require any training data. We propose using their location, number and orientation in order to group them into geographically close clusters. Keypoint clusters also indicate the exact position of the defect at the same time. We furthermore compared our approach to supervised methods using deep learning. The presented processing pipeline shows how normalization and classification methods need to be combined, in order to reliably detect fiber defects such as cuts and holes. We evaluate the performance of our system in real-world settings with images of piles of textiles, taken in stereo vision. Our results show that our novel unsupervised classification method using keypoint clustering achieves comparable results to other supervised methods.

Tracking multiple persons is a challenging task especially when persons move in groups and occlude one another. Existing research have investigated the problems of group division and segmentation; however, lacking overall person-group topology modeling limits the ability to handle complex person and group dynamics. We propose a Graphical Social Topology (GST) model in the RGB-D data domain, and estimate object group dynamics by jointly modeling the group structure and states of persons using RGB-D topological representation. With our topology representation, moving persons are not only assigned to groups, but also dynamically connected with each other, which enables in-group individuals to be correctively associated and the cohesion of each group to be precisely modeled. Using the learned typical topology pattern and group online update modules, we infer the birth/death and merging/splitting of dynamic groups. With the GST model, the proposed multi-person tracker can naturally facilitate the occlusion problem by treating the occluded object and other in-group members as a whole, while leveraging overall state transition. Experiments on different RGB-D and RGB datasets confirm that the proposed multi-person tracker improves the state-of-the-arts.

Biometric recognition technology has made significant advances over the last decade and is now used across a number of services and applications. However, this widespread deployment has also resulted in privacy concerns and evolving societal expectations about the appropriate use of the technology. For example, the ability to automatically extract age, gender, race, and health cues from biometric data has heightened concerns about privacy leakage. Face recognition technology, in particular, has been in the spotlight, and is now seen by many as posing a considerable risk to personal privacy. In response to these and similar concerns, researchers have intensified efforts towards developing techniques and computational models capable of ensuring privacy to individuals, while still facilitating the utility of face recognition technology in several application scenarios. These efforts have resulted in a multitude of privacy-enhancing techniques that aim at addressing privacy risks originating from biometric systems and providing technological solutions for legislative requirements set forth in privacy laws and regulations, such as GDPR. The goal of this overview paper is to provide a comprehensive introduction into privacy-related research in the area of biometrics and review existing work on Biometric Privacy-Enhancing Techniques (B-PETs) applied to face biometrics. To make this work useful for as wide of an audience as possible, several key topics are covered as well, including evaluation strategies used with B-PETs, existing datasets, relevant standards, and regulations and critical open issues that will have to be addressed in the future.

Soft-biometrics play an important role in face biometrics and related fields since these might lead to biased performances, threaten the user's privacy, or are valuable for commercial aspects. Current face databases are specifically constructed for the development of face recognition applications. Consequently, these databases contain a large number of face images but lack in the number of attribute annotations and the overall annotation correctness. In this work, we propose a novel annotation-transfer pipeline that allows to accurately transfer attribute annotations from multiple source datasets to a target dataset. The transfer is based on a massive attribute classifier that can accurately state its prediction confidence. Using these prediction confidences, a high correctness of the transferred annotations is ensured. Applying this pipeline to the VGGFace2 database, we propose the MAAD-Face annotation database. It consists of 3.3M faces of over 9k individuals and provides 123.9M attribute annotations of 47 different binary attributes. Consequently, it provides 15 and 137 times more attribute annotations than CelebA and LFW. Our investigation on the annotation quality by three human evaluators demonstrated the superiority of the MAAD-Face annotations over existing databases. Additionally, we make use of the large number of high-quality annotations from MAAD-Face to study the viability of soft-biometrics for recognition, providing insights into which attributes support genuine and imposter decisions. The MAAD-Face annotations dataset is publicly available.

Most existing person re-identification methods compute pairwise similarity by extracting robust visual features and learning the discriminative metric. Owing to visual ambiguities, these content-based methods that determine the pairwise relationship only based on the similarity between them, inevitably produce a suboptimal ranking list. Instead, the pairwise similarity can be estimated more accurately along the geodesic path of the underlying data manifold by exploring the rich contextual information of the sample. In this paper, we propose a lightweight post-processing person re-identification method in which the pairwise measure is determined by the relationship between the sample and the counterpart's context in an unsupervised way. We translate the point-to-point comparison into the bilateral point-to-set comparison. The sample's context is composed of its neighbor samples with two different definition ways: the first order context and the second order context, which are used to compute the pairwise similarity in sequence, resulting in a progressive post-processing model. The experiments on four large-scale person re-identification benchmark datasets indicate that (1) the proposed method can consistently achieve higher accuracies by serving as a post-processing procedure after the content-based person re-identification methods, showing its state-of-the-art results, (2) the proposed lightweight method only needs about 6 milliseconds for optimizing the ranking results of one sample, showing its high-efficiency. Code is available at: https://github.com/123ci/PBCmodel.

Online services are used for all kinds of activities, like news, entertainment, publishing content or connecting with others. But information technology enables new threats to privacy by means of global mass surveillance, vast databases and fast distribution networks. Current news are full of misuses and data leakages. In most cases, users are powerless in such situations and develop an attitude of neglect for their online behaviour. On the other hand, the GDPR (General Data Protection Regulation) gives users the right to request a copy of all their personal data stored by a particular service, but the received data is hard to understand or analyze by the common internet user. This paper presents TransparencyVis - a web-based interface to support the visual and interactive exploration of data exports from different online services. With this approach, we aim at increasing the awareness of personal data stored by such online services and the effects of online behaviour. This design study provides an online accessible prototype and a best practice to unify data exports from different sources.

Face image quality is an important factor to enable high-performance face recognition systems. Face quality assessment aims at estimating the suitability of a face image for the purpose of recognition. Previous work proposed supervised solutions that require artificially or human labelled quality values. However, both labelling mechanisms are error prone as they do not rely on a clear definition of quality and may not know the best characteristics for the utilized face recognition system. Avoiding the use of inaccurate quality labels, we proposed a novel concept to measure face quality based on an arbitrary face recognition model. By determining the embedding variations generated from random subnetworks of a face model, the robustness of a sample representation and thus, its quality is estimated. The experiments are conducted in a cross-database evaluation setting on three publicly available databases. We compare our proposed solution on two face embeddings against six state-of-the-art approaches from academia and industry. The results show that our unsupervised solution outperforms all other approaches in the majority of the investigated scenarios. In contrast to previous works, the proposed solution shows a stable performance over all scenarios. Utilizing the deployed face recognition model for our face quality assessment methodology avoids the training phase completely and further outperforms all baseline approaches by a large margin. Our solution can be easily integrated into current face recognition systems, and can be modified to other tasks beyond face recognition.

The growing popularity of Virtual Reality and Augmented Reality (VR/AR) devices in many applications also demands authentication of users. As the devices inherently capture the eye image while capturing the user interaction, the authentication can be devised using the iris and periocular recognition. While both iris and periocular data being non-ideal unlike the data captured from standard biometric sensors, the authentication performance is expected to be lower. In this work, we present and evaluate a fusion framework for improving the biometric authentication performance. Specifically, we employ score-level fusion for two independent biometric systems of iris and periocular region to avoid expensive feature-level fusion. With a detailed evaluation of three different score-level fusion after the score normalization on a dataset of 12579 images, we report the performance gain in authentication using score-level fusion for iris and periocular recognition.

Iris presentation attack detection (PAD) algorithms are developed to address the vulnerability of iris recognition systems to presentation attacks. Taking into account that the deep features successfully improved computer vision performance in various fields including iris recognition, it is natural to use features extracted from deep neural networks for iris PAD. Each layer in a deep learning network carries features of different level of abstraction. The features extracted from the first layer to the higher layers become more complex and more abstract. This might point our complementary information in these features that can collaborate towards an accurate PAD decision. Therefore, we propose an iris PAD solution based on multi-layer fusion. The information extracted from the last several convolutional layers are fused on two levels, feature-level and score-level. We demonstrated experiments on both, off-the-shelf pre-trained network and network trained from scratch. An extensive experiment also explores the complementary between different layer combinations of deep features. Our experimental results show that feature-level based multi-layer fusion method performs better than the best single layer feature extractor in most cases. In addition, our fusion results achieve similar or better results than the state-of-the-art algorithms on the Notre Dame and IIITD-WVU databases of the Iris Liveness Detection Competition 2017 (LivDet-Iris 2017).

Augmented and virtual reality is being deployed in different fields of applications. Such applications might involve accessing or processing critical and sensitive information, which requires strict and continuous access control. Given that Head-Mounted Displays (HMD) developed for such applications commonly contains internal cameras for gaze tracking purposes, we evaluate the suitability of such setup for verifying the users through iris recognition. In this work, we first evaluate a set of iris recognition algorithms suitable for HMD devices by investigating three well-established handcrafted feature extraction approaches, and to complement it, we also present the analysis using four deep learning models. While taking into consideration the minimalistic hardware requirements of stand-alone HMD, we employ and adapt a recently developed miniature segmentation model (EyeMMS) for segmenting the iris. Further, to account for non-ideal and non-collaborative capture of iris, we define a new iris quality metric that we termed as Iris Mask Ratio (IMR) to quantify the iris recognition performance. Motivated by the performance of iris recognition, we also propose the continuous authentication of users in a non-collaborative capture setting in HMD. Through the experiments on a publicly available OpenEDS dataset, we show that performance with EER = 5% can be achieved using deep learning methods in a general setting, along with high accuracy for continuous user authentication.

Launched in 2013, LivDet-Iris is an international competition series open to academia and industry with the aim to assess and report advances in iris Presentation Attack Detection (PAD). This paper presents results from the fourth competition of the series: LivDet-Iris 2020. This year's competition introduced several novel elements: (a) incorporated new types of attacks (samples displayed on a screen, cadaver eyes and prosthetic eyes), (b) initiated LivDet-Iris as an on-going effort, with a testing protocol available now to everyone via the Biometrics Evaluation and Testing (BEAT)* open-source platform to facilitate reproducibility and benchmarking of new algorithms continuously, and (c) performance comparison of the submitted entries with three baseline methods (offered by the University of Notre Dame and Michigan State University), and three open-source iris PAD methods available in the public domain. The best performing entry to the competition reported a weighted average APCER of 59.10% and a BPCER of 0.46% over all five attack types. This paper serves as the latest evaluation of iris PAD on a large spectrum of presentation attack instruments.

Iris recognition systems are vulnerable to the presentation attacks, such as textured contact lenses or printed images. In this paper, we propose a lightweight framework to detect iris presentation attacks by extracting multiple micro-stripes of expanded normalized iris textures. In this procedure, a standard iris segmentation is modified. For our Presentation Attack Detection (PAD) network to better model the classification problem, the segmented area is processed to provide lower dimensional input segments and a higher number of learning samples. Our proposed Micro Stripes Analyses (MSA) solution samples the segmented areas as individual stripes. Then, the majority vote makes the final classification decision of those micro-stripes. Experiments are demonstrated on five databases, where two databases (IIITD-WVU and Notre Dame) are from the LivDet-2017 Iris competition. An in-depth experimental evaluation of this framework reveals a superior performance compared with state-of-the-art (SoTA) algorithms. Moreover, our solution minimizes the confusion between textured (attack) and soft (bona fide) contact lens presentations.

Deeply-learned face representations enable the success of current face recognition systems. Despite the ability of these representations to encode the identity of an individual, recent works have shown that more information is stored within, such as demographics, image characteristics, and social traits. This threatens the user's privacy, since for many applications these templates are expected to be solely used for recognition purposes. Knowing the encoded information in face templates helps to develop bias-mitigating and privacy-preserving face recognition technologies. This work aims to support the development of these two branches by analysing face templates regarding 113 attributes. Experiments were conducted on two publicly available face embeddings. For evaluating the predictability of the attributes, we trained a massive attribute classifier that is additionally able to accurately state its prediction confidence. This allows us to make more sophisticated statements about the attribute predictability. The results demonstrate that up to 74 attributes can be accurately predicted from face templates. Especially non-permanent attributes, such as age, hairstyles, haircolors, beards, and various accessories, found to be easily-predictable. Since face recognition systems aim to be robust against these variations, future research might build on this work to develop more understandable privacy preserving solutions and build robust and fair face templates.

Face quality assessment aims at estimating the utility of a face image for the purpose of recognition. It is a key factor to achieve high face recognition performances. Currently, the high performance of these face recognition systems come with the cost of a strong bias against demographic and non-demographic sub-groups. Recent work has shown that face quality assessment algorithms should adapt to the deployed face recognition system, in order to achieve highly accurate and robust quality estimations. However, this could lead to a bias transfer towards the face quality assessment leading to discriminatory effects e.g. during enrolment. In this work, we present an in-depth analysis of the correlation between bias in face recognition and face quality assessment. Experiments were conducted on two publicly available datasets captured under controlled and uncontrolled circumstances with two popular face embeddings. We evaluated four state-of-the-art solutions for face quality assessment towards biases to pose, ethnicity, and age. The experiments showed that the face quality assessment solutions assign significantly lower quality values towards subgroups affected by the recognition bias demonstrating that these approaches are biased as well. This raises ethical questions towards fairness and discrimination which future works have to address.

The paper presents a summary of the 2020 Sclera Segmentation Benchmarking Competition (SSBC), the 7th in the series of group benchmarking efforts centred around the problem of sclera segmentation. Different from previous editions, the goal of SSBC 2020 was to evaluate the performance of sclera-segmentation models on images captured with mobile devices. The competition was used as a platform to assess the sensitivity of existing models to i) differences in mobile devices used for image capture and ii) changes in the ambient acquisition conditions. 26 research groups registered for SSBC 2020, out of which 13 took part in the final round and submitted a total of 16 segmentation models for scoring. These included a wide variety of deep-learning solutions as well as one approach based on standard image processing techniques. Experiments were conducted with three recent datasets. Most of the segmentation models achieved relatively consistent performance across images captured with different mobile devices (with slight differences across devices), but struggled most with low-quality images captured in challenging ambient conditions, i.e., in an indoor environment and with poor lighting.

We show how to overcome the single weakness of an existing fully automatic system for acquisition of spatially varying optical material behavior of real object surfaces. While the expression of spatially varying material behavior with spherical dependence on incoming light as a 4D texture (an ABTF material model) allows flexible mapping onto arbitrary 3D geometry, with photo-realistic rendering and interaction in real time, this very method of texture-like representation exposes it to common problems of texturing, striking in two disadvantages. Firstly, non-seamless textures create visible artifacts at boundaries. Secondly, even a perfectly seamless texture causes repetition artifacts due to their organised placement in large numbers over a 3D surface. We have solved both problems through our novel texture synthesis method that generates a set of seamless texture variations randomly distributed over the surface at shading time. When compared to regular 2D textures, the inter-dimensional coherence of the 4D ABTF material model poses entirely new challenges to texture synthesis, which includes maintaining the consistency of material behavior throughout the 4D space spanned by the spatial image domain and the angular illumination hemisphere. In addition, we tackle the increased memory consumption caused by the numerous variations through a fitting scheme specifically designed to reconstruct the most prominent effects captured in the material model.

Indexing of multi-biometric data is required to facilitate fast search in large-scale biometric systems. Previous works addressing this issue in multi-biometric databases focused on multi-instance indexing, mainly iris data. Few works addressed the indexing in multi-modal databases, with basic candidate list fusion solutions limited to joining face and fingerprint data. Iris and fingerprint are widely used in large-scale biometric systems where fast retrieval is a significant issue. This work proposes joint multi-biometric retrieval solution based on fingerprint and iris data. This solution is evaluated under eight different candidate list fusion approaches with variable complexity on a database of 10,000 reference and probe records of irises and fingerprints. Our proposed multi-biometric retrieval of fingerprint and iris data resulted in a reduction of the miss rate (1- hit rate) at 0.1% penetration rate by 93% compared to fingerprint indexing and 88% compared to iris indexing.

Accurate fingerprint gender estimation can positively affect several applications, since fingerprints are one of the most widely deployed biometrics. For example, gender classification in criminal investigations may significantly minimize the list of potential subjects. Previous work mainly offered solutions for the task of gender classification based on complete fingerprints. However, partial fingerprint captures are frequently occurring in many applications, including forensics and the fast growing field of consumer electronics. Moreover, partial fingerprints are not well-defined. Therefore, this work improves the gender decision performance on a well-defined partition of the fingerprint. It enhances gender estimation on the level of a single minutia. Working on this level, we propose three main contributions that were evaluated on a publicly available database. First, a convolutional neural network model is offered that outperformed baseline solutions based on hand crafted features. Second, several multi-algorithmic fusion approaches were tested by combining the outputs of different gender estimators that help further increase the classification accuracy. Third, we propose including minutia detection reliability in the fusion process, which leads to enhancing the total gender decision performance. The achieved gender classification performance of a single minutia is comparable to the accuracy that previous work reported on a quarter of aligned fingerprints including more than 25 minutiae.

Face recognition technology is spreading into a wide range of applications. This is mainly driven by social acceptance and the performance boost achieved by the deep learning-based solutions in the recent years. Perspective distortion is an understudied distortion in face recognition that causes converging verticals when imaging 3D objects depending on the distance to the object. The effect of this distortion on face recognition was previously studied for algorithms based on hand-crafted features with a clear negative effect on verification performance. Possible solutions were proposed by compensating the distortion effect on the face image level, which requires knowing the camera settings and capturing a high quality image. This work investigates the effect of perspective distortion on the performance of a deep learning-based face recognition solution. It also provides a device parameter-independent solution to decrease this effect by creating more perspective-robust face representations. This was achieved by training the deep learning model on perspective-diverse data, without increasing the size of the training data. Experiments performed on the deep model in hand and a specifically collected database concluded that the perspective distortion effects face verification performance if not considered in the training process, and that this can be improved by our proposal of creating robust face representations by properly selecting the training data.

Due to driver assistance systems and the trend of industry towards automated driving, the hands and feet of the driver in a vehicle require less intervention, becoming even idle. Recent gesture recognition focuses on hand interaction. This paper provides feet gesture interaction. Many gesture recognition systems rely on computing intensive video systems, causing privacy concerns. Furthermore, these systems require a line of sight and therefore a visible interior design integration. Our system proves that invisibly integrated capacitive proximity sensors can do this as well. They do not cause privacy issues and can be integrated under non-conductive materials. Therefore, there is no impact on the design of the visible interior. Our proposed solution distinguishes between four feet gestures. There is no limitation to feet movement. Further, we prove the functionality of the system in an evaluation with six participants, using a prototypical mock-up of a vehicle legroom. %This project contributes to the basis of driver feet gesture recognition pointing to further applications and more comprehensive investigations.

The choice of poses for camera calibration with planar patterns is only rarely considered - yet the calibration precision heavily depends on it. This work presents a pose selection method that finds a compact and robust set of calibration poses and is suitable for interactive calibration. Consequently, singular poses that would lead to an unreliable solution are avoided explicitly, while poses reducing the uncertainty of the calibration are favoured. For this, we use uncertainty propagation. Our method takes advantage of a self-identifying calibration pattern to track the camera pose in real-time. This allows to iteratively guide the user to the target poses, until the desired quality level is reached. Therefore, only a sparse set of key-frames is needed for calibration. The method is evaluated on separate training and testing sets, as well as on synthetic data. Our approach performs better than comparable solutions while requiring 30% less calibration frames.

In 3D object retrieval, additional knowledge like user input, classification information or database dependent configured parameters are rarely available in real scenarios. For example, meta data about 3D objects is seldom if the objects are not within a well-known evaluation database.

In the application of 3D object retrieval we search for 3D objects similar to a given query object. When a user searches for a certain class of objects like 'planes' the results can be unsatisfying: Many object variations are possible for a single class and not all of them are covered with one or a few example objects. We propose a meta model representation which corresponds to a procedural model with meta-parameters. Changing the meta-parameters leads to different variations of a 3D object. For the meta model generation a single object is constructed with a modeling tool. We automatically extract a procedural representation of the object. By inserting meta-parameters we generate our meta model. The meta model defines a whole object class. The user can choose a meta model and search for all objects similar to any instance of the meta model to retrieve all objects of a certain class from a 3D object database. We show that the retrieval precision is significantly improved using the meta model as retrieval query.

One of the major diseases affecting the global population, depression has a strong emotional impact on its sufferers. In this design study, "Talis" is presented as a wearable device which uses emotion recognition as an interface between patient and machine to support psychotherapeutic treatment. We combine two therapy methods, "Cognitive Behavioral Therapy" and "Well-Being Therapy", with interactive methods thought to increase their practical application potential. In this study, we draw on the results obtained in the area of "affective computing" for the use of emotions in empathic devices. The positive and negative phases experienced by the patient are identified through speech recognition and used for direct communication and later evaluation. After considering the design possibilities and suitable hardware, the future realization of such technology appears feasible. In order to design the wearable, user studies and technical experiments were carried out. The results of these suggest that the device could be beneficial for the treatment of patients with depression.

In recent time devices like Google Glass and Oculus Rift gained a lot of public attention. So the field of Virtual and Augmented Reality has become a more and more attractive field of study. Optical Stereo See-Through Head Worn Displays (OST-HWD or OST-HMD) can be used for Augmented Reality, but have to be calibrated. This means, that one has to find a configuration, that aligns the image shown on the displays with the environment, which is observed by the built-in camera. If this is not done, the augmented virtual image would not align with the real world. In this paper, the process of this calibration approach is divided into two stages, hardware and user calibration, but with less constraints for the positions of the cameras, which makes it easier to use. We aim at a more user friendly suite for the calibration of OST-HWD devices. Therefore both of the aforementioned stages are combined in a new quick step-by-step installation wizard, which is written in HTML and JavaScript to ensure easy usability. We apply a new minimization model in order to simplify and robustify the calculations of the virtual plane. In addition to that the required hardware components, including camera and calibration rig, were simplified. The implemented software has been evaluated for its results of the computed virtual plane, intrinsic data and eye positions of the user. Finally a user study was conducted to rate the usability of the calibration process.

Indexing of multi-biometric data is required to facilitate fast search in large-scale biometric systems. Previous works addressing this issue were challenged by including biometric sources of different nature, utilizing the knowledge about the biometric sources, and optimizing and tuning the retrieval performance. This work presents a generalized multi-biometric retrieval approach that adapts the Borda count algorithm within an optimizable structure. The approach was tested on a database of 10k reference and probe instances of the left and the right irises. The experiments and comparisons to five baseline solutions proved to achieve advances in terms of general indexing performance, tunability to certain operating points, and response to missing data. A clear advantage of the proposed solution was noticed when faced by candidate lists of low quality.

Foot interfaces, such as pressure-sensitive insoles, still yield unused potential such as for implicit interaction. In this paper, we introduce CapSoles, enabling smart insoles to implicitly identify who is walking on what kind of floor. Our insole prototype relies on capacitive sensing and is able to sense plantar pressure distribution underneath the foot, plus a capacitive ground coupling effect. By using machine-learning algorithms, we evaluated the identification of 13 users, while walking, with a confidence of ∼95% after a recognition delay of ∼1s. Once the user's gait is known, again we can discover irregularities in gait plus a varying ground coupling. While both effects in combination are usually unique for several ground surfaces, we demonstrate to distinguish six kinds of floors, which are sand, lawn, paving stone, carpet, linoleum, and tartan with an average accuracy of ∼82%. Moreover, we demonstrate the unique effects of wet and electrostatically charged surfaces.

The large scale of the recently demanded biometric systems has put a pressure on creating a more efficient, accurate, and private biometric solutions. Iris biometrics is one of the most distinctive and widely used biometric characteristics. High-performing iris representations suffer from the curse of rotation inconsistency. This is usually solved by assuming a range of rotational errors and performing a number of comparisons over this range, which results in a high computational effort and limits indexing and template protection. This work presents a generic and parameter-free transformation of binary iris representation into a rotation-invariant space. The goal is to perform accurate and efficient comparison and enable further indexing and template protection deployment. The proposed approach was tested on a database of 10 000 subjects of the ISYN1 iris database generated by CASIA. Besides providing a compact and rotational-invariant representation, the proposed approach reduced the equal error rate by more than 55% and the computational time by a factor of up to 44 compared to the original representation.

Tracking multiple persons is a challenging task when persons move in groups and occlude each other. Existing group-based methods have extensively investigated how to make group division more accurately in a tracking-by-detection framework; however, few of them quantify the group dynamics from the perspective of targets' spatial topology or consider the group in a dynamic view. Inspired by the sociological properties of pedestrians, we propose a novel socio-topology model with a topology-energy function to factor the group dynamics of moving persons and groups. In this model, minimizing the topology-energy-variance in a two-level energy form is expected to produce smooth topology transitions, stable group tracking, and accurate target association. To search for the strong minimum in energy variation, we design the discrete group-tracklet jump moves embedded in the gradient descent method, which ensures that the moves reduce the energy variation of group and trajectory alternately in the varying topology dimension. Experimental results on both RGB and RGB-D data sets show the superiority of our proposed model for multiple person tracking in crowd scenes.

Naturalness of the mid-air interaction interface for virtual assembly and shape modeling is important. In order to design an interface perceived as "natural" by most people, common behaviors and mental patterns for mid-air interaction of people have to be recognized, which is an area merely explored yet. This paper serves this purpose of understanding the users' mental interaction models, in order to provide standards and recommendation for devising a natural virtual interaction interface. We tested three kinds of tasks --- manipulating tasks, deforming tasks and tool-based operating tasks on 16 participants. We have found that: 1) different features of mental models were observed for different types of tasks. Interaction techniques should be designed to match these features; 2) virtual hand self-avatar helps estimate size of virtual objects, as well as helps plan and visualize the complex process and procedures of a task, which is especially helpful for tool-based tasks; 3) bimanual interaction is witnessed as a dominant interaction mode preferred by the majority; 4) natural gestures for deforming tasks always reflect forces exerted. These suggestions are useful for designing a midair interaction interface matching users' mental models.

In this paper, we propose a web-enabled shape modeling system with natural free-hand interaction, which can be easily learned by users while imposing least mental load on them. The deformation interface allows for performing various deformations, including stretching, compressing, squeezing, enlarging, twisting and tapering, on shapes interactively mimicking how they are done in real life. The manipulation interface allows an object to be directly grabbed and manipulated with either one or two hands, while also smoothly switching between them. Constrained methods are also provided for precise manipulation. An intuitive metaphor is designed to help the users to discover the interaction techniques by themselves without any manuals or instructions. A rendering pipeline, based on function-based extension of VRML/X3D, is designed with hidden complexity to support the proposed functionalities of the system. Hands motions are captured by Leap Motion controller. The user study proves the naturalness of the modeling system, and its easiness to be learned and remembered.

Platypus is the first system to localize and identify people by remotely and passively sensing changes in their body electric potential which occur naturally during walking. While it uses three or more electric potential sensors with a maximum range of 2 m, as a tag-free system it does not require the user to carry any special hardware. We describe the physical principles behind body electric potential changes, and a predictive mathematical model of how this affects a passive electric field sensor. By inverting this model and combining data from sensors, we infer a method for localizing people and experimentally demonstrate a median localization error of 0.16 m. We also use the model to remotely infer the change in body electric potential with a mean error of 8.8 % compared to direct contact-based measurements. We show how the reconstructed body electric potential differs from person to person and thereby how to perform identification. Based on short walking sequences of 5 s, we identify four users with an accuracy of 94 %, and 30 users with an accuracy of 75 %. We demonstrate that identification features are valid over multiple days, though change with footwear.

S.347-353

In this paper we present new constraints for calibration of underwater stereo-camera-systems and 3D-reconstruction. These constraints are both intuitive and simple to realize. We show that additionally needed refractive parameters in such a system can be calibrated simultaneously. Our constraints partially build upon each other. A subset of them even enables the calibration from stereo-correspondences alone, making known calibration targets unnecessary.

We present a novel multi-view photometric stereo technique that recovers the surface of texture less objects with unknown BRDF and lighting. The camera and light positions are allowed to vary freely and change in each image. We exploit orientation consistency between the target and an example object to develop a consistency measure. Motivated by the fact that normals can be recovered more reliably than depth, we represent our surface as both a depth map and a normal map. These maps are jointly optimized and allow us to formulate constraints on depth that take surface orientation into account. Our technique does not require the visual hull or stereo reconstructions for bootstrapping and solely exploits image intensities without the need for radiometric camera calibration. We present results on real objects with varying degree of specularity and show that these can be used to create globally consistent models from multiple views.

Smart objects within instrumented environments offer an always available and intuitive way of interacting with a system. Connecting these objects to other objects in range or even to smartphones and computers, enables substantially innovative interaction and sensing approaches. In this paper, we investigate the concept of Capacitive Near-Field Communication to enable ubiquitous interaction with everyday objects in a short-range spatial context. Our central contribution is a generic framework describing and evaluating this communication method in Ubiquitous Computing. We prove the relevance of our approach by an open-source implementation of a low-cost object tag and a transceiver offering a high-quality communication link at typical distances up to 15 cm. Moreover, we present three case studies considering tangible interaction for the visually impaired, natural interaction with everyday objects, and sleeping behavior analysis.

Stub autonomous systems usually utilize multiple links to single or multiple ISPs. Today, inbound traffic engineering is considered hard, as there is no direct way to influence routing decisions on remote systems with BGP. Current traffic engineering methods built on top of BGP are heuristic and time-consuming. The Locator/Identifier Separation Protocol (LISP) promises to change that. In this paper, we conduct the first comprehensive evaluation of LISP and its built-in traffic engineering methods on a real-world testbed. First, we compare LISP to plain BGP and BGP advertising more specific prefixes. This comparison shows that LISP allows effective load-balancing with an accuracy of approximately 5%, while being easier to configure than BGP and its variants. Further experiments show that these results are independent from the number of concurrent streams.

Computing ambient occlusion in screen-space (SSAO) is a common technique in real-time rendering applications which use rasterization to process 3D triangle data. However, one of the most critical problems emerging in screen-space is the lack of information regarding occluded geometry which does not pass the depth test and is therefore not resident in the G-buffer. These occluded fragments may have an impact on the proximity-based shadowing outcome of the ambient occlusion pass. This not only decreases image quality but also prevents the application of SSAO on multiple layers of transparent surfaces where the shadow contribution depends on opacity. We propose a novel approach to the SSAO concept by taking advantage of per-pixel fragment lists to store multiple geometric layers of the scene in the G-buffer, thus allowing order independent transparency (OIT) in combination with high quality, opacity-based ambient occlusion (OITAO). This A-buffer concept is also used to enhance overall ambient occlusion quality by providing stable results for low-frequency details in dynamic scenes. Furthermore, a flexible compression-based optimization strategy is introduced to improve performance while maintaining high quality results.

Swiss-Cheese Extended proposes a novel real-time method for recognizing objects with capacitive proximity sensors. Applying this technique to ubiquitous user interfaces, it is possible to detect the 3D-position of multiple human hands in different configurations above a surface that is equipped with a small number of sensors. The retrieved object configurations can significantly improve a user's interaction experience or an application's execution context, for example by detecting multi-hand zoom and rotation gestures or recognizing a grasping hand. We emphasize the broad applicability of the proposed method with a study of a multi-hand gesture recognition device.

Capacitive sensing allows the creation of unobtrusive user interfaces that are based on measuring the proximity to objects or recognizing their dielectric properties. Combining the data of many sensors, applications such as in-the-air gesture recognition, location tracking or fluid-level sensing can be realized. We present OpenCapSense, a highly flexible open-source toolkit that enables researchers to implement new types of pervasive user interfaces with low effort. The toolkit offers a high temporal resolution with sensor update rates up to 1 kHz. The typical spatial resolution varies between one millimeter at close object proximity and around one centimeter at distances of 35cm or above.

The 3D-segmentation of lymph nodes in computed tomography images is required for staging and disease progression monitoring. Major challenges are shape and size variance, as well as low contrast, image noise, and pathologies. In this paper, radial ray based segmentation is applied to lymph nodes. From a seed point, rays are cast into all directions and an optimization technique determines a radius for each ray based on image appearance and shape knowledge. Lymph node specific appearance cost functions are introduced and their optimal parameters are determined. For the first time, the resulting segmentation accuracy of different appearance cost functions and optimization strategies is compared. Further contributions are extensions to reduce the dependency on the seed point, to support a larger variety of shapes, and to enable interaction. The best results are obtained using graph-cut on a combination of the direction weighted image gradient and accumulated intensities outside a predefined intensity range. Evaluation on 100 lymph nodes shows that with an average symmetric surface distance of 0.41 mm the segmentation accuracy is close to manual segmentation and outperforms existing radial ray and model based methods. The method's inter-observer-variability of 5.9% for volume assessment is lower than the 15.9% obtained using manual segmentation.

A widely used technique to recover a 3D surface from photographs is patch-based (multi-view) stereo reconstruction. Current methods are able to reproduce fine surface details, they are however limited by the sampling density and the patch size used for reconstruction. We show that there is a systematic error in the reconstruction depending on the details in the unknown surface (frequencies) and the reconstruction resolution. For this purpose we present a theoretical analysis of patch-based depth reconstruction. We prove that our model of the reconstruction process yields a linear system, allowing us to apply the transfer (or system) function concept. We derive the modulation transfer function theoretically and validate it experimentally on synthetic examples using rendered images as well as on photographs of a 3D test target. Our analysis proves that there is a significant but predictable amplitude loss in reconstructions of fine scale details. In a first experiment on real-world data we show how this can be compensated for within the limits of noise and reconstruction accuracy by an inverse transfer function in frequency space.

Recently, depth cameras have emerged which capture dense depth images in real-time. To benefit from their 3D imaging capabilities in interactive applications which support an arbitrary camera movement, the position and orientation of the depth camera needs to be robustly estimated in real time for each captured depth image. Therefore, this paper describes how to combine a depth camera with a mechanical measurement arm to fuse real-time depth imaging with real-time, high precision pose estimation. Estimating the pose of a depth camera with a measurement arm has three major advantages over 2D/3D image based optical pose estimation: The measurement arm has a very precise guaranteed accuracy better than 0.1mm, the pose estimation accuracy is not influenced by the captured scene and the computational load is much lower than for optical pose estimation, leaving more processing power for the applications themselves.

Concerning gestural interaction in realistic environments there often is an offset between perceived and actual direction of pointing that makes it difficult to reliably select elements in the environment. This work presents a visual support system that provides feedback to a user gesturing freely in an environment and thus enabling reliable selection of and interaction with reactive elements in intelligent environments. A prototype has been created that is showcasing this feedback method based on gesture recognition using the Microsoft Kinect and feedback provision using a custom laser-robot. Finally an evaluation has been performed, in order to prove the efficiency of such a system, acquire usability feedback and determine potential learning effects for gesture-based interaction.

An pipeline for reconstructing the 3D face model from an uncontrolled video sequence is presented which involves three major steps. Firstly, a generic deformable 3D face model is built from the 3D scans of one hundred individuals. Secondly, the 3D face shape from a video sequence is constructed by estimating poses of images using structure-from-motion technique and dense correspondences between those images by employing Huber-L1optical flow algorithm. Finally, the generated generic deformable 3D face model can be fitted to the reconstructed 3D face-shape from a video sequence provided that the deviation from the real 3D face is less than certain thresholds. The application is developed to reconstruct the 3D face-shape in nearly uncontrolled environment so the results cannot be expected to be very accurate. We discuss the steps taken to perform the first and second steps. The factors affecting the depth estimation in face region cause major accuracy problems. They are analyzed and possible improvements to enhance the 3D face-shape reconstruction are presented.

3D Object Retrieval is an important field of research with many application possibilities. One of the main goals in this research is the development of discriminative methods for similarity search. The descriptor-based approach to date has seen a lot of research attention, with many different extraction algorithms proposed. In previous work, we have introduced a simple but effective scheme for 3D model retrieval based on a spatially fixed combination of 3D object fragment descriptors. In this work, we propose a novel flexible combination scheme based on finding the best matching fragment descriptors to use in the combination. By an exhaustive experimental evaluation on established benchmark data we show the capability of the new combination scheme to provide improved retrieval effectiveness. The method is proposed as a versatile and inexpensive method to enhance the effectiveness of a given global 3D descriptor approach.

A personal computer can be considered as a one-node heterogeneous cluster that simultaneously processes several application tasks. It can be composed by, for example, asymmetric CPU and GPUs. This way, a high-performance heterogeneous platform is built on a desktop for data intensive engineering calculations. In our perspective, a workload distribution over the Processing Units (PUs) plays a key role in such systems. This issue presents challenges since the cost of a task at a PU is non-deterministic and can be affected by parameters not known a priori. This paper presents a context-aware runtime and tuning system based on a compromise between reducing the execution time of engineering applications - due to appropriate dynamic scheduling - and the cost of computing such scheduling applied on a platform composed of CPU and GPUs. Results obtained in experimental case studies are encouraging and a performance gain of 21.77% was achieved in comparison to the static assignment of all tasks to the GPU.

Fuzzy commitment is an efficient template protection algorithm that can improve security and safeguard privacy of biometrics. Existing theoretical security analysis has proved that although privacy leakage is unavoidable, perfect security from information-theoretical points of view is possible when bits extracted from biometric features are uniformly and independently distributed. Unfortunately, this strict condition is difficult to fulfill in practice. In many applications, dependency of binary features is ignored and security is thus suspected to be highly overestimated. This paper gives a comprehensive analysis on security and privacy of fuzzy commitment regarding empirical evaluation. The criteria representing requirements in practical applications are investigated and measured quantitatively in an existing protection system for 3D face recognition. The evaluation results show that a very significant reduction of security and enlargement of privacy leakage occur due to the dependency of biometric features. This work shows that in practice, one has to explicitly measure the security and privacy instead of trusting results under non-realistic assumptions.

Abstract The analysis of large graphs plays a prominent role in various fields of research and is relevant in many important application areas. Effective visual analysis of graphs requires appropriate visual presentations in combination with respective user interaction facilities and algorithmic graph analysis methods. How to design appropriate graph analysis systems depends on many factors, including the type of graph describing the data, the analytical task at hand and the applicability of graph analysis methods. The most recent surveys of graph visualization and navigation techniques cover techniques that had been introduced until 2000 or concentrate only on graph layouts published until 2002. Recently, new techniques have been developed covering a broader range of graph types, such as time‐varying graphs. Also, in accordance with ever growing amounts of graph‐structured data becoming available, the inclusion of algorithmic graph analysis and interaction techniques becomes increasingly important. In this State‐of‐the‐Art Report, we survey available techniques for the visual analysis of large graphs. Our review first considers graph visualization techniques according to the type of graphs supported. The visualization techniques form the basis for the presentation of interaction approaches suitable for visual graph exploration. As an important component of visual graph analysis, we discuss various graph algorithmic aspects useful for the different stages of the visual graph analysis process. We also present main open research challenges in this field.

GPUs (Graphics Processing Units) have become one of the main co-processors that contributed to desktops towards high performance computing. Together with multi-core CPUs, a powerful heterogeneous execution platform is built for massive calculations. To improve application performance and explore this heterogeneity, a distribution of workload in a balanced way over the PUs (Processing Units) plays an important role for the system. However, this problem faces challenges since the cost of a task at a PU is non-deterministic and can be influenced by several parameters not known a priori, like the problem size domain. We present a comparison of iterative SLE (Systems of Linear Equations) solvers, used in many scientific and engineering applications, over a heterogeneous CPU-GPUs platform and characterize scenarios where the solvers obtain better performances. A new technique to improve memory access on matrix-vector multiplication used by SLEs on GPUs is described and compared to standard implementations for CPU and GPUs. Such timing profiling is analyzed and break-even points based on the problem sizes are identified for this implementation, pointing whether our technique is faster to use GPU instead of CPU. Preliminary results show the importance of this study applied to a real-time CFD (Computational Fluid Dynamics) application with geometry modification.

S.3740-3743

The number of available 3D models in various areas increase steadily. Effective methods to search for those 3D models by content, rather than textual annotations, are crucial. For this purpose, we propose a new approach for content based 3D model retrieval by hand-drawn sketch images. This approach to retrieve visually similar mesh models from a large database consists of three major steps: (1) suggestive contour renderings from different viewpoints to compare against the user drawn sketches; (2) descriptor computation by analyzing diffusion tensor fields of suggestive contour images or the query sketch respectively; (3) similarity measurement to retrieve the models and the most probable view-point from which a model was sketched. Our proposed sketch based 3D model retrieval system is very robust against variations of shape, pose or partial occlusion of the user draw sketches. Experimental results are presented and indicate the effectiveness of our approach for sketch-based 3D mode retrieval.

With the huge amount of cell images produced in bio-imaging, automatic methods for segmentation are needed in order to evaluate the content of the images with respect to types of cells and their sizes. Traditional PDE-based methods using level-sets can perform automatic segmentation, but do not perform well on images with clustered cells containing sub-structures. Furthermore, DIC images contain a phase gradient, which should be removed first. We present a method that removes this gradient, finds the cell centres and derives the relevant individual cells.

In this work, we take a novel line of approaches to evolve images. It is motivated by the total variation method, known for its denoising and edge-preserving effect. Our approach generalises the TV method by taking a general LPnorm of the gradients instead of the L1in the TV method. We generalise this method in a series of first and second order derivatives in terms of gauge coordinates. This method also incorporates the well-known blurring by a Gaussian filter and the balanced forward -backward diffusion. The method and its properties are briefly discussed. The practical results are visualised on a real-life image, showing the expected behaviour. When a constraint is added that penalises the distance of the results to the input image, one can vary the desired amount of blurring and denoising.

In this paper we present a novel method to derive the skeleton of a shape by means of its symmetry set. We use the property that the medial axis is a subset of the symmetry set. This set is obtained as zero crossings of two equations, based on geometric arguments. The resulting skeleton can easily be annotated with respect to main axis and branches, since this information follows directly from the symmetry set.

Using a Gaussian scale space, one can use the extra dimension, viz. scale, for investigation of "built-in" properties of the image in scale space. We show that one of such induced properties is the nesting of special iso-intensity manifolds, which yield an implicitly present hierarchy of the critical points and regions of their influence, in the original image. Its very nature allows one not only to segment the original image automatically, but also to apply "logical filters" to it, obtaining simplified images. We give an algorithm deriving this hierarchy and show its effectiveness on two different kinds of images, both with respect to segmentation and simplification.