This paper presents one of the first studies on estimating group emotional states in immersive communication environments. We designed a two-phase experiment to assess group emotion recognition in both controlled and natural settings. In the forced-emotion phase, 32 participants mimicked predefined emotions in 2D and 360° video. In the natural-emotion phase, the OpenLAV dataset elicited stronger and more diverse affective responses than LIRIS-ACCEDE, indicating better suitability for triggering emotions. We also found that neutral states are mostly detected as slightly negative valence, suggesting a systematic bias in affect estimation. The study also found greater variability in valence than in arousal, potentially due to the professional setting of the experiment. Overall, the findings demonstrate the feasibility of coarse group emotion detection in 360° video showing a high correlation with participants’ self-perceptions. We believe this tool could be useful for making immersive communication more effective for audience monitoring, enhancing presentation skills, but also more inclusive for people with emotion understanding difficulties.

This paper evaluates a virtual teleportation system based on 360-degree video communication in a field study at a secondary school and a guided debate training session. Socioemotional aspects, including social presence and avatar representation, were assessed by eight teens. Key findings include a higher "wow" effect for those joining the immersive environment, and challenges with avatar interaction such as raising their hand. The system shows promise as an educational tool for debate training and student engagement.

We present a novel dataset for the evaluation of face detection and recognition algorithms in challenging surveillance scenarios. The dataset consists in 4K images of different subjects captured at annotated distances ranging from 1 to 30 meters, both in indoor and outdoor environments, and under two face mask conditions (with and without). To the best of our knowledge, this is the only existing dataset that addresses the joint impact of masks and distances in a rigorous manner. We also propose an end-to-end fully automatic face detection and recognition system to provide baseline results on this dataset. Face detection is performed using Tiny Faces network, while face recognition is performed using VGG Face network. Experimental results show very high detection and recognition rates up to a distance of 20 meters, where the impact of distance is clear (especially for the latter). The use of face masks degrades the detection range and produces less consistent recognition results.

Media forensics has attracted a tremendous attention in the last years in part due to the increasing concerns around DeepFakes. Since the release of the initial DeepFakes databases of the 1st generation such as UADFV and FaceForensics++ up to the latest databases of the 2nd generation such as Celeb-DF and DFDC, many visual improvements have been carried out, making fake videos almost indistinguishable to the human eye. This study provides an in-depth analysis of both 1st and 2nd DeepFakes generations in terms of fake detection performance. Two different methods are considered in our experimental framework: (i) the traditional one followed in the literature based on selecting the entire face as input to the fake detection system, and (ii) a novel approach based on the selection of specific facial regions as input to the fake detection system. Fusion techniques are applied both to the facial regions and also to three different state-of-the-art fake detection systems (Xception, Capsule Network, and DSP-FWA) in order to further increase the robustness of the detectors considered. Finally, experiments regarding intra- and inter-database scenarios are performed. Among all the findings resulting from our experiments, we highlight: (i) the very good results achieved using facial regions and fusion techniques with fake detection results above 99% Area Under the Curve (AUC) for UADFV, FaceForensics++, and Celeb-DF v2 databases, and (ii) the necessity to put more efforts on the analysis of inter-database scenarios to improve the ability of the fake detectors against attacks unseen during learning.

In this paper we evaluate body static information to improve the performance of face recognition at a distance. To this aim, we assess one state-of-the-art face recognition system based on deep features and three body-based person recognition systems, namely: i) row profiles with correlation coefficient, ii) row and column profiles with Support Vector Machines, and iii) contour coordinates with Dynamic Time Warping. Results are reported using the Multi-Biometric Tunnel Database, emphasizing on three distance settings: far, medium, and close, ranging from full body exposure to head and shoulders exposure. Several conclusions can be drawn from this work: a) row and column profiles are more robust than contour coordinates, b) face-based systems perform poorly at far distances, being body-based information more reliable at that distances, c) in general face-based systems perform better than body-based approaches at medium and close distances, and d) the multimodal fusion approach manages to outperform face-only recognition at distance in all distance-settings considered.

The role of soft biometrics to enhance person recognition systems in unconstrained scenarios has not been extensively studied. Here, we explore the utility of the following modalities: gender, ethnicity, age, glasses, beard, and moustache. We consider two assumptions: 1) manual estimation of soft biometrics and 2) automatic estimation from two commercial off-the-shelf systems (COTS). All experiments are reported using the labeled faces in the wild (LFW) database. First, we study the discrimination capabilities of soft biometrics standalone. Then, experiments are carried out fusing soft biometrics with two state-of-the-art face recognition systems based on deep learning. We observe that soft biometrics is a valuable complement to the face modality in unconstrained scenarios, with relative improvements up to 40%/15% in the verification performance when using manual/automatic soft biometrics estimation. Results are reproducible as we make public our manual annotations and COTS outputs of soft biometrics over LFW, as well as the face recognition scores.

Due to the ability of millimeter waves (mmWs) to penetrate dielectric materials, such as plastic, polymer, and clothes, the mmW imaging technology has been widely used for the detection of concealed weapons and objects. The use of mmW images has also recently been proposed for biometric person recognition to overcome certain limitations in image acquisition at visible frequencies. This paper proposes a biometric person recognition system based on the shape information extracted from real mmW images. To this aim, we report experimental results using the mmW images with different body shape-based feature approaches, such as contour coordinates, shape contexts, Fourier descriptors, and row and column profiles. We also study various distance-based and classifier-based matching schemes. Experimental results suggest the potential of performing person recognition through mmW imaging using only shape information, a functionality that could be integrated in the security scanners deployed in airports.

Gender estimation has received increased attention due to its use in a number of pertinent security and commercial applications. Automated gender estimation algorithms are mainly based on extracting representative features from face images. In this work we study gender estimation based on information deduced jointly from face and body, extracted from single-shot images. The approach addresses challenging settings such as low-resolution-images, as well as settings when faces are occluded. Specifically the face-based features include local binary patterns (LBP) and scale-invariant feature transform (SIFT) features, projected into a PCA space. The features of the novel body-based algorithm proposed in this work include continuous shape information extracted from body silhouettes and texture information retained by HOG descriptors. Support Vector Machines (SVMs) are used for classification for body and face features. We conduct experiments on images extracted from video-sequences of the Multi-Biometric Tunnel database, emphasizing on three distance-settings: close, medium and far, ranging from full body exposure (far setting) to head and shoulders exposure (close setting). The experiments suggest that while face-based gender estimation performs best in the close-distance-setting, body-based gender estimation performs best when a large part of the body is visible. Finally we present two score-level-fusion schemes of face and body-based features, outperforming the two individual modalities in most cases.

The use of Millimetre wave images has been proposed recently in the biometric field to overcome certain limitations when using images acquired at visible frequencies. In this paper, several body shape-based techniques were applied to model the silhouette of images of people acquired at 94 GHz. We put forward several methods for the parameterization and classification stage with the objective of finding the best configuration in terms of biometric recognition performance. Contour coordinates, shape contexts, Fourier descriptors and silhouette landmarks were used as feature approaches and for classification we utilized Euclidean distance and a dynamic programming method. Results showed that the dynamic programming algorithm improved the performance of the system with respect to the baseline Euclidean distance and the necessity of a minimum resolution of the contour to achieve promising equal error rates. The use of the contour coordinates is the most suitable feature to use in the system regarding the performance and the computational cost involved when having at least 3 images for model training. Besides, Fourier descriptors are more robust against rotations, which may be of interest when dealing with few training images.