Dynamic Vision Sensor (DVS)-based solutions have recently garnered significant interest across various computer vision tasks, offering notable benefits in terms of dynamic range, temporal resolution, and inference speed. However, as a relatively nascent vision sensor compared to Active Pixel Sensor (APS) devices such as RGB cameras, DVS suffers from a dearth of ample labeled datasets. Prior efforts to convert APS data into events often grapple with issues such as a considerable domain shift from real events, the absence of quantified validation, and layering problems within the time axis. In this paper, we present a novel method for video-to-events stream conversion from multiple perspectives, considering the specific characteristics of DVS. A series of carefully designed losses helps enhance the quality of generated event voxels significantly. We also propose a novel local dynamic-aware timestamp inference strategy to accurately recover event timestamps from event voxels in a continuous fashion and eliminate the temporal layering problem. Results from rigorous validation through quantified metrics at all stages of the pipeline establish our method unquestionably as the current state-of-the-art (SOTA). The code can be found at bit.ly/v2ce.

Technology-mediated dance experiences, as a medium of entertainment, are a key element in both traditional and virtual reality-based gaming platforms. These platforms predominantly depend on unobtrusive and continuous human pose estimation as a means of capturing input. Current solutions primarily employ RGB or RGB-Depth cameras for dance gaming applications; however, the former is hindered by low-light conditions due to motion blur and reduced sensitivity, while the latter exhibits excessive power consumption, diminished frame rates, and restricted operational distance. Boasting ultra-low latency, energy efficiency, and a wide dynamic range, neuromorphic cameras present a viable solution to surmount these limitations. Here, we introduce YeLan, a neuromorphic camera-driven, three-dimensional, high-frequency human pose estimation (HPE) system capable of withstanding low-light environments and dynamic backgrounds. We have compiled the first-ever neuromorphic camera dance HPE dataset and devised a fully adaptable motion-to-event, physics-conscious simulator. YeLan surpasses baseline models under strenuous conditions and exhibits resilience against varying clothing types, background motion, viewing angles, occlusions, and lighting fluctuations.

A set of advanced approaches and models on human action, activity, gesture, and behavior recognition related aspects along with associated challenges are summarized in this note. Notably, Video-based human activity recognition, sensor-based activity analysis, skeleton-based activity recognition, assisted daily living, anomaly detection, facial expression and emotion analysis, gesture and sign language are highlighted in the works. This special issue also introduced six new datasets, while exploring a total of 55 different datasets in its 21 research articles. Apart from the the areas covered in this issue, research on multi-modal analysis, action quality assessment, real-time applications, and activity and behavior computing on edge devices are some of the dominant future challenges to deal with. We firmly believe that the original works and ideas presented in this special issue will serve as helpful references for the relevant research communities in the journey towards a brighter future.

Given the relevance of smartphones for accessing personalized services in smart cities, Continuous Authentication (CA) mechanisms are attracting attention to avoid impersonation attacks. Some of them leverage Data Stream Mining (DSM) techniques applied over sensorial information. Injection attacks can undermine the effectiveness of DSM-based CA by fabricating artificial sensorial readings.The goal of this paper is to study the impact of injection attacks in terms of accuracy and immediacy to illustrate the time the adversary remains unnoticed. Two well-known DSM techniques (K-Nearest Neighbours and Hoeffding Adaptive Trees) and three data sources (location, gyroscope and accelerometer) are considered due to their widespread usage Results show that even if the attacker does not previously know anything about the victim, a significant attack surface arises – 1.35 min are needed, in the best case, to detect the attack on gyroscope and accelerometer and 7.27 min on location data. Moreover, we show that the type of sensor at stake and configuration settings may have a dramatic effect on countering this threat.

State-of-the-art methods of attribute detection from faces almost always assume the presence of a full, unoccluded face. Hence, their performance degrades for partially visible and occluded faces. In this paper, we introduce SPLITFACE, a deep convolutional neural network-based method that is explicitly designed to perform attribute detection in partially occluded faces. Taking several facial segments and the full face as input, the proposed method takes a data driven approach to determine which attributes are localized in which facial segments. The unique architecture of the network allows each attribute to be predicted by multiple segments, which permits the implementation of committee machine techniques for combining local and global decisions to boost performance. With access to segment-based predictions, SPLITFACE can predict well those attributes which are localized in the visible parts of the face, without having to rely on the presence of the whole face. We use the CelebA and LFWA facial attribute datasets for standard evaluations. We also modify both datasets, to occlude the faces, so that we can evaluate the performance of attribute detection algorithms on partial faces. Our evaluation shows that SPLITFACE significantly outperforms other recent methods especially for partial faces.

Generic face detection algorithms do not perform very well in the mobile domain due to significant presence of occluded and partially visible faces. One promising technique to handle the challenge of partial faces is to design face detectors based on facial segments. In this paper two such face detectors namely, SegFace and DeepSegFace, are proposed that detect the presence of a face given arbitrary combinations of certain face segments. Both methods use proposals from facial segments as input that are found using weak boosted classifiers. SegFace is a shallow and fast algorithm using traditional features, tailored for situations where real time constraints must be satisfied. On the other hand, DeepSegFace is a more powerful algorithm based on a deep convolutional neutral network (DCNN) architecture. DeepSegFace offers certain advantages over other DCNN-based face detectors as it requires relatively small amount of data to train by utilizing a novel data augmentation scheme and is very robust to occlusion by design. Extensive experiments show the superiority of the proposed methods, specially DeepSegFace, over other state-of-the-art face detectors in terms of precision-recall and ROC curve on two mobile face datasets.

In this paper, a part-based technique for real time detection of users' faces on mobile devices is proposed. This method is specifically designed for detecting partially cropped and occluded faces captured using a smartphone's front-facing camera for continuous authentication. The key idea is to detect facial segments in the frame and cluster the results to obtain the region which is most likely to contain a face. Extensive experimentation on a mobile dataset of 50 users shows that our method performs better than many state-of-the-art face detection methods in terms of accuracy and processing speed.

In this paper, we propose a novel approach towards human action recognition using spectral domain feature extraction. Action representations can be considered as image templates, which can be useful for understanding various actions or gestures as well as for recognition and analysis. An action recognition scheme is developed based on extracting spectral features from the frames of a video sequence using the two-dimensional discrete Fourier transform (2D-DFT). The proposed spectral feature selection algorithm offers the advantage of very low feature dimensionality and thus lower computational cost. We show that using frequency domain features enhances the distinguishability of different actions, resulting in high within-class compactness and between-class separability of the extracted features, while certain undesirable phenomena, such as camera movement and change in camera distance, are less severe in the frequency domain. Principal component analysis is performed to further reduce the dimensionality of the feature space. Experimental results on a benchmark action recognition database confirm that our proposed method offers not only computational savings but also a high degree of accuracy.

This paper presents an approach to detect voice disorders based on wavelet and prosody-related voice properties. First, several statistical measures of the normalized energy contents of the Discrete Wavelet Transform (DWT) coefficients over all voice frames are determined. Then, similar statistical measures of some prosody-related voice properties, such as mean pitch, jitter and shimmer are also computed over all the frames. In order to form a feature vector to be used in both training and testing phases, a set of statistical measure of the normalized energy contents of the DWT coefficients is combined with a set of statistical measure of the extracted prosody-related voice properties. Here, the voice samples under consideration are assumed to be of two categories, namely healthy and disordered thus formulating the problem in the proposed method as a two-class problem to be solved. Finally, the feature vector as obtained above is fed to an Euclidean Distance based classifier to detect the disordered voice. By performing extensive simulations, it is shown that the statistical analysis based on wavelet and prosody-related properties are able to provide effective detection of a variety of voice disorders from the mixture of healthy and disordered voices.