Human Activity Recognition (HAR) benefits various application domains, including health and elderly care. Traditional HAR involves constructing pipelines reliant on centralized user data, which can pose privacy concerns as they necessitate the uploading of user data to a centralized server. This work proposes multi-frequency Federated Learning (FL) to enable: (1) privacy-aware ML; (2) joint ML model learning across devices with varying sampling frequency. We focus on head-worn devices (e.g., earbuds and smart glasses), a relatively unexplored domain compared to traditional smartwatch- or smartphone-based HAR. Results have shown improvements on two datasets against frequency-specific approaches, indicating a promising future in the multi-frequency FL-HAR task. The proposed network’s implementation is publicly available for further research and development.**

Sensor-based sleep monitoring systems can be used to track sleep behavior on a daily basis and provide feedback to their users to promote health and well-being. Such systems can provide data visualizations to enable self-reflection on sleep habits or a sleep coaching service to improve sleep quality. To provide useful feedback, sleep monitoring systems must be able to recognize whether an individual is sleeping or awake. Existing approaches to infer sleep-wake phases, however, typically assume continuous streams of data to be available at inference time. In real-world settings, though, data streams or data samples may be missing, causing severe performance degradation of models trained on complete data streams. In this paper, we investigate the impact of missing data to recognize sleep and wake, and use regression- and interpolation-based imputation strategies to mitigate the errors that might be caused by incomplete data. To evaluate our approach, we use a data set that includes physiological traces - collected using wristbands -, behavioral data - gathered using smartphones - and self-reports from 16 participants over 30 days. Our results show that the presence of missing sensor data degrades the balanced accuracy of the classifier on average by 10–35 percentage points for detecting sleep and wake depending on the missing data rate. The impu-tation strategies explored in this work increase the performance of the classifier by 4–30 percentage points. These results open up new opportunities to improve the robustness of sleep monitoring systems against missing data.

Flow is a positive affective state occurring when individuals are fully immersed into an activity. Being in flow during work activities can lead to higher performance and productivity. Despite the importance of flow at work, few approaches have been proposed for its automatic recognition using sensor data and most existing studies are conducted in laboratory settings with simulated work activities. In this paper, we investigate the use of physiological data, collected using wrist-worn devices, combined with context information, obtained through self-reports, to automatically distinguish between low and high levels of flow. We investigate the role of the context for flow perceptions and in its automatic recognition. Further, we compare the performance of several sensor fusion strategies based on shallow and deep learning. To evaluate our approach we use a data set of 390 activities collected during actual work days. Our results show that using raw blood volume pulse, electrodermal activity and the type of activity as input to a sensor-based late fusion approach, implemented using convolutional neural networks, allows to reach a balanced accuracy of 70.93%.

Head gestures and facial expressions – like, e.g., nodding or smiling – are important indicators of the quality of human interactions in physical meetings as well as in computer-mediated settings. Computer systems able to recognize such behavioral cues can support and improve human interactions. Several researchers have thus tackled the problem of automatically recognizing head gestures and facial expressions, mainly leveraging video data. In this paper, we instead consider inertial signals collected from unobtrusive, ear-mounted devices. We focus on typical activities performed during social interactions – head shaking, nodding, smiling, talking and yawning – and propose a hierarchical classification approach to discriminate them from each other. Further, we investigate whether the transfer of knowledge learned from publicly available datasets leads to further performance improvements. Our results show that the combined use of our hierarchical approach and transfer learning allows the classifier to discriminate head and mouth activities with an F1 score of 84.79, smile, talk and yawn with an F1 score of 45.42, and nodding and head shaking with an F1 score of 88.24, outperforming shallow classifiers by 2-9 percentage points.