Human Activity Recognition (HAR) with mobile and wearable devices has been deeply studied in the last decades. Research groups working on this topic evaluated their proposed methods mostly on public datasets. However, most of the existing datasets only include inertial sensor data, while it is well-known that additional context data (e.g., semantic location) has the potential to significantly improve the recognition rate. Only a few datasets for context-aware HAR are publicly available, and their annotations were mostly self-reported in-the-wild by the subjects involved in data acquisition. This method harms the quality of annotations, thus discouraging the application of supervised models. In this paper, we propose DOMINO, a new public dataset for context-aware HAR. DOMINO includes 25 users (wearing a smartphone and a smartwatch) performing 14 activities. During data acquisition, the mobile devices recorded both inertial and high-level context data while our team monitored the quality of the self-reported annotations. Our experiments on DOMINO show the positive impact of considering high-level context information for Human Activity Recognition.

The use of supervised learning for Human Activity Recognition (HAR) on mobile devices leads to strong classification performances. Such an approach, however, requires large amounts of labeled data, both for the initial training of the models and for their customization on specific clients (whose data often differ greatly from the training data). This is actually impractical to obtain due to the costs, intrusiveness, and time-consuming nature of data annotation. Moreover, even with the help of a significant amount of labeled data, model deployment on heterogeneous clients faces difficulties in generalizing well on unseen data. Other domains, like Computer Vision or Natural Language Processing, have proposed the notion of pre-trained models, leveraging large corpora, to reduce the need for annotated data and better manage heterogeneity. This promising approach has not been implemented in the HAR domain so far because of the lack of public datasets of sufficient size. In this paper, we propose a novel strategy to combine publicly available datasets with the goal of learning a generalized HAR model that can be fine-tuned using a limited amount of labeled data on an unseen target domain. Our experimental evaluation, which includes experimenting with different state-of-the-art neural network architectures, shows that combining public datasets can significantly reduce the number of labeled samples required to achieve satisfactory performance on an unseen target domain.

Sensor-based Human Activity Recognition (HAR) has been a hot topic in pervasive computing for several years mainly due to its applications in healthcare and well-being. Centralized supervised approaches reach very high recognition rates, but they incur privacy and scalability issues. Federated Learning (FL) has been recently proposed to mitigate these issues. Each subject only shares the weights of a personal model trained locally, instead of sharing data. A cloud server is in charge of aggregating the weights to generate a global model. However, since activity data is non-independently and identically distributed (non-IID), a single model may not be sufficiently accurate for a large number of diverse users. In this work, we propose FedCLAR, a novel federated clustering method for HAR. Based on the similarity of the local model updates, the cloud server in FedCLAR derives groups of users that exhibit similar ways of performing activities. For each group, FedCLAR uses a specialized global model to mitigate the non-IID problem. We evaluated FedCLAR on two well-known public datasets, showing that it outperforms state-of-the-art FL solutions.

Abstract One of the major open problems in sensor-based Human Activity Recognition (HAR) is the scarcity of labeled data. Among the many solutions to address this challenge, semi-supervised learning approaches represent a promising direction. However, their centralized architecture incurs in the scalability and privacy problems that arise when the process involves a large number of users. Federated learning (FL) is a promising paradigm to address these problems. However, the FL methods that have been proposed for HAR assume that the participating users can always obtain labels to train their local models (i.e., they assume a fully supervised setting). In this work, we propose FedAR: a novel hybrid method for HAR that combines semi-supervised and federated learning to take advantage of the strengths of both approaches. FedAR combines active learning and label propagation to semi-automatically annotate the local streams of unlabeled sensor data, and it relies on FL to build a global activity model in a scalable and privacy-aware fashion. FedAR also includes a transfer learning strategy to fine-tune the global model on each user. We evaluated our method on two public datasets, showing that FedAR reaches recognition rates and personalization capabilities similar to state-of-the-art FL supervised approaches. As a major advantage, FedAR only requires a very limited number of annotated data to populate a pre-trained model and a small number of active learning questions that quickly decrease while using the system, leading to an effective and scalable solution for the data scarcity problem of HAR.

In the field of Human Activity Recognition (HAR) the majority of approaches exploit fully supervised methodologies to process inertial sensor data collected from the users' wearable devices. Unfortunately, those solutions require users to collect a large number of annotated examples to train the recognition model, which is costly, unpractical, and time-consuming. In this paper, we propose diverse semi-supervised methodologies to tackle the data scarcity issue in the field of HAR. In particular, in Caviar and ProCaviar we introduce novel knowledge-based reasoning engines that exploiting the context data (e.g. semantic location, weather condition) allows a statistical classifier trained with a limited number of example to recognise a wide set of activities. Then, we propose FedHAR an hybrid semi-supervised and Federated-learning based system that enables distributing the training of an activity recognition model among a large number of subject, reducing the effort required from users to collect annotated data while preserving their privacy.

Recognizing the activities of daily living (ADLs) in multi-inhabitant settings is a challenging task. One of the major challenges is the so-called data association problem: how to assign to each user the environmental sensor events that he/she actually triggered? In this paper, we tackle this problem with a contextaware approach. Each user in the home wears a smartwatch, which is used to gather several high-level context information, like the location in the home (thanks to a micro-localization infrastructure) and the posture (e.g., sitting or standing). Context data is used to associate sensor events to the users which more likely triggered them. We show the impact of context reasoning in our framework on a dataset where up to 4 subjects perform ADLs at the same time (collaboratively or individually). We also report our experience and the lessons learned in deploying a running prototype of our method.