Patterns of activities of daily living (ADLs) can provide early indicators of changes for many diseases and health conditions. Radio frequency (RF) sensing is promising for its contactless and privacy-preserving properties. However, most RF-based human activity recognition (HAR) studies focus on short, well-segmented actions, and their applicability to continuous, longer-duration ADLs remains largely uncertain. Existing continuous activity detection methods are typically trained in a closed set with labeled boundaries, limiting their ability to handle unseen, unsegmented ADLs in real life. In this work, we address both challenges by introducing ADL-CLIP, a framework for RF-based continuous ADL detection in home environments without pre-known boundaries and generalizable to unseen activities. We observe that intra-ADL radio frames cluster closely in the representation space, while those across ADLs are more separated. Building on this insight, we formulate ADL segmentation as a label-free optimization problem that identifies the boundaries by minimizing intra-segment while maximizing across-segment distances. To reconcile the discrepancy between data-driven segmentation and human annotated boundaries, we alternate between refining segmentation boundaries in the representation space and fine-tuning the representation model combining both boundaries using CLIP-style contrastive learning, progressively aligning the two while preserving prior knowledge from language models. We evaluate ADL-CLIP on a dataset collected from 46 non-researcher participants performing 25 ADLs continuously with 16 UWB sensors in an instrumented one-bedroom, one-bathroom apartment. In an open-set setting where 5 ADLs are held out during training, ADL-CLIP achieves 76.6% accuracy on all 25 ADLs, outperforming temporal activity detection baselines by 26% and NLS-based HAR by 34%. To our knowledge, this is the first RF-based open-set, boundary-free ADL detection study in an instrumented home environment.

A data collection infrastructure is vital for generating sufficient amounts and diversity of data necessary for developing algorithms in home-based health monitoring. However, the manageability—deployment and operation efforts—of such an infrastructure has long been overlooked. Even a small size of a dozen homes may incur enormous manual efforts on the research team. In this article, we present Proteus, an easily managed infrastructure designed to automate much of the work in deploying and operating such systems. We develop new components and combine with mature technologies to minimize the human efforts required. Proteus includes: 1) scalable, continuous deployment, operation, and update of devices with automatic bootstrapping; 2) automatic fault and error monitoring and recovery with watchdogs and LED feedback, and complementary edge and cloud storage backups; and 3) an easy-to-use data-agnostic pipeline for integrating new modalities. We demonstrate our system’s robustness through different sets of experiments: three sensor nodes (SNs) running for 24 days sending data (17.4 Mb/s aggregate rate), 10 SNs for 14 days (58 Mb/s aggregate rate), and 32 emulated sensors (419.2 Mb/s aggregate rate). All such experiments have data loss rates less than 1%. Further we reduce human efforts by 25-fold and code required for adding new data modality by 25-fold. We also share our experience and lessons learned during the design, development, and pilot deployment of Proteus. Our results show that Proteus is a promising solution for enabling research teams to effectively manage home-based health monitoring at small to medium sizes.

Decentralized model training for on-road vehicles offers the potential to harness huge amounts of data at low costs. However, existing approaches usually depend on the existence of a coordinator, tight synchronization, or a connected cluster, all of which can be challenging or infeasible for fast-moving vehicles. In this work, we propose Learning by Chatting (LbChat), a fully decentralized and asynchronous model training approach leveraging coreset-sharing to eliminate the need for a coordinator, tight synchronization, or even a connected cluster. Different from conventional decentralized learning methods, a vehicle not only exchanges its local model but also a coreset, a condensed abstract of its local training data, with opportunistically encountered peers. A vehicle measures its model's performance on a peer's coreset, and a lower performance indicates more different data, thus a more “valuable” model from the peer. Such models are compressed less during exchange to maximize the aggregate gain from each encounter. Extensive evaluations on the driving decision-making task demonstrate that LbChat is strongly competitive with the central server or roadside infrastructure-based approaches (e.g., federated learning). Compared to recent fully decentralized vehicular learning benchmarks, LbChat out-performs them significantly by up to 20% higher driving success rate in the most challenging driving condition, demonstrating the power of insights gained from coresets on peer models' value.

Fully decentralized model training for on-road vehicles can leverage crowdsourced data while not depending on central servers, infrastructure or Internet coverage. However, under unreliable wireless communication and short contact duration, model sharing among peer vehicles may suffer severe losses thus fail frequently. To address these challenges, we propose “RoADTrain”, a route-assisted decentralized peer model training approach that carefully chooses vehicles with high chances of successful model sharing. It bounds the per round communication time yet retains model performance under vehicle mobility and unreliable communication. Based on shared route information, a connected cluster of vehicles can estimate and embed the link reliability and contact duration information into the communication topology. We decompose the topology into subgraphs supporting parallel communication, and identify a subset of them with the highest algebraic connectivity that can maximize the speed of the information flow in the cluster with high model sharing successes, thus accelerating model training in the cluster. We conduct extensive evaluation on driving decision making models using the popular CARLA simulator. RoADTrain achieves comparable driving success rates and 1.2–4.5× faster convergence than representative decentralized learning methods that always succeed in model sharing (e.g., SGP), and significantly outperforms other benchmarks that consider losses by 17–27% in the hardest driving conditions. These demonstrate that route sharing enables shrewd selection of vehicles for model sharing, thus better model performance and faster convergence against wireless losses and mobility.

To tap into the gold mine of data generated by Internet of Things (IoT) devices with unprecedented volume and value, there is an urgent need to efficiently and accurately label raw sensor data. To this end, we explore and leverage the hidden connections among the multimodal data collected by various sensing devices and propose to let different modal data complement and learn from each other. But it is challenging to align and fuse multimodal data without knowing their perception (and thus the correct labels). In this work, we propose MultiSense , a paradigm for automatically mining potential perception, cross-labelling each modal data, and then updating the learning models for recognizing human activity to achieve higher accuracy or even recognize new activities. We design innovative solutions for segmenting, aligning, and fusing multimodal data from different sensors, as well as model updating mechanism. We implement our framework and conduct comprehensive evaluations on a rich set of data. Our results demonstrate that MultiSense significantly improves the data usability and the power of the learning models. With nine diverse activities performed by users, our framework automatically labels multimodal sensing data generated by five different sensing mechanisms (video, smart watch, smartphone, audio, and wireless-channel) with an average accuracy 98.5%. Furthermore, it enables models of some modalities to learn unknown activities from other modalities and greatly improves the activity recognition ability.

One of the major challenges for fully enjoying the power of machine learning is the need for the high-quality labelled data. To tap-in the gold-mine of data generated by IoT devices with unprecedented volume and value, we discover and leverage the hidden connections among the multimodal data collected by various sensing devices. Different modal data can complement and learn from each other, but it is challenging to fuse multimodal data without knowing their perception (and thus the correct labels). In this work, we propose MultiSense, a paradigm for automatically mining potential perception, cross labelling each modal data, and then improving the learning models for recognizing human activity accurately. We design innovative solutions for segmenting, aligning, and fusing multimodal data from different sensors. We implement our framework and conduct comprehensive evaluations on a rich set of data. Our results demonstrate that MultiSense significantly improves the data usability and the power of the learning models. With 9 diverse activities performed by users, our framework automatically labels multimodal sensing data generated by five different sensing mechanisms (video, smart watch, smartphone, audio, and wireless-channel) with an average accuracy 98.5%, while for each single-modal data model, the accuracy is 95.2%, 92.1%, 71.1%, 91.5%, and 34.8% respectively. Furthermore, it enables the existing models to learn unknown activities from other modalities and thus greatly improves the activity recognition accuracy.

One of the major challenges for fully enjoying the power of machine learning is the need for the high-quality labelled data. To tap-in the gold-mine of data generated by IoT devices with unprecedented volume and value, we discover and leverage the hidden connections among the multimodal data collected by various sensing devices. Different modal data can complete and learn from each other, but it is challenging to fuse multimodal data without knowing their perception (and thus the correct labels). In this work, we propose MultiSense, a paradigm for automatically mining potential perception, cross-labelling each modal data, and then improving the learning models over the set of multimodal data. We design innovative solutions for segmenting, aligning, and fusing multimodal data from different sensors. We implement our framework and conduct comprehensive evaluations on a rich set of data. Our results demonstrate that MultiSense significantly improves the data usability and the power of the learning models.