Abstract Foundation models have gained growing interest in the IoT domain due to their reduced reliance on labeled data and strong generalizability across tasks, which address key limitations of traditional machine learning approaches. However, most existing foundation model based methods are developed for specific IoT tasks, making it difficult to compare approaches across IoT domains and limiting guidance for applying them to new tasks. This survey aims to bridge this gap by providing a comprehensive overview of current methodologies and organizing them around four shared performance objectives by different domains: efficiency , context-awareness , safety , and security & privacy . For each objective, we review representative works, summarize commonly-used techniques and evaluation metrics. This objective-centric organization enables meaningful cross-domain comparisons and offers practical insights for selecting and designing foundation model based solutions for new IoT tasks. We conclude with key directions for future research to guide both practitioners and researchers in advancing the use of foundation models in IoT applications.

Indoor occupant sensing enables many smart home applications, and various sensing systems have been explored. Based on their installation requirements, we consider two categories of sensors – on- and off-body – and we look into the combination of them for occupant sensing due to their spatial and temporal complementarity. We focus on an example modality pair of wearable IMU and structural vibration that demonstrate modality complementarity in prior work. However, current efforts are built upon the assumption that the knowledge of the signal segments from two modalities are known, which is challenged in a multiple occupants co-living scenario. Therefore, establishing accurate cross-modal signal segment associations is essential to ensure that a correct complementary relationship is learned.

No abstract available.

This paper examines fault resilience among citizen-science air quality monitoring networks in California's economically challenged San Joaquin Valley (SJV). We examine disparities in monitoring capabilities and data equity between the SJV and the San Francisco Bay Area. We found significant inequities through experimental analysis simulating sensor failures. Our results emphasize the need for reliable monitoring systems and advanced modeling algorithms in resource-limited areas.

The growth of the Internet of Things (IoT) sensing systems leads to a large number of multimodal datasets over different deployments. Labeling costs for these datasets, especially fine-grained labels, are often tremendous. On the other hand, different data distributions (domain variance) of these datasets prevent models built with labels of one dataset (source domain) from being directly used in another (target domain). This domain variance may be caused by one or more physical factors change in the deployments, such as buildings and/ or people. Existing model transfer studies mainly focus on adapting the model to the domain variance caused by only one physical factor change. When multiple factors change between the source and target domains, the model transfer often yields low accuracy due to significant domain variance. We present VMA, a model transfer framework for multimodal IoT sensing data that handles multi-factor domain variance. VMA first decouples the multi-factor domain variance between two datasets to multiple single-factor domain variance dataset pairs with other available datasets. Then, VMA leverages sensing modalities robust to each single-factor domain variance for accurate prediction by weighing them more in the fusion. We apply VMA to the fine-grained occupant activity recognition application with a multi-modal sensing system of structural vibration and wearable IMU. We collect real-world datasets to evaluate the proposed framework. VMA achieves a model transfer accuracy up to 76.1% on the target domain with multi-factor domain variance, demonstrating a 1.6x and 1.9x error reduction compared to direct prediction baselines with and without modality-aware learning design.

Approximately 3.5 billion people worldwide have oral diseases [8], which significantly impact people's quality of life [1] and may lead to mortality if left unattended [7]. Out of these oral diseases, occlusal diseases, such as temporomandibular joint and muscle dis-order (TMD), gum recession, fractured teeth, and undesired tooth mobility, are especially hard to diagnose due to the subjective inter-pretation of many current practices used to test for this condition [9]. Occlusal diseases, associated with the alignment of a person's teeth, are usually caused by excessive wearing of teeth, bruxism, and unbalanced biting [5].

With the rise of desktop computers and televisions, people around the world have been leading increasingly sedentary lifestyles. It is estimated that people spend between 8–10 hours sitting each day, occupationally or otherwise [10], which has translated to increased reports of neck and back pain as well. In 2018, neck and back pain was the third most reason for taking days off work, accounting for more than 264 million workdays lost in a single year [1]. In America alone, approximately 40% of adults experience some form of back pain by the age of 30 [5]. Not only can this condition be debilitating -left unchecked, it can also progress into nerve damage, disc compression, spinal disorders, or loss of lung capacity [1], [12].

Occupant information inference with IoT sensor data enables many smart applications, such as patients'/older adults' in-home monitoring. The difficulty of collecting labeled real-world IoT sensor data often leads to reliability and scalability issues for those systems. Extensive prior works (e.g., domain adaptation) focus on the domain shift issues, i.e., the inconsistent data feature and label relationship, and dataset bias is often neglected. Dataset bias is commonly caused by limited and varied accessibility to labeled data for each class, and it is inevitable for real-world datasets. The model trained with a biased dataset fits into the bias, hence cannot further generalize to the testing data for accurate inference.

Footstep-induced floor vibration sensing has been used in many smart home applications, such as elderly/patient monitoring. These systems often leverage data-driven models to infer human information. Therefore, characterizing datasets is crucial for the generalization of this new modality. This dataset contains 144-minute floor vibration signals from two pedestrians in eight environments. We analyze the reusability of this dataset in three different research areas, including vibration-based information inference, knowledge transferring, and multimodal learning. We further characterize the dataset transferability on the occupant identification task, to provide quantitative insights for the transfer learning problems in the real-world floor vibration sensing applications. The characterization is conducted with three metrics, including distribution distance, information dependency, and influencing factor bias. Analysis results depict that the dataset covers different levels of transferability caused by multiple influencing factors. As a result, there are multiple future directions in which the dataset can be reused.

Abstract In this paper, we introduce AutoQual, a mobile-based assessment scheme for infrastructure sensing task performance prediction under new deployment environments. With the growth of the Internet-of-Things (IoT), many non-intrusive sensing systems have been explored for various indoor applications, such as structural vibration sensing. This indirect sensing approach’s learning performance is prone to deployment variance when signals propagate through the environment. As a result, current systems heavily rely on expert knowledge and manual assessment to achieve effective deployments and high sensing task performance. In order to mitigate this expert effort, we propose to systematically study factors that reflect deployment environment characteristics and methods to measure them autonomously. We present AutoQual that measures a series of assessment factors (AFs) reflecting how the deployment environment impacts the system performance. AutoQual outputs a task-oriented sensing quality (TSQ) score by integrating measured AFs trained from known deployments as a prediction of untested system’s performance. In addition, AutoQual achieves this assessment without manual effort by leveraging co-located mobile sensing context to extract structural vibration signal for processing automatically. We evaluate AutoQual by using it to predict untested systems’ performance over multiple sensing tasks. We conduct real-world experiments and investigate 48 deployments in 11 environments. AutoQual achieves less than 0.10 average absolute error when auto-assessing multiple tasks at untested deployments, which shows a $$\le 0.018$$ ≤ 0.018 absolute error difference compared to the manual assessment approach.

Existing machine learning algorithms for human information inference are typically data-driven models trained on carefully labeled datasets. Given the significant labeling effort, traditional pure data-driven approaches are challenging to implement for emerging smart applications requiring long-term finer-grained information. Taking activities of daily life (ADL) tracking for elders as an example, prior work mostly focused on context-level information learning such as cooking and cleaning. [8]. However, new applications such as evaluating elders' cognitive impairments progress by tracking their ADL engagement requires finer-grained, i.e., action-level information [7]. In practice, labeling the day-length data at such granularity can be very expensive and requires a lot of human efforts [9]. My research focuses on the inference problems in the scope of human physical condition monitoring and activity recognition with limited labeled data. To alleviate the effort of labeling large amounts of data, prior works on semi-supervised learning combine a small amount of labeled data with a large amount of unlabeled data to train the model. However, as the label granularity (number of classes) increasing, the difficulty to distinguish nuance distinctions between finer-grained classes escalates as well. This makes training a robust semi-supervised model for finer-grained classification with less labels difficult if not impossible. Fortunately, coarse-grained (context-level) labels is usually available or cheaper to obtain in practice. In this case, the multi-granularity hierarchy between finer and coarse labels follows the aggregation relation defined in [5]. This hierarchical relation can be leveraged in the tasks of inferring finer-grained information. In addition, it is illustrated by the previous study that co-located multi modality sensing systems capture complementary aspects of the same event [6]. The research question I focus on is how to infer finer-grained human information with coarse-grained labeled data leveraging complementary multi-modal sensing? I target three directions: 1) a cross-granularity semi-supervised setting: how to utilize coarse-grained labeled data with a small amount of finer-grained labeled data to infer finer-grained human information, 2) cross-granularity relationship learning: how to learn the multi-granularity class hierarchy from data and further help the finer-grained human information acquisition, 3) enhancing inference granularity by leveraging multi-model sensing: how to leverage the complimentary co-located multiple sensing modalities to accurately infer finer-grained human information?