Internet of Things (IoT) provides a wide range of services to improve convenience and comfort in our daily lives. However, various sensors equipped on IoT devices often raise privacy concerns. Prior works on privacy focus on passive protection from the data and device perspective, such as data encryption and communication protocol design. In this work, we introduce PrivaSee, an augmented reality (AR)-enabled privacy visualization platform to empower users with proactive privacy protection by enhancing their understanding of privacy perception for multimodal sensors.

Indoor occupant sensing enables many smart home applications, and various sensing systems have been explored. Based on their installation requirements, we consider two categories of sensors – on- and off-body – and we look into the combination of them for occupant sensing due to their spatial and temporal complementarity. We focus on an example modality pair of wearable IMU and structural vibration that demonstrate modality complementarity in prior work. However, current efforts are built upon the assumption that the knowledge of the signal segments from two modalities are known, which is challenged in a multiple occupants co-living scenario. Therefore, establishing accurate cross-modal signal segment associations is essential to ensure that a correct complementary relationship is learned.

No abstract available.

Medication intake monitoring is pivotal for healthcare management. While several medication management technologies have been developed, limitations include cost, adoption and adherence, and learning how to use a new system, which is particularly challenging for older adults with cognitive impairment. In this work, we present PA-Pill, a vibration-based sensing system for medication intake monitoring. PA-Pill leverages a physical structure with a unique vibration signature for medication intake event detection and recognition. We implement a prototype and depict preliminary results to verify the feasibility.

The rising older adults population has led to an increased demand for in-home health monitoring to support their well-being in daily life. For instance, localization and tracking are essential applications in elderly monitoring since they can provide various information on health, mobility and can detect falls. The required power and computational resources of traditional acoustic sensor-array solutions make them unavailable on power- and computation- constrained embedded devices. In this paper, we present LEVO, a single-point directional acoustic sensing system that leverages simple LEGO® bricks to build up a physical structure that can embed directional information into a signal waveform. Our preliminary results verifies the feasibility of adopting LEVO for signal direction recognition from signal-point sensing data.

The growth of the Internet of Things (IoT) sensing systems leads to a large number of multimodal datasets over different deployments. Labeling costs for these datasets, especially fine-grained labels, are often tremendous. On the other hand, different data distributions (domain variance) of these datasets prevent models built with labels of one dataset (source domain) from being directly used in another (target domain). This domain variance may be caused by one or more physical factors change in the deployments, such as buildings and/ or people. Existing model transfer studies mainly focus on adapting the model to the domain variance caused by only one physical factor change. When multiple factors change between the source and target domains, the model transfer often yields low accuracy due to significant domain variance. We present VMA, a model transfer framework for multimodal IoT sensing data that handles multi-factor domain variance. VMA first decouples the multi-factor domain variance between two datasets to multiple single-factor domain variance dataset pairs with other available datasets. Then, VMA leverages sensing modalities robust to each single-factor domain variance for accurate prediction by weighing them more in the fusion. We apply VMA to the fine-grained occupant activity recognition application with a multi-modal sensing system of structural vibration and wearable IMU. We collect real-world datasets to evaluate the proposed framework. VMA achieves a model transfer accuracy up to 76.1% on the target domain with multi-factor domain variance, demonstrating a 1.6x and 1.9x error reduction compared to direct prediction baselines with and without modality-aware learning design.

With the rise of desktop computers and televisions, people around the world have been leading increasingly sedentary lifestyles. It is estimated that people spend between 8–10 hours sitting each day, occupationally or otherwise [10], which has translated to increased reports of neck and back pain as well. In 2018, neck and back pain was the third most reason for taking days off work, accounting for more than 264 million workdays lost in a single year [1]. In America alone, approximately 40% of adults experience some form of back pain by the age of 30 [5]. Not only can this condition be debilitating -left unchecked, it can also progress into nerve damage, disc compression, spinal disorders, or loss of lung capacity [1], [12].

Current autonomous checkout is often enable by the use of multiple overhead cameras and/or load sensors on shelf, which is limited by the occlusion and dense deployment. We present MOOCA, an origami-inspired low-cost configurable surface structure as the smart shelf liner. MOOCA leverage conductive threads and copper wires integrated in to the origami structure to detect and recognize pick-up and put-down products. We build our prototype with 3D printed structure using elastic resin. We will demonstrate MOOCA's functionality of predicting the item that is picked up from it.

High-fidelity sensor data quality is the fundamental base of smart agriculture. Since crop information inference and cultivating strategy optimization mainly depend on data-driven methods; the data quality assessment is essential to ensure the reliability of the IoT systems for smart agriculture. The traditional data quality assessment methods focus on sensor data consistency with the costly reference truth, which is often not scalable for agricultural applications.

Footstep-induced floor vibration sensing has been used in many smart home applications, such as elderly/patient monitoring. These systems often leverage data-driven models to infer human information. Therefore, characterizing datasets is crucial for the generalization of this new modality. This dataset contains 144-minute floor vibration signals from two pedestrians in eight environments. We analyze the reusability of this dataset in three different research areas, including vibration-based information inference, knowledge transferring, and multimodal learning. We further characterize the dataset transferability on the occupant identification task, to provide quantitative insights for the transfer learning problems in the real-world floor vibration sensing applications. The characterization is conducted with three metrics, including distribution distance, information dependency, and influencing factor bias. Analysis results depict that the dataset covers different levels of transferability caused by multiple influencing factors. As a result, there are multiple future directions in which the dataset can be reused.

Abstract In this paper, we introduce AutoQual, a mobile-based assessment scheme for infrastructure sensing task performance prediction under new deployment environments. With the growth of the Internet-of-Things (IoT), many non-intrusive sensing systems have been explored for various indoor applications, such as structural vibration sensing. This indirect sensing approach’s learning performance is prone to deployment variance when signals propagate through the environment. As a result, current systems heavily rely on expert knowledge and manual assessment to achieve effective deployments and high sensing task performance. In order to mitigate this expert effort, we propose to systematically study factors that reflect deployment environment characteristics and methods to measure them autonomously. We present AutoQual that measures a series of assessment factors (AFs) reflecting how the deployment environment impacts the system performance. AutoQual outputs a task-oriented sensing quality (TSQ) score by integrating measured AFs trained from known deployments as a prediction of untested system’s performance. In addition, AutoQual achieves this assessment without manual effort by leveraging co-located mobile sensing context to extract structural vibration signal for processing automatically. We evaluate AutoQual by using it to predict untested systems’ performance over multiple sensing tasks. We conduct real-world experiments and investigate 48 deployments in 11 environments. AutoQual achieves less than 0.10 average absolute error when auto-assessing multiple tasks at untested deployments, which shows a $$\le 0.018$$ ≤ 0.018 absolute error difference compared to the manual assessment approach.

Anomaly detection is an important area for both research and real-world applications. In the surface mounting technology (SMT) process, the defectives of solder paste printing need to be detected immediately or it may cause great effort for recycling and slow down the whole process. In this paper, we propose a novel model, MM-DNN, for anomaly detection with multi-modal data. We collect a multi-modal dataset from different sensors in the factory. Our method efficiently extracts both predictive features for classification and correlative features between multi-modal data to achieve a higher detection rate. As shown in the experiment, our method can further reduce 77% false alarm rate of the detection result in the factory while keeping 95% of real defectives be correctly detected.

Real-time emotion detection has being attracted to human attention recently. Recognizing the inner emotion not only assists people to communicate and understand with each other, but also prevents the occurrence of the serious diseases (e.g., autism) and the emergency (i.e., child abuse, sexual invasion). Existing works usually adopt the professional and cumbersome devices to learn the emotions, and therefore limited in the daily usage. In this work, we design a pervasive and wearable device E-See that enables to recognize the emotion in real time. The prototype of the device is deployed in a microcomputer currently, and it can be resized as a small button worn on the collar or extend as a platform to detect the real-time emotion.

No abstract available.

E-Loc is an indoor localization system, which, through using existing indoor electric wiring, detects occupants' location. While many indoor localization technologies require intensive infrastructural supports, E-Loc obtain locations by injecting a signal into the protected earth line of existing residential power network. Caused by human body inside a room, the electromagnetic character changes can be detected to deduce a resident's location. We evaluate our system through experiments inside multiple rooms and our system is able to reach meter-level accuracy.