Abstract Foundation models have gained growing interest in the IoT domain due to their reduced reliance on labeled data and strong generalizability across tasks, which address key limitations of traditional machine learning approaches. However, most existing foundation model based methods are developed for specific IoT tasks, making it difficult to compare approaches across IoT domains and limiting guidance for applying them to new tasks. This survey aims to bridge this gap by providing a comprehensive overview of current methodologies and organizing them around four shared performance objectives by different domains: efficiency , context-awareness , safety , and security & privacy . For each objective, we review representative works, summarize commonly-used techniques and evaluation metrics. This objective-centric organization enables meaningful cross-domain comparisons and offers practical insights for selecting and designing foundation model based solutions for new IoT tasks. We conclude with key directions for future research to guide both practitioners and researchers in advancing the use of foundation models in IoT applications.

LLMs have immense potential for generating plans, transforming an initial world state into a desired goal state.A large body of research has explored the use of LLMs for various planning tasks, from web navigation to travel planning and database querying.However, many of these systems are tailored to specific problems, making it challenging to compare them or determine the best approach for new tasks.There is also a lack of clear and consistent evaluation criteria.Our survey aims to offer a comprehensive overview of current LLM planners to fill this gap.It builds on foundational work by Kartam and Wilkins (1990) and examines six key performance criteria: completeness, executability, optimality, representation, generalization, and efficiency.For each, we provide a thorough analysis of representative works and highlight their strengths and weaknesses.Our paper also identifies crucial future directions, making it a valuable resource for both practitioners and newcomers interested in leveraging LLM planning to support agentic workflows.1

The widespread adoption of Internet of Things (IoT) devices has significantly enhanced convenience for consumers, yet the privacy implications of these devices remain unclear to most users, even with the availability of privacy policies. To address this challenge, we introduce a novel visualization tool that provides an informative and expressive visual representation of the sensors, data processing workflows, and associated privacy risks of IoT devices. This user-friendly tool is designed to enhance user understanding, empowering them to make informed decisions about their privacy.

As wearable devices increase, there's a growing need for intuitive, private, and accessible interaction methods. This position paper builds on the research on unvoiced speech interaction and authentication to propose a vision for interaction in next-generation wearables. This paper draws upon our previous work on unvoiced speech interfaces that leverage jaw movements and facial vibrations for command recognition and user authentication. We argue that unvoiced speech interaction can provide a robust, privacy-preserving, and noise-resistant alternative to traditional interfaces, enhancing accessibility and offering discrete interaction in public spaces. We discuss the potential integration of these systems into commercial devices and explore gesture-based interactions as an alternative to touch. Additionally, we discuss the future direction of unvoiced speech interfaces. This paper sets the stage for implementing unvoiced speech and gesture-based interaction in mainstream wearables in our daily interactions with technology.

Internet of Things (IoT) provides a wide range of services to improve convenience and comfort in our daily lives. However, various sensors equipped on IoT devices often raise privacy concerns. Prior works on privacy focus on passive protection from the data and device perspective, such as data encryption and communication protocol design. In this work, we introduce PrivaSee, an augmented reality (AR)-enabled privacy visualization platform to empower users with proactive privacy protection by enhancing their understanding of privacy perception for multimodal sensors.

This paper presents OMSense, an intelligent surface solution that leverages origami-inspired metasurfaces to allow scalable and precise surface pressure sensing. People interact with various surfaces daily, and these interactions cause the surfaces to deform, a process that can be captured by sensors. This interaction can be utilized in various forms of human-computer interaction and human monitoring, enabling new use cases. However, existing surface sensing schemes are either expensive, difficult to scale, or low-precision due to signal leakage in multiplex design. To solve this problem, we propose OMSense, which adopts the multiplex matrix sensing design and incorporates a 3D metastructure to reduce the shared physical connection-induced signal leakage. In addition to this physical augmentation, OMSense adopts a circuit-guided CNN to mitigate the circuit connection-induced signal leakage (ghosting). We 3D print a circuit-integrated metastructure and evaluate the sensor unit accuracy. OMSense achieves up to 2× sensor unit activation detection F1 score compared to the baselines.

We present Unvoiced, a novel unvoiced user interface that leverages jaw motion to enable users to silently interact with their devices using earables. The core idea is to translate low-frequency jaw motion signals into high-frequency information-rich mel spectrograms. Our proposed cross-modal translation incorporates phonetic, contextual, and syntactic information, while the specialized loss function optimizes for these linguistic features. This ensures that the generated spectrograms capture nuanced speech characteristics. Evaluated for 19 users across four tasks, Unvoiced demonstrates >94% task completion rate and <9% word error rate for over 90% of phrases. Further, Unvoiced maintains >90% task completion rate in noisy conditions.

This poster presents the design and implementation of Unvoiced, a silent speech interaction system. Unvoiced transforms subtle jaw movements into rich speech spectrograms, enabling seamless and private device interaction. Our system captures low-frequency jaw motion signals using ear-worn IMUs and translates them into high-fidelity mel-spectrograms through cross-modal translation techniques. By incorporating phonetic, contextual, and syntactic information, Unvoiced generates high-fidelity spectrograms that existing speech recognition systems can process. In our evaluation with 19 users across four common tasks, Unvoiced achieved a remarkable >94% task completion rate and <9% Word Error Rate (WER) for over 90% of phrases, maintaining robust performance even in noisy conditions.

Indoor occupant sensing enables many smart home applications, and various sensing systems have been explored. Based on their installation requirements, we consider two categories of sensors – on- and off-body – and we look into the combination of them for occupant sensing due to their spatial and temporal complementarity. We focus on an example modality pair of wearable IMU and structural vibration that demonstrate modality complementarity in prior work. However, current efforts are built upon the assumption that the knowledge of the signal segments from two modalities are known, which is challenged in a multiple occupants co-living scenario. Therefore, establishing accurate cross-modal signal segment associations is essential to ensure that a correct complementary relationship is learned.

No abstract available.

This paper examines fault resilience among citizen-science air quality monitoring networks in California's economically challenged San Joaquin Valley (SJV). We examine disparities in monitoring capabilities and data equity between the SJV and the San Francisco Bay Area. We found significant inequities through experimental analysis simulating sensor failures. Our results emphasize the need for reliable monitoring systems and advanced modeling algorithms in resource-limited areas.

Medication intake monitoring is pivotal for healthcare management. While several medication management technologies have been developed, limitations include cost, adoption and adherence, and learning how to use a new system, which is particularly challenging for older adults with cognitive impairment. In this work, we present PA-Pill, a vibration-based sensing system for medication intake monitoring. PA-Pill leverages a physical structure with a unique vibration signature for medication intake event detection and recognition. We implement a prototype and depict preliminary results to verify the feasibility.

In this paper, we present Jawthenticate, an earable system that authenticates a user using audible or inaudible speech without using a microphone. This system can overcome the shortcomings of traditional voice-based authentication systems like unreliability in noisy conditions and spoofing using microphone-based replay attacks. Jawthenticate derives distinctive speech-related features from the jaw motion and associated facial vibrations. This combination of features makes Jawthenticate resilient to vocal imitations as well as camera-based spoofing. We use these features to train a two-class SVM classifier for each user. Our system is invariant to the content and language of speech. In a study conducted with 41 subjects, who speak different native languages, Jawthenticate achieves a Balanced Accuracy (BAC) of 97.07%, True Positive Rate (TPR) of 97.75%, and True Negative Rate (TNR) of 96.4% with just 3 seconds of speech data.

The rising older adults population has led to an increased demand for in-home health monitoring to support their well-being in daily life. For instance, localization and tracking are essential applications in elderly monitoring since they can provide various information on health, mobility and can detect falls. The required power and computational resources of traditional acoustic sensor-array solutions make them unavailable on power- and computation- constrained embedded devices. In this paper, we present LEVO, a single-point directional acoustic sensing system that leverages simple LEGO® bricks to build up a physical structure that can embed directional information into a signal waveform. Our preliminary results verifies the feasibility of adopting LEVO for signal direction recognition from signal-point sensing data.

IoT (Internet of Things) devices, such as network-enabled wearables, are carried by increasingly more people throughout daily life. Information from multiple devices can be aggregated to gain insights into a person’s behavior or status. For example, an elderly care facility could monitor patients for falls by combining fitness bracelet data with video of the entire class. For this aggregated data to be useful to each person, we need a multi-modality association of the devices’ physical ID (i.e., location, the user holding it, visual appearance) with a virtual ID (e.g., IP address/available services). Existing approaches for multi-modality association often require intentional interaction or direct line-of-sight to the device, which is infeasible for a large number of users or when the device is obscured by clothing. We present IDIoT , a calibration-free passive sensing approach that fuses motion sensor information with camera footage of an area to estimate the body location of motion sensors carried by a user. We characterize results across three baselines to highlight how different fusing methodology results better than earlier IMU-vision fusion algorithms. From this characterization, we determine IDIoT is more robust to errors such as missing frames or miscalibration that frequently occur in IMU-vision matching systems.

The growth of the Internet of Things (IoT) sensing systems leads to a large number of multimodal datasets over different deployments. Labeling costs for these datasets, especially fine-grained labels, are often tremendous. On the other hand, different data distributions (domain variance) of these datasets prevent models built with labels of one dataset (source domain) from being directly used in another (target domain). This domain variance may be caused by one or more physical factors change in the deployments, such as buildings and/ or people. Existing model transfer studies mainly focus on adapting the model to the domain variance caused by only one physical factor change. When multiple factors change between the source and target domains, the model transfer often yields low accuracy due to significant domain variance. We present VMA, a model transfer framework for multimodal IoT sensing data that handles multi-factor domain variance. VMA first decouples the multi-factor domain variance between two datasets to multiple single-factor domain variance dataset pairs with other available datasets. Then, VMA leverages sensing modalities robust to each single-factor domain variance for accurate prediction by weighing them more in the fusion. We apply VMA to the fine-grained occupant activity recognition application with a multi-modal sensing system of structural vibration and wearable IMU. We collect real-world datasets to evaluate the proposed framework. VMA achieves a model transfer accuracy up to 76.1% on the target domain with multi-factor domain variance, demonstrating a 1.6x and 1.9x error reduction compared to direct prediction baselines with and without modality-aware learning design.

Approximately 3.5 billion people worldwide have oral diseases [8], which significantly impact people's quality of life [1] and may lead to mortality if left unattended [7]. Out of these oral diseases, occlusal diseases, such as temporomandibular joint and muscle dis-order (TMD), gum recession, fractured teeth, and undesired tooth mobility, are especially hard to diagnose due to the subjective inter-pretation of many current practices used to test for this condition [9]. Occlusal diseases, associated with the alignment of a person's teeth, are usually caused by excessive wearing of teeth, bruxism, and unbalanced biting [5].

With the rise of desktop computers and televisions, people around the world have been leading increasingly sedentary lifestyles. It is estimated that people spend between 8–10 hours sitting each day, occupationally or otherwise [10], which has translated to increased reports of neck and back pain as well. In 2018, neck and back pain was the third most reason for taking days off work, accounting for more than 264 million workdays lost in a single year [1]. In America alone, approximately 40% of adults experience some form of back pain by the age of 30 [5]. Not only can this condition be debilitating -left unchecked, it can also progress into nerve damage, disc compression, spinal disorders, or loss of lung capacity [1], [12].

We demonstrate an ear-worn technology that recognizes unvoiced human commands by tracking jaw motion. The ear-worn system is designed to achieve continual unvoiced command recognition for robust human-computer interaction (HCI) applications. First, the system reliably extracts the jaw motion signals buried under the noise caused by head motion, walking, and other motion artifacts to track single secondary voice articulator (i.e., word). Then, learning from linguistics and human speech anatomy, we design a novel algorithm that localizes the phonemes in the command, and reconstructs the word. We evaluate the proposed system in real-world experiments with 15 volunteers. Our preliminary results show that the proposed system obtains a word recognition accuracy of 95.6% in noise-free conditions and 93.2% and 91.6%, while head nodding and walking.

Occupant information inference with IoT sensor data enables many smart applications, such as patients'/older adults' in-home monitoring. The difficulty of collecting labeled real-world IoT sensor data often leads to reliability and scalability issues for those systems. Extensive prior works (e.g., domain adaptation) focus on the domain shift issues, i.e., the inconsistent data feature and label relationship, and dataset bias is often neglected. Dataset bias is commonly caused by limited and varied accessibility to labeled data for each class, and it is inevitable for real-world datasets. The model trained with a biased dataset fits into the bias, hence cannot further generalize to the testing data for accurate inference.

Current autonomous checkout is often enable by the use of multiple overhead cameras and/or load sensors on shelf, which is limited by the occlusion and dense deployment. We present MOOCA, an origami-inspired low-cost configurable surface structure as the smart shelf liner. MOOCA leverage conductive threads and copper wires integrated in to the origami structure to detect and recognize pick-up and put-down products. We build our prototype with 3D printed structure using elastic resin. We will demonstrate MOOCA's functionality of predicting the item that is picked up from it.

High-fidelity sensor data quality is the fundamental base of smart agriculture. Since crop information inference and cultivating strategy optimization mainly depend on data-driven methods; the data quality assessment is essential to ensure the reliability of the IoT systems for smart agriculture. The traditional data quality assessment methods focus on sensor data consistency with the costly reference truth, which is often not scalable for agricultural applications.

Automated monitoring of livestock behavior can help farmers economically by detecting changes in animal welfare. Prior approaches use video, which requires light and high storage capability, or motion detection, which has difficulty separating subtle activities. Wearable sensors can address these issues but are vulnerable to destruction by the animals. To the best of our knowledge, we present the first system that uses structural vibration to track animal behavior, and the first system to automatically detect piglet nursing. PigNet uses vibration sensors attached to a pig pen to sense the unique vibration patterns and changes in structural response caused by the animals' movement and position within the pen. Combined with our knowledge of pig behavior, we use this physical knowledge of vibration characteristics to detect pig activities and track piglet growth in a real farm environment. Our system is designed to be robust to the harsh environment, which can create unpredictable noise, as well as physically damage or disconnect sensor nodes. When deployed in a real-world farm environment, our system was able to achieve a daily pen-level status profile of up to 90% accuracy, which tracks nursing activity, sow lying activity, and changes in piglet growth over the weeks-long pre-weaning period.

Footstep-induced floor vibration sensing has been used in many smart home applications, such as elderly/patient monitoring. These systems often leverage data-driven models to infer human information. Therefore, characterizing datasets is crucial for the generalization of this new modality. This dataset contains 144-minute floor vibration signals from two pedestrians in eight environments. We analyze the reusability of this dataset in three different research areas, including vibration-based information inference, knowledge transferring, and multimodal learning. We further characterize the dataset transferability on the occupant identification task, to provide quantitative insights for the transfer learning problems in the real-world floor vibration sensing applications. The characterization is conducted with three metrics, including distribution distance, information dependency, and influencing factor bias. Analysis results depict that the dataset covers different levels of transferability caused by multiple influencing factors. As a result, there are multiple future directions in which the dataset can be reused.

Abstract In this paper, we introduce AutoQual, a mobile-based assessment scheme for infrastructure sensing task performance prediction under new deployment environments. With the growth of the Internet-of-Things (IoT), many non-intrusive sensing systems have been explored for various indoor applications, such as structural vibration sensing. This indirect sensing approach’s learning performance is prone to deployment variance when signals propagate through the environment. As a result, current systems heavily rely on expert knowledge and manual assessment to achieve effective deployments and high sensing task performance. In order to mitigate this expert effort, we propose to systematically study factors that reflect deployment environment characteristics and methods to measure them autonomously. We present AutoQual that measures a series of assessment factors (AFs) reflecting how the deployment environment impacts the system performance. AutoQual outputs a task-oriented sensing quality (TSQ) score by integrating measured AFs trained from known deployments as a prediction of untested system’s performance. In addition, AutoQual achieves this assessment without manual effort by leveraging co-located mobile sensing context to extract structural vibration signal for processing automatically. We evaluate AutoQual by using it to predict untested systems’ performance over multiple sensing tasks. We conduct real-world experiments and investigate 48 deployments in 11 environments. AutoQual achieves less than 0.10 average absolute error when auto-assessing multiple tasks at untested deployments, which shows a $$\le 0.018$$ ≤ 0.018 absolute error difference compared to the manual assessment approach.

Recent work has shown that two wearable devices worn on the same user can exploit gait as a secret source to generate a common key for secure pairing. The main threat of using gait comes from side-channel attackers who can use cameras to record the walking user and extract accelerations from the video to pair with legitimate devices. We propose a novel pre-step that uses a CNN-LSTM deep learning model to classify the acceleration measurement devices, i.e., between IMU vs. Camera. We prototype the pre-step and evaluate it using real subjects. Our results show that the proposed pre-step can achieve high classification success rates. The experiments with different cut-off frequencies show that the higher acceleration frequencies appear to contain more distinguishable features to classify camera from IMU.

We present Deskbuddy, a vibration-based system that can track a user's activities through their desk. Tracking sitting and other office related activities let us remind the user to have healthier working habits, as well as giving information about how office spaces are used. Many solutions have been proposed for office activity tracking, but they either require the user to wear a device, or they use cameras or microphones, which can make subjects uncomfortable. Our demo includes a small vibration sensor that sits on a table that can detect four office related activities. We capture the signal from the vibration sensor, extract features, and perform classification on the resulting features. The full functionality of the system will be shown in a video. In order for our demo to be more effective in a crowded environment, we have re-trained it to detect only typing versus not typing.

Gait health monitoring is critical for condition diagnosis and fall prediction in elderly populations. Existing methods for gait health monitoring (e.g. direct observation and sensing) are not suitable for non-clinical environments due to qualitative assessments or operational limitations. Our method utilizes footstep-induced floor vibration sensing to provide a passive gait health monitoring platform that can be used in non-clinical environments (e.g. home settings) to provide gait health information in a timely manner. We decompose vibration responses to obtain signal peaks that correspond to temporal gait information and leverage foot dominance to learn a signal amplitude-footstep ground reaction force transfer function. Preliminary results show that temporal gait parameters can be estimated with up to 99% accuracy and gait balance symmetry can be estimated with as low as 10.4% error.

Secure pairing is an important problem especially due to large number of IoT devices. In this paper, we propose PosePair++, to enable a camera to securely pair with IoT devices which are equipped with IMU sensors. Existing context-based pairing approaches do not adequately address this problem due to differing sensing modalities. To address this challenge, we propose to translate the signals from heterogeneous sensing modalities to a common space, namely 2D acceleration. In this demo, we present PosePair++'s robustness against different types of attackers (i.e., attackers that observe the user's motion, or attackers performing mimicking attack).

In the Internet of Things (IoT) paradigm, it is important to easily setup and control devices, which is achieved by pairing. In this work, we present a novel pairing scheme that utilizes heterogeneous sensing. The core idea is that devices with different sensing capabilities can still detect common information about their user. We demonstrate this idea through an example application consisting of a camera and IoT devices with inertial sensors. As the user holds a device and moves it around, the camera captures the human's pose and compares it to the IoT device motion. If the motion features are similar enough, the device can be successfully paired to the camera's network.

No abstract available.

Context-based pairing solutions increase the usability of IoT device pairing by eliminating any human involvement in the pairing process. This is possible by utilizing on-board sensors (with same sensing modalities) to capture a common physical context (e.g., ambient sound via each device's microphone). However, in a smart home scenario, it is impractical to assume that all devices will share a common sensing modality. For example, a motion detector is only equipped with an infrared sensor while Amazon Echo only has microphones. In this paper, we develop a new context-based pairing mechanism called Perceptio that uses time as the common factor across differing sensor types. By focusing on the event timing, rather than the specific event sensor data, Perceptio creates event fingerprints that can be matched across a variety of IoT devices. We propose Perceptio based on the idea that devices co-located within a physically secure boundary (e.g., single family house) can observe more events in common over time, as opposed to devices outside. Devices make use of the observed contextual information to provide entropy for Perceptio's pairing protocol. We design and implement Perceptio, and evaluate its effectiveness as an autonomous secure pairing solution. Our implementation demonstrates the ability to sufficiently distinguish between legitimate devices (placed within the boundary) and attacker devices (placed outside) by imposing a threshold on fingerprint similarity. Perceptio demonstrates an average fingerprint similarity of 94.9% between legitimate devices while even a hypothetical impossibly well-performing attacker yields only 68.9% between itself and a valid device.

Occupant identification proves crucial in many smart home applications such as automated home control and activity recognition. Previous solutions are limited in terms of deployment costs, identification accuracy, or usability. We propose SenseTribute , a novel occupant identification solution that makes use of existing and prevalent on-object sensors that are originally designed to monitor the status of objects to which they are attached. SenseTribute extracts richer information content from such on-object sensors and analyzes the data to accurately identify the person interacting with the objects. This approach is based on the physical phenomenon that different occupants interact with objects in different ways. Moreover, SenseTribute may not rely on users’ true identities, so the approach works even without labeled training data. However, resolution of information from a single on-object sensor may not be sufficient to differentiate occupants, which may lead to errors in identification. To overcome this problem, SenseTribute operates over a sequence of events within a user activity, leveraging recent work on activity segmentation. We evaluate SenseTribute using real-world experiments by deploying sensors on five distinct objects in a kitchen and inviting participants to interact with the objects. We demonstrate that SenseTribute can correctly identify occupants in 96% of trials without labeled training data, while per-sensor identification yields only 74% accuracy even with training data.

Touch surfaces are intuitive interfaces for computing devices. Most of the traditional touch interfaces (vision, IR, capacitive, etc.) have mounting requirements, resulting in specialized touch surfaces limited by their size, cost, and mobility. More recent work has shown that vibration-based touch sensing techniques can localize taps/knocks, which provides a low-cost flexible alternative. These surfaces are envisioned as intuitive inputs for applications such as interactive meeting tables, smart kitchen appliance control, etc. However, due to dispersive and reflective properties of various vibrating mediums, it is difficult to localize taps accurately on ubiquitous surfaces. Furthermore, no work has been done on tracking continuous swipe interactions through vibration sensing.

Indoor emergency response situations, such as urban fire, are characterized by dangerous constantly changing operating environments with little access to situational information for first responders. In situ information about the conditions, such as the extent and evolution of an indoor fire, can augment rescue efforts and reduce risk to emergency personnel. Static sensor networks that are pre-deployed or manually deployed have been proposed but are less practical due to need for large infrastructure, lack of adaptivity, and limited coverage. Controlled-mobility in sensor networks, that is, the capability of nodes to move as per network needs can provide the desired autonomy to overcome these limitations. In this article, we present SensorFly, a controlled-mobile aerial sensor network platform for indoor emergency response application. The miniature, low-cost sensor platform has capabilities to self deploy, achieve three-dimensional sensing, and adapt to node and network disruptions in harsh environments. We describe hardware design trade-offs, the software architecture, and the implementation that enables limited-capability nodes to collectively achieve application goals. Through the indoor fire monitoring application scenario, we validate that the platform can achieve coverage and sensing accuracy that matches or exceeds static sensor networks and provide higher adaptability and autonomy.

Tracking multiple people in an indoor environment enables various smart building applications such as HVAC energy saving, patient/child monitoring, etc. Researchers have explored various sensing methods including vision, motion, and RF, which either require specific installation requirements or high deployment density. We introduce a passive sparse sensing method based on ambient structural vibration induced by foot strikes. Our system tracks multiple people based on the premise that human foot strikes have spatio-temporal variation, and hence do not fully overlap. The system achieved less than 0.4m accuracy in both one and two persons stepping conditions.

We describe a step-level indoor localization system which uses the ground vibration induced by human footsteps. Indoor localization is important for various smart building applications, including resources arrangement optimization, patient/customer tracking, etc. Geophones are used to measure the ground vibrations and time difference of arrival (TDoA) for different sensors are used to solve the multilateration localization problem. The advantages of this system include its sparsity and also its stability over time. Lesser dependency on instrument people is another upside of this system. The results of pilot tests show that this system can be successfully used for indoor localization.

The dynamic responses of the structure provide a variety of information about the structure as well as people inside. Compared with traditional sensing methods, structural sensing method serves more general sensing purposes due to the diversity of information it can infer from structural responses. For example, by sensing the structural vibration, a system can track and identify a person through vibration caused by their gaits [5, 6]. Such non-intrusive identification and tracking system enables various smart building applications, such as patient monitoring system at advanced hospitals and nursing homes.

Heading information becomes widely used in ubiquitous computing applications for mobile devices. Digital magnetometers, also known as geomagnetic field sensors, provide absolute device headings relative to the earth's magnetic north. However, magnetometer readings are prone to significant errors in indoor environments due to the existence of magnetic interferences, such as from printers, walls, or metallic shelves. These errors adversely affect the performance and quality of user experience of the applications requiring device headings. In this paper, we propose Headio, a novel approach to provide reliable device headings in indoor environments. Headio achieves this by aggregating ceiling images of an indoor environment, and by using computer vision-based pattern detection techniques to provide directional references. To achieve zero-configured and energy-efficient heading sensing, Headio also utilizes multimodal sensing techniques to dynamically schedule sensing tasks. To fully evaluate the system, we implemented Headio on both Android and iOS mobile platforms, and performed comprehensive experiments in both small-scale controlled and large-scale public indoor environments. Evaluation results show that Headio constantly provides accurate heading detection performance in diverse situations, achieving better than 1 degree average heading accuracy, up to 33X improvement over existing techniques.

A system that helps people navigate in indoor environments on a fine-grained level can enable a variety of pervasive computing applications in retail environments. Existing indoor navigation systems rely on extensive RF tagging surveys and accurate floor plans. These prerequisites are often impractical in indoor environments. In this paper, we present SugarTrail, a system for indoor navigation assistance in retail environments that minimizes the need for active tagging and does not require existing maps. By leveraging the structured movement patterns of shoppers in retail store environments, the system provides higher accuracy than existing radio finger-printing approaches. With minimal setup and active user participation, the system automatically learns user movement pathways in indoor environments from radiofrequency and magnetic signatures. These pathways are clustered and used to automatically build a navigable virtual roadmap of the environment. We present results from a campus testbed and from actual radio measurements collected in an operational supermarket to show that SugarTrail system can navigate users with a success rate of > 85% and an average accuracy of 0.7m.

Energy saving is the main goal in most building energy monitoring applications. These systems, however, are operated by people. For this reason, an intuitive user interface is an essential element that will affect users' data understandability, thereby determining the system's usability. Traditional energy monitoring generally focuses on getting energy information by utilizing graphs or static text interfaces. However, these approaches are not related to the physical space. With the increase of information in energy monitoring systems, intuitive and efficient ways of displaying information are needed. In this demo, we present iCEnergy, a vision-based mobile information interface that provides power monitoring using augmented reality. Using existing system data, the system overlays an interactive "energy cloud" over corresponding devices in order to illustrate information about the physical environment. This approach aims to provide users with a comfortable interaction experience through its intuitive information display.

Future ubiquitous home environments can contain 10s or 100s of devices. Ubiquitous services running on these devices (i.e. localizing users, routing, security algorithms) will commonly require an accurate location of each device. In order to obtain these locations, existing techniques require either a manual survey, active sound sources, or estimation using wireless radios. These techniques, however, need additional hardware capabilities and are intrusive to the user. Non-intrusive, automatic localization of ubiquitous computing devices in the home has the potential to greatly facilitate device deployments.