Emotion recognition is critical for various applications, including the early detection of mental health disorders and emotion-based smart home systems. Previous studies utilized various sensing methods for emotion recognition, such as wearable sensors, cameras, and microphones. However, these methods are often intrusive or raise significant privacy concerns, which may reduce user acceptance for continuous, long-term deployment. This paper introduces a non-intrusive and privacy-friendly personalized emotion recognition system, EmotionVibe, which leverages footstep-induced floor vibrations for emotion recognition. The main idea of EmotionVibe is that individuals' emotional states influence their gait patterns, subsequently affecting the floor vibrations induced by their footsteps. However, there are two main research challenges: 1) the complex and indirect relationship between human emotions and footstep-induced floor vibrations and 2) the large between-person variations within the relationship between emotions and gait patterns. To address these challenges, we first empirically characterize this complex relationship and develop an emotion-sensitive feature set including gait-related and vibration-related features from footstep-induced floor vibrations. Furthermore, we personalize the emotion recognition system for each user by calculating gait similarities between the target person (i.e., the person whose emotions we aim to recognize) and those in the training dataset and assigning greater weights to training people with similar gait patterns in the loss function. We evaluated our system in human walking experiments with 20 participants, summing up to 37,001 footstep samples. EmotionVibe achieved the mean absolute error (MAE) of 1.11 and 1.07 for valence (unpleasant to pleasant) and arousal (calm to excited) score estimations, respectively, reflecting 19.0% and 25.7% error reduction compared to the baseline method (using only gait-related features without personalization).

Crowd monitoring in sports stadiums is important to enhance public safety and improve audience experience. Existing approaches mainly rely on cameras and microphones, which can cause significant disturbances and often raise privacy concerns. In this paper, we sense floor vibration, which provides a less disruptive and more non-intrusive way of crowd sensing, to predict crowd behavior. However, since the vibration-based crowd monitoring approach is newly developed, one main challenge is the lack of training data due to sports stadiums are usually large public spaces with complex physical activities.

Monitoring vital signs and activities in pig pens during the farrowing period is crucial for reducing pre-weaning piglet mortality and enhancing farm productivity. Traditional methods focus on either vital signs or activities separately, falling short in the dynamically changing farm environment. This paper introduces a multiscale vibration sensing method which dynamically adjusts sensor amplifier gain to detect both vital signs (e.g. heartbeats and respirations) and larger-scale activities (e.g. walking, eating, nursing, etc.). Preliminary trials demonstrate the system's potential to adapt to rapidly changing conditions by switching between high sensitivity for vital signs and reduced sensitivity for activity sensing depending on the detected vibration signal.

This paper addresses challenges in monitoring human behavior in crowds through floor vibration sensing, overcoming limitations like subjective manual observation, visual occlusions, and audio interference. Our approach involves tackling limited-data vibration signal tasks by conducting pre-training across modalities, leveraging publicly available audio datasets. By leveraging self-supervised representation learning to pre-train on publicly available audio datasets, our approach reduces data requirements, improves robustness, and minimizes the need for human labeling efforts. Evaluation using in-game stadium vibration data with YouTube audio dataset demonstrates up to 5.8 × error reduction for crowd behavior.

Computer vision has been used more ubiquitously in recent years to understand and measure the environment around us, particularly in our neighborhoods. However, many city-wide sensing applications using vision require large labeling efforts, making various applications difficult on a wide scale. We propose a framework for labeling and self-training of in-car video to detect trash on the roads. Our approach requires minimal manual labeling to identify items not meant to be in the street, sidewalk, or public places, from a front-viewing car camera. Our system provides each frame of a video with a score indicating the amount of trash. To prevent overfitting, due to minimal available data, we remove data with high certainty of trash from the training dataset. The results show that our prediction with manually labeled ground truth yield an R2 of 0.66.

Detecting gait abnormalities is crucial for assessing fall risks and early identification of neuromusculoskeletal disorders such as Parkinson's and stroke. Traditional assessments in gait clinics are infrequent and pose barriers, particularly for disadvantaged populations. Previous efforts have explored sensor-based approaches for in-home gait assessments, yet they face limitations such as visual obstructions (cameras), limited coverage (pressure mats), and the need for device carrying (wearables and insoles). To overcome these limitations, we introduce an in-home gait abnormality detection system using footstep-induced floor vibrations, enabling low-cost, non-intrusive, device-free gait health monitoring. The main research challenge is the high uncertainty in floor vibrations due to gait variations among people, making it challenging to develop a generalizable model for new patients. To address this, we analyze time-frequency-domain features of floor vibration data during specific gait phases and develop a feature transformation method through contrastive learning to address the between-people gait variation challenge. Our method transforms the features from vibrations to an embedding space where samples from different people stay close to each other (robust to people variation) while normal and abnormal gait samples are far apart (sensitive to gait abnormalities). Then, gait abnormalities are detected by a downstream classifier after feature transformation. We evaluated our approach through a real-world walking experiment with 21 participants and achieved an 85% to 95% mean accuracy in detecting various gait abnormalities. This novel method overcomes prior limitations in in-home gait assessments, offering accessible gait abnormality detection without the need for intrusive devices or labels for new patients.

Distributed Acoustic Sensing (DAS) that transforms city-wide fiber-optic cables into a large-scale strain sensing array has shown the potential to revolutionize urban traffic monitoring by providing a fine-grained, scalable, and low-maintenance monitoring solution. However, the real-world application of DAS is hindered by challenges such as noise contamination and interference among closely traveling cars. In response, we introduce a self-supervised U-Net model that can suppress background noise and compress car-induced DAS signals into high-resolution pulses through spatial deconvolution. Our work extends recent research by introducing three key advancements. Firstly, we perform a comprehensive resolution analysis of DAS-recorded traffic signals, laying a theoretical foundation for our approach. Secondly, we incorporate space-domain vehicle wavelets into our U-Net model, enabling consistent high-resolution outputs regardless of vehicle speed variations. Finally, we employ L-2 norm regularization in the loss function, enhancing our model’s sensitivity to weaker signals from vehicles in remote traffic lanes. We evaluate the effectiveness and robustness of our method through field recordings under different traffic conditions and various driving speeds. Our results show that our method can enhance the spatial-temporal resolution and better resolve closely traveling cars. The spatial deconvolution U-Net model also enables the characterization of large-size vehicles to identify axle numbers and estimate the vehicle length. Monitoring large-size vehicles also benefits imaging deep earth by leveraging the surface waves induced by the dynamic vehicle-road interaction.

Precision Swine Farming has the potential to directly benefit swine health and industry profit by automatically monitoring the growth and health of pigs. We introduce the first system to use structural vibration to track animals and the first system for automated characterization of piglet group activities, including nursing, sleeping, and active times. PigSense uses physical knowledge of the structural vibration characteristics caused by pig-activity-induced load changes to recognize different behaviors of the sow and piglets. For our system to survive the harsh environment of the farrowing pen for three months, we designed simple, durable sensors for physical fault tolerance, then installed many of them, pooling their data to achieve algorithmic fault tolerance even when some do stop working. The key focus of this work was to create a robust system that can withstand challenging environments, has limited installation and maintenance requirements, and uses domain knowledge to precisely detect a variety of swine activities in noisy conditions while remaining flexible enough to adapt to future activities and applications. We provided an extensive analysis and evaluation of all-round swine activities and scenarios from our one-year field deployment across two pig farms in Thailand and the USA. To help assess the risk of crushing, farrowing sicknesses, and poor maternal behaviors, PigSense achieves an average of 97.8% and 94% for sow posture and motion monitoring, respectively, and an average of 96% and 71% for ingestion and excretion detection. To help farmers monitor piglet feeding, starvation, and illness, PigSense achieves an average of 87.7%, 89.4%, and 81.9% in predicting different levels of nursing, sleeping, and being active, respectively. In addition, we show that our monitoring of signal energy changes allows the prediction of farrowing in advance, as well as status tracking during the farrowing process and on the occasion of farrowing issues. Furthermore, PigSense also predicts the daily pattern and weight gain in the lactation cycle with 89% accuracy, a metric that can be used to monitor the piglets’ growth progress over the lactation cycle.

Heart rate is a critical metric for human cardiovascular health. Most common methods for measuring human heart rate involve wearable devices (e.g., electrocardiography, smart watches). However, such devices can cause discomfort to some patients, especially the elderly or young children. This paper presents FreePulse, a heart rate monitoring system for seated subjects using ambient vibrations. FreePulse builds on our past work using vibrations in the building structures around us to measure human activities and health. As people’s hearts beat, they push on the surfaces the body is touching, creating vibrations in those structures. We combine structure response characterization with human pulse modelling to identify these pulse-induced vibrations from ambient vibration. In testing, FreePulse has shown up to 96% pulse rate accuracy on average, competitive with consumer-grade wearable devices.

The standard way to keep track of inventory is through human counting or with a hand-held device and inputting the data into the warehouse management system (WMS). WMS can be quite expensive for small warehouses and human counting is susceptible to errors. Alternatively, we propose an automated way to keep track of inventory by leveraging the surveillance cameras readily available in the warehouses. In practice, there are challenges such as incorrect stock numbers, overfitting issues, and a mundane labeling process. To identify and mitigate those challenges, we present real-world deployment to various warehouses for a period of 4 months with up to 97 percent counting accuracy.

Object classification is a vital technology that is widely used to track and identify misplaced and out-of-stock items in shopping centers. While there have been a number of studies utilizing various sensing modalities such as computer vision, RFID, and vibration sensors, these methods are limited in their use due to privacy concerns, scalability, and the inability to identify stationary objects. To overcome these limitations, we propose a novel active vibration-sensing approach for object classification by utilizing music as an excitation source. Different objects can induce different deformations of the surface and further change the surface structural response. Therefore, we leverage vibrations from music on a store shelf and measure the structural responses on the surface when different objects are placed. Our evaluation of a store shelf demonstrates that distinct object characteristics lead to unique vibration responses, enabling accurate classification of 98.6% accuracy in distinguishing five common store objects. This study provides a promising avenue for a reliable, privacy-preserving, and scalable object classification system in various settings beyond shopping centers.

Sudden-onset hazards like earthquakes often induce cascading secondary hazards (e.g., landslides, liquefaction, debris flows, etc.) and subsequent impacts (e.g., building and infrastructure damage) that cause catastrophic human and economic losses. Rapid and accurate estimates of these hazards and impacts are critical for timely and effective post-disaster responses. Emerging remote sensing techniques provide pre- and post-event satellite images for rapid hazard estimation. However, hazards and damage often co-occur or colocate with underlying complex cascading geophysical processes, making it challenging to directly differentiate multiple hazards and impacts from satellite imagery using existing single-hazard models. We introduce DisasterNet, a novel family of causal Bayesian networks to model processes that a major hazard triggers cascading hazards and impacts and further jointly induces signal changes in remotely sensed observations. We integrate normalizing flows to effectively model the highly complex causal dependencies in this cascading process. A triplet loss is further designed to leverage prior geophysical knowledge to enhance the identifiability of our highly expressive Bayesian networks. Moreover, a novel stochastic variational inference with normalizing flows is derived to jointly approximate posteriors of multiple unobserved hazards and impacts from noisy remote sensing observations. Integrating with the USGS Prompt Assessment of Global Earthquakes for Response (PAGER) system, our framework is evaluated in recent global earthquake events. Evaluation results show that DisasterNet significantly improves multiple hazard and impact estimation compared to existing USGS products.

IoT (Internet of Things) devices, such as network-enabled wearables, are carried by increasingly more people throughout daily life. Information from multiple devices can be aggregated to gain insights into a person’s behavior or status. For example, an elderly care facility could monitor patients for falls by combining fitness bracelet data with video of the entire class. For this aggregated data to be useful to each person, we need a multi-modality association of the devices’ physical ID (i.e., location, the user holding it, visual appearance) with a virtual ID (e.g., IP address/available services). Existing approaches for multi-modality association often require intentional interaction or direct line-of-sight to the device, which is infeasible for a large number of users or when the device is obscured by clothing. We present IDIoT , a calibration-free passive sensing approach that fuses motion sensor information with camera footage of an area to estimate the body location of motion sensors carried by a user. We characterize results across three baselines to highlight how different fusing methodology results better than earlier IMU-vision fusion algorithms. From this characterization, we determine IDIoT is more robust to errors such as missing frames or miscalibration that frequently occur in IMU-vision matching systems.

Monitoring cardiovascular risk factors requires a method for con-tinuous heart rate monitoring. However, this typically relies on a wearable device which must be issued to, then charged and worn by the user. Ambient sensors can address these challenges, but have only been usable in scenarios where the body is in proximity or contact with these sensors, such as on a bed. In this paper, we present SeatBeats, a system that can monitor heart rate using ambient vibrations in the surface a user sits on. The system relies on the surface's response to heart beat-induced vibrations and utilizes autocorrelation to capture and extract the repeated pattern beats. Our experiments in a real office environment show up to 96% accuracy relative to a smartwatch heart rate sensor.

Pig vital sign monitoring (e.g., estimating the heart rate (HR) and respiratory rate (RR)) is essential to understand the stress level of the sow and detect the onset of parturition. It helps to maximize peri-natal survival and improve animal well-being in swine production. The existing approach mainly relies on manual measurement, which is labor-intensive and only provides a few points of information. Other sensing modalities such as wearables and cameras are developed to enable more continuous measurement, but are still limited due to animal discomfort, data transfer, and storage challenges. In this paper, we introduce PigV2, the first system to monitor pig heart rate and respiratory rate through ground vibrations. Our approach leverages the insight that both heartbeat and respiration generate ground vibrations when the sow is lying on the floor. We infer vital information by sensing and analyzing these vibrations. The main challenge in developing PigV2 is the overlap of vital- and non-vital-related information in the vibration signals, including pig movements, pig postures, pig-to-sensor distances, and so on. To address this issue, we first characterize their effects, extract their current status, and then reduce their impact by adaptively interpolating vital rates over multiple sensors. PigV2 is evaluated through a real-world deployment with 30 pigs. It has 3.4% and 8.3% average errors in monitoring the HR and RR of the sows, respectively.

In-home gait analysis is important for providing early diagnosis and adaptive treatments for individuals with gait disorders. Existing systems include wearables and pressure mats, but they have limited scalability due to dense deployment and device carrying/charging requirements. Recently, vision-based systems have been developed to enable scalable, accurate in-home gait analysis, but it faces privacy concerns due to the exposure of people's appearances and daily activities. To overcome these limitations, our prior work developed footstep-induced structural vibration sensing for in-home gait monitoring, which is device-free, wide-ranged, and perceived as more privacy-friendly. Although it has succeeded in temporal parameter estimation, it shows limited performance for spatial gait parameter estimation due to the low accuracy in footstep localization. In particular, the localization error mainly comes from the estimation error of the wave arrival time at the vibration sensors and its error propagation to wave velocity estimations. To this end, we present GaitVibe+, a vibration-based footstep localization method fused with temporarily installed cameras for in-home gait analysis. Our method has two stages: fusion and operating stages. In the fusion stage, both cameras and vibration sensors are installed to record only a few trials of the subject's footstep data, through which we characterize the uncertainty in wave arrival time and model the wave velocity profiles for the given structure. In the operating stage, we remove the camera to preserve privacy at home. The footstep localization is conducted by estimating the time difference of arrival (TDoA) over multiple vibration sensors, whose accuracy is improved through the reduced uncertainty and velocity modeling during the fusion stage. We evaluate GaitVibe+ through a real-world experiment with 50 walking trials. With only 3 trials of multi-modal fusion, our approach has an average localization error of 0.22 meters, which reduces the spatial gait parameter error by 4.1x (from 111.4% to 27.1%) compared to the existing work.

We present a passive and non-intrusive sensing system for monitoring hand washing activity using structural vibration sensing. Proper hand washing is one of the most effective ways to limit the spread and transmission of disease, and has been especially critical during the COVID-19 pandemic. Prior approaches include direct observation and sensing-based approaches, but are limited in non-clinical settings due to operational restrictions and privacy concerns in sensitive areas such as restrooms. Our work introduces a new sensing modality for hand washing monitoring, which measures hand washing activity-induced vibration responses of sink structures, and uses those responses to monitor the presence and duration of hand washing. Primary research challenges are that vibration responses are similar for different activities, occur on different surfaces/structures, and tend to overlap/coincide. We overcome these challenges by extracting information about signal periodicity for similar activities through cepstrum-based features, leveraging hierarchical learning to differentiate activities on different surfaces, and denoting “primary/secondary” activities based on their relative frequency and importance. We evaluate our approach using real-world hand washing data across four different sink structures/locations, and achieve an average F1-score for hand washing activities of 0.95, which represents an 8.8X and 10.2X reduction in error over two different baseline approaches.

Fine-grained city-scale outdoor air pollution maps provide important environmental information for both city managers and residents. Installing portable sensors on vehicles (e.g., taxis, Ubers) provides a low-cost, easy-maintenance, and high-coverage approach to collecting data for air pollution estimation. However, as non-dedicated platforms, vehicles like taxis usually prefer gathering at busy areas of a city where it is more likely to pick up riders. This leaves many parts of the city unsensed or less-sensed. In addition, due to the natural changes in a city and the movements of the vehicles, the sensed and unsensed areas change over time. Consequently, challenges of air pollution estimation with data collected by non-dedicated mobile platforms are twofold:i.data coverage is sparse;ii.data coverage changes over time. Therefore, the major research question is: how can we derive accurate and robust fine-grained field (e.g., air pollution) estimation given dynamic and sparse data collected from uncontrollable mobile sensing platforms? This paper presents adaptiveHMSS, an adaptivehybridmodel-enabledsensingsystem for fine-grained air pollution estimation with dynamic and sparse data collected from uncontrollable mobile sensing platforms, which is achieved by combining the advantages of aphysics guided modeland adata driven model. To address the challenge of sparse coverage, the physical understanding of the spatiotemporal correlation for air pollution distribution in thephysics guided modelis utilized to infer values at unsensed sparse areas. Meanwhile, thedata driven modelis adopted to estimate the air pollution influential factors (e.g., buildings) not included in thephysics guided model. To address the challenge of time-varying coverage, an adaptive model combination algorithm is designed to enable the system bias to either of the two models according to the amount of data collection and uncertainty of the model. To evaluate the system performance, we deployed 47 air pollution sensing devices on taxis and fixed locations in 2 cities for both controlled and uncontrolled experiments for over two weeks. The results show that with a resolution of$500 \;\mathrm m$by$500\;\mathrm m$by$1\;\mathrm {hour}$, our system achieves up to$3.2\times$error reduction when compared to the baseline approaches.

Automated monitoring of livestock behavior can help farmers economically by detecting changes in animal welfare. Prior approaches use video, which requires light and high storage capability, or motion detection, which has difficulty separating subtle activities. Wearable sensors can address these issues but are vulnerable to destruction by the animals. To the best of our knowledge, we present the first system that uses structural vibration to track animal behavior, and the first system to automatically detect piglet nursing. PigNet uses vibration sensors attached to a pig pen to sense the unique vibration patterns and changes in structural response caused by the animals' movement and position within the pen. Combined with our knowledge of pig behavior, we use this physical knowledge of vibration characteristics to detect pig activities and track piglet growth in a real farm environment. Our system is designed to be robust to the harsh environment, which can create unpredictable noise, as well as physically damage or disconnect sensor nodes. When deployed in a real-world farm environment, our system was able to achieve a daily pen-level status profile of up to 90% accuracy, which tracks nursing activity, sow lying activity, and changes in piglet growth over the weeks-long pre-weaning period.

Detecting a previously unknown person (newcomer detection) is critical for visitor management, intruder prevention, and access control in smart buildings. Biometrics have been used to detect newcomers, including face, fingerprint, voice, iris, etc. These approaches often require active participation of the users or require dense instrumentation. Prior work using footstep-induced floor vibration to identify people removes these requirements, but only functions for known people due to the high variability in footstep-induced vibrations and the limited number of predicted classes in supervised learning. To overcome the limitations, we introduce a newcomer detection system based on non-parametric Bayesian learning, which models the variability and distribution of consecutive footstep-induced floor vibration for newcomer walking patterns. Preliminary results from real-world experiments with 6 people show up to 92% accuracy in newcomer detection with an average of 4 consecutive footsteps.

Monitoring the compliance of social distancing is critical for schools and offices to recover in-person operations in indoor spaces from the COVID-19 pandemic. Existing systems focus on vision- and wearable-based sensing approaches, which require direct line-of-sight or device-carrying and may also raise privacy concerns. To overcome these limitations, we introduce a new monitoring system for social distancing compliance based on footstep-induced floor vibration sensing. This system is device-free, non-intrusive, and perceived as more privacy-friendly. Our system leverages the insight that footsteps closer to the sensors generate vibration signals with larger amplitudes. The system first estimates the location of each person relative to the sensors based on signal energy and then infers the distance between two people. We evaluated the system through a real-world experiment with 8 people, and the system achieves an average accuracy of 97.8% for walking scenario classification and 80.4% in social distancing violation detection.

We introduce a physics-guided signal processing approach to extract a damage-sensitive and domain-invariant (DS & DI) feature from acceleration response data of a vehicle traveling over a bridge to assess bridge health. Motivated by indirect sensing methods' benefits, such as low-cost and low-maintenance, vehicle-vibration-based bridge health monitoring has been studied to efficiently monitor bridges in real-time. Yet applying this approach is challenging because 1) physics-based features extracted manually are generally not damage-sensitive, and 2) features from machine learning techniques are often not applicable to different bridges. Thus, we formulate a vehicle bridge interaction system model and find a physics-guided DS & DI feature, which can be extracted using the synchrosqueezed wavelet transform representing non-stationary signals as intrinsic-mode-type components. We validate the effectiveness of the proposed feature with simulated experiments. Compared to conventional time-and frequency-domain features, our feature provides the best damage quantification and localization results across different bridges in five of six experiments.

This paper presents the Active Structural Occupant Detector, an active vibration sensing system that detects stationary occupants through injection of vibration signals into the floor. Many smart buildings require occupant detection in order to provide personalized services. Some examples include optimized energy usage and security. Several methods currently exist for occupant detection, each with their own drawbacks, such as installation requirements. Structural vibration sensing overcomes many of these drawbacks by measuring impulses created by occupants to infer their movements, but cannot detect stationary occupants. The ASOD utilizes active vibration sources, which inject acoustic waves into the structure then measure how the structure responds to them. Any occupants present interact with these waves, causing changes to the measured signal. By characterizing the changes in how the waves travel, we can predict the presence or lack of an occupant with up to a 97.7% accuracy, as demonstrated by experiments in a real- world environment.

Recent work has shown that two wearable devices worn on the same user can exploit gait as a secret source to generate a common key for secure pairing. The main threat of using gait comes from side-channel attackers who can use cameras to record the walking user and extract accelerations from the video to pair with legitimate devices. We propose a novel pre-step that uses a CNN-LSTM deep learning model to classify the acceleration measurement devices, i.e., between IMU vs. Camera. We prototype the pre-step and evaluate it using real subjects. Our results show that the proposed pre-step can achieve high classification success rates. The experiments with different cut-off frequencies show that the higher acceleration frequencies appear to contain more distinguishable features to classify camera from IMU.

Small unmanned autonomous vehicles (UAVs) swarms are becoming ubiquitous in a number of applications (e.g., surveying, monitoring, and situational awareness). Indoor environments may contain metal equipment that temporarily disrupts radio reception. During these momentary interruptions, a small UAV needs to be able to broadcast a 'heartbeat' to indicate that it is not damaged or lost. Considering alternative messaging modalities, we observe that light-based methods require line-of sight, which is not guaranteed when UAVs are moving through a cluttered environment, while a naive sound-based method is easily drowned out by the UAV's own loud motor and propeller noise.

Vehicular mobile crowdsensing (MCS) enables many smart city applications. Ridesharing vehicle fleets provide promising solutions to MCS due to the advantages of low cost, easy maintenance, high mobility, and long operational time. However, as nondedicated mobile sensing platforms, the first priorities of these vehicles are delivering passengers, which may lead to poor sensing coverage quality. Therefore, to help MCS derive good (large and balanced) sensing coverage quality, an actuation system is required to dispatch vehicles with a limited amount of monetary budget. This article presents PAS, a prediction-based actuation system for city-wide ridesharing vehicular MCS to achieve optimal sensing coverage quality with a limited budget. In PAS, two prediction models forecast probabilities of potential near-future vehicle routes and ride requests across the city. Based on prediction results, a prediction-based actuation planning algorithm is proposed to decide which vehicles to actuate and the corresponding routes. Experiments on city-scale deployments and physical feature-based simulations show that our PAS achieves up to 40% more improvement in sensing coverage quality and up to 20% higher ride request matching rate than baselines. In addition, to achieve a similar level of sensing coverage quality as the baseline, our PAS only needs 10% budget.

Vehicular crowd sensing systems are designed to achieve large spatio-temporal sensing coverage with low-cost in deployment and maintenance. For example, taxi platforms can be utilized for sensing city-wide air quality. However, the goals of vehicle agents are often inconsistent with the goal of the crowdsourcer. Vehicle agents like taxis prioritize searching for passenger ride requests (defined as task requests), which leads them to gather in busy regions. In contrast, sensing systems often need to sample data over the entire city with a desired distribution (e.g., Uniform distribution, Gaussian Mixture distribution, etc.) to ensure sufficient spatio-temporal information for further analysis. This inconsistency decreases the sensing coverage quality and thus impairs the quality of the collected information. A simple approach to reduce the inconsistency is to greedily incentivize the vehicle agents to different regions. However, incentivization brings challenges, including the heterogeneity of desired target distributions, limited budget to incentivize more vehicle agents, and the high computational complexity of optimizing incentivizing strategies. To this end, we present a vehicular crowd sensing system to efficiently incentivize the vehicle agents to match the sensing distribution of the sampled data to the desired target distribution with a limited budget. To make the system flexible to various desired target distributions, we formulate the incentivizing problem as a new type of non-linear multiple-choice knapsack problem, with the dissimilarity between the collected data distribution and the desired distribution as the objective function. To utilize the budget efficiently, we design a customized incentive by combining monetary incentives and potential task (ride) requests at the destination. Meanwhile, an efficient optimization algorithm, iLOCuS, is presented to plan the incentivizing policy for vehicle agents to decompose the sensing distribution into two distinct levels: time-location level and vehicle level, to approximate the optimal solution iteratively and reduce the dissimilarity objective. Our experimental results based on real-world data show that our system can reduce up to 26.99 percent of the dissimilarity between the sensed and target distributions compared to benchmark methods.

We present Deskbuddy, a vibration-based system that can track a user's activities through their desk. Tracking sitting and other office related activities let us remind the user to have healthier working habits, as well as giving information about how office spaces are used. Many solutions have been proposed for office activity tracking, but they either require the user to wear a device, or they use cameras or microphones, which can make subjects uncomfortable. Our demo includes a small vibration sensor that sits on a table that can detect four office related activities. We capture the signal from the vibration sensor, extract features, and perform classification on the resulting features. The full functionality of the system will be shown in a video. In order for our demo to be more effective in a crowded environment, we have re-trained it to detect only typing versus not typing.

Gait health monitoring is critical for condition diagnosis and fall prediction in elderly populations. Existing methods for gait health monitoring (e.g. direct observation and sensing) are not suitable for non-clinical environments due to qualitative assessments or operational limitations. Our method utilizes footstep-induced floor vibration sensing to provide a passive gait health monitoring platform that can be used in non-clinical environments (e.g. home settings) to provide gait health information in a timely manner. We decompose vibration responses to obtain signal peaks that correspond to temporal gait information and leverage foot dominance to learn a signal amplitude-footstep ground reaction force transfer function. Preliminary results show that temporal gait parameters can be estimated with up to 99% accuracy and gait balance symmetry can be estimated with as low as 10.4% error.

Secure pairing is an important problem especially due to large number of IoT devices. In this paper, we propose PosePair++, to enable a camera to securely pair with IoT devices which are equipped with IMU sensors. Existing context-based pairing approaches do not adequately address this problem due to differing sensing modalities. To address this challenge, we propose to translate the signals from heterogeneous sensing modalities to a common space, namely 2D acceleration. In this demo, we present PosePair++'s robustness against different types of attackers (i.e., attackers that observe the user's motion, or attackers performing mimicking attack).

Mobile crowd sensing (MCS) collects city-scale sensing data with low cost and high efficiency. One important goal of MCS is to ensure high quality of sensing coverage to provide sufficient information to data analysis end. However, the goal of the MCS may be inconsistent with the goal of vehicles. This inconsistency between goals results in a bad sensing coverage and decreases the quality of the collected information. Key challenges to resolve this inconsistency include the heterogeneous target desired spatio-temporal distributions, limited budget constraining the ability to incentivize more taxis, and high computational complexity.

The surveillance of air pollution is becoming a highly concerned issue for city residents and urban administrators. Fixed air quality stations as well as mobile gas sensors have been deployed for air quality monitoring but with sparse observations over the entire temporal-spatial space. Therefore, an inference algorithm is essential for comprehensive fine-grained air pollution sensing. Conventional physically-based models can hardly be applied to all the scenarios, while pure data-driven methods suffer from sampling bias and overfitting problems. This paper presents a hybrid algorithm for air pollution inference by guiding the data learning process with physical model. The quantitative combination of knowledge from observed dataset and a discretized convective-diffusion model is performed within a multi-task learning scheme. Evaluations show that, benefited from physical guidance, our hybrid method obtains higher extrapolation ability and more robustness, achieving the same performance with 1/8 sample amount and obtaining 31.9% less error in noisy synthesized environment. In a real-world deployment in Tianjin, our algorithm outperforms the pure data-driven model with 9.69% less inference error over a 9-day PM2.5data collection.

Deep learning models have been successfully used in medical image analysis problems but they require a large amount of labeled images to obtain good performance. However, such large labeled datasets are costly to acquire. Active learning techniques can be used to minimize the number of required training labels while maximizing the model's performance. In this work, we propose a novel sampling method that queries the unlabeled examples that maximize the average distance to all training set examples in a learned feature space. We then extend our sampling method to define a better initial training set, without the need for a trained model, by using Oriented FAST and Rotated BRIEF (ORB) feature descriptors. We validate MedAL on 3 medical image datasets and show that our method is robust to different dataset properties. MedAL is also efficient, achieving 80% accuracy on the task of Diabetic Retinopathy detection using only 425 labeled images, corresponding to a 32% reduction in the number of required labeled examples compared to the standard uncertainty sampling technique, and a 40% reduction compared to random sampling.

In the Internet of Things (IoT) paradigm, it is important to easily setup and control devices, which is achieved by pairing. In this work, we present a novel pairing scheme that utilizes heterogeneous sensing. The core idea is that devices with different sensing capabilities can still detect common information about their user. We demonstrate this idea through an example application consisting of a camera and IoT devices with inertial sensors. As the user holds a device and moves it around, the camera captures the human's pose and compares it to the IoT device motion. If the motion features are similar enough, the device can be successfully paired to the camera's network.

Indoor localization systems cannot rely on the same mechanisms, like GPS, that are used for outdoor or large-scale localization. Instead, autonomous or user-carried devices are often localized by measuring the time taken for an emitted signal to reach a known location; this signal can be sound, light, radio waves, or another similar sensed quantity. Autonomous mobile devices already contain motors, which produce sounds as a side effect of their operation, and so can potentially be included in a localization scheme without new hardware. In this paper, we briefly outline the challenges that need to be met for accurate detection and identification of motor-produced signals. We present a method for improving signal resolution for linear chirps that improves cross-correlation based signal detection by up to 2.8X.

Mobile sensing systems are deployed for urban air pollution monitoring to increase coverage over a city. However, the sampling irregularity brings great challenges for fine-grained pollution field recovery. To address this problem, we proposed a generative model based inference algorithm. By modeling the air pollution evolution and data sampling process separately, the temporal-spatial correlation of pollution field can be considered with irregular sampled data. We use a convolutional long-short term memory structure in the generative model and train it with the scattered observations from mobile sensing. Evaluations on synthesized data and a deployment in the city of Tianjin show that our algorithm accurately captures fine-grained PM2.5 pollution patterns and changes. The average inference error is 6.7μg/m3, which achieves 23.8% improvement over existing techniques.

No abstract available.

Context-based pairing solutions increase the usability of IoT device pairing by eliminating any human involvement in the pairing process. This is possible by utilizing on-board sensors (with same sensing modalities) to capture a common physical context (e.g., ambient sound via each device's microphone). However, in a smart home scenario, it is impractical to assume that all devices will share a common sensing modality. For example, a motion detector is only equipped with an infrared sensor while Amazon Echo only has microphones. In this paper, we develop a new context-based pairing mechanism called Perceptio that uses time as the common factor across differing sensor types. By focusing on the event timing, rather than the specific event sensor data, Perceptio creates event fingerprints that can be matched across a variety of IoT devices. We propose Perceptio based on the idea that devices co-located within a physically secure boundary (e.g., single family house) can observe more events in common over time, as opposed to devices outside. Devices make use of the observed contextual information to provide entropy for Perceptio's pairing protocol. We design and implement Perceptio, and evaluate its effectiveness as an autonomous secure pairing solution. Our implementation demonstrates the ability to sufficiently distinguish between legitimate devices (placed within the boundary) and attacker devices (placed outside) by imposing a threshold on fingerprint similarity. Perceptio demonstrates an average fingerprint similarity of 94.9% between legitimate devices while even a hypothetical impossibly well-performing attacker yields only 68.9% between itself and a valid device.

Occupant identification proves crucial in many smart home applications such as automated home control and activity recognition. Previous solutions are limited in terms of deployment costs, identification accuracy, or usability. We propose SenseTribute , a novel occupant identification solution that makes use of existing and prevalent on-object sensors that are originally designed to monitor the status of objects to which they are attached. SenseTribute extracts richer information content from such on-object sensors and analyzes the data to accurately identify the person interacting with the objects. This approach is based on the physical phenomenon that different occupants interact with objects in different ways. Moreover, SenseTribute may not rely on users’ true identities, so the approach works even without labeled training data. However, resolution of information from a single on-object sensor may not be sufficient to differentiate occupants, which may lead to errors in identification. To overcome this problem, SenseTribute operates over a sequence of events within a user activity, leveraging recent work on activity segmentation. We evaluate SenseTribute using real-world experiments by deploying sensors on five distinct objects in a kitchen and inviting participants to interact with the objects. We demonstrate that SenseTribute can correctly identify occupants in 96% of trials without labeled training data, while per-sensor identification yields only 74% accuracy even with training data.

Incorrect muscle activation can lead to sub-optimal performance, muscle imbalance, and eventually bodily injury. Consequently, assessing muscle activation is important for both excelling in exercise, athletics, and professional sports in general. Existing techniques for assessing muscle activation, such as electromyography, are invasive, requiring needles inserted directly into the muscle or electrodes that have considerable placement requirements (shaving, gels, etc.). This makes them unsuitable for active environments. In addition, factors such as body motion noise that results from the high-impact movements encountered in active sports environments easily corrupt sensor data. This compounds the unsuitability of these systems in the sports and exercise arena. As a result, such systems have been explored mostly in clinical rather than sports-based scenarios. We present MyoVibe, a system for sensing and determining muscle activation in high-mobility, high-impact exercise scenarios. MyoVibe senses and interprets multiple muscle vibration signals obtained from a wearable network of accelerometers to determine muscle activation. By utilizing a diverse feature set combined with the simple yet effective motion artifact mitigation technique, MyoVibe can reduce inertial sensor noise in these high-mobility exercises. As a result, MyoVibe can detect muscle activation with greater than 97% accuracy.

No abstract available.

In this paper, we present Intercom, a simulator framework that provides separate components to address the interdependent aspects of IoT systems, such as sensing, physical interaction, wireless communication, and computation. We initially evaluate a scalable sensing and communication model, which simulates wireless signal strength measurements with an average error of 6.1dBm.

Touch surfaces are intuitive interfaces for computing devices. Most of the traditional touch interfaces (vision, IR, capacitive, etc.) have mounting requirements, resulting in specialized touch surfaces limited by their size, cost, and mobility. More recent work has shown that vibration-based touch sensing techniques can localize taps/knocks, which provides a low-cost flexible alternative. These surfaces are envisioned as intuitive inputs for applications such as interactive meeting tables, smart kitchen appliance control, etc. However, due to dispersive and reflective properties of various vibrating mediums, it is difficult to localize taps accurately on ubiquitous surfaces. Furthermore, no work has been done on tracking continuous swipe interactions through vibration sensing.

In this paper, given the scenario of a mobile sensing system for air pollution monitoring, we aim at the cause and influence of delay effect on measurement and present a filter-based solution to calibrate the sensing data. We also validate the idea and solution by a real-data experiment. It indicates that the solution decreases deviation on spatial measurement and can be applied in mobile sensing systems to improve the sensing data quality.

Low-cost sensors are widely used to realize large-scale deployment for sensing systems. In this paper, we discuss challenges in using industrial-grade gas sensors for air quality monitoring. To overcome variation due to system errors, we present a framework for individualized calibration. Within the framework, multiple regression and interpolation methods are prepared for alternative optimization on fitting sensors' response to gas concentration.

Skeletal muscles are pivotal for sports and exercise. However, overexertion of skeletal muscles causes muscle fatigue which can lead to injury. Consequently, understanding skeletal muscle fatigue is important for injury prevention. Current ways to estimate exhaustion revolve around self-estimation or inference from such sensors as force sensors, electromyography e.t.c. These methods are not always reliable, especially during isotonic exercises. Toward this end, we present Burnout - a wearable system for quantifying skeletal muscle fatigue in an exercise setting. Burnout uses accelerometers to sense skeletal muscle vibrations. From these vibrations, Burnout obtains a region based feature (R- Feature), in the case of this work, the region mean power frequency (R-MPF) gradient to correlate the sensed vibrations to a known ground truth measure of skeletal muscle fatigue, i.e., Dimitrov's spectral fatigue index gradient. We evaluate Burnout on the biceps and quadriceps of 5 healthy participants through four different exercises, collected in a real world environment. Our results show that by using this R-MPF feature on our real world data set, Burnout is able to reduce the error of estimating the ground truth fatigue index gradient by up to 50% on average compared to using the standard MPF feature.

This paper presents a hybrid adaptive particle filter (HAP) with online feedback to dynamically reconstruct high spatial-temporal resolution air pollution information from sparse vehicular based sensors. To deal with data sparsity, we apply both spatial and temporal correlation of air dispersion to reduce data dimension requirement. HAP adaptively predicts when the accumulated prediction error is low and then uses data compensation for correction whenever the prediction error becomes high. The preliminary results based on the city scale deployments with 10 taxis show that our system achieves up to 50% reduction on system errors.

We introduce a sensing system which leverages footstep-induced structural vibration for occupant localization. Such localization is important for many smart building applications, such as efficient building management, senior/health care, and security. Compared to other sensing approaches, footstep-induced vibration provides a sensing system which is sparse and non-intrusive. The main challenge of achieving high accuracy using such approach is frequency-dependent wave propagation characteristics, such as dispersion, in floor structure. These characteristics result in distortions in the shape of signal. To overcome such distortions, we decompose the vibration signals into different frequency components using wavelet transform and focus on specific components in all the sensors. In a set of experiments in a real structure, our approach results in average localization errors of 0.41 meters, a 4.4X reduction compared to a baseline approach using raw data.

Tracking multiple people in an indoor environment enables various smart building applications such as HVAC energy saving, patient/child monitoring, etc. Researchers have explored various sensing methods including vision, motion, and RF, which either require specific installation requirements or high deployment density. We introduce a passive sparse sensing method based on ambient structural vibration induced by foot strikes. Our system tracks multiple people based on the premise that human foot strikes have spatio-temporal variation, and hence do not fully overlap. The system achieved less than 0.4m accuracy in both one and two persons stepping conditions.

According to the World Health Organization (WHO), outdoor air pollution led to an estimated 3.7 million premature deaths worldwide in 2012. To address this problem, it is necessary for both residents and city administrations to understand air quality in their immediate environment with fine-grained temporal-spatial resolution. Currently both fixed and mobile systems are used to attempt to sense the pollution field. However, they generally are expensive, cover small areas and thus result in lower accuracy.

No abstract available.

Skeletal muscles are activated to generate the force needed for movement in most high motion sports and exercises. However, incorrect skeletal muscle activation during these sports and exercises, can lead to sub-optimal performance and injury. Existing techniques are susceptible to motion artifacts, particularly when used in high motion sports (e.g. jumping, cycling, etc.). They require limited body movement, or experts to manually interpret results, making them unsuitable in sports scenarios.

Globally, infrastructure is a vital asset for economic prosperity, but condition assessments tend to be subjective and infrequent [2]. The lack of objective information leads to sub-optimal capital replacement projects, and the information lag prevents timely repair. In this poster we focus on techniques for monitoring rail-based transit infrastructure, although many of the findings could easily be applied in other types of infrastructure. One monitoring solution is to instrument the tracks and track structures, but given the expanse of our transit networks, the installation and maintenance cost of such a sensor network would be prohibitively high. A second solution is to use a custom instrumentation vehicle capable of monitoring the infrastructure as it moves [1]. However, such dedicated vehicles tend to be expensive, particularly in rail where monitoring is more specialized, so to keep costs down, infrastructure owners use these vehicles infrequently.

We describe a step-level indoor localization system which uses the ground vibration induced by human footsteps. Indoor localization is important for various smart building applications, including resources arrangement optimization, patient/customer tracking, etc. Geophones are used to measure the ground vibrations and time difference of arrival (TDoA) for different sensors are used to solve the multilateration localization problem. The advantages of this system include its sparsity and also its stability over time. Lesser dependency on instrument people is another upside of this system. The results of pilot tests show that this system can be successfully used for indoor localization.

The dynamic responses of the structure provide a variety of information about the structure as well as people inside. Compared with traditional sensing methods, structural sensing method serves more general sensing purposes due to the diversity of information it can infer from structural responses. For example, by sensing the structural vibration, a system can track and identify a person through vibration caused by their gaits [5, 6]. Such non-intrusive identification and tracking system enables various smart building applications, such as patient monitoring system at advanced hospitals and nursing homes.

This paper introduces a "multi-choice" offer strategy for an automated agent conducting bilateral multi-issue negotiations in an agent-to-human negotiation setting. Assuming that a rational human counterpart is more likely to concede on less important issues, we developed a modified dynamic weighted majority (DWM) learning algorithm for the negotiation agent to estimate the issue weights and issue ranks of the human counterpart. The agent then utilizes these estimates to strategically propose counter-offers with multiple choices to the human counterpart. This strategy allows the agent to expedite the negotiation process and increase the chance of agreement by improving the satisfaction level of the counterpart. We validated this offer strategy using two sets of buyer behavior data: one simulated based on time-dependent behavior models used in the literature, and another collected from a human experiment on automated negotiations. Results indicate that, when compared to other offer strategies described in the literature with similar learning speeds, (i) the modified DWM-based learning algorithm estimates the counterpart's issue weight/rank more accurately, and (ii) the multi-choice offer strategy utilizing the learning algorithm makes more attractive offers to the counterpart while maintaining the same utility for the agent.