Although WiFi-based contactless sensing has made significant progress in the past decade, most prior work still focus on the reflection zone far from WiFi transceivers, while few studies explore the diffraction zone near transceivers. Additionally, previous diffraction models only consider the CSI amplitude signal and ignore the impact of multipath. In this work, we develop an accurate diffraction model to characterize the relationship between both CSI amplitude and phase and target's movement in the diffraction zone. We further put forward the deformation forms of the model under static multipath conditions and find that the CSI patterns vary significantly with multipath. Consequently, the common assumption of a one-to-one mapping between CSI patterns and activities in existing work fails due to multipaths, degrading sensing performance when multipath changes. To address this challenge, we propose to extract a relative change pattern from CSI signals to recover the one-to-one mapping relations and eliminate the impact of static multipath. Extensive experiments under various multipath conditions demonstrate an accuracy higher than 96% for the coarse-grained intrusion detection and an average error rate of 0.6 bpm for the fine-grained respiration monitoring.

WiFi-based device-free motion tracking systems track persons without requiring them to carry any device. Existing work has explored signal parameters such as time-of-flight (ToF), angle-of-arrival (AoA), and Doppler-frequency-shift (DFS) extracted from WiFi channel state information (CSI) to locate and track people in a room. However, they are not robust due to unreliable estimation of signal parameters. ToF and AoA estimations are not accurate for current standards-compliant WiFi devices that typically have only two antennas and limited channel bandwidth. On the other hand, DFS can be extracted relatively easily on current devices but is susceptible to the high noise level and random phase offset in CSI measurement, which results in a speed-sign-ambiguity problem and renders ambiguous walking speeds. This paper proposes WiTraj, a device-free indoor motion tracking system using commodity WiFi devices. WiTraj improves tracking robustness from three aspects: 1) It significantly improves DFS estimation quality by using the ratio of the CSI from two antennas of each receiver, 2) To better track human walking, it leverages multiple receivers placed at different viewing angles to capture human walking and then intelligently combines the best views to achieve a robust trajectory reconstruction, and, 3) It differentiates walking from in-place activities, which are typically interleaved in daily life, so that non-walking activities do not cause tracking errors. Experiments show that WiTraj can significantly improve tracking accuracy in typical environments compared to existing DFS-based systems. Evaluations across 9 participants and 3 different environments show that the median tracking error$<2.5\%$for typical room-sized trajectories.

Deviant eating behavior such as skipping meals and consuming unhealthy meals has a significant association with mental well-being in college students. However, there is more to what an individual eats. While eating patterns form a critical component of their mental well-being, insights and assessments related to the interplay of eating patterns and mental well-being remain under-explored in theory and practice. To bridge this gap, we use an existing real-time eating detection system that captures context during meals to examine how college students’ eating context associates with their mental well-being, particularly their affect, anxiety, depression, and stress. Our findings suggest that students’ irregularity or skipping meals negatively correlates with their mental well-being, whereas eating with family and friends positively correlates with improved mental well-being. We discuss the implications of our study in designing dietary intervention technologies and guiding student-centric well-being technologies.

We describe a general-purpose end-to-end audio embeddings generator that can be easily adapted to various acoustic scene and event classification applications. In contrast to many other models for audio classification, this end-to-end embeddings generator does not require a separate feature extraction step, but processes audio samples directly which simplifies its porting into hardware platforms. Our approach learns a generic audio embedding representation that is pre-trained on a large audio dataset. It can then be fine-tuned via transfer learning with limited data requirements for an audio classification target application. We compare different transfer learning strategies and evaluate the proposed embeddings generator on three popular publicly available datasets, and report results for different model sizes. The experimental results show that this end-to-end approach presents a scalable, efficient, and competitive alternative to more common spectral-based audio classifiers.

Determining the mode of transport of an individual is an important element of contextual information. In particular, we focus on differentiating between different forms of motorized transport such as car, bus, subway etc. Our approach uses location information and features derived from transit route information (schedule information, not real-time) published by transit agencies. This enables no up-front training or learning of routes and can be deployed instantly to a new place since most transit agencies publish this information. Combined with motion detection using phone accelerometers, we obtain a classification accuracy of around 90% on 50+ hours of car and transit data.

Stress can have long term adverse effects on individuals' physical and mental well-being. Changes in the speech production process is one of many physiological changes that happen during stress. Microphones, embedded in mobile phones and carried ubiquitously by people, provide the opportunity to continuously and non-invasively monitor stress in real-life situations. We propose StressSense for unobtrusively recognizing stress from human voice using smartphones. We investigate methods for adapting a one-size-fits-all stress model to individual speakers and scenarios. We demonstrate that the StressSense classifier can robustly identify stress across multiple individuals in diverse acoustic environments: using model adaptation StressSense achieves 81% and 76% accuracy for indoor and outdoor environments, respectively. We show that StressSense can be implemented on commodity Android phones and run in real-time. To the best of our knowledge, StressSense represents the first system to consider voice based stress detection and model adaptation in diverse real-life conversational situations using smartphones.

We demonstrate CarSafe, a driver safety application for Android phones that fuses information from both front and back cameras and others embedded sensors on the phone to detect and alert drivers to dangerous driving conditions in and outside of the car. In this demonstration, we set up an emulated driving environment to show how CarSafe works.

Driving while being tired or distracted is dangerous. We are developing the CafeSafe app for Android phones, which fuses information from both front and back cameras and others embedded sensors on the phone to detect and alert drivers to dangerous driving conditions in and outside of the car. CarSafe uses computer vision and machine learning algorithms on the phone to monitor and detect whether the driver is tired or distracted using the front camera while at the same time tracking road conditions using the back camera. CarSafe is the first dual-camera application for smart-phones.

Sensor-enabled smartphones are opening a new frontier in the development of mobile sensing applications. The recognition of human activities and context from sensor-data using classification models underpins these emerging applications. However, conventional approaches to training classifiers struggle to cope with the diverse user populations routinely found in large-scale popular mobile applications. Differences between users (e.g., age, sex, behavioral patterns, lifestyle) confuse classifiers, which assume everyone is the same. To address this, we propose Community Similarity Networks (CSN), which incorporates inter-person similarity measurements into the classifier training process. Under CSN every user has a unique classifier that is tuned to their own characteristics. CSN exploits crowd-sourced sensor-data to personalize classifiers with data contributed from other similar users. This process is guided by similarity networks that measure different dimensions of inter-person similarity. Our experiments show CSN outperforms existing approaches to classifier training under the presence of population diversity.

Modeling human behavior requires vast quantities of accurately labeled training data, but for ubiquitous people-aware applications such data is rarely attainable. Even researchers make mistakes when labeling data, and consistent, reliable labels from low-commitment users are rare. In particular, users may give identical labels to activities with characteristically different signatures (e.g., labeling eating at home or at a restaurant as "dinner") or may give different labels to the same context (e.g., "work" vs. "office"). In this scenario, labels are unreliable but nonetheless contain valuable information for classification. To facilitate learning in such unconstrained labeling scenarios, we propose Community-Guided Learning (CGL), a framework that allows existing classifiers to learn robustly from unreliably-labeled user-submitted data. CGL exploits the underlying structure in the data and the unconstrained labels to intelligently group crowd-sourced data. We demonstrate how to use similarity measures to determine when and how to split and merge contributions from different labeled categories and present experimental results that demonstrate the effectiveness of our framework.

Supporting continuous sensing applications on mobile phones is challenging because of the resource demands of long-term sensing, inference and communication algorithms. We present the design, implementation and evaluation of the Jigsaw continuous sensing engine, which balances the performance needs of the application and the resource demands of continuous sensing on the phone. Jigsaw comprises a set of sensing pipelines for the accelerometer, microphone and GPS sensors, which are built in a plug and play manner to support: i) resilient accelerometer data processing, which allows inferences to be robust to different phone hardware, orientation and body positions; ii) smart admission control and on-demand processing for the microphone and accelerometer data, which adaptively throttles the depth and sophistication of sensing pipelines when the input data is low quality or uninformative; and iii) adaptive pipeline processing, which judiciously triggers power hungry pipeline stages (e.g., sampling the GPS) taking into account the mobility and behavioral patterns of the user to drive down energy costs. We implement and evaluate Jigsaw on the Nokia N95 and the Apple iPhone, two popular smartphone platforms, to demonstrate its capability to recognize user activities and perform long term GPS tracking in an energy-efficient manner.

Top end mobile phones include a number of specialized (e.g., accelerometer, compass, GPS) and general purpose sensors (e.g., microphone, camera) that enable new people-centric sensing applications. Perhaps the most ubiquitous and unexploited sensor on mobile phones is the microphone - a powerful sensor that is capable of making sophisticated inferences about human activity, location, and social events from sound. In this paper, we exploit this untapped sensor not in the context of human communications but as an enabler of new sensing applications. We propose SoundSense, a scalable framework for modeling sound events on mobile phones. SoundSense is implemented on the Apple iPhone and represents the first general purpose sound sensing system specifically designed to work on resource limited phones. The architecture and algorithms are designed for scalability and Soundsense uses a combination of supervised and unsupervised learning techniques to classify both general sound types (e.g., music, voice) and discover novel sound events specific to individual users. The system runs solely on the mobile phone with no back-end interactions. Through implementation and evaluation of two proof of concept people-centric sensing applications, we demostrate that SoundSense is capable of recognizing meaningful sound events that occur in users' everyday lives.

We present the design, implementation, evaluation, and user ex periences of theCenceMe application, which represents the first system that combines the inference of the presence of individuals using off-the-shelf, sensor-enabled mobile phones with sharing of this information through social networking applications such as Facebook and MySpace. We discuss the system challenges for the development of software on the Nokia N95 mobile phone. We present the design and tradeoffs of split-level classification, whereby personal sensing presence (e.g., walking, in conversation, at the gym) is derived from classifiers which execute in part on the phones and in part on the backend servers to achieve scalable inference. We report performance measurements that characterize the computational requirements of the software and the energy consumption of the CenceMe phone client. We validate the system through a user study where twenty two people, including undergraduates, graduates and faculty, used CenceMe continuously over a three week period in a campus town. From this user study we learn how the system performs in a production environment and what uses people find for a personal sensing system.