The prevalence of voice-related interaction and communication has raised concerns about privacy leakage and security. For example, millimeter-wave (mmWave) radio signals have been exploited as a potential attacker for acoustic eavesdropping. However, speaker variability and low-quality input pose significant challenges for the practical deployment of mmWave-based eavesdropping. In this paper, we proposeSPACE, an acoustic eavesdropping system to recover intelligible speech from low-quality mmWave signals, which can adapt to numerous different speakers and unseen ones.SPACEis a two-stage system that first reconstructs the spectrogram using a novelRadio TransUNetand then synthesizes the waveform through a neural vocoder. Specifically, to alleviate the negative effect of speaker variability, we introduce a speaker encoder to capture speaker features and a fusion network to condition the spectrogram reconstruction based on the extracted speaker characteristics. Further, to facilitate intelligible speech recovery from low-quality input, we design a Frequency Transformation Layer to exploit the correlation among all frequency harmonics and incorporate the neural vocoder to synthesize the speech waveform from the reconstructed spectrogram without using the contaminated phase. The experimental results show thatSPACEoutperforms existing mmWave-based approaches in scenarios with numerous different speakers and unseen speakers.

Inertial tracking is vital for robotic IoT and has gained popularity thanks to the ubiquity of low-cost inertial measurement units and deep learning-powered tracking algorithms. Existing works, however, have not fully utilized IMU measurements, particularly magnetometers, nor have they maximized the potential of deep learning to achieve the desired accuracy. To address these limitations, we introduceNeurIT, which elevates tracking accuracy to a new level.NeurITemploys a Time-Frequency Block-recurrent Transformer (TF-BRT) at its core, combining both RNN and Transformer to learn representative features in both time and frequency domains. To fully utilize IMU information, we strategically employ body-frame differentiation of magnetometers, considerably reducing the tracking error. We implementNeurITon a customized robotic platform and conduct evaluation in various indoor environments. Experimental results demonstrate thatNeurITachieves a mere 1-meter tracking error over a 300-meter distance. Notably, it significantly outperforms state-of-the-art baselines by 48.21% on unseen data. Moreover,NeurITdemonstrates robustness in large urban complexes and performs comparably to the visual-inertial approach (Tango Phone) in vision-favored conditions while surpassing it in feature-sparse settings. We believeNeurITtakes an important step forward toward practical neural inertial tracking for ubiquitous and scalable tracking of robotic things.NeurITis open-sourced here:https://github.com/aiot-lab/NeurIT.

Facial Expression Recognition (FER) is crucial for understanding human emotions, with applications spanning from mental health assessment to marketing recommendation systems. However, existing camera-based methods raise privacy concerns, while RF-based approaches suffer from limited environmental generalizability and high cost. In this work, we propose ToFace, a FER system leveraging a low-cost (4.8$) Direct Time-of-Flight (DToF) sensor that has been available on commodity smartphones. This sensor provides an extremely low-resolution 8 × 8 depth map and a clear Field of View (FoV), significantly mitigating privacy concerns while avoiding the impact of ambient objects. Despite the benefits, the low-resolution depth map introduces significant challenges for precise expression recognition due to limited facial structure information. We first develop a physical model to extract additional spatial information from the intermediate sensor output, i.e., the transient histograms. We then propose a physics-integrated neural network to reconstruct a facial structure map comprising both depth and orientation for accurate expression recognition. We conduct real-world experiments with 12 users and compare our model with several baselines. The results demonstrate that ToFace achieves the highest recognition accuracy of 75%.

Inertial tracking is vital for autonomous robots and has gained popularity with the ubiquity of low-cost Inertial Measurement Units (IMUs) and deep learning-powered tracking algorithms. Existing works, however, have not fully utilized IMU measurements, particularly magnetometers, nor maximized the potential of deep learning to achieve the desired accuracy. To bridge the gap, we introduce NeurIT, which employs a Time-Frequency Block-recurrent Transformer (TF-BRT) at its core, combining RNN and Transformer to learn both time-frequency representative features. To fully utilize IMU information, we strategically employ differentiation of body-frame magnetometers for orientation calibration in a sensor fusion manner. Experiments conducted in diverse environments show that NeurIT maintains a mere 1 -meter tracking error over a 300 - meter distance, surpassing state-of-the-art baselines by 48.21 % on unseen data. NeurIT also performs comparably to the visual-inertial approach (Tango Phone) in vision-favored conditions and surpasses it in plain environments. We share the code and data to promote further research: https://github.com/aiot-lab/NeurIT.

We introduce OctoNet, a large-scale, multi-modal, multi-view human activity dataset designed to advance human activity understanding and multi-modal learning. OctoNet comprises 12 heterogeneous modalities (including RGB, depth, thermal cameras, infrared arrays, audio, millimeter-wave radar, Wi-Fi, IMU, and more) recorded from 41 participants under multi-view sensor setups, yielding over 67.72M synchronized frames. The data encompass 62 daily activities spanning structured routines, freestyle behaviors, human-environment interaction, healthcare tasks, etc. Critically, all modalities are annotated by high-fidelity 3D pose labels captured via a professional motion-capture system, allowing precise alignment and rich supervision across sensors and views. OctoNet is one of the most comprehensive datasets of its kind, enabling a wide range of learning tasks such as human activity recognition, 3D pose estimation, multi-modal fusion, cross-modal supervision, and sensor foundation models. Extensive experiments have been conducted to demonstrate the sensing capacity using various baselines. OctoNet offers a unique and unified testbed for developing and benchmarking generalizable, robust models for human-centric perceptual AI.

Localization is critical to numerous applications. The performance of classical localization protocols is limited by the specific form of distance information and suffer from considerable ranging errors. This paper foresees a new opportunity by utilizing the exceptional property of entangled quantum states to measure a linear combination of target-anchor distances. Specifically, we consider localization with quantum-based TDoA measurements. Classical TDoA ranging takes the difference of two separate measurements. Instead, quantum ranging allows TDoA estimation within a single measurement, thereby reducing the ranging errors. Numerical simulations demonstrate that the new quantum-based localization significantly outperforms conventional algorithms based on classical ranging, with over 50% gains on average.

Predicting blood glucose levels is fundamental for precise primary care of type-1 diabetes (T1D) patients. However, it is challenging to predict glucose levels accurately, not to mention the early alarm of adverse events (hyperglycemia and hypoglycemia), namely the minority class. In this paper, we propose BG-BERT, a novel self-supervised learning framework for blood glucose level prediction. In particular, BG-BERT incorporates masked autoencoder to capture rich contextual information of blood glucose records for accurate prediction. More specifically, SMOTE data augmentation and shrinkage loss are employed to effectively handle adverse events without discrimination. We evaluate BG-BERT on two benchmark datasets against two state-of-the-art base-line models. The experimental results highlight the significant improvements achieved by BG-BERT in glucose level prediction accuracy (measured by RMSE) and sensitivity to adverse events, with average lifting ratios of 9.5% and 44.9%, respectively.

Sensing and communication are at the core of the Internet of Things, which usually function independently. For example, a smartphone can communicate over Wi-Fi or cellular networks while continuously acquiring sensory data from the environment through various sensors. This paper presents a novel framework, MuSAC (Mutualistic Sensing and Commu-nication), which seamlessly integrates the collection of sensory data with existing communication systems, without adding any extra communication overhead. The framework leverages the mutualistic relationship between specific communication data and sensory data to effectively crowdsource heterogeneous sensory data without harming communication performance in practical distributed systems. To embed massive sensory data into the current transmission of communication data, MuSAC presents novel neural networks to distill universal features from the raw data for compression at the sender side and then extract invariant features on the server side. By doing so, MuSAC eliminates additional communication costs for sensory data collection while also mitigating privacy concerns and data heterogeneity in crowd-sensing. Our real-world experimental validation in Wi-Fi and cellular Massive MIMO communication scenarios demonstrates the effectiveness of the MuSAC framework, shedding light on efficient mobile crowdsensing for massive IoT data collection.

Along with AIGC shines in CV and NLP, its potential in the wireless domain has also emerged in recent years. Yet, existing RF-oriented generative solutions are ill-suited for generating high-quality, time-series RF data due to limited representation capabilities. In this work, inspired by the stellar achievements of the diffusion model in CV and NLP, we adapt it to the RF domain and propose RF-Diffusion. To accommodate the unique characteristics of RF signals, we first introduce a novel Time-Frequency Diffusion theory to enhance the original diffusion model, enabling it to tap into the information within the time, frequency, and complex-valued domains of RF signals. On this basis, we propose a Hierarchical Diffusion Transformer to translate the theory into a practical generative DNN through elaborated design spanning network architecture, functional block, and complex-valued operator, making RF-Diffusion a versatile solution to generate diverse, high-quality, and time-series RF data. Performance comparison with three prevalent generative models demonstrates the RF-Diffusion's superior performance in synthesizing Wi-Fi and FMCW signals. We also showcase the versatility of RF-Diffusion in boosting Wi-Fi sensing systems and performing channel estimation in 5G networks.

Human sensing has gained increasing attention in various applications. Among the available technologies, visual images offer high accuracy, while sensing on the RF spectrum preserves privacy, creating a conflict between imaging resolution and privacy preservation. In this paper, we explore thermal array sensors as an emerging modality that strikes an excellent resolution-privacy balance for ubiquitous sensing. To this end, we present TADAR, the first multi-user Thermal Array-based Detection and Ranging system that estimates the inherently missing range information, extending thermal array outputs from 2D thermal pixels to 3D depths and empowering them as a promising modality for ubiquitous privacy-preserving human sensing. We prototype TADAR using a single commodity thermal array sensor and conduct extensive experiments in different indoor environments. Our results show that TADAR achieves a mean F1 score of 88.8% for multi-user detection and a mean accuracy of 32.0 cm for multi-user ranging, which further improves to 20.1 cm for targets located within 3 m. We conduct two case studies on fall detection and occupancy estimation to showcase the potential applications of TADAR. We hope TADAR will inspire the vast community to explore new directions of thermal array sensing, beyond wireless and acoustic sensing. TADAR is open-sourced on GitHub: https://github.com/aiot-lab/TADAR.

In this demo, we present an all-in-one real-time system for breathing monitoring and presence detection using statistical acoustic sensing. By applying Auto-Correlation Function (ACF) to the Channel Frequency Response (CFR), our system captures both motion statistics and breathing rates. We devise novel weight combining schemes to enhance the SNR of the weak sensing signals. We then enable human presence detection by integrating both motion statistics and breathing rate as vital indicators. Our system operates using a single microphone without relying on a bulky microphone array. Our demo functions in real-time and supports any device that is equipped with a commodity microphone and speaker. Our demo can be accessed through https://youtu.be/JUB6yQ1rQUo

Addressing the pivotal challenge of discerning human and nonhuman activities in smart environments, in this demo, we present a system utilizing commercial WiFi transceivers for precise human and non-human motion differentiation through the walls. This system effectively filters non-human interference in smart home systems by extracting physically and statistically explainable features from ubiquitous WiFi signals. It passively recognizes moving subjects in real time without constraining their movement, even in complex environments. Tailored for edge computing, it ensures minimal resource consumption and generalizes well across various settings. Our long-term field tests confirm a high accuracy rate of 97.34% and a low false alarm rate of 1.75%, underscoring its robustness and readiness for practical deployment. Please find the companion video with the URL: https://youtu.be/6xkJZ_VvL9Q.

Voice interfaces have become one of the most ubiquitous human–computer interaction methods in recent years. Voice activity detection (VAD) is typically the first building block of a complex voice interface, often relying on audio signals. Acoustics-based VAD systems do not perform well in noisy and interference-prone environments. Smart assistants mitigate this problem by using a dictionary-based detection system. However, this approach is limited in its applicability. For instance, users may still need to manually mute and unmute their microphones during online meetings to prevent detection of interfering users, and speech leakage. In order to automate voice detection in challenging environments without these limitations, we propose RadioVAD, a noise and interference-resilient VAD system that uses radio modality, which is already available in various smartphones and home assistants. RadioVAD works by detecting possible human presence in the Field of View of the device, extracting the vocal fold’s vibration signal from the target speaker, and utilizing a time-domain neural network on raw radio signals to detect voice activity. Extensive experiments reveal that RadioVAD can detect voice activity in challenging environments with high accuracy and outperforms audio-based VAD when the audio signal has signal-to-noise ratio below 5 dB. Furthermore, RadioVAD reduces false alarm rate in interference-prone environments by 52%–72%, bringing significant improvements to VAD task. RadioVAD lays the foundation for future voice interfaces utilizing radio modality.

Indoor intelligent perception systems have gained significant attention in recent years. However, accurately detecting human presence can be challenging in the presence of non-human subjects such as pets, robots, and electrical appliances, limiting the practicality of these systems for widespread use. In this paper, we propose a novel system (“WI-MOID") that passively and unobtrusively distinguishes moving human and various non-human subjects using a single pair of commodity WiFi transceivers, without requiring any device on the subjects or restricting their movements. WI-MOID leverages a novel statistical electromagnetic wave theory-based multipath model to detect moving subjects, extracts physically and statistically explainable features of their motion, and accurately differentiates human and various non-human movements through walls, even in complex environments. In addition, WI-MOID is suitable for edge devices, requiring minimal computing resources and storage, and is environment-independent, making it easy to deploy in new environments with minimum effort. We evaluate the performance of WI-MOID in five distinct buildings with various moving subjects, including pets, vacuum robots, humans, and fans, and the results demonstrate that it achieves 97.34% accuracy and 1.75% false alarm rate for identification of human and non-human motion, and 95.98% accuracy in unseen environments without model tuning, demonstrating its robustness for ubiquitous use.

Leaf wetness detection is one of the key technologies for preventing plant diseases in agriculture. In this poster, we propose mmLeaf, leveraging a commercial off-the-shelf millimeter-wave (mmWave) radar to detect actual leaf wetness in diverse environments and lighting conditions. mmLeaf captures mmWave signals reflected by monitored leaves with a two-dimensional (2D) scanning system. Then, we use a multiple-input multiple-output (MIMO) array and synthetic aperture radar (SAR) to reconstruct the signal distribution of different planes of the leaves. A deep learning model takes the fused signal distribution as inputs to classify the leaf wetness. We implement mmLeaf using a frequency-modulated continuous-wave (FMCW) radar and evaluate its performance with a potted plant indoors. By exploring the use of mmWave signals, mmLeaf delivers an end-to-end detection framework that achieves up to 90% accuracy in classifying leaf wetness under different distances.

Indoor target detection through motion sensing based on Wi-Fi signals has gained much attention recently. However, most of the existing motion detection approaches can only detect motion in a large coverage area without knowing the distance of the target motion from the transmitter (Tx)/receiver (Rx). Passive positioning techniques can provide the location of a target, which, however, requires high deployment efforts without robust performance. In this article, we present a novel technique for detecting motion in proximity by exploring the physics behind the indoor radio frequency (RF) multipath propagation. We discover that motion in the proximity of the Rx/Tx produces distinct time dispersion over the radio channel at the Rx/Tx side. By exploring two novel metrics and linking them with the distance of the motions to antennas, we are able to precisely distinguish motions in nearby proximity from the motions far away. Extensive experiments in various real-world scenarios demonstrate that the proposed scheme can achieve true positive rates (TPRs) greater than 95% and 99% in distance-based and room-level proximity detection, respectively, while maintaining the corresponding false positive rates (FPRs) less than 5% and 0.5%. The detection delays for a detection distance of 2 m are within 0.6 s, which verifies the responsiveness of the proposed scheme.

Voice interfaces have become an integral part of our lives with the proliferation of smart devices. Today, Internet of Things devices mainly rely on microphones to sense sound. Microphones, however, have fundamental limitations, such as weak source separation, limited range in the presence of acoustic insulation, and being prone to multiple side-channel attacks. In this article, we propose RadioMic, a radio-based sound sensing system to mitigate these issues and enrich sound applications. RadioMic constructs sound based on tiny vibrations on active sources (e.g., a speaker diaphragm) or object surfaces (e.g., paper bag), and can work through walls, even a soundproof one. To convert the extremely weak sound vibration in the radio signals into sound signals, RadioMic introduces radio acoustics, and presents training-free approaches for robust sound detection and high-fidelity sound recovery. It then exploits a neural network to further enhance the recovered sound by expanding the recoverable frequencies and reducing the noises. RadioMic translates massive online audios to synthesized data to train the network and, thus, minimizes the need for radio-frequency (RF) data. We thoroughly evaluate different components of RadioMic under different scenarios using a commodity mmWave radar. The results show RadioMic outperforms the state-of-the-art systems significantly. We believe RadioMic provides new horizons for sound sensing and inspires attractive sensing capabilities of mmWave sensing devices.

Indoor-location-based services rely on indoor maps, which are yet widely available despite numerous efforts from the industry. Existing solutions employ costly hardware (e.g., lidar) to achieve accurate mapping of indoor environments, or resort to crowdsourcing for floor plan generation at the cost of precision due to inaccurate inertial sensing. In this article, we leverage a new opportunity enabled by recent advances in RF-based inertial tracking that achieves centimeter accuracy. We present EZMAP, a high-accuracy, low-cost floor plan construction system that fuses RF and inertial sensing. EZMAP combines the fine-grained yet local information from RF tracking with the coarse grained but global contexts from inertial sensing (e.g., magnetic field strength), which together makes for an accurate map. Our system employs a robot for trajectory collection and requires only a single access point to be arbitrarily installed in the space, both of which are widely available nowadays. Furthermore, it can generate a map even only a small amount of data is available, allowing it to scale for different buildings, such as malls, office buildings, and homes with little cost. We validate the performance using a Dji RoboMaster S1 robot with commodity WiFi in three different buildings. The results show that our system can efficiently generate faithful maps for the targeted areas. With the ubiquity of the WiFi infrastructure and the rise of home robots, we believe our approach will pave the way for pervasive indoor maps services.

Wearable devices have been important components for ubiquitous computing. However, text input remains challenging on wearables due to the lack of a physical keyboard. In this paper, we propose a novel ring-shaped virtual keyboard system named RingVKB for convenient text input using low-cost IMUs available on any wearables. At the core of RingVKB are two novel designs: 1) A circular keyboard layout with 12 equal sectors, which assembles all common keys on classical keyboards while allowing users to type with only one finger effectively, and 2) an error control algorithm that calculates the relative displacement of keystrokes from the noisy IMU sensor data. The two components, coupled together, enable high-accuracy and efficient text input for ubiquitous scenarios.We implement RingVKB using a small device consisting of a microcontroller and a MEMS sensor, which can be attached to the user's index finger.Experimental results show that RingVKB can effectively improve the relative displacement estimation accuracy, and achieves an overall keystroke recognition accuracy of 93% for 25 key positions. A user study also shows that RingVKB is easy to learn and use. Using only low-cost IMU sensors, RingVKB provides a virtual keyboard solution that can be widely adopted on wearables.

Speech enhancement and separation have been a long-standing problem, especially with the recent advances using a single microphone. Although microphones perform well in constrained settings, their performance for speech separation decreases in noisy conditions. In this work, we proposeRadioSES, an audioradio speech enhancement and separation system that overcomes inherent problems in audio-only systems. By fusing a complementary radio modality,RadioSEScan estimate the number of speakers, solve the source association problem, separate and enhance noisy mixture speeches, and improve both intelligibility and perceptual quality. We perform millimeter-wave sensing to detect and localize speakers and introduce an audioradio deep learning framework to fuse the separate radio features with the mixed audio features. Extensive experiments using commercial off-the-shelf devices show thatRadioSESoutperforms a variety of state-of-the-art baselines, with consistent performance gains in different environmental settings. Similar to the audiovisual methods,RadioSESprovides significant performance improvements (e.g. 3 dB gains in SiSDR, when compared with the corresponding audio-only method), along with the benefits of lower computational complexity and better privacy preservation.

Speech enhancement and separation have been a long-standing problem with recent advances using a single microphone. With the help of video modality, improvements have been shown for these tasks. In this work, we explore a multimodal approach using mmWave radio devices, as these devices can measure vocal folds vibration. Thorough data collection and extensive experiments with two different neural networks indicate that radio modality can bring significant improvements in speech enhancement and separation.

Heat-stroke and death of children being left alone in a parked car has attracted more and more attentions. As a result, car manufactures start to reward solutions for in-car Child Presence Detection (CPD) system to save lives recently. However, most of the existing works rely on dedicated sensors and only achieve limited accuracy and coverage. This paper presents the first-of-its-kind intelligent CPD system using commodity Wi-Fi. Based on a statistical electromagnetic wave model to fully leverage the information in all the multi-path components, the proposed CPD system mainly consists of a motion target detector to detect a child in awake/motion status, a stationary target detector to detect a sleeping child by extracting breathing rate information, and a transition target detector based on a Naive Bayes Classifier using multipath profiles as features. We build a real-time testbed and show through extensive experiments that the proposed system can achieve ≥ 99.34% detection rate and ≤ 4.38% false alarm rate, regardless of the location and motion status of a child. Built upon 2.4/5GHz Wi-Fi, the proposed system can integrate with the existing in-car Wi-Fi system with no additional hardware and calls for low CPU and memory consumptions, thus promising a practical candidate for CPD applications.

Indoor maps are essential to indoor location-based services, but are not widely accessible despite considerable efforts from the industry. Existing solutions employ costly hardware to achieve accurate mapping, or resort to laborious crowdsourcing methods, which may suffer from low accuracy due to inaccurate inertial sensing. In this paper, we leverage advanced RF-based inertial tracking and present a high-accuracy and low-cost floor plan reconstruction system. The proposed system combines local information from RF tracking with the global contexts from inertial sensing (e.g., magnetic field strength) for an accurate map. We validate the performance with commodity WiFi in an office building, which shows that the proposed system can efficiently generate faithful maps for a targeted area. With the ubiquitous deployment of WiFi devices, our approach will make a wide range of indoor location-based systems possible.

Critical robotic tasks such as rescue and disaster response are more prevalently leveraging ML (Machine Learning) models deployed on a team of wireless robots, on which data parallel (DP) training over Internet of Things of these robots (robotic IoT) can harness the distributed hardware resources to adapt their models to changing environments as soon as possible. Unfortunately, due to the need for DP synchronization across all robots, the instability in wireless networks (i.e., fluctuating bandwidth due to occlusion and varying communication distance) often leads to severe stall of robots, which affects the training accuracy within a tight time budget and wastes energy stalling. Existing methods to cope with the instability of datacenter networks are incapable of handling such straggler effect. That is because they are conducting model-granulated transmission scheduling, which is much more coarse-grained than the granularity of transient network instability in real-world robotic IoT networks, making a previously reached schedule mismatch with the varying bandwidth during transmission. We present ROG, the first ROw-Granulated distributed training system optimized for ML training over unstable wireless networks. ROG confines the granularity of transmission and synchronization to each row of a layer’s parameters and schedules the transmission of each row adaptively to the fluctuating bandwidth. In this way the ML training process can update partial and the most important gradients of a stale robot to avoid triggering stalls, while provably guaranteeing convergence. The evaluation shows that, given the same training time, ROG achieved about 4.9%~6.5% training accuracy gain compared with the baselines and saved 20.4%~50.7% of the energy to achieve the same training accuracy.

After years of boom, drones and their applications are now entering indoors. Six-degree-of-freedom (6-DoF) pose tracking is the core of drone flight control, but existing solutions cannot be directly applied to indoor scenarios due to insufficient accuracy, low robustness to adverse texture and light conditions, and signal obstruction in indoor scenarios. To overcome the above limitations, we propose Wi-Drone, a Wi-Fi standalone 6-DoF tracking system for indoor drone flight control. Wi-Drone takes full advantage of both exte-roceptive and proprioceptive measurements of Wi-Fi to estimate the drone's absolute pose and relative motion, and fuse them in a tight-coupling manner to achieve their complementary benefits. We implement Wi-Drone and integrate it into a flight control system. The evaluation results show that Wi-Drone achieves a real-time performance with the average location accuracy of 26.1 cm and the rotation accuracy of 3.8°, which demonstrates its competency of flight control, compared to visual-inertial-based flight control. Such results also outperform existing Wi-Fi-based tracking solutions in terms of both dimensionality and accuracy.

Indoor moving direction and rotation angle measurements are crucial to many ubiquitous mobile computing applications. Most of the state-of-the-art approaches rely on inertial sensors, e.g., accelerometers, gyroscopes, and magnetometers, which suffer from severe accumulative errors or accuracy degradation indoors. This article presents an RF-based direction estimation method, which utilizes the off-the-shelf commodity WiFi devices for accurate moving direction and in-place rotation angle estimation. The proposed approach employs a novel 2-D antenna array and leverages the spatial decay property of the time-reversal resonating strength. First, the moving speeds along different directions, specified by the 2-D array, are derived using virtual antenna alignment. The precise estimation of the device’s moving direction is then achieved by combining the obtained velocity information and the a prior knowledge of the array’s geometry layout. Experiments in a multipath-rich indoor environment have shown that the median error for moving direction estimation is 6.9°, which outperforms the accelerometer counterpart. The results also verify the good accuracy of in-place rotation angle estimation without any accumulative error, which beats the gyroscope in long-term tests. Because the proposed approach can achieve high accuracy without accumulative drifts, it is a promising candidate solution to applications that require accurate direction information.

Keyboard acts as one of the most commonly used mediums for human–computer interaction. Today, massive Internet-of-Things (IoT) devices are designed without a physical keyboard as they go tiny, but are almost all equipped with a wireless module for networks. In this work, we aim to enable a universal virtual keyboard using wireless signals, which would allow a typing interface for tiny IoT devices or serve as a portable alternative to the unwieldy physical keyboards. To this end, we presentmmKey, the first universal virtual keyboard system using a single millimeter-wave (mmWave) radio. By leveraging the unique advantages of mmWave signals,mmKeyconverts any flat surface, with a printed paper keyboard, into an effective typing medium.mmKeyenables concurrent keystrokes and supports multiple keyboard layouts (e.g., computer keyboard, piano keyboard, or phone keypad). We design a novel signal processing pipeline to detect, segment and separate, and finally, recognize keystrokes.mmKeydoes not need any training except for a minimal one-time effort of only three key-presses for keyboard calibration upon the initial setup. We prototypemmKeyusing a commodity 802.11ad/ay chipset, customized to support radar-like operations, and evaluate it with different keyboard layouts under various settings. Experimental results with ten participants demonstrate a keystroke recognition accuracy of >95% for single-key case and >90% for multikey scenario, which leads to a word recognition accuracy of >97%.

Fall is recognized as one of the most frequent accidents among elderly people. Many solutions, either wearable or noncontact, have been proposed for fall detection (FD) recently. Among them, WiFi-based noncontact approaches are gaining popularity due to the ubiquity and noninvasiveness. The existing works, however, usually rely on labor-intensive and time-consuming training before it can achieve a reasonable performance. In addition, the trained models often contain environment-specific information and, thus, cannot be generalized well for new environments. In this article, we propose DeFall, a WiFi-based passive FD system that is independent of the environment and free of prior training in new environments. Unlike previous works, our key insight is to probe the physiological features inherently associated with human falls, i.e., the distinctive patterns of speed and acceleration during a fall. DeFall consists of an offline template-generating stage and an online decision-making stage, both taking the speed estimates as input. In the offline stage, augmented dynamic time-warping (DTW) algorithms are performed to generate a representative template of the speed and acceleration patterns for a typical human fall. In the online phase, we compare the patterns of the real-time speed/acceleration estimates against the template to detect falls. To evaluate the performance of DeFall, we built a prototype using commercial WiFi devices and conducted experiments under different settings. The results demonstrate that DeFall achieves a detection rate above 95% with a false alarm rate lower than 1.50% under both line-of-sight (LOS) and non-LOS (NLOS) scenarios with one single pair of transceivers. Extensive comparison study verifies that DeFall can be generalized well to new environments without any new training.

Monitoring and identifying gait has recently emerged as a promising solution candidate for unobtrusive human recognition. In order to enable ubiquitous and reliable application, a gait recognition system must be robust to environment changes and easy to use without requiring too much user cooperation and recalibration, while maintaining high accuracy, which is often not satisfied in conventional approaches. In this article, we present$\boldsymbol {GaitCube}$, a high-accuracy gait recognition system with the minimal training requirement using a single commodity millimeter-wave (mmWave) radio. To reduce the training overhead, we proposegait data cube, a novel 3-D joint-feature representation of micro-Doppler and micro-range signatures over time that can comprehensively embody the physical relevant features of one’s gait. With a pipeline of signal processing,$\boldsymbol {GaitCube}$can automatically detect and segment human walking and effectively extract thegait data cubes. We implement and evaluate$\boldsymbol {GaitCube}$through experiments conducted at six different locations in a typical indoor space with ten subjects over a month, resulting in >50000 gait instances. The results show that$\boldsymbol {GaitCube}$achieves an accuracy of 96.1% with a single gait cycle using one receive antenna, and the accuracy increases to 98.3% when combining all the receive antennas. Further, it achieves an average recognition accuracy of 79.1% for testing over different times and unseen locations by using only 2 min of training data collected in a single location, enabling a practical and ubiquitous gait-based identification.

As automobiles have become an essential part to facilitate our daily life, advanced driver assistance systems (ADASs) have been gaining more and more interest in assisting drivers to enhance both safety and convenience. To respond timely in case of an emergency, ADAS needs to keep track of the driver’s health/consciousness, which is generally achieved by monitoring the driver’s vital signs, including respiration rate (RR), heart rate (HR), and heart rate variability (HRV). However, most of the state-of-art solutions need to assume that the human is stationary, which does not hold in practical driving scenarios. To tackle the problem, we propose a novel system, which can estimate driver’s RR, HR, and interbeat intervals (IBIs) in the presence of driver’s motion artifacts using commercial millimeter-wave (mmWave) radio. The system consists of two key components. First, to extract the reflection signals containing vital signals, the motion artifacts are first removed by a novel motion compensation module, followed by the periodicity check to identify the components with vital signals. Second, the respiration and heartbeat signals are reconstructed by jointly optimizing the decomposition of all the extracted compound vital signals over different range-azimuth bins. We evaluate the system performance in a real driving environment and investigate the impact of different parameters, including the device locations, pavement conditions, and motion types. The experimental results show that the proposed system can achieve a median error of 0.16 respiration per minute (RPM), 0.82 beat per minute (BPM), and 46 ms for RR, HR, and IBI estimations, corresponding to the relative accuracy of 99.17%, 98.94%, and 94.11%, respectively.

Child presence detection (CPD) is becoming a regulatory requirement for car manufacturers to save children’s lives when they are left alone in unattended vehicles. However, most of the existing solutions require dedicated devices and suffer from limited accuracy and coverage. In this article, we build WiCPD, the first-of-its-kind in-car CPD system using commodity Wi-Fi, which can cover the entire interior of a car with no blind spot. First, we introduce a statistical electromagnetic model which accounts for the impact of motion on all the multipaths inside a car, followed by a motion statistics metric indicating the ambient motion intensity and a signal-to-noise-ratio (SNR) boosting scheme to extract the minute chest movement. Then, we design a unified CPD framework consisting of three target detector modules, including a motion target detector to detect a child in motion/awake, a stationary target detector to detect a stationary/sleeping child, and a transition target detector to detect a sleeping child with sporadic motion who is missed by both the motion and stationary target detectors. We implement a real-time WiCPD system by using commercial Wi-Fi chipsets, deploy it over 20 different cars, and collect data for multiple children aging from 4 to 50 months. The results show that WiCPD can achieve 100% detection rate within 8 s when the child is awake/in-motion and 96.56% detection rate within 20 s for a static/sleeping child. Extensive experiments also demonstrate that WiCPD can be easily deployed in minutes without calibration and enjoys very low CPU and memory consumption, thus promising a practical candidate for CPD applications.

With the development of signal processing technology, the ubiquitous Wi-Fi devices open an unprecedented opportunity to solve the challenging human gesture recognition problem by learning motion representations from wireless signals. Wi-Fi-based gesture recognition systems, although yield good performance on specific data domains, are still practically difficult to be used without explicit adaptation efforts to new domains. Various pioneering approaches have been proposed to resolve this contradiction but extra training efforts are still necessary for either data collection or model re-training when new data domains appear. To advance cross-domain recognition and achieve fully zero-effort recognition, we propose Widar3.0, a Wi-Fi-based zero-effort cross-domain gesture recognition system. The key insight of Widar3.0 is to derive and extract domain-independent features of human gestures at the lower signal level, which represent unique kinetic characteristics of gestures and are irrespective of domains. On this basis, we develop a one-fits-all general model that requires only one-time training but can adapt to different data domains. Experiments on various domain factors (i.e. environments, locations, and orientations of persons) demonstrate the accuracy of 92.7% for in-domain recognition and 82.6%-92.4% for cross-domain recognition without model re-training, outperforming the state-of-the-art solutions.

Indoor localization has gained increasing attention in the era of the Internet of Things. Among various technologies, WiFi fingerprint-based localization has become a mainstream solution. However, RSS fingerprints suffer from critical drawbacks of spatial ambiguity and temporal instability that root in multipath effects and environmental dynamics, which degrade the performance of these systems and therefore impede their wide deployment in the real world. Pioneering works overcome these limitations at the costs of ubiquity as they mostly resort to additional information or extra user constraints. In this article, we present the design and implementation of ViViPlus, an indoor localization system purely based on WiFi fingerprints, which jointly mitigates spatial ambiguity and temporal instability and derives reliable performance without impairing the ubiquity. The key idea is to embrace the spatial awareness of RSS values in a novel form of RSS Spatial Gradient (RSG) matrix for enhanced WiFi fingerprints. We devise techniques for the representation, construction, and localization of the proposed fingerprint form and integrate them all in a practical system. Extensive experiments across 7 months in different environments demonstrate that ViViPlus significantly improves the accuracy in localization scenarios by about 30% to 50% compared with the state-of-the-art approaches.

With the proliferation of smart devices, voice interfaces have become an integral part of our lives, which typically senses sound by microphones through converting the changes in air pressure into electrical signals. Sound sensing through another modality can enable various sensing applications in the absence of a microphone. In fact, environmental sound creates tiny vibrations on object surfaces, which could be captured by radio signals. In this work, we model the vibration on object surfaces due to sound for mmWave devices. We propose a method for the recovery of sound and conduct experiments with various materials to investigate the feasibility of sound reconstruction. We further evaluate the effect of distance and placement to understand the practical limits on the sound reconstruction. The results show that, by using a commodity off-the-shelf radar, it is possible to capture a significant amount of sound from the environment.

Heart Rate Variability (HRV), which measures the fluctuation of heartbeat intervals, has been considered as an important indicator for general health evaluation. In this paper, we present mmHRV, a contact-free HRV monitoring system using commercial millimeter-wave (mmWave) radio. We devise a heartbeat signal extractor, which can optimize the decomposition of the phase of the channel information modulated by the chest movement, and thus estimate the heartbeat signal. The exact time of heartbeats is estimated by finding the peak location of the heartbeat signal while the Inter-Beat Intervals (IBIs) can be further derived for evaluating the HRV metrics. Experimental results show that mmHRV can measure the HRV accurately with 3.68ms average error of mean IBI (w.r.t. 99.49% accuracy) based on the experiments over 10 participants.

In the era of pervasively connected and sensed Internet of Things, many of our interactions with machines have been shifted from conventional computer keyboards and mouses to hand gestures and writing in the air. While gesture recognition and handwriting recognition have been well studied, many new methods are being investigated to enable pervasive handwriting tracking. Most of the existing handwriting tracking systems either require cameras and handheld sensors or involve dedicated hardware restricting user convenience and the scale of usage. In this article, we present mmWrite, the first high-precision passive handwriting tracking system using a single commodity millimeter-wave (mmWave) radio. Leveraging the short wavelength and large bandwidth of 60-GHz signals and the radar-like capabilities enabled by the large phased array, mmWrite transforms any flat region into an interactive writing surface that supports handwriting tracking at millimeter accuracy. MmWrite employs an end-to-end pipeline of signal processing to enhance the range and spatial resolution limited by the hardware, boost the coverage, and suppress interference from backgrounds and irrelevant objects. We implement and evaluate mmWrite on a commodity 60-GHz device. The experimental results show that mmWrite can track a finger/pen with a median error of 2.8 mm and thus can reproduce handwritten characters as small as 1 cm × 1 cm, with a coverage of up to 8 m2supported. With minimal infrastructure needed, mmWrite promises ubiquitous handwriting tracking for new applications in the field of human-computer interactions.

The continuous development of 802.11ad technology provides new opportunities in wireless sensing. In this work, we propose ViMo, a calibration-free remote vital sign monitoring system that can detect stationary/nonstationary users and estimate the respiration rates (RRs) as well as heart rates (HRs) built upon a commercial 60-GHz WiFi. The design of ViMo consists of two key components. First, we design an adaptive object detector that can identify static objects, stationary human subjects, and human in motion without any calibration. Second, we devise a robust HR estimator, which eliminates the respiration signal from the phase of the channel impulse response (CIR) to remove the interference of the harmonics from breathing and adopts dynamic programming (DP) to resist the random measurement noise. The influence of different settings, including the distance between a human and the device, user orientation and incidental angle, blockage material, body movement, and conditions of multiuser separation is investigated by extensive experiments. The experimental results show that ViMo monitors user’s vital signs accurately, with a median error of 0.19 and 0.92 breaths per minute (BPM), respectively, for RR and HR estimation.

Heart rate variability (HRV), which measures the fluctuation of heartbeat intervals, has been considered as an important indicator for general health evaluation. To alleviate the user burden and explore the usability for long-term health monitoring, noncontact methods for HRV monitoring have drawn tremendous attention. In this article, we present mmHRV, the first contact-free multiuser HRV monitoring system using commercial millimeter-wave (mmWave) radio. The design of mmHRV consists of two key components. First, we develop a calibration-free target detector to identify each user’s location. Second, a heartbeat signal extractor is devised, which can optimize the decomposition of the phase of the channel information modulated by the chest movement and, thus, estimate the heartbeat signal. The exact time of heartbeats is estimated by finding the peak location of the heartbeat signal while the interbeat intervals (IBIs) can be further derived for evaluating the HRV metrics of each target. We evaluate the system performance and the impact of different settings, including the distance between human and the device, user orientation, incidental angle, and blockage. Experimental results show that mmHRV can measure the HRV accurately with a median IBI estimation error of 28 ms (with respect to 96.16% accuracy). In addition, the root-mean-square error (RMSE) measured in the nonline-of-sight (NLOS) scenarios is 31.71 ms based on the experiments with 11 participants. The performance of the multiuser scenario is slightly degraded compared with the single-user case; however, the median error of the 3-user case is within 52 ms for all three tested locations.

RF imaging is a dream that has been pursued for years yet not achieved in the evolving wireless sensing. The existing solutions on WiFi bands, however, either require specialized hardware with large antenna arrays or suffer from poor resolution due to fundamental limits in bandwidth, the number of antennas, and the carrier frequency of 2.4 GHz/5 GHz WiFi. In this article, we observe a new opportunity in the increasingly popular 60-GHz WiFi, which overcomes such limits. We present mmEye, a super-resolution imaging system toward a millimeter-wave camera by reusing a single commodity 60-GHz WiFi radios. The key challenge arises from the extremely small aperture (antenna size), e.g., <; 2 cm, which physically limits the spatial resolution. mmEye's core contribution is a super-resolution imaging algorithm that breaks the resolution limits by leveraging all available information at both the transmitter and receiver sides. Based on the MUSIC algorithm, we devise a novel technique of joint transmitter smoothing, which jointly uses the transmit and receive arrays to boost the spatial resolution while not sacrificing the aperture of the antenna array. Built upon this core, we design and implement a functional system on commodity 60-GHz WiFi chipsets. We evaluate mmEye on different persons and objects under various settings. Results show that it achieves a median silhouette (shape) difference of 27.2% and a median boundary keypoint precision of 7.6 cm, and it can image a person even through a thin drywall. The visual results show that the imaging quality is close to that of commercial products like Kinect, making for the first-time super-resolution imaging available on the commodity 60-GHz WiFi devices.

Interests in monitoring and recognizing gait have surged significantly over the past decades. Traditional approaches rely on camera array, floor sensors (e.g., pressure mats), or wearables (e.g., accelerometers), none of which are suitable for continuous and ubiquitous everyday use. In this article, we present GaitWay, the first system that monitors and recognizes an individual's gait through the walls via wireless radios. GaitWay passively and unobtrusively monitors an individual's gait speed by a single pair of commodity WiFi transceivers, without requiring the user to wear any device or walk on a restricted walkway. On this basis, GaitWay automatically identifies stable walking periods, extracts physically plausible and environmentally irrelevant speed features, and accordingly recognizes a subject's gait. Built upon a distinct rich-scattering multipath model, GaitWay can capture one's gait speed when one is $>$ >10 meters away behind the walls. We conduct experiments in a typical indoor space and perform eight sessions of data collection with 11 subjects across six months, resulting in $>$ >5,000 gait instances. The results show that GaitWay achieves a median 0.12 m/s and 90%tile 0.35 m/s error in speed estimation, with a mean error of 3.36 cm in stride lengths. Further, it achieves a verification rate of 90.4% and a recognition rate of 81.2% for five users and 69.8% for 11 users, confirming its comfort and accuracy for continuous and ubiquitous use.

We present the model, design, and implementation of SMARS, the first practical Sleep Monitoring system that exploits Ambient Radio Signals to recognize sleep stages and assess sleep quality. This will enable a future smart home that monitors daily sleep in a ubiquitous, non-invasive and contactless manner, without instrumenting the subject's body or the bed. The key enabler underlying SMARS is a statistical model that accounts for all reflecting and scattering multipaths, allowing highly accurate and instantaneous breathing estimation with best-ever performance achieved on commodity devices. On this basis, SMARS then recognizes different sleep stages, including wake, rapid eye movement (REM), and non-REM (NREM), which was previously only possible with dedicated hardware. We implement a real-time system on commercial WiFi chipsets and deploy it in 6 homes, resulting in 32 nights of data in total. Our results demonstrate that SMARS yields a median absolute error of 0.47 breaths per minute (BPM) and a 95 percent-tile error of only 2.92 BPM for breathing estimation, and detects breathing robustly even when a person is 10 meters away from the link, or behind a wall. SMARS achieves a sleep staging accuracy of 88 percent, outperforming the prevalent unobtrusive commodity solutions using bed sensor or UWB radar. The performance is also validated upon a public sleep dataset of 20 patients. By achieving promising results with merely a single commodity RF link, we believe that SMARS will set the stage for a practical in-home sleep monitoring solution.

Hidden screen-camera communication techniques emerge as a new paradigm that embeds data imperceptibly into regular videos while remaining unobtrusive to human viewers. Three key goals on imperceptible, high rate, and reliable communication are desirable but conflicting, and existing solutions usually made a trade-off among them. In this paper, we present the design and implementation of CHROMACODE, a screen-camera communication system that achieves all three goals simultaneously. In our design, we consider for the first time color space for perceptually uniform lightness modifications. On this basis, we design an outcome-based adaptive embedding scheme, which adapts to both pixel lightness and regional texture. Last, we propose a concatenated code scheme for robust coding and devise multiple techniques to overcome various screen-camera channel errors. Our prototype and experiments demonstrate that CHROMACODE achieves remarkable raw throughputs of >700 kbps, data goodputs of 120 kbps with BER of 0.05, and with fully imperceptible flicker for viewing proved by user study, which significantly outperforms previous works.

Existing indoor navigation solutions usually require pre-deployed comprehensive location services with precise indoor maps and, more importantly, all rely on dedicatedly installed or existing infrastructure. In this article, we present Pair-Navi, an infrastructure-free indoor navigation system that circumvents all these requirements by reusing a previous traveler’s (i.e., leader) trace experience to navigate future users (i.e., followers) in a Peer-to-Peer mode. Our system leverages the advances of visual simultaneous localization and mapping ( SLAM ) on commercial smartphones. Visual SLAM systems, however, are vulnerable to environmental dynamics in the precision and robustness and involve intensive computation that prohibits real-time applications. To combat environmental changes, we propose to cull non-rigid contexts and keep only the static and rigid contents in use. To enable real-time navigation on mobiles, we decouple and reorganize the highly coupled SLAM modules for leaders and followers. We implement Pair-Navi on commodity smartphones and validate its performance in three diverse buildings and two standard datasets (TUM and KITTI). Our results show that Pair-Navi achieves an immediate navigation success rate of 98.6%, which maintains as 83.4% even after 2 weeks since the leaders’ traces were collected, outperforming the state-of-the-art solutions by >50%. Being truly infrastructure-free, Pair-Navi sheds lights on practical indoor navigations for mobile users.

Extensive efforts have been devoted to human gesture recognition with radio frequency (RF) signals. However, their performance degrades when applied to novel gesture classes that have never been seen in the training set. To handle unseen gestures, extra efforts are inevitable in terms of data collection and model retraining. In this article, we present XGest, a cross-label gesture recognition system that can accurately recognize gestures outside of the predefined gesture set with zero extra training effort. The key insight of XGest is to build a knowledge transfer framework between different gesture datasets. Specifically, we design a novel deep neural network to embed gestures into a high-dimensional Euclidean space. Several techniques are designed to tackle the spatial resolution limits imposed by RF hardware and the specular reflection effect of RF signals in this model. We implement XGest on a commodity mmWave device, and extensive experiments have demonstrated the significant recognition performance.

Heading direction information is crucial to many ubiquitous computing applications. The main stream has been resorting to inertial sensors, such as accelerometer, gyroscope and magnetometer, which suffer from severe accumulative errors or large degradations indoors. In this paper, we utilize the radio frequency (RF) signals, received from the commercial off-the-shelf (COTS) WiFi devices, to accurately estimate the heading direction in indoor environments. Based on the time- reversal (TR) technique, we make use of the channel state information (CSI) and the geometry of the antenna array to design the proposed algorithm. A prototype is built using a single access point (AP), without knowing its location, and a two dimensional (2D) antenna array to validate the proposed method. Experiments, conducted in strong non-line-of-sight (NLOS) scenarios with rich multipaths indoors, have shown that the median error for heading direction estimation is 6.9°, which surpasses the inertial sensors. With the high accuracy and low cost, it illustrates the proposed system as a promising solution to large varieties of applications that require accurate heading direction information.

Fall detection systems based on WiFi signals are gaining popularity recently. However, most of the existing works relying on training are environment-dependent. In this paper, we propose DeFall, a novel WiFi-based environment-independent fall detection system by leveraging the features inherently associated with human falls - the patterns of speed and acceleration over time. The system consists of an offline template-generating stage and an online decision-making stage. In the offline stage, the speed of human falls is first estimated based on a statistical modeling about the Channel State Information (CSI). Dynamic Time Warping (DTW) based algorithms are applied to generate a representative template for typical human falls. Then fall event is detected in the online stage by evaluating the similarity between the patterns of realtime speed/acceleration estimates and the representative template. Extensive experiment results show that with a single pair of WiFi transceivers, the proposed system can achieve a detection rate of 96% and a false alarm rate smaller than 1.5% under both line-of-sight (LOS) and non-LOS (NLOS) scenarios.

Accurate monitoring of human vital signs (e.g. breathing and heart rates) is crucial in detecting medical problems. In this paper, we propose ViMo, a calibration-free remote Vital sign Monitoring system that can simultaneously monitor multiple users by leveraging the channel impulse response (CIR) of 60GHz WiFi. By exploiting the periodicity introduced by respiration, we first propose a human detection algorithm which does not require any prior calibration. Then, we apply the auto-correlation function (ACF) of the CIR phase to estimate the breathing rate. Lastly, to mitigate the impact of the breathing signal on the weak heartbeat signal, the cubic spline interpolation is used to eliminate the breathing signal before the estimation of the heart rate. Extensive experiments show that ViMo can achieve a median accuracy of 0.19 BPM for breathing rate estimation and 1 BPM for heart rate estimation, out-performing the existing non-contact solutions that are purely based on frequency analysis.

Passive human localization and tracking using RF signals have been studied for over a decade. Most of the existing solutions, however, can only track a single moving subject due to the coarse multipath resolvability limited by bandwidth and antenna number. In this paper, we break down the limitations by leveraging the emerging 60GHz millimeter-wave radios. We present mmTrack, the first system that passively localizes and tracks multiple users simultaneously using a single commodity 60GHz radio. The design of mmTrack consists of three key components. First, we significantly improve the spatial resolution, limited by the small aperture of the compact 60GHz array, by performing digital beamforming over all receive antennas. Second, we propose a novel multi-target detection approach that tackles the near-far-effect and measurement noise. Finally, we devise a robust clustering technique to accurately recognize multiple targets and estimate the respective locations, from which their individual trajectories are further derived by a continuous tracking algorithm. We implement mmTrack on a commodity 802.11ad device and evaluate it in indoor environments. Our experiments demonstrate that mmTrack detects and counts multiple users precisely with an error ≤ 1 person for 97.8% of the time and achieves a respective median location error of 9.9 cm and 19.7 cm for dynamic and static targets.

Existing indoor location systems do not easily scale at low cost while maintaining high accuracy. We present EasiTrack, an indoor tracking system that achieves decimeter accuracy using a single commodity WiFi Access Point (AP) under Non-Line-Of-Sight conditions and can deploy at scale with (almost) zero cost. We build a fully functional real-time system with a satellite-like architecture, which enables EasiTrack to support an unlimited number of clients. We have demonstrated EasiTrack in a number of different scenarios to track both humans and machines. The results reveal that EasiTrack achieves a decimeter median accuracy and a <2m maximum error and supports a broad coverage of 50 m×60 m using a single AP.

Wireless detection of respiration rates is crucial for many applications. Most of the state-of-the-art solutions estimate breathing rates with the prior knowledge of crowd numbers as well as assuming the distinct breathing rates of different users, which is neither natural nor realistic. However, few of them can leverage the estimated breathing rates to recognize human subjects (also known as identity matching). In this article, using the channel state information (CSI) of a single pair of commercial WiFi devices, a novel system is proposed to continuously track the breathing rates of multiple persons without such impractical assumptions. The proposed solution includes an adaptive subcarrier combination method that boosts the signal-to-noise ratio (SNR) of breathing signals, and iterative dynamic programming and a trace concatenating algorithm that continuously tracks the breathing rates of multiple users. By leveraging both the spectrum and time diversity of the CSI, our system can correctly extract the breathing rate traces even if some of them merge together for a short time period. Furthermore, by utilizing the breathing traces obtained, our system can do people counting and recognition simultaneously. Extensive experiments are conducted in two environments (an on-campus lab and a car). The results show that 86% of average accuracy can be achieved for people counting up to four people for both cases. For 97.9% out of all the testing cases, the absolute error of crowd number estimates is within 1. The system achieves an average accuracy of 85.78% for people recognition in a smart home case.

Despite decades of efforts, existing indoor location systems do not easily scale with low cost while maintaining high accuracy. We present EasiTrack, an indoor tracking system that achieves decimeter accuracy using a single commodity WiFi access point (AP) under non-line-of-sight (NLOS) conditions and can deploy at scale with almost zero cost. EasiTrack makes two key technical contributions. First, it incorporates RF-based inertial measurement algorithms that can accurately infer a target's moving distance purely using the RF signals received by itself. Second, EasiTrack devises a map-augmented tracking algorithm that outputs fine-grained locations by jointly leveraging the distance estimates and an indoor map that is ubiquitously available nowadays. We build a fully functional real-time system centering around a satellite-like architecture, which enables EasiTrack to support an unlimited number of clients. We have deployed EasiTrack in seven different scenarios (including offices, hotels, museums, and manufacturing facilities) to track both humans and machines. The results reveal that EasiTrack achieves a median 0.25 m and 90%tile 0.69-m accuracy in distance measurement, a median 0.58 m and 90%tile 1.33-m location accuracy for tracking objects, and a median 0.70 m and 90%tile 1.97-m accuracy for tracking humans in both line-of-sight and NLOS scenarios and supports a broad coverage of 50 m × 60 m using a single AP. It is also verified that EasiTrack can be easily deployed in massive buildings with little cost, promising a practical solution for ubiquitous indoor tracking.

Existing indoor navigation solutions usually require pre-deployed comprehensive location services with precise indoor maps and, more importantly, all rely on dedicatedly installed or existed infrastructure. In this paper, we present Pair-Navi, an infrastructure-free indoor navigation system that circumvents all these requirements by reusing a previous traveler's (i.e. leader) trace experience to navigate future users (i.e. followers) in a Peer-to-Peer (P2P) mode. Our system leverages the advances of visual SLAM on commercial smartphones. Visual SLAM systems, however, are vulnerable to environmental dynamics in the precision and robustness and involve intensive computation that prohibits real-time applications. To combat environmental changes, we propose to cull non-rigid contexts and keep only the static and rigid contents in use. To enable real-time navigation on mobiles, we decouple and reorganize the highly coupled SLAM modules for leaders and followers. We implement Pair-Navi on commodity smartphones and validate its performance in three diverse buildings. Our results show that Pair-Navi achieves an immediate navigation success rate of 98.6%, which maintains as 83.4% even after two weeks since the leaders' traces were collected, outperforming the state-of-the-art solutions by >50%. Being truly infrastructure-free, Pair-Navi sheds lights on practical indoor navigations for mobile users.

Wi-Fi based sensing systems, although sound as being deployed almost everywhere there is Wi-Fi, are still practically difficult to be used without explicit adaptation efforts to new data domains. Various pioneering approaches have been proposed to resolve this contradiction by either translating features between domains or generating domain-independent features at a higher learning level. Still, extra training efforts are necessary in either data collection or model re-training when new data domains appear, limiting their practical usability. To advance cross-domain sensing and achieve fully zero-effort sensing, a domain-independent feature at the lower signal level acts as a key enabler. In this paper, we propose Widar3.0, a Wi-Fi based zero-effort cross-domain gesture recognition system. The key insight of Widar3.0 is to derive and estimate velocity profiles of gestures at the lower signal level, which represent unique kinetic characteristics of gestures and are irrespective of domains. On this basis, we develop a one-fits-all model that requires only one-time training but can adapt to different data domains. We implement this design and conduct comprehensive experiments. The evaluation results show that without re-training and across various domain factors (i.e. environments, locations and orientations of persons), Widar3.0 achieves 92.7% in-domain recognition accuracy and 82.6%-92.4% cross-domain recognition accuracy, outperforming the state-of-the-art solutions. To the best of our knowledge, Widar3.0 is the first zero-effort cross-domain gesture recognition work via Wi-Fi, a fundamental step towards ubiquitous sensing.

Inertial measurements are critical to almost any mobile applications. It is usually achieved by dedicated sensors (e.g., accelerometer, gyroscope) that suffer from significant accumulative errors. This paper presents RIM, an RF-based Inertial Measurement system for precise motion processing. RIM turns a commodity WiFi device into an Inertial Measurement Unit (IMU) that can accurately track moving distance, heading direction, and rotating angle, requiring no additional infrastructure but a single arbitrarily placed Access Point (AP) whose location is unknown. RIM makes three key technical contributions. First, it presents a spatial-temporal virtual antenna retracing scheme that leverages multipath profiles as virtual antennas and underpins measurements of distance and orientation using commercial WiFi. Second, it introduces a super-resolution virtual antenna alignment algorithm that resolves sub-centimeter movements. Third, it presents an approach to handle measurement noises and thus delivers an accurate and robust system. Our experiments, over a multipath rich area of > 1,000 m2 with one single AP, show that RIM achieves a median error in moving distance of 2.3 cm and 8.4 cm for short-range and long-distance tracking respectively, and 6.1° mean error in heading direction, all significantly outperforming dedicated inertial sensors. We also demonstrate multiple RIM-enabled applications with great performance, including indoor tracking, handwriting, and gesture control.

Cloud computing-based big data processing and intelligent analyticsCloud and big data have become the big things today in many systems especially regarding of information processing and intelligent analytics.Big data analytics is the use of advanced analytic techniques against very large, diverse data sets, and it allows analysts, researchers to make better decisions using data that was previously inaccessible or unusable.Due to the urgent demand on high capacity of computation and storage resources, cloud computing has been acknowledged as the primary computing paradigm for massive data storage, processing under various circumstances and different requirement.1 Moreover, edge computing pushes the cloud frontier to the edge of the network and extends cloud computing to be able to address more application scenarios.This special track plans to solicit novel and original manuscripts in the above topics with an emphasize on ''Cloud Computing-based Big Data Processing and Intelligent Analytics.''From those submitted papers for the 6th International Conference on Advanced Cloud and Big Data (CBD 2018) held in Lanzhou, China on August 12 to August 14, 2018, nine papers are selected that target the following research issues in cloud computing and big data:• Service deployment and task scheduling in cloud computing and edge computing.• Cloud storage system design and optimization.• Case studies of big data in cloud-based system.• Network performance optimization in data centers.• Approximate big data analysis and processing.• Security threats and solutions in cloud computing and big data processing.Recently, mobile edge computing (MEC) has become a fascinating technology trend for future computing paradigm, while at the same time, it is also facing a great challenge that is how to make full use of edge resources to provide a seamless support for compute-intensive latency-sensitive applications.Most of the existing works assume that tasks can be executed upon every edge server, but the assumption does not hold in practical scenarios because a specific application task often corresponds to a certain service that provides the corresponding running environment.How to decide service deployment of so many types of services among multiple edge servers is also a big challenge.To address the challenge, Zhou et al 2 study dynamic service deployment for latency-sensitive applications and first model the long-term budget-constrained latency minimization problem as a multi-slot latency minimization problem based on the Lyapunov framework.Furthermore, the task scheduling optimization is also considered here, which makes every edge server be fully utilized in an even more efficient collaborative manner.How to repair data blocks in the erasure coding storage system is of great challenge as considering the incast problem introduced at the new node.Current solutions mainly rely on path planning and resource allocation and waste a large amount of storage and bandwidth resources unavoidably.Xia et al 3 propose the incast problem to be resolved economically via the in-network aggregation, where a set of in-network methods to repair a failed data block in the erasure coding storage systems.Compared with the existing methods, the in-network methods are capable of reducing the bandwidth consumption as well as achieving higher repair speed.With the continuous development of intelligent transportation systems (ITSs), various sensor data can be used to detect traffic information, but there is a lack of practical value to guide public traveling.Xu et al 4 propose a real-time traffic index model of expressways by using a traffic index to evaluate the actual conditions of expressways.The model considers the actual situation of floating and non-floating vehicles on expressways.Included is the realization of the complete calculation model of real-time traffic index estimation, including highway section division, spatial topology map matching, driving route calculation, and road congestion status judgment.For roads without floating car coverage, the weighted-moving-average time-series prediction method is used to predict the traffic index, so that the running condition of all roads in the network can be analyzed completely.A spotlight has shined on scientific workflows in recent years, as a result of their enormous impact on big data-related scientific areas.Large-scale scientific workflow scheduling across global data centers requires the scheduling framework to optimize data movement cost by leveraging these distributed data centers.However, challenges regarding of data-intensive workflow execution in multiple geo-distributed data centers still exist, such as data dependency, intermediate data placement, etc. Scientific workflow's data and task co-scheduling aim to solve the above problem, which is known to be NP-hard.Zhang et al 5 propose a novel approach based on the multilevel graph coarsening and un-coarsening framework, together with a specialized hybrid genetic algorithm having distinctive graph partition driven features of repair and local improvement, for scheduling data-intensive scientific workflows in geo-distributed data centers and optimizing the cross-data center data transfer volume.

Ubiquitous connectivity among objects is the future of the coming Internet of Things era. Technologies are competing fiercely to fulfill this goal, but none of them can fit into all application scenarios. However, efforts are still made to expand application ranges of certain technologies. Shortly after the adoption of its newest version, Bluetooth 5.0, the Bluetooth Special Interest Group released another new specification on network topology: Bluetooth Mesh. Combined together, those two bring Bluetooth to a brand new stage. However, current works related to it only focus on part of the new Bluetooth, and discussion over the entire one is lacking. Therefore, in this survey, we conduct an investigation toward the new Bluetooth from a comprehensive perspective. Through this, we show that the new Bluetooth not only consolidates its strengths in original application fields but also brings alterations and opportunities to new ones, making it a strong competitor in the future for providing complete solutions to meet the demands of seamless communications in the Internet of Things area.

Monitoring physical assault is critical for the prevention of juvenile delinquency and promotion of school harmony. A large portion of assault events, particularly school violence among teenagers, usually happen at indoor secluded places. Pioneering approaches employ always-on-body sensors or cameras in the limited surveillance area, which are privacy-invasive and cannot provide ubiquitous assault monitoring. In this paper, we present Wi-Dog, a noninvasive physical assault monitoring scheme that enables privacy-preserving monitoring in ubiquitous circumstances. Wi-Dog is based on widely deployed commodity Wi-Fi infrastructures. The key intuition is that Wi-Fi signals are easily distorted by human motions, and motion-induced signals could convey informative characteristics, such as intensity, regularity, and continuity. Specifically, to explicitly reveal the substantive properties of physical assault, we innovatively propose a set of signal processing methods for informative components extraction by selecting sensitive antenna pairs and subcarriers. Then a novel signal-complexity-based segmentation method is developed as a location-independent indicator to monitor targeted movement transitions. Finally, holistic analysis is employed based on domain knowledge, and we distinguish the violence process from both local and global perspective using time-frequency features. We implement Wi-Dog on commercial Wi-Fi devices and evaluate it in real indoor environments. Experimental results demonstrate the effectiveness of Wi-Dog which consistently outperforms the advanced abnormal detection methods with a higher true detection rate of 94% and a lower false alarm rate of 8%.

The past years have witnessed the rapid conceptualization and development of wireless sensing based on Channel State Information (CSI) with commodity WiFi devices.Many research efforts have been devoted to promote WiFi sensing by innovating applications, refining models and optimizing algorithms. A critical issue of Cross-Technology Interference (CTI), however, is surprisingly unnoticed and largely unexplored in the existing literature. In this paper, we demonstrate that CTI poses severe impacts on CSI measurements and further degrades the performance of CSI-based sensing. Based on in-depth understanding of such impacts, we present PERFIC to deal with CTI for CSI on commercial WiFi. We first exploit the inherent cyclostationarity property of different signals to detect CTI and further identify the specific distorted subcarriers on CSI. For each interfered CSI, we then propose to mitigate the impacts of CTI by amending the abnormal subcarriers. We conduct experiments on typical wireless sensing applications, including human detection and activity classification, using off-the-shelf WiFi devices. The results demonstrate that PERFIC yields a remarkable performance gain of >30% with high efficiency and outperforms existing robust classifiers.By providing interference-free CSI that is amendable to existing and emerging CSI-based sensing applications, PERFIC underpins new insights for improving the sensitivity and reliability of wireless sensing.

Vital signs such as heart rate and heartbeat interval are currently measured by electrocardiograms (ECG) or wearable physiological monitors. These techniques either require contact with the patient's skin or are usually uncomfortable to wear, rendering them too expensive and user-unfriendly for daily monitoring. In this paper, we propose a new noninvasive technology to generate an Acousticcardiogram (ACG) that precisely monitors heartbeats using inaudible acoustic signals. ACG uses only commodity microphones and speakers commonly equipped on ubiquitous off-the-shelf devices, such as smartphones and laptops. By transmitting an acoustic signal and analyzing its reflections off human body, ACG is capable of recognizing the heart rate as well as heartbeat rhythm. We employ frequency-modulated sound signals to separate reflection of heart from that of background motions and breath, and continuously track the phase changes of the acoustic data. To translate these acoustic data into heart and breath rates, we leverage the dual microphone design on COTS mobile devices to suppress direct echo from speaker to microphones, identify heart rate in frequency domain, and adopt an advanced algorithm to extract individual heartbeats. We implement ACG on commercial devices and validate its performance in real environments. Experimental results demonstrate ACG monitors user's heartbeat accurately, with median heart rate estimation error of 0.6 beat per minute (bpm), and median heartbeat interval estimation error of 19 ms.

Hidden screen-camera communication techniques emerge as a new paradigm that embeds data imperceptibly into regular videos while remaining unobtrusive to human viewers. Three key goals on imperceptible, high rate, and reliable communication are desirable but conflicting, and existing solutions usually made a trade-off among them. In this paper, we present the design and implementation of ChromaCode, a screen-camera communication system that achieves all three goals simultaneously. In our design, we consider for the first time color space for perceptually uniform lightness modifications. On this basis, we design an outcome-based adaptive embedding scheme, which adapts to both pixel lightness and regional texture. Last, we propose a concatenated code scheme for robust coding and devise multiple techniques to overcome various screen-camera channel errors. Our prototype and experiments demonstrate that ChromaCode achieves remarkable raw throughputs of >700 kbps, data goodputs of 120 kbps with BER of 0.05, and with fully imperceptible flicker for viewing proved by user study, which significantly outperforms previous works.

This paper presents Widar2.0, the first WiFi-based system that enables passive human localization and tracking using a single link on commodity off-the-shelf devices. Previous works based on either specialized or commercial hardware all require multiple links, preventing their wide adoption in scenarios like homes where typically only one single AP is installed. The key insight underlying Widar2.0 to circumvent the use of multiple links is to leverage multi-dimensional signal parameters from one single link. To this end, we build a unified model accounting for Angle-of-Arrival, Time-of-Flight, and Doppler shifts together and devise an efficient algorithm for their joint estimation. We then design a pipeline to translate the erroneous raw parameters into precise locations, which first finds parameters corresponding to the reflections of interests, then refines range estimates, and ultimately outputs target locations. Our implementation and evaluation on commodity WiFi devices demonstrate that Widar2.0 achieves better or comparable performance to state-of-the-art localization systems, which either use specialized hardwares or require 2 to 40 Wi-Fi links.

Device-free passive detection is an emerging technology to detect whether there exist any moving entities in the areas of interest without attaching any device to them. It is an essential primitive for a broad range of applications including intrusion detection for safety precautions, patient monitoring in hospitals, child and elder care at home, and so forth. Despite the prevalent signal feature Received Signal Strength (RSS), most robust and reliable solutions resort to a finer-grained channel descriptor at the physical layer, e.g., the Channel State Information (CSI) in the 802.11n standard. Among a large body of emerging techniques, however, few of them have explored the full potential of CSI for human detection. Moreover, space diversity supported by nowadays popular multiantenna systems are not investigated to a comparable extent as frequency diversity. In this article, we propose a novel scheme for device-free PAssive Detection of moving humans with dynamic Speed (PADS). Both full information (amplitude and phase) of CSI and space diversity across multiantennas in MIMO systems are exploited to extract and shape sensitive metrics for accuracy and robust target detection. We prototype PADS on commercial WiFi devices, and experiment results in different scenarios demonstrate that PADS achieves great performance improvement in spite of dynamic human movements.

Modern mobile devices are equipped with multiple antennas, which brings various wireless sensing applications such as accurate localization, contactless human detection, and wireless human-device interaction. A key enabler for these applications is phased array signal processing, especially Angle of Arrival (AoA) estimation. However, accurate AoA estimation on commodity devices is non-trivial due to limited number of antennas and uncertain phase offsets. Previous works either rely on elaborate calibration or involve contrived human interactions. In this paper, we aim to enable practical AoA measurements on commodity off-the-shelf (COTS) mobile devices. The key insight is to involve users' natural rotation to formulate a virtual spatial-temporal antenna array and conduce a relative incident signal of measurements at two orientations. Then by taking the differential phase, it is feasible to remove the phase offsets and derive the accurate AoA of the equivalent incoming signal, while the rotation angle can also be captured by built-in inertial sensors. On this basis, we propose Differential MUSIC (D-MUSIC), a relative form of the standard MUSIC algorithm that eliminates the unknown phase offsets and achieves accurate AoA estimation on COTS mobile devices with only one rotation. We further extend DMUSIC to 3-D space, integrate extra measurements during rotations for higher estimation accuracy, and fortify it in multipath-rich scenarios. We prototype D-MUSIC on commodity WiFi infrastructure and evaluate it in typical indoor environments. Experimental results demonstrate a superior performance with average AoA estimation errors of 130 with only three measurements and 50 with at most 10 measurements. Requiring no modifications or calibration, D-MUSIC is envisioned as a promising scheme for practical AoA estimation on COTS mobile devices.

The proliferation of mobile computing has prompted WiFi-based indoor localization to be one of the most attractive and promising techniques for ubiquitous applications. A primary concern for these technologies to be fully practical is to combat harsh indoor environmental dynamics, especially for long-term deployment. Despite numerous research on WiFi fingerprint-based localization, the problem of radio map adaptation has not been sufficiently studied and remains open. In this work, we propose AcMu, an automatic and continuous radio map self-updating service for wireless indoor localization that exploits the static behaviors of mobile devices. By accurately pinpointing mobile devices with a novel trajectory matching algorithm, we employ them as mobile reference points to collect real-time RSS samples when they are static. With these fresh reference data, we adapt the complete radio map by learning an underlying relationship of RSS dependency between different locations, which is expected to be relatively constant over time. Extensive experiments for 20 days across six months demonstrate that AcMu effectively accommodates RSS variations over time and derives accurate prediction of fresh radio map with average errors of less than 5dB, outperforming existing approaches. Moreover, AcMu provides 2x improvement on localization accuracy by maintaining an up-to-date radio map.

In-air interaction acts as a key enabler for ambient intelligence and augmented reality. As an increasing popular example, exergames, and the alike gesture recognition applications, have attracted extensive research in designing accurate, pervasive and low-cost user interfaces. Recent advances in wireless sensing show promise for a ubiquitous gesture-based interaction interface with Wi-Fi. In this work, we extract complete information of motion-induced Doppler shifts with only commodity Wi-Fi. The key insight is to harness antenna diversity to carefully eliminate random phase shifts while retaining relevant Doppler shifts. We further correlate Doppler shifts with motion directions, and propose a light-weight pipeline to detect, segment, and recognize motions without training. On this basis, we present WiDance, a Wi-Fi-based user interface, which we utilize to design and prototype a contactless dance-pad exergame. Experimental results in typical indoor environment demonstrate a superior performance with an accuracy of 92%, remarkably outperforming prior approaches.

In recent years, WiFi-based sensing applications have been proliferated due to growing capacities of the physical layer. Channel State Information (CSI), which depicts the characteristics of propagation environment and reflects different human behaviors, can be easily obtained on commodity WiFi devices with slight driver modification. For the sake of higher accuracy and robustness of CSI-based sensing, a variety of research efforts have been devoted to model refinement, algorithm optimization and data sanitization. Radio frequency interference (RFI) is a crucial problem, which, however, is surprisingly overlooked and largely unexplored. The sensing performance can be significantly boosted by identifying and properly handling the interfered CSI measurements. In this paper, we demonstrate that it is feasible to identify the interfered CSI measurements due to the unique properties induced by RFI. We propose two RFI detection algorithms by utilizing cyclostationary analysis from different angles. Experimental results on off-the-shelf WiFi devices show that both algorithms are robustly stable for different scenarios and can achieve a remarkable overall accuracy of > 90%.

Sensorless sensing using wireless signals has been rapidly conceptualized and developed recently. Among numerous applications of WiFi-based sensing, human presence detection acts as a primary and fundamental function to boost applications in practice. Many complicated approaches have been proposed to achieve high detection accuracy, which, however, frequently omit various practical constraints like real-time capability, computation efficiency, sampling rates, deployment efforts, etc. A practical detection system that works in real world lacks. In this paper, we design and implement WiSH, a real-time system for contactless human detection that is applicable for whole-day usage. WiSH employs lightweight yet effective methods and thus enables detection under practical conditions even on resource-limited devices with very low signal sampling rates. We deploy WiSH on commodity desktops and customized tiny nodes in different everyday scenarios. The experimental results demonstrate superior performance of WiSH, achieving a detection accuracy of >98% using a sampling rate of 20Hz with an average detection delay of merely 1.5s, which renders it a promising system for real-world deployment.

Various pioneering approaches have been proposed for Wi-Fi-based sensing, which usually employ learning-based techniques to seek appropriate statistical features, yet do not support precise tracking without prior training. Thus to advance passive sensing, the ability to track fine-grained human mobility information acts as a key enabler. In this paper, we propose Widar, a Wi-Fi-based tracking system that simultaneously estimates a human's moving velocity (both speed and direction) and location at a decimeter level. Instead of applying statistical learning techniques, Widar builds a theoretical model that geometrically quantifies the relationships between CSI dynamics and the user's location and velocity. On this basis, we propose novel techniques to identify frequency components related to human motion from noisy CSI readings and then derive a user's location in addition to velocity. We implement Widar on commercial Wi-Fi devices and validate its performance in real environments. Our results show that Widar achieves decimeter-level accuracy, with a median location error of 25 cm given initial positions and 38 cm without them and a median relative velocity error of 13%.

Most of existing indoor navigation systems work in a client/server manner, which needs to deploy comprehensive localization services together with precise indoor maps a prior. In this paper, we design and realize a peer-to-peer navigation system (ppNav), on smartphones, which enables the fast-to-deploy navigation services, avoiding the requirements of pre-deployed location services and detailed floorplans. ppNav navigates a user to the destination by tracking user mobility, promoting timely walking tips and alerting potential deviations, according to a previous traveller's trace experience. Specifically, we utilize the ubiquitous WiFi fingerprints in a novel diagrammed form and extract both radio and visual features of the diagram to track relative locations and exploit fingerprint similarity trend for deviation detection. We further devise techniques to lock on a user to the nearest reference path in case he/she arrives at an uncharted place. Consolidating these techniques, we implement ppNav on commercial mobile devices and validate its performance in real environments. Our results show that ppNav achieves delightful performance, with an average relative error of 0.9 m in trace tracking and a maximum delay of nine samples (about 4.5 s) in deviation detection.

Industry 4.0 brings forward higher requirements on the monitoring of industrial production. Video surveillance based on Wireless Mesh Networks (WMNs) has demonstrated its effectiveness in a number of applications. Different from some typical applications of WMN that solve the "last mile" Internet access problem, WMN-based video surveillance for industrial monitoring is very likely to work in extreme circumstances or requiring high performance. Thus, the guarantee of video quality is the key for the success of industrial video surveillance. Through extensive experimental research, we find that the state of the art mapping and queuing algorithms are approaching complication and the room for improvement is decreasing. The possible solution for performance breakthrough lies in exploiting the potentials of data granularity for mapping. In this work, we propose IMesh, a video transmission solution based on WMN, which takes frame type, frame location, data packets and other factors into consideration and quantifies their impacts on video quality. The proposed approach is particularly suitable for video surveillance in industrial production under aggressive conditions. To the best of our knowledge, IMesh is the first one that differentiates, prioritizes, and schedules video data in the packet level, which is the finest granularity one can achieve while keep the MAC layer protocol unchanged. Experiment results show that the proposed solution outperforms previous works both in terms of video quality and packet delay.

Most of existing indoor navigation systems work in a client/server manner, which needs to deploy comprehensive localization services together with precise indoor maps a prior. In this paper, we design and realize a Peer-to-Peer navigation system, named ppNav, on smartphones, which enables the fast-to-deploy navigation services, avoiding the requirements of pre-deployed location services and detailed floorplans. ppNav navigates a user to the destination by tracking user mobility, promoting timely walking tips, and alerting potential deviations, according to a previous traveller's trace experience. Specifically, we utilize the ubiquitous WiFi fingerprints in a novel diagrammed form and extract both radio and visual features of the diagram to track relative locations and exploit fingerprint similarity trend for deviation detection. Consolidating these techniques, we implement ppNav on commercial mobile devices and validate its performance in real environments. Our results show that ppNav achieves delightful performance, with an average relative error of 0.9m in trace tracking and a maximum delay of 9 samples (about 4.5s) in deviation detection.

Modern mobile devices are equipped with multiple antennas, which brings various wireless sensing applications such as accurate localization, contactless human detection and wireless human-device interaction. A key enabler for these applications is phased array signal processing, especially Angle of Arrival (AoA) estimation. However, accurate AoA estimation on commodity devices is non-trivial due to limited number of antennas and uncertain phase offsets. Previous works either rely on elaborate calibration or involve contrived human interactions. In this paper, we aim to enable practical AoA measurements on commodity off-the-shelf (COTS) mobile devices. The key insight is to involve users' natural rotation to formulate a virtual spatial-temporal antenna array and conduce a relative incident signal of measurements at two orientations. Then by taking the differential phase, it is feasible to remove the phase offsets and derive the accurate AoA of the equivalent incoming signal, while the rotation angle can also be captured by built-in inertial sensors. On this basis, we propose Differential MUSIC (D-MUSIC), a relative form of the standard MUSIC algorithm that eliminates the unknown phase offsets and achieves accurate AoA estimation on COTS mobile devices with only one rotation. We further extend D-MUSIC to 3-D space and fortify it in multipath-rich scenarios. We prototype D-MUSIC on commodity WiFi infrastructure and evaluate it in typical indoor environments. Experimental results demonstrate a superior performance with an average AoA estimation error of 13°. Requiring no modifications or calibration, D-MUSIC is envisioned as a promising scheme for practical AoA estimation on COTS mobile devices.

Wireless-based device-free human sensing has raised increasing research interest and stimulated a range of novel location-based services and human-computer interaction applications for recreation, asset security and elderly care. A primary functionality of these applications is to first detect the presence of humans before extracting higher-level contexts such as physical coordinates, body gestures, or even daily activities. In the presence of dense multipath propagation, however, it is non-trivial to even reliably identify the presence of humans. The multipath effect can invalidate simplified propagation models and distort received signal signatures, thus deteriorating detection rates and shrinking detection range. In this paper, we characterize the impact of human presence on wireless signals via ray-bouncing models, and propose a measurable metric on commodity WiFi infrastructure as a proxy for detection sensitivity. To achieve higher detection rate and wider sensing coverage in multipath-dense indoor scenarios, we design a lightweight sub carrier and path configuration scheme harnessing frequency diversity and spatial diversity. We prototype our scheme with standard WiFi devices. Evaluations conducted in two typical office environments demonstrate a detection rate of 92.0% with a false positive of 4.5%, and almost 1x gain in detection range given a minimal detection rate of 90%.

The proliferation of mobile computing has prompted WiFi-based indoor localization to be one of the most attractive and promising techniques for ubiquitous applications. A primary concern for these technologies to be fully practical is to combat harsh indoor environmental dynamics, especially for long-term deployment. Despite numerous research on WiFi fingerprint-based localization, the problem of radio map adaptation has not been sufficiently studied and remains open. In this work, we propose AcMu, an automatic and continuous radio map self-updating service for wireless indoor localization that exploits the static behaviors of mobile devices. By accurately pinpointing mobile devices with a novel trajectory matching algorithm, we employ them as mobile reference points to collect real-time RSS samples when they are static. With these fresh reference data, we adapt the complete radio map by learning an underlying relationship of RSS dependency between different locations, which is expected to be relatively constant over time. Extensive experiments for 20 days across 6 months demonstrate that AcMu effectively accommodates RSS variations over time and derives accurate prediction of fresh radio map with average errors of less than 5dB. Moreover, AcMu provides 2x improvement on localization accuracy by maintaining an up-to-date radio map.

WiFi technology has fostered numerous mobile computing applications, such as adaptive communication, finegrained localization, gesture recognition, etc., which often achieve better performance or rely on the availability of Line-Of-Sight (LOS) signal propagation. Thus the awareness of LOS and Non-Line-Of-Sight (NLOS) plays as a key enabler for them. Realtime LOS identification on commodity WiFi devices, however, is challenging due to limited bandwidth of WiFi and resulting coarse multipath resolution. In this work, we explore and exploit the phase feature of PHY layer information, harnessing both space diversity with antenna elements and frequency diversity with OFDM subcarriers. On this basis, we propose PhaseU, a real-time LOS identification scheme that works in both static and mobile scenarios on commodity WiFi infrastructure. Experimental results in various indoor scenarios demonstrate that PhaseU consistently outperforms previous approaches, achieving overall LOS and NLOS detection rates of 94.35% and 94.19% in static cases and both higher than 80% in mobile contexts. Furthermore, PhaseU achieves real-time capability with millisecond-level delay for a connected AP and 1-second delay for unconnected APs, which is far beyond existing approaches.

Non-invasive human sensing based on radio signals has attracted a great deal of research interest and fostered a broad range of innovative applications of localization, gesture recognition, smart health-care, etc., for which a primary primitive is to detect human presence. Previous works have studied the detection of moving humans via signal variations caused by human movements. For stationary people, however, existing approaches often employ a prerequisite scenario-tailored calibration of channel profile in human-free environments. Based on in-depth understanding of human motion induced signal attenuation reflected by PHY layer channel state information (CSI), we propose DeMan, a unified scheme for non-invasive detection of moving and stationary human on commodity WiFi devices. DeMan takes advantage of both amplitude and phase information of CSI to detect moving targets. In addition, DeMan considers human breathing as an intrinsic indicator of stationary human presence and adopts sophisticated mechanisms to detect particular signal patterns caused by minute chest motions, which could be destroyed by significant whole-body motion or hidden by environmental noises. By doing this, DeMan is capable of simultaneously detecting moving and stationary people with only a small number of prior measurements for model parameter determination, yet without the cumbersome scenario-specific calibration. Extensive experimental evaluation in typical indoor environments validates the great performance of DeMan in various human poses and locations and diverse channel conditions. Particularly, DeMan provides a detection rate of around 95% for both moving and stationary people, while identifies human-free scenarios by 96%, all of which outperforms existing methods by about 30%.

Indoor localization is of great importance for a range of pervasive applications, attracting many research efforts in the past decades. Most radio-based solutions require a process of site survey, in which radio signatures of an interested area are annotated with their real recorded locations. Site survey involves intensive costs on manpower and time, limiting the applicable buildings of wireless localization worldwide. In this study, we investigate novel sensors integrated in modern mobile phones and leverage user motions to construct the radio map of a floor plan, which is previously obtained only by site survey. Considering user movements in a building, originally separated RSS fingerprints are geographically connected by user moving paths of locations where they are recorded, and they consequently form a high dimension fingerprint space, in which the distances among fingerprints are preserved. The fingerprint space is then automatically mapped to the floor plan in a stress-free form, which results in fingerprints labeled with physical locations. On this basis, we design LiFS, an indoor localization system based on off-the-shelf WiFi infrastructure and mobile phones. LiFS is deployed in an office building covering over 1,600 m2, and its deployment is easy and rapid since little human intervention is needed. In LiFS, the calibration of fingerprints is crowdsourced and automatic. Experiment results show that LiFS achieves comparable location accuracy to previous approaches even without site survey.

Global positioning system (GPS) has enabled a number of geographical applications over many years. Quite a lot of location-based services, however, still suffer from considerable positioning errors of GPS (usually 1 to 20 m in practice). In this study, we design and implement a high-accuracy global positioning solution based on GPS and human mobility captured by mobile phones. Our key observation is that smartphone-enabled dead reckoning supports accurate but local coordinates of users' trajectories, while GPS provides global but inconsistent coordinates. Considering them simultaneously, we devise techniques to refine the global positioning results by fitting the global positions to the structure of locally measured ones, so the refined positioning results are more likely to elicit the ground truth. We develop a prototype system, named GloCal, and conduct comprehensive experiments in both crowded urban and spacious suburban areas. The evaluation results show that GloCal can achieve 30 percent improvement on average error with respect to GPS. GloCal uses merely mobile phones and requires no infrastructure or additional reference information. As an effective and light-weight augmentation to global positioning, GloCal holds promise in real-world feasibility.

Wireless LANs, particularly WiFi, have been pervasively deployed and have fostered myriad wireless communication services and ubiquitous computing applications. A primary concern in designing these applications is to combat harsh indoor propagation environments, particularly Non-Line-Of-Sight (NLOS) propagation. The ability to identify the existence of the Line-Of-Sight (LOS) path acts as a key enabler for adaptive communication, cognitive radios, and robust localization. Enabling such capability on commodity WiFi infrastructure, however, is prohibitive due to the coarse multipath resolution with MAC-layer received signal strength. In this paper, we propose two PHY-layer channel-statistics-based features from both the time and frequency domains. To further break away from the intrinsic bandwidth limit of WiFi, we extend to the spatial domain and harness natural mobility to magnify the randomness of NLOS paths while retaining the deterministic nature of the LOS component. We propose LiFi, a statistical LOS identification scheme with commodity WiFi infrastructure, and evaluate it in typical indoor environments covering an area of 1500 m2. Experimental results demonstrate that LiFi achieves an overall LOS detection rate of 90.42% with a false alarm rate of 9.34% for the temporal feature and an overall LOS detection rate of 93.09% with a false alarm rate of 7.29% for the spectral feature.

Crossroad is among the most dangerous parts outside for the visually impaired people. Numerous studies have exploited navigating systems for the visually impaired community, providing services ranging from block detection, route planning to realtime localization. However, none of them have addressed the safety issue in crossroad and integrated three key factors necessary for a practical crossroad navigation system: detecting the crossroad, locating zebra patterns, and guiding the user within zebra crossing when passing the road. Our CrossNavi application responds to these needs, providing an integrated crossroad navigation service that incorporates all the essential functionalities mentioned above. The overall service is fulfilled by the collaboration of built-in sensors on commodity phones, and requires minimal human participation. We describe the technical aspects of its design, implementation, interface, and further improvements to make the system practical on a wider basis. Experimental results from three visually impaired volunteers show that the system exhibits promising behavior in both urban and rural areas.

Device-free passive detection is an emerging technology to detect whether there exists any moving entities in the area of interests without attaching any device to them. It is an essential primitive for a broad range of applications including intrusion detection for safety precautions, patient monitoring in hospitals, child and elder care at home, etc. Despite of the prevalent signal feature Received Signal Strength (RSS), most robust and reliable solutions resort to finer-grained channel descriptor at physical layer, e.g., the Channel State Information (CSI) in the 802.11n standard. Among a large body of emerging techniques, however, few of them have explored full potentials of CSI for human detection. Moreover, space diversity supported by nowadays popular multi-antenna systems are not investigated to the comparable extent as frequency diversity. In this paper, we propose a novel scheme for device-free PAssive Detection of moving humans with dynamic Speed (PADS). Both amplitude and phase information of CSI are extracted and shaped into sensitive metrics for target detection; and CSI across multi-antennas in MIMO systems are further exploited to improve the detection accuracy and robustness. We prototype PADS on commercial WiFi devices and experiment results in different scenarios demonstrate that PADS achieves great performance improvement in spite of dynamic human movements.

Wireless LANs, especially WiFi, have been pervasively deployed and have fostered myriad wireless communication services and ubiquitous computing applications. A primary concern in designing each scenario-tailored application is to combat harsh indoor propagation environments, particularly Non-Line-Of-Sight (NLOS) propagation. The ability to distinguish Line-Of-Sight (LOS) path from NLOS paths acts as a key enabler for adaptive communication, cognitive radios, robust localization, etc. Enabling such capability on commodity WiFi infrastructure, however, is prohibitive due to the coarse multipath resolution with mere MAC layer RSSI. In this work, we dive into the PHY layer and strive to eliminate irrelevant noise and NLOS paths with long delays from the multipath channel responses. To further break away from the intrinsic bandwidth limit of WiFi, we extend to the spatial domain and harness natural mobility to magnify the randomness of NLOS paths while retaining the deterministic nature of the LOS component. We prototype LiFi, a statistical LOS identification scheme for commodity WiFi infrastructure and evaluate it in typical indoor environments covering an area of 1500 m2. Experimental results demonstrate an overall LOS identification rate of 90.4% with a false alarm rate of 9.3%.

Context-awareness is getting increasingly important for a range of mobile and pervasive applications on nowadays smartphones. Whereas human-centric contexts (e.g., indoor/ outdoor, at home/in office, driving/walking) have been extensively researched, few attempts have studied from phones' perspective (e.g., on table/sofa, in pocket/bag/hand). We refer to such immediate surroundings as micro-environment, usually several to a dozen of centimeters, around a phone. In this study, we design and implement Sherlock, a micro-environment sensing platform that automatically records sensor hints and characterizes the micro-environment of smartphones. The platform runs as a daemon process on a smartphone and provides finer-grained environment information to upper layer applications via programming interfaces. Sherlock is a unified framework covering the major cases of phone usage, placement, attitude, and interaction in practical uses with complicated user habits. As a long-term running middleware, Sherlock considers both energy consumption and user friendship. We prototype Sherlock on Android OS and systematically evaluate its performance with data collected on fifteen scenarios during three weeks. The preliminary results show that Sherlock achieves low energy cost, rapid system deployment, and competitive sensing accuracy.

Crowdsourcing-based mobile applications are becoming more and more prevalent in recent years, as smartphones equipped with various built-in sensors are proliferating rapidly. The large quantity of crowdsourced sensing data stimulates researchers to accomplish some tasks that used to be costly or impossible, yet the quality of the crowdsourced data, which is of great importance, has not received sufficient attention. In reality, the low-quality crowdsourced data are prone to containing outliers that may severely impair the crowdsourcing applications. Thus in this work, we conduct pioneer investigation considering crowdsourced data quality. Specifically, we focus on estimating user motion trajectory information, which plays an essential role in multiple crowdsourcing applications, such as indoor localization, context recognition, indoor navigation, etc. We resort to the family of robust statistics and design a robust trajectory estimation scheme, name TrMCD, which is capable of alleviating the negative influence of abnormal crowdsourced user trajectories, differentiating normal users from abnormal users, and overcoming the challenge brought by spatial unbalance of crowdsourced trajectories. Two real field experiments are conducted and the results show that TrMCD is robust and effective in estimating user motion trajectories and mapping fingerprints to physical locations.

Device-free Passive (DfP) human detection acts as a key enabler for emerging location-based services such as smart space, human-computer interaction, and asset security. A primary concern in devising scenario-tailored detecting systems is coverage of their monitoring units. While disk-like coverage facilitates topology control, simplifies deployment analysis, and is crucial for proximity-based applications, conventional monitoring units demonstrate directional coverage due to the underlying transmitter-receiver link architecture. To achieve omnidirectional coverage under such link-centric architecture, we propose the concept of omnidirectional passive human detection. The rationale is to exploit the rich multipath effect to blur the directional coverage. We harness PHY layer features to robustly capture the fine-grained multipath characteristics and virtually tune the shape of the coverage of the monitoring unit, which is previously prohibited with mere MAC layer RSSI. We design a fingerprinting scheme and a threshold-based scheme with off-the-shelf WiFi infrastructure and evaluate both schemes in typical clustered indoor scenarios. Experimental results demonstrate an average false positive of 8 percent and an average false negative of 7 percent for fingerprinting in detecting human presence in 4 directions. And both average false positive and false negative remain around 10 percent even with threshold-based methods.

Indoor localization has enabled a great number of mobile and pervasive applications, attracting attentions from researchers worldwide. Most of current solutions rely on Received Signal Strength (RSS) of wireless signals as location fingerprint, to discriminate locations of interest. Fingerprint uniqueness with respect to locations is a basic requirement in these fingerprinting-based solutions. However, due to insufficient number of signal sources, temporal variations of wireless signals, and rich multipath effects, such requirement is not always met in complex indoor environments, which we refer to as fingerprint ambiguity. In this work, we explore the potential of leveraging user motion against fingerprint ambiguity. Our basic idea is that user motion patterns collected by built-in sensors of mobile phones add to the diversity built by RSS fingerprints. On this basis, we propose MoLoc, a motion-assisted localization scheme implemented on mobile phones. MoLoc can easily be integrated in existing localization systems by simply adding a motion database that is constructed automatically by crowdsourcing. We conducted experiments in a large office hall. The experiment results show that MoLoc doubles the localization accuracy achieved by the fingerprinting method, and limits the mean localization error to less than 1m.

Global Positioning System (GPS) has enabled a number of geographical applications over many years. Quite a lot of location-based services, however, still suffer from considerable positioning errors of GPS (usually 1m to 20m in practice). In this study, we design and implement a high-accuracy global positioning solution based on GPS and human mobility captured by mobile phones. Our key observation is that smart phone-enabled dead reckoning supports accurate but local coordinates of users' trajectories, while GPS provides global but inconsistent coordinates. Considering them simultaneously, we devise techniques to refine the global positioning results by fitting the global positions to the structure of locally measured ones, so the refined positioning results are more likely to elicit the ground truth. We develop a prototype system, named GloCal, and conduct comprehensive experiments in both crowded urban and spacious suburban areas. The evaluation results show that GloCal can achieve 30% improvement on average error with respect to GPS.

Passive human detection and localization serve as key enablers for various pervasive applications such as smart space, human-computer interaction and asset security. The primary concern in devising scenario-tailored detecting systems is the coverage of their monitoring units. In conventional radio-based schemes, the basic unit tends to demonstrate a directional coverage, even if the underlying devices are all equipped with omnidirectional antennas. Such an inconsistency stems from the link-centric architecture, creating an anisotropic wireless propagating environment. To achieve an omnidirectional coverage while retaining the link-centric architecture, we propose the concept of Omnidirectional Passive Human Detection, and investigate to harness the PHY layer features to virtually tune the shape of the unit coverage by fingerprinting approaches, which is previously prohibited with mere MAC layer RSSI. We design the scheme with ubiquitously deployed WiFi infrastructure and evaluate it in typical multipath-rich indoor scenarios. Experimental results show that our scheme achieves an average false positive of 8% and an average false negative of 7% in detecting human presence in 4 directions.

Knowing accurate positions of nodes in wireless ad hoc and sensor networks is essential for a wide range of pervasive and mobile applications. However, errors are inevitable in distance measurements and we observe that a small number of outliers can degrade localization accuracy drastically. To deal with noisy and outlier ranging results, triangle inequality, is often employed in existing approaches. Our study shows that triangle inequality has many limitations, which make it far from accurate and reliable. In this study, we formally define the outlier detection problem for network localization and build a theoretical foundation to identify outliers based on graph embeddability and rigidity theory. Our analysis shows that the redundancy of distance measurements plays an important role. We then design a bilateration generic cycles-based outlier detection algorithm, and examine its effectiveness and efficiency through a network prototype implementation of MicaZ motes as well as extensive simulations. The results show that our design significantly improves the localization accuracy by wisely rejecting outliers.

Indoor localization is of great importance for a range of pervasive applications, attracting many research efforts in the past two decades. Most radio-based solutions require a process of site survey, in which radio signatures are collected and stored for further comparison and matching. Site survey involves intensive costs on manpower and time. In this work, we study unexploited RF signal characteristics and leverage user motions to construct radio floor plan that is previously obtained by site survey. On this basis, we design WILL, an indoor localization approach based on off-the-shelf WiFi infrastructure and mobile phones. WILL is deployed in a real building covering over 1600 m2, and its deployment is easy and rapid since site survey is no longer needed. The experiment results show that WILL achieves competitive performance comparing with traditional approaches.

Indoor localization is of great importance for a range of pervasive applications, attracting many research efforts in the past two decades. Most radio-based solutions require a process of site survey, in which radio signatures are collected and stored for further comparison and matching. Site survey involves intensive costs on manpower and time. In this work, we study unexploited RF signal characteristics and leverage user motions to construct radio floor plan that is previously obtained by site survey. On this basis, we design WILL, an indoor localization approach based on off-the-shelf WiFi infrastructure and mobile phones. WILL is deployed in a real building covering over 1600m2, and its deployment is easy and rapid since site survey is no longer needed. The experiment results show that WILL achieves competitive performance comparing with traditional approaches.

Indoor localization is of great importance for a range of pervasive applications, attracting many research efforts in the past decades. Most radio-based solutions require a process of site survey, in which radio signatures of an interested area are annotated with their real recorded locations. Site survey involves intensive costs on manpower and time, limiting the applicable buildings of wireless localization worldwide. In this study, we investigate novel sensors integrated in modern mobile phones and leverage user motions to construct the radio map of a floor plan, which is previously obtained only by site survey. On this basis, we design LiFS, an indoor localization system based on off-the-shelf WiFi infrastructure and mobile phones. LiFS is deployed in an office building covering over 1600m2, and its deployment is easy and rapid since little human intervention is needed. In LiFS, the calibration of fingerprints is crowdsourced and automatic. Experiment results show that LiFS achieves comparable location accuracy to previous approaches even without site survey.

A majority of localization approaches of wireless sensor networks rely on the measurements of inter-node distance. Errors are inevitable in distance measurements and we observe that a small number of outliers can degrade localization accuracy drastically. To deal with noisy and outlier ranging results, a straight-forward method, triangle inequality, is often employed in previous studies. However, triangle inequality has its own limitations that make itself far from accurate and reliable. In this study, we first analyze how much information are needed to identify outlier measurements. Applying rigidity theory, we propose the concept of verifiable edges and derive the conditions of an edge being verifiable. On this basis, we design a localization approach with outlier detection, which explicitly eliminates the rangings with large errors before location computation. Considering entire networks, we define verifiable graphs in which all edges are verifiable. If a wireless network meets the requirements of graph verifiability, it is not only localizable, but also outlier-resistant. Extensive simulations are conducted to examine the effectiveness of the proposed approach. The results show the remarkable improvements of location accuracy by sifting outliers.