This paper presents E-Cube, a framework designed to enhance the efficiency of mobile video streaming systems. Our core insight is that content correlations among frames, which are critical for video streaming but challenging to extract in dynamic scenes, are often already available as intermediate outputs from other mobile computing subsystems. By adopting a cross-subsystem design, E-Cube repurposes these intermediate results obtained from event cameras to reduce computation, energy consumption, and bandwidth usage of existing video streaming systems, while simultaneously improving video quality. We implement E-Cube on a drone-embedded chip through software-hardware co-design, and plugged E-Cube into three prevalent drone video streaming systems for industrial inspection. Evaluations in one of the world's largest oil fields and on public datasets demonstrate E-Cube can save over 35% network bandwidth overhead and reduce drone video streaming energy consumption by over 12% for H.265 and AV1, and 47% for advanced H.266, all while achieving improved video quality.

This paper introduces CLEAR-APG, a novel acoustic sensing approach that enables reliable heart rate monitoring in unconstrained environments using off-the-shelf active noise cancellation (ANC) headphones. By emitting ultrasonic signals into the user's ear canal via the headphone speaker and analyzing their echoes, which can detect the frequency of a pulsating vein along the canal wall. However, everyday activities such as exercising, speaking, or eating cause jaw movements that deform the ear canal, overwhelming the subtle deformation caused by blood flowing. To overcome this challenge, we employ the ANC headphone's built-in gyroscope to capture body motion and identify how various motion patterns influence the heartbeat waveform. Building on this insight, we propose a multi-modal method that effectively denoises the heartbeat waveform measurements and further accurately extracts heart rate. We implement CLEARAPG on ANC earbuds and conduct comprehensive field studies on 14 users. The results show that CLEAR-APG achieves an average heart rate error of 4.01% across seven different activities, satisfying industry-required margin of 10% heart rate error.

Event cameras are increasingly integrated into SLAM systems to mitigate motion blur and enhance tracking performance in low-light environments. However, the event stream by nature lacks essential semantic and color information, posing a fundamental challenge to visual feature extraction and mapping. This paper introduces Ev-SLAM, a novel event-based visual SLAM system that harnesses implicit neural 3D scene representation to enhance visual feature extraction and SLAM performance. We design three modules to achieve this goal, including (1) a dynamic-slicing algorithm to address the impact of polarity neutralization in event frame generation; (2) a point cloud representation that minimizes redundant sampling in free space, significantly reducing the computational overhead of neural radiance field (NeRF) training; and (3) an optical-flow-based approach for coarse camera pose and depth estimation, coupled with a scale feedback mechanism to bootstrap NeRF training. Extensive evaluations demonstrate that Ev-SLAM outperforms SOTA baselines Point-SLAM and Co-SLAM, achieving up to an 82.9% improvement in tracking accuracy under normal-speed conditions and 85.2% under fast-speed conditions.

We present LeakyFeeder, a mobile application that explores the acoustic signals leaked from headphones to reconstruct gesture motions around the ear for fine-grained gesture control. To achieve this goal, LeakyFeeder repurposes the speaker and a single feedforward microphone on active noise cancellation (ANC) headphones as a SONAR system, using inaudible frequency-modulated continuous-wave (FMCW) signals to track gesture reflections for accurate sensing. Since this single-receiver SONAR system is unable to differentiate reflection angles and further disentangle signal reflections from different gesture parts, we draw on principles of multi-modal learning to frame gesture motion reconstruction as a multi-modal translation task and propose a deep learning-based approach to fill the information gap between low-dimensional FMCW ranging readings and high-dimensional 3D hand movements. We implement LeakyFeeder on a pair of Google Pixel Buds and conduct experiments to examine the efficacy and robustness of LeakyFeeder in various conditions. Experiments based on six gesture types inspired by Apple Vision Pro demonstrate that LeakyFeeder achieves a PCK performance of 89% at 3cm across ten users, with an average MPJPE and MPJRPE error of 2.71cm and 1.88cm, respectively.

This paper explores sensor-informed personal agents that can take advantage of sensor hints on wearables to enhance the personal agent's response. We demonstrate that such a sensor-in-the-loop AI agent design can be easily integrated into existing LLM agents by building a prototype named WellMax based on existing well-developed techniques such as structured prompt templates and few-shot prompting. The head-to-head comparison with a non-sensor-informed agent across five use scenarios demonstrates that this sensor-in-the-loop design can effectively improve users' needs and their overall experience. The deep-dive into agents' replies and participants' feedback further reveals that sensor-in-the-loop agents not only provide more contextually relevant responses but also exhibit a better understanding of user priorities and situational nuances. In addition, we conduct two case studies to examine the potential pitfalls and distill key insights from this sensor-in-the-loop agent. We hope this work can spawn new ideas for building more intelligent, empathetic, and effective AI-driven personal assistants.

As the size of Deep Neural Networks (DNNs) continues to grow, their runtime latency also scales. While model pruning and Neural Architecture Search (NAS) can effectively reduce the computation workload, their effectiveness fails to consistently translate into runtime latency reduction. In this paper, we identify the root cause behind the mismatch between workload reduction and latency reduction is GPU tail effect – a classic system issue caused by resource under-utilization in the last processing wave of the GPU. We conduct detailed DNN workload characterization and demonstrate the prevalence of GPU tail effect across different DNN architectures, and meanwhile reveal that the unique deep structure and the light-weight layer workload of DNNs exacerbate the tail effect for DNN inference. We then propose a tail-awareness design space enhancement and DNN optimization algorithm to optimize existing NAS and pruning designs and achieve better runtime latency and model accuracy performance. Extensive experiments show 11%-27% latency reduction over SOTA DNN pruning and NAS methods.

Whilesmalldrone video streaming systems create unprecedented video content, they also place a power burden exceeding 20% on the drone's battery, limiting flight endurance. We present${\sf Mighty}$, a hardware-software solution to minimize the power consumption of a drone's video streaming system by offloading power overheads associated with both video compression and transmission to a ground controller.${\sf Mighty}$innovates a high performance co-design among:(1)a ring oscillator-based, ultra-low power backscatter radio;(2)a spectrally-efficient, non-linear, low-power physical layer modulation and multi-chain radio architecture; and(3)a lightweight video compression codec-bypassing software design. Our co-design exploits synergies among these components, resulting in joint throughput and range performance that pushes the known envelope. We prototype${\sf Mighty}$on PCB board and conduct extensive field studies both indoors and outdoors. The power efficiency of${\sf Mighty}$is about 16.6 nJ/bit. A head-to-head comparison with aDJI Mini2drone's default video streaming system shows that${\sf Mighty}$achieves similar throughput at a drone-to-controller distance of up to 150 meters, with 34–55× improvement of power efficiency than WiFi-based video streaming solutions.

Fast and accurate obstacle avoidance is crucial to drone safety. Yet existing on-board sensor modules such as frame cameras and radars are ill-suited for doing so due to their low temporal resolution or limited field of view. This paper presentsBioDrone, a new design paradigm for drone obstacle avoidance using stereo event cameras. At the heart of BioDrone are three simple yet effective system designs inspired by the mammalian visual system, namely, a chiasm-inspired event filtering, a lateral geniculate nucleus (LGN)-inspired event matching, and a dorsal stream-inspired obstacle tracking. We implement BioDrone on FPGA through software-hardware co-design and deploy it on an industrial drone. In comparative experiments against two state-of-the-art event-based systems, BioDrone consistently achieves an obstacle detection rate of$> $90%, and an obstacle tracking error of$<$5.8 cm across all flight modes with an end-to-end latency of$<$6.4 ms, outperforming both baselines by over 44%.

We present MAF, a novel acoustic sensing approach that leverages the commodity hardware in bone conduction earphones for hand-to-face gesture interactions. Briefly, by shining audio signals with bone conduction earphones, we observe that these signals not only propagate along the surface of the human face but also dissipate into the air, creating an acoustic field that envelops the individual’s head. We conduct benchmark studies to understand how various hand-to-face gestures and human factors influence this acoustic field. Building on the insights gained from these initial studies, we then propose a deep neural network combined with signal preprocessing techniques. This combination empowers MAF to effectively detect, segment, and subsequently recognize a variety of hand-to-face gestures, whether in close contact with the face or above it. Our comprehensive evaluation based on 22 participants demonstrates that MAF achieves an average gesture recognition accuracy of 92% across ten different gestures tailored to users’ preferences.

Edge-assisted visual SLAM stands as a pivotal enabler for emerging mobile applications, such as search-and-rescue, smart logistics, and industrial inspection. Limited by the computing capability of lightweight mobile devices like MAVs, current innovations balance system accuracy and efficiency by allocating lightweight and time-sensitive tracking tasks to mobile devices, while offloading the more resource-intensive yet delay-tolerant map optimization tasks to the edge. However, our pilot study in a large-scale oil field reveals several limitations of such a tracking-optimization decoupled paradigm, arising due to the disruption of inter-dependencies between the two tasks concerning data, resources, and threads.In this paper, we design and implement edgeSLAM2, an innovative system that reshapes the edge-assisted visual SLAM paradigm by tightly integrating tracking and partial-yet-crucial optimization on mobile. edgeSLAM2 harnesses the hierarchical and heterogeneous computing units offered by the latest commercial systems-on-chip (SoCs) to enhance the computational capacity of mobile devices, which in turn, allows edgeSLAM2 to design a suit of novel algorithms for map sync, optimization, and tracking that accommodate such architectural upgrade. By fully embracing the on-chip intelligence, edgeSLAM2 simultaneously enhances system accuracy and efficiency through software-hardware co-design. We deploy edgeSLAM2 on an industrial drone and conduct comprehensive experiments in a large-scale oil field over three months. The results show that edgeSLAM2 surpasses comparative methods by achieving an 80% reduction in bandwidth consumption, a 32% improvement in accuracy, and a 26% reduction in tracking delay.

The elderly over 65 accounts for 80% of COVID deaths in the United States. In response to the pandemic, the federal, state governments, and commercial insurers are promoting video visits, through which the elderly can access specialists at home over the Internet, without the risk of COVID exposure. However, the current video visit practice barely relies on video observation and talking. The specialist could not assess the patient's health conditions by performing auscultations.

This paper presents the design, implementation, and evaluation of MagWear, a novel biomagnetism-based system that can accurately and inclusively monitor the heart rate and respiration rate of mobile users with diverse skin tones. MagWear's contributions are twofold. Firstly, we build a mathematical model that characterizes the magnetic coupling effect of blood flow under the influence of an external magnetic field. This model uncovers the variations in accuracy when monitoring vital signs among individuals. Secondly, leveraging insights derived from this mathematical model, we present a softwarehardware co-design that effectively handles the impact of human diversity on the performance of vital sign monitoring, pushing this generic solution one big step closer to real adoptions. We have implemented a prototype of MagWear on a two-layer PCB board and followed IRB protocols to conduct system evaluations. Our extensive experiments involving 30 volunteers demonstrate that MagWear achieves high monitoring accuracy with a mean percentage error (MPE) of 1.55% for heart rate and 1.79% for respiration rate. The head-to-head comparison with Apple Watch 8 further demonstrates MagWear's consistently high performance in different user conditions.

Constructing precise 3D maps is crucial for the development of future map-based systems such as self-driving and navigation. However, generating these maps in complex environments, such as multi-level parking garages or shopping malls, remains a formidable challenge. In this paper, we introduce a participatory sensing approach that delegates map-building tasks to map users, thereby enabling cost-effective and continuous data collection. The proposed method harnesses the collective efforts of users, facilitating the expansion and ongoing update of the maps as the environment evolves.

We present the design and implementation of EarVoice, a lightweight mobile service that enables hands-free voice assistant activation on commodity earphones. EarVoice comprises two design modules: one for joint speech detection and primary user identification that explores the attributes of the air channel and in-body audio pathway to differentiate between the primary user and others nearby; and another for accurate wakeup word enhancement, which employs a "copy, paste, and adapt" approach to reconstruct the missing high-frequency component in speech recordings. To minimize false positives, enhance agility, and preserve privacy, we deploy EarVoice on a dongle where the proposed signal processing algorithms are streamlined with a gating mechanism to permit only the primary user's speech to enter the pairing device (e.g., a smartphone) for wakeup word recognition, preventing unintended disclosure of ambient conversations. We implemented the dongle on a 4-layer PCB board and conducted extensive experiments with 23 participants in both controlled and uncontrolled scenarios. The experiment results show that EarVoice achieves around 90% wakeup word recognition accuracy in stationary scenarios, which is on par with the high-end, multi-sensor fusion-based Airpods Pro earbud. EarVoice's performance drops to 84% on mobile cases, slightly worse than Airpods (around 90%).

Edge-assisted visual SLAM plays a crucial role in enabling innovative mobile applications, such as autonomous swarm inspection, search-and-rescue, and smart logistics. Constrained by the computational capacities of lightweight mobile devices, current approaches delegate lightweight, time-sensitive tracking tasks to the mobile end while offloading resource-intensive, latency-tolerant map optimization tasks to the edge. However, our pilot study reveals several limitations of the tracking-optimization decoupled paradigm, stemming from the disruption of inter-dependencies between the two tasks. In this paper, we design and implement edgeSLAM2, an innovative system that reshapes the edge-assisted visual SLAM paradigm by tightly integrating tracking and partial-yet-crucial optimization on mobile. edgeSLAM2 harnesses the heterogeneous computing units offered by the commercial systems-on-chip (SoCs) to enhance the computational capacity of mobile devices, which in turn, allows edgeSLAM2 to design a suit of novel algorithms for map sync, optimization, and tracking that accommodate such architectural upgrade. By capitalizing on the full potential of on-chip intelligence, edgeSLAM2 supports both solitary and collaborative SLAM with accuracy and immediacy, underpinned by a cohesive software-hardware co-design. We deploy edgeSLAM2 on drones for industrial inspection. Comprehensive experiments in one of the world’s largest oil fields over three months demonstrate its superior performance.

The edge-based multi-agent visual SLAM is crucial for emerging mobile applications like search-and-rescue, inventory automation, and industrial inspection. It uses a central node to manage the global map and schedule tasks for agents. However, as the number of agents increases, the system faces scalability challenges due to operational overhead, such as data redundancy, bandwidth consumption, and localization errors. In this paper, we introduce, a framework designed to enhance the scalability of collaborative visual SLAM service in edge offloading settings. consists of three system modules: a change log-based server-client synchronization mechanism, a priority-aware task scheduler, and a lean global map representation. These modules work together to address the challenges of data explosion problems. is open-source and compatible with the robotic operating system (ROS). Existing visual SLAM applications could incorporate through SwarmAPI, a set of well-packaged APIs, to compose SwarmMap’s function modules to enhance their performance and capacity in multi-agent scenarios. Comprehensive evaluations and a three-month case study at one of the world’s largest oilfields demonstrate that can serve 2$\times$more agents ($>$20 agents) than the state-of-the-arts with the same resource overhead, meanwhile maintaining an average trajectory error of 38$cm$, outperforming existing works by$>$55%.

The radio range of backscatter systems continues growing as new wireless communication primitives are continuously invented. Nevertheless, both the bit error rate and the packet loss rate of backscatter signals increase rapidly with the radio range, thereby necessitating the cooperation between the access point and the backscatter tags through a feedback loop. Unfortunately, the low-power nature of backscatter tags limits their ability to demodulate feedback signals from a remote access point and scales down to such circumstances. This paper presents, an ultra-low-power demodulator for long-range LoRa backscatter systems. is based on an observation that a frequency-modulated chirp signal can be transformed into an amplitude-modulated signal using a differential circuit. Moreover, we redesign a LoRa backscatter tag which integrates and a ring oscillator-based modulator. The LoRa backscatter tag enables re-transmission, rate adaption and channel hopping – three PHY-layer operations that are important to channel efficiency yet unavailable on existing long-range backscatter systems. We prototype and the LoRa backscatter tag on two PCB boards and evaluate their performance in different environments. Results show that achieves 3.5–5$\times$gain on the demodulation range, compared with state-of-the-art systems. Our ASIC simulation shows that the power consumption of and the LoRa backscatter tag are around 93.2$\mu W$and 94.7$\mu W$. Code and hardware schematics can be found at: https://github.com/ZangJac/Saiyan.

This paper presents RF-Transformer, a unified backscatter radio hardware abstraction that allows a low-power IoT device to directly communicate with heterogeneous wireless receivers. Unlike existing backscatter systems that are tailored to a specific wireless communication protocol, RF-Transformer provides a programmable interface to the micro-controller, allowing IoT devices to synthesize different types of protocol-compliant backscatter signals in the PHY layer. By leveraging the nonlinear characteristics of the negative impedance, RF-Transformer also achieves a cross-frequency backscatter design that enables IoT devices in harmonic frequency bands to communicate with each other. We implement a PCB prototype of RF-Transformer on 2.4 GHz ISM band and conduct extensive experiments. We leverage the software defined platform USRP to transmit the carrier signal and receive the backscatter signal to verify the efficacy of our design. Our extensive field studies show that RF-Transformer achieves 23.8 Mbps, 247.1 Kbps, 986.5 Kbps, and 27.3 Kbps throughput when generating standard Wi-Fi, ZigBee, Bluetooth, and LoRa signals.

Low-power wide-area networks (LPWAN) evolve rapidly with advanced communication primitives (e.g., coding, modulation) being continuously invented. This rapid iteration on LPWAN, however, forms a communication barrier between legacy wireless sensor nodes deployed years ago (e.g., ZigBee-based sensor node) with their latest competitor running a different communication protocol (e.g., LoRa-based IoT node): they work on the same frequency band but share different MAC- and PHY-layer regulations and thus cannot talk to each other directly. To break this barrier, we propose LigBee, a cross-technology communication (CTC) solution that enables symbol-level communication from the latest LPWAN LoRa node to legacy ZIGBEE node. We have implemented LigBee on both software-defined radios and commercial-off-the-shelf (COTS) LoRa and ZigBee nodes, and demonstrated that LigBee builds a reliable CTC link from LoRa node to ZigBee node on both platforms. Our experimental results show that i) LigBee achieves a bit error rate (BER) in the order of 10−3with 70 ∼ 80% frame reception ratio (FRR), ii) the range of LigBee link is over 300m, which is 6 ∼ 7.5× the typical range of legacy ZigBee and state-of-the-art solution, and iii) the throughput of LigBee link is maintained on the order of kbps, which is close to the LoRa’s throughput.

This paper presents Audioplethysmography (APG), a novel cardiac monitoring modality for active noise cancellation (ANC) headphones. APG sends a low intensity ultrasound probing signal using an ANC headphone's speakers and receives the echoes via the on-board feedback microphones. We observed that, as the volume of ear canals slightly changes with blood vessel deformations, the heartbeats will modulate these ultrasound echoes. We built mathematical models to analyze the underlying physics and propose a multi-tone APG signal processing pipeline to derive the heart rate and heart rate variability in both constrained and unconstrained settings. APG enables robust monitoring of cardiac activities using mass-market ANC headphones in the presence of music playback and body motion such as running.

This paper tries to answer a question: "Can we achieve spatial-selective transmission on IoT devices?" A positive answer would enable more secure data transmission among IoT devices. The challenge, however, is how to manipulate signal propagation without relying on beamforming antenna arrays which are usually unavailable on low-end IoT devices.

Fast and accurate obstacle avoidance is crucial to drone safety. Yet existing on-board sensor modules such as frame cameras and radars are ill-suited for doing so due to their low temporal resolution or limited field of view. This paper presents BioDrone, a new design paradigm for drone obstacle avoidance using stereo event cameras. At the heart of BioDrone is two simple yet effective system design inspired by the mammalian visual system, namely, a chiasm-inspired signal processing pipeline for fast event filtering and obstacle detection, and a lateral geniculate nucleus (LGN)-inspired event matching algorithm for accurate obstacle localization. To make BioDrone a practical solution, we further take significant engineering efforts to deploy the software stack on FPGA through software and hardware co-design. The performance comparison with two state-of-the-art event-based obstacle avoidance systems shows BioDrone achieves a consistently high obstacle detection rate of 96.1%. The average localization error of BioDrone is 6.8cm with a 4.7ms latency, outperforming both baselines by over 40%.

Localization and navigation play a key role in many location-based services and have attracted numerous research efforts. In recent years, visual SLAM has been prevailing for autonomous driving. However, the ever-growing computation resources demanded by SLAM impede its applications to resource-constrained mobile devices. In this paper, we present the design, implementation, and evaluation ofedgeSLAM, an edge-assisted real-time semantic visual SLAM service running on mobile devices.edgeSLAMleverages the state-of-the-art semantic segmentation algorithm to enhance localization and mapping accuracy, and speeds up the computation-intensive SLAM and semantic segmentation algorithms by computation offloading. The key innovations ofedgeSLAMinclude an efficient computation offloading strategy, an opportunistic data sharing method, an adaptive task scheduling algorithm, and a multi-user support mechanism. We fully implementedgeSLAMand plan to open-source it. Extensive experiments are conducted under 3 datasets. The results show thatedgeSLAMcan run on mobile devices at 35fps and achieve 5cm localization accuracy from real-world experiments, outperforming existing solutions by more than 15%. We also demonstrate the usability ofedgeSLAMthrough 2 case studies of pedestrian localization and robot navigation. To the best of our knowledge,edgeSLAMis the first edge-assisted real-time semantic visual SLAM for mobile devices.

This article presents SoundSticker, a system for steganographic, in-band data communication over an acoustic channel. In contrast with recent works that hide bits in inaudible frequency bands, SoundSticker embeds hidden bits in the audible sounds, making them more reliably survive audio codecs and bandpass filtering, while achieving a higher data rate and remaining imperceptible to a listener. The key observation behind SoundSticker is that the human ear is less sensitive to the audio phase changes than the frequency and amplitude changes, which leaves us an opportunity to alter the phase of an audio clip to convey hidden information. We take advantage of this opportunity and build an OFDM-based physical layer. To make this PHY-layer design work for a variety of end devices with heterogeneous computation resources, SoundSticker addresses multiple technical challenges including perceivable waveform artifacts caused by the phase-based modulation, bit rate adaptation without channel sounding and real-time preamble detection. Our prototype on both smartphones and ESP32 platforms demonstrates SoundSticker’s superior performance against the state of the arts, while preserving excellent sound quality and remaining unaffected by common audio codecs like MP3 and AAC. Audio clips produced by SoundSticker can be found at https://soundsticker.github.io/ .

This paper presents RF-Transformer, a unified backscatter radio hardware abstraction that allows a low-power IoT device to directly communicate with heterogeneous wireless receivers at the minimum power consumption. Unlike existing backscatter systems that are tailored to a specific wireless communication protocol, RF-Transformer provides a programmable interface to the micro-controller, allowing IoT devices to synthesize different types of protocol-compliant backscatter signals sharing radically different PHY-layer designs. To show the efficacy of our design, we implement a PCB prototype of RF-Transformer on 2.4 GHz ISM band and showcase its capability on generating standard ZigBee, Bluetooth, LoRa, and Wi-Fi 802.11b/g/n/ac packets. Our extensive field studies show that RF-Transformer achieves 23.8 Mbps, 247.1 Kbps, 986.5 Kbps, and 27.3 Kbps throughput when generating standard Wi-Fi, ZigBee, Bluetooth, and LoRa signals while consuming 7.6--74.2X less power than their active counterparts. Our ASIC simulation based on the 65-nm CMOS process shows that the power gain of RF-Transformer can further grow to 92--678X. We further integrate RF-Transformer with pressure sensors and present a case study on detecting foot traffic density in hallways. Our 7-day case studies demonstrate RF-Transformer can reliably transmit sensor data to a commodity gateway by synthesizing LoRa packets on top of Wi-Fi signals. Our experimental results also verify the compatibility of RF-Transformer with commodity receivers. Code and hardware schematics can be found at: https://github.com/LeFsCC/RF-Transformer.

Virtual visits (a.k.a., telehealth) have been promoted in response to the COVID pandemic since early 2020. Despite its convenience, the current virtual visit practice barely relies on video observation and talking. The specialist, however, cannot accurately assess the patient's health condition by listening to acoustic cardiopulmonary signals emanating from the patient's heart with a stethoscope. In this poster, we explore the feasibility of remote auscultation in virtual visits settings by reusing the patient's earphones as a stethoscope. The proposed hardware-software system captures the minute heartbeats from the patient's ear canal. It then offloads these noisy cardiac signals to the pairing device (e.g., a smartphone or a laptop) to reconstruct fine-grained Phonocardiogram (PCG) signals. By listening to the reconstructed PCG signals, the specialist can easily assess the patient's health condition and make the most informed diagnosis. We describe the design challenges and explain our technical roadmap.

Earables are embedded devices that can be placed in, on, or around the ear to sense human motions and physiological activities over an extended period of time. However, today's earable design principle heavily relies on dedicated sensors (e.g., accelerometer, gyroscope, proximity sensor), which inevitably adds cost, weight, and power consumption to earable devices, constituting a critical bottleneck in their wide adoption. Moreover, the tight coupling of sensors with onboard microcontrollers makes existing earables difficult to program, raising the barrier of entry to earable computing.

Backscatter communication holds potential for ubiquitous and low-cost connectivity among low-power IoT devices. To avoid interference between the carrier signal and the backscatter signal, recent works propose a frequency-shifting technique to separate these two signals in the frequency domain. Such proposals, however, have to occupy the precious wireless spectrum that is already overcrowded, and increase the power, cost, and complexity of the backscatter tag. In this paper, we revisit the classic ON-OFF Keying (OOK) modulation and propose Aloba, a backscatter system that takes the ambient LoRa transmissions as the excitation and piggybacks the in- band OOK modulated signals over the LoRa transmissions. Our design enables the backscatter signal to work in the same frequency band of the carrier signal, meanwhile achieving flexible data rate at different transmission range. The key contributions of Aloba include: i) the design of a low-power backscatter tag that can pick up the ambient LoRa signals from other signals; ii) a novel decoding algorithm to demodulate both the carrier signal and the backscatter signal from their superposition. We further adopt link coding mechanism and interleave operation to enhance the reliability of backscatter signal decoding. We implement Aloba and conduct head-to-head comparison with the state-of-the-art LoRa backscatter system PLoRa in various settings. The experiment results show Aloba can achieve 39.5–199.4 Kbps data rate at various distances, 10.4–$52.4\times $higher than PLoRa.

With the fast development of deep neural networks (DNNs), many real-world applications are adopting multiple models to conduct compound tasks, such as co-running classification, detection, and segmentation models on autonomous vehicles. Such multi-tenant DNN inference cases greatly exacerbate the computational complexity and call for comprehensive collaboration for graph-level operator scheduling, runtime-level resource awareness, as well as hardware scheduler support. However, the current scheduling support for such multi-tenant inference is still relatively backward. In this work, we propose a resource-aware scheduling framework for efficient multi-tenant DNN inference on GPU, which automatically coordinates DNN computing in different execution levels. Leveraging the unified scheduling intermediate representation and the automated ML-based searching algorithm, optimal schedules could be generated to wisely adjust model concurrency and interleave DNN model operators, maintaining a continuously balanced resource utilization across the entire inference process, and eventually improving the runtime efficiency. Experiments show that we could consistently achieve$1.3\times\sim 1.7\times$speed-up, comparing to regular DNN runtime libraries (e.g., CuDNN, TVM) and particular concurrent scheduling methods (e.g., NVIDIA Multi-Stream).

Headphones continue to become more intelligent as new functions (e.g., touch-based gesture control) appear. These functions usually rely on auxiliary sensors (e.g., accelerometer and gyroscope) that are available in smart headphones. However, for those headphones that do not have such sensors, supporting these functions becomes a daunting task. This paper presents HeadFi, a new design paradigm for bringing intelligence to headphones. Instead of adding auxiliary sensors into headphones, HeadFi turns the pair of drivers that are readily available inside all headphones into a versatile sensor to enable new applications spanning across mobile health, user-interface, and context-awareness. HeadFi works as a plug-in peripheral connecting the headphones and the pairing device (e.g., a smartphone). The simplicity (can be as simple as only two resistors) and small form factor of this design lend itself to be embedded into the pairing device as an integrated circuit. We envision HeadFi can serve as a vital supplementary solution to existing smart headphone design by directly transforming large amounts of existing "dumb" headphones into intelligent ones. We prototype HeadFi on PCB and conduct extensive experiments with 53 volunteers using 54 pairs of non-smart headphones under the institutional review board (IRB) protocols. The results show that HeadFi can achieve 97.2%--99.5% accuracy on user identification, 96.8%--99.2% accuracy on heart rate monitoring, and 97.7%--99.3% accuracy on gesture recognition.

Presents corrections to the acknowledgement section for the above named article.

Visual peer-to-peer navigation is a suitable solution for indoor navigation for it relieves the labor of site-survey and eliminates infrastructure dependence. However, a major drawback hampers its application, as the peer-to-peer mode suffers from a deficiency of paths in large indoor scenarios with multifarious places-of-interest. Nevertheless, we propose one with a profound crowdsourcing scheme that addresses the drawback by merging the paths of different leaders' into a global map. To realize the idea, we further deal with entailed challenges, namely the unidirectional disadvantage, the scale ambiguity, and large computational overhead. We design a navigation strategy to solve the unidirectional problem and turn to VIO to tackle scale ambiguity. We devise a mobile-edge architecture to enable real-time navigation (30fps, 100ms end-to-end delay) and lighten the burden of smartphones (35% battery life for 2h35min) while assuring the accuracy of localization and map construction. Through experimental validations, we show that P2P navigation, previously relying on the abundance of independent paths, can enjoy a sufficiency of navigation paths with a crowdsourced global map. The experiments demonstrate a navigation success rate of 100% and spatial offset of less than 3.2m, better than existing works.

Localization and navigation play a key role in many location-based services and have attracted numerous research efforts from both academic and industrial community. In recent years, visual SLAM has been prevailing for robots and autonomous driving cars. However, the ever-growing computation resource demanded by SLAM impedes its application to resource-constrained mobile devices. In this paper we present the design, implementation, and evaluation of edgeSLAM, an edge assisted real-time semantic visual SLAM service running on mobile devices. edgeSLAM leverages the state-of-the-art semantic segmentation algorithm to enhance localization and mapping accuracy, and speeds up the computation-intensive SLAM and semantic segmentation algorithms by computation offloading. The key innovations of edgeSLAM include an efficient computation offloading strategy, an opportunistic data sharing mechanism, and an adaptive task scheduling algorithm. We fully implement edgeSLAM on an edge server and different types of mobile devices (2 types of smartphones and a development board). Extensive experiments are conducted under 3 data sets, and the results show that edgeSLAM is able to run on mobile devices at 35fps frame rate and achieves a 5cm localization accuracy, outperforming existing solutions by more than 15%. We also demonstrate the usability of edgeSLAM through 2 case studies of pedestrian localization and robot navigation. To the best of our knowledge, edgeSLAM is the first real-time semantic visual SLAM for mobile devices.

This paper presents the design, implementation and evaluation of In-N-Out, a software-hardware solution for far-field wireless power transfer. In-N-Out can continuously charge a medical implant residing in deep tissues at near-optimal beamforming power, even when the implant moves around inside the human body. To accomplish this, we exploit the unique energy ball pattern of distributed antenna array and devise a backscatter-assisted beamforming algorithm that can concentrate RF energy on a tiny spot surrounding the medical implant. Meanwhile, the power levels on other body parts stay in low level, reducing the risk of overheating. We proto-type In-N-Out on 21 software-defined radios and a printed circuit board (PCB). Extensive experiments demonstrate that In-N-Out achieves 0.37 mW average charging power inside a 10 cm-thick pork belly, which is sufficient to wirelessly power a range of commercial medical devices. Our head-to-head comparison with the state-of-the-art approach shows that In-N-Out achieves 5.4X-18.1X power gain when the implant is stationary, and 5.3X-7.4X power gain when the implant is in motion.

Backscatter communication holds potential for ubiquitous and low-cost connectivity among low-power IoT devices. To avoid interference between the carrier signal and the backscatter signal, recent works propose a frequency-shifting technique to separate these two signals in the frequency domain. Such proposals, however, have to occupy the precious wireless spectrum that is already overcrowded, and increase the power, cost, and complexity of the backscatter tag. In this paper, we revisit the classic ON-OFF Keying (OOK) modulation and propose Aloba, a backscatter system that takes the ambient LoRa transmissions as the excitation and piggybacks the in-band OOK modulated signals over the LoRa transmissions. Our design enables the backsactter signal to work in the same frequency band of the carrier signal, meanwhile achieving good tradeoff between transmission range and link throughput. The key contributions of Aloba include: i) the design of a low-power backscatter tag that can pick up the ambient LoRa signals from other signals; ii) a novel decoding algorithm to demodulate both the carrier signal and the backscatter signal from their superposition. The design of Aloba completely unleashes the backscatter tag's ability in OOK modulation and achieves flexible data rate at different transmission range. We implement Aloba and conduct head-to-head comparison with the state-of-the-art LoRa backscatter system PLoRa in various settings. The experiment results show Aloba can achieve 39.5--199.4 Kbps data rate at various distances, 10.4--52.4X higher than PLoRa.

Passive Radio Frequency Identification (RFID) tags have been widely applied in many applications, such as logistics, retailing, and warehousing. In many situations, the order of objects is more important than their absolute locations. However, state-of-art ordering methods need a continuing movement of tags and readers, which limit the application domain and scalability. In this paper, we propose a 2-dimension ordering approach for passive tags that requires no device movement. Instead, our method utilizes signal changes caused by arbitrary movement of human beings around tags, who carry no device for horizontal dimension ordering. Hence, our method is called Human Movement based Ordering (HMO). The basic idea of HMO is that when people pass between the reader antenna and tags, the received signal strength will change. By observing the time-series RSS changes of tags, HMO can obtain the order of tags along with a specific horizontal direction. For vertical dimension, we employ a linear programming method that is tolerant of tiny errors in practice. We implement HMO with commodity off-the-shelf RFID devices. The experimental results show that HMO can achieve up to 88.71 and 90.86 percent average accuracies in the signal-and multi-person cases, respectively.

Low-power wide-area networks (LPWANs) are a compelling answer to the networking challenges faced by many Internet of Things devices. Their combination of low power, long range, and deployment ease has motivated a flurry of research, including exciting results on backscatter and interference cancellation that further lower power budgets and increase capacity. But despite the interest, we argue that unlicensed LPWAN technologies can only serve a narrow class of Internet of Things applications due to two principal challenges: capacity and coexistence. We propose a metric, bit flux, to describe networks and applications in terms of throughput over a coverage area. Using bit flux, we find that the combination of low bit rate and long range restricts the use case of LPWANs to sparse sensing applications. Furthermore, this lack of capacity leads networks to use as much available bandwidth as possible, and a lack of coexistence mechanisms causes poor performance in the presence of multiple, independently-administered networks. We discuss a variety of techniques and approaches that could be used to address these two challenges and enable LPWANs to achieve the promise of ubiquitous connectivity.

This paper presents PLoRa, an ambient backscatter design that enables long-range wireless connectivity for batteryless IoT devices. PLoRa takes ambient LoRa transmissions as the excitation signals, conveys data by modulating an excitation signal into a new standard LoRa "chirp" signal, and shifts this new signal to a different LoRa channel to be received at a gateway faraway. PLoRa achieves this by a holistic RF front-end hardware and software design, including a low-power packet detection circuit, a blind chirp modulation algorithm and a low-power energy management circuit. To form a complete ambient LoRa backscatter network, we integrate a light-weight backscatter signal decoding algorithm with a MAC-layer protocol that work together to make coexistence of PLoRa tags and active LoRa nodes possible in the network. We prototype PLoRa on a four-layer printed circuit board, and test it in various outdoor and indoor environments. Our experimental results demonstrate that our prototype PCB PLoRa tag can backscatter an ambient LoRa transmission sent from a nearby LoRa node (20 cm away) to a gateway up to 1.1 km away, and deliver 284 bytes data every 24 minutes indoors, or every 17 minutes outdoors. We also simulate a 28-nm low-power FPGA based prototype whose digital baseband processor achieves 220 μW power consumption.

Hand gesture is becoming an increasingly popular means of interacting with consumer electronic devices, such as mobile phones, tablets and laptops. In this paper, we present AudioGest, a device-free gesture recognition system that can accurately sense the hand in-air movement around user's devices. Compared to the state-of-the-art techniques, AudioGest is superior in using only one pair of built-in speaker and microphone, without any extra hardware or infrastructure support and with no training, to achieve multimodal hand detection. Specifically, our system is not only able to accurately recognize various hand gestures, but also reliably estimate the hand in-air duration, average moving speed and waving range. We achieve this by transforming the device into an active sonar system that transmits inaudible audio signal and decodes the echoes of hand's movement at its microphone. We address various challenges including cleaning the noisy reflected sound signal, interpreting the echo spectrogram into hand gestures, decoding the Doppler frequency shifts into the hand waving speed and range, as well as being robust to the environmental motion and signal drifting. We extensively evaluate our system on three electronic devices under four real-world scenarios using overall 3,900 hand gestures collected by five users for more than two weeks. Our results show that AudioGest detects six hand gestures with an accuracy up to 96 percent. By distinguishing the gesture attributions, it can provide more fine-grained control commands for various applications.

Smart spaces, such as smart homes and smart offices, are common Internet of Things (IoT) scenarios for building automation with networked sensors. In this paper, we suggest a different notion of smart spaces, where the radio environment is programmable to achieve desirable link quality within the space. We envision deploying low-cost devices embedded in the walls of a building to passively reflect or actively transmit radio signals. This is a significant departure from typical approaches to optimizing endpoint radios and individual links to improve performance. In contrast to previous work combating or leveraging per-link multipath fading, we actively reconfigure the multipath propagation. We sketch design and implementation directions for such a programmable radio environment, highlighting the computational and operational challenges our architecture faces. Preliminary experiments demonstrate the efficacy of using passive elements to change the wireless channel, shifting frequency "nulls" by nine Wi-Fi subcarriers, changing the 2 x 2 MIMO channel condition number by 1.5 dB, and attenuating or enhancing signal strength by up to 26 dB.

Increasing numbers of everyday objects in libraries, stores and warehouses are instrumented with passive RFID tags, resulting in a ripe opportunity for gesture-based interactions with people. By a simple act of picking up and gesturing with an RFID-tagged object, users can send their opinions and sentiments about that object to the cloud. Prior work in RFID-based gesture tracking relies on multiple bulky and expensive antennas and readers to function, which incurs unacceptable infrastructure costs for large-scale ubiquitous deployment (over an entire warehouse or mall, for example) thus hindering practical adoption. In this paper, we propose Pantomime, the first RFID-based gesture recognition system that uses just a single antenna per geographical area of coverage. Our key insight is to replace the conventional multiple antenna single tag tracking framework with an equivalent multiple tag single antenna system. Through a novel tag coordination protocol and a lightweight tracking algorithm, Pantomime enables accurate gesture tracking that works for objects tagged with just two RFID tags. We implement a real-time prototype of Pantomime with commercial off-the-shelf (COTS) RFID readers and antennas. Extensive evaluations and real-world case studies in a classroom and a retail store demonstrate that Pantomime achieves comparable gesture tracking accuracy (87%) to state-of-the-art multi-antenna schemes (88%) at a minimal deployment cost.

Passive Radio Frequency Identification (RFID) tags have been widely applied in many applications, such as logistics, retailing, and warehousing. In many situations the relative locations of objects are more important than their absolute locations. However, state-of-art relative localization methods need continuing movement of tags and readers, which limit the application domain and scalability. In this paper, we propose a relative localization approach for passive tags that requires no device movement. Instead, our method utilizes signal changes caused by arbitrary movement of human beings around tags, who carry no device. Hence our method is called Human Movement based Relative Localization (HMRL). The basic idea of HMRL is that when people pass between reader antenna and tags, the received signal strength will change. By observing the time-series RSS changes of tags, HMRL can obtain the order of tags along a specific horizontal direction. HMRL can also get the order of tags in a vertical direction using hyperbolic positioning. We implement HMRL with commodity off-the-shelf RFID devices. The experimental results show that HMRL achieves high accuracy for relative localization of passive tags.

This paper presents the design and implementation of Wi-Fi Goes to Town, the first Wi-Fi based roadside hotspot network designed to operate at vehicular speeds with meter-sized picocells. Wi-Fi Goes to Town APs make delivery decisions to the vehicular clients they serve at millisecond-level granularities, exploiting path diversity in roadside networks. In order to accomplish this, we introduce new buffer management algorithms that allow participating APs to manage each others' queues, rapidly quenching each others' transmissions and flushing each others' queues. We furthermore integrate our fine-grained AP selection and queue management into 802.11's frame aggregation and block acknowledgement functions, making the system effective at modern 802.11 bit rates that need frame aggregation to maintain high spectral efficiency. We have implemented our system in an eight-AP network alongside a nearby road, and evaluate its performance with mobile clients moving at up to 35 mph. Depending on the clients' speed, Wi-Fi Goes to Town achieves a 2.4-4.7x TCP throughput improvement over a baseline fast handover protocol that captures the state of the art in Wi-Fi roaming, including the recent IEEE 802.11k and 802.11r standards.

Regular free-weight exercise helps to strengthen natural movements and stabilize muscles that are important to strength, balance, and posture of human beings. Prior works have exploited wearable sensors or RF signal changes for activity sensing, recognition, and counting, etc.. However, none of them have incorporated three key factors necessary for a practical free-weight exercise monitoring system: recognizing free-weight activities on site, assessing their qualities, and providing useful feedbacks to the bodybuilder promptly. Our FEMO system provides an integrated free-weight exercise monitoring service that incorporates all the essential functionalities mentioned above. FEMO achieves this by attaching passive RFID tags on the dumbbells and leveraging the Doppler shift profile of the reflected backscatter signals for on-site free-weight activity recognition and assessment. The rationale behind FEMO is 1) since each free-weight activity owns unique arm motions, the corresponding Doppler shift profile should be distinguishable to each other. 2) Doppler profile of each activity has a strong spatial-temporal correlation that implicitly reflects the quality of the activity. We implement FEMO with COTS RFID devices and conduct a two-week experiment. The preliminary result from 15 volunteers demonstrates that FEMO can be applied to a variety of free-weight activities, and provide valuable feedbacks for activity alignment.

Many object localization applications need the relative locations of a set of objects as oppose to their absolute locations. Although many schemes for object localization using radio frequency identification (RFID) tags have been proposed, they mostly focus on absolute object localization and are not suitable for relative object localization because of large error margins and the special hardware that they require. In this paper, we propose an approach called spatial-temporal phase profiling (STPP) to RFID-based relative object localization. The basic idea of STPP is that by moving a reader over a set of tags during which the reader continuously interrogating the tags, for each tag, the reader obtains a sequence of RF phase values, which we call a phase profile, from the tag's responses over time. By analyzing the spatial-temporal dynamics in the phase profiles, STPP can calculate the spatial ordering among the tags. In comparison with prior absolute object localization schemes, STPP requires neither dedicated infrastructure nor special hardware. We implemented STPP and evaluated its performance in two real-world applications: locating misplaced books in a library and determining the baggage order in an airport. The experimental results show that STPP achieves about 84% ordering accuracy for misplaced books and 95% ordering accuracy for baggage handling. We further leverage the controllable reader antenna and upgrade STPP to infer the spacing between each pair of tags. The result shows that STPP could achieve promising performance on distance ranging.

Even though indoor smoking ban is being put into practice in civilized countries, existing vision or sensor-based smoking detection methods cannot provide ubiquitous detection service. In this paper, we take the first attempt to build a ubiquitous passive smoking detection system, Smokey, which leverages the patterns smoking leaves on WiFi signal to identify the smoking activity even in the non-line-of-sight and through-wall environments. We study the behaviors of smokers and leverage the common features to recognize the series of motions during smoking, avoiding the target-dependent training set to achieve the high accuracy. We design a foreground detection-based motion acquisition method to extract the meaningful information from multiple noisy subcarriers even influenced by posture changes. Without the requirement of target's compliance, we leverage the rhythmical patterns of smoking to detect the smoking activities. We also leverage the diversity of multiple antennas to enhance the robustness of Smokey. Due to the convenience of integrating new antennas, Smokey is scalable in practice for ubiquitous smoking detection. We prototype Smokey with the commodity WiFi infrastructure and evaluate its performance in real environments. Experimental results show Smokey is accurate and robust in various scenarios.

Shopping behavior data is of great importance in understanding the effectiveness of marketing and merchandising campaigns. Online clothing stores are capable of capturing customer shopping behavior by analyzing the click streams and customer shopping carts. Retailers with physical clothing stores, however, still lack effective methods to comprehensively identify shopping behaviors. In this paper, we show that backscatter signals of passive RFID tags can be exploited to detect and record how customers browse stores, which garments they pay attention to, and which garments they usually pair up. The intuition is that the phase readings of tags attached to items will demonstrate distinct yet stable patterns in a time-series when customers look at, pick out, or turn over desired items. We design ShopMiner, a framework that harnesses these unique spatial-temporal correlations of time-series phase readings to detect comprehensive shopping behaviors. We have implemented a prototype of ShopMiner with a COTS RFID reader and four antennas, and tested its effectiveness in two typical indoor environments. Empirical studies from two-week shopping-like data show that ShopMiner is able to identify customer shopping behaviors with high accuracy and low overhead, and is robust to interference.

Wireless sensing, tracking, and drawing technologies are enabling exciting new possibilities for human-machine interaction. They primarily rely on measurements of backscattered phase, amplitude, and Doppler signal distortions, and often require many measurements of these quantities---in time, or from multiple antennas. In this paper we present the design and implementation of PolarDraw, the first whiteboard in the air that sends differentially-polarized wireless signals to glean more precise tracking information from a tag. Leveraging information received from each polarization angle, our novel algorithms infer orientation and position of an RFID-tagged pen using just two antennas, when the user writes in the air or on a physical whiteboard. An experimental comparison in a cluttered indoor office environment compares two-antenna PolarDraw with recent state-of-the-art object tracking systems that use double the number of antennas, demonstrating comparable centimeter-level tracking accuracy and character recognition rates (88--94%), thus making a case for the use of polarization in many other tracking systems.

Hand gesture is becoming an increasingly popular means of interacting with consumer electronic devices, such as mobile phones, tablets and laptops. In this paper, we present AudioGest, a device-free gesture recognition system that can accurately sense the hand in-air movement around user's devices. Compared to the state-of-the-art, AudioGest is superior in using only one pair of built-in speaker and microphone, without any extra hardware or infrastructure support and with no training, to achieve fine-grained hand detection. Our system is able to accurately recognize various hand gestures, estimate the hand in-air time, as well as average moving speed and waving range. We achieve this by transforming the device into an active sonar system that transmits inaudible audio signal and decodes the echoes of hand at its microphone. We address various challenges including cleaning the noisy reflected sound signal, interpreting the echo spectrogram into hand gestures, decoding the Doppler frequency shifts into the hand waving speed and range, as well as being robust to the environmental motion and signal drifting. We implement the proof-of-concept prototype in three different electronic devices and extensively evaluate the system in four real-world scenarios using 3,900 hand gestures that collected by five users for more than two weeks. Our results show that AudioGest can detect six hand gestures with an accuracy up to 96%, and by distinguishing the gesture attributions, it can provide up to 162 control commands for various applications.

Indoor localization is of great importance to a wide range of applications in shopping malls, office buildings and public places. The maturity of computer vision (CV) techniques and the ubiquity of smartphone cameras hold promise for offering sub-meter accuracy localization services. However, pure CV-based solutions usually involve hundreds of photos and pre-calibration to construct image database, a labor-intensive overhead for practical deployment. We present ClickLoc, an accurate, easy-to-deploy, sensor-enriched, image-based indoor localization system. With core techniques rooted in semantic information extraction and optimization-based sensor data fusion, ClickLoc is able to bootstrap with few images. Leveraging sensor-enriched photos, ClickLoc also enables user localization with a single photo of the surrounding place of interest (POI) with high accuracy and short delay. Incorporating multi-modal localization with Manifold Alignment and Trapezoid Representation, ClickLoc not only localizes efficiently, but also provides image-assisted navigation. Extensive experiments in various environments show that the 80-percentile error is within 0.26m for POIs on the floor plan, which sheds light on sub-meter level indoor localization.

Even though indoor smoking ban is being put into practice in civilized countries, existing vision or sensor-based smoking detection methods cannot provide ubiquitous smoking detection. In this paper, we take the first attempt to build a ubiquitous passive smoking detection system, which leverages the patterns smoking leaves on WiFi signals to identify the smoking activity even in the non-line-of-sight and through-wall environments. We study the behaviors of smokers and leverage the common features to recognize the series of motions during smoking, avoiding the target-dependent training set to achieve the high accuracy. We design a foreground detection based motion acquisition method to extract the meaningful information from multiple noisy subcarriers even influenced by posture changes. Without requirements of target's compliance, we leverage the rhythmical patterns of smoking to reduce the detection false positives. We prototype Smokey with the commodity WiFi infrastructure and evaluate its performance in real environments. Experimental results show Smokey is accurate and robust in various scenarios.

In this work, we propose RAD, a RApid Deployment localization framework without human sampling. The basic idea of RAD is to automatically generate a fingerprint database through space partition, of which each cell is fingerprinted by its maximum influence APs. Based on this robust location indicator, fine-grained localization can be achieved by a discretized particle filter utilizing sensor data fusion. We devise techniques for CIVD-based field division, graph-based particle filter, EM-based individual character learning, and build a prototype that runs on commodity devices. Extensive experiments show that RAD provides a comparable performance to the state-of-the-art RSS-based methods while relieving it of prior human participation.

Libraries, manufacturing lines, and offices of the future all stand to benefit from knowing the exact spatial order of RFID-tagged books, components, and folders, respectively. To this end, radio-based localization has demonstrated the potential for high accuracy. Key enabling ideas include motion-based synthetic aperture radar, multipath detection, and the use of different frequencies (channels). But indoors in real-world situations, current systems often fall short of the mark, mainly because of the prevalence and strength of multipath reflections of the radio signal off nearby objects. In this paper we describe the design and implementation of MobiTagbot, an autonomous wheeled robot reader that conducts a roving survey of the above such areas to achieve an exact spatial order of RFID-tagged objects in very close (1--6 cm) spacings. Our approach leverages a serendipitous correlation between the changes in multipath reflections that occur with motion and the effect of changing the carrier frequency (channel) of the RFID query. By carefully observing the relationship between channel and phase, MobiTagbot detects if multipath is likely prevalent at a given robot reader location. If so, MobiTagbot excludes phase readings from that reader location, and generates a final location estimate using phase readings from other locations as the robot reader moves in space. Experimentally, we demonstrate that cutting-edge localization algorithms including Tagoram are not accurate enough to exactly order items in very close proximity, but MobiTagbot is, achieving nearly 100% ordering accuracy for items at low (3--6 cm) spacings and 86% accuracy for items at very low (1--3 cm) spacings.

Device-free indoor localization aims to localize people without requiring them to carry any devices or being actively involved in the localizing process. It underpins a wide range of applications including older people surveillance, intruder detection and indoor navigation. However, in a cluttered environment such as a residential home, the Received Signal Strength Indicator (RSSI) is heavily obstructed by furniture or metallic appliances, thus reducing the localization accuracy. This environment is important to observe as human-object interaction (HOI) events, detected by pervasive sensors, can potentially reveal people's interleaved locations during daily living activities, such as watching TV, opening the fridge door. This paper aims to enhance the performance of commercial off-the-shelf (COTS) RFID-based localization system by leveraging HOI contexts in a furnished home. Specifically, we propose a general Bayesian probabilistic framework to integrate both RSSI signals and HOI events to infer the most likely location and trajectory. Experiments conducted in a residential house demonstrate the effectiveness of our proposed method, in which we can localize a resident with average 95% accuracy and track a moving subject with 0.58m mean error distance.

Sleep quality plays a vital role in personal health. A great deal of effort has been paid to design sleep quality monitoring systems, providing services ranging from bedtime monitoring to sleep activity detection. However, as sleep quality is closely related to the distribution of sleep duration over different sleep stages, neither the bedtime nor the intensity of sleep activities is able to reflect sleep quality precisely. We present Sleep Hunter, a mobile service that provides a fine-grained detection of sleep stage transition for sleep quality monitoring and intelligent wake-up call. The rationale is that each sleep stage is accompanied by specific body movements and acoustic signals. Leveraging the built-in sensors on smartphones, Sleep Hunter integrates these physical activities with sleep environment, inherent temporal relation, and personal factors by a statistical model for a fine-grained sleep stage detection. Based on the duration of each sleep stage, Sleep Hunter further provides sleep quality report and smart call service for users. Experimental results from over 30 sets of nocturnal sleep data show that our system is superior to existing actigraphy-based sleep quality monitoring systems, and achieves satisfying detection accuracy compared with dedicated polysomnography-based devices.

Different from online shopping, in-store shopping has few ways to collect the customer behaviors before purchase. In this paper, we present the design and implementation of an on-site Customer Behavior IDentification system based on passive RFID tags, named CBID. By collecting and analyzing wireless signal features, CBID can detect and track tag movements and further infer corresponding customer behaviors. We model three main objectives of behavior identification by concrete problems and solve them using novel protocols and algorithms. The design innovations of this work include a Doppler effect based protocol to detect tag movements, an accurate Doppler frequency estimation algorithm, an image-based human count estimation protocol and a tag clustering algorithm using cosine similarity. We have implemented a prototype of CBID in which all components are built by off-the-shelf devices. We have deployed CBID in real environments and conducted extensive experiments to demonstrate the accuracy and efficiency of CBID in customer behavior identification.

Indoor localization is of great importance to a wide range of applications in the era of mobile computing. Current mainstream solutions rely on Received Signal Strength (RSS) of wireless signals as fingerprints to distinguish and infer locations. However, those methods suffer from fingerprint ambiguity that roots in multipath fading and temporal dynamics of wireless signals. Though pioneer efforts have resorted to motion-assisted or peer-assisted localization, they neither work in real time nor work without the help of peer users, which introduces extra costs and constraints, and thus degrades their practicality. To get over these limitations, we propose Argus, an image-assisted localization system for mobile devices. The basic idea of Argus is to extract geometric constraints from crowdsourced photos, and to reduce fingerprint ambiguity by mapping the constraints jointly against the fingerprint space. We devise techniques for photo selection, geometric constraint extraction, joint location estimation, and build a prototype that runs on commodity phones. Extensive experiments show that Argus triples the localization accuracy of classic RSS-based method, in time no longer than normal WiFi scanning, with negligible energy consumption.

We introduce SmartGuide, a light-weighted and efficient approach to localize and recognize a distant unknown building. Our approach relies on shooting only a single photo of a target building via a smartphone and a local 2D Google map. SmartGuide first extracts a partial top view contour of a building from its side-view photo by applying vanishing point and the Manhattan World Assumption, and then fetches a candidate building set from a local 2D Google map based on smartphone's GPS readings. Partial top view shape, orientation and distance relative to the camera are used as input parameters in a probability model, which adversely recognizes the best candidate building in the local map. Our model is developed based on kernel density estimation that helps reduce noise in the smartphone sensors, such as GPS readings and camera ray direction reported by noisy accelerometer and compass. Experimental results demonstrate that our approach recognizes buildings ranging from 20m to 520m and achieves 92.7% accuracy in downtown areas where the Manhattan World Assumption is applicable. In addition, the processing time is no more than 6 seconds for 87% of cases. Compared with existing building localization schemes, SmartGuide offers numerous advantages. Our method avoids taking multiple photos, intricate 3D reconstruction or any initial deployment cost of database construction, making it faster and less labor-intensive than existing solutions.

Regular free-weight exercise helps to strengthen the body's natural movements and stabilize muscles that are important to strength, balance, and posture of human beings. Prior works have exploited wearable sensors or RF signal changes (e.g., WiFi and Blue tooth) for activity sensing, recognition and countingetc.. However, none of them have incorporate three key factors necessary for a practical free-weight exercise monitoring system: recognizing free-weight activities on site, assessing their qualities, and providing useful feedbacks to the bodybuilder promptly. Our FEMO system responds to these demands, providing an integrated free-weight exercise monitoring service that incorporates all the essential functionalities mentioned above. FEMO achieves this by attaching passive RFID tags on the dumbbells and leveraging the Doppler shift profile of the reflected backscatter signals for on-site free-weight activity recognition and assessment. The rationale behind FEMO is 1): since each free-weight activity owns unique arm motions, the corresponding Doppler shift profile should be distinguishable to each other and serves as a reliable signature for each activity. 2): the Doppler profile of each activity has a strong spatial-temporal correlation that implicitly reflects the quality of each performed activity. We implement FEMO with COTS RFID devices and conduct a two-week experiment. The preliminary result from 15 volunteers demonstrates that FEMO can be applied to a variety of free-weight activities and users, and provide valuable feedbacks for activity alignment.

Shopping behavior data are of great importance to understand the effectiveness of marketing and merchandising efforts. Online clothing stores are capable capturing customer shopping behavior by analyzing the click stream and customer shopping carts. Retailers with physical clothing stores, however, still lack effective methods to identify comprehensive shopping behaviors. In this paper, we show that backscatter signals of passive RFID tags can be exploited to detect and record how customers browse stores, which items of clothes they pay attention to, and which items of clothes they usually match with. The intuition is that the phase readings of tags attached on desired items will demonstrate distinct yet stable patterns in the time-series when customers look at, pick up or turn over desired items. We design ShopMiner,, a framework that harnesses these unique spatial-temporal correlations of time-series phase readings to detect comprehensive shopping behaviors. We have implemented a prototype of ShopMiner, with a COTS RFID reader and four antennas, and tested its effectiveness in two typical indoor environments. Empirical studies from two-week shopping-like data show that ShopMiner, could achieve high accuracy and efficiency in customer shopping behavior identification.

Mobile sensing apps have proliferated rapidly over the recent years. Most of them rely on inference components heavily for detecting interesting activities or contexts. Existing work implements inference components using traditional models designed for balanced data sets, where the sizes of interesting (positive) and non-interesting (negative) data are comparable. Practically, however, the positive and negative sensing data are highly imbalanced. For example, a single daily activity such as bicycling or driving usually occupies a small portion of time, resulting in rare positive instances. Under this circumstance, the trained models based on imbalanced data tend to mislabel positive ones as negative. In this paper, we propose a new inference framework SLIM based on several machine learning techniques in order to accommodate the imbalanced nature of sensing data. Especially, guided under-sampling is employed to obtain balanced labelled subsets, followed by a similarity-based sampling that draws massive unlabelled data to enhance training. To the best of our knowledge, SLIM is the first model that considers data imbalance in mobile sensing. We prototype two sensing apps and the experimental results show that SLIM achieves higher recall (activity recognition rate) while maintaining the precision compared with five classical models. In terms of the overall recall and precision, SLIM is around 12 percent better than the compared solutions on average.

Wireless LANs, particularly WiFi, have been pervasively deployed and have fostered myriad wireless communication services and ubiquitous computing applications. A primary concern in designing these applications is to combat harsh indoor propagation environments, particularly Non-Line-Of-Sight (NLOS) propagation. The ability to identify the existence of the Line-Of-Sight (LOS) path acts as a key enabler for adaptive communication, cognitive radios, and robust localization. Enabling such capability on commodity WiFi infrastructure, however, is prohibitive due to the coarse multipath resolution with MAC-layer received signal strength. In this paper, we propose two PHY-layer channel-statistics-based features from both the time and frequency domains. To further break away from the intrinsic bandwidth limit of WiFi, we extend to the spatial domain and harness natural mobility to magnify the randomness of NLOS paths while retaining the deterministic nature of the LOS component. We propose LiFi, a statistical LOS identification scheme with commodity WiFi infrastructure, and evaluate it in typical indoor environments covering an area of 1500 m2. Experimental results demonstrate that LiFi achieves an overall LOS detection rate of 90.42% with a false alarm rate of 9.34% for the temporal feature and an overall LOS detection rate of 93.09% with a false alarm rate of 7.29% for the spectral feature.

Sleep quality plays a significant role in personal health. A great deal of effort has been paid to design sleep quality monitoring systems, providing services ranging from bedtime monitoring to sleep activity detection. However, as sleep quality is closely related to the distribution of sleep duration over different sleep stages, neither the bedtime nor the intensity of sleep activities is able to reflect sleep quality precisely. To this end, we present Sleep Hunter, a mobile service that provides a fine-grained detection of sleep stage transition for sleep quality monitoring and intelligent wake-up call. The rationale is that each sleep stage is accompanied by specific yet distinguishable body movements and acoustic signals. Leveraging the built-in sensors on smartphones, Sleep Hunter integrates these physical activities with sleep environment, inherent temporal relation and personal factors by a statistical model for a fine-grained sleep stage detection. Based on the duration of each sleep stage, Sleep Hunter further provides sleep quality report and smart call service for users. Experimental results from over 30 sets of nocturnal sleep data show that our system is superior to existing actigraphy-based sleep quality monitoring systems, and achieves satisfying detection accuracy compared with dedicated polysomnography-based devices.

Crossroad is among the most dangerous parts outside for the visually impaired people. Numerous studies have exploited navigating systems for the visually impaired community, providing services ranging from block detection, route planning to realtime localization. However, none of them have addressed the safety issue in crossroad and integrated three key factors necessary for a practical crossroad navigation system: detecting the crossroad, locating zebra patterns, and guiding the user within zebra crossing when passing the road. Our CrossNavi application responds to these needs, providing an integrated crossroad navigation service that incorporates all the essential functionalities mentioned above. The overall service is fulfilled by the collaboration of built-in sensors on commodity phones, and requires minimal human participation. We describe the technical aspects of its design, implementation, interface, and further improvements to make the system practical on a wider basis. Experimental results from three visually impaired volunteers show that the system exhibits promising behavior in both urban and rural areas.

Different from online shopping, in-store shopping has few ways to collect the customer behaviors before purchase. In this paper, we present the design and implementation of an on-site Customer Behavior Identification system based on passive RFID tags, named CBID. By collecting and analyzing wireless signal features, CBID can detect and track tag movements and further infer corresponding customer behaviors. We model three main objectives of behavior identification by concrete problems and solve them using novel protocols and algorithms. The design innovations of this work include a Doppler effect based protocol to detect tag movements, an accurate Doppler frequency estimation algorithm, a multi-RSS based tag localization protocol, and a tag clustering algorithm using cosine similarity. We have implemented a prototype of CBID in which all components are built by off-the-shelf devices. We have deployed CBID in real environments and conducted extensive experiments to demonstrate the accuracy and efficiency of CBID in customer behavior identification.

Near field authentication is of great importance for a range of applications, and has attracted many research efforts in the past decades. Several approaches have been developed and demonstrated their feasibility. The state-of-art works, however, still have much room to improve their automation and usability. First, user assistance is required in most existing approaches, which will be easily observed and imitated by attackers. Second, the authentications of several works heavily depend on special hardware, e.g., Server or high resolution screen, which greatly restricts their application scenarios. In this paper, we present a near field authentication system Tooth that needs little human assistance and is compatible with most smart phones. ToAuth is based on the key insight that the acceleration traces are similar for a pair of smart phones when they are contacting physically and vibrating. The random vibration patterns are sufficiently uncertain to provide high entropy to generate a pair of cryptographic keys yet are inimitable for a third party who does not get in touch with the vibration source. ToAuth leverages the keys to make authentication for smart phones. We implement ToAuth on Android platform and evaluate its performance under various scenarios. Extensive experiments demonstrate ToAuth could achieve around 90% success rate in stable environment, and prevent attacks depended on vibration noise.

In many sensor network applications, multiple data forwarding tasks usually exist with different source-destination node pairs. Due to limitations of the duty-cycling operation and interference, however, not all tasks can be guaranteed to be scheduled within their required delay constraints. We investigate a fundamental scheduling problem of both theoretical and practical importance, called multi-task schedulability problem, i.e., given multiple data forwarding tasks, to determine the maximum number of tasks that can be scheduled within their deadlines and work out such a schedule. We formulate the multi-task schedulability problem, prove its NP-Hardness, and propose an approximate algorithm with analysis on the performance bound and complicity. We further extend the proposed algorithm by explicitly altering duty cycles of certain sensor nodes so as to fully support applications with stringent delay requirements to accomplish all tasks. We then design a practical scheduling protocol based on proposed algorithms. We conduct extensive trace-driven simulations to validate the effectiveness and efficiency of our approach with various settings.

In many logistics applications of RFID technology, luggage attached with tags are placed on moving conveyor belts for processing. It is important to figure out the order of goods on the belts so that further actions like sorting can be accurately taken on proper goods. Due to arbitrary goods placement or the irregularity of wireless signal propagation, neither of the order of tag identification nor the received signal strength provides sufficient evidence on their relative positions on the belts. In this study, we observe, from experiments, a critical region of reading rate when a tag gets close enough to a reader. This phenomenon, as well as other signal attributes, yields the stable indication of tag order. We establish a probabilistic model for recognizing the transient critical region and propose the OTrack protocol to continuously monitor the order of tags. To validate the protocol, we evaluate the accuracy and effectiveness through a one-month experiment conducted through a working conveyor at Beijing Capital International Airport.

In this study, we explore the possibility of locating remote objects via cameras together with built-in inertial sensors of off-the-shelf smartphones. Our solution, CamLoc, enables a user taking two photos of an object using a smartphone at a fixed location and immediately knowing the location of the object in global coordinates, thus facilitating myriad location-based services. Such usage is user-friendly but error prone. We devise several techniques to mitigate the errors caused by cheap and noisy sensors, upgrading the positioning accuracy to an applicable level. We prototype CamLoc on Android OS, and evaluate its performance across different scenarios with various building densities. Experiment results show that our system achieves 89 percent and 72 percent physical location mapping accuracy in rural and downtown areas, respectively, which is competitive with existing solutions.

Context-awareness is getting increasingly important for a range of mobile and pervasive applications on nowadays smartphones. Whereas human-centric contexts (e.g., indoor/ outdoor, at home/in office, driving/walking) have been extensively researched, few attempts have studied from phones' perspective (e.g., on table/sofa, in pocket/bag/hand). We refer to such immediate surroundings as micro-environment, usually several to a dozen of centimeters, around a phone. In this study, we design and implement Sherlock, a micro-environment sensing platform that automatically records sensor hints and characterizes the micro-environment of smartphones. The platform runs as a daemon process on a smartphone and provides finer-grained environment information to upper layer applications via programming interfaces. Sherlock is a unified framework covering the major cases of phone usage, placement, attitude, and interaction in practical uses with complicated user habits. As a long-term running middleware, Sherlock considers both energy consumption and user friendship. We prototype Sherlock on Android OS and systematically evaluate its performance with data collected on fifteen scenarios during three weeks. The preliminary results show that Sherlock achieves low energy cost, rapid system deployment, and competitive sensing accuracy.

Device-free Passive (DfP) human detection acts as a key enabler for emerging location-based services such as smart space, human-computer interaction, and asset security. A primary concern in devising scenario-tailored detecting systems is coverage of their monitoring units. While disk-like coverage facilitates topology control, simplifies deployment analysis, and is crucial for proximity-based applications, conventional monitoring units demonstrate directional coverage due to the underlying transmitter-receiver link architecture. To achieve omnidirectional coverage under such link-centric architecture, we propose the concept of omnidirectional passive human detection. The rationale is to exploit the rich multipath effect to blur the directional coverage. We harness PHY layer features to robustly capture the fine-grained multipath characteristics and virtually tune the shape of the coverage of the monitoring unit, which is previously prohibited with mere MAC layer RSSI. We design a fingerprinting scheme and a threshold-based scheme with off-the-shelf WiFi infrastructure and evaluate both schemes in typical clustered indoor scenarios. Experimental results demonstrate an average false positive of 8 percent and an average false negative of 7 percent for fingerprinting in detecting human presence in 4 directions. And both average false positive and false negative remain around 10 percent even with threshold-based methods.

In many logistics applications of RFID technology, goods attached with tags are placed on moving conveyor belts for processing. It is important to figure out the order of goods on the belts so that further actions like sorting can be accurately taken on proper goods. Due to arbitrary goods placement or the irregularity of wireless signal propagation, neither of the order of tag identification nor the received signal strength provides sufficient evidence on their relative positions on the belts. In this study, we observe, from experiments, a critical region of reading rate when a tag gets close enough to a reader. This phenomenon, as well as other signal attributes, yields the stable indication of tag order. We establish a probabilistic model for recognizing the transient critical region and propose the OTrack protocol to continuously monitor the order of tags. To validate the protocol, we evaluate the accuracy and effectiveness through a one-month experiment conducted through a working conveyor at Beijing Capital International Airport.

Passive human detection and localization serve as key enablers for various pervasive applications such as smart space, human-computer interaction and asset security. The primary concern in devising scenario-tailored detecting systems is the coverage of their monitoring units. In conventional radio-based schemes, the basic unit tends to demonstrate a directional coverage, even if the underlying devices are all equipped with omnidirectional antennas. Such an inconsistency stems from the link-centric architecture, creating an anisotropic wireless propagating environment. To achieve an omnidirectional coverage while retaining the link-centric architecture, we propose the concept of Omnidirectional Passive Human Detection, and investigate to harness the PHY layer features to virtually tune the shape of the unit coverage by fingerprinting approaches, which is previously prohibited with mere MAC layer RSSI. We design the scheme with ubiquitously deployed WiFi infrastructure and evaluate it in typical multipath-rich indoor scenarios. Experimental results show that our scheme achieves an average false positive of 8% and an average false negative of 7% in detecting human presence in 4 directions.

We study the rendezvous data collection problem for the mobile sink in wireless sensor networks. We introduce to jointly optimize trajectory planning for the mobile sink and workload balancing for the network. By doing so, the mobile sink is able to efficiently collect network-wide data within a given delay bound and the network can eliminate the energy bottleneck to dramatically prolong its lifetime. Such a joint optimization problem is shown to be NP-hard and we propose an approximation algorithm, named RPS-LB, to approach the optimal solution. In RPS-LB, according to observed properties of the median reference structure in the network, a series of Rendezvous Points (RPs) are selected to construct the trajectory for the mobile sink and the derived approximation ratio of RPSLB guarantees that the formed trajectory is comparable with the optimal solution. The workload allocated to each RP is proven to be balanced mathematically. We then relax the assumption that mobile sink knows the location of each sensor node and present a localized, fully distributed version, RPS-LB-D, which largely improves the system applicability in practice. We verify the effectiveness of our proposals via extensive experiments.