Cameras are key enablers for a wide range of IoT use cases including smart cities, intelligent transportation, AI-enabled farms, and more. These IoT applications require cloud software (including models) to act on the images. However, traditional task oblivious compression techniques are a poor fit for delivering images over low power IoT networks that are lossy and limited in capacity. The key challenge is their brittleness against packet loss; they are highly sensitive to small amounts of packet loss requiring retransmission for transport, which further reduces the available capacity of the network. We propose Starfish, a design that achieves better compression ratios and is graceful with packet loss. In addition to that, Starfish features content-awareness and task-awareness, meaning that we can build specialized codecs for each application scenario and optimized for task objectives, including objective/perceptual quality as well as AI tasks directly. We carefully design the DNN architecture and use an AutoML method to search for TinyML models that work on extremely low power/cost AIoT accelerators. Starfish is not only the first image compress framework that works on a $3 AIoT accelerators but also outperforms JPEG, a well-established baseline, by up to 3x, in terms of bandwidth efficiency and up to 2.5x as efficient in energy consumption. It also features graceful and gradual performance degradation in the presence of packet loss. The application-level simulation indicates that Starfish could deliver 3.7x images while providing better image quality.

Recent years have seen exciting developments in the use of RFID tags as sensors to enable a range of applications including home automation, health and wellness, and augmented reality. However, widespread use of RFIDs as sensors requires significant instrumentation to deploy tethered readers, which limits usability in mobile settings. Our solution is WearID, a low-power wrist-worn backscatter reader that bridges this gap and allows ubiquitous sensing of interaction with tagged objects. Our end-to-end design includes innovations in hardware architecture to reduce power consumption and deal with wrist attenuation and blockage, as well as signal processing architecture to reliably detect grasping, touching, and other hand-based interactions. We show via exhaustive characterization that WearID is roughly 6× more power efficient than state-of-art commercial readers, provides 3D coverage of 30 to 50 cm around the wrist despite body blockage, and can be used to reliably detect hand-based interactions. We also open source the design of WearID with the hope that this can enable a range of new and unexplored applications of wearables.

Modern mobile networks are facing unprecedented growth in demand due to a new class of traffic from Internet of Things (IoT) devices such as smart wearables and autonomous cars. Future networks must schedule delay-tolerant software updates, data backup, and other transfers from IoT devices while maintaining strict service guarantees for conventional real-time applications such as voice-calling and video. This problem is extremely challenging because conventional traffic is highly dynamic across space and time, so its performance is significantly impacted if all IoT traffic is scheduled immediately when it originates. In this paper, we present a reinforcement learning (RL) based scheduler that can dynamically adapt to traffic variation, and to various reward functions set by network operators, to optimally schedule IoT traffic. Using 4 weeks of real network data from downtown Melbourne, Australia spanning diverse traffic patterns, we demonstrate that our RL scheduler can enable mobile networks to carry 14.7% more data with minimal impact on existing traffic, and outpeforms heuristic schedulers by more than 2x. Our work is a valuable step towards designing autonomous, "self-driving" networks that learn to manage themselves from past data.

The human eye offers a fascinating window into an individual's health, cognitive attention, and decision making, but we lack the ability to continually measure these parameters in the natural environment. We demonstrate CIDER, a system that operates in a highly optimized low-power mode under indoor settings by using a fast Search-Refine controller to track the eye, but detects when the environment switches to more challenging outdoor sunlight and switches models to operate robustly under this condition. Our design is holistic and tackles a) power consumption in digitizing pixels, estimating pupillary parameters, and illuminating the eye via near-infrared and b) error in estimating pupil center and pupil dilation. We demonstrate that CIDER can estimate pupil center with error less than two pixels (0.6°), and pupil diameter with error of one pixel (0.22mm). Our end-to-end results show that we can operate at power levels of roughly 7mW at a 4Hz eye tracking rate, or roughly 32mW at rates upwards of 250Hz.

While many radio technologies are available for mobile devices, none of them are designed to deal with asymmetric available energy. Battery capacities of mobile devices vary by up to three orders of magnitude between laptops and wearables, and our inability to deal with such asymmetry has limited the lifetime of constrained portable devices.

In this paper, we look at making backscatter practical for ultra-low power on-body sensors by leveraging radios on existing smartphones and wearables (e.g. WiFi and Bluetooth). The difficulty lies in the fact that in order to extract the weak backscattered signal, the system needs to deal with self-interference from the wireless carrier (WiFi or Bluetooth) without relying on built-in capability to cancel or reject the carrier interference.

The human eye offers a fascinating window into an individual's health, cognitive attention, and decision making, but we lack the ability to continually measure these parameters in the natural environment. The challenges lie in: a) handling the complexity of continuous high-rate sensing from a camera and processing the image stream to estimate eye parameters, and b) dealing with the wide variability in illumination conditions in the natural environment. This paper explores the power-robustness tradeoffs inherent in the design of a wearable eye tracker, and proposes a novel staged architecture that enables graceful adaptation across the spectrum of real-world illumination. We propose CIDER, a system that operates in a highly optimized low-power mode under indoor settings by using a fast Search-Refine controller to track the eye, but detects when the environment switches to more challenging outdoor sunlight and switches models to operate robustly under this condition. Our design is holistic and tackles a) power consumption in digitizing pixels, estimating pupillary parameters, and illuminating the eye via near-infrared, b) error in estimating pupil center and pupil dilation, and c) model training procedures that involve zero effort from a user. We demonstrate that CIDER can estimate pupil center with error less than two pixels (0.6°), and pupil diameter with error of one pixel (0.22mm). Our end-to-end results show that we can operate at power levels of roughly 7mW at a 4Hz eye tracking rate, or roughly 32mW at rates upwards of 250Hz.

Backscatter provides dual-benefits of energy harvesting and low-power communication, making it attractive to a broad class of wireless sensors. But the design of a protocol that enables extremely power-efficient radios for harvesting-based sensors as well as high-rate data transfer for data-rich sensors presents a conundrum. In this paper, we present a new {\em fully asymmetric} backscatter communication protocol where nodes blindly transmit data as and when they sense. This model enables fully flexible node designs, from extraordinarily power-efficient backscatter radios that consume barely a few micro-watts to high-throughput radios that can stream at hundreds of Kbps while consuming a paltry tens of micro-watts. The challenge, however, lies in decoding concurrent streams at the reader, which we achieve using a novel combination of time-domain separation of interleaved signal edges, and phase-domain separation of colliding transmissions. We provide an implementation of our protocol, LF-Backscatter, and show that it can achieve an order of magnitude or more improvement in throughput, latency and power over state-of-art alternatives.

Different battery chemistries perform better on different axes, such as energy density, cost, peak power, recharge time, longevity, and efficiency. Mobile system designers are constrained by existing technology, and are forced to select a single chemistry that best meets their diverse needs, thereby compromising other desirable features. In this paper, we present a new hardware-software system, called Software Defined Battery (SDB), which allows system designers to integrate batteries of different chemistries. SDB exposes APIs to the operating system which control the amount of charge flowing in and out of each battery, enabling it to dynamically trade one battery property for another depending on Application And/Or User Needs. Using microbenchmarks from our prototype SDB implementation, and through detailed simulations, we demonstrate that it is possible to combine batteries which individually excel along different axes to deliver an enhanced collective performance when compared to traditional battery packs.

Continuous, real-time tracking of eye gaze is valuable in a variety of scenarios including hands-free interaction with the physical world, detection of unsafe behaviors, leveraging visual context for advertising, life logging, and others. While eye tracking is commonly used in clinical trials and user studies, it has not bridged the gap to everyday consumer use. The challenge is that a real-time eye tracker is a power-hungry and computation-intensive device which requires continuous sensing of the eye using an imager running at many tens of frames per second, and continuous processing of the image stream using sophisticated gaze estimation algorithms. Our key contribution is the design of an eye tracker that dramatically reduces the sensing and computation needs for eye tracking, thereby achieving orders of magnitude reductions in power consumption and form-factor. The key idea is that eye images are extremely redundant, therefore we can estimate gaze by using a small subset of carefully chosen pixels per frame. We use a sparse pixel-based gaze estimation algorithm that is a multi-layer neural network learned using a state-of-the-art sparsity-inducing regularization function which minimizes the gaze prediction error while simultaneously minimizing the number of pixels used. Our results show that we can operate at roughly 70mW of power, while continuously estimating eye gaze at the rate of 30 Hz with errors of roughly 4 degrees.

This paper argues for a clean-slate redesign of wireless sensor systems to take advantage of the extremely low power consumption of backscatter communication and emerging ultra-low power sensor modalities. We make the case that existing sensing architectures incur substantial overhead for a variety of computational blocks between the sensor and RF front end - while these overheads were negligible on platforms where communication was expensive, they become the bottleneck on backscatter-based systems and increase power consumption while limiting throughput. We present a radically new design that is minimalist, yet efficient, and designed to operate end-to-end at tens of μWs while enabling high-data rate backscatter at rates upwards of many hundreds of Kbps. In addition, we demonstrate a complex reader-driven MAC layer that jointly considers energy, channel conditions, data utility, and platform constraints to enable network-wide throughput optimizations. We instantiate this architecture on a custom FPGA-based platform connected to microphones, and show that the platform consumes 73x lower power and has 12.5x higher throughput than existing backscatter-based sensing platforms.

Continuous, real-time tracking of eye gaze is valuable in a variety of scenarios including hands-free interaction with the physical world, detection of unsafe behaviors, leveraging visual context for advertising, life logging, and others. While eye tracking is commonly used in clinical trials and user studies, it has not bridged the gap to everyday consumer use. The challenge is that a real-time eye tracker is a power-hungry and computation-intensive device which requires continuous sensing of the eye using an imager running at many tens of frames per second, and continuous processing of the image stream using sophisticated gaze estimation algorithms. Our key contribution is the design of an eye tracker that dramatically reduces the sensing and computation needs for eye tracking, thereby achieving orders of magnitude reductions in power consumption and form-factor. The key idea is that eye images are extremely redundant, therefore we can estimate gaze by using a small subset of carefully chosen pixels per frame. We instantiate this idea in a prototype hardware platform equipped with a low-power image sensor that provides random access to pixel values, a low-power ARM Cortex M3 microcontroller, and a bluetooth radio to communicate with a mobile phone. The sparse pixel-based gaze estimation algorithm is a multi-layer neural network learned using a state-of-the-art sparsity-inducing regularization function that minimizes the gaze prediction error while simultaneously minimizing the number of pixels used. Our results show that we can operate at roughly 70mW of power, while continuously estimating eye gaze at the rate of 30 Hz with errors of roughly 3 degrees.

Indoor physical analytics calls for high-accuracy localization that existing indoor (e.g., WiFi-based) localization systems may not offer. By exploiting the ever increasingly wider adoption of LED lighting, in this paper, we study the problem of using visible LED lights for accurate localization. We identify the key challenges and tackle them through the design of Pharos. In particular, we establish and experimentally verify an optical channel model suitable for localization. We adopt BFSK and channel hopping to achieve reliable location beaconing from multiple, uncoordinated light sources over shared light medium. Preliminary evaluation shows that Pharos achieves the 90th percentile localization accuracy of 0.4m and 0.7m for two typical indoor environments. We believe visible light based localization holds the potential to significantly improve the position accuracy, despite few potential issues to be conquered in real deployment.

We present ViRi -- an intriguing system that enables a user to enjoy a frontal view experience even when the user is actually at a slanted viewing angle. ViRi tries to restore the front-view effect by enhancing the normal content rendering process with an additional geometry correction stage. The necessary prerequisite is effectively and accurately estimating the actual viewing angle under natural viewing situations and under the constraints of the device's computational power and limited battery deposit. We tackle the problem with face detection and augment the phone camera with a fisheye lens to expand its field of view so that the device can recognize its user even the phone is placed casually. We propose effective pre-processing techniques to ensure the applicability of face detection tools onto highly distorted fisheye images. To save energy, we leverage information from system states, employ multiple low power sensors to rule out unlikely viewing situations, and aggressively seek additional opportunities to maximally skip the face detection. For situations in which face detection is unavoidable, we design efficient prediction techniques to further speed up the face detection. The effectiveness of the proposed techniques have been confirmed through thorough evaluations. We have also built a straw man application to allow users to experience the intriguing effects of ViRi.

Auditeur is a general-purpose, energy-efficient, and context-aware acoustic event detection platform for smartphones. It enables app developers to have their app register for and get notified on a wide variety of acoustic events. Auditeur is backed by a cloud service to store user contributed sound clips and to generate an energy-efficient and context-aware classification plan for the phone. When an acoustic event type has been registered, the smartphone instantiates the necessary acoustic processing modules and wires them together to execute the plan. The phone then captures, processes, and classifies acoustic events locally and efficiently. Our analysis on user-contributed empirical data shows that Auditeur's energy-aware acoustic feature selection algorithm is capable of increasing the device lifetime by 33.4%, sacrificing less than 2% of the maximum achievable accuracy. We implement seven apps with Auditeur, and deploy them in real-world scenarios to demonstrate that Auditeur is versatile, 11.04% - 441.42% less power hungry, and 10.71% - 13.86% more accurate in detecting acoustic events, compared to state-of-the-art techniques. We present a user study to demonstrate that novice programmers can implement the core logic of interesting apps with Auditeur in less than 30 minutes, using only 15 - 20 lines of Java code.

Mobile phones have become an ideal platform for physiological and environmental sensing. A number of research and commercial smartphone "accessories" have emerged in recent years that try to extend the sensing capabilities of a mobile phone. However, the major drawback of these devices is that they either require the user to act in some specific way or change their lifestyle and habit to some extent. In this demo, we present Septimu V2 (Septimu2) -- a novel non-intrusive physiological and environmental sensing platform which is fully embedded in a conventional earphone, works with existing smartphones, and does not require the user to change habits in any way. Septimu2 is a continuation of [1], and integrates a suite of new sensors. In addition to 3-axis accelerometer and gyroscope, Septimu2 incorporates remote IR temperature sensor, IR LED, IR photodiode and two additional microphones. The baseboard performs signal condition and sends the data to cellphone via Bluetooth. Septimu2 enables a number of applications, including heart-rate monitoring, fine grained posture detection, and external sound source localization and classification.