Gaze estimation models are widely used in applications such as driver attention monitoring and human-computer interaction. While many methods for gaze estimation exist, they rely heavily on data-hungry deep learning to achieve high performance. This reliance often forces practitioners to harvest training data from unverified public datasets, outsource model training, or rely on pre-trained models. However, such practices expose gaze estimation models to backdoor attacks. In such attacks, adversaries inject backdoor triggers by poisoning the training data, creating a backdoor vulnerability: the model performs normally with benign inputs, but produces manipulated gaze directions when a specific trigger is present. This compromises the security of many gaze-based applications, such as causing the model to fail in tracking the driver's attention. To date, there is no defense that addresses backdoor attacks on gaze estimation models. In response, we introduce SecureGaze, the first solution designed to protect gaze estimation models from such attacks. Unlike classification models, defending gaze estimation poses unique challenges due to its continuous output space and globally activated backdoor behavior. By identifying distinctive characteristics of backdoored gaze estimation models, we develop a novel and effective approach to reverse-engineer the trigger function for reliable backdoor detection. Extensive evaluations in both digital and physical worlds demonstrate that SecureGaze effectively counters a range of backdoor attacks and outperforms seven state-of-the-art defenses adapted from classification models.

Gaze estimation is of great importance to many scientific fields and daily applications, ranging from fundamental research in cognitive psychology to attention-aware systems. While recent advancements in deep learning have led to highly accurate gaze estimation systems, these solutions often come with high computational costs and depend on large-scale labeled gaze data for supervised learning, posing significant practical challenges. To move beyond these limitations, we present EfficientGaze, a resource-efficient framework for gaze representation learning. We introduce the frequency-domain gaze estimation, which exploits the feature extraction capability and the spectral compaction property of discrete cosine transform to substantially reduce the computational cost of gaze estimation systems for both calibration and inference. Moreover, to overcome the data labeling hurdle, we design a novel multi-task gaze-aware contrastive learning framework to learn gaze representations that are generic across subjects in an unsupervised manner. Our evaluation on two gaze estimation datasets demonstrates that EfficientGaze achieves comparable gaze estimation performance to existing supervised learning-based approaches, while enabling up to 6.80 times and 1.67 times speedup in system calibration and gaze estimation, respectively.

Deep learning models are increasingly deployed on edge Internet of Things (IoT) devices. However, these models typically operate under supervised conditions and fail to recognize unseen classes different from training. To address this, zero-shot learning (ZSL) aims to classify data of unseen classes with the help of semantic information. Foundation models (FMs) trained on web-scale data have shown impressive ZSL capability in natural language processing and visual understanding. However, leveraging FMs’ generalized knowledge for zero-shot IoT sensing using signals such as mmWave, IMU, and Wi-Fi has not been fully investigated. In this work, we align the IoT data embeddings with the semantic embeddings generated by an FM’s text encoder for zero-shot IoT sensing. To utilize the physics principles governing the generation of IoT sensor signals to derive more effective prompts for semantic embedding extraction, we propose to use cross-attention to combine a learnable soft prompt that is optimized automatically on training data and an auxiliary hard prompt that encodes domain knowledge of the IoT sensing task. To address the problem of IoT embeddings biasing to seen classes due to the lack of unseen class data during training, we propose using data augmentation to synthesize unseen class IoT data for fine-tuning the IoT feature extractor and embedding projector. We evaluate our approach on multiple IoT sensing tasks. Results show that our approach achieves superior open-set detection and generalized zero-shot learning performance compared with various baselines. Our code is available at https://github.com/schrodingho/FM_ZSL_IoT.

Diverse Wi-Fi-based wireless applications have been proposed, ranging from daily activity recognition to vital sign monitoring. Despite their remarkable sensing accuracy, the high energy consumption and the requirement for customized hardware modification hinder the wide deployment of the existing sensing solutions. In this paper, we propose REHSense, an energy-efficient wireless sensing solution based on Radio-Frequency (RF) energy harvesting. Instead of relying on a power-hungry Wi-Fi receiver, REHSense leverages an RF energy harvester as the sensor and utilizes the voltage signals harvested from the ambient Wi-Fi signals to enable simultaneous context sensing and energy harvesting. We design and implement REHSense using a commercial-off-the-shelf (COTS) RF energy harvester. Extensive evaluation of three fine-grained wireless sensing tasks (i.e., respiration monitoring, human activity recognition, and hand gesture recognition) shows that REHSense can achieve comparable sensing accuracy with conventional Wi-Fi-based solutions while adapting to different sensing environments, reducing the power consumption of sensing by 98.7% and harvesting up to 4.5 mW of power from RF energy.

In this work, we introduce SEESys, the first system to provide online pose error estimation for Simultaneous Localization and Mapping (SLAM). Unlike prior offline error estimation approaches, the SEESys framework efficiently collects real-time system features and delivers accurate pose error magnitude estimates with low latency. This enables real-time quality-of-service information for downstream applications. To achieve this goal, we develop a SLAM system run-time status monitor (RTS monitor) that performs feature collection with minimal overhead, along with a multi-modality attention-based Deep SLAM Error Estimator (DeepSEE) for error estimation. We train and evaluate SEESys using both public SLAM benchmarks and a diverse set of synthetic datasets, achieving an RMSE of 0.235 cm of pose error estimation, which is 15.8% lower than the baseline. Additionally, we conduct a case study showcasing SEESys in a real-world scenario, where it is applied to a real-time audio error advisory system for human operators of a SLAM-enabled device. The results demonstrate that SEESys provides error estimates with an average end-to-end latency of 37.3 ms, and the audio error advisory reduces pose tracking error by 25%.

Solar cells have been widely used for offering energy for Internet of Things (IoT) devices. Recently, solar cells have also been used as sensors for context awareness sensing due to their sensitivity to varying lighting conditions. In this article, we are the first to use solar cells for symmetric key generation. To generate symmetric keys, we take advantage of photovoltage measurements generated from solar cells equipped with a pair of IoT devices. Symmetric keys are essential for pairing IoT devices and further securing wireless communication. Despite the sensitivity to varying lighting conditions, challenges still remain for the use of solar cells for key generation, such as time unsynchronisation and noisy measurements. To solve these challenges, we design a novel key generation framework, SolarKey, which includes the starting point detection and a compressed sensing-based two-tier key reconciliation method. Extensive experiments have been conducted to evaluate the performance of our proposed key generation method in various environments, which shows the proposed method can improve the key matching rate by up to 25%. We also conduct security analysis and the randomness test, which shows that SolarKey is resilient to common attacks such as the eavesdropping attack and the imitating attack and sufficiently random.

Simultaneous localization and mapping (SLAM) takes in sensor data, e.g., camera frames, and estimates the user's trajectory while creating a map of the surrounding environment. However, existing SLAM evaluation methods are not reference-free, requiring ground-truth trajectories collected from external systems that are infeasible for most scenarios. In this demo, we present Deep SLAM Error Estimator (DeepSEE), a framework that collects features from a standard visual SLAM pipeline as multivariate time series and uses an attention-based neural network to estimate the tracking error at run time. We evaluate DeepSEE in a game engine-based virtual environment, which generates the visual input for DeepSEE and provides the ground-truth trajectory. Demo participants can navigate the virtual environment to create their own trajectories and view the online pose error estimation. This demo showcases how DeepSEE can act as a quality-of-service indicator for downstream applications.

Smartphones are moving towards the fullscreen design for better user experience. This trend forces front cameras to be placed under screen, leading to Under-Screen Cameras (USC). Accordingly, a small area of the screen is made translucent to allow light to reach the USC. In this paper, we utilize the translucent screen's features to inconspicuously modify its pixels, imperceptible to human eyes but inducing perturbations on USC images. These screen perturbations affect deep learning models in image classification and face recognition. They can be employed to protect user privacy, or disrupt the front camera's functionality in the malicious case. We design two methods, one-pixel perturbation and multiple-pixel perturbation, that can add screen perturbations to images captured by USC and successfully fool various deep learning models. Our evaluations, with three commercial full-screen smartphones on testbed datasets and synthesized datasets, show that screen perturbations significantly decrease the average image classification accuracy, dropping from 85% to only 14% for one-pixel perturbation and 5.5% for multiple-pixel perturbation. For face recognition, the average accuracy drops from 91% to merely 1.8% and 0.25%, respectively.

In this paper, we present EV-Eye, a first-of-its-kind large scale multimodal eye tracking dataset aimed at inspiring research on high-frequency eye/gaze tracking. EV-Eye utilizes an emerging bio-inspired event camera to capture independent pixel-level intensity changes induced by eye movements, achieving sub-microsecond latency. Our dataset was curated over a two-week period and collected from 48 participants encompassing diverse genders and age groups. It comprises over 1.5 million near-eye grayscale images and 2.7 billion event samples generated by two DAVIS346 event cameras. Additionally, the dataset contains 675 thousands scene images and 2.7 million gaze references captured by Tobii Pro Glasses 3 eye tracker for cross-modality validation. Compared with existing event-based high-frequency eye tracking datasets, our dataset is significantly larger in size, and the gaze references involve more natural eye movement patterns, i.e., fixation, saccade and smooth pursuit. Alongside the event data, we also present a hybrid eye tracking method as benchmark, which leverages both the near-eye grayscale images and event data for robust and high-frequency eye tracking. We show that our method achieves higher accuracy for both pupil and gaze estimation tasks compared to the existing solution.

This paper presents EMGSense, a low-effort self-supervised domain adaptation framework for sensing applications based on Electromyography (EMG). EMGSense addresses one of the fundamental challenges in EMG cross-user sensing—the significant performance degradation caused by time-varying biological heterogeneity—in a low-effort (data-efficient and label-free) manner. To alleviate the burden of data collection and avoid labor-intensive data annotation, we propose two EMG-specific data augmentation methods to simulate the EMG signals generated in various conditions and scope the exploration in label-free scenarios. We model combating biological heterogeneity-caused performance degradation as a multi-source domain adaptation problem that can learn from the diversity among source users to eliminate EMG heterogeneous biological features. To relearn the target-user-specific biological features from the unlabeled data, we integrate advanced self-supervised techniques into a carefully designed deep neural network (DNN) structure. The DNN structure can seamlessly perform two training stages that complement each other to adapt to a new user with satisfactory performance. Comprehensive evaluations on two sizable datasets collected from 13 participants indicate that EMGSense achieves an average accuracy of 91.9% and 81.2% in gesture recognition and activity recognition, respectively. EMGSense outperforms the state-of-the-art EMG-oriented domain adaptation approaches by 12.5%-17.4% and achieves a comparable performance with the one trained in a supervised learning manner.

We design a system, SolarGest, which can recognize hand gestures near a solar-powered device by analyzing the patterns of the photocurrent. SolarGest is based on the observation that each gesture interferes with incident light rays on the solar panel in a unique way, leaving its discernible signature in harvested photocurrent. Using solar energy harvesting laws, we develop a model to optimize design and usage of SolarGest. To further improve the robustness of SolarGest under non-deterministic operating conditions, we combine dynamic time warping with Z-score transformation in a signal processing pipeline to pre-process each gesture waveform before it is analyzed for classification. We evaluate SolarGest with both conventional opaque solar cells as well as emerging see-through transparent cells. Our experiments demonstrate that SolarGest achieves 99% for six gestures with a single cell and 95% for fifteen gesture with a$2\times 2$solar cell array. The power measuement study suggests that SolarGest consume 44% less power compared to light sensor based systems.

Recent advances in eye tracking have given birth to a new genre of gaze-based context sensing applications, ranging from cognitive load estimation to emotion recognition. To achieve state-of-the-art recognition accuracy, a large-scale, labeled eye movement dataset is needed to train deep learning-based classifiers. However, due to the heterogeneity in human visual behavior, as well as the labor-intensive and privacy-compromising data collection process, datasets for gaze-based activity recognition are scarce and hard to collect. To alleviate the sparse gaze data problem, we present EyeSyn, a novel suite of psychology-inspired generative models that leverages only publicly available images and videos to synthesize a realistic and arbitrarily large eye movement dataset. Taking gaze-based museum activity recognition as a case study, our evaluation demonstrates that EyeSyn can not only replicate the distinct pat-terns in the actual gaze signals that are captured by an eye tracking device, but also simulate the signal diversity that results from dif-ferent measurement setups and subject heterogeneity. Moreover, in the few-shot learning scenario, EyeSyn can be readily incorpo-rated with either transfer learning or meta-learning to achieve 90% accuracy, without the need for a large-scale dataset for training.

The personalization of augmented reality (AR) experiences based on environmental and user context is key to unlocking their full potential. The recent addition of eye tracking to AR headsets provides a convenient method for detecting user context, but complex analysis of raw gaze data is required to detect where a user's attention and thoughts truly lie. In this demo we present Catch My Eye, the first system to incorporate deep neural network (DNN)-based activity recognition from user gaze into a realistic mobile AR app. We develop an edge computing-based architecture to offload context computation from resource-constrained AR devices, and present a working example of content adaptation based on user context, for the scenario of a virtual art gallery. It shows that user activities can be accurately recognized and employed with sufficiently low latency for practical AR applications.

Kinetic energy harvesting technologies have been progressively used to power wearable devices and to sense the context through energy generation patterns. However, detecting human activities with signals from kinetic harvesters still needs improvement due to the use of approaches based on handcrafted features and the overfitting to device location or subjects. Hence, in this article, we present a deep learning architecture that leverages the feature extraction capability of the convolutional neural networks and the construction of the temporal sequences of recurrent neural networks to improve existing classification results. To provide sufficient data for the deep learning classifier, we propose three data augmentation methods to increase intraclass variance simulating new users performing the same activities. The proposed architecture outperforms existing approaches of kinetic harvesting-based human activity recognition by 13% of accuracy when the training data are augmented with the proposed methods. Finally, given the dependency of kinetic harvesting signals on device location and subjects, we employ transfer learning to improve the classification performance when the system is exposed to new subjects and locations. Transfer learning helps to increase classification performance by 30% when the device location is changed and 35% when the data come from a new subject.

Smart space has emerged as a new paradigm that combines sensing, communication, and artificial intelligence technologies to offer various customized services. A fundamental requirement of these services is person identification. Although a variety of person-identification approaches has been proposed, they suffer from several limitations in practical applications, such as low energy efficiency, accuracy degradation, and privacy issue. This article proposes an energy-harvesting-based privacy-preserving gait recognition scheme for smart space, which is named PrivGait. In PrivGait, we extract discriminative features from 1-D gait signal and design an attention-based long short-term memory (LSTM) network to classify different people. Moreover, we leverage a novel Bloom filter-based privacy-preserving technique to address the privacy leakage problem. To demonstrate the feasibility of PrivGait, we design a proof-of-concept prototype using off-the-shelf energy-harvesting hardware. Extensive evaluation results show that the proposed scheme outperforms state of the art by 6%–10% and incurs low system cost while preserving user’s privacy.

Piezoelectric energy harvester (PEH), which generates electricity from stress or vibrations, is attracting tremendous attention as a viable solution to extend battery life of wearable devices. More interestingly, besides the energy harvesting capability, recent research has demonstrated the feasibility of leveraging PEH as an power-free sensor for gait recognition as its stress or vibration patters are significantly influenced by the gait. However, as PEHs are not designed for precise motion sensing, the gait recognition accuracy remains low with conventional classification algorithms. The accuracy deteriorates further when the generated electricity is stored simultaneously. In this work, to achieve high performance gait recognition and efficient energy harvesting at the same time, we make two distinct contributions. First, we propose a preprocessing algorithm to filter out the effect of energy storage on PEH electricity signals. Second, we propose long short-term memory (LSTM) network-based classifiers to accurately capture temporal information in gait-induced electricity generation. We prototype the proposed gait recognition architecture in the form factor of an insole and evaluate its gait recognition as well as energy harvesting performance with 20 subjects. Our results show that the proposed architecture detects human gait with 12 percent higher recall and harvests up to 127 percent more energy while consuming 38 percent less power compared to the state-of-the-art.

Mobile Augmented Reality (AR), which overlays digital content on the real-world scenes surrounding a user, is bringing immersive interactive experiences where the real and virtual worlds are tightly coupled. To enable seamless and precise AR experiences, an image recognition system that can accurately recognize the object in the camera view with low system latency is required. However, due to the pervasiveness and severity of image distortions, an effective and robust image recognition solution for “in the wild” mobile AR is still elusive. In this article, we present CollabAR, an edge-assisted system that provides distortion-tolerant image recognition for mobile AR with imperceptible system latency . CollabAR incorporates both distortion-tolerant and collaborative image recognition modules in its design. The former enables distortion-adaptive image recognition to improve the robustness against image distortions, while the latter exploits the spatial-temporal correlation among mobile AR users to improve recognition accuracy. Moreover, as it is difficult to collect a large-scale image distortion dataset, we propose a Cycle-Consistent Generative Adversarial Network-based data augmentation method to synthesize realistic image distortion. Our evaluation demonstrates that CollabAR achieves over 85% recognition accuracy for “in the wild” images with severe distortions, while reducing the end-to-end system latency to as low as 18.2 ms.

Conventional radio-frequency (RF) sensing systems rely on either frequency diversity or spatial diversity to ensure high sensing accuracy. Such reliance introduces several practical limitations that hinder the pervasive deployment of existing solutions. To circumvent this prevalent reliance, we present MetaSense, a system that leverages antenna pattern diversity for fine-grained RF sensing. MetaSense incorporates the dynamic metasurface antenna (DMA) and the auxiliary-assisted ensemble multimask learning (AEMML) framework in its design. The DMA is a novel type of antenna that can provide a diverse set of uncorrelated radiation patterns in a low-cost and low-complexity manner. The AEMML is a quality-aware learning framework that can dynamically assess and aggregate the heterogeneous channel measurements from different antenna patterns to ensure high sensing accuracy. It also incorporates a transfer learning model that allows it to generalize to new sensing conditions with few training instances required. We prototype MetaSense and demonstrate its effectiveness on a writing motion recognition task using a custom-designed 2-D DMA. The results show that MetaSense achieves 92% to 98% accuracy in classifying ten miniature writing motions, outperforming a nontunable antenna by 20% in all scenarios. Moreover, when deployed in new sensing positions where limited training instances are available, MetaSense requires as few as five training instances per class to achieve over 90% accuracy.

Internet of Things (IoT) is flourishing and has penetrated deeply into people's daily life. With the seamless connection to the physical world, IoT provides tremendous opportunities to a wide range of applications. However, potential risks exist when the IoT system collects sensor data and uploads it to the Cloud. The leakage of private data can be severe with curious database administrator or malicious hackers who compromise the Cloud. In this work, we propose Kryptein, a compressive-sensing-based lightweight encryption scheme for Cloud-enabled IoT systems to secure the interaction between the IoT devices and the Cloud. Kryptein supports random compressed encryption, statistical computation over cipher, and accurate raw data decryption. According to our evaluation based on two real datasets, Kryptein provides strong protection to the data. It is 250 times faster than other state-of-the-art systems and incurs 120 times less energy consumption. The performance of Kryptein is also measured on off-the-shelf IoT devices, and the result shows Kryptein can run efficiently on IoT devices. After comparing with other state-of-the-art lightweight ciphers on IoT (Simon and Speck), IoT system with Kryptein is expected to have a much more longevity with about 35 percent extended lifetime. Further, experiments illustrated IoT data variance will not affect Kryptein's accuracy in a long term usage, and Krpytein is also able to support basic analytics tasks like machine learning (e.g., classification).

Mobile Augmented Reality (AR), which overlays digital content on the real-world scenes surrounding a user, is bringing immersive interactive experiences where the real and virtual worlds are tightly coupled. To enable seamless and precise AR experiences, an image recognition system that can accurately recognize the object in the camera view with low system latency is required. However, due to the pervasiveness and severity of image distortions, an effective and robust image recognition solution for mobile AR is still elusive. In this paper, we present CollabAR, an edge-assisted system that provides distortion-tolerant image recognition for mobile AR with imperceptible system latency. CollabAR incorporates both distortion-tolerant and collaborative image recognition modules in its design. The former enables distortion-adaptive image recognition to improve the robustness against image distortions, while the latter exploits the ‘spatial-temporal’ correlation among mobile AR users to improve recognition accuracy. We implement CollabAR on four different commodity devices, and evaluate its performance on two multi-view image datasets. Our evaluation demonstrates that CollabAR achieves over 96% recognition accuracy for images with severe distortions, while reducing the end-to-end system latency to as low as 17.8ms for commodity mobile devices.

In this work, we present GazeGraph, a system that leverages human gazes as the sensing modality for cognitive context sensing. GazeGraph is a generalized framework that is compatible with different eye trackers and supports various gaze-based sensing applications. It ensures high sensing performance in the presence of heterogeneity of human visual behavior, and enables quick system adaptation to unseen sensing scenarios with few-shot instances. To achieve these capabilities, we introduce the spatial-temporal gaze graphs and the deep learning-based representation learning method to extract powerful and generalized features from the eye movements for context sensing. Furthermore, we develop a few-shot gaze graph learning module that adapts the `learning to learn' concept from meta-learning to enable quick system adaptation in a data-efficient manner. Our evaluation demonstrates that GazeGraph outperforms the existing solutions in recognition accuracy by 45% on average over three datasets. Moreover, in few-shot learning scenarios, GazeGraph outperforms the transfer learning-based approach by 19% to 30%, while reducing the system adaptation time by 80%.

We propose the use of the conventional energy storage component, i.e., capacitor, in the kinetic-powered wearable IoTs as the sensor to detect human activities. Since activities accumulate energy in the capacitor at different rates, the charging rate of the capacitor can be used to detect the activities. The key advantage of the proposed capacitor-based activity sensing mechanism, called CapSense, is that it obviates the need for sampling the motion signal at a high rate, and thus, significantly reduces power consumption of the wearable device. The challenge we face is that capacitors are inherently non-linear energy accumulators, which leads to significant variations in the charging rates. We solve this problem by jointly configuring the parameters of the capacitor and the associated energy harvesting circuits, which allows us to operate in the charging cycles that are approximately linear. We design and implement a kinetic-powered shoe and conduct experiments with 10 subjects. Our results show that CapSense can classify five different daily activities with 95% accuracy while consuming 57% less system power compared to conventional motion-sensor-based approaches.

Monitoring the daily transportation modes of an individual provides useful information in many application domains, such as urban design, real-time journey recommendation, and providing location-based services. In existing systems, accelerometer and GPS are the dominantly used signal sources for transportation context monitoring which drain out the limited battery life of the wearable devices very quickly. To resolve the high energy consumption issue, in this paper, we present EnTrans, which enables transportation mode detection by using only the kinetic energy harvester as an energy-efficient signal source. The proposed idea is based on the intuition that the vibrations experienced by the passenger during traveling with different transportation modes are distinctive. Thus, voltage signal generated by the energy harvesting devices should contain sufficient features to distinguish different transportation modes. We evaluate our system using over 28 h of data, which is collected by eight individuals using a practical energy harvesting prototype. The evaluation results demonstrate that EnTrans is able to achieve an overall accuracy over 92% in classifying five different modes while saving more than 34% of the system power compared to conventional accelerometer-based approaches.

We design a system, SolarGest, which can recognize hand gestures near a solar-powered device by analyzing the patterns of the photocurrent. SolarGest is based on the observation that each gesture interferes with incident light rays on the solar panel in a unique way, leaving its distinguishable signature in harvested photocurrent. Using solar energy harvesting laws, we develop a model to optimize design and usage of SolarGest. To further improve the robustness of SolarGest under non-deterministic operating conditions, we combine dynamic time warping with Z-score transformation in a signal processing pipeline to pre-process each gesture waveform before it is analyzed for classification. We evaluate SolarGest with both conventional opaque solar cells as well as emerging see-through transparent cells. Our experiments with 6,960 gesture samples for 6 different gestures reveal that even with transparent cells, SolarGest can detect 96% of the gestures while consuming 44% less power compared to light sensor based systems.

Mobile Augmented Reality (AR), which overlays digital information with real-world scenes surrounding a user, provides an enhanced mode of interaction with the ambient world. Contextual AR applications rely on image recognition to identify objects in the view of the mobile device. In practice, due to image distortions and device resource constraints, achieving high performance image recognition for AR is challenging. Recent advances in edge computing offer opportunities for designing collaborative image recognition frameworks for AR. In this demonstration, we present CollabAR, an edge-assisted collaborative image recognition framework. CollabAR allows AR devices that are facing the same scene to collaborate on the recognition task. Demo participants develop an intuition for different image distortions and their impact on image recognition accuracy. We showcase how heterogeneous images taken by different users can be aggregated to improve recognition accuracy and provide a better user experience in AR.

With the rapid development of sensor networks and embedded computing technologies, miniaturized wearable healthcare monitoring devices have become practically feasible. For many of these devices, accelerometer-based user authentication systems by gait analysis are becoming a hot research topic. However, a major bottleneck of such system is it requires continuous sampling of accelerometer, which reduces battery life of wearable sensors. In this paper, we present KEH-Gait, which advocates use of output voltage signal from kinetic energy harvester (KEH) as the source for gait recognition. KEH-Gait is motivated by the prospect of significant power saving by not having to sample the accelerometer at all. Indeed, our measurements show that, compared to conventional accelerometer-based gait detection, KEH-Gait can reduce energy consumption by 82.15 percent. The feasibility of KEH-Gait is based on the fact that human gait has distinctive movement patterns for different individuals, which is expected to leave distinctive patterns for KEH as well. We evaluate the performance of KEH-Gait using two different types of KEH hardware on a data set of 20 subjects. Our experiments demonstrate that, although KEH-Gait yields slightly lower accuracy than accelerometer-based gait detection when single step is used, the accuracy problem can be overcome by the proposed Probability-based Multi-Step Sparse Representation Classification (PMSSRC). Moreover, the security analysis shows that the EER of KEH-Gait against an active spoofing attacker is 11.2 and 14.1 percent using two different types of KEH hardware, respectively.

The feasibility of using vibration energy harvesting (VEH) as an energy-efficient receiver for short-range acoustic data communication has been investigated recently. When data was encoded in acoustic signal within the energy harvesting frequency band and transmitted through a speaker, a VEH receiver was capable of decoding the data by processing the harvested energy signal. Although previous work created new opportunities for simultaneous energy harvesting and communication using the same hardware, the communication makes annoying sounds as the energy harvesting frequency band lies within the sensitive region of human auditory system. In this work, we present a novel modulation scheme to completely hide all communications within background music sound. The proposed modulation exploits sound masking theory to maximize signal to noise ratio of data communication without being audible to the music listener. We capitalize on the existence of repetitive sound patterns within popular music to realize synchronization between the transmitter and the receiver. We implement the proposed modulation within multiple hit songs and demonstrate its efficacy using a real VEH prototype made from off-the-shelf hardware. A user study involving 30 subjects confirms that the proposed modulation can completely hide VEH-based data communication from human perception while achieving up to 14 bps data rate, which is sufficient to transmit short codes or coupons of practical use.

Kinetic energy harvesting (KEH) may help combat battery issues in wearable devices. While the primary objective of KEH is to generate energy from human activities, the harvested energy itself contains information about human activities that most wearable devices try to detect using motion sensors. In principle, it is therefore possible to use KEH both as a power generator and a sensor for human activity recognition (HAR), saving sensor-related power consumption. Our aim is to quantify the potential of human activity recognition from kinetic energy harvesting (HARKE). We evaluate the performance of HARKE using two independent datasets: (i) a public accelerometer dataset converted into KEH data through theoretical modeling; and (ii) a real KEH dataset collected from volunteers performing activities of daily living while wearing a data-logger that we built of a piezoelectric energy harvester. Our results show that HARKE achieves an accuracy of 80 to 95 percent, depending on the dataset and the placement of the device on the human body. We conduct detailed power consumption measurements to understand and quantify the power saving opportunity of HARKE. The results demonstrate that HARKE can save 79 percent of the overall system power consumption of conventional accelerometer-based HAR.

Internet of Things (IoT) is flourishing and has penetrated deeply into people's daily life. With the seamless connection to the physical world, IoT provides tremendous opportunities to a wide range of applications. However, potential risks exist when the IoT system collects sensor data and uploads it to the cloud. The leakage of private data can be severe with curious database administrator or malicious hackers who compromise the cloud. In this work, we propose Kryptein, a compressive-sensing-based encryption scheme for cloud-enabled IoT systems to secure the interaction between the IoT devices and the cloud. Kryptein supports random compressed encryption, statistical decryption, and accurate raw data decryption. According to our evaluation based on two real datasets, Kryptein provides strong protection to the data. It is 250 times faster than other state-of-the-art systems and incurs 120 times less energy consumption. The performance of Kryptein is also measured on off-the-shelf IoT devices, and the result shows Kryptein can run efficiently on IoT devices.

We propose a new activity sensing method, CapSense, which detects activities of daily living (ADL) by sampling the voltage of the kinetic energy harvesting (KEH) capacitor at an ultra low sampling rate. Unlike conventional sensors that generate only instantaneous motion information of the subject, KEH capacitors accumulate and store human generated energy over time. Given that humans produce kinetic energy at distinct rates for different ADL, the KEH capacitor can be sampled only once in a while to observe the energy generation rate and identify the current activity. Thus, with CapSense, it is possible to avoid collecting time series motion data at high frequency, which promises significant power saving for the sensing device. We prototype a shoe-mounted KEH-powered wearable device and conduct experiments with 10 subjects for detecting 5 different activities. Our results show that compared to the existing time-series-based activity recognition, CapSense reduces sampling-induced power consumption by 99% and the overall system power, after considering wireless transmissions, by 75%. CapSense recognizes activities with up to 90%.

With the capability to harvest energy from low frequency motions or vibrations, piezoelectric energy harvesting has become a promising solution to achieve self-powered wearable system. Apart from generating energy to power the wearable devices, the output electricity signal of the PEH can also be used as an information source as it reflects the activity or motion patterns of the user. In this paper, we have designed and built an insole-based user authentication system by leveraging the AC voltage generated by the PEH during human walking. Meanwhile, the generated power is also collected and stored, which could be later used as the power source of the mobile system. By using a dataset of 20 subjects, we have demonstrated that our system can achieve 89.76% of human recognition accuracy when using only one gait cycle signal, and the accuracy can be further increased to 95.86% when two gait cycles are utilized.

This paper investigates the possibility of using a vibration energy harvesting (VEH) device as a communication receiver. By modulating the ambient vibration energy using a transmitting speaker, and demodulating the harvested power at the receiving VEH, we aim to transmit small amounts of data at low rates between two proximate devices. The key advantage of using VEH as a receiver is that the modulated sound waves can be successfully demodulated directly from the harvested power without employing the power-consuming digital signal processing (DSP), which makes a VEH receiver significantly more power efficient than a conventional microphone-based decoder. To address the extremely narrow bandwidth of VEH, we design a simple ON-OFF keying modulation, but optimized for VEH hardware. Experiments with a real VEH device shows that, at a distance of 2 cm, a laptop speaker with the proposed modulation scheme can achieve 30 bps communication for a target bit error rate of less than 1%, which would enable many emerging short range applications, such as mobile payment. The communication range of a laptop can be extended to 80 cm for 5 bps, allowing a range of other audio-based device-to-device communications, such as a web advertisement on a laptop browser transferring tokens to a nearby smartphone. We also demonstrate that the proposed VEH-based sound decoding is resilient to background noise, thanks to its extremely narrow power harvesting bandwidth, which works as a natural noise filter.