The rise of wearables such as fitness trackers and smartwatches has increased the need for strong security to protect personal data. Although two-factor authentication methods improve security, they often require additional user input, making them inconvenient. Recently, hardware flaws in accelerometers and WiFi interfaces have been leveraged to create low-effort two-factor authentication methods. However, these hardware-based device credentials are static, necessitating device replacement if the credentials are compromised. In this study, we introduce an innovative device authentication system that identifies wearables using vibration-based credentials. By utilizing built-in vibration motors and motion sensors (i.e., accelerometers and gyroscopes), our system establishes a unique communication channel to capture the distinct characteristics of each device. Unlike existing methods, our vibration-based credentials are reprogrammable and user-friendly. We develop advanced data processing techniques to minimize the impact of noise, body motion artifacts, and wearing position. We design a lightweight convolutional neural network for feature extraction and device authentication, with a majority vote mechanism to improve identification robustness. Extensive experiments with five different smartwatches demonstrate that our system achieves an average precision of 98% and a recall of 94% under various attacks, demonstrating that including gyroscope data significantly improves performance across different wearing poses and watch orientations.

With the extensive deployment of smart environments and IoT devices, WiFi sensing has proven its significant convenience and contact-free sensing capabilities in supporting a wide range of applications. However, designing a ubiquitous WiFi sensing system for diverse real-world scenarios presents a substantial dilemma, as the system performance deteriorates when the testing data diverges significantly from the training data due to domain variations. To address this dilemma, existing studies need extra efforts to develop new features or even retrain the original model under environmental variations. However, these approaches have not efficiently resolved the dilemma. In this study, we conduct a comprehensive study on the domain variation problem to make WiFi sensing robust and accurate in practical applications. Our definition of domains is comprehensive and includes environmental conditions, surrounding settings, user differences, user orientations, user's positions relative to WiFi sensors, and user participation time frames. We design a novel conformal prediction framework that quantifies the conformity (i.e., similarity) between the testing and training WiFi samples, then labels the testing samples with the most probable class(es). Unlike traditional conformal prediction which relies on data from a single domain, we develop a new statistical (Type I) approach to assess the conformity of the testing WiFi samples to individual training domains and aggregate the outcomes. To further improve the framework's generalization, we design a (Type II) fusion approach that utilizes the inter-relationships among domains for more accurate conformity quantification. Built upon these two methods, kernel density estimation-based and SVM-based methods are developed to compute the conformity scores for new testing samples to make conformal predictions. Extensive experiments, utilizing both self-collected and publicly available datasets show that our framework can improve prediction accuracies ranging from 20.1% to 77.1% in three of the most representative WiFi-based applications across six types of domain variations.

Extended Reality (XR) headsets are increasingly serving as repositories for substantial volumes of sensitive data and gateways to web applications. This transition highlights the need for convenient and secure user authentication solutions. Traditional password/PIN-based schemes are ill-suited to the XR's gesture- and voice-based interfaces and are prone to shoulder-surfing attacks. Some recent XR systems incorporate two-factor authentication, but it requires additional operations on a second device (e.g., a smartphone or wearable). In this work, we introduce the first effortless and inbuilt XR user authentication system by leveraging the harmonics of vibrations excited by users' vital signs. The system is transparent to users (no efforts during enrollment and authentication) and requires no additional hardware. The key idea is that vital signs (i.e., breathing and heart beating) naturally generate low-frequency mechanical vibrations, causing human skull to vibrate and produces harmonic signals. When the harmonics pass the human head, they carry rich biometrics associated with the wearer's skull structure and soft tissues, which can be captured by the XR motion sensors. Instead of directly utilizing the vibrations, we extract more reliable biometrics from the ratios among different harmonic frequencies, which capture wearers' unique head and facial attenuation properties and are non-volatile when the periodicity and amplitude of vital signs fluctuate. We further design an adaptive filter to mitigate the body motion distortions in common XR interactions. By adopting advanced deep learning models with the attention mechanism, our system realizes effective and robust authentication across XR scenarios. Evaluations across 10 months, with 52 users and two popular XR headsets, show that our system can accurately authenticate users with over 95% true positive rates and rejects unauthorized users with over 98% true negative rates under various XR scenarios, with biometrics remaining consistent over long-term periods.

Maintaining concentration is essential for productivity, learning and safety, yet it remains difficult to assess objectively in everyday settings. Traditional methods such as self-reporting and observational studies are subjective and labor-intensive. Wearable sensors can provide physiological data but require constant contact with the user, while camera-based systems raise privacy concerns and are sensitive to illumination and occlusion.

Wearable devices (e.g., fitness trackers and smartwatches) integrating sophisticated sensors are pervasively used in our daily lives these days. Recent research has demonstrated that the vibration motors and motion sensors in these devices offer a powerful sensing channel for various applications including human-computer interaction (HCI) [6, 7], health monitoring [5], and user authentication [1, 3]. However, vibration signals collected from wearable devices are highly susceptible to distortion from body motion artifacts and variations across different devices [4]. Therefore, a comprehensive and systematic sensing testbed is essential to facilitate research in vibration sensing for wearable devices across a wide range of applications.

Studying cognitive impairment and its impact on driving behaviors is crucial for enhancing public safety. To facilitate cognitive impairment studies, we devise a testbed for realworld driving data collection using ubiquitous mobile edge devices (i.e., smartphones) [4]. Toward this end, we develop an application for autonomous data collection using smartphones. To enable robust data collection in real-world driving scenarios, we design a coordinate alignment method that automatically aligns the smartphone's coordinate system with the vehicle's by continuously detecting stationary and straight-line acceleration periods. We also design a two-step segmentation algorithm that first utilizes gyroscope readings to segment rotation-based behaviors (e.g., turning) and then employs accelerometer data to segment non-rotation-based behaviors (e.g., braking). The processed data is then uploaded to a cloud server through WiFi connections for further analysis.

Reliable and ubiquitous user authentication has become essential in smart cities, connected vehicles, and smart homes where users interact with multiple devices in their daily lives. However, existing biometric approaches, such as fingerprint, facial, or voice recognition, often require expensive hardware intrusive interaction, or raise privacy concerns, limiting their scalability in everyday settings [1–3]. To address these limitations, we explore a millimeter-wave (mmWave) testbed that enables palm-based user authentication through fine-grained sensing of palm geometry, skin thickness, and surface texture. By leveraging the widespread integration of mmWave technology in WiGig and 5G, this approach provides a low-cost, contactless, and privacy-preserving alternative to conventional biometrics. This work presents how the mmWave testbed is utilized to investigate cross-environment modeling and robustness in palm-based user authentication. Our system, named mmPalm, captures the reflections of Frequency-Modulated Continuous Wave (FMCW) signals from a user's palm to construct a distinctive palm profile that represents both structural and material characteristics of the hand. These reflections contain rich information about the three-dimensional geometry of the palm, sub-surface tissue variations, and fine surface textures, allowing unique identification without visual or physical contact. The mmWave testbed allows us to systematically collect palm data under varied distances, angles, and environments, providing a consistent platform for model development and evaluation.

Blood pressure (BP) is one of the most essential biomarkers for human health, widely used to diagnose cardiovascular diseases [3] and assess mental states [2, 5]. It is considered Protected Health Information (PHI) under HIPAA, and access to it typically requires explicit user consent. In this work, we uncover a novel privacy breach in the metaverse usage: a user's private BP information can be covertly and continuously surveilled using the unrestricted in-built motion sensors present in commodity VR headsets.

The adoption of augmented reality/virtual reality (AR/VR) has dramatically risen over the past few years across various application sectors, including immersive gaming, social communication, education, and tourism. The emerging use of AR/VR headsets has also created an excellent opportunity to promote pervasive health monitoring service as most AR/VR devices are already equipped with enriched sensing paradigm and will interact with users for a long time. In this talk, we aim to explore innovative technologies that enable fine-grained and personalized health status monitoring (e.g. vital signs and user identities) leveraging facial vibrations captured by the in-built motion sensor testbed on commodity AR/VR headsets. On one hand, it provides real-time health information required in virtual healthcare applications. For instance, a doctor can continuously monitor a patient's vital signs during the tele-medicine session at home, which helps the doctor to realize timely and precise diagnoses [2]. On the other hand, as people are spending increasing time in cyberspace (e.g., Metaverse), exposure to virtual and immersive contents requires high concentration on users' mind. Such usage cases may significantly increase the visual and psychological burden and induce potential health issues (e.g., anxiety, hypertension, sleep disorders) [1, 3, 5].

Blood pressure (BP) is one of the most essential biomarkers for various diseases. It is considered protected health information under HIPAA and usually needs the user's consent for access. In this work, we uncover an insidious privacy breach in metaverse usage: private BP information can be covertly obtained from unrestricted motion sensors in virtual reality (VR) headsets. The insight is that the motion sensors can capture the subtle vibrations induced by the blood waves in the major arteries. Such vibrations are highly correlated with users' cardiac cycles and BP. As adversaries can continuously obtain motion sensor data from VR headsets without users' consent, they can derive and collect users' BP information in metaverse apps or websites, leading to more severe consequences, such as discrimination, exploitation, and targeted harassment. To demonstrate this severe privacy leakage in the meta-verse, we develop a practical attack, BPSniff, which can reconstruct fine-grained blood flow patterns and derive BP based on motion sensor data from users' VR headsets. BP-Sniff is the first practical attack revealing the BP leakage in the metaverse without using dedicated equipment. Unlike previous mobile sensing approaches that require user-specific calibration, BPSniff bypasses this constraint, enabling truly stealthy passive BP attacks at scale. Our attack first employs a variational autoencoder to reconstruct high-fidelity blood flow patterns from VR headset motion sensor data. We then develop an Adam-optimized long short-term memory (LSTM) regression model that leverages BP-related fiducial features from successive blood flow patterns to continuously estimate the user's BP. We evaluate BPSniff through extensive experiments and a longitudinal study of 8 weeks, involving 37 participants and two VR headset models. The results show that BPSniff can achieve low mean errors of 1.75 mmHg for systolic blood pressure (SBP) and 1.34 mmHg for diastolic blood pressure (DBP), which are comparable to commercial BP monitors and satisfy the standard (i.e., mean error ≤ 5.0 mmHg) specified by FDA's AAMI protocol.

In AR/VR devices, the voice interface, serving as one of the primary AR/VR control mechanisms, enables users to interact naturally using speeches (voice commands) for accessing data, controlling applications, and engaging in remote communication/meetings. Voice authentication can be adopted to protect against unauthorized speech inputs. However, existing voice authentication mechanisms are usually susceptible to voice spoofing attacks and are unreliable under the variations of phonetic content. In this work, we propose SAFARI, a spoofing-resistant and text-independent speech authentication system that can be seamlessly integrated into AR/VR voice interfaces. The key idea is to elicit phonetic-invariant biometrics from the facial muscle vibrations upon the headset. During speech production, a user's facial muscles are deformed for articulating phoneme sounds. The facial deformations associated with the phonemes are referred to as visemes. They carry rich biometrics of the wearer's muscles, tissue, and bones, which can propagate through the head and vibrate the headset. SAFARI aims to derive reliable facial biometrics from the viseme-associated facial vibrations captured by the AR/VR motion sensors. Particularly, it identifies the vibration data segments that contain rich viseme patterns (prominent visemes) less susceptible to phonetic variations. Based on the prominent visemes, SAFARI learns on the correlations among facial vibrations of different frequencies to extract biometric representations invariant to the phonetic context. The key advantages of SAFARI are that it is suitable for commodity AR/VR headsets (no additional sensors) and is resistant to voice spoofing attacks as the conductive property of the facial vibrations prevents biometric disclosure via the air media or the audio channel. To mitigate the impacts of body motions in AR/VR scenarios, we also design a generative diffusion model trained to reconstruct the viseme patterns from the data distorted by motion artifacts. We conduct extensive experiments with two representative AR/VR headsets and 35 users under various usage and attack settings. We demonstrate that SAFARI can achieve over 96% true positive rate on verifying legitimate users while successfully rejecting different kinds of spoofing attacks with over 97% true negative rates.

Smartphones have emerged as indispensable devices, seamlessly integrating into our daily lives. However, traditional smartphone interfaces, primarily relying on touchscreens, raise privacy concerns and are susceptible to privacy leakages. We thus propose CasePad, an innovative system that leverages low-cost smartphone cases to achieve fine-grained finger activity sensing while preserving users’ privacy. Toward this end, we devise a passive system to exploit acoustic signals generated from finger interactions on the back of the smartphone case. Our novel approach leverages acoustic mini-structures embedded within the smartphone case to regulate the acoustic signals from finger interactions and enhance their diversity. We further develop a multi-task learning framework including a multi-scale shared encoder and task-specific decoders to extract comprehensive acoustic features of finger activities. To achieve precise predictions, we utilize the Multilayer Perceptron (MLP) as an encoder and design a series of loss functions in decoding tailored to the specific characteristics of finger activities. During the offline training, CasePad utilizes raw passive finger activity sound as input and leverages the camera for supervision. With the use of Siamese network to extract feature files that only contain finger activity-specific information, the user does not need to collect data to train their own model. Extensive experimental evaluations with different smartphone models validate CasePad’s high performance, achieving 98.76% classification accuracy in detecting finger activity direction. Additionally, CasePad demonstrates remarkable precision in deriving detailed finger activity characteristics that closely match the ground truth measurements across various finger activities, including position tracking with a mean squared error (MSE) of 10.28 mm, distance estimation with an MSE of 9.32 mm, and speed derivation with a mean absolute error (MAE) of 7.29 mm/s, respectively.

Biometric authentication systems are increasingly needed across a broad range of applications including in smart city environments (e.g., entering hotels, high-rise buildings, train stations, hospitals, and personalizing vehicles settings), and in smart home environments (e.g., controlling smart devices, en-hancing VR/AR experience). Traditional methods, such as face-based and fingerprint-based authentication, usually incur high cost to be installed in all this kind of environments, making them hard to become a ubiquitous authentication approach. In this paper, we develop a ubiquitous low-effort user authentication approach based on palm recognition using millimeter wave (mmWave) signals. Extensive experiments demonstrate that our system achieves 99% authentication accuracy.

Traditional smartphone touchscreens often raise privacy concerns. We thus propose a novel system using low-cost smartphone cases for privacy-preserving finger activity sensing via passive acoustic signals from the back of the smartphone case. It leverages embedded mini-structures to regulate and enhance the acoustic signals gen-erated by finger activities on the case. We develop a multi-task learning framework with a multi-scale shared encoder and task-specific decoders to extract comprehensive acous-tic features of finger activities. During offline training, our system uses raw passive finger activity sound as input and camera supervision. A Siamese network is utilized to extract finger activity-specific feature files, eliminating the need for users to collect training data. Initial experimental evaluations validate the system's superior performance.

The pervasive use of WiFi has driven the recent research in WiFi sensing, converting communication tech into sensing for applications such as activity recognition, user authentication, and vital sign monitoring. Despite the integration of deep learning into WiFi sensing systems, potential security vulnerabilities to adversarial attacks remain unexplored. This paper introduces the first physical attack focusing on deep learning-based WiFi sensing systems, demonstrating how adversaries can subtly manipulate WiFi packet preambles to affect channel state information (CSI), a critical feature in such systems, and thereby influence underlying deep learning models without disrupting regular communication. To realize the proposed attack in practical scenarios, we rigorously analyze and derive the intricate relationship between the pilot symbol and CSI. A novel mechanism is proposed to facilitate quantitive control of receiver-side CSI through minimal modifications to the pilot symbols of WiFi packets at the transmitter. We further develop a perturbation optimization method based on the Carlini & Wagner (CW) attack and a penalty-based training process to ensure the attack's universal efficacy across various CSI responses and noise. The physical attack is implemented and evaluated in two representative WiFi sensing systems (i.e., activity recognition and user authentication) with 35 participants over 3 months. Extensive experiments demonstrate the remarkable attack success rates of 90.47% and 83.83% for activity recognition and user authentication, respectively.

Deep learning-enabled Voice User Interfaces (VUIs) have surpassed human-level performance in acoustic perception tasks. However, the significant cost associated with training these models compels users to rely on third-party data or outsource training services. Such emerging trends have drawn substantial attention to training-phase attacks, particularly backdoor attacks. Such attacks implant hidden trigger patterns (e.g., tones, environmental sounds) into the model during training, thereby manipulating the model's predictions in the inference phase. However, existing backdoor attacks can be easily undermined in practice as the inserted triggers are audible. Users may notice such attacks when listening to the training data and remaining alert for suspicious sounds. In this work, we present a novel audio backdoor attack that exploits completely inaudible triggers in the frequency domain of the audio spectrograms. Specifically, we optimize the trigger to be a frequency-domain pattern with the energy below the noise floor (e.g., background and hardware noises) at any given frequency, thereby rendering the trigger inaudible. To realize such attacks, we design a strategy that automatically generates inaudible triggers in the spectrum supported by commodity playback devices (e.g., smartphones and laptops). We further develop optimization techniques to enhance the trigger's robustness against speech content and onset variations. Experiments on hotword and speaker recognition indicate that our attack can achieve attack success rates of more than 98.2% and 81.0% under digital and physical attack scenarios. The results also demonstrate the trigger's inaudibility with a Signal-to-Noise Ratio (SNR) less than -3.54 dB against background noises. We further verify that our attack can successfully bypass state-of-the-art backdoor defense strategies based on learning and audio processing.

Privacy concerns over the security of personal information have grown in tandem with the spread of smartwatches. However, effective methods for protecting private data on smartwatches are very limited. Personal identity number (PIN) input is the only privacy protection method on off-the-shelf smartwatches, which requires tedious user effort. This is ineffective at securing information such as notifications and attention-grabbing alerts, which may leak personal data to passersby and adversaries, causing embarrassment or revealing sensitive communications. In this work, we propose a novel privacy protection system, TouchTone, that verifies users and secure personal data in a convenient and low-effort manner. Our system employs a challenge-response process to passively capture finger biometrics from an unobtrusive touch gesture using only microphones, speakers, and accelerometer sensors already built in smartwatches. To address smartwatch incompatibility with traditional high-frequency sensing techniques, we develop non-intrusive low-frequency challenge signals and cross-domain sensing techniques (i.e., measuring acoustic signals in the vibration domain) to capture robust and effective features specific to user fingers. A low-cost profile matching-based classifier is designed to enable stand-alone privacy protection on smartwatches. We conduct extensive experiments with 54 participants using varied hardware, environments, noise levels, user motions, and other impact factors, achieving around 97% true positive rate and 2% false positive rate in recognizing participants' identities for privacy protection.

The physical layer secret key generation exploiting wireless channel reciprocity has demonstrated its viability and effectiveness in various wireless scenarios, such as the Internet of Things (IoT) network, mobile communication network, and industrial control system. Most of the existing studies rely on the quantization technique to convert channel measurements into secret bits for confidential communications. However, non-simultaneous packet exchanges in time-division duplex systems and noise effects usually induce inconsistent quantization results and mismatched secret bits. Although recent research has spent significant effort mitigating such non-reciprocity, it is still far from practical error-free key generation. Unlike previous quantization-based approaches, we take a different viewpoint to match the randomly manipulated (i.e., permuted or edited) channel measurements between a pair of users by minimizing their discrepancy holistically. Specifically, two novel secret key generation algorithms based on bipartite graph matching (BMSKG) and edited sequence alignment (SA-SKG) are developed. BM-SKG allows two users to generate the same secret key based on the permutation order of channel measurements, while SASKG aims to align the edited channel measurements between a pair of users for secret key agreement. In both algorithms, one user can preset the secret key and embed encrypted messages in the exchanged data packets, which reduces communication overheads in key generation. Extensive experimental results show that both BM-SKG and SA-SKG algorithms achieve error-free key agreement on channel measurements at a low cost under various scenarios.

Deep learning (DL) has recently become a key technology supporting radio frequency (RF) signal classification applications. Given the heavy DL training requirement, adopting outsourced training is a practical option for RF application developers. However, the outsourcing process exposes a security vulnerability that enables a backdoor attack. While backdoor attacks have been explored in the vision domain, it is rarely explored in the RF domain. In this work, we present a stealthy backdoor attack that targets DL-based RF signal classification. To realize such an attack, we extensively explore the characteristics of the RF data in different applications, which include RF modulation classification and RF fingerprint-based device identification. Then, we design a training-based backdoor trigger generation approach with different optimization procedures for two backdoor attack scenarios (i.e., poison-label and clean-label). Extensive experiments on two RF signal classification datasets show that the attack success rate is over 99.2%, while its classification accuracy for the clean data remains high (i.e., less than a 0.6% drop compared to the clean model). The low NMSE (less than 0.091) indicates the stealthiness of the attack. Additionally, we demonstrate that our attack can bypass existing defense strategies, such as Neural Cleanse and STRIP.

Many mobile applications have resorted to deep neural networks (DNNs) because of their strong inference capabilities. Since both input data and DNN architectures could be sensitive, there is an increasing demand for secure DNN execution on mobile devices. Towards this end, hardware-based trusted execution environments on mobile devices (mobile TEEs), such as ARM TrustZone, have recently been exploited to execute CNN securely. However, running entire DNNs on mobile TEEs is challenging as TEEs have stringent resource and performance constraints. In this work, we develop a novel mobile TEE-based security framework that can efficiently execute the entire DNN in a resource-constrained mobile TEE with minimal inference time overhead. Specifically, we propose a progressive pruning to gradually identify and remove the redundant neurons from a DNN while maintaining a high inference accuracy. Next, we develop a memory optimization method to deallocate the memory storage of the pruned neurons utilizing the low-level programming technique. Finally, we devise a novel adaptive partitioning method that divides the pruned model into multiple partitions according to the available memory in the mobile TEE and loads the partitions into the mobile TEE separately with a minimal loading time overhead. Our experiments with various DNNs and open-source datasets demonstrate that we can achieve 2-30 times less inference time with comparable accuracy compared to existing approaches securing entire DNNs with mobile TEE.

The market size of augmented reality and virtual reality (AR/VR) has been expanding rapidly in recent years, with the use of face-mounted headsets extending beyond gaming to various application sectors, such as education, healthcare, and the military. Despite the rapid growth, the understanding of information leakage through sensor-rich headsets remains in its infancy. Some of the headset's built-in sensors do not require users' permission to access, and any apps and websites can acquire their readings. While theseunrestricted sensors are generally considered free of privacy risks, we find that an adversary could uncover private information by scrutinizing sensor readings, making existing AR/VR apps and websites potential eavesdroppers. In this work, we investigate a novel, unobtrusive privacy attack called FaceReader, which reconstructs high-quality vital sign signals (breathing and heartbeat patterns) based on unrestricted AR/VR motion sensors. FaceReader is built on the key insight that the headset is closely mounted on the user's face, allowing the motion sensors to detect subtle facial vibrations produced by users' breathing and heartbeats. Based on the reconstructed vital signs, we further investigate three more advanced attacks, including gender recognition, user re-identification, and body fat ratio estimation. Such attacks pose severe privacy concerns, as an adversary may obtain users' sensitive demographic/physiological traits and potentially uncover their real-world identities. Compared to prior privacy attacks relying on speeches and activities, FaceReader targets spontaneous breathing and heartbeat activities that are naturally produced by the human body and are unobtrusive to victims. In particular, we design an adaptive filter to dynamically mitigate the impacts of body motions. We further employ advanced deep-learning techniques to reconstruct vital sign signals, achieving signal qualities comparable to those of dedicated medical instruments, as well as deriving sensitive gender, identity, and body fat information. We conduct extensive experiments involving 35 users on three types of mainstream AR/VR headsets across 3 months. The results reveal that FaceReader can reconstruct vital signs with low mean errors and accurately detect gender (over 93.33%). The attack can also link/re-identify users across different apps, websites, and longitudinal sessions with over 97.83% accuracy. Furthermore, we present the first successful attempt at revealing body fat information from motion sensor data, achieving a remarkably low estimation error of 4.43%.

Recently, deep learning (DL) has become one of the key technologies supporting radio frequency (RF) signal classification applications. Given the heavy DL training requirement, adopting outsourced training is a practical option for RF application developers. However, the outsourcing process exposes a security vulnerability that enables a backdoor attack. While backdoor attacks have been explored in the computer vision domain, it is rarely explored in the RF domain. In this work, we present a stealthy backdoor attack that targets DL-based RF signal classification. To realize such an attack, we extensively explore the characteristics of the RF data in different applications, which include RF modulation classification and RF fingerprint-based device identification. Particularly, we design a training-based backdoor trigger generation approach with an optimization procedure that not only accommodates dynamic application inputs but also is stealthy to RF receivers. Extensive experiments on two RF signal classification datasets show that the average attack success rate of our backdoor attack is over 99.2%, while its classification accuracy for the clean data remains high (i.e., less than a 0.6% drop compared to the clean model). Additionally, we demonstrate that our attack can bypass existing defense strategies, such as Neural Cleanse and STRIP.

Emotional state leakage attracts increasing concerns as it reveals rich sensitive information, such as intent, demo graphic, personality, and health information. Existing emotion recognition techniques rely on vision and audio data, which have limited threat due to the requirements of accessing restricted sensors (e.g., cameras and microphones). In this work, we first investigate the feasibility of detecting the emotional state of people in the vibration domain via zero-permission motion sensors. We find that when voice is being played through a smartphone's loudspeaker or ear speaker, it generates vibration signals on the smartphone surface, which encodes rich emotional information. As the smartphone is the go-to device for almost everyone nowadays, our attack based only on motion sensors raises severe concerns about emotion state leakage. We comprehensively study the relationship between vibration data and human emotion based on several publicly available emotion datasets (e.g., SAVEE, TESS). Time-frequency features and machine learning techniques are developed to determine the emotion of the victim based on speech vibrations. We evaluate our attack on both the ear speakers and loudspeakers on a diverse set of smartphones. The results demonstrate our attack can achieve a high accuracy, with around 95.3% (random guess 14.3%) accuracy for the loudspeaker setting and 60.52% (random guess 14.3%) accuracy for the ear speaker setting.

Personal Identification Number (PIN), as one of the primary means of protecting digital properties and privacy on mobile devices, has been suffering from shoulder surfing attacks and weak password guessing for the long term. Recent years witness the growing interest in two-factor authentication that takes advantage of two different ways for mutual verification, thereby strengthening user authentication's accuracy and reliability. Especially with the popularity of smartwatches, more physiological signals are readily available to facilitate two-factor authentication. This paper presents a lightweight and unobtrusive two-factor authentication scheme, P2Auth, integrating the PIN and unique keystroke-related Photoplethysmography (PPG) measurement on wearables. Specifically, we propose the transformation of the multivariate PPG signal induced by the keystrokes to extract reliable biometric features. We develop short-time energy-based methods to identify the input cases, thus enabling support the authentication for both one-handed and two-handed input cases. Furthermore, we also consider the situation where there is no fixed PIN and design a new enhanced privacy scheme by combining the PPG measurements of different keystrokes to improve authentication security. The experiments involving 15 volunteers demonstrate that our prototype system can achieve an average authentication accuracy of over 95% for one-handed cases and over 90% for two-handed cases.

As years pass, smartphones are becoming a larger part of daily lives, causing users to interact with them more than ever. There are moments, however, when it becomes difficult for the user to operate their device directly. Currently, a user can either touch their devices for direct interaction, or use voice commands for simpler tasks. Although these two methods are very capable means of interacting with the devices, they have their limitations. Touching a physical device is not always practical, while voice commands become ineffective in loud environments. A good example would be if the user is washing dishes in a noisy environment, where neither physical control nor voice commands are convenient. Existing systems of smartphone CSI gesture recognition rely on manual feature extraction which could be hard to implement as gestures grow in number and complexity. We study the feasibility of using lightweight image classification models with minimal preprocessing by implementing and testing the performance of such an architecture. We collect data for five gestures from three setups and two phones, on which our system is able to obtain 90.0% accuracy. Additionally, we investigate the impact of different people, distances, and phones on the system's performance.

Sound source localization is vital for daily tasks such as communication or navigating environments. However, millions of adults struggle with hearing impairment, which limits their ability to identify the direction and distance of sound sources. Traditional methods for sound spatial sensing, such as microphone arrays, are not suitable for resource-constrained IoT devices like smartphones due to power consumption or hardware complexity. To overcome these limitations, this paper proposes EarCase, an alternative scheme that utilizes commercial smartphones with only two microphones to recognize 3D acoustic spatial information. EarCase draws inspiration from the human auditory system, where two ears amplify minute differences in acoustic signals to help pinpoint sound sources. This ability can be regarded as a response function trained through a large amount of sound source information, which can be used to extract spectral cues from a sound source position to the ears drums. We imitate this effect by designing a smartphone case with perforated mini-structures covering the microphones to help the smartphone infer the location of the sound source. Sound waves that pass through the mini-structure will undergo unique changes in diffraction at the hole, amplifying directional information similar to ears. Our scheme uses the top and bottom microphones to eliminate noises and multi-path effects, making the design robust to different sound sources in varying environments. By using only built-in microphones and low-cost phone cases, EarCase provides an accessible tool to enhance the quality of life for hearing impaired individuals. Extensive experimental results show that EarCase achieves high accuracy in localizing sounds, with a mean error of 3.7° at a distance of 200cm and 96% accuracy for real-world sounds (e.g., car horns).

Despite the rapid growth of augmented reality and virtual reality (AR/VR) in various applications, the understanding of information leakage through sensor-rich headsets remains in its infancy. In this poster, we investigate an unobtrusive privacy attack, which exposes users' vital signs and embedded sensitive information (e.g., gender, identity, body fat ratio), based on unrestricted AR/VR motion sensors. The key insight is that the headset is closely mounted on the user's face, allowing the motion sensors to detect facial vibrations produced by users' breathing and heartbeats. Specifically, we employ deep-learning techniques to reconstruct vital signs, achieving signal qualities comparable to dedicated medical instruments, as well as deriving users' gender, identity, and body fat information. Experiments on three types of commodity AR/VR headsets reveal that our attack can successfully reconstruct high-quality vital signs, detect gender (accuracy over 93.33%), re-identify users (accuracy over 97.83%), and derive body fat ratio (error less than 4.43%).

Finger biometrics are widely used by smartphones as a secure and user-friendly credential for privacy protection. However, this information is difficult to measure without high-resolution images, leaving most works to treat this as an image-domain problem. We demonstrate that low-effort alternatives on smartphones are possible through the use of sound propagation in ubiquitous smartphone cases. Inexpensive and widely adopted, smartphone cases are always in contact with fingers, making them ideal for collecting finger biometrics. We thus design BioCase, an acoustic sensing system that leverages smartphone cases equipped with mini-structures to capture unique biometric-hybrid signatures (i.e., reflections influenced by the user's fingertip physiology and behavior) for smartphone privacy protection. The system generates inaudible structure-borne sound and measure the propagation through the smartphone case, mini-structures, and user finger. The design of the mini-structure controls the behavior of structure-borne sound such that unique responses are produced when different users and fingers touch the smartphone case. This enables low-cost, low-effort privacy protection, merely touching the smartphone case can authenticate users. Comprehensive experiments with 46 users over 10 weeks demonstrate BioCase can differentiate users with over 94% accuracy at a 5% false positive rate.

Vital signs (e.g., breathing and heart rates) and personal identities are essential information for personalized medicine and healthcare. The popularity of augmented reality/virtual reality (AR/VR) provides an excellent opportunity for enabling long-term health monitoring in a broad range of scenarios, including virtual entertainment, education, and telemedicine. However, commercial-off-the-shelf AR/VR devices do not have dedicated biosensors for providing vital signs and personal identities. In this work, we propose a novel framework that can generate fine-grained vital sign signals and other personalized health information of an AR/VR user through passive sensing on AR/VR devices. In particular, we find that the user's minute facial vibrations induced by breathing and heart beating can impact the readily available motion sensors on AR/VR headsets, which encode rich vital sign patterns and unique biometrics. The proposed framework further estimates the breathing and heartbeat rates, detects the gender and identity, and derives the body fat percentage of the user. To mitigate the impacts of body movement, we design an adaptive filtering scheme to cancel the spontaneous and non-spontaneous motion artifacts. We also develop unique facial vibration features and deep learning techniques to facilitate vital sign signal reconstruction and user identification. Extensive experiments demonstrate that our framework can achieve a low error of vital sign signal reconstruction and rate measurement, along with 95.51% and 93.33% accuracy on identity and gender recognition.

There is a growing trend for people to perform work-outs at home due to the global pandemic of COVID-19 and the stay-at-home policy of many countries. Since a self-designed fitness plan often lacks professional guidance to achieve ideal outcomes, it is important to have an in-home fitness monitoring system that can track the exercise process of users. Traditional camera-based fitness monitoring may raise serious privacy concerns, while sensor-based methods require users to wear dedicated devices. Recently, researchers propose to utilize RF signals to enable non-intrusive fitness monitoring, but these approaches all require huge training efforts from users to achieve a satisfactory performance, especially when the system is used by multiple users (e.g., family members). In this work, we design and implement a fitness monitoring system using a single COTS mm Wave device. The proposed system integrates workout recognition, user identification, multi-user monitoring, and training effort reduction modules and makes them work together in a single system. In particular, we develop a domain adaptation framework to reduce the amount of training data collected from different domains via mitigating impacts caused by domain characteristics embedded in mm Wave signals. We also develop a GAN-assisted method to achieve better user identification and workout recognition when only limited training data from the same domain is available. We propose a unique spatialtemporal heatmap feature to achieve personalized workout recognition and develop a clustering-based method for concurrent workout monitoring. Extensive experiments with 14 typical workouts involving 11 participants demonstrate that our system can achieve 97% average workout recognition accuracy and 91% user identification accuracy.

The open nature of voice input makes voice assistant (VA) systems vulnerable to various acoustic attacks (e.g., replay and voice synthesis attacks). A simple yet effective way for adversaries to launch these attacks is to hide behind barriers (e.g., a wall, a window, or a door) and give unauthorized voice commands without being observed by legitimate users. In this work, we develop an automated, training-free defense system that can protect VA systems from such thru-barrier acoustic attacks. Our study finds that acoustic signals passing through the barriers generally present a unique frequency-selective effect in the vibration domain. Thus, we propose to devise a system to capture this unique effect of barriers by leveraging low-cost, cross-domain sensing available in users’ wearables. The system replays the audio-domain signals with the wearable’s speaker and captures the conductive vibrations caused by the audio sounds in the vibration domain via the built-in accelerometer. To improve the proposed system’s reliability, we develop a unique vibration-domain enhancement method to extract the phonemes most sensitive to the frequency-selective effect of barriers. We identify effective vibration-domain features that capture the barriers’ effects in the vibration domain. A 2D-correlation-based method is developed to examine the speech similarity between the recordings from the VA system and the user’s wearable and detect thru-barrier attacks. Extensive experiments with various barriers and environments demonstrate that the proposed defense system can effectively defend random, replay, synthesis, and hidden voice attacks with less than 4% equal error rates.

Deep learning models have become key enablers of voice user interfaces. With the growing trend of adopting outsourced training of these models, backdoor attacks, stealthy yet effective training-phase attacks, have gained increasing attention. They inject hidden trigger patterns through training set poisoning and overwrite the model's predictions in the inference phase. Research in backdoor attacks has been focusing on image classification tasks, while there have been few studies in the audio domain. In this work, we explore the severity of audio-domain backdoor attacks and demonstrate their feasibility under practical scenarios of voice user interfaces, where an adversary injects (plays) an unnoticeable audio trigger into live speech to launch the attack. To realize such attacks, we consider jointly optimizing the audio trigger and the target model in the training phase, deriving a position-independent, unnoticeable, and robust audio trigger. We design new data poisoning techniques and penalty-based algorithms that inject the trigger into randomly generated temporal positions in the audio input during training, rendering the trigger resilient to any temporal position variations. We further design an environmental sound mimicking technique to make the trigger resemble unnoticeable situational sounds and simulate played over-the-air distortions to improve the trigger's robustness during the joint optimization process. Extensive experiments on two important applications (i.e., speech command recognition and speaker recognition) demonstrate that our attack can achieve an average success rate of over 99% under both digital and physical attack settings.

For decades, one-time verification has been the standard for user verification at entry points, office rooms, etc. However, such approaches request users to provide their secrets (e.g., entering passwords and collecting fingerprints) and re-verify (e.g., screen shutdown) manually. Thus, they cannot confirm whether the user is a legitimate or an imposter after verification, which raises the urgent demand for a more convenient and secure solution to perform continuous user verification. However, existing continuous verification methods heavily rely on users' active participation, which is inconvenient. Toward this end, we propose a continuous user verification system, BioTag, which utilizes the low-cost radio frequency identification (RFID) technology to capture unique physiological characteristics rooted in the users' respiration motions for continuous user verification. Specifically, we use two RFID tags attached to a user's chest and abdomen to capture the user's intrinsic respiratory patterns via RFID signals. We develop respiratory feature extraction methods based on waveform morphology analysis and fuzzy wavelet transformation (FWPT) to derive unique biometric information from the user's respiration signals. Furthermore, we develop an adaptive classifier using the gradient boosting decision tree (GBDT) to identify legitimate users and attackers accurately. Extensive experiments involving 41 participants demonstrate that BioTag can robustly authenticate users and detect various types of adversaries with low training effort. In particular, our system can achieve over 95.2% and 94.8% verification accuracy on random attack and imitation attack scenarios, respectively.

The Augmented reality/Virtual reality (AR/VR) industry has ushered in a period of rapid development. The next decade leaves a massive imagination for AR/VR in terms of end product form, software, content, applications, and user increment. The AR & VR technology offers a gazillion of possibilities for smart healthcare. In this poster, we develop an innovative continuous blood pressure (CBP) estimation system leveraging the built-in motion sensors of AR/VR headsets for users. We design a deep learning-based PPG construction scheme using the motion sensor-based cardiac signal and estimate the continuous blood pressure using the regression model. Our experimental results show that our system can continuously estimate both systolic blood pressure (SBP) and diastolic blood pressure (DBP) with a mean error of less than 4 mmHg and 0.9 mmHg respectively within a day.

Millimeter wave (mmWave)-based human activity recognition (HAR) systems have emerged in recent years due to their better privacy preservation and higher-resolution sensing. However, these systems are vulnerable to adversarial attacks. In this work, we propose a universal targeted attack method for mmWave-based HAR system. In particular, a universal perturbation is generated in advance which can be added to new-coming mmWave data to deceive the HAR system, causing it to output our desired label. We validate our proposed attack using a public mmWave dataset. We demonstrate the effectiveness of our proposed universal attack with a high attack success rate of over 95%.

Augmented reality/virtual reality (AR/VR) headsets have attracted millions of users and gained predictable popularity. However, long-period usage of immersive technology may lead to health issues (e.g., cybersickness, anxiety). In this poster, we design a low-cost and personalized healthcare monitoring system grounded on vital sign tracking (i.e., breathing and heartbeat rate tracking), by exploiting built-in AR/VR motion sensors. The key insight is that the conductive vibrations induced by chest and heart movements can propagate through the user's cranial bones, thereby vibrating the AR/VR headset mounted on the user's head. To realize this system, we design signal processing techniques to cancel the human motions and derive the periods of breathing and heartbeat through frequency-domain analyses. We further design a user identification scheme based on respiratory and cardiac biometrics, which works with vital sign monitoring to provide personalized healthcare recommendations. Our experiment shows that the proposed scheme can achieve less than 5.7% error rate on breathing/heartbeat rate estimation and 95% accuracy on user identification.

With the wide deployment of smart environments and IoT devices, WiFi sensing has demonstrated its great convenience and contactless sensing capabilities in supporting a broad array of applications. However, designing a ubiquitous WiFi sensing system for heterogeneous scenarios in practice is still a big dilemma as the system performs poorly when the testing data is significantly different from the training data caused by domain variations. To address this dilemma, existing studies involve extra efforts to develop new features or even to retrain the original model under environmental variations. However, none of them can resolve the dilemma completely. In this work, we conduct a comprehensive study on the domain variation problem to make WiFi sensing robust and accurate in reality. Our definition of domains is comprehensive and includes environments, surrounding settings, user differences, user's facing directions, user's positions relative to WiFi sensors, and user participating time frames. Our innovation is to achieve reliable WiFi sensing across all the domains based on the conformal prediction framework. Our approach quantifies the conformity (i.e., similarity) between the testing WiFi samples and the training samples, then labels the testing samples with the most probable class(es). We develop a novel cross-domain transformal prediction scheme based on the multivariate kernel density estimation to effectively assess and learn the conformity of each domain in the training data. To meet various application-specific requirements, we further develop two approaches to fuse the knowledge of conformity derived from the training domains to perform predictions. Extensive experiments with both self-collected and public datasets show that our framework can improve prediction accuracies from 30% to 74% improvements in three most representative WiFi-based applications across six types of domain variations.

Traditional one-time user authentication is vulnerable to attacks when an adversary can obtain unauthorized privileges after a user’s initial login. Continuous user authentication (CA) has recently shown its great potential by enabling seamless user authentication with few users’ participation. We devise a low-cost system that can exploit users’ pulsatile signals from photoplethysmography (PPG) sensors in commodity wearable devices to perform CA. Our system requires zero user effort and applies to practical scenarios that have nonclinical PPG measurements with human motion artifacts (MAs). We explore the uniqueness of the human cardiac system and develop adaptive MA filtering methods to mitigate the impacts of transient and continuous activities from daily life. Furthermore, we identify general fiducial features and develop an adaptive classifier that can authenticate users continuously based on their cardiac characteristics with little additional training effort. Experiments with our wrist-worn PPG sensing platform on 20 participants under practical scenarios demonstrate that our system can achieve a high CA accuracy of over 90% and a low false detection rate of 4% in detecting random attacks. We show that our MA mitigation approaches can improve the CA accuracy by around 39% under both transient and continuous daily activity scenarios.

Deep learning (DL) has demonstrated great performance in various applications on powerful computers and servers. Recently, with the advancement of more powerful mobile devices (e.g., smartphones and touch pads), researchers are seeking DL solutions that could be deployed on mobile devices. Compared to traditional DL solutions using cloud servers, deploying DL on mobile devices have unique advantages in data privacy, communication overhead, and system cost. This article provides a comprehensive survey for the current studies of adopting and deploying DL on mobile devices. Specifically, we summarize and compare the state-of-the-art DL techniques on mobile devices in various application domains involving vision, speech/speaker recognition, human activity recognition, transportation mode detection, and security. We generalize an optimization pipeline for bringing DL to mobile devices, including model-oriented optimization mechanisms (e.g., pruning and quantization) and nonmodel-oriented optimization mechanisms (e.g., software accelerator and hardware design). Moreover, we summarize popular DL libraries regarding their support to state-of-the-art models (software) and processors (hardware). Based on our summarization, we further provide insights into potential research opportunities for developing DL for mobile devices.

Neuron networks pruning is effective in compressing pre-trained CNNs for their deployment on low-end edge devices. However, few works have focused on reducing the computational cost of pruning and inference. We find that existing pruning methods usually remove parameters without fine-grained impact analysis, making it hard to achieve an optimal solution. This work develops a novel mixture pruning mechanism, MIXP, which can effectively reduce the computational cost of CNNs while maintaining a high weight compression ratio and model accuracy. We propose to remove neuron bond that can effectively reduce convolution computations and weight size in CNNs. We also design an influence factor to analyze the importance of neuron bonds and weights in a fine-grained way so that MIXP could achieve precise pruning with few retraining iterations. Experiments with MNIST, CIFAR-10, and ImageNet datasets demonstrate that MIXP could achieve significantly fewer FLOPs and retraining iterations on four widely-used CNNs than existing pruning methods.

The physical layer secret key generation exploiting wireless channel reciprocity has attracted considerable attention in the past two decades. On-going research have demonstrated its viability in various radio frequency (RF) systems. Most of existing work rely on quantization technique to convert channel measurements into digital binaries that are suitable for secret key generation. However, non-simultaneous packet exchanges in time division duplex systems and noise effects in practice usually create random channel measurements between two users, leading to inconsistent quantization results and mismatched secret bits. While significant efforts were spent in recent research to mitigate such non-reciprocity, no efficient method has been found yet. Unlike existing quantization-based approaches, we take a different viewpoint and perform the secret key agreement by solving a bipartite graph matching problem. Specifically, an efficient dual-permutation secret key generation method, DP-SKG, is developed to match the randomly permuted channel measurements between a pair of users by minimizing their discrepancy holistically. DP-SKG allows two users to generate the same secret key based on the permutation order of channel measurements despite the non-reciprocity over wireless channels. Extensive experimental results show that DP-SKG could achieve error-free key agreement on received signal strength (RSS) with a low cost under various scenarios.

Low-cost in-baggage object identification is highly demanded in enhancing public safety and smart manufacturing. Existing approaches usually require specialized equipment and heavy deployment overhead, making them hard to scale for wide deployment. The recent WiFi-based approach is unsuitable for practical deployment as it did not address dynamic environmental impacts. In this work, we propose an environment-independent in-baggage object identification system by leveraging low-cost WiFi. We exploit the channel state information (CSI) to capture material and shape characteristics to facilitate fine-grained inbaggage object identification. A major challenge of building such a system is that CSI measurements are sensitive to real-world dynamics, such as different types of baggage, time-varying ambient noises and interferences, and different deployment environments. To tackle these problems, we develop WiFi features based on polarized directional antennas that can capture objects’ material and shape characteristics. A convolutional neural network-based model is developed to constructively integrate the WiFi features and perform accurate in-baggage object identification. We also develop a material-based domain adaptation using adversarial learning to facilitate fast deployments in different environments. We conduct extensive experiments involving 14 representation objects, 4 types of bags in 3 different room environments. The results show that our system can achieve over 97% in the same environment, and our domain adaptation method can improve the object identification accuracy by 42% when the system is deployed in a new environment with little training.

Gesture recognition has the potential to become a part of contactless interactions with devices to improve accessibility and ease with applications. As the presence of portable devices remains standard, WiFi will continue to constantly connect these devices. Leveraging this availability, instead of relying on installing special sensors, ubiquitous WiFi sensing devices can decipher motion, thus mitigating additional costs. We develop a low-cost hand gesture recognition system utilizing Channel State Information (CSI) from a few subcarriers in prevalent WiFi signals. This information is sent through a lightweight signal segmentation algorithm and Convolutional Neural Network (CNN) that learns the gestures and successfully distinguishes them. Computationally demanding feature extraction is avoided as it increases processing time and does not scale well with additional gestures. Our model obtains an 96% accuracy rate across three different gestures on average.

Different from traditional gestures, sign language gestures involve a lot of finger-level gestures without wrist or arm movements. They are hard to detect using existing motion sensors-based approaches. We introduce the first low-cost sign language gesture recognition system that can differentiate fine-grained finger movements using the Photoplethysmography (PPG) and motion sensors in commodity wearables. By leveraging the motion artifacts in PPG, our system can accurately recognize sign language gestures when there are large body movements, which cannot be handled by the traditional motion sensor-based approaches. We further explore the feasibility of using both PPG and motion sensors in wearables to improve the sign language gesture recognition accuracy when there are limited body movements. We develop a gradient boost tree (GBT) model and deep neural network-based model (i.e., ResNet) for classification. The transfer learning technique is applied to ResNet-based model to reduce the training effort. We develop a prototype using low-cost PPG and motions sensors and conduct extensive experiments and collect over 7000 gestures from 10 adults in the static and body-motion scenarios. Results demonstrate that our system can differentiate nine finger-level gestures from the American Sign Language with an average recognition accuracy over 98 percent.

Due to the open nature of voice input, voice assistant (VA) systems (e.g., Google Home and Amazon Alexa) are vulnerable to various security and privacy leakages (e.g., credit card numbers, passwords), especially when issuing critical user commands involving large purchases, critical calls, etc. Though the existing VA systems may employ voice features to identify users, they are still vulnerable to various acoustic-based attacks (e.g., impersonation, replay, and hidden command attacks). In this work, we propose a training-free voice authentication system, WearID, leveraging the cross-domain speech similarity between the audio domain and the vibration domain to provide enhanced security to the ever-growing deployment of VA systems. In particular, when a user gives a critical command, WearID exploits motion sensors on the user’s wearable device to capture the aerial speech in the vibration domain and verify it with the speech captured in the audio domain via the VA device’s microphone. Compared to existing approaches, our solution is low-effort and privacy-preserving, as it neither requires users’ active inputs (e.g., replying messages/calls) nor to store users’ privacy-sensitive voice samples for training. In addition, our solution exploits the distinct vibration sensing interface and its short sensing range to sound (e.g., 25cm) to verify voice commands. Examining the similarity of the two domains’ data is not trivial. The huge sampling rate gap (e.g., 8000Hz vs. 200Hz) between the audio and vibration domains makes it hard to compare the two domains’ data directly, and even tiny data noises could be magnified and cause authentication failures. To address the challenges, we investigate the complex relationship between the two sensing domains and develop a spectrogram-based algorithm to convert the microphone data into the lower-frequency “ motion sensor data” to facilitate cross-domain comparisons. We further develop a user authentication scheme to verify that the received voice command originates from the legitimate user based on the cross-domain speech similarity of the received voice commands. We report on extensive experiments to evaluate the WearID under various audible and inaudible attacks. The results show WearID can verify voice commands with 99.8% accuracy in the normal situation and detect 97.2% fake voice commands from various attacks, including impersonation/replay attacks and hidden voice/ultrasound attacks.

Many existing identification approaches require active user input, specialized sensing hardware, or personally identifiable information such as fingerprints or face scans. In this paper, we propose EchoLock, a low-effort identification scheme that validates the user by sensing hand geometry via commodity microphones and speakers. EchoLock can serve as a complementary verification method for high-end devices or as a stand-alone user identification scheme for lower-end devices without using privacy-sensitive features. In addition to security applications, our system can also personalize user interactions with smart devices, such as automatically adapting settings or preferences when different people are holding smart remotes. To this end, we study the impact of hands on structure borne sound propagation in mobile devices and develop a user identification scheme that can measure, quantify, and exploit distinct sound reflections in order to differentiate distinct identities. Particularly, we propose a non-intrusive hand sensing technique to derive unique acoustic features in both time and frequency domain, which can effectively capture the physiological and behavioral traits of a user's hand (e.g., hand contours, finger sizes, holding strengths, and holding styles). Furthermore, learning-based algorithms are developed to robustly identify the user under various environments and conditions. We conduct extensive experiments with 20 participants, gathering 80,000 hand geometry samples using different hardware setups across 160 key use case scenarios. Our results show that EchoLock is capable of identifying users with over 94% accuracy, without requiring any active user input.

Drivers' identities are essential information that can facilitate a broad range of applications. For example, by understanding who is driving the vehicle when an accident happens, insurance companies could determine the liability and payment in a car accident claim case with high confidence. Another example, pick-up service companies could track the identities of their drivers to ensure that authorized drivers are driving esteemed clients to their destinations. While there are existing studies that can utilize video cameras and dedicated sensors to identify drivers, they either have privacy issues or require additional hardware, which is not practical enough for daily uses. In this paper, we devise a low-cost driver identification system, which can determine drivers' identities by using sensors readily available in wearable devices. Our system captures the unique driving behaviors during pervasive but momentary driving events (i.e., turning at intersections) with motion sensors, which are widely integrated into commodity wearable devices (e.g., smartphones and activity trackers). Toward this end, we extensively analyze people's driving behaviors and identify the critical turning events that capture people's unique behavioral patterns for driver identification. We design a fine-grained turning segmentation method that divides sensor data into critical turning stages (i.e., before, during, and after-turn stages), which provide multiple dimensions of turning behavioral metrics facilitating driver identification. The system extracts unique turning behavior features from time and frequency domains to enable driver identification based on drivers' turning behaviors at different types of turns. Extensive experiments are conducted with 12 drivers and various types of turns in real-road conditions. The results demonstrate that our system can identify drivers with high accuracy and low falsepositive rate based on one single turning event.

The ever-growing security issues in various mobile applications and smart devices create an urgent demand for a reliable and convenient user verification method. Traditional verification methods request users to provide their secrets (e.g., entering passwords and collecting fingerprints). We envision that the essential trend of user verification is to free users from active participation in the verification process. Toward this end, we propose a continuous user verification system, which re-uses the widely deployed WiFi infrastructure to capture the unique physiological characteristics rooted in user's respiratory motions. Different from the existing continuous verification approaches, posing dependency on restricted scenarios/user behaviors (e.g., keystrokes and gaits), our system can be easily integrated into any WiFi infrastructure to provide non-intrusive continuous verification. Specifically, we extract the respiration-related signals from the channel state information (CSI) of WiFi. We then derive the user-specific respiratory features based on the waveform morphology analysis and fuzzy wavelet transformation of the respiration signals. Additionally, a deep learning based user verification scheme is developed to identify legitimate users accurately and detect the existence of spoofing attacks. Extensive experiments involving 20 participants demonstrate that the proposed system can robustly verify/identify users and detect spoofers under various types of attacks.

The rapid advancement of social media and communication technology enables video chat to become an important and convenient way of daily communication. However, such convenience also makes personal video clips easily obtained and exploited by malicious users who launch scam attacks. Existing studies only deal with the attacks that use fabricated facial masks, while the liveness detection that targets the playback attacks using a virtual camera is still elusive. In this work, we develop a novel video chat liveness detection system, LiveScreen, which can track the weak light changes reflected off the skin of a human face leveraging chromatic eigenspace differences. We design an inconspicuous challenge frame with minimal intervention to the video chat and develop a robust anomaly frame detector to verify the liveness of the remote user in the video chat using the response to the challenge frame. Furthermore, we propose resilient defense strategies to defeat both naive and intelligent playback attacks leveraging spatial and temporal verification. We implemented a prototype over both laptop and smartphone platforms and conducted extensive experiments in various realistic scenarios. We show that our system can achieve robust liveness detection with accuracy and false detection rates 97.7% (94.8%) and 1% (1.6%) on smartphones (laptops), respectively.

Traditional one-time user authentication processes might cause friction and unfavorable user experience in many widely-used applications. This is a severe problem in particular for security-sensitive facilities if an adversary could obtain unauthorized privileges after a user's initial login. Recently, continuous user authentication (CA) has shown its great potential by enabling seamless user authentication with few active participation. We devise a low-cost system exploiting a user's pulsatile signals from the photoplethysmography (PPG) sensor in commercial wrist-worn wearables for CA. Compared to existing approaches, our system requires zero user effort and is applicable to practical scenarios with non-clinical PPG measurements having motion artifacts (MA). We explore the uniqueness of the human cardiac system and design an MA filtering method to mitigate the impacts of daily activities. Furthermore, we identify general fiducial features and develop an adaptive classifier using the gradient boosting tree (GBT) method. As a result, our system can authenticate users continuously based on their cardiac characteristics so little training effort is required. Experiments with our wrist-worn PPG sensing platform on 20 participants under practical scenarios demonstrate that our system can achieve a high CA accuracy of over 90% and a low false detection rate of 4% in detecting random attacks.

Mobile payment has drawn considerable attention due to its convenience of paying via personal mobile devices at anytime and anywhere, and passcodes (i.e., PINs or patterns) are the first choice of most consumers to authorize the payment. This paper demonstrates a serious security breach and aims to raise the awareness of the public that the passcodes for authorizing transactions in mobile payments can be leaked by exploiting the embedded sensors in wearable devices (e.g., smartwatches). We present a passcode inference system, WristSpy, which examines to what extent the user's PIN/pattern during the mobile payment could be revealed from a single wrist-worn wearable device under different passcode input scenarios involving either two hands or a single hand. In particular, WristSpy has the capability to accurately reconstruct fine-grained hand movement trajectories and infer PINs/patterns when mobile and wearable devices are on two hands through building a Euclidean distance-based model and developing a training-free parallel PIN/pattern inference algorithm. When both devices are on the same single hand, a highly challenging case, WristSpy extracts multi-dimensional features by capturing the dynamics of minute hand vibrations and performs machine-learning based classification to identify PIN entries. Extensive experiments with 15 volunteers and 1600 passcode inputs demonstrate that an adversary is able to recover a user's PIN/pattern with up to 92% success rate within 5 tries under various input scenarios.

The rapid advancement of social media and communication technology enables video chat to become an important and convenient way of daily communication. However, such convenience also makes personal video clips easily obtained and exploited by malicious users who launch scam attacks. Existing studies only deal with the attacks that use fabricated facial masks, while the liveness detection that targets the playback attacks using a virtual camera is still elusive. In this work, we develop a novel video chat liveness detection system, which can track the weak light changes reflected off the skin of a human face leveraging chromatic eigenspace differences. We design an inconspicuous challenge frame with minimal intervention to the video chat and develop a robust anomaly frame detector to verify the liveness of remote user in a video chat session. Furthermore, we propose a resilient defense strategy to defeat both naive and intelligent playback attacks leveraging spatial and temporal verification. The evaluation results show that our system can achieve accurate and robust liveness detection with the accuracy and false detection rate as high as 97.7% (94.8%) and 1% (1.6%) on smartphones (laptops), respectively.

We present a photoplethysmography (PPG)-based continuous user authentication (CA) system leveraging the pervasively equipped PPG sensor in commodity wrist-worn wearables such as the smartwatch. Compared to existing approaches, our system does not require any users' interactions (e.g., performing specific gestures) and is applicable to practical scenarios where the user's daily activities cause motion artifacts (MA). Notably, we design a robust MA removal method to mitigate the impact of MA. Furthermore, we explore the uniqueness of the human cardiac system and extract the fiducial features in the PPG measurements to train the gradient boosting tree (GBT) classifier, which can effectively differentiate users continuously using low training effort. In particular, we build the prototype of our system using a commodity smartwatch and a WebSocket server running on a laptop for CA. In order to demonstrate the practical use of our system, we will demo our prototype under different scenarios (i.e., static and moving) to show it can effectively detect MA caused by daily activities and achieve a high authentication success rate.

This paper subverts the traditional understanding of Photoplethysmography (PPG) and opens up a new direction of the utility of PPG in commodity wearable devices, especially in the domain of human computer interaction of fine-grained gesture recognition. We demonstrate that it is possible to leverage the widely deployed PPG sensors in wrist-worn wearable devices to enable finger-level gesture recognition, which could facilitate many emerging human-computer interactions (e.g., sign-language interpretation and virtual reality). While prior solutions in gesture recognition require dedicated devices (e.g., video cameras or IR sensors) or leverage various signals in the environments (e.g., sound, RF or ambient light), this paper introduces the first PPG-based gesture recognition system that can differentiate fine-grained hand gestures at finger level using commodity wearables. Our innovative system harnesses the unique blood flow changes in a user's wrist area to distinguish the user's finger and hand movements. The insight is that hand gestures involve a series of muscle and tendon movements that compress the arterial geometry with different degrees, resulting in significant motion artifacts to the blood flow with different intensity and time duration. By leveraging the unique characteristics of the motion artifacts to PPG, our system can accurately extract the gesture-related signals from the significant background noise (i.e., pulses), and identify different minute finger-level gestures. Extensive experiments are conducted with over 3600 gestures collected from 10 adults. Our prototype study using two commodity PPG sensors can differentiate nine finger-level gestures from American Sign Language with an average recognition accuracy over 88%, suggesting that our PPG-based finger-level gesture recognition system is promising to be one of the most critical components in sign language translation using wearables.

This work proposes a continuous user verification system based on unique human respiratory-biometric characteristics extracted from the off-the-shelf WiFi signals. Our system innovatively re-uses widely available WiFi signals to capture the unique physiological characteristics rooted in respiratory motions for continuous authentication. Different from existing continuous authentication approaches having limited applicable scenarios due to their dependence on restricted user behaviors (e.g., keystrokes and gaits) or dedicated sensing infrastructures, our approach can be easily integrated into any existing WiFi infrastructure to provide non-invasive continuous authentication independent of user behaviors. Specifically, we extract representative features leveraging waveform morphology analysis and fuzzy wavelet transformation of respiration signals derived from the readily available channel state information (CSI) of WiFi. A respiration-based user authentication scheme is developed to accurately identify users and reject spoofers. Extensive experiments involving 20 subjects demonstrate that the proposed system can achieve a high authentication success rate of over 93% and robustly defend against various types of attacks.

Due to the extensive use of smart devices using them to study the driving behaviors has attracted a lot of researchers. This work demonstrates the problem of identifying drivers based on their driving style using smart phones. For this purpose the turns done by the drivers are being studied. Different sensors are embedded in the smart phones which are being used in order to extract some features to distinguish different drivers. Experiments are being done with four drivers and the results show that our system can distinguish them with high accuracy of 92% using only one turn.

Mobile payment has drawn considerable attention due to its convenience of paying via personal mobile devices at anytime and anywhere, and passcodes (i.e., PINs) are the first choice of most consumers to authorize the payment. This work demonstrates a serious security breach and aims to raise the awareness of the public that the passcodes for authorizing transactions in mobile payments can be leaked by exploiting the embedded sensors in wearable devices (e.g., smartwatches). We present a passcode inference system, which examines to what extent the user's PIN during mobile payment could be revealed from a single wrist-worn wearable device under different input scenarios involving either two hands or a single hand. Extensive experiments with 15 volunteers demonstrate that an adversary is able to recover a user's PIN with high success rate within 5 tries under various input scenarios.

This paper presents a photoplethysmography (PPG)-based continuous user authentication (CA) system, which especially leverages the PPG sensors in wrist-worn wearable devices to identify users. We explore the uniqueness of the human cardiac system captured by the PPG sensing technology. Existing CA systems require either the dedicated sensing hardware or specific gestures, whereas our system does not require any users' interactions but only the wearable device, which has already been pervasively equipped with PPG sensors. Notably, we design a robust motion artifacts (MA) removal method to mitigate the impact of MA from wrist movements. Additionally, we explore the characteristic fiducial features from PPG measurements to efficiently distinguish the human cardiac system. Furthermore, we develop a cardiac-based classifier for user identification using the Gradient Boosting Tree (GBT). Experiments with the prototype of the wrist-worn PPG sensing platform and 10 participants in different scenarios demonstrate that our system can effectively remove MA and achieve a high average authentication success rate over $90%$.

Tracking human sleeping postures and vital signs of breathing and heart rates during sleep is important as it can help to assess the general physical health of a person and provide useful clues for diagnosing possible diseases. Traditional approaches (e.g., polysomnography) are limited to clinic usage. Recent radio frequency-based approaches require specialized devices or dedicated wireless sensors and are only able to track breathing rate. In this paper, we propose to track the vital signs of both breathing rate and heart rate during sleep by using off-the-shelf WiFi without any wearable or dedicated devices. Our system reuses existing WiFi network and exploits the fine-grained channel information to capture the minute movements caused by breathing and heart beats. Our system thus has the potential to be widely deployed and perform continuous long-term monitoring. The developed algorithm makes use of the channel information in both time and frequency domain to estimate breathing and heart rates, and it works well when either individual or two persons are in bed. Our extensive experiments demonstrate that our system can accurately capture vital signs during sleep under realistic settings, and achieve comparable or even better performance comparing to traditional and existing approaches, which is a strong indication of providing noninvasive, continuous fine-grained vital signs monitoring without any additional cost.

User authentication is the critical first step in detecting identity-based attacks and preventing subsequent malicious attacks. However, the increasingly dynamic mobile environments make it harderto always apply cryptographic-based methods for user authentication due to their infrastructural and key management overhead. Exploiting non-cryptographic based techniques grounded on physical layer properties to perform user authentication appears promising. In this work, the use of channel state information (CSI), which is available from off-the-shelf WiFi devices, to perform fine-grained user authentication is explored. Particularly, a user-authentication framework that can work with both stationary and mobile users is proposed. When the user is stationary, the proposed framework builds a user profile for user authentication that is resilient to the presence of a spoofer. The proposed machine learning based user-authentication techniques can distinguish between two users even when they possess similar signal fingerprints and detect the existence of a spoofer. When the user is mobile, it is proposed to detect the presence of a spoofer by examining the temporal correlation of CSI measurements. Both office building and apartment environments show that the proposed framework can filter out signal outliers and achieve higher authentication accuracy compared with existing approaches using received signal strength (RSS).

The proliferation of wearable devices, e.g., smartwatches and activity trackers, with embedded sensors has already shown its great potential on monitoring and inferring human daily activities. This paper reveals a serious security breach of wearable devices in the context of divulging secret information (i.e., key entries) while people are accessing key-based security systems. Existing methods of obtaining such secret information rely on installations of dedicated hardware (e.g., video camera or fake keypad), or training with labeled data from body sensors, which restrict use cases in practical adversary scenarios. In this work, we show that a wearable device can be exploited to discriminate mm-level distances and directions of the user's fine-grained hand movements, which enable attackers to reproduce the trajectories of the user's hand and further to recover the secret key entries. In particular, our system confirms the possibility of using embedded sensors in wearable devices, i.e., accelerometers, gyroscopes, and magnetometers, to derive the moving distance of the user's hand between consecutive key entries regardless of the pose of the hand. Our Backward PIN-Sequence Inference algorithm exploits the inherent physical constraints between key entries to infer the complete user key entry sequence. Extensive experiments are conducted with over 7,000 key entry traces collected from 20 adults for key-based security systems (i.e., ATM keypads and regular keyboards) through testing on different kinds of wearables. Results demonstrate that such a technique can achieve 80 percent accuracy with only one try and more than 90 percent accuracy with three tries. Moreover, the performance of our system is consistently good even under low sampling rate and when inferring long PIN sequences. To the best of our knowledge, this is the first technique that reveals personal PINs leveraging wearable devices without the need for labeled training data and contextual information.

Advanced driver assistance systems and, in particular automated driving offers an unprecedented opportunity to transform the safety, efficiency, and comfort of road travel. Developing such safety technologies requires an understanding of not just common highway and city traffic situations but also a plethora of widely different unusual events (e.g., object on the road way and pedestrian crossing highway, etc.). While each such event may be rare, in aggregate they represent a significant risk that technology must address to develop truly dependable automated driving and traffic safety technologies. By developing technology to scale road data acquisition to a large number of vehicles, this paper introduces a low-cost yet reliable solution, BigRoad, that can derive internal driver inputs (i.e., steering wheel angles, driving speed and acceleration) and external perceptions of road environments (i.e., road conditions and front-view video) using a smartphone and an IMU mounted in a vehicle. We evaluate the accuracy of collected internal and external data using over 140 real-driving trips collected in a 3-month time period. Results show that BigRoad can accurately estimate the steering wheel angle with 0.69 degree median error, and derive the vehicle speed with 0.65 km/h deviation. The system is also able to determine binary road conditions with 95% accuracy by capturing a small number of brakes. We further validate the usability of BigRoad by pushing the collected video feed and steering wheel angle to a deep neural network steering wheel angle predictor, showing the potential of massive data acquisition for training self-driving system using BigRoad.

VibSense pushes the limits of vibration-based sensing to determine the location of a touch on extended surface areas as well as identify the object touching the surface leveraging a single sensor. Unlike capacitive sensing, it does not require conductive materials and compared to audio sensing it is more robust to acoustic noise. It supports a broad array of applications through either passive or active sensing using only a single sensor. In VibSense's passive sensing, the received vibration signals are determined by the location of the touch impact. This allows location discrimination of touches precise enough to enable emerging applications such as virtual keyboards on ubiquitous surfaces for mobile devices. Moreover, in the active mode, the received vibration signals carry richer information of the touching object's characteristics (e.g., weight, size, location and material). This further enables VibSense to match the signals to the trained profiles and allows it to differentiate personal objects in contact with any surface. VibSense is evaluated extensively in the use cases of localizing touches (i.e., virtual keyboards), object localization and identification. Our experimental results demonstrate that VibSense can achieve high accuracy, over 95%, in all these use cases.

The proliferation of wearable devices, e.g., smartwatches and activity trackers, with embedded sensors has already shown its great potential on monitoring and inferring human daily activities. This paper reveals a serious security breach of wearable devices in the context of divulging secret information (i.e., key entries) while people accessing key-based security systems. Existing methods of obtaining such secret information relies on installations of dedicated hardware (e.g., video camera or fake keypad), or training with labeled data from body sensors, which restrict use cases in practical adversary scenarios. In this work, we show that a wearable device can be exploited to discriminate mm-level distances and directions of the user's fine-grained hand movements, which enable attackers to reproduce the trajectories of the user's hand and further to recover the secret key entries. In particular, our system confirms the possibility of using embedded sensors in wearable devices, i.e., accelerometers, gyroscopes, and magnetometers, to derive the moving distance of the user's hand between consecutive key entries regardless of the pose of the hand. Our Backward PIN-Sequence Inference algorithm exploits the inherent physical constraints between key entries to infer the complete user key entry sequence. Extensive experiments are conducted with over 5000 key entry traces collected from 20 adults for key-based security systems (i.e. ATM keypads and regular keyboards) through testing on different kinds of wearables. Results demonstrate that such a technique can achieve 80% accuracy with only one try and more than 90% accuracy with three tries, which to our knowledge, is the first technique that reveals personal PINs leveraging wearable devices without the need for labeled training data and contextual information.

Given the increasing popularity of wearable devices, this paper explores the potential to use wearables for steering and driver tracking. Such capability would enable novel classes of mobile safety applications without relying on information or sensors in the vehicle. In particular, we study how wrist-mounted inertial sensors, such as those in smart watches and fitness trackers, can track steering wheel usage and angle. In particular, tracking steering wheel usage and turning angle provide fundamental techniques to improve driving detection, enhance vehicle motion tracking by mobile devices and help identify unsafe driving. The approach relies on motion features that allow distinguishing steering from other confounding hand movements. Once steering wheel usage is detected, it further uses wrist rotation measurements to infer steering wheel turning angles. Our on-road experiments show that the technique is 99% accurate in detecting steering wheel usage and can estimate turning angles with an average error within 3.4 degrees.

Recent advancements in audio recording on mobile devices have improved audio localization capabilities using phones. Previous research has shown that millimeter level accuracy is capable using an off-the-shelf smartphone. This work demonstrates that such valuable resources in smartphones are readily available for developing consumer applications. We develop a smartphone application called HearHere that utilizes geometric features of sound to categorize a tapping sound within a grid. The geometric features are based on the Time Difference of Arrival (TDoA) of the sound between two microphones. By using this classification, any off the shelf phone with stereo recording capability can be made into a music producing device. The application consists of three major components: system calibration, data collection, and audio localization engine based on TDoA. We demonstrate that we can accurately map the tapping sound six region on various solid surface and turn it into a MIDI controller with six different types of instruments.

This paper utilizes smartphone sensing of vehicle dynamics to determine driver phone use, which can facilitate many traffic safety applications. Our system uses embedded sensors in smartphones, i.e., accelerometers and gyroscopes, to capture differences in centripetal acceleration due to vehicle dynamics. These differences combined with angular speed can determine whether the phone is on the left or right side of the vehicle. Our low infrastructure approach is flexible with different turn sizes and driving speeds. Extensive experiments conducted with two vehicles in two different cities demonstrate that our system is robust to real driving environments. Despite noisy sensor readings from smartphones, our approach can achieve a classification accuracy of over 90 percent with a false positive rate of a few percent. We also find that by combining sensing results in a few turns, we can achieve better accuracy (e.g., 95 percent) with a lower false positive rate. In addition, we seek to exploit the electromagnetic field measurement inside a vehicle to complement vehicle dynamics for driver phone sensing under the scenarios when little vehicle dynamics is present, for example, driving straight on highways or standing at roadsides.

Many smartphone apps routinely gather various private user data and send them to advertisers. Despite recent study on protection mechanisms and analysis on apps' behavior, the understanding about the consequences of such privacy losses remains limited. In this paper we investigate how much an advertiser can infer about users' social and community relationships by combining data from multiple applications and across many users. After one month's user study involving about 200 most popular Android apps, we find that an advertiser can infer 90% of the social relationships. We further propose a privacy leakage inference framework and use real mobility traces and Foursquare data to quantify the consequences of privacy leakage. We find that achieving 90% inference accuracy of the social and community relationships requires merely 3 weeks' user data. The discoveries underscore the importance of early adoption of privacy protection mechanisms.

This paper explores the limits of audio ranging on mobile devices in the context of a keystroke snooping scenario. Acoustic keystroke snooping is challenging because it requires distinguishing and labeling sounds generated by tens of keys in very close proximity. Existing work on acoustic keystroke recognition relies on training with labeled data, linguistic context, or multiple phones placed around a keyboard --- requirements that limit usefulness in an adversarial context. In this work, we show that mobile audio hardware advances can be exploited to discriminate mm-level position differences and that this makes it feasible to locate the origin of keystrokes from only a single phone behind the keyboard. The technique clusters keystrokes using time-difference of arrival measurements as well as acoustic features to identify multiple strokes of the same key. It then computes the origin of these sounds precise enough to identify and label each key. By locating keystrokes this technique avoids the need for labeled training data or linguistic context. Experiments with three types of keyboards and off-the-shelf smartphones demonstrate scenarios where our system can recover $94\%$ of keystrokes, which to our knowledge, is the first single-device technique that enables acoustic snooping of passwords.

Tracking human vital signs of breathing and heart rates during sleep is important as it can help to assess the general physical health of a person and provide useful clues for diagnosing possible diseases. Traditional approaches (e.g., Polysomnography (PSG)) are limited to clinic usage. Recent radio frequency (RF) based approaches require specialized devices or dedicated wireless sensors and are only able to track breathing rate. In this work, we propose to track the vital signs of both breathing rate and heart rate during sleep by using off-the-shelf WiFi without any wearable or dedicated devices. Our system re-uses existing WiFi network and exploits the fine-grained channel information to capture the minute movements caused by breathing and heart beats. Our system thus has the potential to be widely deployed and perform continuous long-term monitoring. The developed algorithm makes use of the channel information in both time and frequency domain to estimate breathing and heart rates, and it works well when either individual or two persons are in bed. Our extensive experiments demonstrate that our system can accurately capture vital signs during sleep under realistic settings, and achieve comparable or even better performance comparing to traditional and existing approaches, which is a strong indication of providing non-invasive, continuous fine-grained vital signs monitoring without any additional cost.

The Global Positioning System (GPS) is widely used in critical infrastructures but is vulnerable to radio frequency (RF) interference. A common source of interference are commercial drivers that use GPS jammers to circumvent vehicle tracking systems. Existing mechanisms to detect and identify such interference emitting vehicles on roadways require a large number of specialized detectors or a manual observation process. In this paper, we design a practical, automated system to facilitate enforcement actions. Our system combines information from roadside monitoring points at key locations along the roadway as well as mobile detectors (e.g., smartphones and other mobile GPS systems). Rather than attempting precise localization at a given time, the system exploits the inherent variation in driving speeds and the resulting diverging trajectories of vehicles to uniquely identify the interfering vehicle. Through our experiments on a local highway with a vehicle transmitting interference in the 900MHz ISM band, we found that the vehicle identification rate of our mechanism is 65% for a single-point setup and 100% for a two-point setup. We performed 200 hours of passive monitoring of GPS L1 band on roadways and found two episodes of real interference. We also demonstrate that our mobile detector-based crowdsourced smartphone profiles are sufficiently consistent in time and space to enable reliable interference detection.

User authentication is the critical first step to detect identity-based attacks and prevent subsequent malicious attacks. However, the increasingly dynamic mobile environments make it harder to always apply the cryptographic-based methods for user authentication due to their infrastructural and key management overhead. Exploiting non-cryptographic based techniques grounded on physical layer properties to perform user authentication appears promising. In this work, we explore to use channel state information (CSI), which is available from off-the-shelf WiFi devices, to conduct fine-grained user authentication. We propose an user-authentication framework that has the capability to build the user profile resilient to the presence of the spoofer. Our machine learning based user-authentication techniques can distinguish two users even when they possess similar signal fingerprints and detect the existence of the spoofer. Our experiments in both office building and apartment environments show that our framework can filter out the signal outliers and achieve higher authentication accuracy compared with existing approaches using received signal strength (RSS).

Activity monitoring in home environments has become increasingly important and has the potential to support a broad array of applications including elder care, well-being management, and latchkey child safety. Traditional approaches involve wearable sensors and specialized hardware installations. This paper presents device-free location-oriented activity identification at home through the use of existing WiFi access points and WiFi devices (e.g., desktops, thermostats, refrigerators, smartTVs, laptops). Our low-cost system takes advantage of the ever more complex web of WiFi links between such devices and the increasingly fine-grained channel state information that can be extracted from such links. It examines channel features and can uniquely identify both in-place activities and walking movements across a home by comparing them against signal profiles. Signal profiles construction can be semi-supervised and the profiles can be adaptively updated to accommodate the movement of the mobile devices and day-to-day signal calibration. Our experimental evaluation in two apartments of different size demonstrates that our approach can achieve over 96% average true positive rate and less than 1% average false positive rate to distinguish a set of in-place and walking activities with only a single WiFi access point. Our prototype also shows that our system can work with wider signal band (802.11ac) with even higher accuracy.

We investigate using smartphone WiFi signals to track human queues, which are common in many business areas such as retail stores, airports, and theme parks. Real-time monitoring of such queues would enable a wealth of new applications, such as bottleneck analysis, shift assignments, and dynamic workflow scheduling. We take a minimum infrastructure approach and thus utilize a single monitor placed close to the service area along with transmitting phones. Our strategy extracts unique features embedded in signal traces to infer the critical time points when a person reaches the head of the queue and finishes service, and from these inferences we derive a person's waiting and service times. We develop two approaches in our system, one is directly feature-driven and the second uses a simple Bayesian network. Extensive experiments conducted both in the laboratory as well as in two public facilities demonstrate that our system is robust to real-world environments. We show that in spite of noisy signal readings, our methods can measure service and waiting times to within a $10$ second resolution.

Highly accurate indoor localization of smartphones is critical to enable novel location based features for users and businesses. In this paper, we first conduct an empirical investigation of the suitability of WiFi localization for this purpose. We find that although reasonable accuracy can be achieved, significant errors (e.g., 6 8m) always exist. The root cause is the existence of distinct locations with similar signatures, which is a fundamental limit of pure WiFi-based methods. Inspired by high densities of smartphones in public spaces, we propose a peer assisted localization approach to eliminate such large errors. It obtains accurate acoustic ranging estimates among peer phones, then maps their locations jointly against WiFi signature map subjecting to ranging constraints. We devise techniques for fast acoustic ranging among multiple phones and build a prototype. Experiments show that it can reduce the maximum and 80-percentile errors to as small as 2m and 1m, in time no longer than the original WiFi scanning, with negligible impact on battery lifetime.

Secret key generation among wireless devices using physical layer information of radio channel has been an attractive alternative for ensuring security in mobile environments. Received signal strength (RSS) based secret key extraction gains much attention due to its easy accessibility in wireless infrastructure. However, the problem of using RSS to generate keys among multiple devices to ensure secure group communication in practice remains open. In this work, we propose a framework for collaborative key generation among multiple wireless devices leveraging RSS. To deal with mobile devices not within each other’s communication range, we employ relay nodes to achieve reliable key extraction. To enable secure group communication, two protocols are developed to perform collaborative group key generation via star and chain topologies respectively. We further provide the theoretic analysis on the achievable secrecy rate for both star and chain topologies in the presence of an eavesdropper. Our prototype development using MICAz motes and extensive experiments using fading trend based key extraction demonstrate the feasibility of using RSS for group key generation in both indoor and outdoor environments, and concurrently achieving a lower bit mismatch rate compared to existing studies.

Mobile wireless devices play important roles in our daily life, e.g., users often use such devices to take pictures and share with friends via opportunistic peer-to-peer links, which however are intermittent in nature, and hence require the store-and-forward feature proposed in Delay Tolerant Networks to provide useful data sharing opportunities. Moreover, mobile devices may not be willing to forward data items to other devices due to the limited resources. Hence, effective data dissemination schemes need to be designed to encourage nodes to collaboratively share data. We propose a Multi-Receiver Incentive-Based Dissemination (MuRIS) scheme that allows nodes to cooperatively deliver information of interest to one another via chosen paths utilizing few transmissions. Our scheme exploits local historical paths and users' interests information maintained by each node. In addition, the charge and rewarding functions incorporated within our scheme stimulate cooperation among nodes such that the nodes have no incentive to launch edge insertion attacks. Furthermore, our charge and rewarding functions are designed such that the chosen delivery paths mimic efficient multicast tree that results in fewer delivery hops. Extensive simulation studies using real human contact-based mobility traces show that our scheme outperforms existing methods in terms of delivery ratio and transmission efficiency.

We investigate using smartphone WiFi signals to track human queues, which are common in many business areas such as retail stores, airports, and theme parks. Real-time monitoring of such queues would enable a wealth of new applications, such as bottleneck analysis, shift assignments, and dynamic workflow scheduling. We take a minimum infrastructure approach and thus utilize a single monitor placed close to the service area along with transmitting phones. Our strategy extracts unique features embedded in the signal traces to infer the critical time points when a person reaches the head of the queue and finishes service, and from these inferences we derive a person's waiting and service times. We develop a feature driven approach in our system. Extensive experiments conducted both in the laboratory demonstrate that our system is robust to queues with different waiting time. We show that in spite of noisy signal readings, our methods can measure important time periods in queue (e.g., service and waiting times) to within a $10$ second resolution.

This paper utilizes smartphone sensing of vehicle dynamics to determine driver phone use, which can facilitate many traffic safety applications. Our system uses embedded sensors in smartphones, i.e., accelerometers and gyroscopes, to capture differences in centripetal acceleration due to vehicle dynamics. These differences combined with angular speed can determine whether the phone is on the left or right side of the vehicle. Our low infrastructure approach is flexible with different turn sizes and driving speeds. Extensive experiments conducted with two vehicles in two different cities demonstrate that our system is robust to real driving environments. Despite noisy sensor readings from smartphones, our approach can achieve a classification accuracy of over $90\%$ with a false positive rate of a few percent. We also find that by combining sensing results in a few turns, we can achieve better accuracy (e.g., $95\%$) with a lower false positive rate.

Mobile wireless devices (e.g., smartphones, PDAs, and notebooks) play important roles in our daily life, e.g., users often use such devices for bank transactions, keep in touch with friends. Users can also store such information and share with one another via opportunistic peer to peer links. However, peer to peer links are opportunistic links which are intermittent in nature and hence require the store-and-forward feature proposed in Delay Tolerant Networks to provide useful data sharing opportunities. Moreover, due to the limited resources, e.g., communication bandwidth and battery consumption, mobile devices can be selfish and may not be willing to forward data items to other devices that are interested in such items. Hence, effective data dissemination schemes need to be designed to encourage nodes to collaboratively share data. In this paper, we propose a Multi-Receiver Incentive-Based Dissemination (MuRIS) scheme that allows nodes to cooperatively deliver information of interest to one another via chosen delivery paths that utilize few transmissions. Our MuRIS scheme utilizes local historical path and tracks users' interests information maintained by each node. In addition, the charge and reward functions incorporated within our MuRIS scheme stimulate cooperation among nodes such that the nodes have no incentive to launch edge insertion attacks. Furthermore, our charge and reward functions are designed such that the chosen delivery paths mimic efficient multicast tree that results in fewest delivery hops. Extensive simulation studies using real human contact-based mobility traces show that our MuRIS scheme outperforms existing methods in terms of delivery ratio and transmission efficiency.

Securing communication in mobile wireless networks is challenging because the traditional cryptographic-based methods are not always applicable in dynamic mobile wireless environments. Using physical layer information of radio channel to generate keys secretly among wireless devices has been proposed as an alternative in wireless mobile networks. And the Received Signal Strength (RSS) based secret key extraction gains much attention due to the RSS readings are readily available in wireless infrastructure. However, the problem of using RSS to generate keys among multiple devices to ensure secure group communication remains open. In this work, we propose a framework for collaborative key generation among a group of wireless devices leveraging RSS. The proposed framework consists of a secret key extraction scheme exploiting the trend exhibited in RSS resulted from shadow fading, which is robust to outsider adversary performing stalking attacks. To deal with mobile devices not within each other's communication range, we employ relay nodes to achieve reliable key extraction. To enable secure group communication, two protocols, namely star-based and chain-based, are developed in our framework by exploiting RSS from multiple devices to perform group key generation collaboratively. Our experiments in both outdoor and indoor environments confirm the feasibility of using RSS for group key generation among multiple wireless devices under various mobile scenarios. The results also demonstrate that our collaborative key extraction scheme can achieve a lower bit mismatch rate compared to existing works when maintaining the comparable bit generation rate.

Highly accurate indoor localization of smartphones is critical to enable novel location based features for users and businesses. In this paper, we first conduct an empirical investigation of the suitability of WiFi localization for this purpose. We find that although reasonable accuracy can be achieved, significant errors (e.g., $6\sim8m$) always exist. The root cause is the existence of distinct locations with similar signatures, which is a fundamental limit of pure WiFi-based methods. Inspired by high densities of smartphones in public spaces, we propose a peer assisted localization approach to eliminate such large errors. It obtains accurate acoustic ranging estimates among peer phones, then maps their locations jointly against WiFi signature map subjecting to ranging constraints. We devise techniques for fast acoustic ranging among multiple phones and build a prototype. Experiments show that it can reduce the maximum and 80-percentile errors to as small as $2m$ and $1m$, in time no longer than the original WiFi scanning, with negligible impact on battery lifetime.