LoRa has emerged as a strong wireless communication technology for IoT devices. The security challenges of LoRa have also raised various concerns. We propose SymScrab, a simple physical layer solution that significantly improves the security of LoRa without affecting the communication performance or consuming more communication resources. SymScrab encrypts the message by scrambling the baseband samples with a pseudo-random permutation, which is easy to recover by the legitimate receiver who has the same permutation, but impossible for an adversary. We further propose a physical layer Message Integrity Check (MIC) that rejects spoofed packets because spoofed packets do not exhibit expected physical layer features. We provide a rigorous security analysis of SymScrab and prove guarantees for confidentiality and integrity. We implement SymScrab as an extension to the open-source implementation of LoRa, and test it with both over-the-air experiments and simulations. The results confirm that SymScrab does not negatively affect communication performance.

With the extensive deployment of smart environments and IoT devices, WiFi sensing has proven its significant convenience and contact-free sensing capabilities in supporting a wide range of applications. However, designing a ubiquitous WiFi sensing system for diverse real-world scenarios presents a substantial dilemma, as the system performance deteriorates when the testing data diverges significantly from the training data due to domain variations. To address this dilemma, existing studies need extra efforts to develop new features or even retrain the original model under environmental variations. However, these approaches have not efficiently resolved the dilemma. In this study, we conduct a comprehensive study on the domain variation problem to make WiFi sensing robust and accurate in practical applications. Our definition of domains is comprehensive and includes environmental conditions, surrounding settings, user differences, user orientations, user's positions relative to WiFi sensors, and user participation time frames. We design a novel conformal prediction framework that quantifies the conformity (i.e., similarity) between the testing and training WiFi samples, then labels the testing samples with the most probable class(es). Unlike traditional conformal prediction which relies on data from a single domain, we develop a new statistical (Type I) approach to assess the conformity of the testing WiFi samples to individual training domains and aggregate the outcomes. To further improve the framework's generalization, we design a (Type II) fusion approach that utilizes the inter-relationships among domains for more accurate conformity quantification. Built upon these two methods, kernel density estimation-based and SVM-based methods are developed to compute the conformity scores for new testing samples to make conformal predictions. Extensive experiments, utilizing both self-collected and publicly available datasets show that our framework can improve prediction accuracies ranging from 20.1% to 77.1% in three of the most representative WiFi-based applications across six types of domain variations.

Extended Reality (XR) headsets are increasingly serving as repositories for substantial volumes of sensitive data and gateways to web applications. This transition highlights the need for convenient and secure user authentication solutions. Traditional password/PIN-based schemes are ill-suited to the XR's gesture- and voice-based interfaces and are prone to shoulder-surfing attacks. Some recent XR systems incorporate two-factor authentication, but it requires additional operations on a second device (e.g., a smartphone or wearable). In this work, we introduce the first effortless and inbuilt XR user authentication system by leveraging the harmonics of vibrations excited by users' vital signs. The system is transparent to users (no efforts during enrollment and authentication) and requires no additional hardware. The key idea is that vital signs (i.e., breathing and heart beating) naturally generate low-frequency mechanical vibrations, causing human skull to vibrate and produces harmonic signals. When the harmonics pass the human head, they carry rich biometrics associated with the wearer's skull structure and soft tissues, which can be captured by the XR motion sensors. Instead of directly utilizing the vibrations, we extract more reliable biometrics from the ratios among different harmonic frequencies, which capture wearers' unique head and facial attenuation properties and are non-volatile when the periodicity and amplitude of vital signs fluctuate. We further design an adaptive filter to mitigate the body motion distortions in common XR interactions. By adopting advanced deep learning models with the attention mechanism, our system realizes effective and robust authentication across XR scenarios. Evaluations across 10 months, with 52 users and two popular XR headsets, show that our system can accurately authenticate users with over 95% true positive rates and rejects unauthorized users with over 98% true negative rates under various XR scenarios, with biometrics remaining consistent over long-term periods.

Monitoring vital signs such as breathing rate (BR) and heart rate (HR) is crucial for early detection of health issues and supports a wide range of health-related applications. Traditional monitoring methods often involve body-attached medical devices, which can be intrusive and inconvenient for continuous use in daily life. Contactless monitoring using radio frequency (RF) signals has emerged as a promising alternative, but acquiring precise vital sign measurements remains challenging due to the limited sensing resolution of RF devices. In this paper, we design a high-resolution contactless vital sensing system by leveraging advanced beamforming in combination with machine learning (ML) methods. The key idea of our system is to reconstruct fine-grained vital sign measurements from RF signals, achieving low estimation error, comparable to that of dedicated medical devices such as photoplethysmography sensors, respiration monitoring belts. To enhance the reconstruction performance, we integrate an antenna array with double phase shifters to acquire RF data that captures precise chest displacement of human subjects. An encoder-decoder model based on a 1D convolutional neural network is then developed to map the RF signals into vital sign measurements. Extensive evaluations show that our system has low errors of 0.3 beat per minute (BPM) for BR estimation and 2.7 BPM for HR estimation.

Maintaining concentration is essential for productivity, learning and safety, yet it remains difficult to assess objectively in everyday settings. Traditional methods such as self-reporting and observational studies are subjective and labor-intensive. Wearable sensors can provide physiological data but require constant contact with the user, while camera-based systems raise privacy concerns and are sensitive to illumination and occlusion.

With the rapid development of mobile and edge-integrated sensing technologies, 3D point clouds have emerged as a fundamental data modality for understanding and interacting with the physical world. They provide rich spatial and geometric information that enables autonomous driving, mobile robotics, and intelligent IoT devices to perceive and reason about their surroundings in real time. In particular, the development of edge-deployed sensors, such as LiDAR, depth cameras, and structured-light sensors, has made it feasible to capture and process 3D point clouds directly at the network edge, empowering low-latency perception and decision-making for safety-critical systems.

Studying cognitive impairment and its impact on driving behaviors is crucial for enhancing public safety. To facilitate cognitive impairment studies, we devise a testbed for realworld driving data collection using ubiquitous mobile edge devices (i.e., smartphones) [4]. Toward this end, we develop an application for autonomous data collection using smartphones. To enable robust data collection in real-world driving scenarios, we design a coordinate alignment method that automatically aligns the smartphone's coordinate system with the vehicle's by continuously detecting stationary and straight-line acceleration periods. We also design a two-step segmentation algorithm that first utilizes gyroscope readings to segment rotation-based behaviors (e.g., turning) and then employs accelerometer data to segment non-rotation-based behaviors (e.g., braking). The processed data is then uploaded to a cloud server through WiFi connections for further analysis.

Reliable and ubiquitous user authentication has become essential in smart cities, connected vehicles, and smart homes where users interact with multiple devices in their daily lives. However, existing biometric approaches, such as fingerprint, facial, or voice recognition, often require expensive hardware intrusive interaction, or raise privacy concerns, limiting their scalability in everyday settings [1–3]. To address these limitations, we explore a millimeter-wave (mmWave) testbed that enables palm-based user authentication through fine-grained sensing of palm geometry, skin thickness, and surface texture. By leveraging the widespread integration of mmWave technology in WiGig and 5G, this approach provides a low-cost, contactless, and privacy-preserving alternative to conventional biometrics. This work presents how the mmWave testbed is utilized to investigate cross-environment modeling and robustness in palm-based user authentication. Our system, named mmPalm, captures the reflections of Frequency-Modulated Continuous Wave (FMCW) signals from a user's palm to construct a distinctive palm profile that represents both structural and material characteristics of the hand. These reflections contain rich information about the three-dimensional geometry of the palm, sub-surface tissue variations, and fine surface textures, allowing unique identification without visual or physical contact. The mmWave testbed allows us to systematically collect palm data under varied distances, angles, and environments, providing a consistent platform for model development and evaluation.

Blood pressure (BP) is one of the most essential biomarkers for human health, widely used to diagnose cardiovascular diseases [3] and assess mental states [2, 5]. It is considered Protected Health Information (PHI) under HIPAA, and access to it typically requires explicit user consent. In this work, we uncover a novel privacy breach in the metaverse usage: a user's private BP information can be covertly and continuously surveilled using the unrestricted in-built motion sensors present in commodity VR headsets.

The adoption of augmented reality/virtual reality (AR/VR) has dramatically risen over the past few years across various application sectors, including immersive gaming, social communication, education, and tourism. The emerging use of AR/VR headsets has also created an excellent opportunity to promote pervasive health monitoring service as most AR/VR devices are already equipped with enriched sensing paradigm and will interact with users for a long time. In this talk, we aim to explore innovative technologies that enable fine-grained and personalized health status monitoring (e.g. vital signs and user identities) leveraging facial vibrations captured by the in-built motion sensor testbed on commodity AR/VR headsets. On one hand, it provides real-time health information required in virtual healthcare applications. For instance, a doctor can continuously monitor a patient's vital signs during the tele-medicine session at home, which helps the doctor to realize timely and precise diagnoses [2]. On the other hand, as people are spending increasing time in cyberspace (e.g., Metaverse), exposure to virtual and immersive contents requires high concentration on users' mind. Such usage cases may significantly increase the visual and psychological burden and induce potential health issues (e.g., anxiety, hypertension, sleep disorders) [1, 3, 5].

Blood pressure (BP) is one of the most essential biomarkers for various diseases. It is considered protected health information under HIPAA and usually needs the user's consent for access. In this work, we uncover an insidious privacy breach in metaverse usage: private BP information can be covertly obtained from unrestricted motion sensors in virtual reality (VR) headsets. The insight is that the motion sensors can capture the subtle vibrations induced by the blood waves in the major arteries. Such vibrations are highly correlated with users' cardiac cycles and BP. As adversaries can continuously obtain motion sensor data from VR headsets without users' consent, they can derive and collect users' BP information in metaverse apps or websites, leading to more severe consequences, such as discrimination, exploitation, and targeted harassment. To demonstrate this severe privacy leakage in the meta-verse, we develop a practical attack, BPSniff, which can reconstruct fine-grained blood flow patterns and derive BP based on motion sensor data from users' VR headsets. BP-Sniff is the first practical attack revealing the BP leakage in the metaverse without using dedicated equipment. Unlike previous mobile sensing approaches that require user-specific calibration, BPSniff bypasses this constraint, enabling truly stealthy passive BP attacks at scale. Our attack first employs a variational autoencoder to reconstruct high-fidelity blood flow patterns from VR headset motion sensor data. We then develop an Adam-optimized long short-term memory (LSTM) regression model that leverages BP-related fiducial features from successive blood flow patterns to continuously estimate the user's BP. We evaluate BPSniff through extensive experiments and a longitudinal study of 8 weeks, involving 37 participants and two VR headset models. The results show that BPSniff can achieve low mean errors of 1.75 mmHg for systolic blood pressure (SBP) and 1.34 mmHg for diastolic blood pressure (DBP), which are comparable to commercial BP monitors and satisfy the standard (i.e., mean error ≤ 5.0 mmHg) specified by FDA's AAMI protocol.

International audience

As one of the most essential accessories, headsets have been widely used in common online conversations. The metal coil vibration patterns of headset speakers/microphones have been proven to be highly correlated with the speaker-produced/microphone-received sound. This paper presents an online conversation eavesdropping system,RFSpy, which uses only one RFID tag attached on a headset to alternately sense metal coil vibrations of headset speaker and microphone for eavesdropping on speaker-produced and microphone-received sound. In some accessible scenarios, assuming attackers secretly attach a small, battery-free RFID tag under one ear cushion of an eavesdropped user’s headset without being noticed. Meanwhile, RFID readers are camouflaged as decorations placed in/out of rooms to transmit and receive RF signals. When the eavesdropped user talks with other users online through the headset,RFSpyfirst activates the RFID tag to capture the metal coil vibration patterns of headset speaker and microphone upon RF signals. Then,RFSpyreconstructs sound spectrograms from the RF signal-based vibration patterns for not only trained words but also untrained (i.e., out-of-vocabulary) words utilizing designed SSR network. Finally,RFSpyconverts the sound spectrograms to conversation content through sound recognition API. Extensive experiments demonstrate thatRFSpycan eavesdrop on online conversations with out-of-vocabulary words effectively.

The technology of facial expression reconstruction has paved the way for various face-centric applications such as virtual reality (VR) modeling, human-computer interaction, and affective computing. Existing vision-based solutions present challenges in privacy leakage and poor lighting conditions. In this paper, we introduce a nonintrusive facial expression reconstruction system,mm3DFace, which uses a millimeter wave (mmWave) radar to reconstruct facial expressions in a privacy-preserving and passive manner.mm3DFacefirst captures and pre-processes mmWave signals reflected by a human face, and extracts intricate facial geometric features using a ConvNeXt model integrated with triple loss embedding. Subsequently,mm3DFacederives pose-invariant facial representations utilizing region-divided affine transformation, and further generates individual facial shapes with 68 facial landmarks. Then, dynamic facial expressions with 3D facial avatars are reconstructed to exhibit realistic facial expressions. Finally,mm3DFaceenables micro-expression recognition with mmWave signals, which ensures the capability of describing tiny facial changes. Through extensive real-world experiments involving 15 participants,mm3DFaceachieves a normalized mean error of 3.94%, a mean absolute error of 2.30 mm, and a 3D-mean absolute error of 4.10 mm in tracking 68 facial landmarks, which demonstrates the efficacy and practicality ofmm3DFacein real-world 3D facial reconstruction scenarios.

In AR/VR devices, the voice interface, serving as one of the primary AR/VR control mechanisms, enables users to interact naturally using speeches (voice commands) for accessing data, controlling applications, and engaging in remote communication/meetings. Voice authentication can be adopted to protect against unauthorized speech inputs. However, existing voice authentication mechanisms are usually susceptible to voice spoofing attacks and are unreliable under the variations of phonetic content. In this work, we propose SAFARI, a spoofing-resistant and text-independent speech authentication system that can be seamlessly integrated into AR/VR voice interfaces. The key idea is to elicit phonetic-invariant biometrics from the facial muscle vibrations upon the headset. During speech production, a user's facial muscles are deformed for articulating phoneme sounds. The facial deformations associated with the phonemes are referred to as visemes. They carry rich biometrics of the wearer's muscles, tissue, and bones, which can propagate through the head and vibrate the headset. SAFARI aims to derive reliable facial biometrics from the viseme-associated facial vibrations captured by the AR/VR motion sensors. Particularly, it identifies the vibration data segments that contain rich viseme patterns (prominent visemes) less susceptible to phonetic variations. Based on the prominent visemes, SAFARI learns on the correlations among facial vibrations of different frequencies to extract biometric representations invariant to the phonetic context. The key advantages of SAFARI are that it is suitable for commodity AR/VR headsets (no additional sensors) and is resistant to voice spoofing attacks as the conductive property of the facial vibrations prevents biometric disclosure via the air media or the audio channel. To mitigate the impacts of body motions in AR/VR scenarios, we also design a generative diffusion model trained to reconstruct the viseme patterns from the data distorted by motion artifacts. We conduct extensive experiments with two representative AR/VR headsets and 35 users under various usage and attack settings. We demonstrate that SAFARI can achieve over 96% true positive rate on verifying legitimate users while successfully rejecting different kinds of spoofing attacks with over 97% true negative rates.

Smartphones have emerged as indispensable devices, seamlessly integrating into our daily lives. However, traditional smartphone interfaces, primarily relying on touchscreens, raise privacy concerns and are susceptible to privacy leakages. We thus propose CasePad, an innovative system that leverages low-cost smartphone cases to achieve fine-grained finger activity sensing while preserving users’ privacy. Toward this end, we devise a passive system to exploit acoustic signals generated from finger interactions on the back of the smartphone case. Our novel approach leverages acoustic mini-structures embedded within the smartphone case to regulate the acoustic signals from finger interactions and enhance their diversity. We further develop a multi-task learning framework including a multi-scale shared encoder and task-specific decoders to extract comprehensive acoustic features of finger activities. To achieve precise predictions, we utilize the Multilayer Perceptron (MLP) as an encoder and design a series of loss functions in decoding tailored to the specific characteristics of finger activities. During the offline training, CasePad utilizes raw passive finger activity sound as input and leverages the camera for supervision. With the use of Siamese network to extract feature files that only contain finger activity-specific information, the user does not need to collect data to train their own model. Extensive experimental evaluations with different smartphone models validate CasePad’s high performance, achieving 98.76% classification accuracy in detecting finger activity direction. Additionally, CasePad demonstrates remarkable precision in deriving detailed finger activity characteristics that closely match the ground truth measurements across various finger activities, including position tracking with a mean squared error (MSE) of 10.28 mm, distance estimation with an MSE of 9.32 mm, and speed derivation with a mean absolute error (MAE) of 7.29 mm/s, respectively.

Hand pose estimation is a key support for a variety of interactive applications including user interface control, sign language understanding, virtual reality modeling, etc. Existing approaches mainly exploit wearable devices such as gloves or bracelets to estimate hand poses, which may introduce high deploying costs and intrusive user experience. Others rely on vision technologies whereas they could face complicated illuminations and privacy leakage. In this paper, we present a millimeter wave (mmWave) signal-based 3D hand pose estimation system, mmHand, which utilizes a mmWave radar to generate 3D hand skeletons and reconstruct 3D hand meshes. mmHand first leverages mmWave signals to sense a hand and pre-process the signals. Then, mmHand extracts spatial and temporal features using a designed attention-based hourglass network (mmSpaceNet) and Long Short-Term Memory (LSTM), respectively. Based on the extracted features, mmHand further regresses hand joints in 3D space to generate 3D hand skeletons. Finally, 3D hand meshes that continuously describe hand poses with detailed surfaces are reconstructed through a hand Model with Articulated and Non-rigid defOrmations (MANO). Extensive experiments demonstrate that mmHand can accurately generate 3D hand skeletons with 18.3mm mean per joint position error and 95.1 % of correct key points, which indicates the effectiveness of mmHand on hand pose estimation.

Biometric authentication systems are increasingly needed across a broad range of applications including in smart city environments (e.g., entering hotels, high-rise buildings, train stations, hospitals, and personalizing vehicles settings), and in smart home environments (e.g., controlling smart devices, en-hancing VR/AR experience). Traditional methods, such as face-based and fingerprint-based authentication, usually incur high cost to be installed in all this kind of environments, making them hard to become a ubiquitous authentication approach. In this paper, we develop a ubiquitous low-effort user authentication approach based on palm recognition using millimeter wave (mmWave) signals. Extensive experiments demonstrate that our system achieves 99% authentication accuracy.

Traditional smartphone touchscreens often raise privacy concerns. We thus propose a novel system using low-cost smartphone cases for privacy-preserving finger activity sensing via passive acoustic signals from the back of the smartphone case. It leverages embedded mini-structures to regulate and enhance the acoustic signals gen-erated by finger activities on the case. We develop a multi-task learning framework with a multi-scale shared encoder and task-specific decoders to extract comprehensive acous-tic features of finger activities. During offline training, our system uses raw passive finger activity sound as input and camera supervision. A Siamese network is utilized to extract finger activity-specific feature files, eliminating the need for users to collect training data. Initial experimental evaluations validate the system's superior performance.

The pervasive use of WiFi has driven the recent research in WiFi sensing, converting communication tech into sensing for applications such as activity recognition, user authentication, and vital sign monitoring. Despite the integration of deep learning into WiFi sensing systems, potential security vulnerabilities to adversarial attacks remain unexplored. This paper introduces the first physical attack focusing on deep learning-based WiFi sensing systems, demonstrating how adversaries can subtly manipulate WiFi packet preambles to affect channel state information (CSI), a critical feature in such systems, and thereby influence underlying deep learning models without disrupting regular communication. To realize the proposed attack in practical scenarios, we rigorously analyze and derive the intricate relationship between the pilot symbol and CSI. A novel mechanism is proposed to facilitate quantitive control of receiver-side CSI through minimal modifications to the pilot symbols of WiFi packets at the transmitter. We further develop a perturbation optimization method based on the Carlini & Wagner (CW) attack and a penalty-based training process to ensure the attack's universal efficacy across various CSI responses and noise. The physical attack is implemented and evaluated in two representative WiFi sensing systems (i.e., activity recognition and user authentication) with 35 participants over 3 months. Extensive experiments demonstrate the remarkable attack success rates of 90.47% and 83.83% for activity recognition and user authentication, respectively.

Detection and assessment of micro vibrations are crucial tasks in both industrial settings and daily life. However, vibration sensors attached to the target vibrator may introduce potential resonance, and wireless detection methods suffer from severe multipath interference. Fortunately, moiré-based sensing methods have gained recognition in recent years due to their ability to perceive micro motion changes. In this paper, we propose MoiréVib, a micro-vibration detection solution based on moiré patterns for dynamic and high-frequency environments. We attach a printed marker with periodic gratings to the surface of vibration devices to generate moiré patterns, which can amplify micro vibrations due to their low-frequency magnification effect. However, moiré pattern's changes caused by micro vibrations are often overwhelmed by random pixel-level noises, and the limited frame rate of the camera fails to capture high-frequency moiré features. To deal with these problems, we propose a spectrum-based method to refine and enhance the dynamic and micro moiré features. Additionally, we propose a dual-frame-rate-based fusion mechanism to realize high-frequency reconstruction of moiré features. Extensive experimental results show that MoiréVib can realize a median amplitude detection error of 4.37 μm and achieve frequency detection up to 300Hz with a frame rate range of 10~30 fps.

Deep learning-enabled Voice User Interfaces (VUIs) have surpassed human-level performance in acoustic perception tasks. However, the significant cost associated with training these models compels users to rely on third-party data or outsource training services. Such emerging trends have drawn substantial attention to training-phase attacks, particularly backdoor attacks. Such attacks implant hidden trigger patterns (e.g., tones, environmental sounds) into the model during training, thereby manipulating the model's predictions in the inference phase. However, existing backdoor attacks can be easily undermined in practice as the inserted triggers are audible. Users may notice such attacks when listening to the training data and remaining alert for suspicious sounds. In this work, we present a novel audio backdoor attack that exploits completely inaudible triggers in the frequency domain of the audio spectrograms. Specifically, we optimize the trigger to be a frequency-domain pattern with the energy below the noise floor (e.g., background and hardware noises) at any given frequency, thereby rendering the trigger inaudible. To realize such attacks, we design a strategy that automatically generates inaudible triggers in the spectrum supported by commodity playback devices (e.g., smartphones and laptops). We further develop optimization techniques to enhance the trigger's robustness against speech content and onset variations. Experiments on hotword and speaker recognition indicate that our attack can achieve attack success rates of more than 98.2% and 81.0% under digital and physical attack scenarios. The results also demonstrate the trigger's inaudibility with a Signal-to-Noise Ratio (SNR) less than -3.54 dB against background noises. We further verify that our attack can successfully bypass state-of-the-art backdoor defense strategies based on learning and audio processing.

Eavesdropping on human sound is one of the most common but harmful ways to threaten personal privacy. As one of the most essential accessories, headsets have been widely used in common online conversations, such as online calls, video meetings, etc. The metal coil vibration patterns of headset speakers/microphones have been proven to be highly correlated with the speaker-produced/microphone-received sound content. This paper presents an online conversation eavesdropping system, RFSpy, which uses only one RFID tag attached on a headset to alternately sense the metal coil vibrations of headset speaker and microphone for eavesdropping on speaker-produced and microphone-received sound. In some accessible scenarios, such as meeting rooms, offices, etc., assuming attackers secretly attach a small, battery-free RFID tag under one ear cushion of an eavesdropped user's headset without being noticed. Meanwhile, RFID readers are camouflaged as decorations placed in/out of rooms to transmit and receive RF signals. When the eavesdropped user talks with other users online by using the headset, RFSpy first activates the RFID tag attached on the headset to capture the metal coil vibration patterns of headset speaker and microphone upon RF signals. Then, RFSpy reconstructs sound spectrograms from the RF signal-based vibration patterns for not only trained words but also untrained (i.e., out-of-vocabulary) words by utilizing a designed Sound Spectrogram Reconstruction (SSR) network. Finally, RFSpy converts the sound spectrograms to conversation content through a sound recognition API. Extensive experiments in real environments demonstrate that RFSpy can eavesdrop on online conversations with out-of-vocabulary (OOV) words effectively.

Privacy concerns over the security of personal information have grown in tandem with the spread of smartwatches. However, effective methods for protecting private data on smartwatches are very limited. Personal identity number (PIN) input is the only privacy protection method on off-the-shelf smartwatches, which requires tedious user effort. This is ineffective at securing information such as notifications and attention-grabbing alerts, which may leak personal data to passersby and adversaries, causing embarrassment or revealing sensitive communications. In this work, we propose a novel privacy protection system, TouchTone, that verifies users and secure personal data in a convenient and low-effort manner. Our system employs a challenge-response process to passively capture finger biometrics from an unobtrusive touch gesture using only microphones, speakers, and accelerometer sensors already built in smartwatches. To address smartwatch incompatibility with traditional high-frequency sensing techniques, we develop non-intrusive low-frequency challenge signals and cross-domain sensing techniques (i.e., measuring acoustic signals in the vibration domain) to capture robust and effective features specific to user fingers. A low-cost profile matching-based classifier is designed to enable stand-alone privacy protection on smartwatches. We conduct extensive experiments with 54 participants using varied hardware, environments, noise levels, user motions, and other impact factors, achieving around 97% true positive rate and 2% false positive rate in recognizing participants' identities for privacy protection.

In the realm of AR applications and particularly camera-to-screen interactions, camera tracking stands as a crucial technology. However, the ever-increasing demand for tracking accuracy makes it essential to explore a six-degrees of freedom (6-DoF) tracking technology with ultra-high precision to facilitate micro-motion sensing. In this paper, we propose a novel sensing method MoiréTracker to achieve camera’s 6-DoF pose tracking with ultra-high precision. MoiréTracker outputs camera’s continuous 3-DoF trajectory and 3-DoF posture changes according to the captured moiré patterns, which can be produced by the superposition of camera’s Color Filter Array (CFA) and the projection of screen raster on the CFA plane. Thanks to moiré pattern’s high sensitivity to 6-DoF motions, we characterize the relationship between moiré features and camera’s micro pose changes, so as to realize the continuous 6-DoF pose tracking for camera with ultra-high precision. Moreover, our proposal involves a thumbnail-based method aimed at expanding the working range of MoiréTracker, enabling the pervasive camera-to-screen interactions. We implement a prototype system and evaluate its performance in real-world environments. Extensive experiment results show that MoiréTracker achieves the average trajectory error of 1.20 cm and the posture error of 1.07°.

The physical layer secret key generation exploiting wireless channel reciprocity has demonstrated its viability and effectiveness in various wireless scenarios, such as the Internet of Things (IoT) network, mobile communication network, and industrial control system. Most of the existing studies rely on the quantization technique to convert channel measurements into secret bits for confidential communications. However, non-simultaneous packet exchanges in time-division duplex systems and noise effects usually induce inconsistent quantization results and mismatched secret bits. Although recent research has spent significant effort mitigating such non-reciprocity, it is still far from practical error-free key generation. Unlike previous quantization-based approaches, we take a different viewpoint to match the randomly manipulated (i.e., permuted or edited) channel measurements between a pair of users by minimizing their discrepancy holistically. Specifically, two novel secret key generation algorithms based on bipartite graph matching (BMSKG) and edited sequence alignment (SA-SKG) are developed. BM-SKG allows two users to generate the same secret key based on the permutation order of channel measurements, while SASKG aims to align the edited channel measurements between a pair of users for secret key agreement. In both algorithms, one user can preset the secret key and embed encrypted messages in the exchanged data packets, which reduces communication overheads in key generation. Extensive experimental results show that both BM-SKG and SA-SKG algorithms achieve error-free key agreement on channel measurements at a low cost under various scenarios.

The two-factor authentication ($2$FA) has drawn increasingly attention as the mobile devices become more prevalent. For example, the user's possession of the enrolled phone could be used by the$2$FA system as the second proof to protect his/her online accounts. Existing$2$FA solutions mainly require some form of user-device interaction, which may severely affect user experience and creates extra burdens to users. In this work, we propose a secure$2$FA system utilizing the proximity of a user's enrolled phone and the login device as the second proof without requiring the user's interactions. The basic idea of our$2$FA system is to derive location signatures based on acoustic beep signals emitted alternately by both devices and sensing the echoes with microphones, and compare the extracted signatures for proximity detection. Moreover, to further enhance the security of our system, we also design a device authentication scheme which derives the acoustic fingerprint between the login device and enrolled phone to verify the identity of two devices. Given the received beep signal, our system designs a period selection scheme to identify two sound segments accurately: the chirp period is the sound segment propagating directly from the speaker to the microphone whereas the echo period is the sound segment reflected back by surrounding objects. To achieve an accurate proximity detection, we develop a new energy loss compensation extraction scheme by utilizing the extracted chirp periods to estimate the intrinsic differences of energy loss between microphones of the enrolled phone and the login device. Our proximity detection component then conducts the similarity comparison between the identified two echo periods after the energy loss compensation to effectively determine whether the enrolled phone and the login device are in proximity for$2$FA. Moreover, to provide higher security, our device fingerprint-assisted proximity detection further utilizes the overall energy loss between the login device and enrolled phone as their hardware fingerprint to authenticate the identity of two devices. Our experimental results show that our system is accurate in providing$2$FA and robust to both man-in-the-middle (MiM) and co-located attacks across different scenarios and device models.

The indoor location awareness has drawn increasing attention as the mobile apps are used extensively in our daily lives. Existing indoor localization solutions either require a pre-installed infrastructure or can only achieve room-level accuracy, which could not provide a function-location service for mobile devices. In this work, we propose a new active sensing system that enables smartphones to identify some pre-defined indoor locations robustly without requiring any additional sensors or pre-installed infrastructure. The main idea behind our system is to utilize the acoustic signatures, which are derived from the mobile device by emitting a beep signal and selecting its echoes created by dominant reflectors, as the robust fingerprint for location identification. Given the microphone samplings, our system designs a correlation based technique to accurately detect the beginning points of echoes from the received beep signal. To achieve a robust location identification, we develop a new echo selection scheme to select echoes created by dominant reflectors by exploiting the relationships between propagation delays of different orders of echoes. To deal with the variable number of selected echoes, our location identification component then derives histograms from selected echoes and uses the one-against-all SVM classifiers to determine the current location. Our experimental results show that our proposed system is accurate and robust for location identification under various real-world scenarios.

Deep learning (DL) has recently become a key technology supporting radio frequency (RF) signal classification applications. Given the heavy DL training requirement, adopting outsourced training is a practical option for RF application developers. However, the outsourcing process exposes a security vulnerability that enables a backdoor attack. While backdoor attacks have been explored in the vision domain, it is rarely explored in the RF domain. In this work, we present a stealthy backdoor attack that targets DL-based RF signal classification. To realize such an attack, we extensively explore the characteristics of the RF data in different applications, which include RF modulation classification and RF fingerprint-based device identification. Then, we design a training-based backdoor trigger generation approach with different optimization procedures for two backdoor attack scenarios (i.e., poison-label and clean-label). Extensive experiments on two RF signal classification datasets show that the attack success rate is over 99.2%, while its classification accuracy for the clean data remains high (i.e., less than a 0.6% drop compared to the clean model). The low NMSE (less than 0.091) indicates the stealthiness of the attack. Additionally, we demonstrate that our attack can bypass existing defense strategies, such as Neural Cleanse and STRIP.

The indoor location awareness has drawn increasing attention for smart speakers as they become essential to provide function-location services. Existing indoor localization solutions either require add-on equipment or could only achieve room-level accuracy, which could not provide a function-location service for smart speakers. In this work, we propose a location identification system utilizing 3-D acoustic images, which are derived from the smart speaker by emitting a beep signal and sensing echoes created by objects in the surrounding environment with its microphone array, as the proof to identify some pre-defined indoor locations. Given the recorded acoustic samplings captured by the microphone array, our image construction component constructs a virtual imaging hemisphere and steers the array towards each grid of the hemisphere to generate a 3-D acoustic image of the surrounding environment. Moreover, we design a transfer-learning based model to derive effective features from the constructed images, and propose a data augmentation scheme for generating synthesized training images. To achieve a more accurate location identification, we further design a distance estimation scheme to identify the distances between the smart speaker and some major surrounding objects by utilizing the constructed 3-D acoustic image, and then adopt such distance information for location identification. Our experimental results show that our proposed system is accurate and robust for location identification under various real world scenarios.

Wi-Fi sensing can classify human activities because each activity causes unique changes to the channel state information (CSI). Existing WiFi sensing suffers from limited scalability as the system needs to be retrained whenever new classes are added, which causes overheads of data collection and retraining. Cross-domain sensing may fail because the mapping between activities and CSI variations is destroyed when a different environment or user (domain) is involved. This paper proposed a few-shot learning-based WiFi sensing system, named FewSense, which can recognise novel classes in unseen domains with only a few samples. Specifically, a feature extractor was pre-trained offline using the source domain data. When the system was applied in the target domain, a few samples were used to fine-tune the feature extractor for domain adaptation. Inference was made by computing the cosine similarity. FewSense can further boost the classification accuracy by collaboratively fusing inference from multiple receivers. We evaluated the performance of FewSense using three public datasets, i.e., SignFi, Widar, and Wiar. The results show that FewSense with five-shot learning recognised novel classes in unseen domains with an accuracy of 93.9%, 96.5%, and 82.7% on the SignFi, Widar, and Wiar datasets, respectively. Our collaborative sensing model improved system performance by an average of 29.2%.

Many mobile applications have resorted to deep neural networks (DNNs) because of their strong inference capabilities. Since both input data and DNN architectures could be sensitive, there is an increasing demand for secure DNN execution on mobile devices. Towards this end, hardware-based trusted execution environments on mobile devices (mobile TEEs), such as ARM TrustZone, have recently been exploited to execute CNN securely. However, running entire DNNs on mobile TEEs is challenging as TEEs have stringent resource and performance constraints. In this work, we develop a novel mobile TEE-based security framework that can efficiently execute the entire DNN in a resource-constrained mobile TEE with minimal inference time overhead. Specifically, we propose a progressive pruning to gradually identify and remove the redundant neurons from a DNN while maintaining a high inference accuracy. Next, we develop a memory optimization method to deallocate the memory storage of the pruned neurons utilizing the low-level programming technique. Finally, we devise a novel adaptive partitioning method that divides the pruned model into multiple partitions according to the available memory in the mobile TEE and loads the partitions into the mobile TEE separately with a minimal loading time overhead. Our experiments with various DNNs and open-source datasets demonstrate that we can achieve 2-30 times less inference time with comparable accuracy compared to existing approaches securing entire DNNs with mobile TEE.

Speech eavesdropping has long been an important threat to the privacy of individuals and enterprises. Recent research has shown the possibility of deriving private speech information from sound-induced vibrations. Acoustic signals transmitted through a solid medium or air may induce vibrations upon solid surfaces, which can be picked up by various sensors (e.g., motion sensors, high-speed cameras and lasers), without using a microphone. To date, these threats are limited to scenarios where the sensor is in contact with the vibration surface or at least in the visual line-of-sight.

The market size of augmented reality and virtual reality (AR/VR) has been expanding rapidly in recent years, with the use of face-mounted headsets extending beyond gaming to various application sectors, such as education, healthcare, and the military. Despite the rapid growth, the understanding of information leakage through sensor-rich headsets remains in its infancy. Some of the headset's built-in sensors do not require users' permission to access, and any apps and websites can acquire their readings. While theseunrestricted sensors are generally considered free of privacy risks, we find that an adversary could uncover private information by scrutinizing sensor readings, making existing AR/VR apps and websites potential eavesdroppers. In this work, we investigate a novel, unobtrusive privacy attack called FaceReader, which reconstructs high-quality vital sign signals (breathing and heartbeat patterns) based on unrestricted AR/VR motion sensors. FaceReader is built on the key insight that the headset is closely mounted on the user's face, allowing the motion sensors to detect subtle facial vibrations produced by users' breathing and heartbeats. Based on the reconstructed vital signs, we further investigate three more advanced attacks, including gender recognition, user re-identification, and body fat ratio estimation. Such attacks pose severe privacy concerns, as an adversary may obtain users' sensitive demographic/physiological traits and potentially uncover their real-world identities. Compared to prior privacy attacks relying on speeches and activities, FaceReader targets spontaneous breathing and heartbeat activities that are naturally produced by the human body and are unobtrusive to victims. In particular, we design an adaptive filter to dynamically mitigate the impacts of body motions. We further employ advanced deep-learning techniques to reconstruct vital sign signals, achieving signal qualities comparable to those of dedicated medical instruments, as well as deriving sensitive gender, identity, and body fat information. We conduct extensive experiments involving 35 users on three types of mainstream AR/VR headsets across 3 months. The results reveal that FaceReader can reconstruct vital signs with low mean errors and accurately detect gender (over 93.33%). The attack can also link/re-identify users across different apps, websites, and longitudinal sessions with over 97.83% accuracy. Furthermore, we present the first successful attempt at revealing body fat information from motion sensor data, achieving a remarkably low estimation error of 4.43%.

Recently, deep learning (DL) has become one of the key technologies supporting radio frequency (RF) signal classification applications. Given the heavy DL training requirement, adopting outsourced training is a practical option for RF application developers. However, the outsourcing process exposes a security vulnerability that enables a backdoor attack. While backdoor attacks have been explored in the computer vision domain, it is rarely explored in the RF domain. In this work, we present a stealthy backdoor attack that targets DL-based RF signal classification. To realize such an attack, we extensively explore the characteristics of the RF data in different applications, which include RF modulation classification and RF fingerprint-based device identification. Particularly, we design a training-based backdoor trigger generation approach with an optimization procedure that not only accommodates dynamic application inputs but also is stealthy to RF receivers. Extensive experiments on two RF signal classification datasets show that the average attack success rate of our backdoor attack is over 99.2%, while its classification accuracy for the clean data remains high (i.e., less than a 0.6% drop compared to the clean model). Additionally, we demonstrate that our attack can bypass existing defense strategies, such as Neural Cleanse and STRIP.

Deep Neural Networks (DNNs) for 3D point cloud recognition are vulnerable to adversarial examples, threatening their practical deployment. Despite the many research endeavors have been made to tackle this issue in recent years, the diversity of adversarial examples on 3D point clouds makes them more challenging to defend against than those on 2D images. For examples, attackers can generate adversarial examples by adding, shifting, or removing points. Consequently, existing defense strategies are hard to counter unseen point cloud adversarial examples. In this paper, we first establish a comprehensive, and rigorous point cloud adversarial robustness benchmark to evaluate adversarial robustness, which can provide a detailed understanding of the effects of the defense and attack methods. We then collect existing defense tricks in point cloud adversarial defenses and then perform extensive and systematic experiments to identify an effective combination of these tricks. Furthermore, we propose a hybrid training augmentation methods that consider various types of point cloud adversarial examples to adversarial training, significantly improving the adversarial robustness. By combining these tricks, we construct a more robust defense framework achieving an average accuracy of 83.45% against various attacks, demonstrating its capability to enabling robust learners. Our codebase are open-sourced on: https://github.com/qiufan319/benchmark_pc_attack.git.

Emotional state leakage attracts increasing concerns as it reveals rich sensitive information, such as intent, demo graphic, personality, and health information. Existing emotion recognition techniques rely on vision and audio data, which have limited threat due to the requirements of accessing restricted sensors (e.g., cameras and microphones). In this work, we first investigate the feasibility of detecting the emotional state of people in the vibration domain via zero-permission motion sensors. We find that when voice is being played through a smartphone's loudspeaker or ear speaker, it generates vibration signals on the smartphone surface, which encodes rich emotional information. As the smartphone is the go-to device for almost everyone nowadays, our attack based only on motion sensors raises severe concerns about emotion state leakage. We comprehensively study the relationship between vibration data and human emotion based on several publicly available emotion datasets (e.g., SAVEE, TESS). Time-frequency features and machine learning techniques are developed to determine the emotion of the victim based on speech vibrations. We evaluate our attack on both the ear speakers and loudspeakers on a diverse set of smartphones. The results demonstrate our attack can achieve a high accuracy, with around 95.3% (random guess 14.3%) accuracy for the loudspeaker setting and 60.52% (random guess 14.3%) accuracy for the ear speaker setting.

The user authentication has drawn increasingly attention as the smart speaker becomes more prevalent. For example, smart speakers that can verify who is sending voice commands can mitigate various types of attacks such as replay attack or impersonation attack. Existing user authentication solutions either cannot be applicable to smart speakers directly or require certain additional user-device interaction or pre-installed infrastructure, which may severely affect the user experience and create extra burdens to users. In this work, we propose a user authentication system EchoImage utilizing acoustic images, which are derived from the smart speaker by emitting beep signals and sensing echoes from the user's body with its microphone array, as the proof for user authentication. Given the acoustic samplings of the reflected beep signal, our system designs a distance estimation component by applying a correlation based technique on the beamformed signal to estimate the distance between the user and microphone array. Our image construction component then constructs a virtual imaging plane using the estimated distance and steers the array towards each grid of the plane to generate an acoustic image of the user. Moreover, we propose a transfer learning-based method to derive efficient features from the constructed images, and employ SVM classifiers for accurate user authentication. Our extensive experiments demonstrate that our system is robust and accurate across various scenarios.

Personal Identification Number (PIN), as one of the primary means of protecting digital properties and privacy on mobile devices, has been suffering from shoulder surfing attacks and weak password guessing for the long term. Recent years witness the growing interest in two-factor authentication that takes advantage of two different ways for mutual verification, thereby strengthening user authentication's accuracy and reliability. Especially with the popularity of smartwatches, more physiological signals are readily available to facilitate two-factor authentication. This paper presents a lightweight and unobtrusive two-factor authentication scheme, P2Auth, integrating the PIN and unique keystroke-related Photoplethysmography (PPG) measurement on wearables. Specifically, we propose the transformation of the multivariate PPG signal induced by the keystrokes to extract reliable biometric features. We develop short-time energy-based methods to identify the input cases, thus enabling support the authentication for both one-handed and two-handed input cases. Furthermore, we also consider the situation where there is no fixed PIN and design a new enhanced privacy scheme by combining the PPG measurements of different keystrokes to improve authentication security. The experiments involving 15 volunteers demonstrate that our prototype system can achieve an average authentication accuracy of over 95% for one-handed cases and over 90% for two-handed cases.

The two-factor authentication (2FA) has become pervasive as the mobile devices become prevalent. Existing 2FA solutions usually require some form of user involvement, which could severely affect user experience and bring extra burdens to users. In this work, we propose a secure 2FA that utilizes the individual acoustic fingerprint of the speaker/microphone on enrolled device as the second proof. The main idea behind our system is to use both magnitude and phase fingerprints derived from the frequency response of the enrolled device by emitting acoustic beep signals alternately from both enrolled and login devices and receiving their direct arrivals for 2FA. Given the input microphone samplings, our system designs an arrival time detection scheme to accurately identify the beginning point of the beep signal from the received signal. To achieve a robust authentication, we develop a new distance mitigation scheme to eliminate the impact of transmission distances from the sound propagation model for extracting stable fingerprint in both magnitude and phase domain. Our device authentication component then calculates a weighted correlation value between the device profile and fingerprints extracted from run-time measurements to conduct the device authentication for 2FA. Our experimental results show that our proposed system is accurate and robust to both random impersonation and Man-in-the-middle (MiM) attack across different scenarios and device models.

Neural networks have already demonstrated attractive performance for solving motion planning problems, especially in static and predictable environments. However, efficient neural planners that can adapt to unpredictable dynamic environments, a highly demanded scenario in many practical applications, are still under-explored. To fill this research gap and enrich the existing motion planning approaches, in this pa-per, we propose DynGMP, a graph neural network (GNN)-based planner that provides high-performance planning solutions in unpredictable dynamic environments. By fully leveraging the prior exploration experience and minimizing the replanning cost incurred by environmental change, DynGMP achieves high planning performance and efficiency simultaneously. Empirical evaluations across different environments show that DynGMP can achieve close to 100% success rate with fast planning speed and short path cost. Compared with existing non-learning and learning-based counterparts, DynGMP shows very significant planning performance improvement, e.g., at least 2.7×, 2.2×,$2.4\times$and$2\times$faster planning speed with low path distance in four environments, respectively.

Speech privacy leakage has long been a public concern. Existing non-microphone-based eavesdropping attacks rely on physical contact or line-of-sight between the sensor (e.g., a motion sensor or a radar) and the victim sound source. In this poster, we investigate a new form of attack that remotely elicits speech from minute surface vibrations upon common room objects (e.g., paper bags, plastic storage bin) via mmWave sensing. We design and implement a highresolution software-defined phased-MIMO radar that integrates transmit beamforming, virtual array, and receive beamforming. The proposed system enhances sensing directivity by focusing all the mmWave beams toward a target room object. We successfully demonstrate such an attack by developing a deep speech recognition scheme grounded on unsupervised domain adaptation. Without prior training on the victim's data, our attack can achieve a high success rate of over 90% in recognizing simple digits.

Sound source localization is vital for daily tasks such as communication or navigating environments. However, millions of adults struggle with hearing impairment, which limits their ability to identify the direction and distance of sound sources. Traditional methods for sound spatial sensing, such as microphone arrays, are not suitable for resource-constrained IoT devices like smartphones due to power consumption or hardware complexity. To overcome these limitations, this paper proposes EarCase, an alternative scheme that utilizes commercial smartphones with only two microphones to recognize 3D acoustic spatial information. EarCase draws inspiration from the human auditory system, where two ears amplify minute differences in acoustic signals to help pinpoint sound sources. This ability can be regarded as a response function trained through a large amount of sound source information, which can be used to extract spectral cues from a sound source position to the ears drums. We imitate this effect by designing a smartphone case with perforated mini-structures covering the microphones to help the smartphone infer the location of the sound source. Sound waves that pass through the mini-structure will undergo unique changes in diffraction at the hole, amplifying directional information similar to ears. Our scheme uses the top and bottom microphones to eliminate noises and multi-path effects, making the design robust to different sound sources in varying environments. By using only built-in microphones and low-cost phone cases, EarCase provides an accessible tool to enhance the quality of life for hearing impaired individuals. Extensive experimental results show that EarCase achieves high accuracy in localizing sounds, with a mean error of 3.7° at a distance of 200cm and 96% accuracy for real-world sounds (e.g., car horns).

Despite the rapid growth of augmented reality and virtual reality (AR/VR) in various applications, the understanding of information leakage through sensor-rich headsets remains in its infancy. In this poster, we investigate an unobtrusive privacy attack, which exposes users' vital signs and embedded sensitive information (e.g., gender, identity, body fat ratio), based on unrestricted AR/VR motion sensors. The key insight is that the headset is closely mounted on the user's face, allowing the motion sensors to detect facial vibrations produced by users' breathing and heartbeats. Specifically, we employ deep-learning techniques to reconstruct vital signs, achieving signal qualities comparable to dedicated medical instruments, as well as deriving users' gender, identity, and body fat information. Experiments on three types of commodity AR/VR headsets reveal that our attack can successfully reconstruct high-quality vital signs, detect gender (accuracy over 93.33%), re-identify users (accuracy over 97.83%), and derive body fat ratio (error less than 4.43%).

Recent years have witnessed the emerging market of 3D facial reconstruction that supports numerous face-driven scenarios including modeling in virtual reality (VR), human-computer interaction, and affective computing applications. Current mainstream approaches rely on vision for 3D facial reconstruction, which may encounter privacy concerns and suffer from obstruction scenes and bad lighting conditions. In this paper, we present a nonintrusive 3D facial reconstruction system, mm3DFace, which leverages a millimeter wave (mmWave) radar to reconstruct 3D human faces that continuously express facial expressions in a privacy-preserving and passive manner. Based on the pre-processed mmWave signals, mm3DFace first extracts facial geometric features that capture subtle changes in facial expressions through a ConvNeXt model with triple loss embedding. Then, mm3DFace derives distance and orientation-robust facial shapes with 68 facial landmarks using region-divided affine transformation. mm3DFace next reconstructs facial expressions through a designed regional amplification method and finally generates 3D facial avatars that continuously express facial expressions. Extensive experiments involving 15 participants in real-world environments show that mm3DFace can accurately track 68 facial landmarks with 3.94% normalized mean error, 2.30mm mean absolute error, and 4.10mm 3D-mean absolute error, which is effective and practical in real-world 3D facial reconstruction.

Finger biometrics are widely used by smartphones as a secure and user-friendly credential for privacy protection. However, this information is difficult to measure without high-resolution images, leaving most works to treat this as an image-domain problem. We demonstrate that low-effort alternatives on smartphones are possible through the use of sound propagation in ubiquitous smartphone cases. Inexpensive and widely adopted, smartphone cases are always in contact with fingers, making them ideal for collecting finger biometrics. We thus design BioCase, an acoustic sensing system that leverages smartphone cases equipped with mini-structures to capture unique biometric-hybrid signatures (i.e., reflections influenced by the user's fingertip physiology and behavior) for smartphone privacy protection. The system generates inaudible structure-borne sound and measure the propagation through the smartphone case, mini-structures, and user finger. The design of the mini-structure controls the behavior of structure-borne sound such that unique responses are produced when different users and fingers touch the smartphone case. This enables low-cost, low-effort privacy protection, merely touching the smartphone case can authenticate users. Comprehensive experiments with 46 users over 10 weeks demonstrate BioCase can differentiate users with over 94% accuracy at a 5% false positive rate.

Vital signs (e.g., breathing and heart rates) and personal identities are essential information for personalized medicine and healthcare. The popularity of augmented reality/virtual reality (AR/VR) provides an excellent opportunity for enabling long-term health monitoring in a broad range of scenarios, including virtual entertainment, education, and telemedicine. However, commercial-off-the-shelf AR/VR devices do not have dedicated biosensors for providing vital signs and personal identities. In this work, we propose a novel framework that can generate fine-grained vital sign signals and other personalized health information of an AR/VR user through passive sensing on AR/VR devices. In particular, we find that the user's minute facial vibrations induced by breathing and heart beating can impact the readily available motion sensors on AR/VR headsets, which encode rich vital sign patterns and unique biometrics. The proposed framework further estimates the breathing and heartbeat rates, detects the gender and identity, and derives the body fat percentage of the user. To mitigate the impacts of body movement, we design an adaptive filtering scheme to cancel the spontaneous and non-spontaneous motion artifacts. We also develop unique facial vibration features and deep learning techniques to facilitate vital sign signal reconstruction and user identification. Extensive experiments demonstrate that our framework can achieve a low error of vital sign signal reconstruction and rate measurement, along with 95.51% and 93.33% accuracy on identity and gender recognition.

Virtual Reality (VR) has gained popularity in numerous fields, including gaming, social interactions, shopping, and education. In this paper, we conduct a comprehensive study to assess the trustworthiness of the embedded sensors on VR, which embed various forms of sensitive data that may put users’ privacy at risk. We find that accessing most on-board sensors (e.g., motion, position, and button sensors) on VR SDKs/APIs, such as OpenVR, Oculus Platform, and WebXR, requires no security permission, exposing a huge attack surface for an adversary to steal the user’s privacy. We validate this vulnerability through developing malware programs and malicious websites and specifically explore to what extent it exposes the user’s information in the context of keystroke snooping. To examine its actual threat in practice, the adversary in the considered attack model doesn’t possess any labeled data from the user nor knowledge about the user’s VR settings. Extensive experiments, involving two mainstream VR systems and four keyboards with different typing mechanisms, demonstrate that our proof-of-concept attack can recognize the user’s virtual typing with over 89.7% accuracy. The attack can recover the user’s passwords with up to 84.9% recognition accuracy if three attempts are allowed and achieve an average of 87.1% word recognition rate for paragraph inference. We hope this study will help the community gain awareness of the vulnerability in the sensor management of current VR systems and provide insights to facilitate the future design of more comprehensive and restricted sensor access control mechanisms.

The growing adoption of voice-enabled devices (e.g., smart speakers), particularly in smart home environments, has introduced many security vulnerabilities that pose significant threats to users' privacy and safety. When multiple devices are connected to a voice assistant, an attacker can cause serious damage if they can gain control of these devices. We ask where and how can an attacker issue clean voice commands stealthily across a physical barrier, and perform the first academic measurement study of this nature on the command injection attack. We present the BarrierBypass attack that can be launched against three different barrier-based scenarios termed across-door, across-window, and across-wall. We conduct a broad set of experiments to observe the command injection attack success rates for multiple speaker samples (TTS and live human recorded) at different command audio volumes (65, 75, 85 dB), and smart speaker locations (0.1-4.0m from barrier). Against Amazon Echo Dot 2, BarrierBypass is able to achieve 100% wake word and command injection success for the across-wall and across-window attacks, and for the across-door attack (up to 2 meters). At 4 meters for the across-door attack, BarrierBypass can achieve 90% and 80% injection accuracy for the wake word and command, respectively. Against Google Home mini BarrierBypass is able to achieve 100% wake word injection accuracy for all attack scenarios. For command injection BarrierBypass can achieve 100% accuracy for all the three barrier settings (up to 2 meters). For the across-door attack at 4 meters, BarrierBypass can achieve 80% command injection accuracy. Further, our demonstration using drones yielded high command injection success, up to 100%. Overall, our results demonstrate the potentially devastating nature of this vulnerability to control a user's device from outside of the device's physical space, and its limitations, without the need for complex and error-prone command injection.

Voice authentication is drawing increasing attention and becomes an attractive alternative to passwords for mobile authentication. Recent advances in mobile technology further accelerate the adoption of voice biometrics in an array of diverse mobile applications. However, recent studies show that voice authentication is vulnerable to replay attacks, where an adversary can spoof a voice authentication system using a pre-recorded voice sample collected from the victim. In this article, we propose VoiceLive, a liveness detection system for both text-dependent and text-independent voice authentication on smartphones. VoiceLive detects a live user by leveraging the user's unique vocal system and the stereo recording of smartphones. In particular, utilizing the built-in gyroscope, loudspeaker and microphone, VoiceLive first measures the smartphone's distance and angle from the user, then it captures the position specific time-difference-of-arrival (TDoA) changes in a sequence of phoneme sounds to the two microphones of the phone, and uses such unique TDoA dynamic which doesn't exist under replay attacks for liveness detection. VoiceLive is practical as it doesn't require additional hardware but two-channel stereo recording that is supported by virtually all smartphones. Our experimental evaluation with 12 participants and different types of phones shows that VoiceLive achieves over 99% detection accuracy at around 1% Equal Error Rate (EER) on the text-dependent system and around 99% accuracy and 2% EER on the text-independent one. Results also show that VoiceLive is robust to different phone positions, i.e., the user are free to hold the smartphone with distinct distances and angles.

Wireless body area network is a key enabler for connected healthcare but recent cyberattacks have compromised its security and trustworthiness. This paper investigates heartbeat-based key generation to secure body area networks. The interpulse intervals (IPIs) between any two adjacent peaks of heartbeat signals are random and state-of-the-art literature has demonstrated that IPI is a good random source to be extracted as cryptographic keys. Heartbeat signals can be measured by electrocardiography (ECG) and photoplethysmography (PPG) sensors. A general heartbeat-based key generation framework applicable to both ECG and PPG signals is proposed. A robust peak detection algorithm is designed to capture noisy peaks and a simple yet efficient IPI alignment algorithm to align the common IPIs. A key establishment protocol is used to convert analog IPIs to digital binaries and reconcile them between legitimate devices. We evaluate the performance for both ECG signals from an online public database, MIT PhysioBank, and PPG signals collected from our testbed. The results demonstrate that our algorithm is robust and heartbeat-based key generation can be completed for both ECG and PPG signals. We finally create a PPG-based prototype and a demonstration video to show the practicality of our framework.

User authentication nowadays has become an important support for not only security guarantees but also emerging novel applications. Although WiFi signal-based user authentication has achieved initial success, it works in single-user scenarios while multi-user authentication remains a challenging task. In this paper, we present MultiAuth, a multi-user authentication system that can authenticate multiple users with a single pair of commodity WiFi devices. The basic idea is to profile multipath components of WiFi signals, and leverage the multipath components to characterize each user individually for multi-user authentication. MultiAuth first profiles multipath components of WiFi signals through a proposed MUltipath Time-of-Arrival estimation algorithm (MUTA). Then, after matching corresponding multipath components to each user in complex multi-user scenarios, MultiAuth constructs individual CSI based on the multipath components to characterize each user individually. An AoA-based approach is exploited to further separate individual CSI constructed by the users with same ToA. To identify users through their activities, MultiAuth extracts user behavior profiles based on the individual CSI, and leverages a dual-task neural network for robust user authentication. Extensive experiments involving 3 simultaneously present users demonstrate that MultiAuth is effective in multi-user authentication with 86.2% average accuracy and 9.5% average false accept rate.

In this work, we propose a WiFi vision-based approach to person re-identification (Re-ID) indoors. Our approach leverages the advances of WiFi to visualize a person and utilizes deep learning to help WiFi devices identify and recognize people. Specifically, we leverage multiple antennas on WiFi devices to estimate the two-dimensional angle of arrival (2D AoA) of the WiFi signal reflections to enable WiFi devices to "see'' a person. We then utilize deep learning techniques to extract a 3D mesh representation of a person and extract the body shape and walking patterns for person Re-ID. Our preliminary study shows that our system achieves high overall ranking accuracies. It also works under non-line-of-sight and different person appearance conditions, where the traditional camera vision-based systems do not work well.

Ear wearables (earables) have become an emerging and wide acceptable platform for various applications. Because of the limited input interface of earables, traditional authentication methods become less desired. However, the feature-rich sensing abilities of earables and the unique human face-ear channel bring us new sensing opportunities to reutilize fingerprints. In this work, we proposed SlidePass, a secure earables authentication system that leverages the finger-face acoustic friction produced by sliding finger gestures on the face. In particular, our system leverages the inward-facing microphone of the earables to reliably capture the acoustic of finger-face frictions. The core insight of our system is to utilize the face as a natural scanner for finger-face friction and earables to capture and reconstruct the fingerprint features. SlidePass is specially designed for earables. Due to the finger-face friction captured and encrypted by the face channel that is unique and hidden in the human skull, SlidePass is more resistant to various spoofing attacks. Our preliminary evaluation included ten different fingerprints showing that SlidePass achieves an average accuracy of 94%.

Compressed deep neural network (DNN) models have been widely deployed in many resource-constrained platforms and devices. However, the security issue of the compressed models, especially their vulnerability against backdoor attacks, is not well explored yet. In this paper, we study the feasibility of practical backdoor attacks for the compressed DNNs. More specifically, we propose a universal adversarial perturbation (UAP)-based approach to achieve both high attack stealthiness and high attack efficiency simultaneously. Evaluation results across different DNN models and datasets with various compression ratios demonstrate our approach’s superior performance compared with the existing solutions.

There is a growing trend for people to perform work-outs at home due to the global pandemic of COVID-19 and the stay-at-home policy of many countries. Since a self-designed fitness plan often lacks professional guidance to achieve ideal outcomes, it is important to have an in-home fitness monitoring system that can track the exercise process of users. Traditional camera-based fitness monitoring may raise serious privacy concerns, while sensor-based methods require users to wear dedicated devices. Recently, researchers propose to utilize RF signals to enable non-intrusive fitness monitoring, but these approaches all require huge training efforts from users to achieve a satisfactory performance, especially when the system is used by multiple users (e.g., family members). In this work, we design and implement a fitness monitoring system using a single COTS mm Wave device. The proposed system integrates workout recognition, user identification, multi-user monitoring, and training effort reduction modules and makes them work together in a single system. In particular, we develop a domain adaptation framework to reduce the amount of training data collected from different domains via mitigating impacts caused by domain characteristics embedded in mm Wave signals. We also develop a GAN-assisted method to achieve better user identification and workout recognition when only limited training data from the same domain is available. We propose a unique spatialtemporal heatmap feature to achieve personalized workout recognition and develop a clustering-based method for concurrent workout monitoring. Extensive experiments with 14 typical workouts involving 11 participants demonstrate that our system can achieve 97% average workout recognition accuracy and 91% user identification accuracy.

The open nature of voice input makes voice assistant (VA) systems vulnerable to various acoustic attacks (e.g., replay and voice synthesis attacks). A simple yet effective way for adversaries to launch these attacks is to hide behind barriers (e.g., a wall, a window, or a door) and give unauthorized voice commands without being observed by legitimate users. In this work, we develop an automated, training-free defense system that can protect VA systems from such thru-barrier acoustic attacks. Our study finds that acoustic signals passing through the barriers generally present a unique frequency-selective effect in the vibration domain. Thus, we propose to devise a system to capture this unique effect of barriers by leveraging low-cost, cross-domain sensing available in users’ wearables. The system replays the audio-domain signals with the wearable’s speaker and captures the conductive vibrations caused by the audio sounds in the vibration domain via the built-in accelerometer. To improve the proposed system’s reliability, we develop a unique vibration-domain enhancement method to extract the phonemes most sensitive to the frequency-selective effect of barriers. We identify effective vibration-domain features that capture the barriers’ effects in the vibration domain. A 2D-correlation-based method is developed to examine the speech similarity between the recordings from the VA system and the user’s wearable and detect thru-barrier attacks. Extensive experiments with various barriers and environments demonstrate that the proposed defense system can effectively defend random, replay, synthesis, and hidden voice attacks with less than 4% equal error rates.

Due to the shared nature of wireless mediums, jamming attacks have long been a great hazard to the security of wireless networks. A plethora of efforts have been spent to mitigate the impact of jamming attacks, and especially the localization technique of malicious jammer emerges in the last decade and enables us to remove the interfering devices from the physical layer. Since it is impossible to directly measure the interfering signal in a jamming scenario, many existing methods rely on inaccurate ranging estimation to locate the jammers. In this paper, we propose an improved least-square jammer localization method leveraging the network distribution properties. Specifically, we exploit the stochastic geometry theory to analyze the coverage changes of wireless devices around the jammer, which are then used to improve the accuracy of ranging estimation. Extensive numerical results demonstrate the effectiveness and robustness of the proposed jammer localization methods under various scenarios. Notably, the improved leastsquares method has a decreasing of 35 % on the mean localization error comparing to the traditional least-square method.

With the development of smart indoor environments, user authentication becomes an essential mechanism to support various secure accesses. Although recent studies have shown initial success on authenticating users with human activities or gestures using WiFi, they rely on predefined body gestures and perform poorly when meeting undefined body gestures. This work aims to enable WiFi-based user authentication with undefined body gestures rather than only predefined body gestures, i.e., realizing a gesture-independent user authentication. In this paper, we first explore physiological characteristics underlying body gestures, and find that statistical distributions under WiFi signals induced by body gestures can exhibit invariant individual uniqueness unrelated to specific body gestures. Inspired by this observation, we propose a user authentication system, which utilizes WiFi signals to identify individuals in a gesture-independent manner. Specifically, we design an adversarial learning-based model, which suppresses specific gesture characteristics, and extracts invariant individual uniqueness unrelated to specific body gestures, to authenticate users in a gesture-independent manner. Extensive experiments in indoor environments show that the proposed system is feasible and effective in gesture-independent user authentication.

Optical camera communication (OCC) has attracted increasing attention recently thanks to the wide usage of LED and high-resolution cameras. The lens-image sensor structure enables the camera distinguish light from various source, which is ideal for spatial MIMO. Hence, OCC can be applied to several emerging application scenarios, such as vehicle and drone communications. However, distance is a major bottleneck for OCC system, because the increase in distance makes it difficult for the camera to distinguish adjacent LEDs, which we call LED spatial mixing.In this paper, we propose a novel hierarchical coding scheme named as OnionCode to support dynamic range of channel capacity in one-to-many OCC scenario. OnionCode adopts a multi-priority receiving scheme, i.e., the receivers can dynamically discard the low-priority bit streams according to the measured channel capacity. OnionCode achieves this based on a key insight that, the luminance level of a mix-LED is distinguishable. We prototype a LED-based OCC system to evaluate the efficacy of OnionCode and the results show that OnionCode achieves a higher coding efficiency and overall throughput compared with the existing hierarchical coding.

The continuously increasing time spent on car trips in recent years brings growing attention to the physical and mental health of drivers on roads. As one of the key vital signs, the heartbeat is a critical indicator of drivers' health states. Most existing studies on heartbeat monitoring either require sensor attachment or could only provide sketchy heart rates. Moreover, most approaches require the subject to remain stationary or a quiet measuring environment, which is hard to apply to dynamic driving environments. In this paper, we propose a contactless cardiac cycle monitoring system, mmECG, which leverages Commercial-Off-The-Shelf mmWave radar to estimate the fine-grained heart movements of drivers in moving vehicles. By exploring the principle of mmWave signal-based sensing, we first perform studies in static environments and find the fine-grained heart movements, represented as stages of atria and ventricles in repetitive cardiac cycles, can be captured by the FMCW-based mmWave radar as phase changes in signals. Whereas in driving environments, such phase changes are caused and influenced by not only the heartbeat of drivers but also driving operations and vehicle dynamics. To further extract the minute heart movements of drivers and eliminate other influences in phase changes, we construct a movement mixture model to represent the phase changes caused by different movements, and further design a hierarchy variational mode decomposition (VMD) approach to extract and estimate the essential heart movement in mmWave signals. Finally, based on the extracted phase changes, mmECG reconstructs the cardiac cycle by estimating fine-grained movements of atria and ventricles leveraging a template-based optimization method. Experimental results involving 25 drivers in real driving scenarios demonstrate that mmECG can accurately estimate not only heart rates but also cardiac cycles of drivers in real driving environments.

Deep learning models have become key enablers of voice user interfaces. With the growing trend of adopting outsourced training of these models, backdoor attacks, stealthy yet effective training-phase attacks, have gained increasing attention. They inject hidden trigger patterns through training set poisoning and overwrite the model's predictions in the inference phase. Research in backdoor attacks has been focusing on image classification tasks, while there have been few studies in the audio domain. In this work, we explore the severity of audio-domain backdoor attacks and demonstrate their feasibility under practical scenarios of voice user interfaces, where an adversary injects (plays) an unnoticeable audio trigger into live speech to launch the attack. To realize such attacks, we consider jointly optimizing the audio trigger and the target model in the training phase, deriving a position-independent, unnoticeable, and robust audio trigger. We design new data poisoning techniques and penalty-based algorithms that inject the trigger into randomly generated temporal positions in the audio input during training, rendering the trigger resilient to any temporal position variations. We further design an environmental sound mimicking technique to make the trigger resemble unnoticeable situational sounds and simulate played over-the-air distortions to improve the trigger's robustness during the joint optimization process. Extensive experiments on two important applications (i.e., speech command recognition and speaker recognition) demonstrate that our attack can achieve an average success rate of over 99% under both digital and physical attack settings.

Faced with the threat of identity leakage during voice data publishing, users are engaged in a privacy-utility dilemma while enjoying convenient voice services. Existing studies employ direct modification or text-based re-synthesis to de-identify users' voices, but resulting in inconsistent audibility for human participants and not adaptive to informed attacks. In this poster, we propose a non-intrusive and adaptive speaker de-identification scheme to balance the privacy and utility of voice services. We generate adversarial examples to conceal user identity from exposure by Automatic Speaker Identification (ASI). By learning a compact distribution with a conditional variational auto-encoder, our system enables on-demand target sampling and diverse identity transformation. We also introduce the acoustic masking effect to construct inaudible perturbations, thus preserving the speech content and perceptual quality. Experiments on 50 speakers show our system could achieve 98.2% successful de-identification on 4 mainstream ASIs with an objective perceptual quality of 4.38 and a subjective mean opinion score of 4.56.

Camera tracking has become a key technology for various application scenarios, especially for AR-based camera-to-screen interaction. Demand for subtle motion detection in camera tracking makes it essential to explore the six degrees of freedom (6-DoF) pose detection with ultra-high precision. In this paper, we propose a novel sensing method MoiréPose to achieve ultra-high precision on the camera's 6-DoF pose estimation. The purpose of MoiréPose is to derive the camera's 3-DoF position and 3-DoF posture relative to the screen according to the captured moiré pattern, which is produced by the superposition of the camera's Color Filter Array (CFA) and the screen raster projected onto the CFA layer. Based on moiré pattern's high sensitivity to 6-DoF pose movement and robustness to the environmental interference in the frequency domain, we propose a spectrogram-based method to realize the camera's 6-DoF detection with ultra-high precision. Moreover, we propose a thumbnail-based method to effectively extend the working range of MoiréPose, so as to realize pervasive camera-to-screen interaction. We have implemented a prototype system and evaluate the performance in real-world environments. Extensive experiment results show that MoiréPose achieves an average position error of 7.5mm and an overall posture error of 1.66°.

Person re-identification (Re-ID) has become increasingly important as it supports a wide range of security applications. In this work, we propose a WiFi-based person Re-ID system in 3D space, which leverages the advances of WiFi and deep learning to extract the static body shape and dynamic walking patterns to recognize people. In particular, we leverage multiple antennas on WiFi devices to capture signal reflections of the human body and produce a WiFi image of a person. We then leverage deep learning to extract both the static body shape and dynamic walking patterns for person Re-ID. Our evaluation results show that our system achieves an overall rank-1 accuracy of 87.1%.

In this work, we present, Wi-Mesh, a WiFi vision-based 3D human mesh construction system. Our system leverages the advances of WiFi to visualize the shape and deformations of the human body for 3D mesh construction. In particular, it estimates the two-dimensional angle of arrival (2D AoA) of the WiFi signal reflections to enable WiFi devices to "see" the physical environment as we humans do. It then extracts only the images of the human body from the physical environment, and leverages deep learning models to digitize the extracted human body into 3D mesh representation. Experimental evaluation under various indoor environments shows that Wi-Mesh achieves an average vertices location error of 2.58cm and joint position error of 2.24cm.

For decades, one-time verification has been the standard for user verification at entry points, office rooms, etc. However, such approaches request users to provide their secrets (e.g., entering passwords and collecting fingerprints) and re-verify (e.g., screen shutdown) manually. Thus, they cannot confirm whether the user is a legitimate or an imposter after verification, which raises the urgent demand for a more convenient and secure solution to perform continuous user verification. However, existing continuous verification methods heavily rely on users' active participation, which is inconvenient. Toward this end, we propose a continuous user verification system, BioTag, which utilizes the low-cost radio frequency identification (RFID) technology to capture unique physiological characteristics rooted in the users' respiration motions for continuous user verification. Specifically, we use two RFID tags attached to a user's chest and abdomen to capture the user's intrinsic respiratory patterns via RFID signals. We develop respiratory feature extraction methods based on waveform morphology analysis and fuzzy wavelet transformation (FWPT) to derive unique biometric information from the user's respiration signals. Furthermore, we develop an adaptive classifier using the gradient boosting decision tree (GBDT) to identify legitimate users and attackers accurately. Extensive experiments involving 41 participants demonstrate that BioTag can robustly authenticate users and detect various types of adversaries with low training effort. In particular, our system can achieve over 95.2% and 94.8% verification accuracy on random attack and imitation attack scenarios, respectively.

The Augmented reality/Virtual reality (AR/VR) industry has ushered in a period of rapid development. The next decade leaves a massive imagination for AR/VR in terms of end product form, software, content, applications, and user increment. The AR & VR technology offers a gazillion of possibilities for smart healthcare. In this poster, we develop an innovative continuous blood pressure (CBP) estimation system leveraging the built-in motion sensors of AR/VR headsets for users. We design a deep learning-based PPG construction scheme using the motion sensor-based cardiac signal and estimate the continuous blood pressure using the regression model. Our experimental results show that our system can continuously estimate both systolic blood pressure (SBP) and diastolic blood pressure (DBP) with a mean error of less than 4 mmHg and 0.9 mmHg respectively within a day.

Speech privacy leakage has long been a public concern. Through speech eavesdropping, an adversary may steal a user's private information or an enterprise's financial/intellectual properties, leading to catastrophic consequences. Existing non-microphone-based eavesdropping attacks rely on physical contact or line-of-sight between the sensor (e.g., a motion sensor or a radar) and the victim sound source. In this poster, we discover a new form of speech eavesdropping attack that senses minor speech-induced vibrations upon common room objects using mmWave. By integrating phasedarray and multiple-input and multiple-output (MIMO) on a single mmWave transceiver, our attack can capture and fuse micrometerlevel vibrations upon the surfaces of multiple objects to reveal speech content in a remote and non-line-of-sight fashion. We successfully demonstrate such an attack by developing a deep speech recognition scheme grounded on unsupervised domain adaptation. Without prior training on the victim's data, our attack can achieve a high success rate of over 90% in recognizing simple speech content.

Nowadays, the market of 3D human posture tracking has extended to a broad range of application scenarios. As current mainstream solutions, vision-based posture tracking systems suffer from privacy leakage concerns and depend on lighting conditions. Towards more privacy-preserving and robust tracking manner, recent works have exploited commodity radio frequency signals to realize 3D human posture tracking. However, these studies cannot handle the case where multiple users are in the same space. In this paper, we present a mmWave-based multi-user 3D posture tracking system, m3Track, which leverages a single commercial off-the-shelf (COTS) mmWave radar to track multiple users' postures simultaneously as they move, walk, or sit. Based on the sensing signals from a mmWave radar in multi-user scenarios, m3Track first separates all the users on mmWave signals. Then, m3Track extracts shape and motion features of each user, and reconstructs 3D human posture for each user through a designed deep learning model. Furthermore. m3Track maps the reconstructed 3D postures of all users into 3D space, and tracks users' positions through a coordinate-corrected tracking method, realizing practical multi-user 3D posture tracking with a COTS mmWave radar. Experiments conducted in real-world multi-user scenarios validate the accuracy and robustness of m3Track on multi-user 3D posture tracking.

Millimeter wave (mmWave)-based human activity recognition (HAR) systems have emerged in recent years due to their better privacy preservation and higher-resolution sensing. However, these systems are vulnerable to adversarial attacks. In this work, we propose a universal targeted attack method for mmWave-based HAR system. In particular, a universal perturbation is generated in advance which can be added to new-coming mmWave data to deceive the HAR system, causing it to output our desired label. We validate our proposed attack using a public mmWave dataset. We demonstrate the effectiveness of our proposed universal attack with a high attack success rate of over 95%.

Augmented reality/virtual reality (AR/VR) headsets have attracted millions of users and gained predictable popularity. However, long-period usage of immersive technology may lead to health issues (e.g., cybersickness, anxiety). In this poster, we design a low-cost and personalized healthcare monitoring system grounded on vital sign tracking (i.e., breathing and heartbeat rate tracking), by exploiting built-in AR/VR motion sensors. The key insight is that the conductive vibrations induced by chest and heart movements can propagate through the user's cranial bones, thereby vibrating the AR/VR headset mounted on the user's head. To realize this system, we design signal processing techniques to cancel the human motions and derive the periods of breathing and heartbeat through frequency-domain analyses. We further design a user identification scheme based on respiratory and cardiac biometrics, which works with vital sign monitoring to provide personalized healthcare recommendations. Our experiment shows that the proposed scheme can achieve less than 5.7% error rate on breathing/heartbeat rate estimation and 95% accuracy on user identification.

The integration of deep learning on Speaker Recognition (SR) advances its development and wide deployment, but also introduces the emerging threat of adversarial examples. However, only a few existing studies investigate its practical threat in physical domain, which either evaluate its feasibility only by directly replaying generated adversarial examples, or explore the partial channel interference for robustness improvement. In this paper, we propose a physical adversarial example attack, PhyTalker, which could generate and inject perturbations on voices in a live-streaming manner on attacking various SR models in different physical channels. Compared with the typical adversarial example for digital attacks, PhyTalker generates a subphoneme-level perturbation dictionary to decouple the perturbation optimization and injection. Moreover, we introduce the channel augmentation to compensate both device and environmental distortions, as well as model ensemble to improve the perturbation transferability. Finally, PhyTalker recognizes and localizes the latest recorded phoneme to determine the corresponding perturbations for real-time broadcasting. Extensive experiments are conducted with a large-scale corpus in real physical scenarios, and results show that PhyTalker achieves an overall Attack Success Rate (ASR) of 85.5% in attacking mainstream SR systems and Mel Cepstral Distortion (MCD) of 2.45dB in human audibility.

The ubiquitous WiFi signals provide us the opportunity to sense human activities and the physical environment. In this work, we take a layered approach to design a vision-based method for commodity WiFi sensing. Specifically, the next-generation WiFi supports a larger number of antennas that can provide spatial information of the signal reflections, which enables a vision-based approach for WiFi sensing. To better leverage the spatial formation of the signal reflections and fulfill emerging applications, we provide a holistic layered framework including hardware, physical, deep learning, and application layers as well as a case study. The proposed layered approach could enlighten the research on future WiFi sensing.

With the wide deployment of smart environments and IoT devices, WiFi sensing has demonstrated its great convenience and contactless sensing capabilities in supporting a broad array of applications. However, designing a ubiquitous WiFi sensing system for heterogeneous scenarios in practice is still a big dilemma as the system performs poorly when the testing data is significantly different from the training data caused by domain variations. To address this dilemma, existing studies involve extra efforts to develop new features or even to retrain the original model under environmental variations. However, none of them can resolve the dilemma completely. In this work, we conduct a comprehensive study on the domain variation problem to make WiFi sensing robust and accurate in reality. Our definition of domains is comprehensive and includes environments, surrounding settings, user differences, user's facing directions, user's positions relative to WiFi sensors, and user participating time frames. Our innovation is to achieve reliable WiFi sensing across all the domains based on the conformal prediction framework. Our approach quantifies the conformity (i.e., similarity) between the testing WiFi samples and the training samples, then labels the testing samples with the most probable class(es). We develop a novel cross-domain transformal prediction scheme based on the multivariate kernel density estimation to effectively assess and learn the conformity of each domain in the training data. To meet various application-specific requirements, we further develop two approaches to fuse the knowledge of conformity derived from the training domains to perform predictions. Extensive experiments with both self-collected and public datasets show that our framework can improve prediction accuracies from 30% to 74% improvements in three most representative WiFi-based applications across six types of domain variations.

In this paper, we present, Wi-Mesh, a WiFi vision-based 3D human mesh construction system. Our system leverages the advances of WiFi to visualize the shape and deformations of the human body for 3D mesh construction. In particular, it leverages multiple transmitting and receiving antennas on WiFi devices to estimate the two-dimensional angle of arrival (2D AoA) of the WiFi signal reflections to enable WiFi devices to "see" the physical environment as we humans do. It then extracts only the images of the human body from the physical environment, and leverages deep learning models to digitize the extracted human body into a 3D mesh representation. Experimental evaluation under various indoor environments shows that Wi-Mesh achieves an average vertices location error of 2.81cm and joint position error of 2.4cm, which is comparable to the systems that utilize specialized and dedicated hardware. The proposed system has the advantage of reusing the WiFi devices that already exist in the environment for potential mass adoption. It can also work in non-line of sight (NLoS), poor lighting conditions, and baggy clothes, where the camera-based systems do not work well.

The location awareness becomes increasingly important as mobile devices such as smartphones are used extensively in our daily lives. Existing indoor localization solutions either require certain preinstalled infrastructures or add-on devices, which could not provide a location semantics identification service for smartphones to infer both type and size of a geographic location. In this work, we propose a new active sensing system that enables smartphones to identify its location semantics without requiring any additional infrastructure. The main idea behind our system is to utilize the acoustic signatures, which are derived from the smartphone by emitting a predesigned beep signal and identifying two echo sets which correspond to sidewalls and other static objects respectively, as the proof to achieve both spatial size estimation and room-type prediction simultaneously for indoor location semantics identification. Given the microphone samplings, our system designs a correlation-based scheme to identify beginning points of echoes corresponding to static reflectors accurately from the received signal. To achieve an accurate location semantics identification, we develop a new echo selection scheme to discriminate echoes created by sidewalls and other static reflectors by utilizing the geometrical relationships between the delays of echoes. To deal with the varying number of identified echoes, our location semantics prediction scheme then derives histograms from echo sets and adopt a deep-learning-based classifier to determine the current location semantics. Our experimental results show that our proposed system is accurate and robust for location semantics identification under various real-world scenarios.

The prevalence of WiFi devices and ubiquitous coverage of WiFi networks provide us the opportunity to extend WiFi capabilities beyond communication, particularly in sensing the physical environment. In this article, we survey the evolution of WiFi sensing systems utilizing commodity devices over the past decade. It groups WiFi sensing systems into three main categories: 1) activity recognition (large scale and small scale); 2) object sensing; and 3) localization. We highlight the milestone work in each category and the underline techniques they adopted. Next, this work presents the challenges faced by existing WiFi sensing systems. Finally, we comprehensively discuss the future trending of commodity WiFi sensing.

The Internet of Things (IoT) has grown over decades to encompass many forms of sensing modalities, and continues to improve in terms of sophistication and lower costs. The trend of hardware miniaturization and emphasis on user convenience has inspired numerous studies to integrate more varied devices within the IoT into modernizing healthcare systems, facilitating applications, such as activity recognition, fitness assistance, vital signs monitoring, daily dietary tracking, and sleep monitoring. These applications are vital for prevention, detection, and treatment of ailments and can be realized using both dedicated health sensors as well as general-purpose sensors not originally designed for health monitoring. This article surveys such studies, detailing smart health monitoring systems, and the types of sensor components utilized within the IoT. We categorize and analyze these works based on their leverage of device-based techniques (i.e., use of sensors worn or carried by the person) and device-free techniques (i.e., wireless sensing without need to carry hardware), as well as signal processing and classification techniques utilized. In particular, we discuss how different combinations of these techniques can be creatively applied to support professional and commercial health-monitoring IoT networks. We also identify limitations and potential directions that future research may explore.

Voice biometrics is drawing increasing attention to user authentication on smart devices. However, voice biometrics is vulnerable to replay attacks, where adversaries try to spoof voice authentication systems using prerecorded voice samples collected from genuine users. To this end, we propose VoiceGesture, a liveness detection solution for voice authentication on smart devices, such as smartphones and smart speakers. With audio hardware advances on smart devices, VoiceGesture leverages built-in speaker and microphone pairs on smart devices as Doppler radar to sense articulatory gestures for liveness detection during voice authentication. The experiments with 21 participants and different smart devices show that VoiceGesture achieves over 99% and around 98% detection accuracy for text-dependent and text-independent liveness detection, respectively. Moreover, VoiceGesture is robust to different device placements, low audio sampling frequency, and supports medium-range liveness detection on smart speakers in various use scenarios, including smart homes and smart vehicles.

Traditional one-time user authentication is vulnerable to attacks when an adversary can obtain unauthorized privileges after a user’s initial login. Continuous user authentication (CA) has recently shown its great potential by enabling seamless user authentication with few users’ participation. We devise a low-cost system that can exploit users’ pulsatile signals from photoplethysmography (PPG) sensors in commodity wearable devices to perform CA. Our system requires zero user effort and applies to practical scenarios that have nonclinical PPG measurements with human motion artifacts (MAs). We explore the uniqueness of the human cardiac system and develop adaptive MA filtering methods to mitigate the impacts of transient and continuous activities from daily life. Furthermore, we identify general fiducial features and develop an adaptive classifier that can authenticate users continuously based on their cardiac characteristics with little additional training effort. Experiments with our wrist-worn PPG sensing platform on 20 participants under practical scenarios demonstrate that our system can achieve a high CA accuracy of over 90% and a low false detection rate of 4% in detecting random attacks. We show that our MA mitigation approaches can improve the CA accuracy by around 39% under both transient and continuous daily activity scenarios.

Deep learning (DL) has demonstrated great performance in various applications on powerful computers and servers. Recently, with the advancement of more powerful mobile devices (e.g., smartphones and touch pads), researchers are seeking DL solutions that could be deployed on mobile devices. Compared to traditional DL solutions using cloud servers, deploying DL on mobile devices have unique advantages in data privacy, communication overhead, and system cost. This article provides a comprehensive survey for the current studies of adopting and deploying DL on mobile devices. Specifically, we summarize and compare the state-of-the-art DL techniques on mobile devices in various application domains involving vision, speech/speaker recognition, human activity recognition, transportation mode detection, and security. We generalize an optimization pipeline for bringing DL to mobile devices, including model-oriented optimization mechanisms (e.g., pruning and quantization) and nonmodel-oriented optimization mechanisms (e.g., software accelerator and hardware design). Moreover, we summarize popular DL libraries regarding their support to state-of-the-art models (software) and processors (hardware). Based on our summarization, we further provide insights into potential research opportunities for developing DL for mobile devices.

Recent years witnessed the emergence of visible light communication (VLC) over screen-to-camera channel, such as barcode and unobtrusive optical pattern, due to the widely adoption of screen and camera in plenty of electronic devices. The prevalence of wide viewing angle screen and high standard cameras also imposes great threat for visible light communication, and the information leakage over screen-to-camera channel has been rarely explored. In this paper, we propose a secret key distribution system leveraging the unique color shift property over screen-to-camera channel. To facilitate such design, two practical secret key distribution methods, key matching-based and nearest next hop-based, are developed to map the secret key into a unique optical pattern on screen, which can only be correctly decoded by the legitimate user situated at an accessible region. We also provide theoretical analysis on the security of both methods. The performance of the proposed system is implemented with off-the-shelf devices and validated under various experimental scenarios. The results demonstrate that our system can achieve high bit-decoding accuracy for the legitimate users while maintaining low recovery accuracy for the attackers.

Gesture recognition has become increasingly important in human-computer interaction and can support different applications such as smart home, VR, and gaming. Traditional approaches usually rely on dedicated sensors that are worn by the user or cameras that require line of sight. In this paper, we present a fine-grained finger gesture recognition system by using commodity WiFi without requiring user to wear any sensors. Our system takes advantages of the fine-grained Channel State Information available from commodity WiFi devices and the prevalence of WiFi network infrastructures. It senses and identifies subtle movements of finger gestures by examining the unique patterns exhibited in the detailed CSI. We devise environmental noise removal mechanism to mitigate the effect of signal dynamic due to the environment changes. Moreover, we propose to capture the intrinsic gesture behavior to deal with individual diversity and gesture inconsistency. Lastly, we utilize multiple WiFi links and larger bandwidth at 5GHz to achieve finger gesture recognition under multi-user scenario. Our experimental evaluation in different environments demonstrates that our system can achieve over 90 percent recognition accuracy and is robust to both environment changes and individual diversity. Results also show that our system can provide accurate gesture recognition under different scenarios.

Given the increasing amount of time people spent on driving, the physical and mental health of drivers is essential to road safety. Breathing patterns are critical indicators of the wellbeing of drivers on the road. Existing studies on breathing monitoring require active user participation of wearing special sensors or relatively quiet environments during sleep, which are hardly applicable to noisy driving environments. In this work, we propose a fine-grained breathing monitoring system,BreathListener, which leverages audio devices on smartphones to estimate the fine-grained breathing waveform in driving environments. By investigating the data collected from real driving environments, we find that energy spectrum density (ESD) of acoustic signals can be utilized to capture breathing procedures in driving environments. To extract breathing pattern in ESD signals,BreathListenereliminates interference from driving environments in ESD signals utilizing background subtraction and variational mode decomposition (VMD). After that, the extracted breathing pattern is transformed into Hilbert spectrum, and we further design a deep learning architecture based on generative adversarial network (GAN) to generate fine-grained breathing waveform from the Hilbert spectrum of extracted breathing patterns in ESD signals. Experiments with ten drivers in real driving environments show thatBreathListenercan accurately capture breathing patterns of drivers in driving environments.

This work enables mobile user authentication via finger inputs on ubiquitous surfaces leveraging low-cost physical vibration. The system we proposed extends finger-input authentication beyond touch screens to any solid surface for IoT devices (e.g., smart access systems and IoT appliances). Unlike passcode or biometrics-based solutions, it integrates passcode, behavioral and physiological characteristics, and surface dependency together to provide a low-cost, tangible and enhanced security solution. The proposed system builds upon a touch sensing technique with vibration signals that can operate on surfaces constructed from a broad range of materials. New algorithms are developed to discriminate fine-grained finger inputs and supports three independent passcode secrets including PIN number, lock pattern, and simple gestures by extracting unique features in the frequency domain to capture both behavioral and physiological characteristics including contacting area, touching force, and etc. The system is implemented using a single pair of low-cost portable vibration motor and receiver that can be easily attached to any surface (e.g., a door panel, a stovetop or an appliance). Extensive experiments demonstrate that our system can authenticate users with high accuracy (e.g., more than 97 percent within two trials), low false positive rate (e.g., less 2 percent) and is robust to various types of attacks.

Recently, the vulnerability of deep neural network (DNN)-based audio systems to adversarial attacks has obtained increasing attention. However, the existing audio adversarial attacks allow the adversary to possess the entire user's audio input as well as granting sufficient time budget to generate the adversarial perturbations. These idealized assumptions, however, make the existing audio adversarial attacks mostly impossible to be launched in a timely fashion in practice (e.g., playing unnoticeable adversarial perturbations along with user's streaming input). To overcome these limitations, in this paper we propose fast audio adversarial perturbation generator (FAPG), which uses generative model to generate adversarial perturbations for the audio input in a single forward pass, thereby drastically improving the perturbation generation speed. Built on the top of FAPG, we further propose universal audio adversarial perturbation generator (UAPG), a scheme to craft universal adversarial perturbation that can be imposed on arbitrary benign audio input to cause misclassification. Extensive experiments on DNN-based audio systems show that our proposed FAPG can achieve high success rate with up to 214X speedup over the existing audio adversarial attack methods. Also our proposed UAPG generates universal adversarial perturbations that can achieve much better attack performance than the state-of-the-art solutions.

Recent advances in speaker verification and speech processing technology have seen voice authentication being adopted on a wide scale in commercial applications like online banking and customer care support and on devices such as smartphones and IoT voice assistant systems. However, it has been shown that the current voice authentication systems can be ineffective against voice synthesis attacks that mimic a user's voice to high precision. In this work, we suggest a paradigm shift from the traditional voice authentication systems operating in the audio domain but susceptible to speech synthesis attacks (in the same audio domain). We leverage a motion sensor's capability to pick up phonatory vibrations, that can help to uniquely identify a user via voice signatures in the vibration domain. The user's speech is played/echoed back by a device's speaker for a short duration (hence our method is termed EchoVib) and the resulting non-linear phonatory vibrations are picked up by the motion sensor for speaker recognition. The uniqueness of the device's speaker and its accelerometer results in a device-specific fingerprint in response to the echoed speech. The use of the vibration domain and its non-linear relationship with audio allows EchoVib to resist the state-of-the-art voice synthesis attacks, shown to be successful in the audio domain.

Recent years have witnessed the rapid development of automatic speech recognition (ASR) systems, providing a practical voice-user interface for widely deployed smart devices. With the ever-growing deployment of such an interface, several voice-based attack schemes have been proposed towards current ASR systems to exploit certain vulnerabilities. Posing one of the more serious threats,hidden voice attack uses the human-machine perception gap to generate obfuscated/hidden voice commands that are unintelligible to human listeners but can be interpreted as commands by machines. However, due to the nature of hidden voice commands (i.e., normal and obfuscated samples exhibit a significant difference in their acoustic features), recent studies show that they can be easily detected and defended by a pre-trained classifier, thereby making it less threatening. In this paper, we validate that such a defense strategy can be circumvented with a more advanced type of hidden voice attack calledHVAC. Our proposed HVAC attack can easily bypass the existing learning-based defense classifiers while preserving all the essential characteristics of hidden voice attacks (i.e., unintelligible to humans and recognizable to machines). Specifically, we find that all classifier-based defenses build on top of classification models that are trained with acoustic features extracted from the entire audio of normal and obfuscated samples. However, only speech parts (i.e., human voice parts) of these samples contain the useful linguistic information needed for machine transcription. We thus propose a fusion-based method to combine the normal sample and corresponding obfuscated sample as a hybrid HVAC command, which can effectively cheat the defense classifiers. Moreover, to make the command more unintelligible to humans, we tune the speed and pitch of the sample and make it even more distorted in the time domain while ensuring it can still be recognized by machines. Extensive physical over-the-air experiments demonstrate the robustness and generalizability of our HVAC attack under different realistic attack scenarios. Results show that our HVAC commands can achieve an average 94.1% success rate of bypassing machine-learning-based defense approaches under various realistic settings.

With the popularity of intelligent audio systems in recent years, their vulnerabilities have become an increasing public concern. Existing studies have designed a set of machine-induced audio attacks, such as replay attacks, synthesis attacks, hidden voice commands, inaudible attacks, and audio adversarial examples, which could expose users to serious security and privacy threats. To defend against these attacks, existing efforts have been treating them individually. While they have yielded reasonably good performance in certain cases, they can hardly be combined into an all-in-one solution to be deployed on the audio systems in practice. Additionally, modern intelligent audio devices, such as Amazon Echo and Apple HomePod, usually come equipped with microphone arrays for far-field voice recognition and noise reduction. Existing defense strategies have been focusing on single- and dual-channel audio, while only few studies have explored using multi-channel microphone array for defending specific types of audio attack. Motivated by the lack of systematic research on defending miscellaneous audio attacks and the potential benefits of multi-channel audio, this paper builds a holistic solution for detecting machine-induced audio attacks leveraging multi-channel microphone arrays on modern intelligent audio systems. Specifically, we utilize magnitude and phase spectrograms of multi-channel audio to extract spatial information and leverage a deep learning model to detect the fundamental difference between human speech and adversarial audio generated by the playback machines. Moreover, we adopt an unsupervised domain adaptation training framework to further improve the model's generalizability in new acoustic environments. Evaluation is conducted under various settings on a public multi-channel replay attack dataset and a self-collected multi-channel audio attack dataset involving 5 types of advanced audio attacks. The results show that our method can achieve an equal error rate (EER) as low as 6.6% in detecting a variety of machine-induced attacks. Even in new acoustic environments, our method can still achieve an EER as low as 8.8%.

Earables (ear wearable) are rapidly emerging as a new platform to enable a variety of personal applications. The traditional authentication methods thus become less applicable and inconvenient for earables due to their limited input interface. Earables, however, often feature rich around the head sensing capability that can be leveraged to capture new types of biometrics. In this work, we propose ToothSonic that leverages the toothprint-induced sonic effect produced by a user performing teeth gestures for user authentication. In particular, we design several representative teeth gestures that can produce effective sonic waves carrying the information of the toothprint. To reliably capture the acoustic toothprint, it leverages the occlusion effect of the ear canal and the inward-facing microphone of the earables. It then extracts multi-level acoustic features to represent the intrinsic acoustic toothprint for authentication. The key advantages of ToothSonic are that it is suitable for earables and is resistant to various spoofing attacks as the acoustic toothprint is captured via the private teeth-ear channel of the user that is unknown to others. Our preliminary studies with 20 participants show that ToothSonic achieves 97% accuracy with only three teeth gestures.

Exercise intensity monitoring of physical activities has drawn increasingly attention as the awareness of the exercise intensity is of great importance for a person to achieve optimal training outcomes. For example, over-training could lead to excessive fatigue and loss of motivation for exercise. Traditional exercise intensity monitoring systems utilize GPS data to track the user’s intensity of cardio activities through his/her position and speed. Such systems however become invalid for indoor exercises on stationary fitness equipments such as the treadmill or exercise bike. Recent work in using body-worn sensors to track the user’s heart rate for exercise intensity monitoring usually involves additional wearable sensors which are only available on some particular fitness equipments, and thus are hard to be used in all occasions. This work presents an exercise intensity monitoring system which is capable of detecting a person’s exercise intensity via smartphones. Our system exploits the off-the-shelf smartphone and its headphone to capture the user’s breathing sound. Given the captured acoustic data, our system performs data pre-processing to remove the environmental noise and identify the non-silent acoustic frames based on the signal energy. Our system then conducts breathing event detection for non-silent frames, and further calibrates the detection results by utilizing the high correlation between breathing cycles to improve the detection accuracy. Moreover, our system can estimate the person’s exercise intensity based on features extracted from the frames which contain breathing sound. Our experiments involving 9 subjects over four-month time period demonstrate that our proposed exercise intensity monitoring system is robust and accurate in both indoor and outdoor environments.

Neuron networks pruning is effective in compressing pre-trained CNNs for their deployment on low-end edge devices. However, few works have focused on reducing the computational cost of pruning and inference. We find that existing pruning methods usually remove parameters without fine-grained impact analysis, making it hard to achieve an optimal solution. This work develops a novel mixture pruning mechanism, MIXP, which can effectively reduce the computational cost of CNNs while maintaining a high weight compression ratio and model accuracy. We propose to remove neuron bond that can effectively reduce convolution computations and weight size in CNNs. We also design an influence factor to analyze the importance of neuron bonds and weights in a fine-grained way so that MIXP could achieve precise pruning with few retraining iterations. Experiments with MNIST, CIFAR-10, and ImageNet datasets demonstrate that MIXP could achieve significantly fewer FLOPs and retraining iterations on four widely-used CNNs than existing pruning methods.

The physical layer secret key generation exploiting wireless channel reciprocity has attracted considerable attention in the past two decades. On-going research have demonstrated its viability in various radio frequency (RF) systems. Most of existing work rely on quantization technique to convert channel measurements into digital binaries that are suitable for secret key generation. However, non-simultaneous packet exchanges in time division duplex systems and noise effects in practice usually create random channel measurements between two users, leading to inconsistent quantization results and mismatched secret bits. While significant efforts were spent in recent research to mitigate such non-reciprocity, no efficient method has been found yet. Unlike existing quantization-based approaches, we take a different viewpoint and perform the secret key agreement by solving a bipartite graph matching problem. Specifically, an efficient dual-permutation secret key generation method, DP-SKG, is developed to match the randomly permuted channel measurements between a pair of users by minimizing their discrepancy holistically. DP-SKG allows two users to generate the same secret key based on the permutation order of channel measurements despite the non-reciprocity over wireless channels. Extensive experimental results show that DP-SKG could achieve error-free key agreement on received signal strength (RSS) with a low cost under various scenarios.

The two-factor authentication (2FA) has drawn increasingly attention as the mobile devices become more prevalent. For example, the user's possession of the enrolled phone could be used by the 2FA system as the second proof to protect his/her online accounts. Existing 2FA solutions mainly require some form of user-device interaction, which may severely affect user experience and creates extra burdens to users. In this work, we propose Proximity-Echo, a secure 2FA system utilizing the proximity of a user's enrolled phone and the login device as the second proof without requiring the user's interactions or pre-constructed device fingerprints. The basic idea of Proximity-Echo is to derive location signatures based on acoustic beep signals emitted alternately by both devices and sensing the echoes with microphones, and compare the extracted signatures for proximity detection. Given the received beep signal, our system designs a period selection scheme to identify two sound segments accurately: the chirp period is the sound segment propagating directly from the speaker to the microphone whereas the echo period is the sound segment reflected back by surrounding objects. To achieve an accurate proximity detection, we develop a new energy loss compensation extraction scheme by utilizing the extracted chirp periods to estimate the intrinsic differences of energy loss between microphones of the enrolled phone and the login device. Our proximity detection component then conducts the similarity comparison between the identified two echo periods after the energy loss compensation to effectively determine whether the enrolled phone and the login device are in proximity for 2FA. Our experimental results show that our Proximity-Echo is accurate in providing 2FA and robust to both man-in-the-middle (MiM) and co-located attacks across different scenarios and device models.

Low-cost in-baggage object identification is highly demanded in enhancing public safety and smart manufacturing. Existing approaches usually require specialized equipment and heavy deployment overhead, making them hard to scale for wide deployment. The recent WiFi-based approach is unsuitable for practical deployment as it did not address dynamic environmental impacts. In this work, we propose an environment-independent in-baggage object identification system by leveraging low-cost WiFi. We exploit the channel state information (CSI) to capture material and shape characteristics to facilitate fine-grained inbaggage object identification. A major challenge of building such a system is that CSI measurements are sensitive to real-world dynamics, such as different types of baggage, time-varying ambient noises and interferences, and different deployment environments. To tackle these problems, we develop WiFi features based on polarized directional antennas that can capture objects’ material and shape characteristics. A convolutional neural network-based model is developed to constructively integrate the WiFi features and perform accurate in-baggage object identification. We also develop a material-based domain adaptation using adversarial learning to facilitate fast deployments in different environments. We conduct extensive experiments involving 14 representation objects, 4 types of bags in 3 different room environments. The results show that our system can achieve over 97% in the same environment, and our domain adaptation method can improve the object identification accuracy by 42% when the system is deployed in a new environment with little training.

Augmented reality/virtual reality (AR/VR) has extended beyond 3D immersive gaming to a broader array of applications, such as shopping, tourism, education. And recently there has been a large shift from handheld-controller dominated interactions to headset-dominated interactions via voice interfaces. In this work, we show a serious privacy risk of using voice interfaces while the user is wearing the face-mounted AR/VR devices. Specifically, we design an eavesdropping attack, Face-Mic, which leverages speech-associated subtle facial dynamics captured by zero-permission motion sensors in AR/VR headsets to infer highly sensitive information from live human speech, including speaker gender, identity, and speech content. Face-Mic is grounded on a key insight that AR/VR headsets are closely mounted on the user's face, allowing a potentially malicious app on the headset to capture underlying facial dynamics as the wearer speaks, including movements of facial muscles and bone-borne vibrations, which encode private biometrics and speech characteristics. To mitigate the impacts of body movements, we develop a signal source separation technique to identify and separate the speech-associated facial dynamics from other types of body movements. We further extract representative features with respect to the two types of facial dynamics. We successfully demonstrate the privacy leakage through AR/VR headsets by deriving the user's gender/identity and extracting speech information via the development of a deep learning-based framework. Extensive experiments using four mainstream VR headsets validate the generalizability, effectiveness, and high accuracy of Face-Mic.

WiFi human sensing has become increasingly attractive in enabling emerging human-computer interaction applications. The corresponding technique has gradually evolved from the classification of multiple activity types to more fine-grained tracking of 3D human poses. However, existing WiFi-based 3D human pose tracking is limited to a set of predefined activities. In this work, we present Winect, a 3D human pose tracking system for free-form activity using commodity WiFi devices. Our system tracks free-form activity by estimating a 3D skeleton pose that consists of a set of joints of the human body. In particular, Winect first identifies the moving limbs by leveraging the signals reflected off the human body and separates the entangled signals for each limb. Then, our system tracks each limb and constructs a 3D skeleton of the body by modeling the inherent relationship between the movements of the limb and the corresponding joints. Our evaluation results show that Winect achieves centimeter-level accuracy for free-form activity tracking under various environments.

With the increasing integration of humans and the cyber world, user authentication becomes critical to support various emerging application scenarios requiring security guarantees. Existing works utilize Channel State Information (CSI) of WiFi signals to capture single human activities for non-intrusive and device-free user authentication, but multi-user authentication remains a challenging task. In this paper, we present a multi-user authentication system, MultiAuth, which can authenticate multiple users with a single commodity WiFi device. The key idea is to profile multipath components of WiFi signals induced by multiple users, and construct individual CSI from the multipath components to solely characterize each user for user authentication. Specifically, we propose a MUltipath Time-of-Arrival measurement algorithm (MUTA) to profile multipath components of WiFi signals in high resolution. Then, after aggregating and separating the multipath components related to users, MultiAuth constructs individual CSI based on the multipath components to solely characterize each user. To identify users, MultiAuth further extracts user behavior profiles based on the individual CSI of each user through time-frequency analysis, and leverages a dual-task neural network for robust user authentication. Extensive experiments involving 3 simultaneously present users demonstrate that MultiAuth is accurate and reliable for multi-user authentication with 87.6% average accuracy and 8.8% average false accept rate.

Distributed computation plays an essential role in cloud and edge computing. Data such as images, audio, and text can be represented as matrices to facilitate efficient computation, especially in the domains of distributed machine learning, computer vision, and signal processing. Many coded computation algorithms have been proposed for big data applications to securely partition and distribute matrices to parallel worker devices. However, these proposals have yet to be adapted for mobile platforms beyond theoretical means. Mobile IoT networks can greatly benefit from secure distributed computing, however, commercial devices such as smartphones and tablets are much more limited in resources compared to platforms in data centers, requiring special design considerations. We investigate existing distribution schemes from an operational complexity and security viewpoint and study their performance in several mobile IoT networks, identifying performance bottlenecks in regards to communication and computation costs. From our findings, we propose new, scalable algorithms optimized to handle the unique constraints of mobile IoT. Extensive evaluations of our proposals on publicly available image classification datasets show how distributed learning can be specially optimized to enhance runtime and battery performance on mobile IoT by over 10×.

This paper presents GoPose, a 3D skeleton-based human pose estimation system that uses commodity WiFi devices at home. Our system leverages the WiFi signals reflected off the human body for 3D pose estimation. In contrast to prior systems that need dedicated sensors, our system does not require a user to wear any sensors and can reuse the WiFi devices that already exist in a home environment for mass adoption. To realize such a system, we leverage the 2D AoA estimation of the signals reflected from the human body and the deep learning techniques. Preliminary results show GoPose achieves a high accuracy of 4.5cm in various scenarios.

In this paper, we build a speech privacy attack that exploits speech reverberations from a smartphone's inbuilt loudspeaker captured via a zero-permission motion sensor (accelerometer). We design our attack Spearphone, and demonstrate that speech reverberations from inbuilt loudspeakers, at an appropriate loudness, can impact the accelerometer, leaking sensitive information about the speech. In particular, we show that by exploiting the affected accelerometer readings and carefully selecting feature sets along with off-the-shelf machine learning techniques, Spearphone can perform gender classification (accuracy over 90%) and speaker identification (accuracy over 80%) for the audio/video playback on the smartphone for our recorded dataset. We use lightweight classifiers and an off-the-shelf machine learning tool so that the attacking effort is minimized, making our attack practical. Our results with testing the attack on a voice call and voice assistant response were also encouraging, showcasing the impact of the proposed attack. In addition, we perform speech recognition and speech reconstruction to extract more information about the eavesdropped speech to an extent. Our work brings to light a fundamental design vulnerability in many currently-deployed smartphones, which may put people's speech privacy at risk while using the smartphone in the loudspeaker mode during phone calls, media playback or voice assistant interactions.

Abstract Human–computer interaction through touch screens plays an increasingly important role in our daily lives. Besides smartphones and tablets, laptops are the most prevalent mobile devices for both work and leisure. To satisfy the requirements of some applications, it is desirable to re-equip a typical laptop with both handwriting and drawing capability. In this paper, we design a virtual writing tablet system, VPad, for traditional laptops without touch screens. VPad leverages two speakers and one microphone, which are available in most commodity laptops, to accurately track hand movements and recognize writing characters in the air without additional hardware. Specifically, VPad emits inaudible acoustic signals from two speakers in a laptop and then analyzes energy features and Doppler shifts of acoustic signals received by the microphone to track the trajectory of hand movements. Furthermore, we propose a state machine-based trajectory optimization method to correct the unexpected trajectory and employ a stroke direction sequence model based on probability estimation to recognize characters users write in the air. Experimental results show that VPad achieves the average error of 1.55 cm for trajectory tracking and the accuracy over 90% of character recognition merely through built-in audio devices on a laptop.

User authentication is a critical process in both corporate and home environments due to the ever-growing security and privacy concerns. With the advancement of smart cities and home environments, the concept of user authentication is evolved with a broader implication by not only preventing unauthorized users from accessing confidential information but also providing the opportunities for customized services corresponding to a specific user. Traditional approaches of user authentication either require specialized device installation or inconvenient wearable sensor attachment. This article supports the extended concept of user authentication with a device-free approach by leveraging the prevalent WiFi signals made available by IoT devices, such as smart refrigerator, smart TV, and smart thermostat, and so on. The proposed system utilizes the WiFi signals to capture unique human physiological and behavioral characteristics inherited from their daily activities, including both walking and stationary ones. Particularly, we extract representative features from channel state information (CSI) measurements of WiFi signals, and develop a deep-learning-based user authentication scheme to accurately identify each individual user. To mitigate the signal distortion caused by surrounding people’s movements, our deep learning model exploits a CNN-based architecture that constructively combines features from multiple receiving antennas and derives more reliable feature abstractions. Furthermore, a transfer-learning-based mechanism is developed to reduce the training cost for new users and environments. Extensive experiments in various indoor environments are conducted to demonstrate the effectiveness of the proposed authentication system. In particular, our system can achieve over 94% authentication accuracy with 11 subjects through different activities.

The development of smart homes has advanced the concept of user authentication to not only protecting user privacy but also facilitating personalized services to users. Along this direction, we propose to integrate user authentication with human-computer interactions between users and smart household appliances through widely-deployed WiFi infrastructures, which is non-intrusive and device-free. In this paper, we propose$FingerPass$which leverages channel state information (CSI) of surrounding WiFi signals to continuously authenticate users through finger gestures in smart homes.$FingerPass$separates the user authentication process into two stages, login and interaction, to achieve high authentication accuracy and low response latency simultaneously. In the login stage, we develop a deep learning-based approach to extract behavioral characteristics of finger gestures for highly accurate user identification. For the interaction stage, to provide continuous authentication in real time for satisfactory user experience, we design a verification mechanism with lightweight classifiers to continuously authenticate the user’s identity during each interaction of finger gestures. Experiments in real environments show that$FingerPass$can achieve the authentication accuracies of 90.6 percent under in-domain scenarios and 87.6 percent under cross-domain scenarios, as well as$186.6\;ms$response time during interactions.

This paper demonstrates the feasibility of a side-channel attack to infer keystrokes on touch screen leveraging an off-the-shelf smartphone. Although there exist some studies on keystroke eavesdropping attacks on touch screen, they are mainly direct eavesdropping attacks, i.e., require the device of victims compromised to provide side-channel information for the adversary, which are hardly launched in practical scenarios. In this work, we show the practicability of an indirect eavesdropping attack, KeyListener, which infers keystrokes on QWERTY keyboards of touch screen leveraging audio devices on a smartphone. We investigate the attenuation of acoustic signals, and find that a user's keystroke fingers can be localized through the attenuation of acoustic signals received by the microphones in the smartphone. We then utilize the attenuation of acoustic signals to localize each keystroke, and further analyze errors induced by ambient noises. To improve the accuracy of keystroke localization, KeyListener further tracks finger movements during inputs through phase change and Doppler effect to reduce errors of acoustic signal attenuation-based keystroke localization. In addition, a binary tree-based search approach is employed to infer keystrokes in a context-aware manner. The proposed keystroke eavesdropping attack is robust to various environments without the assistance of additional infrastructures. Extensive experiments demonstrate that the accuracy of keystroke inference in top-5 candidates can approach 90 percent with a top-5 error rate of around 6 percent, which is a strong indication of the possible user privacy leakage of inputs on QWERTY keyboard.

Different from traditional gestures, sign language gestures involve a lot of finger-level gestures without wrist or arm movements. They are hard to detect using existing motion sensors-based approaches. We introduce the first low-cost sign language gesture recognition system that can differentiate fine-grained finger movements using the Photoplethysmography (PPG) and motion sensors in commodity wearables. By leveraging the motion artifacts in PPG, our system can accurately recognize sign language gestures when there are large body movements, which cannot be handled by the traditional motion sensor-based approaches. We further explore the feasibility of using both PPG and motion sensors in wearables to improve the sign language gesture recognition accuracy when there are limited body movements. We develop a gradient boost tree (GBT) model and deep neural network-based model (i.e., ResNet) for classification. The transfer learning technique is applied to ResNet-based model to reduce the training effort. We develop a prototype using low-cost PPG and motions sensors and conduct extensive experiments and collect over 7000 gestures from 10 adults in the static and body-motion scenarios. Results demonstrate that our system can differentiate nine finger-level gestures from the American Sign Language with an average recognition accuracy over 98 percent.

Finding a small object (e.g., earbuds, keys or a wallet) in an indoor environment (e.g., in a house or an office) can be frustrating. In this paper, we propose an innovative system, calledHyperEar, to localize such an object using only a single smartphone, based on enhanced time-difference-of-arrival (TDoA) measurements over acoustic signals issued from the object. One major challenge is the hardware limitations of a Commercial-Off-The-Shelf (COTS) phone with a short separation between the two microphones and the low sampling rate of such microphones. HyperEar enhances the accuracy of TDoA measurements by virtually increasing distances between microphones through sliding the phone in the air. HyperEar requires no communication for synchronization between the phone and the object and is a low-cost and easy-to-use system. We evaluate the performance of HyperEar via extensive experiments in various indoor conditions and the results demonstrate that, for an object of 7 m away, HyperEar can achieve a mean localization accuracy of about 15 cm when the object in normal indoor environments.

Due to the open nature of voice input, voice assistant (VA) systems (e.g., Google Home and Amazon Alexa) are vulnerable to various security and privacy leakages (e.g., credit card numbers, passwords), especially when issuing critical user commands involving large purchases, critical calls, etc. Though the existing VA systems may employ voice features to identify users, they are still vulnerable to various acoustic-based attacks (e.g., impersonation, replay, and hidden command attacks). In this work, we propose a training-free voice authentication system, WearID, leveraging the cross-domain speech similarity between the audio domain and the vibration domain to provide enhanced security to the ever-growing deployment of VA systems. In particular, when a user gives a critical command, WearID exploits motion sensors on the user’s wearable device to capture the aerial speech in the vibration domain and verify it with the speech captured in the audio domain via the VA device’s microphone. Compared to existing approaches, our solution is low-effort and privacy-preserving, as it neither requires users’ active inputs (e.g., replying messages/calls) nor to store users’ privacy-sensitive voice samples for training. In addition, our solution exploits the distinct vibration sensing interface and its short sensing range to sound (e.g., 25cm) to verify voice commands. Examining the similarity of the two domains’ data is not trivial. The huge sampling rate gap (e.g., 8000Hz vs. 200Hz) between the audio and vibration domains makes it hard to compare the two domains’ data directly, and even tiny data noises could be magnified and cause authentication failures. To address the challenges, we investigate the complex relationship between the two sensing domains and develop a spectrogram-based algorithm to convert the microphone data into the lower-frequency “ motion sensor data” to facilitate cross-domain comparisons. We further develop a user authentication scheme to verify that the received voice command originates from the legitimate user based on the cross-domain speech similarity of the received voice commands. We report on extensive experiments to evaluate the WearID under various audible and inaudible attacks. The results show WearID can verify voice commands with 99.8% accuracy in the normal situation and detect 97.2% fake voice commands from various attacks, including impersonation/replay attacks and hidden voice/ultrasound attacks.

Existing efforts in audio adversarial attacks only focus on the scenarios where an adversary has prior knowledge of the entire speech input so as to generate an adversarial example by aligning and mixing the audio input with corresponding adversarial perturbation. In this work we consider a more practical and challenging attack scenario where the intelligent audio system takes streaming audio inputs (e.g., live human speech) and the adversary can deceive the system by playing adversarial perturbations simultaneously. This change in attack behavior brings great challenges, preventing existing adversarial perturbation generation methods from being applied directly. In practice, (1) the adversary cannot anticipate what the victim will say: the adversary cannot rely on their prior knowledge of the speech signal to guide how to generate adversarial perturbations; and (2) the adversary cannot control when the victim will speak: the synchronization between the adversarial perturbation and the speech cannot be guaranteed. To address these challenges, in this paper we propose AdvPulse, a systematic approach to generate subsecond audio adversarial perturbations, that achieves the capability to alter the recognition results of streaming audio inputs in a targeted and synchronization-free manner. To circumvent the constraints on speech content and time, we exploit penalty-based universal adversarial perturbation generation algorithm and incorporate the varying time delay into the optimization process. We further tailor the adversarial perturbation according to environmental sounds to make it inconspicuous to humans. Additionally, by considering the sources of distortions occurred during the physical playback, we are able to generate more robust audio adversarial perturbations that can remain effective even under over-the-air propagation. Extensive experiments on two representative types of intelligent audio systems (i.e., speaker recognition and speech command recognition) are conducted in various realistic environments. The results show that our attack can achieve an average attack success rate of over 89.6% in indoor environments and 76.0% in inside-vehicle scenarios even with loud engine and road noises.

Many existing identification approaches require active user input, specialized sensing hardware, or personally identifiable information such as fingerprints or face scans. In this paper, we propose EchoLock, a low-effort identification scheme that validates the user by sensing hand geometry via commodity microphones and speakers. EchoLock can serve as a complementary verification method for high-end devices or as a stand-alone user identification scheme for lower-end devices without using privacy-sensitive features. In addition to security applications, our system can also personalize user interactions with smart devices, such as automatically adapting settings or preferences when different people are holding smart remotes. To this end, we study the impact of hands on structure borne sound propagation in mobile devices and develop a user identification scheme that can measure, quantify, and exploit distinct sound reflections in order to differentiate distinct identities. Particularly, we propose a non-intrusive hand sensing technique to derive unique acoustic features in both time and frequency domain, which can effectively capture the physiological and behavioral traits of a user's hand (e.g., hand contours, finger sizes, holding strengths, and holding styles). Furthermore, learning-based algorithms are developed to robustly identify the user under various environments and conditions. We conduct extensive experiments with 20 participants, gathering 80,000 hand geometry samples using different hardware setups across 160 key use case scenarios. Our results show that EchoLock is capable of identifying users with over 94% accuracy, without requiring any active user input.

As the popularity of voice user interface (VUI) exploded in recent years, speaker recognition system has emerged as an important medium of identifying a speaker in many security-required applications and services. In this paper, we propose the first real-time, universal, and robust adversarial attack against the state-of-the-art deep neural network (DNN) based speaker recognition system. Through adding an audio-agnostic universal perturbation on arbitrary enrolled speaker's voice input, the DNN-based speaker recognition system would identify the speaker as any target (i.e., adversary-desired) speaker label. In addition, we improve the robustness of our attack by modeling the sound distortions caused by the physical over-the-air propagation through estimating room impulse response (RIR). Experiment using a public dataset of 109 English speakers demonstrates the effectiveness and robustness of our proposed attack with a high attack success rate of over 90%. The attack launching time also achieves a 100× speedup over contemporary non-universal attacks.

Drivers' identities are essential information that can facilitate a broad range of applications. For example, by understanding who is driving the vehicle when an accident happens, insurance companies could determine the liability and payment in a car accident claim case with high confidence. Another example, pick-up service companies could track the identities of their drivers to ensure that authorized drivers are driving esteemed clients to their destinations. While there are existing studies that can utilize video cameras and dedicated sensors to identify drivers, they either have privacy issues or require additional hardware, which is not practical enough for daily uses. In this paper, we devise a low-cost driver identification system, which can determine drivers' identities by using sensors readily available in wearable devices. Our system captures the unique driving behaviors during pervasive but momentary driving events (i.e., turning at intersections) with motion sensors, which are widely integrated into commodity wearable devices (e.g., smartphones and activity trackers). Toward this end, we extensively analyze people's driving behaviors and identify the critical turning events that capture people's unique behavioral patterns for driver identification. We design a fine-grained turning segmentation method that divides sensor data into critical turning stages (i.e., before, during, and after-turn stages), which provide multiple dimensions of turning behavioral metrics facilitating driver identification. The system extracts unique turning behavior features from time and frequency domains to enable driver identification based on drivers' turning behaviors at different types of turns. Extensive experiments are conducted with 12 drivers and various types of turns in real-road conditions. The results demonstrate that our system can identify drivers with high accuracy and low falsepositive rate based on one single turning event.

The proliferation of mobile devices along with their rich functionalities/applications have made people form addictive and potentially harmful usage behaviors. Though this problem has drawn considerable attention, existing solutions (e.g., text notification or setting usage limits) are insufficient and cannot provide timely recommendations or control of inappropriate usage of mobile devices. This paper proposes a generalized context inference framework, which supports timely usage recommendations using low-power sensors in mobile devices Comparing to existing schemes that rely on detection of single type user contexts (e.g., merely on location or activity), our framework derives a much larger-scale of user contexts that characterize the phone usages, especially those causing distraction or leading to dangerous situations. We propose to uniformly describe the general user context with context fundamentals, i.e., physical environments, social situations, and human motions, which are the underlying constituent units of diverse general user contexts. To mitigate the profiling efforts across different environments, devices, and individuals, we develop a deep learning-based architecture to learn transferable representations derived from sensor readings associated with the context fundamentals. Based on the derived context fundamentals, our framework quantifies how likely an inferred user context would lead to distractions/dangerous situations, and provides timely recommendations for mobile device access/usage. Extensive experiments during a period of 7 months demonstrate that the system can achieve 95% accuracy on user context inference while offering the transferability among different environments, devices, and users.

Eating well plays a key role in people's overall health and wellbeing. Studies have shown that many health-related problems such as obesity, diabetes and anemia are closely associated with people's unhealthy eating habits (e.g., skipping meals, eating irregularly and overeating). Thus, keeping track of diet is becoming more important. Traditional eating monitoring solutions relying on self-report remain an onerous task, while the recent trends requiring users to wear dedicated yet expensive hardware are cumbersome. To overcome these limitations, in this paper, we develop a device-free eating monitoring system using WiFi-enabled devices (e.g., smartphone or laptop). Our system aims to automatically monitor users' eating activities by identifying the fine-grained eating motions and detecting the minute movements during chewing and swallowing. In particular, our system distinguishes eating from non-eating activities by using K-means clustering with principal component analysis on the extracted Channel State Information (CSI) from WiFi signals. It further adopts a soft decision-based eating motion classification through identifying the utensils (e.g., using a folk, knife, spoon or bare hands) in use. Moreover, we propose a minute motion reconstruction method to identify chewing and swallowing through detecting users' minute facial muscle movements. The derived fine-grained eating monitoring results are beneficial to the understanding of users' eating behaviors and estimation of food intake types and amounts. Extensive experiments with 20 users over 1600-minute eating show that the proposed system can recognize the user's eating motions with up to 95% accuracy and estimate the chewing and swallowing amount within 10% percentage error.

The ever-growing security issues in various mobile applications and smart devices create an urgent demand for a reliable and convenient user verification method. Traditional verification methods request users to provide their secrets (e.g., entering passwords and collecting fingerprints). We envision that the essential trend of user verification is to free users from active participation in the verification process. Toward this end, we propose a continuous user verification system, which re-uses the widely deployed WiFi infrastructure to capture the unique physiological characteristics rooted in user's respiratory motions. Different from the existing continuous verification approaches, posing dependency on restricted scenarios/user behaviors (e.g., keystrokes and gaits), our system can be easily integrated into any WiFi infrastructure to provide non-intrusive continuous verification. Specifically, we extract the respiration-related signals from the channel state information (CSI) of WiFi. We then derive the user-specific respiratory features based on the waveform morphology analysis and fuzzy wavelet transformation of the respiration signals. Additionally, a deep learning based user verification scheme is developed to identify legitimate users accurately and detect the existence of spoofing attacks. Extensive experiments involving 20 participants demonstrate that the proposed system can robustly verify/identify users and detect spoofers under various types of attacks.

The rapid advancement of social media and communication technology enables video chat to become an important and convenient way of daily communication. However, such convenience also makes personal video clips easily obtained and exploited by malicious users who launch scam attacks. Existing studies only deal with the attacks that use fabricated facial masks, while the liveness detection that targets the playback attacks using a virtual camera is still elusive. In this work, we develop a novel video chat liveness detection system, LiveScreen, which can track the weak light changes reflected off the skin of a human face leveraging chromatic eigenspace differences. We design an inconspicuous challenge frame with minimal intervention to the video chat and develop a robust anomaly frame detector to verify the liveness of the remote user in the video chat using the response to the challenge frame. Furthermore, we propose resilient defense strategies to defeat both naive and intelligent playback attacks leveraging spatial and temporal verification. We implemented a prototype over both laptop and smartphone platforms and conducted extensive experiments in various realistic scenarios. We show that our system can achieve robust liveness detection with accuracy and false detection rates 97.7% (94.8%) and 1% (1.6%) on smartphones (laptops), respectively.

Multi-user identification could facilitate various large-scale identity-based services such as access control, automatic surveillance system, and personalized services, etc. Although existing solutions can identify multiple users using cameras, such vision-based approaches usually raise serious privacy concerns and require the presence of line-of-sight. Differently, in this paper, we propose MU-ID, a gait-based multi-user identification system leveraging a single commercial off-the-shelf (COTS) millimeter-wave (mmWave) radar. Particularly, MU-ID takes as input frequency-modulated continuous-wave (FMCW) signals from the radar sensor. Through analyzing the mmWave signals in the range-Doppler domain, MU-ID examines the users' lower limb movements and captures their distinct gait patterns varying in terms of step length, duration, instantaneous lower limb velocity, and inter-lower limb distance, etc. Additionally, an effective spatial-temporal silhouette analysis is proposed to segment each user's walking steps. Then, the system identifies steps using a Convolutional Neural Network (CNN) classifier and further identifies the users in the area of interest. We implement MU-ID with the TI AWR1642BOOST mmWave sensor and conduct extensive experiments involving 10 people. The results show that MU-ID achieves up to 97% single-person identification accuracy, and over 92% identification accuracy for up to four people, while maintaining a low false positive rate.

Traditional one-time user authentication processes might cause friction and unfavorable user experience in many widely-used applications. This is a severe problem in particular for security-sensitive facilities if an adversary could obtain unauthorized privileges after a user's initial login. Recently, continuous user authentication (CA) has shown its great potential by enabling seamless user authentication with few active participation. We devise a low-cost system exploiting a user's pulsatile signals from the photoplethysmography (PPG) sensor in commercial wrist-worn wearables for CA. Compared to existing approaches, our system requires zero user effort and is applicable to practical scenarios with non-clinical PPG measurements having motion artifacts (MA). We explore the uniqueness of the human cardiac system and design an MA filtering method to mitigate the impacts of daily activities. Furthermore, we identify general fiducial features and develop an adaptive classifier using the gradient boosting tree (GBT) method. As a result, our system can authenticate users continuously based on their cardiac characteristics so little training effort is required. Experiments with our wrist-worn PPG sensing platform on 20 participants under practical scenarios demonstrate that our system can achieve a high CA accuracy of over 90% and a low false detection rate of 4% in detecting random attacks.

With the increasing prevalence of smart mobile and Internet of things (IoT) environments, user authentication has become a critical component for not only preventing unauthorized access to security-sensitive systems but also providing customized services for individual users. Unlike traditional approaches relying on tedious passwords or specialized biometric/wearable sensors, this paper presents a device-free user authentication via daily human behavioral patterns captured by existing WiFi infrastructures. Specifically, our system exploits readily available channel state information (CSI) in WiFi signals to capture unique behavioral biometrics residing in the user’s daily activities, without requiring any dedicated sensors or wearable device attachment. To build such a system, one major challenge is that wireless signals always carry substantial information that is specific to the user’s location and surrounding environment, rendering the trained model less effective when being applied to the data collected in a new location or environment. This issue could lead to significant authentication errors and may quickly ruin the whole system in practice. To disentangle the behavioral biometrics for practical environment-independent user authentication, we propose an end-to-end deep-learning based approach with domain adaptation techniques to remove the environment-and location-specific information contained in the collected WiFi measurements. Extensive experiments in a residential apartment and an office with various scales of user location variations and environmental changes demonstrate the effectiveness and generalizability of the proposed authentication system.

With increasing private and sensitive data stored in mobile devices, secure and effective mobile-based user authentication schemes are desired. As the most natural way to contact with mobile devices, finger touches have shown potentials for user authentication. Most existing approaches utilize finger touches as behavioral biometrics for identifying individuals, which are vulnerable to spoofer attacks. To resist attacks for on-touch user authentication on mobile devices, this paper exploits physical characters of touching fingers by investigating active vibration signal transmission through fingers, and we find that physical characters of touching fingers present unique patterns on active vibration signals for different individuals. Based on the observation, we propose a behavior-irrelevant on-touch user authentication system, TouchPass, which leverages active vibration signals on smartphones to extract only physical characters of touching fingers for user identification. TouchPass first extracts features that mix physical characters of touching fingers and behavior biometrics of touching behaviors from vibration signals generated and received by smartphones. Then, we design a Siamese network-based architecture with a specific training sample selection strategy to reconstruct the extracted signal features to behavior-irrelevant features and further build a behavior-irrelevant on-touch user authentication scheme leveraging knowledge distillation. Our extensive experiments validate that TouchPass can accurately authenticate users and defend various attacks.

Acoustic communication is an increasingly popular alternative to existing short-range wireless communication technologies for mobile devices, such as NFC and QR codes. Unlike the current standards, there are no requirements for extra hardware, lighting conditions, or Internet connection. However, the audibility and limited throughput of existing studies hinder their deployment on a wide range of applications. In this paper, we aim to redesign acoustic communication mechanism to push the boundary of potential throughput while keeping the inaudibility. Specifically, we propose BatComm, a high-throughput and inaudible acoustic communication system for mobile devices capable of throughput rates 12X higher than contemporary state-of-the-art acoustic communication for mobile devices. We theoretically model the non-linearity of microphone and use orthogonal frequency division multiplexing (OFDM) to transmit data bits over multiple orthogonal channels with an ultrasound frequency carrier. We also design a series of techniques to mitigate interference caused by sources such as the signal's unbalanced frequency response, ambient noise, and unrelated residual signals created through OFDM, amplitude modulation (AM), and related processes. Extensive evaluations under multiple realistic settings demonstrate that our inaudible acoustic communication system can achieve over 47kbps within a 10cm communication range. We also show the possibility of increasing the communication range to room scale (i.e., around 2m) while maintaining high-throughput and inaudibility. Our findings offer a new direction for future inaudible acoustic communication techniques to pursue in emerging mobile and IoT applications.

The explosive usage of mobile devices enables conducting electronic transactions involving direct signature on such devices. Thus, user signature verification becomes critical to ensure the success deployment of online transactions such as approving legal documents and authenticating financial transactions. Existing approaches mainly focus on user verification targeting the unlocking of mobile devices or performing continuous verification based on a user's behavioral traits. Few studies provide efficient real-time user signature verification. In this work, we propose a critical segment based online signature verification system to secure mobile transactions on multi-touch mobile devices. Our system identifies and exploits the segments which remain invariant within a user's signature to capture the intrinsic signing behavior embedded in each user's signature. Our system extracts useful features from a user's signature that describe both the geometric layout of the signature as well as behavioral and physiological characteristics in the user's signing process. Given the input signatures for user enrollment, our system further designs a quality score to identify the problematic signature sets to achieve robust user signature profile construction. Moreover, we develop the signature normalization and interpolation methods to achieve robust signature verification in the presence of signature geometric distortions caused by different writing sizes, orientations and locations on touch screens. Our experimental evaluation of 25 subjects over six months time period shows that our system is highly accurate in provide signature verification and robust to signature forging attacks.

Logistics monitoring is a fundamental application that utilizes RFID systems to manage numerous tagged-objects. Due to the frequent rearrangement of tagged-objects, a fast RFID-based tracking approach is highly desired for accurate logistics distribution. However, traditional RFID systems usually take tens of seconds to interrogate hundreds of RFID tags, not to mention the time delay involved to locate all the tags, which severely prevents from in-time tracking. To address this issue, we reduce the problem domain by first distinguishing the motion status of the tagged-objects, i.e., “stationary” or “moving”, and then tracking the moving objects with the state-of-the-art localization schemes, which significantly reduces the efforts of tracking all the objects. Toward this end, we propose a moving tag detection mechanism, which achieves the time efficiency by exploiting the useless collision signal in RFID systems. In particular, we extract two kinds of physical-layer features (namely, phase profile and backscatter link frequency) from the collision signal received by the USRP to distinguish tags at different positions. We further develop the Graph Matching (GM) method and Coherent Phase Variance (CPV) method to detect the moving tagged-objects. Experiment results show that our approach can accurately detect the moving objects while reducing 80 percent inventory time compared with the state-of-art solutions.

Given the increasing popularity, mobile devices are exploited to enhance active driving safety nowadays. Among all safety services provided for vehicles, tracking the rotation angle of steering wheel in real time can monitor the vehicles' dynamics and drivers' behaviors at the same time. In this paper, we propose a steering tracking system, SteerTrack, which tracks the rotation angle of the steering wheel in real time leveraging audio devices on smartphones. SteerTrack seeks a device-free approach for steering tracking without requiring installation of specialized sensors on the steering wheels nor asking drivers to wear sensors on their wrists. Since the steering wheel is operated by a driver's hands, the rotation angle of the steering wheel can be tracked based on movements of the driver's hands. SteerTrack first builds an acoustic signal field inside of a vehicle and then analyzes the echoes reflected from the driver's hands with relative correlation coefficient (RCC) and reference frame to track the movement trajectory of hands under different steering maneuvers. Given the tracked movement trajectory, SteerTrackfurther develops a geometrical transformation-based method for estimating the rotation angle of the steering wheel in 3D driving environments by projecting the steering wheel to a 2D ellipse. Through extensive experiments in real driving environments with five volunteers for several weeks, SteerTrack can achieve an average steering wheel estimation error of 1.48 degree during driving, and 4.61 degree for turns.

The number of Internet of Things is growing exponentially, among which the ZigBee devices are being widely deployed, incurring severe collision problem in ZigBee networks. Instead of collision avoidance or packet retransmissions which introduce extra time/energy overhead, existing methods try to decompose multi-packet collision directly. For example, state-of-the-art mZig exploits collision-free chips to decompose the collided chips iteratively, however, suffers from the high bit error rate and low frame reception rate which limit the practical applications. Toward this end, we observe three major issues of existing solutions: 1) all existing solutions adopt the priori-chip-dependent decomposition pattern, leading to the error propagation; 2) the available samples for chip decoding can be scarce, resulting in severe scarce-sample errors; 3) existing solutions assume the consistent frequency offset for consecutive packets, leading to inaccurate frequency offset estimation. To solve these issues in collision decomposition, we propose FORWARD, a novel physical layer design to enable accurate collision decoding in ZigBee. The key idea is to generate all possible overlapping combinations as reference waveforms. The decomposition is determined by comparing the collided signal with the reference waveforms. Such a priori-chip-independent design has the advantages to eliminate the error propagation. To ensure sufficient samples for decoding, FORWARD always choose the longest segment as reference. Furthermore, the real-time channel estimation and frequency offset calibration ensure the accurate collision decoding. We implement FORWARD on USRP platform and evaluate its performance. Experimental results demonstrate that FORWARD reduces bit error rate by order of magnitude and increases frame reception rate by 10% ~ 50% compared with the state-of-the-art.

Voice access technologies are widely adopted in mobile devices and voice assistant systems as a convenient way of user interaction. Recent studies have demonstrated a potentially serious vulnerability of the existing voice interfaces on these systems to "hidden voice commands". This attack uses synthetically rendered adversarial sounds embedded within a voice command to trick the speech recognition process into executing malicious commands, without being noticed by legitimate users.

Finding a small object (e.g., keys or a wallet) in an indoor environment (e.g., in a house or an office) can be frustrating. In this paper, we propose an innovative system, called HyperEar, to localize such an object using only one single smartphone, based on enhanced time-difference-of-arrival (TDoA) measurements over acoustic signals issued from the object. One major challenge is the hardware limitations of a Commercial-Off-The-Shelf (COTS) phone with a short separation between the two microphones and the low sampling rate of such microphones. HyperEar enhances the accuracy of TDoA measurements by virtually increasing distances between microphones through sliding the phone in the air. HyperEar requires no communication for synchronization between the phone and the object and is a low-cost and easy-to-use system. We evaluate the performance of HyperEar via extensive experiments in various indoor conditions and the results demonstrate that, for an object of 7m away, HyperEar can achieve a mean localization accuracy of about 15cm when the object in normal indoor environments.

As wireless communication is tailored for low-power devices while the number of Internet of Things is growing exponentially, the collision problem in ZigBee is worsen. The classical approaches of solving collision problems lie in collision avoidance and packet retransmission, which could incur considerable overhead. The new trend is to decompose multipacket collision directly, however, the high bit error rate limits its practical applications. Toward this end, we observe three major issues in the existing solutions: 1) all existing solutions adopt the priori-chip-dependent decomposition pattern, leading to the error propagation; 2) the available samples for chip decoding can be scarce, resulting in severe scarce-sample errors; 3) existing solutions assume the consistent frequency offset for consecutive packets, leading to inaccurate frequency offset estimation. To solve the issues of collision decomposition in ZigBee, we propose FORWARD, a novel physical layer design to enable highly accurate collision decomposition in ZigBee. The key idea is to generate all possible collided combinations as reference waveforms. The decomposition is determined by comparing the collided signal with the reference waveforms. Such a priori-chip-independent design has the advantages to eliminate the cumulative errors incurred from error propagation. When decoding, FORWARD always choose the longest segment to ensure sufficient samples for decoding. Furthermore, the recursive calibration design is approaching the real-time frequency offset and dynamically compensates the reference waveform. We implement FORWARD on USRP based testbed and evaluate its performance. Experimental results demonstrate that FORWARD reduces bit error rate by 4.96× and increases throughput 1.46~2.8× compared with the state-of-the-art mZig.

This paper demonstrates the feasibility of a side-channel attack to infer keystrokes on touch screen leveraging an off-the-shelf smartphone. Although there exist some studies on keystroke eavesdropping attacks on touch screen, they are mainly direct eavesdropping attacks, i.e., require the device of victims compromised to provide side-channel information for the adversary, which are hardly launched in practical scenarios. In this work, we show the practicability of an indirect eavesdropping attack, KeyListener, which infers keystrokes on QWERTY keyboards of touch screen leveraging audio devices on a smartphone. We investigate the attenuation of acoustic signals, and find that a user's keystroke fingers can be localized through the attenuation of acoustic signals received by the microphones in the smartphone. We then utilize the attenuation of acoustic signals to localize each keystroke, and further analyze errors induced by ambient noises. To improve the accuracy of keystroke localization, KeyListener further tracks finger movements during inputs through phase change and Doppler effect to reduce errors of acoustic signal attenuation-based keystroke localization. In addition, a binary tree-based search approach is employed to infer keystrokes in a context-aware manner. The proposed keystroke eavesdropping attack is robust to various environments without the assistance of additional infrastructures. Extensive experiments demonstrate that the accuracy of keystroke inference in top-5 candidates can approach 90% with a top-5 error rate of around 6%, which is a strong indication of the possible user privacy leakage of inputs on QWERTY keyboard.

Mobile payment has drawn considerable attention due to its convenience of paying via personal mobile devices at anytime and anywhere, and passcodes (i.e., PINs or patterns) are the first choice of most consumers to authorize the payment. This paper demonstrates a serious security breach and aims to raise the awareness of the public that the passcodes for authorizing transactions in mobile payments can be leaked by exploiting the embedded sensors in wearable devices (e.g., smartwatches). We present a passcode inference system, WristSpy, which examines to what extent the user's PIN/pattern during the mobile payment could be revealed from a single wrist-worn wearable device under different passcode input scenarios involving either two hands or a single hand. In particular, WristSpy has the capability to accurately reconstruct fine-grained hand movement trajectories and infer PINs/patterns when mobile and wearable devices are on two hands through building a Euclidean distance-based model and developing a training-free parallel PIN/pattern inference algorithm. When both devices are on the same single hand, a highly challenging case, WristSpy extracts multi-dimensional features by capturing the dynamics of minute hand vibrations and performs machine-learning based classification to identify PIN entries. Extensive experiments with 15 volunteers and 1600 passcode inputs demonstrate that an adversary is able to recover a user's PIN/pattern with up to 92% success rate within 5 tries under various input scenarios.

In recent decades, countless efforts have been put into the research and development of short-range wireless communication, which offers a convenient way for numerous applications (e.g., mobile payments, mobile advertisement). Regarding the design of acoustic communication, throughput and inaudibility are the most vital aspects, which greatly affect available applications that can be supported and their user experience. Existing studies on acoustic communication either use audible frequency band (e.g., <20kHz) to achieve a relatively high throughput or realize inaudibility using near-ultrasonic frequency band (e.g., 18-20kHz) which however can only achieve limited throughput. Leveraging the non-linearity of microphones, voice commands can be demodulated from the ultrasound signals, and further recognized by the speech recognition systems. In this poster, we design an acoustic communication system, which achieves high-throughput and inaudibility at the same time, and the highest throughput we achieve is over 17x higher than the state-of-the-art acoustic communication systems.

The rapid advancement of social media and communication technology enables video chat to become an important and convenient way of daily communication. However, such convenience also makes personal video clips easily obtained and exploited by malicious users who launch scam attacks. Existing studies only deal with the attacks that use fabricated facial masks, while the liveness detection that targets the playback attacks using a virtual camera is still elusive. In this work, we develop a novel video chat liveness detection system, which can track the weak light changes reflected off the skin of a human face leveraging chromatic eigenspace differences. We design an inconspicuous challenge frame with minimal intervention to the video chat and develop a robust anomaly frame detector to verify the liveness of remote user in a video chat session. Furthermore, we propose a resilient defense strategy to defeat both naive and intelligent playback attacks leveraging spatial and temporal verification. The evaluation results show that our system can achieve accurate and robust liveness detection with the accuracy and false detection rate as high as 97.7% (94.8%) and 1% (1.6%) on smartphones (laptops), respectively.

We present a photoplethysmography (PPG)-based continuous user authentication (CA) system leveraging the pervasively equipped PPG sensor in commodity wrist-worn wearables such as the smartwatch. Compared to existing approaches, our system does not require any users' interactions (e.g., performing specific gestures) and is applicable to practical scenarios where the user's daily activities cause motion artifacts (MA). Notably, we design a robust MA removal method to mitigate the impact of MA. Furthermore, we explore the uniqueness of the human cardiac system and extract the fiducial features in the PPG measurements to train the gradient boosting tree (GBT) classifier, which can effectively differentiate users continuously using low training effort. In particular, we build the prototype of our system using a commodity smartwatch and a WebSocket server running on a laptop for CA. In order to demonstrate the practical use of our system, we will demo our prototype under different scenarios (i.e., static and moving) to show it can effectively detect MA caused by daily activities and achieve a high authentication success rate.

The development of smart homes has advanced the concept of user authentication to not only protecting user privacy but also facilitating personalized services to users. Along this direction, we propose to integrate user authentication with human-computer interactions between users and smart household appliances through widely-deployed WiFi infrastructures, which is non-intrusive and device-free. In this paper, we propose FingerPass which leverages channel state information (CSI) of surrounding WiFi signals to continuously authenticate users through finger gestures in smart homes. We investigate CSI of WiFi signals in depth and find CSI phase can be used to capture and distinguish the unique behavioral characteristics from different users. FingerPass separates the user authentication process into two stages, login and interaction, to achieve high authentication accuracy and low response latency simultaneously. In the login stage, we develop a deep learning-based approach to extract behavioral characteristics of finger gestures for highly accurate user identification. For the interaction stage, to provide continuous authentication in real time for satisfactory user experience, we design a verification mechanism with lightweight classifiers to continuously authenticate the user's identity during each interaction of finger gestures. Experiments in real environments show that FingerPass can achieve 91.4% authentication accuracy, and 186.6ms response time during interactions.

With the increasing prevalence of mobile and IoT devices (e.g., smartphones, tablets, smart-home appliances), massive private and sensitive information are stored on these devices. To prevent unauthorized access on these devices, existing user verification solutions either rely on the complexity of user-defined secrets (e.g., password) or resort to specialized biometric sensors (e.g., fingerprint reader), but the users may still suffer from various attacks, such as password theft, shoulder surfing, smudge, and forged biometrics attacks. In this paper, we propose, CardioCam, a low-cost, general, hard-to-forge user verification system leveraging the unique cardiac biometrics extracted from the readily available built-in cameras in mobile and IoT devices. We demonstrate that the unique cardiac features can be extracted from the cardiac motion patterns in fingertips, by pressing on the built-in camera. To mitigate the impacts of various ambient lighting conditions and human movements under practical scenarios, CardioCam develops a gradient-based technique to optimize the camera configuration, and dynamically selects the most sensitive pixels in a camera frame to extract reliable cardiac motion patterns. Furthermore, the morphological characteristic analysis is deployed to derive user-specific cardiac features, and a feature transformation scheme grounded on Principle Component Analysis (PCA) is developed to enhance the robustness of cardiac biometrics for effective user verification. With the prototyped system, extensive experiments involving $25$ subjects are conducted to demonstrate that CardioCam can achieve effective and reliable user verification with over $99%$ average true positive rate (TPR) while maintaining the false positive rate (FPR) as low as $4%$.

Given the increasing amount of time people spent on driving, the physical and mental health of drivers is essential to road safety. Breathing patterns are critical indicators of the well-being of drivers on the road. Existing studies on breathing monitoring require active user participation of wearing special sensors or relatively quiet environments during sleep, which are hardly applicable to noisy driving environments. In this work, we propose a fine-grained breathing monitoring system, BreathListener, which leverages audio devices on smartphones to estimate the fine-grained breathing waveform in driving environments. By investigating the data collected from real driving environments, we find that Energy Spectrum Density (ESD) of acoustic signals can be utilized to capture breathing procedures in driving environments. To extract breathing pattern in ESD signals, BreathListener eliminates interference from driving environments in ESD signals utilizing background subtraction and Ensemble Empirical Mode Decomposition (EEMD). After that, the extracted breathing pattern is transformed into Hilbert spectrum, and we further design a deep learning architecture based on Generative Adversarial Network (GAN) to generate fine-grained breathing waveform from the Hilbert spectrum of extracted breathing patterns in ESD signals. Experiments with 10 drivers in real driving environments show that BreathListener can accurately capture breathing patterns of drivers in driving environments.

Sleep monitoring has drawn increasing attention as sleep quality is important to maintain a person's well-being. For instance, serious health problems, such as cardiovascular disease, fatigue, or depression, are usually associated with inadequate and irregular sleep. Traditional sleep monitoring systems involve wearable sensors with professional installation, and thus are usually limited to clinical usage. Recent work for sleep monitoring can detect several sleep events, such as coughing and snoring, using smartphone sensors. However, such coarse-grained sleep monitoring is unable to detect the breathing rate which is an important health indicator. In this paper, we present a fine-grained sleep monitoring system to detect the breathing rate and sleep events simultaneously by leveraging smartphones. Our system exploits the readily available smartphone earphone placed close to the user to reliably capture the human breathing sound. Given the captured acoustic sound, noise reduction is performed to remove the environmental noise and the breathing rate is then identified based on the signal envelope detection. Our system can further detect some sleep events, including snoring, coughing, turning over, and getting up, based on the features extracted from the acoustic sound. Moreover, we develop a body movement-assisted sleep event detection method to provide higher detection accuracy by further exploiting the user's body movement patterns captured by the accelerometer embedded on smartphones. Our extensive experiments involving nine subjects over six months confirm the effectiveness of our proposed system on breathing rate monitoring and sleep events detection under various environments. By combining breathing rate and sleep events, our system can provide noninvasive and continuous fine-grained sleep monitoring for healthcare related applications, such as sleep apnea monitoring, as evidenced by our experimental study.

To prevent users privacy from leakage, more and more mobile devices employ biometric-based authentication approaches, such as fingerprint, face recognition, voiceprint authentications, and so on, to enhance the privacy protection. However, these approaches are vulnerable to replay attacks. Although the state-of-art solutions utilize liveness verification to combat the attacks, existing approaches are sensitive to ambient environments, such as ambient lights and surrounding audible noises. Toward this end, we explore liveness verification of user authentication leveraging users mouth movements, which are robust to noisy environments. In this paper, we propose a lip reading-based user authentication system, LipPass, which extracts unique behavioral characteristics of users speaking mouths through acoustic sensing on smartphones for user authentication. We first investigate Doppler profiles of acoustic signals caused by users' speaking mouths and find that there are unique mouth movement patterns for different individuals. To characterize the mouth movements, we propose a deep learning-based method to extract efficient features from Doppler profiles and employ softmax function, support vector machine and support vector domain description to construct multi-class identifier, binary classifiers and spoofer detectors for mouth state identification, user identification and spoofer detection, respectively. Afterward, we develop a balanced binary tree-based authentication approach to accurately identify each individual leveraging these binary classifiers and spoofer detectors with respect to registered users. Through extensive experiments involving 48 volunteers in four real environments, LipPass can achieve 90.2% accuracy in user identification and 93.1% accuracy in spoofer detection.

Human-computer interaction based on touch screens plays an increasing role in our daily lives. Besides smartphones and tablets, laptops are the most popular mobile devices used in both work and leisure. To satisfy requirements of many emerging applications, it becomes desirable to equip both writing and drawing functions directly on laptop screens. In this paper, we design a virtual writing tablet system, VPad, for traditional laptops without touch screens. VPad leverages two speakers and one microphone, which are available in most commodity laptops, for trajectory tracking without additional hardware. It employs acoustic signals to accurately track hand movements and recognize characters user writes in the air. Specifically, VPad emits inaudible acoustic signals from two speakers in a laptop. Then VPad applies Sliding-window Overlap Fourier Transformation technique to find Doppler frequency shift with higher resolution and accuracy in real time. Furthermore, we analyze frequency shifts and energy features of acoustic signals received by the microphone to track the trajectory of hand movements. Finally, we employ a stroke direction sequence model based on possibility estimation to recognize characters users write in the air. Our experimental results show that VPad achieves the average trajectory tracking error of only 1.55cm and the character recognition accuracy of above 90% merely through two speakers and one microphone on a laptop.

To prevent users' privacy from leakage, more and more mobile devices employ biometric-based authentication approaches, such as fingerprint, face recognition, voiceprint authentications, etc., to enhance the privacy protection. However, these approaches are vulnerable to replay attacks. Although state-of-art solutions utilize liveness verification to combat the attacks, existing approaches are sensitive to ambient environments, such as ambient lights and surrounding audible noises. Towards this end, we explore liveness verification of user authentication leveraging users' lip movements, which are robust to noisy environments. In this paper, we propose a lip reading-based user authentication system, LipPass, which extracts unique behavioral characteristics of users' speaking lips leveraging build-in audio devices on smartphones for user authentication. We first investigate Doppler profiles of acoustic signals caused by users' speaking lips, and find that there are unique lip movement patterns for different individuals. To characterize the lip movements, we propose a deep learning-based method to extract efficient features from Doppler profiles, and employ Support Vector Machine and Support Vector Domain Description to construct binary classifiers and spoofer detectors for user identification and spoofer detection, respectively. Afterwards, we develop a binary tree-based authentication approach to accurately identify each individual leveraging these binary classifiers and spoofer detectors with respect to registered users. Through extensive experiments involving 48 volunteers in four real environments, LipPass can achieve 90.21% accuracy in user identification and 93.1% accuracy in spoofer detection.

Recently, gesture recognition has gained considerable attention in emerging applications (e.g., AR/VR systems) to provide a better user experience for human-computer interaction. Existing solutions usually recognize the gestures based on wearable sensors or specialized signals (e.g., WiFi, acoustic and visible light), but they are either incurring high energy consumption or susceptible to the ambient environment, which prevents them from efficiently sensing the fine-grained finger movements. In this paper, we present RF-finger, a device-free system based on Commercial-Off-The-Shelf (COTS) RFID, which leverages a tag array on a letter-size paper to sense the fine-grained finger movements performed in front of the paper. Particularly, we focus on two kinds of sensing modes: finger tracking recovers the moving trace of finger writings; multi-touch gesture recognition identifies the multi-touch gestures involving multiple fingers. Specifically, we build a theoretical model to extract the fine-grained reflection feature from the raw RF -signal, which describes the finger influence on the tag array in cm- level resolution. For the finger tracking, we leverage K-Nearest Neighbors (KNN) to pinpoint the finger position relying on the fine-grained reflection features, and obtain a smoothed trace via Kalman filter. Additionally, we construct the reflection image of each multi-touch gesture from the reflection features by regarding the multiple fingers as a whole. Finally, we use a Convolutional Neural Network (CNN) to identify the multi-touch gestures based on the images. Extensive experiments validate that RF -finger can achieve as high as 88% and 92% accuracy for finger tracking and multi-touch gesture recognition, respectively.

This paper subverts the traditional understanding of Photoplethysmography (PPG) and opens up a new direction of the utility of PPG in commodity wearable devices, especially in the domain of human computer interaction of fine-grained gesture recognition. We demonstrate that it is possible to leverage the widely deployed PPG sensors in wrist-worn wearable devices to enable finger-level gesture recognition, which could facilitate many emerging human-computer interactions (e.g., sign-language interpretation and virtual reality). While prior solutions in gesture recognition require dedicated devices (e.g., video cameras or IR sensors) or leverage various signals in the environments (e.g., sound, RF or ambient light), this paper introduces the first PPG-based gesture recognition system that can differentiate fine-grained hand gestures at finger level using commodity wearables. Our innovative system harnesses the unique blood flow changes in a user's wrist area to distinguish the user's finger and hand movements. The insight is that hand gestures involve a series of muscle and tendon movements that compress the arterial geometry with different degrees, resulting in significant motion artifacts to the blood flow with different intensity and time duration. By leveraging the unique characteristics of the motion artifacts to PPG, our system can accurately extract the gesture-related signals from the significant background noise (i.e., pulses), and identify different minute finger-level gestures. Extensive experiments are conducted with over 3600 gestures collected from 10 adults. Our prototype study using two commodity PPG sensors can differentiate nine finger-level gestures from American Sign Language with an average recognition accuracy over 88%, suggesting that our PPG-based finger-level gesture recognition system is promising to be one of the most critical components in sign language translation using wearables.

This work proposes a continuous user verification system based on unique human respiratory-biometric characteristics extracted from the off-the-shelf WiFi signals. Our system innovatively re-uses widely available WiFi signals to capture the unique physiological characteristics rooted in respiratory motions for continuous authentication. Different from existing continuous authentication approaches having limited applicable scenarios due to their dependence on restricted user behaviors (e.g., keystrokes and gaits) or dedicated sensing infrastructures, our approach can be easily integrated into any existing WiFi infrastructure to provide non-invasive continuous authentication independent of user behaviors. Specifically, we extract representative features leveraging waveform morphology analysis and fuzzy wavelet transformation of respiration signals derived from the readily available channel state information (CSI) of WiFi. A respiration-based user authentication scheme is developed to accurately identify users and reject spoofers. Extensive experiments involving 20 subjects demonstrate that the proposed system can achieve a high authentication success rate of over 93% and robustly defend against various types of attacks.

Due to the extensive use of smart devices using them to study the driving behaviors has attracted a lot of researchers. This work demonstrates the problem of identifying drivers based on their driving style using smart phones. For this purpose the turns done by the drivers are being studied. Different sensors are embedded in the smart phones which are being used in order to extract some features to distinguish different drivers. Experiments are being done with four drivers and the results show that our system can distinguish them with high accuracy of 92% using only one turn.

Mobile payment has drawn considerable attention due to its convenience of paying via personal mobile devices at anytime and anywhere, and passcodes (i.e., PINs) are the first choice of most consumers to authorize the payment. This work demonstrates a serious security breach and aims to raise the awareness of the public that the passcodes for authorizing transactions in mobile payments can be leaked by exploiting the embedded sensors in wearable devices (e.g., smartwatches). We present a passcode inference system, which examines to what extent the user's PIN during mobile payment could be revealed from a single wrist-worn wearable device under different input scenarios involving either two hands or a single hand. Extensive experiments with 15 volunteers demonstrate that an adversary is able to recover a user's PIN with high success rate within 5 tries under various input scenarios.

This paper presents a photoplethysmography (PPG)-based continuous user authentication (CA) system, which especially leverages the PPG sensors in wrist-worn wearable devices to identify users. We explore the uniqueness of the human cardiac system captured by the PPG sensing technology. Existing CA systems require either the dedicated sensing hardware or specific gestures, whereas our system does not require any users' interactions but only the wearable device, which has already been pervasively equipped with PPG sensors. Notably, we design a robust motion artifacts (MA) removal method to mitigate the impact of MA from wrist movements. Additionally, we explore the characteristic fiducial features from PPG measurements to efficiently distinguish the human cardiac system. Furthermore, we develop a cardiac-based classifier for user identification using the Gradient Boosting Tree (GBT). Experiments with the prototype of the wrist-worn PPG sensing platform and 10 participants in different scenarios demonstrate that our system can effectively remove MA and achieve a high average authentication success rate over $90%$.

Given the increasing popularity, mobile devices are exploited to enhance active driving safety nowadays. Among all safety services provided for vehicles, tracking the rotation angle of steering wheel in real time can monitor the vehicles' dynamics and drivers' behaviors at the same time. In this paper, we propose a steering tracking system, SteerTrack, which tracks the rotation angle of steering wheel in real time leveraging audio devices on smartphones. SteerTrack seeks a device-free approach for steering tracking without requiring installation of specialized sensors on steering wheels nor asking drivers to wear sensors on their wrists. Since the steering wheel is operated by a driver's hands, the rotation angle of steering wheel can be tracked based on movements of the driver's hands. SteerTrack first builds an acoustic signal field inside of a vehicle and then analyzes the echoes reflected from the driver's hands with relative correlation coefficient(RCC) and reference frame to track the movement trajectory of hands under different steering maneuvers. Given the tracked movement trajectory, SteerTrack further develops a geometrical transformation-based method for estimating the rotation angle of steering wheel in 3D driving environments by projecting the steering wheel to a 2D ellipse. Through extensive experiments in real driving environments with 5 volunteers for several weeks, SteerTrack can achieve an average error of 4.61 degree for estimating the rotation angle of steering wheel.

Tracking human sleeping postures and vital signs of breathing and heart rates during sleep is important as it can help to assess the general physical health of a person and provide useful clues for diagnosing possible diseases. Traditional approaches (e.g., polysomnography) are limited to clinic usage. Recent radio frequency-based approaches require specialized devices or dedicated wireless sensors and are only able to track breathing rate. In this paper, we propose to track the vital signs of both breathing rate and heart rate during sleep by using off-the-shelf WiFi without any wearable or dedicated devices. Our system reuses existing WiFi network and exploits the fine-grained channel information to capture the minute movements caused by breathing and heart beats. Our system thus has the potential to be widely deployed and perform continuous long-term monitoring. The developed algorithm makes use of the channel information in both time and frequency domain to estimate breathing and heart rates, and it works well when either individual or two persons are in bed. Our extensive experiments demonstrate that our system can accurately capture vital signs during sleep under realistic settings, and achieve comparable or even better performance comparing to traditional and existing approaches, which is a strong indication of providing noninvasive, continuous fine-grained vital signs monitoring without any additional cost.

User authentication is the critical first step in detecting identity-based attacks and preventing subsequent malicious attacks. However, the increasingly dynamic mobile environments make it harderto always apply cryptographic-based methods for user authentication due to their infrastructural and key management overhead. Exploiting non-cryptographic based techniques grounded on physical layer properties to perform user authentication appears promising. In this work, the use of channel state information (CSI), which is available from off-the-shelf WiFi devices, to perform fine-grained user authentication is explored. Particularly, a user-authentication framework that can work with both stationary and mobile users is proposed. When the user is stationary, the proposed framework builds a user profile for user authentication that is resilient to the presence of a spoofer. The proposed machine learning based user-authentication techniques can distinguish between two users even when they possess similar signal fingerprints and detect the existence of a spoofer. When the user is mobile, it is proposed to detect the presence of a spoofer by examining the temporal correlation of CSI measurements. Both office building and apartment environments show that the proposed framework can filter out signal outliers and achieve higher authentication accuracy compared with existing approaches using received signal strength (RSS).

The proliferation of wearable devices, e.g., smartwatches and activity trackers, with embedded sensors has already shown its great potential on monitoring and inferring human daily activities. This paper reveals a serious security breach of wearable devices in the context of divulging secret information (i.e., key entries) while people are accessing key-based security systems. Existing methods of obtaining such secret information rely on installations of dedicated hardware (e.g., video camera or fake keypad), or training with labeled data from body sensors, which restrict use cases in practical adversary scenarios. In this work, we show that a wearable device can be exploited to discriminate mm-level distances and directions of the user's fine-grained hand movements, which enable attackers to reproduce the trajectories of the user's hand and further to recover the secret key entries. In particular, our system confirms the possibility of using embedded sensors in wearable devices, i.e., accelerometers, gyroscopes, and magnetometers, to derive the moving distance of the user's hand between consecutive key entries regardless of the pose of the hand. Our Backward PIN-Sequence Inference algorithm exploits the inherent physical constraints between key entries to infer the complete user key entry sequence. Extensive experiments are conducted with over 7,000 key entry traces collected from 20 adults for key-based security systems (i.e., ATM keypads and regular keyboards) through testing on different kinds of wearables. Results demonstrate that such a technique can achieve 80 percent accuracy with only one try and more than 90 percent accuracy with three tries. Moreover, the performance of our system is consistently good even under low sampling rate and when inferring long PIN sequences. To the best of our knowledge, this is the first technique that reveals personal PINs leveraging wearable devices without the need for labeled training data and contextual information.

Real-time driving behavior monitoring is a corner stone to improve driving safety. Most of the existing studies on driving behavior monitoring using smartphones only provide detection results after an abnormal driving behavior is finished, not sufficient for driver alerting and avoiding car accidents. In this paper, we leverage built-in audio devices on smartphones to realize early recognition of inattentive driving events including Fetching Forward, Picking up Drops, Turning Back, and Eating or Drinking. Through empirical studies of driving traces collected in real driving environments, we find that each type of inattentive driving event exhibits unique patterns on Doppler profiles of audio signals. This enables us to develop an Early Recognition system, ER, which can recognize inattentive driving events at an early stage and alert drivers timely. ER employs machine learning methods to first generate binary classifiers for every pair of inattentive driving events, and then develops a modified vote mechanism to form a multi-classifier for all four types of inattentive driving events, for atypical inattentive driving events along with other driving behaviors. It next turns the multi-classifier into a gradient model forestto achieve early recognition of inattentive driving. Through extensive experiments with eight volunteers driving for about two months, ER can achieve an average total accuracy of 94.80 percent for inattentive driving recognition and recognize over 80 percent inattentive driving events before the event is 50 percent finished.

The goal of this work is to enable user authentication via finger inputs on ubiquitous surfaces leveraging low-cost physical vibration. We propose VibWrite that extends finger-input authentication beyond touch screens to any solid surface for smart access systems (e.g., access to apartments, vehicles or smart appliances). It integrates passcode, behavioral and physiological characteristics, and surface dependency together to provide a low-cost, tangible and enhanced security solution. VibWrite builds upon a touch sensing technique with vibration signals that can operate on surfaces constructed from a broad range of materials. It is significantly different from traditional password-based approaches, which only authenticate the password itself rather than the legitimate user, and the behavioral biometrics-based solutions, which usually involve specific or expensive hardware (e.g., touch screen or fingerprint reader), incurring privacy concerns and suffering from smudge attacks. VibWrite is based on new algorithms to discriminate fine-grained finger inputs and supports three independent passcode secrets including PIN number, lock pattern, and simple gestures by extracting unique features in the frequency domain to capture both behavioral and physiological characteristics such as contacting area, touching force, and etc. VibWrite is implemented using a single pair of low-cost vibration motor and receiver that can be easily attached to any surface (e.g., a door panel, a desk or an appliance). Our extensive experiments demonstrate that VibWrite can authenticate users with high accuracy (e.g., over 95% within two trials), low false positive rate (e.g., less 3%) and is robust to various types of attacks.

While the mobile users enjoy the anytime anywhere Internet access by connecting their mobile devices through Wi-Fi services, the increasing deployment of access points (APs) have raised a number of privacy concerns. This paper explores the potential of smartphone privacy leakage caused by surrounding APs. In particular, we study to what extent the users' personal information such as social relationships and demographics could be revealed leveraging simple signal information from APs without examining the Wi-Fi traffic. Our approach utilizes users' activities at daily visited places derived from the surrounding APs to infer users' social interactions and individual behaviors. Furthermore, we develop two new mechanisms: the Closeness-based Social Relationships Inference algorithm captures how closely people interact with each other by evaluating their physical closeness and derives fine-grained social relationships, whereas the Behavior-based Demographics Inference method differentiates various individual behaviors via the extracted activity features (e.g., activeness and time slots) at each daily place to reveal users' demographics. Extensive experiments conducted with 21 participants' real daily life including 257 different places in three cities over a 6-month period demonstrate that the simple signal information from surrounding APs have a high potential to reveal people's social relationships and infer demographics with an over 90% accuracy when using our approach.

Acknowledging the powerful sensors on wearables and smartphones enabling various applications to improve users' life styles and qualities (e.g., sleep monitoring and running rhythm tracking), this paper takes one step forward developing FitCoach, a virtual fitness coach leveraging users' wearable mobile devices (including wrist-worn wearables and arm-mounted smartphones) to assess dynamic postures (movement patterns & positions) in workouts. FitCoach aims to help the user to achieve effective workout and prevent injury by dynamically depicting the short-term and long-term picture of a user's workout based on various sensors in wearable mobile devices. In particular, FitCoach recognizes different types of exercises and interprets fine-grained fitness data (i.e., motion strength and speed) to an easy-to-understand exercise review score, which provides a comprehensive workout performance evaluation and recommendation. FitCoach has the ability to align the sensor readings from wearable devices to the human coordinate system, ensuring the accuracy and robustness of the system. Extensive experiments with over 5000 repetitions of 12 types of exercises involve 12 participants doing both anaerobic and aerobic exercises in indoors as well as outdoors. Our results demonstrate that FitCoach can provide meaningful review and recommendations to users by accurately measure their workout performance and achieve 93% accuracy for workout analysis.

Real-time driving behavior monitoring is a corner stone to improve driving safety. Most of the existing studies on driving behavior monitoring using smartphones only provide detection results after an abnormal driving behavior is finished, not sufficient for driver alert and avoiding car accidents. In this paper, we leverage existing audio devices on smartphones to realize early recognition of inattentive driving events including Fetching Forward, Picking up Drops, Turning Back and Eating or Drinking. Through empirical studies of driving traces collected in real driving environments, we find that each type of inattentive driving event exhibits unique patterns on Doppler profiles of audio signals. This enables us to develop an Early Recognition system, ER, which can recognize inattentive driving events at an early stage and alert drivers timely. ER employs machine learning methods to first generate binary classifiers for every pair of inattentive driving events, and then develops a modified vote mechanism to form a multi-classifier for all inattentive driving events along with other driving behaviors. It next turns the multi-classifier into a gradient model forest to achieve early recognition of inattentive driving. Through extensive experiments with 8 volunteers driving for about half a year, ER can achieve an average total accuracy of 94.80% for inattentive driving recognition and recognize over 80% inattentive driving events before the event is 50% finished.

Dancing is always challenging especially for beginners who may lack sense of rhythm. Salsa, as a popular style of dancing, is even harder to learn due to its unique overlapped rhythmic patterns made by different Latin instruments (e.g., Clave sticks, Conga drums, Timbale drums) together. In order to dance in synchronization with the Salsa beats, the beginners always need prompts (e.g., beat counting voice) to remind them of the beat timing. The traditional way to generate the Salsa music with beat counting voice prompts requires professional dancers or musicians to count Salsa beats manually, which is only possible in dance studios. Additionally, the existing music beat tracking solutions cannot well capture the Salsa beats due to its intricacy of rhythms. In this work, we propose a mobile device enabled beat counting system, SalsaAsst, which can perform rhythm deciphering and fine-grained Salsa beat tracking to assist Salsa dancers with beat counting voice/vibration prompts. The proposed system can be used conveniently in many scenarios, which can not only help Salsa beginners make accelerated learning progress during practice at home but also significantly reduce professional dancers' errors during their live performance. The developed Salsa beat counting algorithm has the capability to track beats accurately in both real-time and offline manners. Our extensive tests using 40 Salsa songs under 8 evaluation metrics demonstrate that SalsaAsst can accurately track the beats of Salsa music and achieve much better performance comparing to existing beat tracking approaches.

Real-time tracking of subway riding will provide great convenience to millions of commuters in metropolitan areas. Traditional approaches using timetables need continuous attentions from the subway riders and are limited to the poor accuracy of estimating the travel time. Recent approaches using mobile devices rely on GSM and WiFi, which are not always available underground. In this work, we present SubTrack, utilizing sensors on mobile devices to provide automatic tracking of subway riding in real time. The real-time automatic tracking covers three major aspects of a passenger: detection of entering a station, tracking the passenger's position, and estimating the arrival time of subway stops. In particular, SubTrack employs the cell ID to first detect a passenger entering a station and exploits inertial sensors on the passenger's mobile device to track the train ride. Our algorithm takes the advantages of the unique vibrations in acceleration and typical moving patterns of the train to estimate the train's velocity and the corresponding position, and further predict the arrival time in real time. Our extensive experiments in two cities in China and USA respectively demonstrate that our system can accurately track the position of subway riders, predict the arrival time and push the arrival notification in a timely manner.

User authentication is a critical process in both corporate and home environments due to the ever-growing security and privacy concerns. With the advancement of smart cities and home environments, the concept of user authentication is evolved with a broader implication by not only preventing unauthorized users from accessing confidential information but also providing the opportunities for customized services corresponding to a specific user. Traditional approaches of user authentication either require specialized device installation or inconvenient wearable sensor attachment. This paper supports the extended concept of user authentication with a device-free approach by leveraging the prevalent WiFi signals made available by IoT devices, such as smart refrigerator, smart TV and thermostat, etc. The proposed system utilizes the WiFi signals to capture unique human physiological and behavioral characteristics inherited from their daily activities, including both walking and stationary ones. Particularly, we extract representative features from channel state information (CSI) measurements of WiFi signals, and develop a deep learning based user authentication scheme to accurately identify each individual user. Extensive experiments in two typical indoor environments, a university office and an apartment, are conducted to demonstrate the effectiveness of the proposed authentication system. In particular, our system can achieve over 94% and 91% authentication accuracy with 11 subjects through walking and stationary activities, respectively.

Advanced driver assistance systems and, in particular automated driving offers an unprecedented opportunity to transform the safety, efficiency, and comfort of road travel. Developing such safety technologies requires an understanding of not just common highway and city traffic situations but also a plethora of widely different unusual events (e.g., object on the road way and pedestrian crossing highway, etc.). While each such event may be rare, in aggregate they represent a significant risk that technology must address to develop truly dependable automated driving and traffic safety technologies. By developing technology to scale road data acquisition to a large number of vehicles, this paper introduces a low-cost yet reliable solution, BigRoad, that can derive internal driver inputs (i.e., steering wheel angles, driving speed and acceleration) and external perceptions of road environments (i.e., road conditions and front-view video) using a smartphone and an IMU mounted in a vehicle. We evaluate the accuracy of collected internal and external data using over 140 real-driving trips collected in a 3-month time period. Results show that BigRoad can accurately estimate the steering wheel angle with 0.69 degree median error, and derive the vehicle speed with 0.65 km/h deviation. The system is also able to determine binary road conditions with 95% accuracy by capturing a small number of brakes. We further validate the usability of BigRoad by pushing the collected video feed and steering wheel angle to a deep neural network steering wheel angle predictor, showing the potential of massive data acquisition for training self-driving system using BigRoad.

VibSense pushes the limits of vibration-based sensing to determine the location of a touch on extended surface areas as well as identify the object touching the surface leveraging a single sensor. Unlike capacitive sensing, it does not require conductive materials and compared to audio sensing it is more robust to acoustic noise. It supports a broad array of applications through either passive or active sensing using only a single sensor. In VibSense's passive sensing, the received vibration signals are determined by the location of the touch impact. This allows location discrimination of touches precise enough to enable emerging applications such as virtual keyboards on ubiquitous surfaces for mobile devices. Moreover, in the active mode, the received vibration signals carry richer information of the touching object's characteristics (e.g., weight, size, location and material). This further enables VibSense to match the signals to the trained profiles and allows it to differentiate personal objects in contact with any surface. VibSense is evaluated extensively in the use cases of localizing touches (i.e., virtual keyboards), object localization and identification. Our experimental results demonstrate that VibSense can achieve high accuracy, over 95%, in all these use cases.

The rapid pace of urbanization and socioeconomic development encourage people to spend more time together and therefore monitoring of human dynamics is of great importance, especially for facilities of elder care and involving multiple activities. Traditional approaches are limited due to their high deployment costs and privacy concerns (e.g., camera-based surveillance or sensor-attachment-based solutions). In this work, we propose to provide a fine-grained comprehensive view of human dynamics using existing WiFi infrastructures often available in many indoor venues. Our approach is low-cost and device-free, which does not require any active human participation. Our system aims to provide smart human dynamics monitoring through participant number estimation, human density estimation and walking speed and direction derivation. A semi-supervised learning approach leveraging the non-linear regression model is developed to significantly reduce training efforts and accommodate different monitoring environments. We further derive participant number and density estimation based on the statistical distribution of Channel State Information (CSI) measurements. In addition, people's walking speed and direction are estimated by using a frequency-based mechanism. Extensive experiments over 12 months demonstrate that our system can perform fine-grained effective human dynamic monitoring with over 90% accuracy in estimating participants number, density, and walking speed and direction at various indoor environments.

A key component of a smart grid is its ability to collect useful information from a power grid for enabling control centers to estimate the current states of the power grid. Such information can be delivered to the control centers via wireless or wired networks. It is envisioned that wireless technology will be widely used for local-area communication subsystems in the smart grid (e.g., in distribution networks). However, various attacks with serious impact can be launched in wireless networks such as channel jamming attacks and denial-of-service attacks. In particular, jamming attacks can cause significant damages to power grids, e.g., delayed delivery of time-critical messages can prevent control centers from properly controlling the outputs of generators to match load demands. In this paper, a communication subsystem with enhanced self-healing capability in the presence of jamming is designed via intelligent local controller switching while integrating a retransmission mechanism. The proposed framework allows sufficient readings from smart meters to be continuously collected by various local controllers to estimate the states of a power grid under various attack scenarios. The jamming probability is also analyzed considering the impact of jammer power and shadowing effects. In addition, guidelines on optimal placement of local controllers to ensure effective switching of smart meters under jamming are provided. Via theoretical, experimental and simulation studies, it is demonstrated that our proposed system is effective in maintaining communications between smart meters and local controllers even when multiple jammers are present in the network.

Rogue access point (AP) has emerged as an important security problem in WLANs. However, it is a challenge task to localize the rogue AP with both high accuracy and minimal infrastructure cost. Either expensive professional infrastructure (e.g., multiple wireless sniffers) or additional hardware (e.g., directional antenna) need to be pre-deployed for rogue AP localization with high cost. Moreover, existing methods using Received Signal Strength (RSS) result in a large error as RSS is suffered from the multipath and shadowing effects in complex wireless environment. In this work, we exploit the channel state information (CSI), which is readily available from commercial Wi-Fi devices, to locate the rogue AP with high accuracy. We use only a single off-the-shelf Wi-Fi device for rogue AP localization which involves minimal infrastructure requirement. Our proposed rogue AP localization framework consists of two components: direction determination and position estimation. The direction determination can be carried out by using the human blocking effect on the CSI amplitude or phase. The multiple antennas on the Wi-Fi devices can be further utilized to enhance the rogue AP direction estimation. Given the estimated direction, two schemes are proposed to pinpoint the position of the rogue AP: determining directions at multiple locations grounded on triangulation and walking towards the rogue AP with direction adjustment. Results from extensive experiments in both indoor and outdoor environments show that our framework can achieve more practical and accurate rogue AP localization when comparing with the existing RSS-based approach.

With the rapid development of information technology, mobile devices have exhibited increasing popularity in recent years. To support the anytime-anywhere service model of mobile devices, one important problem related to the screen display arises when using these devices (e.g., smartphone and tablet) under various lighting conditions. On one hand, it is hard for users to see the display clearly under strong lighting conditions (e.g., sunlight). On the other hand, the screen appears dazzling under weak lighting conditions. This problem related to the mobile device display can significantly degrade user experience and undermine the successful deployment of the anytime-anywhere mobile service model. Existing solutions mainly focus on the automatic adjustment of brightness level under different light conditions. We show that merely utilizing brightness level to solve the display problem is not enough to maintain the user experience under both strong and weak lighting scenarios through experimenting with over 200 volunteers. In this work, we take a different approach by investigating automatic color scheme adjustment to improve user experience. We find that Readability, Comfort level, and Similarityare major factors that contribute to user experience. In recognizing these problems, we propose a system, ColorVert, which utilizes the DKL color space to adaptively transform color schemes by sensing ambient light to improve user experience under various lighting scenarios. Our experimental evaluation with over 200 precipitants and various mobile devices demonstrates that ColorVert is more effective in both maintaining as well as improving user experience compared with the existing automatic brightness adjustment system.

Real-time abnormal driving behaviors monitoring is a corner stone to improving driving safety. Existing works on driving behaviors monitoring using smartphones only provide a coarse-grained result, i.e., distinguishing abnormal driving behaviors from normal ones. To improve drivers’ awareness of their driving habits so as to prevent potential car accidents, we need to consider a fine-grained monitoring approach, which not only detects abnormal driving behaviors but also identifies specific types of abnormal driving behaviors, i.e.,Weaving,Swerving,Sideslipping,Fast U-turn,Turning with a wide radius, andSudden braking. Through empirical studies of the 6-month driving traces collected from real driving environments, we find that all of the six types of driving behaviors have their unique patterns on acceleration and orientation. Recognizing this observation, we further propose a fine-grained abnormalDriving behaviorDetection and iDentification system,$D^{3}$, to perform real-time high-accurate abnormal driving behaviors monitoring using smartphone sensors. We extract effective features to capture the patterns of abnormal driving behaviors. After that, two machine learning methods,Support Vector Machine(SVM) andNeuron Networks(NN), are employed, respectively, to train the features and output a classifier model which conducts fine-grained abnormal driving behaviors detection and identification. From results of extensive experiments with 20 volunteers driving for another four months in real driving environments, we show that$D^{3}$achieves an average total accuracy of 95.36 percent with SVM classifier model, and 96.88 percent with NN classifier model.

The proliferation of wearable devices, e.g., smartwatches and activity trackers, with embedded sensors has already shown its great potential on monitoring and inferring human daily activities. This paper reveals a serious security breach of wearable devices in the context of divulging secret information (i.e., key entries) while people accessing key-based security systems. Existing methods of obtaining such secret information relies on installations of dedicated hardware (e.g., video camera or fake keypad), or training with labeled data from body sensors, which restrict use cases in practical adversary scenarios. In this work, we show that a wearable device can be exploited to discriminate mm-level distances and directions of the user's fine-grained hand movements, which enable attackers to reproduce the trajectories of the user's hand and further to recover the secret key entries. In particular, our system confirms the possibility of using embedded sensors in wearable devices, i.e., accelerometers, gyroscopes, and magnetometers, to derive the moving distance of the user's hand between consecutive key entries regardless of the pose of the hand. Our Backward PIN-Sequence Inference algorithm exploits the inherent physical constraints between key entries to infer the complete user key entry sequence. Extensive experiments are conducted with over 5000 key entry traces collected from 20 adults for key-based security systems (i.e. ATM keypads and regular keyboards) through testing on different kinds of wearables. Results demonstrate that such a technique can achieve 80% accuracy with only one try and more than 90% accuracy with three tries, which to our knowledge, is the first technique that reveals personal PINs leveraging wearable devices without the need for labeled training data and contextual information.

Affordable cloud computing technologies allow users to efficiently store, and manage their Personal Health Records (PHRs) and share with their caregivers or physicians. This in turn improves the quality of healthcare services, and lower health care cost. However, serious security and privacy concerns emerge because people upload their personal information and PHRs to the public cloud. Data encryption provides privacy protection of medical information but it is challenging to utilize encrypted data. In this paper, we present a privacy-preserving disease treatment, complication prediction scheme (PDTCPS), which allows authorized users to conduct searches for disease diagnosis, personalized treatments, and prediction of potential complications. $PDTCPS$ uses a tree-based structure to boost search efficiency, a wildcard approach to support fuzzy keyword search, and a Bloom-filter to improve search accuracy and storage efficiency. In addition, our design also allows health care providers and the public cloud to collectively generate aggregated training models for disease diagnosis, personalized treatments and complications prediction. Moreover, our design provides query unlinkability and hides both search & access patterns. Finally, our evaluation results using two UCI datasets show that our scheme is more efficient and accurate than two existing schemes.

Voice authentication is drawing increasing attention and becomes an attractive alternative to passwords for mobile authentication. Recent advances in mobile technology further accelerate the adoption of voice biometrics in an array of diverse mobile applications. However, recent studies show that voice authentication is vulnerable to replay attacks, where an adversary can spoof a voice authentication system using a pre-recorded voice sample collected from the victim. In this paper, we propose VoiceLive, a practical liveness detection system for voice authentication on smartphones. VoiceLive detects a live user by leveraging the user's unique vocal system and the stereo recording of smartphones. In particular, with the phone closely placed to a user's mouth, it captures time-difference-of-arrival (TDoA) changes in a sequence of phoneme sounds to the two microphones of the phone, and uses such unique TDoA dynamic which doesn't exist under replay attacks for liveness detection. VoiceLive is practical as it doesn't require additional hardware but two-channel stereo recording that is supported by virtually all smartphones. Our experimental evaluation with 12 participants and different types of phones shows that VoiceLive achieves over 99% detection accuracy at around 1% Equal Error Rate (EER). Results also show that VoiceLive is robust to different phone placements and is compatible to different sampling rates and phone models.

Recently, Visible Light Communication (VLC) over a screen-camera channel has drawn considerable attention to unobtrusive design. It overcomes the distractive nature of traditional coded image approaches (e.g., barcodes). Previous unobtrusive methods fall into two categories: 1) utilizing alpha channel, a well known concept in computer graphics, to encode bits into the pixel translucency change with off-the-shelf smart devices; and 2) leveraging the spatial-temporal flicker-fusion property of human vision system with the fast frame rate of modern displays. However, these approaches heavily rely on high-end devices to achieve both unobtrusive and high accuracy screen-camera-based data communication without affecting video-viewing experience. Unlike previous approaches, we propose Uber-in-light, a novel unobtrusive and accurate VLC system, that enables real-time screen-camera communication, applicable to any screen and camera. The proposed system encodes the data as complementary intensity changes over Red, Green, and Blue (RGB) color channels that could be successfully decoded by camera while leaving the human visual perception unaffected. We design a MFSK modulation scheme with dedicated frame synchronization signal embedded in an orthogonal color channel to achieve high throughput. Furthermore, together with the complementary color intensity, an enhanced MUSIC-based demodulation scheme is developed to ensure highly accurate data transmission. Our user experience experiments confirmed the effectiveness of delivering unobtrusive data across different types of video content and resolutions. Extensive real-time performance evaluations are conducted using our prototype implementation to demonstrate the efficiency and reliability of the proposed system under diverse wireless environments.

Given the increasing popularity of wearable devices, this paper explores the potential to use wearables for steering and driver tracking. Such capability would enable novel classes of mobile safety applications without relying on information or sensors in the vehicle. In particular, we study how wrist-mounted inertial sensors, such as those in smart watches and fitness trackers, can track steering wheel usage and angle. In particular, tracking steering wheel usage and turning angle provide fundamental techniques to improve driving detection, enhance vehicle motion tracking by mobile devices and help identify unsafe driving. The approach relies on motion features that allow distinguishing steering from other confounding hand movements. Once steering wheel usage is detected, it further uses wrist rotation measurements to infer steering wheel turning angles. Our on-road experiments show that the technique is 99% accurate in detecting steering wheel usage and can estimate turning angles with an average error within 3.4 degrees.

Acknowledging the powerful sensors on wearable mobile devices enabling various applications to improve users' life styles and qualities, this paper takes one step forward developing a automatic personal fitness assistance through wearable mobile devices to assess dynamic postures in workouts. In particular, our system recognizes different types of exercises and interprets fine-grained fitness data to an easy-to-understand exercise review score. The system has the ability to align the sensor readings from wearable devices to the earth coordinate system, ensuring the accuracy and robustness of the system. Experiments with 12 types of exercises involve multiple participants doing both anaerobic and aerobic exercises in indoors as well as outdoors. Our results demonstrate that the proposed system can provide meaningful review and recommendations to users by accurately measure their workout performance and achieve 93% accuracy for workout analysis.

Recent advancements in audio recording on mobile devices have improved audio localization capabilities using phones. Previous research has shown that millimeter level accuracy is capable using an off-the-shelf smartphone. This work demonstrates that such valuable resources in smartphones are readily available for developing consumer applications. We develop a smartphone application called HearHere that utilizes geometric features of sound to categorize a tapping sound within a grid. The geometric features are based on the Time Difference of Arrival (TDoA) of the sound between two microphones. By using this classification, any off the shelf phone with stereo recording capability can be made into a music producing device. The application consists of three major components: system calibration, data collection, and audio localization engine based on TDoA. We demonstrate that we can accurately map the tapping sound six region on various solid surface and turn it into a MIDI controller with six different types of instruments.

This work explores vibration-based sensing to determine the location of a touch on extended surface areas as well as identify the object touching the surface leveraging a single sensor. It supports a broad array of applications through either passive or active sensing using only a single sensor. In the passive sensing, the received vibration signals are determined by the location of the touch impact. This allows location discrimination of touches precise enough to enable emerging applications such as virtual keyboards on ubiquitous surfaces for mobile devices. Moreover, in the active mode, the received vibration signals carry richer information of the touching object's characteristics (e.g., weight, size, location and material). This further enables our work to match the signals to the trained profiles and allows it to differentiate personal objects in contact with any surface. We evaluated extensively in the use cases of localizing touches (i.e., virtual keyboards), object localization and identification. Our experimental results demonstrate that the proposed vibration-based solution can achieve high accuracy, over 95%, in all these use cases.

VibKeyboard could accurately determine the location of a keystroke on extended surface areas leveraging a single vibration sensor. Unlike capacitive sensing, it does not require conductive materials and compared to audio sensing it is more robust to acoustic noise. In VibKeyboard, the received vibration signals are determined by the location of the touch impact. This allows location discrimination of touches precise enough to enable emerging applications such as virtual keyboards on ubiquitous surfaces for mobile devices. VibKeyboard seeks to extract unique features in frequency domain embedded in the vibration signal attenuation and interference and perform fine grained localization. Our experimental results demonstrate that VibKeyboard could accurately recognize keystrokes from close-by keys on a nearby virtual keyboard.

In this work, we propose the first PIN number based authentication system, which can be deployed on ubiquitous surfaces, leveraging physical vibration signals. The proposed system aims to integrate PIN number, behavioral and physiological characteristics together to provide enhanced security. Different from the existing password-based approaches, the proposed system builds upon a touch sensing technique using vibration signals that can operate on any solid surface. In this poster, we explore the feasibility of using vibration signals for ubiquitous user authentication and develop algorithms that identify fine-grained finger inputs with different password secrets (e.g., PIN sequences). We build a prototype using a vibration transceiver that can be attached to any surface (e.g., a door or a desk) easily. Our experiments in office environments with multiple users demonstrate that we can achieve high authentication accuracy with a low false negative rate.

In this paper, we address the problem of how to make the wireless service provider (WSP) earn profits in a wireless video broadcast network with consideration of advertisement insertion. At the beginning, this study examines the profit components by analyzing traffic provision and advertisement insertion. This study considers using two components for profit maximization-one is the function for allocating video rates, and the other is the function for inserting advertisement duration. The maximum achievable profit depends on joint optimization of optimal video-rate vectors and advertisement-duration vectors, which are usually computationally intensive. To resolve such a complexity problem, this work also proposes an effective algorithm for joint optimization. First, the overall profit is formulated as the solution of four local optimization problems through horizontal and vertical decomposition. Second, a theoretic polymatroidal framework is introduced in our work for optimization as this framework is proved effective in profit maximization of multiuser systems. Third, this study shows that the overall profit can be maximized by finding the optimal profit points on the boundary of the rate and duration regions. As a result, the optimum points and the total profit can be obtained through a hierarchical greedy algorithm. Experimental results demonstrate that the proposed method is capable of making maximum profits for WSPs in a wide range of broadcasting rates.

This paper utilizes smartphone sensing of vehicle dynamics to determine driver phone use, which can facilitate many traffic safety applications. Our system uses embedded sensors in smartphones, i.e., accelerometers and gyroscopes, to capture differences in centripetal acceleration due to vehicle dynamics. These differences combined with angular speed can determine whether the phone is on the left or right side of the vehicle. Our low infrastructure approach is flexible with different turn sizes and driving speeds. Extensive experiments conducted with two vehicles in two different cities demonstrate that our system is robust to real driving environments. Despite noisy sensor readings from smartphones, our approach can achieve a classification accuracy of over 90 percent with a false positive rate of a few percent. We also find that by combining sensing results in a few turns, we can achieve better accuracy (e.g., 95 percent) with a lower false positive rate. In addition, we seek to exploit the electromagnetic field measurement inside a vehicle to complement vehicle dynamics for driver phone sensing under the scenarios when little vehicle dynamics is present, for example, driving straight on highways or standing at roadsides.

Many smartphone apps routinely gather various private user data and send them to advertisers. Despite recent study on protection mechanisms and analysis on apps' behavior, the understanding about the consequences of such privacy losses remains limited. In this paper we investigate how much an advertiser can infer about users' social and community relationships by combining data from multiple applications and across many users. After one month's user study involving about 200 most popular Android apps, we find that an advertiser can infer 90% of the social relationships. We further propose a privacy leakage inference framework and use real mobility traces and Foursquare data to quantify the consequences of privacy leakage. We find that achieving 90% inference accuracy of the social and community relationships requires merely 3 weeks' user data. The discoveries underscore the importance of early adoption of privacy protection mechanisms.

Sleep monitoring has drawn increasingly attention as the quality and quantity of the sleep are important to maintain a person's health and well-being. For example, inadequate and irregular sleep are usually associated with serious health problems such as fatigue, depression and cardiovascular disease. Traditional sleep monitoring systems, such as PSG, involve wearable sensors with professional installations, and thus are limited to clinical usage. Recent work in using smartphone sensors for sleep monitoring can detect several events related to sleep, such as body movement, cough and snore. Such coarse-grained sleep monitoring however is unable to detect the breathing rate which is an important vital sign and health indicator. This work presents a fine-grained sleep monitoring system which is capable of detecting the breathing rate by leveraging smartphones. Our system exploits the readily available smartphone earphone placed close to the user to reliably capture the human breathing sound. Given the captured acoustic sound, our system performs noise reduction to remove environmental noise and then identifies the breathing rate based on the signal envelope detection. Our system can further detect detailed sleep events including snore, cough, turn over and get up based on the acoustic features extracted from the acoustic sound. Our experimental evaluation of six subjects over six months time period demonstrates that the breathing rate monitoring and sleep events detection are highly accurate and robust under various environments. By combining breathing rate and sleep events, our system can provide continuous and noninvasive fine-grained sleep monitoring for healthcare related applications, such as sleep apnea monitoring as evidenced by our experimental study.

This paper explores the limits of audio ranging on mobile devices in the context of a keystroke snooping scenario. Acoustic keystroke snooping is challenging because it requires distinguishing and labeling sounds generated by tens of keys in very close proximity. Existing work on acoustic keystroke recognition relies on training with labeled data, linguistic context, or multiple phones placed around a keyboard --- requirements that limit usefulness in an adversarial context. In this work, we show that mobile audio hardware advances can be exploited to discriminate mm-level position differences and that this makes it feasible to locate the origin of keystrokes from only a single phone behind the keyboard. The technique clusters keystrokes using time-difference of arrival measurements as well as acoustic features to identify multiple strokes of the same key. It then computes the origin of these sounds precise enough to identify and label each key. By locating keystrokes this technique avoids the need for labeled training data or linguistic context. Experiments with three types of keyboards and off-the-shelf smartphones demonstrate scenarios where our system can recover $94\%$ of keystrokes, which to our knowledge, is the first single-device technique that enables acoustic snooping of passwords.

Tracking human vital signs of breathing and heart rates during sleep is important as it can help to assess the general physical health of a person and provide useful clues for diagnosing possible diseases. Traditional approaches (e.g., Polysomnography (PSG)) are limited to clinic usage. Recent radio frequency (RF) based approaches require specialized devices or dedicated wireless sensors and are only able to track breathing rate. In this work, we propose to track the vital signs of both breathing rate and heart rate during sleep by using off-the-shelf WiFi without any wearable or dedicated devices. Our system re-uses existing WiFi network and exploits the fine-grained channel information to capture the minute movements caused by breathing and heart beats. Our system thus has the potential to be widely deployed and perform continuous long-term monitoring. The developed algorithm makes use of the channel information in both time and frequency domain to estimate breathing and heart rates, and it works well when either individual or two persons are in bed. Our extensive experiments demonstrate that our system can accurately capture vital signs during sleep under realistic settings, and achieve comparable or even better performance comparing to traditional and existing approaches, which is a strong indication of providing non-invasive, continuous fine-grained vital signs monitoring without any additional cost.

Motivated by safety challenges resulting from distracted pedestrians, this paper presents a sensing technology for fine-grained location classification in an urban environment. It seeks to detect the transitions from sidewalk locations to in-street locations, to enable applications such as alerting texting pedestrians when they step into the street. In this work, we use shoe-mounted inertial sensors for location classification based on surface gradient profile and step patterns. This approach is different from existing shoe sensing solutions that focus on dead reckoning and inertial navigation. The shoe sensors relay inertial sensor measurements to a smartphone, which extracts the step pattern and the inclination of the ground a pedestrian is walking on. This allows detecting transitions such as stepping over a curb or walking down sidewalk ramps that lead into the street. We carried out walking trials in metropolitan environments in United States (Manhattan) and Europe (Turin). The results from these experiments show that we can accurately determine transitions between sidewalk and street locations to identify pedestrian risk.

This video is a demonstration of the work discussed in our full paper available in the MobiSys'15 proceedings. The video illustrates a sensing technology for fine-grained location classification in an urban environment, for enhancing pedestrian safety. Our system seeks to detect the transitions from sidewalk locations to in-street locations, to enable applications such as alerting texting pedestrians when they step into the street. Existing positioning technologies are not sufficiently precise to allow distinguishing a position on the sidewalk from a position in the street, as explored in our previous work. To this end, we use shoe-mounted inertial sensors for location classification based on surface gradient profile and step patterns. This approach is different from existing shoe sensing solutions that focus on dead reckoning and inertial navigation. The shoe sensors relay inertial sensor measurements to a smartphone, which extracts the step pattern and the inclination of the ground a pedestrian is walking on. This allows detecting transitions such as stepping over a curb or walking down sidewalk ramps that lead into the street. We carried out walking trials in metropolitan environments in United States (Manhattan) and Europe (Turin). The results from these experiments show that we can accurately determine transitions between sidewalk and street locations to identify pedestrian risk.

Real-time abnormal driving behaviors monitoring is a corner stone to improving driving safety. Existing works on driving behaviors monitoring using smartphones only provide a coarsegrained result, i.e. distinguishing abnormal driving behaviors from normal ones. To improve drivers' awareness of their driving habits so as to prevent potential car accidents, we need to consider a finegrained monitoring approach, which not only detects abnormal driving behaviors but also identifies specific types of abnormal driving behaviors, i.e. Weaving, Swerving, Sideslipping, Fast U-turn, Turning with a wide radius and Sudden braking. Through empirical studies of the 6-month driving traces collected from real driving environments, we find that all of the six types of driving behaviors have their unique patterns on acceleration and orientation. Recognizing this observation, we further propose a finegrained abnormal Driving behavior Detection and iDentification system, D3, to perform real-time high-accurate abnormal driving behaviors monitoring using smartphone sensors. By extracting unique features from readings of smartphones' accelerometer and orientation sensor, we first identify sixteen representative features to capture the patterns of driving behaviors. Then, a machine learning method, Support Vector Machine (SVM), is employed to train the features and output a classifier model which conducts fine-grained identification. From results of extensive experiments with 20 volunteers driving for another 4 months in real driving environments, we show that D3achieves an average total accuracy of 95.36%.

With the rapid development of information technology, mobile devices have exhibited increasing popularity in recent years. To support the anytime-anywhere service model of mobile devices, one important problem related to the screen display arises when using these devices (e.g., smartphone and tablet) under various lighting conditions. On one hand, it is hard for users to see the display clearly under strong lighting conditions (e.g., sunlight). On the other hand, the screen appears dazzling under weak lighting conditions. This problem related to the mobile device display can significantly degrade user experience and undermine the successful deployment of the anytime-anywhere mobile service model. Existing solutions mainly focus on the automatic adjustment of brightness level under different light conditions. We show that merely utilizing brightness level to solve the display problem is not enough to maintain the user experience under both strong and weak lighting scenarios through experimenting with over 200 volunteers. In this work, we take a different approach by investigating automatic color scheme adjustment to improve user experience. We find that Readability, Comfort level and Similarity are major factors that contribute to user experience. In recognizing these problems, we propose a system, ColorVert, which utilizes the DKL color space to adaptively transform color schemes by sensing ambient light to improve user experience under various lighting scenarios. Our experimental evaluation with over 200 precipitants and various mobile devices demonstrates that ColorVert is more effective in both maintaining as well as improving user experience compared with the existing automatic brightness adjustment system.

The rapid deployment of sensing technology in smartphones and the explosion of their usage in people's daily lives provide users with the ability to collectively sense the world. This leads to a growing trend of mobile healthcare systems utilizing sensing data collected from smartphones with/without additional external sensors to analyze and understand people's physical and mental states. However, such healthcare systems are vulnerable to user spoofing, in which an adversary distributes his registered device to other users such that data collected from these users can be claimed as his own to obtain more healthcare benefits and undermine the successful operation of mobile healthcare systems. Existing mitigation approaches either only rely on a secret PIN number (which can not deal with colluded attacks) or require an explicit user action for verification. In this paper, we propose a user verification system leveraging unique gait patterns derived from acceleration readings to detect possible user spoofing in mobile healthcare systems. Our framework exploits the readily available accelerometers embedded within smartphones for user verification. Specifically, our user spoofing mitigation framework (which consists of three components, namely Step Cycle Identification, Step Cycle Interpolation, and Similarity Comparison) is used to extract gait patterns from run-time accelerometer measurements to perform robust user verification under various walking speeds. We show that our framework can be implemented in two ways: user-centric and server-centric, and it is robust to not only random but also mimic attacks. Our extensive experiments using over 3,000 smartphone-based traces with mobile phones placed on different body positions confirm the effectiveness of the proposed framework with users walking at various speeds. This strongly indicates the feasibility of using smartphone based low grade accelerometer to conduct gait recognition and facilitate effective user verification without active user cooperation.

Touch-screen technique has gained the large popularity in human-screen interaction with modern smartphones. Due to the limited size of equipped screens, scrolling operations are indispensable in order to display the content of interest on screen. While power consumption caused by hardware and software installed within smartphones is well studied, the energy cost made by human-screen interaction such as scrolling remains unknown. In this paper, we analyze the impact of scrolling operations to the power consumption of smartphones, finding that the state-of-art strategy of smartphones in responding a scrolling operation is to always use the highest frame rate which arouses huge computation burden and can contribute nearly 50 percent to the total power consumption of smartphones. In recognizing this significance, we further propose a novel system, energy-efficient engine (E3), which automatically tracks the scrolling speed and adaptively adjusts the frame rate according to user preference. The goal of E3is to guarantee the user experience and minimize the energy consumption caused by scrolling at the same time. Extensive experiment results demonstrate the efficiency of E3design. On average, E3can save up to 60 percent of the energy consumed by CPU and 35 percent of the overall energy consumption.

The problem of device-free passive localization aims on locating moving objects which do not carry any localization devices. The rationale of this problem is based on the fact that a moving object can result in the changes of received signal strength (RSS) of the wireless links. Existing studies on this problem usually do not consider the impact of dynamic target speed on device-free passive localization. However, the experiments show that the localization performance degrades substantially when an object is moving at dynamic speed. To meet this challenge, in this paper, we propose an adaptive device-free passive localization framework which has three components to detect target speed change and perform adaptive localization. This framework can be easily adapted for existing device-free localization methods which are based on the detection of signal strength changes. As demonstrated in the experiments, the proposed framework can lead to 50 and 30 percent improvement on median and maximum error respectively over the localization algorithms without considering dynamic moving speeds of the target.

Location-based services (LBSs) have raised serious privacy concerns in the society, due to the possibility of leaking a mobile user's location information in enabling location-dependent services. While existing location-privacy studies are mainly focused on preventing the leakage of a user's location in accessing the LBS server, the possible privacy leakage in the calculation of the user's location, i.e., the localization, has been largely ignored. Such a privacy leakage stems from the fact that a localization algorithm typically takes the location of anchors (reference points for localization) as input, and generates the target's location as output. As such, the location of anchors and target could be leaked to others. An adversary could further utilize the leakage of anchor's locations to attack the localization infrastructure and undermine the accurate estimation of the target's location. To address this issue, in this paper, we study the multi-lateral privacy-preserving localization problem, whereby the location of a target is calculated without the need of revealing anchors' location, and the knowledge of the localization outcome, i.e., the target's location, is strictly limited to the target itself. To fully protect the user's privacy, our study protects not only the user's exact location information (the geo-coordinates), but also any side information that may lead to a coarse estimate of the location. We formulate the problem as a secure least-squared-error (LSE) estimation for an overdetermined linear system and develop three privacy-preserving solutions by leveraging combinations of information-hiding and homomorphic encryption. These solutions provide different levels of protection for location-side information and resilience to node collusion and have the advantage of being able to trade a user's privacy requirements for better computation and communication efficiency. Through numerical results, we verify the significant efficiency improvement of the proposed schemes over existing multiparty secure LSE algorithms.

We study the distributed spectrum sensing in cognitive radio networks. Existing distributed consensus-based fusion algorithms only ensure equal gain combining of local measurements, whose performance may be incomparable to various centralized soft combining schemes. Motivated by this fact, we consider practical channel conditions and link failures, and develop new weighted soft measurement combining without a centralized fusion center. Following the measurement by its energy detector, each secondary user exchanges its own measurement statistics with its local one-hop neighbors, and chooses the information exchanging rate according to the measurement channel condition, e.g., the signal-to-noise ratio (SNR). We rigorously prove the convergence of the new consensus algorithm, and show all secondary users hold the same global decision statistics from the weighted soft measurement combining throughout the network. We also provide distributed optimal weight design under uncorrelated measurement channels. The convergence rate of the consensus iteration is given under the assumption that each communication link has an independent probability to fail, and the upper bound of the iteration number of the$ \epsilon$-convergence is explicitly given as a function of system parameters. Simulation results show significant improvement of the sensing performance compared to existing consensus-based approaches, and the performance of the distributed weighted design is comparable to the centralized weighted combining scheme.

The Global Positioning System (GPS) is widely used in critical infrastructures but is vulnerable to radio frequency (RF) interference. A common source of interference are commercial drivers that use GPS jammers to circumvent vehicle tracking systems. Existing mechanisms to detect and identify such interference emitting vehicles on roadways require a large number of specialized detectors or a manual observation process. In this paper, we design a practical, automated system to facilitate enforcement actions. Our system combines information from roadside monitoring points at key locations along the roadway as well as mobile detectors (e.g., smartphones and other mobile GPS systems). Rather than attempting precise localization at a given time, the system exploits the inherent variation in driving speeds and the resulting diverging trajectories of vehicles to uniquely identify the interfering vehicle. Through our experiments on a local highway with a vehicle transmitting interference in the 900MHz ISM band, we found that the vehicle identification rate of our mechanism is 65% for a single-point setup and 100% for a two-point setup. We performed 200 hours of passive monitoring of GPS L1 band on roadways and found two episodes of real interference. We also demonstrate that our mobile detector-based crowdsourced smartphone profiles are sufficiently consistent in time and space to enable reliable interference detection.

User authentication is the critical first step to detect identity-based attacks and prevent subsequent malicious attacks. However, the increasingly dynamic mobile environments make it harder to always apply the cryptographic-based methods for user authentication due to their infrastructural and key management overhead. Exploiting non-cryptographic based techniques grounded on physical layer properties to perform user authentication appears promising. In this work, we explore to use channel state information (CSI), which is available from off-the-shelf WiFi devices, to conduct fine-grained user authentication. We propose an user-authentication framework that has the capability to build the user profile resilient to the presence of the spoofer. Our machine learning based user-authentication techniques can distinguish two users even when they possess similar signal fingerprints and detect the existence of the spoofer. Our experiments in both office building and apartment environments show that our framework can filter out the signal outliers and achieve higher authentication accuracy compared with existing approaches using received signal strength (RSS).

The rapid deployment of cloud computing provides users with the ability to outsource their data to public cloud for economic savings and flexibility. To protect data privacy, users have to encrypt the data before outsourcing to the cloud, which makes the data utilization, such as data retrieval, a challenging task. It is thus desirable to enable the search service over encrypted cloud data for supporting effective and efficient data retrieval over a large number of data users and documents in the cloud. Existing approaches on encrypted cloud data search either focus on single keyword search or become inefficient when a large amount of documents are present, and thus have little support for the efficient multi-keyword search. In this paper, we propose a light-weight search approach that supports efficient multi-keyword ranked search in cloud computing system. Specifically, we first propose a basic scheme using polynomial function to hide the encrypted keyword and search patterns for efficient multi-keyword ranked search. To enhance the search privacy, we propose a privacy-preserving scheme which utilizes the secure inner product method for protecting the privacy of the searched multi-keywords. We analyze the privacy guarantee of our proposed scheme and conduct extensive experiments based on the real-world dataset. The experiment results demonstrate that our scheme can enable the encrypted multi-keyword ranked search service with high efficiency in cloud computing.

Acquiring instant vehicle speed is desirable and a corner stone to many important vehicular applications. This paper utilizes smartphone sensors to estimate the vehicle speed, especially when GPS is unavailable or inaccurate in urban environments. In particular, we estimate the vehicle speed by integrating the accelerometer's readings over time and find the acceleration errors can lead to large deviations between the estimated speed and the real one. Further analysis shows that the changes of acceleration errors are very small over time which can be corrected at some points, called reference points, where the true vehicle speed is known. Recognizing this observation, we propose an accurate vehicle speed estimation system, SenSpeed, which senses natural driving conditions in urban environments including making turns, stopping and passing through uneven road surfaces, to derive reference points and further eliminates the speed estimation deviations caused by acceleration errors. Extensive experiments demonstrate that SenSpeed is accurate and robust in real driving environments. On average, the real-time speed estimation error on local road is 1.32mph, and the offline speed estimation error is as low as 0.75mph. Whereas the average error of GPS is 3.1mph and 2.8mph respectively.

Location based services (LBSs) have raised serious privacy concerns in the society, due to the possibility of leaking a mobile user's location information in enabling location-dependent services. While existing location-privacy studies are mainly focused on preventing the leakage of user's location in accessing the LBS server, the possible privacy leakage during the localization process has been largely ignored. Such a privacy leakage stems from the fact that a localization algorithm typically takes the location of anchors (i.e., reference points for localization) as input, and generates the target's location as output. As such, the location of anchors, and consequently the target's location, could be leaked to others. An adversary could further utilize the leakage of anchor's locations to attack the localization infrastructure and undermine the accurate estimation of the target's location. To address this issue, in this paper, we study the multi-lateral privacy preserving localization problem, whereby the location of a target is calculated without the need of revealing anchors' location, and the knowledge of the localization outcome is strictly limited to the target itself. To fully protect user's privacy, our study protects not only the user's exact location information (the geo-coordinates), but also any side information that may lead to a coarse estimate of the location. Three privacy-preserving localization solutions are developed by leveraging combinations of information hiding and homomorphic encryption. These solutions provide different levels of protection for location side information and resilience to node collusion, and have the advantage of being able to trade user's privacy requirements for better computation/communication efficiency.

Human indoor localization was previously implemented using wireless sensor networks at the cost of sensing infrastructure deployment. Motivated by high density of smartphones in public spaces, we propose to use a robot-assisted localization system in which the low-cost Kinect sensor and smartphone-based acoustic relative ranging are used to localize moving human targets in indoor environments. An extended Kalman filter based localization algorithm is developed for real-time dynamic position estimation. We present both simulations and real robot-smartphone experiments demonstrating the performance with a localization accuracy of approximately 0.5m.

Sleep monitoring has drawn increasingly attention as the quality and quantity of the sleep are important for maintaining a person's health and well-being. For example, inadequate and irregular sleep are usually associated with serious health problems such as fatigue, depression and cardiovascular disease. Traditional sleep monitoring systems, such as PSG, involve wearable sensors with professional installations, and thus are limited to clinical usage. Recent work in using smartphone sensors for sleep monitoring can detect several events related to sleep, such as body movement, cough and snore. Such coarse-grained sleep monitoring however is unable to detect the breathing rate which is a vital sign and health indicator. This work presents a fine-grained sleep monitoring system which is capable of detecting the breathing rate by leveraging smartphones. Our system exploits the readily available smartphone earphone that placed close to the user to capture the breath sound reliably. Given the captured acoustic signal, our system performs noise reduction to remove environmental noise and then identifies the breathing rate based on the signal envelope detection. Our experimental evaluation of six subjects over six months time period demonstrates that the breathing rate monitoring is highly accurate and robust under various environments. This strongly indicates the feasibility of using the smartphone and its earphone to perform continuous and noninvasive fine-grained sleep monitoring.

Activity monitoring in home environments has become increasingly important and has the potential to support a broad array of applications including elder care, well-being management, and latchkey child safety. Traditional approaches involve wearable sensors and specialized hardware installations. This paper presents device-free location-oriented activity identification at home through the use of existing WiFi access points and WiFi devices (e.g., desktops, thermostats, refrigerators, smartTVs, laptops). Our low-cost system takes advantage of the ever more complex web of WiFi links between such devices and the increasingly fine-grained channel state information that can be extracted from such links. It examines channel features and can uniquely identify both in-place activities and walking movements across a home by comparing them against signal profiles. Signal profiles construction can be semi-supervised and the profiles can be adaptively updated to accommodate the movement of the mobile devices and day-to-day signal calibration. Our experimental evaluation in two apartments of different size demonstrates that our approach can achieve over 96% average true positive rate and less than 1% average false positive rate to distinguish a set of in-place and walking activities with only a single WiFi access point. Our prototype also shows that our system can work with wider signal band (802.11ac) with even higher accuracy.

We investigate using smartphone WiFi signals to track human queues, which are common in many business areas such as retail stores, airports, and theme parks. Real-time monitoring of such queues would enable a wealth of new applications, such as bottleneck analysis, shift assignments, and dynamic workflow scheduling. We take a minimum infrastructure approach and thus utilize a single monitor placed close to the service area along with transmitting phones. Our strategy extracts unique features embedded in signal traces to infer the critical time points when a person reaches the head of the queue and finishes service, and from these inferences we derive a person's waiting and service times. We develop two approaches in our system, one is directly feature-driven and the second uses a simple Bayesian network. Extensive experiments conducted both in the laboratory as well as in two public facilities demonstrate that our system is robust to real-world environments. We show that in spite of noisy signal readings, our methods can measure service and waiting times to within a $10$ second resolution.

This paper investigates the issue of cooperative spectrum sensing with a crowd of low-end personal spectrum sensors (such as smartphones, tablets, and in-vehicle sensors), where the sensing data from crowd sensors that may be unreliable, untrustworthy, or even malicious. Moreover, due to either unexpected equipment failures or malicious behaviors, every crowd sensor could sporadically and randomly contribute with abnormal data, which makes the existing cooperative sensing schemes ineffective. To tackle these challenges, we first propose a generalized modeling approach for sensing data with an arbitrary abnormal component. Under this model, we then analyze the impact of general abnormal data on the performance of the cooperative sensing, by deriving closed-form expressions of the probabilities of global false alarm and global detection. To improve sensing data quality and enhance cooperative sensing performance, we further formulate an optimization problem as stable principal component pursuit, and develop a data cleansing-based robust spectrum sensing algorithm to solve it, where the under-utilization of licensed spectrum bands and the sparsity of nonzero abnormal data are jointly exploited to robustly cleanse out the potential nonzero abnormal data component from the original corrupted sensing data. Extensive simulation results demonstrate that the proposed robust sensing scheme performs well under various abnormal data parameter configurations.

Highly accurate indoor localization of smartphones is critical to enable novel location based features for users and businesses. In this paper, we first conduct an empirical investigation of the suitability of WiFi localization for this purpose. We find that although reasonable accuracy can be achieved, significant errors (e.g., 6 8m) always exist. The root cause is the existence of distinct locations with similar signatures, which is a fundamental limit of pure WiFi-based methods. Inspired by high densities of smartphones in public spaces, we propose a peer assisted localization approach to eliminate such large errors. It obtains accurate acoustic ranging estimates among peer phones, then maps their locations jointly against WiFi signature map subjecting to ranging constraints. We devise techniques for fast acoustic ranging among multiple phones and build a prototype. Experiments show that it can reduce the maximum and 80-percentile errors to as small as 2m and 1m, in time no longer than the original WiFi scanning, with negligible impact on battery lifetime.

Secret key generation among wireless devices using physical layer information of radio channel has been an attractive alternative for ensuring security in mobile environments. Received signal strength (RSS) based secret key extraction gains much attention due to its easy accessibility in wireless infrastructure. However, the problem of using RSS to generate keys among multiple devices to ensure secure group communication in practice remains open. In this work, we propose a framework for collaborative key generation among multiple wireless devices leveraging RSS. To deal with mobile devices not within each other’s communication range, we employ relay nodes to achieve reliable key extraction. To enable secure group communication, two protocols are developed to perform collaborative group key generation via star and chain topologies respectively. We further provide the theoretic analysis on the achievable secrecy rate for both star and chain topologies in the presence of an eavesdropper. Our prototype development using MICAz motes and extensive experiments using fading trend based key extraction demonstrate the feasibility of using RSS for group key generation in both indoor and outdoor environments, and concurrently achieving a lower bit mismatch rate compared to existing studies.

As wireless networks become more pervasive, the amount of the wireless data is rapidly increasing. One of the biggest challenges of wide adoption of distributed data storage is how to store these data securely. In this work, we study the frequency-based attack, a type of attack that is different from previously well-studied ones, that exploits additional adversary knowledge of domain values and/or their exact/approximate frequencies to crack the encrypted data. To cope with frequency-based attacks, the straightforward 1-to-1 substitution encryption functions are not sufficient. We propose a data encryption strategy based on 1-to- n substitution via dividing and emulating techniques to defend against the frequency-based attack, while enabling efficient query evaluation over encrypted data. We further develop two frameworks, incremental collection and clustered collection, which are used to defend against the global frequency-based attack when the knowledge of the global frequency in the network is not available. Built upon our basic encryption schemes, we derive two mechanisms, direct emulating and dual encryption, to handle updates on the data storage for energy-constrained sensor nodes and wireless devices. Our preliminary experiments with sensor nodes and extensive simulation results show that our data encryption strategy can achieve high security guarantee with low overhead.

Jammers can severely disrupt the communications in wireless networks, and jammers' position information allows the defender to actively eliminate the jamming attacks. Thus, in this paper, we aim to design a framework that can localize one or multiple jammers with a high accuracy. Most of existing jammer-localization schemes utilize indirect measurements (e.g., hearing ranges) affected by jamming attacks, which makes it difficult to localize jammers accurately. Instead, we exploit a direct measurement-the strength of jamming signals (JSS). Estimating JSS is challenging as jamming signals may be embedded in other signals. As such, we devise an estimation scheme based on ambient noise floor and validate it with real-world experiments. To further reduce estimation errors, we define an evaluation feedback metric to quantify the estimation errors and formulate jammer localization as a nonlinear optimization problem, whose global optimal solution is close to jammers' true positions. We explore several heuristic search algorithms for approaching the global optimal solution, and our simulation results show that our error-minimizing-based framework achieves better performance than the existing schemes. In addition, our error-minimizing framework can utilize indirect measurements to obtain a better location estimation compared with prior work.

Wireless localization using the received signal strength (RSS) can have tremendous savings over using specialized positioning infrastructures. In this work, we explore improving RSS localization performance in multipath environments by varying the transmitter's signal power and frequency. We first derive and analyze the Cramér-Rao Lower Bound (CRLB) of RSS-based localization based on the frequency dependent path loss propagation model that considers the transmitter's signal power and frequency. The derived CRLB shows the feasibility of improving localization performance by applying frequency and power level selection for RSS-based localization. Using this analysis, we develop two new selection metrics based on the observed standard deviations of RSS as well as residuals. We then show a set of selection methods that attempt to select the combinations of power and frequencies which minimize the localization error in a representative class of localization algorithms. Our simulation results confirm the proposed selection methods can improve the localization accuracy under CRLB. Additionally, using active RFID tags, we experimentally characterize the effect of using multiple signal powers and frequencies on a wide spectrum of RSS-based algorithms. We found that the performance of all the algorithms improves when leveraging on multiple power levels and frequencies, although different algorithms present different sensitivity in terms of localization accuracy under different selection methods.

Mobile wireless devices play important roles in our daily life, e.g., users often use such devices to take pictures and share with friends via opportunistic peer-to-peer links, which however are intermittent in nature, and hence require the store-and-forward feature proposed in Delay Tolerant Networks to provide useful data sharing opportunities. Moreover, mobile devices may not be willing to forward data items to other devices due to the limited resources. Hence, effective data dissemination schemes need to be designed to encourage nodes to collaboratively share data. We propose a Multi-Receiver Incentive-Based Dissemination (MuRIS) scheme that allows nodes to cooperatively deliver information of interest to one another via chosen paths utilizing few transmissions. Our scheme exploits local historical paths and users' interests information maintained by each node. In addition, the charge and rewarding functions incorporated within our scheme stimulate cooperation among nodes such that the nodes have no incentive to launch edge insertion attacks. Furthermore, our charge and rewarding functions are designed such that the chosen delivery paths mimic efficient multicast tree that results in fewer delivery hops. Extensive simulation studies using real human contact-based mobility traces show that our scheme outperforms existing methods in terms of delivery ratio and transmission efficiency.

Data sharing in the cloud, fueled by favorable cloud technology trends, has emerging as a promising pattern in regard to enabling data more accessible to users in a convenient manner. To achieve data sharing, enterprises and customers in increasing numbers keep their data stored into cloud server. In this paper, we focus on seeking a solution that allows secure and effective access to the cloud data. We propose an effective and flexible privacy-preserving data policy, P2E, utilizing ciphertext policy attribute-based encryption (CP-ABE) and combining it with technique of identity-based encryption (IBE). In addition to ensuring strong data sharing security, the policy succeeds in preserving the privacy of cloud users. Security analysis indicates that the proposed policy is security and enforces fine-grained access control and full collusion resistance simultaneously. Furthermore, our performance analysis and experimental results show that P2E is as light as possible.

Securing wireless communication remains challenging in dynamic mobile environments due to the shared nature of wireless medium and lacking of fixed key management infrastructures. Generating secret keys using physical layer information thus has drawn much attention to complement traditional cryptographic-based methods. Although recent work has demonstrated that Received Signal Strength (RSS) based secret key extraction is practical, existing RSS-based key generation techniques are largely limited in the rate they generate secret bits and are mainly applicable to mobile wireless networks. In this paper, we show that exploiting the channel response from multiple Orthogonal Frequency-Division Multiplexing (OFDM) subcarriers can provide fine-grained channel information and achieve higher bit generation rate for both static and mobile cases in real-world scenarios. We further develop a Channel Gain Complement (CGC) assisted secret key extraction scheme to cope with channel non-reciprocity encountered in practice. Our extensive experiments using WiFi networks in both indoor as well as outdoor environments demonstrate that our approach can achieve significantly faster secret bit generation rate at 60 ~ 90bit/packet, and is resilient to malicious attacks identified to be harmful to RSS-based techniques including predictable channel attack and stalking attack.

Device-free passive localization enables locating targets (e.g., intruders or victims) that do not carry any radio devices nor do they actively participate in the wireless localization process. This is because the wireless environments will get affected when people move into the area, which result in the changes of Received Signal Strength (RSS) of the wireless links. In this paper, we first show that the localization performance degrades significantly when people are moving in dynamic speeds. This is because existing studies in device-free passive localization system have an implicit assumption that the target is moving at a constant speed, which is not always true in practical scenarios. To cope with targets moving with dynamic speeds, we propose an adaptive speed change detection framework including three components: speed change detection, determination of time-window size and adaptive localization. Two speed change detection schemes have been developed to capture the changes of moving speed and adjust the time-window size adaptively to facilitate effective localization. We demonstrate that our framework is flexible to work with any device-free localization method using signal strength. Results from the real experiments confirm that our approach has over 30% improvement on both median and max localization error, under dynamically changing speed of the target.

The rapid advancement of smartphones has instigated tremendous data applications for cell phones. Supporting simultaneous voice and data services in a cellular network is not only desirable but also becoming indispensable. However, if the voice and data are serviced through the same antenna (like the 3G UMTS network), a voice call with data sessions requires better radio connection than a voice-only call. In this paper, we systematically study the coordination between the voice and data transmissions in UMTS networks. From analyzing a large carrier's UMTS network recording data, we first identify the most relevant network measurements/features indicating a potential call drop, then propose a drop-call predictor based on AdaBoost. Moreover, we develop an intelligent call management strategy to voluntarily block data sessions when the voice is predicted to be dropped. Our analysis utilizing real service provider's data sets shows that our proposed scheme can not only predict drop calls with a very high accuracy but also achieve the highest user satisfaction compared to the other existing call management strategies.

Cloud storage services enable users to remotely store their data and eliminate excessive local installation of software and hardware. One critical issue is how to enable a secure data collaboration service including data access and update in cloud computing. A data collaboration service is to support the availability and consistency of the shared data among multi-users. In this paper, we propose a secure and efficient data collaboration scheme SECO. In SECO, we employ a two-level hierarchical identity based encryption (HIBE) to guarantee data confidentiality against untrusted cloud. This paper is the first attempt to explore secure cloud data collaboration service that precludes information leakage and enables a one-to-many encryption paradigm, data writing operation and fine-grained access control simultaneously. Security analysis indicates that the SECO enforces fine-grained access control and collusion resistant. Extensive performance analysis and experiment results demonstrate that SECO is highly efficient and low overhead on computation and communication.

We investigate using smartphone WiFi signals to track human queues, which are common in many business areas such as retail stores, airports, and theme parks. Real-time monitoring of such queues would enable a wealth of new applications, such as bottleneck analysis, shift assignments, and dynamic workflow scheduling. We take a minimum infrastructure approach and thus utilize a single monitor placed close to the service area along with transmitting phones. Our strategy extracts unique features embedded in the signal traces to infer the critical time points when a person reaches the head of the queue and finishes service, and from these inferences we derive a person's waiting and service times. We develop a feature driven approach in our system. Extensive experiments conducted both in the laboratory demonstrate that our system is robust to queues with different waiting time. We show that in spite of noisy signal readings, our methods can measure important time periods in queue (e.g., service and waiting times) to within a $10$ second resolution.

This paper utilizes smartphone sensing of vehicle dynamics to determine driver phone use, which can facilitate many traffic safety applications. Our system uses embedded sensors in smartphones, i.e., accelerometers and gyroscopes, to capture differences in centripetal acceleration due to vehicle dynamics. These differences combined with angular speed can determine whether the phone is on the left or right side of the vehicle. Our low infrastructure approach is flexible with different turn sizes and driving speeds. Extensive experiments conducted with two vehicles in two different cities demonstrate that our system is robust to real driving environments. Despite noisy sensor readings from smartphones, our approach can achieve a classification accuracy of over $90\%$ with a false positive rate of a few percent. We also find that by combining sensing results in a few turns, we can achieve better accuracy (e.g., $95\%$) with a lower false positive rate.

The rapid deployment of sensing technology in smartphones and the explosion of their usage in people's daily lives provide users with the ability to collectively sense the world. This leads to a growing trend of mobile healthcare systems utilizing sensing data collected from smartphones with/without additional external sensors to analyze and understand people's physical and mental states. However, such healthcare systems are vulnerable to user spoofing attacks, in which an adversary distributes his registered device to other users such that data collected from these users can be claimed as his own to obtain more healthcare benefits and undermine the successful operation of mobile healthcare systems. Existing mitigation approaches either only rely on a secret PIN number (which can not deal with colluded attacks) or require an explicit user action for verification. In this paper, we propose a user verification scheme leveraging unique gait patterns derived from acceleration readings in mobile healthcare systems to detect possible user spoofing attacks. Our framework exploits the readily available accelerometers embedded within smartphones for user verification. Specifically, our user spoofing attack mitigation scheme (which consists of three components, namely Step Cycle Identification, Step Cycle Interpolation, and Similarity Score Computation) is used to extract gait patterns from run-time accelerometer measurements to perform robust user verification under various walking speeds. Our experiments using 322 smartphone-based traces over a period of 6 months confirm that our scheme is highly effective for detecting user spoofing attacks. This strongly indicates the feasibility of using smartphone based low grade accelerometer to conduct gait recognition and facilitate effective user verification without active user cooperation.

Touch-screen technique has gained the large popularity in human-screen interaction with modern smartphones. Due to the limited size of equipped screens, scrolling operations are indispensable in order to display the content of interest on screen. While power consumption caused by hardware and software installed within smartphones is well studied, the energy cost made by human-screen interaction such as scrolling remains unknown. In this paper, we analyze the impact of scrolling operations to the power consumption of smartphones, finding that the state-of-art strategy of smartphones in responding a scrolling operation is to always use the highest frame rate which arouses huge computation burden and can contribute nearly 50% to the total power consumption of smartphones. In recognizing this significance, we further propose a novel system, Energy-Efficient Engine(E3), which automatically tracks the scrolling speed and adaptively adjusts the frame rate according to individual user preference. The goal of E3 is to guarantee the user experience and minimize the energy consumption caused by scrolling at the same time. Extensive experiment results demonstrate the efficiency of E3 design. On average, E3 can save up to 58% of the energy consumed by CPU and 34% of the overall energy consumption.

Wireless spoofing attacks are easy to launch and can significantly impact the performance of networks. Although the identity of a node can be verified through cryptographic authentication, conventional security approaches are not always desirable because of their overhead requirements. In this paper, we propose to use spatial information, a physical property associated with each node, hard to falsify, and not reliant on cryptography, as the basis for 1) detecting spoofing attacks; 2) determining the number of attackers when multiple adversaries masquerading as the same node identity; and 3) localizing multiple adversaries. We propose to use the spatial correlation of received signal strength (RSS) inherited from wireless nodes to detect the spoofing attacks. We then formulate the problem of determining the number of attackers as a multiclass detection problem. Cluster-based mechanisms are developed to determine the number of attackers. When the training data are available, we explore using the Support Vector Machines (SVM) method to further improve the accuracy of determining the number of attackers. In addition, we developed an integrated detection and localization system that can localize the positions of multiple attackers. We evaluated our techniques through two testbeds using both an 802.11 (WiFi) network and an 802.15.4 (ZigBee) network in two real office buildings. Our experimental results show that our proposed methods can achieve over 90 percent Hit Rate and Precision when determining the number of attackers. Our localization results using a representative set of algorithms provide strong evidence of high accuracy of localizing multiple adversaries.

Mobile wireless devices (e.g., smartphones, PDAs, and notebooks) play important roles in our daily life, e.g., users often use such devices for bank transactions, keep in touch with friends. Users can also store such information and share with one another via opportunistic peer to peer links. However, peer to peer links are opportunistic links which are intermittent in nature and hence require the store-and-forward feature proposed in Delay Tolerant Networks to provide useful data sharing opportunities. Moreover, due to the limited resources, e.g., communication bandwidth and battery consumption, mobile devices can be selfish and may not be willing to forward data items to other devices that are interested in such items. Hence, effective data dissemination schemes need to be designed to encourage nodes to collaboratively share data. In this paper, we propose a Multi-Receiver Incentive-Based Dissemination (MuRIS) scheme that allows nodes to cooperatively deliver information of interest to one another via chosen delivery paths that utilize few transmissions. Our MuRIS scheme utilizes local historical path and tracks users' interests information maintained by each node. In addition, the charge and reward functions incorporated within our MuRIS scheme stimulate cooperation among nodes such that the nodes have no incentive to launch edge insertion attacks. Furthermore, our charge and reward functions are designed such that the chosen delivery paths mimic efficient multicast tree that results in fewest delivery hops. Extensive simulation studies using real human contact-based mobility traces show that our MuRIS scheme outperforms existing methods in terms of delivery ratio and transmission efficiency.

Securing communication in mobile wireless networks is challenging because the traditional cryptographic-based methods are not always applicable in dynamic mobile wireless environments. Using physical layer information of radio channel to generate keys secretly among wireless devices has been proposed as an alternative in wireless mobile networks. And the Received Signal Strength (RSS) based secret key extraction gains much attention due to the RSS readings are readily available in wireless infrastructure. However, the problem of using RSS to generate keys among multiple devices to ensure secure group communication remains open. In this work, we propose a framework for collaborative key generation among a group of wireless devices leveraging RSS. The proposed framework consists of a secret key extraction scheme exploiting the trend exhibited in RSS resulted from shadow fading, which is robust to outsider adversary performing stalking attacks. To deal with mobile devices not within each other's communication range, we employ relay nodes to achieve reliable key extraction. To enable secure group communication, two protocols, namely star-based and chain-based, are developed in our framework by exploiting RSS from multiple devices to perform group key generation collaboratively. Our experiments in both outdoor and indoor environments confirm the feasibility of using RSS for group key generation among multiple wireless devices under various mobile scenarios. The results also demonstrate that our collaborative key extraction scheme can achieve a lower bit mismatch rate compared to existing works when maintaining the comparable bit generation rate.

Jammer can jeopardize the dependability of wireless networks, and jammer's position information allows the network to cope with jamming leveraging varieties of defense strategies. Thus, in this paper, we address the problem of localizing jammer. Prior work relies on indirect measurements derived from jamming effects, which makes it difficult to accurately localize jammer. We localize jammer by directly using the strength of jamming signals (JSS). Estimating JSS is challenging as they may be embedded in other signals. As such, we devise an estimation scheme based on ambient noise floor and validate it with real world experiments. To improve localization accuracy, we define an evaluation feedback metric to quantify the estimation errors and formulate jammer localization as a nonlinear optimization problem, whose optimal solution approaches jammer's true position. We exploit a heuristic search based algorithm for approximating the global optimal solution, and our extensive simulation shows that our error-minimizing-based algorithm outperforms existing algorithms.

The inclusion of embedded sensors in mobile phones, and the explosion of their usage in people's daily lives provide users with the ability to collectively sense the world. The collected sensing data from such a mobile phone enabled social network can be mined for users' behaviors and their social communities, and to support a broad range of applications including mobile healthcare systems. However, such mobile healthcare systems built upon social networks are vulnerable to clone attacks, in which the adversary replicates the legitimate nodes and distributes the clones throughout the network to undermine the successful application deployment. Existing clone attack mitigation approaches either only focus on the prevention techniques or can only work in static or well-connected networks, and hence are not applicable to our targeted mobile healthcare systems. In this paper, we propose a social closeness based method in a mobile healthcare disease control system to detect any clone attacks that may be launched to disrupt the normal operations of the system. Our social closeness based method exploits the social relationships among users for clone attack detection. Specifically, we define a new metric called community betweenness, which considers mobile users' community information. We find that the value of this metric changes significantly under the clone attack, which is suitable to be used for clone attack detection. We derive both analytical and training based approaches to determine the threshold setting of the community betweenness for robust clone attack detection. Extensive trace-driven simulation studies reveal that our social closeness based method can detect clone attacks with high detection ratio and low false positive rate.

Highly accurate indoor localization of smartphones is critical to enable novel location based features for users and businesses. In this paper, we first conduct an empirical investigation of the suitability of WiFi localization for this purpose. We find that although reasonable accuracy can be achieved, significant errors (e.g., $6\sim8m$) always exist. The root cause is the existence of distinct locations with similar signatures, which is a fundamental limit of pure WiFi-based methods. Inspired by high densities of smartphones in public spaces, we propose a peer assisted localization approach to eliminate such large errors. It obtains accurate acoustic ranging estimates among peer phones, then maps their locations jointly against WiFi signature map subjecting to ranging constraints. We devise techniques for fast acoustic ranging among multiple phones and build a prototype. Experiments show that it can reduce the maximum and 80-percentile errors to as small as $2m$ and $1m$, in time no longer than the original WiFi scanning, with negligible impact on battery lifetime.

No abstract available.

Technological advancements have made it possible to use information associated with a mobile's location to form new computing and services. One concern with these emerging location-based services (LBS) is their ability to provide security while remaining reliable and accurate. In this paper, we focus on securing Mobile Location-based Services (MLBS), where certain goods or services are provided by mobile vendors to a clientele based on the proximity of vendors to potential customers. We identify different attacks and misuse faced by MLBS, and show that position verification is a critical step in providing a secure and trustworthy MLBS. To provide position verification, we propose a scheme called Key Distribution-based Position Verification (KEPI), which takes advantage of an auxiliary network of transponders to facilitate trustworthy location-based services. We derive an analytical model to evaluate our approach and our simulation results provide useful insights about how auxiliary networks can help provide trustworthy mobile services.

ABSTRACT Trustworthy location information is important because it is a critical input to a wide variety of location‐based applications. However, the localization infrastructure is vulnerable to physical attacks, and consequently, the localization results are affected. In this paper, we aim to achieve robust localization under infrastructure attacks. We first investigated the impact of infrastructure attacks on localization and showed that the performance of location estimations degraded significantly under the attack. We then derived an attack‐resistant scheme that is not algorithm specific and can be integrated with existing localization algorithms. Our attack‐resistant scheme exploited the characteristics of the geometric patterns returned by location estimates under the attack; that is, the localization results of a wireless device under the normal situation were clearly clustered together, whereas the localization results were scattered when an attack was present. Thus, our attack‐resistant scheme is grounded on K‐means clustering analysis of intra‐distance of localization results from all possible combinations of any three access points. To evaluate the effectiveness and scalability of our proposed scheme, we used received signal strength for validation and applied our approach to three broad classes of localization algorithms: lateration based, fingerprint matching, and Bayesian networks. We validated our scheme in the ORBIT test bed (North Brunswick, NJ, USA) using an 802.11 (Wi‐Fi) network and in a real office building environment using an 802.15.4 (ZigBee) network. The extensive experimental results demonstrated that the application of our scheme could help the broad range of localization algorithms to achieve comparable or even better localization performance when under infrastructure attacks as compared with normal situations without attack, thus, effectively eliminating the effects of infrastructure attacks. Copyright © 2011 John Wiley & Sons, Ltd.

This work addresses the fundamental problem of distinguishing between a driver and passenger using a mobile phone, which is the critical input to enable numerous safety and interface enhancements. Our detection system leverages the existing car stereo infrastructure, in particular, the speakers and Bluetooth network. Our acoustic approach has the phone send a series of customized high frequency beeps via the car stereo. The beeps are spaced in time across the left, right, and if available, front and rear speakers. After sampling the beeps, we use a sequential change-point detection scheme to time their arrival, and then use a differential approach to estimate the phone's distance from the car's center. From these differences a passenger or driver classification can be made. To validate our approach, we experimented with two kinds of phones and in two different cars. We found that our customized beeps were imperceptible to most users, yet still playable and recordable in both cars. Our customized beeps were also robust to background sounds such as music and wind, and we found the signal processing did not require excessive computational resources. In spite of the cars' heavy multipath environment, our approach had a classification accuracy of over 90 percent, and around 95 percent with some calibrations. We also found, we have a low false positive rate, on the order of a few percent.

Jamming attacks are especially harmful when ensuring the dependability of wireless communication. Finding the position of a jammer will enable the network to actively exploit a wide range of defense strategies. In this paper, we focus on developing mechanisms to localize a jammer by exploiting neighbor changes. We first conduct jamming effect analysis to examine how the communication range alters with the jammer's location and transmission power using free-space model. Then, we show that a node's affected communication range can be estimated purely by examining its neighbor changes caused by jamming attacks and thus, we can perform the jammer location estimation by solving a least-squares (LSQ) problem that exploits the changes of communication range. Compared with our previous iterative-search-based virtual force algorithm, our LSQ-based algorithm exhibits lower computational cost (i.e., one step instead of iterative searches) and higher localization accuracy. Furthermore, we analyze the localization challenges in real systems by building the log-normal shadowing model empirically and devising an adaptive LSQ-based algorithm to address those challenges. The extensive evaluation shows that the adaptive LSQ-based algorithm can effectively estimate the location of the jammer even in a highly complex propagation environment.

Community association is an important attribute of a social network because people may belong to varying groups with different characteristics at different times. Traditional community detection approaches often rely on a centralized server and are only useful for offline data analysis. In this paper, we propose and evaluate a distributed community detection approach that allows individual users to detect their own communities based on local observations. Our proposed template- matching method derives dynamic spatial and temporal characteristics of social communities by exploiting human's mobility patterns. Our template matching method allows users with similar moving patterns to be grouped together as one community. Our results using both simulation as well as real experiments demonstrate that our method can detect local communities effectively with high detection rate and low false positive rate.

In this paper, we study the distributed spectrum sensing in cognitive radio networks. Using weighted average consensus algorithm, we develop a weighted soft measurement combining scheme without the centralized fusion center. After the measurement by the energy detector, each secondary user (SU) exchanges their own measurement statistics with its local neighbors, and chooses the information exchanging rate according to the estimated average signal-to-noise ratio (SNR). We prove the convergence of the consensus iteration, and each SU will hold the global decision statistics from the weighted soft measurement combining throughout the network. The proposed scheme is robust with respect to temporary communication link failures. Simulation results show our method has a better performance than the existing average consensus-based approach.

Jamming attacks and unintentional radio interference are one of the most urgent threats harming the dependability of wireless communication and endangering the successful deployment of pervasive applications built on top of wireless networks. Unlike the traditional approaches focusing on developing jamming defense techniques without considering the location of jammers, we take a different viewpoint that the jammers' position should be identified and exploited for building a wide range of defense strategies to alleviate jamming. In this paper, we address the problem of localizing multiple jamming attackers coexisting in wireless networks by leveraging the network topology changes caused by jamming. We systematically analyze the jamming effects and develop a framework that can partition network topology into clusters and can successfully estimate the positions of multiple jammers even when their jamming areas are overlapping. Our experiments on a multi-hop network setup using MicaZ sensor nodes validate the feasibility of real-time collection of network topology changes under jamming and our extensive simulation results demonstrate that our approach is highly effective in localizing multiple attackers with or without the prior knowledge of the order that the jammers are turned on.

Received Signal Strength (RSS) based localization algorithms are sensitive to a set of non-cryptographic attacks. For example, the attacker can perform signal strength attacks by placing an absorbing or reflecting material around a wireless device to modify its RSS readings. In this work, we first formulate the all-around signal strength attacks, where similar attacks are launched towards all landmarks, and experimentally show the feasibility of launching such attacks. We then propose a general principle for designing RSS-based algorithms so that they are robust to all-around signal strength attacks. To evaluate our approach, we adapt two RSS-based localization algorithms according to our principle and experiment with real attack scenarios. All the experiments show that our design principle can be applied to achieve comparable performance with much better robustness.

New mobile phones equipped with multiple sensors provide users with the ability to sense the world at a microscopic level. The collected mobile sensing data can be comprehensive enough to be mined not only for the understanding of human behaviors but also for supporting multiple applications ranging from monitoring/tracking, to medical, emergency and military applications. In this work, we investigate the feasibility and effectiveness of using human contact traces collected from mobile phones to derive social community information to control the disease propagation rate in the healthcare domain. Specifically, we design a community-based framework that extracts the dynamic social community information from human contact based traces to make decisions on who will receive disease alert messages and take vaccination. We have experimentally evaluated our framework via a trace-driven approach by using data sets collected from mobile phones. The results confirmed that our approach of utilizing mobile phone enabled dynamic community information is more effective than existing methods, without utilizing social community information or merely using static community information, at reducing the propagation rate of an infectious disease. This strongly indicates the feasibility of exploiting the social community information derived from mobile sensing data for supporting healthcare related applications.

This work addresses the fundamental problem of distinguishing between a driver and passenger using a mobile phone, which is the critical input to enable numerous safety and interface enhancements. Our detection system leverages the existing car stereo infrastructure, in particular the speakers and Bluetooth network. Our acoustic approach has the phone send a series of customized high frequency beeps via the car stereo. The beeps are spaced in time across the left, right, and if available, front and rear speakers. After sampling the beeps, we use a sequential change-point detection scheme to time their arrival, and then use a differential approach to estimate the phone's distance from the car's center. From these differences a passenger or driver classification can be made. To validate our approach, we experimented with two kinds of phones and in two different cars. We found that our customized beeps were imperceptible to most users, yet still playable and recordable in both cars. Our customized beeps were also robust to background sounds such as music and wind, and we found the signal processing did not require excessive computational resources. In spite of the cars' heavy multi-path environment, our approach had a classification accuracy of over 90%, and around 95% with some calibrations. We also found we have a low false positive rate, on the order of a few percent.

In this paper, we consider the problem of tracking fine-grained speeds variations of vehicles using signal strength traces from GSM enabled phones. Existing speed estimation techniques using mobile phone signals can provide longer-term speed averages but cannot track short-term speed variations. Understanding short-term speed variations, however, is important in a variety of traffic engineering applications-for example, it may help distinguish slow speeds due to traffic lights from traffic congestion when collecting real time traffic information. Using mobile phones in such applications is particularly attractive because it can be readily obtained from a large number of vehicles. Our approach is founded on the observation that the large-scale path loss and shadow fading components of signal strength readings (signal profile) obtained from the mobile phone on any given road segment appear similar over multiple trips along the same road segment except for distortions along the time axis due to speed variations. We therefore propose a speed tracking technique that uses a Derivative Dynamic Time Warping (DDTW) algorithm to realign a given signal profile with a known training profile from the same road. The speed tracking technique then translates the warping path (i.e., the degree of stretching and compressing needed for alignment) into an estimated speed trace. Using 6.4 hours of GSM signal strength traces collected from a vehicle, we show that our algorithm can estimate vehicular speed with a median error of ± 5mph compared to using a GPS and can capture significant speed variations on road segments with a precision of 68% and a recall of 84%.

This paper introduces an algorithm that estimates the speed of a mobile phone by matching time-series signal strength data to a known signal strength trace from the same road. Knowing a mobile phone's speed is useful, for example, to estimate traffic congestion or other transportation performancemetrics. The proposed algorithmcan be implemented in the carrier's infrastructure with Network Measurement Reports obtained by a base station or on a mobile phone with signal strength readings obtained by the handset and depending on implementation choices, promises lower energy consumption than Global Positioning System (GPS) receivers. We evaluate the effectiveness of our algorithm on highway and arterial roads using GSM signal strength traces obtained from several phones over a one month period. The results show that the Correlation algorithm is significantly more accurate than existing techniques based on handoffs or phone localization.

Cell tower triangulation is a popular technique for determining the location of a mobile device. However, cell tower triangulation methods require the knowledge of the actual locations of cell towers. Because the locations of cell towers are not publicly available, these methods often need to use estimated tower locations obtained through wardriving. This paper provides the first large scale study of the accuracy of two existing methods for cell tower localization using wardriving data. The results show that naively applying these methods results in very large localization errors. We analyze the causes for these errors and conclude that one can localize a cell accurately only if it falls within the area covered by the wardriving trace. We further propose a bounding technique to select the cells that fall within the area covered by the wardriving trace and identify a cell combining optimization that can further reduce the localization error by half.

Recent years have witnessed increasing interests in passive intrusion detection for wireless environments, e.g., asset protection in industrial facilities and emergency rescue of trapped people. Most previous studies have focused primarily on exploiting a single intrusion indicator, such as moving variance, for capturing an intrusion pattern at a time. However, in real-world, there are many intrusion patterns which may be only detectable by combining different intrusion indicators and performing detection jointly. To this end, we propose a joint intrusion learning approach, which has the ability in combining the detection power of several complementary intrusion indicators and detects different intrusion patterns at the same time. We developed the GREEK algorithm, which utilizes grid-based clustering over K-neighborhood to effectively diagnose the presence of intrusions. Further, we show that the performance of intrusion detection can be enhanced by utilizing the collaborative detecting efforts among multiple transmitter-receiver pairs. To validate the effectiveness of the joint intrusion learning method, we conducted experiments in a real-office environment using an IEEE 802.15.4 (Zigbee) network. Our experimental results provide strong evidence of the effectiveness of our joint learning approach in performing passive intrusion detection with a minimized false positive rate.

Wireless localization using the received signal strength (RSS) can have tremendous savings over using specialized positioning infrastructures. In this work, we explore improving RSS localization performance in multipath environments by varying the transmitter's signal power and frequency. Using a theoretical analysis, we first show how selection of different signal powers and frequencies can improve localization accuracy for the least squares algorithm. We next develop a set of selection methods that attempt to select the combinations of power and frequencies which minimize the localization error. Our selection methods are based on the observed standard deviations of RSS as well as algorithm specific residuals. Using active RFID tags, we experimentally characterize the effect of using multiple signal powers and frequencies on a wide spectrum of RSS-based algorithms. We found that the performance of all the algorithms improves when leveraging on multiple power levels and frequencies, although different algorithms present different sensitivity in terms of localization accuracy under different selection methods.

The wireless data collected in mobile environments provides tremendous opportunities to build new applications in various domains such as Vehicular Ad Hoc Networks and mobile social networks. One of the biggest challenges is how to store these data. Storing the data decentralized in wireless devices is an attractive approach because of its major advantages over centralized ones. In this work, to facilitate effective access control of the wireless data in distributed data storage, we propose a fully decentralized key management scheme by utilizing a cryptography-based secret sharing method. The secret sharing method splits the keys into multiple shares and distributes them to multiple nodes, which brings the challenge that due to node mobility, these key shares may not be available in the neighborhood when they are needed for key reconstruction. To address this challenge arising from mobile environments, we propose the Transitive Prediction(TRAP) protocol that distributes key shares among devices that are traveling together. We derive a theoretical analysis of the robustness of our approach. Furthermore, inside TRAP, we develop three key distribution schemes that utilize the correlation relationship embedded among devices that are traveling together. Our key distribution schemes maximize the chance of successful key reconstruction and minimize the communication overhead. Our extensive simulation results demonstrate that our key distribution schemes are highly effective, and thus provide strong evidence of the feasibility of applying our approach to support distributed data storage in wireless networks.

The Disruption Tolerant Networks (DTNs) are vulnerable to insider attacks, in which the legitimate nodes are compromised and the adversary modifies the delivery metrics of the node to launch harmful attacks in the networks. The traditional detection approaches of secure routing protocols can not address such kind of insider attacks in DTNs. In this paper, we propose a mutual correlation detection scheme (MUTON) for addressing these insider attacks. MUTON takes into consideration of the transitive property when calculating the packet delivery probability of each node and correlates the information collected from other nodes. We evaluated our approach through extensive simulations using both Random Way Point and Zebranet mobility models. Our results show that MUTON can detect insider attacks efficiently with high detection rate and low false positive rate.

The Disruption Tolerant Networks (DTNs) are especially useful in providing mission critical services such as in emergency networks or battlefield scenarios. However, DTNs are vulnerable to insider attacks, in which the legitimate nodes are compromised and the adversary nodes launch blackhole attacks by dropping packets in the networks. The traditional approaches of securing routing protocols can not address such insider attacks in DTNs. In this paper, we propose a method to secure the history records of packet delivery information at each contact so that other nodes can detect insider attacks by analyzing these packet delivery records. We evaluated our approach through extensive simulations using both Random Way Point and Zebranet mobility models. Our results show that our method can detect insider attacks efficiently with high detection rate and low false positive rate.

Wireless sensor networks are typically deployed to monitor phenomena that vary over the spatial region the sensor network covers. The sensor readings may also be dual-used for additional purposes. In this paper, we propose to use the inherent spatial variability in physical phenomena, such as temperature or ambient acoustic energy, to support localization and position verification. We first present the problem of localization using general spatial information fields, and then, propose a theory for exploiting this spatial variability for localization. Our Spatial Correlation Weighting Mechanism (SCWM) uses spatial correlation across different phenomena to isolate an appropriate subset of environmental parameters for better location accuracy. We then develop an array of algorithms employing environmental parameters using a two-level approach: first, we develop the strategies on how the subset of parameters should be chosen, and second, we derive mapping functions for position estimation. Our algorithms support our theoretical model for performing localization utilizing environmental properties. Finally, we provide an experimental evaluation of our approach by using a collection of physical phenomena measured across 100 locations inside a building. Our results provide strong evidence of the viability of using general sensor readings for location-aware applications.

We show that signal strength variability can be reduced by employing multiple low-cost antennas at fixed locations. We further explore the impact of this reduction on wireless localization by analyzing a representative set of algorithms ranging from fingerprint matching, to statistical maximum likelihood estimation, to threshold bounding of signal fingerprints, and to multilateration. Using an indoor wireless testbed, we provide experimental evaluation of the localization performance under multiple antennas. We found that in nearly all cases the performance of localization algorithms improved when using multiple antennas. Specifically, the median and the 90th percentile error can be reduced up to 70 percent. Additionally, we found that multiple antennas improve the localization stability significantly, up to 100 percent improvement, when there are small-scale three-dimensional movements of a mobile device around a given location.

Location estimation is a critical step for many location-aware applications. To obtain location information, localization methods employing received signal strength (RSS) are attestative since it can reuse the existing wireless infrastructure for localization. Among the large class of localization schemes, RSS-based lateration methods have the advantage of providing closed-form solutions for mathematical analysis as compared to heuristic-based localization approaches. However, the localization accuracy of RSS-based lateration methods are significantly affected by the unpredictable setup in indoor environments. To improve the applicability of RSS-based lateration methods in indoors, we propose two approaches, regression-based and correlation-based. The regression-based approach uses linear regression to discover a better fit of signal propagation model between RSS and the distance, while the correlation-based approach utilizes the correlation among RSS in local area to obtain more accurate signal propagation. Our results using both simulation as well as real experiments demonstrate that our improved methods outperform the original RSS-based lateration methods significantly.

Dynamic spectrum access has been proposed as a means to share scarce radio resources, and requires devices to follow protocols that use resources in a proper, disciplined manner. For a cognitive radio network to achieve this goal, spectrum policies and the ability to enforce them are necessary. Detection of an unauthorized (anomalous) usage is one of the critical issues in spectrum etiquette enforcement. In this paper, we present a network structure for dynamic spectrum access and formulate the anomalous usage detection problem using statistical significance testing. The detection problem is classified into two subproblems. For the case where no authorized signal is present, we describe the existing cooperative sensing schemes and investigate the impact of signal path loss on their performance. For the case where an authorized signal is present, we propose three methods that detect anomalous transmissions by making use of the characteristics of radio propagation. Analytical models are formulated for two special cases and, due to the intractability of the general problem, we present an algorithm using machine learning techniques to solve the general case. Our simulation results show that our approaches can effectively detect unauthorized spectrum usage with high detection rate and low false positive rate.

Wireless spoofing attacks are easy to launch and can significantly impact the performance of networks. Although the identity of a node can be verified through cryptographic authentication, conventional security approaches are not always desirable because of their overhead requirements. In this paper, we propose to use location information, a physical property associated with each node, hard to falsify, and not reliant on cryptography, as the basis for (1) detecting spoofing attacks; (2) determining the number of attackers when multiple adversaries masquerading as a same node identity; and (3) localizing multiple adversaries. We formulate the problem of determining the number of attackers as a multi-class detection problem. We first propose two cluster-based mechanisms to determine the number of attackers. We then develop SILENCE that employs the minimum distance testing of RSS values in addition to cluster analysis and can achieve better accuracy than other methods under study that merely use cluster analysis alone. We further developed an integrated detection and localization system that can localize the positions of multiple attackers. We evaluated our techniques through two testbeds using both an 802.11 (WiFi) network and an 802.15.4 (ZigBee) network in two real office buildings. Our experimental results show that SILENCE can achieve over 90% Hit Rate and Precision when determining the number of attackers. Additionally, our localization results using a representative set of algorithms provide strong evidence of high accuracy of localizing multiple adversaries.

This work investigates the lower bounds of wireless localization accuracy using signal strength on commodity hardware. Our work relies on trace-driven analysis using an extensive indoor experimental infrastructure. First, we report the best experimental accuracy, twice the best prior reported accuracy for any localization system. We experimentally show that adding more and more resources (e.g., training points or landmarks) beyond a certain limit, can degrade the localization performance for lateration-based algorithms, and that it could only be improved further by "cleaning" the data. However, matching algorithms are more robust to poor quality RSS measurements. We next compare with a theoretical lower bound using standard Cramer Rao Bound (CRB) analysis for unbiased estimators, which is frequently used to provide bounds on localization precision. Because many localization algorithms are based on different mathematical foundations, we apply a diverse set of existing algorithms to our packet traces and found that the variance of the localization errors from these algorithms are smaller than the variance bound established by the CRB. Finally, we found that there exists a wide discrepancy from what free- space models predict in the signal to distance function even in an environment with limited shadowing and multipath, thereby imposing a fundamental limit on the achievable localization accuracy indoors.

The flexibility and openness of wireless networks enables an adversary to masquerade as other devices easily. Identity-based spoofing attacks are serious network threats as they can facilitate a variety of advanced attacks to undermine the normal operation of networks. However, the existing mechanisms can only detect spoofing attacks when the victim node and the spoofing node are static. In this paper, we propose a method for detecting spoofing attacks in the mobile wireless environment, that is when wireless devices, such as the victim node and/or the spoofing node are moving. We develop the DEMOTE system, which exploits received signal strength (RSS) traces collected over time and achieves an optimal threshold to partition the RSS traces into classes for attack detection. Further, our novel algorithm alignment prediction (ALP), when without the knowledge of spatial constraint of the wireless nodes, utilizes temporal constraints to predict the best RSS alignment of partitioned RSS classes for RSS trace reconstruction over time. Our approach does not require any changes or cooperation from wireless devices other than packet transmissions. Through experiments from an office building environment, we show that DEMOTE achieves accurate attack detection both in signal space as well as in physical space using localization and is generic across different technologies including IEEE 802.11 b/g and IEEE 802.15.4.

We present the DECODE technique to determine whether a set of transmitters are comoving, i.e., moving together in close proximity. Comovement information can find use in applications ranging from inventory tracking to social network sensing and to optimizing mobile device localization. The positioning errors from indoor RSS-based localization systems tend to be too large, making it difficult to detect whether two devices are moving together based on the interdevice distances. DECODE achieves accurate comovement detection by exploiting the correlations in positioning errors over time. DECODE can not only be implemented in the position space but also in the signal space where a correlation in shadow fading due to objects blocking the path between the transmitter and receiver exists. This technique requires no change in or cooperation from the tracked devices other than sporadic transmission of packets. Using experiments from an office environment, we show that DECODE can achieve near-perfect comovement detection at walking speed mobility using correlation coefficients computed over approximately 60-second time intervals. We further show that DECODE is generic and could accomplish detection for mixed mobile transmitters of different technologies (IEEE 802.11b/g and IEEE 802.15.4), and our results are not very sensitive to the frequency at which transmitters communicate.

Recently, it has been noted that localization algorithms that use signal strength are susceptible to noncryptographic attacks, which consequently threatens their viability for sensor applications. In this work, we examine several localization algorithms and evaluate their robustness to attacks where an adversary attenuates or amplifies the signal strength at one or more landmarks. We study both point-based and area-based methods that employ received signal strength for localization, and propose several performance metrics that quantify the estimator's precision, bias, and error, including Hölder metrics, which quantify the variability in position space for a given variability in signal strength space. We then conduct a trace-driven evaluation of a set of representative algorithms, where we measured their performance as we applied attacks on real data from two different buildings. We found the median error degraded gracefully, with a linear response as a function of the attack strength. We also found that area-based algorithms experienced a decrease and a spatial-shift in the returned area under attack, implying that precision increases though bias is introduced for these schemes. Additionally, we observed similar values for the average Hölder metric across most of the algorithms, thereby providing strong experimental evidence that nearly all the algorithms have similar average responses to signal strength attacks with the exception of the Bayesian Networks algorithm.

Accurately obtaining the position of mobile devices is critical to high-level applications. In indoor environments, localization approaches employing RF-based fingerprint matching is an active research area because it can reuse the existing communication infrastructure, as well as reduce the signal uncertainty to achieve better location accuracy. In this paper, we provide a theoretical analysis of the localization performance when using fingerprint matching schemes. Specifically, we derived an analytic expression for the Cumulative Distribution Function (CDF) of the location error and investigated the mathematical relationship between the location error and the sampling points. Particularly, we studied the effects of the number of sampling points and the distance between adjacent sampling points. We further conducted experiments using an 802.11 network in a real office building environment. Our performance evaluation provides strong evidence that theoretical analysis is effective as the experimental results match our analytic study very well.

We present the DECODE technique to determine from a remote receiver whether a set of transmitters are co-moving, i.e., moving together in close proximity. Co-movement information can find use in applications ranging from inventory tracking, to social network sensing, and to optimizing mobile device localization. DECODE detects co-moving transmitters by identifying correlations in communication signal strength due to shadow fading. Unlike localization systems, it can operate using measurements from only a single receiver. It requires no changes in or cooperation from the tracked devices other than sporadic transmission of packets. Using experiments from an office environment, we show that DECODE can achieve near perfect co-movement detection at walking-speed mobility using correlation coefficients computed over approximately 60-second time intervals.

Wireless networks are vulnerable to sybil attacks, in which a sybil node forges multiple identifications to trick the system and conduct harmful attacks. The traditional approach to address sybil attacks is to employ cryptographic-related methods. However, conventional security approaches may not always desirable due to their infrastructural overhead. In this paper, we propose to utilize K-means cluster analysis for detecting sybil attacks based on the spatial correlation between the signal strength and physical locations. Our approach requires minimal overhead to wireless devices. We have evaluated our methods through experimentation using both an 802.11 (WiFi) network as well as an 802.15.4 (ZigBee) network in two office buildings. Our results show that the proposed sybil attack detector is highly effective with over 95% detection rates and under 5% false positive rates.

Wireless sensor networks monitor phenomena that vary over the spatial region the sensor network covers. The sensor readings may also be dual-used for additional purposes. In this paper we propose to use the inherent spatial variability in physical phenomena to support localization and position verification. We first present the problem of localization using general spatial information fields, and then propose a theory for exploiting this spatial variability for localization. Our spatio-correlation weighting mechanism (SCWM) uses spatial relationships of measured physical phenomena to determine an appropriate subset of environmental parameters for better location accuracy. We next present the Flex - EP algorithm, which supports our theoretical model for performing localization. Finally, we provide an experimental evaluation of our approach by using a collection of physical phenomena measured across one hundred locations inside a building. Our results provide strong evidence of the viability of using general sensor readings for location applications.

We show that signal strength variability can be reduced by employing multiple low-cost antennas at fixed locations. We further explore the impact of this reduction on wireless localization by analyzing a representative set of algorithms ranging from fingerprint matching, to statistical maximum likelihood estimation, and to multilateration. We provide experimental evaluation using an indoor wireless testbed of the localization performance under multiple antennas. We found that in nearly all cases the performance of localization algorithms improved when using multiple antennas. Specifically, the median and the 90th percentile error can be reduced up to 70%. Additionally, we found that multiple antennas improve the localization stability significantly, up to 100% improvement, when there are small scale 3-dimensional movements of a mobile device around a given location.

Accurately positioning nodes in wireless and sensor networks is important because the location of sensors is a critical input to many higher-level networking tasks. However, the localization infrastructure can be subjected to non-cryptographic attacks, such as signal attenuation and amplification, that cannot be addressed by traditional security services. We propose several attack detection schemes for wireless localization systems. We first formulate a theoretical foundation for the attack detection problem using statistical significance testing. Next, we define test metrics for two broad localization approaches: multilateration and signal strength. We then derived both mathematical models and analytic solutions for attack detection for any system that utilizes those approaches. We also studied additional test statistics that are specific to a diverse set of algorithms. Our trace-driven experimental results provide strong evidence of the effectiveness of our attack detection schemes with high detection rates and low false positive rates across both an 802.11 (WiFi) network as well as an 802.15.4 (ZigBee) network in two real office buildings. Surprisingly, we found that of the several methods we describe, all provide qualitatively similar detection rates which indicate that the different localization systems all contain similar attack detection capability.

We investigate the impact of landmark placement on localization performance using a combination of analytic and experimental analysis. For our analysis, we have derived an upper bound for the localization error of the linear least squares algorithm. This bound reflects the placement of landmarks as well as measurement errors at the landmarks. We next develop a novel algorithm, maxL minE, that using our analysis, finds a pattern for landmark placement that minimizes the maximum localization error. To show our results are applicable to a variety of localization algorithms, we then conducted a series of localization experiments using both an 802.11 (WiFi) network as well as an 802.15.4 (ZigBee) network in a real building environment. We use both received signal strength (RSS) and time-of-arrival (ToA) as ranging modalities. Our experimental results show that our landmark placement algorithm is generic because the resulting placements improve localization performance across a diverse set of algorithms, networks, and ranging modalities

No abstract available.