Radio Frequency Fingerprinting (RFF) exploits inherent hardware-level imperfections of wireless transmitters as unclonable identifiers for device identification. These unique signatures, concealed in transmitted signals, inevitably experience complex distortions during wireless propagation (i.e., coupled with ambient noise and channel fading), making it extremely challenging for reliable extraction. Despite substantial research efforts dedicated to advancing effective fingerprint extraction techniques, current approaches still struggle in handling fingerprint robustness under distance variations, leading to severe SNR fluctuations and complex multipath effects. To address this gap, we propose the first unsupervised framework for distance-invariant radio frequency fingerprinting, eliminating dependence on labeled target domain data. Specifically, we first preprocess raw RF samples by confining them within a specified variation range and filtering noisy high-frequency components while avoiding aliasing. For source domain data, we then propose a set of physics-inspired data augmentation techniques designed to emulate realistic wireless signal propagation effects. Building on this, we introduce a dual alignment contrastive learning method to explicitly decouple identity-discriminative features, ensuring the model focuses on device-specific traits. Furthermore, we incorporate a pseudo-labeling-based domain adaptation module to refine the model for the unlabeled target domain, enhancing its generalization to unseen distances. Extensive experiments on public datasets show that our method achieves the identification accuracy outperforming state-of-the-art approaches by 40%, while maintaining computational efficiency suitable for edge deployment.

Physical layer secret key generation has emerged as a promising approach for secret key establishment in wireless networks. Unlike traditional quantization-based methods, recent studies have explored matching the patterns of segmented channel samples of equal length for key agreement. However, equal-length segmentation either suffers from inconsistencies between users for short segments or a reduced key generation rate for long ones. To address these issues, we propose a Variable-length Segment Matching-based Secret Key Generation method, VSM-SKG, which adaptively partitions channel samples into variable-length segments to enhance overall matching accuracy, key generation rate, and encryption strength. Specifically, we introduce a dissimilarity-enhanced segmentation and calibration strategy that partitions channel samples into variable-length segments to enlarge segment-wise dissimilarity. To achieve consistent key recovery between users, we develop a dynamic path-aware key generation method that identifies potential segmentation patterns and generates agreed-upon secret keys using a recursive approach combined with a fast retrieval mechanism. Theoretical analyses and real-world experiments validate the attributes of VSM-SKG in terms of accuracy, efficiency, and security in key generation.

Physical layer authentication (PLA), which leverages device-specific physical layer features to achieve information-theoretic security with low complexity, offers a hardware-rooted security solution for next-generation Internet of Things (IoT) networks. While existing PLA approaches primarily rely on passive extraction of inherent hardware-induced radio frequency features, they remain vulnerable to adversarial spoofing that replicates legitimate radio frequency fingerprints (RFF). To address this critical vulnerability, we propose an active PLA framework that employs a challenge–response protocol to embed session-specific perturbation into each transmission. Upon receiving a nonce from the receiver, the legitimate transmitter generates a hash value and embeds a corresponding in-phase and quadrature (IQ) imbalance-based perturbation into the baseband signal. This design conceals inherent hardware-specific RFF and injects dynamic, unpredictable fingerprints that vary across sessions and are resilient to forgery. At the receiver, authentication is performed using a learning-based method that combines a CNN-based feature extractor with a lightweight logistic regression classifier trained on augmented samples. Extensive simulations demonstrate that the proposed framework achieves high authentication accuracy under both static and dynamic channel conditions, while effectively resisting advanced spoofing attacks, including GAN-based impersonation. These results confirm the robustness, generalization capability, and applicability of the proposed scheme for secure IoT communications.

As one of the most common ways for user authentication, Personal Identification Number (PIN), due to its simplicity and convenience, has suffered from plenty of side-channel attacks, which pose a severe threat to people’s privacy and property. The success of existing attacks is usually built upon the premise of no occlusion between the attacker and the victim’s hand gesture, but it increases the difficulty of launching the attack and the possibility of exposure. To overcome such limitation, we propose ArmSpy++, an improved video-assisted PIN inference attack built upon our previous research, ArmSpy. Specifically, ArmSpy++ employs new modules to leverage more features like the keystroke-induced elbow bending, wrist speed variation, and the spatial relationship between different arm joints, to correctly detect Keystrokes. ArmSpy++ delves into the perspective relationship and natural typing habits to ensure a high success rate of PIN inference. We also re-designed the inferred PIN pattern coordination mechanism to accurately deduce the PINs. By using a pre-trained HigherHRNet model for posture estimation ArmSpy++ eliminates the necessity of additional training. The extensive experiments demonstrate that ArmSpy++ can achieve over 83.1% average accuracy with 3 attempts and even 92.5% for some victims, indicating the severity of the threat posed by ArmSpy++.

Secret key generation exploiting inherent channel randomness stands as an important paradigm for physical-layer security in wireless networks. However, existing work relying on quantization has some difficulties in eliminating inconsistent key bits due to the impact of ambient noise. Recent studies propose to match the segmented channel samples (i.e., channel episodes) of similar variation patterns between legitimate peers to achieve error-free key generation, but they also suffer from high computational overhead and reduced accuracy for large key lengths. This work proposes a secret key generation method based on sorted indices matching (SIM-SKG), aiming at efficient and error-free key generation. Specifically, we sort the channel samples to ensure each channel episode with a unique variation pattern for accurate matching. To avoid the impact of half-duplex communication mode and ambient noise, we propose to match the indices instead of the channel samples as in existing studies. We also develop a noise perturbation scheme that further mitigates the ambiguity during indices matching. Extensive experimental studies demonstrate the high efficiency and accuracy of SIM-SKG under various scenarios for both RSS and CSI channel measurements. Specifically, SIM-SKG achieves error-free key generation with a length of 2048 bits within as little as 1.7$msec$. Moreover, theoretical analyses and experiments also confirm the security of the SIM-SKG method against various attacks.

The two-factor authentication (2FA) has been increasingly used with the popularity of mobile devices. Currently, many existing 2FA schemes extract the devices’ acoustic fingerprints as the second factor. Nevertheless, they mainly consider deriving fingerprints from the raw acoustic waveforms for authentication, which are susceptible to the fingerprint variations caused by the environmental noise or the varying distance between devices. To address these vulnerabilities, we propose a robust system utilizing the distortions of modulated signals, which are incurred by the acoustic elements of mobile devices, as the proof for 2FA. Specifically, our system first designs a channel delay estimation scheme to accurately estimate the propagation delay from the speaker to the microphone by deriving the phase change of the received sinusoidal signal. To perform a robust authentication, we design a new acoustic fingerprinting scheme to remove the impacts of the varying distance and environmental noise from the demodulated PSK signals for fingerprint extraction. Moreover, our device authentication component designs a transfer learning-based scheme to capture the subtle differences in devices’ fingerprints for accurate device authentication. To the best of our knowledge, this is the first 2FA system that could extract acoustic fingerprints in modulation domain and can effectively withstand the impacts of channel distortions. We also confirm the accuracy and security of our system through extensive user experiments.

Secret key generation plays an important role in device-to-device communication security in wireless networks. For consistent key generation between two communicating parties, existing matching-based key generation methods match segmented channel measurements (channel episodes) of similar patterns between two parties. However, these methods suffer diminished accuracy for large key lengths or in the presence of ambient noise. This work takes a different perspective to produce the desired channel measurements through adaptive manipulation of pilot signals for robust and accurate physical layer secret key generation. Specifically, an adaptive pilot manipulation scheme is designed not only to ensure that each channel episode has a unique pattern but also to improve pattern similarity between a pair of matched channel episodes, thus enabling high matching consistency. To validate the effectiveness of our method, we implement it by re-configuring software modules in GNU radio running on the USRP platform. Extensive experiments demonstrate that our method outperforms existing representative quantization-based and matching-based methods with improved key generation performance.

Hand pose estimation is a key support for a variety of interactive applications including user interface control, sign language understanding, virtual reality modeling, etc. Existing approaches mainly exploit wearable devices such as gloves or bracelets to estimate hand poses, which may introduce high deploying costs and intrusive user experience. Others rely on vision technologies whereas they could face complicated illuminations and privacy leakage. In this paper, we present a millimeter wave (mmWave) signal-based 3D hand pose estimation system, mmHand, which utilizes a mmWave radar to generate 3D hand skeletons and reconstruct 3D hand meshes. mmHand first leverages mmWave signals to sense a hand and pre-process the signals. Then, mmHand extracts spatial and temporal features using a designed attention-based hourglass network (mmSpaceNet) and Long Short-Term Memory (LSTM), respectively. Based on the extracted features, mmHand further regresses hand joints in 3D space to generate 3D hand skeletons. Finally, 3D hand meshes that continuously describe hand poses with detailed surfaces are reconstructed through a hand Model with Articulated and Non-rigid defOrmations (MANO). Extensive experiments demonstrate that mmHand can accurately generate 3D hand skeletons with 18.3mm mean per joint position error and 95.1 % of correct key points, which indicates the effectiveness of mmHand on hand pose estimation.

Physical layer-based secret key generation has garnered significant attention due to its inherent advantages of lightweight implementation, information-theoretic security, and broad applicability for mobile devices. The reciprocal randomness of the wireless channel ensures the consistent generation of secret bits between two communicating parties. However, it also suffers from the degradation of the efficiency of key generation attributed to the adverse impact of ambient noise, despite sustained efforts to mitigate the inconsistency during quantization. We find that a slight perturbation of the pilot signal, without affecting the correct reception of data frames, induces a corresponding change in the channel response, making it possibly adaptable to the target quantization strategies, thereby reducing the probability of key mismatch. Therefore, we take a different viewpoint on proactive contamination of the pilot signals to obtain the desired channel measurements for accurate physical layer secret key generation. Specifically, we design an adaptive pilot manipulation to avoid the expected channel measurements being too close to the quantization thresholds, enabling high quantization consistency. Furthermore, we also develop a random cross-threshold mechanism to prevent attackers from inferring the quantization results by monitoring the trend of pilot signal variations. A reliable long training sequence (LTS) modification mechanism is incorporated into our method to ensure communication performance by adaptively adjusting the scale of the pilot signal. To validate the effectiveness of our proposed method, we implement a prototype by re-configuring software modules in GNU radio running on the USRP platform. Extensive experiments demonstrate that our scheme outperforms existing representative quantization schemes with better key generation performance.

Privacy concerns over the security of personal information have grown in tandem with the spread of smartwatches. However, effective methods for protecting private data on smartwatches are very limited. Personal identity number (PIN) input is the only privacy protection method on off-the-shelf smartwatches, which requires tedious user effort. This is ineffective at securing information such as notifications and attention-grabbing alerts, which may leak personal data to passersby and adversaries, causing embarrassment or revealing sensitive communications. In this work, we propose a novel privacy protection system, TouchTone, that verifies users and secure personal data in a convenient and low-effort manner. Our system employs a challenge-response process to passively capture finger biometrics from an unobtrusive touch gesture using only microphones, speakers, and accelerometer sensors already built in smartwatches. To address smartwatch incompatibility with traditional high-frequency sensing techniques, we develop non-intrusive low-frequency challenge signals and cross-domain sensing techniques (i.e., measuring acoustic signals in the vibration domain) to capture robust and effective features specific to user fingers. A low-cost profile matching-based classifier is designed to enable stand-alone privacy protection on smartwatches. We conduct extensive experiments with 54 participants using varied hardware, environments, noise levels, user motions, and other impact factors, achieving around 97% true positive rate and 2% false positive rate in recognizing participants' identities for privacy protection.

Optical image stabilization (OIS), powered by a special micro-electromechanical structure in the camera lenses to compensate for the optical distortion caused by camera shakes, has become an indispensable feature in many smartphones. However, we discover that this seemingly benign component can be exploited to eavesdrop on nearby audio signals, posing a significant threat to people's privacy during conversations or phone calls. Specifically, the OIS component can be influenced by external acoustic stimuli leading to slight vibrations, and at the same time, the coil and magnetized components inside the OIS induce electromagnetic leakage as they vibrate, according to Faraday's Law of Electromagnetic Induction. This electro-magnetic leakage contains voice information that can be used to recover the audio signals if intercepted by individuals with malicious intent. Inspired by the above discovery, we propose OISMic, a new acoustic eavesdropping attack that takes advantage of sound-induced OIS vibrations on smartphones. Unlike other existing acoustic eavesdropping attacks, eavesdropping exploiting OIS vibrations not only overcomes the constraints imposed by system permissions for many sensor-based approaches but is also immune to ultrasonic jammer that hinders the methods relying on microwave or light reflections to sense sound-induced vibrations. To execute this non-trivial attack in practical scenarios, we developed a prototype circuit that has a compact design capable of capturing the electromagnetic leakage caused by OIS vibrations. After converting the collected leaked electromagnetic signals into audio signals, a software-based phase-locked loop (PLL) method is developed to enhance the representation of voice components. Meanwhile, to reconstruct the weak audio signals, we also designed a diffusion-based neural network to learn the distribution of electromagnetic noise within the audio spectrum. Extensive experiments indicate that OISMic can accurately reconstruct voice under various scenarios, achieving an average word correct rate of 90.57 % across different devices.

The physical layer secret key generation exploiting wireless channel reciprocity has demonstrated its viability and effectiveness in various wireless scenarios, such as the Internet of Things (IoT) network, mobile communication network, and industrial control system. Most of the existing studies rely on the quantization technique to convert channel measurements into secret bits for confidential communications. However, non-simultaneous packet exchanges in time-division duplex systems and noise effects usually induce inconsistent quantization results and mismatched secret bits. Although recent research has spent significant effort mitigating such non-reciprocity, it is still far from practical error-free key generation. Unlike previous quantization-based approaches, we take a different viewpoint to match the randomly manipulated (i.e., permuted or edited) channel measurements between a pair of users by minimizing their discrepancy holistically. Specifically, two novel secret key generation algorithms based on bipartite graph matching (BMSKG) and edited sequence alignment (SA-SKG) are developed. BM-SKG allows two users to generate the same secret key based on the permutation order of channel measurements, while SASKG aims to align the edited channel measurements between a pair of users for secret key agreement. In both algorithms, one user can preset the secret key and embed encrypted messages in the exchanged data packets, which reduces communication overheads in key generation. Extensive experimental results show that both BM-SKG and SA-SKG algorithms achieve error-free key agreement on channel measurements at a low cost under various scenarios.

Physical-layer key generation has emerged as a promising avenue for establishing secret keys using reciprocal channel measurements between wireless devices. However, channel reciprocity may suffer degradation from ambient noise and cause mismatched secret bits, while existing methods mitigating this issue may yet face limitations in key efficiency. The root cause behind such limitations is the heavy reliance on channel measurements, which can be naturally susceptible to channel non-reciprocity attributed to environmental factors. Instead of direct key extraction from channel measurements, we seek to share a pre-defined key and utilize channel measurements as a bearer to facilitate key transmission. We propose an accurate and efficient key generation method (KeyCome) to ensure secure key sharing by encapsulating it with channel state information (CSI) obfuscation matrices through circulant convolution. To this end, we develop a reliable key derivation through a quadratic programming method with matrix equilibration, ensuring stable and rapid solutions. Notably, the transmitter can control the key beforehand for enhanced communication efficiency and combine it with an error correction mechanism for accurate key derivation. Furthermore, a lightweight reconciliation scheme is designed to minimize mismatched bits caused by occasional non-reciprocity. Comprehensive experiments demonstrate KeyCome's high accuracy and efficiency in key generation.

The two-factor authentication ($2$FA) has drawn increasingly attention as the mobile devices become more prevalent. For example, the user's possession of the enrolled phone could be used by the$2$FA system as the second proof to protect his/her online accounts. Existing$2$FA solutions mainly require some form of user-device interaction, which may severely affect user experience and creates extra burdens to users. In this work, we propose a secure$2$FA system utilizing the proximity of a user's enrolled phone and the login device as the second proof without requiring the user's interactions. The basic idea of our$2$FA system is to derive location signatures based on acoustic beep signals emitted alternately by both devices and sensing the echoes with microphones, and compare the extracted signatures for proximity detection. Moreover, to further enhance the security of our system, we also design a device authentication scheme which derives the acoustic fingerprint between the login device and enrolled phone to verify the identity of two devices. Given the received beep signal, our system designs a period selection scheme to identify two sound segments accurately: the chirp period is the sound segment propagating directly from the speaker to the microphone whereas the echo period is the sound segment reflected back by surrounding objects. To achieve an accurate proximity detection, we develop a new energy loss compensation extraction scheme by utilizing the extracted chirp periods to estimate the intrinsic differences of energy loss between microphones of the enrolled phone and the login device. Our proximity detection component then conducts the similarity comparison between the identified two echo periods after the energy loss compensation to effectively determine whether the enrolled phone and the login device are in proximity for$2$FA. Moreover, to provide higher security, our device fingerprint-assisted proximity detection further utilizes the overall energy loss between the login device and enrolled phone as their hardware fingerprint to authenticate the identity of two devices. Our experimental results show that our system is accurate in providing$2$FA and robust to both man-in-the-middle (MiM) and co-located attacks across different scenarios and device models.

The indoor location awareness has drawn increasing attention as the mobile apps are used extensively in our daily lives. Existing indoor localization solutions either require a pre-installed infrastructure or can only achieve room-level accuracy, which could not provide a function-location service for mobile devices. In this work, we propose a new active sensing system that enables smartphones to identify some pre-defined indoor locations robustly without requiring any additional sensors or pre-installed infrastructure. The main idea behind our system is to utilize the acoustic signatures, which are derived from the mobile device by emitting a beep signal and selecting its echoes created by dominant reflectors, as the robust fingerprint for location identification. Given the microphone samplings, our system designs a correlation based technique to accurately detect the beginning points of echoes from the received beep signal. To achieve a robust location identification, we develop a new echo selection scheme to select echoes created by dominant reflectors by exploiting the relationships between propagation delays of different orders of echoes. To deal with the variable number of selected echoes, our location identification component then derives histograms from selected echoes and uses the one-against-all SVM classifiers to determine the current location. Our experimental results show that our proposed system is accurate and robust for location identification under various real-world scenarios.

The two-factor authentication (2FA) has become pervasive as the mobile devices become prevalent. Existing 2FA solutions usually require some form of user involvement, which could severely affect user experience and bring extra burdens to users. In this work, we propose a secure 2FA that utilizes the individual acoustic fingerprint of the speaker/microphone on enrolled device as the second proof. The main idea behind our system is to use both magnitude and phase fingerprints derived from the frequency response of the enrolled device by emitting acoustic beep signals alternately from both enrolled and login devices and receiving their direct arrivals for 2FA. Given the input microphone samplings, our system designs an arrival time detection scheme to accurately identify the beginning point of the beep signal from the received signal. To achieve a robust authentication, we develop a new distance mitigation scheme to eliminate the impact of transmission distances from the sound propagation model for extracting stable fingerprint in both magnitude and phase domain. Our device authentication component then calculates a weighted correlation value between the device profile and fingerprints extracted from run-time measurements to conduct the device authentication for 2FA. Moreover, to thwart the possible co-located attacks, our proximity detection component further makes the enrolled phone to generate an active random vibration signal by its built-in motor, and then matches the signal received by the microphone of login device with the signal received by the accelerometer of enrolled phone to verify the proximity of two devices. Our experimental results show that our proposed system is accurate and robust to various attacks across different scenarios and device models.

The indoor location awareness has drawn increasing attention for smart speakers as they become essential to provide function-location services. Existing indoor localization solutions either require add-on equipment or could only achieve room-level accuracy, which could not provide a function-location service for smart speakers. In this work, we propose a location identification system utilizing 3-D acoustic images, which are derived from the smart speaker by emitting a beep signal and sensing echoes created by objects in the surrounding environment with its microphone array, as the proof to identify some pre-defined indoor locations. Given the recorded acoustic samplings captured by the microphone array, our image construction component constructs a virtual imaging hemisphere and steers the array towards each grid of the hemisphere to generate a 3-D acoustic image of the surrounding environment. Moreover, we design a transfer-learning based model to derive effective features from the constructed images, and propose a data augmentation scheme for generating synthesized training images. To achieve a more accurate location identification, we further design a distance estimation scheme to identify the distances between the smart speaker and some major surrounding objects by utilizing the constructed 3-D acoustic image, and then adopt such distance information for location identification. Our experimental results show that our proposed system is accurate and robust for location identification under various real world scenarios.

The user authentication has drawn increasingly attention as the smart speaker becomes more prevalent. For example, smart speakers that can verify who is sending voice commands can mitigate various types of attacks such as replay attack or impersonation attack. Existing user authentication solutions either cannot be applicable to smart speakers directly or require certain additional user-device interaction or pre-installed infrastructure, which may severely affect the user experience and create extra burdens to users. In this work, we propose a user authentication system EchoImage utilizing acoustic images, which are derived from the smart speaker by emitting beep signals and sensing echoes from the user's body with its microphone array, as the proof for user authentication. Given the acoustic samplings of the reflected beep signal, our system designs a distance estimation component by applying a correlation based technique on the beamformed signal to estimate the distance between the user and microphone array. Our image construction component then constructs a virtual imaging plane using the estimated distance and steers the array towards each grid of the plane to generate an acoustic image of the user. Moreover, we propose a transfer learning-based method to derive efficient features from the constructed images, and employ SVM classifiers for accurate user authentication. Our extensive experiments demonstrate that our system is robust and accurate across various scenarios.

Personal Identification Number (PIN), as one of the primary means of protecting digital properties and privacy on mobile devices, has been suffering from shoulder surfing attacks and weak password guessing for the long term. Recent years witness the growing interest in two-factor authentication that takes advantage of two different ways for mutual verification, thereby strengthening user authentication's accuracy and reliability. Especially with the popularity of smartwatches, more physiological signals are readily available to facilitate two-factor authentication. This paper presents a lightweight and unobtrusive two-factor authentication scheme, P2Auth, integrating the PIN and unique keystroke-related Photoplethysmography (PPG) measurement on wearables. Specifically, we propose the transformation of the multivariate PPG signal induced by the keystrokes to extract reliable biometric features. We develop short-time energy-based methods to identify the input cases, thus enabling support the authentication for both one-handed and two-handed input cases. Furthermore, we also consider the situation where there is no fixed PIN and design a new enhanced privacy scheme by combining the PPG measurements of different keystrokes to improve authentication security. The experiments involving 15 volunteers demonstrate that our prototype system can achieve an average authentication accuracy of over 95% for one-handed cases and over 90% for two-handed cases.

The two-factor authentication (2FA) has become pervasive as the mobile devices become prevalent. Existing 2FA solutions usually require some form of user involvement, which could severely affect user experience and bring extra burdens to users. In this work, we propose a secure 2FA that utilizes the individual acoustic fingerprint of the speaker/microphone on enrolled device as the second proof. The main idea behind our system is to use both magnitude and phase fingerprints derived from the frequency response of the enrolled device by emitting acoustic beep signals alternately from both enrolled and login devices and receiving their direct arrivals for 2FA. Given the input microphone samplings, our system designs an arrival time detection scheme to accurately identify the beginning point of the beep signal from the received signal. To achieve a robust authentication, we develop a new distance mitigation scheme to eliminate the impact of transmission distances from the sound propagation model for extracting stable fingerprint in both magnitude and phase domain. Our device authentication component then calculates a weighted correlation value between the device profile and fingerprints extracted from run-time measurements to conduct the device authentication for 2FA. Our experimental results show that our proposed system is accurate and robust to both random impersonation and Man-in-the-middle (MiM) attack across different scenarios and device models.

The social trust assessment can spur extensive applications such as social recommendations, shopping, financial investment strategies, etc, but remain a challenging problem having limited exploration. Such explorations mainly limit their studies to static network topology or simplified dynamic networks, toward the social trust relationship prediction. In contrast, in this paper, we explore the social trust by taking into account the time-varying online social networks whereas the social trust relationship may vary over time. The DTrust, a dynamic graph neural network-based solution, will be proposed for accurate social trust prediction. In particular, DTrust is composed of a static aggregation unit and a dynamic unit, respectively responsible for capturing both the spatial dependence features and temporal dependence features. In the former unit, we stack multiple NNConv layers derived from the edge-conditioned convolution network for capturing the spatial dependence features correlated to the network topology and the observed social relationships. In the latter unit, a gated recurrent unit (GRU) is employed for learning the evolution law of social interaction and social trust relationships. Based on the extracted spatial and temporal features, we then employ a fully connected neural network for learning, able to predict the social trust relationships for both current and future time slots. Extensive experimental results exhibit that our DTrust can outperform the benchmark counterparts on two real-world datasets.

The increasingly popular usage of wearable devices provides users with the ability to continuously track their health conditions or physical activities. Such system is however vulnerable to user spoofing, in which a user distributes his/her device to other users such that the data collected from these users could be claimed to be his/her own. Thus, it is critical to identify the user for many wearable devices, allowing the sensing data to be labeled properly. In this paper, we propose a user identification system by leveraging the users whispered sound to mitigate user spoofing for wearable devices. Our system exploits the contact microphone placed into contact with the body to capture the users whispered sound for user identification. Given the captured acoustic data, our system first identifies frames which contain whispered events. Our system then calculates acoustic features from the identified whispered frames to determine whether the voice is collected when the microphone is on the body. Moreover, to make our system robust, we assign different quality weights to the whispers phonemes by considering their consistency (i.e., intra users differences) and distinctiveness (i.e., inter users differences) simultaneously. Our experiments demonstrate that our system is robust and accurate across various scenarios.

PIN inference attack leveraging keystroke-induced side-channel information poses a substantial threat to the security of people’s privacy and properties. Among various PIN inference attacks, video-assisted method provide more intuitive and robust side-channel information to infer PINs. But it usually requires there is no visual occlusion between the attacker and the victims or their hand gestures, making the attackers either easy to expose themselves or inapplicable to the scenarios such as ATM or POS terminals. In this paper, we present a novel and practical video-assisted PIN inference system, ArmSpy, which infers victim’s PIN by observing from behind the victims in a stealthy way. Specifically, ArmSpy explores the subtle keystroke-induced arm posture changes, including elbow bending angle changes and the spatial relationship between different arm joints, to infer the PIN entries. We develop the keystroke inference mechanism to detect the keystroke events and pinpoint the keystroke positions, and then accurately infer the PINs with the proposed inferred PIN coordination mechanism. Extensive experimental results demonstrate that ArmSpy can achieve over 67% average accuracy on inferring the PIN with 3 attempts and even over 80% for some victims, indicating the severity of the threat posed by ArmSpy.

The continuously increasing time spent on car trips in recent years brings growing attention to the physical and mental health of drivers on roads. As one of the key vital signs, the heartbeat is a critical indicator of drivers' health states. Most existing studies on heartbeat monitoring either require sensor attachment or could only provide sketchy heart rates. Moreover, most approaches require the subject to remain stationary or a quiet measuring environment, which is hard to apply to dynamic driving environments. In this paper, we propose a contactless cardiac cycle monitoring system, mmECG, which leverages Commercial-Off-The-Shelf mmWave radar to estimate the fine-grained heart movements of drivers in moving vehicles. By exploring the principle of mmWave signal-based sensing, we first perform studies in static environments and find the fine-grained heart movements, represented as stages of atria and ventricles in repetitive cardiac cycles, can be captured by the FMCW-based mmWave radar as phase changes in signals. Whereas in driving environments, such phase changes are caused and influenced by not only the heartbeat of drivers but also driving operations and vehicle dynamics. To further extract the minute heart movements of drivers and eliminate other influences in phase changes, we construct a movement mixture model to represent the phase changes caused by different movements, and further design a hierarchy variational mode decomposition (VMD) approach to extract and estimate the essential heart movement in mmWave signals. Finally, based on the extracted phase changes, mmECG reconstructs the cardiac cycle by estimating fine-grained movements of atria and ventricles leveraging a template-based optimization method. Experimental results involving 25 drivers in real driving scenarios demonstrate that mmECG can accurately estimate not only heart rates but also cardiac cycles of drivers in real driving environments.

The location awareness becomes increasingly important as mobile devices such as smartphones are used extensively in our daily lives. Existing indoor localization solutions either require certain preinstalled infrastructures or add-on devices, which could not provide a location semantics identification service for smartphones to infer both type and size of a geographic location. In this work, we propose a new active sensing system that enables smartphones to identify its location semantics without requiring any additional infrastructure. The main idea behind our system is to utilize the acoustic signatures, which are derived from the smartphone by emitting a predesigned beep signal and identifying two echo sets which correspond to sidewalls and other static objects respectively, as the proof to achieve both spatial size estimation and room-type prediction simultaneously for indoor location semantics identification. Given the microphone samplings, our system designs a correlation-based scheme to identify beginning points of echoes corresponding to static reflectors accurately from the received signal. To achieve an accurate location semantics identification, we develop a new echo selection scheme to discriminate echoes created by sidewalls and other static reflectors by utilizing the geometrical relationships between the delays of echoes. To deal with the varying number of identified echoes, our location semantics prediction scheme then derives histograms from echo sets and adopt a deep-learning-based classifier to determine the current location semantics. Our experimental results show that our proposed system is accurate and robust for location semantics identification under various real-world scenarios.

Exercise intensity monitoring of physical activities has drawn increasingly attention as the awareness of the exercise intensity is of great importance for a person to achieve optimal training outcomes. For example, over-training could lead to excessive fatigue and loss of motivation for exercise. Traditional exercise intensity monitoring systems utilize GPS data to track the user’s intensity of cardio activities through his/her position and speed. Such systems however become invalid for indoor exercises on stationary fitness equipments such as the treadmill or exercise bike. Recent work in using body-worn sensors to track the user’s heart rate for exercise intensity monitoring usually involves additional wearable sensors which are only available on some particular fitness equipments, and thus are hard to be used in all occasions. This work presents an exercise intensity monitoring system which is capable of detecting a person’s exercise intensity via smartphones. Our system exploits the off-the-shelf smartphone and its headphone to capture the user’s breathing sound. Given the captured acoustic data, our system performs data pre-processing to remove the environmental noise and identify the non-silent acoustic frames based on the signal energy. Our system then conducts breathing event detection for non-silent frames, and further calibrates the detection results by utilizing the high correlation between breathing cycles to improve the detection accuracy. Moreover, our system can estimate the person’s exercise intensity based on features extracted from the frames which contain breathing sound. Our experiments involving 9 subjects over four-month time period demonstrate that our proposed exercise intensity monitoring system is robust and accurate in both indoor and outdoor environments.

The physical layer secret key generation exploiting wireless channel reciprocity has attracted considerable attention in the past two decades. On-going research have demonstrated its viability in various radio frequency (RF) systems. Most of existing work rely on quantization technique to convert channel measurements into digital binaries that are suitable for secret key generation. However, non-simultaneous packet exchanges in time division duplex systems and noise effects in practice usually create random channel measurements between two users, leading to inconsistent quantization results and mismatched secret bits. While significant efforts were spent in recent research to mitigate such non-reciprocity, no efficient method has been found yet. Unlike existing quantization-based approaches, we take a different viewpoint and perform the secret key agreement by solving a bipartite graph matching problem. Specifically, an efficient dual-permutation secret key generation method, DP-SKG, is developed to match the randomly permuted channel measurements between a pair of users by minimizing their discrepancy holistically. DP-SKG allows two users to generate the same secret key based on the permutation order of channel measurements despite the non-reciprocity over wireless channels. Extensive experimental results show that DP-SKG could achieve error-free key agreement on received signal strength (RSS) with a low cost under various scenarios.

The two-factor authentication (2FA) has drawn increasingly attention as the mobile devices become more prevalent. For example, the user's possession of the enrolled phone could be used by the 2FA system as the second proof to protect his/her online accounts. Existing 2FA solutions mainly require some form of user-device interaction, which may severely affect user experience and creates extra burdens to users. In this work, we propose Proximity-Echo, a secure 2FA system utilizing the proximity of a user's enrolled phone and the login device as the second proof without requiring the user's interactions or pre-constructed device fingerprints. The basic idea of Proximity-Echo is to derive location signatures based on acoustic beep signals emitted alternately by both devices and sensing the echoes with microphones, and compare the extracted signatures for proximity detection. Given the received beep signal, our system designs a period selection scheme to identify two sound segments accurately: the chirp period is the sound segment propagating directly from the speaker to the microphone whereas the echo period is the sound segment reflected back by surrounding objects. To achieve an accurate proximity detection, we develop a new energy loss compensation extraction scheme by utilizing the extracted chirp periods to estimate the intrinsic differences of energy loss between microphones of the enrolled phone and the login device. Our proximity detection component then conducts the similarity comparison between the identified two echo periods after the energy loss compensation to effectively determine whether the enrolled phone and the login device are in proximity for 2FA. Our experimental results show that our Proximity-Echo is accurate in providing 2FA and robust to both man-in-the-middle (MiM) and co-located attacks across different scenarios and device models.

Eating well plays a key role in people's overall health and wellbeing. Studies have shown that many health-related problems such as obesity, diabetes and anemia are closely associated with people's unhealthy eating habits (e.g., skipping meals, eating irregularly and overeating). Thus, keeping track of diet is becoming more important. Traditional eating monitoring solutions relying on self-report remain an onerous task, while the recent trends requiring users to wear dedicated yet expensive hardware are cumbersome. To overcome these limitations, in this paper, we develop a device-free eating monitoring system using WiFi-enabled devices (e.g., smartphone or laptop). Our system aims to automatically monitor users' eating activities by identifying the fine-grained eating motions and detecting the minute movements during chewing and swallowing. In particular, our system distinguishes eating from non-eating activities by using K-means clustering with principal component analysis on the extracted Channel State Information (CSI) from WiFi signals. It further adopts a soft decision-based eating motion classification through identifying the utensils (e.g., using a folk, knife, spoon or bare hands) in use. Moreover, we propose a minute motion reconstruction method to identify chewing and swallowing through detecting users' minute facial muscle movements. The derived fine-grained eating monitoring results are beneficial to the understanding of users' eating behaviors and estimation of food intake types and amounts. Extensive experiments with 20 users over 1600-minute eating show that the proposed system can recognize the user's eating motions with up to 95% accuracy and estimate the chewing and swallowing amount within 10% percentage error.

The explosive usage of mobile devices enables conducting electronic transactions involving direct signature on such devices. Thus, user signature verification becomes critical to ensure the success deployment of online transactions such as approving legal documents and authenticating financial transactions. Existing approaches mainly focus on user verification targeting the unlocking of mobile devices or performing continuous verification based on a user's behavioral traits. Few studies provide efficient real-time user signature verification. In this work, we propose a critical segment based online signature verification system to secure mobile transactions on multi-touch mobile devices. Our system identifies and exploits the segments which remain invariant within a user's signature to capture the intrinsic signing behavior embedded in each user's signature. Our system extracts useful features from a user's signature that describe both the geometric layout of the signature as well as behavioral and physiological characteristics in the user's signing process. Given the input signatures for user enrollment, our system further designs a quality score to identify the problematic signature sets to achieve robust user signature profile construction. Moreover, we develop the signature normalization and interpolation methods to achieve robust signature verification in the presence of signature geometric distortions caused by different writing sizes, orientations and locations on touch screens. Our experimental evaluation of 25 subjects over six months time period shows that our system is highly accurate in provide signature verification and robust to signature forging attacks.

Recent years have witnessed tremendous academic efforts and industry growth in machine learning. The security of machine learning has become increasingly prominent. Poisoning attack is one of the most relevant security threats to machine learning which focuses on polluting the training data that machine learning needs during the training process. Specifically, the attacker blends crafted poisoning samples into training data in order to make the learned model beneficial to him. To the best of our knowledge, existing researches about poisoning attack focused on either integrity attack or availability attack, which did not unify these two attacks together. Aside from that, from the attacker's perspective, attacker's strategy is not flexible enough. Finally, existing proposals only concentrated on increasing the test error of the learned model but ignored the importance of the concealment of attack. To overcome these issues, we firstly present a thorough adversarial model for poisoning attack in which attacker's strategy is defined from two aspects, i.e., the effect of attack and the concealment of attack. Then we unify integrity attack and availability attack together in similar formulations. Furthermore, in order to enhance flexibility, a tradeoff parameter is inserted into attacker's objective function which means the attacker can balance the attraction of effect against the requirement of concealment. Finally, as examples, extensive experiments are conducted on linear regression and logistic regression to demonstrate the effectiveness of attack.

Sleep monitoring has drawn increasing attention as sleep quality is important to maintain a person's well-being. For instance, serious health problems, such as cardiovascular disease, fatigue, or depression, are usually associated with inadequate and irregular sleep. Traditional sleep monitoring systems involve wearable sensors with professional installation, and thus are usually limited to clinical usage. Recent work for sleep monitoring can detect several sleep events, such as coughing and snoring, using smartphone sensors. However, such coarse-grained sleep monitoring is unable to detect the breathing rate which is an important health indicator. In this paper, we present a fine-grained sleep monitoring system to detect the breathing rate and sleep events simultaneously by leveraging smartphones. Our system exploits the readily available smartphone earphone placed close to the user to reliably capture the human breathing sound. Given the captured acoustic sound, noise reduction is performed to remove the environmental noise and the breathing rate is then identified based on the signal envelope detection. Our system can further detect some sleep events, including snoring, coughing, turning over, and getting up, based on the features extracted from the acoustic sound. Moreover, we develop a body movement-assisted sleep event detection method to provide higher detection accuracy by further exploiting the user's body movement patterns captured by the accelerometer embedded on smartphones. Our extensive experiments involving nine subjects over six months confirm the effectiveness of our proposed system on breathing rate monitoring and sleep events detection under various environments. By combining breathing rate and sleep events, our system can provide noninvasive and continuous fine-grained sleep monitoring for healthcare related applications, such as sleep apnea monitoring, as evidenced by our experimental study.

Sleep monitoring has drawn increasingly attention as the quality and quantity of the sleep are important to maintain a person's health and well-being. For example, inadequate and irregular sleep are usually associated with serious health problems such as fatigue, depression and cardiovascular disease. Traditional sleep monitoring systems, such as PSG, involve wearable sensors with professional installations, and thus are limited to clinical usage. Recent work in using smartphone sensors for sleep monitoring can detect several events related to sleep, such as body movement, cough and snore. Such coarse-grained sleep monitoring however is unable to detect the breathing rate which is an important vital sign and health indicator. This work presents a fine-grained sleep monitoring system which is capable of detecting the breathing rate by leveraging smartphones. Our system exploits the readily available smartphone earphone placed close to the user to reliably capture the human breathing sound. Given the captured acoustic sound, our system performs noise reduction to remove environmental noise and then identifies the breathing rate based on the signal envelope detection. Our system can further detect detailed sleep events including snore, cough, turn over and get up based on the acoustic features extracted from the acoustic sound. Our experimental evaluation of six subjects over six months time period demonstrates that the breathing rate monitoring and sleep events detection are highly accurate and robust under various environments. By combining breathing rate and sleep events, our system can provide continuous and noninvasive fine-grained sleep monitoring for healthcare related applications, such as sleep apnea monitoring as evidenced by our experimental study.

Motivated by safety challenges resulting from distracted pedestrians, this paper presents a sensing technology for fine-grained location classification in an urban environment. It seeks to detect the transitions from sidewalk locations to in-street locations, to enable applications such as alerting texting pedestrians when they step into the street. In this work, we use shoe-mounted inertial sensors for location classification based on surface gradient profile and step patterns. This approach is different from existing shoe sensing solutions that focus on dead reckoning and inertial navigation. The shoe sensors relay inertial sensor measurements to a smartphone, which extracts the step pattern and the inclination of the ground a pedestrian is walking on. This allows detecting transitions such as stepping over a curb or walking down sidewalk ramps that lead into the street. We carried out walking trials in metropolitan environments in United States (Manhattan) and Europe (Turin). The results from these experiments show that we can accurately determine transitions between sidewalk and street locations to identify pedestrian risk.

This video is a demonstration of the work discussed in our full paper available in the MobiSys'15 proceedings. The video illustrates a sensing technology for fine-grained location classification in an urban environment, for enhancing pedestrian safety. Our system seeks to detect the transitions from sidewalk locations to in-street locations, to enable applications such as alerting texting pedestrians when they step into the street. Existing positioning technologies are not sufficiently precise to allow distinguishing a position on the sidewalk from a position in the street, as explored in our previous work. To this end, we use shoe-mounted inertial sensors for location classification based on surface gradient profile and step patterns. This approach is different from existing shoe sensing solutions that focus on dead reckoning and inertial navigation. The shoe sensors relay inertial sensor measurements to a smartphone, which extracts the step pattern and the inclination of the ground a pedestrian is walking on. This allows detecting transitions such as stepping over a curb or walking down sidewalk ramps that lead into the street. We carried out walking trials in metropolitan environments in United States (Manhattan) and Europe (Turin). The results from these experiments show that we can accurately determine transitions between sidewalk and street locations to identify pedestrian risk.

The rapid deployment of sensing technology in smartphones and the explosion of their usage in people's daily lives provide users with the ability to collectively sense the world. This leads to a growing trend of mobile healthcare systems utilizing sensing data collected from smartphones with/without additional external sensors to analyze and understand people's physical and mental states. However, such healthcare systems are vulnerable to user spoofing, in which an adversary distributes his registered device to other users such that data collected from these users can be claimed as his own to obtain more healthcare benefits and undermine the successful operation of mobile healthcare systems. Existing mitigation approaches either only rely on a secret PIN number (which can not deal with colluded attacks) or require an explicit user action for verification. In this paper, we propose a user verification system leveraging unique gait patterns derived from acceleration readings to detect possible user spoofing in mobile healthcare systems. Our framework exploits the readily available accelerometers embedded within smartphones for user verification. Specifically, our user spoofing mitigation framework (which consists of three components, namely Step Cycle Identification, Step Cycle Interpolation, and Similarity Comparison) is used to extract gait patterns from run-time accelerometer measurements to perform robust user verification under various walking speeds. We show that our framework can be implemented in two ways: user-centric and server-centric, and it is robust to not only random but also mimic attacks. Our extensive experiments using over 3,000 smartphone-based traces with mobile phones placed on different body positions confirm the effectiveness of the proposed framework with users walking at various speeds. This strongly indicates the feasibility of using smartphone based low grade accelerometer to conduct gait recognition and facilitate effective user verification without active user cooperation.

The rapid deployment of cloud computing provides users with the ability to outsource their data to public cloud for economic savings and flexibility. To protect data privacy, users have to encrypt the data before outsourcing to the cloud, which makes the data utilization, such as data retrieval, a challenging task. It is thus desirable to enable the search service over encrypted cloud data for supporting effective and efficient data retrieval over a large number of data users and documents in the cloud. Existing approaches on encrypted cloud data search either focus on single keyword search or become inefficient when a large amount of documents are present, and thus have little support for the efficient multi-keyword search. In this paper, we propose a light-weight search approach that supports efficient multi-keyword ranked search in cloud computing system. Specifically, we first propose a basic scheme using polynomial function to hide the encrypted keyword and search patterns for efficient multi-keyword ranked search. To enhance the search privacy, we propose a privacy-preserving scheme which utilizes the secure inner product method for protecting the privacy of the searched multi-keywords. We analyze the privacy guarantee of our proposed scheme and conduct extensive experiments based on the real-world dataset. The experiment results demonstrate that our scheme can enable the encrypted multi-keyword ranked search service with high efficiency in cloud computing.

Sleep monitoring has drawn increasingly attention as the quality and quantity of the sleep are important for maintaining a person's health and well-being. For example, inadequate and irregular sleep are usually associated with serious health problems such as fatigue, depression and cardiovascular disease. Traditional sleep monitoring systems, such as PSG, involve wearable sensors with professional installations, and thus are limited to clinical usage. Recent work in using smartphone sensors for sleep monitoring can detect several events related to sleep, such as body movement, cough and snore. Such coarse-grained sleep monitoring however is unable to detect the breathing rate which is a vital sign and health indicator. This work presents a fine-grained sleep monitoring system which is capable of detecting the breathing rate by leveraging smartphones. Our system exploits the readily available smartphone earphone that placed close to the user to capture the breath sound reliably. Given the captured acoustic signal, our system performs noise reduction to remove environmental noise and then identifies the breathing rate based on the signal envelope detection. Our experimental evaluation of six subjects over six months time period demonstrates that the breathing rate monitoring is highly accurate and robust under various environments. This strongly indicates the feasibility of using the smartphone and its earphone to perform continuous and noninvasive fine-grained sleep monitoring.

The rapid deployment of sensing technology in smartphones and the explosion of their usage in people's daily lives provide users with the ability to collectively sense the world. This leads to a growing trend of mobile healthcare systems utilizing sensing data collected from smartphones with/without additional external sensors to analyze and understand people's physical and mental states. However, such healthcare systems are vulnerable to user spoofing attacks, in which an adversary distributes his registered device to other users such that data collected from these users can be claimed as his own to obtain more healthcare benefits and undermine the successful operation of mobile healthcare systems. Existing mitigation approaches either only rely on a secret PIN number (which can not deal with colluded attacks) or require an explicit user action for verification. In this paper, we propose a user verification scheme leveraging unique gait patterns derived from acceleration readings in mobile healthcare systems to detect possible user spoofing attacks. Our framework exploits the readily available accelerometers embedded within smartphones for user verification. Specifically, our user spoofing attack mitigation scheme (which consists of three components, namely Step Cycle Identification, Step Cycle Interpolation, and Similarity Score Computation) is used to extract gait patterns from run-time accelerometer measurements to perform robust user verification under various walking speeds. Our experiments using 322 smartphone-based traces over a period of 6 months confirm that our scheme is highly effective for detecting user spoofing attacks. This strongly indicates the feasibility of using smartphone based low grade accelerometer to conduct gait recognition and facilitate effective user verification without active user cooperation.

The inclusion of embedded sensors in mobile phones, and the explosion of their usage in people's daily lives provide users with the ability to collectively sense the world. The collected sensing data from such a mobile phone enabled social network can be mined for users' behaviors and their social communities, and to support a broad range of applications including mobile healthcare systems. However, such mobile healthcare systems built upon social networks are vulnerable to clone attacks, in which the adversary replicates the legitimate nodes and distributes the clones throughout the network to undermine the successful application deployment. Existing clone attack mitigation approaches either only focus on the prevention techniques or can only work in static or well-connected networks, and hence are not applicable to our targeted mobile healthcare systems. In this paper, we propose a social closeness based method in a mobile healthcare disease control system to detect any clone attacks that may be launched to disrupt the normal operations of the system. Our social closeness based method exploits the social relationships among users for clone attack detection. Specifically, we define a new metric called community betweenness, which considers mobile users' community information. We find that the value of this metric changes significantly under the clone attack, which is suitable to be used for clone attack detection. We derive both analytical and training based approaches to determine the threshold setting of the community betweenness for robust clone attack detection. Extensive trace-driven simulation studies reveal that our social closeness based method can detect clone attacks with high detection ratio and low false positive rate.

Community association is an important attribute of a social network because people may belong to varying groups with different characteristics at different times. Traditional community detection approaches often rely on a centralized server and are only useful for offline data analysis. In this paper, we propose and evaluate a distributed community detection approach that allows individual users to detect their own communities based on local observations. Our proposed template- matching method derives dynamic spatial and temporal characteristics of social communities by exploiting human's mobility patterns. Our template matching method allows users with similar moving patterns to be grouped together as one community. Our results using both simulation as well as real experiments demonstrate that our method can detect local communities effectively with high detection rate and low false positive rate.

New mobile phones equipped with multiple sensors provide users with the ability to sense the world at a microscopic level. The collected mobile sensing data can be comprehensive enough to be mined not only for the understanding of human behaviors but also for supporting multiple applications ranging from monitoring/tracking, to medical, emergency and military applications. In this work, we investigate the feasibility and effectiveness of using human contact traces collected from mobile phones to derive social community information to control the disease propagation rate in the healthcare domain. Specifically, we design a community-based framework that extracts the dynamic social community information from human contact based traces to make decisions on who will receive disease alert messages and take vaccination. We have experimentally evaluated our framework via a trace-driven approach by using data sets collected from mobile phones. The results confirmed that our approach of utilizing mobile phone enabled dynamic community information is more effective than existing methods, without utilizing social community information or merely using static community information, at reducing the propagation rate of an infectious disease. This strongly indicates the feasibility of exploiting the social community information derived from mobile sensing data for supporting healthcare related applications.

The Disruption Tolerant Networks (DTNs) are vulnerable to insider attacks, in which the legitimate nodes are compromised and the adversary modifies the delivery metrics of the node to launch harmful attacks in the networks. The traditional detection approaches of secure routing protocols can not address such kind of insider attacks in DTNs. In this paper, we propose a mutual correlation detection scheme (MUTON) for addressing these insider attacks. MUTON takes into consideration of the transitive property when calculating the packet delivery probability of each node and correlates the information collected from other nodes. We evaluated our approach through extensive simulations using both Random Way Point and Zebranet mobility models. Our results show that MUTON can detect insider attacks efficiently with high detection rate and low false positive rate.

The Disruption Tolerant Networks (DTNs) are especially useful in providing mission critical services such as in emergency networks or battlefield scenarios. However, DTNs are vulnerable to insider attacks, in which the legitimate nodes are compromised and the adversary nodes launch blackhole attacks by dropping packets in the networks. The traditional approaches of securing routing protocols can not address such insider attacks in DTNs. In this paper, we propose a method to secure the history records of packet delivery information at each contact so that other nodes can detect insider attacks by analyzing these packet delivery records. We evaluated our approach through extensive simulations using both Random Way Point and Zebranet mobility models. Our results show that our method can detect insider attacks efficiently with high detection rate and low false positive rate.