As wearables continue to gain widespread popularity, ensuring secure and convenient authentication becomes imperative to safeguard the data stored within these devices. However, existing wearable-based authentication solutions often rely on specialized and costly hardware, have limited applicability to specific scenarios, and are vulnerable to permanent biometrics leakages. To address these limitations, this paper explores the uniqueness of hand motions and subtle vibrations associated with face tapping. Specifically, we propose TapPass as a secure and convenient authentication solution that leverages face tapping signals captured via the Inertial Measurement Unit (IMU) in wrist-worn wearables for user authentication. To address significant interference from other body motions, we utilize the energy ratio and duration analysis of IMU measurements, followed by deep learning-based extraction of clean face tapping signals. Additionally, we explore the uniqueness of face tapping signals and extract effective features encompassing motion, vibration, and integral aspects. Based on these features, we generate cancelable biometrics leveraging a linear convolution-based approach, bestowing re-registration capabilities upon traditionally invariant biometrics. This solution effectively eliminates concerns surrounding permanent biometrics leakage and enables accurate authentication in both single-user and multi-user scenarios. Extensive experiments with 24 volunteers over three months demonstrate that TapPass achieves accurate authentication while effectively tackling major attacks and motion interference, all while maintaining user-friendliness.

Blood pressure (BP) is a critical indicator for life-threatening conditions. While invasive catheter-based methods offer high accuracy, non-invasive techniques typically require placement on specific body areas, introducing discomfort and rendering their accuracy sensitive to wearing conditions. To overcome these limitations, recent efforts have explored contactless BP monitoring using RF sensing. However, existing approaches often rely on deep learning models without grounding in physiological principles, resulting in poor generalization and limited clinical trustworthiness. In this paper, we proposehBP-Fi, a contactless BP measurement system driven byhemodynamicsacquired via RF sensing. In addition to its contactless convenience,hBP-Fi outperforms existing RF-based approaches by i) employing a physiologically grounded hemodynamic model of pulse generation that forms the basis for RF-based BP estimation, ii) enabling super-resolution arterial pulse tracking via beam-steerable RF scanning, iii) ensuring output trustworthiness through an interpretable (transparent-by-design) deep learning model, and iv) achieving robust generalizability to unseen users and scenarios via a CycleGAN-based training strategy. Extensive experiments with 35 subjects under practical scenarios demonstrate thathBP-Fi can achieve errors of -2.95$\pm$7.66 mmHg and 2.63$\pm$6.05 mmHg for systolic and diastolic blood pressures, respectively.

Deep learning has significantly advanced Radio Frequency (RF) sensing, leading to extensive research and practical applications in both academia and industry. However, these advancements have also introduced potential privacy and security threats to RF sensing data. In this paper, we present RFInv, the first model inversion attack targeting deep learning classifier-empowered RF sensing systems. RFInv can recover users' private sensing data without knowledge of the RF sensing model's structure, relying solely on the output prediction vector of the deep learning classifier. Consequently, this recovered sensitive data can be exploited for malicious purposes such as identity impersonation and unauthorized device control. To realize the proposed attack, we develop a deep generative adversarial network that integrates an inversion module and a critic module, enabling effective RF data recovery in black-box scenarios. To address the unique challenge of preserving physical consistency in RF data, we incorporate attention mechanisms and deformable convolutions to model their complex temporal and spatial dynamics, ensuring physical consistency. Furthermore, a spectrogram alignment loss is introduced to further enhance reconstruction accuracy. The network is trained using an auxiliary dataset, circumventing the need for access to the target model's training data. We systematically evaluate our proposed attack across multiple datasets for various RF sensing tasks and target models with different network architectures. Extensive experiments demonstrate that RFInv can recover diverse types of RF privacy data with an average Structural Similarity Index Measure (SSIM) of 0.78 and achieves an 86.21% Relative Attack Success Rate (RASR).

Palm vein recognition has gained traction in secure authentication due to its unique, stable, and inherently concealed biometric characteristics. However, the electromagnetic (EM) emissions from subcutaneous vein imaging sensors (SVIS) may unintentionally leak sensitive biometric information, fundamentally challenging the assumed security guarantees. In this paper, we propose VeinPhantom, a novel side-channel attack that reconstructs palm vein patterns from unintended EM emissions of SVIS, ultimately enabling spoofing attacks against biometric authentication systems. To overcome the challenge of low information entropy caused by weak EM signals and complex environmental interference, VeinPhantom first analyzes palm vein information from EM signals, then employs a cascaded enhancement strategy and incorporates a Dynamic Guidance Diffusion framework to progressively reconstruct high-fidelity palm vein patterns. Extensive experiments demonstrate that VeinPhantom achieves an average structure similarity index measure (SSIM) of 0.56 on commercial devices, along with a 56.96% spoofing success rate against state-of-the-art authentication systems. We further discuss potential mitigation strategies to defend against the attack.

As automotive voice assistants (AVAs) become increasingly cen- tral to modern vehicles, their vulnerability to attacks exploiting inaudible sounds should raise security concerns. However, such concerns are often deemed low priority, because it is widely be- lieved that an attacker to AVAs should be strategically positioned inside the concerned vehicle for two main reasons: i) inaudible signals can barely penetrate vehicle hulls and ii) a line-of-sight (LoS) path is needed between the attacker (sound source) and the AVA's microphone. In this paper, we disprove this common belief by proposing ShieldSpear to launch AVA attacks outside vehicle hulls. ShieldSpear exploits a tiny piezo-element placed on the exterior of the windshield to convert it into both a speaker and microphone. While this setting naturally brings the attacking sound source into a vehicle, strategically placing this compactly integrated element may further yield i) covertness (blended into stickers), ii) LoS path to AVA's microphones, and iii) real-time attacking capability dur- ing vehicle motion. To maintain sufficient volume while evading detection, we design novel hardware and signal carriers for deliver- ing attack (voice) commands. Moreover, ShieldSpear leverages the windshield-converted microphone to acquire drivers' voiceprint so as to accurately emulate it in the faked commands. Extensive experiments involving five mainstream vehicles have demonstrated the effectiveness of ShieldSpear by a 90.9% end-to-end success rate in injecting faked voice commands into AVAs.

As a crucial tool for assessing health, spirometry provides valuable insights into pulmonary functions. Recent advancements have enabled more convenient measurements by shifting spirometry solutions from cumbersome clinical devices to portable devices. However, the forced maneuvers and burdensome procedures, which necessitate repeated maximal forced breathing, often lead to dizziness and discomfort, rendering them unsuitable for vulnerable populations. In this paper, we present ESPIRO (Earphone-enabled Speech sPIROmetry) system to furnish user-friendly pulmonary function monitoring for diverse populations. Basically, ESPIRO records normal speech using microphone-embedded earphones and characterizes pulmonary function-related glottal flow during speech production. ESPIRO advances existing spirometry solutions in i) leveraging phonetics to associate pulmonary function with glottal flow in normal speech, thereby eliminating the need for forced breathing; ii) identifying effective speech features according to physiological basis, ensuring reliable spirometry measurements; and iii) effectively addressing ambient noise, making it suitable for various real-world settings. Extensive experiments with 38 subjects on 18 commodity earphones confirm that ESPIRO accurately estimates pulmonary function indices in practice.

Biometrics has been increasingly integrated into wearables for enhanced data security in recent years. Meanwhile, wearable popularity offers a unique chance to capture novel biometrics via embedded sensors. In this article, we study new intracorporal biometrics combining the uniqueness of heart motion, bone conduction, and body asymmetry. Specifically, we introduce HeartPrint, a passive yet secure user authentication system exploiting the bone-conducted heart sounds captured by (widely available) dualin-ear microphones (IEMs). To eliminate interference, we devise a novel method combining modified non-negative matrix factorization and adaptive filtering. This extracts clean heart sounds while addressing interference of body sounds and audio produced by the earphones. We further explore the uniqueness of IEM-recorded heart sounds in three aspects to extract a novel biometric representation, based on which HeartPrint leverages a convolutional neural model equipped with a continual learning method to achieve accurate authentication under drifting body conditions. Furthermore, user-friendly registration and energy-effective authentication are facilitated by a data augmentation method using transformer-based GAN and an authentication interval control method. Extensive experiments with 45 participants confirm that HeartPrint can achieve 1.6% FAR and 1.8% FRR, while effectively coping with major attacks, complicated interference, and hardware diversity, while exhibiting robustness in real-world environments.

Free weight training (FWT) is of utmost importance for physical well-being. The success of FWT depends largely on choosing the suitable workload, as improper selections can lead to suboptimal outcomes or injury. Current workload estimation approaches rely on manual recording and specialized equipment with limited feedback. Therefore, we introducePPGSpotter, a wearable PPG-based FWT monitoring system in a convenient, low-cost, and fine-grained manner. By characterizing the arterial geometry compressions caused by the deformation of distinct muscle groups,PPGSpottercan infer essential FWT factors such as current workload, repetitions, and exercise type and provide recommendations for workload adjustment. To remove pulse-related interference, we develop an arterial interference elimination approach based on adaptive filtering, effectively extracting the pure motion-derived signal (MDS). Furthermore, we explore 2D representations of MDS within the phase space to extract spatiotemporal information, enablingPPGSpotterto address the challenge of resisting sensor shifts. Finally, we leverage a multi-task CNN-based network and workload adjustment guidance to achieve personalized FWT monitoring. Extensive experiments with 15 participants confirm thatPPGSpottercan achieve promising workload estimation (0.59 kg RMSE), repetitions estimation (0.96 reps RMSE), and exercise type recognition (91.57% F1-score) while providing valid workload adjustment recommendations (0.22 kg RMSE).

Blood pressure (BP) measurement is significant to the assessment of many dangerous health conditions. Apart from invasively inserting catheters into arteries, non-invasive approaches typically rely on wearing devices on specific skin areas with consistent pressure. However, this can be uncomfortable and unsuitable for certain individuals, and the accuracy of these methods may significantly decrease due to improper device placements and wearing states. Recently, contactless methods leveraging RF technology have emerged as a potential alternative. However, these methods suffer from the drawback of overfitting deep learning (DL) models without a sound physiological basis, resulting in a lack of clear explanations for their outputs. Consequently, such limitations lead to skepticism and distrust among medical experts. In this paper, we propose hBP-Fi, a contactless BP measurement system driven by hemodynamics acquired via RF sensing. In addition to its contactless convenience, hBP-Fi is superior to other RF sensing approaches in i) grounding on hemodynamics as the key physical process of heart-pulse activities, ii) exploiting beam-steerable RF devices to achieve a super-resolution scan on the fine-grained pulse activities along arm arteries, and iii) ensuring the trustworthiness of system outputs via an explainable (decision-understandable) DL model. Extensive experiments with 35 subjects demonstrate that hBP-Fi can achieve the error of -2.05±6.83 mmHg and 1.99 ± 6.30 mmHg for monitoring systolic and diastolic blood pressures, respectively.

Free weight training (FWT) is of utmost importance for physical well-being. However, the success of FWT depends on choosing the suitable workload, as improper selections can lead to suboptimal outcomes or injury. Current workload estimation approaches rely on manual recording and specialized equipment with limited feedback. Therefore, we introduce PPGSpotter, a novel PPG-based system for FWT monitoring in a convenient, low-cost, and fine-grained manner. By characterizing the arterial geometry compressions caused by the deformation of distinct muscle groups during various exercises and workloads in PPG signals, PPGSpotter can infer essential FWT factors such as workload, repetitions, and exercise type. To remove pulse-related interference that heavily contaminates PPG signals, we develop an arterial interference elimination approach based on adaptive filtering, effectively extracting the pure motion-derived signal (MDS). Furthermore, we explore 2D representations within the phase space of MDS to extract spatiotemporal information, enabling PPGSpotter to address the challenge of resisting sensor shifts. Finally, we leverage a multi-task CNN-based model with workload adjustment guidance to achieve personalized FWT monitoring. Extensive experiments with 15 participants confirm that PPGSpotter can achieve workload estimation (0.59 kg RMSE), repetitions estimation (0.96 reps RMSE), and exercise type recognition (91.57% F1-score) while providing valid workload adjustment recommendations.

Past tongue-jaw movement interaction systems typically require dedicated hardware and are uncomfortable to use, limiting their scalability and generalizability. This paper introducesCanalScan, the first system that recognizes tongue-jaw movements using commodity speakers and microphones mounted on ubiquitous off-the-shelf devices (e.g., smartphones). What inspires us is that tongue-jaw movements always cause ear canal deformations, and we find that for different tongue-jaw movements, dynamic features of ear canal deformations present unique patterns on acoustic reflections in the ear canal. Specifically,CanalScanfirst sends an acoustic signal to the ear canal, then parses the reflection signals for tongue-jaw movements recognition. To eliminate the impacts of body movements, we develop a body movement noise filtering method and a dynamic segmentation method to identify and separate the tongue-jaw movements-associated ear canal deformations from other types of body movements. We further propose a sensor position detection method and a data transformation mechanism to reduce the impacts of diversities in-ear canal shapes and relative positions between sensors and the ear canal.CanalScanexplores twelve unique and consistent features and applies a random forest classifier to distinguish tongue-jaw movements. Extensive experiments with twenty participants validate the generalizability, effectiveness, robustness, and high accuracy ofCanalScan.

Recent literature advances motion sensors mounted on smartphones and AR/VR headsets to speech eavesdropping due to their sensitivity to subtle vibrations. The popularity of motion sensors in earphones has fueled a rise in their sampling rate, which enables various enhanced features. This paper investigates a new threat of eavesdropping via motion sensors of earphones by developing EarSpy, which builds on our observation that the earphone's accelerometer can capture bone conduction vibrations (BCVs) and ear canal dynamic motions (ECDMs) associated with speaking; they enable EarSpy to derive unique information about the wearer's speech. Leveraging a study on the motion sensor measurements captured from earphones, EarSpy gains abilities to disentangle the wearer's live speech from interference caused by body motions and vibrations generated when the earphone's speaker plays audio. To enable user-independent attacks, EarSpy involves novel efforts, including a trajectory instability reduction method to calibrate the waveform of ECDMs and a data augmentation method to enrich the diversity of BCVs. Moreover, EarSpy explores effective representations from BCVs and ECDMs, and develops a neural network model with character-level and word-level speech recognition models to realize speech recognition. Extensive experiments involving 14 participants demonstrate that EarSpy reaches a promising recognition for the wearer's speech.

With the ever-increasing demand for improving communication and independence for visually impaired people, automatic Braille recognition has gained increasing attention in facilitating Braille learning and reading. However, current approaches mainly require high-cost hardware, involve inconvenient operation, and disturb the normal touch function. In this paper, we proposeBrailleReaderas a low-cost and effortless Braille character recognition system without disturbing normal Braille touching. It exploits the wrist motion of Braille reading captured by the motion sensor available in the ubiquitous wrist-worn device to infer the encoded character information. To address the noise caused by other body and hand movements, we propose a novel noise cancellation method using the wavelet packet decomposition and reconstruction technique to separate clean wrist movement induced by the Braille dot. Moreover, we further explore the unique wrist movement pattern in three aspects to extract a novel and effective feature set. Based on this,BrailleReaderleverages a spiking neural network-based model to robustly recognize Braille characters across different people and different surface materials. Extensive experiments with 48 participants demonstrate thatBrailleReadercan perform accurate and robust recognition of 26 Braille characters.

Recent literature advances motion sensors mounted on smartphones and AR/VR headsets to speech eavesdropping due to their sensitivity to subtle vibrations. The popularity of motion sensors in earphones has fueled a rise in their sampling rate, which enables various enhanced features. This paper investigates a new threat of eavesdropping via motion sensors of earphones by developing EarSpy, which builds on our observation that the earphone’s accelerometer can capture bone conduction vibrations (BCVs) and ear canal dynamic motions (ECDMs) associated with speaking; they enable EarSpy to derive unique information about the wearer’s speech. Leveraging a study on the motion sensor measurements captured from earphones, EarSpy gains abilities to disentangle the wearer’s live speech from interference caused by body motions and vibrations generated when the earphone’s speaker plays audio. To enable user-independent attacks, EarSpy involves novel efforts, including a trajectory instability reduction method to calibrate the waveform of ECDMs and a data augmentation method to enrich the diversity of BCVs. Moreover, EarSpy explores effective representations from BCVs and ECDMs, and develops a convolutional neural model with Connectionist Temporal Classification (CTC) to realize accurate speech recognition. Extensive experiments involving 14 participants demonstrate that EarSpy reaches a promising recognition for the wearer’s speech.

Proper handwashing, having a crucial effect on reducing bacteria, serves as the cornerstone of hand hygiene. For elders with dementia, they suffer from a gradual loss of memory and difficulty coordinating handwashing steps. Proper assistance should be provided to them to ensure their hand hygiene adherence. Toward this end, we propose AWash, leveraging inertial measurement unit (IMU) readily available in most wrist-worn devices (e.g., smartwatches) to characterize handwashing actions and provide assistance. To monitor handwashing scenarios round-the-clock while achieving energy efficiency, we design methods that distinguish handwashing from other daily activities and dynamically adjust the sampling duty cycle. Upon detecting handwashing actions, we design several novel techniques to segment different handwashing actions and extract sensor-body inclination angles that handle particular interference of senile dementia patients. Moreover, a user-independent network model is built to recognize the handwashing actions of senile dementia patients without requiring their training data. Furthermore, we propose a transfer learning method that improves system performance. To meet users’ diverse needs, we use a state machine to make prompt decisions, supporting customized assistance. Extensive experiments on a prototype with eight older participants demonstrate that AWash can increase the user’s independence in the execution of handwashing.

Mobile devices are promising to apply two-factor authentication to improve system security. Existing solutions have certain limits of requiring extra user effort, which might seriously affect user experience and delay authentication time. In this paper, we propose PPGPass, a novel mobile two-factor authentication system, which leverages Photoplethysmography (PPG) sensors available in most wrist-worn wearables. PPGPass simultaneously performs a password/pattern/signature authentication and a physiological-based authentication. To realize both nonintrusive and secure, we design a two-stage algorithm to separate clean heartbeat signals from PPG signals contaminated by motion artifacts so that users do not have to deliberately keep their bodies still. In addition, to deal with noncancelable issues when biometrics are compromised, we design a repeatable and non-invertible method to generate cancelable feature templates as alternative credentials. We leverage the great power ofRandom ForestandSupport Vector Data Descriptionto detect adversaries and verify a user's identity. To the best of our knowledge, PPGPass is the first nonintrusive and secure mobile two-factor authentication based on PPG sensors. Extensive experiments demonstrate that PPGPass can achieve the false acceptance rate of 3.11% and the false recognition rate of 3.71%, which confirms its high effectiveness, security, and usability.

Driver drowsiness is a significant factor in road crashes. However, existing solutions for driver drowsiness detection have major drawbacks of requiring special hardware, constrained recording conditions, and cannot handle the asynchronous and contradictory nature of multiple indicators. In view of this, we propose FDWatch, a novel drowsiness detection system that exploits the low-cost Photoplethysmogram (PPG) sensor and motion sensor integrated into wrist-worn devices. We design a set of novel algorithms to extract multiple drowsiness-related indicators covering major categories of human factors. In particular, we demonstrate that commodity PPG sensors can be utilized to detect yawning behavior; it contributes as an important indicator for drowsiness detection. The core of FDWatch is based on the Dempster-Shafer evidence theory. It considers different indicators as evidence describing the state of the driver from different angles. To make the extracted indicators applicable to Dempster-Shafer evidence theory, we employ backpropagation neural networks to obtain the basic probability assignment. Moreover, we propose a similarity-distance-based method to handle evidence conflicts. Extensive experiments with real-road driving data show that FDWatch can accurately detect driver drowsiness with a missing alarm rate of 3.57% and a false alarm rate of 3.68%.

Handwritten signature authentication is an important service to defend against fraudulent activities. Current automated solutions rely heavily on dedicated devices and require certain user efforts. In this work, we explore the feasibility of a new type of signature authentication system, SAP - Signature Authentication with PPG Sensor, which leverages Photoplethysmography (PPG) sensors in wrist-worn wearable devices. To make SAP non-intrusive and secure, we design effective algorithms to separate the signature signals from the heartbeat signals in the raw PPG signals. We implement a low-cost hardware prototype of SAP. Our preliminary experimental results show that SAP can achieve an average F1 score of up to 98%.

Handwritten signature authentication is a crucial service to defend against fraudulent activities. Existing automated solutions rely heavily on dedicated devices that are expensive and require different user efforts that affect the user experience. In this paper, we propose a new signature authentication system, PPGSign, which leverages Photoplethysmography (PPG) sensors in the existing wrist-worn wearable devices. The unique blood flow changes in the supplicant’s hand movement are exploited in this system to validate the signature. To make PPGSign nonintrusive and secure, we explore effective algorithms to separate the signature signals from the heartbeat signals in the raw PPG signals. We build a low-cost hardware prototype to verify our proposed method. Our experimental results show that PPGSign can achieve an average F1 score of up to 98%, which verifies the feasibility and efficiency of the proposed solution.

Hand hygiene has a significant impact on human health. Proper handwashing, having a crucial effect on reducing bacteria, serves as the cornerstone of hand hygiene. For the elder with dementia, they suffer from a gradual loss of memory and difficulty in coordinating steps in the execution of handwashing. Proper assistance should be provided to them to ensure their hand hygiene adherence. Toward this end, we propose AWash, leveraging only commodity IMU sensor mounted on most wrist-worn devices (e.g., smartwatches) to characterize hand motions and provide assistance accordingly. To handle particular interference of senile dementia patients in IMU sensor readings, we design a number of effective techniques to segment handwashing actions, transform sensory input to body coordinate system, and extract sensor-body inclination angles. A hybrid neural network model is used to enable AWash to generalize to new users without retraining or adaptation, avoiding the trouble of collecting behavior information of every user. To meet the diverse needs of users with various executive functioning, we use a state machine to make prompt decisions, which supports customized assistance. Extensive experiments on a prototype with eight older participants demonstrate that AWash can increase the user's independence in the execution of handwashing.

Human-machine interface based on tongue-jaw movements has recently become one of the major technological trends. However, existing schemes have several limitations, such as requiring dedicated hardware and are usually uncomfortable to wear. This paper presents CanalScan, a nonintrusive system for tongue-jaw movement recognition using only commodity speaker and microphone mounted on ubiquitous off-the-shelf devices (e.g., smartphones). The basic idea is to send an acoustic signal, then captures its reflections and derive unique patterns of ear canal deformation caused by tongue-jaw movements. A dynamic segmentation method with Support Vector Domain Description is used to segment tongue-jaw movements. To combat sensor position-sensitive deficiency and ear-canal-shape-sensitive deficiency in multi-path reflections, we first design algorithms to assist users in adjusting the acoustic sensors to the same valid zone. Then we propose a data transformation mechanism to reduce the impacts of diversities in ear canal shapes and relative positions between sensors and the ear canal. CanalScan explores twelve unique and consistent features and applies a Random Forest classifier to distinguish tongue-jaw movements. Extensive experiments with twenty participants demonstrate that CanalScan achieves promising recognition for six tongue-jaw movements, is robust against various usage scenarios, and can be generalized to new users without retraining and adaptation.

Arterial blood pressure (ABP) monitoring using wearables has emerged as a promising approach to empower users with self-monitoring for effective diagnosis and control of hypertension. However, existing schemes mainly monitor ABP at discrete time intervals, involve some form of user effort, have insufficient accuracy, and require collecting sufficient training data for model development. To tackle these problems, we propose Crisp-BP, a novel ABP monitoring system leveraging the PPG sensor available in commercial wrist-worn devices (e.g., smartwatches or fitness trackers). It enables continuous, accurate, user-independent ABP monitoring and requires no behavior changes during collecting PPG data. The basic idea is to illuminate a skin/tissue, measure the light absorption, and characterize ABP-related blood volume change in the artery. To obtain accurate measurements and relieve the pain of training data collection, we use an arterial pulse extraction method that removes interference caused by capillary pulses. Moreover, we design a contact pressure estimation method to combat the deficiency of PPG waveform being sensitive to the contact pressure between the sensor and the skin. In addition, we leverage the great power of Bidirectional Long Short Term Memory and design a hybrid neural network model to enable user-independent ABP monitoring, so that users do not have to provide training data for model development. Furthermore, we propose a transfer learning method that first extracts general knowledge from online PPG data, then use it to improve the learning of a new model on our target problem. Extensive experiments with 35 participants demonstrate that Crisp-BP obtains the average estimation error of 0.86 mmHg and 1.67 mmHg and the standard deviation error of 6.55 mmHg and 7.31 mmHg for diastolic pressure and systolic pressure, respectively. These errors are within the acceptable range regulated by the FDA's AAMI protocol, which allows average errors of up to 5 mmHg and a standard deviation of up to 8 mmHg. Our results demonstrate that Crisp-BP is promising for improving the diagnosis and control of hypertension as it provides continuousness, comfort, convenience, and accuracy.

Micro finger gesture recognition is an emerging approach to realize more friendly interaction between human and smart devices, especially for small wearable devices, such as smartwatches and virtual reality glasses. This paper proposes airFinger, a novel solution utilizing NIR light sensing to realize both real-time gesture recognition and finger tracking aiming at micro finger gestures. Using a custom NIR-based sensor with novel algorithms to capture subtle finger movements, airFinger enables to detect a rich set of micro finger gestures and track finger movements in terms of scrolling direction, velocity, and displacement. Besides, airFinger is capable of effective noise mitigation, gesture segmentation, and reducing false recognition due to the unintentional actions of users. Extensive experimental results demonstrate that airFinger has robustness against individual diversity, gesture inconsistency, and many other impacts. The overall performance reaches an average accuracy as high as 98.72% over a set of 8 micro finger gestures among 10, 000 gesture samples collected from 10 volunteers.

Mobile devices are promising to apply two-factor authentication in order to improve system security and enhance user privacy-preserving. Existing solutions usually have certain limits of requiring some form of user effort, which might seriously affect user experience and delay authentication time. In this paper, we propose PPGPass, a novel mobile two-factor authentication system, which leverages Photoplethysmography (PPG) sensors in wrist-worn wearables to extract individual characteristics of PPG signals. In order to realize both nonintrusive and secure, we design a two-stage algorithm to separate clean heartbeat signals from PPG signals contaminated by motion artifacts, which allows verifying users without intentionally staying still during the process of authentication. In addition, to deal with non-cancelable issues when biometrics are compromised, we design a repeatable and non-invertible method to generate cancelable feature templates as alternative credentials, which enables to defense against man-in-the-middle attacks and replay attacks. To the best of our knowledge, PPGPass is the first nonintrusive and secure mobile two-factor authentication based on PPG sensors in wearables. We build a prototype of PPGPass and conduct the system with comprehensive experiments involving multiple participants. PPGPass can achieve an average F1 score of 95.3%, which confirms its high effectiveness, security, and usability.