Due to the widespread use of mobile devices, it is essential to authenticate users on mobile devices to prevent sensitive information leakage. Biometrics-based authentication is prevalent on smart devices to verify the legitimacy of users, but is vulnerable to replay attacks. In this paper, we propose to leverage the distinctive finger tap gesture during unlocking smartphone to establish a secure multi-factor authentication system, named FingerVib. Compared with other biometric-based authentication systems, FingerVib does not require users to remember any complicated information (e.g., hand gestures, doodles) and the working type is unobtrusive. When users unlock their phones by tapping, FingerVib utilizes the microphone to record the sound produced by fingers tapping on the phone and adopts IMU (Inertial Measurement Unit) to extract the vibration of users’ smartphones. One key contribution is that we model the inherent correlation between sounds and vibration signals. Specifically, FingerVib captures two novel reactions to describe how the individual’s contact palm modulates signals in two different domains. Based on these two responses, we develop a real-time noise-resistant unlocking activity detection algorithm, which allows accurate unlocking signal segmentation even if the two modalities are interfered. Further, we develop a modal fusion model where the model extracts cross-modal features and acquires inter-modal correlation features to ensure consistent performance of inference even when modalities are disturbed. In a user study with 41 participants, FingerVib achieves an authentication accuracy of 98.53% and an average performance of 1.36% FAR, 2.76% FRR and 2.72% EER against replay attacks and impersonation attacks. FingerVib’s fusion approach improves identification performance by roughly 9.7% and 11.6% over Wavocie and AUDIOIMU, respectively, within existing multi-modal fusion systems. Extensive experimental results demonstrate the effectiveness and robustness of FingerVib under various conditions.

Blood pressure (BP) is a critical physiological parameter closely associated with severe diseases such as heart failure and kidney damage. Current methods either require additional or dedicated hardware, or closing touching to the devices, causing discomfort and inconvenience. Therefore, a convenient, contactless BP measurement solution is highly desired. In this work, we present Ubicon-BP, a ubiquitous, device-free, and contactless BP detection application. Ubicon-BP calculates BP based on the pulse transit time (PTT), a key feature that is medically proven correlated with BP. However, using smartphone sensors to contactless calculate PTT is non-trivial since it requires a micro-second level precision for cardiac event detection. To address this issue, we propose leveraging the acoustic sensors in smartphone to detect vibrations caused by heart valve movements, as well as camera sensors to measure finger pulses. To accurately measure heartbeat signal that are susceptible to motion, we first improve the sensing granularity of acoustic signals and then introduce the IQ-MVED model to eliminate motion interference. Furthermore, when recovering pulse signals from video signals, issues such as poor generalization performance arise. Consequently, we propose the TS-CAN and meta-learning models to obtain personalized pulse signals. Finally, we transform the extracted time-frequency features from the recovered heartbeats and pulse signals to the corresponding BP. Comprehensive testing involving 50 subjects reveal a standard deviation error of$ 4.27 \; \text{mmHg}$for diastolic pressure and$ 6.36 \; \text{mmHg}$for systolic pressure, respectively.

The interaction mode of smart eyewear has garnered significant research attention. Most smart eyewear relies on touchpads for user interaction. This article identifies a drawback arising from the use of touchpads, which can be obtrusive and unfriendly to users. In this article, we propose TeethFa, a novel fabric sensor-based system for recognizing teeth gestures. TeethFa serves as a hands-free interaction method for smart eyewear. TeethFa utilizes fabric sensors embedded in the glasses frame to capture pressure changes induced by facial muscle movements linked to teeth movements. This enables the identification of subtle teeth gestures. To detect teeth gestures, TeethFa designs a novel template-based signal segmentation method to determine the boundary of teeth gestures from fabric sensors, even in the presence of motion interference. To improve TeethFa’s generalization, we employ a meta-learning technique based on generalization adjustment to extend the model to new users. We conduct extensive experiments to assess TeethFa’s performance on 30 volunteers. The results demonstrate that our system accurately identifies five different teeth gestures with an average accuracy of 93.57%, and even for new users, the accuracy can reach 89.58%. TeethFa shows promise in offering a new interaction paradigm for smart eyewear in the future.