Benefiting from smartphones' powerful computing and sensing capabilities, biometric authentication is widely applied to them for conveniently verifying users' identities. However, most biometric features can be easily acquired or reproduced, making them vulnerable to replay and impersonation attacks. To address this issue, we propose DBreathLock, a non-contact deep breath-based authentication system that utilizes a smartphone emitting inaudible frequency-modulated continuous waves (FMCW)-based sonar signals and synchronously records breath sounds and sonar echoes of chest-abdominal-joint (C-A-joint) movements. Then, we implement a dual-protection barrier to defend against advanced replay attacks (ARAs). First, by analyzing the energy features of C-A-joint movements, we develop a Deep Breath Activity Detection method to detect deep breath fragments alongside the capability of resisting ARAs. Second, we take C-A-joint movements and smartphone vibrations caused by holding a smartphone as features and design a liveness detection mechanism to further fortify the resistance to ARAs. Furthermore, a multi-stream identity authentication model is designed to verify legitimate users by fusing biometric features from C-A-joint movements, deep breath sounds, and correlation sequences of both. Extensive real-world experiments with 40 users demonstrate DBreathLock's authentication accuracy of 95.97%. Additionally, it successfully defends against advanced replay, impersonation, simple hybrid, and advanced hybrid attacks, achieving the AUC of 0.9792, and FPRs of 2.17%, 2%, and 4.17%, respectively.

Frequency Modulated Continuous Wave (FMCW)-based mmWave radar has attracted widespread attention because of its non-contact and high spatial resolution for vital signs monitoring. Meanwhile, current studies focus mainly on how to improve the detection performance of steady multiple objects or unsteady single objects. In this work, we propose an innovative method for identity-based multi-object vital signs monitoring under unsteady scenarios. The method automatically distinguishes between steady and motion states, and conducts a best-effort vital signs monitoring during unsteady scenarios. To this end, we design a weight vector enhancement method combined with object spatial positioning for differentiating multiple objects, and identify each object according to the gait-based EfficientNet model. We also design a steady-state detector based on the MobileNet-V2 network to find the slots of object keeping steady for vital signs monitoring and then apply the variational mode decomposition (VMD) algorithm to extract the respiratory and heart rates of a single object. The experimental results showed that the mean absolute error of respiratory rate and heart rate decreased to 1.37 bpm and 2.56 bpm respectively in the case of multiple objects. In addition, the steady-state detector achieves close to 98.1% accuracy in recognizing motion types, and the average recognition rate of identity recognition based on gait features reaches about 93.26%.

Frequency Modulated Continuous Wave (FMCW)-based mmWave radar has attracted widespread attention due to its non-contact and high spatial resolution for multi-target vital signs monitoring. However, current works mostly focus on how to improve detection performance under the steady scenarios with a single target, while unsteady scenarios and physical mutual interference from multiple targets are rarely considered. In this work, we propose an innovative method for multi-target vital signs for unsteady scenarios, such as aerobic exercise. The method automatically distinguishes between steady state and motion state, and completes a best-effort vital signs detection during unsteady state. To differentiate multiple targets, we design a weight vector enhancement method combined with target space localization, and then, apply the variational mode decomposition (VMD) algorithm to extract the respiratory and heart rates of a single target. Moreover, for evaluating the efforts of exercise, we propose Multi-target Motion Recognition (MMR) based on MobileNet-V2 network to recognize the motion states of multiple targets. The experimental results showed that the mean absolute error of respiratory rate and heart rate decreased to 1.37 bpm and 2.56 bpm, respectively. Meanwhile, the MMR algorithm achieves close to 98.1% accuracy in recognizing motion states.

Benefiting from smartphones’ powerful computing and sensing capabilities, biometric authentication is widely used on them to conveniently verify user’s identity. However, most biometric features can be easily acquired or reproduced, making them vulnerable to replay and impersonation attacks. To address these issues, we propose DBreathLock, a non-contact identity authentication system that leverages a smartphone to capture unique biometric features from users’ deep breaths. Specifically, the smartphone emits inaudible frequency-modulated continuous waves (FMCW)-based sonar signals to capture chest-abdominal-joint (C-A-joint) movements, increasing the difficulty of impersonating legitimate users. By analyzing the energy features of CA-joint movements, we develop a Deep Breath Activity Detection (DBAD) method to detect deep breath fragments alongside the capability of resisting advanced replay attacks. To further fortify the resistance to advanced replay attacks, we employ the morphological features of C-A-joint movements and SVC model to recognize these attacks. Following this, we construct mutual information (MI) sequences via calculating the correlation between C-A-joint movements and breath sounds to reduce identity authentication failure caused by physiological changes. Then, a multi-stream identity authentication model is designed to verify legitimate users by fusing the features from C-A-joint movements, deep breath sounds, and MI sequences. Extensive real-world experiments involving 20 users demonstrate that DBreathLock achieves an authentication accuracy of 98.33%. Additionally, it successfully defends against both advanced replay and impersonation attacks.

In product quality monitoring, surface defect detection (SDD) is an important part related to the appearance and performance of the product. Machine vision-based SDD usually needs to construct a product-specific detection model and manually label the dataset for training model by paid labeler. We try to transform the single labeler’s work into collaborative labeling works of multiple Internet users to reduce the labeling cost, and propose CrowdLab, a crowd-sourcing dataset labeling system for SDD. In CrowdLab, we tactfully segment the product surface image and employ verification image in web login to make multiple users collaboratively select the part of defect regions and finally recognize these regions. The experiment results illustrate that SDD model using CrowdLab dataset can achieve 96.7% accuracy, which is close to the personnel work.

Simultaneous localization and mapping (SLAM) using an RGB-D camera is a key enabling technique for many augmented reality (AR) applications. However, most existing RGB-D SLAM methods could fail in dynamic scenarios due to non-trivial pose estimation errors arising from moving objects. In this study, we present an accurate and robust RGB-D SLAM system for dynamic scenarios which can run real-time on a single dual-core CPU. The core of our system is a robust and efficient dynamic keypoint exclusion method which consists of three steps: 1) grouping spatially and appearance related pixels of a keyframe into regions; 2) identifying dynamic regions by checking motion consistency of keypoints in every region; 3) excluding keypoints in the identified dynamic regions as well as the matching points in the 3D local map. The dynamic keypoint exclusion method can be easily integrated into any keypoint based RGB-D SLAM system for improving the accuracy and robustness in dynamic scenes with trivial time increase (16.6ms per frame). Experimental results on the TUM dataset demonstrates that our method which runs on an Intel i7-4900 CPU is even 2.3X faster than the state-of-the-art method DS-SLAM [1] which runs parallel on a P4000 GPU and a comparable CPU. In addition, our system outperforms the state-of-the-art methods [1]–[4] in terms of smaller absolute trajectory errors (ATE). We also apply our system to a real AR application and live experiments with a hand-held RGB-D camera demonstrate the robustness and generalizability of our method in practical scenarios.11A demo video is provided onhttps://github.com/cc-qy/Dynamic-RGB-D-SLAM

Mobile devices equipped with a monocular camera and an inertial measurement unit (IMU) are ideal platforms for augmented reality (AR) applications. However, nontrivial noises in low-cost IMUs, which are usually equipped in consumer-level mobile devices, could lead to large errors in pose estimation and in turn significantly degrade the user experience in mobile AR apps. In this study, we propose a novel monocular visual-inertial state estimation approach for robust and accurate pose estimation even for low-cost IMUs. The core of our method is an IMU pre-integration correction approach which effectively reduces the negative impact of IMU noises using the visual constraints in a sliding window and the kinematic constraint. We seamlessly integrate the IMU pre-integration correction module into a tightly-coupled,sliding-window based optimization framework for state estimation. Experimental results on public dataset EUROC demonstrate the superiority of our method to the state-of-the-art VINS-Mono in terms of smaller absolute trajectory errors (ATE) and relative pose errors (RPE). We further apply our method to real AR applications on two types of consumer-level mobile devices equipped with low-cost IMUs, i.e. an off-the-shelf smartphone and an AR glass. Experimental results demonstrate that our method can facilitate robust AR with little drifts on the two devices.