Automatic modulation classification (AMC) in complex electromagnetic environments is essential for ensuring reliable spectrum surveillance. However, most existing AMC algorithms primarily rely on the data engineer and focus on improving accuracy under high signal-to-noise ratio (SNR) conditions, making it difficult to maintain robust and accurate performance in real-world unstable SNR scenarios. In order to address this challenge, a knowledge-guided spatial-temporal fusion network for low-SNR modulation recognition is proposed, named STFNet. Firstly, aiming at the instability of the model classification caused by the single input mode, the enhanced constellation diagram and the I/Q signal are simultaneously introduced as the inputs of the STFNet. Furthermore, the STFNet is designed as a dual-path adaptive multimodal fusion architecture to simultaneously exploit the spatial features of the enhanced constellation diagram and the temporal features of the I/Q signal. Secondly, an AMC-specific transfer learning (AMC-TL) strategy is introduced to enhance the global robustness of spatial representations through contrastive learning. More importantly, a domain knowledge-guided mixture of experts (DKG-MoE) is proposed to incorporate traditional features into the expert routing process, which improves temporal recognition accuracy. Then, an adaptive modality attention (AMA) module is developed to balance the accuracy under high SNR and the robustness under low SNR. Finally, extensive experiments on four benchmark datasets demonstrate that the proposed method achieves a 4.13% accuracy improvement under low SNR conditions (-20 dB ∼ 0 dB) and outperforms all state-of-the-art (SOTA) methods in overall average accuracy (65.57%). The code is publicly available at https://github.com/yoho78/STFNet.

Headphones have become ubiquitous in daily work and communication, leading users to assume a sense of privacy and security during confidential conversations while overlooking the potential risk of eavesdropping. In this paper, we present mmEar, an end-to-end eavesdropping system that demonstrates the feasibility of compromising headphones using a commercial off-the-shelf (COTS) mmWave radar. Unlike previous approaches that rely on relatively strong vibrations, mmEar targets extremely faint, low-SNR speech-induced vibrations on headphone surfaces. To address this challenge, we introduce a Faint Vibration Emphasis (FVE) technique that amplifies phase variations on the IQ plane, followed by a deep denoising network for enhanced signal quality. Furthermore, we design a diffusion-based generative model within a pretrain–finetune framework, leveraging large-scale synthetic data to significantly improve generalization and robustness across diverse scenarios. Extensive experiments on multiple headphone and earphone models validate the practicality and effectiveness of the proposed attack, revealing that most tested devices can be compromised to recover intelligible speech.

Face authentication has become an increasingly attractive and prevalent technology in human-computer interaction. Recent works have explored radio frequency (RF) signals for illumination-robust and privacy-preserving face authentication. However, these approaches require all users to undergo compulsory prior registration and cannot effectively handle unregistered users, which restricts their practical applicability in real-world scenarios. In this paper, we present ammWave-basedmulti-user face authentication system,m3FacePass, which performs zero-shot face authentication in complex multi-user scenarios using a commercial off-the-shelf (COTS) mmWave radar. First, m3FacePass collects mmWave signals and reconstructs spatial mappings by generating 3D heatmaps. Based on the 3D heatmaps,m3FacePassdetects and separates faces in multi-user scenarios through template matching. Then, unique facial features are extracted and transformed to a hypersphere manifold, which reserves feature space for unregistered users. By introducing dummy classifiers in the authentication model,m3FacePassoptimizes decision boundaries between registered and unregistered users. Furthermore,m3FacePassenhances its generalization ability for zero-shot authentication by incorporating pseudo unknowns during model training. Extensive experiments in real-world environments demonstrate thatm3FacePassachieves an authentication accuracy of 96.6% for registered users and 91.1% for unregistered users in multi-user scenarios.

Face authentication has become an increasingly attractive and prevalent technology in human-computer interaction. Recent works have explored radio frequency (RF) signals for illumination-robust and privacy-preserving face authentication. However, these approaches require all users to undergo compulsory prior registration and cannot effectively handle unregistered users, which restricts their practical applicability in real-world scenarios. In this paper, we present a mmWave-based multiuser face authentication system,$m^{3}$FacePass, which performs zero-shot face authentication in complex multi-user scenarios using a commercial off-the-shelf (COTS) mm Wave radar. First,$m^{3}$FacePass collects mmWave signals and reconstructs spatial mappings by generating 3D heatmaps. Based on the 3D heatmaps,$m^{3}$FacePass detects and separates faces in multi-user scenarios through template matching. Then, unique facial features are extracted and transformed to a hypersphere manifold, which reserves feature space for unregistered users. By introducing dummy classifiers in the authentication model,$\boldsymbol{m}^{\mathbf{3}}$FacePass optimizes decision boundaries between registered and unregistered users. Furthermore,$\boldsymbol{m}^{\mathbf{3}}$FacePass enhances its generalization ability for zero-shot authentication by incorporating pseudo unknowns during model training. Extensive experiments in realworld environments demonstrate that$\boldsymbol{m}^{3}$FacePass achieves an authentication accuracy of 96.6% for registered users and 91.1% for unregistered users in multi-user scenarios.

Photoplethysmography (PPG) has been extensively employed in commercial and medical products to assess human cardiac activities. However, despite PPG’s active role in improving people’s daily lives, research on the vulnerabilities of PPG systems is still in its infancy. This paper investigates the feasibility of deceiving PPG sensors in the physical domain. We propose FakePPG, which utilizes a low-cost Liquid Crystal Modulator (LCM) device to mimic the PPG signals of a legitimate user, thus deceiving both the PPG-based health assessment and potential authentication applications. To implement FakePPG in practical scenarios, we build the attack prototype using commercial off-the-shelf electronic components and further design an automated optimization and attack framework. By leveraging the modified multi-Gaussian model for parameterization, the evolutionary strategy for optimization, and the reference heart rate model for heartbeat variability alignment, FakePPG can achieve efficient, flexible, and automated PPG forgery against arbitrary users and heart states. Extensive experimental results show that FakePPG can achieve a success rate of 96.7% for Atrial Fibrillation (AFib) spoofing and 91.2% for identity spoofing, respectively, revealing a realistic threat to PPG systems.

The technology of facial expression reconstruction has paved the way for various face-centric applications such as virtual reality (VR) modeling, human-computer interaction, and affective computing. Existing vision-based solutions present challenges in privacy leakage and poor lighting conditions. In this paper, we introduce a nonintrusive facial expression reconstruction system,mm3DFace, which uses a millimeter wave (mmWave) radar to reconstruct facial expressions in a privacy-preserving and passive manner.mm3DFacefirst captures and pre-processes mmWave signals reflected by a human face, and extracts intricate facial geometric features using a ConvNeXt model integrated with triple loss embedding. Subsequently,mm3DFacederives pose-invariant facial representations utilizing region-divided affine transformation, and further generates individual facial shapes with 68 facial landmarks. Then, dynamic facial expressions with 3D facial avatars are reconstructed to exhibit realistic facial expressions. Finally,mm3DFaceenables micro-expression recognition with mmWave signals, which ensures the capability of describing tiny facial changes. Through extensive real-world experiments involving 15 participants,mm3DFaceachieves a normalized mean error of 3.94%, a mean absolute error of 2.30 mm, and a 3D-mean absolute error of 4.10 mm in tracking 68 facial landmarks, which demonstrates the efficacy and practicality ofmm3DFacein real-world 3D facial reconstruction scenarios.

Hand pose estimation is a key support for a variety of interactive applications including user interface control, sign language understanding, virtual reality modeling, etc. Existing approaches mainly exploit wearable devices such as gloves or bracelets to estimate hand poses, which may introduce high deploying costs and intrusive user experience. Others rely on vision technologies whereas they could face complicated illuminations and privacy leakage. In this paper, we present a millimeter wave (mmWave) signal-based 3D hand pose estimation system, mmHand, which utilizes a mmWave radar to generate 3D hand skeletons and reconstruct 3D hand meshes. mmHand first leverages mmWave signals to sense a hand and pre-process the signals. Then, mmHand extracts spatial and temporal features using a designed attention-based hourglass network (mmSpaceNet) and Long Short-Term Memory (LSTM), respectively. Based on the extracted features, mmHand further regresses hand joints in 3D space to generate 3D hand skeletons. Finally, 3D hand meshes that continuously describe hand poses with detailed surfaces are reconstructed through a hand Model with Articulated and Non-rigid defOrmations (MANO). Extensive experiments demonstrate that mmHand can accurately generate 3D hand skeletons with 18.3mm mean per joint position error and 95.1 % of correct key points, which indicates the effectiveness of mmHand on hand pose estimation.

Recent years have witnessed the emerging market of 3D facial reconstruction that supports numerous face-driven scenarios including modeling in virtual reality (VR), human-computer interaction, and affective computing applications. Current mainstream approaches rely on vision for 3D facial reconstruction, which may encounter privacy concerns and suffer from obstruction scenes and bad lighting conditions. In this paper, we present a nonintrusive 3D facial reconstruction system, mm3DFace, which leverages a millimeter wave (mmWave) radar to reconstruct 3D human faces that continuously express facial expressions in a privacy-preserving and passive manner. Based on the pre-processed mmWave signals, mm3DFace first extracts facial geometric features that capture subtle changes in facial expressions through a ConvNeXt model with triple loss embedding. Then, mm3DFace derives distance and orientation-robust facial shapes with 68 facial landmarks using region-divided affine transformation. mm3DFace next reconstructs facial expressions through a designed regional amplification method and finally generates 3D facial avatars that continuously express facial expressions. Extensive experiments involving 15 participants in real-world environments show that mm3DFace can accurately track 68 facial landmarks with 3.94% normalized mean error, 2.30mm mean absolute error, and 4.10mm 3D-mean absolute error, which is effective and practical in real-world 3D facial reconstruction.

User authentication nowadays has become an important support for not only security guarantees but also emerging novel applications. Although WiFi signal-based user authentication has achieved initial success, it works in single-user scenarios while multi-user authentication remains a challenging task. In this paper, we present MultiAuth, a multi-user authentication system that can authenticate multiple users with a single pair of commodity WiFi devices. The basic idea is to profile multipath components of WiFi signals, and leverage the multipath components to characterize each user individually for multi-user authentication. MultiAuth first profiles multipath components of WiFi signals through a proposed MUltipath Time-of-Arrival estimation algorithm (MUTA). Then, after matching corresponding multipath components to each user in complex multi-user scenarios, MultiAuth constructs individual CSI based on the multipath components to characterize each user individually. An AoA-based approach is exploited to further separate individual CSI constructed by the users with same ToA. To identify users through their activities, MultiAuth extracts user behavior profiles based on the individual CSI, and leverages a dual-task neural network for robust user authentication. Extensive experiments involving 3 simultaneously present users demonstrate that MultiAuth is effective in multi-user authentication with 86.2% average accuracy and 9.5% average false accept rate.

With the development of smart indoor environments, user authentication becomes an essential mechanism to support various secure accesses. Although recent studies have shown initial success on authenticating users with human activities or gestures using WiFi, they rely on predefined body gestures and perform poorly when meeting undefined body gestures. This work aims to enable WiFi-based user authentication with undefined body gestures rather than only predefined body gestures, i.e., realizing a gesture-independent user authentication. In this paper, we first explore physiological characteristics underlying body gestures, and find that statistical distributions under WiFi signals induced by body gestures can exhibit invariant individual uniqueness unrelated to specific body gestures. Inspired by this observation, we propose a user authentication system, which utilizes WiFi signals to identify individuals in a gesture-independent manner. Specifically, we design an adversarial learning-based model, which suppresses specific gesture characteristics, and extracts invariant individual uniqueness unrelated to specific body gestures, to authenticate users in a gesture-independent manner. Extensive experiments in indoor environments show that the proposed system is feasible and effective in gesture-independent user authentication.

Nowadays, the market of 3D human posture tracking has extended to a broad range of application scenarios. As current mainstream solutions, vision-based posture tracking systems suffer from privacy leakage concerns and depend on lighting conditions. Towards more privacy-preserving and robust tracking manner, recent works have exploited commodity radio frequency signals to realize 3D human posture tracking. However, these studies cannot handle the case where multiple users are in the same space. In this paper, we present a mmWave-based multi-user 3D posture tracking system, m3Track, which leverages a single commercial off-the-shelf (COTS) mmWave radar to track multiple users' postures simultaneously as they move, walk, or sit. Based on the sensing signals from a mmWave radar in multi-user scenarios, m3Track first separates all the users on mmWave signals. Then, m3Track extracts shape and motion features of each user, and reconstructs 3D human posture for each user through a designed deep learning model. Furthermore. m3Track maps the reconstructed 3D postures of all users into 3D space, and tracks users' positions through a coordinate-corrected tracking method, realizing practical multi-user 3D posture tracking with a COTS mmWave radar. Experiments conducted in real-world multi-user scenarios validate the accuracy and robustness of m3Track on multi-user 3D posture tracking.

With the increasing integration of humans and the cyber world, user authentication becomes critical to support various emerging application scenarios requiring security guarantees. Existing works utilize Channel State Information (CSI) of WiFi signals to capture single human activities for non-intrusive and device-free user authentication, but multi-user authentication remains a challenging task. In this paper, we present a multi-user authentication system, MultiAuth, which can authenticate multiple users with a single commodity WiFi device. The key idea is to profile multipath components of WiFi signals induced by multiple users, and construct individual CSI from the multipath components to solely characterize each user for user authentication. Specifically, we propose a MUltipath Time-of-Arrival measurement algorithm (MUTA) to profile multipath components of WiFi signals in high resolution. Then, after aggregating and separating the multipath components related to users, MultiAuth constructs individual CSI based on the multipath components to solely characterize each user. To identify users, MultiAuth further extracts user behavior profiles based on the individual CSI of each user through time-frequency analysis, and leverages a dual-task neural network for robust user authentication. Extensive experiments involving 3 simultaneously present users demonstrate that MultiAuth is accurate and reliable for multi-user authentication with 87.6% average accuracy and 8.8% average false accept rate.

The development of smart homes has advanced the concept of user authentication to not only protecting user privacy but also facilitating personalized services to users. Along this direction, we propose to integrate user authentication with human-computer interactions between users and smart household appliances through widely-deployed WiFi infrastructures, which is non-intrusive and device-free. In this paper, we propose$FingerPass$which leverages channel state information (CSI) of surrounding WiFi signals to continuously authenticate users through finger gestures in smart homes.$FingerPass$separates the user authentication process into two stages, login and interaction, to achieve high authentication accuracy and low response latency simultaneously. In the login stage, we develop a deep learning-based approach to extract behavioral characteristics of finger gestures for highly accurate user identification. For the interaction stage, to provide continuous authentication in real time for satisfactory user experience, we design a verification mechanism with lightweight classifiers to continuously authenticate the user’s identity during each interaction of finger gestures. Experiments in real environments show that$FingerPass$can achieve the authentication accuracies of 90.6 percent under in-domain scenarios and 87.6 percent under cross-domain scenarios, as well as$186.6\;ms$response time during interactions.

The development of smart homes has advanced the concept of user authentication to not only protecting user privacy but also facilitating personalized services to users. Along this direction, we propose to integrate user authentication with human-computer interactions between users and smart household appliances through widely-deployed WiFi infrastructures, which is non-intrusive and device-free. In this paper, we propose FingerPass which leverages channel state information (CSI) of surrounding WiFi signals to continuously authenticate users through finger gestures in smart homes. We investigate CSI of WiFi signals in depth and find CSI phase can be used to capture and distinguish the unique behavioral characteristics from different users. FingerPass separates the user authentication process into two stages, login and interaction, to achieve high authentication accuracy and low response latency simultaneously. In the login stage, we develop a deep learning-based approach to extract behavioral characteristics of finger gestures for highly accurate user identification. For the interaction stage, to provide continuous authentication in real time for satisfactory user experience, we design a verification mechanism with lightweight classifiers to continuously authenticate the user's identity during each interaction of finger gestures. Experiments in real environments show that FingerPass can achieve 91.4% authentication accuracy, and 186.6ms response time during interactions.