The scarcity of high-quality millimeter-wave (mmWave) radio frequency (RF) data for human sensing poses significant challenges to the generalization and robustness of deep learning models in this domain. The acquisition of RF data is constrained by the need for specialized equipment and specific application scenarios, resulting in a significant gap in both quantity and diversity compared to visual data. To address this critical bottleneck, we propose PODM-mmHPE, a novel framework that leverages Physical Optics (PO) and Diffusion Models (DM) to synthesize high-quality human RF data for mmWave Human Pose Estimation (mmHPE). Our framework introduces two key innovations: (1) a PO-based RF simulation pipeline that generates coarse RF data from 3D human mesh models by modeling electromagnetic wave interactions with body surface geometry and (2) a DM-based refinement module that transforms the coarse data into realistic RF heatmaps, capturing the complexity and noise of real-world RF signals. Furthermore, we develop map2pose, a Graph Convolutional Network (GCN)-based model that establishes an end-to-end mapping from RF heatmaps to 3D human poses, serving both as a downstream task benchmark and a validation mechanism for the quality of our synthetic data. Experiments demonstrate PODM-mmHPE’s effectiveness with 23.9% and 25.1% MPJPE reductions on HuPR and mmHPE datasets under synthetic augmentation. This validates our physics-guided synthesis enables generalizable mmWave pose estimation.

In the realm of autonomous driving, precise and robust 3D perception is paramount. Multi-modal fusion for 3D object detection is crucial for improving accuracy, generalization, and robustness in autonomous driving. In this paper, we introduce the Depth Enhancement Network (DEN), an innovative camera-radar fusion framework that generates an accurate depth estimation for 3D object detection. To overcome the lack of spatial information To overcome the limitations caused by the lack of spatial information in an image, DEN estimates image depth using accurate radar points. Furthermore, to extract more comprehensive and fine-grained scene depth information, we present an innovative label optimization strategy that enhances label density and quality. DEN achieves an 18.78% reduction in Mean Absolute Error (MAE) and a 12.8% decrease in Root Mean Square Error (RMSE) for depth estimation. Additionally, it improves 3D object detection accuracy by 0.8% compared to the baseline model. Under low visibility conditions, DEN demonstrates a 6.7% reduction in MAE and a 9.6% reduction in RMSE compared to the baseline. These improvements demonstrated its robustness and enhanced performance under challenging conditions.

Nowadays, human pose estimation (HPE) is widely used in several application areas. The current mainstream method based on vision suffers from privacy leakage and relies on lighting conditions. To adopt a more privacy-preserving and pervasive HPE approach, recent studies have implemented 3-D HPE using commodity radio frequency (RF) signals. However, RF-based HPE faces issues, such as resolution limitations and complex data processing, which makes it challenging to extract and utilize multiscale human activity features. In this article, we propose mmHPE, a novel approach to detect and reconstruct 3-D human posture in multiscale scenarios using a single millimeter wave radar. mmHPE consists of three main parts. Specifically, we develop a 3-D target detection network (TDN) and design an optimized loss function for it to enhance its 3-D target bounding box (BBox) detection capability in radar 3-D space. Next, an enhanced point cloud generator (EPCG) algorithm based on the 3-D target BBox is proposed to generate a stable and accurate point cloud of the target. Furthermore, we design a multiscale coarse-fine HPE network (CFN) ranging from approximate to precise estimation for reconstructing a 3-D skeleton from point cloud data. Extensive experiments demonstrate that our method surpasses other methods for 3-D human pose reconstruction in multiscale scenes, with an average error of 4.50 cm. Our method is robust enough to accurately estimate the target pose even in occluded or low-light scenes.

The hand localization problem has been a longstanding focus due to its many applications. The task involves modeling the hand as a singular point and determining its position within a defined coordinate system. However, due to data modality limitations, existing hand localization technologies face several challenges. For example, vision-based localization raises privacy concerns, while wearable-based methods compromise user comfort. In this article, we introduce mmHand, a new device-free, privacy-preserving dynamic hand localization system with pixel-level accuracy, using a single commodity mmWave device. We first propose a mmImage generation tool to fully extract spatial information from raw mmWave data and introduce a novel 2-D image-format representation of mmWave data. Next, we design a framework that provides a new quality evaluation method and pixel space labeling for the mmWave data. Finally, we present a cross-modality spatial feature-enhanced model with high spatial feature extraction capabilities, which can accurately localize hand positions at the pixel level in the mmWave radar U-V pixel coordinate system. We evaluate the system with experiments on 12 subjects in three scenarios, and the results across four metrics demonstrate the effectiveness of our hand localization system.

In this paper, we introduce mmOrbit, a mmWave-based rotor orbit measurement system that can estimate rotor orbit by analyzing machinery surface vibration. To measure the 2D rotor orbit, two synchronized mmWave radars are deployed to build the orbit from different viewpoints. The existing literature shows that the micro-displacement measurement accuracy of mmWave radar is not enough to meet the application requirements of micron-level resolution without appropriate fine-gained processing methods. Therefore, we propose a three-step vibration displacement extraction algorithm with sliding table to extract mechanical vibration micro-displacement and increase measurement precision. Accurate mechanical vibration displacements in two vertical directions are used to estimate the 2D rotor orbit. Our extensive experiments indicate that mmOrbit can accurately measure mechanical vibration micro-displacement with an error of 4.4 μm for the 80th percentile. Furthermore, mmOrbit can estimate 2D rotor orbit with high precision, showing an orbit eccentricity error of 7%, an orbit direction error of 7 °, and a disjoint area proportion of 17% for the 80th percentile. The imbalance detection experiment verifies the accuracy and dependability of our measured rotor orbit.

Non-invasive human vital sign detection has gained significant attention in recent years, with its potential for contactless, long-term monitoring. Advances in radar systems have enabled non-contact detection of human vital signs, emerging as a crucial area of research. The movements of key human organs influence radar signal propagation, offering researchers the opportunity to detect vital signs by analyzing received electromagnetic (EM) signals. In this review, we provide a comprehensive overview of the current state-of-the-art in millimeter-wave (mmWave) sensing for vital sign detection. We explore human anatomy and various measurement methods, including contact and non-contact approaches, and summarize the principles of mmWave radar sensing. To demonstrate how EM signals can be harnessed for vital sign detection, we discuss four mmWave-based vital sign sensing (MVSS) signal models and elaborate on the signal processing chain for MVSS. Additionally, we present an extensive review of deep learning-based MVSS and compare existing studies. Finally, we offer insights into specific applications of MVSS (e.g., biometric authentication) and highlight future research trends in this domain.