This article proposes a three-dimensional (3D) human pose estimation method using through-wall radar (TWR) systems, which extends and supplements new applications in the era of the Internet of Things (IoT). TWR system can penetrate non-metallic obstacles and perceive wall-occlusive human targets, but the physical characteristics of radio frequency (RF) signals, such as poor imaging resolution and specularity effect, make the pose estimation process highly ill-posed. In this work, we propose a mutual information maximizing deeply-supervised network (MIMDSN), which aims to extract accurate and robust 3D human skeletons from TWR images. Inspired by past works, an optical system is attached to the TWR system to provide cross-modal pseudo labels. Based on a depth design philosophy of convolutional neural networks that meets radar resolution constraints, we design a resolution-guided pose estimation network for keypoint coordinate regression. To alleviate the ill-posed problem, supervising solely the network output is insufficient. The cross-modal supervision is not only built on predictions, but also on features of the network’s hidden layer. With the help of information theory, the mutual information between features and pseudo labels is maximized for feature alignment and discriminability enhancement. Experiments show competitive performance against state-of-the-art RF-based human pose estimation methods and can reconstruct accurate 3D skeletons in multi-target, low-visibility, and wall-occlusive scenes.

Ground-penetrating radar (GPR) has been widely used to detect subsurface objects. In recent years, deep learning techniques have achieved significant success in image recognition, which has potential implications for interpreting GPR data. However, reliable training of deep learning models requires massive amounts of labeled data, which can be difficult to obtain due to the high costs of data acquisition and field validation. This letter proposes a GPR data augmentation method based on generative adversarial networks (GANs) - Wasserstein GANs (WAEGAN). This proposed method utilizes a GANs model with an encoder E, a joint generator G, and a discriminator D to generate data. A pre-trained classifier C imposes target category constraints on the generated data, while a Wasserstein loss function is employed to stabilize the training process.The performance of the proposed method is evaluated in terms of the validity, diversity, and impact on the classifier performance of the generated fake GPR data. The experimental results verify the superiority of the proposed method in simultaneously generating multiple target categories and generating GPR data that conforms to reality.

As a nondestructive and nonintrusive geophysical electromagnetic technology, ground-penetrating radar (GPR) has been widely applied to subsurface target detection, such as landmine detection, pipeline detection, and underground cavity detection. The target response received by the GPR system is generally contaminated by clutter, which greatly affects the detection performance of the buried targets. In this letter, a novel clutter removal method combining migration and dictionary learning (DL) is presented. First, the proposed method applies the frequency–wavenumber (F–K) migration to the received GPR B-scan data. Then, since the focused target response and the clutter in the migrated GPR B-scan data present different morphological components, DL can be applied to the migrated GPR B-scan data to separate the focused target response (point-shaped structure) from the clutter (horizontal strip-shaped structure). Both numerical simulated data and experimental data collected by a real GPR system are used to evaluate the performance of the proposed clutter removal method. The experimental results demonstrate the effectiveness of the proposed clutter removal method under irregular clutter conditions, which improves the detection ability of the buried targets.

Clutter removal in ground-penetrating radar (GPR) B-scan data has been widely studied in recent years. In this letter, we propose a novel data-driven clutter suppression method in GPR data based on conditional generative adversarial nets (cGANs). The proposed method learns a function that maps the cluttered data to the clutter-free data from the training set. The training set consists of pairs of cluttered data and corresponding clutter-free data. Different from the traditional method that only uses the simulation training set, we simulate the clutter-free data and add the real collected non-target data to the simulated clutter-free data as cluttered data, so that the trained network can generalize well to the real GPR data. The proposed method is compared with the subspace method, sparse representation-based method, and low-rank and sparse matrix decomposition (LRSD) methods on both simulation data and real collected data. The results show that the proposed method has higher performance in terms of computational complexity, clutter suppression results, and applicability than those state-of-the-art methods.

Ultrawideband (UWB) multiple-input–multiple-output (MIMO) radar is widely used for through-wall imaging (TWI) due to its excellent penetrability and large aperture. Multichannels in the MIMO radar system are usually time-division multiplexing based on microwave switches to reduce the complexity of the system in engineering. The switching process of the channel will bring time delay, which cannot be ignored in the TWI of the moving target. The switching time delay will cause the defocus and position shift of the TWI of the moving target. This letter proposes a motion compensation method based on multiframe data fusion (MFDF) used for correcting the echo of the through-wall moving target. A geometric model is established in the proposed method through the echo of the current frame and the next frame, and the compensated signal is obtained through the geometric solution. The proposed method is compared with before compensation and the traditional single-channel motion compensation algorithm (SCMCA) through simulation and experimental data verification. The visual images and quantitative results show that the proposed motion compensation method can obtain a good focus image of the through-wall moving target and reduce the positioning error.

Low imaging spatial resolution hinders through-the-wall radar imaging (TWRI) from reconstructing complete human postures. This letter mainly discusses a convolutional neural network (CNN)-based human posture reconstruction method for TWRI. The training process follows a supervision-prediction learning pipeline inspired by the cross-modal learning technique. Specifically, optical images and TWRI signals are collected simultaneously using a self-develop radar containing an optical camera. Then, the optical images are processed with a computer-vision-based supervision network to generate ground-truth human skeletons. Next, the same type of skeleton is predicted from corresponding TWRI signals using a prediction network. After training, the model shows complete predictions in wall-occlusive scenarios solely using TWRI signals. Experiments show comparable quantitative results with the state-of-the-art vision-based methods in nonwall-occlusive scenarios and accurate qualitative results with wall occlusion.

Although the through-the-wall radar imaging (TWRI) system working in the appropriate frequency band can penetrate the nonmetallic obstacles and sense the targets behind, its low imaging spatial resolution hinders the acquisition of more detailed information, such as human pose and shape. This article mainly discusses a deep learning-based human pose and shape recovery method from TWRI images. Inspired by cross-modal learning, the method follows a teacher–student learning pipeline that avoids the heavy cost of manual labeling. Specifically, a camera is attached to the self-develop radar system to simultaneously capture paired red-green-blue (RGB) images and TWRI images in a scenario without wall occlusion. A pose estimation framework (Hourglass) and a semantic segmentation framework (UNet) serve as the teacher network to convert the RGB images into the pose keypoints and the shape masks. By taking inspiration from the topological architecture of these frameworks, a student network radar pose shape network (RPSNet) is designed to extract the information from the corresponding radar images and predict the keypoints and masks that are close to the results above. Instead of learning two single-task objectives independently, multitasking learning is introduced to adaptatively learn common features. When applied to wall-occlusive scenarios, only the radar images are collected and fed into the student network for pose and shape recovery. The advantages of this method over computer vision-based methods for human recovery are demonstrated in scenarios both without and with wall occlusion.