With the continuous evolution of radar RF sensing technology in the Internet of Things (IoT) field, the deployment of co-frequency communication devices within smart home and healthcare environments is becoming increasingly prevalent. Concurrently, the proliferation of RF signals has rendered the challenge of spectrum resource allocation increasingly prominent, exacerbating the issue of severe mutual interference among these co-frequency devices. This severe interference exhibits intricate attributes, characterized by prolonged duration, a broad frequency range, and high level power intensity. When experiencing interference, the target’s echo is notably veiled, rendering recovery through standard technologies exceedingly challenging. From our perspective, in the presence of severe interference, the emphasis ought to be on elegantly reconstructing the target’s echo located in the period of interference, rather than merely eliminating the negative impact on interferences from the raw radar signals. Based on this thought, we have explored the feasibility of a deep network model for interference mitigation of radar signals in this paper, and introduced an interference mitigation method, namely Ra-SPD. The Ra-SPD is crafted through a dual design methodology, integrating the mask-guided spectral decomposition mechanism alongside the region-aware spatial decomposition scheme. The primary objective of the initial design is to enhance the capabilities of the standard deep model, enabling it to distinguish the duration of interference present in radar signals, particularly when the semantic alignment between the restored radar signal and the corresponding ground truth is at risk of being disrupted. Meanwhile, the improvement of local information related to the restored radar signal is accomplished using the spatial decomposition scheme, aimed at increasing the robustness of low-intensity signal segments against severe interference. Experimental results demonstrate superior performance of Ra-SPD at 15%, 40%, and 80% three different signal-to-interference ratio conditions, achieving PSNR: 35.73/32.99/30.28 dB; SSIM: 0.96/0.95/0.92; FID: 23.41/30.97/43.92; and MAE: 0.014/0.024/0.036 across conditions. Our method consistently outperforms six benchmark algorithms in all objective metrics and subjective evaluations, exhibiting 7.6%, 1.5%, 5.5%, and 17.2% average improvements in four metrics relative to the suboptimal method, highlighting significant advantages in interference mitigation and signal quality preservation.

In practice, radar measurements are hindered by unavoidable noise, which lowers the signal-to-noise ratio (SNR) and raises the problem of radar signal denoising. Thanks to the development of deep learning techniques, recently proposed denoisers are progressively capable of blind denoising. On the other hand, due to the great fitting capacity of deep neural networks, the deep-learning-based denoising model would prefer to overfit on the training set, hence diminishing the generalization of a denoiser and impeding its use in a broader situation. This article focuses on this “blind universal denoising” problem for the first time and introduces a novel generative-adversarial-network-based (GAN-based) denoiser for radar spectrograms. The core idea of the proposed model lies in minimizing the generalization error during the model’s training, and to this end, our model incorporates a proposed identical dual learning (IDL) scheme and a reciprocal adversarial training (RAT) strategy to avoid the overfitting risk in the denoiser’s training. We perform the radar simulation using a motion capture database, and verify our model’s effectiveness under three different setups of training and testing datasets. For each setup, the noise level in the training and testing sets is configured to be different so to simulate the unknown measurement situations. Eleven algorithms are selected as comparisons, and the experimental results on two criteria illustrate that our method outperforms the others with a significant improvement.

The fundamental goal of hyperspectral anomaly detection (HAD) is the identification of pixels manifesting substantial deviations in spectral attributes when compared to their neighboring pixels. Nevertheless, the intrinsic attributes of hyperspectral images (HSI), characterized by their high-dimensional essence and the interdependencies among spectral bands, frequently exert an influence on the efficacy of anomaly detection( AD). Furthermore, current detection algorithms often fall short in harnessing the inherent information encapsulated within HSI, thereby constraining the network’s expressive potential. In response to these challenges, we introduce a multiview model tailored that amalgamates both global and local features for HAD. Specifically, the proposed method employs an unsupervised learning-based multiview network to simultaneously conduct feature analysis on both global and local attributes within HSI. The model incorporates a dual-component structure, featuring a global module utilizing axial attention for comprehensive global attribute analysis, and a local module employing a convolutional neural network with residual connections to capture fine-grained local features. Subsequently, the global-local multiview anomaly coupling mechanism is applied to consolidate the strengths of distinct perspectives, resulting in the ultimate AD outcomes. Experimental evaluations are performed on seven public HSI datasets, demonstrating superior performance of the proposed method in comparison to other state-of-the-art approaches.

Small object detection in large-scale remote sensing images (RSIs) is crucial for military and civil applications, but it remains challenging. Since small objects occupy few pixels, their features are easily interfered with by complex backgrounds and large objects. In addition, they are susceptible to localization offsets, which are prone to false or missed detections as there are few predicted bounding boxes matching the ground truth. To overcome these issues, this article proposes a filter progressive small object detection (FPSOD) model that is based on the progressive mechanism. With the proposed attention-based soft-threshold filtering module, FPSOD significantly filters out redundant information in high-level feature maps thus enhancing the semantic features of small objects. Furthermore, a progressive regression loss (PR-Loss) function is proposed to facilitate the precise localization, which mitigates predicted bounding box drift by limiting the fluctuated range of the gradients. The experimental results show that the proposed model substantially improves the precision and recall of small objects, effectively reduces missed detections, and improves detection performance.

Decomposing a single mixed image into individual image layers is the common crux of a classical category of tasks in image restoration. Several unified frameworks have been proposed that can handle different types of degradation in superimposed image decomposition. However, there are always undesired structural distortions in the separated images when dealing with complicated degradation patterns. In this paper, we propose a unified framework for superimposed image decomposition that can cope with intricate degradation patterns adaptively. Considering the different mixing patterns between the layers, we introduce a degeneration representation in the latent space to mine the intrinsic relationship between the superimposed image and the degeneration pattern. Moreover, by extracting structure-guided knowledge from the superimposed image, we further propose structural guidance refinement to avoid confusing content caused by structure distortion. Extensive experiments have demonstrated that our method remarkably outperforms other popular image separation frameworks. The method also achieves competitive results on related applications including image deraining, image reflection removal, and image shadow removal, which validates the generalization of the framework.

Gesture and sign language (SL) recognition technology enables machines to understand the meaning of human hand movements. In human–computer interaction, it is expected that gesture/SL recognition technologies will overcome equipment size and application environment constraints; in information communication, gesture/SL recognition technology will assist healthy people in more easily entering the world of deaf people and better understanding and meeting their inner emotional needs; and in patient monitoring, gesture/SL recognition technologies will detect abnormal behavior in the elderly or patients and reduce possible safety issues. As a result, it has significant implications for both research and broad application. Because the radar sensor can work normally in a wide range of illumination and weather conditions, as well as penetrate the shelter to receive the moving object echo signal and preserve individual privacy, it is becoming increasingly popular in a variety of recognition tasks. A primary step in using radar sensors for gesture/SL recognition is to suppress clutter to highlight useful motion information. To evaluate the clutter suppression effect, however, an objective metric is required. We present an objective assessment metric based on the pseudo-reference image (PRI) and an automatic threshold selection method based on Otsu for clutter suppression, as well as subjective and objective experiments demonstrating their effectiveness and universality in gesture/SL recognition. Notably, our proposed metric can be used for any recognition task that uses micro-Doppler (MD) spectrograms as the dataset.

Micro-Doppler-based human activity recognition has been extensively researched in remote sensing. However, a well-performing classifier requires sufficient radar data of omnidirectional human movements due to the “angle sensitivity”, resulting in high costs for radar data acquisition. To address this issue, this study defines for the first time the task of “omnidirectional spectrogram generation” and proposes a method to obtain enough omnidirectional spectrograms based on the spectrograms of human movements in one direction. It significantly reduces the dependence on radar measurements with omnidirectional setups. Our method is founded upon an image translation framework that is enhanced by incorporating the concept of information disentanglement and a proposed feature-level unbiased domain translation strategy. They enable us to generate high-quality omnidirectional spectrograms at various aspect angles. The generated spectrograms are then used as training support of omnidirectional micro-Doppler-based classifiers. Subsequently, we conduct an in-depth analysis of the metric correlation between the quality of generated spectrograms and the performance of these classifiers. Finally, we introduce a method for evaluating this correlation by proposed criterion. Our method is evaluated based on a radar simulation dataset, and the results show that it significantly exceeds the compared methods, demonstrating its great potential for the task of omnidirectional recognition of human activities. Besides, we find that there is a significant correlation between classification accuracy and several image quality assessment metrics, and we believe that this investigation will serve as the foundation for future research on assessing the contribution of data generation methods to downstream tasks using quantitative measures.

In recent years, radar-based human activity recognition has received considerable interest, but it faces the difficulty of a shortage of training data. In this research, a two-stage domain adaption method is proposed for this issue. This method combines simulated radar spectrograms with an elaborated domain-translation network to perform domain adaptation, hence enabling human activity recognition with limited measurement data support. To validate the efficacy of our method, we conduct radar simulation and measurements, and the experimental results demonstrate that the proposed method outperforms other comparisons in terms of accuracy, showing its great capacity for radar-based human activity recognition.

In recent years, gait-based person identification has gained significant interest for a variety of applications, including security systems and public security forensics. Meanwhile, this task is faced with the challenge of disguised gaits. When a human subject changes what he or she is wearing or carrying, it becomes challenging to reliably identify the subject’s identity using gait data. In this paper, we propose an unsupervised domain adaptation (UDA) model, namedGuided Subspace Alignment under the Class-awarecondition (G-SAC), to recognize human subjects based on their disguised gait data by fully exploiting the intrinsic information in gait biometrics. To accomplish this, we employ neighbourhood component analysis (NCA) to create an intrinsic feature subspace from which we can obtain similarities between normal and disguised gaits. With the aid of a proposed constraint for adaptive class-aware alignment, the class-level discriminative feature representation can be learned guided by this subspace. Our experimental results on a measured micro-Doppler radar dataset demonstrate the effectiveness of our approach. The comparison results with several state-of-the-art methods indicate that our work provides a promising domain adaptation solution for the concerned problem, even in cases where the disguised pattern differs significantly from the normal gaits. Additionally, we extend our approach to more complex multi-target domain adaptation (MTDA) challenge and video-based gait recognition tasks, the superior results demonstrate that the proposed model has a great deal of potential for tackling increasingly difficult problems.

In this paper, the interference mitigation problem is tackled as a regression problem. A prior-guided deep learning (DL) based interference mitigation approach is proposed for frequency modulated continuous wave (FMCW) radars. Considering the complex-valued nature of radar signals, complex-valued convolutional neural network, which is different from the conventional real-valued counterparts, is utilized as an architecture for implementation. Meanwhile, as the desired beat signals of FMCW radars and interferences exhibit different distributions in the time-frequency domain, this prior feature is exploited as a regularization term to avoid overfitting of the learned representation. The effectiveness and accuracy of our proposed complex-valued fully convolutional network (CV-FCN) based interference mitigation approach are verified and analyzed through both simulated and measured radar signals. Compared with the real-valued counterparts, the CV-FCN shows a better interference mitigation performance with a potential of half memory reduction in low Signal to Interference plus Noise Ratio (SINR) scenarios. The average SINR of interfered signals has been improved from -9.13 dB to 10.46 dB. Moreover, the CV-FCN trained using only simulated data can be directly utilized for interference mitigation in various measured radar signals and shows a superior generalization capability. Furthermore, by incorporating the prior feature, the CV-FCN trained on only 1/8 of the full data achieves comparable performance as that on the full dataset in low SINR scenarios, and the training procedure converges faster.

Human movement direction determination (HMDD) is a significant task in human detection and recognition applications, but it remains a challenge when utilizing single-input and single-output (SISO) radar because angle information cannot be accessed without multiple receiving antennas. Moreover, adopting multiple-output radar systems for this task would limit their applicability in a broader range of scenarios, as these systems require a larger placement area and a more extensive calibration procedure than SISO radar. Tackling this problem, this paper presents an effective method for the SISO-radar-based human movement direction determination task. Our method combines the joint time-frequency analysis (JTFA) technique with a proposed bio-inspired feature extraction process, thereby producing an accurate perception of moving direction based on the analysis of micro-Doppler signatures. The radar simulation and measurements are separately conducted and used to establish the corresponding dataset, so the superior performance of our method could be verified on the HMDD task. Furthermore, this article delves into why existing criteria for evaluating an HMDD model’s performance are insufficient in multi-direction situations, followed by an introduction of “small error concentration" and “omnidirectional error uniformity", as well as their evaluation protocols, to describe and measure the bias of an HMDD model’s results on multi-direction determination problems. By comparing with existing both traditional and deep-learning methods, we confirm our method’s superior in the HMDD task, and we believe that our research will aid in the advancement of human detection and recognition applications using SISO radar.

In this letter, a radar-based gait authentication method is proposed. We focus on the overfitting problem on the target category caused by limited training data in authentication models and propose a one-class classification model to alleviate this problem. The effectiveness of such model is verified by establishing a radar-based gait dataset, which is composed of gait micro-Doppler spectrograms derived from nine human subjects. The experimental results demonstrate that, under the condition of limited training data, the performances of an authentication model degrade because misclassification of the non-target samples easily occurs. The proposed method effectively avoids this risk, performing the other existing authentication and one-class classification methods on the metric Equal Error Rate.

The performance of deep learning (DL) algorithms for radar-based human motion recognition (HMR) is hindered by the diversity and volume of the available training data. In this article, to tackle the issue of insufficient training data for HMR, we propose an instance-based transfer learning (ITL) method with limited radar micro-Doppler (MD) signatures, alleviating the burden of collecting and annotating a large number of radar samples. ITL is a unique algorithm that consists of three interconnected parts, including DL model pretraining, correlated source data selection, and adaptive collaborative fine-tuning (FT). Any of the three components cannot be excluded; otherwise, the performance of the entire algorithm decreases. The experiments with a radar data set of six human motions show that ITL achieves state-of-the-art performance for HMR with limited training samples, outperforming several existing transfer learning approaches. Especially, when there are only 100 samples per person per class, ITL yields an F1 score of 96.7%. Last but not least, ITL is more generalized to human motion differences. Though adapted to recognize the persons’ motions in a small-scale target data set, ITL can also classify the persons’ motion data used for pretraining, achieving up to 11.0% F1 score enhancement over the conventional FT method.

In remote sensing, micro-Doppler signatures are widely used in moving target detection and automatic target recognition. However, since Doppler signatures are easily affected by the moving direction of the target, prior information of aspect angle is essential for spectral analysis. Thus, a micro-Doppler-based classifier is considered to be “angle-sensitive.” In this article, we propose an angle-insensitive classifier for the omnidirectional classification problem using the monostatic radar through a proposed new convolutional neural network. We further provide a sensible definition of “angle sensitivity,” and perform experiments on two data sets obtained through simulations and measurements. The results demonstrate that the proposed algorithm outperforms both feature-based and existing deep-learning-based counterparts, and resolve the issue of angle sensitivity in micro-Doppler-based classification.

In a smart home environment, assisted living has been a topic of great research over the past decade. Human motion analysis is considered as a key technology for living states recognition in an assisted living system. Recent research has proved that rich information can be obtained from human movements, such as the motion category, moving patterns, and human identity. In this paper, a nonintrusive human movement sensing system is established with a mono-static ultrawide bandwidth radar. Then, we propose a well-designed joint motion classification (MCL) and person identification (PID) convolutional neural network (named as “JMI-CNN”). To recognize human motions and identities simultaneously, the network employs a multitask learning scheme as well as the attention mechanism and the hierarchical feature reuse strategies. We report the experimental result on the data from 15 individuals, each performing six motions. It shows that the model achieves a promising performance of 80.57% on the joint task, while the accuracy for MCL and PID are 98.50% and 80.92%, respectively. Moreover, we carry out ablation studies to evaluate the design principles of the proposed method. Discussions on the impact of signal noise ratio and slow-time window length are also conducted.

Micro-Doppler-based human motion classification has become a topical area of research recently. However, the current research is limited by the lack of labeled training data. Domain adaptation, namely, the ability to take advantage of knowledge from an available source data set and apply it to an unlabeled target data set, is useful in this situation. A typical strategy for this transfer learning technique is to extract domain-invariant feature representations. In this letter, an unsupervised domain adaptation method for micro-Doppler classification is proposed. Given no available measurement training samples, we creatively utilize the motion capture database as an auxiliary and adapt its interior knowledge to the measurement data set. To achieve domain-invariant features, three types of features are extracted and fused including low-level deep features from the convolutional neural network, empirical features, and statistical features. After feature fusion, a k-nearest neighbor classifier is applied to the measurement data to classify seven human activities. Experimental results show that our approach outperforms several state-of-the-art unsupervised domain adaptation methods. The impact of the output from different convolution layers is further investigated, and ablation studies of the efficacy of each feature are also carried out in this letter.