Moving infrared small target detection (IRSTD) plays a critical role in practical applications, such as surveillance of unmanned aerial vehicles (UAVs) and UAV-based search system. Moving IRSTD still remains highly challenging due to weak target features and complex background interference. Accurate spatio-temporal feature modeling is crucial for moving target detection, typically achieved through either temporal differences or spatio-temporal (3D) convolutions. Temporal difference can explicitly leverage motion cues but exhibits limited capability in extracting spatial features, whereas 3D convolution effectively represents spatio-temporal features yet lacks explicit awareness of motion dynamics along the temporal dimension. In this paper, we propose a novel moving IRSTD network (TDCNet), which effectively extracts and enhances spatio-temporal features for accurate target detection. Specifically, we introduce a novel temporal difference convolution (TDC) re-parameterization module that comprises three parallel TDC blocks designed to capture contextual dependencies across different temporal ranges. Each TDC block fuses temporal difference and 3D convolution into a unified spatio-temporal convolution representation. This re-parameterized module can effectively capture multi-scale motion contextual features while suppressing pseudo-motion clutter in complex backgrounds, significantly improving detection performance. Moreover, we propose a TDC-guided spatio-temporal attention mechanism that performs cross-attention between the spatio-temporal features extracted from the TDC-based backbone and a parallel 3D backbone. This mechanism models their global semantic dependencies to refine the current frame’s features, thereby guiding the model to focus more accurately on critical target regions. To facilitate comprehensive evaluation, we construct a new challenging benchmark, IRSTD-UAV, consisting of 15,106 real infrared images with diverse low signal-to-clutter ratio scenarios and complex backgrounds. Extensive experiments on IRSTD-UAV and public infrared datasets demonstrate that our TDCNet achieves state-of-the-art detection performance in moving target detection.

Infrared unmanned aerial vehicle (UAV) target images often suffer from motion blur degradation caused by rapid sensor movement, significantly reducing contrast between target and background. Generally, detection performance heavily depends on the discriminative feature representation between target and background. Existing methods typically treat deblurring as a preprocessing step focused on visual quality, while neglecting the enhancement of task-relevant features crucial for detection. Improving feature representation for detection under blur conditions remains challenging. In this paper, we propose a novel Joint Feature-Domain Deblurring and Detection end-to-end framework, dubbed JFD³. We design a dual-branch architecture with shared weights, where the clear branch guides the blurred branch to enhance discriminative feature representation. Specifically, we first introduce a lightweight feature restoration network, where features from the clear branch serve as feature-level supervision to guide the blurred branch, thereby enhancing its distinctive capability for detection. We then propose a frequency structure guidance module that refines the structure prior from the restoration network and integrates it into shallow detection layers to enrich target structural information. Finally, a feature consistency self-supervised loss is imposed between the dual-branch detection backbones, driving the blurred branch to approximate the feature representations of the clear one. We also construct a benchmark, named IRBlurUAV, containing 30,000 simulated and 4,118 real infrared UAV target images with diverse motion blur. Extensive experiments on IRBlurUAV demonstrate that JFD³ achieves superior detection performance while maintaining real-time efficiency.

Unmanned aerial vehicle (UAV) target tracking based on thermal infrared imaging has been one of the most important sensing technologies in anti-UAV applications. However, the infrared UAV targets often exhibit weak features and complex backgrounds, posing significant challenges to accurate tracking. To address these problems, we introduce SiamDFF, a novel dynamic feature fusion Siamese network that integrates feature enhancement and global contextual attention knowledge distillation for infrared UAV target (IRUT) tracking. The SiamDFF incorporates a selective target enhancement network (STEN), a dynamic spatial feature aggregation module (DSFAM), and a dynamic channel feature aggregation module (DCFAM). The STEN employs intensity-aware multi-head cross-attention to adaptively enhance important regions for both template and search branches. The DSFAM enhances multi-scale UAV target features by integrating local details with global features, utilizing spatial attention guidance within the search frame. The DCFAM effectively integrates the mixed template generated from STEN in the template branch and original template, avoiding excessive background interference with the template and thereby enhancing the emphasis on UAV target region features within the search frame. Furthermore, to enhance the feature extraction capabilities of the network for IRUT without adding extra computational burden, we propose a novel tracking-specific target-aware contextual attention knowledge distiller (TCAKD). It transfers the target prior from the teacher network to the student model, significantly improving the student network's focus on informative regions at each hierarchical level of the backbone network. Extensive experiments on real infrared UAV datasets demonstrate that the proposed approach outperforms state-of-the-art target trackers under complex backgrounds while achieving a real-time tracking speed.

Infrared unmanned aerial vehicle (UAV) images captured using thermal detectors are often affected by temperature-dependent low-frequency nonuniformity, which significantly reduces the contrast of the images. Detecting UAV targets under nonuniform conditions is crucial in UAV surveillance applications. Existing methods typically treat infrared nonuniformity correction (NUC) as a preprocessing step for detection, which leads to suboptimal performance. Balancing the two tasks while enhancing detection-beneficial information remains challenging. In this paper, we present a detection-friendly union framework, termed UniCD, that simultaneously addresses both infrared NUC and UAV target detection tasks in an end-to-end manner. We first model NUC as a small number of parameter estimation problem jointly driven by priors and data to generate detection-conducive images. Then, we incorporate a new auxiliary loss with target mask supervision into the backbone of the infrared UAV target detection network to strengthen target features while suppressing the background. To better balance correction and detection, we introduce a detection-guided self-supervised loss to reduce feature discrepancies between the two tasks, thereby enhancing detection robustness to varying nonuniformity levels. Additionally, we construct a new benchmark composed of 50,000 infrared images in various nonuniformity types, multi-scale UAV targets and rich backgrounds with target annotations, called IRBFD. Extensive experiments on IRBFD demonstrate that our UniCD is a robust union framework for NUC and UAV target detection while achieving real-time processing capabilities. Dataset can be available at https://github.com/IVPLaboratory/UniCD.

Unmanned aerial vehicle (UAV) detection based on thermal infrared imaging has been one of the most important sensing technologies in the anti-UAV system. However, the technical limitations and long-range detection of thermal sensors often lead to acquiring low-resolution (LR) infrared images, thereby bringing great challenges for the subsequent target detection task. In this article, we propose a novel spatial and contrast interactive super-resolution network (SCINet) for assisting infrared UAV target detection. The network consists of two main subnetworks: a spatial enhancement branch (SEB) and a contrast enhancement branch (CEB). The SEB embeds the lightweight convolution module and attention mechanism to highlight the spatial structure detail features of infrared UAV targets. The proposed CEB incorporates the center-oriented contrast-aware module and multibranch collapsible module, which can provide local contrast priors to reconstruct the super-resolved UAV target image. The spatial features of the intermediate layers from the SEB are integrated into the CEB as a rich gradient prior. Besides, the output features of the CEB are aggregated into those of the SEB for further supplementing the contrast of spatial features in return. Dual-branch feature interaction (DBFI) of the SEB and CEB can further enhance the spatial details and target saliency of the targets. In addition, we also introduce an infrared UAV detection network via a new dual-dimensional feature calibration module (DFCM) for boosting the detection performance. Extensive experiments demonstrate that the SCINet outperforms the state-of-the-art (SOTA) SR methods on real infrared UAV sequences and improves the detection performance of infrared small UAV targets. The code is available athttps://github.com/IVPLaboratory/SCINet.

Unmanned aerial vehicles (UAVs) have been popular in many commercial and industrial applications, but they also pose great threats to urban safety and aerial security. Intelligent UAV surveillance based on thermal infrared (TIR) imaging has attracted increasing attention for its long-range monitoring ability in both day and night scenarios. However, weak UAV target features, dynamically changing UAV states, and complex background interferences present serious challenges to the accurate tracking of UAVs. To tackle these problems, we propose a novel online multiscale infrared UAV target (IRUT) tracking network (SiamCAP) incorporating enhanced context feature awareness and pixel-wise attention modulation. We first introduce a novel contrast-enhanced multiscale online re-parameterization block (CMORB) to effectively extract contrast difference intensity information between the target and the background, and transform it into a single branch for both training and inference without introducing computational overhead. Then, we construct a feature fusion modulation module (FFMM) to guide cross-layer feature aggregation. It uses low-level attention to highlight the UAV target feature in the deep layer with the novel full spatial resolution channel attention (FSRCA), which calculates pixel-wise importance without dimensionality compression. Finally, we propose a cross-attention-based updatable feature interaction module (CUFIM) to model the correlation between online updating multitemplate and search frame, which improves the model’s robustness to changes in the state of UAVs and complex backgrounds. Extensive experiments on real infrared UAV datasets demonstrate that the proposed approach outperforms the state-of-the-art (SOTA) target trackers under complex backgrounds while achieving a real-time tracking speed.

Multi-scale infrared unmanned aerial vehicle (UAV) targets (IRUTs) detection under dynamic scenarios remains a challenging task due to weak target features, varying shapes and poses, and complex background interference. Current detection methods find it difficult to address the above issues accurately and efficiently. In this paper, we design a dynamic attentive network (DANet) incorporating a scale-adaptive feature enhancement mechanism (SaFEM) and an attention-guided cross-weighting feature aggregator (ACFA). The SaFEM adaptively adjusts the network's receptive fields at hierarchical network levels leveraging separable deformable convolution (SDC), which enhances the network's multi-scale IRUT awareness. The ACFA, modulated by two crossing attention mechanisms, strengthens structural and semantic properties on neighboring levels for the accurate representation of multi-scale IRUT features from different levels. A plug-and-play anti-distractor contrastive regularization (ADCR) is also imposed on our DANet, which enforces similarity on features of targets and distractors from a new uncompressed feature projector (UFP) to increase the network's anti-distractor ability in complex backgrounds. To further increase the multi-scale UAV detection performance of DANet while maintaining its efficiency superiority, we propose a novel scale-specific knowledge distiller (SSKD) based on a divide-and-conquer strategy. For the "divide'' stage, we intendedly construct three task-oriented teachers to learn tailored knowledge for small-, medium-, and large-scale IRUTs. For the "conquer'' stage, we propose a novel element-wise attentive distillation module (EADM), where we employ a pixel-wise attention mechanism to highlight teacher and student IRUT features, and incorporate IRUT-associated prior knowledge for the collaborative transfer of refined multi-scale IRUT features to our DANet. Extensive experiments on real infrared UAV datasets demonstrate that our DANet is able to detect multi-scale UAVs with a satisfactory balance between accuracy and efficiency.

The low-rank tensor model has made great progress for hyperspectral image (HSI) restoration. Recently, the low-rank tensor methods have further been boosted with subspace learning by transforming original HSI into a low-dimensional subspace with reduced computational burden and discriminative feature representation. However, existing subspace-based methods consistently employ a fixed subspace dimension for all patches, which may violate the intrinsic dimension discrepancy of different image content, leading to information loss or redundancy. In this work, our key observation is that the intrinsic subspace of different image patches along different dimensions is different, which should be adaptively modelled for compact feature extraction. Therefore, we propose a content-aware subspace low-rank tensor recovery (CSLRTR) methods by leveraging both deep network and low-rank tensor model. Specifically, we first analyze the intrinsic discrepancy of different HSI patches among both spatial and spectral dimensions, and design a simple network to adaptively learn the optimal subspace dimension. The adaptive subspace learning and low-rank tensor recovery are iteratively performed and mutually promote each other. On one hand, the learned subspace would contribute to more compact low-rank representation for better restoration; on the other hand, the low rank tensor recovery with less degradations would definitely ease the difficulty of the subspace estimation. Note that, the adaptive content-aware subspace strategy has been simultaneously employed on both spectral and nonlocal dimensions, where the spectral-spatial relationship has been further strengthened with better restoration. We have performed extensive experiments on different datasets and restoration tasks, and extended the content-aware subspace strategy to previous methods.

Intelligent unmanned aerial vehicle (UAV) surveillance based on infrared imaging has wide applications in the anti-UAV system for protecting urban security and aerial safety. However, weak target features and complex background distraction pose great challenges for the accurate detection of UAVs. To address this issue, we propose a novel differentiated attention guided network to adaptively strengthen the discriminative features between UAV targets and complex background. First, a novel spatial-aware channel attention (SCA) is introduced into deep layers via preserving critical spatial features and leveraging channel interdependencies to focus on the large-scale targets. The channel-modulated deformable spatial attention is introduced into shallow layers via refining channel context and dynamically perceiving the spatial features for focusing on the small-scale targets. A combination of the above two attention mechanisms is employed in intermediate layers of the network for concentrating on the medium-scale targets. Then, we embed a feature aggregator at the detection branches to guide the information exchange of high-level feature maps and low-level feature maps with a bottom-up context modulation, and integrate an SCA at the end to further boost the distinctive feature representation for task-awareness. The above design can adaptively enhance multiscale UAV target features and suppress complex background interferences, leading to better detection performance, especially for small targets. Extensive experiments on real infrared UAV datasets reveal that the proposed method outperforms the baseline object detectors by a large margin, validating its feasibility in real-world infrared UAV detection. The source code can be found athttps://github.com/KALEIDOSCOPEIP/DAGNet.

Thermal infrared imaging possesses the ability to monitor unmanned aerial vehicles (UAVs) in both day and night conditions. However, long-range detection of the infrared UAVs often suffers from small/dim targets, heavy clutter, and noise in the complex background. The conventional local prior-based and the nonlocal prior-based methods commonly have a high false alarm rate and low detection accuracy. In this letter, we propose a model that converts small UAV detection into a problem of predicting the residual image (i.e., background, clutter, and noise). Such novel reformulation allows us to directly learn a mapping from the input infrared image to the residual image. The constructed image-to-image network integrates the global and the local dilated residual convolution blocks into the U-Net, which can capture local and contextual structure information well and fuse the features at different scales both for image reconstruction. Additionally, subpixel convolution is utilized to upscale the image and avoid image distortion during upsampling. Finally, the small UAV target image is obtained by subtracting the residual image from the input infrared image. The comparative experiments demonstrate that the proposed method outperforms state-of-the-art ones in detecting real-world infrared images with heavy clutter and dim targets.

Local contrast has been proven to be effective in infrared (IR) small target detection, but existing algorithms still have some drawbacks. In this letter, a ratio-difference (RD) local feature contrast (LFC) method is proposed. First, a new nested window is designed, and the local gray contrast is selected as the features of inner and outer windows to investigate the characteristics of a local area. Then, the RD LFC is calculated. At last, a simple but useful weighting function is proposed. Experiments show that the proposed algorithm can achieve better performance in terms of detection rate and false alarm rate and has good robustness against noise.

The blind image deblurring methods have achieved great progress for Gaussian random noise. Few works have paid attention to the image deblurring under the structural noise, which is a very common degradation in multi-detectors imaging systems. This paper considers the practical yet challenging problem of blind deblurring in presence of the line-pattern stripe noise for remote sensing images. To overcome this issue, we explicitly formulate the structural noise into a novel and robust blind image deblurring framework. We observe that the structural line-pattern stripe noise would deteriorate both the kernel estimation and non-blind deblurring, and propose a three-stage restoration framework to progressively estimate the blur kernel and clean image. Specifically, we first estimate an intermediate blur kernel by getting rid of the negative influence of the stripe noise in the unidirectional gradient domain. Next, a learning-based kernel refinement network is introduced to rectify the missing details of the inaccurate kernel. Finally, a low-rank decomposition-based non-blind deblurring model is proposed to simultaneously estimate the clean image and stripe noise. Experimental results on real and synthetic datasets demonstrate that the proposed RBDS method outperforms the state-of-the-art blind deblurring methods.

Hyperspectral imaging, providing abundant spatial and spectral information simultaneously, has attracted a lot of interest in recent years. Unfortunately, due to the hardware limitations, the hyperspectral image (HSI) is vulnerable to various degradations, such as noises (random noise), blurs (Gaussian and uniform blur), and downsampled (both spectral and spatial downsample), each corresponding to the HSI denoising, deblurring, and super-resolution tasks, respectively. Previous HSI restoration methods are designed for one specific task only. Besides, most of them start from the 1-D vector or 2-D matrix models and cannot fully exploit the structurally spectral-spatial correlation in 3-D HSI. To overcome these limitations, in this article, we propose a unified low-rank tensor recovery model for comprehensive HSI restoration tasks, in which nonlocal similarity within spectral-spatial cubic and spectral correlation are simultaneously captured by third-order tensors. Furthermore, to improve the capability and flexibility, we formulate it as a weighted low-rank tensor recovery (WLRTR) model by treating the singular values differently. We demonstrate the reweighed strategy, which has been extensively studied in the matrix, also greatly benefits the tensor modeling. We also consider the stripe noise in HSI as the sparse error by extending WLRTR to robust principal component analysis (WLRTR-RPCA). Extensive experiments demonstrate the proposed WLRTR models consistently outperform state-of-the-art methods in typical HSI low-level vision tasks, including denoising, destriping, deblurring, and super-resolution.

Infrared (IR) images are often contaminated by obvious intensity bias and stripes, which severely affect the visual quality and subsequent applications. It is challenging to eliminate simultaneously the mixed nonuniformity noise without blurring the fine-image details in low-textured IR images. In this article, we present a new model for simultaneous intensity bias correction and destriping through introducing two sparsity constraints. One is that model fit on the intensity bias should be as accurate as possible. A bivariate polynomial model is built to characterize the global smoothness of the intensity bias. The other constraint is that the unidirectional variational sparse model can concisely represent the direction characteristic of stripe noise. A computationally efficient numerical algorithm based on split Bregman iteration is used to solve the complex optimization problem. The proposed method is fundamentally different from the existing denoising techniques and simultaneously estimates the sharp image, intensity bias, and stripe components. Significant improvement on image quality is achieved on both simulated and real studies. Both qualitative and quantitative comparisons with the state-of-the-art correction methods demonstrate its superiority.

In the infrared focal plane arrays imaging systems, the temperature-dependent nonuniformity effects severely degrade the image quality. In this letter, we propose a very deep convolutional neural network for unified infrared aerothermal nonuniform correction. Our network is built with the multiscale and residual training. The multiscale subnetworks utilize the multiscale property in the images, and the long-short-term residual learning contributes to the information propagation. Compared with the previous methods, the proposed method is more robust to various nonuniform artifacts and more efficient at processing time. Experimental results validate the superiority of our method for infrared nonuniform correction.

The spectral and the spatial information in hyperspectral images (HSIs) are the two sides of the same coin. How to jointly model them is the key issue for HSIs' noise removal, including random noise, structural stripe noise, and dead pixels/lines. In this paper, we introduce the deep convolutional neural network (CNN) to achieve this goal. The learned filters can well extract the spatial information within their local receptive filed. Meanwhile, the spectral correlation can be depicted by the multiple channels of the learned 2-D filters, namely, the number of filters in each layer. The consequent advantages of our CNN-based HSI denoising method (HSI-DeNet) over previous methods are threefold. First, the proposed HSI-DeNet can be regarded as a tensor-based method by directly learning the filters in each layer without damaging the spectral-spatial structures. Second, the HSI-DeNet can simultaneously accommodate various kinds of noise in HSIs. Moreover, our method is flexible for both single image and multiple images by slightly modifying the channels of the filters in the first and last layers. Last but not least, our method is extremely fast in the testing phase, which makes it more practical for real application. The proposed HSI-DeNet is extensively evaluated on several HSIs, and outperforms the state-of-the-art HSI-DeNets in terms of both speed and performance.

Multispectral remote sensing images often suffer from the common problem of stripe noise, which greatly degrades the imaging quality and limits the precision of the subsequent processing. The conventional destriping approaches usually remove stripe noise band by band, and show their limitations on different types of stripe noise. In this paper, we tentatively categorize the stripes in remote sensing images in a more comprehensive manner. We propose to treat the multispectral images as a spectral-spatial volume and pose an anisotropic spectral-spatial total variation regularization to enhance the smoothness of solution along both the spectral and spatial dimension. As a result, a more comprehensive stripes and random noise are perfectly removed, while the edges and detail information are well preserved. In addition, the split Bregman iteration method is employed to solve the resulting minimization problem, which highly reduces the computational load. We extensively validate our method under various stripe categories and show comparison with other approaches with respect to result quality, running time, and quantitative assessments.

Remote sensing images destriping and denoising are both classical problems, which have attracted major research efforts separately. This letter shows that the two problems can be successfully solved together within a unified variational framework. To do this, we proposed a joint destriping and denoising method by integrating the unidirectional total variation and sparse representation regularizations. Experimental results on simulated and real data in terms of qualitative and quantitative assessments show significant improvements over conventional methods.

Deblurring and destriping are both classical problems for remote sensing images, which are known to be difficult. Treating deblurring and destriping separately, such a straightforward approach, however, suffers greatly from the defective output. This paper shows that the two problems can be successfully solved together and benefit greatly from each other within a unified variational framework. To do this, we propose a joint deblurring and destriping method by combining the framelet regularization and unidirectional total variation. Extensive experiments on simulation and real remote sensing images are carried out and the results of our joint model show significant improvement over conventional methods of treating the two tasks separately.

Atmospheric turbulence affects imaging systems by virtue of wave propagation through a medium with a nonuniform index of refraction. It can lead to blurring in images acquired from a long distance away. In this letter, it is observed that blurring increases the second-order central moment (SOCM) of images, and we introduce a new parametric blur identification method by minimizing SOCM. The method applies to finite-support images, in which the scene consists of a finite-extent object against a uniformly black, gray, or white background. The SOCM method has been validated by direct comparisons with other methods on simulated and real degraded images.