Recently, cross-spectral image patch matching based on feature relation learning has attracted extensive attention. However, existing methods focus on mining richer feature relations by building complex relation extraction structures. Meanwhile, performance bottlenecks have gradually emerged. To address this, we make the first attempt to explore a stable and efficient bridge between descriptor learning and metric learning, and construct a Knowledge-Guided Learning Network (KGL-Net), which achieves significant performance improvements while abandoning complex network structures. Specifically, we find that there is feature extraction consistency between metric learning based on feature difference learning and descriptor learning based on Euclidean distance. This provides the foundation for bridge building. To ensure the stability and efficiency of the constructed bridge, on the one hand, we conduct an in-depth exploration of 20 combined network architectures. On the other hand, a feature-guided loss is constructed to achieve mutual guidance of features. In addition, unlike existing methods, we consider that the feature mapping ability of the metric branch should receive more attention. Therefore, a hard negative sample mining for metric learning (HNSM-M) strategy is constructed. To the best of our knowledge, this is the first time that hard negative sample mining for metric networks has been implemented and brings significant performance gains. Extensive experimental results show that our KGL-Net achieves SOTA performance in multiple cross-spectral image patch matching datasets. Our code is available at https://github.com/YuChuang1205/KGL-Net.

Recently, single-frame infrared small target (SIRST) detection with single point supervision has drawn wide-spread attention. However, the latest label evolution with single point supervision (LESPS) framework suffers from instability, excessive label evolution, and difficulty in exerting embedded network performance. Inspired by organisms gradually adapting to their environment and continuously accumulating knowledge, we construct an innovative Progressive Active Learning (PAL) framework, which drives the existing SIRST detection networks progressively and actively recognizes and learns harder samples. Specifically, to avoid the early low-performance model leading to the wrong selection of hard samples, we propose a model pre-start concept, which focuses on automatically selecting a portion of easy samples and helping the model have basic task-specific learning capabilities. Meanwhile, we propose a refined dual-update strategy, which can promote reasonable learning of harder samples and continuous refinement of pseudo-labels. In addition, to alleviate the risk of excessive label evolution, a decay factor is reasonably introduced, which helps to achieve a dynamic balance between the expansion and contraction of target annotations. Extensive experiments show that existing SIRST detection networks equipped with our PAL framework have achieved state-of-the-art (SOTA) results on multiple public datasets. Furthermore, our PAL framework can build an efficient and stable bridge between full supervision and single point supervision tasks. Our code is available at https://github.com/YuChuang1205/PAL

Infrared small target detection faces the problem that it is difficult to effectively separate the background and the target. Existing deep learning-based methods focus on edge and shape features, but ignore the richer structural differences and detailed information embedded in high-frequency components from different directions, thereby failing to fully exploit the value of high-frequency directional features in target perception. To address this limitation, we propose a multi-scale direction-aware network (MSDA-Net), which is the first attempt to integrate the high-frequency directional features of infrared small targets as domain prior knowledge into neural networks. Specifically, to fully mine the high-frequency directional features, on the one hand, a high-frequency direction injection (HFDI) module without trainable parameters is constructed to inject the high-frequency directional information of the original image into the network. On the other hand, a multi-scale direction-aware (MSDA) module is constructed, which promotes the full extraction of local relations at different scales and the full perception of key features in different directions. In addition, considering the characteristics of infrared small targets, we construct a feature aggregation (FA) structure to address target disappearance in high-level feature maps, and a feature calibration fusion (FCF) module to alleviate feature bias during cross-layer feature fusion. Extensive experimental results show that our MSDA-Net achieves state-of-the-art (SOTA) results on multiple public datasets. The code can be available at https://github.com/YuChuang1205/MSDA-Net.

Deep learning has contributed to the rapid development of building extraction tasks from remote sensing(RS) images. Existing models typically leverage a segmentation-head to predict results, where multi-channel feature maps extracted by the network are directly output as single-channel predictions. However, it is rarely noticed that this process results in a loss of features, which can lead to incomplete extraction of smaller buildings. Besides, boundary-blurring is also a common problem in the task. Therefore, in this letter, we propose a Siamese Prototype Contrastive Learning Network (SPCL-Net) to address these two problems. In the network, a novel Prototype Contrastive Learning (PCL) module is proposed to alleviate feature loss problem by applying contrastive learning between prototype vectors. In addition, a Reverse Boundary Enhancement (RBE) module is proposed to facilitate the representation of building boundaries and mitigate the boundary-blurring problem. Experiments are conducted on two datasets, INRIA and WHU. Compared with existing models, the final results show that the proposed approach is better than theirs in terms of evaluation metrics IOU.

Recently, infrared small target detection has attracted extensive attention. However, due to the small size and the lack of intrinsic features of infrared small targets, the existing methods generally have the problem of inaccurate edge positioning and the target is easily submerged by the background. Therefore, we propose an innovative gradient-guided learning network (GGL-Net). Specifically, we are the first to explore the introduction of gradient magnitude images into the deep learning-based infrared small target detection method, which is conducive to emphasizing the edge details and alleviating the problem of inaccurate edge positioning of small targets. On this basis, we propose a novel dual-branch feature extraction network that utilizes the proposed gradient supplementary module (GSM) to encode raw gradient information into deeper network layers and embeds attention mechanisms reasonably to enhance feature extraction ability. In addition, we construct a two-way guidance fusion module (TGFM), which fully considers the characteristics of feature maps at different levels. It can facilitate the effective fusion of multi-scale feature maps and extract richer semantic information and detailed information through reasonable two-way guidance. Extensive experiments prove that GGL-Net has achieves state-of-the-art results on the public real NUAA-SIRST dataset and the public synthetic NUDT-SIRST dataset.

Recently, cross-spectral image patch matching methods based on feature difference aggregation have achieved excellent performance, but they introduce a large number of parameters, limit matching speed and have poor scalability. At the same time, only using feature difference learning to extract differential features will lead to the loss of consistent features between cross-spectral image patches. Therefore, we construct a novel four-branch efficient feature relation learning network (EFR-Net) without feature difference aggregation. Specifically, a new four-branch feature relation learning strategy is proposed, which reasonably combines multiple feature relation learning to comprehensively and effectively extract the differential features and consistent features between image patches. At the same time, we construct an efficient local attention (ELA) module with negligible parameters, which can learn some global context information, enhance the interaction of local information and promote the extraction of discriminative features. In addition, a combined metric network is introduced to facilitate network optimization and improve network generalization. Furthermore, a public optical and SAR image patch matching dataset with a patch size of 64 × 64 pixels is constructed based on the OS dataset, which is called the OS patch dataset. We also establish an experimental benchmark on this new dataset. Extensive experimental results show that the proposed EFR-Net achieves excellent performance on cross-spectral image patch matching (OS patch dataset, VIS-NIR patch dataset) and single spectral image patch matching (Brown dataset).

Recently, feature relation learning has attracted extensive attention in cross-spectral image patch matching. However, most feature relation learning methods can only extract shallow feature relations and are accompanied by the loss of useful discriminative features or the introduction of disturbing features. Although the latest multi-branch feature difference learning network can relatively sufficiently extract useful discriminative features, the multi-branch network structure it adopts has a large number of parameters. Therefore, we propose a novel two-branch feature interaction learning network (FIL-Net). Specifically, a novel feature interaction learning idea for cross-spectral image patch matching is proposed, and a new feature interaction learning module is constructed, which can effectively mine common and private features between cross-spectral image patches, and extract richer and deeper feature relations with invariance and discriminability. At the same time, we re-explore the feature extraction network for the cross-spectral image patch matching task, and a new two-branch residual feature extraction network with stronger feature extraction capabilities is constructed. In addition, we propose a new multi-loss strong-constrained optimization strategy, which can facilitate reasonable network optimization and efficient extraction of invariant and discriminative features. Furthermore, a public VIS-LWIR patch dataset and a public SEN1-2 patch dataset are constructed. At the same time, the corresponding experimental benchmarks are established, which are convenient for future research while solving few existing cross-spectral image patch matching datasets. Extensive experiments show that the proposed FIL-Net achieves state-of-the-art performance in three different cross-spectral image patch matching scenarios.

Infrared small target suffers from the lack of intrinsic features, context and samples. Conventional detection methods are usually unable to sufficiently and effectively extract the features of infrared small targets. Therefore, we propose a novel attention-based local contrast learning network (ALCL-Net). Considering the scarcity of intrinsic features of infrared small targets, we propose ResNet32, which enhances the ability to extract infrared small target features and avoids the problem that the target features are overwhelmed by the background features due to too deep network. At the same time, we construct a simplified bilinear interpolation attention module (SBAM), which is used for fusion of hierarchical feature maps. It has fast inference speed and can focus on the feature of the target in the lack of context. Furthermore, local contrast learning (LCL) is introduced, which adopts the local contrast idea of non-deep learning methods. It can alleviate the dependence on dataset samples, thereby improving detection accuracy on datasets with few samples. Compared with the state-of-the-art methods, the proposed ALCL-Net achieves superior performance with an intersection-over-union (IoU) of 0.792 and normalized IoU (nIoU) of 0.771 on the public SIRST dataset.

Cross-spectral image patch matching is still challenging due to significant nonlinear differences between image patches. Recently, image patch matching methods based on feature relation learning have attracted increasing attention and achieved good performance. However, we find that the metric learning methods based on feature difference cannot comprehensively and effectively extract useful discriminative information between image patch pairs by only adopting two branches network structure. Therefore, we propose a novel multi-branch feature difference learning network (MFD-Net). Specifically, we build a multi-branch parallel feature difference extraction network, which can capture richer and more discriminative feature difference information and achieve significant improvements on matching tasks. Furthermore, we propose a combined metric network composed of a master metric network module and multiple branch metric network modules, which promotes the forward update of network weights and reduces the similarity of features extracted by each feature difference extraction module with negligible increase in inference time. Extensive experimental results show that the proposed MFD-Net achieves superior performances on cross-spectral image patch matching and single spectral image patch matching.