To address the challenges posed by extensive modality differences in image matching tasks, this paper proposes a cascaded learning framework. It guides the optimization of an end-to-end matching model via a dynamic data engine, which can provide sufficient cross-modal training data to support the model’s full adaptation to cross-modal features. The data engine integrates a random homography transformation module and a lightweight image generation model, enabling the online synthesis of cross-modal image pairs with geometric variations and diverse styles. This provides the matching model with rich cross-modal stimulation. The matching model adopts a hybrid architecture combining a CNN backbone and Transformer attention mechanisms, which integrates multi-scale local feature extraction with global context modeling. By adopting the proposed stepwise aggregation strategy, the efficiency of feature extraction is well ensured. Subsequently, a coarse-to-fine matching strategy is employed to achieve high accuracy and robustness of feature alignment. Comprehensive experiments on both self-collected and public cross-modal image matching datasets demonstrate that the proposed DGIM outperforms existing state-of-the-art approaches in cross-modal matching performance while achieving a good balance between efficiency and effectiveness. It also exhibits broad practical potential across multiple fields and scenes. This work provides novel solutions and evaluation benchmarks for cross-modal image matching tasks. The code and testing dataset will be made publicly available at https://github.com/LotrL/DGIM.

Few-shot learning enables rapid generalization from extremely limited training examples. While previous efforts have utilized meta-learning or data augmentation methods to mitigate the problem of data scarcity, such approaches may struggle to maintain robustness and generalize effectively due to overfitting and noise sensitivity. In this paper, we propose a novel approach, the Semi-Supervised Label Correction method for Few-Shot Learning (SSLC-FSL), which leverages the data distribution of readily available and easily obtainable unlabeled data. SSLC-FSL iteratively corrects the labels of testing samples with alternating steps of pseudo-labeling and sample selection. The objective of pseudo-labeling is to repurpose graph-based semi-supervised learning for joint prediction of the entire testing set. We then introduce a Modulation Selection Network (MSN) to rank testing samples by learning with noisy labels. The training set is expanded by selecting confident pseudo-labeled samples. In the MSN, a Modulation Aggregation Layer is designed to encode support class information into each testing sample, thereby highlighting target category features and mitigating the negative impact of incorrect labels. The iterative label correction process is repeated until all testing samples are recalled to the expanded support set. To boost the SSLC-FSL algorithm, we pre-train a feature extractor to produce general-purpose representations. Particularly, we investigate two types of auxiliary tasks and their collaborative learning to acquire transferable visual information via an end-to-end multi-task learning model. Our SSLC-FSL outperforms current state-of-the-art methods in any shot and all data settings, with up to +27.74% on standard remote sensing benchmarks and +5.70% on standard natural scene benchmarks.

In this letter, a novel neural network (CDMI-Net) that combines change detection and multiple instance learning (MIL) is proposed for landslide mapping. After obtaining a score map of landslides provided by the network, the final binary map is generated by fast postprocessing. The benefits of the proposed method are threefold. First, using the MIL framework, the network is trained only by the scene-level samples and it reduces the need for pixel-level samples. Second, a change-detection network architecture using a two-stream U-Net with shared weights is designed to learn the deep features of the landslide from the two-period aerial images, reducing the false-positive results. Third, integrating a gated attention-based pooling layer and a fast level-set evolution algorithm can finally produce the pixel-level results. Experimental results show that the proposed CDMI-Net achieves comparable and even better performance on the testing image pairs than all other methods and has great potential for the landslide mapping application.