Achieving pixel-level registration between SAR and optical images remains a challenging task due to their fundamentally different imaging mechanisms and visual characteristics. Although deep learning has achieved great success in many cross-modal tasks, its performance on SAR-Optical registration tasks is still unsatisfactory. Gradient-based information has traditionally played a crucial role in handcrafted descriptors by highlighting structural differences. However, such gradient cues have not been effectively leveraged in deep learning frameworks for SAR-Optical image matching. To address this gap, we propose SOMA, a dense registration framework that integrates structural gradient priors into deep features and refines alignment through a hybrid matching strategy. Specifically, we introduce the Feature Gradient Enhancer (FGE), which embeds multi-scale, multi-directional gradient filters into the feature space using attention and reconstruction mechanisms to boost feature distinctiveness. Furthermore, we propose the Global-Local Affine-Flow Matcher (GLAM), which combines affine transformation and flow-based refinement within a coarse-to-fine architecture to ensure both structural consistency and local accuracy. Experimental results demonstrate that SOMA significantly improves registration precision, increasing the CMR@1px by 12.29% on the SEN1-2 dataset and 18.50% on the GFGE_SO dataset. In addition, SOMA exhibits strong robustness and generalizes well across diverse scenes and resolutions.

Semantic segmentation of river ice image serves as a critical technological foundation for hydrological monitoring and ice flood early warning system. Current publicly available river ice datasets predominantly utilize UAV-captured image and ground-based photographic observations. To address the limitations of spatial coverage in existing datasets, we present NWPU_YRCC_GFICE - a satellite remote sensing dataset constructed from multi-spectral GF-2 satellite images. The dataset innovatively categorizes river ice into six fine-grained classes across freeze-thaw cycles and covers river ice data from Yellow River (Ningxia-Inner Mongolia section) spanning the past 10 years. We further establish a comprehensive deep learning benchmark, which evaluates 33 state-of-the-art segmentation models and two improved segmentation models based on YOLO and Segformer architecture, separately. Experiments are conducted on the NWPU_YRCC_GFICE dataset and three public river ice datasets (NWPU_YRCC_EX, NWPU_YRCC2, and Alberta river ice segmentation dataset). The proposed models exhibit excellent performance, surpassing the state-of-the-art methods. The presented NWPU_YRCC_GFICE dataset and benchmark enriches the river ice dataset and favors in promoting fine-grained river ice segmentation research from satellite view. Our dataset and code is available at https://github.com/ASGOLabMultisourceCooperationGroup/NWPU_YRCC_GFICE.

Real-time semantic segmentation of remote-sensing images demands a trade-off between speed and accuracy, which makes it challenging. Apart from manually designed networks, researchers seek to adopt neural architecture search (NAS) to discover a real-time semantic segmentation model with optimal performance automatically. Most existing NAS methods stack up no more than two types of searched cells, omitting the characteristics of resolution variation. This paper proposes the Hierarchical shared Architecture Search (HAS) method to automatically build a real-time semantic segmentation model for remote sensing images. Our model contains a lightweight backbone and a multi-scale feature fusion module. The lightweight backbone is carefully designed with low computational cost. The multi-scale feature fusion module is searched using the NAS method, where only the blocks from the same layer share identical cells. Extensive experiments reveal that our searched real-time semantic segmentation model of remote sensing images achieves the state-of-the-art trade-off between accuracy and speed. Specifically, on the LoveDA, Potsdam, and Vaihingen datasets, the searched network achieves 54.5% mIoU, 87.8% mIoU, and 84.1% mIoU, respectively, with an inference speed of 132.7 FPS. Besides, our searched network achieves 72.6% mIoU at 164.0 FPS on the CityScapes dataset and 72.3% mIoU at 186.4 FPS on the CamVid dataset.

Detecting changes in heterogeneous images without the supervision of changed label is a challenging yet critical task for quick responding natural disaster relief. Nevertheless, most of available unsupervised heterogeneous change detection methods strong rely on the quality of pseudo labels, and they suffer from performance degradation, even irreversible model collapse, when encounter the low-quality pseudo labels, leading to unreliable detection results. In order to improve the reliability of unsupervised heterogeneous change detection, in this paper, we propose a novel change detection paradigm based on sample synthesis and knowledge transfer. We address the issue of label reliability by artificially creating a changed region and assigning labels rather than constructing pseudo labels. These constructed labels guide the network in automatically learning the correspondence between heterogeneous images, confirming the reliability of changed regions. Moreover, an augmentation with synthetic samples on real samples makes it possible to generate more transferable samples while reducing the domain gap coarsely. A dual-branch joint training with feature contrastive learning is further developed to transfer the knowledge of changes from the synthetic sample domain to real sample domain. Experimental results on five public datasets demonstrate that our proposed method has superior performance when compared with available state-of-the-art methods. Our code is available at https://github.com/zhangqiiii/SS-KT.

Multi-modal image matching is very challenging due to the significant diversities in visual appearance of different modal images. Typically, the existing well-performed methods mainly focus on learning invariant and discriminative features for measuring the relation between multi-modal image pairs. However, these methods often take the features as a whole and largely overlook the fact that different scale features for a same image pair may have different similarity, which may lead to sub-optimal results only. In this work, we propose a Scale-Separative Metric Learning Quadruplet network (SSML-QNet) for multi-modal image patch matching. Specifically, SSML-QNet can extract both relevant and irrelevant features of imaging modality with the proposed quadruplet network architecture. Then, the proposed Scale-Separative Metric Learning module separately encodes the similarity of different scale features with the pyramid structure. And for each scale, cross-modal consistent features are extracted and measured by coordinate and channel-wise attention sequentially. This makes our network robust to appearance divergence caused by different imaging mechanism. Experiments on the benchmark dataset (VIS-NIR, VIS-LWIR, Optical-SAR, and Brown) have verified that the proposed SSML-QNet is able to outperform other state-of-the-art methods. Furthermore, the cross-dataset transferring experiments on these four datasets also have shown that the proposed method has powerful ability of cross-dataset transferring.

Recent RGB-D semantic segmentation networks are usually manually designed. However, due to limited human efforts and time costs, their performance might be inferior for complex scenarios. To address this issue, we propose the first Neural Architecture Search (NAS) method that designs the network automatically. Specifically, the target network consists of an encoder and a decoder. The encoder is designed with two independent branches, where each branch specializes in extracting features from RGB and depth images, respectively. The decoder fuses the features and generates the final segmentation result. Besides, for automatic network design, we design a grid-like network-level search space combined with a hierarchical cell-level search space. By further developing an effective gradient-based search strategy, the network structure with hierarchical cell architectures is discovered. Extensive results on two datasets show that the proposed method outperforms the state-of-the-art approaches, which achieves a mIoU score of 55.1% on the NYU-Depth v2 dataset and 50.3% on the SUN-RGBD dataset.

Change detection based on heterogeneous images is of great importance in some applications, such as disaster monitoring and damage assessment. However, due to the huge modality discrepancy in heterogeneous images, it is difficult to accurately detect the changed regions. In this paper, we analyze the interference of modality-alignment and changed areas to each other, and propose a progressive modality-alignment based unsupervised change detection model for heterogeneous images. Specifically, the modality alignment is achieved in an iterative manner, which can improve the detection accuracy progressively. To reduce the influence of modality discrepancy and the changed regions to each other, a pseudo-label self-learning strategy is designed, where the pseudo-labels learned by the model itself are used to act as a guidance of change detection, and they are in turn refined by the proposed progressive model. Experimental results on different real heterogeneous images verify the effectiveness and robustness of proposed method.

Image-based geo-localization is estimating the location of a query image by matching it to a large amount of images in geo-tagged database. This matching task is very challenging due to the vast differences in visual appearance or modality of image pairs on different platforms, for example, one image from the RGB camera, the other from the light detection and ranging (LiDAR) sensor. The spatial layout of the scene can provide important clues and significantly reduce matching ambiguity. Therefore, we propose a novel deep network that embeds spatial configuration of the scenes into feature representation. Specifically, we design a spatial-scale attention (SSA) module to highlight the salience correspondence layout features at different scales. The encoded features not only represent the emergence of certain objects, but also reflect the relative locations of the objects. By this way, we learn more discriminative deep feature representations, leading to a higher recall. The experimental results on two standard cross-view benchmark datasets (CVUSA and CVACT) and a cross-modal dataset (GRAL) demonstrate that our method performs better than the state-of-the-art methods. Remarkably, the recall rate@top-1 improves from 27.6% to 40.5% on the GRAL dataset.

Change detection (CD) is one of the most important topics in the field of remote sensing. In this letter, we propose an effective satellite images CD network named DifUnet++. As the presentation of explicit difference is more conducive to extract change features, we design a differential pyramid of two input images as the input of Unet++. Considering the scale diversity of changed regions in remote sensing images, a multiply side-outs fusion strategy is adopted to predict the detection results of different scales. Furthermore, a learning upsampling method is utilized to refine the details of CD. The proposed architecture is evaluated on two public satellite image CD data sets. The experimental results show that our method performs much better than state-of-the-art methods.

With the development of deep learning, change detection technology has gained great progress. However, how to effectively extract multi-scale substantive changed features and accurately detect small changed objects as well as the accurate details is still a challenge. To solve the problem, we propose Attentived Differential High-Resolution Change Detection Network (ADHR-CDNet) for remote sensing images. In ADHR-CDNet, a novel high-resolution backbone with a Differential Pyramid Module (DPM) is proposed to extract multi-level and multi-scale substantive changed features. The backbone structure with four interconnected sub-network branches of different resolution is helpful to extract multi-level and multi-scale features. DPM is capable of distinguishing between substantive changes and pseudo changes induced by illumination, shadow, seasonal variation, and so on. Then, a novel Multi-Scale Spatial feature Attention Module (MSSAM) is presented to effectively fuse the spatial detail information of different scale features produced by our backbone to generate finer prediction. We conduct quantitative and qualitative experiments on three public change detection datasets: the Lebedev, the LEVIR-CD, and the WHU Building dataset. The proposed ADHR-CDNet reaches F1-score of 97.2% (improved 3.1%) on the Lebedev dataset, 91.4% (improved 1.6%) on the LEVIR-CD dataset, and 90.9% (improved 1.2%) on the WHU Building dataset. The experimental results demonstrate that our method performs much better than the state-of-the-art methods. The visualization comparison results show that our method can effectively detect small changed objects and significantly improve the details of detected changed objects. Our code is available at https://github.com/w-here/ASGO-113lab/tree/main/ADHR-CDNet.

To extend existing distance metrics in the$L^{p}$space, we define a novel distance, named relative Euclidean distance (RED), and prove that it is positive definite, symmetric, and satisfies the triangle inequality. This distance has a value range of [0, 1]. We prove a uniqueness of it in the$L^{p}$space. The technique for order preference by similarity to ideal solution (TOPSIS) is widely used for multiple-attribute decision problems. The commonly used relative closeness measure in the TOPSIS is not a mathematical distance and thus does not have all the nice properties of such a distance. To improve the TOPSIS, particularly its relative closeness measure, we propose two measures, named optimistic distance (OD) and pessimistic distance (PD), based on our RED, and use them to measure the relative closeness in the TOPSIS. Both measures are positive definite, symmetric, and satisfy the triangle inequality. The best choice in OD (or PD) is on the Pareto frontier. Three examples of performance ranking are given to illustrate the usefulness of the TOPSIS with our proposed measures.

Given multiple statistical estimators, how to rank their performance is a worthwhile problem. We propose an approach to estimation performance ranking based on pairwise comparison. Because ranking is about two or more estimators and is relative, pairwise comparison (e.g., Pitman’s measure of closeness) suits the needs. However, pairwise comparison cannot guarantee transitivity, which is needed for ranking. To get around this problem, we propose a ranking vector (RV) approach based on pairwise comparison. Here, an RV is obtained by using pairwise comparison results without using pairwise ranks directly. An RV provides ordinal information determining the rank and also supplementary cardinal information exhibiting how much one estimator is better than another. Ordinal information is more important and thus is guaranteed by using an order-preserving mapping in obtaining an RV. Our RV approach based on pairwise comparison is also applied to multiple-attribute decision problems. The approach is easily applicable and it does not need data normalization.

If a data set has a large range (e.g., the large elements are several orders of magnitude greater than the small elements), then the median is usually applied to measure its central tendency. However, it has two drawbacks. A novel measure of central tendency called iterative mid-range (IMR) is proposed. It has several attractive properties and can overcome the drawbacks of the median. Estimation performance is often evaluated in a statistical sense by the Monte Carlo method. Given a set of estimation errors, estimation performance is evaluated by measures of central tendency of error, that is, by finding a typical value (e.g., root-mean-square error) to represent the errors. The proposed IMR is applied to estimation performance evaluation, and it is named IMR error (IMRE). This letter advocates replacing the median by our proposed IMR in many cases.