The multi-modality remote sensing foundation model (MM-RSFM) has made notable progress recently. However, most existing approaches remain limited to medium-resolution, single-modality, restricting their performance in fine-grained downstream applications such as disaster response and urban planning. In this work, MaRS is proposed, a multi-modality very-high-resolution (VHR) remote sensing foundation model designed for cross-modality granularity interpretation of complex scenes. To achieve this, a multi-modality VHR SAR-optical dataset, MaRS-16M, is constructed through large-scale collection and semi-automated processing, comprising over 16 million paired samples. Unlike previous work, MaRS tackles two fundamental challenges in VHR SAR-optical self-supervised learning (SSL) techniques. Cross-granularity contrastive learning (CGCL) is introduced to alleviate alignment inconsistencies caused by imaging differences, and meta-modality attention (MMA) is designed to unify heterogeneous physical characteristics across modalities. Compared to existing remote sensing foundation models (RSFMs) and general vision foundation models (VFMs), MaRS performs better as a pre-trained backbone across nine multi-modality VHR downstream tasks.

Semantic change detection (SCD) involves detecting changed regions and classifying their corresponding semantic change categories in remote sensing images. However, in most SCD scenarios, only the land-cover category information of the changed regions is provided. The mainstream multi-task Siamese network optimization process fails to fully leverage the information from unchanged land-cover types, which limits its ability to recognize land-cover change types and ultimately limits SCD performance. To address the issue of suppressed change type recognition caused by sparse labels in the SCD task, this paper proposes a confusion-aware contrastive-based semi-supervised semantic change detection method through space collaboration (SC2A-SCD). The proposed SC2A-SCD framework first integrates bi-temporal land-cover classification (LCC)-predicted logits from both the classification and representation spaces using joint classification and representation space modeling (JSM), providing high-quality pseudo-labels for unchanged land-cover types and enhancing the model’s ability to recognize bi-temporal land-cover types. Meanwhile, confusion-aware contrastive learning (CACL) is conducted in the representation space, utilizing confusion sampling strategies to sample more confusable anchors and negatives that are prone to misclassification, thereby effectively improving the representation capability of bi-temporal land-cover types and enhancing the quality of the pseudo-labels obtained via JSM, ultimately boosting the ability of SC2A-SCD to recognize semantic changes. Compared to the state-of-the-art SCD methods, the comprehensive experimental results confirm that the proposed SC2A-SCD framework can effectively improve the recognition ability for change types, demonstrating its effectiveness in enhancing SCD performance.

Local Climate Zones (LCZs) offer a standardized urban classification system critical for studying climate variations and the urban heat island effect. Large-scale LCZ mapping facilitates cross-city climate comparisons. Remote sensing (RS), particularly supervised learning, has become the primary LCZ classification method due to satellite imagery’s broad coverage and high resolution. However, current RS approaches face challenges, including high labeling demands and limited transferability. In this work, we aim to leverage limited labeled data from a source city (i.e., source domain) to improve LCZ classification performance for multiple unlabeled target cities (i.e., target domains). To address these challenges, this work proposes a novel Source-free Multi-target Unsupervised Domain Adaptation (SFMT-UDA) framework for cross-city LCZ classification. Our approach operates in two stages: first performing single-target adaptation between source and target domains using a self-supervised pseudo-labeling module enhanced by an LCZ class similarity matrix to reduce label noise, then conducting multi-target adaptation among target domains through a Multi-head Multi-target Domain Adaptation (MH-MTDA) network with an integrated style-transfer module for consistent self-training. Extensive experiments on the newly developed VHRLCZ dataset demonstrate that the proposed SFMT-UDA framework achieves superior performance compared to state-of-the-art methods, showing significant improvements in overall accuracy across multiple target domains. The related code and data are available at https://github.com/ctrlovefly/SFMT-UDA.

Semantic Change Detection (SCD) is recognized as both a crucial and challenging task in the field of image analysis. Traditional methods for SCD have predominantly relied on the comparison of image pairs. However, this approach is significantly hindered by substantial imaging differences, which arise due to variations in shooting times, atmospheric conditions, and angles. Such discrepancies lead to two primary issues: the under-detection of minor yet significant changes, and the generation of false alarms due to temporal variances. These factors often result in unchanged objects appearing markedly different in multi-temporal images. In response to these challenges, the MapChange framework has been developed. This framework introduces a novel paradigm that synergizes temporal-invariant historical map data with contemporary high-resolution images. By employing this combination, the temporal variance inherent in conventional image pair comparisons is effectively mitigated. The efficacy of the MapChange framework has been empirically validated through comprehensive testing on two public datasets. These tests have demonstrated the framework's marked superiority over existing state-of-the-art SCD methods.

The advent of high-resolution satellite technology has significantly improved the accuracy of land cover classification, but it also presents new challenges. Deep learning methods, particularly Convolutional Neural Networks (CNNs) and Transformers, have shown promise in overcoming these challenges, yet they are highly reliant on extensive annotated datasets. An alternative, using lower resolution land cover products for label generation, introduces Label Noise, which can adversely affect the learning process and model performance. To address the lack of quality labels and the Label Noise problem, this study introduces the LabelAgent framework. This framework combines high-resolution, recent remote sensing data with historical, low-resolution product data to produce higher quality training labels. Furthermore, it employs a deep reinforcement learning-based approach to intelligently identify and mitigate noisy labels, thereby stabilizing the training process and enhancing the classification efficacy. The outcome of this research offers a novel solution to the challenges in remote sensing image classification, promising improved accuracy and generalization capability for land cover classification models.

Tidal wetlands, situated crucially at the land-sea intersection, play a pivotal role in sustainable development. Remote sensing has become a useful method for large-scale tidal wetland mapping. However, it encounters challenges due to tidal wetlands exhibit considerable similarity in spectral characteristics and show significant seasonal variations. The existing machine-learning-based methods are difficult to achieve high-efficiency mapping, hence restricting their utility in wetland monitoring. Conversely, deep learning methods possess a significant edge in managing intricate data, resulting in better classification accuracy. However, the requirement of pixel-level labels for deep learning has posed difficulties for extensive network training. In this article, a framework based on historical point samples is introduced for large-scale tidal wetland mapping. The proposed method uses existing point labels to generate per-pixel pseudo-labels, maximizing the utilization of historical data, and utilizing methods such as CutMix and bootstrapping loss to deal with label noise.

Extracting road network structures with high accuracy from very high-resolution (VHR) remote sensing imagery is a challenging task in the field of computer vision. Previous end-to-end algorithms modeled the road graph as a general graph structure with defined vertices and edges, represented as G = (V, E), which is a default approach in graph generation tasks. However, the use of excessively small edge units conflicts with the network’s geometric structure and potential object features of roads, leading to issues such as false alarm roads and disconnected roads. In this study, we propose a road graph generation model based on road segments, represented as G = (I, S), leveraging the long-distance connectivity of minimal road topological units. This approach aims to balance the connectivity and precision of the generated road graph. Subsequently, we introduce a road graph generation framework named RoadSegment, based on the aforementioned modeling, to validate the feasibility of the proposed road segment modeling. RoadSegment utilizes an Element Detector to identify intersections and road segment objects. Additionally, we propose an Intersection Connection Strategy (ICS) to establish connectivity between road segments. Empirical experiments conducted on the SpaceNet3 Dataset demonstrate the superiority of road segment modeling.

Understanding land cover dynamics, especially in wetlands, is crucial for environmental monitoring and ecosystem management, given the threats posed by climate change and human activities. Despite the existence of global-scale land cover maps and detailed wetland thematic maps, a gap remains in their integration, limiting comprehensive ecosystem analysis. Typically, these methods utilize medium-resolution data, which fail to capture the finer details of land cover and wetland interfaces, highlighting the need for high-resolution mapping. As high-resolution mapping becomes more prevalent, the classification of land cover and wetland classes grows increasingly complex. Deep learning methods, particularly semantic segmentation, have emerged as solutions, yet they require extensive training labels, which are challenging and resource-intensive to acquire. This paper proposes a novel framework utilizing historical land cover products as training labels for high-resolution mapping, addressing the challenges of resolution mismatches and semantic errors. The developed cross-resolution mapping dataset, evaluated against a manually labeled test set, demonstrates the framework's effectiveness in providing detailed classification at a provincial scale.

Extracting road vector graphs with high accuracy from high-resolution remote sensing imagery presents a significant challenge. The prior end-to-end algorithms have typically modeled the road graph as a general graph structure with vertices and edges, denoted as$G=(V,E)$, which is a standard approach in graph generation tasks. However, the traditional$G=(V,E)$graph representation with vertices and edges, utilizing very small edge units, can conflict with the road network’s geometric structure and the inherent features of road instances, leading to issues such as false positives and disconnected roads. In this article, the IS-RoadDet framework is proposed to generate a road vector graph with intersections (I) and road segments (S), denoted as$G=(I,S)$, which leverages the minimum road topology unit features of road to improve road topology. Compared to$G=(V,E)$, instead of detecting a large number of vertices to maintain the road topology connectivity,$G=(I,S)$uses road segments with minimum road units to avoid false positives. In IS-RoadDet, the intersection and road segment detector (ISDetector) is introduced to detect intersections and road segments as independent instance objects with joint learning, and an intersection connectivity strategy (ICS) is designed to establish connectivity between road segments with intersections. Empirical experiments conducted on the SpaceNet3 and Sat2Graph datasets substantiated the superior performance of the proposed road segment modeling method. The code will be made available at:https://github.com/WanderRainy/IS-Roadandhttps://rsidea.whu.edu.cn/is-road.htm.

Road occlusion seriously affects the connectivity of extracted roads, and has a negative effect in practical applications. The dense road occlusion problem is caused by high-rise buildings and street trees, and is a more serious and unique problem than simple occlusion caused by low buildings and scattered trees. The existing methods mainly solve the road occlusion problem by enhancing the encoder ability to capture the long connectivity feature of roads. Unfortunately, the existing methods can only solve small and sparse road occlusion situations, and they cannot deal with the dense road occlusions caused by dense high-rise buildings or trees. In this article, to solve the dense road occlusion problem, the occlusion-aware road extraction network, namely OARENet, is proposed for road extraction from high-resolution remote sensing imagery. In OARENet, an occlusion-aware decoder (OADecoder) is designed by explicit modeling the texture feature for road regions with dense occlusions. The OADecoder is made up of a regular occlusion-aware (ROA) module and a stochastic occlusion-aware (SOA) module. The ROA module is implemented by adopting different dilation rates to fit the texture feature in the semantic feature maps. The SOA module is proposed by designing stochastic convolutions to adaptively fit the spatial details of road regions with dense occlusions. In order to evaluate the dense occlusion problem, a dense occlusion road dataset (JHWV) was built and annotated. The experimental results obtained on the DeepGlobe dataset, the newly built JHWV dataset, and large-scale urban images demonstrate the superiority of OARENet, especially when faced with a dense road occlusion situation. Code has been made available at: https://github.com/WanderRainy/OARENet.

In remote sensing imagery analysis, patch-based methods have limitations in capturing information beyond the sliding window. This shortcoming poses a significant challenge in processing complex and variable geo-objects, which results in semantic inconsistency in segmentation results. To address this challenge, we propose a dynamic scale perception framework, named GeoAgent, which adaptively captures appropriate scale context information outside the image patch based on the different geo-objects. In GeoAgent, each image patch’s states are represented by a global thumbnail and a location mask. The global thumbnail provides context beyond the patch, and the location mask guides the perceived spatial relationships. The scale-selection actions are performed through a Scale Control Agent (SCA). A feature indexing module is proposed to enhance the ability of the agent to distinguish the current image patch’s location. The action switches the patch scale and context branch of a dual-branch segmentation network that extracts and fuses the features of multi-scale patches. The GeoAgent adjusts the network parameters to perform the appropriate scale-selection action based on the reward received for the selected scale. The experimental results, using two publicly available datasets and our newly constructed dataset WUSU, demonstrate that GeoAgent outperforms previous segmentation methods, particularly for large-scale mapping applications.

Wetlands are among the most valuable environmental resources and are crucial to achieving sustainable development strategies. However, the number, distribution, and types of wetlands are poorly understood. Widely used medium-resolution wetland products do not provide enough surface feature for fine-grained wetland mapping. To fill this gap, a high-resolution wetland remote sensing mapping dataset covering the contiguous United States was constructed. On this dataset, the performance of deep semantic segmentation networks with various backbones and architecture for wetland mapping was evaluated. Several loss functions were implemented to address the issue of class imbalance of this dataset, resulting in an improvement in classification accuracy. High spatial resolution images offer an abundance of surface texture, shape, structure, and neighborhood relationship data that can be applied to the classification of large-scale wetlands. The combination of high-resolution imagery and deep semantic segmentation models enables the automatic classification of wetlands to be refined.

Urban land-use types, such as residential and administration, can be inferred through semantic objects and their relationships. Point of interest (POI) data can serve as the semantic objects for urban land-use mapping. However, the previous POI-based approaches have rarely considered the relationships between the semantic objects in the urban land-use mapping, and three main challenges remain: 1) the lack of paired semantic object/land-use samples; 2) the lack of a unified model for semantic objects and the relationships between sematic objects and urban land use; and 3) the difficulty of automatically learning semantic object/land-use mapping relationships. In this paper, to address these issues, a graph-based urban land-use mapping framework integrating semantic object/land-use relationships (GOLR) is proposed. Based on open-source area of interest (AOI) and POI data, an urban object/land-use (UOLU) dataset covering 34 cities in China was built. To model the spatial and mapping relationships, the semantic objects and their relationships are used to jointly build an urban land-use graph. The mapping from semantic objects to urban land use can then be learned by the urban land-use graph isomorphic network (ULGIN) model. Finally, the GOLR framework was applied to obtain accurate land-use mapping results for multiple Chinese cities.

In this paper, a new semantic change detection (CD) dataset based on Chinese Gaofen-2 (GF-2) satellite images with high spatial resolution (HSR) namely Wuhan Urban Semantic Understanding (WUSU) dataset is built up and a CD framework combining binary and semantic CD tasks based on deep learning and a conditional random field model (SDCRF) is proposed. Existing CD datasets mostly focus on “change/no change”. Traditional CD methods pay attention only on either of the binary CD task or the semantic CD task. Although there are methods to handle both tasks simultaneously but they ignore the inconsistency between the two tasks. In the SDCRF framework, any state-of-the-art feature extraction model can be used to extract the class and change probabilities as the unary potential of a fully connected conditional random field (FC-CRF) model which is adopted as a post-processing to enhance the location information of deep networks and reduce outlier noise.

High spatial resolution (HSR) remote sensing images can reflect more subtle changes and more specific types of land use and land cover (LULC) due to the abundant spatial geometric information. In this article, a class-prior object-oriented conditional random field (COCRF) framework consisting of a binary change detection (CD) task and a multiclass CD task is proposed to fill the application gap. In the proposed framework, the class-prior knowledge is used to improve the construction of the unary potential in both the binary and multiclass CD tasks, to reduce the influence of spectral variability. The binary CD result provides a constraint to the multiclass CD result. As a result, both parts have effective interaction. The class posterior probability images of two dates can be obtained automatically with the class-prior knowledge by sample migration. Furthermore, an object constraint described by the class dispersion within the objects is added to improve the smoothness in local objects, while the pairwise potential improves the smoothness of the whole area by using the eight-neighborhood spectral information of the center pixel. By integrating the above approaches, the problems of error accumulation and the manual intervention required in the traditional multiclass CD methods can be relieved. An adaptive parameter estimation strategy is also adopted in the proposed framework, to save the time required for manual parameter setting. The proposed COCRF framework was validated on two HSR remote sensing image data sets, where it achieved a better performance than the other state-of-the-art CD methods.

Semantic change detection is a meaningful but challenging task in the remote sensing community. The currently dominant approaches are mainly based on deep learning. However, the lack of high-resolution annotations is the main bottleneck for semantic change detection at scale when using these state-of-the-art deep learning models. In this paper, the label refinement framework is proposed for weakly-supervised semantic change detection, which allows the deep network to learn from low-resolution labels and produce high-resolution semantic change maps, thus alleviating the data-hungry problem. This framework contains four parts: coarse label training’ pseudo-label refinement, multitask change detection and post-process. The experimental results on 2021 IEEE GRSS Data Fusion Contest Track MSD dataset confirmed the effectiveness of the proposed method. Additionally, our method wins 4th place in the 2021 IEEE GRSS Data Fusion Contest Track MSD (DFC21-MSD).