Reference-based super-resolution (RefSR) has the potential to build bridges across spatial and temporal resolutions of remote sensing images. However, existing RefSR methods are limited by the faithfulness of content reconstruction and the effectiveness of texture transfer in large scaling factors. Conditional diffusion models have opened up new opportunities for generating realistic high-resolution images, but effectively utilizing reference images within these models remains an area for further exploration. Furthermore, content fidelity is difficult to guarantee in areas without relevant reference information. To solve these issues, we propose a change-aware diffusion model named Ref-Diff for RefSR, using the land cover change priors to guide the denoising process explicitly. Specifically, we inject the priors into the denoising model to improve the utilization of reference information in unchanged areas and regulate the reconstruction of semantically relevant content in changed areas. With this powerful guidance, we decouple the semantics-guided denoising and reference texture-guided denoising processes to improve the model performance. Extensive experiments demonstrate the superior effectiveness and robustness of the proposed method compared with state-of-the-art RefSR methods in both quantitative and qualitative evaluations. The code and data are available at https://github.com/dongrunmin/RefDiff.

Change detection in remote sensing is crucial for land use/land cover (LULC) change awareness. However, large-scale change detection suffers from limited contextual understanding of spatial intricacies of large changes in existing CNN-based methods. Overlapped receptive fields lead to weight sharing across feature sliders, contributing to limited detection ability on large dense multi-class changes. To address this problem, this paper presents a receptive convolution operation for large-scale multi-class change detection from high-resolution remote sensing images. Different from existing CNN-based networks, our architecture involves receptive convolutions with a large kernel size to guarantee focus on different receptive field features. Experiments on SECOND datasets show that the proposed method achieves better performance than previous counterparts. Furthermore, a large-scale LULC change detection is conducted to demonstrate the ability in large-scale applications.

Efficient and timely assessment of road network dynamic changes is crucial for the comprehensive evaluation of urban development, transportation accessibility, and environmental impacts. While existing methods mainly focus on optimizing performance with very-high-resolution remote sensing images on public road datasets, their practical applicability remains to unlock when confronted with large-scale real-world applications utilizing multi-spectral remote sensing images. The limitations manifest in unsatisfactory model generalization and fragmented segmentation, reducing the effectiveness of road extraction outcomes. To address these challenges, this study introduces a novel connectivity-aware approach tailored to address road extraction challenges in real-world scenarios. Leveraging Sentinel-2 multi-spectral imagery, this study conducts a 6-year road change mapping over an expansive 10,097 square kilometers in Xi’an, China. Experimental and evaluation results underscore the efficacy of the proposed methodology for widespread applications in urban planning and environmental management, offering a robust solution for practical and efficient road extraction in diverse and extensive large-scale urban investigation.

With the increasing spatial and temporal resolutions of obtained remote sensing (RS) images, effective compression becomes critical for storage, transmission, and large-scale in-memory processing. Although image compression methods achieve a series of breakthroughs for daily images, a straightforward application of these methods to RS domain underutilizes the properties of the RS images, such as content duplication, homogeneity, and temporal redundancy. This paper proposes a Spatial-Temporal Context model (STCM) for RS image compression, jointly leveraging context from a broader spatial scope and across different temporal images. Specifically, we propose a stacked diagonal masked module to expand the contextual reference scope, which is stackable and maintains its parallel capability. Furthermore, we propose spatial-temporal contextual adaptive coding to enable the entropy estimation to reference context across different temporal RS images at the same geographic location. Experiments show that our method outperforms previous state-of-the-art compression methods on rate-distortion (RD) performance. For downstream tasks validation, our method reduces the bitrate by 52 times for single temporal images in the scene classification task while maintaining accuracy.

Fine urban change segmentation using multi-temporal remote sensing images is essential for understanding human-environment interactions in urban areas. Although there have been advances in high-quality land cover datasets that reveal the physical features of urban landscapes, the lack of fine-grained land use datasets hinders a deeper understanding of how human activities are distributed across landscapes and the impact of these activities on the environment, thus constraining proper technique development. To address this, we introduce FUSU, the first fine-grained land use change segmentation dataset for Fine-grained Urban Semantic Understanding. FUSU features the most detailed land use classification system to date, with 17 classes and 30 billion pixels of annotations. It includes bi-temporal high-resolution satellite images with 0.2-0.5 m ground sample distance and monthly optical and radar satellite time series, covering 847 km^2 across five urban areas in the southern and northern of China with different geographical features. The fine-grained land use pixel-wise annotations and high spatial-temporal resolution data provide a robust foundation for developing proper deep learning models to provide contextual insights on human activities and urbanization. To fully leverage FUSU, we propose a unified time-series architecture for both change detection and segmentation. We benchmark FUSU on various methods for several tasks. Dataset and code are available at: https://github.com/yuanshuai0914/FUSU.

In high-resolution remote sensing images (RSIs), complex composite object detection (e.g., coal-fired power plant detection and harbor detection) is challenging due to multiple discrete parts with variable layouts leading to complex weak inter-relationship and blurred boundaries, instead of a clearly defined single object. To address this issue, this article proposes an end-to-end framework, i.e., relational part-aware network (REPAN), to explore the semantic correlation and extract discriminative features among multiple parts. Specifically, we first design a part region proposal network (P-RPN) to locate discriminative yet subtle regions. With butterfly units (BFUs) embedded, feature-scale confusion problems stemming from aliasing effects can be largely alleviated. Second, a feature relation Transformer (FRT) plumbs the depths of the spatial relationships by part-and-global joint learning, exploring correlations between various parts to enhance significant part representation. Finally, a contextual detector (CD) classifies and detects parts and the whole composite object through multirelation-aware features, where part information guides to locate the whole object. We collect three remote sensing object detection datasets with four categories to evaluate our method. Consistently surpassing the performance of state-of-the-art methods, the results of extensive experiments underscore the effectiveness and superiority of our proposed method.

Semi-supervised learning has attracted increasing attention in the large-scale land cover mapping task. However, existing methods overlook the potential to alleviate the class imbalance problem by selecting a suitable set of unlabeled data. Besides, in class-imbalanced scenarios, existing pseudo-labeling methods mostly only pick confident samples, failing to exploit the hard samples during training. To tackle these issues, we propose a unified Class-Aware Semi-Supervised Semantic Segmentation framework. The proposed framework consists of three key components. To construct a better semi-supervised learning dataset, we propose a class-aware unlabeled data selection method that is more balanced towards the minority classes. Based on the built dataset with improved class balance, we propose a Class-Balanced Cross Entropy loss, jointly considering the annotation bias and the class bias to re-weight the loss in both sample and class levels to alleviate the class imbalance problem. Moreover, we propose the Class Center Contrast method to jointly utilize the labeled and unlabeled data. Specifically, we decompose the feature embedding space using the ground truth and pseudo-labels, and employ the embedding centers for hard and easy samples of each class per image in the contrast loss to exploit the hard samples during training. Compared with state-of-the-art class-balanced pseudo-labeling methods, the proposed method improves the mean accuracy and mIoU by 4.28% and 1.70%, respectively, on the large-scale Sentinel-2 dataset with 24 land cover classes.

Plastic pollution is becoming an increasingly prominent problem and the function of plastics determines whether they need to be recycled or not. In order to explore the possibility of using satellite imagery to classify the functionality of plastics, this study proposes a two-stage workflow: firstly, a classification map is obtained based on hyperspectral satellite imagery to generate plastic types, and then using these identified plastic coverage areas, a deep learning algorithm is used to assign functionality to these classified plastic areas based on sentinel-2 imagery. By comparing five leading-edge image classification models, classification accuracies of up to 74% were achieved, demonstrating the feasibility of using deep learning models trained on satellite images to identify plastic features.

High-resolution land cover mapping (LCM) is an important application for studying and understanding the change of the earth surface. While deep learning (DL) methods demonstrate great potential in analyzing satellite images, they largely depend on massive high-quality labels. This paper proposes SW-LCM, a Scalable and Weakly-supervised two-stage Land Cover Mapping method on a new Sunway Supercomputer. Our method consists of a k-means clustering module as a first stage, and an iterative deep learning module as a second stage. With the k-means module providing a good enough starting point (taking inaccurate results as noisy labels), the deep learning module improves the classification results in an iterative way, without any labelling efforts required for processing large scenarios. To achieve efficiency for country-level land cover mapping, we design a customized data partition scheme and an on-the-fly assembly for k-means. Through careful parallelization and optimization, our k-means module scales to 98,304 computing nodes (over 38 million cores), and provides a sustained performance of 437.56 PFLOPS, in a real LCM task of the entire region of China; the iterative updating part scales to 24,576 nodes, with a performance of 11 PFLOPS. We produce a 10-m resolution land cover map of China, with an accuracy of 83.5% (10-class) or 73.2% (25-class), 7% to 8% higher than best existing products, paving ways for finer land surveys to support sustainability-related applications.

Commonly recognized as an important symbol of the tropics and global warming, the glacier loss on Mt. Kilimanjaro has received worldwide attention for decades. In this paper, we propose a high-resolution glacier-cover (GC) record of Mt. Kilimanjaro over the period from 1984 to 2020, using a novel deep learning-based semantic segmentation method and Google Earth images, as well as digital elevation model (DEM) and ERA5-Land (ERA5) for snowline and temperature variations analysis. Our method achieves an accuracy of 94.37%, which proves the model's capability to record the GC areas precisely. The results show that (1) the GC area dramatically decreases from 19.2 km2to 3.6 km2during 37 years, which decreases about 4% and 2% per year from 1984 to 2000 and from 2000 to 2020 respectively, (2) the snowline altitude rises from$4,651 m$to$5,088 m$by about$437 m$, and (3) the average$5,000 m$air temperature on Mt. Kilimanjaro increases from −2.1 °C to −1.1 °C by about 1 °C. This study indicates that there will be no GC within a few decades if the current loss continues.

Automatic and accurate extraction of buildings from remote sensing images plays a vital role in many applications. However, existing approaches for building extraction generally apply high-resolution remote sensing images as input to attain high-resolution extraction results, which is time consuming and limited due to its spatiotemporal accessibility and cost. To address this challenge, in this paper, we propose an end-to-end approach, i.e., SRBuildingSeg-E2, to achieve higher-resolution building extraction from relatively low resolution remote sensing images. By integrating super resolution and semantic segmentation techniques, our proposed approach can attain high-resolution representations using low-resolution input. The quantitative assessment results reveal its promising performance in higher-resolution building extraction.

Counting and detecting oil palm trees from high-resolution remotely sensed images is a significant work for improving economy of several countries such as Malaysia, Indonesia, etc. However, rare attention have been paid on accelerating tree crown detection algorithms on high performance platforms. In this paper, we design a parallel approach for oil palm tree detection on a SW26010 many-core processor, which is used in a world-leading supercomputer, Sunway TaihuLight. Our parallel framework contains three steps, local maximum filtering, oil palm tree crown center reassignment and oil palm tree crown center merging. Experimental results indicates that our parallel approach of oil palm tree detection obtains the speedup of 32.30 times and 1.74 times for a QuickBird image with a size of$12,188\times 12,576$pixels compared with the well-optimized software implementation of the original algorithm on an Intel 12-core CPU and FPGAs.

Remote sensing image super-resolution (SR) plays an important role by supplementing the lack of original high-resolution (HR) images in the study scenarios of large spatial areas or long time series. However, due to the lack of imagery information in low-resolution (LR) images, single-image super-resolution (SISR) is an inherently ill-posed problem. Especially, it is difficult to reconstruct the fine textures of HR images at large upscaling factors (e.g., four times). In this work, based on Google Earth HR images, we explore the potential of the reference-based super-resolution (RefSR) method on remote sensing images, utilizing rich texture information from HR reference (Ref) images to reconstruct the details in LR images. This method can use existing HR images to help reconstruct the LR images of long time series or a specific time. We build a reference-based remote sensing SR data set (RRSSRD). Furthermore, by adopting the generative adversarial network (GAN), we propose a novel end-to-end reference-based remote sensing GAN (RRSGAN) for SR. RRSGAN can extract the Ref features and align them to the LR features. Eventually, the texture information in the Ref features can be transferred to the reconstructed HR images. In contrast to the existing RefSR methods, we propose a gradient-assisted feature alignment method that adopts the deformable convolutions to align the Ref and LR features and a relevance attention module (RAM) to improve the robustness of the model in different scenarios (e.g., land cover changes and cloud coverage). The experimental results demonstrate that RRSGAN is robust and outperforms the state-of-the-art SISR and RefSR methods in both quantitative evaluation and visual results, which indicates the great potential of the RefSR method for remote sensing tasks. Our code and data are available athttps://github.com/dongrunmin/RRSGAN.

Over the past decade, domain adaptation (DA) algorithms have been proposed to address domain gap problems as they do not need any interpretation in the target domain. However, most existing efforts focus on scenarios with only one source domain and one target domain. In this article, we explore the scenario with one source domain and mixed multiple target domains for remote sensing applications and propose a new algorithm, named the two-stage adaptation network (TSAN). First, we utilize the adversarial learning approach to confuse the classifier to discriminate between the source domain and the whole mixed-multiple-target domain. Second, we adopt self-supervised learning to divide the mixed-multiple-target domain with automated generation of “pseudo”-domain labels, which guides our network to learn intrinsic features of multiple target domains. Finally, these two steps are combined as an iterative procedure. We integrate a test dataset that includes five remote sensing datasets and ten classes. Our method achieves an average accuracy of 63.25% and 73.68% with two typical backbones, considerably outperforming other DA methods with an average accuracy improvement of 4.84%–20.19% and 9.06%–17.04%, respectively. Furthermore, we identify the negative transfer effect in existing mainstream DA methods in remote sensing image classification with multiple different domains.

Single image super-resolution (SISR) is essential in many remote sensing applications. Most of the existing SISR methods on remote sensing images assume that the low resolution (LR) images are synthesized from high-resolution (HR) images by bicubic downscaling. However, the performance of those methods is limited in the real-world remote sensing scenario as the actual degradation is sometimes different from the assumption. Therefore, we introduce the blind super-resolution (SR) concept and propose a super-resolution method with blur kernel prediction (BKPSR). BKPSR first predicts the blur kernel code for an image and then utilizes the blur kernel code to assist the image super-resolution. Experimental results indicate that our method outperforms existing SISR methods on real-world remote sensing images.

Energy consumption management shapes the route to meet greenhouse gas emission reduction goals. Although certain types of or even total energy consumption monitor have been realized using nighttime light remote sensing data, there still leaves a gap to model energy consumption in different sectors timely and broadly. Using parcel oriented temporal linear unmixing method (POTLUM), nighttime light was decomposed into light from different land uses as sources. A pilot study in Shanghai shows that in each year, statistical energy consumptions in living, secondary and tertiary sectors correlate well with unmixed nighttime light from 2014 to 2018 (R2=0.75, 0.61, 0.24, 0.99, 0.71, respectively). This study proves the capability of unmixed nighttime light to estimate sectoral energy consumption using POTLUM, which helps to better monitor energy consumption and serves GHG emission reduction.

Satellite-observed night-time light has been utilized as a significant indicator for human activities and its impact on environment. Up to present, existing nighttime light (NTL) data still faces challenges in detecting the short-term human-related events due to limitation of the satellite revisit time and data quality. In this paper, we propose a promising approach for monitoring daily light during night based on deep learning and Moderate Resolution Imaging Spectroradiometer (MODIS). By modelling the relationship between MODIS and observed nighttime light, our proposed approach achieves the capability for conversion from MODIS image to Luojia-I-like daily NTL images. The quantitatively assessment of our generated NTL images demonstrates its great performance in terms of both similarity and NTL pattern.

Although deep learning has been successfully applied in the field of remote sensing image classification, it still requires time-consuming and costly annotations. In recent years, domain adaptation has been witnessed to address this problem as they do not need any human interpreted in the target domain dataset. However, most of the existing works dedicate effort on the circumstance where there is only one source domain and only one target domain. In this paper, we firstly explore one source and multiple target domains issue for remote sensing application and build a challenging mixed multi-target dataset to contribute to the community. Our method constitutes three parts. Firstly, as we are blind for the multitarget domain, we adopt meta learning to divide the mixed multi-target dataset and insert sub-target domain loss as the part of the loss function. Secondly, we apply the adversarial learning to confuse the classifier to discriminate between the source domain images and the whole mixed multi-target domain images. Finally, the meta learning and the adversarial learning are dynamically iterative procedures and the labels for domain classification in mixed multi-target dataset will be updated for a particular iteration. Our method is well-performed in the four common remote sensing dataset (AID, NWPU-RESISC45, UC Merced and WHU-RS19), including five classes (agriculture, forest, river, residential and parking). Our method achieved an average accuracy of 81.59% and outperformed other domain adaptation method. The experiment results indicate our method is promising for large-scale, multi-regional and multi-temporal remote sensing applications.