Accurately mapping large-scale cropland is crucial for agricultural production management and planning. Currently, the combination of remote sensing data and deep learning techniques has shown outstanding performance in cropland mapping. However, those approaches require massive precise labels, which are labor-intensive. To reduce the label cost, this study presented a weakly supervised framework considering multi-temporal information for large-scale cropland mapping. Specifically, we extract high-quality labels according to their consistency among global land cover (GLC) products to construct the supervised learning signal. On the one hand, to alleviate the over-fitting problem caused by the model’s over-trust of remaining errors in high-quality labels, we encode the similarity/ aggregation of cropland in the visual/spatial domain to construct the unsupervised learning signal, and take it as the regularization term to constrain the supervised part. On the other hand, to sufficiently leverage the plentiful information in the samples without high-quality labels, we also incorporate the unsupervised learning signal in these samples, enriching the diversity of the feature space. After that, to capture the phenological features of croplands, we introduce dense satellite image time series (SITS) to extend the proposed framework in the temporal dimension. We also visualized the high-dimensional phenological features to uncover how multi-temporal information benefits cropland extraction, and assessed the method’s robustness under conditions of data scarcity. The proposed framework has been experimentally validated for strong adaptability across three study areas (Hunan Province, Southeast France, and Kansas) in large-scale cropland mapping, and the internal mechanism and temporal generalizability are also investigated. The source codes are available at https://github.com/wangyuze-csu/WSFCMI.

Deep learning has achieved great success in learning features from massive remote sensing images (RSIs). To better understand the connection between three feature learning paradigms, which are unsupervised feature learning (USFL), supervised feature learning (SFL), and self-supervised feature learning (SSFL), this paper analyzes and compares them from the perspective of feature learning signals, and gives a unified feature learning framework. Under this unified framework, we analyze the advantages of SSFL over the other two learning paradigms in RSI understanding tasks and give a comprehensive review of existing SSFL works in RS, including the pre-training dataset, self-supervised feature learning signals, and the evaluation methods. We further analyze the effects of SSFL signals and pre-training data on the learned features to provide insights into RSI feature learning. Finally, we briefly discuss some open problems and possible research directions.

The imaging process of optical remote sensing images are easily affected by external conditions. Therefore, remote sensing images under different imaging conditions often show color differences, resulting in feature distribution differences between the source and target domain, hindering the migration of semantic segmentation models between domains. Currently, most domain adaptation methods are for single-source and single-target domains. Here, we proposed a novel and concise method, coined MDANet, for the adaptation of patch images of multi-source and multi-target domains and for reducing the distribution differences of different patch images by projecting them onto the virtual center of a mixed-domain. MDANet is a lightweight and self-supervised network that can be grafted with any semantic segmentation model. Our method significantly improved the segmentation accuracy of semantic segmentation models and showed higher stability and competitiveness than existing methods.

With the development of deep learning, supervised learning methods perform well in remote sensing image (RSI) scene classification. However, supervised learning requires a huge number of annotated data for training. When labeled samples are not sufficient, the most common solution is to fine-tune the pretraining models using a large natural image data set (e.g., ImageNet). However, this learning paradigm is not a panacea, especially when the target RSIs (e.g., multispectral and hyperspectral data) have different imaging mechanisms from RGB natural images. To solve this problem, we introduce a new self-supervised learning (SSL) mechanism to obtain the high-performance pretraining model for RSI scene classification from large unlabeled data. Experiments on three commonly used RSI scene classification data sets demonstrated that this new learning paradigm outperforms the traditional dominant ImageNet pretrained model. Moreover, we analyze the impacts of several factors in SSL on RSI scene classification, including the choice of self-supervised signals, the domain difference between the source and target data sets, and the amount of pretraining data. The insights distilled from this work can help to foster the development of SSL in the remote sensing community. Since SSL could learn from unlabeled massive RSIs, which are extremely easy to obtain, it will be a promising way to alleviate dependence on labeled samples and thus efficiently solve many problems, such as global mapping.

Over the past decades, the successful invention and employment of multiple sensors have marked the advent of a new era in multisensor remote-sensing (RS) images acquisition. To effectively utilize the massive multisensor images for RS scene understanding, we expect that a scene classification model learned with particular sensor data can generalize well to other sensor data. However, this is a very challenging task due to the cross-sensor data differences. In the deep learning (DL) pipeline, a common way to handle this challenging task is to fine-tune the models pretrained on source sensor data with limited labeled data from the target sensor. Unfortunately, fine-tune technique is usually applied between homogeneous networks, which may not be the best choice if the source and target data are largely different. To address these issues, we formulate the cross-sensor RS scene understanding problem as a heterogeneous network-oriented transfer learning problem, in which the source and the target networks are different and data-oriented selected. Afterward, the knowledge between heterogeneous networks is transferred using the pseudo-label recursive propagation mechanism inspired by the concept of knowledge distillation. To the best of our knowledge, this is the first time to investigate the cross-sensor scene classification problem by constructing such a heterogeneous networks’ transfer scheme in RS fields. Our experiments using two cross-sensor RS datasets [aerial images$\rightarrow $multispectral images (MSIs) and aerial images$\rightarrow $hyper-spectral images (HSIs)] demonstrated that the proposed transfer learning strategy based on heterogeneous networks outperforms the supervised learning (SL) and fine-tune scheme for cross-sensor scene classification.

Thick clouds seriously impact the quality of optical remote sensing images (RSIs) and limit their application. For removing the cloud, some learning-based methods have been proposed and attracted considerable attention. However, these methods need to train paired multitemporal images with/without cloud, which are difficult and costly to collect. To solve this problem, we propose a novel texture complexity-guided self-paced learning (SPL) framework to remove the thick cloud from single RSIs. The framework does not need paired images and it exploits a texture complexity-guided mechanism to rank the self-generated cloud-corrupted training samples by texture complexity from low to high and then trains the generative adversarial cloud removal network using the SPL technique. In this way, the cloud removal network learns to restore the cloud-corrupted areas from easy to hard and thus to realize the image reconstruction for different difficulty levels. In addition, we introduce a structural similarity (SSIM) loss function to optimize the training network and improve the coherence of the image structure. Simulated and real experiments are performed on the single images acquired by Gao Fen-1 (GF-1) and Sentinel-2 satellites to validate the effectiveness of the proposed method. The results show that the proposed method has a better performance in cloud removal than other state-of-the-art methods, especially for the images of the areas with complex textures. The source codes are available athttps://github.com/GeoX-Lab/TPL.

Reducing the feature distribution shift caused by the factor of visual-environment changes, namely as VE-changes, is a hot issue in domain adaptation learning. However, in the semantic segmentation task of remote sensing imageries, besides VE-changes, the change of semantic-scenes (SS-changes) is another factor raising domain gap, which brings the label distribution shift. For example, although urban and rural share the same landcover label, there is still a gap in label distribution. If there is little relation that can be found in neither feature nor label space, forcibly adapting to a new domain could have a high risk of negative transfer. Hence, we propose a new Transitive Domain Adaptation method for Remote Sensing images (TDARS). Firstly, we introduce an intermediate domain to enlarge the relation between the given source and target domains. Secondly, we learn from primary and non-primary confident classes to increase the likelihood of transferring valuable information. As a result, TDARS enables the given source and target domains to be connected through the selected intermediate domain and performs effective knowledge transfer among all domains. The proposed method is evaluated on three domain adaptation datasets of remote sensing images. Extensive experiments show the approach can effectively handle the domain shift problem from remote sensing images compared to other state-of-the-art domain adaptation methods.

Remote sensing image (RSI) scene classification (RSISC) is a fundamental problem for understanding high-resolution RSIs. More recently, deep learning methods, especially convolutional neural networks (CNNs), and large data sets have greatly promoted the RSISC. However, deep learning methods rely heavily on the visual features extracted from the patches cropped from original RSIs, so the intraclass diversity and interclass similarity are two big challenges. To address these problems, in this letter, we propose a spatial information considered model to learn more discriminative features. By combining CNN and recurrent neural network, the proposed method can exploit both local and long-range spatial relation information to enhance the representational ability of the learned features. As the initial visual features of a single patch are transformed into higher-level features with spatial information, the proposed method achieves more accurate scene classification. Besides, we present an RSISC data set named as CSU-RSISC10 data set to preserve the spatial information between scenes in a new way of organization. Experiments demonstrate that the proposed method outperforms other three state-of-the-art methods in scene classification using CSU-RSISC10 data set.

Optical satellites are affected by factors such as seasonal and atmospheric variation, illumination, and sensor distortion. Thus, satellite images covering large-scale area often show conspicuous color differences, resulting in poor color continuity of the mosaicked satellite image. This study proposes a novel combined model color correction (CMCC) method for high-resolution optical satellite images, which constructively combines a defogging model with a radiation correction model. First, this study analyzed the feasibility of using easily available low-resolution satellite images as external references to correct the color of high-resolution images and describes the selection criteria for external references. Second, considering the negative effects of atmosphere on the color and clarity of remote sensing images, we proposed an optical satellite image enhancement method, which is based on the content characteristics of remote sensing images and the dark channel prior defogging method. Finally, we designed a two-stage color correction process: 1) correcting the color of downsampled images via low-frequency modeling and replacement and 2) mapping the color of downsampled images to original images through local modeling and super-resolution color correction. Furthermore, this study proposes an indicator of quality considered mean absolute error (QCMAE) for quantitative evaluation of the color correction result. We selected 328 Gaofen-1 (GF-1) high-resolution images for the experiments. Visual effects and statistical results of images after being processed by the proposed CMCC are both superior to the three state-of-the-art methods, which verifies the effectiveness and reliability of the proposed method.

For road extraction tasks in VHR satellite imagery, a deep neural network may perform well. But a network with certain reasoning ability as human will get a more satisfying result. To this end, we focus on how to effectively model the context information of the road and propose a well-designed spatial information inference structure (SIIS) which can add into any typical semantic segmentation network. The network with SIIS called SII-Net can not only learn the local visual characteristic of the road but also the global spatial structure information (such as the continuity and trend of the road). So, it can effectively solve the challenging occlusion problem in road detection and well preserve the continuity of the extracted road. The experimental results of two datasets show that the proposed method can improve the comprehensive performance of road extraction.

Though Hyperspectral Image (HSI) Classification has been extensively investigated over recent decades, it is still a challenge task especially when the number of labeled samples is extremely limited. In this paper, we overcome this challenge by using synthetic samples, and proposed a semi-supervised variational Generative Adversarial Networks(GANs) for this purpose. Compared to the conditional GAN which is recently used for generating samples for HSI classification, the proposed approach has two novel aspects. First, we extend the classic variational generative adversarial network to the semi-supervised context through an ensemble prediction technique. By this way, our model can be trained using limited labeled samples (only 5 samples per class) with a large number of unlabeled samples. Second, we adopt an encoder-decoder network to explicitly learn the relationship between the latent space and the real image space. This property enables our model producing diverse samples by simply varying some latent parameters, which is desirable for enriching the training dataset. We have shown that the proposed model can achieve better and robust performance for HSI classification compared to conditional GAN, especially when the labeled data is limited.

As the spatial resolution of remote sensing images is improving gradually, it is feasible to realize “scene-object” collaborative image interpretation. Unfortunately, this idea is not fully utilized in vehicle detection from high-resolution aerial images, and most of the existing methods may be promoted by considering the variability of vehicle spatial distribution in different image scenes and treating vehicle detection tasks scene-specific. With this motivation, a scene context-driven vehicle detection method is proposed in this paper. At first, we perform scene classification using the deep learning method and, then, detect vehicles in roads and parking lots separately through different vehicle detectors. Afterward, we further optimize the detection results using different postprocessing rules according to different scene types. Experimental results show that the proposed approach outperforms the state-of-the-art algorithms in terms of higher detection accuracy rate and lower false alarm rate.