Clouds are a very important factor in the availability of optical remote sensing images. Recently, deep learning (DL)-based cloud detection methods have surpassed classical methods based on rules and physical models of clouds. However, most of these deep models are very large, which limits their applicability and explainability, while other models do not make use of the full spectral information in multispectral images, such as Sentinel-2. In this article, we propose a lightweight network for cloud detection, fusing multiscale spectral and spatial features (CD-FM3SFs) and tailored for processing all spectral bands in Sentinel-2A images. The proposed method consists of an encoder and a decoder. In the encoder, three input branches are designed to handle spectral bands at their native resolution and extract multiscale spectral features. Three novel components are designed: a mixed depthwise separable convolution (MDSC) and a shared and dilated residual block (SDRB) to extract multiscale spatial features, and a concatenation and sum (CS) operation to fuse multiscale spectral and spatial features with little calculation and no additional parameters. The decoder of CD-FM3SF outputs three cloud masks at the same resolution as input bands to enhance the supervision information of small, middle, and large clouds. To validate the performance of the proposed method, we manually labeled 36 Sentinel-2A scenes evenly distributed over mainland China. The experiment results demonstrate that CD-FM3SF outperforms traditional cloud detection methods and state-of-the-art DL-based methods in both accuracy and speed.

As the only natural satellite of the Earth, the Moon provides vital location resources and supportive environment for Earth observations, and the Moon-based Earth observation (MEO) has unparalleled advantages in global climate change and large-scale phenomena. The ocean plays an important role in regulating climate and global water and carbon cycle. With an attempt to explore the feasibility of MEO-based marine environment monitoring, this study aimed to investigate the observing geometry and revisiting frequency of the MEO-based ocean color remote sensing and further to explore its quantitative application potentials. Results showed that MEO-based ocean color remote sensing, capturing the Earth on an hourly basis, could observe most part of the ocean for over five times per day; however, both solar zenith angle and view zenith angle were high at high-latitude regions; atmospheric reflectance accounted for most of sensor-measured signal, especially at high solar and view zenith angle, while surface-reflected glint reflectance was also notable at low solar zenith angle; and the remote sensing reflectance retrieved from MEO-based ocean color remote sensing could be used for chlorophyll retrieval. In further studies, more efforts should be paid on how to accurately retrieve remote sensing reflectance at high solar and view zenith angle, which would improve the application capability of MEO for polar regions. Overall, this study demonstrated the great potentials of MEO-based ocean color remote sensing, and MEO would be a new observing perspective and long-term consistent data source for marine environment monitoring.

The optical images acquired from satellite platforms are commonly multiresolution images, and converting multiresolution satellite images into full higher-resolution (HR) images has been a critical technique for improving the image quality. In this study, we introduced the generative adversarial network (GAN) and proposed a new fusion GAN (FusGAN) approach for solving the remote sensing image fusion problem. Specifically, we developed a new adversarial training strategy: 1) downscaled multiresolution images are adopted for generative network (G-Net) training, and 2) the discriminative network (D-Net) is used to adversarially train the G-Net by discriminating whether the original multiresolution images have been fused well enough. To further improve the capability of the network, we structured our G-Net with residual dense blocks by combining state-of-the-art residual and dense connection ideas. Our proposed FusGAN approach is evaluated both visually and quantitatively on Sentinel-2 and Landsat Operational Land Imager (OLI) multiresolution images. As demonstrated by the results, the proposed FusGAN approach outperforms the selected benchmark methods and both perfectly preserves spectral information and reconstructs spatial information in image fusion. Considering the common resolution disparities among intra- and intersatellite images, the proposed FusGAN approach can contribute to the quality improvement of satellite images and thus improve remote sensing applications.

Cloud detection is an important step in the processing of remote sensing images. Most methods based on convolutional neural networks (CNNs) for cloud detection require pixel-level labels, which are time-consuming and expensive to annotate. To overcome this challenge, this letter proposes a novel semisupervised algorithm for cloud detection by training a self-attentive generative adversarial network (SAGAN) to extract the feature difference between cloud images and cloud-free images. Our main idea is to introduce visual attention into the process of generating “real” cloud-free images. The training of SAGAN is based on three guiding principles: expansion of attention maps of cloud regions which will be replaced with translated cloud-free images, reduction of attention maps to coincide with cloud boundaries, and optimization of a self-attentive network to handle the extreme cases. The inputs for SAGAN training are the images and image-level labels, which are easier, cheaper, and more time-saving than the existing methods based on CNN. To test the performance of SAGAN, experiments are conducted on the Sentinel-2A Level 1C image data. The results show that the proposed method achieves very promising results with only the image-level labels of training samples.

Subsidence has been reported in land reclamation area of Shenzhen. In this study, time series InSAR technique is employed to determine the surface movements in Shenzhen in the period between 2015 and 2018 using ALOS-2 images. Subsidence can be observed in Qianhai, Houhai, and the area to the south of Bao'an International Airport. Subsidence detected are in land reclamation area. This study highlight the necessity of continuous displacement monitoring in Shenzhen.

Scale parameter selection is a key step in an object-based image analysis (OBIA) work. In existing works, the first step is the selection of optimal scale parameter, followed by feature description and later analysis. However, only low-level image features are used at this step, which are not directly related to the purpose of the application. To overcome the limitation, we propose a multiscale object-based image analysis framework, in which, the multiscale classification is performed first, and the optimal scale parameter is estimated using the multiscale classification results and training samples. The experiments have demonstrated the effectiveness of our approach in estimating optimal scale parameter for object-based landcover classification, and showed great potential in automatic analysis of high spatial resolution remote sensing images.

Region-based hierarchical image representation is crucial in many computer vision applications. However, in practice, an image hierarchy is usually dense, and contains many less informative branches. It is expected that a hierarchy should be accurate and simplified, which is not only desirable for different applications, but also saves considerable computational load for the further analysis. To achieve this target, this paper proposes a novel approach for unsupervised simplification of region-based image hierarchies, which employs the global and local evolution analyses of a hierarchy. First, we introduce a global evolution analysis in the scale-sets framework, which provides clues for eliminating less informative branches. Moreover, a hybrid unsupervised simplification method is designed, utilizing the information from global and local evolution functions. A number of experiments on various images have shown that the proposed approach is effective and efficient in removing less informative nodes (averagely about 90% of the whole nodes), while preserving salient image details and retaining the accuracy.

Human vision benefits a lot from pattern collocations in visual activities such as object detection and recognition. Usually, pattern collocations display as the co-occurrences of visual primitives, e.g., colors, gradients, or textons, in neighboring regions. In the past two decades, many sophisticated local feature descriptors have been developed to describe visual primitives, and some of them even take into account the co-occurrence information for improving their discriminative power. However, most of these descriptors only consider feature co-occurrence within a very small neighborhood, e.g., 8-connected or 16-connected area, which would fall short in describing pattern collocations built up by feature co-occurrences in a wider neighborhood. In this paper, we propose to describe local pattern collocations by using a new and general regional co-occurrence approach. In this approach, an input image is first partitioned into a set of homogeneous superpixels. Then, features in each superpixel are extracted by a variety of local feature descriptors, based on which a number of patterns are computed. Finally, pattern co-occurrences within the superpixel and between the neighboring superpixels are calculated and factorized into a final descriptor for local pattern collocation. The proposed regional co-occurrence framework is extensively tested on a wide range of popular shape, color, and texture descriptors in terms of image and object categorizations. The experimental results have shown significant performance improvements by using the proposed framework over the existing popular descriptors.

Shadows often incur uneven illumination to pavement images, which brings great challenges to image-based pavement crack detection. Thus, it is desired to remove pavement shadows before detecting pavement cracks. However, due to the large penumbras cast by trees, light poles, etc., it is difficult to locate shadows in a pavement image. In this paper, an automatic pavement shadow removal method is proposed based on geodesic analysis. First, a geodesic shadow model is used to partition a pavement shadow into a number of geodesic regions. Then, an optimal background region is selected for reference by statistic analysis. Finally, a texture-balanced illuminance compensation is applied on all geodesic regions over the image. Experiments demonstrate the effectiveness of the proposed method.

Due to the diversity of geographical objects, it makes great sense to introduce multiscale segmentation/representation into the analysis and interpretation of high-spatial-resolution remote sensing images. However, with the increasing use of high-resolution images, traditional multiscale segmentation methods gradually show their lack in efficiency, particularly when handling large-scale images. In this paper, a novel bilevel scale-sets model (BSM) is proposed for multiscale region-based representation of large-scale remote sensing images. In the BSM, first, an image is divided into blocks with overlapped margins, and a low-level scale-sets model is blockwisely implemented. Second, a segmentation result is obtained by retrieving and mosaicking the blockwise segmentation results, based on which a high-level scale-sets model is implemented covering the whole image. To further improve the efficiency of the BSM, a parallel implementation is presented for the blockwise scale-sets model. In the experiments, first, the effectiveness of the BSM is validated using a WorldView2 image covering a coastal area of Shenzhen, where the BSM obtains accurate multiscale representation results without any mosaic artifacts. Then, the efficiency of the BSM is demonstrated by comparing with the state-of-the-art multiscale segmentation method, i.e., the one integrated in the commercial software eCognition v9.2, where the proposed BSM takes about 7 min to process a 24 000 × 24 000 multispectral ZY3 image and is two to three times faster than the competing method.

As a pre-processing tool, superpixel algorithms have been popular used in many computer-vision applications. High efficiency is a desired property of superpixel algorithms, especially in real-time vision systems. In this paper, a novel high-efficient superpixel algorithm is developed based on the watershed algorithm, namely the spatial-constrained watershed (SCoW). SCoW performs watersheding in a marker-controlled manner, with a set of evenly placed markers. To align superpixel boundaries to image edges, an edge-preserving scheme is embedded into the SCoW which makes a balance between the homogeneity and the compactness. Without any complex computing, the proposed superpixel algorithm is found to produce high quality superpixels as traditional superpixel algorithms, while holding much higher efficiency.

Both image registration and fusion are essential steps to produce high-resolution multispectral images in remote sensing. Traditionally, they are viewed as two independent processes. As a result, the registration errors ignored in the fusion process can significantly affect the fusion quality. In this context, an iterative optimization approach, which jointly considers the registration and fusion processes, is proposed for panchromatic (PAN) and multispectral (MS) images. Given a registration method and a fusion method, the joint optimization process is described as finding the optimal registration parameters to gain the optimal fusion performance. In our approach, the downhill simplex algorithm is adopted to refine the registration parameters iteratively. Experiments on a set of PAN and MS images of ZY-3 and GeoEye-1 show that the proposed approach outperforms several competing ones in terms of registration accuracy and fusion quality.

As the first Chinese civilian high-resolution remote-sensing satellite, Ziyuan 3 (ZY-3) is equipped with nadir, forward, and backward cameras for stereo mapping. In order to acquire high-resolution mapping accuracy, in-orbit sensor calibration is a particularly crucial procedure after the satellite is launched and orbiting. The key problem of in-orbit sensor calibration is analyzed based on the overall structural design and imaging geometric characteristics of ZY-3. In this paper, a rigorous imaging model of ZY-3 and an interior orientation model are established based on the satellite's designed configuration, charge-coupled device splicing, and imaging characteristics. A new trajectory model, which is called the exterior orientation model, is proposed, which establishes a systematic-error model for the sensors of ZY-3. In addition, the 609th track data are used to validate the new trajectory model, a piece point with a weight polynomial model, which is compared with other trajectory models. Furthermore, different ground control points are selected for the bundle block adjustment experiments with the systematic error and trajectory models of the proposed sensor. It is observed by experiment that the piece point with weight polynomial model has a higher orbit and attitude interpolation accuracy than the other models. For self-calibration with additional parameters, the model can reduce the correlation of the parameters, improve the solved accuracy of the parameters in the exterior and interior orientation models of the ZY-3 sensors, and ensure the correctness and stability of the calibration results.

This letter presents a novel spatially-constrained color–texture model for hierarchical segmentation of very high resolution images. The segmentation starts with an initial partition, where the image is partitioned into many homogeneous regions. Then, the regions are regarded as node sets of a region adjacency graph, in which the distances of each pair of adjacent regions are calculated combining color and textural features with spatial constraint. Finally, a stepwise optimized region merging process is applied to obtain hierarchical segmentation results. Experiments and comparisons by using different satellite images are carried out to demonstrate the encouraging performance as well as the high efficiency of the proposed method.