Mapping surface water bodies from fine spatial resolution optical remote sensing imagery is essential for the understanding of the global hydrologic cycle. Although satellite data are useful for mapping, the limited spectral information captured by some satellite systems can be suboptimal for the task. For example, the very high-resolution images of Google Earth (GE) only contain RGB bands, which often means many water bodies and land objects are confused. Sentinel-2 (S2) imagery has a spectral resolution more suitable for mapping water bodies, but its medium spatial resolution limits the ability for detailed mapping of water-land boundaries. This letter proposes a deep feature and domain knowledge fusion network (DFDKFNet) for mapping surface water bodies by fusing GE and S2 images while incorporating domain knowledge. DFDKFNet uses the remote sensing indices of normalized difference water index (NDWI) and normalized difference vegetation index (NDVI) derived from the S2 image as the representative domain knowledge to better extract water bodies from terrestrial features. A similar pixel-based approach is used to downscale the NDWI and NDVI maps to match the spatial resolution between the GE and S2 images. The DFDKFNet uses the GE and downscaled NDWI and NDVI images to extract the deep semantic features of water bodies, which are fused with the domain knowledge extracted from the NDWI and NDVI images. DFDKFNet was compared with several state-of-the-art algorithms, and the results show that DFDKFNet can enhance water body mapping accuracy.

Spatiotemporal satellite image fusion (STIF) has been widely applied in land surface monitoring to generate high spatial and high temporal reflectance images from satellite sensors. This paper proposed a new unmixing-based spatiotemporal fusion method that is composed of a self-trained random forest machine learning regression (R), low resolution (LR) endmember estimation (E), high resolution (HR) surface reflectance image reconstruction (R), and residual compensation (C), that is, RERC. RERC uses a self-trained random forest to train and predict the relationship between spectra and the corresponding class fractions. This process is flexible without any ancillary training dataset, and does not possess the limitations of linear spectral unmixing, which requires the number of endmembers to be no more than the number of spectral bands. The running time of the random forest regression is about ~1% of the running time of the linear mixture model. In addition, RERC adopts a spectral reflectance residual compensation approach to refine the fused image to make full use of the information from the LR image. RERC was assessed in the fusion of a prediction time MODIS with a Landsat image using two benchmark datasets, and was assessed in fusing images with different numbers of spectral bands by fusing a known time Landsat image (seven bands used) with a known time very-high-resolution PlanetScope image (four spectral bands). RERC was assessed in the fusion of MODIS-Landsat imagery in large areas at the national scale for the Republic of Ireland and France. The code is available at https://www.researchgate.net/proiile/Xiao_Li52.

The spatiotemporal reflectance fusion method is used to blend high-temporal and low-spatial resolution images with their low-temporal and high-spatial resolution counterparts that were previously acquired by various satellite sensors. Recently, a wide variety of learning-based solutions have been developed, but challenges remain. These solutions usually require two sets of data acquired before and after the prediction time, making them unsuitable for near-real-time predicting. The solutions are always trained band by band and thus do not consider the spectral correlation. High-resolution temporal changes are difficult to reconstruct accurately with the network structure used, which lowers the accuracy of the fusion result. To address these problems, this study proposes a novel spatiotemporal adaptive reflectance fusion model using a generative adversarial network (GASTFN). In GASTFN, an end-to-end network, including a generative and discriminative network, is simultaneously trained for all spectral bands. The proposed model can be applied to the one-pair case, consider the spectral correlation of each band, and improve the process of producing super-resolution imagery by adopting the discriminative network for image reflectance values rather than temporal changes in reflectance. The proposed model has been verified with two actual satellite data sets acquired in heterogeneous landscapes and areas with abrupt changes, with a comparison of the state-of-art methods. The results show that GASTFN can generate the most accurate fusion images with more detailed textures, more realistic spatial shapes, and higher accuracy, demonstrating that the GASTFN is effective for predicting near-real-time changes in image reflectance and preserves the most valuable spatial information.

Rivers are important elements of the earth’s ecosystem, and their spatial distribution information is critical for the study of hydrological and biogeochemical processes. Moderate Resolution Imaging Spectroradiometer (MODIS) imagery has been widely used for river mapping due to its high temporal resolution and long-term observation records, which are essential to capture the rapid fluctuation of rivers. However, when the conventional hard classification methods are used, the accuracy of the river maps produced from MODIS data (and especially for those rivers with narrow widths) is often limited because mixed pixels are common in MODIS imagery, due to coarse spatial resolution. In this article, a cascaded spectral–spatial information combined deep convolutional neural network (CNN) model for super-resolution river mapping (DeepRivSRM) is proposed to produce Landsat-like fine-resolution river maps from MODIS images. In DeepRivSRM, a CNN made up of a spectral unmixing module and a super-resolution mapping (SRM) module is introduced to handle the spectral and spatial information simultaneously. Moreover, for training the DeepRivSRM model, an adaptive cross-entropy loss function incorporating the fraction information of the rivers is designed to improve the performance of the DeepRivSRM model for small rivers. The proposed method was evaluated with MODIS images from three test sites and was compared with hard classification, a conventional SRM method, and a CNN-based SRM method. The results show that DeepRivSRM can generate more accurate river maps by effectively learning the subpixel-scale spatial–spectral information in MODIS imagery.

Optical earth observation satellite sensors often provide a coarse spatial resolution (CR) multispectral (MS) image together with a fine spatial resolution (FR) panchromatic (PAN) image. Pansharpening is a technique applied to such satellite sensor images to generate an FR MS image by injecting spatial detail taken from the FR PAN image while simultaneously preserving the spectral information of MS image. Pansharpening methods are mostly applied on a per-pixel basis and use the PAN image to extract spatial detail. However, many land cover objects in FR satellite sensor images are not illustrated as independent pixels, but as many spatially aggregated pixels that contain important semantic information. In this article, an object-based pansharpening approach, termed object-based area-to-point regression kriging (OATPRK), is proposed. OATPRK aims to fuse the MS and PAN images at the object-based scale and, thus, takes advantage of both the unified spectral information within the CR MS images and the spatial detail of the FR PAN image. OATPRK is composed of three stages: image segmentation, object-based regression, and residual downscaling. Three data sets acquired from IKONOS and Worldview-2 and 11 benchmark pansharpening algorithms were used to provide a comprehensive assessment of the proposed OATPRK approach. In both the synthetic and real experiments, OATPRK produced the most superior pan-sharpened results in terms of visual and quantitative assessment. OATPRK is a new conceptual method that advances the pixel-level geostatistical pansharpening approach to the object level and provides more accurate pan-sharpened MS images.

Super-resolution mapping (SRM) aims to determine the spatial distribution of the land cover classes contained in the area represented by mixed pixels to obtain a more appropriate and accurate map at a finer spatial resolution than the input remotely sensed image. The image-based SRM models directly use the observed images as input and can mitigate the uncertainty caused by class fraction errors. However, existing image-based SRM models always adopt a fixed set of endmembers used in the entire image, ignoring the spatial variability and spectral uncertainty of endmembers. To address this problem, this letter proposed an optimal endmember-based SRM (OESRM) model, which considers the spatial variations in endmembers, and determines the best-fit one for each coarse resolution pixel using the spectral angle and the spectral distance as the spectral similarity indexes. A Sentinel-2A and a Landsat-8 multispectral images were used to analyze the performance of OESRM, by comparing with three other SRM methods which adopt a fixed endmember set or multiple endmember sets. The results showed that OESRM generated resultant land cover maps with more spatial detail, and reduced the confusion between land cover classes with similar spectral features. The proposed OESRM model produced the results with the highest overall accuracy in both experiments, showing its effectiveness in reducing the effect of endmember uncertainty on SRM.

The mixed pixel problem is common in remote sensing. A soft classification can generate land cover class fraction images that illustrate the areal proportions of the various land cover classes within pixels. The spatial distribution of land cover classes within each mixed pixel is, however, not represented. Super-resolution land cover mapping (SRM) is a technique to predict the spatial distribution of land cover classes within the mixed pixel using fraction images as input. Spatial–temporal SRM (STSRM) extends the basic SRM to include a temporal dimension by using a finer-spatial resolution land cover map that pre- or postdates the image acquisition time as ancillary data. Traditional STSRM methods often use one land cover map as the constraint, but neglect the majority of available land cover maps acquired at different dates and of the same scene in reconstructing a full state trajectory of land cover changes when applying STSRM to time-series data. In addition, the STSRM methods define the temporal dependence globally, and neglect the spatial variation of land cover temporal dependence intensity within images. A novel local STSRM (LSTSRM) is proposed in this paper. LSTSRM incorporates more than one available land cover map to constrain the solution, and develops a local temporal dependence model, in which the temporal dependence intensity may vary spatially. The results show that LSTSRM can eliminate speckle-like artifacts and reconstruct the spatial patterns of land cover patches in the resulting maps, and increase the overall accuracy compared with other STSRM methods.

Sentinel-2 is a wide-swath and fine spatial resolution satellite imaging mission designed for data continuity and enhancement of the Landsat and other missions. The Sentinel-2 data are freely available at the global scale, and have similar wavelengths and the same geographic coordinate system as the Landsat data, which provides an excellent opportunity to fuse these two types of satellite sensor data together. In this paper, a new approach is presented for the fusion of Landsat 8 Operational Land Imager and Sentinel-2 Multispectral Imager data to coordinate their spatial resolutions for continuous global monitoring. The 30 m spatial resolution Landsat 8 bands are downscaled to 10 m using available 10 m Sentinel-2 bands. To account for the land-cover/land-use (LCLU) changes that may have occurred between the Landsat 8 and Sentinel-2 images, the Landsat 8 panchromatic (PAN) band was also incorporated in the fusion process. The experimental results showed that the proposed approach is effective for fusing Landsat 8 with Sentinel-2 data, and the use of the PAN band can decrease the errors introduced by LCLU changes. By fusion of Landsat 8 and Sentinel-2 data, more frequent observations can be produced for continuous monitoring (this is particularly valuable for areas that can be covered easily by clouds, thereby, contaminating some Landsat or Sentinel-2 observations), and the observations are at a consistent fine spatial resolution of 10 m. The products have great potential for timely monitoring of rapid changes.

Forest mapping from satellite sensor imagery provides important information for the timely monitoring of forest growth and deforestation, bioenergy potential assessment, and modeling of carbon flux, among others. Due to the daily global revisit rate and wide swath width, MODerate-resolution Imaging Spectroradiometer (MODIS) images are used commonly for satellite-derived forest mapping at both regional and global scales. However, the spatial resolution of MODIS images is too coarse to observe fine spatial variation in forest cover. The last few decades have seen the production of several fine-spatial-resolution satellite-derived global forest cover maps, such as Hansen’s global tree canopy cover map of 2000, which includes abundant spectral, temporal, and spatial prior information about forest cover at a fine spatial resolution. In this paper, a novel learning-based spatial–temporal superresolution mapping approach is proposed to integrate both current MODIS images and prior maps of Hansen’s tree canopy cover, to map present forest cover with a fine spatial resolution. The novel approach is composed of three main stages: 1) automatic generation of 240-m forest proportion images from both 240- and 480-m MODIS images using a nonlinear learning-based spectral unmixing method; 2) downscaling the 240-m forest proportion images to 30 m to predict the class possibilities at the subpixel scale using a temporal-example learning-based downscaling method; and 3) final production of the fine-spatial-resolution forest map by solving a regularization-based optimization problem. The novel approach produced more accurate fine-spatial-resolution forest cover maps in terms of both visual and quantitative evaluation than traditional pixel-based classification and the latest subpixel based superresolution mapping methods. The results show the great efficiency and potential of the novel approach for producing fine-spatial-resolution forest maps from MODIS images.

Superresolution mapping (SRM) is a technique for generating a fine-spatial-resolution land cover map from coarse-spatial-resolution fraction images estimated by soft classification. The prior model used to describe the fine-spatial-resolution land cover pattern is a key issue in SRM. Here, a novel learning-based SRM algorithm, whose prior model is learned from other available fine-spatial-resolution land cover maps, is proposed. The approach is based on the assumption that the spatial arrangement of the land cover components for mixed pixel patches with similar fractions is often similar. The proposed SRM algorithm produces a learning database that includes a large number of patch pairs for which there is a fine- and coarse-spatial-resolution representation for the same area. From the learning database, patch pairs that have similar coarse-spatial-resolution patches as those in the input fraction images are selected. Fine-spatial-resolution patches in these selected patch pairs are then used to estimate the latent fine-spatial-resolution land cover map by solving an optimization problem. The approach is illustrated by comparison against state-of-the-art SRM methods using land cover map subsets generated from the USA's National Land Cover Database. Results show that the proposed SRM algorithm better maintains the spatial pattern of land covers for a range of different landscapes. The proposed SRM algorithm has the highest overall accuracy and kappa values in all of these SRM algorithms, by using the entire maps in the accuracy assessment.

The Moderate Resolution Imaging Spectroradiometer (MODIS) is an important data set in global burned-area mapping. The MODIS global burned-area product has a coarse spatial resolution at approximately 500 m, which often introduces errors to the mapped burned areas. In this letter, a novel subpixel mapping (SPM) approach was proposed to produce burned-area maps at the fine spatial resolution similar to Landsat imagery, by exploring the spectral and spatial information provided by the second and fifth bands of MODIS. The proposed SPM approach aims to refine the estimate of burned areas, which have been detected by the MODIS global burned-area product. The performance of the proposed SPM approach was assessed with an experiment area containing six burned areas, by comparing with the MODIS burned-area product MCD45. The result shows that the average omission error decreased from 52.26% for MCD45 to 16.74% for SPM, and the average commission error decreased from 21.76% for MCD45 to 12.54% for SPM. The kappa value increased from 0.5583 for MCD45 to 0.8756 for SPM, indicating that the proposed SPM approach is effective in reducing the influence of the coarse spatial resolution of MODIS imagery in mapping a burned area and refining existing global burned-area products.

Super-resolution mapping (SRM) is a method for generating a fine-resolution land cover map from coarse-resolution fraction images. Example-regression-based SRM algorithms can estimate a fine-resolution land cover map with detailed spatial information by learning land cover spatial patterns from available land cover maps. Existing example-regression-based SRM algorithms are sensitive to fraction errors, and the results often include many linear artifacts and speckles. To overcome these shortcomings, this study proposes an improved example-regression-based SRM algorithm. The objective function of the proposed SRM algorithm comprises three terms. The first term is used to minimize the difference between the fraction values of the estimated fine-resolution land cover map and the input fraction values. The second term is used to maximize the class membership possibility values of the fine pixels in the result. The final term is used to make the result locally smooth. The proposed SRM algorithm is compared with several popular SRM algorithms using both synthetic and real fraction images. Experimental results indicate that the proposed SRM algorithm can produce results with less speckles and linear artifacts, more spatial details, smoother boundaries, and higher accuracies than the SRM results used for comparison.

Subpixel mapping (SPM) is a technique to obtain a land cover map with finer spatial resolution than the original remotely sensed imagery. An image-based SPM model that directly uses the original image data as input by integrating both the spectral and spatial information has been demonstrated as a promising SPM model. However, all existing image-based SPM models are based on a supervised approach, since the spectral term in these SPM models is composed of a supervised unmixing method. The endmembers and training samples for different land cover classes must be determined before implementing these supervised SPM algorithms. In this letter, a novel unsupervised image-based SPM model based on the fuzzy c-means (FCM) clustering approach (usFCM_SPM) was proposed. By incorporating the unsupervised unmixing criterion of the FCM clustering algorithm and the maximal land cover spatial-dependence principle, the proposed usFCM_SPM can generate a subpixel land cover map without any prior endmember information. Both synthetic multispectral image and real IKONOS image experiments demonstrate that the usFCM_SPM can generate higher accuracy subpixel land cover maps than the traditional unsupervised pixel-scale classification approaches and the unsupervised pixel-swapping model.

Superresolution land cover mapping (SRM) is a technique for generating land cover maps with a finer spatial resolution than the input image. In general, either the original multispectral (MS) images or the spectral unmixing results of the MS image are used as input for SRM models. Panchromatic (PAN) images are often captured together with MS images by many remote sensors and provide more spatial information due to their higher spatial resolution compared with the MS image. In this paper, a spatially adaptive spatial–spectral managed SRM model (SA_SSMSRM) that incorporates both MS and PAN images is proposed. SA_SSMSRM aims to better smooth homogeneous regions of objects (which represent a territory within which there is a uniformity in terms of land cover class) and preserve land cover class boundaries simultaneously by using the PAN image pixel photometric distance (i.e., gray-level distance or pixel value difference). Homogeneous regions in the PAN images are usually characterized by the photometric (pixel value) similarity, whereas class boundaries are usually characterized by photometric dissimilarity. The SA_SSMSRM smoothing parameter, which controls the contribution of the prior term (which encodes prior knowledge about land cover spatial patterns), is designed to be spatially adaptive, with its value decreasing if the photometric similarity of neighboring PAN image pixels decreases. SA_SSMSRM was examined on high-spatial-resolution QuickBird images, IKONOS images, and Advanced Land Observing Satellite (ALOS) images with both MS and PAN data. Results showed that the proposed SA_SSMSRM can generate more accurate superresolution maps than other SRM models.

Superresolution mapping (SRM) is a technique for translating original coarse-resolution fractions into a fine-resolution land cover map by dividing a coarse-resolution pixel into a few finer resolution pixels and determining their class labels. SRM can be solved by considering it a maximum a posteriori principle-based classification problem and by assigning each fine pixel as the class with the highest probability. Fine-pixel class membership probabilities (CMPs) can be calculated at two different scales: at the fine scale, in which the target fine pixel is compared with other fine pixels, and at the coarse scale, in which coarse fractions are downscaled into fine-pixel probabilities. The fine-scale CMP is suitable for representing local land cover features but not for maintaining global features. The coarse-scale CMP is the opposite of the fine-scale CMP. This paper proposes a novel multiscale approach to overcome this shortcoming by fusing the CMP calculated at both fine and coarse scales with the tau model. With the fused CMP, a simulated-annealing algorithm is applied to produce a fine-resolution land cover map. The land cover maps generated from QuickBird and IKONOS images and the National Land Cover Database were used to validate the effectiveness of the proposed SRM algorithms. The proposed SRM algorithms were evaluated visually and quantitatively by comparing them with several existing SRM algorithms. The results indicate that the accuracy of land cover maps at fine spatial resolution increased significantly compared with that obtained from all existing SRM algorithms.