Deep learning has achieved promising progress for digital elevation model (DEM) super-resolution (SR). However, the existing methods rarely consider the integration of multi-modal data with auxiliary high-frequency information. A primary challenge stems from the heterogeneous feature representations among these data sources, which complicates the effective learning of the terrain feature mapping relationships. In this paper, we propose a novel framework for DEM SR by integrating optical remote sensing imagery as the auxiliary data. A terrain-guided texture-edge feature fusion network is constructed to transfer the feature representation of high-resolution image textures with the guidance of informative terrain features, for adapting DEM SR learning. By exploiting the multi-dimensional attention mechanism, the meaningful components from the image conforming to the terrain features provide high-frequency information for DEM SR, while noisy features related to spectral variations are excluded from modelling. The terrain-oriented textural and edge features are then fused to generate the SR result with the constraint of a terrain feature-aware loss function. Extensive experiments on both simulated and real datasets indicate that the proposed method can reconstruct a DEM with high-accuracy elevation and sharper terrain details, and outperforms the state-of-art methods.

Recent advancements in neural radiance fields (NeRF) have demonstrated significant potential in 3D reconstruction and novel view synthesis. However, their application in photogrammetry remains limited due to challenges in geometric accuracy for unbounded scenes and training inefficiency. We propose Bidirectional Depth-Space Optimized NeRF (BiDS-NeRF), a framework that integrates depth estimation with space warping optimization to balance high-precision geometric reconstruction and high-fidelity rendering. A lightweight architecture leveraging multi-resolution hash encoding is first established, achieving significant training acceleration while preserving rendering quality. Building upon this foundation, an implicit surface segmentation mechanism with dynamic space warping is proposed, effectively resolving distant object representation challenges in unbounded scenes. Finally, geometric consistency is enhanced through the integration of a monocular depth estimation pre-training model, coupled with a novel bidirectional optimization strategy that synchronizes photometric and geometric constraints. Complementing our technical contributions, an open-source UAV benchmark dataset with multiple categories of unbounded scenes is established, providing quantifiable metrics for cross-scene applicability assessment. Experiments on the OMMO dataset and our UAV dataset demonstrate improvements in relevant metrics, particularly in complex geometric regions. Our method provides a practical solution for UAV-based 3D reconstruction, advancing NeRF’s applicability in large-scale scene modeling.

Land surface temperature (LST) serves as a key indicator for studying the thermal characteristics of the land-atmosphere interface. Nevertheless, cloud cover and atmospheric particulates frequently obstruct the thermal infrared (TIR) spectral range in satellite observations, leading to significant data missing in derived LST products. Multi-source data fusion methods are the most effective means for all-weather TIR LST reconstruction. However, most of the existing multi-source data fusion reconstruction methods only consider the spatial and temporal differences individually, which makes it challenging to ensure spatio-temporal consistency in LST gap-filling. In this study, a novel framework for LST reconstruction is proposed by constructing a spatio-temporal consistency-guided global-local fusion network (STCGL-Net) to fuse TIR and reanalysis data. Utilizing a deep convolutional network architecture, the STCGL-Net efficiently captures the local spatial features and texture details of multi-source observations. At the same time, spatio-temporal consistency Transformer and coordinate attention mechanisms are cleverly embedded to capture the global spatiotemporal dependencies that drive changes in LST. Using spatio-temporal consistency as the core physical constraint to guide the STCGL-Net's training process ensures that the reconstructed LST is not only numerically accurate, but also evolves in accordance with physical laws. As a key input parameter, ERA5 surface solar radiation data quantitatively characterize solar forcing on land surface thermal dynamics and enhance LST reconstruction accuracy. Evaluation demonstrates STCGL-Net achieves cloud-contaminated LST reconstruction with R²=0.87 and MAE=0.54K. Comparative evaluations with three established classic methods confirm the STCGL-Net’s consistent performance across various spatial and temporal scales. Validation against Surface Radiation Budget station LST, confirms reconstruction robustness (R²:0.8-0.9, MAE:3-5K), establishing reliable all-weather LST generation capability.

Polarimetric synthetic aperture radar (SAR) can capture rich polarization information of targets, but it is inherently affected by speckle. Learning-based methods have demonstrated superior speckle suppression potential. Most existing methods use optical images to simulate SAR noise for model training. Because of the significant differences in the imaging mechanisms between optical and SAR images, the data characteristics of these two types differ significantly, resulting in poor generalization performance. To this end, an Information Flow Switching-based Despeckling Network (IFSDN) is proposed for dual-polarization SAR image. By using the long time series data, the first dual-polarization SAR real dataset is constructed. The hybrid feature extraction module (HFEM) is constructed to independently extract and integrate features from both the diagonal and nondiagonal elements of the covariance matrix. Additionally, the multihierarchical residual attention despeckling (MRAD) module performs despeckling on feature maps from low to high levels. On this basis, the information flow switching mechanism facilitates the interaction of dominant features before and after despeckling, injecting spatial details into the despeckled results, reducing speckle noise, and preserving polarization information. By considering temporal changes, an adaptive joint loss function is, furthermore, constructed to guide the network training process, achieving high-fidelity despeckling while maintaining spatial-polarization information. Experiments show that IFSDN outperforms existing state-of-the-art methods in the speckle removal task for real dual-polarization SAR images, which can effectively preserve spatial and polarization information while suppressing speckles. Besides, generalization experiments demonstrate that the proposed model can be effectively applied to diverse datasets across various climate zones, showcasing its strong robustness.

Shadows significantly reduce the available information in remote sensing images, obstructing downstream tasks such as object detection, scene classification, and localization. However, shadow removal from remote sensing images is still an open issue, for the following reasons. Firstly, deep neural networks are difficult to train since the corresponding ground truths of shadows are almost always unavailable in practice. Secondly, the existing shadow removal methods still suffer from blurry details and boundary artifacts. In this paper, we describe how a generative shadow synthesis and removal framework that couples data-driven methods with illumination models was developed to address the above challenges effectively. Various shadows were synthesized in shadow-free regions of remote sensing images by GSS-Net, which is a generative shadow synthesis network that considers the physical process of shadow illumination attenuation. In this way, a large-scale, diverse, and realistic shadow dataset (RS-SynShadow) was built. A generative shadow removal network—GSR-Net—embedding a histogram-enhanced illumination model, was then developed for high-fidelity shadow removal without artifacts. Extensive experiments conducted on synthetic and real data demonstrate that the proposed shadow synthesis and removal framework significantly outperforms the state-of-the-art methods, both visually and quantitatively. The dataset and code will be made available at https://github.com/fzzfRS/RS-GSSR.

The data quality issue induced by atmospheric and other disturbances can significantly impede the application of remote sensing vegetation indices (VIs). Despite the development of numerous VI reconstruction techniques, two major challenges remain, i.e., the reliance on quality flag inputs and parameter settings. Quality flags are usually necessary as inputs for the different methods to improve the reconstruction accuracy, but mislabeling can be common in the quality flag data, which can directly introduce uncertainties. Furthermore, constant parameter schemes are usually assigned during the reconstruction applications, but the optimal parameters for all the models can show great spatial heterogeneity. Accordingly, in this paper, we propose a parameter-free and flag-free adaptive time-series method based on a variational reconstruction framework (PF-Free) to address the afore-mentioned issues, which can be applied without any parameter or flag inputs. PF-Free makes full use of the time-series temporal smoothness and inter-annual similarity to label the data after time-series rearrangement, and the parameters are adaptively selected using an improved generalized cross-validation (GCV) technique. Simulation and real-data experiments all demonstrate that PF-Free can achieve better and more stable reconstruction results, compared to other comparative methods. The adaptive quality flags can denote the data quality robustly and accurately, while guaranteeing better reconstruction performance, as long as there is any mislabeling in the original flags. Moreover, the adaptive parameter selection strategy considers the great spatial heterogeneity in the optimal parameters on a pixel-by-pixel basis, leading to more stable reconstruction outcomes under complex conditions. Further experiments also prove the effectiveness of PF-Free in processing data without quality flags or severely contaminated data, using Advanced Very High-Resolution Radiometer (AVHRR) data and Moderate Resolution Imaging Spectroradiometer (MODIS) daily normalized difference vegetation index (NDVI) data. This work provides a practical reconstruction method for VI time series, which is both flexible and convenient, without requiring any parameter or flag input, which we believe will advance VI reconstruction applications.

Image super-resolution (SR) is an effective way to enhance the spatial resolution and detail information of remote sensing images to obtain a superior visual quality. As SR is severely ill-conditioned, effective image priors are necessary to regularize the solution space and generate the corresponding high-resolution (HR) image. In this article, we propose a novel gradient-guided multiframe SR (MFSR) framework for remote sensing imagery reconstruction. The framework integrates a learned gradient prior as the regularization term into a model-based optimization method. Specifically, the local gradient regularization (LGR) prior is derived from the deep residual attention network (DRAN) through gradient profile transformation (GPT). The nonlocal total variation (NLTV) prior is characterized using the spatial structure similarity of the gradient patches with the maximum a posteriori (MAP) model. The modeled prior performs well in preserving edge smoothness and suppressing visual artifacts, while the learned prior is effective in enhancing sharp edges and recovering fine structures. By incorporating the two complementary priors into an adaptive norm-based reconstruction framework, the mixed L1 and L2 regularization minimization problem is optimized to achieve the required HR remote sensing image. Extensive experimental results on remote sensing data demonstrate that the proposed method can produce visually pleasant images and is superior to several of the state-of-the-art SR algorithms in terms of the quantitative evaluation.

Data quantity and quality are both critical for information extraction and analyzation in remote sensing. The current remote sensing datasets, however, often fail to meet these two requirements, for which the cloud is a primary factor degrading the data quantity and quality. This limitation affects the precision of results in remote sensing applications, particularly those derived from data-driven techniques. In this article, a physical law embedded generative cloud synthesis (PGCS) method is proposed to generate diverse,ealistic cloud images to enhance real data and promote the development of algorithms for subsequent tasks, such as cloud correction, cloud detection, and data augmentation for classification, recognition, and segmentation. The PGCS method involves two key phases: spatial synthesis and spectral synthesis. In the spatial synthesis phase, a style-based generative adversarial network is used to simulate the spatial characteristics, generating an infinite number of single-channel clouds. In the spectral synthesis phase, the atmospheric scattering law is embedded through a local statistics and global fitting method, converting the single-channel clouds into multispectral clouds. The experimental results demonstrate that PGCS achieves a high accuracy in both phases and performs better than three other existing cloud synthesis methods. Two cloud correction methods are developed from PGCS and exhibits a superior performance compared to state-of-the-art methods in the cloud correction task. The application of PGCS with data from various sensors was, furthermore, investigated and successfully extended. Code will be provided athttps://github.com/Liying-Xu/PGCS.

Shadow removal is beneficial for various remote sensing applications, such as semantic segmentation and object detection. However, traditional shadow removal methods perform poorly when applied to high-resolution remote sensing images. Some deep learning-based models exhibit high-level accuracy of removal, but it is difficult to be expanded to multi-scenarios due to insufficient pairs of shadow/shadow-free data in reality. In this paper, a novel framework is proposed from shadow synthesis to shadow removal, aiming to improve the usability of deep learning-based models in the task of shadow removal in high-resolution remote sensing images. First, we combine the physical shadow illumination model and a domain alignment network to synthesize a large variety of realistic shadows. Then, a shadow removal network considering global-local features is built to restore the ground surface details finely. Numerous experiments have shown that the proposed framework can effectively remove various shadows and is superior to existing methods.

There is no "cloudy & cloud-free" paired images with totally identical surface information under the same spatial and temporal condition in reality, which limits the development of supervised deep learning methods in the field of cloud removal and detection. In this regard, a high-fidelity thin cloud synthetic method is proposed, which is more challenging than the synthesis of thick clouds. This method combines physical model and data-driven methods. The expression of scattering laws at the pixel level is extended to the channel level to synthesize multi-channel cloud from cirrus band with controlled cloud thickness. Besides, spatial and spectral features are learned from real data using generative adversarial networks and transformed to synthetic data. Based on it, a dataset containing infinite number of "cloudy & cloud-free" pairs can be constructed. Experimental results show that the proposed method has the best visual effect with highest quantitative evaluations compared with current methods.

Radiometric normalization is a critical step in producing consistent vegetation index data. However, to date, there have been few studies in the field of high-resolution vegetation index mapping. In this letter, a set of reference-based radiometric normalization methods, combining absolute and relative approaches, are evaluated for high-resolution vegetation index mapping. A framework incorporating a region-to-region [moment matching (MM)] method is introduced and compared to a traditional pixel-to-pixel method (M-estimation). Three modeling strategies (global, classification, and blocked) are evaluated for the two methods. Specifically, we describe the six sets of normalization experiments conducted with high-resolution images captured by China’s Gaofen-2 (GF-2) satellite and compare the results qualitatively and quantitatively. In addition, the sensitivity of the three parameters (purity threshold, number of categories, and block size) is analyzed. The results show that the MM method outperforms the M-estimation method visually. However, opposite performances are found in the quantitative evaluation. In summary, the blocked modeling strategy is recommended if the data volume is small. The findings of this letter will have important implications for the production of GF-2 vegetation index products at a large scale.

High-resolution seamless remote sensing (HRSRS) images are essential foundational data for natural resource monitoring and land-use assessment, etc. However, high spatial resolution (HR) Earth observation from satellites typically has a long revisit period, and optical images can be widely obscured by clouds. Furthermore, the geometry and radiation issues make the generation of HRSRS images a challenging task. In this letter, to systematically address these issues, we propose a robust and efficient framework designed to generate HRSRS images for regional-scale areas, integrating various mature image processing technologies and jointing super-resolution (SR) reconstruction and thick cloud removal for the first time. In particular, the frame-work can realize adaptive reconstruction of lost spatial information on demand based on the satellite observation coverage and thick cloud coverage information. The effectiveness and reliability of the framework was demonstrated by its successful application in generating a 1-m resolution quarterly seamless image of the city of Wuhan, Hubei province, China, using 34 images acquired by the Chinese Gaofen (GF)-1/2/6/7 satellites. The experimental results show that both the intermediate results and final results achieve a satisfactory visual and quantitative effect. For example, SR reconstruction improves the spatial resolution of low-resolution images from 2-m to 1-m, thus increasing the spatial frequency and entropy by about 0.62 and 0.12, respectively.

Image super-resolution (SR) enhances the spatial resolution and details of remote sensing images to obtain a better visual quality. However, the existing SR methods rarely consider the imaging degradation factors caused by unfavorable atmospheric conditions, which hinders them from accurately modeling the scale dependencies for the real texture features and results in artifacts in the reconstructed images. In this letter, we propose the degradation-oriented progressive learning SR (DoPSR) framework, which handles the degradation issues posed by the low resolution, together with haze, in remote sensing images. Specifically, DoPSR progressively optimizes the SR estimation for the haze-corrupted input by decomposing the challenging restoration task into feasible sub-problems. It begins with two parallel sub-networks for haze component prediction and detail enhancement, respectively, and then generates the preliminary haze removal result. Furthermore, a novel collaborative refinement mechanism is introduced to encourage the refinement of the coarse result by leveraging the extracted degradation component of the former stage as prior information. In particular, a degradation-aware fusion module is designed to integrate the degradation prior through cross-stage feature aggregation. Extensive experimental results demonstrate the superiority of the proposed method, which outperforms competitive methods by 0.3-1.6 dB in the peak signal-to-noise ratio (PSNR) on the synthetic dataset.

Transportation network extraction based on deep learning has become a hotspot. However, the existing models all aim to distinguish between background and transportation network, while ignoring the class attributes within the transportation networks. In this letter, we propose a multi-class transportation network extraction network (MCTN-Net) to simultaneously extract railways, roadways, trails and bridges. Inspired by multi-task learning, the network first extracts the semantic and information together by the use of a dense feature shared encoder (DFSE). The orientation and semantic features are then fused in the orientation-guided stacking module (OGSM) to enhance the connection between transportation network pixels. Furthermore, a semantic refinement branch (SRB) is designed to improve the ability of classifying different transportation network types through deep supervised fusion and class attention. A multi-class transportation network dataset was constructed and used in the experiments. The experiential results indicate that the proposed method achieves an MIoU of 64.29% and an FWIoU of 71.20% without the background, which is significantly better than the other road extraction models and semantic segmentation methods. The code and dataset are available at https://github.com/fzzfRS/MCTN-Net.

Remote sensing (RS) images change detection (CD) plays a crucial role in monitoring surface dynamic. However, current deep learning (DL)-based CD methods still suffer from pseudo changes and scale variations due to inadequate exploration of temporal differences and under-utilization of multiscale features. Based on the aforementioned considerations, a spatiotemporal enhancement and interlevel fusion network (SEIFNet) is proposed to improve the ability of feature representation for changing objects. Firstly, the multilevel feature maps are acquired from Siamese hierarchical backbone. To highlight the disparity in the same location at different times, the spatiotemporal difference enhancement modules (ST-DEM) are introduced to capture global and local information from bitemporal feature maps at each level. Coordinate attention and cascaded convolutions are adopted in subtraction and connection branches, respectively. Then, an adaptive context fusion module (ACFM) is designed to integrate interlevel features under the guidance of different semantic information, constituting a progressive decoder. Additionally, a plain refinement module and a concise summation-based prediction head are employed to enhance the boundary details and internal integrity of CD results. The experimental results validate the superiority of our lightweight network over 8 state-of-the-art (SOTA) methods on LEVIR-CD, SYSU-CD and WHU-CD datasets, both in accuracy and efficiency. Also, the effects of different types of backbones and differential enhancement modules are discussed in the ablation experiments in details. The code will be available at https://github.com/lixinghua5540/SEIFNet.

Surface soil moisture plays a pivotal role in various hydrological processes. Precisely assessing soil moisture with high resolution is crucial for effective water resource management, informed agricultural decision-making, and in-depth climate change research. While passive microwave remote sensing is a primary technology for regional soil moisture monitoring, its practical application is hindered by data discontinuity and low resolution. To address these challenges, we propose an integrated learning framework to enhance the continuity and resolution of soil moisture data across China. Leveraging low-resolution passive microwave soil moisture data, moderate-resolution assimilated soil moisture data, and multiple high-resolution ancillary inputs, the network effectively captures the spatiotemporal dynamics of soil moisture through integrating gap-filling, multisource fusion, and spatial downscaling processes. Validation against in situ data demonstrates the significant enhancements achieved by the proposed method, with an average R value of 0.706 and an average unbiased root mean square error of 0.055 m3/m3. Comparative analysis further confirms its superior accuracy and robustness across diverse regions. These findings highlight the potential of this integrated learning framework to advance hydrological applications, enhance agricultural production, and support climate research.

Hyperspectral images (HSIs) are inevitably degraded by a mixture of various types of noise, such as Gaussian noise, impulse noise, stripe noise, and dead pixels, which greatly limits the subsequent applications. Although various denoising methods have already been developed, accurately recovering the spatial-spectral structure of HSIs remains a challenging problem to be addressed. Furthermore, serious stripe noise, which is common in real HSIs, is still not fully separated by the previous models. In this paper, we propose an adaptive hyper-Laplacian regularized low-rank tensor decomposition (LRTDAHL) method for HSI denoising and destriping. On the one hand, the stripe noise is separately modeled by the tensor decomposition, which can effectively encode the spatial-spectral correlation of the stripe noise. On the other hand, adaptive hyper-Laplacian spatial-spectral regularization is introduced to represent the distribution structure of different HSI gradient data by adaptively estimating the optimal hyper-Laplacian parameter, which can reduce the spatial information loss and over-smoothing caused by the previous total variation regularization. The proposed model is solved using the alternating direction method of multipliers (ADMM) algorithm. Extensive simulation and real-data experiments all demonstrate the effectiveness and superiority of the proposed method.

Supervised deep learning despeckling methods usually use optical images to simulate multiplicative noise for training. However, due to the different imaging mechanisms of optical images and SAR images, the data characteristics of the two are significantly different, resulting in poor generalization performance of the model trained through the above form. Besides, the existing deep learning models do not fully consider the physical scattering mechanism, which causes the loss of polarization information. To solve those problems, an unsupervised deep learning method is proposed for dual-polarization SAR image despeckling. Under this framework, we combine the dual-polarization SAR covariance matrix and polarization decomposition information to construct a Dual-branch SAR image Despeckling Network (DSDN). The residual channel and the spatial attention mechanism are embedded to calibrate the polarization and spatial feature maps. The cross-attention mechanism is designed to mine the association of feature maps before and after denoising. Besides, the dual-branch joint loss function is proposed to constrain the training process. Spatial information experiments and polarization information experiments indicate that, compared with the existing state-of-the-art SAR despeckling methods, the proposed method can effectively remove the coherent speckle noise of dual-polarization SAR images, and can better preserve the polarization information. Codes are available at https://github.com/LiupengLin/DSDN.

Deep learning has achieved significant success in pan sharpening, but there are still two nonnegligible challenges. First, most of the existing methods rely on reduced-resolution training samples, limiting their performance when migrating from simulated to real-world scenes. Second, they pay insufficient attention to the imaging mechanism and the complexity and heterogeneity of remote sensing information, which leads to unclear representation of the relationships between images and the underutilization of the full-resolution features. In response to the abovementioned issues, this article presents an unsupervised pan-sharpening network incorporating imaging spectral prior and spatial-spectral compensation, named USCPNet. First, a structure-guided cross-attention (CA) residual (SCAR) block is constructed, deriving the desired high-resolution multispectral (HRMS) image guided by the panchromatic texture-structure features. A spectrally adaptive degradation network (SNet) coupling imaging spectral prior is then introduced, which characterizes the pixel-by-pixel spectral mapping between the HRMS image and the high-resolution panchromatic (HRPAN) image, to implement a precise spatial constraint driven by the imaging mechanism. In addition, given the difficulty of comprehensively extracting and integrating complementary features within an unsupervised framework through single-stream fusion, spatial-spectral joint progressive compensated (SSPC) stages are employed to achieve refined enhancement of the effective information in the predicted HRMS image through iterative rounds of residual fusion. Experiments conducted on Gaofen-1 (GF-1), Gaofen-2 (GF-2), and WorldView-2 (WV-2) satellite images reveal that the proposed USCPNet excels in spatial enhancement (SE) while preserving spectral fidelity. USCPNet also demonstrates advantages in specific applications, such as large-scale image fusion and vegetation index generation, compared with state-of-the-art methods.

Local climate zone (LCZ) mapping can explore the variability of the impact of urban form on the thermal environment in different urban contexts, and large-scale LCZ mapping can help us to better understand the spatial and temporal dynamics of the climate in urban areas around the world. Studies have indicated that deep learning-based methods can effectively perform LCZ classification. However, the accuracy of LCZ classification on large-scale datasets is still unsatisfactory, mainly due to the fact that the traditional convolutional neural networks are not good at mining contextual information, which is crucial for fully understanding remote sensing scenes. In this paper, to solve this problem, we propose an LCZ mapping method based on remote sensing images by coupling multi-level features mined from global and local ranges with prior knowledge, named LCZ-MFKNet. The global and local features are extracted through Swin Transformer and space-maintained ResNet (SM-ResNet) model branches, respectively, and then fused through an improved squeeze-and-excitation (iSE) module. The prior knowledge studied from the theoretical definition and experimental tests is that two typical sets of LCZ categories are easily confounded in multi-class classification but separable in two-class classification. Experiments are conducted on the large publicly available So2Sat LCZ42 dataset, where the proposed LCZ-MFKNet method achieved the highest LCZ mapping accuracy. Moreover, six megacities were selected globally for LCZ mapping, and the results verified the accuracy and the general applicability of the proposed LCZ-MFKNet method in large-scale LCZ mapping.

Recently, deep learning has been widely used for cloud detection in satellite images. However, due to radiometric and spatial resolution differences in images from different sensors and time-consuming process of manually labeling cloud detection datasets, it is difficult to effectively generalize deep learning models for cloud detection in multi-sensor images. This paper propose a weakly supervised learning method for transferring deep models for cloud detection in multi-sensor images (TransMCD), which leverages the generalization of deep models and the spectral features of clouds to construct pseudo-label dataset to improve the generalization of models. A deep model is first pre-trained using a well-annotated cloud detection dataset, which is used to obtain a rough cloud mask of unlabeled target image. The rough mask can be used to determine the spectral threshold adaptively for cloud segmentation of target image. Block-level pseudo-labels with high confidence in target image are selected using the rough mask and spectral mask. Unsupervised segmentation technique is utilized to construct a high-quality pixel-level pseudo-label dataset. Finally, the pseudo-label dataset is utilized as supervised information for transferring the pre-trained model to target image. The TransMCD method was validated by transferring model trained on 16-m Gaofen-1 wide field of view images to 8-m Gaofen-1, 4-m Gaofen-2 and 10-m Sentinel-2 images. The F1-score of the transferred models on target images achieves improvements of 1.23–9.63% over the pre-trained models, which is comparable to the fully-supervised models trained with well-annotated target images, suggesting the efficiency of the TransMCD method for cloud detection in multi-sensor images.

Because of the presence of clouds, the available information in optical remote sensing images is greatly reduced. These temporal‐based methods are widely used for cloud removal. However, the temporal differences in multitemporal images have consistently been a challenge for these types of methods. Towards this end, a bishift network (BSN) model is proposed to remove thick clouds from optical remote sensing images. As its name implies, BSN is combined of two dependent shifts. Moment matching (MM) and deep style transfer (DST) are the first shift to preliminarily eliminate temporal differences in multitemporal images. In the second shift, an improved shift net is proposed to reconstruct missing information under cloud covers. It introduces multiscale feature connectivity with shift connections and depthwise separable convolution (DSC), which can capture local details and global semantics effectively. Through experiments with Sentinel‐2 images, it has been demonstrated that the proposed BSN has great advantages over traditional methods and state‐of‐the‐art methods in cloud removal.

Seamline detection is one of the most important issues in mosaicking high-resolution remote sensing images (HRRSIs). However, it is difficult to make a balance between efficiency and accuracy. On that account, a shortest matrix path-based dynamic programming (SMP-DP) algorithm is proposed to find the optimal seamline for HRRSI mosaicking. First, a pixel cost matrix defined by intensity difference, gradient similarity, and geometric difference is constructed in the overlapping area. Second, the least average path cost from the starting pixel to each pixel is calculated and the SMP-DP algorithm is applied to find the optimal path. Experimental results on HRRSI prove that the proposed method detects high-quality seamline and crosses much fewer objects with the highest computational efficiency, compared with the state-of-the-art method and commercial software.

High-resolution nighttime light (HR-NTL) imagery is a reliable indicator of human activity in urban areas at night. However, HR-NTL images on Earth are blurred because of issues such as blooming, noise, and overexposure in the imaging system and transmission path. Toward this end, a nighttime light image deblooming network (NTL-Db-Net) is proposed, which aims to make the Earth clearer at night. The NTL-Db-Net employs a generative adversarial network (GAN) as its deblooming framework. As the lack of ideal clear NTL data for GAN training, our method integrates blur transfer algorithm to simulate degradation for generating training data. This method was tested using HR-NTL data on Wuhan, Macau, Changchun, Xiamen, Atlanta and Jerusalem from HR Jilin-1 satellite constellation. Compared with the results of state-of-the-art SEAM, Uformer, and DRBNet, those from NTL-Db-Net demonstrated superior visual features and achieved the highest scores across all selected non-reference evaluation metrics. NTL-Db-Net also exhibited a lower number of network parameters and a smaller time cost. In addition, this study conducted a spatial clustering analysis of nighttime light with road networks and permeable surfaces. The results indicated that nightlights and roads exhibited a distinct spatially aggregated distribution, while permeable surfaces displayed a clear spatially dispersed distribution. Code and data are available at https://github.com/lixinghua5540/NTL-Db-Net.

Shadow correction is an important task in the analysis of high-resolution remote sensing images, as the existence of shadows reduces radiometric information and causes changes in the energy distribution. This is especially the case when the spatial resolution is very high, as the shadows disturb the subsequent processing and applications, such as image mosaicking, classification, segmentation, etc. Traditional shadow correction methods are limited by the shadow detection accuracy and the available non-shaded samples in the imagery. In this paper, we propose an evolutionary shadow correction network (ESCNet) and describe how we built a benchmark unmanned aerial vehicle (UAV) image dataset to achieve shadow correction directly, without shadow detection. The proposed ESCNet is made up of two sub-networks with an evolutionary relationship: a shadow removal network (SRNet) followed by a radiation adjustment network (RANet). The shadows are first removed by SRNet trained on the UAV image dataset to achieve the primary shadow-corrected image, and the global radiation is then adjusted to a sunlit-like status by RANet. Shadow detection is not required in the proposed method, which effectively overcomes the error accumulation and shadow edge artifact problem of the traditional methods. Experiments were carried out and the results were compared with those of both traditional and deep learning-based shadow correction methods, for which both qualitative and quantitative evaluations were performed. The results suggest that the proposed method shows obvious advantages in information recovery for shadow regions, and the global brightness of the corrected imagery is consistent with that of sunlit conditions.

The separation of land surface temperature (LST) and land surface emissivity (LSE) is an ill-posed problem in thermal infrared (TIR) remote sensing. By building a new observation matrix to compress the LSE unknows and a dictionary training method to reconstruct complete LSE spectra, a new dictionary-based sparse representation for emissivity (DSRE) method has been proposed to retrieve LST and LSE from the atmospherically corrected hyperspectral TIR data. The proposed method fully utilizes the sparsity of compressed sensing and the empirical knowledge of the trained emissivity dictionary. The sensitivity analysis shows that the modeling accuracies of the proposed method are 0.215Kand 0.0060 for LST and LSE, respectively. Even with the instrument noise of 0.3Kand the uncertainties in atmospheric transmittance, atmospheric upwelling, and downwelling radiance of 10 %, the retrieval accuracies are 0.811Kfor LST and 0.0241 for LSE, respectively. Then a field experiment was conducted to validate the proposed method, and a comparison was executed to three published methods, including ASTER temperature-emissivity separation (ASTERTES), linear spectral emissivity constraint TES (LSECTES), and iterative spectrally smooth TES (ISSTES). The accuracies of retrieved LST and spectral LSE are 1.41K/ 0.009, 2.57K/ 0.071, 1.59K/ 0.038, and 2.00K/ 0.077 for DSRE, ASTERTES, LSECTES, and ISSTES. In contrast to the three published methods, our proposed method is more accurate and effective than other published methods. Especially in the atmospheric absorption band, the proposed method has a strong anti-noise capability to the residuals of environmental downwelling radiance.

Diatom blooms frequently observed in the river tributaries pose a threat to the regional water environment. Satellite remote sensing provides us with an effective tool to delineate the extent of large-scale harmful algal blooms (HABs) in both oceans and inland waters. However, the diatom bloom detection in river systems remains challenging, due to the high demand for a robust model to characterize the spatial and spectral features from the satellite images with high resolution and limited spectral bands. In this study, we developed a novel deep learning-based framework for the detection of the status of diatom blooms in river tributaries using Sentinel-2 MultiSpectral Imager (MSI) images. Distinct from the previous works for detection of bloom extent, the water pixels are categorized as ‘non-bloom’, ‘mild bloom’, and ‘severe bloom’, corresponding to the different bloom intensities. To achieve this, a simple convolutional neural network (CNN) is trained using the collected spectral samples from MSI images characterizing the different bloom statuses. The input features include the spectral variables highly correlated to the pivotal absorption and backscattering properties of diatoms, the chlorophyll-a (Chla), and water temperature. The classification results can then be obtained based on the spectral and environmental characteristics learned by the constructed model. The trained model was extensively tested in tributaries of Yangtze River, where both visual and quantitative validation demonstrated that the proposed model can achieve reliable detection results.

Cloud contamination is inevitable in optical satellite images, especially for those in visible and near-infrared (VNIR) spectra. A general thin cloud correction method for satellite VNIR images is proposed in this study by coupling statistical information with a scattering model to solve the abovementioned problem. A thin cloud map (TCM) is created by utilizing the characteristics of land surface and thin clouds to depict the thin cloud spatial distribution and relative intensity. Furthermore, different cloud reflectance estimation (CRE) algorithms are proposed for different VNIR bands by considering the scattering properties of thin clouds. For coastal and blue bands with short wavelengths, the images are divided into multiple layers to search for dark pixels based on TCM so that thin clouds can be estimated via robust regression. The CRE of the green, red, and near-infrared bands with long wavelengths is realized via the scattering model by taking the thin clouds of coastal or blue bands as a reference. Experiments are performed on cloud-covered VNIR images captured by different satellites to validate the universality of the proposed method. Two traditional methods and one deep learning method are utilized for a comparison. Compared with the benchmark methods, the proposed method yields totally cloud-free images and more credible color. The quantitative measures obtained by the proposed method are the closest to the ideal values among the four methods. Discussions of the novelties of the proposed method, the extended applications of TCM, and the parallax problem in experiments is also performed to complete the evaluation.

High temporal resolution aerosol optical depth (AOD) observations derived from new-generation geostationary (GEO) satellite possess unique advantages in analyzing aerosol fast variation processes and thereby providing more accurate assessments on their climate effects and health risks. Unfortunately, the expected advantages and values are dramatically limited by relatively large proportion of data missing in the GEO AOD products due to cloud obscuration and intrinsic retrieval algorithm. Although several data recovery algorithms have been proposed in recent years to improve the spatial coverage for GEO AOD products, yet most of them aims at filling up the data blanks rather than reconstructing the temporally continuous variation of aerosol. Accordingly, in this study, a novel framework of nested spatiotemporal fusion blending GEO with sun-synchronous orbit (SSO) satellite observations based on Bayesian maximum entropy (BME) theorem is developed for GEO Advanced Himawari-8 Imager (AHI) AOD recovery with the sufficient excavation of complementary information from GEO and SSO satellite observations, where the minute-stage and hour-stage BME fusion are jointly employed to reconcile temporal inconsistency and data discrepancies between GEO and SSO observations. The results demonstrate that the AOD spatial coverage is dramatically increased by 240.9% (from 20.5% to 70%) with ensured accuracy after Nested-BME fusion. Additionally, two case analyses, during the development and dispersion processes of haze respectively, both demonstrate that the proposed Nested-BME fusion framework could reconstruct the reliable aerosol diurnal variation trends on the basis of recovering missing data for Himawari-8 AHI AOD datasets, while the AHI official level-2 and level-3 AOD products fail to capture these key trends. Furthermore, the developed Nested-BME AOD fusion framework is also applicable for other geostationary satellites over other regions, which could substantially enhance the availability and value of high temporal resolution AOD products for better scientific applications.

Soil moisture (SM) downscaling aims to solve the coarse resolution problem of passive microwave SM products. On the basis of SMAP SM products and related MODIS products, this study develops a deep residual cycle generative adversarial network (GAN) based heterogeneous spatial-spectral fusion method to downscale SMAP SM from 36km to 9km. On the one hand, the proposed method creatively regards the MODIS products that can reflect the SM state as the spectral features of SM in a broad sense and performs the heterogeneous spatial-spectral fusion between the low-resolution (LR) SM product and high-resolution (HR) MODIS products. On the other hand, considering the spatial correlation of SM, the proposed method utilizes a deep residual cycle generative adversarial network (GAN) to extract and fuse features of heterogeneous images through convolutions. Both qualitative and quantitative evaluation of experimental results shows that the proposed method can generate high accuracy SM products.

Precipitation is a critical process in the terrestrial hydrological circulation, affecting climate change, water resource management, and agricultural production. Satellite-borne observations have prominent advantages in macro and mesoscopic quantitative precipitation estimation. Nevertheless, they are subject to low spatial resolution and inherent biases. Therefore, this study utilizes the surface-surface downscaling network and point-surface fusion network for fine-resolution and high-precision precipitation mapping over China. To deeply explore the complicated relationships between various ancillary factors, ground measurements and satellite precipitation, an attention mechanism based convolutional network (AMCN) is used for spatial downscaling and a geo-intelligent deep belief network (Geoi-DBN) is used for ground-satellite fusion. Experimental results indicate that cascaded networks toward two different objectives are superior to baseline methods, achieving R2 and RMSE of about 0.84 and 27.23 mm/month, respectively. Besides, the assistance of geo-intelligent items and ancillary factors contributes to fusion accuracy. This study provides an effective way for precipitation estimation over China.

In the task of change detection (CD), high-resolution remote sensing images (HRSIs) can provide rich ground object information. However, the interference from noise and complex background information can also bring some challenges to CD. In recent years, deep learning methods represented by convolutional neural networks (CNNs) have achieved good CD results. However, the existing methods have difficulty in detecting the detailed change information of the ground objects effectively. The imbalance of positive and negative samples can also seriously affect the CD results. In this letter, to solve the above problems, we propose a method based on a multiscale fully convolutional neural network (MFCN), which uses multiscale convolution kernels to extract the detailed features of the ground object features. A loss function combining weighted binary cross-entropy (WBCE) loss and dice coefficient loss is also proposed, so that the model can be trained from unbalanced samples. The proposed method was compared with six state-of-the-art CD methods on the DigitalGlobe dataset. The experiments showed that the proposed method can achieve a higherF1-score, and the detection effect of the detailed changes was better than that of the other methods.

Deep learning shows potential superiority in the image fusion field. To solve the problem of the spatial resolution degradation of polarimetric synthetic aperture radar (PolSAR) images caused by system limitation, we propose a fully PolSAR images and DualSAR images fusion network (FDFNet). We use low resolution (LR)-PolSAR super-resolution (LPSR) and modified cross attention mechanism (MCroAM) to perform data fusion on LR-PolSAR and high resolution (HR)-dual-polarization synthetic aperture radar (DualSAR) and design a polarimetric decomposition attention module to introduce the polarimetric parameters of LR-PolSAR images to maintain polarimetric information. Besides, we use the differential information between LR-PolSAR and HR-DualSAR to guide spatial resolution reconstruction. The loss function based on the$L_{1} $norm is used to constrain the network training process. The experimental results show the superiority of the proposed method over the existing methods in visual and quantitative evaluation. In addition, polarimetric decomposition experiments verify the effectiveness of the proposed method to maintain polarimetric information.

Pansharpening aims to sharpen a low spatial resolution (LR) multispectral (MS) image using a high spatial resolution (HR) panchromatic (PAN) image to obtain the HR MS image. Though large numbers of pansharpening methods have been proposed, and many advanced methods have shown high quantitative results, few of them are widely used in real applications. This may be attributed to their instability for different images with different ground surface features, or the complexity to be implemented and the time-consuming process for some state-of-the-art methods. In this letter, we proposed a simple, adjustable, and robust fusion (SARF) method. In the proposed method, a spatial-spectral coenhanced strategy was proposed, and several details of the proposed fusion model were specifically designed for the “simple, adjustable, robust” features. It was tested and verified by four-band and eight-band MS images based on reduced resolution (RR) and full resolution (FR) experiments. The experimental results demonstrated the promising spatial visuality of the proposed method, and the spectral fidelity was more robust than most of component substitution (CS)-based and multiresolution analysis (MRA)-based methods.

For radiometric normalization (RN) of multiple remote sensing (MRS) images within large-scale coverage, the traditional methods ignore the error accumulation and adaptive allocation of cumulative errors caused by the transfer paths in the classical one-after-another pipeline. To this end, a block adjustment-based RN method of MRS images is proposed by considering the global and local radiometric differences (RDs) in this letter. First, the block adjustment-based global RN is conducted to eliminate the global differences of MRS images. This step is independent of transfer paths so that it breaks through the corresponding error accumulation and uneven distribution in the one-after-another pipeline. Second, two local strategies based on block adjustment and edge optimization are further adopted to remove the local residual RDs. In the experiments, it demonstrates that the proposed method can obtain MRS images with a balanced and appealing visual effect, which outperforms the moment matching (MM) method and the popular ENVI software.

It is a challenging task to integrate the spatial, temporal, and spectral information of multi-source remote sensing images, especially in the case of heterogeneous images. To this end, for the first time, this paper proposes a heterogeneous integrated framework based on a novel deep residual cycle generative adversarial network (GAN). The proposed network consists of a forward fusion part and a backward degeneration feedback part. The forward part generates the desired fusion result from the various observations; the backward degeneration feedback part considers the imaging degradation process and regenerates the observations inversely from the fusion result. The heterogeneous integrated fusion framework supported by the proposed network can simultaneously merge the complementary spatial, temporal, and spectral information of multi-source heterogeneous observations to achieve heterogeneous spatio-spectral fusion, spatio-temporal fusion, and heterogeneous spatio-temporal-spectral fusion. Furthermore, the proposed heterogeneous integrated fusion framework can be leveraged to relieve the two bottlenecks of land-cover change and thick cloud cover. Thus, the inapparent and unobserved variation trends of surface features, which are caused by the low-resolution imaging and cloud contamination, can be detected and reconstructed well. Images from many different remote sensing satellites, i.e., Moderate Resolution Imaging Spectroradiometer (MODIS), Landsat 8, Sentinel-1, and Sentinel-2, were utilized in the experiments conducted in this study, and both the qualitative and quantitative evaluations confirmed the effectiveness of the proposed image fusion method.

Fine particulate matter (PM2.5) pollution can cause serious public health problems worldwide. A novel geographically and temporally weighted neural network constrained by global training (GC-GTWNN) is proposed in this article for the remote sensing estimation of surface PM2.5. The global neural network (NN) is trained to learn the overall effect of the influencing variables on surface PM2.5, and the local geographically and temporally weighted NN (GTWNN) addresses the spatiotemporal heterogeneity of the relationship between PM2.5and the influencing variables. Specifically, a global NN is trained with all samples collected from the entire study domain and period. Then, initialized with the global NN, the GTWNN models are built for each location and time and fine-tuned via spatiotemporally localized samples. Meanwhile, the geographically weighted loss function is designed for GTWNN. The proposed GC-GTWNN modeling is tested with a case study across China, which integrates satellite aerosol optical depth, surface PM2.5measurements, and auxiliary variables. Cross-validation results indicate that a remarkable improvement is observed from the global NN to GC-GTWNN modeling ($R^{2}$value increasing from 0.49 to 0.80), and GC-GTWNN modeling also notably outperforms the conventionally popular PM2.5estimation models.

Surface ozone (O3) pollution is a severe environmental problem that endangers human health. It is necessary to obtain high spatio-temporal resolution O3data to provide support for pollution monitoring and prevention. For this purpose, this study makes comprehensive use of remote sensing data, reanalysis data, and ground station observations, and develops an enhanced geographically and temporally weighted neural network (EGTWNN) model to acquire high spatial and temporal resolutions of O3data. The EGTWNN model is nested by two neural networks. The first neural network automatically learns the spatio-temporal proximity relationship to obtain spatio-temporal weights for the samples, and the spatio-temporal weights are then inputted into the second neural network to conduct weighted modeling of the relationship between O3and influencing variables. The contribution of the proposed model is that the first neural network replaces the traditional empirical weighting method, and represents the spatio-temporal proximity relationship more accurately to improve estimation accuracy. Results indicate that the cross-validation R2and RMSE of EGTWNN are 0.81 and 21.24 μg/m3, respectively, which are increased by 0.02 and decreased by ~1 μg/m3relative to those of the traditional empirical weighting method based geographically and temporally weighted neural network model. The results also show that compared with the geographically and temporally weighted regression model, the proposed model achieves superior performance. In addition, the spatio-temporal weights obtained by the first neural network of EGTWNN are highly consistent with those obtained by the traditional empirical weighting method, indicating that the results of neural networks are highly interpretable.

The data fusion technology aims to aggregate the characteristics of different data and to obtain products with multiple data advantages. To solve the problem of reduced resolution of polarimetric synthetic aperture radar (PolSAR) images due to system limitations, we propose a fully PolSAR images and single-polarization synthetic aperture radar (SinSAR) images fusion network to generate high-resolution PolSAR (HR-PolSAR) images. To take advantage of the polarimetric information of the low-resolution PolSAR (LR-PolSAR) images and the spatial information of the high-resolution single-polarization SAR (HR-SinSAR) images, we propose a fusion framework for joint LR-PolSAR images and HR-SinSAR images and design a cross-attention mechanism to extract features from the joint input data. Besides, based on the physical imaging mechanism, we designed the PolSAR polarimetric loss functions for constrained network training. The experimental results confirm the superiority of the fusion network over traditional algorithms. The average peak signal-to-noise ratio (PSNR) is increased by more than 3.6 dB, and the average mean absolute error (MAE) is reduced to less than 0.07. Experiments on polarimetric decomposition and polarimetric signature show that it maintains polarimetric information well.

The normalized difference vegetation index (NDVI) can reflect the plant life cycle of growth and senescence and has become a widely used tool for many applications related to phenology, ecology, and environment. However, unwanted disturbance from cloud, snow, and other atmospheric effects greatly lowers the NDVI quality and hinders its further application. In this article, differing from the previous research attempting to approach the upper NDVI envelope by local adjustment or threshold-related iteration, a novel one-step global variational reconstruction (OGVR) method for NDVI time series is proposed via joint modeling of the gradual vegetation change and negatively biased atmospheric contamination. Two versions of the proposed method are designed for processing NDVI data with or without auxiliary flag information. Long-term and global-scale Advanced Very High Resolution Radiometer (AVHRR) global inventory monitoring and modeling system (GIMMS) data were applied in simulated and real-data experiments to verify the proposed method. The results show that the proposed method can successfully estimate the natural vegetation change from seriously contaminated NDVI time series and can conquer the problem of continuous low-value gaps. The qualitative and quantitative comparisons with five other widely used methods indicate that the proposed method has significant advantages in terms of both effectiveness and stability.

Image super-resolution (SR) is an effective solution to the limitation of the spatial resolution of video satellite images, which is caused by the degradation and compression in the imaging phase. For the processing of satellite videos, the commonly employed deep-learning-based single-frame SR (SFSR) framework has limited performance without using complementary information between the video frames. On the other side, the multiframe SR (MFSR) can utilize temporal subpixel information to super-resolve the high-resolution (HR) imagery. However, although deeper and wider deep learning network provides powerful feature representations for SR methods, it has always been a challenge to accurately reconstruct the boundaries of ground objects in video satellite images. In this article, to address these issues, we propose an edge-guided video SR (EGVSR) framework for video satellite image SR, which couples the MFSR model and the edge-SFSR (E-SFSR) model in a unified network. The EGVSR framework is composed of an MFSR branch and an edge branch. The MFSR branch is used to extract the complementary features from the consecutive video frames. Concurrently, the edge branch acts as an SFSR model to translate the edge maps from the low-resolution modality to the HR one. At the final SR stage, the DBFM is built to focus on the promising inner representations of the features of the two branches and fuse them. Extensive experiments on video satellite imagery show that the proposed EGVSR method can achieve superior performance compared to the representative deep-learning-based SR methods.

Solar-induced chlorophyll fluorescence (SIF) is an intuitive and accurate way to measure vegetation photosynthesis. Orbiting Carbon Observatory-2 (OCO-2)-retrieved SIF has shown great potential in estimating terrestrial gross primary production (GPP), but the discontinuous spatial coverage limits its application. Although some researchers have reconstructed OCO-2 SIF data, few have considered the uneven spatial and temporal distribution of the swath-distributed data, which can induce large uncertainties. In this article, we propose a spatiotemporal constrained light gradient boosting machine model (ST-LGBM) to reconstruct a contiguous OCO-2 SIF product (eight days, 0.05°), considering the data distribution characteristics. Two spatial and temporal constraining factors are introduced to utilize the relationships between the swath-distributed OCO-2 samples, combining the geographical regularity and vegetation phenological characteristics. The results indicate that the ST-LGBM method can improve the reconstruction accuracy in the missing data areas ($R^{2}= 0.79$), with an increment of 0.05 in$R^{2}$. The declined accuracy of the traditional light gradient boosting machine (LightGBM) method in the missing data areas is well alleviated in our results. The real-data comparison with TROPOspheric Monitoring Instrument (TROPOMI) SIF observations also shows that the results of the ST-LGBM method can achieve a much better consistency, in both spatial distribution and temporal variation. The sensitivity analysis also shows that the ST-LGBM can support stable results when using various input combinations or different machine learning models. This approach represents an innovative way to reconstruct a more accurate globally continuous OCO-2 SIF product and also provides references to reconstruct other data with a similar distribution.

High-resolution precipitation products are essential for accurate hydrological and meteorological applications. To improve the spatial resolution and accuracy of monthly satellite precipitation products, we developed a new downscaling-calibration framework with three key steps: 1) coarse-resolution satellite precipitation data are downscaled to 1-km resolution precipitation data with multivariate adaptive regression spline (MARS) model; 2) residual correction is applied to bridge the difference between the satellite precipitation data and downscaled precipitation data; and 3) the geographical differential analysis (GDA) calibration method is implemented to improve accuracy by merging the residual-corrected data with rain gauge data. In this study, geolocation variables (longitude and latitude), a digital elevation model (DEM) data, daytime and nighttime land surface temperatures, and four remote sensing indices were used to downscale monthly Tropical Rainfall Measuring Mission (TRMM) 3B43 precipitation datasets over the Yangtze River Economic Belt. The downscaled results showed that the MARS model can avoid “boxy artifact” and pixel-level anomalies, which are often found in geographically weighted regression (GWR) and random forest (RF) results. According to the validation, the step of residual correction is not necessary. With GDA calibration, the MARS-based estimated results were more accurate than the results of the other methods (i.e., GWR and RF) and original TRMM products. Therefore, the developed MARS-based downscaling-calibration procedure can improve not only the spatial resolution but also the quality of the TRMM 3B43 products.

With the development of video satellites, multimoving object tracking in satellite video is possible and has become a new challenging task. The difficulties are mainly caused by the characteristics of satellite videos: 1) small objects; 2) low contrast between objects and background; and 3) background in a state of continuous motion. These characteristics make it difficult for the advanced multiobject tracking algorithms in the natural video to give full play to their advantages, resulting in vast false alarms, missed objects, ID switches, and low-confidence bounding boxes. To tackle these problems, a novel multimoving object tracking method considering slow features (SFs) and motion features has been proposed in this research, named SF and motion feature-guided multiobject tracking (SFMFMOT), which realizes the continuous tracking of moving vehicles in satellite videos. A nonmaximum suppression (NMS) module guided by bounding box proposals based on SFs is designed to assist the object detection part by utilizing the sensitivity of SF analysis to the changed pixels. While removing a large number of static false alarms and supplementing missed objects, it improves the recall rate by increasing the confidence score of the correctly detected object bounding boxes. In order to improve the tracking performance, a set of optimization strategies based on motion features and time accumulation information are proposed to smooth the trajectory, remove static false alarms, and duplicate bounding boxes. The proposed method is evaluated in three satellite videos and its superiority is demonstrated.

As a new earth observation tool, satellite video has been widely used in remote-sensing field for dynamic analysis. Video super-resolution (VSR) technique has thus attracted increasing attention due to its improvement to spatial resolution of satellite video. However, the difficulty of remote-sensing image alignment and the low efficiency of spatial–temporal information fusion make poor generalization of the conventional VSR methods applied to satellite videos. In this article, a novel fusion strategy of temporal grouping projection and an accurate alignment module are proposed for satellite VSR. First, we propose a deformable convolution alignment module with a multiscale residual block to alleviate the alignment difficulties caused by scarce motion and various scales of moving objects in remote-sensing images. Second, a temporal grouping projection fusion strategy is proposed, which can reduce the complexity of projection and make the spatial features of reference frames play a continuous guiding role in spatial–temporal information fusion. Finally, a temporal attention module is designed to adaptively learn the different contributions of temporal information extracted from each group. Extensive experiments on Jilin-1 satellite video demonstrate that our method is superior to current state-of-the-art VSR methods.

Hyperspectral images (HSIs) have a wealth of applications in many areas, due to their fine spectral discrimination ability. However, in the practical imaging process, HSIs are often degraded by a mixture of various types of noise, for example, Gaussian noise, impulse noise, dead pixels, dead lines, and stripe noise. Low-rank matrix decomposition theory has been widely used in HSI denoising, and has achieved competitive results by modeling the impulse noise, dead pixels, dead lines, and stripe noise as sparse components. However, the existing low-rank-based methods for HSI denoising cannot completely remove stripe noise when the stripe noise is no longer sparse. In this article, we extend the HSI observation model and propose a double low-rank (DLR) matrix decomposition method for HSI denoising and destriping. By simultaneously exploring the low-rank characteristic of the lexicographically ordered noise-free HSI and the low-rank structure of the stripe noise on each band of the HSI, the two low-rank constraints are formulated into one unified framework, to achieve separation of the noise-free HSI, stripe noise, and other mixed noise. The proposed DLR model is then solved by the augmented Lagrange multiplier (ALM) algorithm efficiently. Both simulation and real HSI data experiments were carried out to verify the superiority of the proposed DLR method.

Hyperspectral images (HSIs) are crucial for many research works. Spectral super-resolution (SSR) is a method used to obtain high-spatial-resolution (HR) HSIs from HR multispectral images. Traditional SSR methods include model-driven algorithms and deep learning. By unfolding a variational method, this article proposes an optimization-driven convolutional neural network (CNN) with a deep spatial-spectral prior, resulting in physically interpretable networks. Unlike the fully data-driven CNN, auxiliary spectral response function (SRF) is utilized to guide CNNs to group the bands with spectral relevance. In addition, the channel attention module (CAM) and the reformulated spectral angle mapper loss function are applied to achieve an effective reconstruction model. Finally, experiments on two types of data sets, including natural and remote sensing images, demonstrate the spectral enhancement effect of the proposed method, and also, the classification results on the remote sensing data set verified the validity of the information enhanced by the proposed method.

Image fusion is a powerful means to integrate complementary spatio-temporal-spectral information among multi-source remote sensing images. The existing remote sensing image fusion is mostly limited to the fusion between optical images, and most of them are limited to the fusion between two sensors. Based on this, this paper proposes a heterogeneous spatio-temporal-spectral fusion method based on deep learning. Specifically, it combines the low-spatial-resolution (LR) cloudy image with the high-spatial-resolution (HR) SAR images and the HR cloud-free optical image to remove the clouds and improve the spatial resolution of the LR cloudy image. The SAR image is acquired at the same date as the LR cloudy image, while the HR cloud-free image is acquired at another date. Experiments are performed on the images of Landsat 8, Sentinel-1, and Sentinel-2. The experimental results show that the proposed method can effectively achieve the joint goal of spatial resolution improvement and cloud removal of the Landsat image.

It is very significant to enhance the spatial resolution of hyperspectral images for more accurate image interpretation. In this paper, we propose an innovative fusion method by the enhanced residual dense network to better extract the spatio-spectral features joint with GRU. The enhanced residual dense blocks (ERDB) contains a modified spatial and spectral attention module. All attention coefficients are calculated based on original images. Then, the idea of gate recurrent unit (GRU) is applied to integrate the useful information from results of all ERDBs followed by a convolution for reconstruction. Finally, a skip connection is added to further maintain the spectrum. The effectiveness of this method can be seen from the experimental results of simulation experiments.

Synthetic aperture radar (SAR) images are inherently affected by speckle noise, for which deep learning-based methods have shown good potential. However, the deep learning-based methods proposed until now directly map low-quality images to high-quality images, and they are unable to characterize the priors for all the kinds of speckle images. The variational method is a classic model optimization approach that establishes the relationship between the clean and noisy images from the perspective of a probability distribution. Therefore, in this article, we propose the recursive deep convolutional neural network (CNN) prior model for SAR image despeckling (SAR-RDCP). First, the data-fitting term and regularization term of the SAR variational model are decoupled into two subproblems, i.e., a data-fitting block and a deep CNN prior block. The gradient descent algorithm is then used to solve the data-fitting block, and a predenoising residual channel attention network based on dilated convolution is used for the deep CNN prior block, which combines an end-to-end iterative optimization training. In the experiments undertaken in this study, the proposed model was compared with several state-of-the-art despeckling methods, obtaining better results in both the quantitative and qualitative evaluations.

In this paper, we present a novel hyperspectral image (HSI) denoising method especially for mixed noise removal. The proposed method combines both data-driven with model-driven strategy via a deep spatio-spectral variational structure. The mixed noise estimation and removal are collaboratively derived through fusing the Bayesian spatio-spectral posterior and deep learning model. The framework can both utilize the logicality of traditional model-driven methods, and the high efficiency of data-driven methods for parameters optimizing. Simulated and actual experiments demonstrate that the presented method outperforms other existing methods for HSI mixed noise removal, on both reconstructing effects and time-consuming.

The lunar hyperspectral remote sensing is one of the most important means to understand the physical properties and chemical constituents of lunar surface materials. Moon Mineral Mapper (M3) is currently the only hyperspectral image (HSI) data of Moon. However, due to the limitations of sensor manufacture and the impact of complex extraterrestrial environment, there are serious stripes on the M3 images, which do harm to subsequent identifications and analysis. In this paper, an effective destriping algorithm for lunar HSIs based on the intrinsic characteristics of the stripes on M3 is proposed. Experimental results demonstrate that our method shows an improvement in terms of visual perception and spectral fidelity.

Visible remotely sensed images usually suffer from the haze, which contaminates the surface radiation and degrades the data quality in both spatial and spectral dimensions. This study proposes a spatial-spectral adaptive haze removal method for visible remote sensing images to resolve spatial and spectral problems. Spatial adaptation is considered from global and local aspects. A globally nonuniform atmospheric light model is constructed to depict spatially varied atmospheric light. Moreover, a bright pixel index is built to extract local bright surfaces for transmission correction. Spectral adaptation is performed by exploring the relationships between image gradients and transmissions among bands to estimate spectrally varied transmission. Visible remote sensing images featuring different land covers and haze distributions were collected for synthetic and real experiments. Accordingly, four haze removal methods were selected for comparison. Visually, the results of the proposed method are completely free from haze and colored naturally in all experiments. These outcomes are nearly the same as the ground truth in the synthetic experiments. Quantitatively, the mean-absolute-error, root-mean-square-error, and spectral angle are the smallest, and the coefficient-of-determination (R2) is the largest among the five methods in the synthetic experiments. R2, structural similarity index measure, and the correlation coefficient between the result of the proposed method and the reference image are closest to 1 in the real data experiments. All experimental analyses demonstrate that the proposed method is effective in removing haze and recovering ground information faithfully under different scenes.

In this work, a progressively spatio-temporal patch group learning framework for cloud and shadow removal in Sentinel-2 data is proposed. Through sorting the spatial and corresponding multi-temporal patches with masks as the patch group fashion, a spatiotemporal patch group recovering model is developed using a global-local deep CNN. Finally, all the ergodic patches are weighted aggregated with integrity measure, then updated spatial data and its mask are regenerated through progressive iteration. Two experiments have been performed to demonstrate the effectiveness of the proposed method on Sentienl-2 MSI data, with single/multiple temporal imageries in small and largescale scenarios.

In this paper, a new pansharpening method with residual convolutional neural network (RCNN)) is proposed. The proposed method utilizes a novel end-to-end CNN, which maps the differential information between the high spatial resolution panchromatic image (HR-PAN) and the low spatial resolution multispectral image (LR-MS) to the differential information between the HR-PAN image and the high spatial resolution multispectral image (HR-MS). Unlike the CNN-based pansharpening methods in other literatures, the proposed method makes full use of the spatial information in the HR-PAN image, and simultaneously preserve the spectral information of the MS image. Experimental results at both reduced resolution and full resolution demonstrate the superior performance of the proposed method comparing to state-of-the-art pansharpening methods in both quantitative and visual assessments.

Vegetation water content (VWC) is recognized as an important parameter in vegetation growth study. Recently, the ground-based GNSS-R method is emerging in monitoring VWC owing to its high accuracy. However, the small footprint and sparse distribution hinder its application. Therefore, we propose a method to improve the spatial resolution of GNSS-R VWC products by downscaling with other products highly correlated with VWC, using random forest (RF). Satisfactory downscaling results with cross-validation R values of 0.83 and RMSE of 0.025 were obtained. VWC images with a 500-m spatial resolution were then acquired, which is consistent with the distribution of NDVI and GPP, further indicating the accuracy of the downscaling results.

In order to solve the problem of full-polarimetric SAR image degradation, this paper proposes a full-polarimetric SAR image super-resolution reconstruction method combined with a convolutional neural network and residual compensation. Through the advantages of the deep convolutional neural network for nonlinear model fitting, this paper performs super-resolution reconstruction on low-resolution full-polarimetric SAR images, and then applies residual compensation to network reconstruction results, using low-resolution image information to the network. The super-resolution reconstruction results are corrected to obtain a high-resolution full-polarimetric SAR image. Compared with the traditional full-polarimetric SAR image super-resolution reconstruction method, the proposed method shows excellent results in both visual and quantitative evaluation indicators, especially the reconstruction of detailed information.

Due to the rapid urbanization, the impact of urban spatial form on surface temperature cannot be ignored and is becoming increasingly significant. This study investigates the impact of the urban spatial form which includes building form and urban land surface moisture (ULSM) on the land surface temperature (LST) of communities in Wuhan, China. The LST was retrieved in the summer and winter by Landsat 8 OLI/TIRS data. Thirty typical communities in the three-ring area were selected to represent general residential area in Wuhan. By extracting the building height (BH), building density (BD), floor area ratio (FAR), sky view factor (SVF), frontal area index (FAI) and ULSM of each community, the result shows that BH, BD, FAR, SVF and ULSM are critical variables in lowering LST in both winter and summer.

Deriving accurate snow depth is of great importance since snow cover is an informative indicator of climate change. The objective of this study is to develop a snow-depth retrieval algorithm based on a deep learning approach by fusing passive microwave remote sensing brightness temperature, station observations and GNSS-R snow-depth product to improve the accuracy of snow-depth retrieval. The results show that DBN performs the best compared with three alternative algorithms.

In this study, the performance of the MODIS C6 multiangle implementation of atmospheric correction (MAIAC) 1-km aerosol optical depth (AOD) products in China during 2008-2016 are validated using 17 AERONET sites. The results show that the correlation coefficient (R) for MAIAC in C6 is 0.942 and the RMSE is 0.2. Meanwhile, the fraction above the EE is shown as 49.85%, indicating that the MAIAC retrievals will integrally overestimate AOD values in China. Especially, significant overestimation of MAIAC is found at vegetation sites with high elevation and in areas round water. The scale of high-resolution AOD distribution is fine while an obvious boundary caused by aerosol models is also observed.

This study attempts to develop a novel solution for the estimation of regional surface soil moisture (SSM) using a machine learning model trained on in situ measurement target data. Specifically, the generalized regression neural network (GRNN) is employed to establish the relationship between in-situ measurements from Sparse Network Stations (SNSs) in the continental U.S. and passive microwave observations from the Soil Moisture Active Passive (SMAP) satellite for April 2015 to March 2018. More importantly, to address the scale mismatch issue resulting from the small spatial support of in situ measurements, we turn to the extended triple collocation technique whereby individual SNSs' reliability at the SMAP coarse footprint is determined before fed into GRNN. The cross-validation results show that the GRNN model trained on reliable SNSs obtains a fairly good performance, with out-of-sample cross-validated R and unbiased RMSE values of 0.92 and 0.043 cm3cm-3, respectively. Moreover, the comparison in space shows that the spatial patterns of GRNN retrievals is the most consistent with in situ measurements than both the SMAPL3SMP and the ERA-Interim SSM data. Furthermore, the GRNN-estimated SSM time series over stations agrees much better with in-situ measurements than the official SMAP passive SSM product. All these results indicate that the statistical GRNN modeling has shown great potential in estimating reliable regional SSM climate records using in-situ measurements as training references.

Enhancing the spatial resolution of hyperspectral (HS) images by fusing with higher spatial resolution multispectral (MS) data is of significance for applications. However, due to the narrow bandwidth, HS images (HSIs) are vulnerable to various types of noise, such as Gaussian noise and stripes, which can severely affect the fusion performance. This paper focuses on antinoise HS and MS image fusion to enhance the spatial details and suppress the noise. By analysis of the intrinsic structure and noise properties, we formulate this problem as the minimization of an objective function. Under the optimization framework, small multilinear ranks in tensor are first used to identify the intrinsic structures of the clean HSI part. Then, considering the high spectral correlation, it is assumed that any bands can be represented by the combination of certain adjacent bands. The difference between one band and its corresponding combination can be used to preserve the spatio-spectral consistency and characterize the distribution of sparse noise (such as stripe noise), based on the variational properties along two directions. The alternating direction method of multipliers (ADMM) is applied to solve and accelerate the model optimization. Experiments with both simulated- and real-data demonstrate the effectiveness of the proposed model and its robustness to the noise, in terms of both qualitative and quantitative perspectives.

Despeckling is a fundamental preprocessing step for applications using polarimetric synthetic aperture radar data in most cases. In this paper, a guided filter with nonlinear weight kernels and adaptive filtering windows is presented for PolSAR image despeckling, in which the guidance image is constructed by a weighted average using the statistical traits of the speckled image. The output result is then estimated by another weighted average, with the aid of the fully polarimetric information from both the guidance image and the speckled image. In the experimental part, the filtering results obtained with both simulated and real PolSAR images reveal the positive performance of the proposed method in both reducing speckle and retaining details, when compared with some of the state-of-the-art algorithms. Furthermore, the relatively low computational complexity is another strength of the proposed method.

The optical remote sensing images not only have to make a fundamental tradeoff between the spatial and spectral resolutions, but also are inevitable to be polluted by the clouds; however, the existing pansharpening methods mainly focus on the resolution enhancement of the optical remote sensing images without cloud contamination. How to fuse the cloud-contaminated images to achieve the joint resolution enhancement and cloud removal is a promising and challenging work. In this paper, a pansharpening method for the challenging cloud-contaminated very high-resolution remote sensing images is proposed. Furthermore, the cloud-contaminated conditions for the practical observations with all the thick clouds, the thin clouds, the haze, and the cloud shadows are comprehensively considered. In the proposed methods, a two-step fusion framework based on multisource and multitemporal observations is presented: 1) the thin clouds, the haze, and the light cloud shadows are proposed to be first jointly removed and 2) a variational-based integrated fusion model is then proposed to achieve the joint resolution enhancement and missing information reconstruction for the thick clouds and dark cloud shadows. Through the proposed fusion method, a promising cloud-free fused image with both high spatial and high spectral resolutions can be obtained. To comprehensively test and verify the proposed method, the experiments were implemented based on both the cloud-free and cloud-contaminated images, and a number of different remote sensing satellites including the IKONOS, the QuickBird, the Jilin (JL)-1, and the Deimos-2 images were utilized. The experimental results confirm the effectiveness of the proposed method.

In the field of spatial–spectral fusion, the variational model-based methods and the deep learning (DL)-based methods are state-of-the-art approaches. This paper presents a fusion method that combines the deep neural network with a variational model for the most common case of spatial–spectral fusion: panchromatic (PAN)/multispectral (MS) fusion. Specifically, a deep residual convolutional neural network (CNN) is first trained to learn the gradient features of the high spatial resolution multispectral image (HR-MS). The image observation variational models are then formulated to describe the relationships of the ideal fused image, the observed low spatial resolution multispectral image (LR-MS) image, and the gradient priors learned before. Then, fusion result can then be obtained by solving the fusion variational model. Both quantitative and visual assessments on high-quality images from various sources demonstrate that the proposed fusion method is superior to all the mainstream algorithms included in the comparison, in terms of overall fusion accuracy.

Hyperspectral image (HSI) denoising is a crucial preprocessing procedure to improve the performance of the subsequent HSI interpretation and applications. In this paper, a novel deep learning-based method for this task is proposed, by learning a nonlinear end-to-end mapping between the noisy and clean HSIs with a combined spatial-spectral deep convolutional neural network (HSID-CNN). Both the spatial and spectral information are simultaneously assigned to the proposed network. In addition, multiscale feature extraction and multilevel feature representation are, respectively, employed to capture both the multiscale spatial-spectral feature and fuse different feature representations for the final restoration. The simulated and real-data experiments demonstrate that the proposed HSID-CNN outperforms many of the mainstream methods in both the quantitative evaluation indexes, visual effects, and HSI classification accuracy.

The existence of hybrid noise in hyperspectral images (HSIs) severely degrades the data quality, reduces the interpretation accuracy of HSIs, and restricts the subsequent HSI applications. In this paper, the spatial-spectral gradient network (SSGN) is presented for mixed noise removal in HSIs. The proposed method employs a spatial-spectral gradient learning strategy, in consideration of the unique spatial structure directionality of sparse noise and spectral differences with additional complementary information for effectively extracting intrinsic and deep features of HSIs. Based on a fully cascaded multiscale convolutional network, SSGN can simultaneously deal with different types of noise in different HSIs or spectra by the use of the same model. The simulated and real-data experiments undertaken in this study confirmed that the proposed SSGN outperforms at mixed noise removal compared with the other state-of-the-art HSI denoising algorithms, in evaluation indices, visual assessments, and time consumption.

In this paper, a novel spatiotemporal fusion model based on deep learning is proposed, which handles the huge spatial resolution gap and the nonlinear mapping between the high spatial resolution (HSR) image and the corresponding high temporal resolution (HTR) image at the same imaging time. Considering the huge spatial resolution gap, a two-layer fusion strategy is adopted. In each layer, the convolutional neural network (CNN) model is employed to exploit the non-linear mapping between the HSR and HTR image and reconstruct the high-spatial and high-temporal (HSHT) resolution images. In the experiment, Landsat data is the representation of the high spatial resolution images, MODIS data is used as the corresponding low spatial resolution images. The experimental results on two different datasets clearly illustrate the superiority of the proposed model.

Satellite remote sensing is a promising approach for the estimation of ground-level PM2.5. In this paper, a deep learning framework for satellite-based PM2.5 estimation is presented. Taking advantage of multi-layer learning and layer-by-layer pre-training, deep learning has the great potential to mine nonlinear relationship between PM2.5 and satellite observations. Firstly, the presented deep learning framework can be employed to estimate ground PM2.5 using satellite-derived aerosol optical depth (AOD). Secondly, the AOD products are retrieved from satellite top-of-atmosphere (TOA) reflectance. The deep learning framework can further be adopted to estimate ground PM2.5 directly from satellite TOA reflectance. The presented framework was tested for AOD-based PM2.5 estimation in China and reflectance-based PM2.5 estimation across Wuhan Metropolitan Area, respectively. The results show that the deep learning framework achieves an outstanding performance for both AOD-based and reflectance-based PM2.5 estimation. This study provides an effective way for the satellite-based estimation of PM2.5.

In this paper, a unified spatial-temporal-spectral framework of missing information reconstruction in remote sensing images is proposed. Based on an end-to-end non-linear mapping structure, the proposed method employs a unified deep convolutional neural network combined with joint spatial-temporal-spectral supplementary information. It should be noted that the proposed model can use multi-source data (spatial, spectral, and temporal) as the input of the unified framework. The results of real-data experiments demonstrate that the proposed model exhibits high effectiveness in missing information reconstruction tasks like dead lines in Aqua MODIS band 6, Landsat ETM+ SLC-off and thick cloud removal.

Striping effects are a common phenomenon in remote-sensing imaging systems, and they can exhibit considerable differences between different sensors. Such artifacts can greatly degrade the quality of the measured data and further limit the subsequent applications in higher level remote-sensing products. Although a lot of destriping methods have been proposed to date, a few of them are robust to different types of stripes. In this paper, we conduct a thorough feature analysis of stripe noise from a novel perspective. With regard to the problem of striping diversity and complexity, we propose a universal destriping framework. In the proposed destriping procedure, a 1-D variational method is first designed and utilized to estimate the statistical feature-based guidance. The guidance information is then incorporated into 2-D optimization to control the image estimation for a reliable and clean output. The iteratively reweighted least-squares method and alternating direction method of multipliers are exploited in the proposed approach to solve the minimization problems. Experiments under various cases of simulated and real stripes confirm the effectiveness and robustness of the proposed model in terms of the qualitative and quantitative comparisons with other approaches.

The land surface reflectance data are indispensable to generate many other land products. Global land surface reflectance data have been routinely produced from remote sensing sensors aboard different satellite platforms. However, the original data, especially the daily data, suffer from a large number of spatial gaps, which result from atmospheric contamination and instrument deficiencies. This seriously limits their further applications. Many composite products with less spatial gaps have been generated to solve the above problem, but they easily sacrifice their temporal resolutions of original data. Even worse, they cannot be directly implemented in realistic applications because of the noise and composite seams. This paper proposes a temporal-spatial reconstruction method (TSRM) to generate daily gapless and seamless land surface reflectance data. The TSRM integrates both temporal and spatial information for recovering different land cover types using three processing steps. First, spatial gaps are coarsely filled with multiyear weighted average (Step1). After that, all the gaps that are not filled in the first step are interpolated by using harmonic analysis of time series with true value constraint (Step2). Finally, the reconstructed results in the last step are seamlessly processed using the Poisson image editing method, and the seamless daily reflectance data set is generated (Step3). The Moderate Resolution Imaging Spectroradiometer reflectance data set (MOD09GA and MYD09GA) on two testing areas is selected to verify the performance of the proposed TSRM. Experimental results show that the TSRM has good performance with regard to maintaining the temporal and spatial integrity of the daily land surface reflectance data. Results on different testing sites also demonstrate that the TSRM preserves spectral integrity with clear seasonal trends for each spectral band.

Registration is a classical problem in the application of remote sensing images. The existing methods prefer to fit the relationship between target and source images with the same model on the whole. In fact, the geometrical relationship between two images is not always consistent, especially for the wide-field-viewed images of GaoFen-1 (GF-1) launched by the China Aerospace Science and Technology Corporation (CASC) in April 2013. Generally, The existing methods didn't take the local deformation into consideration. Towards this end, we solve the problem with three stages in this paper. Firstly, the coarse registration obtains the integral perspective transformation model of images. Secondly, the fine registration partitions image into many blocks and improves the relationship of every block with the inverse distance weighting (IDW) function. Finally, the coordinate transformation and resampling are the final step. Compared to other methods, the experiments demonstrate that the proposed algorithm is capable of generating satisfied results which are robust against deformation at local area.

The presence of shadows in urban aerial images can degrade the quality of the images and cause problems in image interpretation. In this paper, a novel shadow removal method based on the clustering correction of illumination field is proposed. We construct a spatially adaptive weighted total variation model to achieve the optimized illumination field. The land surface types are considered to correct the distribution of the illumination field based on the clustering moment matching method. Then the shadows are recovered in the illumination field with the texture well preserved and the shadow boundary smoothed. Experiments and comparisons are presented to verify the effectiveness of the proposed method.

Soil Moisture Active Passive (SMAP) satellite mission, launched on Jan. 31, 2015, can provide a 9 km soil moisture product globally by merging passive and active observations. However, the radar sensor of SMAP was failed since Jul. 7, 2015 and SMAP 9 km SM product (SMAP_AP) is only available for 85 days. To ameliorate the vacancy of SMAP_AP, a spatio-temporal fusion model STNLFFM combined with the SMAP 36 km soil moisture product (SMAP_P) is utilized to generate 9 km soil moisture SM product (SMAP_F). Generation of SMAP_F was implemented over one year from Apr. 13, 2015 to Apr. 12, 2016 in the paper. Then SMAP_F was evaluated by SMAP_AP and in-situ soil moisture from international soil moisture network. It is revealed that the STNLFFM can be taken as an effective method for SMAP 9 km soil moisture generation.

Pan-sharpening is a fundamental and significant task in the field of remote sensed imagery fusion, which demands fusion of panchromatic and multi-spectral images with the rich information accurately preserved in both spatial and spectral domains. In this paper, to overcome the drawbacks of traditional pan-sharpening methodologies, we employed the advanced concept of deep learning to propose a Multi-Scale-and-Depth Convolutional Neural Network (MSDCNN) as an end-to-end pan-sharpening model. By the results of a large number of quantitative and visual assessments, the qualities of images fused by the proposed network have been confirmed superior to compared state-of-the-art methods.

Missing data reconstruction (e.g., dead pixel repair and cloud removing) in remote sensing images is a very important problem for the subsequent image analysis. It is well-known that missing data reconstruction is an ill-posed problem. In remote sensing images, there is a strong correlation in spectral frequencies or in temporal frames, and also there are a lot of self-similarity patterns in spatial domain. We can make use of these properties to derive low rank matrices according to their spectral, temporal and spatial dimensions. In this paper, we propose a tensor completion model based on these low rank matrices to deal with missing data reconstruction problem. We also present a weighting method for spectral, temporal and spatial dimensions and for their distribution of singular values. Our experimental results demonstrate that the weighting method can recover remote images very well. In particular, we show the effectiveness of the proposed method for both simulated and real data sets, and the performance of the proposed in terms of visual and quantitative measures is better than those of the other testing methods.

Remote sensing images often suffer from cloud cover. Cloud removal is required in many applications of remote sensing images. Multitemporal-based methods are popular and effective to cope with thick clouds. This paper contributes to a summarization and experimental comparison of the existing multitemporal-based methods. Furthermore, we propose a spatiotemporal-fusion with poisson-adjustment method to fuse multi-sensor and multitemporal images for cloud removal. The experimental results show that the proposed method is able to obtain more accurate results than the current multitemporal-based methods, especially when the multi-temporal images suffer from significant changes.

Remote sensing images are widely used in various fields. However, they are usually degraded by haze. In this paper, a spatial-spectral adaptive haze removal method for remote sensing images is proposed. The proposed method is based on haze imaging model and the dark channel prior. Our method is able to diminish the phenomenon of color distortion and haze removed not completely in different bands. The experimental results verify that the proposed method can remove the haze completely and yield visually haze free images, even the haze distribution is uneven.

In the field of multispectral (MS) and panchromatic image fusion (pansharpening), the impressive effectiveness of deep neural networks has recently been employed to overcome the drawbacks of the traditional linear models and boost the fusion accuracy. However, the existing methods are mainly based on simple and flat networks with relatively shallow architectures, which severely limits their performance. In this letter, the concept of residual learning is introduced to form a very deep convolutional neural network to make the full use of the high nonlinearity of the deep learning models. Through both quantitative and visual assessments on a large number of high-quality MS images from various sources, it is confirmed that the proposed model is superior to all the mainstream algorithms included in the comparison, and achieves the highest spatial-spectral unified accuracy.

Filling missing information or removing special objects is often required in the applications of high spatial resolution images. A novel single-image reconstruction method is presented in this letter to solve this task, without the use of any complementary data. First, the spatial pattern of the image is obtained by the statistics of similar patch offsets in the known regions, which provide reliable information for reconstructing the image. The missing regions are then filled by combining a series of shifted pixels via global optimization. The proposed method was tested on a cloudy image for cloud removal and on a public image for military object concealment. The experimental results show that the proposed method can produce visually convincing and coherent reconstructed images, and the accuracy of the reconstruction is better than the existing noncomplementation methods.

The tradeoff in remote sensing instruments that balances the spatial resolution and temporal frequency limits our capacity to monitor spatial and temporal dynamics effectively. The spatiotemporal data fusion technique is considered as a cost-effective way to obtain remote sensing data with both high spatial resolution and high temporal frequency, by blending observations from multiple sensors with different advantages or characteristics. In this paper, we develop the spatial and temporal nonlocal filter-based fusion model (STNLFFM) to enhance the prediction capacity and accuracy, especially for complex changed landscapes. The STNLFFM method provides a new transformation relationship between the fine-resolution reflectance images acquired from the same sensor at different dates with the help of coarse-resolution reflectance data, and makes full use of the high degree of spatiotemporal redundancy in the remote sensing image sequence to produce the final prediction. The proposed method was tested over both the Coleambally Irrigation Area study site and the Lower Gwydir Catchment study site. The results show that the proposed method can provide a more accurate and robust prediction, especially for heterogeneous landscapes and temporally dynamic areas.

Mapping and monitoring urban area extents is a very relevant task for many applications related to risks, economic activities, population mapping and the interaction between people and the natural environment. This task can be accomplished using VHR and HR optical data. The main drawback of using the optical data is the irregular presence of the clouds, making mapping by these sensors completely useless in many cases. Even when the clouds are small and irregular, the extracted urban area extents may be erroneous and/or incomplete, and many uncertainties show up in thematic maps. In this paper, we prove that the joint use of algorithms for urban extent extraction, cloud detection and masking, and image reconstruction provide a clear processing chain able to overcome this issue and to increasing the reliability of the extracted urban area extents.

Soil Moisture and Ocean Salinity (SMOS) is the first L-band passive microwave mission dedicated to soil moisture (SM) monitoring but has coarse resolution. SMOS SM was downscaled by Back propagation neural network (BPNN) and MODIS LST and EVI, and was evaluated by intensive 56 stations in-situ obtained from the mesoscale Tibetan Plateau Soil Moisture/Temperature Monitoring Network (SMTMN) during 2010-2012 over Naqu. It showed that downscaled rmse and bias were close for BPNN and regression, downscaled r was significantly improved by BPNN. In terms of r, it was improved greatly by downscaling comparing with original SMOS evaluations. Besides, descending of SMOS (D_SMOS) got better downscaled SM than ascending of SMOS and evaluation of downscaled D_SMOS (r=0.849, rmse=0.056m3/m3and bias=0.016 m3/m3) was nice in SMTMN scale. The fine SM BPNN downscaled by SMOS can be used for regional or local studies.

The cloud and cloud shadow are difficult to capture accurately in optical imagery because of insufficient spectral information. In this paper, an automatic multiple features combined (MFC) method is proposed for cloud and cloud shadow detection in GF-1 WFV imagery which includes three visible and one near-infrared bands. The local optimization strategy with guided filtering, and the proposed object-based filter combining geometry and texture features are used in the proposed method to refine cloud detection results and exclude non-cloud bright objects. The experimental results indicate that MFC performs well under different conditions.

China is currently suffering from a heavy PM2.5pollution. To estimate ground-level PM2.5from satellite-observed aerosol optical depth (AOD), many regional studies have been undertaken, but a few at national scale in China. Moreover, due to the wide spatial range and complex meteorological fields, the previous models' estimation accuracy of PM2.5still has space to improve. In this paper, using the newly available national PM2.5measurements, we develop a generalized regression neural network (GRNN) model to better describe the PM2.5-AOD relationship in China. Besides, a direct average of satellite-derived PM2.5can only reflect the level of PM2.5pollution on some certain days when AOD data is available. To address this issue, a pixel-based merging scheme is proposed. The results suggest that the cross validation R and RMSE are 0.811 and 20.11 μg / m3, respectively. The results also show that our study can provide useful information for global monitoring of PM2.5pollution in China.

Impervious surface has been recognized as a key indicator in assessing urban environments. Referring to the previous research, linear spectral mixture analysis has been widely used to extract impervious surface. In this paper, a material-based endmember selection is applied to support linear spectral unmixing, which suggests that the impervious surface should be classified by their essential impervious materials. Taking Landsat images of Wuhan city for experiment, the results show that the classification accuracy is around 95%. Besides, the extracted impervious surface distribution is highly similar to the ground truth and its variation possesses a similar tendency with Urban Heat Island Intensity.

In this paper, we introduced a deep learning model: Convolutional neural network(CNN) from the field of natural image classification and restoration, to solve general quality improving tasks for remote sensing images, including super-resolution, denoising and haze removal. To take advantage of the content similarity among aerial images and the learning ability of deep learning models, we proposed the idea of training CNN on datasets collected from aerial images with specific degenerating factors, then apply the model to matched tasks. Experiments showed that our network achieved superior performance in quantified results, and visually reconstructed a satisfying majority of missing details from low-quality observations.

Rapidly and accurately obtaining collapsed buildings information of the earthquake-stricken areas can help to effectively guide the implementation of the emergency rescue and can reduce disaster losses and casualties. This work is focused on rapid building earthquake damage information detection in urban areas using a single post-earthquake PolSAR data. In this paper, the methods of polarization orientation angle (POA) compensation and Wishart supervised classification are employed to extract the collapsed buildings and undamaged buildings. In addition, the two parameters of the normalized difference of the dihedral component (NDDC) and the HH-HV Correlation Coefficient (ρHHHV) are proposed to improve the extraction accuracy of the collapsed buildings and undamaged buildings. The building damage assessment is carried out at the city block scale according to the building collapse rate.

Complementary information from multi-sensor can be integrated to effectively solve many problems in remote sensing application. Synthetic Aperture Radar (SAR) imaging can be a feasible alternative to traditional optical remote sensing techniques because it is independent of solar illumination and weather conditions. This paper proposes a novel fusion framework combining IHS transform with sparse representation theory to fuse multispectral and SAR images. In addition, the simultaneous orthogonal matching pursuit (SOMP) technique is introduced to guarantee the efficiency. Experiments on various datasets have verified the effectiveness of proposed method.

Due to the limitation of hyperspectral sensors and optical imaging systems, there are several irreconcilable conflicts between high spatial resolution and high spectral resolution of hyperspectral images (HSIs). Therefore, HSI super-resolution (SR) is regarded as an important preprocessing task for subsequent applications. In this letter, we use sparse representation to analyze the spectral and spatial feature of HSIs. Considering the sparse characteristic of spectral unmixing and high pattern repeatability of spatial-spectral blocks, we proposed a novel HSI SR framework utilizing spectral mixture analysis and spatial-spectral group sparsity. By simultaneously combining the sparsity and the nonlocal self-similarity of the images in the spatial and spectral domains, the method not only maintains the spectral consistency but also produces plenty of image details. Experiments on three hyperspectral data sets confirm that the proposed method is robust to noise and achieves better results than traditional methods.

In the commonly employed regularization models of image restoration and super-resolution (SR), the norm determination is often challenging. This paper proposes a method to adaptively determine the optimal norms for both fidelity term and regularization term in the (SR) restoration model. Inspired by a generalized likelihood ratio test, a piecewise function is proposed to solve the norm of the fidelity term. This function can find the stable norm value in a certain number of iterations, regardless of whether the noise type is Gaussian, impulse, or mixed. For the regularization norm, the main advantage of the proposed method is that it is locally adaptive. Specifically, it assigns different norms for different pixel locations, according to the local activity measured by a structure tensor metric. The proposed method was tested using different types of images. The experimental results and error analyses verify the efficacy of the method.

In this paper, we present a spatial spectral hyperspectral image (HSI) mixed-noise removal method named total variation (TV)-regularized low-rank matrix factorization (LRTV). In general, HSIs are not only assumed to lie in a low-rank subspace from the spectral perspective but also assumed to be piecewise smooth in the spatial dimension. The proposed method integrates the nuclear norm, TV regularization, and L1-norm together in a unified framework. The nuclear norm is used to exploit the spectral low-rank property, and the TV regularization is adopted to explore the spatial piecewise smooth structure of the HSI. At the same time, the sparse noise, which includes stripes, impulse noise, and dead pixels, is detected by the L1-norm regularization. To tradeoff the nuclear norm and TV regularization and to further remove the Gaussian noise of the HSI, we also restrict the rank of the clean image to be no larger than the number of endmembers. A number of experiments were conducted in both simulated and real data conditions to illustrate the performance of the proposed LRTV method for HSI restoration.

Hyperspectral image (HSI) denoising is a crucial preprocessing task that is used to improve the quality of images for object detection, classification, and other subsequent applications. It has been reported that noise can be effectively removed using the sparsity in the nonnoise part of the image. With the appreciable redundancy and correlation in HSIs, the denoising performance can be greatly improved if this redundancy and correlation is utilized efficiently in the denoising process. Inspired by this observation, a noise reduction method based on joint spectral-spatial distributed sparse representation is proposed for HSIs, which exploits the intraband structure and the interband correlation in the process of joint sparse representation and joint dictionary learning. In joint spectral-spatial sparse coding, the interband correlation is exploited to capture the similar structure and maintain the spectral continuity. The intraband structure is utilized to adaptively code the spatial structure differences of the different bands. Furthermore, using a joint dictionary learning algorithm, we obtain a dictionary that simultaneously describes the content of the different bands. Experiments on both synthetic and real hyperspectral data show that the proposed method can obtain better results than the other classic methods.

Remote sensing images are often contaminated by varying degrees of stripes, which severely affects the visual quality and subsequent application of the data. Unlike with conventional methods, we achieve the destriping by separating the stripe component based on a full analysis of the various stripe properties. Under an optimization framework, an ℓ0-norm-based regularization is used to characterize the global sparse distribution of the stripes. In addition, difference-based constraints are adopted to describe the local smoothness and discontinuity in the along-stripe and across-stripe directions, respectively. The alternating direction method of multipliers is applied to solve and accelerate the model optimization. Experiments with both simulated and real data demonstrate the effectiveness of the proposed model, in terms of both qualitative and quantitative perspectives.

In this paper, we focus on the despeckling of synthetic aperture radar (SAR) images by variational methods which introduce nonlocal regularization functionals. To achieve this goal, two models are investigated from different aspects. The first model is derived for the logarithmically transformed (homomorphic) domain of the SAR data, and the other is derived for the original (nonhomomorphic) domain. The statistical properties of the speckle and the log-transformed speckle are analyzed, and the similarity measurements between pixels in the homomorphic domain and nonhomomorphic domain are then derived for constructing the corresponding nonlocal regularization functionals. Meanwhile, in the proposed models, we develop a strategy to adaptively choose the regularization parameters based on both the local heterogeneity information and the noise level of the images, aiming at getting a better balance between the goodness of fit of the original data and the amount of smoothing. A quasi-Newton iteration method is employed to quickly minimize the proposed adaptive nonlocal functionals. Experiments conducted on both simulated images and real SAR images confirm the good performances of the proposed methods, both in reducing speckle and preserving image quality.

Remote sensing satellite sensors feature a tradeoff between the spatial, temporal, and spectral resolutions. In this paper, we propose an integrated framework for the spatio-temporal-spectral fusion of remote sensing images. There are two main advantages of the proposed integrated fusion framework: it can accomplish different kinds of fusion tasks, such as multiview spatial fusion, spatio-spectral fusion, and spatio-temporal fusion, based on a single unified model, and it can achieve the integrated fusion of multisource observations to obtain high spatio-temporal-spectral resolution images, without limitations on the number of remote sensing sensors. The proposed integrated fusion framework was comprehensively tested and verified in a variety of image fusion experiments. In the experiments, a number of different remote sensing satellites were utilized, including IKONOS, the Enhanced Thematic Mapper Plus (ETM+), the Moderate Resolution Imaging Spectroradiometer (MODIS), the Hyperspectral Digital Imagery Collection Experiment (HYDICE), and Système Pour l' Observation de la Terre-5 (SPOT-5). The experimental results confirm the effectiveness of the proposed method.

Most of the empirical topographic correction methods are based on the universal assumptions of the relationship between radiations and solar incident angles. The correction accuracy is hardly to be accessed quantitatively. This paper introduces a land cover adaptive C (LCAC) method for topographic correction, and verifies its advantage quantitatively. Experiments on synthetic and real remote sensing data are performed. The synthetic data derives from the SMARTS2 model, which simulates the surface reflectance and the atmospheric conditions. The global land cover map produced by the National Geomatics Center of China is taken as the auxiliary data. The LCAC outperforms the traditional C method in experiments on both synthetic and real remote sensing data by visual and quantitative assessments.

In this paper, we propose a method to refine the similarity measurement for constructing the diffusion coefficients in each iteration in the PolSAR anisotropic diffusion filter, by considering both the information of the original speckled image and the filtered image of last iteration. In addition, to alleviate the over-smoothing problem the anisotropic diffusion based methods often encountered, an adaptive noise level based data-fitting term is also added into the diffusion equation. Experiments on both simulated and real PolSAR images revealed the good performances of the proposed method in both suppressing speckle and retaining image quality.

In this paper, a unified framework for the spatio-temporal-spectral fusion of remote sensing images is proposed. The relationships between the observed images and the desired image are first established based on general image observation models. Maximum a posteriori (MAP) theory is then employed to formulate the unified fusion framework. The proposed method is able to fuse images from an arbitrary number of optical sensors with different spatial, temporal, and spectral resolutions. The experimental results verify the effectiveness of the proposed method.

As a significant digital representation of terrain surface, varieties of DEM products have been available to the public. The most widely used global DEM products are SRTM and ASTER GDEM. Given the comparable horizontal resolution and vertical error, accuracy validation and comparison have been of interest since the release, however, usually on a wide range. In this paper, we presented the results of accuracy assessment for ASTER GDEM v2 and SRTM v4.1 in Yulong Mountain, Yunnan province, China. Topographic map was chosen as the benchmark. The results and discussions were centered on the relationship between error distribution in elevation and mountainous hypsography based on data causes. The results revealed their levels of reliability for applied glaciology and hydrology in the typical snow mountain area.

The digital elevation model (DEM) is a significant digital representation of a terrain surface. Although a variety of DEM products are available, they often suffer from problems varying in spatial coverage, data resolution, and accuracy. However, the multi-source DEMs often contain supplementary information, which makes it possible to produce a higher-quality DEM through blending the multi-scale data. Inspired by super-resolution (SR) methods, we propose a regularized framework for the production of high-resolution (HR) DEM data with extended coverage. To deal with the registration error and the horizontal displacement among multi-scale measurements, robust data fidelity with weighted norm is employed to measure the conformance of the reconstructed HR data to the observed data. Furthermore, a slope-based Markov random field (MRF) regularization is used as the spatial regularization. The proposed method can simultaneously handle complex terrain features, noises, and data voids. Using the proposed method, we can reconstruct a seamless DEM data with the highest resolution among the input data, and an extensive spatial coverage. The experiments confirmed the effectiveness of the proposed method under different cases.

The Moderate Resolution Imaging Spectroradiometer (MODIS) land surface temperature (LST) product can offer accurate LST with high temporal and spatial resolution, but the quality is often degraded by cloud. To improve the usability of the MODIS LST, this letter proposes a reconstruction method based on multitemporal data. First, a multitemporal classification is employed to distinguish the different land surface types. The invalid LST values can then be predicted using a robust regression with the multitemporal information from the other LSTs. Finally, postprocessing is proposed to eliminate outliers. Simulated and actual experiments show that the method can accurately reconstruct the missing values.

Global or regional environmental change is of wide concern. Extensive studies have indicated that long-term vegetation cover change is one of the most important factors reflecting environmental change, and normalized difference vegetation index (NDVI) time-series data sets have been widely used in vegetation dynamic change monitoring. However, the significant residual effects and noise levels impede the application of NDVI time-series data in environmental change research. This study develops a novel and robust filter method, i.e., the moving weighted harmonic analysis (MWHA) method, which incorporates a moving support domain to assign the weights for all the points, making the determination of the frequency number much easier. Additionally, a four-step process flow is designed to make the data approach the upper NDVI envelope, so that the actual change in the vegetation can be detected. A total of 487 test pixels selected from SPOT VEGETATION 10-day MVC NDVI time-series data from January 1999 to December 2001 were used to illustrate the effectiveness of the new method by comparing the MWHA results with the results of another four existing methods. Finally, the long-term SPOT VEGETATION 10-day maximum-value compositing (MVC) NDVI time series for China from April 1998 to May 2014 was reconstructed by the use of the proposed method, and a test region in China was utilized to validate the effectiveness of the proposed MWHA method. All the results indicate that the reconstructed high-quality NDVI time series fits the actual growth profile of the vegetation and is suitable for use in further remote sensing applications.

In this paper, a new denoising algorithm is proposed for hyperspectral image data cubes. With the strong correlations of the image bands, the low-rank structure of the hyperspectral image is explored by lexicographically ordering the 3-D data cube into 2-D matrix. Based on this property, the traditional principal component analysis (PCA) denoising model is established. For hyperspectral images (HSIs), the noise intensity in different bands is different. Therefore, a noise-adjusted iterative randomized singular value decomposition (NAIRSVD) algorithm is proposed to solve this PCA model. Combined with adaptive noise estimation and upper bound rank estimation, the proposed NAIRSVD algorithm is free from manual parameter determination. Several experiments were conducted to illustrate the performance of the proposed algorithm.

In the past decade, the synthesis-based methods have drawn people's attention more and more in the sparse representation community. The synthesis model decomposes the data into a combination of a few atoms of the overcomplete dictionary. However, the dual analysis-based methods have not been studied deeply. The analysis model results in a sparse outcome by multiplying an analysis dictionary. This work proposes an analysis-based recovery of the missing information of remote sensing data, by extracting supplementary information from another term of data at a different period. Our method is verified by the qualitative and quantitative assessments in the experiments.

In this paper, we present two spatially adaptive nonlocal total variation methods for the speckle filtering of synthetic aperture radar (SAR) images. One is for the despeckling of SAR intensity images, and the other one is for full polarimetric SAR (PolSAR) images. Experiments were conducted on two simulated images and an airborne PolSAR images to illustrate the filtering performances, and the results show that the proposed methods effectively reduces speckle, retains edges and targets, and preserves the polarimetric scattering mechanisms.

The Earth Observing System of the National Aeronautics and Space Administration pays a great deal of attention to the long-term global observations of the land surface, biosphere, atmosphere, and oceans. Specifically, the Moderate Resolution Imaging Spectroradiometer (MODIS) instrument on board the twin satellites Terra and Aqua plays a vital role in the mission. Unfortunately, around 70% of the detectors in Aqua MODIS band 6 have malfunctioned or failed. Consequently, many of the derivatives related to band 6, such as the normalized difference snow index, suffer from the adverse impact of dead or noisy pixels. In this letter, the missing or noisy information in Aqua MODIS band 6 is successfully completed using a robust multilinear regression (M-estimator) based on the spectral relations between working detectors in band 6 and all the other spectra. The experimental results indicate that the proposed robust M-estimator multiregression (RMEMR) algorithm can effectively complete the large areas of missing information while retaining the edges and textures, compared to the state-of-the-art methods.

Land surfaces are commonly obstructed by haze in remote sensing images, which reduces the available land cover information. Haze detection is therefore important for locating, avoiding, or restoring hazy regions. In this letter, a principal component (PC)-based haze masking (PCHM) method is developed for the masking of haze in visible remote sensing images covering land surfaces at middle latitudes. Owing to the evidence of haze in the second PC, the PCHM method results in accurate haze masks. The complete procedure comprises two steps: haze construction and spatial optimization. The validity of the PCHM method is demonstrated through its application to several hazy visible images clipped from Landsat Enhanced Thematic Mapper Plus scenes. The quantitative assessments verify the superiority of the proposed method over the haze optimized transformation method for the production of binary haze masks. In addition, the resulting haze masks are compared with a MODIS cloud product, which further proves the necessity and validity of the proposed method.

Filling dead pixels or removing uninteresting objects is often desired in the applications of remotely sensed images. In this paper, an effective image inpainting technology is presented to solve this task, based on multichannel nonlocal total variation. The proposed approach takes advantage of a nonlocal method, which has a superior performance in dealing with textured images and reconstructing large-scale areas. Furthermore, it makes use of the multichannel data of remotely sensed images to achieve spectral coherence for the reconstruction result. To optimize the proposed variation model, a Bregmanized-operator-splitting algorithm is employed. The proposed inpainting algorithm was tested on simulated and real images. The experimental results verify the efficacy of this algorithm.

With regard to quantitative remote sensing products in the visible and infrared ranges, thick clouds and accompanying shadows are an inevitable source of noise. Due to the absence of adequate supporting information from the data themselves, it is a formidable challenge to accurately restore the surficial information underlying large-scale clouds. In this paper, dictionary learning is expanded into the multitemporal recovery of quantitative data contaminated by thick clouds and shadows. This paper proposes two multitemporal dictionary learning algorithms, expanding on their KSVD and Bayesian counterparts. In order to make better use of the temporal correlations, the expanded KSVD algorithm seeks an optimized temporal path, and the expanded Bayesian method adaptively weights the temporal correlations. In the experiments, the proposed algorithms are applied to a reflectance product and a land surface temperature product, and the respective advantages of the two algorithms are investigated. The results show that, from both the qualitative visual effect and the quantitative objective evaluation, the proposed methods are effective.

Shadows are evident in most aerial images with high resolutions, particularly in urban scenes, and their existence obstructs the image interpretation and the following application, such as classification and target detection. Most current shadow removal methods were proposed for natural images, whereas shadows in remote sensing images show distinct characteristics. We have therefore analyzed the characteristics of shadows in aerial images, and in this paper, we propose a new shadow removal method for aerial images, using nonlocal (NL) operators. In the proposed method, the soft shadow is introduced to replace the traditional binary hard shadow. NL operators are used to regularize the shadow scale and the updated shadow-free image. Furthermore, a spatially adaptive NL regularization is introduced to handle compound shadows. The combination of the soft shadow and NL operators yields satisfying shadow-free results, preserving textures and holding regular color. Different types of shadowed aerial images are employed to verify the proposed method, and the results are compared with two other methods. The experimental results confirm the validity of the proposed method and the advantage of the soft-shadow approach.

Because of malfunction or noise in 15 out of the 20 detectors, band 6 (1.628-1.652 μm) of the Moderate Resolution Imaging Spectroradiometer (MODIS) sensor aboard the Aqua satellite contains large areas of dead pixel stripes. Therefore, the corresponding high-level products of MODIS are corrupted by this periodic phenomenon. This paper proposes an improved Bayesian dictionary learning algorithm based on the burgeoning compressed sensing theory to solve this problem. Compared with other state-of-the-art methods, the proposed method can adaptively exploit the spectral relations of band 6 and other spectra. The performance of the proposed method is demonstrated by experiments on both simulated Terra and real Aqua images.

In this paper, we propose a hyperspectral image denoising algorithm with a Spatial-spectral view fusion strategy. The idea is to denoise a noisy hyperspectral 3-D cube using the hyperspectral total variation algorithm, but applied to both the spatial and spectral views. A metric Q-weighted fusion algorithm is then adopted to merge the denoising results of the two views together, so that the denoising result is improved. A number of experiments illustrate that the proposed approach can produce a better denoising result than both the individual spatial and spectral view denoising results.

Hyperspectral images (HSIs) are often degraded by a mixture of various kinds of noise in the acquisition process, which can include Gaussian noise, impulse noise, dead lines, stripes, and so on. This paper introduces a new HSI restoration method based on low-rank matrix recovery (LRMR), which can simultaneously remove the Gaussian noise, impulse noise, dead lines, and stripes. By lexicographically ordering a patch of the HSI into a 2-D matrix, the low-rank property of the hyperspectral imagery is explored, which suggests that a clean HSI patch can be regarded as a low-rank matrix. We then formulate the HSI restoration problem into an LRMR framework. To further remove the mixed noise, the “Go Decomposition” algorithm is applied to solve the LRMR problem. Several experiments were conducted in both simulated and real data conditions to verify the performance of the proposed LRMR-based HSI restoration method.

In this paper, a local scale measure is presented for the detection of homogeneous regions in an image. Then, based on region homogeneity, an adaptive nonlocal means filter (ANLMF) is proposed. In this method, the neighborhood window size for denoising varies adaptively, according to the local scale measure. Experiments show that the proposed filter (ANLMF) is better than the state-of-the-art nonlocal means filter (NLMF).

Hyperspectral images (HSIs) provide abundant information to solve various kinds of problems like object identification and classification. However, HSIs often inevitably suffer many factors from various resources [1], such as imperfect imaging optics, sensor noise, and atmospheric effects, which degrade the acquired image quality [2]. Thus, HSI image super resolution reconstruction, used to achieve sub-pixel mapping, is an active research topic due to its effectiveness in improving the resolution of hyperspectral image. In the paper, a HSI super-resolution method, in which the different dictionaries are learnt for different bands and sparse structure from wavelength range with high correlation is regarded with similar sparse coefficients , is proposed to obtain the high-resolution image.

Total variation is used as a popular and effective image prior model in the regularization-based image processing fields. However, as the total variation model favors a piecewise constant solution, the processing result under high noise intensity in the flat regions of the image is often poor, and some pseudoedges are produced. In this paper, we develop a regional spatially adaptive total variation model. Initially, the spatial information is extracted based on each pixel, and then two filtering processes are added to suppress the effect of pseudoedges. In addition, the spatial information weight is constructed and classified with k-means clustering, and the regularization strength in each region is controlled by the clustering center value. The experimental results, on both simulated and real datasets, show that the proposed approach can effectively reduce the pseudoedges of the total variation regularization in the flat regions, and maintain the partial smoothness of the high-resolution image. More importantly, compared with the traditional pixel-based spatial information adaptive approach, the proposed region-based spatial information adaptive total variation model can better avoid the effect of noise on the spatial information extraction, and maintains robustness with changes in the noise intensity in the super-resolution process.

The sub-pixel mapping technique, which can provide a fine-resolution map of class labels, has attracted more and more attention in recent years. Generally speaking, there are two kinds of methods used to realize the sub-pixel labeling. The first kind are image reconstruction based methods, which first improve the spatial resolution of an image by the super-resolution technique, and then perform a hard classification on the super-resolved image. The second kind are pixel unmixing based methods, where the sub-pixel mapping is implemented based on the results of image unmixing. In this paper, we present a sparse representation method and a back-propagation (BP) neural network method for image reconstruction based and pixel unmixing based mapping, respectively. The advantages and disadvantages of both kinds of methods are analyzed and discussed.

High-resolution multispectral (HRM) images are widely used in many remote sensing applications. Using the pan-sharpening technique, a low-resolution multispectral (LRM) image and a high-resolution panchromatic (HRP) image can be fused to an HRM image. This letter proposes a new compressed sensing (CS)-based pan-sharpening method which views the image observation model as a measurement process in the CS theory and constructs a joint dictionary from LRM and HRP images in which the HRM is sparse. The novel joint dictionary makes the method practical in fusing real remote sensing images, and a tradeoff parameter is added in the image observation model to improve the results. The proposed algorithm is tested on simulated and real IKONOS images, and it results in improved image quality compared to other well-known methods in terms of both objective measurements and visual evaluation.

This letter proposes a blind image restoration method for the deblurring of remote sensing images. A simple but robust identification method of point spread function (PSF) support is proposed, and a joint estimation method is presented to simultaneously solve the PSF coefficients and restoration image. To narrow the solution space for the best possible definition, the Huber-Markov (Huber-Markov random field) prior model is employed to regularize the two series of unknowns. Experiments were performed to demonstrate the effectiveness of the proposed approach.

Total variation (TV) has been used as a popular and effective image prior model in regularization-based image processing fields, such as denoising, deblurring, super-resolution (SR), and others, because of its ability to preserve edges. However, as the TV model favors a piecewise constant solution, the processing results in the flat regions of the image being poor, and it cannot automatically balance the processing strength between different spatial property regions in the image. In this paper, we propose a spatially weighted TV image SR algorithm, in which the spatial information distributed in different image regions is added to constrain the SR process. A newly proposed and effective spatial information indicator called difference curvature is used to identify the spatial property of each pixel, and a weighted parameter determined by the difference curvature information is added to constrain the regularization strength of the TV regularization at each pixel. Meanwhile, a majorization-minimization algorithm is used to optimize the proposed spatially weighted TV SR model. Finally, a significant amount of simulated and real data experimental results show that the proposed spatially weighted TV SR algorithm not only efficiently reduces the “artifacts” produced with a TV model in fat regions of the image, but also preserves the edge information, and the reconstruction results are less sensitive to the regularization parameters than the TV model, because of the consideration of the spatial information constraint.

Perceptually inspired color correction methods are characterized by human visual system properties. In this paper, we propose a perceptually inspired variational method for uneven intensity correction of remote sensing images. The proposed method shares the same intrinsic scheme as the Retinex theory, but the reflectance in this method is solved directly within the limited dynamic range and is supposed to comply with the gray world assumption. Considering the smoothness of illumination and the complexity of reflectance, the proposed method integrates L2 norm and total variation prior to inflict varying constraints to different components and regions. The minimum of this variational model is calculated using the steepest descent approach. Experimental results are presented to validate the effectiveness of the proposed method.

The amount of noise included in a hyperspectral image limits its application and has a negative impact on hyperspectral image classification, unmixing, target detection, and so on. In hyperspectral images, because the noise intensity in different bands is different, to better suppress the noise in the high-noise-intensity bands and preserve the detailed information in the low-noise-intensity bands, the denoising strength should be adaptively adjusted with the noise intensity in the different bands. Meanwhile, in the same band, there exist different spatial property regions, such as homogeneous regions and edge or texture regions; to better reduce the noise in the homogeneous regions and preserve the edge and texture information, the denoising strength applied to pixels in different spatial property regions should also be different. Therefore, in this paper, we propose a hyperspectral image denoising algorithm employing a spectral-spatial adaptive total variation (TV) model, in which the spectral noise differences and spatial information differences are both considered in the process of noise reduction. To reduce the computational load in the denoising process, the split Bregman iteration algorithm is employed to optimize the spectral-spatial hyperspectral TV model and accelerate the speed of hyperspectral image denoising. A number of experiments illustrate that the proposed approach can satisfactorily realize the spectral-spatial adaptive mechanism in the denoising process, and superior denoising results are produced.

In this paper, an adjustable model-based image fusion method for multispectral (MS) and panchromatic (PAN) images is developed. The relationships of the desired high spatial resolution (HR) MS images to the observed low-spatial-resolution MS images and HR PAN image are formulated with image observation models. The maximum a posteriori framework is employed to describe the inverse problem of image fusion. By choosing particular probability density functions, the fused HR MS images are solved using a gradient descent algorithm. In particular, two functions are defined to adaptively determine most regularization parameters using the partially fused results at each iteration, retaining one parameter to adjust the tradeoff between the enhancement of spatial information and the maintenance of spectral information. The proposed method has been tested using QuickBird and IKONOS images and compared to several known fusion methods using quantitative evaluation indices. The experimental results verify the efficacy of this method.

Image super-resolution (SR) reconstruction has been a hot research topic in recent years. This technique allows the recovery of a high-resolution (HR) image from several low-resolution (LR) images that are noisy, blurred and down-sampled. Among the available reconstruction frameworks, the maximum a posteriori (MAP) model is widely used. In this model, the regularization parameter plays an important role. If the parameter is too small, the noise will not be effectively restrained; conversely, the reconstruction result will become blurry. Therefore, how to adaptively select the optimal regularization parameter has been widely discussed. In this paper, we propose an adaptive MAP reconstruction method based upon a U-curve. To determine the regularization parameter, a U-curve function is first constructed using the data fidelity term and prior term, and then the left maximum curvature point of the curve is regarded as the optimal parameter. The proposed algorithm is tested on both simulated and actual data. Experimental results show the effectiveness and robustness of this method, both in its visual effects and in quantitative terms.

Image super-resolution reconstruction (SRR) refers to a signal processing approach which produces a high-resolution (HR) image from observed multiple low-resolution (LR) images. In this paper, we propose a joint MAP formulation combining image registration, blur identification, and SRR together to deal with heavy aliasing in the observed LR images. A cyclic coordinate decent optimization procedure is used to solve the formulation, in which the registration parameters, blurring information, and HR image are found in an alternate manner given the others, respectively. The proposed algorithm has been tested on a synthetic image sequence. The experiment results and error analyses verify the efficacy of this algorithm.

In this paper, a new sub-pixel mapping method inspired by the clonal selection algorithm (CSA) in artificial immune systems (AIS) is proposed, namely clonal selection subpixel mapping (CSSM). In CSSM, the sub-pixel mapping problem becomes one of assigning land cover classes to the sub-pixels while maximizing the spatial dependence by clonal selection algorithm. CSSM inherits the biologic properties of human immune systems, i.e. clone, mutation, memory, to build a memory-cell population with a diverse set of local optimal solutions. Based on the memory-cell population, CSSM outputs the value of the memory cell and find the optimal sub-pixel mapping result. The proposed method was tested using the synthetic and degraded real imagery. Experimental results demonstrate that the proposed approach outperform traditional sub-pixel mapping algorithms, and hence provide an effective option for sub-pixel mapping of remote sensing imagery.

In this paper, we propose a super-resolution image reconstruction algorithm to moderate-resolution imaging spectroradiometer (MODIS) remote sensing images. This algorithm consists of two parts: registration and reconstruction. In the registration part, a truncated quadratic cost function is used to exclude the outlier pixels, which strongly deviate from the registration model. Accurate photometric and geometric registration parameters can be obtained simultaneously. In the reconstruction part, the L1 norm data fidelity term is chosen to reduce the effects of inevitable registration error, and a Huber prior is used as regularization to preserve sharp edges in the reconstructed image. In this process, the outliers are excluded again to enhance the robustness of the algorithm. The proposed algorithm has been tested using real MODIS band-4 images, which were captured in different dates. The experimental results and comparative analyses verify the effectiveness of this algorithm.

Remotely sensed images often suffer from the common problems of stripe noise and random dead pixels. The techniques to recover a good image from the contaminated one are called image destriping (for stripes) and image inpainting (for dead pixels). This paper presents a maximumaposteriori(MAP)-based algorithm for both destriping and inpainting problems. The main advantage of this algorithm is that it can constrain the solution space according toaprioriknowledge during the destriping and inpainting processes. In the MAP framework, the likelihood probability density function (PDF) is constructed based on a linear image observation model, and a robust Huber-Markov model is used as the prior PDF. The gradient descent optimization method is employed to produce the desired image. The proposed algorithm has been tested using moderate resolution imaging spectrometer images for destriping and China-Brazil Earth Resource Satellite and QuickBird images for simulated inpainting. The experiment results and quantitative analyses verify the efficacy of this algorithm.

Super resolution image reconstruction allows the recovery of a high-resolution (HR) image from several low-resolution images that are noisy, blurred, and down sampled. In this paper, we present a joint formulation for a complex super-resolution problem in which the scenes contain multiple independently moving objects. This formulation is built upon the maximum a posteriori (MAP) framework, which judiciously combines motion estimation, segmentation, and super resolution together. A cyclic coordinate descent optimization procedure is used to solve the MAP formulation, in which the motion fields, segmentation fields, and HR images are found in an alternate manner given the two others, respectively. Specifically, the gradient-based methods are employed to solve the HR image and motion fields, and an iterated conditional mode optimization method to obtain the segmentation fields. The proposed algorithm has been tested using a synthetic image sequence, the "Mobile and Calendar" sequence, and the original "Motorcycle and Car" sequence. The experiment results and error analyses verify the efficacy of this algorithm.

Super-resolution image reconstruction has been one of the most active research areas in recent years. In this paper, a new super-resolution algorithm is proposed to the problem of obtaining a high-resolution image from several low-resolution images that have been sub-sampled and displaced by different amounts of sub-pixel shifts. The algorithm is based on the MAP framework, solving the optimization by proposed iteration steps. At each iteration step, the regularization parameter is updated using the partially reconstructed image solved at the last step. The proposed algorithm is tested on synthetic images, and the reconstructed images are evaluated by the PSNR method. The results indicate that the proposed algorithm has considerable effectiveness in terms of both objective measurements and visual evaluation.