Hyperspectral images (HSIs) often suffer from various noises, such as Gaussian noise, stripe noise, impulse noise, and deadlines due to the influence of sensors and external environments. These noises significantly degrade the quality of HSI and hinder subsequent applications. While most current transformer-based methods can effectively remove certain types of noise, they struggle with wide stripe noise. In addition, transformers are typically applied within local windows due to the limitation of computational complexity. Although windowshifting operations enhance the interaction between windows to a certain extent, this interaction remains insufficient for comprehensive global modeling. In light of these limitations, we propose a Bidirectional-aware network combining Transformer and Mamba (BTMnet), which consists of Bidirectional Long-Short Distance Attention (BLSDA) and Channel-Split Mamba (CSM). To better remove wide stripe noise, BLSDA is designed with two rectangular windows adapted to wide stripes in both vertical and horizontal directions, utilizing transformers to compute attention relationships within windows and across different windows. To further integrate global information and enhance the interaction of features between adjacent windows, CSM extracts global features by scanning in four directions across different feature channels. In BLSDA, we applied bidirectional windows in vertical and horizontal directions, and in CSM, we conducted bidirectional scanning in vertical and horizontal directions. The combination of these techniques allows for the simultaneous extraction of bidirectional features from HSI. By evaluating the metrics and visualization, the experimental results on simulated and real experiments prove that our method can achieve better results.

As typical structural noise, stripes commonly occur in remote sensing images captured by linear array sensors, which seriously lowers the image quality and hinders the downstream applications. Differing from the conventional methods, this article explores the ability of the ordering domain in separable stripe representation and provides a new perspective for destriping. To enhance the model flexibility to adapt to different types of stripes, the additive and multiplicative stripe components are fully considered and creatively incorporated into the observation model. Based on the additive-multiplicative observation model and the ordering domain transformation, we propose a novel destriping model, called ordering domain destriping (ODD), which constrains the additive and multiplicative stripe components in line with their statistical distribution characteristics. The results obtained on simulated and real striped images show that the proposed method can successfully estimate the latent clean images without losing stripe-like object details in challenging test scenarios, such as mixed additive-multiplicative stripes, wide stripes, and deadlines. The qualitative and quantitative comparisons with six other destriping methods verify the effectiveness and stability of the proposed model.

Deep learning (DL)-based methods have achieved tremendous success in remote sensing (RS) change detection (CD). However, most DL-based methods heavily rely on high-quality labeled samples, where noisy labels are inevitably introduced during the annotation process, especially at edge regions. Under the supervision of such noisy labels, the performance of RS CD models will deteriorate significantly. To break the limitation, a novel uncertainty-aware noisy label learning network is proposed for RS CD, termed UNLLNet. Specifically, a joint detection strategy based on uncertainty analysis is proposed, which leverages samples characterized by low uncertainty and high probability to detect and correct potential noisy labels, thereby mitigating the risk of erroneous correction. Given that noisy labels are more likely to be introduced at edge regions, an edge-guided inter-level difference refinement module (EIDRM) is designed to effectively calibrate the edge structure of change regions, thereby facilitating better identification of noisy labels at edge regions. Moreover, a noise-robust loss function with adaptive hard sample enhancement is introduced, assigning loss weights to each sample based on their uncertainty to further enhance the model robustness over noisy labels. Experimental results on LEVIR-CD, CDD, and CLCD datasets validate the effectiveness and advantages of UNLLNet compared to other state-of-the-art RS CD methods.

Hyperspectral images (HSIs) with rich spectral information have been widely used in surface classification, object detection, and other real application problems. However, due to the hardware limitations, the low spatial resolution HSIs hinder the exploration of their application potential. Deep learning-based methods are currently the most common solutions for single HSI super-resolution (HSI SR) tasks. However, such methods often overlook the degradation principle from high-resolution HSI to low-resolution HSI. In this article, we propose a CNN-transformer embedded unfolding network (CTUNet), in which an unfolding framework with an effective spatial-spectral prior network is designed for HSI SR by incorporating the degradation principle of HSIs. Specifically, a maximum posterior-based energy model is employed, enabling alternate optimization to seek the optimal solution in an iterative mechanism. To effectively utilize the structure prior of HSI, multiscale self-calibrated convolution (MSSC) and edge-guided transformer module are combined to learn latent spatial-spectral priors. Additionally, hidden feature connections between adjacent iterations enhance the representation of the image features. Extensive experiments conducted on three available HSI datasets demonstrate that our method outperforms several state-of-the-art HSI SR methods. The code will be available athttps://github.com/YoeTon/CTUNet.

With the rapid development of remote sensing imaging technology, change detection (CD) with heterogeneous images has become a hot topic in the community. Given the distinct physical properties of heterogeneous images, it is difficult for direct extraction of change information. Some models that transform heterogeneous images into a mutual feature domain can be beneficial. However, the transformation may be influenced by the changed areas that are not the discrepancy of the domains, which further decreases the accuracy of CD. To solve the problem, we propose a reciprocal transformation-based joint deep and broad learning (RTDBL) model for CD with heterogeneous images. In the RTDBL model, in order to rapidly extract features, a deep feature extraction (DFE) module is designed without the need for training. In addition, for directly highlighting change information and eliminating the influence of changed areas, a reciprocal heterogeneous nodes transformation (RHNT) module is designed to construct regression functions for achieving reciprocal transformation. Subsequently, to achieve cross-spatial information interaction, a structural nodes extraction (SNE) module is proposed for obtaining structural nodes. For effectively utilizing aforementioned information and exploring the connections of heterogeneous nodes, a heterogeneous dual broad learning (HDBL) is developed to predict the change map. According to the best of our knowledge, this is the first attempt that joints deep learning and broad learning for CD with heterogeneous images. The efficacy of the proposed RTDBL is demonstrated through experimental analysis on four widely used datasets, in comparison with ten state-of-the-art models.

Deep learning-based hyperspectral image (HSI) change detection (CD) approaches have a strong ability to leverage spectral-spatial-temporal information through automatic feature extraction, and currently dominate in the research field. However, their efficiency and universality are limited by the dependency on labeled data. Although the newly applied untrained networks can avoid the need for labeled data, their feature volatility from the simple difference space easily leads to inaccurate CD results. Inspired by the interesting finding that salient changes appear as bright "stripes" in a new feature space, we propose a novel unsupervised CD method that represents and models changes in stripes for HSIs (named as StripeCD), which integrates optimization modeling into an untrained network. The StripeCD method constructs a new feature space that represents change features in stripes and models them in a novel optimization manner. It consists of three main parts: 1) dual-branch untrained convolutional network, which is utilized to extract deep difference features from bitemporal HSIs and combined with a two-stage channel selection strategy to emphasize the important channels that contribute to CD. 2) multiscale forward-backward segmentation framework, which is proposed for salient change representation. It transforms deep difference features into a new feature space by exploiting the structure information of ground objects and associates salient changes with the stripe-shaped change component. 3) stripe-shaped change extraction model, which characterizes the global sparsity and local discontinuity of salient changes. It explores the intrinsic properties of deep difference features and constructs model-based constraints to better identify changed regions in a controllable manner. The proposed StripeCD method outperformed the state-of-the-art unsupervised CD approaches on three widely used datasets. In addition, the proposed StripeCD method indicates the potential for further investigation of untrained networks in facilitating reliable CD.

As the foundation of image interpretation, semantic segmentation is an active topic in the field of remote sensing. Facing the complex combination of multiscale objects existing in remote sensing images (RSIs), the exploration and modeling of contextual information have become the key to accurately identifying the objects at different scales. Although several methods have been proposed in the past decade, insufficient context modeling of global or local information, which easily results in the fragmentation of large-scale objects, the ignorance of small-scale objects, and blurred boundaries. To address the above issues, we propose a contextual representation enhancement network (CRENet) to strengthen the global context (GC) and local context (LC) modeling in high-level features. The core components of the CRENet are the local feature alignment enhancement module (LFAEM) and the superpixel affinity loss (SAL). The LFAEM aligns and enhances the LC in low-level features by constructing contextual contrast through multilayer cascaded deformable convolution and is then supplemented with high-level features to refine the segmentation map. The SAL assists the network to accurately capture the GC by supervising semantic information and relationship learned from superpixels. The proposed method is plug-and-play and can be embedded in any FCN-based network. Experiments on two popular RSI datasets demonstrate the effectiveness of our proposed network with competitive performance in qualitative and quantitative aspects.

Due to the limitations of sensor hardware devices, the hyperspectral image (HSI) often suffers from various types of noise, such as Gaussian noise, impulse noise, stripe noise, and deadlines, which can significantly degrade their quality. Although many data-driven methods have been proposed to deal with complex noise, few of them consider the structural characteristics of noise. This not only leads to a lack of interpretability but also results in poor performance when dealing with structural noise in practical applications. To address this issue, this article proposes KPInet, a convolutional neural network (CNN) driven by the structural knowledge of noise for HSI denoising. First and foremost, the knowledge optimization-driven module (KODM) utilizes the deep unrolling method to unfold a total variation (TV) algorithm that considers the structural characteristics of noise. This approach improves the network’s interpretability and results in better performance on structural noise, while maintaining the effect of removing Gaussian noise. Second, the statistical feature injection module (SFIM) extracts more features by utilizing spectral gradients, medians, and means of the HSI. Third, the multiscale degradation guidance module (MDGM) utilizes a dual-stream decoder with a low-resolution upsampling guidance branch to better distinguish the real structure and noise structure in the HSI. Experimental results on simulated and real datasets indicate that the approach achieves favorable denoising performance, as evidenced by both quantitative evaluation metrics and visual results. Furthermore, it also demonstrates the robustness and generalization capacity of the proposed KPInet.

Cloud-free image reconstruction is of great significance for improving the quality of optical satellite images that are vulnerable to bad weather. When cloud cover makes it impossible to obtain information under the cloud, auxiliary data is indispensable to guide the reconstruction of the cloud-contaminated area. Additionally, the areas that require continuous observation are mostly regions with complex features, which puts higher demands on the restoration of texture, color, and other details in data reconstruction. In this paper, we propose a Transformer-based generative adversarial network for cloud-free multispectral image reconstruction via multi-sensor data fusion in satellite images (TransGAN-CFR). Synthetic Aperture Radar (SAR) images that are not affected by clouds are used as auxiliary data and paired with cloudy optical images into the GAN generator. To take advantage of the deep-shallow features and global-local geographical proximity in remote sensing images, the proposed generator employs a hierarchical Encoder-Decoder structure, in which the Transformer blocks adopt a non-overlapping window multi-head self-attention (WMSA) mechanism and a modified feed-forward network though depth-wise convolutions and the gating mechanism. Besides, we introduce a Triplet loss function specifically designed for cloud removal tasks to provide the generated cloud-less image with greater proximity to the ground truth. Compared with seven state-of-the-art deep learning-based cloud removal models, our network can yield more natural cloud-free images with better visual performance and more accurate results in quantitative evaluation on the SEN12MS-CR dataset.

Due to the impact of climate change, the frequency of flood events has increased in recent years, which puts forward an urgent need for timely and accurate flood mapping for emergency response. As the synthetic aperture radar (SAR) enables all-time monitoring regardless of bad weather conditions, it fits far better than passive optical sensors to delineate submerged areas during flood events. However, the universal, rapid, and accurate detection of flood extent remains a challenge. Drawing inspiration from the analysis of time-series variation in representative ground objects caused by flood events, as observed in a dual-polarized SAR time series over a hydrological year, we construct a novel window-based variation model. This model can be used to capture both long-term trends and short-term fluctuations of flood features across different polarization modes. Subsequently, we introduce an unsupervised flood mapping framework that integrates spatiotemporal flood features extracted by fuzzy-based methods. Given the distinct backscatter value of short vegetation, a flooded short vegetation activation model is designed and performed to enhance flood mapping accuracy in complex regions. The proposed method, tested on the 2020 East Dongting Lake flood in China, surpasses three unsupervised flood mapping methods and two deep learning methods in terms of quantitative evaluation and visual performance. The uncertainty of our proposed framework is tested through parameter sensitivity analyses, comparisons with flood mapping results from other sensor images, and extensive experiments on floods at different locations and times, thereby demonstrating its effectiveness, stability, and universality.

Deep learning approaches have been extensively applied to change detection in hyperspectral images (HSIs). However, the majority of them encounter scarcity of training samples or rely on complex structures and learning strategies. Although untrained change detection models have been proved to be effective in relief above problems, they were constructed using regular convolutions and treated spatial locations and channels equally, which are insufficient to extract discriminative features and lead to limited accuracy. Given this, a novel untrained framework using randomly initialized models with spatial-channel augmentation (RICD) is proposed for HSI change detection in this paper. It consists of two major modules: 1) an enhanced feature extraction network using successive dilation-deformable feature extraction blocks, which can extract multiscale spatial-spectral features over unfixed sampling locations. It enlarges the field of view of convolutions and takes arbitrary neighborhood into consideration, which helps to increase the discriminativeness of the extracted features; 2) a change sensitive feature augmentation and comparison module integrating feature selection and spatial-channel augmentation strategies, which can exploit spatial context and channel importance. It magnifies difference between changed pixels and unchanged ones and emphasizes contribution of significant channels of the selected change sensitive features. Despite that convolution operations are included in RICD, all the weights are untrained and fixed once they are randomly initialized, indicating that the RICD can work in an unsupervised manner. Its performance is tested over three widely used hyperspectral datasets. Quantitative and qualitative comparisons with several state-of-the-art unsupervised methods reveal the effectiveness of the RICD method.

Deep learning (DL)-based methods incorporating convolutional neural networks (CNNs) and recurrent neural networks (RNNs) have been successfully applied to change detection in satellite time series. However, traditional RNNs assume identical time interval between image sequences, which hardly meets the real case in satellite time series because of clouds and shadows. In this letter, a novel irregular-time-distanced recurrent CNN (IRCNN) is proposed. IRCNN consists of three sub-networks: a multi-branch Siamese CNN, irregular-time-distanced long short-term memory (ILSTM), and fully connected (FC) layers. Superior to the existing methods, IRCNN can account for temporal dependency among time series with irregular time distances. It is end-to-end trainable with samples generated using an automatic annotation generation method, which is proposed based on the prior knowledge from the continuous change detection and classification (CCDC) approach. IRCNN was tested over five study areas using Landsat time series collected between 2013 and 2020. Experiments demonstrate the effectiveness and stability of the proposed network with better performance, compared to the state-of-the-art approaches in terms of both qualitative and quantitative aspects. Our IRCNN Pytorch code and data are available athttps://github.com/thebinyang/IRCNN.

The normalized difference vegetation index (NDVI) can reflect the plant life cycle of growth and senescence and has become a widely used tool for many applications related to phenology, ecology, and environment. However, unwanted disturbance from cloud, snow, and other atmospheric effects greatly lowers the NDVI quality and hinders its further application. In this article, differing from the previous research attempting to approach the upper NDVI envelope by local adjustment or threshold-related iteration, a novel one-step global variational reconstruction (OGVR) method for NDVI time series is proposed via joint modeling of the gradual vegetation change and negatively biased atmospheric contamination. Two versions of the proposed method are designed for processing NDVI data with or without auxiliary flag information. Long-term and global-scale Advanced Very High Resolution Radiometer (AVHRR) global inventory monitoring and modeling system (GIMMS) data were applied in simulated and real-data experiments to verify the proposed method. The results show that the proposed method can successfully estimate the natural vegetation change from seriously contaminated NDVI time series and can conquer the problem of continuous low-value gaps. The qualitative and quantitative comparisons with five other widely used methods indicate that the proposed method has significant advantages in terms of both effectiveness and stability.

Change detection in time series is among the most critical problems in earth monitoring and attracts extensive attention in the remote sensing community. The task is, however, nontrivial because available images are irregularly collected due to interference by clouds and shadows. Traditional recurrent neural networks neglect such information and thus degrade the possibility of distinguishing pseudochanges (caused by intra-annual and inter-annual dynamics) and real changes. To this end, we proposed an unsupervised time-distance-guided convolutional recurrent network (UTRNet) for change detection in irregularly collected images. UTRNet is distinctive because the influence of pseudochanges can be suppressed by adopting a novel time-distance-guided long short-term memory (TLSTM) unit, in which input and forget gates are modified to adapt to irregular time distances. To the best of our knowledge, this is the first time that the influence of pseudochanges can be suppressed using irregular time distances. Moreover, to make UTRNet more applicable, a weighted prechange detection model is proposed to extract the most reliable training samples automatically. In the training process, unlike existing approaches that only care about changed and unchanged sample imbalance, our UTRNet also pays attention to imbalance between hard and easy samples and proposes a new focal weighted cross-entropy loss, which helps to make the training process focus on hard changed samples. The proposed UTRNet is validated on Landsat 8 time series data collected over nine typical scenes in the 2013-2021 period. Qualitative and quantitative comparisons with several state-of-the-art methods suggest the superior performance of UTRNet. Our dataset and codes are available at https://github.com/thebinyang/UTRNet.

Hyperspectral imagery (HSI) is one of the emerging tools to explore the physical properties and chemical composition of the lunar surface. Moon mineralogy mapper (M3) is the most widely used lunar HSI data set with the widest coverage and the excellent resolution; however, dense and nonperiodic stripes distributed across all bands in M3images hinder visual interpretation as well as their use in subsequent applications. In this article, a fast destriping approach for M3is proposed using the Hodrick–Prescott decomposition embedded in the low-rank framework (LRHP) to overcome this limitation. The integration of a statistical filter and variational model tackles the problem stemming from a lack of the correct residual information when certain pixels are corrupted in every band, thereby restoring severely degraded hyperspectral images (HSIs). Simulated and real experiments conducted on typical regions on the Moon with various levels of corruption demonstrate that the proposed LRHP rapidly achieves favorable performance against state-of-the-art approaches. Also, expanding tests on interference imaging spectrometer (IIM) data of Chang’E-1 and commonly used terrestrial remote sensing images show that LRHP has good generalization capability. Moreover, the integrated band depth (IBD) maps further verify the necessity of destriping and the high spectral fidelity of LRHP that benefits further applications.

Hyperspectral image classification has been a constant hot topic in remote sensing field, and achieved significant progress recently. Until now, most of the existing works are based on high-quality noise-free datasets, whereas in real applications, the images are often degraded by different types of noise, which makes the noise sensitivity become one of the key issues for classification assessment. In this paper, we study the noise effects on hyperspectral image classification including Guassian, salt-and-pepper, and stripe noise. The experimental results shows that noise has varying degrees of negative effects on different hyperspectral classification methods, which provides instructional information for method design and selection under noise environment in actual classification applications.

Fusion of hyperspectral images (HSIs) and multispectral images (MSIs) with different resolutions is an active research topic in the field of remote sensing. However, HSI-MSI fusion assessments in existing researches are basically conducted on simulated data gone through spectral or spatial downsampling, which cannot reflect the actual performances of fusion methods in application scenarios. To conquer this problem, a new remote sensing dataset- Hunan UAV-borne HSIs and MSIs fusion(HNU-HMiF) dataset is provided in this paper. The proposed dataset contains fine registered hyperspectral and multispectral image pairs captured by unmanned aerial vehicle (UAV) covering different ground objects, and can be used to evaluate, select, and even develop fusion methods for users or researchers. Successful applications including method evaluation and comparison confirm the validity and reliability of the proposed dataset.

In this paper, a new pansharpening method with residual convolutional neural network (RCNN)) is proposed. The proposed method utilizes a novel end-to-end CNN, which maps the differential information between the high spatial resolution panchromatic image (HR-PAN) and the low spatial resolution multispectral image (LR-MS) to the differential information between the HR-PAN image and the high spatial resolution multispectral image (HR-MS). Unlike the CNN-based pansharpening methods in other literatures, the proposed method makes full use of the spatial information in the HR-PAN image, and simultaneously preserve the spectral information of the MS image. Experimental results at both reduced resolution and full resolution demonstrate the superior performance of the proposed method comparing to state-of-the-art pansharpening methods in both quantitative and visual assessments.

Enhancing the spatial resolution of hyperspectral (HS) images by fusing with higher spatial resolution multispectral (MS) data is of significance for applications. However, due to the narrow bandwidth, HS images (HSIs) are vulnerable to various types of noise, such as Gaussian noise and stripes, which can severely affect the fusion performance. This paper focuses on antinoise HS and MS image fusion to enhance the spatial details and suppress the noise. By analysis of the intrinsic structure and noise properties, we formulate this problem as the minimization of an objective function. Under the optimization framework, small multilinear ranks in tensor are first used to identify the intrinsic structures of the clean HSI part. Then, considering the high spectral correlation, it is assumed that any bands can be represented by the combination of certain adjacent bands. The difference between one band and its corresponding combination can be used to preserve the spatio-spectral consistency and characterize the distribution of sparse noise (such as stripe noise), based on the variational properties along two directions. The alternating direction method of multipliers (ADMM) is applied to solve and accelerate the model optimization. Experiments with both simulated- and real-data demonstrate the effectiveness of the proposed model and its robustness to the noise, in terms of both qualitative and quantitative perspectives.

The existence of hybrid noise in hyperspectral images (HSIs) severely degrades the data quality, reduces the interpretation accuracy of HSIs, and restricts the subsequent HSI applications. In this paper, the spatial-spectral gradient network (SSGN) is presented for mixed noise removal in HSIs. The proposed method employs a spatial-spectral gradient learning strategy, in consideration of the unique spatial structure directionality of sparse noise and spectral differences with additional complementary information for effectively extracting intrinsic and deep features of HSIs. Based on a fully cascaded multiscale convolutional network, SSGN can simultaneously deal with different types of noise in different HSIs or spectra by the use of the same model. The simulated and real-data experiments undertaken in this study confirmed that the proposed SSGN outperforms at mixed noise removal compared with the other state-of-the-art HSI denoising algorithms, in evaluation indices, visual assessments, and time consumption.

Striping effects are a common phenomenon in remote-sensing imaging systems, and they can exhibit considerable differences between different sensors. Such artifacts can greatly degrade the quality of the measured data and further limit the subsequent applications in higher level remote-sensing products. Although a lot of destriping methods have been proposed to date, a few of them are robust to different types of stripes. In this paper, we conduct a thorough feature analysis of stripe noise from a novel perspective. With regard to the problem of striping diversity and complexity, we propose a universal destriping framework. In the proposed destriping procedure, a 1-D variational method is first designed and utilized to estimate the statistical feature-based guidance. The guidance information is then incorporated into 2-D optimization to control the image estimation for a reliable and clean output. The iteratively reweighted least-squares method and alternating direction method of multipliers are exploited in the proposed approach to solve the minimization problems. Experiments under various cases of simulated and real stripes confirm the effectiveness and robustness of the proposed model in terms of the qualitative and quantitative comparisons with other approaches.

Remote sensing images are often contaminated by varying degrees of stripes, which severely affects the visual quality and subsequent application of the data. Unlike with conventional methods, we achieve the destriping by separating the stripe component based on a full analysis of the various stripe properties. Under an optimization framework, an ℓ0-norm-based regularization is used to characterize the global sparse distribution of the stripes. In addition, difference-based constraints are adopted to describe the local smoothness and discontinuity in the along-stripe and across-stripe directions, respectively. The alternating direction method of multipliers is applied to solve and accelerate the model optimization. Experiments with both simulated and real data demonstrate the effectiveness of the proposed model, in terms of both qualitative and quantitative perspectives.