Weakly supervised semantic segmentation (WSSS) based on image-level labels significantly reduces the labeling burden. However, current mainstream approaches optimise solely using single image information, neglecting the rich semantic correlation among images and struggling to dynamically suppress interfering information. When confronted with complex backgrounds and multi-category remote sensing (RS) images, intra-class consistency and inter-class discrimination pose significant challenges. To address these challenges, this paper proposes the cross-image class token constraints network (CICTC-Net). CICTC-Net establishes semantic correlations across multi-category RS images and implements two modules for targeted optimisation. Specifically, the cross-image token enhancement (CITE) module constructs intra-class token relationship graphs and applies cross-image consistency constraints to enhance semantic consistency among objects of the same category. The class-patch interaction refinement (CPIR) module constructs a directed graph of class-patch relationships and employs a neighbourhood selection mechanism to refine class tokens, thereby enhancing inter-class discriminability. Experiments on two RS datasets demonstrate that this approach significantly outperforms existing state-of-the-art solutions.

Landslide extraction plays a critical role in disaster prevention and mitigation. However, acquiring sufficient landslide samples is often challenging, significantly limiting the performance of existing extraction methods. Self-supervised learning approaches, such as masked autoencoder (MAE) reconstruction and contrastive learning, offer a promising solution by reducing reliance on labeled data. Nonetheless, Transformer-based MAEs are computationally intensive, while CNN-based MAEs often struggle to capture global contextual information. To address these limitations, this letter proposes a CNN-Transformer hybrid masked autoencoder integrated with contrastive learning (CT-MAE) for pretraining a landslide extraction model. HRNet is employed for landslide extraction. An HRNet-based MAE, which has the c-apability to preserve the resolution and effectively integrate both shallow and deep features, is proposed for pretraining the HRNet. Moreover, the traditional Transformer-based MAE is applied as an auxiliary branch, and contrastive learning is utilized on the features extracted from the encoders of both MAEs. This auxiliary branch serves as a form of data augmentation, providing supplementary supervision, and enables the CNN-based MAE to capture richer global features without modifying the underlying CNN architecture. Experimental results on two publicly available datasets demonstrate that the proposed pretraining approach significantly enhances landslide extraction performance, highlighting its potential for large-scale applications.

Employing multimodal remote sensing images (MRSIs) enhances ground object identification, yet the heterogeneity of MRSIs can increase the risk of model overfitting, particularly when training samples are limited. Self-supervised methods, such as masked autoencoder (MAE) reconstruction or contrastive learning, efficiently extract features from MRSIs with minimal reliance on labeled samples. However, current methods have not adequately considered both the shared and complementary features of MRSIs during the pretraining process, limiting feature representation and classification performance. To address these limitations, we propose a self-supervised pretraining framework called shared and complementary information interaction enhancement network (SCIIENet) for hyperspectral images (HSIs) and light detection and ranging (LiDAR)/synthetic aperture radar (SAR) data joint classification. Specifically, the framework incorporates an decoupling-interaction-enhancement approach through a triple-branch MAE architecture. The triple-branch architecture seperately captures the spectral features of HSIs that provide complementary information relative to LiDAR/SAR data, as well as the spatial features of HSIs and LiDAR/SAR data. We also introduce an interaction enhancement strategy that utilizes hierarchical contrastive learning (HCL) to bridge significant gaps in MRSI to emphasize the shared spatial information of MRSIs and incorporates a two-stage cross-modal fusion encoder (TCFE) to integrate the shared and complementary spatial information of different modalities. Moreover, we propose multi-stage skip connections (MSCs) for the reconstruction, which effectively preserve complementary spatial information. Extensive experiments on four datasets show that our method significantly improves classification accuracy compared to several state-of-the-art methods.

High-resolution remote sensing image (HRSI) scene classification often faces challenges; for example, the intraclass similarity is low, but the interclass similarity is high due to complex backgrounds and variable scene scales. Convolutional neural networks (CNNs), the leading methods for HRSI scene classification, offer excellent performance. However, traditional CNNs require fixed-size inputs, which are a limitation when dealing with HRSI that represent large image domains, potentially degrading classification performance. To overcome these problems, we propose a CNN-based model named GLR-CNN in this article. First, to capitalize on the information from large-scale scenes adequately, VGG16 is utilized to extract the deep representative features, fine-tuned by the target HRSI of any size. Furthermore, a multilayer feature fusion block based on the channel–spatial attention algorithm is integrated into the CNN to capture more discriminative features from arbitrary-size images. Finally, to enhance the consistency between image features and similarities, a global latent relationship is used to measure the similarities among image features, then embed it into the fully connected layers (FCLs), and construct the latent relationship constraint. The model is optimized by the joint objective function including the latent relationship constraint and cross-entropy loss with label smoothing. Extensive experiments on three HRSI datasets obtained improvements of 3.93%, 6.5%, and 2.2% in overall accuracy compared to the finetuned VGG16 model, proving the effectiveness of the GLR-CNN method.

Satellite synthetic aperture radar interferometry (InSAR) has become a recognized and reliable surface deformation monitoring technology in recent years. However, the complexity of surface deformation, driven by various physical mechanisms, presents itself in different forms such as secular trend, cyclical fluctuations, and irregular variations in time series. Consequently, the limitations of conventional InSAR technology, which relies on linear deformation assumptions, make it challenging to meet the requirements of monitoring complex nonlinear deformation. Additionally, the commonly used parameter inversion method based on the least squares approach is unsuitable for non-Gaussian observation error distribution with gross errors. To address these issues, we propose an InSAR interferometric phase nonlinear function model that considers zero-mean second-order stochastic differential equations and periodic changes. This model can quantitatively describe the law of surface deformation driven by multiple physical factors. Furthermore, we utilize an efficient M-estimation method, known for its high robustness, to optimize the model parameters and mitigate the impact of non-Gaussian noise and/or gross errors in InSAR observation and data processing. By conducting simulation experiments, it verifies that the proposed method is more robust than the conventional InSAR method. Finally, the processing and analysis of Sentinel-1 data in the overlying rock glacier area confirm the effectiveness of the proposed method in extracting nonlinear surface deformation.

Deep subspace learning (DSL) plays an essential role in hyperspectral image classification, providing an effective solution tool to reduce the redundant information of hyperspectral image (HSI) pixels. Semi-supervised convolutional neural network (CNN)-based DSL methods can extract a more representative representation of latent subspace with the help of the labeled and unlabeled data. However, CNN-based DSL methods may lose the information of the class boundaries leading to misclassifications within regular input. We develop the deep subspace learning method with boundary consistency (DSL-BC) for the HSI classification to address this problem. In DSL-BC, the convolutional autoencoder (CAE) is first applied to extract the deep subspace representation (DSR). The DSR is used to model the boundary consistency. The graph convolutional network (GCN) is further adapted to enforce the boundary consistency by conducting the graph convolution on arbitrarily structured non-Euclidean data and irregular image regions. In addition, the adaptive entropy rate (ER) superpixel segmentation algorithm is applied to generate superpixels, and superpixel constraint is employed to improve the ability of DSL and GCN construction. DSL-BC integrates the DSL, the GCN, and the superpixel constraint into a unified objective function. A customized iterative algorithm is used to solve the objective function of the DSL-BC. The experimental results on three challenging public HSI datasets demonstrate that the DSL-BC can outperform the related state-of-the-art HSI classification methods.

Coal fires pose great threats to valuable energy resources, the ecological environment, and human safety. They are one of the most persistent fires on the Earth, which can burn for an extremely long-term period from decades to hundreds or even thousands of years. Remote sensing detection of coal fires is of significance for mitigating coal fire hazards. Nevertheless, short-term or temporal discrete land surface temperature (LST) data have limited capability in characterizing the persistent coal fire. This study proposed a methodology to monitor persistent coal fires using long-term Landsat thermal images and further to analyze spatiotemporal dynamics of coal fires. A total of 1118 high-quality Landsat images (each image containing$446\times446$pixels) spanning 35 years from 1986 to 2020 in the Wuda coalfield area (China) were processed to retrieve the LST. LST time series of each pixel was decomposed into the seasonal, trend, and remainder components. Coal fire areas were demarcated by using the range of the trend components. To trace the trend and change point of the LST time series, the Mann–Kendall test was applied to the trend components, and the Pettitt test was employed to the original time series vectors of those pixels located in the coal fire areas, respectively. The random sample consensus algorithm was utilized to identify the background temperature (inliers) and high temperature (outliers) and, thus, judge the coal fire burning period, and the symbolic aggregate approximation algorithm was used to evaluate the robustness of the judgment. The calibration was conducted according to the filed surveys, obtaining spatiotemporal 3-D coal fire dynamics. The proposed methodology was testified by comparisons with fieldwork and regional anomaly extractor algorithm, demonstrating good performance in comprehensive monitoring of persistent coal fires.

For deep networks, accurate image similarities cannot be well characterized with limited iterations, so the latent relationships between images can be embedded to enhance image retrieval performance. In this article, we propose a method named DFLLR to learn deep image features and accurate image similarities for remote sensing image retrieval (RSIR) simultaneously. First, the AlexNet is employed to extract high-level semantic features. Second, to obtain accurate image similarities, latent relationships between images are constructed with manifold learning and embedded in the AlexNet model with fully connected layers; in this way, the latent relationships and image features can be jointly learned. Third, to boost the RSIR performance further, the constraints of central and margin for jointly learning latent relationships and image features are integrated into our DFLLR. The central constraint is used to reduce the discrepancy of the latent relationships at the intraclass level and enhance the accuracies of image features. Moreover, the margin constraint is designed to enhance the accuracies of the latent relationships by maximizing the manifold margin between the latent relationships at the intraclass and interclass levels. To validate our method, we perform comprehensive experiments on three publicly available remote sensing image datasets, and the results demonstrate that it significantly outperforms other state-of-the-art methods.

Although accurate training and initialization information is difficult to acquire, unsupervised hyperspectral subpixel mapping (SPM) without relying on this predefined information is an insufficiently addressed research issue. This letter presents a novel Bayesian approach for unsupervised SPM of hyperspectral imagery (HSI) based on the Markov random field (MRF) and a band-weighted discrete spectral mixture model (BDSMM), with the following key characteristics. First, this is an unsupervised approach that allows adjustment of abundance and endmember information adaptively for less relying on algorithm initialization. Second, this approach consists of the BDSMM for accommodating the noise heterogeneity and the hidden label field of subpixels in HSI. The BDSMM also integrates SPM into the spectral mixture analysis and allows enhanced SPM by fully exploring the endmember-abundance patterns in HSI. Third, the MRF and BDSMM are integrated into a Bayesian framework to use both the spatial and spectral information efficiently, and an expectation-maximization (EM) approach is designed to solve the model by iteratively estimating the endmembers and the label field. Experiments on both simulated and real HSI demonstrate that the proposed algorithm can yield better performance than traditional methods.

Subspace learning (SL) plays an essential role in hyperspectral image (HSI) classification since it can provide an effective solution to reduce the redundant information in the image pixels of HSIs. Previous works about SL aim to improve the accuracy of HSI recognition. Using a large number of labeled samples, related methods can train the parameters of the proposed solutions to obtain better representations of HSI pixels. However, the data instances may not be sufficient to learn a precise model for HSI classification in real applications. Moreover, it is well known that it takes much time, labor, and human expertise to label HSI images. To avoid the abovementioned problems, a novel SL method that includes the probability assumption called SL with the conditional random field (SLCRF) is developed. In SLCRF, the 3-D convolutional autoencoder (3DCAE) is first introduced to remove the redundant information in HSI pixels. Besides, the relationships are also constructed using spectral-spatial information among the adjacent pixels. Then, the conditional random field (CRF) framework can be constructed and further embedded into the HSI SL procedure with the semisupervised approach. Through the linearized alternating direction method termed LADMAP, the objective function of SLCRF is optimized using a defined iterative algorithm. The proposed method is comprehensively evaluated using the challenging public HSI data sets. We can achieve state-of-the-art performance using these HSI sets.

Semisupervised deep learning methods (DLMs) can mitigate the dependence on large amounts of labeled samples using a small number of labeled samples. However, for semisupervised deep feature learning (SDFL), the quality of extracted features cannot be well ensured without a certain amount of labeled samples. To address this issue, we develop the SDFL method with feature consistency (SDFL-FC) for the hyperspectral image (HSI) classification. The SDFL-FC first adopts the convolutional neural network (CNN) to extract spectral–spatial features of HSI and then uses the fully connected layers (FCLs) to model the feature consistency. Moreover, two constraints that enforce both the feature consistency of single pixel (FCS) and feature consistency of group pixels (FCG) are introduced to obtain the representative and discriminative features. The FCS is achieved by the generative adversarial network (GAN) regularization, which can reconstruct the original data from extracted features. The FCG is based on the assumption that the features of group pixels should have similar characteristics within a superpixel, which is embedded in each FCL. The final FCL outputs the class labels, and the cross-entropy (CE) loss is calculated with the labeled samples, while the two losses of FCS and FCG are calculated with all the training samples (both labeled and unlabeled). SDFL-FC integrates the FCS, FCG, and CE loss into a unified objective function and uses a customized iterative optimization algorithm to optimize it. Experiments demonstrate that the SDFL-FC can outperform the related state-of-the-art HSI classification methods.

Unmixing is a key but difficult issue in hyperspectral image (HSI) processing, and many unmixing methods have been proposed. However, an effective introduction of the spatial context in unmixing remains a challenge but is a necessary condition for many real scene applications. In this letter, a new nonnegative matrix factorization (NMF) method that combines nonlocal spatial information with spatial group sparsity (NLNMF) is proposed. Each superpixel generated by the simple linear iterative clustering (SLIC) segmentation method was used as a group. The search region of the nonlocal means method was adaptively set using a superpixel label from each spectrum to find the similar spectra to reestimate the reference spectrum. Additionally, the sparsity of spectra in the same superpixel was considered to be the same. Experiment results for synthetic and real HSI showed that the proposed method not only can more accurately estimate the endmember and abundance compared with other unmixing methods but also has good performance regarding antinoise.

With a small number of labeled samples for training, it can save considerable manpower and material resources, especially when the amount of high spatial resolution remote sensing images (HSR-RSIs) increases considerably. However, many deep models face the problem of overfitting when using a small number of labeled samples. This might degrade HSR-RSI retrieval accuracy. Aiming at obtaining more accurate HSR-RSI retrieval performance with small training samples, we develop a deep metric learning approach with generative adversarial network regularization (DML-GANR) for HSR-RSI retrieval. The DML-GANR starts from a high-level feature extraction (HFE) to extract high-level features, which includes convolutional layers and fully connected (FC) layers. Each of the FC layers is constructed by deep metric learning (DML) to maximize the interclass variations and minimize the intraclass variations. The generative adversarial network (GAN) is adopted to mitigate the overfitting problem and validate the qualities of extracted high-level features. DML-GANR is optimized through a customized approach, and the optimal parameters are obtained. The experimental results on the three data sets demonstrate the superior performance of DML-GANR over state-of-the-art techniques in HSR-RSI retrieval.

Classification is an important application of hyperspectral image (HSI). However, it is also a challenging research topic due to the spatial variability of spectral signature and limited training samples. To address these problems, a novel unsupervised feature learning method called latent relationship guided the stacked sparse autoencoder (LRSSAE) is developed in this article, which can effectively exploit the latent relationship under feature space to improve the ability of feature learning. Moreover, the superpixels constraint is employed on the feature representation to avoid the “salt-and-pepper” problem, and it is enforced on the latent relationship to enhance the latent relationship learning additionally. In LRSSAE, combining the stacked sparse autoencoder (SSAE) with the graph regularizations of latent relationship in each hidden layer and the superpixel constraints in the top layer, we extract feature representation in an unsupervised manner. And then, we present a customized iterative algorithm to optimize the LRSSAE. We evaluate the proposed method on three widely used HSI data sets comprehensively. The results demonstrate that our method achieves promising classification performance on these data sets and obtains improvements of 5.06%, 5.77%, and 2.11% in overall accuracy compared to the best SSAE method.

Although efficient hyperspectral image (HSI) denoising relies on complete and accurate description and modeling the spatial-spectral signal in HSI, the current approaches do not fully account for key characteristics of HSI, i.e., the mixed spectra effect, the spatial nonstationarity effect, and noise variance heterogeneity effect. To address this issue, this article presents a linear spectral mixture model with nonlocal means constraint (LSMM-NLMC), with the following advantages. First, LSMM-NLMC can effectively learn the signal in mixed pixels in HSI by estimating clean endmembers and abundances for image restoration. Second, LSMM-NLMC can efficiently address nonstationary spatial correlation effect by imposing NLMC on the latent scene signal. Last, LSMM-NLMC provides accurate noise characterization by accounting for noise variance heterogeneity effect using a band-dependent noise model and a band-weighted Mahalanobis distance for similarity measurement. A novel optimization method based on the expectation-maximization (EM) algorithm and the purified means approach is used to efficiently solve the resulting maximum a posterior (MAP) problem. The experiments on both simulated and real HSI data sets demonstrate that the visual quality and denoising accuracy are significantly improved by the proposed LSMM-NLMC compared with previous methods.

Unsupervised classification of hyperspectral imagery (HSI) relies on a data generative model, based on which the labels of pixels and the model parameters are iteratively estimated. Traditionally, the generative model is based on the Gaussian mixture model (GMM) that describes the data generation process from a statistical perspective. However, considering the fact that a semantic class is always dominated by a particular endmember, classifying the spectral pixels based on the associated endmember-abundance pattern as described by the spectral mixture model (SMM) is more meaningful from a physical perspective. In this paper, we explore the potential of spectral mixture model for assisting unsupervised classification of HSI based on a recently proposed K-P-Means unmixing algorithm. Moreover, we investigate modeling the spatial information using Markov random field in this new context. We incorporate SMM and MRF into the Bayesiam framework and solve it via the maximum a posterior (MAP) approach. The results on both simulated and real hyperspectral images demonstrate that this new approach can effectively exploit the spatial-spectral information of HSI for improved unsupervised classification of HSI.

Efficient super-resolution of hyperspectral images (HSI) relies on the representational model (RM) that is capable of capturing the spatial and spectral correlation in hyperspectral images. In this paper, the spectral information in hyperspectral images is explained by linear spectral mixture model (LSMM), which expressed the observed pixels as a linear combination of endmembers, and the spatial information is captured by a spatial auto-regression model. The two component is combined in the maximum likelihood estimation (MLE) framework and solved by the expectation and maximization (EM) algorithm. Experiments on both simulated and real hyperspectral images demonstrate that the proposed method is not only capable of providing an accurate and effective super-resolution reconstruction of the image, but also capable of resisting the influence of noise.

Efficient denoising of hyperspectral imagery (HSI) relies on an representational model that is capable of capturing the spatial and spectral correlation in HSI. Recently, an intrinsic representation (IR) approach based on the linear spectral mixture model (LSMM) was proposed for unsupervised feature extraction. The IR model constitutes a sound representational model due to its ability to account for the physical data generation process of HSI, the spatial correlation effect, and the noise variance heterogeneity effect. In this paper, we explore the potential of IR for the denoising of HSI. A noisy pixel in HSI is expressed as a nonnegative linear combination of several endmembers, plus some Gaussian noise with heterogeneous noise variances. In order to perform denoising, the IR approach is used to adaptively estimate both the endmembers and the nonnegative coefficients (i.e., the abundances), which are finally used to reconstruct the clean image. The experiments on both simulated and real hyperspectral images demonstrate that the IR approach not only can resist the influence of noise, but also can preserve the image details.

This letter presents K-P-Means, a novel approach for hyperspectral endmember estimation. Spectral unmixing is formulated as a clustering problem, with the goal of K-P-Means to obtain a set of “purified” hyperspectral pixels to estimate endmembers. The K-P-Means algorithm alternates iteratively between two main steps (abundance estimation and endmember update) until convergence to yield final endmember estimates. Experiments using both simulated and real hyperspectral images show that the proposed K-P-Means method provides strong endmember and abundance estimation results compared with existing approaches.

The combination of nonlocal grouping and transformed domain filtering has led to the state-of-the-art denoising techniques. In this paper, we extend this line of study to the denoising of synthetic aperture radar (SAR) images based on clustering the noisy image into disjoint local regions with similar spatial structure and denoising each region by the linear minimum mean-square error (LMMSE) filtering in principal component analysis (PCA) domain. Both clustering and denoising are performed on image patches. For clustering, to reduce dimensionality and resist the influence of noise, several leading principal components identified by the minimum description length criterion are used to feed the K-means clustering algorithm. For denoising, to avoid the limitations of the homomorphic approach, we build our denoising scheme on additive signal-dependent noise model and derive a PCA-based LMMSE denoising model for multiplicative noise. Denoised patches of all clusters are finally used to reconstruct the noise-free image. The experiments demonstrate that the proposed algorithm achieved better performance than the referenced state-of-the-art methods in terms of both noise reduction and image detail preservation.