Snapshot compressive imaging (SCI) captures multispectral images (MSIs) using a single coded two-dimensional (2-D) measurement, but reconstructing high-fidelity MSIs from these compressed inputs remains a fundamentally ill-posed challenge. Recent diffusion-based methods improve quality but are limited by scarce MSI training data, domain shifts from RGB-pretrained models, and slow multi-step sampling. These drawbacks restrict their practicality in real-world applications. Unlike prior approaches that rely on expensive iterative refinement or subspace-based diffusion embeddings (e.g., DiffSCI, PSR-SCI)—we introduce a fundamentally different paradigm: a self-supervised One-Step Diffusion (OSD) framework designed specifically for SCI. The key novelty lies in using a single-step diffusion refiner to correct an initial reconstruction, eliminating iterative denoising entirely while preserving generative quality. Moreover, we adopt a self-supervised equivariant learning strategy to train both the predictor and refiner directly from raw 2-D measurements, enabling generalization to unseen domains without ground-truth MSI. To further address limited MSI data, we design a band-selection–driven distillation strategy that transfers core generative priors from large-scale RGB datasets, effectively bridging the domain gap. Extensive experiments confirm that our approach sets a new standard—yielding PSNR gains of 3.44dB, 1.61dB, and 0.28dB on the Harvard, NTIRE, and ICVL datasets respectively, while cutting reconstruction time from 8.9s to just 0.22s per image. These gains in efficiency and adaptability advance SCI reconstruction, enabling accurate and practical real-world deployment.

Binary segmentation is used to distinguish objects of interest from background, and is an active area of convolutional encoder-decoder network research. The current decoders are designed for specific objects based on the common backbones as the encoders, but cannot deal with complex backgrounds. Inspired by the way human eyes detect objects, we propose a new unified dual-branch decoder paradigm, termed the difference-aware decoder, to better explore the differences between foreground and background and to separate objects of interest in optical images. This decoder operates in two stages, leveraging multi-level features from the encoder. In the first stage, coarse detection of foreground objects is achieved by directly utilizing high-level semantic features, mimicking the initial rough observation of human vision. In the second stage, the decoder refines segmentation by exploring differences in low-level features, guided by the coarse map from the first stage. To enhance this process, we introduce two key innovations. First, a difference-aware prototype generation strategy leverages the guide map to extract foreground and background prototypes from high-level features, and calculates the similarity between these prototypes and corresponding representations in low-level feature spaces. Second, an overlapped window cross-level semantic guidance mechanism integrates high-level semantic information into low-level features through channel grouping and multi-scale aligned window pairs, guided by the similarities computed in the first strategy. Together, these innovations significantly enhance the DAD’s ability to discern subtle differences, enabling precise foreground extraction and effectively addressing the challenges of complex and varied backgrounds. To verify the performance of the proposed difference-aware decoder, we choose three well known backbones including ResNet, Res2Net, PVT, and two binary segmentation tasks,i.e., salient object detection, and camouflaged object detection, for comparative experiments. The results demonstrate that the difference-aware decoder can achieve higher accuracy than the other state-of-the-art binary segmentation methods for these tasks. The source code will be available on https://github.com/Henryjiepanli/DAD.

The presence of metallic implants introduces bright and dark streaks that appear in computed tomography (CT) images, degrading image quality and interfering with medical diagnosis. To reduce these artifacts, deep learning approaches have been applied for metal-corrupted restoration, which usually requires a large amount of simulated degraded-clean pairs for training. To achieve metal artifact reduction (MAR) without reference images, implicit neural representation (INR) has emerged and shown capabilities for image restoration in an unsupervised manner. However, existing INR methods for MAR usually treat the spatial coordinates independently and ignore their correlation, resulting in detail loss and artifacts remaining. In this paper, we propose an INR-based unsupervised MAR framework and design a High-order Line Attention Network to capture local contextual and geometric representations from X-rays, which maps the spatial coordinates into discrete linear attenuation coefficients of imaged objects for artifact-free CT image reconstruction. The second-order feature interaction can effectively improve the spectral bias problems and fit low and high-frequency details of real signals well. The proposed line-attention module with linear complexity can establish global relationships among spatial point tokens from sampled rays. To provide more local contextual information, a multiple local adjacent ray sampling strategy is adopted to compose several sub-fan beams with more context as a training batch. With the help of these components, the unsupervised MAR framework can approximate the implicit continuous function to estimate measurements and generate artifact-free CT images. Simulated and real experiments indicated that the proposed approach achieved superior MAR performance compared with other state-of-the-art methods.

Few-shot learning (FSL) has been employed in hyperspectral image (HSI) classification, achieving excellent performance with limited training data. However, existing HSI few-shot classification methods often encounter the problem of insufficient domain-transferable knowledge learning that is either from natural images or HSI solely. In this paper, we propose a two-stage cross-domain few-shot classification method of HSI, which for the first time makes use of heterogeneous labeled natural images and HSIs in the source domain (SD) to support the classification of novel classes in the target HSI domain. We first use a large amount of labeled natural images at the first stage to pre-train a backbone, which will be used to extract the spatial feature of HSIs at the second stage with fine-tuning. In the second stage, we propose a cross-domain few-shot classification method, which allows for effective discriminative feature learning in the target HSI domain with the transferred knowledge of the old classes obtained from natural images and HSIs in the source domain. To obtain domain-transferable knowledge, FSL is employed on the HSI source and target domain. To deal with the domain shift problem, we propose a class-matching based cross-domain contrastive loss. In addition, we take into account the large spectral variations problem in the target HSI domain and introduce an instance-level self-supervised loss. Experimental results on real data sets demonstrate that our method outperforms the recent state-of-the-art.

Road graph extraction from remote sensing images is essential in navigation and urban planning. However, shadows and occlusions in these images frequently disrupt the continuity of road representations, resulting in fragmented and poorly connected road graphs extracted by existing methods. To overcome these challenges, we introduce KDGraph, a novel keypoint detection method for road graph extraction from remote sensing images. Specifically, keypoints are defined as vertices situated at the endpoints, corners, or junctions of roads, characterized by their positions and multiple directional attributes. In addition, we develop a greedy parsing algorithm to connect these keypoints and construct the road graph based on their directional information. The key innovation of our method lies in reformulating road mask extraction as a keypoint detection task. By positioning keypoints in areas less affected by shadows and occlusions, KDGraph effectively reduces their impact on road graph connectivity. Extensive experiments are conducted using the SpaceNet3 dataset and a newly constructed shadow-occluded road graph extraction (SoR) dataset, which includes road segments from 15 cities with varying degrees of shadows and occlusions. A patch expansion strategy is introduced for large-scale inference on the SoR dataset. Results show that KDGraph outperforms all comparative methods in both generalization and the capability of handling shadows and occlusions. Code and SoR dataset available at:https://github.com/ruoyxue/KDGraph.

Recently, transformer-based models have achieved remarkable performance in the hyperspectral image (HSI) classification. However, due to the limited training data, existing methods often show limited capability of capturing fine-grained local features. Although attempts have been made to solve this problem, the large amount of parameters imposes the risk of overfitting. In this paper, we propose a locally enhanced transformer network for HSI classification with fewer network parameters, which mainly consists of a multi-branch spatial-spectral tokenization (MSST) module and a dual-branch transformer encoder (DTE) module. The MSST generates effective spatialspectral tokens through diverse convolutions with a residual connection. The DTE consists of a global transformer branch and a locally enhanced transformer branch, which are used to capture the global and local spatial dependencies of HSI, respectively. Unlike the conventional self-attention module used in the global branch, we propose an improved multi-head selfattention (IMSA) module in the local branch by incorporating the local prior information of HSI with graph convolution, to enhance the local information extraction. To fuse the global and local features from the two branches, we introduce an adaptive strategy by using learnable weights for both branches. We devise our MSST and DTE with a shallow architecture, significantly reducing the number of parameters. Experimental results on benchmark datasets demonstrate that the proposed method outperforms the state-of-the-art.

Cross-domain few-shot learning (FSL) has shown promising performance in hyperspectral image classification (HSIC) under limited labeled data. However, existing approaches often suffer from insufficient meta-knowledge transfer due to reliance on a single source domain, and fail to fully bridge the domain gap owing to the use of single-level alignment strategies. In this paper, we propose a heterogeneous data-driven, global-to-local cross-domain FSL framework for HSIC, leveraging richly labeled natural RGB images and hyperspectral data as source domains to support classification in the target domain with only a few labeled samples. The proposed method consists of two stages. In the first stage, we employ natural RGB images to learn a powerful spatial feature extractor with FSL and self-supervised learning, which will be fine-tuned for HSI in the second stage. To alleviate the domain gap between the source and target HSI domains at the second cross-domain FSL stage, we propose a global-to-local domain adaptation strategy that performs alignment both at the domain level and class level, effectively reducing the learning bias toward the source HSI while promoting discriminative feature learning. Additionally, to address large intra-class variance in the target HSI domain, we introduce a self-supervised contrastive loss based on positive pairs only, enhancing the within-class representation compactness. Extensive experiments on three benchmark datasets demonstrate that our method outperforms the state-of-the-art.

Semantic segmentation is a critical procedure in remote sensing image analysis that backs up various applications. High resolution remote sensing images contain a wealth of ground object features, which are organized into various describable scenes. The visual content offset towards each scene is an intuitive sense for understanding geospatial objects. However, existing semantic segmentation methods for remote sensing images generally neglect this intuition and lack the ability to adjust their perception preference on different images. To address this problem, we propose a paradigm for collecting scene information and dynamically adjusting the model inference process to be scene-aware. Specifically, our method leverages the class feature from the image to enhance the fixed class representation from the model. The interaction of these information is facilitated by a neighbor-friendly embedding space, making it more faithful to associate the image features and model parameters. For the model to better understand the complex scenes, a manifold mixup method is proposed to expand the effective embedding space on intra-class and inter-class regions, forcing the model to challenge the ambiguous instances in remote sensing images. Extensive experiments on four publicly available datasets demonstrated that our proposed improved the accuracy of semantic segmentation models on remote sensing images, overcoming the state-of-the-art methods.

Despite advances in deep learning for remote sensing building extraction (RSBE), Multi-target Domain RSBE (MD-RSBE) remains challenging, as it requires transferring knowledge from a labeled source domain to multiple unlabeled target domains, with domain shifts in texture, style, and semantics. Existing domain adaptation (DA) and generalization (DG) methods face significant limitations: DA requires target-domain training, while DG needs multi-source training, leading to high training costs and low generalization in practical MD-RSBE scenarios. To address this, we propose a Multi-view Augmented Single-source Domain Generalization (MASDG) method, which effectively mitigates domain shifts across RS source and target domains for robust MD-RSBE performance by enriching the diversity of the source domain through multi-view augmentation and enforcing semantic consistency. Specifically, MASDG consists of three key components: Texture-level Domain Augmentation (TDA) module, Style-level Domain Augmentation (SDA) module and Semantic-invariant Representation Learning (SRL). To mitigate texture-level domain shift, TDA first introduces parameter-optimized multi-layer random convolution to modify the texture of source image, generating texture-augmented image pairs for simulating real-world texture diversity across various RS domains. Then, with each image pair from TDA, SDA employs two paralleled encoders, namely the general feature encoder and the batch-guided style encoder, to formulate multi-view building features, further mitigating style-level domain shift. Finally, SRL ensures semantic-invariant representation learning via a dual mechanism, including multi-view segmentation loss and semantic consistency loss. The former generates predictions from diverse feature views (original, texture-augmented, style-augmented, etc.), while the latter performs semantic alignment by minimizing distribution discrepancies among predictions, bridging semantic inconsistency to enable robust segmentation. Extensive experiments across three different MD-RSBE settings with 7 different target domains demonstrate that our MASDG outperforms existing state-of-the-art methods by a significant margin.

Multimodal remote sensing image segmentation (MRSIS) is important for intelligent remote sensing image (RS) interpretation, which encompasses three distinct tasks: semantic segmentation, instance segmentation, and panoptic segmentation. Existing methods typically address individual tasks with specialized models, limiting generalization and real-world applicability. Multi-task learning approaches have introduced separated task heads to unify tasks, yet we identify two key challenges when directly applying them to MRSIS: (1) the modality gap, arising from semantic discrepancies and granularity discrepancies across RS modalities, and (2) the task gap, due to varying preferences in learning different segmentation tasks. To overcome these challenges, we propose PMTSeg—a novel Prompt-driven Multimodal Transformer for task-adapted MRSIS. PMTSeg integrates three key components: (1) Task-common Multimodal Affinity Approximation (TMAA), (2) Task-common Multi-scale Semantic Fusion (TMSF), and (3) a unified Prompt-driven Segmentation Head (PSH). First, TMAA addresses the modality gap by approximating inter-modal affinity matrices, extracting task-common features across modalities and aligning semantic information. Then, TMSF further integrates these features using the scale-matched fusion at multiple scales to produce enriched, multi-scale task-common features. Moreover, to address the task gap, the PSH leverages task-adapted text prompts and task-adapted contrastive loss to model relationships across tasks, enabling adaptive optimization for robust and universal MRSIS performance. Extensive experiments on three MRSIS datasets—VALID, SEMCITY TOULOUSE, and UBCV2—demonstrate that PMTSeg significantly surpasses state-of-the-art methods in all three segmentation tasks, offering a unified and accurate solution to MRSIS.

Accurate cloud detection is essential for subsequent optical remote sensing imagery processing. Although many deep learning (DL)-based cloud detection methods have been proposed, the accurate detection of thin clouds still remains a challenge. To solve this issue, this article introduces a thin cloud-aware network (TANet). TANet tackles the problem from the aspects of color, texture, spatial distribution, and feature by constructing unique strategies to enhance the sensitivity of the network to thin cloud regions, thereby improving the overall accuracy of cloud detection. On the one hand, the TANet utilizes a color prior guidance module (CPGM) to incorporate robust dehaze priors, guiding the network to pay more attention to thin cloud areas. On the other hand, the global information aggregation module (GIAM) is employed to deeply extract long-distance dependence between pixels, mining potential correlations between thin and thick clouds, and addressing the challenge of recognizing thin clouds in a local perspective. In addition, we construct a plug-and-play cloud feature difference (CFD) loss, encouraging the network to learn more distinctive features between pixels of thin clouds and cloud-free regions, thereby strengthening the network’s ability to distinguish highly similar samples of different classes. The experimental results substantiate that our proposed method attains the lowest omission error and the highest detection accuracy. This affirms the superior capability of TANet in thin cloud detection, thereby yielding more dependable cloud detection results.

Large-scale high-resolution (HR) land-cover mapping is a vital task to survey the Earth's surface and resolve many challenges facing humanity. However, it is still a nontrivial task hindered by complex ground details, various landforms, and the scarcity of accurate training labels over a wide-span geographic area. In this paper, we propose an efficient, weakly supervised framework (Paraformer) to guide large-scale HR land-cover mapping with easy-access historical land-cover data of low resolution (LR). Specifically, existing land-cover mapping approaches reveal the dominance of CNNs in preserving local ground details but still suffer from insufficient global modeling in various landforms. Therefore, we design a parallel CNN-Transformer feature extractor in Paraformer, consisting of a downsampling-free CNN branch and a Transformer branch, to jointly capture local and global contextual information. Besides, facing the spatial mismatch of training data, a pseudo-label-assisted training (PLAT) module is adopted to reasonably refine LR labels for weakly supervised semantic segmentation of HR images. Experiments on two large-scale datasets demonstrate the superiority of Paraformer over other state-of-the-art methods for automatically up-dating HR land-cover maps from LR historical labels.

Tree species classification is essential for forest resource surveys and monitoring activities. Although airborne hyperspectral images (HSIs) can provide rich spatial and spectral information, the lack of labeled samples and the high similarity between spectra remain challenges for achieving fine-grained tree species classification mapping. In this article, a cross modal few shot learning framework is presented for multiple tree species classification (CMTSC). Notably, we innovatively introduce language prior knowledge to guide the generation of discriminative visual features, aiming to use additional modalities to improve the uni-modal classification task. Firstly, an improved three-dimensional ghost attention network (TGAN) with strong learning capability without massive parameters is constructed. Secondly, we use linguistic features from class names to optimize the decision boundary of the visual classifier by cross-modal adaptation. Thirdly, the cross domian few shot learning (FSL) strategy is employed to overcome the dilemma of sparse labeled samples and fixed application scenarios. Experiments on Gaofeng Forest Farm B (GFF-B) in Nanning City demonstrate the effectiveness of the proposed method compared to other state-of-the-art methods. The codes will be available at: https://github.com/HlEvag/CMTSC.

The escalating intensity and frequency of floods, exacerbated by global climate change, emphasize the urgent need to address the growing risks of floods. Rapid and precise flood detection is paramount for efficiently responding to emergencies and executing disaster relief measures, enabling swift reactions to flood disasters and minimizing the losses incurred thus. The 2024 IEEE GRSS Data Fusion Contest Track 1 is centered on leveraging multi-source remote sensing data, particularly synthetic aperture radar (SAR) data, to classify flood and non-flood areas. In this contest, we acknowledge the significance of managing uncertain predictions and present an efficient Uncertainty-Aware Fusion Network (UAFNet). Specifically, we build on the traditional encoder-decoder architecture, initially employing the pyramid visual transformer (PVT) as a feature extractor. Subsequently, we apply a typical decoding strategy, namely the feature pyramid network, to obtain a flood extraction map with relatively high uncertainty. Furthermore, leveraging the uncertain extraction map, we introduce an Uncertainty Rank Algorithm to quantify the uncertainty level of each pixel of the foreground and background. We seamlessly integrate this algorithm with our proposed Uncertainty-Aware Fusion Module, enabling level-by-level feature refinement and ultimately yielding a refined extraction map with minimal uncertainty. Employing the proposed UAFNet, we utilize diverse versions of PVT as encoders to train multiple UAFNets. Additionally, we enhance our approach with online testing augmentation and multi-model fusion strategy, aiming to enhance the final flood extraction accuracy. Our technical solution has exhibited outstanding performance, earning the first-place ranking in the 2024 IEEE GRSS Data Fusion Contest Track 1 and achieving an impressive F1 score of 82.985% on the official test set.

As global climate change worsens, floods are becoming more severe and frequent, urgently demanding effective flood risk mitigation strategies. Timely and precise flood inundation mapping is crucial for emergency response and relief. The 2024 IEEE GRSS Data Fusion Contest Track 2 aims to pioneer innovative algorithms for accurate flood extraction using multi-source optical remote sensing (RS) data. However, data diversity introduces aleatoric uncertainty, especially with synthetic, non-real data. Meanwhile, the vast coverage of RS imagery and the small proportion of flood areas cause a significant class imbalance, leading to epistemic uncertainty. In this paper, we propose an Uncertainty-aware Detail-Preserving Network (UADPNet) for rapid flood mapping of multi-source optical data. Firstly, we design an Aleatoric Uncertainty Estimator to model aleatoric uncertainty in multi-source data. Secondly, we introduce a Multi-Scale Convolution Block to extract multi-scale information without downsampling. Thirdly, we utilize a multi-level supervised strategy to quantify epistemic uncertainty and highlight uncertain pixels via the Uncertainty-Aware Fusion Module. With UADPNet, we adopt a multi-model fusion and post-processing strategy to enhance the final flood extraction accuracy. Our outstanding experimental results in the official test set showcase the superiority of our method, which secured the first-place ranking in the 2024 IEEE GRSS Data Fusion Contest Track 2, boasting an impressive F1 score of 89.843%.

Buildings, as fundamental man-made structures in urban environments, serve as crucial indicators for understanding various city function zones. Rapid urbanization has raised an urgent need for efficiently surveying building footprints and functions. In this study, we proposed a semi-supervised framework to identify every building’s function in large-scale urban areas with multi-modality remote-sensing data. In detail, optical images, building height, and nighttime-light data are collected to describe the morphological attributes of buildings. Then, the area of interest (AOI) and building masks from the volunteered geographic information (VGI) data are collected to form sparsely labeled samples. Furthermore, the multi-modality data and weak labels are utilized to train a segmentation model with a semi-supervised strategy. Finally, results are evaluated by 20,000 validation points and statistical survey reports from the government. The evaluations reveal that the produced function maps achieve an OA of 82% and Kappa of 71% among 1,616,796 buildings in Shanghai, China. This study has the potential to support large-scale urban management and sustainable urban development. All collected data and produced maps are open access at https://github.com/LiZhuoHong/BuildingMap.

Multimodal image matching is essential in image stitching, image fusion, change detection, and land cover mapping. However, the severe nonlinear radiometric distortion and geometric distortion of multimodal images severely limit the accuracy of multimodal image matching. To solve these problems, we propose a detector-free multimodal image matching approach to establish pixel-level dense correspondences. We mitigate the impact of modality differences on feature point extraction by establishing robust reference points. Specifically, we design a phase congruency module to keep the location of the reference point centered on the image edge structures. Simultaneously, a guiding correction module exploits the geometric relationships between pixels and reference points to establish accurate pixel correspondences. Finally, refined correspondences are obtained by finely positioning highly correlated pixel matches. Experiments show that our method can obtain sufficient and robust correspondences on multimodal images.

Convolutional neural networks (CNN) have demonstrated excellent performance in the classification of hyperspectral image (HSI). However, CNN-based models often fail to capture long-range contextual information due to the limited receptive fields. In this paper, we propose a locally enhanced transformer network for HSI classification. Firstly, we propose a multi-branch spatial-spectral token (SST) module based on CNN to transform HSI into the spatial-spectral tokens, facilitating the reduction of information loss during tokenization. Secondly, after SST we propose a dual-branch transformer module, which consists of a global transformer and a locally enhanced transformer, to capture the global and local spatial features of HSI. Particularly, in the local branch we develop an improved multi-head self-attention (IMSA) by incorporating the neighbourhood information derived from super-pixel segmentation to improve the local feature extraction ability of the conventional transformer. Experimental results on benchmark datasets demonstrate that the proposed method achieves better performance over the state-of-the-art.

Self-supervised learning (SSL) has gained significant traction within the remote sensing community, with pretraining a foundation model on large-scale unlabeled datasets for the interpretation of remote sensing images (RSIs) emerging as a trending direction. This approach aims to supplant the conventional practice of loading ImageNet pretrained weights, offering a more versatile and potentially more effective solution for handling RSIs. Among SSL techniques, contrastive learning excels in extracting general representations in the field of remote sensing. However, its excessive focus on inter-instance discrimination hinders the effectiveness of pretraining due to the diverse and complex geographical information present in RSIs. Moreover, the typical two-variations-as-one-pair pattern may be suboptimal, particularly given the temporal information specific to RSIs. In this article, we propose a novel method called promoting intra-instance similarity (PIS) for short, which leverages the temporal information specific to RSIs and increases the intra-instance variations to expand the positive representation space. Additionally, by PIS within this space, our foundation models develop the ability to extract more general and instance-invariant features that prove beneficial for various downstream tasks. Experiments show that our PIS method achieves state-of-the-art (SOTA) performance on ten datasets across four downstream remote sensing tasks, demonstrating the generalizability and efficacy of the proposed method. Through our preliminary investigation into intra-instance characteristics, we believe there exists substantial potential in this aspect, holding considerable promise for further exploration. The codes are available on the website:https://github.com/ShawnAn-WHU/PIS.git.

Haze pollution in hyperspectral images (HSIs) leads to surface information lack and image clarity degradation, which seriously affects the performance of subsequent image interpretation. Existing model-based hyperspectral haze removal methods enjoy good interpretability and generalization, but they can only process images in a specific wavelength range due to the principle limitation. Deep learning-based dehazing methods have good feature extraction capability, but the cost of acquiring sufficient training data is high in practical applications. At the same time, taking into account that HSIs have spectral low-rank structures, fully utilizing the low-rank property will facilitate the reconstruction of HSIs. In order to combine the complementary benefits of deep learning-based and physical model-based approaches, we decide to formulate HSI dehazing reconstruction as an unsupervised DIP framework. Specifically, we propose an unsupervised dehazing network with hybrid prior constraints (HPC-UDN) for HSI haze removal, which effectively integrates low-rank prior, deep priors, and physical haze prior. First, the low-rank prior of hyperspectral data is characterized by matrix decomposition, where the decomposition factors are learned through two generative networks. Then, multiple spectral groups are divided based on the correlation and complementarity between spectral bands. In order to exchange information between adjacent spectral groups, a novel spectral grouping feature fusion module is designed, which connects neighboring spectral groups to transfer spectral and spatial features. Finally, high-quality HSI is recovered by merging the features extracted from each spectral group. Extensive simulated and real-data experiments certify the effectiveness and robustness of the presented unsupervised approach and potential applications in the GF-5 image dehazing task.

Subspace clustering has achieved remarkable performance for hyperspectral image (HSI). However, existing methods are often computationally expensive and have limited ability to capture the intrinsic structural information of HSI. In this paper, we propose a structural prior-guided subspace clustering method, which simultaneously incorporates the local and non-local spatial information and the cluster prior information. Accordingly, three efficient regularizations are developed. Considering the local connectivity of pixels, we propose an ℓ2,1norm based constraint on the representation difference matrix to improve the homogeneity of clustering result. Next, to capture the non-local geometric structure of HSI, we propose a manifold-based regularization with an adaptively learned landmark graph. Furthermore, we explore the block-diagonal cluster structure of HSI and develop a landmark-based clustering constraint, which makes the representations more favorable for clustering. Our local constraint is imposed on all the data points due to its efficiency and the latter two are solely imposed on landmarks, leading to computationally efficient regularizations. Due to the local constraint, the manifold and cluster structure of the landmarks can be effectively propagated to all the data points. To make our model scalable to large-scale data, we learn a compact dictionary with an orthogonal constraint, significantly reducing the number of parameters. In addition, we propose a novel landmark selection method to support our landmark-based constraints using multi-scale super-pixel segmentation and clustering, which improves the uniformity and diversity of landmarks. We also develop an efficient algorithm to solve the proposed model. Experimental results demonstrate that our model outperforms the state-of-the-art.

Building extraction aims to segment building pixels from remote sensing images and plays an essential role in many applications, such as city planning and urban dynamic monitoring. Over the past few years, deep learning methods with encoder–decoder architectures have achieved remarkable performance due to their powerful feature representation capability. Nevertheless, due to the varying scales and styles of buildings, conventional deep learning models always suffer from uncertain predictions and cannot accurately distinguish the complete footprints of the building from the complex distribution of ground objects, leading to a large degree of omission and commission. In this paper, we realize the importance of uncertain prediction and propose a novel and straightforward Uncertainty-Aware Network (UANet) to alleviate this problem. Specifically, we first apply a general encoder–decoder network to obtain a building extraction map with relatively high uncertainty. Second, in order to aggregate the useful information in the highest-level features, we design a Prior Information Guide Module to guide the highest-level features in learning the prior information from the conventional extraction map. Third, based on the uncertain extraction map, we introduce an Uncertainty Rank Algorithm to measure the uncertainty level of each pixel belonging to the foreground and the background. We further combine this algorithm with the proposed Uncertainty-Aware Fusion Module to facilitate level-by-level feature refinement and obtain the final refined extraction map with low uncertainty. To verify the performance of our proposed UANet, we conduct extensive experiments on three public building datasets, including the WHU building dataset, the Massachusetts building dataset, and the Inria aerial image dataset. Results demonstrate that the proposed UANet outperforms other state-of-the-art algorithms by a large margin. The source code of the proposed UANet is available at https://github.com/Henryjiepanli/Uncertainty-aware-Network.

Image-level weakly supervised semantic segmentation (WSSS) methods have gained prominence in remote sensing image building extraction tasks, primarily due to their cost-effectiveness in manual annotation. However, owing to the intricate details present in remote sensing building images, the pseudolabels generated from existing image-level weakly supervised methods often encounter issues of incorrect activation and unclear boundaries. In this article, we propose a global-to-local dynamic information enhancement (G2LDIE) framework. This framework effectively extracts global information from remote sensing building images and supplements local details through a local information enhancement (LIE) module, generating more accurate pseudolabels. Additionally, we propose a dynamic label guide strategy (DLGS) to enhance model consistency in category representation across various scale images. To address the challenge of unclear building boundaries issue in pseudolabels, we introduce a segment anything model (SAM) postprocessing (SPP) method, which can better correct the boundaries of building images at different resolutions while reducing computational costs. Extensive and detailed experiments on three datasets confirm that our framework can generate refined pseudolabels and outperform other image-level weakly supervised methods in terms of accuracy and generalization performance in building extraction.

Landsat optical sensor is crucial for the long-term observations of the Earth’s surface with a 30 m spatial resolution. However, the 16-day revisit cycle and severe atmospheric interference have impeded the monitoring of rapid surface changes. Spatiotemporal fusion (STF) is a classic method of predicting Landsat surface reflectance with multi-temporal and multi-source data, but it is limited by unpredictable temporal changes and cloudy Landsat-MODIS image pairs. Another emerging solution is synthetic aperture radar (SAR)-to-optical image translation (S2OIT), which always produces spectral distortions. To tackle these defects, we propose a new data-driven solution, SAR-optical data-based spatial–spectral fusion (SOSSF), which combines the high-spatial and cloud-free advantages of Sentinel-1 data and the high-spectral and high-temporal advantages of MODIS images to synthesize high-spatial and high-temporal Landsat-8 images. To achieve this solution, we first establish a worldwide benchmark dataset, namely SMILE, with various land cover types and all meteorological seasons, satisfying the big data requirements of deep learning. Second, we design an attention-based dual-path fusion network (ADFNet) to respectively extract and fully fuse spatial and spectral information from SAR-optical data. Extensive experiments suggest that the proposed SOSSF solution outperforms the state-of-the-art STF and S2OIT solutions, robustly performing in the continuously changing and frequently cloudy regions. The proposed ADFNet model achieves the best visual effect and the highest accuracy in different scenes, seasons, and bands. Furthermore, the proposed SOSSF solution is proven to be a practical way to simulate time-series and large-scale Landsat-8 surface reflectance, considerably enriching raw Landsat-8 products.

Deep learning-based building instance extraction on remote sensing imagery (RSI) has achieved tremendous success under the large-scale labeled training data. However, multi-target domain adaptation building instance extraction (MD-BIE) is still a challenge task that involves transferring knowledge from a source domain to multiple unlabeled target domains, which poses various semantic gaps between and within multiple domains,e.g., style, illumination, resolution, density, scale, etc. Most current methods for single-target domain adaptation are not applicable to the more realistic MD-BIE task. To this end, we propose a novel Domain-common Approximation Learning (DAL) for both modelling intra-domain and inter-domain adaptation, thus obtaining robust MD-BIE. DAL contains three main modules: multi-domain style transfer (MST), multi-domain feature approximation (MFA), and multi-domain cascaded instance extraction (MCIE). To alleviate the semantic gaps between multiple domains for inter-domain adaptation, we first employ the MST to learn multiple target-domain-like features that preserve both the styles of target domains and the content of the source domain, and then use the MFA to approximate these features towards a central domain-common space, thus producing domain-common semantic representations. Moreover, we develop the MCIE with hierarchical extraction losses for intra-domain adaptation to extract precise building instance contours from the domain-common semantic representations, further eliminating the potential gaps within multiple domains. By co-learning these three modules in an end-to-end manner, the DAL bridges the semantic gaps between and within multiple domains. Extensive experiments on different popular MD-BIS tasks (SAB → Crowd & WHU, Crowd → SAB & WHU, SAB → Crowd & SAB & WHU and SAB → WHU) show that our DAL outperforms the current methods by a significant margin.

Hyperspectral image (HSI) classification is a core processing procedure in the remote sensing community, which has been recently well studied using vision transformers (ViTs). However, due to the high computational and memory complexities, existing transformer-based classification methods tend to restrict the spatial extent of the transformer to small cropped HSI patches instead of the whole HSI data, thus sacrificing the essential strength of transformers in long-range interaction modeling and overlooking the beneficial multiscale features in HSI data. Inspiringly, here we propose PSFormer, a novel pyramid superpixel transformer (PSFormer) method specifically for HSI classification, in order to make full use of the transformer to excavate multiscale local-global features in HSI data. Specifically, a progressive superpixel merging strategy is introduced to flexibly control the scale of feature maps. Furthermore, a unique transformer backbone design based on a spectral attention layer and a classification head with a gate mechanism are developed, to adaptively exploit valuable local-global information at different scales with low computational cost. Extensive experimental results on five widely used datasets demonstrate the superiority of PSFormer over other state-of-the-art networks. For the sake of reproducibility, the related code of the PSFormer method will be open-sourced at:https://github.com/immortal13.

Multimodal image matching is essential in image stitching, image fusion, change detection, and land cover mapping. However, the severe nonlinear radiometric distortion (NRD) and geometric distortions in multimodal images severely limit the accuracy of multimodal image matching, posing significant challenges to existing methods. Additionally, detector-based methods are prone to feature point offset issues in regions with substantial modal differences, which also hinder the subsequent fine registration and fusion of images. To address these challenges, we propose a guided refinement for detector-free multimodal image matching (GRiD) method, which weakens feature point offset issues by establishing pixel-level correspondences and utilizes reference points to guide and correct matches affected by NRD and geometric distortions. Specifically, we first introduce a detector-free framework to alleviate the feature point offset problem by directly finding corresponding pixels between images. Subsequently, to tackle NRD and geometric distortion in multimodal images, we design a guided correction module that establishes robust reference points (RPs) to guide the search for corresponding pixels in regions with significant modality differences. Moreover, to enhance RPs reliability, we incorporate a phase congruency module during the RPs confirmation stage to concentrate RPs around image edge structures. Finally, we perform finer localization on highly correlated corresponding pixels to obtain the optimized matches. We conduct extensive experiments on four multimodal image datasets to validate the effectiveness of the proposed approach. Experimental results demonstrate that our method can achieve sufficient and robust matches across various modality images and effectively suppress the feature point offset problem.

Hyperspectral image (HSI) qualities are limited by a mixture of Gaussian noise, impulse noise, stripes, and deadlines during the sensor imaging process, resulting in weak application performance. To enhance HSI qualities, methods based on convolutional neural networks have been successively applied to restore clean data from the observed data. However, the architecture of these methods lacks spectral and spatial constraints, and the convolution operators have limited receptive fields and inflexible model inferences. Thus, in this study, we propose an efficient end-to-end transformer, named HSI denoising transformer (Hider), for mixed HSI noise removal. First, a U-shaped 3-D transformer architecture is built for multiscale feature aggregation. Second, a multihead global spectral attention module within the spectral transformer block is designed to excavate information in different spectral patterns. Finally, an additional locally enhanced cross-spatial attention module within the spatial-spectral transformer block is constructed to build the long-range spatial relationship to avoid the high computational complexity of global spatial self-attention. Through the imposition of global correlations along spectrum and spatial self-similarity constraints on the transformer, our proposed Hider aims to capture long-range spatial contextual information and cluster objects with the same spectral pattern for HSI denoising. To verify the effectiveness and efficiency of Hider, we conducted extensive simulated and real experiments. The denoising results on both simulated and real-world datasets show that Hider achieves superior evaluation metrics and visual assessments compared with other state-of-the-art methods.

Research has shown that most of the Earth’s surface is covered by clouds, which have reduced the usability of optical remote sensing images. Therefore, it is critical to detect clouds quickly and accurately. Cloud detection methods based on deep learning have been widely studied in recent years. However, the existing methods still face a significant challenge that thin clouds are often translucent and easily missed. To enhance the detection of thin clouds by convolutional neural networks, we propose a cloud feature difference (CFD) loss, which gathers the samples in thin clouds, thick clouds and subsurface as a sample group, namely Cloud-Triplet. By measuring the features difference between samples, the CFD loss forces the network to model discriminative features during training, thus improving the ability of detecting thin clouds. Experiments show that our proposed CFD loss is effective in enhancing the detection of clouds.

Hyperspectral image (HSI) denoising has an essential effect on HSI analysis and interpretation. In recent years, denoising methods based on convolutional neural networks (CNNs) have made great progress. However, the convolution kernel in the CNN model is content-independent, and the ability to capture long-distance correlation is weak, which leads to spectral distortion and edge blurring. To address this problem, we propose a spatial–spectral transformer network for HSI denoising, which introduces the shifted window-based transformer method to denoise HSIs by modeling image content correlation while preserving the local inductive bias. In detail, to jointly explore the spatial–spectral features, we first formulate the spatial–spectral cubes as network input, which are composed of the current band and its adjacent fixed K-bands. Second, these spatial–spectral cubes are forwarded to cascaded hyperspectral transformer blocks (HTBs) with skip connection for deep feature extraction. The HTB contains multiple transformer layers based on different window partitioning, which not only reduces memory cost, but also enhances the feature extraction capability. Finally, for the denoised${B}$spatial–spectral cubes, we average the pixels of overlapping spectral bands to generate a complete HSI. Furthermore, we introduce the total variation (TV) to preserve the smoothness structures of HSIs. The experimental results on simulated and real data indicate that the proposed spatial-spectral transformer denoising (SSTD) is superior to other mainstream learning-based HSI denoising algorithms.

Hyperspectral image (HSI) classification with limited training samples has been well studied in recent years. Among them, the few-shot learning (FSL) technique demonstrates excellent processing capability under limited labeled samples. Nevertheless, the current FSL-based works provide scarce attention to effective class prototypes and metric types, resulting in high generalization error and poor interpretation during the cross-domain testing phase. A dual-adjustment mode-based cross-domain meta-learning (DMCM) method for few-shot HSI classification is proposed to tackle this issue. Specifically, a three-dimensional ghost attention network (TGAN) with strong learning capability without massive parameters is first constructed. Meanwhile, a dual-adjustment mode comprising intra-correction (IC) and inter-alignment (IA) learning strategies is then adopted to solve domain shift issue via episode-level meta tasks, where IC and IA focus on effective class prototypes and data distribution differences between domains, respectively. Afterward, considering that the traditional Euclidean distance metric is insensitive to the distribution of within-class samples, the class-covariance metric is employed to account for the distribution in feature space of each class to optimize decision boundary and alleviate the misclassification problem. Extensive experiments on three publicly available target hyperspectral datasets demonstrate the effectiveness of the proposed method in comparison with other state-of-the-art methods. The codes will be available on the website: https://github.com/HlEvag/DMCM.git.

Feature matching is a fundamental and critical task for remote sensing image registration. However, numerous outliers (false matches) harm the feature point neighborhood structure due to the view transformation from the camera. Meanwhile, unknown local distortion obscures the distinction between inliers (correct matches) and outliers. To solve these problems, we propose a global-local consistency network (GLoCNet) for feature matching to exclude the interference of outliers under various transformation patterns and provide stable neighborhood support for the similarity metric of feature points. Specifically, a global transformation consistency module is proposed to obtain a neighbor pool by exploiting the compact nature of the inlier distribution under different transformation patterns. In addition, feature points interact with information through the local neighborhood consistency module in center-based graph construction. Finally, the difference between inliers and outliers is increased by dynamically adjusting the upper limitation distance of the outlier and suppressing its effect in the neighborhood. We conducted rich experiments on extensive datasets to verify the effectiveness of the proposed method. The experimental results illustrate that the proposed GLoCNet can effectively handle numerous outliers and achieve satisfactory registration results under various transformation patterns.

As an important yet challenging task in Earth observation, change detection (CD) is undergoing a technological revolution, given the broadening application of deep learning. Nevertheless, existing deep learning-based CD methods still suffer from two salient issues: 1) incomplete temporal modeling, and 2) space-time coupling. In view of these issues, we propose a more explicit and sophisticated modeling of time and accordingly establish a pair-to-video change detection (P2V-CD) framework. First, a pseudo transition video that carries rich temporal information is constructed from the input image pair, interpreting CD as a problem of video understanding. Then, two decoupled encoders are utilized to spatially and temporally recognize the type of transition, and the encoders are laterally connected for mutual promotion. Furthermore, the deep supervision technique is applied to accelerate the model training. We illustrate experimentally that the P2V-CD method compares favorably to other state-of-the-art CD approaches in terms of both the visual effect and the evaluation metrics, with a moderate model size and relatively lower computational overhead. Extensive feature map visualization experiments demonstrate how our method works beyond making contrasts between bi-temporal images. Source code is available at https://github.com/Bobholamovic/CDLab.

Band selection (BS) reduces effectively the spectral dimension of a hyperspectral image (HSI) by selecting relatively few representative bands, which allows efficient processing in subsequent tasks. Existing unsupervised BS methods based on subspace clustering are built on matrix-based models, where each band is reshaped as a vector. They encode the correlation of data only in the spectral mode (dimension) and neglect strong correlations between different modes, i.e., spatial modes and spectral mode. Another issue is that the subspace representation of bands is performed in the raw data space, where the dimension is often excessively high, resulting in a less efficient and less robust performance. To address these issues, in this article, we propose a tensor-based subspace clustering model for hyperspectral BS. Our model is developed on the well-known Tucker decomposition. The three factor matrices and a core tensor in our model encode jointly the multimode correlations of HSI, avoiding effectively to destroy the tensor structure and information loss. In addition, we propose well-motivated heterogeneous regularizations (HRs) on the factor matrices by taking into account the important local and global properties of HSI along three dimensions, which facilitates the learning of the intrinsic cluster structure of bands in the low-dimensional subspaces. Instead of learning the correlations of bands in the original domain, a common way for the matrix-based models, our model learns naturally the band correlations in a low-dimensional latent feature space, which is derived by the projections of two factor matrices associated with spatial dimensions, leading to a computationally efficient model. More importantly, the latent feature space is learned in a unified framework. We also develop an efficient algorithm to solve the resulting model. Experimental results on benchmark datasets demonstrate that our model yields improved performance compared to the state-of-the-art.

Change detection (CD), as one of the central problems in Earth observation, has attracted a lot of research interest over recent decades. Due to the rapid development of satellite sensors in recent years, we have witnessed an enrichment of the CD source data with the availability of very-high-resolution (VHR) multispectral imagery, which provides abundant change clues. However, precisely locating real changed areas still remains a challenge. In this article, we propose an end-to-end superpixel-enhanced CD network (ESCNet) for VHR images, which combines differentiable superpixel segmentation and a deep convolutional neural network (DCNN). Two weight-sharing superpixel sampling networks (SSNs) are tailored for the feature extraction and superpixel segmentation of bitemporal image pairs. A UNet-based Siamese neural network is then employed to mine the different information. The superpixels are then leveraged to reduce the latent noise in the pixel-level feature maps while preserving the edges, where a novel superpixelation module is used to serve this purpose. Furthermore, to compensate for the dependence on the number of superpixels, we propose an innovative adaptive superpixel merging (ASM) module, which has a concise form and is fully differentiable. A pixel-level refinement module making use of the multilevel decoded features is also appended to the end of the framework. Experiments on two public datasets confirmed the superiority of ESCNet compared to the traditional and state-of-the-art (SOTA) deep learning-based CD (DLCD) methods.

The main purpose of anomaly detection in hyperspectral images is to detect targets that are different from their surroundings. With the development of deep learning technology, anomaly detection in hyperspectral images using deep neural networks has drawn great attention in recent years. However, most of the existing deep learning-based anomaly detection algorithms fail to consider the low-rank properties of the background and underutilize the rich spectral information of the image. In this paper, we propose a novel autoencoder in autoencoder network based on low-rank module embedding for anomaly detection in hyperspectral images. Firstly, the background is purified by using the low-rank module (LRM), and then the image background is reconstructed by using autoencoder in autoencoder network (AiANet), which is a spatial-spectral dual encoding-decoding network. AiANet fully considers the differences between anomalies and backgrounds in spatial and spectral dimensions to better reconstruct the background. Finally, the anomaly appears in images as reconstruction errors. Our proposed method effectively exploits the low-rank property of the backgrounds and makes full use of the spectral information to extract pure backgrounds to separate anomalies. Experiments on two real hyperspectral images demonstrate that the proposed method outperforms the other competitors.

The primary purpose of anomaly detection in hyperspectral images (HSI) is to detect different anomaly targets from their surrounding backgrounds. Recently, anomaly detection has been well developed by deep learning technology. However, previous works utilize deep neural networks as a feature extractor followed by a traditional detector to detect anomalies, which cannot separate background and anomaly effectively. In this paper, we try to extract low-rank background features using neural networks to make full use of the low-rank properties of the background and then reconstruct the background with these features to directly separate the anomaly from the background, so we propose an multi-scale background reconstruction network with low-rank embedding (MSBRNet) for anomaly detection in HSI. Firstly, we use a low-rank background features extraction module (LBM) to extract low-rank background features. Then the background is reconstructed using a multi-scale background reconstruction module (MBRM). Finally, we calculate the mean square error of the input image and the output background to measure the effect of background reconstruction, and anomalies appear as reconstruction errors. Experiments on two publicly available experimental datasets demonstrate the significant advantage of the proposed method over other competitors.

Land-cover mapping is a pivotal pathway for Earth observation. Nevertheless, the lack of labeled data and the domain gap of different mapping regions are still challenges inhibiting the large-scale implementation of common mapping methods. In this article, a multi-stage pseudo-label iteration framework is presented for the semi-supervised land-cover mapping track of the 2022 Data Fusion Contest (DFC-SLM). The proposed framework combines the multi-stage training process with the pseudo-label technique to tackle the issues of large-scale land-cover mapping task when limited labeled samples are available. Firstly, the multi-stage training process promotes to sufficiently explore the discriminative and robust features of land covers with limited labeled data, where the proportion of samples in confusing classes gradually increases. Secondly, the pseudo-label technique generates extra supervision information from the unlabeled imagery. Overall, experimental results obtained from several cities in France achieved a mIoU of 52.96%, and won 2nd place on the final leaderboard of the 2022 DFC-SLM.

In order to unmix the hyperspectral imagery (HSI) with better performance, this letter proposes a correntropy-based autoencoder-like nonnegative matrix factorization (NMF) (CANMF) with total variation (CANMF-TV) method. NMF is extensively applied to unmix the mixed pixels. However, it only reconstructs the original data from the abundances in endmember space. To directly project the original data space into the endmember space, and then achieve the abundance matrix, we first exploit an autoencoder-like NMF for hyperspectral unmixing, which integrates bothdecoderandencoder. Considering that HSI is typically degraded by noise, the correntropy-induced metric (CIM) is introduced to construct a CANMF model. In addition, TV regularizer is imposed into the CANMF model so as to preserve the spatial-contextual information by promoting the piecewise smoothness of abundances. Finally, a series of experiments are conducted on both synthetic and real data sets, demonstrating the effectiveness of the proposed CANMF-TV method over comparison.

In the above article[1], it should be noted that the second value of each column in the last row ofTable I(i.e., 0.52%, 0.35%, 0.51%, and 0.72%) is calculated using average deviation rather than standard deviation. In order to be consistent with the title ofTable Iin[1], the corresponding standard deviations are provided here.

Non-local low-rank tensor approximation has been developed as a state-of-the-art method for hyperspectral image (HSI) restoration, which includes the tasks of denoising, compressed HSI reconstruction and inpainting. Unfortunately, while its restoration performance benefits from more spectral bands, its runtime also substantially increases. In this paper, we claim that the HSI lies in a global spectral low-rank subspace, and the spectral subspaces of each full band patch group should lie in this global low-rank subspace. This motivates us to propose a unified paradigm combining the spatial and spectral properties for HSI restoration. The proposed paradigm enjoys performance superiority from the non-local spatial denoising and light computation complexity from the low-rank orthogonal basis exploration. An efficient alternating minimization algorithm with rank adaptation is developed. It is done by first solving a fidelity term-related problem for the update of a latent input image, and then learning a low-dimensional orthogonal basis and the related reduced image from the latent input image. Subsequently, non-local low-rank denoising is developed to refine the reduced image and orthogonal basis iteratively. Finally, the experiments on HSI denoising, compressed reconstruction, and inpainting tasks, with both simulated and real datasets, demonstrate its superiority with respect to state-of-the-art HSI restoration methods.

Hyperspectral images (HSIs) can finely discriminate distinct objects with a high spectral resolution, and they are widely employed in various applications. However, mixed noise severely degrades the quality of HSI and restricts the performance of subsequent tasks. As one of the critical pre-processing steps, HSI denoising has been developed rapidly, among which low-rank (LR) prior-based methods have achieved superior performance. Nevertheless, the existing approaches frequently fail to completely remove noise and reconstruct high-quality HSIs when tackling complicated mixed noise with multiple types of high-intensity stripe noise. To solve this problem, we propose an HSI denoising and destriping method based on anisotropic spatial and spectral total variation regularized double low-rank approximation (ATVDLR). The double low-rank approximation framework is devoted to separating the clean image from the mixed noise by exploiting both the global correlations of HSI tensor and the LR structure of stripe noise. Furthermore, the anisotropic spatial and spectral total variation regularization is introduced to preserve the spatial–spectral smoothness of HSI and the directional feature of stripes, thereby further suppressing high-level stripes and Gaussian noise. Finally, the alternating direction method of multipliers (ADMM) technique is designed to solve the proposed ATVDLR model. Extensive experimental results indicate that the proposed method outperforms other state-of-the-art techniques in multi-type high-intensity mixed noise reduction and image structural information protection, and has superior performance in complex mixed noise removal of real Gaofen-5 HSIs.

Hyperspectral image (HSI) mixed noise removal is a fundamental problem and an important preprocessing step in remote sensing fields. The low-rank approximation-based methods have been verified effective to encode the global spectral correlation for HSI denoising. However, due to the large scale and complexity of real HSI, previous low-rank HSI denoising techniques encounter several problems, including coarse rank approximation (such as nuclear norm), the high computational cost of singular value decomposition (SVD) (such as Schatten$p$-norm), and adaptive rank selection (such as low-rank factorization). In this article, two novel factor group sparsity-regularized nonconvex low-rank approximation (FGSLR) methods are introduced for HSI denoising, which can simultaneously overcome the mentioned issues of previous works. The FGSLR methods capture the spectral correlation via low-rank factorization, meanwhile utilizing factor group sparsity regularization to further enhance the low-rank property. It is SVD-free and robust to rank selection. Moreover, FGSLR is equivalent to Schatten$p$-norm approximation (Theorem 1), and thus FGSLR is tighter than the nuclear norm in terms of rank approximation. To preserve the spatial information of HSI in the denoising process, the total variation regularization is also incorporated into the proposed FGSLR models. Specifically, the proximal alternating minimization is designed to solve the proposed FGSLR models. Experimental results have demonstrated that the proposed FGSLR methods significantly outperform existing low-rank approximation-based HSI denoising methods.

Although deep learning has made semantic segmentation of very-high-resolution (VHR) remote sensing (RS) images practical and efficient, its large-scale application is still limited. Given the diversity of imaging sensors, acquisition conditions, and regional styles, a deep learning network well-trained on one source domain dataset often suffers from drastic performance drops when applied to other target domain datasets. Thus, we propose a novel end-to-end mutual information domain adaptation network (MIDANet) that can shift between semantic segmentation domains by integrating multitask learning in the convolutional neural networks within an entropy adversarial learning (EAL) framework. Through the joint learning of semantic segmentation and elevation estimation, the features extracted by MIDANet can concentrate more on the elevation clues while dropping the domain-variant information (i.e., texture, spectral information). First, one encoder is applied to excavate general semantic features. Two decoders that share the same architecture are used to perform pixel-level classification and digital surface model (DSM) regression. Second, feature interaction modules (FIMs) and a mutual information attention unit (MIAU) are designed to mine the latent relationships between the two tasks and enhance their feature representations. Finally, a final MIDANet is obtained for semantic segmentation that does not require any semantic segmentation labels in the target domain after the adversarial learning of the classification entropy at the output level. Extensive comparative experiments and ablation studies were conducted on the International Society for Photogrammetry and Remote Sensing (ISPRS) Potsdam and Vaihingen test datasets. The results show that MIDANet outperforms other state-of-the-art domain adaptation (DA) methods in both evaluation metrics and visual assessment.

Clustering algorithms play an essential and unique role in classification tasks, especially when annotated data are unavailable or very scarce. Current clustering approaches in remote sensing are mostly designed for a single data source, such as hyperspectral image (HSI), while, nowadays, multisensor data are being routinely acquired. In this article, we propose a multiview subspace clustering model that exploits effectively the rich information from multiple features extracted either from a single data source (HSI) or from multiple sources that we call generically multiviews of the same scene. An important novelty of our approach is that it integrates local and nonlocal spatial information from each view in a unified framework. Our model learns a common intrinsic cluster structure from view-specific subspace representations by a new decomposition-based scheme. In addition, we develop innovative manifold-based spatial regularization as a hybrid hypergraph, which merges local and nonlocal spatial context and improves, thereby, the learning of view-specific structures. We develop an efficient algorithm to solve the resulting optimization problem. Extensive experiments on real data sets demonstrate the superior clustering performance over the state of the art.

Band selection, which removes irrelevant bands from hyperspectral images (HSIs) and keeps essential spectral information contained in a relatively few bands, allows huge savings in data storage, computation time, and imaging hardware. In this article, we propose a novel structural subspace clustering (STSC) method for hyperspectral band selection, which leverages the self-representation property of data and structural prior information to learn the cluster structure of bands. Particularly, we propose a general clustering model where the coarse coefficients matrix derived from a self-representation model is decomposed as a combination of a desirable coefficients matrix and a sparse matrix. This strategy adaptively adjusts the coarse coefficients matrix to learn the intrinsic data structure in low-dimensional subspaces. To guide this learning process, we introduce a structural regularization approach which makes use of the prior information about local and global properties of spectral bands. Moreover, we incorporate also prior knowledge about the dictionary, which demonstrates to yield a better clustering performance. We develop an adaptive method to estimate the number of selected bands by analyzing eigenvalue gaps of Laplacian matrix. To solve the resulting model, an efficient algorithm based on alternating direction method of multipliers (ADMMs) is developed. Extensive experiments on benchmark HSIs show that our method outperforms the state-of-the-art band selection methods.

Sparse subspace clustering (SSC) has emerged as an effective approach for the automatic analysis of hyperspectral images (HSI). Traditional SSC-based approaches employ the input HSI data as a dictionary of atoms, in terms of which all the data samples are linearly represented. This leads to highly redundant dictionaries of huge size, and the computational complexity of the resulting optimization problems becomes prohibitive for large-scale data. In this article, we propose a scalable subspace clustering method, which integrates the learning of a concise dictionary and robust subspace representation in a unified model. This reduces significantly the size of the involved optimization problems. We introduce a new adaptive spatial regularization for the representation coefficients, which incorporates spatial information of HSI and improves the robustness of the model to noise. We derive an effective solver based on alternating minimization and alternating direction method of multipliers (ADMMs) to solve the resulting optimization problem. Experimental results on four representative hyperspectral images show the effectiveness of the proposed method and excellent clustering performance relative to the state of the art.

Hyperspectral images (HSIs) are often contaminated by several types of noise, which significantly limits the accuracy of subsequent applications. Recently, low-rank modeling based on tensor singular value decomposition (T-SVD) has achieved great success in HSI restoration. Most of them use the convex and nonconvex surrogates of the tensor rank, which cannot well approximate the tensor singular values and obtain suboptimal restored results. We suggest a novel HSI restoration model by introducing a fibered rank constrained tensor restoration framework with an embedded plug-and-play (PnP)-based regularization (FRCTR-PnP). More precisely, instead of using the convex and nonconvex surrogates to approximate the fibered rank, the proposed model directly constrains the tensor fibered rank of the solution, leading to a better approximation to the original image. Since exploiting the low-fibered-rankness of HSI is mainly to capture the global structure, we further employ an implicit PnP-based regularization to preserve the image details. Particularly, the above two building blocks are complementary to each other, rather than isolated and uncorrelated. Based on the alternating direction multiplier method (ADMM), we propose an efficient algorithm to tackle the proposed model. For robustness, we develop a three-directional randomized T-SVD (3DRT-SVD), which preserves the intrinsic structure of the clean HSI and removes partial noise by projecting the HSI onto a low-dimensional essential subspace. Extensive experimental results including simulated and real data demonstrate that the proposed method achieves superior performance over compared methods in terms of quantitative evaluation and visual inspection.

Spatiotemporal fusion (STF) is an effective solution to promote the application of remote sensing images, given that the tradeoff between the temporal resolution and the spatial resolution is ubiquitous in the production of remote sensing images. However, cloud coverage makes it difficult to obtain dense cloud-free Landsat–Moderate Resolution Imaging Spectroradiometer (MODIS) image pairs on the timeline, which limits the application of existing STF methods. Considering the lack of prior image pairs and the huge spatial resolution gap between Landsat and MODIS images, this article presents a novel remote sensing image STF method based on a generative adversarial network to handle one Landsat–MODIS prior image pair case (OPGAN), which contains a generator and a discriminator simultaneously trained in a min–max game. OPGAN is built based on the STF observation model that learns the base information from the prior Landsat image and then captures temporal change (TC) information from a difference image constructed from MODIS images collected at times 1 and 2 and sensor difference information from the difference image between Landsat and MODIS images at time 1. They are combined together to reconstruct the Landsat image at time 2 at both high spatial and high temporal resolution. Moreover, a change loss is proposed to further improve the accuracy of TC prediction. Extensive experiments on the STF dataset illustrate that the proposed OPGAN method can obtain more accurate prediction of spatial information and TCs in the case of insufficient prior information.

Can we automatically learn meaningful semantic feature representations when training labels are absent? Several recent unsupervised deep learning approaches have attempted to tackle this problem by solving the data reconstruction task. However, these methods can easily latch on low-level features. To solve this problem, we propose an end-to-end spectral–spatial semantic feature learning network (S3FN) for unsupervised deep semantic feature extraction (FE) from hyperspectral images (HSIs). Our main idea is to learn spectral-spatial features from high-level semantic perspective. First, we utilize the feature transformation to obtain two feature descriptions of the same source data from different views. Then, we propose the spectral–spatial feature learning network to project the two feature descriptions into the deep embedding space. Subsequently, a contrastive loss function is introduced to align the two projected features, which should have the same implied semantic meaning. The proposed S3FN learns the spectral and spatial features separately, and then merges them. Finally, the learned spectral–spatial features by S3FN are processed by a classifier to evaluate their effectiveness. Experimental results on three publicly available HSI datasets show that our proposed S3FN can produce promising classification results with a lower time cost than other state-of-the-art (SOTA) deep learning-based unsupervised FE methods.

Hyperspectral images (HSIs) have a wealth of applications in many areas, due to their fine spectral discrimination ability. However, in the practical imaging process, HSIs are often degraded by a mixture of various types of noise, for example, Gaussian noise, impulse noise, dead pixels, dead lines, and stripe noise. Low-rank matrix decomposition theory has been widely used in HSI denoising, and has achieved competitive results by modeling the impulse noise, dead pixels, dead lines, and stripe noise as sparse components. However, the existing low-rank-based methods for HSI denoising cannot completely remove stripe noise when the stripe noise is no longer sparse. In this article, we extend the HSI observation model and propose a double low-rank (DLR) matrix decomposition method for HSI denoising and destriping. By simultaneously exploring the low-rank characteristic of the lexicographically ordered noise-free HSI and the low-rank structure of the stripe noise on each band of the HSI, the two low-rank constraints are formulated into one unified framework, to achieve separation of the noise-free HSI, stripe noise, and other mixed noise. The proposed DLR model is then solved by the augmented Lagrange multiplier (ALM) algorithm efficiently. Both simulation and real HSI data experiments were carried out to verify the superiority of the proposed DLR method.

The lack of labels is one of the major challenges in hyperspectral image (HSI) classification. Widely used Deep Learning (DL) models such as convolutional neural networks (CNNs) experience serious performance degradation when training samples are limited. In contrast, graph convolutional networks (GCNs) can simultaneously exploit the insufficient labeled data and massive unlabeled data of HSI in a semisupervised learning fashion. However, in order to reduce computational cost and mitigate noise, existing GCN-based classification methods usually perform superpixel segmentation as a preprocessing step and implement feature extraction as well as node classification on the predefined superpixel graph, where one superpixel might incorporate pixels with different labels. Moreover, the local spectral–spatial information within superpixels is generally ignored. To alleviate these two issues, we propose an end-to-end mixhop superpixel-based GCN (EMS-GCN) framework for HSI classification. Specifically, we first introduce the differentiable superpixel segmentation algorithm to map the pixel representations into a superpixel feature space, which allows refining the superpixel boundary with the training of the network. After that, a superpixel graph is constructed and fed into a novel mixhop superpixel-based GCN, where both the local information within superpixels and long-range information among superpixels are extracted, while the structure of the superpixel graph is updated at the same time. Finally, the enhanced superpixel representations are mapped back into a pixel feature space to conduct pixel-wise classification. Extensive experiments demonstrate the effectiveness of the proposed EMS-GCN method compared with other state-of-the-art methods.

Convolutional neural network (CNN)-based feature fusion of RGB and auxiliary remote sensing data is known to enable improved semantic segmentation. However, such fusion is challengeable because of the substantial variance in data characteristics and quality (e.g., data uncertainties and misalignment) between two modality data. In this article, we propose a unified gather-to-guide network (G2GNet) for remote sensing semantic segmentation of RGB and auxiliary data. The key aspect of the proposed architecture is a novel gather-to-guide module (G2GM) that consists of a feature gatherer and a feature guider. The feature gatherer generates a set of cross-modal descriptors by absorbing the complementary merits of RGB and auxiliary modality data. The feature guider calibrates the RGB feature response by using the channel-wise guide weights extracted from the cross-modal descriptors. In this way, the G2GM can perform RGB feature calibration with different modality data in a gather-to-guide fashion, thus preserving the informative features while suppressing redundant and noisy information. Extensive experiments conducted on two benchmark datasets show that the proposed G2GNet is robust to data uncertainties while also improving the semantic segmentation performance of RGB and auxiliary remote sensing data.

Currently, the convolutional neural networks (CNNs) have become the mainstream methods for hyperspectral image (HSI) classification, due to their powerful ability to extract local features. However, CNNs fail to effectively and efficiently capture the long-range contextual information and diagnostic spectral information of HSI. In contrast, the leading-edge vision transformers are capable of capturing long-range dependencies and processing sequential data such as spectral signatures. Nevertheless, pre-existing transformer-based classification methods generally generate inaccurate token embeddings from a single spectral or spatial dimension of raw HSIs and encounter difficulty modeling locality with insufficient training data. To mitigate these limitations, we propose a novel local-enhanced spectral-spatial transformer method (i.e., LESSFormer) specifically devised for HSI classification. Two effective and efficient modules are designed in LESSFormer, i.e., the HSI2Token module and the local-enhanced transformer encoder. The former is devised to transform HSI into the adaptive spectral-spatial tokens, and the latter is built to further enhance the representation ability of tokens by reinforcing the local information explicitly with a simple attention mask as well as retaining the long-range information in the meantime. Extensive experimental results on the new Xiong’an dataset and the widely used Pavia University and Houston University datasets have shown the superiority of LESSFormer over other state-of-the-art networks.

Buildings constitute one of the most important landscapes in remote sensing (RS) images and have been broadly analyzed in a wide range of applications from urban planning to other socioeconomic studies. As very-high-resolution (VHR) RS imagery becomes more accessible, the current building extraction methods are confronted with the challenges of the diverse appearances, various scales, and complicated structures of buildings in complex scenes. With the development of context-aware deep learning methods, it has been proven by numerous works that capturing contextual information can offer spatial relation cues for robust recognition and detection of the objects. In this article, we propose a novel local-global dual-stream network (DS-Net) that adaptively captures local and long-range information for the accurate mapping of building rooftops in VHR RS images. The local branch and the global branch of DS-Net work in a complementary manner to each other with different fields of view on the input image. Through a well-defined dual-stream architecture, DS-Net learns hierarchical representations for both the local and global branches, and a deep feature sharing strategy is further developed to enforce more collaborative integration of the two branches. Extensive experiments were carried out to verify the effectiveness of our model on three widely used VHR RS data sets: the Massachusetts buildings data set, the Inria Aerial Image Labeling data set, and the DeepGlobe Building Detection Challenge data set. Empirically, the proposed DS-Net achieves competitive or superior performance compared with the current state-of-the-art methods in terms of quantitative measures and visual evaluations.

Change detection is a geospatial application for social good whose development is restricted by a slow-growing labeling technology and outdated classification labels for remotely sensed images. In this paper, a change cross-detection method based on label improvements and multi-model fusion is proposed for Multi-temporal Semantic change Detection (MSD) with weak, noisy, and low-resolution labels. For unmatched labels, a Siamese Skip_FCN network is proposed to generate preliminary labels at high-resolution. Subsequently, a multi-model fusion method is introduced to perform accurate and stable land cover classification. In addition, a cross-detection structure is used to generate high precision change maps and a post-processing step further improves the final results. In the track MSD of the 2021 Data Fusion Contest (DFC21-MSD), the proposed method achieved a mean intersection over union (mIoU) of 70.25% in phase 1 and 67.72% in phase 2, ranking first in both phases [1].

In this paper, a multi-model fusion framework is proposed for automatic detection of settlements without electricity (DSE) based on the multimodal and multitemporal remote sensing data. To settle the problems of data noise and data redundancy, the data preprocessing step, which consists of band selection, cloud removal, grayscale stretch and data augmentation, is firstly applied. Two models of single-task and dual-task are further constructed for DSE. The single-task model builds a global context convolutional neural network (GC-CNN) for the detection of settlements without electricity and the dual-task model employs the GC-CNN for settlement detection and the random forest classifier for electricity detection. Moreover, a model fusion principle and a post-processing method is designed to integrate and improve the results above, thus producing the final segmentation result. Verified through the competition website, the proposed method achieved a F1-score of 0.8806, ranking second in the first track of 2021 IEEE GRSS Data Fusion Contest.

Drought, leading to water supply deficits and vegetation growth anomalies, is one of the most serious but least understood natural phenomenon. Different vegetation types vary in sensitivity to drought and exhibit diverse kinds of responses. However, few studies have focused on the sensitivity and the response of various types of vegetation. Thus, in this paper, a new Drought Index based on land cover Types Response Adjustment (TRADI) is proposed based on response adjustment for vegetation, to gauge the response factors among various vegetation types and provide a comprehensive assessment of drought conditions. Multisource data from the Moderate Resolution Imaging Spectroradiometer (MODIS), the Tropical Rainfall Measuring Mission (TRMM) and the Global Land Data Assimilation System (GLDAS) was used to derive drought condition indices (CIs) and construct the TRADI. A correlation analysis between TRADI and Palmer Drought Severity Index (PDSI) monitoring results for typical drought events in the CONtinental United States (CONUS) was carried out to evaluate the reliability and efficiency of the proposed method. These results showed that drought was underestimated or overestimated by varying degrees when using traditional condition-based indices (CIs), but the TRADI can correct these misestimates and improve the drought monitoring accuracy.

Supervised hyperspectral image (HSI) classification has been widely studied and used in many different applications. However, the performance of the supervised classifiers, including the traditional machine learning methods and the deep neural networks, is significantly affected by the inaccurate labeling of training samples, which is a common problem in HSI supervised classification. In this article, we propose a superpixel guided sample selection neural network (S3Net) framework with end-to-end training for handling noisy labels in HSI classification. It includes two stages: sample selection and sample correction. In sample selection, a sample with a small training loss has a higher probability of being the correct label and hence selected from the noisy labels for model training. In order to avoid the error propagation caused by the noisy labels, we utilize a cross-selection update strategy that exchanges selected samples between two neural networks during conventional loss backpropagation. Sample selection is a pruning process, which may cause insufficient training sample problem in HSI classification. To solve this problem, we propose the sample correction strategy to correct the noisy labels by propagating clean label information in the homogeneous regions obtained by superpixel. Experimental results on three public HSI data sets demonstrate the effectiveness of the proposed S3Net framework when handling noisy labels.

Clustering is a common method for hyperspectral image (HSI) interpretation in the case of no labeled samples. Many subspace clustering methods have now been proposed for HSIs and have obtained remarkable success. However, because of the prohibitively large computational complexity induced by the self-dictionary representation, these methods suffer from the scalability issue and are ineffective for large HSIs. In this article, to address this issue, we focus on a scalable subspace clustering scheme and introduce the recently developed sketched subspace clustering (sketched-SC) model to HSI. The sketched-SC model is computationally inexpensive and is suitable for the large HSI clustering task as it constructs a compact yet expressive dictionary. However, several problems degrade the performance of sketched-SC, i.e., the inadequate mining of the structural information and no consideration of spatial information. In view of this, a novel scalable nonlocal means regularized sketched reweighted sparse and low-rank (NL-SSLR) SC algorithm is proposed for use with large HSIs. On the one hand, the SSLR representation model is constructed to explore the underlying local and global structural information of the HSIs at the same time. On the other hand, the nonlocal means regularization is used to fully explore the spatial correlation information and better account for the self-similarity of HSIs, to further boost the clustering performance. The experimental results obtained on two well-known hyperspectral data sets corroborate the superiority of the proposed algorithm over the other state-of-the-art HSI clustering methods.

Hyperspectral image (HSI) clustering is extremely challenging because of the complexity of the image structure. Recently, the subspace clustering algorithms have achieved competitive performance for HSIs. However, these methods generally are computationally complex and time-and-memory-consuming, given their reliance on large-scale adjacency matrix learning and graph segmentation, which limits their application to large HSIs and reduces their attractiveness in real applications. In this article, in view of this, two novel sparsity-based clustering algorithms are proposed for large HSIs, named sparse coding-based clustering (SCC) and joint SCC (JSCC). To the best of our knowledge, we are the first to use the sparse representation recovery residual to cluster HSIs. Based on a structured dictionary constructed by$k$-means and$k$-nearest neighbor (KNN), an SCC model is constructed to cluster HSIs according to the recovery residual minimization criterion. By dealing with a pixel-wise sparse recovery problem instead of the large-scale graph optimization problem of the whole image, the computational complexity and the time-and-memory cost are reduced to a large degree, which makes sense for practical applications. Then, by introducing the super-pixel neighborhood, a JSCC model is constructed to better explore the interpixel correlation of HSIs and further improve the clustering performance. The proposed algorithms were verified on three widely used HSIs. All the three experiments confirm the effectiveness of the proposed algorithms, which can be considered as competitive tools for use with large HSIs.

Due to technological limitations and budget constraints, spatiotemporal fusion is considered a promising way to deal with the tradeoff between the temporal and spatial resolutions of remote sensing images. Furthermore, the generative adversarial network (GAN) has shown its capability in a variety of applications. This article presents a remote sensing image spatiotemporal fusion method using a GAN (STFGAN), which adopts a two-stage framework with an end-to-end image fusion GAN (IFGAN) for each stage. The IFGAN contains a generator and a discriminator in competition with each other under the guidance of the optimization function. Considering the huge spatial resolution gap between the high-spatial, low-temporal (HSLT) resolution Landsat imagery and the corresponding low-spatial, high-temporal (LSHT) resolution MODIS imagery, a feature-level fusion strategy is adopted. Specifically, for the generator, we first super-resolve the MODIS images while also extracting the high-frequency features of the Landsat images. Finally, we integrate the features from the MODIS and Landsat images. STFGAN is able to learn an end-to-end mapping between the Landsat-MODIS image pairs and predicts the Landsat-like image for a prediction date by considering all the bands. STFGAN significantly improves the accuracy of phenological change and land-cover-type change prediction with the help of residual blocks and two prior Landsat-MODIS image pairs. To examine the performance of the proposed STFGAN method, experiments were conducted on three representative Landsat-MODIS data sets. The results clearly illustrate the effectiveness of the proposed method.

Due to the physical limitations of the imaging devices, hyperspectral images (HSIs) are commonly distorted by a mixture of Gaussian noise, impulse noise, stripes, and dead lines, leading to the decline in the performance of unmixing, classification, and other subsequent applications. In this paper, we propose a novel end-to-end low-rank spatial-spectral network (LR-Net) for the removal of the hybrid noise in HSIs. By integrating the low-rank physical property into a deep convolutional neural network (DCNN), the proposed LR-Net simultaneously enjoys the strong feature representation ability from DCNN and the implicit physical constraint of clean HSIs. Firstly, spatial-spectral atrous blocks (SSABs) are built to exploit spatial-spectral features of HSIs. Secondly, these spatial-spectral features are forwarded to a multi-atrous block (MAB) to aggregate the context in different receptive fields. Thirdly, the contextual features and spatial-spectral features from different levels are concatenated before being fed into a plug-and-play low-rank module (LRM) for feature reconstruction. With the help of the LRM, the workflow of low-rank matrix reconstruction can be streamlined in a differentiable manner. Finally, the low-rank features are utilized to capture the latent semantic relationships of the HSIs to recover clean HSIs. Extensive experiments on both simulated and real-world datasets were conducted. The experimental results show that the LR-Net outperforms other state-of-the-art denoising methods in terms of evaluation metrics and visual assessments. Particularly, through the collaborative integration of DCNNs and the low-rank property, the LR-Net shows strong stability and capacity for generalization.

Sparse subspace clustering (SSC) has achieved the state-of-the-art performance in clustering of hyperspectral images. However, the computational complexity of SSC-based methods is prohibitive for large-scale problems. We propose a large-scale SSC-based method, which processes efficiently large-scale HSIs without sacrificing the clustering accuracy. The proposed approach incorporates sketching of the self-representation dictionary reducing thereby largely the number of optimization variables. In addition, we employ a total variation (TV) regularization of the sparse matrix, resulting in a robust sparse representation. We derive a solver based on the alternating direction method of multipliers (ADMM) for the resulting optimization problem. Experimental results on real data show improvements over the traditional SSC-based methods in terms of accuracy and running time.

Buildings constitute one of the most important landscapes in remote sensing images. Automatic building extraction methods towards the very high resolution remote sensing imagery feature both the local refinement of segmentation results and the context-aware reasoning for segmentation. In this paper, we propose a novel dual-stream convolutional neural network (DS-Net) to collaboratively incorporate local and global features for accurately segmenting buildings in very high resolution aerial images. We develop a hierachical representation and a deep feature sharing strategy for both the local branch and global branch in DS-Net to effectively exploit the complementarity between the two branches. Through extensive experiments on the large-scale building detection datasets, we show that the proposed DS-Net can benefit from both the local and global features, which significantly improves the accuracy of building extraction over diversified remote sensing scenes.

Spatio-temporal fusion is a promising way to deal with the tradeoff between the temporal resolution and spatial resolution of the remote sensing images. This paper presents a novel remote sensing image spatio-temporal fusion model to expand the application of spatio-temporal fusion with insufficient data, based on a generative adversarial network to handle one prior image pair cases (OPGAN). Considering the huge spatial resolution gap between the high-spatial, low-temporal (HSLT) resolution Landsat imagery and the corresponding low-spatial, high-temporal (LSHT) resolution MODIS imagery, the proposed OPGAN simultaneously trains a generator and a discriminator in a min-max game to reconstruct the high-spatial-high-temporal (HSHT) resolution Landsat images, significantly improving the accuracy of change prediction with the help of the temporal changes and sensor differences. Experimental results on three representative Landsat-MODIS datasets illustrate the effectiveness of the proposed OPGAN method.

In this paper, a multi-branch fusion framework is proposed to address the land cover mapping issue with low-resolution labels. To obtain homogeneous target objects, a multi-resolution segmentation (MRS) algorithm is applied to yield unsupervised object-based segmentation maps. Through an index-based judgement mechanism, a label filtering principle was designed and employed to screen out samples with noisy labels while retaining samples with clean labels, thus acquiring more accurate training data. A patch-to-point classification network was established based on these filtered training patches, which fully extracts the contextual features and generates pixel-based prediction results. A post-processing step, consisting of fusion and voting operations, was developed to merge the pixel-based and object-based results, and produce a final segmentation map. Verified through the competition website, the proposed method achieved an average accuracy (AA) of 57.22%, ranking second in the first track of 2020 IEEE GRSS Data Fusion Contest.

Hyperspectral images (HSIs) are normally corrupted by a mixture of various noise types, which degrades the quality of the acquired image and limits the subsequent application. In this article, we propose a novel denoising method for the HSI restoration task by combining nonlocal low-rank tensor decomposition and total variation regularization, which we refer to as TV-NLRTD. To simultaneously capture the nonlocal similarity and high spectral correlation, the HSI is first segmented into overlapping 3-D cubes that are grouped into several clusters by the k-means++ algorithm and exploited by low-rank tensor approximation. Spatial-spectral total variation (SSTV) regularization is then investigated to restore the clean HSI from the denoised overlapping cubes. Meanwhile, the ℓ1-norm facilitates the separation of the clean nonlocal low-rank tensor groups and the sparse noise. The proposed TV-NLRTD method is optimized by employing the efficient alternating direction method of multipliers (ADMM) algorithm. The experimental results obtained with both simulated and real hyperspectral data sets confirm the validity and superiority of the proposed method compared with the current state-of-the-art HSI denoising algorithms.

Crop classification and mapping are important to socioeconomic, food safety, and policy-making. Accurate and timely spatial distribution of crop types based on remote sensing is important for both scientific and practical purposes. However, the existing studies mostly focused on main crops, such as winter wheat, paddy rice and some others. For the garlic crop, there is few relevant research available. So in this paper, in order to explore the feasibility of garlic extraction, a preliminary experiment of garlic mapping based on random forest (GMRF) is carried out with multi-temporal 10-m resolution Sentinel-2 images, by taking Jinxiang County, Shandong Province, China as the experimental area. The experimental results suggest that the proposed GMRF method can achieve a good performance, with an overall accuracy (OA) of 98.56% and a kappa coefficient of 0.967.

In this paper, we propose a multi-level fusion framework to address the pairwise semantic stereo issue. For disparity estimation, we adopt the pyramid stereo matching network. For semantic segmentation, the single segmentation network is proposed with respect to the left image, along with the disparity fusion segmentation network for the combination of semantic features and disparity features. Specifically, the multi-receptive fusion block is designed and employed to fully extract and fuse the contextual information. Finally, the refined segmentation result is obtained via yet another fusion of the multi-model results. The proposed method achieved a mean intersection over union (mIoU) of 79.05%, an average endpoint error (EPE) of 1.3966, and an mIoU-3 of 77.75%, ranking first in the Pairwise Semantic Stereo Challenge of the 2019 IEEE GRSS Data Fusion Contest [1],[2].

Sparse subspace clustering (SSC) has achieved the state-of-the-art performance in the clustering of hyperspectral images (HSIs). However, the high computational complexity and sensitivity to noise limit its clustering performance. In this paper, we propose a scalable SSC method for the large-scale HSIs, which significantly accelerates the clustering speed of SSC without sacrificing clustering accuracy. A small landmark dictionary is first generated by applying k-means to the original data, which results in the significant reduction of the number of optimization variables in terms of sparse matrix. In addition, we incorporate spatial reg-ularization based on total variation (TV) and improve this way strongly robustness to noise. A landmark-based spectral clustering method is applied to the obtained sparse matrix, which further improves the clustering speed. Experimental results on two real HSIs demonstrate the effectiveness of the proposed method and the superior performance compared to both traditional SSC-based methods and the related large-scale clustering methods.

Optical remote sensing images are inevitably corrupted by clouds during the acquisition process. To reconstruct the missing information contaminated by clouds, this paper introduces a new cloud removal method based on X-fork generative adversarial network with multitemporal data, which can be named X-MTGAN. By utilizing the auxiliary differential image between two imaging times, X-MTGAN can be well trained with multitemporal SAR-optical data. Then, the target optical image is synthesized with an end-to-end generator of the X-MTGAN, which has advantages in capturing change information between two temporal images. Finally, the cloud-free image can be subsequently acquired by replacing cloud-contaminated regions with the simulated image. By utilizing Setinel-1 and Sentinel-2 data, experiments are conducted to validate the feasibility of the proposed approach. Compared with the state-of-the-art methods, the results illustrate that X-MTGAN is visually and quantitatively effective in the removal of clouds, which has favorable applicability and competitive performance.

Clustering is a very challenging task for hyperspectral imagery (HSI) because of the complex spectral-spatial structures found in such data. Recently, the sparse recovery-based approaches have been introduced to deal with hyperspectral clustering, and have achieved state-of-the-art performances. Several recent works have shown that it is the collaborative representation mechanism over all the dictionary atoms, rather than the sparse constraint that determines the recognition performance. Based on this fact, in this paper, we focus on the working mechanism of collaborative representation to explore its potential in HSI clustering. However, directly introducing collaborative representation clustering (CRC) to HSIs results in several problems, i.e., the high redundancy of the global dictionary atoms and the absence of spatial information, which greatly limit the clustering performance. In view of this, we propose a novel total variation regularized CRC with a locally adaptive dictionary (TV-CRC-LAD) algorithm for HSI. First, the LAD construction strategy is introduced instead of the global dictionary to relieve the high redundancy and the interference of unrelated atoms in the representation process, to more precisely represent each pixel only with the highly correlated atoms. Second, TV regularization is integrated to better account for the rich spatial-contextual information and promotes the piecewise smoothness of the HSI clustering result. The proposed algorithm was tested on three widely used hyperspectral data sets, and the experimental results clearly illustrate that the proposed algorithm outperforms the corresponding sparsity-based clustering methods and the other state-of-the-art methods.

Band selection is an effective approach to mitigate the “Hughes phenomenon” of hyperspectral image (HSI) classification. Recently, sparse representation (SR) theory has been successfully introduced to HSI band selection, and many SR-based methods have been developed and shown great potential and superiority. However, due to the inherent limitations of the SR scheme, i.e., individually representing each band with only a few other bands from the same subspace, the SR-based methods cannot effectively capture the global structures of the data, which limit the band selection performance. In this paper, to overcome this obstacle, the novel Laplacian-regularized low-rank subspace clustering (LLRSC) algorithm is proposed for HSI band selection. On the one hand, the low-rank subspace clustering model is introduced to capture the global structure information for the learned representation coefficient matrix and deal with the HSI band selection task in the clustering framework. On the other hand, considering the high correlation between adjacent bands, 1-D Laplacian regularization is utilized to incorporate the neighboring band information and further reduce the representation bias. Lastly, an eigenvalue analysis algorithm based on band mutation information is utilized to estimate the appropriate size of the band subset. The experimental results indicate that the proposed LLRSC algorithm outperforms the other state-of-the-art methods and achieves a very competitive band selection performance for HSIs.

Sparse subspace clustering (SSC) has been widely applied in remote sensing demonstrating excellent performance. Recent extensions incorporate spatial information, typically via smoothness-enforcing regularization. We propose an alternative approach: a joint sparsity SSC model, where pixels within a local region are enforced to select a common set of samples in the subspace-sparse representation. The corresponding optimization problem is solved by the alternating direction method of multipliers (ADMM). Experimental results on real data show a significant improvement over SSC and related state-of-the-art methods.

In this paper, a novel spatiotemporal fusion model based on deep learning is proposed, which handles the huge spatial resolution gap and the nonlinear mapping between the high spatial resolution (HSR) image and the corresponding high temporal resolution (HTR) image at the same imaging time. Considering the huge spatial resolution gap, a two-layer fusion strategy is adopted. In each layer, the convolutional neural network (CNN) model is employed to exploit the non-linear mapping between the HSR and HTR image and reconstruct the high-spatial and high-temporal (HSHT) resolution images. In the experiment, Landsat data is the representation of the high spatial resolution images, MODIS data is used as the corresponding low spatial resolution images. The experimental results on two different datasets clearly illustrate the superiority of the proposed model.

This paper focuses on dimension reduction (DR) technique for hyperspectral image (HSI). In this paper, we proposed a superpixel-based linear discriminant analysis (SP-LDA) dimension reduction method for HSI classification. Pixels within a local spatial neighborhood are expected to have similar spectral curves and share the same class label. To fully exploit the spatial structure, superpixel segmentation is firstly introduced to generate the superpixel map, which can adaptively explore the neighborhood structure information. Moreover, we extend the SP-LDA algorithm by combining the extracted feature from spectral and spatial dimensions, which can fully exploit complementary and consistent information from both dimensions. The experimental results on two standard hyperspectral datasets confirm the superiority of the proposed algorithms.

Sparsity-based classification methods have been widely used in hyperspectral image (HSI) classification. These methods typically assumed Gaussian noise, neglecting the fact that HSIs are often corrupted by different types of noise in practice. In this paper, we develop a robust super-pixel level joint sparse representation classification model (RSJSRC) to address the mixed noise problem in sparsity-based HSI classification. Our method takes into account both Gaussian and sparse noise. Experimental results on simulated and real data demonstrate the efficiency of the proposed method and clear benefits from the introduced mixed-noise model.

In this paper, we propose total variation regularized collaborative representation clustering with a locally adaptive dictionary for hyperspectral remote sensing imagery. With regard to the high redundancy of the global dictionary and the interference of unrelated dictionary atoms in the representation process, the collaborative representation clustering model with a locally adaptive dictionary is introduced to more precisely represent each pixel only with highly correlated atoms. In addition, total variation regularization is integrated to better account for the rich spatial contextual information. The extensive experimental results clearly illustrate the superiority of the proposed algorithm.

Robust techniques such as sparse subspace clustering (SSC) have been recently developed for hyperspectral images (HSIs) based on the assumption that pixels belonging to the same land-cover class approximately lie in the same subspace. In order to account for the spatial information contained in HSIs, SSC models incorporating spatial information have become very popular. However, such models are often based on a local averaging constraint, which does not allow for a detailed exploration of the spatial information, thus limiting their discriminative capability and preventing the spatial homogeneity of the clustering results. To address these relevant issues, in this letter, we develop a new and effective ℓ2-norm regularized SSC algorithm which adds a four-neighborhood ℓ2-norm regularizer into the classical SSC model, thus taking full advantage of the spatial-spectral information contained in HSIs. The experimental results confirm the potential of including the spatial information (through the newly added ℓ2-norm regularization term) in the SSC framework, which leads to a significant improvement in the clustering accuracy of SSC when applied to HSIs.

Blind hyperspectral unmixing (HU), which includes the estimation of endmembers and their corresponding fractional abundances, is an important task for hyperspectral analysis. Recently, nonnegative matrix factorization (NMF) and its extensions have been widely used in HU. Unfortunately, most of the NMF-based methods can easily lead to an unsuitable solution, due to the nonconvexity of the NMF model and the influence of noise. To overcome this limitation, we make the best use of the structure of the abundance maps, and propose a new blind HU method named total variation regularized reweighted sparse NMF (TV-RSNMF). First, the abundance matrix is assumed to be sparse, and a weighted sparse regularizer is incorporated into the NMF model. The weights of the weighted sparse regularizer are adaptively updated related to the abundance matrix. Second, the abundance map corresponding to a single fixed endmember should be piecewise smooth. Therefore, the TV regularizer is adopted to capture the piecewise smooth structure of each abundance map. In our multiplicative iterative solution to the proposed TV-RSNMF model, the TV regularizer can be regarded as an abundance map denoising procedure, which improves the robustness of TV-RSNMF to noise. A number of experiments were conducted in both simulated and real-data conditions to illustrate the advantage of the proposed TV-RSNMF method for blind HU.

In this paper, a unified superresolution framework, which simultaneously integrates image reconstruction and image registration, is proposed for multiangle-multispectral remote sensing images. With the strong correlation of different angle images, the low-rank structure of multiangle image is explored by lexicographically ordering the 3-D data cube into a 2-D matrix. Based on this property, the observed images is decomposed to a background image and a foreground image. Considering the high relationship between each band of the multispectral image, a multichannel total variation (MTV) constraint was applied on the target high resolution background image. Therefore, a robust superresolution model of multiangle-multispectral remote sensing image based on rank minimization (RSRRM) is proposed to register the low resolution multi-angle images and reconstruct high-resolution image. This method can achieve image registration and superresolution at the same time. Several experiments are conducted to illustrate the performance of the method.

Recently, non-negative matrix factorization (NMF) model has been widely used in hyperspectral unmixing (HU). In this paper, based on NMF, we explore the properties of abundance maps, and propose a new blind HU algorithm named total variation regularized reweighted sparse NMF (TV-RSNMF). Typically, only a subset of endmembers are assumed to generate the fixed pixel. As a result, the abundance maps are assumed to be sparse. So we introduce a weighted sparse regularization to explore the sparsity of abundance maps in the NMF model. In addition, the abundance maps related to fixed material are assumed to be piecewise smooth and we adopt a total variation (TV) regularizer to promote the piecewise smooth property. TV regularizer can be regarded as an abundance maps denoising procedure, which significantly improves the robustness of the proposed method to noise. Several experiments were conducted to illustrate the performance of the proposed algorithm.

Hyperspectral and LiDAR data, can provide plentiful information about the objects on the Earths surface. However there are some shortages for each of them, where hyperspectral sensor is easily influenced by cloud and difficult to distinguish different objects contained same materials, LiDAR cannot discriminate different objects which are similar in altitude. Fusion of these multi-source data for reliable classification attracts increasing interests but remains challenging. In this paper, we propose a new framework to fuse multi-source data for classification. The proposed method contains three main works: 1) cloud shadows extraction; 2) feature fusion of spectral and spatial information extracted from hyperspectral image, elevation information extracted from LiDAR data; 3) decision fusion of cloud and non-cloud regions. Experimental results on real HSI and LiDAR data demonstrate effectiveness of the proposed method both visually and quantitatively.

Band selection is an effective approach to mitigate the “Hughes phenomenon” of hyperspectral image (HSI) classification. In this paper, a novel squaring weighted low-rank subspace clustering band selection (SWLRSC) algorithm is proposed for hyperspectral imagery. The SWLRSC method can effectively capture the global structure information of the HSI band set by constructing a strongly connected adjacency matrix with accurate representation coefficients, and can adaptively determine an appropriate size for the selected band subset. The experimental results indicate that the proposed SWLRSC algorithm outperforms the state-of-the-art band selection algorithms.

Dimensionality reduction (DR) is an important and helpful preprocessing step for hyperspectral image (HSI) classification. Recently, sparse graph embedding (SGE) has been widely used in the DR of HSIs. SGE explores the sparsity of the HSI data and can achieve good results. However, in most cases, locality is more important than sparsity when learning the features of the data. In this letter, we propose an extended SGE method: the weighted sparse graph based DR (WSGDR) method for HSIs. WSGDR explicitly encourages the sparse coding to be local and pays more attention to those training pixels that are more similar to the test pixel in representing the test pixel. Furthermore, WSGDR can offer data-adaptive neighborhoods, which results in the proposed method being more robust to noise. The proposed method was tested on two widely used HSI data sets, and the results suggest that WSGDR obtains sparser representation results. Furthermore, the experimental results also confirm the superiority of the proposed WSGDR method over the other state-of-the-art DR methods.

In this paper, we present a spatial spectral hyperspectral image (HSI) mixed-noise removal method named total variation (TV)-regularized low-rank matrix factorization (LRTV). In general, HSIs are not only assumed to lie in a low-rank subspace from the spectral perspective but also assumed to be piecewise smooth in the spatial dimension. The proposed method integrates the nuclear norm, TV regularization, and L1-norm together in a unified framework. The nuclear norm is used to exploit the spectral low-rank property, and the TV regularization is adopted to explore the spatial piecewise smooth structure of the HSI. At the same time, the sparse noise, which includes stripes, impulse noise, and dead pixels, is detected by the L1-norm regularization. To tradeoff the nuclear norm and TV regularization and to further remove the Gaussian noise of the HSI, we also restrict the rank of the clean image to be no larger than the number of endmembers. A number of experiments were conducted in both simulated and real data conditions to illustrate the performance of the proposed LRTV method for HSI restoration.

Clustering for hyperspectral images (HSIs) is a very challenging task due to its inherent complexity. In this paper, we propose a novel spectral-spatial sparse subspace clustering S4C algorithm for hyperspectral remote sensing images. First, by treating each kind of land-cover class as a subspace, we introduce the sparse subspace clustering (SSC) algorithm to HSIs. Then, considering the spectral and spatial properties of HSIs, the high spectral correlation and rich spatial information of the HSIs are taken into consideration in the SSC model to obtain a more accurate coefficient matrix, which is used to build the adjacent matrix. Finally, spectral clustering is applied to the adjacent matrix to obtain the final clustering result. Several experiments were conducted to illustrate the performance of the proposed S4C algorithm.

With regard to the specific role of each pixel within a spatial parcel of a hyperspectral image (HSI), we propose a novel superpixel-oriented sparse representation classification method with a multi-task learning approach. The proposed algorithm exploits the class-level sparsity prior for multiple-feature fusion, and also the correlation and distinctiveness of pixels in a spatial local region. Compared with the state-of-the-art hyperspectral classifiers, the superiority of the spatial prior utilization, the multiple-feature fusion, and the computational efficiency are maintained at the same time in the proposed method. The proposed classification framework was tested on two HSIs. The experimental results suggest that the proposed algorithm performs better than the other representation-based classification algorithms and some popular hyperspectral multiple-feature classifiers.

In this paper, we propose a superpixel-level sparse representation classification framework with multitask learning for hyperspectral imagery. The proposed algorithm exploits the class-level sparsity prior for multiple-feature fusion, and the correlation and distinctiveness of pixels in a spatial local region. Compared with some of the state-of-the-art hyperspectral classifiers, the superiority of the multiple-feature combination, the spatial prior utilization, and the computational complexity are maintained at the same time in the proposed method. The proposed classification algorithm was tested on three hyperspectral images. The experimental results suggest that the proposed algorithm performs better than the other sparse (collaborative) representation-based algorithms and some popular hyperspectral multiple-feature classifiers.

In this paper, a new denoising algorithm is proposed for hyperspectral image data cubes. With the strong correlations of the image bands, the low-rank structure of the hyperspectral image is explored by lexicographically ordering the 3-D data cube into 2-D matrix. Based on this property, the traditional principal component analysis (PCA) denoising model is established. For hyperspectral images (HSIs), the noise intensity in different bands is different. Therefore, a noise-adjusted iterative randomized singular value decomposition (NAIRSVD) algorithm is proposed to solve this PCA model. Combined with adaptive noise estimation and upper bound rank estimation, the proposed NAIRSVD algorithm is free from manual parameter determination. Several experiments were conducted to illustrate the performance of the proposed algorithm.

A novel sparsity-based sub-pixel anomaly detection framework is proposed for hyperspectral imagery. The proposed approach consists of the following steps. First, a joint sparsity model is utilized to simultaneously represent the surrounding local background pixels and to automatically prune the rough overcomplete dictionary as a reliable, compact base for the following center test pixel representation. An unconstrained linear unmixing approach based on the compact dictionary is then utilized to decompose the abundance of the center test pixel. The unmixing result is finally compared to the former background joint sparse representation step, and the energy disparity is utilized to reflect the anomaly test result. The experimental results confirm that the proposed algorithm outperforms the classical RX-based anomaly detector and the orthogonal subspace projection based detector, and gives a desirable and stable performance.

The Earth Observing System of the National Aeronautics and Space Administration pays a great deal of attention to the long-term global observations of the land surface, biosphere, atmosphere, and oceans. Specifically, the Moderate Resolution Imaging Spectroradiometer (MODIS) instrument on board the twin satellites Terra and Aqua plays a vital role in the mission. Unfortunately, around 70% of the detectors in Aqua MODIS band 6 have malfunctioned or failed. Consequently, many of the derivatives related to band 6, such as the normalized difference snow index, suffer from the adverse impact of dead or noisy pixels. In this letter, the missing or noisy information in Aqua MODIS band 6 is successfully completed using a robust multilinear regression (M-estimator) based on the spectral relations between working detectors in band 6 and all the other spectra. The experimental results indicate that the proposed robust M-estimator multiregression (RMEMR) algorithm can effectively complete the large areas of missing information while retaining the edges and textures, compared to the state-of-the-art methods.

In this letter, a novel supervised segmentation technique based on sparsely representing the stacked extended morphological attribute profiles (EAPs) and maximum a posteriori probability (MAP) is presented for very high resolution (VHR) images. Attribute profiles (APs), which are extracted by using several attributes, are applied to the multispectral VHR image, leading to a set of extended EAPs. Using the sparse prior of representing the pixel with all training samples, the extended multi-AP (EMAP) feature stacked by the EAP features is transformed into a class-dependent residual feature, which can be normalized as a posterior probability distribution of the pixel. A graph-cut approach is utilized to segment the image scene and obtain the final classification result. Experiments were conducted on IKONOS and WorldView-2 data sets. Compared with SVM, object-oriented SVM with majority voting, and some other state-of-the-art methods, the proposed method shows stable and effective results.

With regard to quantitative remote sensing products in the visible and infrared ranges, thick clouds and accompanying shadows are an inevitable source of noise. Due to the absence of adequate supporting information from the data themselves, it is a formidable challenge to accurately restore the surficial information underlying large-scale clouds. In this paper, dictionary learning is expanded into the multitemporal recovery of quantitative data contaminated by thick clouds and shadows. This paper proposes two multitemporal dictionary learning algorithms, expanding on their KSVD and Bayesian counterparts. In order to make better use of the temporal correlations, the expanded KSVD algorithm seeks an optimized temporal path, and the expanded Bayesian method adaptively weights the temporal correlations. In the experiments, the proposed algorithms are applied to a reflectance product and a land surface temperature product, and the respective advantages of the two algorithms are investigated. The results show that, from both the qualitative visual effect and the quantitative objective evaluation, the proposed methods are effective.

Sparse representation has been widely used in image classification. Sparsity-based algorithms are, however, known to be time consuming. Meanwhile, recent work has shown that it is the collaborative representation (CR) rather than the sparsity constraint that determines the performance of the algorithm. We therefore propose a nonlocal joint CR classification method with a locally adaptive dictionary (NJCRC-LAD) for hyperspectral image (HSI) classification. This paper focuses on the working mechanism of CR and builds the joint collaboration model (JCM). The joint-signal matrix is constructed with the nonlocal pixels of the test pixel. A subdictionary is utilized, which is adaptive to the nonlocal signal matrix instead of the entire dictionary. The proposed NJCRC-LAD method is tested on three HSIs, and the experimental results suggest that the proposed algorithm outperforms the corresponding sparsity-based algorithms and the classical support vector machine hyperspectral classifier.

Hyperspectral images (HSIs) are often degraded by a mixture of various kinds of noise in the acquisition process, which can include Gaussian noise, impulse noise, dead lines, stripes, and so on. This paper introduces a new HSI restoration method based on low-rank matrix recovery (LRMR), which can simultaneously remove the Gaussian noise, impulse noise, dead lines, and stripes. By lexicographically ordering a patch of the HSI into a 2-D matrix, the low-rank property of the hyperspectral imagery is explored, which suggests that a clean HSI patch can be regarded as a low-rank matrix. We then formulate the HSI restoration problem into an LRMR framework. To further remove the mixed noise, the “Go Decomposition” algorithm is applied to solve the LRMR problem. Several experiments were conducted in both simulated and real data conditions to verify the performance of the proposed LRMR-based HSI restoration method.

A column generation kernel technology based nonlinear regression classification method for hyperspectral image is proposed in this paper. The nonlinear extension for the collaborative representation regression is utilized in the joint collaboration model framework. The proposed algorithm is tested on two hyperspectral images. Experimental results suggest that the proposed nonlinear algorithm shows superior performance over other linear regression-based algorithms and the classical hyperspectral classifier SVM.

High-resolution multispectral (HRM) images are widely used in many remote sensing applications. Using the pan-sharpening technique, a low-resolution multispectral (LRM) image and a high-resolution panchromatic (HRP) image can be fused to an HRM image. This letter proposes a new compressed sensing (CS)-based pan-sharpening method which views the image observation model as a measurement process in the CS theory and constructs a joint dictionary from LRM and HRP images in which the HRM is sparse. The novel joint dictionary makes the method practical in fusing real remote sensing images, and a tradeoff parameter is added in the image observation model to improve the results. The proposed algorithm is tested on simulated and real IKONOS images, and it results in improved image quality compared to other well-known methods in terms of both objective measurements and visual evaluation.

In this paper, an adjustable model-based image fusion method for multispectral (MS) and panchromatic (PAN) images is developed. The relationships of the desired high spatial resolution (HR) MS images to the observed low-spatial-resolution MS images and HR PAN image are formulated with image observation models. The maximum a posteriori framework is employed to describe the inverse problem of image fusion. By choosing particular probability density functions, the fused HR MS images are solved using a gradient descent algorithm. In particular, two functions are defined to adaptively determine most regularization parameters using the partially fused results at each iteration, retaining one parameter to adjust the tradeoff between the enhancement of spatial information and the maintenance of spectral information. The proposed method has been tested using QuickBird and IKONOS images and compared to several known fusion methods using quantitative evaluation indices. The experimental results verify the efficacy of this method.

Image super-resolution reconstruction (SRR) refers to a signal processing approach which produces a high-resolution (HR) image from observed multiple low-resolution (LR) images. In this paper, we propose a joint MAP formulation combining image registration, blur identification, and SRR together to deal with heavy aliasing in the observed LR images. A cyclic coordinate decent optimization procedure is used to solve the formulation, in which the registration parameters, blurring information, and HR image are found in an alternate manner given the others, respectively. The proposed algorithm has been tested on a synthetic image sequence. The experiment results and error analyses verify the efficacy of this algorithm.