Detecting small air target is an important task in civil aviation. However, the weak characteristics of these targets make detection challenging. Hyperspectral image (HSI), provides a new approach for the small air target detection task due to its strong ability of capturing both spatial and spectral information simultaneously. In this article, we propose a spectral-spatial enhanced local contrast strategy for hyperspectral small air target detection. An unsupervised band selection step based on the local contrast strategy has been designed based on local contrast (LC-UBSM) to choose bands with better distinguish ability between the target and background in HSI. Then, we have developed an improved RX detection algorithm with combined spatial and spectral variance (CSSV-RX) to detect the target while suppressing both background and noise. Experimental results on both real GAOFEN-5 dataset and simulated dataset based on EO-1 (Earth Observing-1) satellite have validated the effectiveness and robustness of the proposed method.

Hyperspectral remote sensing images often suffer from mixed noise-Gaussian, stripe, and impulse-due to atmospheric interference, solar variability, and sensor imperfections. These noises are typically band-dependent and diverse in distribution, making unified denoising particularly challenging. Existing deep denoising methods rely on clean/noisy pairs, which are unavailable in real-world remote sensing, while traditional approaches require manual tuning and lack adaptability. We propose SpecEStop, a fully self-supervised spectral vector denoising framework that requires only a single noisy HSI for training. Leveraging a novel deep spectral prior, SpecEStop exploits the spectral bias of neural networks, which tend to learn low-frequency (clean) signal components with Gaussian noise before overfitting to non-Gaussian noise ones. An adaptive early stopping strategy halts training before non-Gaussian noise is learned, enabling effective suppression of complex noise patterns. To address remaining Gaussian noise, we purposely design the network architecture to preserve its statistical properties in the latent space, allowing the use of off-the-shelf Gaussian denoisers during inference. Without any clean supervision, SpecEStop achieves effective, stage-wise removal of mixed noise, as validated across diverse real-world scenarios. Code will be released at https://github.com/ruobing-Zhang and the permanent code repository maintained by the corresponding author at http://github.com/LinaZhuang.

Tiny object detection (TOD) in remote sensing imagery remains challenging because foreground signals are extremely weak in deep feature hierarchies and are easily overwhelmed by high-response background interference. To mitigate this observed foreground-background signal modulation imbalance (FBSMI) difficulty, we propose a signal modulation network (SMN) for remote-sensing TOD. SMN comprises two complementary components. First, an adaptive Wiener filter modulator (AWFM) is inserted after backbone stages to suppress background-dominated noise while preserving weak target-related responses at multiple resolutions. Second, we introduce the novel denoising diffusion transformer (DDT), a feature-space conditional diffusion module that operates on detector feature tensors rather than image pixels. DDT generates multiple diffusion-guided semantic feature variants from high-level fused features and expands the local representation space around weak tiny object evidence. Extensive experiments on AI-TOD, SODA-A, DOTAv2.0, and DIOR-R demonstrate that SMN not only effectively mitigates the FBSMI problem, but also improves detection accuracy, particularly for very tiny and tiny objects, compared with state-of-the-art methods.

Efficient tiny object detection (TOD) in large-size remote sensing imagery (LSRSI) is particularly challenging in real-world remote sensing applications. We observe that as the input size of the remote sensing scene increases, TOD faces more severe foreground signal identification issues. To address this, we are the first to design a backbone network from the perspective of low-level spatial feature preservation and utilization, specifically for tiny object feature extraction in large-size remote sensing scene patches. The proposed architecture, referred to as the resolution preserving and utilization network (RPUN), demonstrates excellent foreground tiny object feature response identification ability when increasing the input size of remote sensing scenes, effectively maintaining detection performance comparable to that of smaller input slices. Additionally, we introduce GF2UBSv2, a large-scale panchromatic satellite imagery dataset focused on tiny urban bridge detection. Extensive experiments conducted on GF2UBSv2, DIOR, SODA-A, and DOTAv2.0 demonstrate the superior performance of RPUN compared with state-of-the-art methods. The code and dataset are available at: https://github.com//Nankle.

Multimodal change detection (MCD) aims to detect changed areas between the bi-temporal multimodal images such as the RGB, panchromatic (PAN), multispectral (MS), and synthetic aperture radar (SAR) images, which has attracted attention in recent years. However, existing deep learning-based methods for MCD tasks still face several heterogeneity factors, the first one is the spatial resolution differences in multimodal data, which leads to the semantic gap between multimodal features. To solve this problem, we propose the heterogeneous collaborative fusion (HCF) module to integrate the multimodal features with spatial gaps. The other one is the consistency and dissimilarity between multimodal data, which lead to unequal detection contributions. To address this dilemma, we propose the heterogeneous adaptive fusion (HAF) module to fuse multimodal decision-making jointly. In this study, we proposed a heterogeneous fusion network for MCD (HF-MCD) with the HCF and the HAF module. We validate the proposed method on four public available MCD datasets. Extensive experimental results have demonstrated the superior performance of HF-MCD over the state-of-the-art methods.

Deep learning-based frameworks have shown great potential in the field of hyperspectral image (HSI) classification owing to their superior modeling capabilities. However, the existence of mixed pixels and spectral heterogeneity limits the discriminant performance of the classifier, which makes it impossible to distinguish the mixed spectra effectively in actual scenarios. To address this gap, we propose a subpixel spectral variability network ($\text {S}^{2}\text {VNet}$) for HSI classification, which incorporates complete subpixel information and class features modeled by spectral variability and nonlinear mixture characteristics to enhance classification performance.$\text {S}^{2}\text {VNet}$is capable of extracting endmembers and abundances based on the nonlinear autoencoder (AE) framework and estimating variability parameters by simultaneously considering scaling factors and perturbation terms to ensure accurate endmember construction. The enhanced subpixel fusion module is further designed to automatically integrate three aspects of abundances, spectral cosine correlation information, and pixel-level class features to provide a robust joint representation for the classifier. Extensive experiments on four public HSI datasets demonstrate the superiority and generalization of the proposed method when benchmarked with state-of-the-art methods. The code will be available athttps://github.com/hanzhu97702/S2VNet.

Depending on a large-scale paired dataset of low-resolution hyperspectral image (LrHSI), high-resolution multispectral image (HrMSI), and corresponding high-resolution hyperspectral image (HrHSI), the supervised paradigm has achieved impressive performance in the hyperspectral image super-resolution (HISR). However, the intrinsic data-intensive manner hinders its further application in real scenarios. Fortunately, deep image prior (DIP) allows us to achieve unsupervised super-resolution (SR) by solely utilizing degraded observations. However, its potential to accurately model complicated hyperspectral priors is still not fully exploited due to the following two factors: 1) existing methods tend to reconstruct the unknown HrHSI directly from a randomly generated noise, leaving it hard to leverage the scene-relevant information for prior learning and 2) the vanilla architecture is handcrafted for the generator network, which shows limitations in feature representation and thus fails to characterize the complicated image properties. To unleash the potential of DIP for the HISR task, we propose an enhanced DIP network, called EDIP-Net, by addressing the aforementioned impediments. Specifically, EDIP-Net is built with a two-stage four-component scheme, with a zero-shot learning (ZSL) stage for input image establishment and a deep image generation (DIG) stage for prior learning. First, we exploit the cross-scale spectral relationship inside the observations and thus design a degradation learning network to generate paired training samples from the observations themselves. As such, two image-coarse estimations are derived in a ZSL manner by learning an interactive spectral learning network. By replacing random noise with two estimations, we design a double U-shape architecture for the generator network to capture their hyperspectral prior, each independently generating one HrHSI candidate. Under this premise, we further propose a degradation-aware decision fusion strategy to integrate the optimal results in a pixel-to-pixel manner. Extensive experiments demonstrate our superiority in achieving high-quality SR performance. The code will be available athttps://github.com/JiaxinLiCAS.

Deep learning has become increasingly popular in hyperspectral image (HSI) and light detection and ranging (LiDAR) data classification, thanks to its powerful feature learning and representation capabilities. However, HSI often contains substantial redundant information, which can hinder efficient data utilization. Furthermore, the significant disparity in information content between HSI and LiDAR data poses a major challenge in representing and aligning semantic information across these two modalities. To address these challenges, we propose a fusion network structure guided by feature reconstruction embedding. This approach employs feature decomposition to reconstruct HSI features and incorporates weight embedding to seamlessly integrate the reconstructed information into classification features. Furthermore, we introduce a cross-modal attention fusion module designed to merge extracted HSI and LiDAR features. This module fully exploits the complementary nature of these two type of feature, facilitating effective information exchange and semantic alignment across multimodal data. We evaluated our method on three widely used HSI and LiDAR datasets: Houston 2013, Augsburg and MUUFL. Experimental results demonstrate that our proposed FRGFNet significantly outperforms traditional probabilistic methods and state-of-the-art deep learning networks, showcasing its effectiveness in multi-source data fusion.

Hyperspectral anomaly detection (HAD) seeks to identify targets of interest within hyperspectral images (HSIs) without requiring prior knowledge. Recent works show that embedding low-rank attributes in the HAD task can yield superior results. While most such methods focus on exploiting global or local geometrical structures in HSIs, largely overlooking similar features in nonlocal regions. To this end, we propose a novel nonlocal framework that captures the nonlocal self-similar (NSS) and global correlation along spectrum (GCS) priors within the subspace of HSIs. Then, we integrate a deep denoiser prior into the nonlocal framework to further leverage the external priors. The proposed approach, referred to as nonlocal and deep priors for anomaly detection (NLDPAD), combines the structural modeling of nonlocal low-rank and the feature extraction capabilities of deep learning (DL). Furthermore, an efficient alternating minimization (AM) algorithm was developed to optimize the framework, while the plug-and-play alternating direction method of multipliers (PnP-ADMM) was used to solve the subproblems. Extensive experiments on real satellite and aerial hyperspectral datasets demonstrate that the proposed NLDPAD framework significantly outperforms state-of-the-art methods, achieving superior detection performance.

Multimodal fusion widely uses convolutional layers to capture local correlations and adjust feature dimensions. However, the progressive expansion of the receptive field in convolutional layers often compromises spatial context retention, leading to the loss of fine details. Furthermore, the fixed-size kernels typically used in standard convolution restrict the network’s ability to capture multiscale contextual details. To address this limitation, this paper develops a dilated transformation-guided unsupervised multimodal learning (DTUML) method to fuse a high-resolution multispectral image (HR-MSI) and a low-resolution hyperspectral image (LR-HSI), thereby generating a high-resolution hyperspectral image (HR-HSI). Our DTUML adopts a dual-stream encoder architecture to conduct multimodal data, where one stream focuses on preserving spectral information from LR-HSIs, while the other emphasizes the acquisition of spatial details from HR-MSIs. These complementary features are subsequently integrated to ensure spectral fidelity and retain spatial detail. Then, a convolutional layer restores dimensional consistency and outputs an HR-HSI. Extensive experiments demonstrate the effectiveness of DTUML, showing superior performance and strong competitiveness compared to state-of-the-art methods. Code:https://github.com/yuanchaosu/TGRS-DTUML.

Object Detection is a fundamental procedure in the interpretation of remote sensing images. In large-scale remote sensing images, it is common to observe that the interesting objects only occupy a small area. Such objects provide limited information gain and exhibit unclear edges, often named as small objects. The inherent characteristics of small objects significantly hinder the precise localization and accurate classification of deep object detection networks. In this paper, we introduce a significant challenge: the presence of similar objects among these small objects, which leads to dramatic misclassification and overall accuracy decrease. To assess this phenomenon, we propose a novel metric, Similar Category Angle (SCA), for classification discrimination, which serves to intuitively describe the network’s effectiveness in discriminating similar category objects in its final predictions. We also propose a one-stage object detection network named Similar Category Enhancement Network (SCENet), designed to tackle the challenges associated with discriminating similar objects in small object detection tasks. Specifically, we design SCA Loss guided by the SCA metric, which integrates SCA into the network training process, thereby enhances the network’s capability to discriminate between similar category objects. Meanwhile, we propose Laplacian Sobel Enhancement FPN, LSE-FPN, a module that incorporates dynamic edge extraction operators into the FPN to enhance the network’s ability to detect small objects by sharpening the explicit edges of objects in the feature map. Extensive experiments conducted on SODA-A, VisDrone2019 and FAIR1M-AIR datasets demonstrate the superiority of SCENet in the small object detection task, with significant improvements in detection results for both the mAP50 and SCA metrics. The code is available at https://github.com/weiziji01/SCENet.

Convolution Neural Networks (CNNs) have demonstrated strong feature extraction capabilities in Euclidean spaces, achieving remarkable success in hyperspectral image (HSI) classification tasks. Meanwhile, Graph convolution networks (GCNs) effectively capture spatial-contextual characteristics by leveraging correlations in non-Euclidean spaces, uncovering hidden relationships to enhance the performance of HSI classification (HSIC). Methods combining GCNs with CNNs have achieved excellent results. However, existing GCN methods primarily rely on single-scale graph structures, limiting their ability to extract features across different spatial ranges. To address this issue, this paper proposes a multiscale segmentation-guided fusion network (MS2FN) for HSIC. This method constructs pixel-level graph structures based on multiscale segmentation data, enabling the GCN to extract features across various spatial ranges. Moreover, effectively utilizing features extracted from different spatial scales is crucial for improving classification performance. This paper adopts distinct processing strategies for different feature types to enhance feature representation. Comparative experiments demonstrate that the proposed method outperforms several state-of-the-art (SOTA) approaches in accuracy. The source code will be released at https://github.com/shengrunhua/MS2FN.

Small vehicles (SV) detection is crucial for urban security and traffic management. However, detecting such targets from a single image presents significant challenges due to the difficulty in discerning their dynamic movements. In this paper, we propose a deep joint image-level and feature-level processing network, IFNet, designed for detecting changes in SV using bi-temporal hyperspectral images. At the image-level, a new Gumbel Softmax trick (GS)-based band selection strategy is introduced to address the problem of inconsistent spectral resolutions of bi-temporal images. At the feature-level, to tackle the challenge of capturing edge and shape details of SV, we propose a feature-based edge enhancement module, it can extract the target edge using high-level difference features, and the refined change map will be generated with the guidance of the edge map. Moreover, current deep learning-based hyperspectral change detection (HCD) methods are limited by HCD datasets. Therefore, we propose a benchmark dataset, the Hyperspectral Vehicle Change Detection (HVCD) dataset, which consists of 201 pairs of aerial hyperspectral images, each with a size of $256\times 256$ , and exhibits inconsistent spectral resolutions across the bi-temporal data. Extensive experiments conducted on the HVCD dataset demonstrate that our IFNet obtains state-of-the-art performance with an acceptable computational cost.

Vision transformer (ViT) has recently been a popular topic in the foundation model field, taking advantage of its strong scalability and outstanding representation capabilities. As a deep model, ViT introduces a new architecture for achieving hyperspectral image (HSI) unmixing. However, traditional ViTs overlook pixel-level spatial continuity by partitioning the input image into nonoverlapping fixed-size patches. This approach disrupts local structural relationships and hinders the model's ability to capture fine-grained spatial dependencies, resulting in suboptimal feature representation for dense prediction tasks in unmixing. To address these challenges, this article proposes the development of a self-supervised relation-aware ViT (SRViT). SRViT incorporates a self-embedded module comprising encoders, a pixel-level position encoder (PLPE), a self-supervised contrastive mechanism (SCM), and a decoder. The self-embedded module and PLPE preserve local correlations in HSI across different views, facilitating cross-view learning through SCM to ensure generalization. In addition, the decoder incorporates Kronecker-factored approximate curvature (K-FAC) to capture the local geometric structure of spectral information. Ultimately, SRViT learns endmembers and fractional abundance as the unmixing result. The effectiveness and competitiveness of SRViT have been systematically validated through comparative experiments, demonstrating its superior performance. The source code is available at the following link: https://github.com/yuanchaosu/TNNLS-SRViT.

Multimodal images can provide richer information for a wide range of applications. However, the physical heterogeneity resulted by the difference of spatial resolution pose great challenges for multimodal change detection. To this end, we propose a change detection method called primary modality guided deep neural network (PMGN), integrating multi-resolution and multimodal data. First, we propose the principal modality and rely more on its information. Second, PMGN compensates for the limitations of low spatial resolution modalities through the primary modality guided feature exchange module. Finally, the adaptive decision fusion module enables the multimodal decision-level features to fuse efficiently. Experiments demonstrate the effectiveness and advantages of the proposed approach.

Hyperspectral super-resolution (SR) can effectively enhance the spatial resolution of hyperspectral images, holding significant application value. Nevertheless, existing methods tend to overlook global information and some detailed aspects of hyperspectral images, resulting in limitations in feature extraction. In addressing this issue, we propose a Dilated Attention With Multi-Stream Learning (DAMS) network to facilitate the fusion of hyperspectral and multispectral images. The network comprises three autoencoders, incorporating dilated residual multipath feature extraction for high-resolution multispectral images and a dense convolutional neural network for low-resolution hyperspectral images. Notably, no prior knowledge of point spread function (PSF) and spectral response function (SRF) is required. Experimental results with DAMS showcase its advantages over other super-resolution fusion methods, demonstrating robust performance across diverse datasets with varying PSF and SRF.

Due to its powerful feature extraction and representation capabilities, deep learning has been successfully applied in the field of hyperspectral image classification. In patchbased hyperspectral image classification, where the central pixel represents the true category, extracting spectral features similar to the central pixel spectrum is crucial. To this end, we develop a new framework, called the inverse Mahalanobis attention network (IMAN), to address the spectral similarity feature extraction problem. The proposed framework develops a self-attention mechanism module based on the Mahalanobis distance to better learn the correlation between feature vectors of pixels, thereby efficiently suppressing noise generated by different land cover categories within patches. The feature extraction capability is further enhanced by integrating a dual-stream network structure that separates spatial and spectral information in hyperspectral images. Experiments conducted on on real hyperspectral datasets demonstrate the effectiveness and superiority of the proposed method compared to several state-of-the-art hyperspectral image classification methods.

Anomaly detection is one of the most popular research topics in hyperspectral remote sensing. A variety of traditional model-driven methods fail to reveal features of data with diversity due to monotonous, fixed analytical modes. This paper analyzes mathematical-statistical properties of hyperspectral images (HSIs) and proposes an interesting approach of information entropy estimation based on point-set topology (IEEPST) to resolve anomaly detection from a brand new perspective, thus eliminating the limitations caused by the data-model discrepancy. Specifically, the original HSI data are mapped into topological spaces to enable ordered arrangements, in preparation for revealing data features. Particularly, information entropy estimation is introduced for the first time in the adoption of point-set topology to adequately unravel the data arrangements in topological spaces, whereby the land cover information is efficiently extracted for detection. Experimental results demonstrate that IEEPST accommodates both detection accuracy and computational efficiency, and is highly competitive with other sophisticated and state-of-the-art methods.

Hyperspectral image (HSI) change detection can detect subtle land surface change information, which is of great significance for promoting the sustainable development of human beings. Different from traditional methods, deep learningbased methods can effectively extract more discriminative features, but the problem of mixed pixels is still a challenge due to the low spatial resolution HSI. In this study, an Adaptive Endmembers Learning (AEL)-based deep unmixing network has been proposed for the change detection task, which can perform an unsupervised unmixing through adaptive endmembers learning and then obtain both the binary and multi-class change detection results. Experiments on the China dataset and the USA dataset have shown that AEL performs better than current state-of-the-art methods.

In recent years, deep learning has been proposed for hyperspectral images(HSIs) super-resolution, and many fusion models for HSI and multispectral images(MSIs) have been developed. However, these networks are constrained to the structure of convolutional neural networks(CNNs), and more attention needs to be paid to the disadvantage of the restricted receptive field of CNNs, such that some of the distal information needs to be included in the process of acquiring features. This approach involves capturing large-scale spatial features through multi-head spatial attention and spectral features of MSI and HSI through multi-head spectral attention. Subsequently, the features are processed by convolution kernels of different scales compactly. The effectiveness and competitiveness of MTSANet are evaluated by comparing it with some state-of-the-art (SOTA) methods.

Vision transformer (ViT) is a prevalent technique for capturing long-distance dependencies and has shown impressive performance in the field of hyperspectral image (HSI) classification. However, the core component of ViT, namely, self-attention, faces challenges in balancing high-computational complexity and global modeling within entire input sequences. To alleviate this issue, a novel Gaussian retentive network, called GRetNet, is devised in this letter to enhance the comprehension of fine-grained spatial and spectral features while reducing computational costs. This method provides a powerful classification backbone and can adaptively generate priors to perceive more effective spatial information by introducing a spatial decay mask to assign different weights at various positions. Furthermore, the Gaussian multi-head attention (GMA) is designed to provide dynamic recalibration of feature significance based on statistical distribution and focuses on distinct spectral patterns across different heads, thereby rendering a more concise and robust modeling for HSI classification. Compared with the state-of-the-art classification algorithms, the proposed GRetNet method can yield better classification results and computational efficiency on four benchmark hyperspectral datasets, which verifies its effectiveness and superiority.

Due to the complementarity of visible and infrared images, it has become more favorable to fuse these two modalities to improve the object detection accuracy in the remote sensing area. However, there are still some problems to be solved. Most of the existing algorithms focus too much on the local information and ignore long-range information when performing feature extraction on different modalities. Besides, coarse weighted fusion strategies do not fully utilize the information from different modalities, and the fusion structure ignores the importance of intermodal information exchange. To tackle these problems, a cross-modal feature fusion and interaction strategy for the convolutional neural network (CNN)-transformer-based object detection in visual and infrared remote sensing imagery is proposed. We adopt a parallel structure to extract the features of different modalities, separately. In visual and infrared modality, the convolutional layers and transformer encoders are cascaded to fully extract both local and long-range information. The cross-modal feature fusion and interaction module (CFFIM) adopts the attention mechanisms to jointly fuse different modal features at the same scale to improve the diversity of fused features, and the feature interaction enables the sharing of visible and infrared information. Experiments on the VEDAI dataset have demonstrated the effectiveness of the proposed scheme compared to other state-of-the-art algorithms.

Hyperspectral anomaly detection (HAD) poses the challenge of distinguishing anomalous targets from the majority of background objects without prior knowledge. Most existing deep learning (DL) models struggle to account for both local and global spatial-spectral features in the image, limiting their performance. In this letter, we introduce PUNNet, which integrates the patch-shuffle downsampling technique and nonlinear activation-free network (NAFNet) block with dilated convolution into an advanced blind-spot network for HAD. Specifically, PUNNet utilizes the patch-shuffle downsampling operation to extend its receptive field and exploits channel attention in the NAFNet block with dilated convolution to capture global contextual information in the image. Meanwhile, PUNNet satisfies the blind-spot requirement, meaning its receptive field excludes the center pixel’s information. This allows for reliable and precise background reconstruction in a self-supervised learning paradigm, further weakening anomalous feature expression and increasing the reconstruction error of anomalies. Experimental results demonstrate that PUNNet achieves a leading position in HAD performance. The code is available athttps://github.com/DegangWang97/IEEE_GRSL_PUNNet.

The detection of objects with larger aspect ratios (OLAR) is a challenging problem in a special application scenario, such as remote sensing object recognition and scene text detection. However, current object detectors perform poorly in OLAR feature extraction because they are incapable of adaptively responding to object shapes, which leads to severe misalignment between impure feature representations and region proposals. In this letter, we aim at solving this problem by proposing our shape-sensitive convolution network (SSC-Net). SSC-Net is carefully embedded with a feature enhancement module (SSC module) specifically suitable for OLAR. This module can use fewer sampling points to achieve more intelligent feature sampling area transformation, thus achieving the goal of enhancing OLAR feature representation. Extensive experiments on benchmark datasets that are rich in OLARs have proved the superiority of our method. Besides, we further verified the plug-and-play performance of the SSC module, and the experimental results show that it can significantly improve the detection performance of the detector for OLAR.

Deep learning (DL) has been widely applied to hyperspectral image (HSI) classification, owing to its promising feature learning and representation capabilities. However, limited by the spatial resolution of sensors, existing DL-based classification approaches mainly focus on pixel-level spectral and spatial information extraction through complex network architecture design while ignoring the existence of mixed pixels in actual scenarios. To tackle this difficulty, we propose a novel dual-branch subpixel-guided network for HSI classification, called DSNet, which automatically integrates subpixel information and convolutional class features by introducing a deep autoencoder unmixing architecture to enhance classification performance. DSNet is capable of fully considering physically nonlinear properties within subpixels and adaptively generating diagnostic abundances in an unsupervised manner to achieve more reliable decision boundaries for class label distributions. The subpixel fusion module is designed to ensure high-quality information fusion across pixel and subpixel features, further promoting stable joint classification. Experimental results on three benchmark datasets demonstrate the effectiveness and superiority of DSNet compared with state-of-the-art DL-based HSI classification approaches. The codes will be available athttps://github.com/hanzhu97702/DSNet, contributing to the remote sensing community.

Cross-scene image classification aims to transfer prior knowledge of ground materials to annotate regions with different distributions and reduce hand-crafted cost in the field of remote sensing. However, existing approaches focus on single-source domain generalization to unseen target domains, and are easily confused by large real-world domain shifts due to the limited training information and insufficient diversity modeling capacity. To address this gap, we propose a novel multi-source collaborative domain generalization framework (MS-CDG) based on homogeneity and heterogeneity characteristics of multi-source remote sensing data, which considers data-aware adversarial augmentation and model-aware multi-level diversification simultaneously to enhance cross-scene generalization performance. The data-aware adversarial augmentation adopts an adversary neural network with semantic guide to generate MS samples by adaptively learning realistic channel and distribution changes across domains. In views of cross-domain and intra-domain modeling, the model-aware diversification transforms the shared spatial-channel features of MS data into the class-wise prototype and kernel mixture module, to address domain discrepancies and cluster different classes effectively. Finally, the joint classification of original and augmented MS samples is employed by introducing a distribution consistency alignment to increase model diversity and ensure better domain-invariant representation learning. Extensive experiments on three public MS remote sensing datasets demonstrate the superior performance of the proposed method when benchmarked with the state-of-the-art methods.

Transformer holds significance in deep learning (DL) research. Node embedding (NE) and positional encoding (PE) are usually two indispensable components in a Transformer. The former can excavate hidden correlations from the data, while the latter can store locational relationships between nodes. Recently, the Transformer has been applied for hyperspectral image (HSI) classification because the model can capture long-range dependencies to aggregate global features for representation learning. In an HSI, adjacent pixels tend to be homogeneous, while the NE does not identify the positional information of pixels. Therefore, PE is crucial for Transformers to understand locational relationships between pixels. However, in this area, most Transformer-based methods randomly generate PEs without considering their physical meaning, which leads to weak representations. This article proposes a new graph generative structure-aware Transformer (GraphGST) to solve the above-mentioned PE problem when implementing HSI classification. In our GraphGST, a new absolute PE (APE) is established to acquire pixels’ absolute positional sequences (APSs) and is integrated into the Transformer architecture. Moreover, a generative mechanism with self-supervised learning is developed to achieve cross-view contrastive learning (CL), aiming to enhance the representation learning of the Transformer. The proposed GraphGST model can capture local-to-global correlations, and the extracted APSs can complement the spectral features of pixels to assist in NE. Several experiments with real HSIs are conducted to evaluate the effectiveness of our GraphGST. The proposed method demonstrates very competitive performance compared with other state-of-the-art (SOTA) approaches. Our source codes will be provided in the following linkhttps://github.com/yuanchaosu/TGRS-graphGST.

By fusing a low-resolution hyperspectral image (LrMSI) with an auxiliary high-resolution multispectral image (HrMSI), hyperspectral image super-resolution (HISR) can generate a high-resolution hyperspectral image (HrHSI) economically. Despite the promising performance achieved by deep learning (DL), there are still two challenges remaining to be solved. First, most DL-based methods heavily rely on large-scale training triplets, which reduces them to limited generalization and poor practicability in real-world scenarios. Second, existing methods pursue higher performance by designing complex structures from off-the-shelf components while ignoring inherent information from the degradation model, hence leading to insufficient integration of domain knowledge and lower interpretability. To address those drawbacks, we propose a model-informed multi-stage unsupervised network, M2U-Net for short, by leveraging both deep image prior (DIP) and degradation model information. Generally, M2U-Net is built with a three-stage scheme, i.e., degradation information learning (DIL), initialized image establishment (IIE), and deep image generation (DIG) stages. The first stage is to exploit the deep information of the degradation model via a tiny network whose parameters and outputs will serve as guidance for the following two stages. Instead of feeding uninformed noise as input for stage three, IIE stage aims to establish an initialized input with expressive HrHSI-relevant information by resorting to a spectral mapping learning network, thus facilitating the extraction of prior information and further magnifying the potential of DIP for high-quality reconstruction. Last, we propose a dual U-shape network as a powerful regularizer to capture image statistics, in which two U-Nets are coupled together by cross-attention guidance (CAG) module to separately achieve spatial feature extraction and final image generation. The CAG module can incorporate abundant spatial information into the reconstruction process and hence guide the network toward a more plausible generation. Extensive experiments demonstrate the effectiveness of our proposed M2U-Net in terms of quantitative evaluation and visual quality. The code will be available at https://github.com/JiaxinLiCAS.

The adequate and finer spectral information in hyperspectral images (HSIs) are benefit for various downstream applications like smart agriculture and environmental monitoring. In HSI classification, dual-stream convolutional networks have gained much attention and have been widely used. In patch-based hyperspectral classification tasks, however, merely using center-labeled patches could lead to an increased unlabeled noise in the data. Moreover, in the application of dual-stream network structures, heterogeneity existed in both the data and feature semantic levels to capture more representative features. To tackle these challenges, we have devised a framework called cross-semantic heterogeneous modeling network (CreatingNet), which aligns more closely with the design principles of dual-stream networks by adjusting the input size. This framework introduces a distance metric attention mechanism (DMAM) based on spectral and spatial distances to strengthen the influence of the center pixel on the entire patch. Additionally, we present a fusion module named CrossViT, which combines features with diverse structures and characteristics, leveraging their complementarity. The proposed multiscale heterogeneous fusion module allows for more effective integration of spatial and spectral features in the images. Extensive experiments on four well-known HSI datasets (Indian Pines, Pavia University, Salinas, and Houston 2013) demonstrate the superior classification performance of the proposed CreatingNet to several state-of-the-art methods. The effectiveness of the proposed model is further validated through ablation studies.

Hyperspectral anomaly detection aims at distinguishing targets of interest from the background without prior knowledge. Although low-rank representation (LRR)-based methods have been broadly applied in anomaly detection tasks, how to approximate the penalties in LRR-based methods more precisely is still a problem that needs to be further investigated. To this end, this article designs a unified nonconvex framework called hyperspectral anomaly detection via generalized shrinkage mappings (HADGSMs) to better approximate the LRR-based methods. The core of the proposed framework is to design new nonconvex penalties to approximate the group sparsity,$l_{0}$gradient, and low-rankness penalties in the LRR-based anomaly detection models, which can be efficiently minimized by means of generalized shrinkage mappings (GSMs). Then, an efficient alternating direction method of multipliers (ADMM) is developed to handle the proposed model. Experiments conducted on several real hyperspectral datasets demonstrate the superiority and effectiveness of the proposed framework in enhancing detection performance with respect to state-of-the-art methods.

In recent years, deep learning (DL) has accelerated the development of hyperspectral image (HSI) processing, expanding the range of applications further. As a typical model of unsupervised DL, the autoencoder framework has been extensively applied for spectral unmixing due to its strong representation ability and scalability. Nowadays, most DL-based unmixing approaches adopt the linear mixture model (LMM) to estimate pure spectral signatures (endmembers) and their corresponding abundance fractions. However, since sunlight scattering is an inevitable physical phenomenon, the spectral mixture problem is inherently nonlinear. Moreover, most existing nonlinear unmixing approaches focus exclusively on spectral information, neglecting the spatial distribution of materials and the intrinsic correlation between pixels, making it challenging to explore latent features. To address these issues, this article develops a new deep autoencoder-based augmented network (DAAN). The proposed DAAN employs the multilinear mixture model (MLMM) to handle the nonlinear influence caused by multiple scattering. Meanwhile, the proposed DAAN constraints homogenous smoothing in the autoencoder architecture, enabling the aggregation of intrinsic correlations by means of spatial relationships to enhance the performance of abundance estimation. We achieve unsupervised nonlinear hyperspectral unmixing by combining spectral and spatial information. The effectiveness and advantages of DAAN are confirmed by several experiments with synthetic and real HSI datasets. The results indicate that the proposed method outperforms other DL-based unmixing approaches. The source codes of the proposed DAAN will be provided in the following linkhttps://github.com/yuanchaosu/TGRS-daan.

Deep-learning (DL)-based methods have been shown to be effective on the hyperspectral image (HSI) anomaly detection task because of their feature extraction ability. However, current DL-based methods lack an effective means of regularizing the background information. In this article, the hyperbolic space-based autoencoder (HSAE) is proposed for the hyperspectral anomaly detection task. We assume that an effective hierarchical structural representation can better model the HSI in the spatial domain, and this enables the background information to be effectively regularized. Motivated by this idea, the HSAE embeds the HSI into hyperbolic space, which is a non-Euclidean geometry with a constant negative curvature and an exponential growth distance between points. Using a wrapped normal prior distribution, the training of the hidden representation is supervised to preserve more hierarchical features. After the training process, a hyperbolic distance-based anomaly detector (HDB) is introduced to discover anomalies in a more robust way. Experimental results on several popular HSI benchmarks fully demonstrate the superiority of our HSAE.

With hyperspectral remote sensors (imaging spectrometers) imaging a scene, the specificity of the target of interest is manifested in the significant differences between it and the surrounding background in terms of quantity, spatial distribution, and spectral characteristics, which provides conditions for the implementation of pixel-level diagnostics for target detection. Traditional model-driven methods utilize specific model assumptions to parse hyperspectral image (HSI) data in scenes with variability and are prone to encounter limitations due to model-data discrepancy. Most data-driven methods are limited in practical applications due to the great demand for training samples, the large number of parameters to be determined, and the costly computational complexity. To address the limitations of the existing methods, this article adopts point-set topology theories to analyze the properties of hyperspectral data at the mathematical-statistical level and seek a solution for the information retrieval task of target detection, whereby a target detection method through information entropy estimation based on point-set topology is proposed. First, parallel topological spaces are constructed to order the original HSI data to ensure that the differences in data features between various classes of land covers are reflected in intuitive properties in the topological spaces. Second, in conjunction with the priori information about the target, information entropy estimation is introduced to select optimal separable spaces for the target and the background by measuring the degree of ordering of data to achieve an accurate separation. Finally, a proper way to quantify and highlight the differences in data features between various land covers in the optimal separable spaces is explored for the algorithmic output to perform the information retrieval task. The proposed target detection through information entropy estimation based on point-set topology (TD-IEEPST) exploits an innovative combination of point set topology theories and information entropy estimation to achieve efficient extraction of land cover information for detection, ensuring both theoretical interpretability and computational efficiency. Extensive experimental results on real hyperspectral datasets verify that the proposed method is ahead of other widely used and state-of-the-art methods in terms of computational cost, detection effects, and robustness, and promising to provide technical support for detection response requirements in practical applications.

As one of the most active research hotspots in hyperspectral remote sensing, anomaly detection is widely used because it takes effect without any priori information about the target or the background. Most of the traditional model-driven methods fail to reveal features of data with diversity due to fixed analytical modes. A variety of data-driven methods encounter difficulties in practical applications due to their costly computational complexity. In this article, an innovative combination of point-set topology and information entropy theories is utilized to analyze the mathematical–statistical properties of hyperspectral images (HSIs), thus eliminating the limitations caused by the data-model discrepancy. Specifically, the original HSI data are mapped into topological spaces in a specific form to enable ordered arrangements, in preparation for revealing data features. In particular, information entropy estimation is introduced for the first time in the adoption of point-set topology to adequately unravel the data arrangements in topological spaces, whereby the land cover information is efficiently extracted for detection. Accordingly, an interesting approach of information entropy estimation based on point-set topology (IEEPST) is proposed to resolve anomaly detection from a brand new perspective, pursuing prominent detection accuracy while ensuring computational efficiency. The experimental results on benchmark HSI datasets demonstrate that IEEPST achieves detection performance with high probabilities of detection (PD) and low false alarm rates (FARs) at an inexpensive computational cost. The proposed IEEPST is highly competitive with other sophisticated and state-of-the-art methods.

Hyperspectral anomaly detection (HAD) aims to identify anomalous objects that deviate from surrounding backgrounds in an unlabeled hyperspectral image (HSI). Most available neural networks that make use of the reconstruction error to perform HAD tend to fit both backgrounds and anomalies, resulting in small reconstruction errors for both and not being effective in separating targets from background. To address this issue, we develop DirectNet, a new background reconstruction network for HAD that seamlessly integrates a sliding dual-window model into a blind-block architecture. Concretely, DirectNet establishes an inner window within the network’s receptive field by erasing the center block information, so that the content of the inner window remains invisible during the reconstruction of the central pixel. Additionally, the depth of our reconstruction network is adaptive to the size of the input image patch, ensuring that the network’s receptive field aligns with the dimensions of the input patch. The receptive field outside the inner window is considered an outer window. This weakens the impact of anomalies on the reconstruction process, causing the reconstructed pixels to converge towards the background distribution in the outer window region. Consequently, the reconstructed HSI can be regarded as a pure background HSI, leading to further amplification of reconstruction errors for anomalous targets. This enhancement improves the discriminatory ability of DirectNet. Specifically, DirectNet solely utilizes the outer window information to predict/reconstruct the central pixel. As a result, when reconstructing pixels inside anomalous targets of different sizes, the targets primarily fall within the inner window. Comprehensive experiments (conducted on four datasets) demonstrate that DirectNet achieves competitive performance compared to other state-of-the-art detectors.

The most cost-effective way to obtain a high spatial resolution hyperspectral image (HrHSI) is to fuse a low spatial resolution hyperspectral image (LrHSI) and corresponding high spatial resolution multispectral image (HrMSI). This article proposes a generalizable unsupervised deep fusion method based on spectral-spatial collaborative constraint to address LrHSI and HrMSI fusion task. First, in view of the limitations of the current spectral-spatial downsampled model, the group convolution enhancement (GCE) module is designed to eliminate the radiometric difference between the images to be fused. Second, to enhance the model’s feature extraction ability, this article introduces the design of the spatial, channel, and filter 3-D attention factor dynamic convolutional kernel (SCFConv). In order to verify the proposed method, we compared and evaluated our method with traditional methods and unsupervised deep learning methods using both simulated and real onboard data, respectively. In the absence of HrHSI validation images in real scenarios, we evaluate the performance of different fusion models through classification results. The experimental results demonstrate the effectiveness of the proposed model and the practical value of the fusion results (the onboard data produced by ours are available athttps://drive.google.com/drive/folders/1JLCCB6ld5R49HDLN5SsMISx1d0fuqRjO).

Hyperspectral imaging enables advanced change detection but struggles with extensive redundant data across spatial and spectral dimensions. This bloats model size and computational loads. To address this problem, we propose a new gated spectral–spatial–temporal attention network with spectral similarity filtering (HyGSTAN) with a lightweight yet accurate architectural design. Specifically, our HyGSTAN introduces three innovative modules: 1) spectral similarity filtering to reduce spectral redundancy via cosine similarity; 2) gated spectral-spatial attention to capture intra-image spatial features using single-head weak self-attention and gated mechanisms; and 3) gated spectral–spatial–temporal attention to extract inter-image temporal changes. Experiments on three benchmark datasets demonstrate HyGSTAN’s ability to balance accuracy, model complexity, and computational efficiency. The proposed attention mechanisms extract more discriminative information without sacrificing performance. The source code of this work will be released at https://github.com/Welcome-to-LISA/HyGSTAN.

Single-image super-resolution (SISR) for high-resolution (HR) remote sensing image (RSI) acquisition is becoming increasingly valuable and important, and convolutional neural networks (CNNs) have produced considerable progress in this field. In RSIs, many similar geo-objects recur within the same scene, maintaining the same positions in both low resolution (LR) and HR. Based on this observation, we found that these similar geo-objects could be utilized to reconstruct texture details in LR by exploiting the consistent non-local relationships between these geo-objects in LR and HR, thereby improving the quality of SISR. Therefore, we propose a novel graph convolutional network (GCN) for SISR including a dynamic graph attention mechanism to learn the in-scale and cross-scale non-local features of RSIs. In scale, we propose a dynamic graph attention block (DGAB) that adaptively determines non-local patches upon the scene correlation derived from RSIs and further fuses patch-wise non-local information weighed by the attention scores of topological relationships and radiation characteristics in RSIs. Across different scales, we also introduce a dynamic graph attention mixing block (DGAMB) to upsample LR non-local information to HR non-local information. Most SISR methods have the upsampling blocks at the end of the network, ignoring feature extraction in high-dimensional space. To address this problem, DGAMB was designed as an upsampler in the middle of the model, enhancing the level of high-dimensional information extraction from the model. The experiments based on the WHU Building and UC Merced datasets show that our proposed method outperforms state-of-the-art methods. Our code is available athttps://github.com/WenjuanZhang-aircas/NSGCN.

Hyperspectral anomaly detection (HAD) aims to highlight the pixels that are different from the surrounding pixels without any prior information. However, as a hyperspectral image (HSI) tends to possess a huge data volume in the spectral domain, the dimension curse is inevitable in HAD. The unsupervised band selection (UBS) method is an effective tool to avoid the dimensionality curse in the HAD task. To obtain a more robust band subset without the help of any HAD detectors, we propose a spectral–spatial out-of-distribution (OOD)-based UBS method for HAD (HADUBS), which can acquire the optimal band subset in a more straightforward way. Our key observation is that the OOD term of pixels can reveal the differences and similarities of anomaly representation ability of different bands. Hence, we developed an OOD-based feature subspace representation module to obtain latent feature spaces with a better indication of the anomaly detection ability. Moreover, we introduced a UBS strategy called mutual information (MI)-based local outlier factor (MILOF) to significantly improve the discriminative ability of the selected band subset by investigating the locally sparse prior of anomalies. Extensive experimental results on five common HAD datasets demonstrate the superior performance of HADUBS. The source code will be made publicly available athttps://github.com/duang33/HADUBS.

In recent years, neural network-based methods have shown promising results in hyperspectral image (HSI) denoising area. Real HSIs exhibit substantial variations in noise distribution due to various factors such as different imaging techniques, camera variations, imaging environments, and hardware aging. In this paper, we develop an eigenimage plus eigennoise level map guided convolutional neural network for HSI denoising. Our main idea is to perform eigendecomposition on HSIs, utilize the low-rank property of HSIs in the spectral dimension and approximate the spectral vectors in a low-dimensional orthogonal subspace, where representation coefficients are called eigenimages. Besides eigenimages, we make use of estimated eigennoise level map as an input to guide the network for denoising. The proposed network can be constructed without restriction in the number of eigencomponents by using all eigenimages and eigennoise level maps of training noisy-clean pairs. In the inference part, the trained network can be used to remove noise in observed eigenimages without restriction in the number of eigencomponents, and an underlying clean image HSI can be estimated by performing orthogonal projection back. Experimental results on both simulated and real HSIs demonstrate the effectiveness of our trained Eigen-CNN compared with state-of-the-art HSI denoising methods. A MATLAB demo of this work is available at https://github.com/LinaZhuang/HSI-denoiser-Eigen-CNN for the sake of reproducibility.

The performance of deep learning-based denoisers highly depends on the quantity and quality of training data. However, paired noisy-clean training images are generally unavailable in hyperspectral remote sensing areas. To solve this problem, this work resorts to the self-supervised learning technique, where our proposed model can train itself to learn one part of noisy input from another part of noisy input. We study a general hyperspectral image (HSI) denoising framework, called Eigenimage2Eigenimage (E2E), which turns the HSI denoising problem into an eigenimage (i.e., the subspace representation coefficients of the HSI) denoising problem and proposes a learning strategy to generate noisy-noisy paired training eigenimages from noisy eigenimages. Consequently, the E2E denoising framework can be trained without clean data and applied to denoise HSIs without the constraint with the number of frequency bands. Experimental results are provided to demonstrate the performance of the proposed method that is better than the other existing deep learning methods for denoising HSIs. A MATLAB demo of this work is available at https://github.com/LinaZhuang/HSI-denoiser-Eigenimage2Eigenimage for the sake of reproducibility.

Owing to the strong ability of feature extraction and representation, deep learning has exhibited powerful potential in the field of multispectral-aided hyperspectral super-resolution (MS-aided HS-SR). Though great strides have been made by existing methods, their superior performance mainly drives from large training datasets, hence failing to handle the real cases with limited samples. In this article, we propose an unsupervised method inspired by the theory of spectral mixing, which is based solely on one pair of HS-MS correspondence. Specifically, two coupled autoencoders are employed as the backbone of our network, aiming at deriving the latent abundance representations and corresponding endmembers of input HS-MS data. To enrich the feature representations and guide the network learning, we embed an interactive module into the encoder part to enhance the information transmission and design a degradation loss to constrain the target image. Experiments in Chikusei dataset demonstrate the effectiveness of our proposed method.

In recent years, fusion methods based on unsupervised deep learning have achieved impressive performance in the fusion of hyperspectral image (HSI) and multispectral image (MSI). However, there are still some limitations in the current research. Most existing fusion methods only apply to simulated data and need more verification on real data sets. To solve these issues, this paper designed an unsupervised dynamic convolutional neural network fusion model (UDCNN), which can adaptively learn the radiometric difference between HSI and MSI. This model achieves better performance on simulated data compared with related unsupervised deep learning methods, and achieves more accurate results on real data through classification-oriented application of the fusion results.

Spatial-spectral classification (SSC) has become a trend for hyperspectral image (HSI) classification. However, most SSC methods mainly consider local information, so that some correlations may not be effectively discovered when they appear in regions that are not contiguous. Although many SSC methods can acquire spatial-contextual characteristics via spatial filtering, they lack the ability to consider correlations in non-Euclidean spaces. To address the aforementioned issues, we develop a new semisupervised HSI classification approach based on normalized spectral clustering with kernel-based learning (NSCKL), which can aggregate local-to-global correlations to achieve a distinguishable embedding to improve HSI classification performance. In this work, we propose a normalized spectral clustering (NSC) scheme that can learn new features under a manifold assumption. Specifically, we first design a kernel-based iterative filter (KIF) to establish vertices of the undirected graph, aiming to assign initial connections to the nodes associated with pixels. The NSC first gathers local correlations in the Euclidean space and then captures global correlations in the manifold. Even though homogeneous pixels are distributed in noncontiguous regions, our NSC can still aggregate correlations to generate new (clustered) features. Finally, the clustered features and a kernel-based extreme learning machine (KELM) are employed to achieve the semisupervised classification. The effectiveness of our NSCKL is evaluated by using several HSIs. When compared with other state-of-the-art (SOTA) classification approaches, our newly proposed NSCKL demonstrates very competitive performance. The codes will be available at https://github.com/yuanchaosu/TCYB-nsckl.

Recently, unmixing-based networks have shown significant potential in unsupervised multispectral-aided hyperspectral image super-resolution task (MS-aided HS-SR). Nevertheless, the representation ability of unsupervised networks and the design of loss functions still have not been fully explored, leaving large room for further improvement. To this end, we propose an enhanced unmixing-inspired unsupervised network with attention-embedded degradation learning, EU2ADL for short, to realize MS-aided HS-SR. First, two coupled autoencoders serve as the backbone of EU2ADL network to simultaneously decompose input modalities into abundances and corresponding endmembers, whose encoder part is composed of a spatial-spectral two-stream subnetwork for modality-salient representation learning and a parameter-shared one-stream subnetwork for modality-interacted representation enhancement. More importantly, a hybrid model-constrained loss containing a perceptual abundance term and a degradation-guided term is introduced to further eliminate the latent distortions. Since the hybrid loss is built on the degradation model, we additionally present an attention-embedded degradation learning network to adaptively estimate the unknown degradation parameters. Extensive experimental results on four datasets demonstrate the effectiveness of our proposed methods when compared with state-of-the-arts.

Without any prior information, hyperspectral anomaly detection is devoted to locating targets of interest within a specific scene by exploiting differences in spectral characteristics between various land covers. Traditional methods originated from the signal processing perspective, and most of them rely heavily on specific model assumptions. Because of the model-driven attributes, such methods cannot mine the deep-level features of data to adapt to the variability of scenes and cannot fully extract the information of land covers contained in images to accurately separate anomalies from the background. By independently designing a chessboard-shaped topological framework that avoids making any distribution assumptions but directly mines high-dimensional data features to break through the limitations of traditional detectors, this article proposes a novel chessboard topology-based anomaly detection (CTAD) method to dissect images and extract detailed information of land covers adaptively, thereby enabling highly accurate detection. Extensive experimental results on hyperspectral images (HSIs) in real scenes demonstrate that the proposed CTAD can be adapted to the variability of scenes by autonomously learning data features and exhibiting strong generalization and detection capabilities, facilitating practical applications.

Recent years have witnessed the flourishing of deep learning-based methods in hyperspectral anomaly detection (HAD). However, the lack of available supervision information persists throughout. In addition, existing unsupervised learning/semisupervised learning methods to detect anomalies utilizing reconstruction errors not only generate backgrounds but also reconstruct anomalies to some extent, complicating the identification of anomalies in the original hyperspectral image (HSI). In order to train a network able to reconstruct only background pixels (instead of anomalous pixels), in this article, we propose a new blind-spot self-supervised learning network (called BS3LNet) that generates training patch pairs with blind spots from a single HSI and trains the network in self-supervised fashion. The BS3LNet tends to generate high reconstruction errors for anomalous pixels and low reconstruction errors for background pixels due to the fact that it adopts a blind-spot architecture, i.e., the receptive field of each pixel excludes the pixel itself and the network reconstructs each pixel using its neighbors. The above characterization suits the HAD task well, considering the fact that spectral signatures of anomalous targets are significantly different from those of neighboring pixels. Our network can be considered a superb background generator, which effectively enhances the semantic feature representation of the background distribution and weakens the feature expression for anomalies. Meanwhile, the differences between the original HSI and the background reconstructed by our network are used to measure the degree of the anomaly of each pixel so that anomalous pixels can be effectively separated from the background. Extensive experiments on two synthetic and three real datasets reveal that our BS3LNet is competitive with regard to other state-of-the-art approaches.

Hyperspectral image super-resolution can compensate for the incompleteness of single-sensor imaging and provide desirable products with both high spatial and spectral resolution. Among them, unmixing-inspired networks have drawn considerable attention owing to their straightforward unsupervised paradigm. However, most do not fully capture and utilize the multi-modal information due to their limited representation ability of constructed networks, hence leaving large room for further improvement. To this end, we propose an X-shaped interactive autoencoders network with cross-modality mutual learning between hyperspectral and multispectral data, XINet for short, to cope with this problem. Generally, it employs a coupled structure equipped with two autoencoders, aiming at deriving latent abundances and corresponding endmembers from input correspondence. Inside the network, a novel X-shaped interactive architecture is designed by coupling two disjointed U-Nets together via a parameter-shared strategy, which not only enables sufficient information flow between two modalities but also leads to informative spatial-spectral features. Considering the complementarity across each modality, a cross-modality mutual learning module is constructed to further transfer knowledge from one modality to another, allowing for better utilization of multi-modal features. Moreover, a joint self-supervised loss is proposed to effectively optimize our proposed XINet, enabling an unsupervised manner without external triplets supervision. Extensive experiments, including super-resolved results in four datasets, robustness analysis, and extension to other applications, are conducted, and the superiority of our method is demonstrated.

International audience

Hyperspectral unmixing aims at estimating pure spectral signatures and their proportions in each pixel. In practice, the atmospheric effects, intrinsic variation of the spectral signatures of the materials, illumination, and topographic changes cause what is known as spectral variability resulting in significant estimation errors being propagated throughout the unmixing task. To this end, we developed a new method, called the orthogonal subspace unmixing (OSU), to address spectral variability by utilizing the orthogonal subspace projection. The proposed OSU method jointly performs orthogonal subspace learning and the unmixing process to find a more suitable subspace for unmixing. The orthogonal subspace projection encourages the representation held in the subspace to be more distinct from each other to remove the complex spectral variability in the subspace. Furthermore, an alternating minimization (AM) was designed to solve the resulting optimization problem. An efficient and convergent symmetric Gauss–Seidel alternating direction method of multipliers (sGS-ADMM), essentially a special case of the semiproximal alternating direction method of multipliers (SPADMM), was developed to solve the subproblem. Experiments conducted on one synthetic data and two real data demonstrate the effectiveness and superiority of the proposed framework in mitigating the effects of spectral variability with respect to classical linear unmixing methods or variability accounting approaches.

Deep learning (DL) has brought many new trends for hyperspectral image classification (HIC). Graph neural networks (GNNs) are models that fuse DL and structured data. Although GNN-based methods have focused on modeling relations, most of them are susceptible to noise, being adverse to capturing hidden correlations from data. Moreover, existing related approaches typically adopt changeless graph structures, which might lead to poor generalization. To solve the problems mentioned above, this paper develops an adaptive cuckoo refinement-based graph transfer network (ACGT-Net) that introduces a meta-heuristic optimization strategy to refine the graph structure. Specifically, we first pre-train a graph convolutional network (GCN) to learn transferable weight parameters. In the undirected graph, nodes are associated with pixels, and edges correspond to similarities between nodes. Afterward, we integrate a cuckoo search strategy (CSS) into the trained GCN to adaptively refine the graph structure. The graph structure refinement (GSR) with the CSS can pay more attention to significant channels by global optimization to improve the generalization of the GNN. Several experiments with real datasets verify the effectiveness and competitiveness of our ACGT-Net compared with other state-of-the-art (SOTA) methods.

Given a priori knowledge, hyperspectral target detection aims to locate objects of interest within specific scenes by utilizing differences in spectral characteristics among various land covers. However, for those traditional model-driven detectors with monotonic analytical mode, they perform mediocrely in the disassembly of hyperspectral image (HSI) data, failing to cope with real scenes with complexity. The discrepancy between fixed model assumptions and HSI data severely reduces detection effects, leading to the inability of such methods to mine deep-level features and adapt to the variability of imaging scenes. To overcome the limitations of traditional methods, we propose a chessboard-shaped topological framework for high-dimensional data structures to disassemble an HSI from both spatial and spectral dimensions adaptively. With hyperspectral target detection is refined into an information retrieval task in a topological space, a target detection method based on chessboard-shaped topology (CTTD) is proposed. In the topological space, latent and hidden data features of original images are presented in an intuitive way. Therefore, the differences in both spatial and spectral dimensions between the two classes of objects, namely target and background, are specifically amplified and exploited to perform the information retrieval task with superior performance. Extensive experimental results on benchmark HSI data sets demonstrate that CTTD can efficiently adapt to the variability of real scenes while extracting abundant and detailed information for accurate target localization. Moreover, both detection effects and computational efficiency exhibited by the proposed method provide a strong support for its popularization in practical applications.

Supervised deep learning (DL)-based hyperspectral images change detection (HSIs-CD) has demonstrated excellent performance; however, current methods require many labeled training samples, and labeling the dataset is labor-intensive, limiting the application of high-precision supervised learning. Besides, there has been a lack of breakthroughs in unsupervised HSIs change detection (CD) methods due to the different feature distributions of bitemporal HSIs. Here, we propose a semi-supervised domain alignment transformer (DA-Former) for HSIs-CD to address the issues with limited samples. Specifically, a dual-branch transformer autoencoder (TAE) is designed, where the middle layer weights of the dual-branch transformer are shared, pulling features from different data into the same space. Moreover, the TAE is also trained cyclically to align the domains. Although the bitemporal HSIs features are cross domain, there is still confusion between the features of different objects. Thus, two fully connected (FC) layers are employed to classify the HSIs middle features extracted by the TAE into changed class or unchanged class with limited labeled data. Three HSIs-CD datasets are used to test this method, showing that TAE can align bitemporal HSIs domains and achieve the highest accuracy compared with benchmark approaches. The code of the proposed method will be published athttps://github.com/yanhengwang-heu/IEEE_TGRS_DA-Former.

Hyperspectral anomaly detection (HAD) aims to identify anomalous targets that deviate from the surrounding background in unlabeled hyperspectral images (HSIs). Most existing deep networks that exploit reconstruction errors to detect anomalies are prone to fit anomalous pixels, thus yielding small reconstruction errors for anomalies, which is not favorable for separating targets from HSIs. In order to achieve a superior background reconstruction network for HAD purposes, this paper proposes a self-supervised blind-block network (termed BockNet) with a guard window. BockNet creates a blind-block (guard window) in the center of the network’s receptive field, rendering it unable to see the information inside the guard window when reconstructing the central pixel. This process seamlessly embeds a sliding dual-window model into our BockNet, in which the inner window is the guard window and the outer window is the receptive field outside the guard window. Naturally, BockNet utilizes only the outer window information to predict/reconstruct the central pixel of the perceptive field. During the reconstruction of pixels inside anomalous targets of varying sizes, the targets typically fall into the guard window, weakening the contribution of anomalies to the reconstruction results so that those reconstructed pixels converge to the background distribution of the outer window area. Accordingly, the reconstructed HSI can be deemed as a pure background HSI, and the reconstruction error of anomalous pixels will be further enlarged, thus improving the discrimination ability of the BockNet model for anomalies. Extensive experiments on four datasets illustrate the competitive and satisfactory performance of our BockNet compared to other state-of-the-art detectors.

In multimodal data fusion and land-cover interpretation tasks, the fusion interpretability between hyperspectral image (HSI) and light detection and ranging (LiDAR) data is always nontrivial to be clarified. Furthermore, the heterogeneous sample and distribution variances of these two remote sensing (RS) modalities impede the joint classification performance. In this paper, a Hyperspectral Intrinsic Image Decomposition guided Data Fusion Network (HI2D2FNet) is proposed. Generally, classic hyperspectral intrinsic image decomposition (HIID) performs well in image enhancement and shadow removal. It decomposes one HSI into one reflectance component and one shading component. Inspired by the core mechanism of HIID, our motivation is to preliminarily exploit its potential for multimodal RS data fusion and explore the inherent modality connection between HSI and LiDAR data from the intrinsic perspective. Specifically, compared with existing techniques, HI2D2FNet is capable of fusing the horizontal geometry information in the shading component with the vertical geometry information in the LiDAR data from both sample and distribution perspectives in the spatial domain. The generated cross-modal geometry feature contributes to guiding the reflectance stream optimization. This unique fusion framework connects both modalities with respect to geometry information and enhances the specific fusion interpretability. The decomposition and fusion modules in HI2D2FNet are optimized simultaneously in a novel alternative optimization pattern. Furthermore, several unique cross-modal constraints in terms of prior RS properties are presented. Experiments conducted on three widely available datasets prove the superiority of HI2D2FNet over state-of-the-art techniques. The source codes will be available at https://github.com/GEOywb/HI2D2FNet.

Transformer is a powerful tool for capturing long-range dependencies and has shown impressive performance in hyperspectral image (HSI) classification. However, such power comes with a heavy memory footprint and huge computation burden. In this paper, we propose two types of lightweight self-attention modules (a channel lightweight multi-head self-attention module and a position lightweight multi-head self-attention module) to reduce both memory and computation while associating each pixel or channel with global information. Moreover, we discover that transformers are ineffective in explicitly extracting local and multi-scale features due to the fixed input size and tend to overfit when dealing with a small number of training samples. Therefore, a lightweight transformer (LiT) network, built with the proposed lightweight self-attention modules, is presented. LiT adopts convolutional blocks to explicitly extract local information in early layers and employs transformers to capture long-range dependencies in deep layers. Furthermore, we design a controlled multi-class stratified sampling strategy to generate appropriately sized input data, ensure balanced sampling, and reduce the overlap of feature extraction regions between training and test samples. With appropriate training data, convolutional tokenization, and lightweight transformers, LiT mitigates overfitting and enjoys both high computational efficiency and good performance. Experimental results on several HSI datasets verify the effectiveness of our design.

Arbitrary-oriented object detection (AOOD) is widely used in aerial images because of its efficient object representation. However, current detectors employ the over-standardized feature extraction structure, resulting in detectors has no ability to adaptively readjust feature representations of detection units. Meanwhile, we observe that many detection units could not focus on the objects of interest in their receptive field and are easily affected by the background information and interference targets, leading to the weaking of feature expression ability. We call them sub-optimal detection units. To address this issue, we propose a novel feature enhancement module called fountain feature enhancement module (FFEM). FFEM ingeniously uses the fountain-like structure to reconstruct the features of sub-optimal detection units, generating fountain features that can automatically condense spatial regional features, which effectively enhances detectors’ overall representation ability. Then, a high-performance AOOD detector called fountain fusion net (FFN) is proposed with FFEM embedded, and many novel AOOD components are tested for their progressiveness. We validated our FFN and FFEM using three remote sensing datasets ‒ DOTA, HRSC2016, and UCAS-AOD as well as one scene text dataset‒ICDAR 2015. Extensive experiments demonstrate the effectiveness of our proposed method on improving current detectors to achieve state-of-the-art performance based on this novel idea.

Owing to the powerful feature extraction and representation capabilities, deep learning (DL) has been successfully applied in hyperspectral unmixing (HU). However, only relying on hyperspectral data for unmixing fails to distinguish objects with similar spectral information, resulting in the degradation of unmixing performance. To this end, this paper presents a novel multimodal unmixing network, MUNet for short, by considering the height information of light detection and ranging (LiDAR) data in a squeeze-and-excitation (SE) attention fashion to guide the unmixing process toward a more accurate performance. MUNet is capable of efficiently embedding the height information obtained from LiDAR data into the autoencoder unmixing architecture through the attention mechanism, thereby fusing more spatial information to obtain ideal unmixing results. Experimental results conducted on the real multimodal dataset demonstrate the effectiveness and superiority of the proposed MUNet compared to several state-of-the-art deep unmixing approaches.

Over the past few decades, a large collection of feature ex-traction and classification algorithms have been developed for land cover mapping using remote sensing data. Although these methods have shown the gradually-increasing performance, their potential inevitably meets the bottleneck due to the lack of high-quality and diversified remote sensing bench-mark datasets, particularly for the multimodal cases. Accordingly, this, to a larger extent, limits the development of the corresponding methodologies and the practical application of land cover classification. To this end, we aim in this pa-per to introduce and build several multimodal remote sensing benchmark datasets for land cover classification. Further-more, two new multimodal land cover classification bench-mark datasets, i.e., Berlin and Augsburg, are openly available. Experiments are conducted on the two datasets for evaluating the performance of several multimodal feature learning and classification methods.

Dimensionality reduction (DR) is a common preprocessing technology for hyperspectral images (HSIs). Recently, many neural networks can implement DR to remove the re-dundant information by node embedding. However, numer-ous hidden-layer parameters limit the generalization ability of the node embedding. In this paper, we develop a graph-cut-based node embedding (GCNE) that can be used for DR of HSIs. The embedding can refine correlations by a graph-cut strategy, and it can avoid numerous parameters when using graph models. Moreover, we combine the graph-cut strategy and extreme learning machine (ELM) to achieve HSI classi-fication. The effectiveness of the proposed method is verified by using real HSIs. Compared with other state-of-the-art DR and classification methods, the proposed approach demon-strates very competitive performance.

In this letter, a novel neural architecture search (NAS) method based on reinforcement learning, called RLNAS, is devised to realize the automatic architecture design in the field of hyperspectral unmixing (HU). This method first train the search network in the constructed self-supervised datasets based on hyperspectral images. The block-based searching and weight-sharing strategies are then introduced to reduce the computational cost in the training phase. The final optimal architecture is obtained by optimizing the multi-objective reward function to balance the trade-off between accuracy and computational efficiency. Compared with the state-of-the-art unmixing algorithms, the proposed RLNAS method can yield better unmixing results on synthetic and real hyperspectral datasets, which verifies its effectiveness and superiority. In addition, the proposed method offers promising potential of the NAS for HU.

Deep learning (DL) has been garnering increasing attention in remote sensing (RS) due to its powerful data representation ability. In particular, deep models have been proven to be effective for RS data classification based on a single given modality. However, with one single modality, the ability in identifying the materials remains limited due to the lack of feature diversity. To overcome this limitation, we present a simple but effective multimodal DL baseline by following a deep encoder–decoder network architecture, EndNet for short, for the classification of hyperspectral and light detection and ranging (LiDAR) data. EndNet fuses the multimodal information by enforcing the fused features to reconstruct the multimodal input in turn. Such a reconstruction strategy is capable of better activating the neurons across modalities compared with some conventional and widely used fusion strategies, e.g., early fusion, middle fusion, and late fusion. Extensive experiments conducted on two popular hyperspectral and LiDAR data sets demonstrate the superiority and effectiveness of the proposed EndNet in comparison with several state-of-the-art baselines in the hyperspectral-LiDAR classification task. The codes will be available athttps://github.com/danfenghong/IEEE_GRSL_EndNet, contributing to the RS community.

Owing to easy acquisition and large coverage from the space, multispectral (MS) imaging has garnered growing interest in various applications of remote sensing. However, the limited spectral information of MS data, to a great extent, leads to difficulties in classifying the materials more accurately, particularly for those classes that have very similar visual appearances. To address this issue effectively, we attempt to enhance the spectral resolution of MS imagery, enabling the identification of materials at a more precise level by the means of richer spectral information. More specifically, we propose to learn locality-constrained sparse coding (LCSC) for short, on partially overlapped hyperspectral (HS)-MS pairs (i.e., dictionary). LCSC is capable of capturing neighboring relations well by enforcing the local constraint for each pixel. Such a strategy makes it possible to better reconstruct HS products from MS images and partially overlapped HS images. Reconstruction and unmixing are explored as potential applications to assess the performance of spectral enhancement. Extensive experiments are conducted on two HS-MS data sets in comparison with several state-of-the-art baselines, which demonstrate the effectiveness of the proposed LCSC algorithm in the task of spectral enhancement.

Hyperspectral images (HSIs) usually have finer spectral resolution but coarser spatial resolution than multispectral images (MSIs). To obtain a desired HSI with higher spatial resolution, great research attention has been paid to achieving hyperspectral super-resolution by fusing the observed HSI with an auxiliary MSI of the same scene. However, most of the existing HSI-MSI fusion methods rely either on prior knowledge of the degradation model or on sufficient training data, hindering their practicality and interpretability. In this letter, we propose a novel unsupervised HSI-MSI fusion network with the ability of degradation adaptive learning, namely, UDALN. Specifically, we propose three modules to straightly encode the spatial and spectral transformations across resolutions, i.e., SpaDnet, SpeUnet, and SpeDnet. Through an elaborately designed three-stage unsupervised training strategy, the estimated network parameters can exhibit clear physical meanings of degradation processes and therefore help guarantee a faithful reconstruction of the desired HSI. The experimental results on two widely used hyperspectral datasets demonstrate the effectiveness of our method in comparison to the state-of-the-art HSI-MSI fusion models. (Code available athttps://github.com/JiaxinLiCAS/UDALN_GRSL.)

Node embedding (NE) is conducive to aggregating correlations and relieving the influence of the Hughes phenomenon when processing high-dimensional data. Although some graph neural networks can capture correlations during achieving NE, the application of NE still faces two rigorous challenges: numerous model parameters and poor generalization. In this letter, we propose a new approach for hyperspectral image (HSI) classification, called the graph-cut-based collaborative NEs (GCCNE). Specifically, we develop a graph-cut-based NE (GCNE) to achieve low-dimensional feature representation, which avoids numerous model parameters when using a graph structure. Considering that the graph-cut in a low-dimensional space does not need to set anchors to decrease the calculation amount, we adopt an ensemble framework based on random subspaces (RSs) to implement the GCNE to obtain the collaborative feature sets, enhancing the generalization of feature representation. Afterward, the collaborative feature sets are input in several kernel-based extreme learning machines (KELMs), respectively, classifying pixels. The number of RSs is the same as the number of KELMs. Finally, we acquire an ensemble result associated with each class. The effectiveness and competitiveness of the proposed method are evaluated by using real HSI datasets.

Hyperspectral images (HSIs) exist in directional stripes commonly due to the failure of pushbroom acquisition. These stripes are not only vertically and horizontally oriented but also tend to be oblique. Furthermore, they can also be aperiodic and heavy. To address this problem, we propose a hyperspectral destriping algorithm, namely, GF-destriping. Taking advantage of the high sparsity and strong directionality of stripes in HSIs, Gabor filters are used to detect the stripes band by band first, and then, an advanced inpainting method, FastHyIn, is used to recover to the striped image. The numerical experiments on simulated data and real data sets show that our proposed algorithm is efficient and superior to state-of-the-art HSI destriping algorithms.

Owing to the powerful and automatic representation capabilities, deep learning (DL) techniques have made significant breakthroughs and progress in hyperspectral unmixing (HU). Among the DL approaches, autoencoders (AEs) have become a widely-used and promising network architecture. However, these AE-based methods heavily rely on manual design and may not be a good fit for specific datasets. To unmix hyperspectral images more intelligently, we propose an automatic neural architecture search model for HU, AutoNAS for short, to determine the optimal network architecture by considering channel configurations and convolution kernels simultaneously. In AutoNAS, the self-supervised training mechanism based on hyperspectral images is first designed for generating the training samples of the supernet. Then, the affine parameter sharing strategy is adopted by applying different affine transformations on the supernet weights in the training phase, which enables finding the optimal channel configuration. Furthermore, on the basis of the obtained channel configuration, the evolutionary algorithm with additional computational constraints is introduced into networks to achieve flexible convolution kernel search by evaluating unmixing results of different architectures in the supernet. Extensive experiments conducted on four hyperspectral datasets demonstrate the effectiveness and superiority of the proposed AutoNAS in comparison with several state-of-the-art unmixing algorithms.

With the continuous progress of computer deep learning technology, convolutional neural network (CNN), as a representative approach, provides a unique solution for hyperspectral image (HSI) classification. However, the parameters of CNN can not be well-tuned when the number of training samples is insufficient, resulting in unsatisfactory classification performance. To tackle the thorny problem, a deep ensemble CNN method based on sample expansion for HSI classification is studied in this paper. Specially, spatial information is first extracted and fused with original spectral bands to help classifiers obtain discriminant spectral-spatial features. Then we use the pixel-pair feature (PPF) to expand the number of training samples so that the parameters of CNN structure can be fully trained. In addition, deep ensemble CNN is employed in this paper, enabling the trained model to obtain better generalization ability and more robust classification results. Ultimately, the proposed method is applied to classify four widely used hyperspectral data sets. Experimental results show that the studied approach yields higher classification accuracy than some CNN-based methods even under the condition of small-size training set.

In recent years, deep learning (DL) has attracted increasing attention in hyperspectral unmixing (HU) applications due to its powerful learning and data fitting ability. The autoencoder (AE) framework, as an unmixing baseline network, achieves good performance in HU by automatically learning low-dimensional embeddings and reconstructing data. Nevertheless, the conventional AE-based architecture, which focuses more on the pixel-level reconstruction loss, tends to lose some significant detailed information of certain materials (e.g., material-related properties) in the reconstruction process. Therefore, inspired by the perception mechanism, we propose a cycle-consistency unmixing network, called CyCU-Net, by learning two cascaded AEs in an end-to-end fashion, to enhance the unmixing performance more effectively. CyCU-Net is capable of reducing the detailed and material-related information loss in the process of reconstruction by relaxing the original pixel-level reconstruction assumption to cycle consistency dominated by the cascaded AEs. More specifically, cycle consistency can be achieved by a newly proposed self-perception loss, which consists of two spectral reconstruction terms and one abundance reconstruction term. By taking advantage of the self-perception loss in the network, the high-level semantic information can be well preserved in the unmixing process. Moreover, we investigate the performance gain of CyCU-Net with extensive ablation studies. Experimental results on one synthetic and three real hyperspectral data sets demonstrate the effectiveness and competitiveness of the proposed CyCU-Net in comparison with several state-of-the-art unmixing algorithms.

Tensor-based methods have been widely studied to attack inverse problems in hyperspectral imaging since a hyperspectral image (HSI) cube can be naturally represented as a third-order tensor, which can perfectly retain the spatial information in the image. In this article, we extend the linear tensor method to the nonlinear tensor method and propose a nonlinear low-rank tensor unmixing algorithm to solve the generalized bilinear model (GBM). Specifically, the linear and nonlinear parts of the GBM can both be expressed as tensors. Furthermore, the low-rank structures of abundance maps and nonlinear interaction abundance maps are exploited by minimizing their nuclear norm, thus taking full advantage of the high spatial correlation in HSIs. Synthetic and real-data experiments show that the low rank of abundance maps and nonlinear interaction abundance maps exploited in our method can improve the performance of the nonlinear unmixing. A MATLAB demo of this work will be available athttps://github.com/LinaZhuangfor the sake of reproducibility.

Deep learning (DL) has aroused wide attention in hyperspectral unmixing (HU) owing to its powerful feature representation ability. As a representative of unsupervised DL approaches, autoencoder (AE) has been proven to be effective to better capture nonlinear components of hyperspectral images than the traditional model-driven linearized methods. However, only using hyperspectral images for unmixing fails to distinguish objects in complex scene, especially for different endmembers with similar materials. To overcome this limitation, we propose a novel multimodal unmixing network for hyperspectral images, called MUNet, by considering the height differences of light detection and ranging (LiDAR) data in a squeeze-and-excitation (SE)-driven attention fashion to guide the unmixing process, yielding performance improvement. MUNet is capable of fusing multimodal information and using the attention map derived by LiDAR to aid network that focuses on more discriminative and meaningful spatial information regarding scenes. Moreover, attribute profile (AP) is adopted to extract the geometrical structures of different objects to better model the spatial information of LiDAR. Experimental results on synthetic and real datasets demonstrate the effectiveness and superiority of the proposed method compared with several state-of-the-art unmixing algorithms. The codes will be available athttps://github.com/hanzhu97702/IEEE_TGRS_MUNet, contributing to the remote sensing community.

Hyperspectral (HS) images are characterized by approximately contiguous spectral information, enabling the fine identification of materials by capturing subtle spectral discrepancies. Owing to their excellent locally contextual modeling ability, convolutional neural networks (CNNs) have been proven to be a powerful feature extractor in HS image classification. However, CNNs fail to mine and represent the sequence attributes of spectral signatures well due to the limitations of their inherent network backbone. To solve this issue, we rethink HS image classification from a sequential perspective with transformers, and propose a novel backbone network called \ul{SpectralFormer}. Beyond band-wise representations in classic transformers, SpectralFormer is capable of learning spectrally local sequence information from neighboring bands of HS images, yielding group-wise spectral embeddings. More significantly, to reduce the possibility of losing valuable information in the layer-wise propagation process, we devise a cross-layer skip connection to convey memory-like components from shallow to deep layers by adaptively learning to fuse "soft" residuals across layers. It is worth noting that the proposed SpectralFormer is a highly flexible backbone network, which can be applicable to both pixel- and patch-wise inputs. We evaluate the classification performance of the proposed SpectralFormer on three HS datasets by conducting extensive experiments, showing the superiority over classic transformers and achieving a significant improvement in comparison with state-of-the-art backbone networks. The codes of this work will be available at https://github.com/danfenghong/IEEE_TGRS_SpectralFormer for the sake of reproducibility.

An important task of spectral unmixing is to recover the signatures of endmembers from a hyperspectral dataset in which no pure signature is exposed. Most algorithms are based on linear mixing model and require that data should be sufficiently scattered to guarantee model uniqueness. However, data may not be scattered sufficiently enough and become incomplete. In this case, can we still recover the endmembers accurately? Moreover, if we wrongly estimate an endmember, the error may be propagated to other endmembers. For these purposes, we propose a new model, namely, affine simplicial cone (ASC), to capture the local geometric feature of data. This model requires less geometric information and relaxes the conditions of model uniqueness. Then, we present analyses on the error propagation of turbulent endmembers and the condition of local uniqueness. Based on ASC, we present an endmember purification problem to recover only one endmember. In this way, error propagation can be alleviated, and local uniqueness can be easily satisfied. Finally, we develop an endmember purifying algorithm (EPA) to solve this problem. Our experiments demonstrate that the performance of EPA is competitive to the state-of-the-art unmixing algorithms not only for the synthetic datasets but also for the real hyperspectral remote sensing datasets. We can conclude that the ASC model and the EPA algorithm have the potential capability for hyperspectral data exploration.

Hyperspectral target detection can be described as locating targets of interest within a hyperspectral image based on prior information of targets. The complexity of actual scenes limits the performance of traditional statistical methods that rely on model assumptions, and traditional machine learning methods rely on mapping functions with limited complexity. To address these problems, we propose a Siamese transformer network for hyperspectral image target detection (STTD). The contribution of this article is threefold. First, we propose a novel method of constructing training samples using only the image itself and the limited prior information, which is suitable for target detection based on the Siamese network framework. Second, the Siamese network framework is utilized to solve the problem of similarity metric learning, i.e., make homogeneous features as close as possible and heterogeneous features as far as possible. Third, the most state-of-the-art network, transformer, is applied as the backbone of our proposed Siamese network to extract global features from spectra with long-range dependencies to achieve target detection. Furthermore, we make adaptive improvements to transformer for hyperspectral images. The proposed method shows its unique advantages in suppressing the background to a low level and highlighting the target with high probability. Experiments on five different datasets demonstrate the superiority of the proposed STTD as compared to the state-of-the-art.

Given the prior information of the target, hyperspectral target detection focuses on exploiting spectral differences to separate objects of interest from the background, which can be treated as information retrieval (IR) task in machine learning (ML). Most traditional detection methods work in the original feature space and rely heavily on specific assumptions, which cannot guarantee effective extraction of features for the target and background in hyperspectral images (HSIs). Mass estimation (ME) is a base modeling mechanism that has been proven to effectively solve problems in IR and is not restricted by specific assumptions. In this article, we propose a novel target detection method through ensemble-based IR with ME (EIRME). By directly deriving the ordering from a sample set to rank data points, ME provides a simple and straightforward ranking measure to ensure that points similar to the given target are far away from dissimilar points. For the estimation of mass distribution, the proposed method utilizes a tree-structured mapping to generate a feature space, in which the separability of the target and background is further improved. In particular, to break through the technical difficulty that the direct migration of IR methods with mass measure cannot specifically meet the high-precision requirements of target detection in HSIs, we develop a specialized measurement, topological mass, which innovatively combines the mass measure with tree topology to quantify the spectral difference for detection output. Moreover, the IR with ME based on parallel measurements through ensemble trees provides a robust solution with better generalization capacity and higher precision for hyperspectral target detection, facilitating practical applications. Experimental results on benchmark HSI datasets prove that the specialized measurement that we developed successfully overcomes the drawbacks of the direct migration of IR methods with ME and exhibits unique advantages. In addition, comparisons with the most classic and advanced detection algorithms demonstrate the superiority of the proposed method.

Convolutional neural networks (CNNs) with excellent spatial feature extraction abilities have become popular in remote sensing (RS) image change detection (CD). However, CNNs often focus on the extraction of spatial information but ignore important spectral and temporal sequences for hyperspectral images (HSIs). In this paper, we propose a joint spectral, spatial, and temporal transformer for hyperspectral image change detection (HSI-CD), named SST-Former. First, the SST-Former position-encodes each pixel on the cube to remember the spectral and spatial sequences. Second, a spectral transformer encoder structure is used to extract spectral sequence information. Then, a class token for storing the class information of a single temporal HSI concatenates the output of the spectral transformer encoder. The spatial transformer encoder is used to extract spatial texture information in the next step. Finally, the features of different temporal HSIs are sent as the input of temporal transformer, which is used to extract useful CD features between the current HSI pairs and obtain the binary CD result through multilayer perception (MLP). We evaluate SST-Former on three HSI-CD datasets by numerous experiments, showing that it performs better than other excellent methods both visually and qualitatively.

Recently, deep learning-based denoising methods for hyperspectral images (HSIs) have been comprehensively studied and achieved impressive performance because they can effectively extract complex and nonlinear image features. Compared with deep learning-based methods, the nonlocal similarity-based denoising methods are more suitable for images containing edges or regular textures. We propose a powerful HSI denoising method, termed NL-3DCNN, combining traditional machine learning and deep learning techniques. NL-3DCNN exploits the high spectral correlation of an HSI by using subspace representation and corresponding representation coefficients are termed eigenimages. The high spatial correlation in eigenimages is exploited by grouping nonlocal similar patches, which are denoised by a 3D convolutional neural network. The numerical and graphical denoising results of simulated and real data show that the proposed method is superior to state-of-the-art methods.

Due to the limitations of imaging systems, satellite hyperspectral imagery (HSI), which yields rich spectral information in many channels, often suffers from poor spatial resolution. HSI super-resolution (SR) refers to the fusion of high spatial resolution multispectral imagery (MSI) and low spatial resolution HSI to generate HSI that has both a high spatial and high spectral resolution. However, most existing SR methods assume that the two original images used are perfectly registered: in reality, nonrigid deformation areas can exist locally in the two images even if prior registration of the control points has been carried out. To address this problem, we propose a novel unsupervised spectral unmixing and image deformation correction network—NonRegSRNet—with multimodal and multitask learning that can be used for the joint registration of HSI and MSI and to produce SR imagery. More specifically, NonRegSRNet integrates the dense registration and SR tasks into a unified model that includes a triplet convolutional neural network. This allows these two tasks to complement each other so that better registration and SR results can be achieved. Furthermore, because the point spread function (PSF) and spectral response function (SRF) are often unavailable, two special convolutional layers are designed to adaptively learn the parameters of the PSF and SRF, which makes the proposed model more adaptable. Experimental results demonstrate that the proposed method has the ability to produce highly accurate and stable reconstructed images under complex nonrigid deformation conditions. (Code available athttps://github.com/saber-zero/NonRegSRNet)

Hyperspectral imaging measures the amount of electromagnetic energy across the instantaneous field of view at a very high resolution in hundreds or thousands of spectral channels. This enables objects to be detected and the identification of materials that have subtle differences between them. However, the increase in spectral resolution often means that there is a decrease in the number of photons received in each channel, which means that the noise linked to the image formation process is greater. This degradation limits the quality of the extracted information and its potential applications. Thus, denoising is a fundamental problem in hyperspectral image (HSI) processing. As images of natural scenes with highly correlated spectral channels, HSIs are characterized by a high level of self-similarity and can be well approximated by low-rank representations. These characteristics underlie the state-of-the-art methods used in HSI denoising. However, where there are rarely occurring pixel types, the denoising performance of these methods is not optimal, and the subsequent detection of these pixels may be compromised. To address these hurdles, in this article, we introduce RhyDe (Robust hyperspectral Denoising), a powerful HSI denoiser, which implements explicit low-rank representation, promotes self-similarity, and, by using a form of collaborative sparsity, preserves rare pixels. The denoising and detection effectiveness of the proposed robust HSI denoiser is illustrated using semireal and real data.

Over the past decades, enormous efforts have been made to improve the performance of linear or nonlinear mixing models for hyperspectral unmixing (HU), yet their ability to simultaneously generalize various spectral variabilities (SVs) and extract physically meaningful endmembers still remains limited due to the poor ability in data fitting and reconstruction and the sensitivity to various SVs. Inspired by the powerful learning ability of deep learning (DL), we attempt to develop a general DL approach for HU, by fully considering the properties of endmembers extracted from the hyperspectral imagery, called endmember-guided unmixing network (EGU-Net). Beyond the alone autoencoder-like architecture, EGU-Net is a two-stream Siamese deep network, which learns an additional network from the pure or nearly pure endmembers to correct the weights of another unmixing network by sharing network parameters and adding spectrally meaningful constraints (e.g., nonnegativity and sum-to-one) toward a more accurate and interpretable unmixing solution. Furthermore, the resulting general framework is not only limited to pixelwise spectral unmixing but also applicable to spatial information modeling with convolutional operators for spatial-spectral unmixing. Experimental results conducted on three different datasets with the ground truth of abundance maps corresponding to each material demonstrate the effectiveness and superiority of the EGU-Net over state-of-the-art unmixing algorithms. The codes will be available from the website: https://github.com/danfenghong/IEEE_TNNLS_EGU-Net.

Owing to the powerful ability in learning low-dimensional representations and reconstruction, autoencoders (AEs) have been successfully applied in hyperspectral unmixing (HU). However, AE-based unmixing architectures, to a great extent, need to be carefully designed in a manual fashion, leading to the bulk of costs in manpower and time. To unmix hyperspectral images more intelligently, we propose an AI-powered evolvable neural architecture search method for HU, EvoNAS for short, to optimally determine the network architecture by the means of the evolutionary algorithm instead of gradient-based or reinforcement learning-based rewards. In EvoNAS, a supernet with all candidate architectures is first trained to learn the unmixing mapping in a self-supervised manner. The optimal network is then constructed by evaluating unmixing results of different architectures in the supernet. EvoNAS is capable of saving tremendous computational cost, since it inherits the weights of the pre-trained supernet and avoids training from scratch during the search phase. Experimental results conducted on two real hyperspectral datasets verify the effectiveness and superiority of the EvoNAS and show the huge potential of the NAS for HU.

With the ever-growing availability of remote sensing (RS) data from either satellite or airborne sensors, simultaneous processing and analysis of multimodal data have been paid more and more attention by researchers in various RS-related applications. In this paper, we propose a multimodal convolutional neural network with an advanced cross-channel reconstruction module, called CCR-Net. As the name suggests, CCR-Net enables a more compact fusion of different RS data sources by the means of the reconstruction strategy across modalities that can mutually exchange information in a more effective way. Experiment are conducted on a widely-used dataset, including hyperspectral and Light Detection and Ranging (LiDAR) data, i.e., Houston2013, to verify the effectiveness and superiority of the proposed CCR - N et in comparison with several state-of-the-art baseline methods.

Convolution neural network (CNN) has been successfully applied to hyperspectral image classification. However, multiclass imbalance is a major problem in the classification of hyper spectral images, and traditional CNN can hardly improve the accuracy of minority classes effectively. In this paper, a new ensemble CNN with enhanced feature subspaces (ECNN-EFSs) algorithm is proposed, which utilizes an imbalanced training set to train the model and achieves accurate classification. Experimental results on two common hyperspectral datasets show that the proposed algorithm outperforms the traditional CNN and ensemble CNN algorithms.

The combined use of spatial information and spectral information has been widely applied to hyperspectral image (HSI) classification. In recent years, multiscale spatial-spectral convolutional neural networks (CNN) have been introduced for hyperspectral image classification (HSIC). However, most of HSIC methods based on CNN mainly use patches as input for classifier. This may cause a lot of redundancy in the training and testing process, and reduce the efficiency of the model. In order to address this problem, we design a novel image-based classification framework. Based on this framework, we propose a multi-scale dense network for HSIs, called HyMSDN. This network merges features from different scales through a feature pyramid structure. Experimental results on real hyperspectral dataset verify the efficiency and effectiveness of the proposed framework, with superior performances compared with other related methods.

Over the past decades, numerous methods have been proposed to solve the linear or nonlinear mixing problems in hyperspectral unmixing (HU). The existence of spectral variabilities and nonlinearity limits, to a great extent, the unmixing ability of most traditional approaches, particularly in complex scenes. In recent years, deep learning (DL) has been garnering increasing attention in nonlinear HU owing to its powerful learning and fitting ability. However, the DL-based methods tend to generate trivial unmixing results due to the lack of considering physically meaningful endmember information. To this end, we propose a novel siamese network, called the deep half-siamese network (Deep HSNet), for HU by fully considering diverse endmember properties extracted using different endmember extraction algorithms. Moreover, the proposed Deep HSNet, beyond the previous autoencoder-like architecture, adopts another subnetwork to learn the endmember information effectively to guide the unmixing process in a reasonable and accurate way. The experimental results conducted on the synthetic and real hyperspectral data sets validate the effectiveness and superiority of the Deep HSNet over several state-of-the-art unmixing algorithms.

Sparse representation (SR)-based models have shown to be a powerful category of frameworks for hyperspectral image classification (HSIC). However, current residual-driven methods mainly focus on the sparsity of the coefficient, which is generally used in conjunction with the dictionary. In fact, the discriminant information hidden behind the value of sparse coefficient is not fully exploited. In this letter, we analyze the SR-based framework from the perspective of sparse coefficient, develop the participation degree (PD)-driven decision mechanism, and establish a concise model called constraint representation (CR). Based on CR, an improved version called adjacent CR (ACR) is further proposed, with consideration of spatial coherence via adjacent constraint. Experimental results using two real hyperspectral datasets verify the improvements of the proposed methods over the other related models and their spatial variants.

Super-resolution (SR) techniques play a crucial role in increasing the spatial resolution of remote sensing data and overcoming the physical limitations of the spaceborne imaging systems. Though the convolutional neural network (CNN)-based methods have obtained good performance, they show limited capacity when coping with large-scale super-resolving tasks. The more complicated spatial distribution of remote sensing data further increases the difficulty in reconstruction. This article develops a dense-sampling super-resolution network (DSSR) to explore the large-scale SR reconstruction of the remote sensing imageries. Specifically, a dense-sampling mechanism, which reuses an upscaler to upsample multiple low-dimension features, is presented to make the network jointly consider multilevel priors when performing reconstruction. A wide feature attention block (WAB), which incorporates the wide activation and attention mechanism, is introduced to enhance the representation ability of the network. In addition, a chain training strategy is proposed to optimize further the performance of the large-scale models by borrowing knowledge from the pretrained small-scale models. Extensive experiments demonstrate the effectiveness of the proposed methods and show that the DSSR outperforms the state-of-the-art models in both quantitative evaluation and visual quality.

Remotely sensed images, especially in urban areas, have highly complex spatial distribution, since the ground objects have diverse ranges of sizes and shapes. This largely increases the difficulty of super-resolution (SR) tasks. Current deep convolutional neural network (CNN)-based SR methods often show limited performance when coping with complicated images. This article develops a second-order multi-scale super-resolution network (SMSR) to explore reconstruction tasks for difficult cases. Specifically, we propose a single-path feature reuse which cleverly captures multi-scale feature information through aggregating the features learned at different depths of a single path. Further, we present a second-order learning mechanism, which double reuses small-difference and large-difference features at local and global levels, makes use of the learned multi-scale information at maximum. The proposed methods achieve multi-scale learning using small-size convolution only, resulting in a lightweight and high-performance SR network. Experimental results show the superiority of our SMSR over state-of-the-art methods in super-resolving complicated image patterns. The effectiveness of SMSR is also demonstrated through its support to object recognition task.

Extensive attention has been widely paid to enhance the spatial resolution of hyperspectral (HS) images with the aid of multispectral (MS) images in remote sensing. However, the ability in the fusion of HS and MS images remains to be improved, particularly in large-scale scenes, due to the limited acquisition of HS images. Alternatively, we super-resolve MS images in the spectral domain by the means of partially overlapped HS images, yielding a novel and promising topic: spectral superresolution (SSR) of MS imagery. This is challenging and less investigated task due to its high ill-posedness in inverse imaging. To this end, we develop a simple but effective method, called joint sparse and low-rank learning (J-SLoL), to spectrally enhance MS images by jointly learning low-rank HS-MS dictionary pairs from overlapped regions. J-SLoL infers and recovers the unknown HS signals over a larger coverage by sparse coding on the learned dictionary pair. Furthermore, we validate the SSR performance on three HS-MS data sets (two for classification and one for unmixing) in terms of reconstruction, classification, and unmixing by comparing with several existing state-of-the-art baselines, showing the effectiveness and superiority of the proposed J-SLoL algorithm. Furthermore, the codes and data sets will be available at https://github.com/danfenghong/IEEE_TGRS_J-SLoL, contributing to the remote sensing (RS) community.

Convolutional neural networks (CNNs) are becoming increasingly popular in modern remote sensing image processing tasks and exhibit outstanding capability for hyperspectral image (HSI) classification. However, for the existing CNN-based HSI-classification methods, most of them only consider single-scale feature extraction, which may neglect some important fine information and cannot guarantee to capture optimal spatial features. Moreover, many state-of-the-art methods have a huge number of network parameters needed to be tuned, which will cause high computational cost. To address the aforementioned two issues, a novel multiscale residual network (MSRN) is proposed for HSI classification. Specifically, the proposed MSRN introduces depthwise separable convolution (DSC) and replaces the ordinary depthwise convolution in DSC with mixed depthwise convolution (MDConv), which mixes up multiple kernel sizes in a single depthwise convolution operation. The DSC with mixed depthwise convolution (MDSConv) can not only explore features at different scales from each feature map but also greatly reduce learnable parameters in the network. In addition, a multiscale residual block (MRB) is designed by replacing the convolutional layer in an ordinary residual block with the MDSConv layer. The MRB is used as the major unit of the proposed MSRN. Furthermore, to enhance further the feature representation ability, the proposed network adds a high-level shortcut connection (HSC) on the cascaded two MRBs to aggregate lower level features and higher level features. Experimental results on three benchmark HSIs demonstrate the superiority of the proposed MSRN method over several state-of-the-art methods.

Classification and identification of the materials lying over or beneath the earth's surface have long been a fundamental but challenging research topic in geoscience and remote sensing (RS), and have garnered a growing concern owing to the recent advancements of deep learning techniques. Although deep networks have been successfully applied in single-modality-dominated classification tasks, yet their performance inevitably meets the bottleneck in complex scenes that need to be finely classified, due to the limitation of information diversity. In this work, we provide a baseline solution to the aforementioned difficulty by developing a general multimodal deep learning (MDL) framework. In particular, we also investigate a special case of multi-modality learning (MML)-cross-modality learning (CML) that exists widely in RS image classification applications. By focusing on “what,” “where,” and “how” to fuse, we show different fusion strategies as well as how to train deep networks and build the network architecture. Specifically, five fusion architectures are introduced and developed, further being unified in our MDL framework. More significantly, our framework is not only limited to pixel-wise classification tasks but also applicable to spatial information modeling with convolutional neural networks (CNNs). To validate the effectiveness and superiority of the MDL framework, extensive experiments related to the settings of MML and CML are conducted on two different multimodal RS data sets. Furthermore, the codes and data sets will be available at https://github.com/danfenghong/IEEE_TGRS_MDL-RS, contributing to the RS community.

Convolutional neural networks (CNNs) have been attracting increasing attention in hyperspectral (HS) image classification due to their ability to capture spatial-spectral feature representations. Nevertheless, their ability in modeling relations between the samples remains limited. Beyond the limitations of grid sampling, graph convolutional networks (GCNs) have been recently proposed and successfully applied in irregular (or nongrid) data representation and analysis. In this article, we thoroughly investigate CNNs and GCNs (qualitatively and quantitatively) in terms of HS image classification. Due to the construction of the adjacency matrix on all the data, traditional GCNs usually suffer from a huge computational cost, particularly in large-scale remote sensing (RS) problems. To this end, we develop a new minibatch GCN (called miniGCN hereinafter), which allows to train large-scale GCNs in a minibatch fashion. More significantly, our miniGCN is capable of inferring out-of-sample data without retraining networks and improving classification performance. Furthermore, as CNNs and GCNs can extract different types of HS features, an intuitive solution to break the performance bottleneck of a single model is to fuse them. Since miniGCNs can perform batchwise network training (enabling the combination of CNNs and GCNs), we explore three fusion strategies: additive fusion, elementwise multiplicative fusion, and concatenation fusion to measure the obtained performance gain. Extensive experiments, conducted on three HS data sets, demonstrate the advantages of miniGCNs over GCNs and the superiority of the tested fusion strategies with regard to the single CNN or GCN models. The codes of this work will be available at https://github.com/danfenghong/IEEE_TGRS_GCN for the sake of reproducibility.

Hyperspectral image (HSI) classification is one of the most active research topics and has achieved promising results boosted by the recent development of deep learning. However, most state-of-the-art approaches tend to perform poorly when the training and testing images are on different domains, e.g., the source domain and target domain, respectively, due to the spectral variability caused by different acquisition conditions. Transfer learning-based methods address this problem by pretraining in the source domain and fine-tuning on the target domain. Nonetheless, a considerable amount of data on the target domain has to be labeled and nonnegligible computational resources are required to retrain the whole network. In this article, we propose a new transfer learning scheme to bridge the gap between the source and target domains by projecting the HSI data from the source and target domains into a shared abundance space based on their own physical characteristics. In this way, the domain discrepancy would be largely reduced such that the model trained on the source domain could be applied to the target domain without extra efforts for data labeling or network retraining. The proposed method is referred to as physically constrained transfer learning through shared abundance space (PCTL-SAS). Extensive experimental results demonstrate the superiority of the proposed method as compared to the state of the art. The success of this endeavor would largely facilitate the deployment of HSI classification for real-world sensing scenarios.

Target detection aims to locate targets of interest within a specific scene. The traditional model-driven detectors based on signal processing have proved to be very effective. However, the detection performance of such traditional methods relies heavily on the model assumption, which is limited by the discrepancy with real hyperspectral images (HSIs) data. In this article, a target detection method through tree-structured encoding (TD-TSE) for HSIs is proposed. Instead of modeling the target and the background to extract valid features, we construct a binary tree based on the features of the data itself and segment the HSI to improve the separability of the target and the background. For the purpose of highlighting the target and suppressing the background, a novel measurement of separation, distance on tree, is calculated via binary encoding based on the constructed tree structure, and the detection output can be obtained according to such distance. To further reduce the generalization error resulting from random subsampling, the statistical average of the distances on multiple independent trees is estimated to improve the robustness of TD-TSE. The proposed method is not constrained by any model assumptions, which is fundamentally different from the most widely used hyperspectral target detectors in the field of signal processing. Moreover, the construction of binary trees without any labeled samples and the linear complexity of the proposed method make it highly practical for the hyperspectral data in real scenes. Extensive experiments on three benchmark HSI data sets demonstrate the effectiveness of the proposed TD-TSE for hyperspectral target detection.

Due to the limitations of hyperspectral imaging systems, hyperspectral imagery (HSI) often suffers from poor spatial resolution, thus hampering many applications of the imagery. Hyperspectral super resolution refers to fusing HSI and MSI to generate an image with both high spatial and high spectral resolutions. Recently, several new methods have been proposed to solve this fusion problem, and most of these methods assume that the prior information of the point spread function (PSF) and spectral response function (SRF) are known. However, in practice, this information is often limited or unavailable. In this work, an unsupervised deep learning-based fusion method-HyCoNet-that can solve the problems in HSI-MSI fusion without the prior PSF and SRF information is proposed. HyCoNet consists of three coupled autoencoder nets in which the HSI and MSI are unmixed into endmembers and abundances based on the linear unmixing model. Two special convolutional layers are designed to act as a bridge that coordinates with the three autoencoder nets, and the PSF and SRF parameters are learned adaptively in the two convolution layers during the training process. Furthermore, driven by the joint loss function, the proposed method is straightforward and easily implemented in an end-to-end training manner. The experiments performed in the study demonstrate that the proposed method performs well and produces robust results for different data sets and arbitrary PSFs and SRFs.

Collaborative Representation Detection (CRD) is a very effective anomaly detection method, which is directly based on the concept that pixel under test (PUT) can be approximately linear represented by its spatial adjacent background pixels. If the adjacent background pixels are contaminated, the approximate value of PUT linearly represented by the surrounding pixels is inaccurate. In this work, an improved method for anomaly detection in hyperspectral imagery is proposed based on CRD. In our proposed method, the least squares technique first is adopted to obtain the preliminary linear representation coefficient, which is positively correlated with its contribution to PUT. Then, the purified background pixels are obtained according to the numerical value of the representation coefficient. Generally, the anomaly pixels are usually different from the background pixels, so saliency weight is imposed on the test pixel to make full use of the spatial information of inner window pixels around the test pixel. Extensive experiments for real hyperspectral datasets show that the proposed method outperforms the CRD method and other traditional detection methods.

Sparse representation (SR)-based models have been widely applied for hyperspectral image classification. However, the original residual-driven frameworks ignore the property of sparse coefficient to some extent, and their spatial variants suffer obstacles of optimization due to the strong constraint. In this paper, based on previous works on sparse coefficient and its spatial expansion, we put forward a novel classifier, called superpixel-level constraint representation (SPCR). In particular, constraint representation (CR) is first applied to interpret the process of SR from perspective of participation degree (PD). Then, a relaxed and adaptive spatial constraint via superpixel segmentation is imposed to transform the individual PD to local relative activity degree (RAD). The final classification is determined based on a concise RAD-driven mechanism. Experimental results on real data set demonstrate the efficiency of the proposed method.

Hyperspectral images (HSIs), featured by high spectral resolution over a wide range of electromagnetic spectra, have been widely used to characterize materials with subtle differences in the spectral domain. However, a large number of bands and an insufficient number of sample pixels for each class are challenging for traditional machine learning-based classifiers. As alternative tools for feature extraction, neural networks have received extensive attention. This letter proposes to combine t-distributed stochastic neighbor embedding (t-SNE) with a convolutional neural network (CNN) for HSI classification. Our framework is designed to automatically capture the potential assembly features, which are extracted from both the dimension-reduced CNN (DR-CNN) and the multiscale-CNN. Experimental results show that the proposed classification framework outperforms several state-of-the-art techniques for three real data sets.

Multisensor fusion is of great importance in Earth observation related applications. For instance, hyperspectral images (HSIs) provide wealthy spectral information while light detection and ranging (LiDAR) data provide elevation information, and using HSI and LiDAR data together can achieve better classification performance. In this paper, an unsupervised feature extraction framework, named as patch-to-patch convolutional neural network (PToP CNN), is proposed for collaborative classification of hyperspectral and LiDAR data. More specific, a three-tower PToP mapping is first developed to seek an accurate representation from HSI to LiDAR data, aiming at merging multiscale features between two different sources. Then, by integrating hidden layers of the designed PToP CNN, extracted features are expected to possess deeply fused characteristics. Accordingly, features from different hidden layers are concatenated into a stacked vector and fed into three fully connected layers. To verify the effectiveness of the proposed classification framework, experiments are executed on two benchmark remote sensing data sets. The experimental results demonstrate that the proposed method provides superior performance when compared with some state-of-the-art classifiers, such as two-branch CNN and context CNN.

Spectral-spatial framework has been widely applied for hyperspectral image classification task. Some well-established models, such as group sparse representation (GSR), have gained a certain advance but still mainly focus on the usage of local spatial similarity and neglect the nonlocal spatial information. Recently, nonlocal self-similarity (NLSS) has been exploited to support the spatial coherence tasks. However, current NLSS-based methods are biased toward the direct use of nonlocal spatial information as a whole, while the underlying spectral information is not well exploited. In this article, we proposed a novel method to exploit local spectral similarity through nonlocal spatial similarity, with the integration of local spatial consistency in a single framework. Specifically, the proposed approach first exploits the NLSS by searching the nonoverlapped similar patches in defined scopes. Then, spectral similarity is determined locally within the found patches. After that, the found similar data and the original data are fused in a designed pattern. Finally, the GSR-based classifier (GSRC) is applied to process the fused data characterized by the manifold learning algorithm. The experimental results based on three real hyperspectral data sets demonstrate the efficiency of the proposed method, with improvements over the other related nonlocal or local similarity-based methods.

Early detection of banana disease can limit the spread of disease, as well as reduce the treatment costs. However, the disease symptoms are so unapparent in the earlier stage that makes the labeled samples acquisition difficult and expensive. Meanwhile, it is much easier to obtain labeled samples at the late stage where the disease symptoms are obvious. In this paper, we exploit machine learning methods to use labeled samples from the late stage to train the model, then detect the banana disease in the earlier stage. Morphological openings and closings are utilized to extract the spectral-spatial features from banana leaves at both earlier and late stages, initial experimental results demonstrate significant improvements over using only spectral information.

Spectral unmixing (SU) is a useful tool for hyperspectral remote sensing image analysis. However, due to the interference of spectral variance and non‐linearity caused by photon multiple‐scattering, the result might be an inaccuracy. In addition, the unmixing performance of typically relies on the prior knowledge of endmembers. Although many classical endmember extraction algorithms have been presented, it is hard to obtain accurate endmembers in practical applications. This study presents a bilinear normal mixing model named as BNMM to tackle these issues. In fact, BNMM employs the polynomial post‐non‐linear mixing model to alleviate the effect of non‐linearity and uses a normal distribution model to reduce the influence of endmembers variability. Based on the BNMM, the authors develop a Hamiltonian Monte Carlo algorithm for SU. The experimental results demonstrate that the proposed algorithm outperforms other classical unmixing algorithms in the case of simulated and benchmark datasets.

Large-scale Landsat image classification is essential for the production of land cover maps. The rise of convolutional neural networks (CNNs) provides a new idea for the implementation of Landsat image classification. However, pixels in Landsat images have higher uncertainty compared with high-resolution images due to its 30-m spatial resolution. In addition, the current deep learning methods tend to lose detailed information such as boundaries along with the stacking of convolutional and pooling layers. To solve these problems, we propose a new method called entropy and MRF model (EMM)-CNN based on Pyramid Scene Parsing Network. The EMM-CNN uses entropy to decrease the uncertainty of pixels. Then, the Markov random filed (MRF) model is employed to construct the connections between neighboring pixels and defined a prior distribution to prevent the cross entropy from sacrificing detailed information for the overall accuracy. Finally, transfer learning based on the pretrained ImageNet is introduced to overcome the shortage of training samples and boost the speed of the training process. Experimental results demonstrate that the proposed EMM-CNN is able to obtain classification results with fine structure by decreasing the uncertainty and retaining detailed information of the detected image.

Ship detection is an important and challenging topic in remote sensing applications. In current literatures, sea-land segmentation is generally requested before ship detection. This makes the implementation of the methods highly complicated. Therefore, based on Faster R-CNN, this paper proposes a ship detection method for large-scale images, which does not need sea-land segmentation as preprocessing step and can detect ships directly from complicated background including sea and land. We use large-scale images consisting of GF-1 and GF-2 satellite images to test our network. Experimental results prove that the proposed method plays a role in removing the interference of objects on land.

A framework based on color feature extraction of social media photos and correlation analysis with air quality parameters is proposed to monitor environmental change. More specifically, photos of the Beijing Olympic Park from Panoramio website have been analyzed as a case study. The aerosol optical depth data at 500 nm wavelength (AOD 500) obtained from sun-photometer observation network station has been used as reference. Results show a proof of concept that social media photos have an interesting potential for air pollution estimate and remote sensing parameter validation with a low cost.

Spectral-spatial classification has been widely exploited for hyperspectral imagery. However, current methods either focus on local spatial similarity or global nonlocal self-similarity (NLSS). In this paper, we propose novel methods to couple both global spatial similarity and local spectral similarity together in a single framework. In particular, our approaches exploit global spatial similarity by searching non-overlap nonlocal patches, whereas spectral similarity is determined locally within the found patches. Experimental results on two real hyperspectral data sets demonstrate the efficiency of the proposed methods, with 5%-7% (overall classification accuracy) improvements over approaches that only consider either global or local similarity.

In this letter, a new method for higher order nonlinear hyperspectral unmixing is introduced. The proposed scheme relies on the harmonic description of the endmembers contributions to characterize the interactions among the materials showing up in the given scenes. Moreover, it aims at directly estimating the probability of occurrence of each material in the images, so to provide an accurate quantification of the endmembers also in complex scenarios. Experimental results carried out on synthetic and real data sets show that the proposed method is able to obtain good unmixing performance when compared to other state-of-the-art architectures.

As a list of remotely sensed data sources is available, how to efficiently exploit useful information from multisource data for better Earth observation becomes an interesting but challenging problem. In this paper, the classification fusion of hyperspectral imagery (HSI) and data from other multiple sensors, such as light detection and ranging (LiDAR) data, is investigated with the state-of-the-art deep learning, named the two-branch convolution neural network (CNN). More specific, a two-tunnel CNN framework is first developed to extract spectral-spatial features from HSI; besides, the CNN with cascade block is designed for feature extraction from LiDAR or high-resolution visual image. In the feature fusion stage, the spatial and spectral features of HSI are first integrated in a dual-tunnel branch, and then combined with other data features extracted from a cascade network. Experimental results based on several multisource data demonstrate the proposed two-branch CNN that can achieve more excellent classification performance than some existing methods.

The added value of multiple data sources on tree species mapping has been widely analyzed. In particular, fusion of hyperspectral (HS) and LiDAR sensors for forest applications is a very hot topic. In this paper, we exploit the use of multi-scale features to fuse HS and LiDAR data for tree species mapping. Hyperspectral data is obtained from the APEX sensor with 286 spectral bands. LiDAR data has been acquired with a TopoSys sensor Harrier 56 at full waveform. We generate multi-scale features on both HS and LiDAR data, by considering the diameter and the height layer of different tree species. Experimental results on a forested area in Belgium demonstrate the effectiveness of using multi-scale features for fusion of HS image and LiDAR data both visually and quantitatively.

This paper proposes a new supervised hyperspectral nonlinear unmixing method based on normalization. The main contribution is presented by reducing the overfitting of model and taking account to spatial correlation, using the normalization. The l2-norm constraints of abundance and nonlinear coefficient are added to the P-Linear spectral mixing model. Moreover different positive parameters are given to control the trade-off between regularity and fitting. Finally, the problem can be expressed as a convex optimization problem, minimizing the cost function and the global optimum can be determined. The proposed method, abbreviated as NPLA (Normalized P-Linear Algorithm), is validated using hyperspectral synthetic and real datasets. The results indicate that the proposed method exhibits better performance on RMSE of abundance, reconstruction error and computed cost compared to other related classical hyperspectral nonlinear unmixing methods.

Hyperspectral images (HSIs) cover hundreds of narrow spectral bands, thus yielding high spectral resolution, enabling precise identification of different materials. However, the existence of dead pixels in the light sensors produces a number of irrelevant measurements, which may compromise the usefulness of HSIs. In this paper, a new hyperspectral inpainting method, named HyInpaint, is proposed. The original HSI is represented on a low dimensional subspace and its estimation is formalized with respect to the subspace representation coefficients on a given basis. The coefficients are estimated by minimizing an objective function which, in addition to the data term, contains a regularizer based on the Criminisi's inpainting method. The optimization is carried out by an instance of the alternating direction method of multipliers (ADMM), adopting the plug-and-play methodology. The effectiveness of the proposed HyInpaint approach is illustrated on Tiangong-1 hyperspectral visible near infrared (VNIR) wavebands data.

This letter proposes to integrate the locality sensitive discriminant analysis (LSDA) with the group sparse representation (GSR) for a hyperspectral imagery classification. The LSDA is to project the data set to a lower-dimensional subspace to preserve local manifold structure and discriminant information, while the GSR is to encode the projected testing set as a sparse linear combination of group-structured training samples for classification. The proposed approach, denoted as LSDA-GSR classifier (GSRC), is evaluated using two real hyperspectral data sets. Experimental results demonstrate that it can provide considerable improvement to the original counterparts, i.e., SRC and GSRC, with a relatively low computational cost.

This letter introduces a new spectral-spatial classification method for hyperspectral images. A multiscale superpixel segmentation is first used to model the distribution of classes based on spatial information. In this context, the original hyperspectral image is integrated with segmentation maps via a feature fusion process in different scales such that the pixel-level data can be represented by multiscale superpixel-level (MSP) data sets. Then, a subspace-based support vector machine (SVMsub) is adopted to obtain the classification maps with multiscale inputs. Finally, the classification result is achieved via a decision fusion process. The resulting method, called MSP-SVMsub, makes use of the spatial and spectral coherences, and contributes to better feature characterization. Experimental results based on two real hyperspectral data sets indicate that the MSP-SVMsub exhibits good performance compared with other related methods.

A large number of world's population lives within coastal areas. These environmentally sensitive zones bear both natural pressures such as erosion, accretion and natural disasters and anthropogenic impacts such as urban growth, resource exploration and pollution. The decay of coastal ecosystem has an interactive influence on surrounding residents. Monitoring the change of coastlines could help the government and coastal managers make sounds policies. This paper presents a track of coastline changes at Tampa Bay (TB) and Xiangshan Harbor (XH) during the last 30 years. There are several methods used to obtain coastlines. One popular way is waterline extraction method using normalized difference water index (NDWI). Several studies recommended on-screen digitizing technique to delineate coastline based on classified NDWI images. However, in this study, a GIS technique, converting raster to polyline, was applied to capture coastlines to reduce laboring work. The preliminary analysis results indicate that the coastlines in TB only had slight changes during these 30 years while the coastlines in XH suffered severe erosions and anthropogenic impacts. In recent decades, TB managements put lots of efforts on ecosystem protection, such as implementing the Tampa Bay Estuary Program to keep a healthy and stable coastal ecosystem. Therefore, the government of XH needs to pay more attention to ecosystem protection during the process of urbanization and to learn the advanced experience and policies of protecting TB coastlines.

This paper proposes a new supervised classification method for hyperspectral images combining the spectral and spatial information. The main contribution is presented by combining subspace-based support vector machine (SVMsub) and Markov random field (MRF). A SVM classifier integrated with a subspace projection is first used to model the posterior distributions of the classes from the spectral information. Then, the spatial information is modeled by a multilevel MRF. Finally, the maximum posterior probability classification is computed via the α-Expansion graph-cut-based optimization algorithm. The proposed method, abbreviated as SVMsub-MRF, is validated using a real typical hyperspectral data set. The results indicate that the proposed method exhibits better performance on accuracy and computational cost compared to other related classical hyperspectral image classification methods.

The kernel minimum noise fraction (KMNF) method is a nonlinear dimensionality reduction method for hyperspectral images. KMNF can transform the original data into higher dimensional feature space by using nonlinear transformation project. The key issue of KMNF is the noise estimation. The original KMNF performs noises estimation based on spatial neighborhood information. However, the spatial resolution of hyperspectral images always is not very high, and the images usually have seriously mixed pixels. Therefore, the spatial information is not enough to precisely estimate noise for KMNF. Differently, we adopt spectral correlation information which is more stable to estimate noise for KMNF. The proposed method is named the optimized KMNF method (OKMNF). Experimental results using real hyperspectral dataset demonstrate that OKMNF has much better performance than KMNF.

Endmember variability is receiving growing attention in the hyperspectral image (HSI) unmixing field. As an extension of linear mixing model (LMM), normal compositional model (NCM) assumes that the pixels of the HSI are linear combinations of random endmembers (as opposed to deterministic for the LMM). NCM explains spectral differences between the observed pixels and endmembers as endmember mixtures and endmember variances, the characteristic of which makes it possible to incorporate the endmember spectral variability in the unmixing process. But the tricky issue for using NCM is the estimation of endmember variances inhering in materials. This letter presents a new approach, termed region-based stochastic expectation maximization, to learn endmember variances from spatial information. The idea is assuming that significant homogeneous regions (composed of similar materials or similar mixture) exist in the HSI, such regions usually give visual indication that spatial-based spectral variability really exists in hyperspectral data. As modeled in NCM, spectral variances in homogeneous region can be approximately linear represented by endmember variances. Hence, given region-based spectral variances, we are able to learn endmember variances. In experiments with simulated data and Moffett field data, the proposed approach competes with other unmixing methods considering endmember variability, with better endmember variance estimates.

Hyperspectral image classification has been a very active area of research in recent years. It faces challenges related with the high dimensionality of the data and the limited availability of training samples. In order to address these issues, subspace-based approaches have been developed to reduce the dimensionality of the input space in order to better exploit the (limited) training samples available. An example of this strategy is a recently developed subspace-projection-based multinomial logistic regression technique able to characterize mixed pixels, which are also an important concern in the analysis of hyperspectral data. In this letter, we extend the subspace-projection-based concept to support vector machines (SVMs), a very popular technique for remote sensing image classification. For that purpose, we construct the SVM nonlinear functions using the subspaces associated to each class. The resulting approach, called SVMsub, is experimentally validated using a real hyperspectral data set collected using the National Aeronautics and Space Administration's Airborne Visible/Infrared Imaging Spectrometer. The obtained results indicate that the proposed algorithm exhibits good performance in the presence of very limited training samples.

Imaging spectroscopic technique has been used for the mineral and rock geological mapping and alteration information extraction successfully with many reasonable results, but it is mainly used in arid and semi-arid land with sparse vegetation covering. In the case of the dense vegetation covering, the outcrop of the altered rocks is small and distributes sparsely, the altered rocks is difficult to be identified directly. The target detection technique using imaging spectroscopic data should be introduced to the extraction of small geological targets under dense vegetation covering area. In the paper, we take Ding-Ma gold deposit as the study area which located in Zhenan country, Shanxi province, China. Some target detection algorithms which are appropriate to the small geological target detection are introduced based on the study of the principle of the algorithms. At last, the small altered rock targets under the covering of vegetation in forest are detected and discriminated using imaging spectroscopy data with the methods of spectral angle map(SAM), Orthogonal Subspace Projection(OSP), Constrained Energy Minimization(CEM), Adaptive Coherence/Cosine Estimator(ACE), Adaptive Matched Filter(AMF), Elliptically Contoured Distributions(ECD). The detection results are reasonable and indicate the ability of target detection algorithms for geological target detection in the forest area.

A new hyperspectral unmixing algorithm is proposed based on the normal compositional model (NCM) to estimate the endmembers and abundance parameters jointly in this paper. The NCM considers the hyperspectral imaging as a stochastic process and interprets each pixel value as a random vector, which is linearly mixed by the endmembers. More precisely, these endmembers are also treated as random variables as opposed to deterministic values in order to capture spectral variability that is not well described by the linear mixing model (LMM). However, the higher complexity of such an unmixing model leads to more difficulty in parameter estimation. A particle swarm optimization-expectation maximization (PSO-EM) algorithm, a “winner-take-all” version of the EM, is proposed to solve the parameter estimation problem, which employs a partial E step. The main contribution of the proposed PSO-EM is making optimum use of particle swarm optimization method (PSO) in the partial E step, which solves the difficulty of the integrals in the NCM model. The performance of the proposed methodology is evaluated through synthetic and real data experiments. Our obtained results demonstrate the superior performance of PSO-EM compared to other NCM-based as well as LMM-based methods.

This letter presents a new technique for clustering hyperspectral images that exploits neighborhood-constrained spatial information. The main feature of the proposed method is the introduction of a neighborhood homogeneity index (NHI) and the use of this index to measure the spatial homogeneity in a local area. A new similarity measurement integrates NHI and spectral information using an adaptive distance norm for clustering. The performance of the proposed neighborhood-constrained-clustering algorithm was assessed through a synthetic image and a real hyperspectral image and compared with those obtained by advanced spectral-spatial clustering algorithms. Experimental results show that the proposed scheme gives better performances.

Rapid detection of collapsed built-up structures is crucial for disaster mitigation in earthquake-affected areas. This paper proposes a method on detecting collapsed built-up structures using only post-event images. Its essence is to track and detect debris of collapsed built-up structures using features obtained through the derivative morphological profile. Experiments are carried out on images of the 2008 Wenchuan earthquake. Results show that the method can extract the debris of collapsed built-up structures in an effective and efficient way and is useful for rapid damage assessment.

An adaptive Markov random field (MRF) approach is proposed for classification of hyperspectral imagery in this letter. The main feature of the proposed method is the introduction of a relative homogeneity index for each pixel and the use of this index to determine an appropriate weighting coefficient for the spatial contribution in the MRF classification. In this way, overcorrection of spatially high variation areas can be avoided. Support vector machines are implemented for improved class modeling and better estimate of spectral contribution to this approach. Experimental results of a synthetic hyperspectral data set and a real hyperspectral image demonstrate that the proposed method works better on both homogeneous regions and class boundaries with improved classification accuracy.

Spectral mixture analysis has been an important research topic in remote sensing applications, particularly for hyperspectral remote sensing data processing. On the basis of linear spectral mixture models, this paper applied directed and weighted graphs to describe the relationship between pixels. In particular, we transformed the endmember extraction problem in the decomposition of mixed pixels into an issue of optimization and built feasible solution space to evaluate the practical significance of the objective function, thereby establishing two ant colony optimization algorithms for endmember extraction. In addition to the detailed process of calculation, we also addressed the effects of different operating parameters on algorithm performance. Finally we designed two sets of simulation data experiments and one set of actual data experiments, and the results of those experiments prove that endmember extraction based on ant colony algorithms can avoid some defects of N-FINDR, VCA and other algorithms, improve the representation of endmembers for all image pixels, decrease the average value of root-mean-square error, and therefore achieve better endmember extraction results than the N-FINDR and VCA algorithms.

This paper described endmember extraction as a combinatorial optimization problem (COP). By defining particles' position and velocity, discrete particle swarm optimization (D-PSO) was proposed based on particle swarm optimization to resolve COP. The algorithm was tested and evaluated by hyperspectral remote sensing data. Experimental results showed that, while extracting the same number of endmembers, D-PSO could get a smaller root-mean-square error between an original image and its remixed image on the precondition of correct extraction results compared to the algorithms of vertex component analysis (VCA) and N-FINDR, which meant that D-PSO could acquire higher extraction precision.

Compression has become a must for efficient storing and transmission of data acquired by satellites with increasing resolution and swath. However, for compression of raw data satellite images, impact of striping noise is inevitable. Variances of Digital Number (DN) values introduced by striping noise will surely impair continuity and smoothness of satellite image and reduce the efficiency of onboard compression. In this paper, using Beijing-1 small satellite images, origin and characteristics of striping noise caused by double channel linear CCD and its impacts on the compression process are analyzed. Then, based on properties of striping noise, an improved method for compression of raw data satellite images is proposed. The new compression method is applied to Beijing-1 small satellite raw data images and yields significant boost in compression performance. Ideas of the proposed algorithm can be easily realized with circuit modification and no adaptation is needed for post processing of the compressed images.

The coastal zone is a complex space where terrestrial environments and marine environments influence each other, including various coast flats and many artificial objects. There were many mixed pixels in hyperspectral image of coastal zone. In this paper, we applied decision tree to classify coastal zone, and adopted pure pixel index (PPI) to extract endmember as training samples during choosing various samples, which can reduce effect of mixed pixels on feature learning, at last using C4.5 decision tree algorithm to classify. We chose hyperspectral image acquired by Operational Modular Imaging Spectrometer (OMIS) in China, Classifying hyperspectral image using the method proposed in this paper, experiment result and classification precision proved efficiency and robustness of our method.

This paper gives a systematic exploration for exposure adjustment of satellite cameras. First, theories of satellite exposure adjustment are summarized and the affecting factors are discussed. Then, based on defects of current adjustment method adopted on Beijing-1 small satellite, an improved exposure adjustment method that combines sun elevation angle and type of imaged target is proposed, a lookup table that consists of the parameters for exposure setting of different land cover types is provided based on DN statistics and apparent radiance simulation, and the exposure adjustment strategy is also discussed. Serial experiments are conducted for testament of the proposed method, results show that the proposed strategy is applicable and can improve quality of the acquired images.

In this article, based on the theory of function series approaching, we change the spectral dimension of the hyperspectral data by using the Discrete Fourier transformation, and get a new feature space which could show the shape point of the spectrum curve. The coefficient, which hyperspectral data's component in the new feature space has against the Fourier series, could tell us the effect of different spectral function to the shape of spectrum curve. The paper especially analyzes the possible effect of this feature space in image shadow recognition and precision improvement of unsupervised classification based on the Euclid distance, and verify via experiments.

A Micro-satellite Constellation for Environment and Disaster Monitoring was successfully launched in China on September 6, 2008, which includes two small satellites, Satellite-A (HJ-1A) and Satellite-B (HJ-1B). The interferometric imaging spectrometer (IFIS) installed on HJ-1A is the first hyperspectral earth observation sensor in China. To assess the data quality of IFIS, a calibration experiment was carried out at the Dunhuang Calibration Site on October 20, 2008. With the simultaneous measurements acquired from the Dunhuang calibration field, the 6s radiative transfer code was used to retrieve the ground surface reflectance. By comparing the in-situ reflectance and 6S reflectance of the Dunhuang calibration target, the radiometric and spectral performance of the IFIS was evaluated. From the homogeneous image of the calibration target, the Signal-to-Noise Ratio (SNR) of IFIS data was estimated based on the high correlation between bands. This noise estimation results was used to estimate noise covariance matrix needed for hyperspectral data dimension reduction, such as Maximum Noise Fractions (MNF). The assessment results indicated that the IFIS has good performance and will be promising in the applications of environment and disaster monitoring.

An extended Bragg scattering model, for fully polarimetric SAR data, is here proposed for describing sea surface scattering. Moreover, the model is considered to examine the scattering contributions from sea surface and detected dark areas due to the presence of anthropogenic and biogenic slicks. Experiments are conducted on fully polarimetric C- and L-band SAR data.

A new method for estimating noise in hyperspectral images is described in this letter. Our method is based on the general internal regularity of Earth objects and the strong spectral correlation of hyperspectral images. It can be used to automatically estimate noise for both radiance and reflectance images. Unlike other methods discussed in this letter, our method is more reliable and adaptable, which we demonstrate using simulated images with different scene contents. Finally, we successfully applied this new method in estimating noise for Pushbroom Hyperspectral Imager (PHI) data.

This paper presents a methodology for radiometric cross-calibration of the solar reflective spectral bands of Moderate Resolution Imaging Spectroradiometer (MODIS) and Chinese Moderate Resolution Imaging Spectroradiometer (CMODIS) sensors based on analysis of two difference time image pair for Dunhuang test site on July 27, 2002. With the well-calibrated MODIS as a reference, we derive top-of-atmosphere(TOA) reflectance using MODIS data and then use these TOA reflectance to compute TOA radiance for CMODIS taking into account the effect of spectral band different and the changes in solar zenith angle due to any temporal differences in the overpass times as well as differences in the view angles between the sensors. This TOA radiance, which is correlated with the sensor digital number (DN) output, determines the in-flight calibration coefficients of CMODIS. The relatively error between the cross-calibration and the reflectance-based method calibration results is within 9%.

The small satellite constellation for environment protection and disaster monitoring (HJ constellation) is a national project proposed by National Committee for Disaster Reduction and State Environmental Protection Administration of China, to construct a network of eight affordable small satellites (two optical satellites and one SAR satellite will be launched in 2006).The payload instrument on board includes a CCD camera, an infrared camera, a hyper-spectrum camera and an S-band SAR. The average revisit time is 48 hours. By the end of 2010, HJ constellation (4 optical satellites and 4 SAR satellites) will be constructed entirely. The objective is to establish an operational system for disaster monitoring and mitigation using remote sensing technology and improve the efficiency of disaster mitigation and relief. Accompany with the construction of the small satellite constellation, a long term and stable national-level disaster reduction operational system in China will be established, as well as the network system at regional and province level. The network system will be a powerful support system for rapid-response disaster monitoring and mitigation.