Natural disasters pose a threat to the safety of human life and buildings. Rapid and accurate building damage assessment (BDA) on remote sensing images is crucial for disaster response and recovery. However, most methods are constructed on the ideal co-registered bitemporal remote sensing images, neglecting the misalignment in practice. In this paper, we propose a novel building damage assessment method, termed FlowMamba, which can effectively handle the offset between the pre- and post-disaster images in BDA task. Specifically, a vision mamba backbone with four stages is utilized to extract multi scale features from the pre- and post-disaster images. Then, a differential optical flow alignment module is designed to estimate shift matrix to align pre- and post-disaster features. Furthermore, a category distance-aware loss function is tailored for the BDA task, which replaces fixed binary values of the penalty factors in to soft values of inter-class distance. Extensive experiments on the xBD dataset, the BRIGHT dataset and four out-of-distribution disaster scenarios validate the robustness and effectiveness of the proposed FlowMamba. Our code is available at https://github.com/flying318/FlowMamba.

Cross-scene hyperspectral image classification aims to identify a new scene in target domain via learned knowledge from source domain using limited training samples. Existing cross-scene alignment approaches focus on aligning the global feature distribution between the source and target domains while overlooking the fine-grained alignment at different levels. Moreover, they mainly use Transformer architectures to model long-range dependencies across different channels but confront efficiency challenges due to their quadratic complexity, which limits classification performance in unsupervised domain adaptation tasks. To address these issues, a new domain-adaptive Mamba (DAMamba) is proposed for cross-scene hyperspectral image classification. First, a spectral-spatial Mamba is developed to extract high-order semantic features from the input data. Then, a domain-invariant prototype alignment method is proposed from three perspectives, i.e., intra-domain, inter-domain, and mini-batch, to produce reliable pseudo-labels and mitigate the spectral shift between the source and target domains. Finally, a fully connected layer is applied to the aligned features in the target domain to obtain the final classification results. Extensive evaluations across diverse cross-scene datasets demonstrate that our DAMamba outperforms existing state-of-the-art methods in classification accuracy and computing time. The code of this paper is available at https://github.com/PuhongDuan/DAMamba.

Multi-source remote sensing data classification refers to the process of categorizing ground objects by integrating complementary strengths of multiple remote sensing data, such as hyperspectral image (HSI), light detection and ranging (LiDAR) and synthetic aperture radar (SAR) data. However, current Mamba-based multisource remote sensing data classification approaches rely on fixed scanning patterns that are inadequate in characterizing spectral-spatial information. Additionally, current fusion techniques adopt concatenation or attention-based fusion rules without considering the complementary characteristics between different modalities. To address these limitations, we propose a spectral-spatial dynamic scan Mamba (SDSM) for multi-source remote sensing data classification. Specifically, a dynamic scan Mamba network is proposed to extract the spectral-spatial features of multi-source remote sensing data, in which a dynamic scan module is designed to adaptively capture the important spatial and spectral information. Furthermore, a bidirectional cross-modal fusion rule is proposed to merge the extracted features, in which a global-local frequency feature extraction module is designed to extract the salient structural features of multi-source remote sensing data as clues to guide heterogeneous feature fusion. Comprehensive experiments on four multi-source remote sensing datasets, i.e., MUUFL, Augsburg, Italy and Yellow River, demonstrate that the proposed method outperforms other state-of-the-art methods with respect to quantitative and qualitative results. The code of this article is available at https://github.com/PuhongDuan/SDSM.

Hyperspectral imaging (HSI) captures abundant spectral information of land covers while light detection and ranging (LiDAR) provides elevation and structural characteristics. Joint classification of HSI and LiDAR data can effectively merge spectral and elevation information to enhance the outcome of land cover classification. Current HSI and LiDAR joint classification approaches mainly employ a three-layer deep network to extract high-order features, followed by a concatenation or weighted fusion scheme which cannot fully exploit the unique properties of different data modalities. Meanwhile, these methods usually require high computational resources. To alleviate these issues, this paper proposes a masked self-attention fusion network (MSAF) for joint HSI and LiDAR classification, where a cascaded cross-attention fusion framework is designed to fully merge different stages of features. First, a mobile convolution block is developed to extract multi-modal data features. Then, a multi-view sequence embedding method is proposed to effectively integrate elevation information and spectral-spatial information so as to obtain token sequences. Finally, an effective masked self-attention mechanism is designed to fuse token sequences. Experimental results on multiple datasets indicate that the proposed framework significantly outperforms other advanced multi-modal fusion methods in terms of classification performance and computing efficiency. The code of this manuscript is available on https://github.com/lulushh/MSAF.

Cross-view localization aims to estimate the precise position and orientation of a ground-view image by aligning it with satellite imagery. However, existing homography-based methods are typically evaluated under limited orientation noise (±45°) and exhibit limited refinement capability, as their local correlation-based refinement relies on a reasonably good initial orientation estimate. To address these limitations, we propose a fine-grained cross-view localization method based on orientation-guided homography (OGH-Net), whose core idea is to predict an initial orientation prior that explicitly guides subsequent homography refinement. Specifically, we first design a hybrid bird's-eye-view (BEV) transformation to generate BEV images with preserved central geometry and expanded coverage. Then, we introduce a lightweight orientation-prior module that provides a coarse yaw estimate across the full ±180° range. Finally, we develop a multiscale iterative homography module that progressively refines the projection matrix through hierarchical iterations across multiple feature resolutions. Under cross-area, unknown-orientation conditions, it reduces mean localization error by 11% and mean orientation error by 27% on VIGOR, and further reduces mean localization error by 27% on KITTI compared with previous state-of-the-art methods. Moreover, OGH-Net runs in real time at up to 107 FPS on a single RTX 3090 GPU, offering a favorable trade-off between accuracy and efficiency. The code and trained models will be released at https://github.com/YC-Zhang2025/OGH-Net.

Segmenting ultra-high-resolution (UHR) images poses a significant challenge due to constraints on GPU memory, leading to a trade-off between detailed local information and a comprehensive contextual understanding. Current UHR methods often employ a multi-branch encoder to handle local and contextual information, which can be memory-intensive. To address the need for both high accuracy and low memory usage in processing UHR images, we introduce a memory-efficient semantic segmentation approach by squeezing context information into local patches (SCPSeg). Our method integrates the processing of local and contextual information within a single-branch encoder. Specifically, we introduce a context squeezing module (CSM) designed to compress global context details into local patches, enabling segmentation networks to perceive broader image contexts. Additionally, we propose a super-resolution guided local feature alignment (LFA) technique to improve segmentation precision by aligning local feature relationships. This approach calculates similarities within sliding windows, avoiding heavy computational costs during the training phase. We evaluate the effectiveness of our proposed method on four widely used UHR segmentation benchmarks. Experimental results demonstrate that our approach enhances UHR segmentation accuracy without incurring additional memory overhead during the inference stage. The code is available at https://github.com/StuLiu/SCPSeg.

Processing high-dimensional data cubes and developing high-performance classifiers are core objectives in the field of hyperspectral image classification (HSIC). Superpixel-based methods are widely used in HSIC due to their efficacy in reducing redundant information and enhancing local features. However, imprecise segmentation, especially in complex structures and textures of hyperspectral images (HSIs), may lead to inconsistencies in the regions extracted by superpixels and the boundaries between different ground objects. Such inconsistencies significantly degrade the classification performance of HSIs. Alternatively, when parameter settings are inaccurate, edge-aware feature extraction methods often introduce sharpening artifacts at the image boundaries, resulting in a decrease in classification accuracy. To effectively address these challenges, we propose a novel probabilistic fusion method for HSIC. This method consists of the following stages. First, spatial information is extracted by a multiscale superpixel segmentation method and then probabilistically optimized by the extended random walk (ERW) method. Next, semantic-aware structural features (S2Fs) are extracted along with edge information of different objects. Lastly, a probabilistic framework is proposed to fuse the class probabilities of superpixel-based spatial information and semantic-aware structural features. Experimental results on three real datasets show state-of-the-art classification performance, even with limited training sets.

Hyperspectral images (HSIs) are often affected by noise originating from both internal imaging mechanisms and external environmental factors. Therefore, denoising serves as a crucial preprocessing step for HSIs. In real-world scenarios, HSI denoising is particularly challenging due to the complex and band-dependent nature of noise. Current Mamba-based models, although capable of sequential modeling, are highly sensitive to input spectral order and still suffer from information loss over relatively long sequences, potentially leading to local over-sharpening and spectral distortion. To address these issues, we propose a novel linear attention Mamba (LaMamba) for HSI denoising. To capture the intrinsic nature of HSI, a 3D selective scan mechanism is designed to convert the input HSI into spectral-spatial continuous sequences using six bidirectional scan orders. Additionally, a linear attention state space model is proposed to capture long-range correlation. Experimental results on both synthetic and real-world hyperspectral datasets demonstrate that our model significantly outperforms other advanced methods in reconstructing spectral fidelity and spatial visual effect. The code is released at https://github.com/PuhongDuan/LaMamba.

Remote sensing scene classification (RSSC) plays a vital role in a variety of applications and has attracted much more attention. In recent years, much progress has been made to release diverse datasets or develop all kinds of techniques for scene classification of multispectral remote sensing images. Nevertheless, very few studies have focused on hyperspectral image scene classification. Moreover, the existing scene classification approaches fail to fully employ the rich spectral information of the input images, which cannot achieve satisfactory performance for hyperspectral images. To alleviate these issues, this work proposes a spectral-spatial fusion network (SSFNet) for hyperspectral RSSC (HRSSC). First, a multiscale regional growth search (MSRGS) method is designed to extract salient object regions from the hyperspectral remote sensing scene. Then, a three-stream network architecture is proposed to extract the global spatial, local spatial, and spectral features, respectively. Finally, the fully connected layer is performed on the extracted features to obtain a class score followed by a decision fusion scheme to generate the final classification result. To evaluate the effectiveness of the proposed SSFNet, we created a publicly available benchmark for the HRSSC dataset, which contains 1445 hyperspectral images, covering 11 scene classes. Experiments on the HRSSC database claim that the proposed SSFNet can attain superior classification performance with respect to other state-of-the-art scene classification techniques. The code of the proposed SSFNet will be available athttps://github.com/PuhongDuan/SSFNet.

Transformer models have achieved remarkable success in hyperspectral image classification (HSIC) owing to their strong global modeling capability. However, their quadratic complexity significantly limits their computational efficiency. Recently, Mamba has been applied to HSIC because of its linear complexity, yet it still suffers from an imbalance between global and local modeling. To overcome these challenges, this paper proposes a novel Learnable Kernel and Mamba with Spatial-Spectral Attention Fusion (LKMA) framework, which enables the extraction of global-local spatial-spectral features (SSF) while enhancing edge feature representation. For local feature extraction, the proposed Multi-Scale Spatial-Spectral Feature Generation (MSSFG) module captures local SSF by employing multi-scale learnable dilation convolutions for spatial features and multi-scale dilation convolutions for spectral features. For global feature extraction, a Global Hidden Mixing Mamba (GHMM) module is introduced, which projects hyperspectral image (HSI) features from the feature space to the hidden state space via a hidden state mixing mechanism. This enables the model to capture contextual semantic information and local details from the HSI. To further explore the synergistic effect between spatial and spectral information, the Spatial-Spectral Attention Fusion (SSAF) module integrates semantic information across multiple feature groups by combining Semantic Grouped Spatial Attention (SGSA) and Progressive Spectral Self-Attention (PSSA), enhancing spatial-spectral representations. Extensive experiments demonstrate that the proposed method outperforms state-of-the-art approaches for HSIC.

Deep learning-based methods for hyperspectral image classification (HSIC) have been widely utilized in recent years. However, existing HSIC methods do not adequately account for illumination variations in HSIs, particularly in urban areas where shadows created by complex ground objects result in significant variations that cannot be ignored. Additionally, when dealing with limited labeled data, most deep learning methods are prone to overfitting, resulting in poor classification performance. To address these challenges, we propose a new Retinex-based dual-branch feature extraction network (RDFEN) for HSIC. First, by incorporating Retinex theory, we propose a hyperspectral Retinex (HyperRetinex) module to extract illumination attributes and reflectance attributes. Then, we propose a dual-branch feature extraction network, which consists of two submodules: illumination attributes feature extraction (IAFE) module and reflectance attributes feature extraction (RAFE) module. Finally, an illumination-reflectance attributes interaction attention fusion (IRAIAF) module is strategically designed to integrate distinct features. Experiments on four benchmark HSI datasets demonstrate that the proposed method outperforms other state-of-the-art HSIC methods, achieving up to 96.85% overall accuracy on the Pavia University dataset, thereby highlighting its effectiveness and robustness in HSIC. For reproducibility, the code is available at https://github.com/JT-shen/Code.

Cross-domain image segmentation plays a crucial role in the field of remote sensing. Current approaches often rely on a mean-teacher model that is integrated from student models to guide the training of the student model itself. However, the feature space of the mean-teacher model exhibits significant domain discrepancy and considerable class overlap, which results in suboptimal performance. Motivated by the idea of learning from stronger teachers, we introduce a robust domain adaptation method called LFMDA. This novel approach is the first to explicitly enhance cross-domain semantic segmentation performance by leveraging vision foundation models (VFMs) within remote sensing applications. Specifically, we propose a prototypical contrastive knowledge distillation loss (PCD) that enables the student model to produce domain-invariant yet category-discriminative features by distilling knowledge from a domain-generalized VFM teacher. Additionally, we introduce a local region homogenization strategy (LRH) to generate high-quality and high-quantity pseudo-labels by incorporating a Segment Anything Model (SAM). Extensive empirical evaluations demonstrate that our method outperforms existing approaches, setting a new state-of-the-art (SOTA) method in domain-adaptive remote sensing image segmentation. The code is available at https://github.com/StuLiu/LFMDA.

Visible and infrared image fusion(VIF) provides more comprehensive understanding of a scene and can facilitate subsequent processing. Although frequency domain contains valuable global information in low frequency and rapid pixel intensity variation data in high frequency of images, existing fusion methods mainly focus on spatial domain. To close this gap, a novel VIF method in frequency domain is proposed. First, a frequency-domain feature extraction module is developed for source images. Then, a frequency-domain transformer fusion method is designed to merge the extracted features. Finally, a residual reconstruction module is introduced to obtain final fused images. To the best of our knowledge, it is the first time that image fusion study is conducted from frequency domain perspective. Comprehensive experiments on three datasets, i.e., MSRS, TNO, and Roadscene, demonstrate that the proposed approach obtains superior fusion performance over several state-of-the-art fusion methods, indicating its great potential as a generic backbone for VIF tasks.

Spectral super-resolution aims to reconstruct a hyperspectral image (HSI) from its corresponding RGB image, which has drawn much more attention in remote sensing field. Recent advances in the application of deep learning models for spectral super-resolution have demonstrated great potential. However, these methods only work in spectral-spatial domain while rarely explore the potential property in the frequency domain. In this work, we first attempt to address spectral super-resolution in the frequency domain. To well merge the frequency information into the super-resolution network, a spectral-spatial-frequency domain fusion network (SSFDF) is designed, which consists of three key parts: frequency-domain feature learning, spectral-spatial domain feature learning, and feature fusion module. In more detail, a frequency-domain feature learning network is first exploited to dig the frequency-domain information of the input data. Then, a symmetric convolutional neural network (CNN) is developed to acquire the spectral-spatial features of the input data, where a parameter-sharing strategy is utilized to reduce network parameters. Finally, a feature fusion module is proposed to reconstruct HSI. Comprehensive experiments on several datasets reveal that our method can attain state-of-the-art reconstruction result with respect to other spectral super-resolution techniques.

Real-time semantic segmentation plays an important role in auto vehicles. However, most real-time small object segmentation methods fail to obtain satisfactory performance on small objects, such as cars and sign symbols, since the large objects usually tend to devote more to the segmentation result. To solve this issue, we propose an efficient and effective architecture, termed small objects segmentation network (SOSNet), to improve the segmentation performance of small objects. The SOSNet works from two perspectives: methodology and data. Specifically, with the former, we propose a dual-branch hierarchical decoder (DBHD) which is viewed as a small-object sensitive segmentation head. The DBHD consists of a top segmentation head that predicts whether the pixels belong to a small object class and a bottom one that estimates the pixel class. In this situation, the latent correlation among small objects can be fully explored. With the latter, we propose a small object example mining (SOEM) algorithm for balancing examples between small objects and large objects automatically. The core idea of the proposed SOEM is that most of the hard examples on small-object classes are reserved for training while most of the easy examples on large-object classes are banned. Experiments on three commonly used datasets show that the proposed SOSNet architecture greatly improves the accuracy compared to the existing real-time semantic segmentation methods while keeping efficiency. The code will be available at https://github.com/StuLiu/SOSNet.

Bipartite graph (BiG) has been proven to be efficient in handling massive multiview data for clustering. However, how to regulate the structural information of view-specific anchors and view-shared BiG is still open and needs to be further studied. Hence, a novel dual-structural BiG learning (DsBiGL) method is proposed in the article. It transforms BiG learning into a joint optimization problem of IntrA-view and InteR-view subspace learning (IASL and IRSL) with the structural constraints, such as k-nearest neighbor (KNN) and low-rank. On one hand, IASL uses the KNN and view-specific low-rank constraints to enhance the discriminativeness of view-specific anchors. On the other hand, IRSL uses an adaptive weighting strategy to obtain view-shared BiG directly from multiview samples, where the KNN and view-shared low-rank constraints are adopted to encode local connectivity and cluster information between samples. Note that IASL and IRSL are integrated into a unified optimization model, which ensures the interactive enhancement of view-specific anchor representation and view-shared BiG learning. Finally, an algorithm based on iterative optimization is designed to solve the proposed DsBiGL model. Experimental results on various multiview datasets have demonstrated the superiority of DsBiGL in terms of clustering results when compared with other comparative methods.

Hyperspectral image (HSI) provides rich spatial and spectral information of ground objects, while synthetic aperture radar (SAR) records scattering information such as shape and structure. Fusion of HSI and SAR can improve the classification performance of land covers. In recent years, graph convolutional networks (GCN) have been widely used in the field of remote sensing due to its advantages in processing non-Euclidean structures, capturing local and global information. In this paper, we propose a spectral-spatial graph convolutional network (SSGCN) for fusion of HSI and SAR. First, the GCN is utilized to extract the spatial information of HSI and SAR. Then, a convolutional neural network is applied to extract the spectral information of HSI. Finally, the extracted spectral and spatial features are merged together followed by a fully connected layer to obtain the final classification result. Experiments on two datasets, i.e., Berlin and Augsburg, reveal that the proposed SSGCN significantly outperforms other representative methods.

Unsupervised domain adaption (UDA) has gained popularity in narrowing performance gaps across domains in remote sensing image semantic segmentation (RSISS). However, current UDA methods suffer from serious noisy pseudo-labels, adversely affecting domain adaptation performance. In this work, a local region homogenizing domain adaptation method (RegDA) is proposed to tackle this issue. Specifically, a generalized segment anything model (SAM) is utilized to obtain the semantic-consistent regions for the images in the target domain. Furthermore, a pixel-level voting scheme is proposed to get the semantic label for each local region and assign it to each pixel within this region. In this way, more reliable pseudo-labels are obtained and domain adaptation performance is improved. Experiment results on ISPRS datasets demonstrate that the proposed RegDA outperforms previous UDA approaches for RSISS. The code will be available at https://github.com/StuLiu/RegDA.

Hyperspectral change detection (HCD) aims to identify the changed and unchanged pixels in bitemporal images, which has been applied in various aspects. Currently, many deep learning-based change detection methods have been developed. However, existing change detection methods only focus on changed or temporal information while neglecting the complementary information between them. To solve this issue, a novel cross-temporal symmetric feature fusion network (CTSFFNet) is proposed for change detection of hyperspectral images. First, we perform pixel-wise subtraction and concatenation on the multi-temporal hyperspectral images to obtain the difference data and temporal data, respectively. Then, a three-layer convolutional neural network is performed on the difference data and temporal data to yield the difference and temporal features. Finally, a cross-temporal symmetric feature fusion (CTSSF) module is designed to merge the extracted features followed by a fully connected layer to obtain the final change regions. Experiments on two popular datasets demonstrate that the proposed CTSSFNet achieves superior detection performance compared to other state-of-the-art methods.

Unsupervised cross-scene hyperspectral image classification transfers the learnable knowledge from a labeled source scene to an unlabeled target scene. Currently, many statistical distribution alignment methods are introduced to mitigate domain discrepancy. However, these methods ignore the finer class specific structure which may cause negative transfer. To solve this issue, a prototype-based inter-intra domain alignment network is proposed for unsupervised cross-scene hyperspectral image classification. Specifically, a prototype-based inter-intra alignment method is proposed to narrow the feature distribution gap. Furthermore, an uncertainty estimation is developed to obtain highly reliable pseudo-labels in the target scene. Experiment results on several datasets imply that the proposed method outperform several cutting-edge unsupervised classification methods.

Remote sensing scene classification has garnered significant interest. However, few studies are dedicated to the classifi-cation of hyperspectral remote sensing scenes. Furthermore, current approaches that are designed for multispectral or visible images are not suitable for hyperspectral remote sensing scene classification. To solve these issues, a spectral-spatial convolutional network is proposed for hyperspectral remote sensing scene classification. First, a multiscale regional growth search method is developed to extract object region of the input data. Then, a three-branch network is designed to extract local spatial, global spatial, and spectral features. Finally, the obtained features are fed into fully connected layer to obtain class scores followed by a decision fusion rule to yield the final classification result. To evaluate the effect of hyperspectral remote sensing scene classification, a hyper-spectral remote sensing scene classification dataset (HRSS-C) is constructed, which consists of 1445 hyperspectral images with 11 different classes. Experiments on the HRSS-C reveal that our method gains superior classification performance with respect to other advanced approaches.

Visible-infrared vehicle target detection aims to pinpoint the location and class of vehicles by fusing the favorable complementary information of visible-infrared image pairs. However, most of detection methods cannot obtain ideal detection performance when visible-infrared image pairs are captured in low lighting environment. To solve this issue, we propose a global-local feature fusion network, which can adaptively integrate the saliency information from visible-infrared image pairs. Initially, a dual-stream ResNet-50 network is designed to extract cross-modal features from visible-infrared image pairs. Then, a global-local feature fusion module (GLF) is proposed to merge the multi-modality features. Finally, the detection head utilizes the fused features of the deep interaction to get the detection results. Experiments on the DroneVehicle and LLVIP datasets show that the proposed method is increased by 7.4% and 1.2 % compared to recently proposed methods, respectively.

Optical remote sensing is the major means of monitoring sea ice, which is beneficial for waterway planning, disaster prevention, and environmental research. Currently, a large number of sea ice segmentation methods have been developed. However, they often ignore the boundaries between sea ice and seawater, leading to unavoidable mis-segmentation. To alleviate this issue, this paper proposes a novel semantic segmentation network applied to sea ice remote sensing images, referred to as SegIceNet. First, we design a partial class activation map to select the feature points whose classes are easily confused in feature layers. By constructing the affinity among these points, the network explores potential semantic information and propagates these feature details among adjacent decoder layers. Second, this paper introduces a point-set optimization strategy, which aims to focus on samples of the aforementioned special points during the training process. Experimental results on a real sea ice dataset show that our method is better than other state-of-the-art semantic segmentation methods. Specifically, even in the imbalanced sea ice dataset, the proposed method achieved 94.88% mIoU.

Hyperspectral and synthetic aperture radar (SAR) image classification, aiming to merge multisource information to boost the precision and reliability of land cover classification, has gained increasing attention. Nevertheless, current techniques still exhibit certain limitations in extracting discriminative features and integrating heterogeneous features. In this work, a graph convolutional fusion network (GCFNet) is proposed for hyperspectral and SAR image classification. First, a spectral residual neural network is employed to extract the spectrum information. Then, a dual-branch graph convolutional network (GCN) is developed to extract the spatial information from hyperspectral and SAR images. Finally, a cross-contextual transformer fusion module is created to merge the spectral and spatial information followed by a dense layer to yield the final prediction outcome. To confirm the performance of the GCFNet, experiments on three datasets (e.g., Berlin, Augsburg, and Yellow River) demonstrate that the GCFNet significantly surpasses other representative methods.

In this article, a two-dimensional (2-D) spectral representation is proposed for the visualization and classification of hyperspectral images (HSIs). First, several sequence data processing methods, i.e., Gramian angular field (GAF) algorithm, Markov transition field (MTF), and recurrence plot (REP), are applied to obtain multiple 2-D features of a one-dimensional (1-D) spectrum. Second, the 2-D spectral features are stacked together to form the final 2-D spectral representation. Finally, many excellent classifiers in computer vision field are applied on the 2-D spectral representation to obtain the final classification result. Furthermore, 114 target spectral visualization maps are established based on their 1-D spectra. Experimental results reveal that the 2-D spectral representation has multiple advantages in terms of better visual quality and classification accuracies. The code of this work is available athttps://github.com/zhuyongxiang1/two-dimensional-spectral-representation.

Oriented object detection with remote sensing scenes has made excellent progress in recent years, especially using anchor-free detectors. Without the limitation of inherent prior spatial information, anchor-free detectors regress the detection boxes from the object center or edge in an elegant way. However, anchor-free detectors suffer severe feature misalignment and inconsistency between classification and regression. Especially in remote sensing scenes, there are densely arranged instances and multi-scale representations, which will affect the detection accuracy. Therefore, a feature augmentation module (FAM) and an oriented feature alignment (OFA) module are proposed for oriented object detection called FAA-Det. More specifically, we first introduce a FAM to enhance the object representation. After that, the augmented feature maps will be fed into OFA for feature alignment and accurate detection. OFA has two independent branches for classification and regression, and their separate structures can alleviate the inconsistency in detection. FAM and OFA comprise the FAA-Head in our detector. Extensive evaluation demonstrates the effectiveness of our proposed FAA-Det that performs the state-of-the-art (SOTA) mean average precision (mAP) on the DOTA and HRSC2016 datasets without bells and whistles. Our code will be available athttps://github.com/jimuIee/FAA-Det.

Hyperspectral image (HSI) classification is commonly influenced by convolution neural networks (CNNs). However, the large number of parameters and computational complexity associated with CNNs can limit their practical application, particularly when computing and storage resources are limited. To address this challenge, we propose a channel-layer-oriented lightweight network for HSI classification. Motivated by existing structures that typically set large channels and stack multiple layers, we give more optimal solutions strategically to further compress the model. For intralayer feature extraction, we develop a channel-oriented spectral–spatial module (COS2M), which introduces a dual-single-channel (DSC) 3-D convolution that works in conjunction with depthwise convolution to fully extract spectral–spatial information. For interlayer information transmission, we propose a novel neighbor-pixel-aware activation function (NPAF), where the activation of a single pixel is determined by the learnable interaction with its neighbor range that enhances information transmission and improves the network’s fitting ability through the single activation layer. By implementing these strategies, we aim to overcome the limitations of traditional CNNs and enable efficient HSI classification within resource-constrained environments. The whole network is designed to be a compact end-to-end structure. It achieves better classification performance than other deep learning methods and lightweight models, even with limited training samples. The network parameters, model complexity, and inference time also demonstrate significant superiority, as confirmed by experiments on three benchmark datasets. The source codes are available publicly at:https://github.com/AchunLee/CLOLN_TGRS

Domain adaptive segmentation has recently gained more and more attention in the remote sensing field. However, current methods often generate a significant number of uncertain examples, i.e., noisy pseudo-labels, in the target domain, which adversely affects model convergence. To solve this issue, an uncertain example mining network is proposed for domain adaptive segmentation of remote sensing images. Specifically, a novel strategy called multilevel pseudo-label correcting (MPC) is proposed to correct the pseudo-labels in class, pixel, and superpixel levels. In this way, more reliable pseudo-labels can be selected for the subsequent training stage. Furthermore, a noise-robust example mining strategy, termed uncertainty-based valuable example mining (UVEM), is proposed to prioritize confident examples with significant gradients for training effectively. Extensive empirical evaluations on IsprsDA and LoveDA datasets demonstrate that the proposed method outperforms previous approaches, establishing state-of-the-art results in domain adaptive remote sensing image segmentation (RSIS). The code will be available athttps://github.com/StuLiu/UemDA.

Hyperspectral images (HSIs) offer detailed and abundant spectral-spatial information, holding great potential for ground object change detection (CD). Currently, numerous deep learning-based hyperspectral CD (HCD) models have been studied. However, these methods only consider single difference information or temporal information while neglecting the complementary advantage between difference and temporal information. To solve this issue, this work proposes a multitemporal symmetric fusion network (MTSFNet) for HCD, which involves three steps. First, the difference and temporal data are calculated by a subtraction operation and concatenation operation. Then, a dual-branch convolutional network is developed to capture the difference and temporal features. Finally, a symmetric feature fusion scheme is designed to integrate the extracted features followed by a fully connected layer to derive the detection results. Experiments conducted on multiple well-known datasets reveal that the proposed MTSFNet outperforms other advanced CD approaches in terms of qualitative and quantitative results.

In the past few years, there has been significant progress in hyperspectral image classification (HSIC). However, when the trained classifier on the source scene is directly applied to a new scene, the classification performance tends to dramatically decrease because of the spectral shift phenomenon. Most existing techniques use feature alignment to learn knowledge from labeled scenes to unlabeled scenes, often overlooking the impact of noisy samples and outliers. To tackle this issue, the classwise prototype-guided alignment network (CPGAN) is proposed for cross-scene HSIC. The core idea is that classwise prototypes across scenes are employed as alignment intermediaries to guide cross-scene feature alignment. Specifically, first, spectral-spatial features from different scenes are extracted with a common feature extractor. Then, an uncertainty-aware pseudolabel selection (UPS) is designed to obtain high-confidence pseudolabels for unlabeled target scenes. Finally, a novel classwise prototype-guided alignment method is proposed to simultaneously achieve interdomain and intradomain alignment (IntraDA). The experimental results conducted on three datasets show that our method achieves superior performance compared to other cutting-edge classification algorithms.

Remote sensing super-resolution (SR) technique, which aims to generate high-resolution image with rich spatial details from its low-resolution counterpart, play a vital role in many applications. Recently, more and more studies attempt to explore the application of Transformer in remote sensing field. However, they suffer from the high computational burden and memory consumption for remote sensing super-resolution. In this paper, we propose an efficient Swin Transformer (ESTNet) via channel attention for SR of remote sensing images, which is composed of three components. First, a three-layer convolutional operation is utilized to extract shallow features of the input low-resolution image. Then, a residual group-wise attention module is proposed to extract the deep features, which contains an efficient channel attention block (ECAB) and a group-wise attention block (GAB). Finally, the extracted deep features are reconstructed to generate high-resolution remote sensing images. Extensive experimental results proclaim that the proposed ESTNet can obtain better super-resolution results with low computational burden. Compared to the recently proposed Transformer-based remote sensing super-resolution method, the number of parameters is reduced by 82.68% while the computational cost is reduced by 87.84%. The code of the proposed ESTNet will be available at https://github.com/PuhongDuan/ESTNet for reproducibility.

Interactive image segmentation (IIS) has been widely used in various fields, such as medicine, industry, etc. However, some core issues, such as pixel imbalance, remain unresolved so far. Different from existing methods based on pre-processing or post-processing, we analyze the cause of pixel imbalance in depth from the two perspectives of pixel number and pixel difficulty. Based on this, a novel and unified Click-pixel Cognition Fusion network with Balanced Cut (CCF-BC) is proposed in this paper. On the one hand, the Click-pixel Cognition Fusion (CCF) module, inspired by the human cognition mechanism, is designed to increase the number of click-related pixels (namely, positive pixels) being correctly segmented, where the click and visual information are fully fused by using a progressive three-tier interaction strategy. On the other hand, a general loss, Balanced Normalized Focal Loss (BNFL), is proposed. Its core is to use a group of control coefficients related to sample gradients and forces the network to pay more attention to positive and hard-to-segment pixels during training. As a result, BNFL always tends to obtain a balanced cut of positive and negative samples in the decision space. Theoretical analysis shows that the commonly used Focal and BCE losses can be regarded as special cases of BNFL. Experiment results of five well-recognized datasets have shown the superiority of the proposed CCF-BC method compared to other state-of-the-art methods. The source code is publicly available at https://github.com/lab206/CCF-BC.

Visible-infrared object detection aims to improve the detector performance by fusing the complementarity of visible and infrared images. However, most existing methods only use local intramodality information to enhance the feature representation while ignoring the efficient latent interaction of long-range dependence between different modalities, which leads to unsatisfactory detection performance under complex scenes. To solve these problems, we propose a feature-enhanced long-range attention fusion network (LRAF-Net), which improves detection performance by fusing the long-range dependence of the enhanced visible and infrared features. First, a two-stream CSPDarknet53 network is used to extract the deep features from visible and infrared images, in which a novel data augmentation (DA) method is designed to reduce the bias toward a single modality through asymmetric complementary masks. Then, we propose a cross-feature enhancement (CFE) module to improve the intramodality feature representation by exploiting the discrepancy between visible and infrared images. Next, we propose a long-range dependence fusion (LDF) module to fuse the enhanced features by associating the positional encoding of multimodality features. Finally, the fused features are fed into a detection head to obtain the final detection results. Experiments on several public datasets, i.e., VEDAI, FLIR, and LLVIP, show that the proposed method obtains state-of-the-art performance compared with other methods.

Hyperspectral image (HSI) classification methods have made great progress in recent years. However, most of these methods are rooted in the closed-set assumption that the class distribution in the training and testing stages is consistent, which cannot handle the unknown class in open-world scenes. In this work, we propose a feature consistency-based prototype network (FCPN) for open-set HSI classification, which is composed of three steps. First, a three-layer convolutional network is designed to extract the discriminative features, where a contrastive clustering module is introduced to enhance the discrimination. Then, the extracted features are used to construct a scalable prototype set. Finally, a prototype-guided open-set module (POSM) is proposed to identify the known samples and unknown samples. Extensive experiments reveal that our method achieves remarkable classification performance over other state-of-the-art classification techniques.

Hyperspectral change detection (HCD) is widely applied in various domains, such as accurate agriculture, disaster assessment, land use, and environmental monitoring. Most of hyperspectral change detection methods aim at extracting and classifying the spectral variation features with dimension reduction and machine learning methods. Different from previous work, this paper proposes an edge-guided hyperspectral change detection method. Specifically, a subtraction operation is adopted to extract difference hyperspectral image. Then, the edge-preserving filtering is performed on the difference HSI so as to extract spectral-spatial features. Next, the number of the extracted features is diminished through the kernel principal component analysis. Finally, the fused features are input into a spectral classifier followed by the edge-preserving filtering to obtain the final change detection result. Experiments on several HCD datasets demonstrate that the proposed method can consistently outperform other advanced approaches in both subjective and objective evaluations when only a limited number of labeled samples are available.

Oil spill detection has attracted increasing attention in recent years since marine oil spill accidents severely affect environments, natural resources, and the lives of coastal inhabitants. Hyperspectral remote sensing images provide rich spectral information which is beneficial for the monitoring of oil spills in complex ocean scenarios. However, most of the existing approaches are based on supervised and semi-supervised frameworks to detect oil spills from hyperspectral images (HSIs), which require a massive amount of effort to annotate a certain number of high-quality training sets. In this study, we make the first attempt to develop an unsupervised oil spill detection method based on isolation forest for HSIs. First, a Gaussian statistical model is designed to remove the bands corrupted by severe noise. Then, kernel principal component analysis (KPCA) is employed to reduce the high dimensionality of the HSIs. Next, the probability of each pixel belonging to one of the classes of seawater and oil spills is estimated with the isolation forest, and a set of pseudo-labeled training samples is automatically produced using the clustering algorithm on the detected probability. Finally, an initial detection map can be obtained by performing the support vector machine (SVM) on the dimension-reduced data, and the initial detection result is further optimized with the extended random walker (ERW) model so as to improve the detection accuracy of oil spills. Experiments on hyperspectral oil spill data (HOSD) created by ourselves demonstrate that the proposed method obtains superior detection performance with respect to other state-of-the-art detection approaches. We will make HOSD and our developed library for oil spill detection publicly available at https://github.com/PuhongDuan/HOSD to further promote this research topic.

Hyperspectral oil spill mapping aims to distinguish the type of oil spill. Recently, most hyperspectral oil spill detection methods are based on supervised methods that work well with rich training samples. However, in the marine oil spill detection scenario, pixel annotations are difficult and costly. Moreover, the labels obtained by domain experts within a hyperspectral image (HSI) are often scarce. To address these issues, a self-supervised spectral-spatial transformer network is proposed for hyperspectral oil spill mapping. First, we propose a transformer-based contrastive learning network to extract the deep discriminative features. Then, the learned features are transferred to the downstream classification network that is fine-tuned with very few labeled samples. Experiments on hyperspectral oil spill database (HOSD) constructed by ourselves indicate that the proposed method can obtain more promising performance than several state-of-the-art oil spill classification techniques in discriminating different types of oil spills, i.e., thick oil, thin oil, sheen, and seawater.

Single-modal image registration methods are generally not feasible for visible and infrared images. Besides, multi-modal image registration methods still suffer from uneven distribution of extracted features, low repeatability, and ambiguous features. To address these issues, a coarse-to-fine infrared and visible image registration approach for dual sensor UAV imaging system is proposed, which is resilient to the difference of focal lengths and field of view. First, in the coarse registration step, the infrared image is transformed to the same scale as the visible image by using the similarity transformation. This operation makes the proposed method robust to the variation of field of view. Then, the feature point pairs are initialized using feature detectors in the infrared image’s blocked phase congruency feature map. Next, the feature point pairs are optimized by estimating the offset based on the relationship between the constructed feature descriptors. Finally, using elastic deformation, the pixel-level registered infrared image is obtained. Extensive experiments demonstrate the superior performance of the proposed coarse-to-fine image registration methodology in the real infrared-visible image pairs. The code and dataset are available at https://drive.google.com/drive/folders/1mpUWwHUbKTrBdOrNMNRRnuJclDUAC7nU?usp=sharing.

With the improvement of imaging equipment, hyperspectral underwater target detection (HUTD) has raised much interest in recent years. The existing HUTD methods do not fully utilize spectral characteristics and need prior information about targets. Besides, the detection performance lacks verification in natural scenarios. In this paper, the authors propose a Feature Bands based Unsupervised underwater target Detection method (FBUD), which aims at finding the optimal feature bands to identify the underwater target near the coastline. Specifically, the normalized difference water index (NDWI) and unmixing technique are adopted to find the target and background pixels. Then, the spectral difference between the target and background is used to find the feature bands. With a simple and fast math operation of the feature bands, the probability map of the underwater target can be easily obtained. Besides, a new unmanned aerial vehicle (UAV)-borne hyperspectral image dataset named HNU-UTD is built for underwater target detection in real-world scenes. Experimental results obtained with the HNU-UTD dataset confirm the accuracy and effectiveness of the proposed detection method, which even outperforms supervised detection methods.

Haze in remote sensing images severely degrades image visibility, making it hard to identify different land covers. This work proposes an edge-preserving filtering-based image dehazing method for remote sensing images, mainly consisting of following several steps. First, the original image contaminated with haze is decomposed by a multiscale guided filtering into base layers that contain haze components and detail layers that reflect the spatial details of input. Then, an optimized atmospheric scattering model is performed on the base layers to eliminate haze. Next, adaptive nonlinear mapping is exploited to enhance image details. Finally, the resulting image is reconstructed by combining the dehazed base layers and the refined detail layers. Experiments on several images demonstrate that the proposed method has an outstanding haze removal capability and yields better dehazing performance compared to other dehazing approaches in terms of objective indexes and subjective results.

Haze in hyperspectral images is a common phenomenon, which severely degrades the quality of the acquired data and limits its applications. In this letter, a dark-object subtraction model is proposed for haze removal of hyperspectral images, which mainly is composed of three steps. First, a haze density map is estimated according to the haze characteristic in different spectral channels. Then, we design a saliency measure method to automatically calculate haze abundance of different channels. Finally, the haze-free image is obtained through solving the dark-object subtraction model. Experiments on real and simulated datasets demonstrate that our method consistently outperforms other state-of-the-art dehazing techniques in terms of reconstructed performance and computational cost.

Shadow removal is a challenging problem in hyperspectral remote sensing images due to its spatial-variant properties and diverse patterns. In this work, a shadow removal framework with multiexposure fusion is proposed for hyperspectral remote sensing images, which consists of three major steps. First, a color space conversion method is exploited to detect the shadow regions. Second, the principle of the intrinsic decomposition model is utilized to generate a set of differently exposed hyperspectral images (HSIs), i.e., multiexposure images. Third, the generated multiexposure images and the original HSIs are fused together with a two-stage image fusion method so as to remove the shadows in hyperspectral remote sensing images effectively. Experiments performed on three real hyperspectral datasets confirm that the performance of the proposed method outperforms other state-of-the-art shadow removal approaches.

Fog in hyperspectral images severely limits the visibility of imaging scene and reduces the image contrast, which has a negative effect on the following image interpretation. Defogging methods aim at restoring a high-quality image from the degraded image. Currently, most dehazing methods mainly depend on the atmospheric scattering model in computer vision and multispectral image communities. However, when these approaches are directly used to remove the fog from HSIs, they cannot produce satisfactory defogging performance. To alleviate this issue, we develop a novel fog model to achieve fog removal from hyperspectral images. First, a fog density map is calculated by differentiating the averaged bands falling into visible and infrared spectral ranges. Then, haze abundance in different spectral bands is estimated based on the pixel reflectance between two selected pixels with different haze levels. Finally, the high-quality hyperspectral image is restored by solving the defogging model. Experiments performed on a new benchmark created by ourselves demonstrate that the proposed method obtains favorable dehazing performance in contrast to other approaches in computer vision and remote sensing fields.

Remote sensing image super-resolution aims to boost the image resolution while recovering rich high-frequency details. Currently, most of super-resolution methods are based on an assumption that the degradation kernel is a specific downsampler. However, the degradation kernel is unknown and sophisticated for real remote sensing scenes, leading to a severe performance drop. To alleviate this problem, we propose a multi-layer degradation representation-guided blind super-resolution method for remote sensing images, which mainly consists of three key steps. First, an unsupervised representation learning is exploited to learn the degradation representation from low-resolution images. Then, a degradation-guided deep residual module is designed to model high-order features across different scales from original images. Finally, a multi-layer degradation-aware feature fusion mechanism is proposed to restore the finer details. Experiments on synthetic and real datasets demonstrate that the proposed method can achieve promising performance with respect to other state-of-the-art super-resolution approaches.

Oil spill mapping is a very challenging problem in marine environmental monitoring. In this paper, the potential of hyperspectral image classification for mapping oil spills is comprehensively investigated. First, several representative hyperspectral image classification methods are reviewed in a general framework. Second, three oil spill mapping cases are designed to analyze the performance of different classification methods in detecting the spatial distribution, classifying the type, and estimating the thickness of oil spills. Finally, the experimental results are analyzed in detail, and some conclusions are given, which bring a comprehensive understanding to scholars who are interested in the fields of hyperspectral remote sensing and oil spill mapping.

Unmanned aerial vehicle (UAV) hyperspectral imaging has been extensively applied in various fields. However, due to the limited imaging width, hyperspectral images (HSIs) captured by UAV need to be stitched, so as to effectively cover the study area. In this article, an effective seamless stitching method with deep feature matching and elastic warp is proposed for HSIs, which consists of the following major steps. First, for each input HSI, a single-band gray-scale image is obtained by fusing the bands corresponding to the red, green, and blue wavelengths. Second, the feature points of each HSI are obtained with a robust VGG-style network and matched with a graph neural network. After point pairs are obtained, the next step is to estimate the transformation matrix of adjacent images, and a spectral correction method based on intrinsic decomposition is proposed to ensure the spectral consistency of adjacent images. In the final stage, a seam-cutting and multiscale blending strategy is adopted to ensure the spatial consistency of the stitching results. Experimental results on real HSIs show that the proposed method is superior to six representative image stitching approaches.

Coastal wetland monitoring plays an important role in the protection and restoration of ecosystems in this world. UAV-hyperspectral imaging, as an emerging technique for Earth observation and space exploration, provides the huge potential ability to identify different wetland species. In this work, a multilayer global spectral–spatial attention network (MGSSAN) is proposed for mapping coastal wetlands, which mainly consists of two major steps. First, a two-branch convolutional neural network (CNN) framework with residual connection is developed to obtain an initial classification probability map, in which one branch is used to capture the spectral information, the other branch is used to extract spatial information, and a global spectral–spatial attention module is designed to guide networks focusing on those features that are more discriminative. Second, an extended random walker method is utilized to optimize the initial classification probabilities, so as to yield the final map. Experiments performed on three wetland HSI datasets created by ourselves verify that the proposed method can obtain superior performance with respect to several state-of-the-art hyperspectral image classification methods.

Feature extraction provides an effective tool to classify hyperspectral images (HSIs). However, most hyperspectral feature extraction methods tend to yield an over-smoothed phenomenon, which leads to inconsistency between the homogeneous regions and the ground objects in the actual scene. To alleviate this problem, an edge-aware feature extractor called contour structural profiles (CSPs) is proposed to extract the discriminative features for hyperspectral images classification (HSIC). The proposed classification method comprises three components. First, the spectral dimension of the HSI is reduced with an averaging-based method. Then, an edge-aware total variation (TV) model is constructed to extract the contour structural profile, in which a learned contour probability map is served as one of the major cues in the feature extraction process. Next, multiscale structural profiles (MSSPs) are constructed using the edge-aware TV model with different parameters so as to fully characterize ground objects with different scales. Finally, the MSSPs are fused with a kernel principal component analysis (KPCA) followed by a spectral classifier to obtain the final classification map. Experimental results on several publicly available hyperspectral datasets illustrate that the proposed method obtains superior classification performance over several state-of-the-art classification approaches, especially when the number of training samples is insufficient.

Combining spectral and spatial information can significantly improve the classification performance of hyperspectral image (HSI). Currently, a lot of spectral–spatial HSI classification methods have been proposed. However, the task of HSI classification has remained challenging since the number of training samples is limited in real scenarios. In this article, we propose a novel HSI classification framework with single sample, in which the spectral self-similarity and spatial polygon structure information are fully combined to improve the classification performance. On the one hand, spectral self-similarity is used to expand training samples, which makes it possible to obtain sufficient samples with minimal cost. On the other hand, polygonal partition is introduced to acquire the geometrical structure of land covers in man-made environments. Specifically, the edge information of geometric objects is captured by polygonal partition, which can be utilized to constrain the spatial range of sample expansion and optimize the classification results. Experimental results on three real HSIs illustrate that the proposed method performs very well under small training sample size even when the number of samples is single per class.

Manual and automatic annotation of Remote Sensing (RS) scenes are rather complex tasks which may unavoidably introduce some degree of mislabeled data in large-scale archives. In this regard, noisy annotations become an important constraint for deep metric learning-based RS characterization methods since most of them are trained in a supervised way. To address this problem, here we investigate the use of deep metric learning for characterizing RS scenes with noisy labels. Specifically, we consider the Normalized Softmax Loss and develop a robust extension, i.e., the Robust Normalized Softmax Loss (RNSL), in order to effectively capture the semantic relationships among RS scenes with mislabeled ground-truth information. The conducted experiments, using the K-NN classifier and two benchmark RS image archives, show the potential of the proposed approach with respect to other state-of-the-art methods.

In this paper, a novel edge guided structure extraction method is proposed for hyperspectral images classification, which consists of the following steps: First, the spectral dimension of the hyperspectral image is reduced with an averaging-based method. Then, the structural features is extracted by an extended relative total variation (ERTV) inspired by a learned edge probability map which serves as one of the major cues in the structure extraction process. Finally, the extracted structural features are fed into SVM for classification. Experimental results on two publicly available hyperspectral data sets demonstrate the competitive performance over several state-of-the-art classification approaches.

It is well known that classification accuracy highly relies on the number of labeled samples. However, it is difficult to obtain sufficient labeled samples in real-world applications. To solve this issue, a novel hyperspectral image (HSI) classification method based on polygonal partition is proposed for crop mapping. This method only needs single sample per class as an initial training set. Specifically, multiscale polygonal partition is applied on the first three components of the HSI. Then, a spectral similarity-based sample expansion method is proposed to obtain more labeled samples. Next, a pixel-wise classifier, the support vector machine (SVM), is used to acquire an initial classification result. Finally, classification result is further optimized according to the partition maps. Experimental results show that classification performance of the proposed method is satisfactory even when the number of labeled sample is single for each class.

The inclusion of spatial information into spectral classifiers for fine-resolution hyperspectral imagery has led to significant improvements in terms of classification performance. The task of spectral-spatial hyperspectral image (HSI) classification has remained challenging because of high intraclass spectrum variability and low interclass spectral variability. This fact has made the extraction of spatial information highly active. In this work, a novel HSI classification framework using the fusion of dual spatial information is proposed, in which the dual spatial information is built by both exploiting pre-processing feature extraction and post-processing spatial optimization. In the feature extraction stage, an adaptive texture smoothing method is proposed to construct the structural profile (SP), which makes it possible to precisely extract discriminative features from HSIs. The SP extraction method is used here for the first time in the remote sensing community. Then, the extracted SP is fed into a spectral classifier. In the spatial optimization stage, a pixel-level classifier is used to obtain the class probability followed by an extended random walker-based spatial optimization technique. Finally, a decision fusion rule is utilized to fuse the class probabilities obtained by the two different stages. Experiments performed on three data sets from different scenes illustrate that the proposed method can outperform other state-of-the-art classification techniques. In addition, the proposed feature extraction method, i.e., SP, can effectively improve the discrimination between different land covers.

Most deep metric learning-based image characterization methods exploit supervised information to model the semantic relations among the remote sensing (RS) scenes. Nonetheless, the unprecedented availability of large-scale RS data makes the annotation of such images very challenging, requiring automated supportive processes. Whether the annotation is assisted by aggregation or crowd-sourcing, the RS large-variance problem, together with other important factors [e.g., geo-location/registration errors, land-cover changes, even low-quality Volunteered Geographic Information (VGI), etc.] often introduce the so-called label noise, i.e., semantic annotation errors. In this article, we first investigate the deep metric learning-based characterization of RS images with label noise and propose a novel loss formulation, named robust normalized softmax loss (RNSL), for robustly learning the metrics among RS scenes. Specifically, our RNSL improves the robustness of the normalized softmax loss (NSL), commonly utilized for deep metric learning, by replacing its logarithmic function with the negative Box–Cox transformation in order to down-weight the contributions from noisy images on the learning of the corresponding class prototypes. Moreover, by truncating the loss with a certain threshold, we also propose a truncated robust normalized softmax loss (t-RNSL) which can further enforce the learning of class prototypes based on the image features with high similarities between them, so that the intraclass features can be well grouped and interclass features can be well separated. Our experiments, conducted on two benchmark RS data sets, validate the effectiveness of the proposed approach with respect to different state-of-the-art methods in three different downstream applications (classification, clustering, and retrieval). The codes of this article will be publicly available fromhttps://github.com/jiankang1991.

Convolutional neural networks (CNNs) have achieved great success when characterizing remote sensing (RS) images. However, the lack of sufficient annotated data (together with the high complexity of the RS image domain) often makes supervised and transfer learning schemes limited from an operational perspective. Despite the fact that unsupervised methods can potentially relieve these limitations, they are frequently unable to effectively exploit relevant prior knowledge about the RS domain, which may eventually constrain their final performance. In order to address these challenges, this article presents a new unsupervised deep metric learning model, called spatially augmented momentum contrast (SauMoCo), which has been specially designed to characterize unlabeled RS scenes. Based on the first law of geography, the proposed approach defines spatial augmentation criteria to uncover semantic relationships among land cover tiles. Then, a queue of deep embeddings is constructed to enhance the semantic variety of RS tiles within the considered contrastive learning process, where an auxiliary CNN model serves as an updating mechanism. Our experimental comparison, including different state-of-the-art techniques and benchmark RS image archives, reveals that the proposed approach obtains remarkable performance gains when characterizing unlabeled scenes since it is able to substantially enhance the discrimination ability among complex land cover categories. The source codes of this article will be made available to the RS community for reproducible research.

Sun glint, as the spectral reflection of solar radiation on non-flat water surfaces, is a serious confounding factor for coastal shallow-water environments. When the coastal areas are observed with a hyperspectral sensor, the existing sun glint in the produced images can seriously influence the quality of the image interpretation. To solve this issue, in this paper, we propose a novel sun glint removal method based on a variation model for hyperspectral images (HSIs). The proposed method aims to decompose the original HSI into a desired clean image and a sun glint image. To achieve this, we exploit a texture-aware total variation to remove the sun glint in HSIs, where the texture information is imposed on the total variation regularization to highlight sun glint. Experiments on simulated and real datasets demonstrate that our method can obtain outstanding performance with respect to other state-of-the-art approaches.

Hyperspectral imaging plays an important role for mineral mapping in a nondestructive and noninvasive way. In this paper, a novel resolution enhancement method is proposed based on the principle of intrinsic image decomposition for mineral mapping. This method is based on an assumption that hyperspectral image (HSI) can be decomposed into a reflectance component and an illumination component. Based on this idea, the RGB image is first transformed into Intensity-Hue-Saturation (IHS) space, and the intensity channel is considered as the illumination component of the HSI with an ideal high spatial resolution. Then, the reflectance component of the ideal HSI is estimated with the downsampled HSI image and the downsampled intensity channel. Finally, the HSI with high resolution can be reconstructed by utilizing the estimated illumination and the reflectance components. Experimental results validate the effectiveness of the proposed method qualitatively and quantitatively by outperforming several state-of-the-art approaches.

Hyperspectral Image (HSI) visualization, which aims at displaying as much material information of original images as possible on a trichromatic monitor with natural color, plays an important role in image interpretation and analysis. However, most of the HSI visualization methods only focus on presenting the detail information of a scene without providing natural colors and distinguishing land covers with similar colors. In order to address this problem, this article proposes a multichannel pulse-coupled neural network (MPCNN)-based HSI visualization method, which consists of the following steps. First, the MPCNN is proposed and explored to fuse the original HSI so as to obtain a fused band with rich spatial details. Then, a color mapping scheme is proposed to determine the weights of red, green, and blue (RGB) channels. Finally, the weighted RGB channels are stacked together for visualization. Experiments performed on four hyperspectral data sets demonstrate that the proposed method not only displays the HSI with nature colors but also improves the details in the image. The effectiveness of the proposed method is demonstrated in terms of both visual effect and objective indexes.

In this article, a novel hyperspectral anomaly detection method with kernel Isolation Forest (iForest) is proposed. The method is based on an assumption that anomalies rather than background can be more susceptible to isolation in the kernel space. Based on this idea, the proposed method detects anomalies as follows. First, the hyperspectral data are mapped into the kernel space, and the first K principal components are used. Then, the isolation samples in the image are detected with the iForest constructed using randomly selected samples in the principal components. Finally, the initial anomaly detection map is iteratively refined with locally constructed iForest in connected regions with large areas. Experimental results on several real hyperspectral data sets demonstrate that the proposed method outperforms other state-of-the-art methods.

In the process of research, development, production, service, and maintenance of silicon photovoltaic (Si-PV) cells and the requirements for detection technology are becoming more and more important. This paper aims to investigate electromagnetic induction (EMI) and image fusion to improve the detection effect of electrothermography (ET) and electroluminescence (EL) of multidefects in Si-PV cells. First, the principles of ET, EL, and other physical processes including EMI, thermal radiation, and luminescence radiation are analyzed in this paper. ET and EL techniques after EMI improvement are used to detect different defects including scratch, broken gridline, surface impurity, hidden crack, and so on. The qualitative results show that EMI can greatly improve the defect detection ability of ET and EL. Then, an image-fusion rule based on L1 norm is proposed to fuse the sparse vector of the ET and EL images. The integration and complementarity of the two wavelength detection data are achieved. Finally, the image-fusion results of sparse representation (SR) algorithm is compared with discrete wavelet transform, curvelet transform, dual-tree complex wavelet transforms, and nonsubsampled contourlet transform. Five objective evaluation indexes including root mean square error, peak signal-to-noise ratio, correlation coefficient, mutual information, and structural similarity index are used to evaluate the fusion results. Overall evaluation results show that the SR algorithm is superior to the other algorithms.

In this paper, a novel deep learning based visualization method is proposed for natural color visualization of hyperspectral images, which consists of the following steps. First, the spectral bands of the hyperspectral image are divided into two groups, i.e., the red, green, and blue (RGB) bands and the remaining bands. Then, a pretrained convolutional neural network (CNN) model, i.e., VGG-19, is explored to fuse the remaining bands so as to obtain a fused band with rich details. Next, the intensity-hue-saturation (IHS) transform is performed on the averaged red, green, and blue bands to obtain three different components, i.e., intensity (I), hue (H), saturation (S). Finally, the fused band is utilized to replace the intensity component followed by an inverse IHS transform. Experiments performed on two hyperspectral data sets demonstrate that the proposed method cannot only obtain a natural color resulting image, but also well preserving image details with respect to several state-of-the-art methods.

Recently, convolutional neural networks (CNNs) as a powerful tool have been introduced for classification of hyperspectral images (HSIs). However, it fails to take the feature redundancy into consideration. Hence, for the pixel-wise HSI classification, the CNN-based methods may not effectively extract the discriminative features from the complex scene in HSIs. In order to overcome this problem, in this letter, a novel dual-path network (DPN)-based HSI classification method is proposed, in which the DPN combines the advantages of the residual network and dense convolutional network. First, the principal component analysis is utilized to extract significant components of HSI. Second, training image patches centered on labeled pixels are constructed to train the DPN. Finally, the labels of test pixels are predicted by using the trained network. Experiments conducted on two hyperspectral data sets demonstrate the state-of-the-art performance of the proposed method over other compared methods in terms of classification accuracies.

In this article, a novel hyperspectral image (HSI) classification method based on fusing multiple edge-preserving operations (EPOs) is proposed, which consists of the following steps. First, the edge-preserving features are obtained by performing different types of EPOs, i.e., local edge-preserving filtering and global edge-preserving smoothing on the dimension-reduced HSI. Then, with the assistance of a superpixel segmentation method, the edge-preserving features are further improved by considering the inter and intra spectral properties of superpixels. Finally, the spectral and edge-preserving features are fused to form one composite kernel, which is fed into the support vector machine (SVM) followed by a majority voting fusion scheme. Experimental results on three data sets demonstrate the superiority of the proposed method over several state-of-the-art classification approaches, especially when the training sample size is limited. Furthermore, 21 well-known methods, including mathematical morphology-based approaches, sparse representation models, and deep learning-based classifiers, are adopted to be compared with the proposed method on Houston data set with standard sets of training and test samples released during 2013 Data Fusion Contest, which also shows the effectiveness of the proposed method.

In this paper, a novel multi-scale structure extraction based spectral-spatial hyperspectral image classification method is proposed, which consists of the following steps. First, the spectral dimension of the hyperspectral image is reduced by averaging adjacent spectral bands. Then, in order to extract the multi-scale significant structural features (MSFs) which are insensitive to image noise and texture, a relative total variation based structure extraction method is applied on the dimension reduced hyperspectral image. Finally, the MSFs are fused together with the kernel principal component analysis (KPCA), so as to obtain the kernel PCA fused multi-scale structural features (KPCA-MSFs) for classification. Experiments conducted on a real hyperspectral image demonstrate the outstanding performance of the proposed approach over several state-of-the-art spectral-spatial classifiers, especially when the image is corrupted by serious scene noise.

In this letter, a novel anomaly detection method is proposed, which can effectively fuse the multiscale information extracted by attribute and edge-preserving filters. The proposed method consists of the following steps. First, multiscale attribute and edge-preserving filters are utilized to obtain multiscale anomaly detection maps. Then, the multiscale detection maps are fused via an averaging approach, and the training samples of the anomalies and background are selected from the fused detection map. Next, the support vector machine classification is performed on the hyperspectral image to obtain an anomaly probability map. Finally, the detection result is obtained by multiplying the fused detection map and the anomaly probability map, followed by an edge-preserving filtering-based postprocessing. Experiments performed on four real hyperspectral data sets demonstrate that the proposed method shows a better detection performance with respect to several state-of-the-art hyperspectral anomaly detection methods.

Image decolorization is known to be an effective way in transferring a color image into a gray one while well preserving the major information of all three bands. In this paper, a simple yet effective hyperspectral image visualization framework based on decolorization, named decolorization based hyperspectral visualization, is proposed, which enables us to fully exploit the benefits of decolorization technique. The proposed framework consists of the following two main steps. First, the hyperspectral image is partitioned into nine subsets of adjacent hyperspectral bands and the averaged band of each subset is calculated. Then, the dimension reduced image is further divided into three groups of adjacent bands, and the bands in each group are fused by using an image decolorization method. The main contribution of this paper is that the strong correlations in two different fields, i.e., image decolorization and hyperspectral image visualization, are first built. Experiments performed on several real hyperspectral data sets demonstrate that the proposed framework can obtain outstanding visualization performance in terms of both subjective and objective evaluations.

In this paper, a novel method is introduced to detect and correct mislabeled training samples for hyperspectral image classification. First, domain transform recursive filtering-based feature extraction is used to improve the separability of the training samples. Then, constrained energy minimization-based object detection is performed on the training set with each training sample serving as the object spectrum. Finally, the label of each training sample is verified or corrected based on the averaged detection probabilities of different classes. Experiments performed on real hyperspectral data sets demonstrate the effectiveness of the proposed method in improving classification performance with respect to the classifier trained with the original training set that contains a number of mislabeled samples.