Effective deep feature extraction via feature-level fusion is crucial for multimodal object detection. However, previous studies often involve complex training processes that integrate modality-specific features by stacking multiple feature-level fusion units, leading to significant computational overhead. To address this issue, we propose a lightweight attention-guided self-modulation feature fusion network (LASFNet). The LASFNet adopts a single feature-level fusion unit to enable high-performance detection, thereby simplifying the training process. The attention-guided self-modulation feature fusion (ASFF) module in the model adaptively adjusts the responses of fused features at both global and local levels, promoting comprehensive and enriched feature generation. Additionally, a lightweight feature attention transformation module (FATM) is designed at the neck of LASFNet to enhance the focus on fused features and minimize information loss. Extensive experiments on three representative datasets demonstrate that our approach achieves a favorable efficiency-accuracy tradeoff. Compared to state-of-the-art methods, LASFNet reduced the number of parameters and computational cost by as much as 90% and 85%, respectively, while improving detection accuracy mean average precision (mAP) by 1%-3%. The code will be open-sourced at https://github.com/leileilei2000/LASFNet.

Deep learning for anomaly detection is one of the current hot topics in remote sensing. However, most deep learning-based methods require retraining or parameter fine-tuning when detecting different hyperspectral images, which increases the complexity of model usage and wastes computing resources. Although a few one-step detection methods exist, they are generally constrained by fixed spatial and spectral dimensions, limiting their flexibility in handling different hyperspectral images. More importantly, this constraint often disrupts the subtle and critical relationships between spectral bands, affecting the comprehensiveness and accuracy of image analysis. To overcome these issues, we propose an adaptive one-step hybrid network for hyperspectral anomaly detection (denoted as AdaptHAD). First, an adaptive sparsity module is constructed to assist in dividing samples in the training phase by utilizing the sparsity property of anomalies in hyperspectral images. Second, a data augmentation method that combines spectral information erasure with random noise is proposed to address the small sample problem. Third, a convolutional attention hybrid network with a transformer and convolutional neural network dual branches is designed to extract global and local features simultaneously to strengthen the model to identify anomalies and backgrounds. Extensive experiments on real-world hyperspectral images demonstrate the effectiveness of the proposed method. The code is available at https://github.com/ismeXQ/AdaptHAD.

Currently, few-shot learning (FSL) is widely used in image classification hyperspectral image classification (HSIC), owing to its exceptional proficiency in achieving good performance with few training samples. Although the FSL has made good progress, there are still some problems to be solved. On the one hand, existing methods rely on linear distance to learn metrics, which cannot capture the subtle similarities and differences between scarce prior samples. On the other hand, many current methods directly superimpose the features of spatial and spectral information, without deeply fusing the internal relationship between these two kinds of information. To address the aforementioned issues, a deep metric learning method based on Brownian distance covariance (DML-BDC) is proposed for few-shot HSIC. A dual-channel Brownian distance covariance feature extraction network is designed, which uses the Brownian covariance representation to model and fuse the spatial and spectral information and uses two different feature extractors to achieve the effect of information complementarity. Then, a metric loss based on Gaussian kernel distance is proposed to learn the complex nonlinear structure and subtle similarities and differences between support samples. Experiments on three benchmark datasets show that DML-BDC has advantages over the existing mainstream methods in terms of classification accuracy, generalization, and model complexity.

The hierarchical classification method can utilize the hierarchical structure to achieve fine classification from coarse-grained to fine-grained. Hierarchical classification has demonstrated outstanding performance across various fields, benefiting from the continuous expansion of dataset sizes. Nevertheless, the majority of current hyperspectral image (HSI) deep learning classification methods employ flat classification strategy, disregarding the hierarchical structure present in HSI data. Can hierarchical structure information assist deep learning in completing HSI classification tasks? In this paper, we validated the feasibility of using hierarchical structure information to assist deep learning in completing HSI classification tasks through intuitive experiments and then proposed a semi-supervised hierarchical classification network (Semi-HCN) to achieve hierarchical classification of HSI. Semi-HCN first generates a large number of pseudo-labels through a fast self-training module for subsequent network training, then extracts hierarchical features through a multi-branch network and progressively consolidates them across layers. Finally, hierarchical constraints are applied to the network output through hierarchical cross entropy loss. Experiments on three real datasets have demonstrated that Semi-HCN can achieve competitive classification performance with limited training samples. The code will be released at https://github.com/jinyaoWHU/Semi-HCN.

Under the broad definition of multimodal data, data presenting different views and complementary information are classified as multimodal. Two-dimensional magnetotelluric (MT) responses, including apparent resistivity and phase of two polarization modes, reflect differing physical properties and offer complementary insights into the subsurface media. Traditional deep learning (DL) approaches often struggle to capture and integrate these complementary features effectively for accurate inversion. In this study, we treat$\rho _{S}^{\text {TE}} $,$\varphi ^{\text {TE}}$,$\rho _{S}^{\text {TM}}$, and$\varphi ^{\text {TM}}$of 2-D MT as multimodal data and introduce a data fusion method of multimodal DL (MDL), which enhances the accuracy of MT inversion by employing a multimodal U-net model to integrate various MT response features effectively. In detail, each MT response is processed in a different encoder to exploit its unique information better. It is densely connected within each encoder and across different encoders, facilitating the fusion of MT data across depths and response modalities. Our method maximizes complementary information from multiple response modalities, resulting in a more precise depiction of nonlinear processes in MT inversion. First, 2-D Gaussian random fields (GRFs) simulate the resistivity model. Then, the multimodal U-net is introduced and improved as the MT inversion framework and compared with the conventional U-net. Moreover, the anti-noise ability and generalization of the multimodal U-net are tested by introducing varying noise levels into the MT responses. Finally, we validate our proposed method with MT field data from the Yanggao area in the Datong Basin, Shanxi Province, China, showing that the performance of our model surpasses conventional U-net and traditional nonlinear conjugate gradient (NCG) inversion methods.

Most of the current hyperspectral image classification (HSIC) methods assume that the interactions among all ground objects in hyperspectral images (HSIs) are static pairwise relationships. However, in real scenarios, multiple ground objects have complex spatial, spectral, or statistical correlations. These correlations are not limited to simple adjacent or pairwise relationships but also include complex higher order interactions involving three or more ground object categories. A hyperedge in a hypergraph can simultaneously connect multiple vertices, effectively capturing the multi-dimensional and high-order relationships among vertices. To address the limitations of the current mainstream methods in modeling the high-order interaction relationships of ground objects, a novel multifeature collaborative attention dynamic hypergraph convolutional (MDHGC) network is proposed to model the entire HSI and capture the high-order relationships among ground objects, thereby achieving accurate classification. Specifically, we designed a static–dynamic collaborative multiview hypergraph convolutional network based on differential attention to learn superpixel-level features, which allows stable and flexible learning of high-order interactions in HSI. To learn the complementary features of pixel-level HSIC, we introduce a branch based on convolutional neural neworks that includes multiscale feature extraction and global–local feature fusion. Comprehensive experiments have been conducted across four distinct datasets to rigorously evaluate the effectiveness of MDHGC.

Deep learning (DL) has received a lot of attention in hyperspectral anomaly detection (HAD) in recent years. While some progress has been made in boosting the generalization of DL-based HAD methods, existing methods still face limitations in labeled sample generation or transfer flexibility. To address these issues, we propose a contrastive self-supervised learning-based background reconstruction method for HAD (CSSBR). By constructing a self-supervised pretraining model based on a pixel- and patch-level masking strategy and a dual attention network (DAN) encoder, the pretraining model can learn general background representation without labeled sample generation. The DAN encoder is constructed by the vision transformer (ViT) and channel attention module (CAM) to extract the global context information and the correlation between spectra. A transfer learning model is constructed based on the DAN encoder and a lightweight decoder, which is simply fine-tuned by background reconstruction guided (BRG) loss to flexibly transfer the pretrained model to various HAD tasks. Experiments on four challenging hyperspectral datasets confirm that the CSSBR method outperforms other state-of-the-art methods.

Due to the limitations of imaging environment and hardware conditions, mixed pixels are common in hyperspectral images, which seriously affects the accuracy of land use coverage mapping. Subpixel mapping (SPM) decomposes mixed pixels to obtain the spatial distribution information of local object components inside the pixel, thereby breaking through the limitations of traditional pixel-level classification and achieving more accurate land use interpretation and refined mapping. Recently, deep convolutional neural networks have demonstrated their potential and effectiveness in SPM. However, in the SPM process, the multiscale spatial context information are not fully utilized in the process of using semantic information for network modulation, and the spatial representation at a more abstract level cannot be fully obtained. Therefore, in response to the above problems, this article proposes a multiscale semantic modulation hybrid convolutional network for SPM. The network obtains multiscale semantic information in semantics by constructing a multiscale semantic modulation module (MSSM) to modulate the backbone network and fully mine the spatial context information. Simultaneously, a hybrid convolutional module integrating, 2-D convolutional neural networks, 3D convolutional neural networks, and attention mechanisms is designed. This module captures joint spatial–spectral features while reducing model complexity and learns more abstract spatial representations to enhance the network’s performance in SPM. Experimental results show that this method outperforms the most advanced SPM methods on three public datasets and a produced wetland dataset, and the details of land cover categories are more prominent. The code and data will be released on GitHub upon acceptance:https://github.com/UPCGIT/MSMCNet

Currently, the research on cross-scene classification of hyperspectral image (HSI) based on domain generalization (DG) has received wider attention. The majority of the existing methods achieve cross-scene classification of HSI via data manipulation that generates more feature-rich samples. The insufficient mining of complex features of HSIs in these methods leads to limiting the effectiveness of the newly generated HSI samples. Therefore, in this paper, we propose a novel single-source frequency transform (SFT), which realizes domain generalization by transforming the frequency features of samples, mainly including frequency transform (FT) and balanced attentional consistency (BAC). Firstly, FT is designed to learn dynamic attention maps in the frequency space of samples filtering frequency components to improve the diversity of features in new samples. Moreover, BAC is designed based on the class activation map to improve the reliability of newly generated samples. Comprehensive experiments on three public HSI datasets demonstrate that the proposed method outperforms the state-of-the-art method, with accuracy at most 5.14% higher than the second place.

Object tracking is considered as a template matching task. Traditional and deep learning-based methods have achieved high performance in satellite video object tracking (SVOT). However, existing methods still suffer from insufficiently discriminative features, complex approaches to handling occlusion, and excessive hyperparameters. In response to these issues, we propose a simple, yet effective Siamese network, termed SiamTITP. A temporal information (TI) submodule is developed, which integrates temporal cues by dynamically updating the template to enhance discriminative features. Furthermore, we propose a structurally simple trajectory prediction (TP) submodule, which solely utilizes polynomial function for fitting historical results to assist the network in addressing occlusion. In an effort to reduce hyperparameters, we forgo feature fusion steps and weighted results, while we propose an adaptive occlusion judgment metrics based on the target size. To validate the efficacy of our approach, we conducted extensive experiments on three large satellite video datasets, namely the SatSOT, SV248S and OOTB datasets. Code and train models are publicly available at https://github.com/jiawei-zhou/SiamTITP.

We consider the computational complexity of spanning tree problems involving the graphical function-index. This index was recently introduced by Li and Peng as a unification of a long list of chemical and topological indices. We present a number of unified approaches to determine the NP-completeness and APX-completeness of maximum and minimum spanning tree problems involving this index. We give many examples of well-studied topological indices for which the associated complexity questions are covered by our results.

We characterize the bipartite graphs that minimize the (first degree-based) entropy among all bipartite graphs of given size. For bipartite graphs given size and (upper bound on the) order, we give a lower bound for this entropy. The extremal graphs turn out to be complete bipartite graphs, or nearly complete bipartite. Here we make use of an equivalent representation of bipartite graphs by means of Young diagrams, which make it easier to compare the entropy of related graphs. We conclude that the general characterization of the extremal graphs is a difficult problem, due to its connections with number theory. However, it is easier to identify them for particular values of the order n and size m because we have narrowed down the possible extremal graphs. We indicate that some of our ideas extend to other degree-based topological indices as well.

As a popular deep learning (DL) algorithm, graph neural network (GNN) has been widely used in hyperspectral image (HSI) classification. However, most of the GNN-based classification algorithms are concentrated in the field of semisupervision, which heavily relies on the quantity and quality of samples. To solve this problem, we propose an unsupervised multiview graph contrastive (UMGC) feature learning algorithm to explore the deep semantic features of HSIs without being constrained by samples. First, we construct multiview adjacency matrixes from spatial and spectral directions. Second, the adaptive data augmentation method is used to selectively enhance the topology and attribute structure of the graph. Thereafter, features are extracted by using a contrastive loss to maximize the similarity between the two views. Finally, we tested the model’s performance based on multiple evaluation methods. Experimental results on three publicly available hyperspectral datasets show that the proposed UMGC can have better classification performance compared with other state-of-the-art unsupervised feature extraction (FE) methods.

Deep learning has shown great success in hyperspectral target detection (HTD). Due to the limited targets of interest in hyperspectral image (HSI), many deep learning based methods focus on spectral feature extraction by expanding samples, exhibiting low model transferability as well as high time consumption. To address these issues, we propose a novel spatial-spectral contrastive self-supervised learning-based HTD method with dual path networks (DPN-CSSTD). This method constructs a contrastive task using a HSI augmentation method and a DPN encoder on a large amount of unlabeled data, enabling the pretrained model with spatial-spectral discrimination ability. The DPN encoder utilizes the convolution neural network and the skip connections to improve the spatial-spectral feature extraction. Additionally, the proposed method constructs a transfer learning model with the pretrained DPN encoder and a newly added detector to improve the pretrained model transferability and the time efficiency of target detection tasks. The transfer learning model is fine-tuned via a proposed fine-tuning strategy with few selected samples to adapt to different TD tasks. Extensive experiments conducted on four challenging hyperspectral datasets demonstrate that the proposed method outperforms other state-of-the-art approaches.

Deep learning (DL) has made significant progress in hyperspectral target detection (HTD) in recent years. However, the existing DL-based HTD methods generally generate numerous labeled samples for network training, which may be impure or too similar to each other. Moreover, most methods with enormous parameters are trained and tested on the same dataset, resulting in single scenario applicability and significant computational consumption issues. To solve these issues, we propose a generative self-supervised learning (GSSL) pretraining model with spectral-spatial masking (S2M). The lightweight vision transformer (ViT) is utilized as the backbone to learn the universal feature representation of images without labeled samples. Subsequently, the pretrained model is transferred to various HTD tasks. The transfer learning model is constructed via the lightweight ViT and a fully connected (FC) layer and fine-tuned via a weighted binary cross entropy (WBCE) loss function and a small number of selected samples. We evaluate its effectiveness on four challenging hyperspectral datasets in terms of the GSSL pretraining model, the S2M strategy, and the WBCE loss function. Our methods achieve improvements in comparison to different pretraining models, masking strategies and loss functions. And our detection results also outperform other state-of-the-art approaches.

Cross-scene classification of hyperspectral image (HSI) based on single-source domain generalization (SDG) focuses on developing a model that can effectively classify images from unseen target domains using only source domain images, without the need for retraining. Most existing SDG approaches for cross-scene classification rely on convolutional neural networks (CNNs). However, the convolutional kernel operation causes the model to emphasize local object features, which can lead to overfitting on the source domain and limits its ability to generalize. Recently, methods based on the state space model (SSM) have demonstrated excellent performance in image classification by capturing global features across different image patches. Building on this inspiration, we propose a novel approach called dynamic token augmentation mamba (DTAM), which aims to explore the potential of SSMs in the cross-scene classification of HSI. The method gradually focuses on the global features of the image by constructing hidden states for HSIs unfolded into long sequences. To further enhance the global features of HSIs, we design a dynamic token augmentation (DTA) module to transform the sample features by perturbing the contextual information while preserving the object information tokens. Additionally, we introduce a loss of classified compensation combined with labels of random samples to suppress the excessive narrowing of the feature range learned by the model. Comprehensive extensive experiments on three publicly available HSI datasets show that the proposed method outperforms the state-of-the-art (SOTA) method. Our code is available athttps://github.com/Varro-pepsi/DTAM.

Object detection is a challenging task that aims to locate and classify instances in an image simultaneously. High-performing deep detectors are typically trained on extensive labeled datasets and will face the problem of accuracy degradation in different scenarios. Through domain adaptation, the detector can attain generalization, making precise detections even in the presence of different data distributions without labels. In this article, we propose a confidence-driven region mixing (CR-Mixing) framework, which is a novel approach that integrates the region-selective mixing (RSM) method into a mean-teacher framework with an unbiased adversarial module (UAM), addressing the domain-shift and knowledge transfer problems in the unsupervised domain adaptation (UDA) object detection task. First, we construct RSM that utilizes pseudodetection labels to compute object information at the region level and integrates this information in a balanced manner to maximize accurate foreground details for the detection model. This enables the detection model to adjust its focus to unlabeled domains gradually. Second, to acquire a backbone network with object-focused features, we develop the UAM method. This method is a multiscale alignment network designed to constrain the backbone network in learning domain-invariant features. We conducted extensive experiments on two benchmarks, and the final results show that the CR-Mixing model demonstrates significant effectiveness in robust UDA for optical remote sensing object detection tasks.

Hyperspectral target detection (HTD) has been a critical issue in the field of Earth observation, with widespread applications in both military and civilian domains. However, existing deep learning-based HTD methods are hindered due to insufficient and low-quality prior training samples, as well as inadequate background suppression capabilities. To address these issues, this article proposes an adversarial sample generation-based multiscale Siamese network (AGMS) for HTD. First, the AGMS utilizes the idea of generative adversarial learning based on the prior few targets and diverse backgrounds to generate adversarial target-background sample pairs, thereby producing high-quality training samples, which enhances the distinctiveness between the target and background samples by adversarially training the generator to produce the target/background samples. In addition, to further highlight the targets and suppress the backgrounds, a difference amplification loss and an adaptive weighted binary cross-entropy loss are proposed. Finally, a multiscale convolutional Siamese network model is designed to explore the generated spectral information at multiple levels and achieve target detection through contrastive learning. Numerous experimental results on four real HSI datasets verify the superiority of the AGMS in comparison to many classical and recently proposed HTD methods. The codes are available athttps://github.com/ShissHAN/AGMS.

Urban road extraction is important for the applications of urban planning and transportation. High-resolution imagery (HRI) has been one of the most popular data sources for extracting roads with high efficiency and low cost. However, roads in HRI are easily obscured by buildings, trees and other landscapes, resulting in discontinuity of the extracted roads. While current road extraction techniques by multi-modal data fusion have shown improved results compared to single-modal methods by incorporating additional information, most existing fusion methods fail to fully exploit the features from different modalities and consider prior knowledge of roads. To address the above problems, a dual encoder-based cross-modal complementary fusion network (DECCFNet) is proposed in this paper. The proposed network takes full advantage of the rich feature information contained in HRI and the immunity of LiDAR data to the influence of shadows. By effectively fusing the complementary information from HRI and LiDAR data, DECCFNet respectively achieved an improvement by at least 2.94% and 2.8% in IOU compared to those only using a single data modality on the two datasets. The proposed DECCFNet mainly contains two modules: 1) Cross-modal feature fusion module (CMFF): In the dual encoder part, CMFF is employed to fuse the deep features of different modalities from the channel and spatial dimension, while a multi-scale fusion strategy is utilized to extract the contextual information. 2) Multi-direction stripe convolution module (MDSC): Since roads have the characteristics of narrowness and continuity, adopting classical convolution kernels directly on road features may introduce irrelevant pixels into the computation, blurring the extraction results. To mitigate this issue, MDSC is applied to strip convolution of road features from multiple directions based on square convolution, and make the network focus more on the specific road features. By comparing several deep learning multimodal data fusion networks in the Erie road dataset, the proposed network exhibits the best road extraction results.

Hyperspectral change detection (HCD) with limited training samples has attracted increasing attention in recent years. The current research works require to train a dedicated model for each dataset, resulting in limited generalization. Moreover, for the same dataset, a model that has been trained for binary change detection (CD) task usually needs to be retrained for multiclass CD task, thereby leading to low efficiency. We propose a cross-task meta-learning network with graph-enhanced attention module (CMGA) for both binary CD tasks and multiclass CD tasks, aiming to fully extract the spatial-spectral features of hyperspectral images (HSIs) and efficiently transfer the prior knowledge to accomplish various HCD tasks with limited training samples. In order to capture general prior knowledge, the multitask meta-optimization is designed to train the pretraining model with the ability of addressing cross-task new classes. Moreover, the diverse tasks are unified into one framework, which aims at identifying the target class from other classes to improve the model efficiency and generalization. The graph-enhanced attention (GA) module is developed to leverage the graph knowledge extract from the graph transformer (GT) module to enhance the pixel-level feature representation. Experiments on four hyperspectral datasets illustrated the effectiveness of the proposed algorithm.

Deep learning can fully exploit the potential information of data, which facilitates effective target-background separation for hyperspectral target detection (HTD). The deep learning model usually requires a large number of labeled samples, yet the available prior target spectra in the hyperspectral image (HSI) are extremely limited. In addition, network training also requires reliable input samples to ensure that the trained model has stronger data discrimination, but current detection methods often suffer from the problem that the background samples are impure. To address the above problems, we propose a lightweight convolutional neural network based on joint correlation distance constraint and density peak clustering (LCNN-CD) for HTD. First, the correlation distance between the prior target and HSI is calculated, and the pixels with high correlation are retained for expanding the target sample, which effectively handles the problem of insufficient target samples. Second, the density peak clustering algorithm is used to extract the main pure background samples, which effectively solves the problem that the background samples are sufficiently impure. Finally, a lightweight convolutional neural network model is designed, which is fed by the training samples (both target and background samples) to obtain the final detection results with low computational efficiency. The proposed method is compared with the classical and recently popular hyperspectral target detection methods on three real HSI datasets. Numerous experiments show that the proposed LCNN-CD method has better target detection performance and effectiveness.

Domain adaptation purposes to transfer well-labelled domain knowledge to poor-labelled domain. However, many domain adaptation methods focus too much on spectral features and simple spatial features, causing failure to eliminate noise that is an important risk in remote sensing image classification. Degenerated features are also a potential risk to feature transformation in domain adaptation. In addition, it is also a typical problem that reduces the difference in probability distribution between different domains for domain adaptation. To alleviate these issues, we propose a multilevel spatial features-based manifold metric learning (MSF-MML) method for domain adaptation in remote sensing image classification. Firstly, it applies and iterates a spatial information filtering to extract multilevel spatial features by calculating the average of sample pixels and neighboring pixels, effectively obtaining more stable feature information. Then, it utilizes the Grassmann manifold space to alleviate the problem of degenerated features of remote sensing data in domain adaptation. Finally, it exploits maximum mean discrepancy to construct metric constraints that reduces discrepancy in domains probability distributions. The effectiveness of MSF-MML was demonstrated on three real-world datasets.

Hyperspectral images (HSI) contain hundreds of bands, which provide a wealth of spectral information and enable better characterization of features. However, the excessive dimensions and redundant information also cause a dimensional disaster for subsequent processing. Band selection is a widely-used dimension reduction technique for hyperspectral images. Traditional methods mainly consider the hyperspectral band selection problem at the level of data, and maintain the information contained in the data, without considering the spatial structures inside hyperspectral images. To fill the gap, in this work, an unsupervised hyperspectral band selection method through structure-conserved and neighborhood-grouped evolutionary algorithm (SNEA) is proposed. Different from other evolutionary algorithms for hyperspectral band selection, firstly, two spatial-structure related optimization objectives are designed, including the locally spatial structure denoted by the pixel’s spatial consistency with its adjacent neighbors and the globally spatial structure denoted by the affinity graph among pixels. With the designed objectives, the hyperspectral band selection is formulated as the problem of conserving spatial structures. Moreover, a neighborhood-grouped pair-wise learning strategy is proposed to generate high-quality offsprings. In this novel strategy, a neighborhood grouping operation is developed to divide the band space into several groups. The population can be initialized efficiently and the offspring solutions can be generated pairwisely under the guidance of grouping. Compared with 9 state-of-the-art comparison algorithms, experimental results on 3 standard hyperspectral datasets demonstrate that the band subset obtained by our proposed SNEA has a better classification performance than the comparison algorithms.

Self-supervised learning has shown remarkable success in image representation learning. Among these methods, masked image modeling and contrastive learning are the most recent and dominant methods. However, these two approaches will behave differently after being transferred into various downstream tasks. In this paper, we propose a RGB-elevation contrastive and image mask prediction pre-training framework. The elevation is normalized digital surface model. Then we evaluate the learned representation by transferring the pre-trained model into change detection task. To this end, we leverage the recently proposed vision transformer’s capability of attending to objects and combine it with the pretext task which is consist of masked image modeling and instance discriminant for fine-tuning the spatial tokens. Besides, the change detection task also requires us to do information interaction between the two temporal remote sensing images. To counter this problem, we propose a plug-in temporal fusion module based on masked cross attention and then we evaluate its effectiveness in three open change detection datasets in terms of initializing the supervised training weights. Our method achieves improvements in comparison to supervised learning methods and two mainstream self-supervised learning methods MoCo and DINO on change detection task. The results of our experiment also achieve state-of-the-art in four change detection datasets. The code will be available at URL.

In hyperspectral target detection, the conventional metric learning-based algorithms provide unique advantages in detecting targets as they do not require specific assumptions and adapt to the condition of limited training samples. Nevertheless, they usually learn a linear transformation for metric space, which is unable to capture nonlinear mapping where the hyperspectral imageries possess, especially occurs in the spectra variability and nonlinear mixing problems. To alleviate this limitation, this study investigates a new spatial-spectral adaptive sample generation and deep metric learning-based method for hyperspectral target detection (denoted as DMLTD). The proposed DMLTD employs a spatial-spectral adaptive sample generation strategy and subpixel synthetic method for background sample generation and target sample augmentation, respectively. With sufficient samples, the proposed DMLTD trains a deep discriminative metric learning network to learn hierarchical nonlinear mappings, so that to address the spectra variability and nonlinear mixing problems, thus exploiting discriminative information between targets and backgrounds for detection. Experiments and analyses conducted on three real-world hyperspectral datasets indicate that our DMLTD yields competitive performance in hyperspectral image target detection.

Most transfer learning methods have the problem of insufficient distance constraint that plays a very important role in improving image classification performance. Therefore, this letter proposes a new method called joint distance transfer metric learning (JDTML) for remote-sensing image classification. First, the JDTML method establishes the constraints of marginal distribution, intraclass distance, interclass distance, and intraclass divergence based on the maximum mean discrepancy. Second, the objective function is to combine these constraints. So, JDTML can not only reduce the differences between the two domains on the whole and in each class, but also gather the samples of the same class and expand the distance from each class to the rest classes. By solving the objective function, the transfer metric matrix is obtained. Finally, the source and target domains are transferred to a common subspace for dimension reduction. The data after dimension reduction is used for classification, and the accuracy of classification is improved by iteration. The experimental results show that JDTML is more accurate than other methods compared.

Traditional target detection methods assume that the background spectrum is subject to the Gaussian distribution, which may only perform well under certain conditions. In addition, traditional target detection methods suffer from the problem of the unbalanced number of target and background samples. To solve these problems, this study presents a novel target detection method based on asymmetric weighted logistic metric learning (AWLML). We first construct a logistic metric-learning approach as an objective function with a positive semidefinite constraint to learn the metric matrix from a set of labeled samples. Then, an asymmetric weighted strategy is provided to emphasize the unbalance between the number of target and background samples. Finally, an accelerated proximal gradient method is applied to identify the global minimum value. Extensive experiments on three challenging hyperspectral datasets demonstrate that the proposed AWLML algorithm improves the state-of-the-art target detection performance.

Dimensionality reduction (DR) technology is an important part of hyperspectral image (HSI) processing. The DR technology can effectively remove the redundant information in the HSIs and avoid the Hughes phenomenon, which is beneficial to image classification. Metric learning is widely used in DR technology, the goal of which is to achieve DR by maximizing the distance of between-class while minimizing the distance of within-class. However, traditional metric learning does not consider the intrinsic structure of data during the training process, resulting in insufficient information utilization. In order to solve the above problems, this article proposes clustered multiple manifold metric learning (CM3L) by combining manifold learning with metric learning for DR and classification of HSI. The proposed CM3L algorithm first divides the original data into multiple independent clusters and regards each sample point and its near point in the cluster as a whole, constructing it as a manifold. Then, CM3L uses the manifold metric distance to replace the traditional metric distance. Finally, by making full use of the local information of HSI in such a way, the discrimination ability is enhanced. Intensive experimental results on three real HSI datasets validate the effectiveness of our proposed CM3L algorithm.

Metric learning, which aims to obtain a metric matrix M such that samples from the same class are close to one another and samples of different classes are far from one another, is widely used in the field of hyperspectral dimensionality reduction (DR) and classification. Traditional metric learning is based on the Mahalanobis distance, which measures the similarity between samples via point-to-point distance, ignoring the structural features of the hyperspectral images (HSIs). To solve the above problem, we proposed clustered multiple manifold metric learning (CM3L), which obtains a manifold distance (MD), aimed at improving discrimination by introducing structural features of the HSIs and achieving good results. However, this manifold distance still has certain shortcomings in specific application situations. MD only considers the labeled data in the construction of the manifold and ignores the unlabeled data, resulting in the destruction of the manifold. Therefore, this article proposes a new spatial–spectral manifold distance (SSMD) to improve the performance of metric learning in hyperspectral DR and classification by maintaining the integrity of the constructed manifolds. The SSMD selects suitable neighboring points in the labeled and unlabeled data through the spectral–spatial information in order to participate in the construction of the manifold. Then, the distance between the manifolds is calculated to replace the traditional Mahalanobis distance. The results of seven sets of comparison experiments on three real HSI datasets demonstrate the effectiveness of SSMD in improving the classification results of HSIs.

Deep learning has achieved impressive results on hyperspectral images (HSIs) classification. Among them, both convolutional neural networks (CNNs) and graph neural networks (GNNs) have great potential for hyperspectral image classification. Supervised CNNs can efficiently extract hierarchical spatial-spectral features of hyperspectral images, but these methods face the problem of high time complexity as the number of network layers increases. Semi-supervised GNNs can rapidly capture the structural information of HSIs, while they cannot be well extended to hyperspectral image applications because of the process of adjacency matrix consuming large amount of memory resources. In this paper, we propose a fast dynamic graph convolutional network (dynamic GCN) and CNN parallel network (FDGC) for HSI classification. We first obtain two classification features by flattening and pooling operations on the results of the convolution layers, which fully exploits the spatial-spectral information contained in the hyperspectral data cube. Then a dynamic graph convolution module is applied to extract the intrinsic structural information of each patch. Finally, we can obtain the HSI classification results based on these spatial, spectral and structural features. By using three branches, FDGC can parallel process multiple features of HSI in a supervised learning manner. In addition, regularization techniques such as DropBlock and label smoothing are applied to further improve the generalization capability of the model. Experimental results on three datasets show that our proposed algorithm is comparable with the state-of-the-art supervised learning models in terms of accuracy while also significantly outperforming in terms of training and inference time.

Due to the lack of high-quality labeled data and poor generalization ability of supervised models in remote sensing scene classification, cross-domain scene classification is proposed to better utilize the existing knowledge to classify the unlabeled data. Since there is a data distribution difference between the training (source domain) and test (target domain) set, many deep domain adaptation methods have been proposed to reduce such distribution discrepancy. However, existing deep domain adaptation methods usually use the discrepancy metric function to align the marginal distribution and do not consider the effect of each sample in different domains on the network weights. In addition, the existing domain adaptation methods cannot adaptively balance the relative importance of marginal and conditional distributions well. To overcome the above shortcomings, we propose a novel Attention-based Dynamic Alignment and Dynamic Distribution Adaptation (ADA-DDA) method to better align the marginal distribution between different distributions by calculating the dynamic weights of each sample in different domains and dynamically balance the relative importance of marginal and conditional distributions. Moreover, the attention mechanism enables purposeful knowledge transfer, so that the extracted features can be highly discriminative. The experimental results demonstrate that our proposed method is superior to the other state-of-the-art deep domain adaptation methods in the comparison, and outperforms the second place in accuracy by 5.36%.

Hyperspectral image (HSI) anomaly detection is an important task in remote sensing domain. In recent years, many scholars have been addicted to constructing deep network-based methods for hyperspectral anomaly detection and have developed numerous related methods. Many of them are designed based on autoencoder, which aims to reconstruct a stable background to identify anomalies. However, these autoencoder-based methods suffer from some problems, such as ignoring the inter-band correlation in HSI. That is, the hyperspectral image presents spectral similarity as well as redundancy between the contiguous bands, which would affect the reconstruction of the HSI. Moreover, the current anomaly detectors lack the use of spatial contextual information that exists in the pixel neighbor region when constructing the detector. To tackle these problems, this study presents a spatial-spectral joint reconstruction with the inter-band correlation based anomaly detector (denoted as SSRICAD) for hyperspectral images. We first divide the original HSI into several sub-HSIs by a band cross-grouping strategy to reduce the redundancy and impose the inter-band correlation constraint into the reconstruction process. Then, an outlier removal constraint is added to alleviate anomaly contamination, which could help rebuild a more stable and pure background component. Finally, spatial information is extracted from the pixel neighbor region to contribute to the spatial-spectral joint reconstruction and further enhance detection performance. Extensive experiments on three benchmark hyperspectral datasets indicate that the proposed SSRICAD can achieve superior performance in anomaly detection.

Convolutional Neural Networks (CNN) and Graph Neural Networks (GNN), such as Graph Attention Networks (GAT), are two classic neural network models, which are applied to the processing of grid data and graph data respectively. They have achieved outstanding performance in hyperspectral images (HSIs) classification field, which have attracted great interest. However, CNN has been facing the problem of small samples and GNN has to pay a huge computational cost, which restrict the performance of the two models. In this paper, we propose Weighted Feature Fusion of Convolutional Neural Network and Graph Attention Network (WFCG) for HSI classification, by using the characteristics of superpixel-based GAT and pixel-based CNN, which proved to be complementary. We first establish GAT with the help of superpixel-based encoder and decoder modules. Then we combined the attention mechanism to construct CNN. Finally, the features are weighted fusion with the characteristics of two neural network models. Rigorous experiments on three real-world HSI data sets show WFCG can fully explore the high-dimensional feature of HSI, and obtain competitive results compared to other state-of-the art methods.

Feature distortions of data are a typical problem in remote sensing image classification, especially in the area of transfer learning. In addition, many transfer learning-based methods only focus on spectral information and fail to utilize spatial information of remote sensing images. To tackle these problems, we propose spectral-spatial weighted kernel manifold embedded distribution alignment (SSWK-MEDA) for remote sensing image classification. The proposed method applies a novel spatial information filter to effectively use similarity between nearby sample pixels and avoid the influence of nonsample pixels. Then, a complex kernel combining spatial kernel and spectral kernel with different weights is constructed to adaptively balance the relative importance of spectral and spatial information of the remote sensing image. Finally, we utilize the geometric structure of features in manifold space to solve the problem of feature distortions of remote sensing data in transfer learning scenarios. SSWK-MEDA provides a novel approach for the combination of transfer learning and remote sensing image characteristics. Extensive experiments have demonstrated that the proposed method is more effective than several state-of-the-art methods.

The sparse representation has been introduced for hyperspectral anomaly detection methods. However, the window parameter tuning and anomaly contamination problems are still the main issues with the background dictionary. In order to solve these problems, this paper proposed the joint sparse representation and multi-task learning method (JSM) for anomaly detection. This method utilizes a global background dictionary construction method to avoid the above window parameter tuning and anomaly contamination problems. Besides, the multi-task learning technology is employed to explore the hyperspectral images similarity within adjacent single-band images. Experiments were carried out on two hyperspectral images, and it was founded that JSM method shows a better detection performance than the other anomaly detection methods.

With the high spectral resolution, hyperspectral image (HSI) can provide a wealth of information for image classification. Many classification methods utilize the training samples to classify the ground materials. However, the small sample problem is still urgent to be solved when considering the cost of labeling training samples. In order to solve this problem, this letter proposes a semisupervised classification method based on the simple linear iterative cluster (SLIC) segmentation for HSI. This method improves the SLIC method to better explore the spectral characteristic of HSI. It explores the learned superpixel map and initial classification map to select the pseudo-labeled samples (PLSs), which is expected to increase the effectiveness of PLSs. The final classification map can be obtained with the integrated labeled training samples and PLSs. Experiments were carried out on three HSIs, and it was founded that the proposed method generally shows a better classification performance than the other methods.

In traditional anomaly detection algorithms, the background information is approximately described by whole hyperspectral imagery. However, the disparity between true and estimated background information would influence the performance of detection algorithms using background information. Considering this problem, a kernel background purification based anomaly target detection method is proposed in this paper. The main idea of the proposed method is to estimate background information more accurately. It contains two main steps: Firstly, the pure background pixel set extraction using the kernel-based method. Secondly, background covariance matrix estimation by extracted pure background pixel set. Experiments implemented on San Diego and PHI data indicate that the proposed method performed better than global Reed-Xiaoli detector (RXD), kernel RXD, and collaborative representation detector (CRD).

Hyperspectral image (HSI) contains a large number of spatial-spectral information, which will make the traditional classification methods face an enormous challenge to discriminate the types of land-cover. Feature learning is very effective to improve the classification performances. However, the current feature learning approaches are most based on a simple intrinsic structure. To represent the complex intrinsic spatial-spectral of HSI, a novel feature learning algorithm, termed discriminant spatial-spectral hypergraph learning (DSSHL), has been proposed on the basis of spatial-spectral information and hypergraph learning. DSSHL constructs an intraclass spatial-spectral hypergraph and an interclass spatial-spectral hypergraph to represent the intrinsic properties of HSI. Then, a feature learning model is designed to compact the intraclass information and separate the interclass information. DSSHL can effectively reveal the complex spatial-spectral structures of HSI for land-cover classification. Experimental results on the Salinas HSI data set shows that DSSHL can achieve better classification accuracies in comparison with some state-of-the-art methods.

Target detection from hyperspectral images is an important problem. Many target detection algorithms have been proposed and have been widely used in real applications during the past decades. However, the performance of these algorithms is highly susceptible to the quality of the target spectrum. This paper proposes a multi-priori learning algorithm to learning the inherent spectral similarity and difference between multiple priori target spectra, which can alleviate the target spectral variation by boosting the priori target spectra. Experiments on two hyperspectral images illustrated the effectiveness of the proposed algorithm.

The high-dimensional data space of hyperspectral images (HSIs) often result in ill-conditioned formulations, which finally leads to many of the high-dimensional feature spaces being empty and the useful data existing primarily in a subspace. To avoid these problems, we use distance metric learning for dimensionality reduction. The goal of distance metric learning is to incorporate abundant discriminative information by reducing the dimensionality of the data. Considering that global metric learning is not appropriate for all training samples, this paper proposes an ensemble discriminative local metric learning (EDLML) algorithm for HSI analysis. The EDLML algorithm learns robust local metrics from both the training samples and the relative neighborhood of them and considers the different local discriminative distance metrics by dealing with the data region by region. It aims to learn a subspace to keep all the samples in the same class are as near as possible, while those from different classes are separated. The learned local metrics are then used to build an ensemble metric. Experiments on a number of different hyperspectral data sets confirm the effectiveness of the proposed EDLML algorithm compared with that of the other dimension reduction methods.

Detecting certain targets from hyperspectral images (HSIs) is of great interest for both civilian and military applications, with the aim being to detect and identify target pixels based on specific spectral signatures. However, the classical algorithms are generally dependent on the specific statistical hypothesis test, and the algorithms may only perform well with certain assumptions. Therefore, in this paper, a novel metric-learning-based target detection framework, named local decision maximum margin metric learning (LDM3L), is proposed for HSI target detection. The proposed method can better separate the target samples from background ones, without the need for certain assumptions. The experimental results demonstrate that the proposed method outperforms both the state-of-the-art target detection algorithms and the other classical metric learning methods.