Hyperspectral images (HSIs) typically have finer spectral resolution but coarser spatial resolution than multispectral images (MSIs). To obtain HSIs with enhanced spatial resolution, considerable emphasis has been placed on achieving hyperspectral super-resolution (SR) by fusing HSIs with MSIs in the same scene. However, most existing HSI-MSI fusion methods either rely on prior knowledge of degradation models or require sufficient training data, hindering their practicality and interpretability. This letter proposes a deep multistream network (DMSN) for HSI SR. Specifically, we introduce the Spa-DNet and the Spe-UNet modules to encode spatial and spectral transformations across resolutions. Furthermore, we design the Int-Net to achieve spatial and spectral information interaction, enhancing the model’s performance. Finally, the proposed approach enables high spatial and spectral resolution HSIs. Using the newly designed three-stage training strategy, the network parameters can exhibit the clear physical significance of the degradation process, thereby helping to ensure faithful reconstruction of the desired HSIs. Experimental results with real datasets demonstrate that the proposed DMSN performs better than other methods. The codes will be available athttps://github.com/yuanchaosu/dmsn-GRSL.

Most existing methods for predicting drug-drug interactions (DDI) predominantly concentrate on capturing the explicit relationships among drugs, overlooking the valuable implicit correlations present between drug pairs (DPs), which leads to weak predictions. To address this issue, this paper introduces a hierarchical multi-relational graph representation learning (HMGRL) approach. Within the framework of HMGRL, we leverage a wealth of drug-related heterogeneous data sources to construct heterogeneous graphs, where nodes represent drugs and edges denote clear and various associations. The relational graph convolutional network (RGCN) is employed to capture diverse explicit relationships between drugs from these heterogeneous graphs. Additionally, a multi-view differentiable spectral clustering (MVDSC) module is developed to capture multiple valuable implicit correlations between DPs. Within the MVDSC, we utilize multiple DP features to construct graphs, where nodes represent DPs and edges denote different implicit correlations. Subsequently, multiple DP representations are generated through graph cutting, each emphasizing distinct implicit correlations. The graph-cutting strategy enables our HMGRL to identify strongly connected communities of graphs, thereby reducing the fusion of irrelevant features. By combining every representation view of a DP, we create high-level DP representations for predicting DDIs. Two genuine datasets spanning three distinct tasks are adopted to gauge the efficacy of our HMGRL. Experimental outcomes unequivocally indicate that HMGRL surpasses several leading-edge methods in performance.

Recently, rumor detection (fake news detection) has seen a surge in research interest, and fact verification (fake news checking) has simultaneously become a significant research aspect. Despite the inherent distinction between fact verification and rumor detection – the former being a three-category task and the latter a binary one – there has yet to be in-depth exploration into the synergies between these two tasks. Furthermore, given the severe scarcity and the time-consuming and costly construction nature of fact verification datasets, few-shot/zero-shot fact verification methods are particularly favored. To tackle these challenges, we conduct a series of studies around “How can rumor detection enhance few-shot fact verification, and to what extent?”. Specifically, we systematically investigate the knowledge transferability between the two tasks, proposing a framework, Det2Ver, that is applicable to both rumor detection and fact verification. Through the construction of adaptive prompt templates and prompt-tuned LLMs like T5, Det2Ver structural-level synchronizes the two tasks and utilizes the external knowledge from rumor detection to reinforce fact verification task. We demonstrate the significance and effectiveness of Det2Ver. Through the few-shot/zero-shot experiments on three widely-used datasets, compared to other LLMs prompt-tuning baselines, the Det2Ver for cross-task knowledge augmentation brings a significant improvement in macro-F1 for fact verification.

Multimodal fusion widely uses convolutional layers to capture local correlations and adjust feature dimensions. However, the progressive expansion of the receptive field in convolutional layers often compromises spatial context retention, leading to the loss of fine details. Furthermore, the fixed-size kernels typically used in standard convolution restrict the network’s ability to capture multiscale contextual details. To address this limitation, this paper develops a dilated transformation-guided unsupervised multimodal learning (DTUML) method to fuse a high-resolution multispectral image (HR-MSI) and a low-resolution hyperspectral image (LR-HSI), thereby generating a high-resolution hyperspectral image (HR-HSI). Our DTUML adopts a dual-stream encoder architecture to conduct multimodal data, where one stream focuses on preserving spectral information from LR-HSIs, while the other emphasizes the acquisition of spatial details from HR-MSIs. These complementary features are subsequently integrated to ensure spectral fidelity and retain spatial detail. Then, a convolutional layer restores dimensional consistency and outputs an HR-HSI. Extensive experiments demonstrate the effectiveness of DTUML, showing superior performance and strong competitiveness compared to state-of-the-art methods. Code:https://github.com/yuanchaosu/TGRS-DTUML.

Vision transformer (ViT) has recently been a popular topic in the foundation model field, taking advantage of its strong scalability and outstanding representation capabilities. As a deep model, ViT introduces a new architecture for achieving hyperspectral image (HSI) unmixing. However, traditional ViTs overlook pixel-level spatial continuity by partitioning the input image into nonoverlapping fixed-size patches. This approach disrupts local structural relationships and hinders the model's ability to capture fine-grained spatial dependencies, resulting in suboptimal feature representation for dense prediction tasks in unmixing. To address these challenges, this article proposes the development of a self-supervised relation-aware ViT (SRViT). SRViT incorporates a self-embedded module comprising encoders, a pixel-level position encoder (PLPE), a self-supervised contrastive mechanism (SCM), and a decoder. The self-embedded module and PLPE preserve local correlations in HSI across different views, facilitating cross-view learning through SCM to ensure generalization. In addition, the decoder incorporates Kronecker-factored approximate curvature (K-FAC) to capture the local geometric structure of spectral information. Ultimately, SRViT learns endmembers and fractional abundance as the unmixing result. The effectiveness and competitiveness of SRViT have been systematically validated through comparative experiments, demonstrating its superior performance. The source code is available at the following link: https://github.com/yuanchaosu/TNNLS-SRViT.

In recent years, deep learning has been proposed for hyperspectral images(HSIs) super-resolution, and many fusion models for HSI and multispectral images(MSIs) have been developed. However, these networks are constrained to the structure of convolutional neural networks(CNNs), and more attention needs to be paid to the disadvantage of the restricted receptive field of CNNs, such that some of the distal information needs to be included in the process of acquiring features. This approach involves capturing large-scale spatial features through multi-head spatial attention and spectral features of MSI and HSI through multi-head spectral attention. Subsequently, the features are processed by convolution kernels of different scales compactly. The effectiveness and competitiveness of MTSANet are evaluated by comparing it with some state-of-the-art (SOTA) methods.

Transformer holds significance in deep learning (DL) research. Node embedding (NE) and positional encoding (PE) are usually two indispensable components in a Transformer. The former can excavate hidden correlations from the data, while the latter can store locational relationships between nodes. Recently, the Transformer has been applied for hyperspectral image (HSI) classification because the model can capture long-range dependencies to aggregate global features for representation learning. In an HSI, adjacent pixels tend to be homogeneous, while the NE does not identify the positional information of pixels. Therefore, PE is crucial for Transformers to understand locational relationships between pixels. However, in this area, most Transformer-based methods randomly generate PEs without considering their physical meaning, which leads to weak representations. This article proposes a new graph generative structure-aware Transformer (GraphGST) to solve the above-mentioned PE problem when implementing HSI classification. In our GraphGST, a new absolute PE (APE) is established to acquire pixels’ absolute positional sequences (APSs) and is integrated into the Transformer architecture. Moreover, a generative mechanism with self-supervised learning is developed to achieve cross-view contrastive learning (CL), aiming to enhance the representation learning of the Transformer. The proposed GraphGST model can capture local-to-global correlations, and the extracted APSs can complement the spectral features of pixels to assist in NE. Several experiments with real HSIs are conducted to evaluate the effectiveness of our GraphGST. The proposed method demonstrates very competitive performance compared with other state-of-the-art (SOTA) approaches. Our source codes will be provided in the following linkhttps://github.com/yuanchaosu/TGRS-graphGST.

Spatial-spectral classification (SSC) has become a trend for hyperspectral image (HSI) classification. However, most SSC methods mainly consider local information, so that some correlations may not be effectively discovered when they appear in regions that are not contiguous. Although many SSC methods can acquire spatial-contextual characteristics via spatial filtering, they lack the ability to consider correlations in non-Euclidean spaces. To address the aforementioned issues, we develop a new semisupervised HSI classification approach based on normalized spectral clustering with kernel-based learning (NSCKL), which can aggregate local-to-global correlations to achieve a distinguishable embedding to improve HSI classification performance. In this work, we propose a normalized spectral clustering (NSC) scheme that can learn new features under a manifold assumption. Specifically, we first design a kernel-based iterative filter (KIF) to establish vertices of the undirected graph, aiming to assign initial connections to the nodes associated with pixels. The NSC first gathers local correlations in the Euclidean space and then captures global correlations in the manifold. Even though homogeneous pixels are distributed in noncontiguous regions, our NSC can still aggregate correlations to generate new (clustered) features. Finally, the clustered features and a kernel-based extreme learning machine (KELM) are employed to achieve the semisupervised classification. The effectiveness of our NSCKL is evaluated by using several HSIs. When compared with other state-of-the-art (SOTA) classification approaches, our newly proposed NSCKL demonstrates very competitive performance. The codes will be available at https://github.com/yuanchaosu/TCYB-nsckl.

Deep learning (DL) has brought many new trends for hyperspectral image classification (HIC). Graph neural networks (GNNs) are models that fuse DL and structured data. Although GNN-based methods have focused on modeling relations, most of them are susceptible to noise, being adverse to capturing hidden correlations from data. Moreover, existing related approaches typically adopt changeless graph structures, which might lead to poor generalization. To solve the problems mentioned above, this paper develops an adaptive cuckoo refinement-based graph transfer network (ACGT-Net) that introduces a meta-heuristic optimization strategy to refine the graph structure. Specifically, we first pre-train a graph convolutional network (GCN) to learn transferable weight parameters. In the undirected graph, nodes are associated with pixels, and edges correspond to similarities between nodes. Afterward, we integrate a cuckoo search strategy (CSS) into the trained GCN to adaptively refine the graph structure. The graph structure refinement (GSR) with the CSS can pay more attention to significant channels by global optimization to improve the generalization of the GNN. Several experiments with real datasets verify the effectiveness and competitiveness of our ACGT-Net compared with other state-of-the-art (SOTA) methods.

This work proposes a new hyperspectral image classification method based on chaotic cuckoos (CC) search with graph convolution network (CC-GCN). Although GCNs can extract inherent features by node embeddings, most models neglect fragmented relations in the spectral domain. The proposed CC-GCN can refine the graph structure and reduce the redundancy information of spectral dimension, further improving the classification accuracy of hyperspectral images. The experiment results demonstrate the effectiveness of the proposed CC-GCN.

Dimensionality reduction (DR) is a common preprocessing technology for hyperspectral images (HSIs). Recently, many neural networks can implement DR to remove the re-dundant information by node embedding. However, numer-ous hidden-layer parameters limit the generalization ability of the node embedding. In this paper, we develop a graph-cut-based node embedding (GCNE) that can be used for DR of HSIs. The embedding can refine correlations by a graph-cut strategy, and it can avoid numerous parameters when using graph models. Moreover, we combine the graph-cut strategy and extreme learning machine (ELM) to achieve HSI classi-fication. The effectiveness of the proposed method is verified by using real HSIs. Compared with other state-of-the-art DR and classification methods, the proposed approach demon-strates very competitive performance.

Node embedding (NE) is conducive to aggregating correlations and relieving the influence of the Hughes phenomenon when processing high-dimensional data. Although some graph neural networks can capture correlations during achieving NE, the application of NE still faces two rigorous challenges: numerous model parameters and poor generalization. In this letter, we propose a new approach for hyperspectral image (HSI) classification, called the graph-cut-based collaborative NEs (GCCNE). Specifically, we develop a graph-cut-based NE (GCNE) to achieve low-dimensional feature representation, which avoids numerous model parameters when using a graph structure. Considering that the graph-cut in a low-dimensional space does not need to set anchors to decrease the calculation amount, we adopt an ensemble framework based on random subspaces (RSs) to implement the GCNE to obtain the collaborative feature sets, enhancing the generalization of feature representation. Afterward, the collaborative feature sets are input in several kernel-based extreme learning machines (KELMs), respectively, classifying pixels. The number of RSs is the same as the number of KELMs. Finally, we acquire an ensemble result associated with each class. The effectiveness and competitiveness of the proposed method are evaluated by using real HSI datasets.

In this letter, we propose a new hyperspectral image (HSI) classification approach, called the random subspace ensemble with enhanced feature (RSE-EF), which trains several individual classifiers with enhanced spatial information. The proposed approach aims to address two common issues: the curses of the imbalanced training samples and high feature-to-instance ratio. Specifically, we first propose a similar-neighboring-sample-search (SNSS) method to address the issue of imbalanced training samples. Afterward, we generate the enhanced random subspaces (ERSs) that possess relatively lower dimensionality and more distinctive information compared with the original random subspaces (RSs) so as to alleviate the curse of high feature-to-instance ratio more effectively. Furthermore, a shallow neural network kernel-based extreme learning machine (KELM) is applied to the RSE-EF to classify image pixels. Experimental results on two public hyperspectral data sets illustrate that the proposed RSE-EF approach outperforms the state-of-the-art HSI classification counterparts.