Due to the limitation of sensors, hyperspectral images contain a large number of mixed pixels. Hyperspectral unmixing techniques decompose these mixed pixels into distinct endmembers and their corresponding abundance values. Traditional methods initialize weights in the decoder and utilize outputs/weights as abundance maps and endmembers——an approach heavily dependent on initial weights that significantly limits performance. This paper proposes a blind hyperspectral unmixing method integrating Recurrent Kolmogorov-Arnold Networks (KAN) with Segment Anything Model (SAM) adapter. The method operates through three sequential stages: feature encoding, endmember extraction, and abundance estimation. Specifically for feature encoding, a HU-SAM adapter is proposed to capture global-local spatial features. For endmember extraction, an iteratively learned Recurrent-KAN module reconstructs endmembers while stabilizing model learning. For abundance estimation, an updated Swin Transformer module is utilized to maintain a lower parameter count. Extensive experiments on real and synthetic datasets demonstrate superior effectiveness of the proposed method over the eight state-of-the-art methods.

With the success of ViT (Vision Transformer), Transformer is being increasingly used for hyperspectral image (HSI) classification given its ability to extract global context dependencies. However, existing methods based on transformers tend to classify HSI in the traditional patch-wise manner. Thus, these methods cannot obtain true global features because the inputs of the model are local patches. To solve these problems, a hybrid convolution and ViT network (HCVN) is proposed for HSI classification. HCVN realizes the classification task from the perspective of semantic segmentation, and its input is the entire HSI, which makes it possible to obtain truly meaningful global features. By improving the original ViT, an HCV module is proposed, which enhances the ability of local structure characterization while extracting global features. The HCVN hybrid convolution layer and HCV module realize the extraction and fusion of local and global features. Finally, the dual branch network architecture is used to integrate the spatial and spectral features. Extensive experiments on two datasets verify the effectiveness of the proposed method.

Transfer learning is an effective way to alleviate the problem of insufficient samples in a hyperspectral image (HSI) classification. However, the present transfer learning-based methods usually transfer knowledge from a single source domain, such as the natural image domain. Therefore, these methods cannot simultaneously transfer spectral and spatial knowledge to the target domain in HSIs. Generally, the natural image has rich spatial structure and texture information, while the HSI has abundant spectral information. To better utilize the knowledge learned from natural image datasets and HSI datasets, we proposed a multimodal transfer feature fusion network (MTFFN) for HSI classification. In MTFFN, a dual-branch network structure is designed to transfer the two-modal knowledge from the natural image domain and the source HSI domain to the target domain in two branches, respectively. A multitask learning strategy is adopted to achieve feature fusion. The fused features are used to generate the final classification result. Moreover, a local attention mechanism is designed to extract more meaningful spectral features. Experiments on two public datasets show that the proposed method is effective (https://github.com/HuaipYan/MTFFN).

To address the limitation of the high-dimensionality features and single spatial scale in the spectral–spatial classification of hyperspectral image (HSI), we propose a random multiscale convolutional network (RMCNet) that combines a multiscale dimensionality reduction module (MDRM) and the RMCNet for improving classification accuracy. The MDRM is based on multiscale superpixel segmentations, which implements dimensionality reduction leading to relieve the Hughes problem and reduce the computation burden in deep learning. Then, the multiscale spectral–spatial features are extracted by the RMCNet to adaptive various complex scenes in HSI. Finally, the multiscale spectral–spatial features act as inputs of support vector machine for classification. In the experiments, three benchmark HSIs are used to evaluate the performance of the proposed method. The experimental results demonstrate that the RMCNet can yield a competitive performance compared with the state-of-the-art methods.

Pan sharpening is used to fuse a low-resolution multispectral (MS) image and a high-resolution panchromatic (PAN) image to obtain a high-resolution MS image. This article proposes PSMD-Net, an end-to-end pan-sharpening method based on a multi-scale dense network. A shallow feature extraction layer (SFEL) extracts the shallow features from the original images, and these are used as an input to a global dense feature fusion (GDFF) network to learn the global features for image reconstruction. A multiscale dense block (MDB) is designed to fully extract the spatial and spectral information from the shallow features in the GDFF network. In the proposed network, multiple MDBs are stacked to extract rich, multi-scale dense hierarchical features, and a global dense connection (GDC) is designed to allow direct connections from the state of the current MDB to all subsequent MDBs to extract more advanced features. The extracted hierarchical features are sent to the global feature fusion layer (GFFL) to adaptively learn the global features for image reconstruction. Finally, global residual learning (GRL) is adopted to force the network to pay more attention to the changing part of the image. We perform experiments on simulated and real data from WorldView-2 and WorldView-3 satellites. Visual and quantitative assessment results demonstrate that PSMD-Net yields higher-resolution fusion images than the state-of-the-art methods.

The image fusion method based on sparse representation in the single‐scale image domain has produced better fusion results than the classic methods based on multi‐scale analysis nowadays. However, due to the limited number of dictionary atoms, it is difficult to provide an accurate description for image details in the sparse‐representation‐based image fusion methods, and it requires a lot of time. A novel dictionary is constructed with non‐subsampled contourlet transform and sparse representation by using the proposed simultaneous strategy. Then the novel dictionary could combine the sparsity attribute of the learning dictionary with a multi‐scale feature of non‐subsampled contourlet transform. Moreover, the simultaneous strategy is combined with this novel dictionary so that sparse coefficients can be represented with the same dictionary atoms and thus they can be compared in a reasonable and accurate way. Finally, the image fusion method along with this novel dictionary is proposed and named non‐subsampled contourlet transform (NSCT)–simultaneous sparse representation (SSR). Experimental results show that the proposed fusion method NSCT–SSR, with its more excellent fusion effect and better anti‐noise capability, outperforms the existing fusion methods, which are based on both multi‐scale domain and sparse representation in the single‐scale image domain.

Abstract Fine‐grained image classification is a challenging topic in the field of computer vision. General models based on first‐order local features cannot achieve acceptable performance because the features are not so efficient in capturing fine‐grained difference. A bilinear convolutional neural network (CNN) model exhibits that a second‐order statistical feature is more efficient in capturing fine‐grained difference than a first‐order local feature. However, this framework only considers the extraction of a second‐order feature descriptor, using a single convolutional layer. The potential effective classification features of other convolutional layers are ignored, resulting in loss of recognition accuracy. In this paper, a multilayer feature descriptors fusion CNN model is proposed. It fully considers the second‐order feature descriptors and the first‐order local feature descriptor generated by different layers. Experimental verification was carried out on fine‐grained classification benchmark data sets, CUB‐200‐2011, Stanford Cars, and FGVC‐aircraft. Compared with the bilinear CNN model, the proposed method has improved accuracy by 0.8%, 1.1%, and 5.5%. Compared with the compact bilinear pooling model, there is an accuracy increase of 0.64%, 1.63%, and 1.45%, respectively. In addition, the proposed model effectively uses multiple 1×1 convolution kernels to reduce dimension. The experimental results show that the multilayer low‐dimensional second‐order feature descriptors fusion model has comparable recognition accuracy of the original model.

For remote sensing image fusion, infrared and visible images have very different brightness due to their disparate imaging mechanisms, the result of which is that nontarget regions in the infrared image often affect the fusion of details in the visible image. This letter proposes a novel infrared and visible image fusion method basing hybrid representation learning by combining dictionary-learning-based joint sparse representation (JSR) and nonnegative sparse representation (NNSR). In the proposed method, different fusion strategies are adopted, respectively, for the mean image, which represents the primary energy information, and for the deaveraged image, which contains important detail features. Since the deaveraged image contains a large amount of high-frequency details information of the source image, JSR is utilized to sparsely and accurately extract the common and innovation features of the deaveraged image, thus, accurately merging high-frequency details in the deaveraged image. Then, the mean image represents low-frequency and overview features of the source image, according to NNSR, mean image is classified well-directed to different feature regions and then fused, respectively. Such proposed method, on the one hand, can eliminate the impact on fusion result suffering from very different brightness causing by different imaging mechanism between infrared and visible image; on the other hand, it can improve the readability and accuracy of the result fusion image. Experimental result shows that, compared with the classical and state-of-the-art fusion methods, the proposed method not only can accurately integrate the infrared target but also has rich background details of the visible image, and the fusion effect is superior.