The joint semantic segmentation of multi-modal remote sensing images can make up for the problem of insufficient features of single-modal images and effectively improve the classification accuracy. Some deep learning methods have achieved good performance, but they face problems such as complex network structure, large number of parameters, and difficulty in deployment. In this paper, more attention is paid to front-end and branch-level feature transformation to obtain multi-scale semantic information. The multi-scale dilated extraction module (MDEM) is constructed to mine the specific features of different modalities. The multi-modal complementary attention module (MCAM) is designed for further acquiring prominent complementary content. The concatenated features are transmitted and reused by the dense convolution to complete the encoding. Ultimately, a general and concise end-to-end model is proposed. Comparative experiments are carried out on three heterogeneous datasets, and the model put forward performs well in qualitative analysis, quantitative comparison and visual effect. Meanwhile, the dexterity and practicability of the model are more prominent, which can provide support for lightweight design and hardware deployment.

Existing mainstream hyperspectral image (HSI) classification frameworks, like multilayer perceptron (MLP) networks, combine learnable linear transformations with nonlearnable nonlinear activation functions. While these feature extraction algorithms achieve high classification accuracy, they require significant computational and memory resources, and many models lack adequate informative representations. Recently, fast Kolmogorov-Arnold networks (FKANs) have emerged as a lightweight alternative to MLPs, offering competitive performance with fewer parameters and lower computational costs. This paper presents a hierarchical progressive fusion network (HPFN) based on FKAN to address these challenges. The proposed network explores HSI information more comprehensively through branches at three scale levels (pixel, patch, global) while remaining lightweight. The orthogonality of information at different levels is preserved to some extent for diverse practical applications. We develop two FKAN based lightweight convolutional networks to extract pixel-level and patch-level features. In the global-level branch, a boundary-enhanced U-Net with an edge-constrained module based on Sobel operators mitigates over-smoothing. The cross-scale guided fusion mechanism allows for dynamic awareness and accurate pairing of pixel-level features with local and global features, reducing information redundancy. Experimental results show that the proposed network achieves competitive classification accuracy on four public datasets at low computational cost, validating its effectiveness.

Object detection plays a crucial role in remote sensing due to the urgent demands of various applications, such as urban planning and environmental monitoring. Despite notable progress, current methods still struggle with detecting challenging small objects. At the object level, the limited pixel representation, blurred details, and background interference of small objects impose greater demands on feature extractors. At the network level, resource bias fails to provide adequate learning signals for these objects. In this paper, we propose a Sparse Transformer-based detector (STDet) to tackle these challenges. Specifically, we design a Local-to-Global Transformer network (LGFormer) to explore essential feature representations. The Local Transformer Block establishes correlations between tokens and their surrounding data, while the Global Transformer Block captures long-distance dependencies related to the objects. Meanwhile, we introduce a Scale-Balanced Label Assignment (SBLA) strategy that considers more samples to small objects. SBLA dynamically shifts the learning focus to easily overlooked objects and alleviates the issue of sample imbalance. Extensive experiments on three large-scale remote sensing datasets demonstrate the effectiveness of STDet and its superiority in small object detection.

Domain Generalization (DG) has led to remarkable achievements in cross-scene Hyperspectral Image (HSI) classification. Inspired by contrastive language-image pre-training (CLIP), the language-aware domain generalization method has been explored for cross-scene HSI classification with language prior knowledge. However, existing methods face some challenges: 1) The weak capacity to extract long-range contextual information and inter-class correlation. 2) Due to the inadequacies of the special pre-training on HSI data, the spatial-spectral features of HSI and linguistic features can not be straightforwardly alignment. To tackle those dilemmas, a novel network has been proposed with a CLIP framework, which consists of an image encoder, based on an encoder-only transformer to obtain the global contextual information and inter-class correlation, a frozen text encoder, and a cross-attention mechanism, named Linguistic-Interact-with-Visual Engager (LIVE), enhances the interaction between two modalities. Extensive experiments demonstrating superior performance over state-of-the-art methods with a CLIP framework, with 83.39% and 83.94% in OA, on the UH dataset and Pavia dataset, respectively.

Recently, remote sensing image object detection based on convolutional neural networks (CNNs) has made significant advancements. However, small objects detection remains a major challenge in this field. Because the small size of the object makes it difficult to extract their features and these features are further weakened after downsampling in the network. In order to improve the detection accuracy of small objects in remote sensing images, this letter provides a feature super-resolution fusion framework based on cross-scale distillation. Specifically, we design a sub-pixel super-resolution feature pyramid network (SSRFPN) replacing the bilinear interpolation with sub-pixel super-resolution (SSR) modules to enhance the feature expression capability. Furthermore, we propose a cross-scale distillation (CSD) mechanism to guide the SSR modules in learning the features of small object regions more accurately. Finally, our method is applied to three detectors on two datasets for validation. We adopt YOLOv7 as the baseline model and achieve the best results, with the average precision at a threshold of 0.5 (AP0.5) of 95.0% and 82.3% on the NWPU VHR-10 dateset and DIOR dataset. And the mean average precision of small objects (mAPS) is improved by 8.5% and 2.5%.

Deep learning has made significant achievements in remote sensing object detection tasks. However, small weak objects located in complex scenes are still not effectively addressed. The lack of feature information and negligible contributions during the optimization stage are the main reasons. To solve the indicated issues, a novel remote-sensing object detection method is proposed in this letter. Firstly, the context feature integration module (CFIM) is designed to extract implicit clues co-occurring with the object to compensate for the lack of features in small weak objects. The receptive field expansive deformable convolution (RFConv) constructed in CFIM can adaptively adjust the information extraction range based on the object’s characteristics, thereby capturing suitable context features. Secondly, to make small weak objects competitive during the optimization process, we propose tailored optimization functions: the balanced sampling strategy (BSS) and the modulated loss function (MLF). BSS makes up for the sample deficiency of small weak objects by balancing sampling. More specifically, BSS dynamically mines potential positive samples from the ignored set in order to increase the chances of matching. MLF progressively adjusts the loss proportion of the object and pushes the detector to be more sensitive to small weak objects during the training stage. Our proposed method achieves 94.79% and 71.23% mAP on the NWPU VHR-10 and DIOR datasets, which also proves the effectiveness.

Hyperspectral image super-resolution technology has made remarkable progress due to the development of deep learning. However, the technique still faces two challenges, i.e., the imbalance between spectral and spatial information extraction, and the parameter deviation and high computational effort associated with 3D convolution. In this article, we propose a super-resolution method for remote sensing hyperspectral images based on multi-domain spatial information and multi-scale spectral information fusion (MSSR). Specifically, inspired by the high degree of self-similarity of remote sensing hyperspectral images, a spatial-spectral attention module based on dilated convolution (DSSA) for capturing global spatial information is proposed. The extraction of local spatial information is then accomplished by residual blocks using small-size convolution kernels. Meanwhile, we propose the 3D Inception module to efficiently mine multi-scale spectral information. The module only retains the scale of the 3D convolution kernel in spectral dimension, which greatly reduces the high computational cost caused by 3D convolution. Comparative experimental results on four benchmark datasets demonstrate that compared with the current cutting-edge models, our method achieves state-of-the-art results and the model computation is greatly reduced.

Graph convolutional networks (GCNs) can extract features of samples in non-Euclidean space, which can be used for hyperspectral image (HSI) classification in collaboration with convolutional neural networks (CNNs). The features of GCNs and CNNs are incompatible to a certain extent, and traditional graph convolution methods use a single channel two-dimensional matrix to extract features. As a result, it is difficult to explore the relationships fully and flexibly between samples. To further exploit the potential of these two networks for collaborative extraction of HSI features, we propose a fusion framework based on multichannel GCNs and class-guided attention mechanism (MG2A). Specifically, a multichannel graph convolutional network (MGCN) module is designed for batchwise network training, where the adjacent matrix of each channel contains different information between samples. In addition, we develop a class-guided attention mechanism to adaptively fuse the features of multiple MGCN modules and learn the transformation process of the features. Finally, the features of CNNs and MGCN modules are fused at multiple layers through a fusion framework. Experimental results on four benchmark HSI datasets show that MG2A achieves better classification performance compared to other state-of-the-art methods.

In remote sensing, it is desirable to improve image resolution by using the image super-resolution (SR) technique. However, there are two challenges: the first one is that high-resolution (HR) images are insufficient or unavailable; another one is that the single degradation model such as bicubic (BIC) cannot super-resolve favorable images in the real world. To address the above two problems, this article presents a multi-degradation, unsupervised SR method based on deep learning. This framework consists of a degrader$ {D}$to fit the image degradation model and a generator$ {G}$to generate SR image. By introducing$ {D}$, calculating the loss function between SR image and HR image as supervised SR methods did can be converted into calculating loss between low resolution (LR) image and image degraded by SR image, thereby realizing unsupervised learning. Experiments on several degradation models show that our method renders the state-of-the-art results compared with existing unsupervised SR methods, and achieves competitive results in contrast with supervised SR methods. Moreover, for real remote sensing images obtained by the Jilin-1 satellite, our method obtained more plausible results visually, which demonstrate the potential in real-world applications.

The development of deep learning has brought new prospects into the field of hyperspectral classification, and the classification ability of this method for the classification of hyperspectral images (HSIs) has been continuously improved. However, there are still some problems that must be solved; for example, the spectral–spatial features of HSIs are not effectively extracted, the labeled samples in the data set are limited, and the number of samples in different categories is imbalanced. To facilitate the progress of hyperspectral classification, a spectral–spatial fractal residual convolutional neural network with data balance augmentation is proposed here. In this network, a data balance augmentation approach that can solve the problems of limited labeled data and imbalanced categories is proposed. In addition, the spectral–spatial residual module is proposed to learn the spectral–spatial information and alleviate the problem of model degradation effectively. In addition, the spectral–spatial focal structure, which can guarantee the integrity of the information, is introduced. Moreover, the spectral–spatial dimensional transformation module, which can reduce the size and number of hyperspectral feature maps without losing the fine features, is presented. In particular, the proposed network has a strong ability to classify the categories that have a small number of samples and reaches the state-of-the-art level for three benchmark data sets.