Prompt tuning has emerged as a powerful approach in the era for foundation models, enabling efficient use of pretrained knowledge while minimizing resource demands. We introduce prompt tuning to the field of remote sensing (RS) and propose a novel prompt-based multimodal fusion framework called ProMF for RS multimodal image classification. ProMF incorporates a small number of learnable parameters into the input space, while keeping the parameters of pretrained networks frozen during model fine tuning. These additional parameters are prepended to the input sequence of each Transformer layer and trained alongside the linear classification head during fine-tuning. Furthermore, to enhance the feature interaction and fusion, we hierarchically incorporate useful prompts through a novel prompt-embedded multihead self-attention (MSA) mechanism. This approach allows for the learning of complementary representations from different modalities layer by layer, improving the model performance while reducing the risk of overfitting. Experimental results on three commonly used datasets demonstrate that the proposed method outperforms state-of-the-art approaches. The demonstrated effectiveness of multimodal prompt tuning offers a new perspective on adapting pretrained models for RS applications. The code will be publicly available at:https://github.com/zhaolin6/ProMF

Hyperspectral images (HSI) offer exceptional precision in distinguishing features due to their broad spectral dimensions. However, their high dimensionality gives rise to a phenomenon known as the ‘dimensional curse’, characterized by data sparsity in high-dimensional feature spaces. This issue is further exacerbated by the limited number of labeled samples, rendering it challenging to effectively define the decision boundary and increasing risk of over-fitting. To address the challenges, we propose a spectral-spatial representation learning framework based on hyperspectral image and light detection and ranging (LiDAR) data, which enhances the generalization of spectral features while reducing dimensionality through the optimization of spectral-wise information. Meanwhile, a local-global spatial feature fusion mechanism is designed for LiDAR spatial features to further alleviating the sparsity of spectral features and to effectively recognize complex land cover. The method fully leverages the complementary strengths of HSI and LiDAR data through self-supervised contrastive learning, effectively mitigates the challenge posed by data properties. Extensive experiments were conducted on three widely used HSI-LiDAR datasets, and the results demonstrate that the proposed algorithm outperforms state-of-art methods in classification accuracy.

3D Gaussian Splatting (3DGS) has gained significant attention for its exceptional performance in real-time rendering and novel view synthesis. However, the traditional Gaussian densification method struggles to effectively capture the complexity of regions with insufficient point cloud density. Although this method improves overall rendering quality by expanding the point cloud to millions of points, blurring and distortion issues still persist in edge details and high-intensity lighting regions. To address these limitations, this paper proposes Gaussian-Enhanced Detail Reconstruction (GEDR), which enhances 3DGS with two key innovations: (1) Multi-scale adaptive Gaussian kernels, dynamically adjusted based on geometric features such as gradient and curvature, enabling finer reconstruction in high-detail regions while maintaining efficiency. (2) Opacity optimization leveraging illumination information, reducing artifacts caused by ambient lighting variations and ensuring stable rendering in large-scale scenes. This strategy ensures efficient and stable rendering, even in large-scale scenes. Evaluations on Mip-NeRF 360 and Tanks & Temples datasets demonstrate that GEDR significantly improves detail preservation, complex region restoration, and robustness to lighting changes while maintaining controlled storage overhead. These results highlight GEDR’s advantages over traditional 3DGS in high-fidelity scene reconstruction.

Detecting small objects in remote sensing images is a significant challenge due to their weak texture, scale variations, and dense spatial arrangements. Existing approaches often overlook the importance of prior contextual information and the aggregation of features in densely packed small objects, both of which are crucial for improving the performance of remote sensing small object detection (RSSOD). In this work, we propose the Prior Guided Context Fusion Network (PGCFNet), which enhances small object detection by decoupling scene contextual information through three novel components: the Prior Guided Context Fusion Module (PGCFM), the DepthWise Aggregator (DWA), and the Prior Guided Small Object Detector (PGSOD). This architecture facilitates a deeper exploration of the relationships between small objects and their surrounding environment. Specifically, PGCFM improves feature representation by integrating multi-scale features and applying prior-guided dynamic channel weighting, addressing the challenge of weak textures. Additionally, DWA refines feature aggregation using dilated convolutions and dynamic feature adjustment, enabling precise multi-scale detection in environments with dense small objects. Furthermore, PGSOD leverages prior knowledge to reduce background interference, enhancing small object detection across varying scales and orientations. Collectively, these modules work synergistically to advance small object detection in remote sensing images, overcoming key challenges in complex environments. Extensive experiments on three public datasets demonstrate that the performance of the proposed method outperforms several state-of-the-art detectors, especially for tiny object detection. Specifically, PGCFNet achieves 86.0% mAP on the DIOR dataset, 95.59% mAP on the NWPU VHR-10 dataset, and 58.5% mAP on the AI-TOD dataset. Additionally, we conducted generalization experiments for PGCFM, DWA and PGSOD, demonstrating its effectiveness across different datasets and detection networks with varying model sizes.

As an effective unsupervised learning method, contrastive learning (CL) has made remarkable progress in the hyperspectral image (HSI) classification. The core idea of CL is to learn representations by attracting positive samples and repelling negative samples. However, due to the patch sampling mode of HSI, the patches with the same semantic information might be undesirably considered as negative samples of each other, which are called “noisy negatives.” The noisy negatives deteriorate the performance of CL. To address the issue, a progressive CL based on noisy negatives cleaning (ProCoL) is proposed for HSI classification. In contrast to existing CL, an adjunct low-dimensional subspace is introduced. Additionally, encoder training was conceptually divided into two stages each with distinct roles. In the rough-training stage, CL is applied concurrently within two different dimensional subspaces to improve the discriminative ability of the encoder. Subsequently, as the training stabilizes, noisy negatives are gradually eliminated in the retraining stage based on the dynamically generated pseudo labels in the low-dimensional space, which further improves the latent representations of the encoder. Experiments show that the ProCoL achieves the best performance compared to the previous state-of-the-art methods.

Contrastive learning has emerged as a promising technique for hyperspectral image (HSI) classification. However, the inherent limitation of sliding window sampling in HSI results in partial samples within a mini-batch exhibiting extremely high similarity. Consequently, there is an increased number of negative sample pairs composed of similar samples, significantly reducing the effectiveness of contrastive learning. Moreover, prevailing classification models heavily depend on convolutional operations, emphasizing the extraction of local features but struggle to capture long-distance dependencies in both spatial and spectral dimensions. To address these problems and fully leverage the abundance of unlabeled samples, we propose a novel purified contrastive learning (PCL) framework for HSI classification. We design a complementary spatial-spectral representation encoder architecture that combines Convolutional Neural Network (CNN) and Transformer to capture local features and global dependencies. More importantly, a purified contrastive loss function is proposed based on super-pixel spatial prior. Extensive experiments on three public datasets demonstrate the superiority of PCL over state-of-the-art methods in HSI classification. The code for this work is available at https://github.com/zhaolin6/PCL for the sake of reproducibility.

Deep learning (DL) methods have achieved impressive performance in hyperspectral image (HSI) classification but are susceptible to adversarial attacks, which can lead to significant accuracy degradation. Although there has been encouraging progress in improving model robustness for HSI classification, the existing approaches primarily concentrate on capturing global pixel-level dependencies while overlooking the intrinsic superpixel priors of HSI—namely, spatial smoothness and spectral correlations within spatially coherent regions. In addition, these methods often lack effective noise suppression mechanisms. To address these challenges, we introduce a novel antiperturbation network, APNet, designed for robust HSI classification against adversarial attacks. APNet incorporates a noise suppression module that includes a U-shaped unit (UsU) and a feature cascade unit (FCU) to extract clear pixel-level features at multiple scales. These features are then combined with superpixel priors, which serve as robust tokens for a Transformer encoder to capture global structural features. By fusing denoised pixel-level features with coarse-grained superpixel-level features, APNet significantly enhances the robustness of feature representations and the network’s intrinsic resistance to adversarial attacks. Extensive experiments on three HSI benchmark datasets show that APNet outperforms the existing state-of-the-art techniques against across various attacks and perturbation intensities. In particular, APNet maintains stable classification performance even under high attack intensities.

The transformer framework has shown great potential in the field of hyperspectral image (HSI) classification due to its superior global modeling capabilities compared to convolutional neural networks (CNNs). To utilize the transformer to model spatial–spectral information, a hybrid transformer that integrates multigranularity tokens and spatial–spectral attention (SSA) is proposed. Specifically, a token generator is designed to embed the multigranularity semantic tokens, which contributes richer image features to the model by exploiting CNN’s local representation capability. Moreover, a transformer encoder with an SSA mechanism is proposed to capture the global dependencies between different tokens, enabling the model to focus on more differentiated channels and spatial locations to improve the classification accuracy. Ultimately, adaptive weighted fusion is applied to different granularity transformer branches to boost HybridFormer’s classification performance. Experiments were conducted on four new challenging datasets, and the results indicate that HybridFormer achieves state-of-the-art results in terms of classification performance. The code of this work will be available athttps://github.com/zhaolin6/HybridFormerfor the sake of reproducibility.

Visual object tracking plays a fundamental role in computer vision. Extracting the unique spectral and spatial features of hyperspectral images (HSIs) can significantly improve tracking performance in complex scenarios, especially in hyperspectral object tracking. However, due to the limited training samples, handcrafted features are employed in most current hyperspectral trackers, although they cannot sufficiently describe the intrinsic nature of the object. This letter proposes a band regrouping and response-level fusion network (BRRF-Net) for hyperspectral object tracking based on deep transfer learning, employing a deep model trained on color videos to represent features to solve this problem. Specifically, a new band regrouping subnetwork that generates the band weights using hyperspectral feature information is proposed. The bands are divided into several groups using band weights and imported into the Siamese network. Finally, the response-level fusion strategy is adopted to integrate the tracker results for the precise location of objects. Experiments on hyperspectral video reveal that the accuracy of the BRRF-Net is up to 0.689, which is the state-of-the-art performance compared with the current hyperspectral object trackers and proves the effectiveness and superiority of the BRRF-Net.

Hyperspectral image (HSI) classification is an active research topic in remote sensing. Supervised learning-based methods have been widely used in HSI classification tasks due to their powerful feature extraction capabilities for cases of sufficiently labeled samples. However, practical applications often have limited samples with accurate labels due to the high cost of labeling or unreliable visual interpretation. We introduce a contrastive self-supervised learning (SSL) algorithm to achieve HSI classification for problems with few labeled samples. First, a new HSI-specific augmentation module is developed to generate sample pairs. Then, a contrastive SSL model based on Siamese networks is used to extract features from these easily accessible sample pairs. Finally, the labeled samples are taken to fine-tune the parameters of the classification model to boost classification performance. Tests of the contrastive self-supervised algorithm have been performed on two widely used HSI datasets. The experimental results reveal that the proposed algorithm requires a few labeled samples to achieve superior performance.

Hyperspectral image (HSI) data have large numbers of bands that probably not all bands are equally informative and predictive for an effective HSI classification. Effective algorithms are highly desired in many real-world HSI applications, especially in cases requiring rapid learning with limited computing power. To address the abovementioned case, we present in this article a novel plug-and-play compact band weighting (CBW) module based on the attention-driven mechanism that evaluates different spectral bands according to their contributions to a given classification task. Compared to existing band weighting (BW) modules with tens of thousands of network parameters by deep learning, the proposed CBW is a lightweight module with only 20 parameters. Both model complexity and time cost are significantly reduced. The CBW module implements BW by making full use of the correlation among the adjacent spectral bands and spectral statistic information and, thereby, leads to the effect of recalibrated HSI. The experimental study has been conducted on three widely used HSI data sets, and results show the superiority of the proposed algorithm over current state-of-the-art methods of BW. The source code is available athttps://github.com/JarvenYi/CBW.

Moving foreground objects detection in complex scenes is a tough job because it requires high recognition accuracy. Adaptive Gaussian mixture model (AGMM) can be used to extract the foreground objects and it shows good performance, however, the detection quality of the foreground objects under complex scenes is not excellent. In this paper, an AGMM and BP neural network hybrid method is proposed, which is used to extract the foreground objects in complex scenes such as, dynamic backgrounds, illumination changes and moving shadows. In this method, an improved BP neural network is used to post-process the images of the foreground objects that are extracted from the AGMM. The neural network has strong robustness by learning the statistical features of the images. Momentum term and adaptive learning rate are added in the BP neural network algorithm to improve the training speed and robustness of the network. The experimental results show that the proposed AGMM and BP neural network hybrid method can extract the complete foreground objects effectively when compared with some other moving objects detection algorithms.

Background modeling in complex scenes is a challenging problem. In this paper, a novel background subtraction method is proposed to address it. First, the textures are modeled with local compact binary patterns (LCBP), which have excellent robustness, strong discriminative power, and fast computation speed. To make LCBP more effective to appearance changes in complex scenarios, spatiotemporal local compact binary patterns (STLCBP) are then considered in which spatial texture information and temporal motion information are combined together. Multiple color spaces are also presented to separate foreground pixels more accurately from the background. To our knowledge, this is the first time that LCBP have been used for background modeling. Extensive experimental results on a widely used dataset clearly show that the proposed method outperforms other state-of-the-art methods and works effectively in complex scenes.

General machine vision algorithms are difficult to detect LCD sub-pixel level defects. By studying the LCD screen images, we found that the pixels in the LCD screen are regularly arranged. The spectrum distribution of LCD images, which is obtained by the Fourier transform, is relatively consistent. According to this feature, a method of sub-pixel defect detection based on notch filter and image registration is proposed. First, we take a defect-free template image to establish registration template and notch-filtering template; then we take the defect images for image registration with registration template, and solve the offset problem. After the notch-filter template filtering the background texture, the defect is more obvious; Finally the defects are obtained by the threshold segmentation method. The experiment results show that the proposed method can detect sub-pixel defects accurately and quickly.

The joint sparse representation (JSR)-based classifier assumes that pixels in a local window can be jointly and sparsely represented by a dictionary constructed by the training samples. The class label of each pixel can be decided according to the representation residual. However, once the local window of each pixel includes pixels from different classes, the performance of the JSR classifier may be seriously decreased. Since correlation coefficient (CC) is able to measure the spectral similarity among different pixels efficiently, this letter proposes a new classification method via fusing CC and JSR, which attempts to use the within-class similarity between training and test samples while decreasing the between-class interference. First, the CCs among the training and test samples are calculated. Then, the JSR-based classifier is used to obtain the representation residuals of different pixels. Finally, a regularization parameter λ is introduced to achieve the balance between the JSR and the CC. Experimental results obtained on the Indian Pines data set demonstrate the competitive performance of the proposed approach with respect to other widely used classifiers.

To improve the rate stability and make a balance for different viewpoints in distributed multi-view video coding (DMVC) system, a novel symmetric DMVC (SDMVC) scheme is proposed in this paper. In the proposed scheme, every frame from all views adopts the same encoding mode and stable output rates are achieved, which are significant to improve the transmission efficiency in the channel. Both temporal and spatial correlations are exploited, in addition, a novel side information (SI) generation algorithm aiming at better exploring the correlations of proposed scheme has been proposed to obtain better performance. The simulation results show that the proposed SDMVC scheme gets a much more stable rate than the asymmetric scheme, only with neglectable bit-rate increasing. Meanwhile, the proposed SI generation algorithm significantly improves the coding performance.