Given the inherently costly and time-intensive nature of pixel-level annotation, the generation of synthetic datasets comprising sufficiently diverse synthetic images paired with ground-truth pixel-level annotations has garnered increasing attention recently for training high-performance semantic segmentation models. However, existing methods necessitate to either predict pseudo annotations after image generation or generate images conditioned on manual annotation masks, which incurs image-annotation semantic inconsistency or scalability problem. To migrate both problems with one stone, we present a novel dataset generative diffusion framework for semantic segmentation, termed JoDiffusion. Firstly, given a standard latent diffusion model, JoDiffusion incorporates an independent annotation variational auto-encoder (VAE) network to map annotation masks into the latent space shared by images. Then, the diffusion model is tailored to capture the joint distribution of each image and its annotation mask conditioned on a text prompt. By doing these, JoDiffusion enables simultaneously generating paired images and semantically consistent annotation masks solely conditioned on text prompts, thereby demonstrating superior scalability. Additionally, a mask optimization strategy is developed to mitigate the annotation noise produced during generation. Experiments on Pascal VOC, COCO, and ADE20K datasets show that the annotated dataset generated by JoDiffusion yields substantial performance improvements in semantic segmentation compared to existing methods.

Meta-learning has emerged as an effective solver for cross-domain few-shot learning (CD-FSL) tasks. Despite achieving obvious progress recently, the typical episodic learning paradigm often causes the feature embedding model collapsing into the simplicity bias pitfall, viz., the model tends to prioritize some shortcut patterns (e.g., color, style, background) that are only sufficient to distinguish categories in source domain, while fail to generalize across domains. To mitigate this problem, we present a novel meta-learning framework which emphasizes meta-exploiting inductive bias to alleviate simplicity bias for CD-FSL promotion, and mainly contributes in the following four aspects. 1) We establish a novel inductive bias for CD-FSL, termed complementary semantic consistency (CSC). The rationale behind lies in that forcing the semantic consistency between two complementary feature learning schemes is beneficial to distill cross-domain transferable features. 2) We establish a solid theoretical foundation, supported by rigorous mathematical proofs and key lemmas, which demonstrates that CSC establishes a tighter generalization bound and facilitates the learning of domain-invariant features. 3) Inspired by CSC, we propose a general meta-learning framework, which implements complementary feature embedding models using parallel networks with the same architecture but different input forms, and introduce proper knowledge distillation losses to encourage the semantic consistency between different branches during meta-training. This framework can be seamlessly integrated with any complementary feature learning schemes. 4) To clarify this point, we instantiate two effective meta-learners based on the proposed framework. The former establishes a two-branch network that simultaneously classifies both the query image and its random local crops. The latter decomposes the query image into high-frequency and low-frequency components, which are then integrated into a parallel feature embedding network for category prediction, analogous to the original query image. Subsequently, a KL divergence based knowledge distillation loss is separately leveraged to force the prediction consistency between the complementary branches (e.g., local-global, spatial-frequency) during meta-training. By doing these, both learners are able to distill cross-domain transferable features with better generalization performance. Empirical results on diverse benchmarks consistently affirm the proposed framework's advantages, while additional analysis provides compelling support for our key claims.

Diffusion model has underpinned much recent advances of dataset augmentation in various computer vision tasks. However, when involving generating multi-object images as real scenarios, most existing methods either rely entirely on text condition, resulting in a deviation between the generated objects and the original data, or rely too much on the original images, resulting in a lack of diversity in the generated images, which is of limited help to downstream tasks. To mitigate both problems with one stone, we propose a prompt-free conditional diffusion framework for multi-object image augmentation. Specifically, we introduce a local-global semantic fusion strategy to extract semantics from images to replace text, and inject knowledge into the diffusion model through LoRA to alleviate the category deviation between the original model and the target dataset. In addition, we design a reward model based counting loss to assist the traditional reconstruction loss for model training. By constraining the object counts of each category instead of pixel-by-pixel constraints, bridging the quantity deviation between the generated data and the original data while improving the diversity of the generated data. Experimental results demonstrate the superiority of the proposed method over several representative state-of-the-art baselines and showcase strong downstream task gain and out-of-domain generalization capabilities. Code is available at \href{https://github.com/00why00/PFCD}{here}.

Cross-domain few-shot learning (FSL) has demonstrated remarkable new classes recognition capabilities in hyperspectral image classification tasks. However, existing domain adaptation methods face two critical challenges in the cross-domain feature alignment process: first, the domain shift leads to misaligned feature transfer and diminished classification accuracy; second, the intra-class feature dispersion and inter-class boundary blurring in few-shot tasks result in degraded classification performance for novel classes. Moreover, the impact of redundant and noisy data on model discriminability is rarely considered in existing approaches. To solve these issues, this article proposes a cross-domain FSL hyperspectral image classification method based on diffusion-augmented prototype knowledge distillation (DAPKD-CFSL). Firstly, we introduce a diffusion-augmented unsupervised domain adaptation pre-training (DA-PT) framework to address the domain shift by performing a domain-adversarial denoising and reconstruction task using visible source data and masked target data. Second, our dual-branch spatial-spectral attention (DB-SSA) captures global and local spectral-spatial dependencies to enhance feature representation. Then, the proposed global-local prototype knowledge distillation (GL-PKD) performs global prototype alignment while conducting local contrastive learning, addressing feature dispersion and boundary ambiguity. Finally, a dynamic learning strategy prioritizes feature alignment early and gradually strengthens classification supervision through adaptive loss weights, and incorporates an SNR-enhanced loss to effectively mitigate noise interference. The experimental results on three HSI datasets demonstrate the superiority and effectiveness of the proposed DAPKD-CFSL.

Fusing a low-resolution (LR) hyperspectral image (HSI) with a high-resolution (HR) multispectral image (MSI) has emerged as a promising strategy for reconstructing high-quality HSIs that combine rich spectral and fine spatial information. However, most existing HSI fusion methods operate under the restrictive assumption that the LR HSI and HR MSI are spatially aligned and fully consistent on the field-of-view (FoV), which significantly limits their applicability in real-world scenarios when such alignment is unavailable. To overcome these limitations, we propose IrregFusion, a generalized HSI fusion framework capable of handling both FoV-consistent and inconsistent fusion scenarios. Specifically, IrregFusion incorporates a Transformer-based reconstruction module that captures both intra- and inter-modal correlations between the diverse spectra data and the MSI, enhancing the model’s ability to perceive and reconstruct non-local spectral–spatial structures. To further address the challenges posed by FoV inconsistencies, we introduce a spectral propagation strategy that diffuses observed spectral information into adjacent spectral-blank regions, thereby easing the reconstruction of missing spectral content. Additionally, a self-supervised adaptation mechanism is integrated into the framework, enabling robust spectral–spatial representation learning and enhancing generalization across diverse and challenging conditions. Extensive experiments conducted on benchmark datasets demonstrate that IrregFusion effectively addresses the challenges of diverse spectral data fusion and consistently outperforms state-of-the-art methods in both reconstruction accuracy and visual fidelity. The source code will be released in https://github.com/JiangtaoNie/IrregFusion.git.

Spectral Super-Resolution (SSR) aims at reconstructing a latent hyperspectral images (HSI) from a RGB image. Recent progress mainly focused on building a deep spectral super-resolution networkto directly map the input RGB image to the corresponding HSI. Their pleasing performance depends on the assumption that the spectral response function determined by the RGB sensor is consistent across training and test data. However, in practice, the training and test data are inevitably captured by different RGB sensors, thus resulting in obvious performance drop when using these networks. To mitigate this problem, we present a novel cognitive feature guided cross-sensor robust spectral super-resolution network. In a specific, a U-shape multi-scale network is first established to learn the deep mapping between input RGB image and the latent HSI. Then, a large-scale foundation cognitive model is introduced to extract multi-level cross-sensor invariant cognitive features from the input RGB. Moreover, these features are separately adapted and injected into different decoder blocks in the U-shape spectral super-resolution network. By doing these, the proposed network learns to appropriately guide the coarse-to-fine spectral reconstruction process using multilevel cognitive features, and thus shows better generalization performance in the cross-sensor SSR tasks. Experiments on two benchmark datasets demonstrate the superiority of the proposed method over several state-of-the-art baselines.

Aircraft detection in synthetic aperture radar (SAR) images is of much significance because of its all-weather, all-day, and strong penetrating characteristics. However, existing algorithms exhibit inadequate capacity for feature extraction due to imaging discontinuities and background interference of SAR images. To overcome these task-specific issues, we proposed a feature enhancement-based SAR aircraft detection algorithm. In detail, we employed Adaptive Contrast Enhancement (ACE) in the preprocessing stage to reduce noises, and then we embed Scatter Point Focused Module (SPFM) into network to enhance the feature extraction of aircraft scattering points. Furthermore, we devised Background Interference Suppression Module (BISM) to accentuate salient points and suppress non-essential pixels. Experimental results on the GaoFen-3 SAR aircraft dataset demonstrate the effectiveness of the proposed feature enhancement-based method.

Meta-learning offers a promising avenue for few-shot learning (FSL), enabling models to glean a generalizable feature embedding through episodic training on synthetic FSL tasks in a source domain. Yet, in practical scenarios where the target task diverges from that in the source domain, meta-learning based method is susceptible to over-fitting. To overcome this, we introduce a novel framework, Meta-Exploiting Frequency Prior for Cross-Domain Few-Shot Learning, which is crafted to comprehensively exploit the cross-domain transferable image prior that each image can be decomposed into complementary low-frequency content details and high-frequency robust structural characteristics. Motivated by this insight, we propose to decompose each query image into its high-frequency and low-frequency components, and parallel incorporate them into the feature embedding network to enhance the final category prediction. More importantly, we introduce a feature reconstruction prior and a prediction consistency prior to separately encourage the consistency of the intermediate feature as well as the final category prediction between the original query image and its decomposed frequency components. This allows for collectively guiding the network's meta-learning process with the aim of learning generalizable image feature embeddings, while not introducing any extra computational cost in the inference phase. Our framework establishes new state-of-the-art results on multiple cross-domain few-shot learning benchmarks.

Few-shot learning (FSL) is an effective approach to address the issue of limited labeled data in hyperspectral image classification (HSIC). However, it overlooks the domain shift between the source domain (SD) and the target domain (TD) in cross-domain tasks. Most existing domain adaptation (DA) methods alleviate the domain shift problem to some extent, but DA methods based on traditional convolutional operators overlook the nonlocal spatial relationships in HSI, while methods based on graph neural networks (GNNs), although effective in leveraging nonlocal spatial information for domain alignment, overly emphasize global relationships, which is disadvantageous for pixel-level classification in HSI. To solve these issues, this article proposes a novel globalp-local graph attention network-based cross-domain FSL (GLGAT-CFSL), which comprehensively reduces domain shift through global-to-local domain alignment. It has the following advantages: 1) an innovative dynamic triplet graph attention network is devised to identify nonlocal spatial relationships in HSI for global graph alignment (GGA) while also addressing common overfitting and oversmoothing issues in GNNs; 2) an ingenious local similarity learning (LSL) strategy is designed after global domain alignment, utilizing intradomain connectivity structures and interdomain node similarities for local DA, promoting cross-domain information propagation and more comprehensive reduction of domain shift; and 3) we propose a novel triaxial dynamic convolutional neural network (TDCNN) as the feature extractor, promoting cross-dimensional interaction between spectral and spatial dimensions, establishing a more generalizable and rich feature representation between the SD and the TD. The experimental results on three HSI datasets demonstrate the superiority and effectiveness of the proposed GLGAT-CFSL.

In recent years, active learning (AL) methods have provided a feasible approach to alleviate the problem of limited labeled samples in deep learning projects. Existing AL algorithms generally tend to select sample without labeled, whose category is difficult to distinguish. However, the sample in the category center is difficult to determine in AL operations, resulting in inaccurate category measuring and inaccurate sample selection. In addition, hyperspectral images (HSIs) have rich spectral reflective bands with strong correlations, which leads to the phenomenon that the spatial distribution between different categories in HSIs characterizes staggered distribution, which undoubtedly influences the HSI classification effect. In this article, we propose a new AL method (called PLGCN) which combines prototype learning (PL) and graph convolution network (GCN) to solve few-shot HSI classification tasks, and this method can add into existing deep learning-based HSI classification models. It includes two advantages: 1) the prototype of each category is iteratively updated to ensure the optimality of prototype in each sampling stage and 2) the spatial distribution of unlabeled samples is extracted via graph convolution neural network in order to obtain the better features in new space for easier discriminating. Experimental results on three commonly used benchmark HSI datasets demonstrate the effectiveness of the PLGCN in HSI classification tasks with limited labeled samples.

Spectral super-resolution (SSR) aims at reconstructing a hyperspectral image (HSI) from an observed RGB image through interpolation in the spectral domain. Recent progress mainly focus on establishing various deep interpolation networks to directly exploit the spatial-spectral information of the RGB image for SSR. However, few of them pay attention on its frequency information, which proves to be orthogonal to the spatial-spectral information and also crucial for SSR, and thus their generalization performance can be further improved. To mitigate this problem, in this study we proposes a wavelet transform based network (WTNet) for SSR. Different from existing SSR networks in image-domain, the Haar wavelet transform is employed to decompose the input RGB image into four different frequency bands. Moreover, a multi-scale convolution and self-attention based feature extraction block and a cross-attention based band interaction block are constructed to separately exploit the statistics within each band as well as the inter-band frequency correlation. By doing these, the proposed WTNet is able to sufficiently exploit the frequency information of the input RGB image for accurate SSR. Experimental results on two datasets demonstrate the efficacy and superior SSR performance of the proposed WTNet.

Fusing infrared and visible images has been an active research topic within the remote sensing community since these two kinds of images can provide complementary information. Though different methods have been proposed, most of the existing infrared and visible image fusion methods only focus on obtaining visually pleasing results without considering if the fusion results fit well for the subsequential object detection task. To obtain better detection results from infrared and visible image fusion, we propose an end-to-end network that incorporates an image fusion module and object detection module into a unified framework. Within the network architecture constructed, attention mechanisms as well as intensity loss and gradient loss are utilized to effectively preserve the distinguishing characteristics of both infrared and visible modalities for object detection, yielding advantageous attributes for detection purposes. By jointly training the image fusion module and the object detection module, our proposed method achieves improved object detection performance. Experimental results corroborate the effectiveness of the proposed approach.

Deep neural networks have underpinned much of recent progress in few-shot object detection (FSOD) in remote sensing images. The key lies in accurately inferring the object categories and bounding boxes depending on the feature of each proposal region. However, due to lack of sufficient labeled samples for training model well-fitting, the feature of each proposal fails to be discriminative and informative enough for accurate inference, thus limiting the generalization capacity. To mitigate this problem, we propose a non-local proposal dynamic enhancement learning (NPDEL) methods for FSOD in remote sensing images. In contrast to directly utilizing the proposal features extracted from the backbone, we propose to enhance them before inference using a non-local dynamic enhancement module which first carries out a non-local graph convolution on all proposal features and then dynamically fuses the convolved results with the original features for enhancement. By doing this, the enhanced proposal features can adaptively aggregate the related semantic information from the whole image, thus improving their discriminability as well as the generalization capacity in FSOD. Experiments results on different FSOD tasks demonstrate the efficacy of the proposed method.

A promising way for low-illumination (LI) remote sensing images high-dynamic range (HDR) reconstruction is to model the mapping function from the input LI images to the corresponding high-quality counterpart using deep convolution neural networks. Due to various image contents, the key for achieving pleasing performance lies on comprehensively exploit the image-specific long-rang (e.g., non-local similarity, low-rank) and short-range (e.g., local similarity, texture etc.) structures in the LI images using appropriate network architecture. However, most existing methods can only exploit either short-range or long-range structures that are contentagnostic shared across all images, thus limiting their generalization capacity. To tackle this problem, we propose a dynamic long-short range structure learning framework for LR remote sensing images HDR reconstruction. In contrast to existing methods, we introduce a novel two-branch network architecture including a pixel-aware dynamic module that can adaptively exploit the pixel-aware short-range structure surrounding each pixel depending on its feature representation, and a long-range transformer module that dynamically exploit the long-range correlation between image patchesin the deep feature space. Then, the learned long-short range structures are integrated and cast into pixel-wise scaling factors of an illumination enhance module to restore the LI image. It empowers us to effectively exploit the image-specific long-short range structures of each input IL images for accurate HDR reconstruction. Experimental results on remote sensing images with different levels of IL demonstrate the effectiveness of the proposed method.

Hyperspectral image (HSI) classification is an essental task of HSI analysis, which aims to assign each pixel a pre-defined class label. Though deep learning based methods dominate the HSI classification methods to date, the existing methods seldom consider how to directly model the uncertainty broadly exists in the HSI applications, which impedes their usage in real applications. To address this problem, we propose to directly model the uncertainty into the deep learning based HSI classification model and construct a specific network based on stochastic differential equation (SDE). The constructed network consists two subnets, in which one is utilized to well fit the HSI classification task and one is exploited to capture the uncertainty within the HSI classification. The constructed network can better depict the uncertainty, and thus result in better HSI classification performance. Experimental results demonstrate the effectiveness of the constructed model for HSI classification.

Fusion based hyperspectral image (HSI) super-resolution has long been the research focus of hyperspectral image processing since it can generate a high-resolution (HR) HSI in both spatial and spectral domains. However, the success of the existing fusion based HSI super-resolution methods depends on the premise that the images utilized for fusion (i.e. the input low-spatial-resolution HSI and the low-spectral-resolution multispectral image) are exactly registered. Although such a premise is too idealistic to comply with in real cases, few efforts have considered this problem. To fill this gap, we propose to incorporate image registration into HSI super-resolution for joint unsupervised learning in this study. Specifically, a spatial transformer network (STN) is introduced to learn the parameters of the affine transformation between the input two images. In order to avoid over-fitting, we constrain the STN with a novel constraint during learning. By doing this, both the STN and super-resolution network can be cast into a weighted joint learning model without any supervision from the latent HR HSI. Experimental results demonstrate the effectiveness of the proposed method in coping with unregistered input images.

Hyperspectral image (HSI) super-resolution, a technique to obtain higher (often spatial) resolution image from the original image, has been extensively studied and applied to lots of fields such as computer vision, remote sensing, etc. Though fusion based method has achieved state-of-the-art result, it always assume the spatial transformation matrix is given in advance, whereas such a matrix is actually unknown in reality. An unsuitable given matrix will deteriorate the superresolution result greatly. To address this issue, we propose a novel fusion based HSI super-resolution method without knowing the spatial transformation matrix. Specifically, we incorporate super-resolution and spatial transformation matrix estimation into a unified framework. We alternately estimate the matrix and the higher spatial resolution HSI. We find that without given the spatial transformation matrix, the proposed method can obtain more accurate reconstruction result compared with other competing methods. Experimental results demonstrate the effectiveness of the proposed method.