The effective fusion of multi-modal remote sensing images, particularly hyperspectral imagery (HSI) and light detection and ranging (LiDAR) data, is pivotal for accurate land use and land cover (LULC) classification. However, this process is hindered by two inherent challenges: pervasive data redundancy and the underutilization of cross-modal complementarity, largely due to the lack of a unifying theoretical framework. To address these limitations, we propose the multi-modal complementary information bottleneck (MCIB) framework, which extends the IB principle to learn compact, sufficient, and complementary representations for multi-modal scenes. From a theoretical perspective, we formalize the MCIB objective and introduce structured priors to derive tractable information-theoretic bounds, providing a principled and computationally feasible approach to reduce redundancy and enhance complementarity simultaneously. Building on the obtained theoretical insights, we design an end-to-end variational optimization strategy with a novel supervised conditional InfoNCE (SCInfoNCE). Efficiently reusing existing model components, this new supervised contrastive method optimizes the conditional mutual information terms crucial for synergy. Extensive experiments on benchmark HSI-LiDAR datasets demonstrate superior classification performance of MCIB. This work not only fills a theoretical gap in multi-modal representation learning, but offers a robust and principled solution for LULC classification using complex heterogeneous remote sensing images.

Hyperspectral target detection (HTD) holds significant promise in numerous earth vision applications, yet it encounters challenges in acquiring high-quality prior target signatures, capturing target spectral variability, and dealing with sample imbalance. To address these issues, we propose a weakly supervised solution, the Proxy-Enhanced Prototype Memory Network (PE-PMN), for HTD tasks. It relies solely on region-level weakly labeled data, eliminating the need for strict prior target knowledge (e.g., handcrafted target signatures or pixel-level annotations). To fully describe target variations and background diversity, two memory prototype networks are introduced to extract, store, and retrieve prototypes of targets and backgrounds, providing comprehensive spectral information. Additionally, a proxy-based enhancement approach is incorporated to enrich the prototypes in the memory banks and boost the separation between target and background features. To mitigate sample imbalance in PE-PMN, we develop the Bag Mix-Up (BMU) strategy based on the Unconstrained Linear Mixture Model (ULMM) to construct a sufficient training dataset. Experimental results on three simulated datasets and three real datasets demonstrate that the proposed PE-PMN significantly outperforms other competitive weakly supervised HTD methods.

The hyperspectral target detection is widely concerned in geoscience and remote sensing due to the abundant spectral information in hyperspectral imagery. However, the detection performance is highly dependent on the high-quality target signature or pixel-level supervised signals, which are extremely challenging and costly. In this article, we propose a variational multiple-instance neural network with embedding correlation modeling (VMIL-ECM) for weakly supervised hyperspectral target detection, which relaxes the rigid target prior (e.g., target signatures and/or pixel-level annotations), and only region-level labels are required. VMIL-ECM explicitly models the location of the targets within the region as a latent variable under the nonindependent and identically distributed (non-i.i.d.) assumption to estimate the underlying ground-truth target locations. The expectation-maximization (EM) algorithm is employed to iteratively optimize the posterior distribution of latent variables and learn discriminative spectral features for the target detection. To fully utilize the contextual information within the hyperspectral region, a permutation-invariant transformer-based structure is devised to explore the embedding correlation among instances. Moreover, a dynamic thresholding strategy is adopted to produce the reliable fine-grained supervised signals. Extensive experiments on three simulated datasets and two real-field datasets are conducted to verify the effectiveness of VMIL-ECM, and the state-of-the-art performance has been achieved over the existing comparison methods. The code for the VMIL-ECM is publicly available at: https://github.com/BoYangXDU/VMIL-ECM.

With the development of multi-modal technology, hyperspectral image (HSI) and light detection and ranging (LiDAR) data has achieved remarkable results in land use and land cover (LULC) classification. Recently, many deep learning based feature extraction and fusion methods have improved the classification performance of LULC tasks. However, most of these methods use a single feature extractor and do not fully utilize the information of HSI and LiDAR data. Moreover, directly fusing various features obtained from feature extractor can lead to feature redundancy, resulting in model overfitting. In this paper, we develop a three-branch excitation network, named TBENet. The three branches extract spectral features, spatial features and elevation features respectively. And an excitation block is used to reduce feature redundancy and improve the generalization of the model. Contrast experiments on Houston dataset show that our proposed method outperforms other state-of-the-art methods, and ablation experiments demonstrate the effectiveness of each block.

Hyperspectral target detection tasks in remote sensing are frequently constrained by the challenge of obtaining pixel-level labels. Despite this challenge, acquiring region-level labels in hyperspectral images is more feasible. Consequently, researchers frequently turn to weakly supervised learning techniques, such as multi-instance learning, to address this issue. This paper proposes a multi-instance neural network with self-attention semantic modeling for hyperspectral target detection. Under semantic modeling, the network adaptively extracts representative spectral features from bag-level annotated hyperspectral data. The spectral features of the target and background are then clearly differentiated by metric learning and classification tasks. The proposed method demonstrates superior effectiveness in weakly labeled hy-perspectral target detection on both simulated and real-field datasets.

Attention-based deep multiple-instance learning (MIL) has been applied to many machine-learning tasks with imprecise training labels. It is also appealing in hyperspectral target detection, which only requires the label of an area containing some targets, relaxing the effort of labeling the individual pixel in the scene. This article proposes an L1 sparsity-regularized attention multiple-instance neural network (L1-attention MINN) for hyperspectral target detection with imprecise labels that enforces the discrimination of false-positive instances from positively labeled bags. The sparsity constraint applied to the attention estimated for the positive training bags strictly complies with the definition of MIL and maintains better discriminative ability. The proposed algorithm has been evaluated on both simulated and real-field hyperspectral (subpixel) target detection tasks, where advanced performance has been achieved over the state-of-the-art comparisons, showing the effectiveness of the proposed method for target detection from imprecisely labeled hyperspectral data.

Target detection becomes increasingly important in hyperspectral image analysis but is limited by difficulties in acquiring accurate pixel-level training labels. This paper proposes a multiple instance metric learning neural network (MIML-Net) for hyperspectral target detection tasks, which only requires region-level labels and greatly alleviates the laborious pixel-level annotation problems. Our method learns the embeddings of regions with weak labels under attention-based multiple instance learning framework. Based on which, we impose a novel metric-based regularizer to constrain target and background embeddings to two learnable compact clusters with distinct centroids, which further boosts the spectral feature representation ability. The proposed metric-based regularizer enforces a discriminative detector due to its capability to reduce the intra-class variations and encourage the inter-class separations simultaneously. Extensive experimental results from both simulated and real-field data sets demonstrate the effectiveness of the proposed MIML-Net in comparison with the state-of-the-art weakly supervised techniques.

Subpixel target detection in hyperspectral imagery is challenging since subpixel targets are smaller in size than the resolution of a single pixel and accurate pixel-level labels on subpixel targets are often unavailable. In particular, this article addresses the problem of learning a prime prototype target signature from imprecisely labeled highly mixed hyperspectral data. Two algorithms, multiple-instance subpixel adaptive cosine estimator (MI-SPACE) and multiple-instance subpixel spectral matched filter (MI-SPSMF), based on multiple-instance learning framework are presented. The proposed methods aim to learn a discriminative prime target signature by maximizing the posterior detection statistics of subpixel hyperspectral targets for the correspondingly proposed subpixel adaptive cosine estimator (SPACE) and subpixel spectral matched filter (SPSMF) detectors, which are also developed in this article. Experimental results demonstrate the effectiveness of the proposed methods on both simulated and real-field hyperspectral subpixel target detection tasks.

In hyperspectral imagery, target detection is challenging since lots of pixels are a mixture of more than one distinct substance and precise pixel-wise labels are often infeasible to obtain. To address this problem, the Multiple Instance Constrained Energy Minimization (MI-CEM) for estimating a discriminative target signature from inaccurately labeled and mixed hyperspectral data is introduced in this paper. The proposed method maximizes the posterior detection statistics of the constrained energy minimization sub-pixel detector and estimates a discriminative target signature. The learned target signature can be applied to CEM for sub-pixel target detection. Experiments on both simulated and real-world data demonstrate that MI-CEM achieves competitive performance compared with the state-of-the-art algorithms.