Transductive Information Maximization (TIM) is a leading transductive few-shot learning method that maximizes the mutual information between query features and their predicted labels, while incorporating supervision from the support set. However, its potential remains underexplored, primarily due to the limited utilization of textual knowledge provided by vision-language models (VLMs) such as CLIP. To address this, we propose TIM++, an enhanced framework that incorporates both visual and textual information for few-shot CLIP adaptation. Specifically, TIM++ introduces a Kullback-Leibler (KL) divergence-based regularization term that encourages the model’s posterior predictions to align with CLIP’s zero-shot output distribution, especially focusing on the most confident predictions. Additionally, we develop an improved prototype initialization strategy that leverages both support and query features enriched with CLIP-guided semantics. Extensive experiments on 11 public datasets demonstrate that TIM++ consistently outperforms the standard TIM, achieving average accuracy gains of 19.25% and 10.88% in 1-shot and 2-shot settings, respectively. TIM++ also surpasses other existing state-of-the-art methods, establishing a new benchmark for few-shot learning with VLMs.

The effective fusion of multi-modal remote sensing images, particularly hyperspectral imagery (HSI) and light detection and ranging (LiDAR) data, is pivotal for accurate land use and land cover (LULC) classification. However, this process is hindered by two inherent challenges: pervasive data redundancy and the underutilization of cross-modal complementarity, largely due to the lack of a unifying theoretical framework. To address these limitations, we propose the multi-modal complementary information bottleneck (MCIB) framework, which extends the IB principle to learn compact, sufficient, and complementary representations for multi-modal scenes. From a theoretical perspective, we formalize the MCIB objective and introduce structured priors to derive tractable information-theoretic bounds, providing a principled and computationally feasible approach to reduce redundancy and enhance complementarity simultaneously. Building on the obtained theoretical insights, we design an end-to-end variational optimization strategy with a novel supervised conditional InfoNCE (SCInfoNCE). Efficiently reusing existing model components, this new supervised contrastive method optimizes the conditional mutual information terms crucial for synergy. Extensive experiments on benchmark HSI-LiDAR datasets demonstrate superior classification performance of MCIB. This work not only fills a theoretical gap in multi-modal representation learning, but offers a robust and principled solution for LULC classification using complex heterogeneous remote sensing images.

Synthetic Aperture Radar (SAR) images have become a primary data source for change detection due to their all-weather, all-day imaging capability, high resolution, and strong sensitivity to ground surface variations. However, the complex scattering characteristics of SAR make distinguishing between changed and unchanged areas particularly challenging. Additionally, the long-tail data distribution, where changed areas constitute only a small portion of the dataset, further exacerbates the difficulty of change detection. To address these challenges, we propose a Fine-Grained Meta-Learning with Semantic Augmentation method for SAR image change detection. First, we propose a fine-grained classification strategy based on edge detection to construct a small, balanced dataset for training the meta-learner and a dataset with reduced imbalance for training the backbone classifier. This strategy enhances the feature learning capability of hard-to-classify samples and reduces the adverse effects of data imbalance. Next, we design a lightweight yet effective Multi-Layer Perceptron-based meta-learner and incorporate semantic data augmentation. The meta-learner automatically augments minority classes along meaningful semantic directions by learning appropriate class-wise covariance matrices, thereby improving the detection performance of the backbone classifier. We evaluate our method on four SAR datasets through cross-dataset experiments, demonstrating its superiority over four methods in effectiveness and robustness. Imbalance analysis further reveals that as the imbalance ratio increases, the performance of comparison methods degrades significantly, whereas our method exhibits the least performance deterioration, confirming its competent capability in handling imbalanced data.

Hyperspectral target detection (HTD) holds significant promise in numerous earth vision applications, yet it encounters challenges in acquiring high-quality prior target signatures, capturing target spectral variability, and dealing with sample imbalance. To address these issues, we propose a weakly supervised solution, the Proxy-Enhanced Prototype Memory Network (PE-PMN), for HTD tasks. It relies solely on region-level weakly labeled data, eliminating the need for strict prior target knowledge (e.g., handcrafted target signatures or pixel-level annotations). To fully describe target variations and background diversity, two memory prototype networks are introduced to extract, store, and retrieve prototypes of targets and backgrounds, providing comprehensive spectral information. Additionally, a proxy-based enhancement approach is incorporated to enrich the prototypes in the memory banks and boost the separation between target and background features. To mitigate sample imbalance in PE-PMN, we develop the Bag Mix-Up (BMU) strategy based on the Unconstrained Linear Mixture Model (ULMM) to construct a sufficient training dataset. Experimental results on three simulated datasets and three real datasets demonstrate that the proposed PE-PMN significantly outperforms other competitive weakly supervised HTD methods.

Ballistocardiograms (BCG) is a passive, non-contact heart rate detection technology that requires no action on the part of the individual. However, during the BCG signal acquisition process, the surface pressure generated by cardiac contraction is easily disturbed by external factors, and as people's health deteriorates, the j-peak (the main peak of the BCG signal) is no longer prominent. Our aim is to establish a non-contact, self-supervised heart rate detection method based on physiological information, to improve the accuracy and robustness of BCG heart rate detection under wider and more adverse conditions. The algorithm is guided by the heart rate estimation based on BCG itself, thereby reconstructing a signal with physiological significance. We also propose a heartbeat mapping algorithm based on Bidirectional Long Short-Term Memory Network (BiLSTM) for extracting global deep features, achieving real-time heartbeat prediction, and eliminating local deviations brought about by reconstruction. To verify the effectiveness of the proposed method, this paper evaluated 40 young subjects and 4 elderly subjects. Compared with the existing state-of-the-art methods, beat-to-beat heart rate estimation and heartbeat detection both performed excellently, surpassing most methods using precise labels. The experimental results show that the proposed method achieves effective heartbeat detection, demonstrating robustness and effectiveness in the face of unavoidable noise and variations.

The hyperspectral target detection is widely concerned in geoscience and remote sensing due to the abundant spectral information in hyperspectral imagery. However, the detection performance is highly dependent on the high-quality target signature or pixel-level supervised signals, which are extremely challenging and costly. In this article, we propose a variational multiple-instance neural network with embedding correlation modeling (VMIL-ECM) for weakly supervised hyperspectral target detection, which relaxes the rigid target prior (e.g., target signatures and/or pixel-level annotations), and only region-level labels are required. VMIL-ECM explicitly models the location of the targets within the region as a latent variable under the nonindependent and identically distributed (non-i.i.d.) assumption to estimate the underlying ground-truth target locations. The expectation-maximization (EM) algorithm is employed to iteratively optimize the posterior distribution of latent variables and learn discriminative spectral features for the target detection. To fully utilize the contextual information within the hyperspectral region, a permutation-invariant transformer-based structure is devised to explore the embedding correlation among instances. Moreover, a dynamic thresholding strategy is adopted to produce the reliable fine-grained supervised signals. Extensive experiments on three simulated datasets and two real-field datasets are conducted to verify the effectiveness of VMIL-ECM, and the state-of-the-art performance has been achieved over the existing comparison methods. The code for the VMIL-ECM is publicly available at: https://github.com/BoYangXDU/VMIL-ECM.

With the development of multi-modal technology, hyperspectral image (HSI) and light detection and ranging (LiDAR) data has achieved remarkable results in land use and land cover (LULC) classification. Recently, many deep learning based feature extraction and fusion methods have improved the classification performance of LULC tasks. However, most of these methods use a single feature extractor and do not fully utilize the information of HSI and LiDAR data. Moreover, directly fusing various features obtained from feature extractor can lead to feature redundancy, resulting in model overfitting. In this paper, we develop a three-branch excitation network, named TBENet. The three branches extract spectral features, spatial features and elevation features respectively. And an excitation block is used to reduce feature redundancy and improve the generalization of the model. Contrast experiments on Houston dataset show that our proposed method outperforms other state-of-the-art methods, and ablation experiments demonstrate the effectiveness of each block.

Convolutional Neural Networks (CNNs) have achieved remarkable performance in remote sensing image classification tasks. To address the issue of high model complexity, we propose a block-level pruning strategy based on the semantic similarity analysis that no fine-tuning is required during the pruning process. By employing this strategy, we effectively reduce the complexity of the model. Furthermore, to restore the overall performance of the pruned model, we propose a teacher-student collaborative distillation strategy that enables knowledge transfer through the collaboration of the original model and the dropped-blocks model to promote the performance of the compact pruned model. Experimental results demonstrate that our pruning and distillation strategies outperform other approaches, thereby achieving favorable performance while reducing model complexity.

Hyperspectral target detection tasks in remote sensing are frequently constrained by the challenge of obtaining pixel-level labels. Despite this challenge, acquiring region-level labels in hyperspectral images is more feasible. Consequently, researchers frequently turn to weakly supervised learning techniques, such as multi-instance learning, to address this issue. This paper proposes a multi-instance neural network with self-attention semantic modeling for hyperspectral target detection. Under semantic modeling, the network adaptively extracts representative spectral features from bag-level annotated hyperspectral data. The spectral features of the target and background are then clearly differentiated by metric learning and classification tasks. The proposed method demonstrates superior effectiveness in weakly labeled hy-perspectral target detection on both simulated and real-field datasets.

Cloth-Changing Person Re-Identification (CC-ReID) aims to accurately identify a target person in the more realistic surveillance scenario where clothes of the pedestrian may change drastically, which is critical in public security systems for tracking down disguised criminal suspects. Existing methods mainly transform the CC-ReID problem into cross-modality feature alignment from the data-driven perspective, without modelling the interference factors such as clothes and camera view changes meticulously. This may lead to over-consideration or under-consideration of the influence of these factors on the extraction of robust and discriminative identity features. This paper proposes a novel algorithm for thoroughly disentangling identity features from interference factors brought by clothes and camera view changes while ensuring the robustness and discriminability. It adopts a dual-stream identity feature learning framework consisting of a raw image stream and a cloth-erasing stream, to explore discriminative and cloth-irrelevant identity feature representations. Specifically, an adaptive cloth-irrelevant contrastive objective is introduced to contrast features extracted by the two streams, aiming to suppress the fluctuation caused by clothes textures in the identity feature space. Moreover, we innovatively mitigate the influence of the interference factors through a generative adversarial interference factor decoupling network. This network is targeted at capturing identity-related information residing in the interference factors and disentangling the identity features from such information. Extensive experimental results demonstrate the effectiveness of the proposed method, achieving superior performances to state-of-the-art methods.

Unsupervised out-of-distribution (OOD) detection aims to identify out-of-domain data by learning only from unlabeled In-Distribution (ID) training samples, which is crucial for developing a safe real-world machine learning system. Current reconstruction-based method provides a good alternative approach, by measuring the reconstruction error between the input and its corresponding generative counterpart in the pixel/feature space. However, such generative methods face the key dilemma, $i.e.$, improving the reconstruction power of the generative model, while keeping compact representation of the ID data. To address this issue, we propose the diffusion-based layer-wise semantic reconstruction approach for unsupervised OOD detection. The innovation of our approach is that we leverage the diffusion model's intrinsic data reconstruction ability to distinguish ID samples from OOD samples in the latent feature space. Moreover, to set up a comprehensive and discriminative feature representation, we devise a multi-layer semantic feature extraction strategy. Through distorting the extracted features with Gaussian noises and applying the diffusion model for feature reconstruction, the separation of ID and OOD samples is implemented according to the reconstruction errors. Extensive experimental results on multiple benchmarks built upon various datasets demonstrate that our method achieves state-of-the-art performance in terms of detection accuracy and speed.

The joint classification of multispectral (MS) and panchromatic (PAN) images aims to provide a more detailed and accurate interpretation of land features. Although deep-learning-based methods have achieved remarkable success in this task, the generalization performance of networks is compromised when labeled samples are insufficient. In this study, we explore the possibility of leveraging unlabeled remote sensing images (RSIs) through contrastive learning and demonstrate the challenges associated with directly applying contrastive learning to RSIs. To end this, we propose a cross-source contrastive learning method for few-shot MS and PAN joint classification (CrossCLMP), which aims to learn sufficient transferable representations in a self-supervised contrastive manner so as to provide a robust pretrained model for fine-tuning the downstream joint classification task. Specifically, we design: 1) intersource and intrasource alignment loss (ER-Align) to achieve self-supervised feature extraction and alignment; 2) the source-unique feature adaptive separation (SUAS) strategy to model source-unique information explicitly; and 3) the auxiliary contrastive learning (ACL) strategy to mitigate the adverse impact of numerous false-negative samples in the pretraining stage. The experimental results and the theoretical analyses on multiple popular datasets comprehensively demonstrate the effectiveness and robustness of the proposed method under few-shot. Our code is available at:https://github.com/Xidian-AIGroup190726/CrossCLMP.

As a hot research topic in remote sensing, effectively integrating the advantageous features of multispectral and panchromatic images is the main challenge for fusing these two remote sensing images. This article proposes a multiscale frequency fusion network based on ConvGRU. To address the underutilization of texture features, we extract multiscale bandpass and low-pass sub-bands representing texture and content features through Contourlet decomposition. Multiscale bandpass sub-bands contain more comprehensive and concentrated texture details. Then, by proposing a multiscale frequency feature extractor based on ConvGRU, we effectively integrate and enhance sub-bands of different scales and frequencies, fully utilizing the characteristics of multispectral and panchromatic images and scale transmission. With these enhanced sub-band features, we obtain more comprehensive scale-enhanced texture features. Simultaneously, content features are also preserved as dual-source image features. Moreover, to reduce redundancy between fused features and make more efficient use of the obtained enhanced features, we designed an Inver-band integrator (IBI) module. It can fuse enhanced features at different scales, improve the complementarity between features, and thus achieve effective fusion. Experimental results demonstrate the effectiveness and robustness of our model on multiple datasets. Our codes are available athttps://github.com/Xidian-AIGroup190726/GMFnet.

In recent years, panchromatic (PAN) images and multispectral (MS) images, as a type of multimodal remote sensing data, are attracting increasingly more attention to their classification problems. However, effectively representing size variations of targets in remote sensing images and reducing redundant representations of different modalities’ deep semantic features to enhance classification accuracy remains a challenge. In this article, we propose a semantically nonredundant continuous-scale feature network (SNCF-Net) for PAN and MS classification, consisting of two modules: the texture-enhanced continuous scale input generation module and the cross-modal feature Kernel interaction (CMKI) module. By simulating the human eye’s adjustment of distance to observe objects of different sizes, we employ 3-D convolution to extract continuous-scale images generated by the texture-enhanced continuous-scale input generation (TCIG) module, enabling optimal feature representation of objects in remote sensing images. Additionally, the texture enhancement (TE) strategy in the TCIG module alleviates texture diffusion in scale space, enhancing the network’s ability to represent texture features. Subsequently, the CMKI module utilizes the response differences between different features to generate convolution kernels from deep feature maps, enabling feature interaction between the PAN modal and MS modal. This reduces redundant representations of essential image content information in deep features of two modalities, facilitating a better mapping between dual-modal features and categories. Our results achieve state-of-the-art performance on multiple datasets. The code is available athttps://github.com/Xidian-AIGroup190726/SNCFNet.

Bi-temporal semantic change detection(SCD) is more sophisticated than binary change detection and it provides more detailed changing information with categories. Naturally, it is more challenging than traditional binary change detection. In this paper, a Siamese CNN is proposed for SCD. For the problems of complex backgrounds of remote sensing images, we use multiscale context information and correlation to enhance SCD performance. For the problem of insufficient feature utilization between subtasks, a channel fusion module is proposed to explore the temporal correlation between bi-temporal images, which benefits the extraction of the final changing map. The experiments in this paper are conducted on the SECOND dataset. Our proposed method outperforms compared methods and obtains more completed changing maps than other methods.

Multiple instance learning (MIL) effectively solves the inaccurate labeled hyperspectral target detection problems, which models the region containing the target as a positive bag, and the area without the target as a negative bag. However, the real labels of instances in the positive bag are unknown, which increases the difficulty of hyperspectral target detection. In this paper, we propose a hyperspectral target detection method based on Co-teaching multiple instance neural network (Coteaching MINN), in which the positive instances in the positive bags are selected to participate in the training of the network by comparing the loss value of the training data. Furthermore, deterministic annealing (DA) algorithm is proposed to enhance the stability of the network during the training process. The experimental results on simulated data and real hyperspectral data demonstrate the advancement and robustness of the proposed method.

Attention-based deep multiple-instance learning (MIL) has been applied to many machine-learning tasks with imprecise training labels. It is also appealing in hyperspectral target detection, which only requires the label of an area containing some targets, relaxing the effort of labeling the individual pixel in the scene. This article proposes an L1 sparsity-regularized attention multiple-instance neural network (L1-attention MINN) for hyperspectral target detection with imprecise labels that enforces the discrimination of false-positive instances from positively labeled bags. The sparsity constraint applied to the attention estimated for the positive training bags strictly complies with the definition of MIL and maintains better discriminative ability. The proposed algorithm has been evaluated on both simulated and real-field hyperspectral (subpixel) target detection tasks, where advanced performance has been achieved over the state-of-the-art comparisons, showing the effectiveness of the proposed method for target detection from imprecisely labeled hyperspectral data.

Target detection becomes increasingly important in hyperspectral image analysis but is limited by difficulties in acquiring accurate pixel-level training labels. This paper proposes a multiple instance metric learning neural network (MIML-Net) for hyperspectral target detection tasks, which only requires region-level labels and greatly alleviates the laborious pixel-level annotation problems. Our method learns the embeddings of regions with weak labels under attention-based multiple instance learning framework. Based on which, we impose a novel metric-based regularizer to constrain target and background embeddings to two learnable compact clusters with distinct centroids, which further boosts the spectral feature representation ability. The proposed metric-based regularizer enforces a discriminative detector due to its capability to reduce the intra-class variations and encourage the inter-class separations simultaneously. Extensive experimental results from both simulated and real-field data sets demonstrate the effectiveness of the proposed MIML-Net in comparison with the state-of-the-art weakly supervised techniques.

In practice, the inaccurate labeling problem in hyperspectral images often has a great impact on the target detection accuracy of hyperspectral images. Modeling this inaccurate labeling problem using multiple instance learning (MIL) is a proven way. In this paper, we propose a deep multiple instance learning method based on ensemble learning, in which 1D convolution neural networks (1D CNN) are adopted to realize an end-to-end hyperspectral target detection structure. In the proposed deep MIL target detection method, the instance-level scores are reconstructed by deep ensemble learning combined with different MIL pooling layers, which helps to estimate the instance labels of positive bags. The method achieved good results in both simulated and real hyperspectral data, showing the effectiveness of the algorithm.

The active millimeter-wave (AMMW) scanner has been widely used for inspecting human security in public places in recent years owing to its ability to detect all kinds of objects under the clothes and be harmless to the body. However, it is really challenging to detect all concealed objects automatically and accurately due to inherent imaging noise, unknown object kind, and uncertain position. Recently, many existing methods, especially deep learning-based, have achieved good performances on concealed object detection. These methods work well for detecting a few kinds of large objects, but fail to perform on dim and incomplete hard objects. To address this task, a concealed object detection model with self-paced feature attention fusion network (SPFAFN) is proposed in this article. To be specific, the features with different scales are fused in a top-down manner to integrate details and global semantics to better detect small objects. During fusing multi-scale features, a hierarchical pyramid attention mechanism composed of channel and spatial attention is developed to perceive the object. Moreover, boosting self-paced learning is exploited to guide the model to learn hard samples that are difficultly detected. The proposed method is validated on two real-world datasets: an AMMW dataset and a publicly available passive millimeter-wave (PMMW) dataset. Experimental results demonstrate that the proposed approach is superior to the state-of-the-art methods, and achieves better performances on the two datasets with Average Precision (AP).

Subpixel target detection in hyperspectral imagery is challenging since subpixel targets are smaller in size than the resolution of a single pixel and accurate pixel-level labels on subpixel targets are often unavailable. In particular, this article addresses the problem of learning a prime prototype target signature from imprecisely labeled highly mixed hyperspectral data. Two algorithms, multiple-instance subpixel adaptive cosine estimator (MI-SPACE) and multiple-instance subpixel spectral matched filter (MI-SPSMF), based on multiple-instance learning framework are presented. The proposed methods aim to learn a discriminative prime target signature by maximizing the posterior detection statistics of subpixel hyperspectral targets for the correspondingly proposed subpixel adaptive cosine estimator (SPACE) and subpixel spectral matched filter (SPSMF) detectors, which are also developed in this article. Experimental results demonstrate the effectiveness of the proposed methods on both simulated and real-field hyperspectral subpixel target detection tasks.

In hyperspectral imagery, target detection is challenging since lots of pixels are a mixture of more than one distinct substance and precise pixel-wise labels are often infeasible to obtain. To address this problem, the Multiple Instance Constrained Energy Minimization (MI-CEM) for estimating a discriminative target signature from inaccurately labeled and mixed hyperspectral data is introduced in this paper. The proposed method maximizes the posterior detection statistics of the constrained energy minimization sub-pixel detector and estimates a discriminative target signature. The learned target signature can be applied to CEM for sub-pixel target detection. Experiments on both simulated and real-world data demonstrate that MI-CEM achieves competitive performance compared with the state-of-the-art algorithms.

Non-invasive heart rate estimation is of great importance in daily monitoring of cardiovascular diseases. In this paper, a bidirectional long short term memory (bi-LSTM) regression network is developed for non-invasive heart rate estimation from the ballistocardiograms (BCG) signals. The proposed deep regression model provides an effective solution to the existing challenges in BCG heart rate estimation, such as the mismatch between the BCG signals and ground-truth reference, multi-sensor fusion and effective time series feature learning. Allowing label uncertainty in the estimation can reduce the manual cost of data annotation while further improving the heart rate estimation performance. Compared with the state-of-the-art BCG heart rate estimation methods, the strong fitting and generalization ability of the proposed deep regression model maintains better robustness to noise (e.g., sensor noise) and perturbations (e.g., body movements) in the BCG signals and provides a more reliable solution for long term heart rate monitoring.

Modeling target detection problem given inaccurate annotations as a multiple instance learning (MIL) problem is an effective way for addressing the ground truth uncertainties of remotely sensed hyperspectral imagery. In this paper, we propose a hyperspectral target detection method based on 1D convolution neural network (1DCNN) feature extraction and long short term memory network (LSTM) under the MIL framework, where the LSTM features for each hyperspectral pixel is further refined by a scoring network as to discriminate the real target instance from the inaccurately labeled hyperspectral regions. The proposed method has achieved superior results on both simulated data and real hyperspectral data over the state-of-the-art methods, showing the prospects for further investigation.

For an image retrieval system, the metric to measure the similarity between target images and queries greatly affects its retrieval performance. However, most of the existing metrics are based on single distance metric, which has been shown lack of robustness for different kinds of queries. In this work, we view the metric learning task as an optimization problem for a robust combination of existing metrics, where the objective function is data-driven rather than analytical. Our contribution is two-fold. Firstly, considering the robustness of image retrieval systems, we formulate the optimization problem as a multiobjective optimization with both the average and worst retrieval performance (precision) for different kinds of queries as objectives. Secondly, we apply a popular multi-objective evolutionary algorithm, NSGA-II, to search the optimal combined metric. With the experiment on two different datasets for image retrieval, we find that the proposed algorithm can find combined metrics with better and more robust retrieval performance than other existing single metrics.

In recent years, graph embedding has become a significant technique to deal with feature extraction and dimension reduction problems. Under the linearization and kernelization, it provides a unified framework in machine learning and other pattern recognition tasks. Polarimetric synthetic aperture (PolSAR) as a typical multi-channel sensor can obtain more geometrical and geophysical information. How to combine those polarimetric scattering signals and target decomposition features and explore the spatial information between pixels become a new research direction to address PolSAR data. In this paper, we utilize the tensorial embedding methods to extract the intrinsic features from a redundant feature space for the PolSAR land cover classification. The effectiveness of the proposed methods is demonstrated using AIRSAR Flevoland data set.

As an useful solution of the approximate nearest neighbor (ANN) search, hashing attracts growing attention in the topic of large-scale image retrieval. In this paper, we propose a semi-supervised deep hashing method based on the adversarial autoencoder (AAE) network for remote sensing image retrieval (RSIR), and we name it SSHAAE. Here, we assume the RS images have been represented by the visual features, and the target of our SSHAAE is mapping those features into the binary codes. First, a hashing layer is adopted to replace the part of original latent layer in AAE. In addition, the classical reconstruction loss function is selected to generate the hash code. Second, two discriminators are added simultaneously to make sure the hash code is bit balanced and the generated label variable is one-hot. Third, we design the hash loss function to guarantee the obtained hash code is discriminative, similarity persevering, and low quantization error. The presented SSHAAE model can be trained by the minimax optimization. The encouraging experimental results counted on a high-resolution RS image archive demonstrate our SSHAAE model is effective to RSIR.

Multiple instance learning (MIL) can be used for solving the imprecisely labeled hyperspectral target detection problems, which only needs the label of an area containing some targets. Furthermore, existing methods decompose this task into a target signature learning task and a follow-on similarity measurement between the estimated signature and the test points. In this paper, we propose a deep multiple instance learning method based on self-attention mechanism, in which the max operation and 1D convolution neural network (1D CNN) are adopted to realize an end-to-end hyperspectral target detection structure without learning the target signature. In the proposed deep MIL target detection method, self-attention mechanism with max operation has advantage in estimating the labels of instances from the positive bag via calculating the contribution of each instance to the bag-level classification. The simulated and real hyperspectral target detection experiments are shown to illustrate the performance of the method.

In this paper, two methods for discriminative multiple instance target characterization, MI-SMF and MI-ACE, are presented. MI-SMF and MI-ACE estimate a discriminative target signature from imprecisely-labeled and mixed training data. In many applications, such as sub-pixel target detection in remotely-sensed hyperspectral imagery, accurate pixel-level labels on training data is often unavailable and infeasible to obtain. Furthermore, since sub-pixel targets are smaller in size than the resolution of a single pixel, training data is comprised only of mixed data points (in which target training points are mixtures of responses from both target and non-target classes). Results show improved, consistent performance over existing multiple instance concept learning methods on several hyperspectral sub-pixel target detection problems.

Signature-based detectors for hyperspectral target detection rely on knowing the specific target signature in advance. However, target signatures are often difficult or impossible to obtain. Furthermore, common methods for obtaining target signatures, such as from laboratory measurements or manual selection from an image scene, usually do not capture the discriminative features of target class. In this paper, an approach for estimating a discriminative target signature from imprecise labels is presented. The proposed approach maximizes the response of the hybrid sub-pixel detector within a multiple instance learning framework and estimates a set of discriminative target signatures. After learning target signatures, any signature based detector can then be applied on test data. Both simulated and real hyperspectral target detection experiments are shown to illustrate the effectiveness of the method.

Dictionary Learning Functions of Multiple Instances (DL-FUMI) is proposed to address target detection problems with inaccurate training labels. DL-FUMI is a multiple instance dictionary learning method that estimates target atoms that describe distinctive and representative features of the target class and background atoms that account for the shared features found across both target and non-target data points. Experimental results show that the target atoms estimated by DL-FUMI are more discriminative and representative of the target class than comparison methods. DL-FUMI is shown to have improved performance on several detection problems as compared to other multiple instance dictionary learning algorithms.