Sparse annotations fundamentally constrain multimodal remote sensing: even recent state-of-the-art supervised methods such as MSFMamba are limited by the availability of labeled data, restricting their practical deployment despite architectural advances. ImageNet-pretrained models provide rich visual representations, but adapting them to heterogeneous modalities such as hyperspectral imaging (HSI) and synthetic aperture radar (SAR) without large labeled datasets remains challenging. We propose UniDiff, a parameter-efficient framework that adapts a single ImageNet-pretrained diffusion model to multiple sensing modalities using only target-domain data. Uni-Diff combines FiLM-based timestep-modality conditioning, parameter-efficient adaptation of approximately 5% of parameters, and pseudo-RGB anchoring to preserve pre-trained representations and prevent catastrophic forgetting. This design enables effective feature extraction from remote sensing data under sparse annotations. Our results with two established multi-modal benchmarking datasets demonstrate that unsupervised adaptation of a pre-trained diffusion model effectively mitigates annotation constraints and achieves effective fusion of multi-modal remotely sensed data.

This paper tackles the intricate challenge of video question-answering (VideoQA). Despite notable progress, current methods fall short of effectively integrating questions with video frames and semantic object-level abstractions to create question-aware video representations. We introduce Local - Global Question Aware Video Embedding (LGQAVE), which incorporates three major innovations to integrate multi-modal knowledge better and emphasize semantic visual concepts relevant to specific questions. LGQAVE moves beyond traditional ad-hoc frame sampling by utilizing a cross-attention mechanism that precisely identifies the most relevant frames concerning the questions. It captures the dynamics of objects within these frames using distinct graphs, grounding them in question semantics with the miniGPT model. These graphs are processed by a question-aware dynamic graph transformer (Q-DGT), which refines the outputs to develop nuanced global and local video representations. An additional cross-attention module integrates these local and global embeddings to generate the final video embeddings, which a language model uses to generate answers. Extensive evaluations across multiple benchmarks demonstrate that LGQAVE significantly outperforms existing models in delivering accurate multi-choice and open-ended answers.

Detecting anomalies in videos is a fundamental challenge in machine learning, particularly for applications like disaster management. Leveraging satellite data, with its high frequency and wide coverage, proves invaluable for promptly identifying extreme events such as wildfires, cyclones, or floods. Geostationary satellites, providing data streams at frequent intervals, effectively create a continuous video feed of Earth from space. This study focuses on detecting anomalies, specifically wildfires and smoke, in these high-frequency satellite videos. In contrast to prior endeavors in anomaly detection within surveillance videos, this study introduces a system tailored for high-frequency satellite videos, placing particular emphasis on two anomalies. Unlike the majority of existing Convolution Neural Network-based methods for wildfire detection that rely on labeled images or videos, our unsupervised approach addresses the challenges posed by high-frequency satellite videos with a high intensity of clouds. These Convolution Neural Network-based methods can only identify fires once they have reached a certain size and are susceptible to false positives. We frame the challenge of wildfire detection as a general anomaly detection problem. Introducing an innovative unsupervised approach involving diffusion models, which are state-of-the-art generative models for anomaly detection in satellite videos, we adopt a “generating-to-detecting” strategy. Performance evaluation, measured through AUC-ROC, underscores the superior efficacy of the diffusion model over CNN and Generative Adversarial Networks-based methods in detecting anomalies in these high-frequency satellite videos characterized by a high intensity of clouds. The dataset utilized can be accessed at this location.

In the past three years, there has been significant interest in hyperspectral imagery (HSI) classification using vision Transformers for the analysis of remotely sensed data. Previous research predominantly focused on the empirical integration of convolutional neural networks (CNNs) to augment the network’s capability to extract local feature information. Yet, the theoretical justification for vision Transformers out-performing CNN architectures in HSI classification remains a question. To address this issue, a unified hierarchical spectral vision Transformer architecture, specifically tailored for HSI classification, is investigated. In this streamlined yet effective vision Transformer architecture, multiple mixer modules are strategically integrated separately. These include the CNN mixer, which executes convolutional operations; the spatial self-attention (SSA) mixer and channel self-attention (CSA) mixer, both of which are adaptations of classical self-attention blocks; and hybrid models, such as the SSA + CNN mixer and CSA + CNN mixer, which merge convolution with self-attention operations. This integration facilitates the development of a broad spectrum of vision Transformer-based models tailored for HSI classification. In terms of the training process, a comprehensive analysis is performed, contrasting classical CNN models and vision Transformer-based counterparts, with particular attention to disturbance robustness and the distribution of the largest eigenvalue of the Hessian. From the evaluations conducted on various mixer models rooted in the unified architecture, it is concluded that the unique strength of vision Transformers can be attributed to their overarching architecture, rather than being exclusively reliant on individual multihead self-attention (MSA) components. Extensive experiments demonstrate that the derived vision Transformer models, based on the unified architecture, surpass the classical methods when applied to multiple hyperspectral benchmark datasets.

Hyperspectral image (HSI) classification using vision Transformers is an area of active research in the field of remote sensing. However, vision Transformers suffer from lack of inductive bias inherent in convolutional neural networks (CNNs), which hinders their generalization ability when working with limited amounts of data. Furthermore, some research indicates that the competitive performance of vision Transformers primarily stems from the overall architecture, characterized by alternating residual layers of sequence mixers and multi-layer perceptron (MLP) blocks, rather than solely relying on the multi-head self-attention (MSA) mixer module. To this end, a novel CNN-mixer hierarchical spectral Transformer applied to hyperspectral image classification is proposed. To extract spectral features, a hierarchical spectral Transformer is designed to aggregate discriminative features from the spectral dimension. To remedy the lack of inductive bias inherent in vision Transformers, a simple CNN mixer is substituted for the MSA mixer to capture the local spatial features of each HSI window patch. Accordingly, the proposed network architecture leverages advantages of both CNNs and vision Transformers. Through extensive experiments, we find that the proposed method outperforms the state-of-the-art (SOTA) methods applied to the University of Houston, Botswana, and Salinas HSI datasets.

In geospatial image analysis applications, many factors can result in statistical differences between training and testing/deployment conditions. Domain adaptation techniques aim to reduce these disparsities with the goal of improving image analysis performance. Despite recent progress, some challenges remain, such as insufficient discriminability in the aligned space and negative transfer. We introduce a novel semi-supervised domain adaptation approach that improves the adversarial discriminative domain adaptation framework, addressing these challenges. We validate its effectiveness using two real-world hyperspectral image analysis datasets with varying acquisition conditions.

Hyperspectral imagery (HSI) has emerged as a highly successful sensing modality for a variety of applications ranging from urban mapping to environmental monitoring and precision agriculture. Despite the efforts made by the scientific community, developing reliable algorithms of HSI classification remains a challenging problem, especially for high-resolution HSI data where there is often larger intraclass variability combined with a scarcity of the ground-truth data and class imbalance. In recent years, deep neural networks have emerged as a promising strategy for problems of HSI classification where they have shown remarkable potential for learning joint spectral-spatial features efficiently via backpropagation. In this letter, we propose a deep learning strategy for HSI classification that combines different convolutional neural networks, especially designed to efficiently learn joint spatial-spectral features over multiple scales. Our method achieves an overall classification accuracy of 66.73% on the 2018 IEEE GRSS hyperspectral data set-a high-resolution data set that includes 20 urban land-cover and land-use classes.

How to efficiently exploit useful information from hyperspectral data by joint analysis of spectral and spatial information is an important problem. In this work, we demonstrate a fusion network that can leverage recent developments in graph convolutional neural networks (GCNs) to effectively analyze reflectance spectra of hyperspectral image pixels and two-dimensional convolutional neural networks (CNNs) which can extract object-specific spatial characteristics from hyperspectral image. To design this fusion model, we study both decision fusion and deep feature fusion approaches and show that the fusion approach outperforms other traditional deep learning methods such as variants of CNNs.

With the emergence of passive and active optical sensors available for geospatial imaging, information fusion across sensors is becoming ever more important. An important aspect of single (or multiple) sensor geospatial image analysis is feature extraction-the process of finding “optimal” lower dimensional subspaces that adequately characterize class-specific information for subsequent analysis tasks, such as classification, change and anomaly detection, and so on. In recent work, we proposed and developed an angle-based discriminant analysis approach that projected data onto the subspaces with maximal “angular” separability in the input (raw) feature space and reproducing kernel Hilbert space. We also developed an angular locality preserving variant of this algorithm. Despite being a promising approach, the resulting subspace does not preserve Euclidean distance information. In this letter, we advance this work to address that limitation and make it suitable for information fusion-we propose and validate a composite kernel-based subspace learning framework that simultaneously preserves Euclidean and angular information, which can operate on an ensemble of feature sources (e.g., from different sources). We validate this method with the multisensor University of Houston hyperspectral and light detection and ranging data set, and demonstrate that a joint discriminant analysis that leverages angular and Euclidean distance information provides superior classification and sensor (information) fusion performance.

Deep neural networks have been shown to be useful for the classification of hyperspectral images, particularly when a large amount of labeled data is available. However, we may not have enough reference data to train a deep neural network for many practical geospatial image analysis applications. To address this issue, in this paper, we propose to use a deep feature alignment neural network to carry out the domain adaptation, where the labeled data from a supplementary data source can be utilized to improve the classification performance in a domain where otherwise limited labeled data are available. In the proposed model, discriminative features for the source and target domains are first extracted using deep convolutional recurrent neural networks and then aligned with each other layer-by-layer by mapping features from each layer to transformed common subspaces at each layer. Experimental results are presented with two data sets. One of these data sets represents domain adaptation between images acquired at different times, while the other data set represents a very unique and challenging domain adaptation problem, representing source and target images that are acquired using different hyperspectral imagers that collect data from different viewpoints and platforms (a ground-based forward-looking street view of objects acquired at the close range and an aerial hyperspectral image). We demonstrate that the proposed deep learning framework enables the robust classification of the target domain data by leveraging information from the source domain.

Deep learning has gained popularity in a variety of computer vision tasks. Recently, it has also been successfully applied for hyperspectral image classification tasks. Training deep neural networks, such as a convolutional neural network for classification requires a large number of labeled samples. However, in remote sensing applications, we usually only have a small amount of labeled data for training because they are expensive to collect, although we still have abundant unlabeled data. In this paper, we propose semi-supervised deep learning for hyperspectral image classification-our approach uses limited labeled data and abundant unlabeled data to train a deep neural network. More specifically, we use deep convolutional recurrent neural networks (CRNN) for hyperspectral image classification by treating each hyperspectral pixel as a spectral sequence. In the proposed semi-supervised learning framework, the abundant unlabeled data are utilized with their pseudo labels (cluster labels). We propose to use all the training data together with their pseudo labels to pre-train a deep CRNN, and then fine-tune using the limited available labeled data. Further, to utilize spatial information in the hyperspectral images, we propose a constrained Dirichlet process mixture model (C-DPMM), a non-parametric Bayesian clustering algorithm, for semi-supervised clustering which includes pairwise must-link and cannot-link constraints-this produces high-quality pseudo-labels, resulting in improved initialization of the deep neural network. We also derived a variational inference model for the C-DPMM for efficient inference. Experimental results with real hyperspectral image data sets demonstrate that the proposed semi-supervised method outperforms state-of-the-art supervised and semi-supervised learning methods for hyperspectral classification.

Sparse representation based classification has gained popularity with geospatial image analysis in general and hyperspectral image analysis in particular. A central idea with such classification approaches is that a test pixel (spectral reflectance vector) can be sparsely represented in a training dictionary of pixels from all classes - in particular, only training pixels in the dictionary that bear the same class membership of the test pixel will contribute significant coefficients in the sparse representation. The traditional applications of such classifiers to hyperspectral imagery utilize pixel (sample) level information, not spatial contextual information. We propose a sparse representation based classification paradigm that effectively and optimally captures the key geometric properties in hyperspectral images - our classifier that is built on this structured sparse representation then offers very robust classification, including in scenarios where training and test objects have rotational variations (a common occurrence with geospatial images). We validate the proposed approach with benchmark hyperspectral data and present results demonstrating the efficacy of the proposed method.

In this paper, we propose a novel domain adaptation method for the classification of hyperspectral images. The proposed method projects the samples from both the source and target domain into a common latent space, where the ratio of within-class distance to between-class distance is minimized. We present a probabilistic framework to learn such transformations and solve the problem with an alternating maximizing algorithm, resulting in a discriminative subspace. The proposed method is evaluated on a challenging real-world hyperspectral dataset, and the experimental results demonstrate the efficacy of the proposed method compared to conventional domain adaptation methods.

Classification of hyperspectral images has recently gained significant popularity due to both the development of remote sensing technologies and the advances in image analysis approaches. One crucial step to achieve accurate classification is to acquire sufficient high-quality training data, which is often a time-consuming and expensive process. To alleviate this burden, in this letter, we propose an active and semisupervised learning (SSL) approach that utilizes morphological component analysis (MCA) for classification of hyperspectral images. First, the original hyperspectral data are decomposed into its morphological components via MCA. In each feature domain, the active learning (AL) and SSL are combined to enlarge the training data set based on superpixels. Finally, decision fusion is carried out to integrate the predictions from the two components. The proposed method is tested on both benchmark and real world application hyperspectral data sets. Experimental results indicate that the proposed method can lead to a better classification with respect to the conventional AL approaches.

Hyperspectral imagery has emerged as a popular sensing modality for a variety of applications, and sparsity-based methods were shown to be very effective to deal with challenges coming from high dimensionality in most hyperspectral classification problems. In this paper, we challenge the conventional approach to hyperspectral classification that typically builds sparsity-based classifiers directly on spectral reflectance features or features derived directly from the data. We assert that hyperspectral image (HSI) processing can benefit very significantly by decoupling data into geometrically distinct components since the resulting decoupled components are much more suitable for sparse representation-based classifiers. Specifically, we apply morphological separation to decouple data into texture and cartoon-like components, which are sparsely represented using local discrete cosine bases and multiscale shearlets, respectively. In addition to providing a structured sparse representation, this approach allows us to build classifiers with invariance properties specific to each geometrically distinct component of the data. The experimental results using real-world HSI data sets demonstrate the efficacy of the proposed framework for classifying multichannel imagery under a variety of adverse conditions-in particular, small training sample size, additive noise, and rotational variabilities between training and test samples.

Active learning is an area of significant ongoing research interest for the classification of remotely sensed data, where obtaining efficient training data is both time consuming and expensive. The goal of active learning is to achieve high classification performance by querying as few samples as possible from a large unlabeled data pool. Traditional active learning frameworks all assume the existence of labeled samples for all classes of interest. However, in real-world applications, the unlabeled data pool may contain data from unknown classes that we are not aware of in advance, and a quick detection of them is useful for enriching our training set. In this scenario, traditional active learning methods may not effectively and rapidly detect the unknown classes. We proposed an active learning framework which provides robust classification performance with minimum manual labeling effort while simultaneously discovering unknown (missing) classes. The discovery of unknown classes is particularly suited to an active learning framework where an annotator is in the loop. A Dirichlet process mixture model is utilized in our proposed method to cluster the labeled and unlabeled samples as a whole. If unknown classes exist, they will emerge as new clusters which are different from other existing clusters occupied by known classes, and then, the proposed query strategy will give priority to querying samples in the new clusters. We present experimental results with hyperspectral data to show that our method provides better classification performance compared to existing active learning methods with or without unknown classes.

In this paper, we propose an adaptive locality weighted multisource joint sparse representation classification (ALWMJ-SRC) model for the classification of multisource remote sensing data. Although the notion of multitask joint sparsity has been recently developed for data fusion and has shown to be effective for various applications, in this paper, we suggest that there are important limitations stemming from the assumptions in such a framework. We propose a formulation that is inspired by this approach yet addresses some of the key shortcomings (e.g., uniform weights and unstable estimation of coefficients), resulting in a more robust formulation for data fusion. Specifically, we impose an adaptive locality weight to constrain the sparse coefficients, which not only considers the locality information between the test sample and the atoms in the dictionary but also helps ensure that the coefficients are adaptively penalized, reducing estimation bias. The adaptive locality weight is calculated for each source, which ensures that complementary information is employed from different sources for fusion. The optimization problem is solved using an alternating-direction-methods-of-multipliers formulation. In addition, the proposed algorithm is extended to the kernel space. The efficacy of the proposed algorithm is validated via experiments for two fusion scenarios-spectral-spatial classification and hyperspectral-LiDAR sensor fusion. The experimental results demonstrate that ALWMJ-SRC consistently performs better than state-of-the-art classification approaches.

There is increasing interest in driving supervised classification of hyperspectral imagery by a support vector machine using a composite kernel employing both spectral and spatial features. While the spectral signature of the current hyper-spectral pixel is often used directly to supply the spectral feature, a statistic - such as the mean - calculated across a spatial window surrounding the pixel is typically employed as a spatial feature. In contrast, a nearest-neighbor spatial feature is proposed in which the nearest neighbors in Euclidean distance to the current pixel are used to calculate the spatial feature. It is argued that the proposed nearest-neighbor spatial feature is more likely to incorporate relevant, same-class neighbor pixels than window-based features for which borders between coherent single-class regions may give rise to misclassification. Experimental results illustrate the performance advantage of the proposed nearest-neighbor framework at supervised hyperspectral classification in comparison to several competing benchmark algorithms that also employ kernel-based support vector machines.

This letter presents a novel superpixel-based approach to hyperspectral image analysis which exploits spatial context within spectrally similar contiguous pixels for robust hyperspectral classification. The proposed approach entails two key steps-first, as a preprocessing step, we compute groupings (superpixels) through graph-based segmentation, following which an object-level classification is undertaken using a decision fusion approach that merges per-pixel outcomes from an ensemble of “per-pixel” Bayesian classifiers. The proposed method provides a robust way to exploit spatial contextual information. Every pixel in a superpixel is classified using statistical Bayesian classification independently, and the decisions are merged to obtain a unique class label for each superpixel. Experimental results with hyperspectral imagery indicate that the proposed method consistently provides a robust classification framework, even when using very limited training data.

Remote sensing is one of the most common ways to extract relevant information about Earth and our environment. Remote sensing acquisitions can be done by both active (synthetic aperture radar, LiDAR) and passive (optical and thermal range, multispectral and hyperspectral) devices. According to the sensor, a variety of information about the Earth's surface can be obtained. The data acquired by these sensors can provide information about the structure (optical, synthetic aperture radar), elevation (LiDAR), and material content (multispectral and hyperspectral) of the objects in the image. Once considered together their complementarity can be helpful for characterizing land use (urban analysis, precision agriculture), damage detection (e.g., in natural disasters such as floods, hurricanes, earthquakes, oil spills in seas), and give insights to potential exploitation of resources (oil fields, minerals). In addition, repeated acquisitions of a scene at different times allows one to monitor natural resources and environmental variables (vegetation phenology, snow cover), anthropological effects (urban sprawl, deforestation), climate changes (desertification, coastal erosion), among others. In this paper, we sketch the current opportunities and challenges related to the exploitation of multimodal data for Earth observation. This is done by leveraging the outcomes of the data fusion contests, organized by the IEEE Geoscience and Remote Sensing Society since 2006. We will report on the outcomes of these contests, presenting the multimodal sets of data made available to the community each year, the targeted applications, and an analysis of the submitted methods and results: How was multimodality considered and integrated in the processing chain? What were the improvements/new opportunities offered by the fusion? What were the objectives to be addressed and the reported solutions? And from this, what will be the next challenges?

Sparse representation of signals for classification is an active research area. Signals can potentially have a compact representation as a linear combination of atoms in an overcomplete dictionary. Based on this observation, a sparse-representation-based classification (SRC) has been proposed for robust face recognition and has gained popularity for various classification tasks. It relies on the underlying assumption that a test sample can be linearly represented by a small number of training samples from the same class. However, SRC implementations ignore the Euclidean distance relationship between samples when learning the sparse representation of a test sample in the given dictionary. To overcome this drawback, we propose an alternate formulation that we assert is better suited for classification tasks. Specifically, class-dependent sparse representation classifier (cdSRC) is proposed for hyperspectral image classification, which effectively combines the ideas of SRC and K-nearest neighbor classifier in a classwise manner to exploit both correlation and Euclidean distance relationship between test and training samples. Toward this goal, a unified class membership function is developed, which utilizes residual and Euclidean distance information simultaneously. Experimental results based on several real-world hyperspectral data sets have shown that cdSRC not only dramatically increases the classification performance over SRC but also outperforms other popular classifiers, such as support vector machine.

A tenet of object classification is that accuracy improves with an increasing number (and variety) of spectral channels available to the classifier. Hyperspectral images provide hundreds of narrowband measurements over a wide spectral range, and offer superior classification performance over color images. However, hyperspectral data is highly redundant. In this paper we suggest that only 6 measurements are needed to obtain classification results comparable to those realized using hyperspectral data. We present classification results for a natural scene using three imaging modalities: 1) using three broadband color filters (RGB) and three narrowband samples, 2) using six narrowband samples, and 3) using six commonly available optical filters. If these results hold for larger datasets of natural images, recently proposed multispectral image sensors [1, 2] can be used to offer material classification results equal to that of hyperspectral data.

Multiple views of a scene - obtained via different sensing modalities - have the potential to significantly enhance image analysis for remote sensing and other applications. This benefit is expected to be significant if the multiple views are providing independent, yet useful, information about the underlying classes in a scene. To exploit such multi-sensor information, a compressive-projection approach to the fusion of multi-sensor imagery is proposed. It is argued that that random projections yield subspaces that preserve the discriminative nature of multi-sensor datasets with profound implications in a practical scenario wherein compressive measurements can directly facilitate data fusion without the need for complicated subspace-learning approaches. A case study fusing experimental hyperspectral and LiDAR data demonstrates that statistical learning in the compressive-measurement domain is not only feasible, but also provides a natural framework for sensor fusion without the need for explicit reconstruction from compressive measurements.

Novelty (new class) detection can be described as the identification of new or “unknown” data that a machine learning system was not aware of during training. The ability to detect new classes can have a significant positive impact on image analysis systems, where the test data (or unlabeled data) may contain information about objects that were not known during training process. Since infinite Gaussian mixture models (IGMM) are capable to fit data with an unknown number of mixtures, the inference scheme based on semi-supervised Gibbs sampling can differentiate between known and novel data by learning the unique data clustering in training and testing modes. In order to deal with non-Gaussian (especially heavy tailed) data, the proposed approach is based on infinite warped mixture models (IWMM). IWMM models assume that each observation has coordinates in a latent space where the data is Gaussian distributed - an IGMM is then learned in that latent space instead. We show that the IWMM model outperforms an IGMM based approach to novelty detection for hyperspectral image analysis.

In this paper, we study a novel approach to active learning in the wavelet domain for classification of Full-waveform LiDAR (FWL) data. Unlike discrete 3-dimensional points obtained from a traditional discrete return LiDAR system, FWL systems have the capability to record the entire backscat-tered signal, which contains additional information about the reflecting objects. With such LiDAR systems, the vertical structure of reflectors is effectively characterized by the shape of the return pulse. Instead of deriving simple structure and statistics-based features, such as pulse amplitude and width, skewness, and kurtosis from the FWL data, in this work, wavelet features are extracted via a Redundant Discrete Wavelet Transform (RDWT) and are then utilized for classification in a multi-view active learning (AL) framework. Additionally, we demonstrate that the proposed approach provides a noise robust framework for analysis and classification of low Signal-to-Noise (SNR) LiDAR data. Experimental results demonstrate the efficacy of the proposed wavelet-based active learning for FWL data.

The Gaussian mixture model is a well-known classification tool that captures non-Gaussian statistics of multivariate data. However, the impractically large size of the resulting parameter space has hindered widespread adoption of Gaussian mixture models for hyperspectral imagery. To counter this parameter-space issue, dimensionality reduction targeting the preservation of multimodal structures is proposed. Specifically, locality-preserving nonnegative matrix factorization, as well as local Fisher's discriminant analysis, is deployed as preprocessing to reduce the dimensionality of data for the Gaussian-mixture-model classifier, while preserving multimodal structures within the data. In addition, the pixel-wise classification results from the Gaussian mixture model are combined with spatial-context information resulting from a Markov random field. Experimental results demonstrate that the proposed classification system significantly outperforms other approaches even under limited training data.

The same high dimensionality of hyperspectral imagery that facilitates detection of subtle differences in spectral response due to differing chemical composition also hinders the deployment of traditional statistical pattern-classification procedures, particularly when relatively few training samples are available. Traditional approaches to addressing this issue, which typically employ dimensionality reduction based on either projection or feature selection, are at best suboptimal for hyperspectral classification tasks. A divide-and-conquer algorithm is proposed to exploit the high correlation between successive spectral bands and the resulting block-diagonal correlation structure to partition the hyperspectral space into approximately independent subspaces. Subsequently, dimensionality reduction based on a graph-theoretic locality-preserving discriminant analysis is combined with classification driven by Gaussian mixture models independently in each subspace. The locality-preserving discriminant analysis preserves the potentially multimodal statistical structure of the data, which the Gaussian mixture model classifier learns in the reduced-dimensional subspace. Experimental results demonstrate that the proposed system significantly outperforms traditional classification approaches, even when few training samples are employed.

In this letter, a fusion-classification system is proposed to alleviate ill-conditioned distributions in hyperspectral image classification. A windowed 3-D discrete wavelet transform is first combined with a feature grouping-a wavelet-coefficient correlation matrix (WCM)-to extract and select spectral-spatial features from the hyperspectral image dataset. The adjacent wavelet-coefficient subspaces (from the WCM) are intelligently grouped such that correlated coefficients are assigned to the same group. Afterwards, a multiclassifier decision-fusion approach is employed for the final classification. The performance of the proposed classification system is assessed with various classifiers, including maximum-likelihood estimation, Gaussian mixture models, and support vector machines. Experimental results show that with the proposed fusion system, independent of the classifier adopted, the proposed classification system substantially outperforms the popular single-classifier classification paradigm under small-sample-size conditions and noisy environments.

The one-against-one (OAO) strategy is commonly employed with classifiers-such as support vector machines-which inherently provide binary two-class classification in order to handle multiple classes. This OAO strategy is introduced for the classification of hyperspectral imagery using discriminant analysis within kernel-induced feature spaces, producing a pair of algorithms-kernel discriminant analysis and kernel local Fisher discriminant analysis-for dimensionality reduction, which are followed by a quadratic Gaussian maximum-likelihood-estimation classifier. In the proposed approach, a multiclass problem is broken down into all possible binary classifiers, and various decision-fusion rules are considered for merging results from this classifier ensemble. Experimental results using several hyperspectral data sets demonstrate the benefits of the proposed approach-in addition to improved classification performance, the resulting classifier framework requires reduced memory for estimating kernel matrices.

A classifier that couples nearest-subspace classification with a distance-weighted Tikhonov regularization is proposed for hyperspectral imagery. The resulting nearest-regularized-subspace classifier seeks an approximation of each testing sample via a linear combination of training samples within each class. The class label is then derived according to the class which best approximates the test sample. The distance-weighted Tikhonov regularization is then modified by measuring distance within a locality-preserving lower-dimensional subspace. Furthermore, a competitive process among the classes is proposed to simplify parameter tuning. Classification results for several hyperspectral image data sets demonstrate superior performance of the proposed approach when compared to other, more traditional classification techniques.

Sparse representation is an active research area in the signal processing and machine learning community in recent years. Recently, sparse representation classifier was proposed for challenging classification tasks - it entails representing a testing sample as a linear combination of all training samples which form an over-complete dictionary. In this paper, we demonstrate that for challenging high-dimensional classification tasks, appropriate dimensionality reduction is beneficial for sparse representation classifiers and it's variants - especially when some features are redundant and/or lack discriminatory power. We propose a new dimensionality reduction algorithm to optimize the performance of greedy pursuit algorithms (required in sparse representation classifiers) by projecting the data into a space where the ratio of intra-class to inter class inner products are maximized. We demonstrate the superiority of the proposed method with standard hyperspectral imagery datasets - both in terms of improved classification accuracy and a speed-up in the run-time.

Current research into the fusion of Hyperspectral Imagery (HI) and full waveform LiDAR (Light detection and ranging) has relied on first processing the full waveform LiDAR (FWL) data to a set of discrete returns before combining. However, more information about target properties can potentially be recovered if the raw waveform is preserved in the fusion with HI. This paper proposes a voxelization method to fuse raw FWL data with HI by dividing the waveform data into voxels, and then synthesizing all waveforms which intersect a voxel into one 3D superposition waveform. The efficacy of this method is evaluated by comparing the synthesized waveform with an actual nadir LiDAR waveform from the voxel of interest. Results show that this method of voxelizing and fusion of FWL data can preserve raw waveform characteristics while effectively representing the FWL data on a 3D raster basis that can be directly co-registered with the HI.

In this work, we propose a new density estimation method for hyperspectral image data based on Dirichlet Process Gaussian mixture models (also known as infinite Gaussian mixture models - IGMMs), which successfully captures the complex multi-modal (potentially non-Gaussian) statistical structure of hyperspectral data. The mixture model we get from this will then be applied to the classification problem. This IGMM based approach is a non-parametric Bayesian method helping circumvent the problem of model selection, which is unavoidable and often difficult when employing traditional parametric Gaussian mixture models (GMM). Inference model based on Gibbs sampling employed during the inference of model parameters. As a preprocessing step, we use Local Fisher's Discriminant Analysis (LFDA) for dimension reduction since we expect it to preserve the multi-modal non-Gaussian structure of the hyperspectral data, which will benefit much in the aspect of computation cost. We compared our proposed IGMM based classification method to the existing state-of-the-art classification methods using popular hyperspectral imagery datasets. The results of our experiments show that the proposed LFDA-IGMM method and GMM method have almost the same performance (sometimes outperforming LFDA-GMM), and they outperform the other commonly used classification approaches when there is a sufficient number of training samples.

Exploiting disparate features from potentially different data sources with multiple-kernel based machine learning is a promising approach for analyzing geo-spatial data. A mixture-of-kernel approach can facilitate construction of a more effective training data pool with Active Learning (AL). In addition, this could alleviate the computational burden in AL implementations. Kernel based learning requires hyperparameter tuning for model selection. Further, an optimal function is required to integrate different features or data sources appropriately in the kernel induced space. Both kernel parameters and kernel combination functions may need to be tuned at each AL learning step, which is potentially very time-consuming. In this paper, a novel multiple kernel active learning algorithm is proposed that promises enhanced classification, improved AL performance, and a mechanism for automatic selection of kernel weights in the mixture-of-kernels. We demonstrate the usefulness of the proposed framework with results for both feature fusion and sensor fusion tasks.

Linear discriminant analysis (LDA) is a popular approach for dimensionality reduction for pattern classification; however, its performance is often degraded when samples are too few, particularly when the dimensionality of the input feature space is excessively high. The classic solution to the small-sample-size problem is to implement LDA in a principal component (PC) subspace, i.e., a strategy known as subspace LDA. This latter approach is extended by coupling LDA and noise-adjusted HSI analysis in order to provide noise-robust feature extraction and classification of high-dimensional data. An extension of the proposed approach in a kernel-induced space is also studied. The resulting noise-adjusted subspace discriminant analysis is evaluated using hyperspectral imagery, with experimental results demonstrating that the proposed approach provides not only superior classification performance, as compared with traditional methods, but also effective dimensionality reduction for classification even in the presence of noise.

Compressive-projection principal component analysis (CPPCA) has been developed to provide reconstruction from random projections of hyperspectral pixels and then subsequently extended by coupling it with classification such that the resulting class-dependent CPPCA yielded improved reconstruction performance. This letter provides an even greater integration of spatial and spectral information to further improve reconstruction performance. Specifically, instead of a pixel-based modulo partitioning employed by the original CPPCA sender, this work proposes an alternative block-based modulo partitioning, which preserves local spatial coherence; spatial segmentation is combined with the pixel-wise classification results using a majority voting rule at the receiver. Experimental results demonstrate not only improved reconstruction performance but also better detection of anomalies, as compared with previous approaches.

There is increasing interest in dimensionality reduction through random projections due in part to the emerging paradigm of compressed sensing. It is anticipated that signal acquisition with random projections will decrease signal-sensing costs significantly; moreover, it has been demonstrated that both supervised and unsupervised statistical learning algorithms work reliably within randomly projected subspaces. Capitalizing on this latter development, several class-dependent strategies are proposed for the reconstruction of hyperspectral imagery from random projections. In this approach, each hyperspectral pixel is first classified into one of several pixel groups using either a conventional supervised classifier or an unsupervised clustering algorithm. After the grouping procedure, a suitable reconstruction method, such as compressive projection principal component analysis, is employed independently within each group. Experimental results confirm that such class-dependent reconstruction, which employs statistics pertinent to each class as opposed to the global statistics estimated over the entire data set, results in more accurate reconstructions of hyperspectral pixels from random projections.

Hyperspectral imaging (HSI) techniques have been widely used for a variety of applications pertaining to vegetation species identification. With its rich spectral information, HSI is a powerful tool to detect and characterize vegetation species and their health. However, due to the high dimensionality of HSI, a the number of training samples required to estimate the parameters of the automated target recognition (ATR) or ground-cover classification algorithms is large. To avoid this over-dimensionality problem, feature selection or feature extraction must be performed to reduce the dimensionality of HSI data. This problem is further exacerbated when spatial information is also exploited in conjunction with spectral information. In this work, we propose a feature selection approach for extracting the most meaningful spatial and spectral features for a vegetative stress detection problem - genetic algorithms based linear discriminant analysis (GA-LDA). Experimental results show that applying GA with an appropriate fitness function in the spatial-spectral feature space is very effective at selecting the most pertinent features and yields very high classification accuracies.

This paper explores different types of gray level co-occurrence matrix (GLCM) [2] texture features for automated detection of landslides on levees using remotely sensed Synthetic Aperture Radar (SAR). Two approaches of texture analysis are investigated: one based on a rubber band straightening transform (RBST) which has been used extensively in the past in the medical imaging community, and one based on the authors' developed approach of spiral straightening transform (SST). The transforms are used to project a circular region in the image to a rectangular representation where texture feature extraction can be applied. Straightforward linear discriminant analysis, for feature reduction and optimization, and maximum likelihood methods, for classification, are also utilized. The proposed system was tested on L-band SAR data with HH, HV, and VV polarizations collected from NASA's UAVSAR of the Mississippi River levee system between Vicksburg, MS and Clarksdale, MS, USA. The proposed approach is shown to detect all known levee landslides in the test area with a low number of false positives.

Feature extraction based on nonnegative matrix factorization is considered for hyperspectral image classification. One shortcoming of most remote-sensing data is low spatial resolution, which causes a pixel to be mixed with several pure spectral signatures, or endmembers. To counter this effect, locality-preserving nonnegative matrix factorization is employed in order to extract an endmembers-based feature representation as well as to preserve the intrinsic geometric structure of hyperspectral data. Subsequently, a Gaussian mixture model classifier is employed in the induced-feature subspace. Experimental results demonstrate that the proposed classification system significantly outperforms traditional approaches even in instances of limited training data and severe pixel mixing.

Locality-preserving projection as well as local Fisher discriminant analysis is applied for dimensionality reduction of hyperspectral imagery based on both spatial and spectral information. These techniques preserve the local geometric structure of hyperspectral data into a low-dimensional subspace wherein a Gaussian-mixture-model classifier is then considered. In the proposed classification system, local spatial information—which is expected to be more multimodal than strictly spectral features—is used. Results with experimental hyperspectral data demonstrate that this system outperforms traditional classification approaches.

Hyperspectral imagery typically provides a wealth of information captured in a wide range of the electromagnetic spectrum for each pixel in the image; however, when used in statistical pattern-classification tasks, the resulting high-dimensional feature spaces often tend to result in ill-conditioned formulations. Popular dimensionality-reduction techniques such as principal component analysis, linear discriminant analysis, and their variants typically assume a Gaussian distribution. The quadratic maximum-likelihood classifier commonly employed for hyperspectral analysis also assumes single-Gaussian class-conditional distributions. Departing from this single-Gaussian assumption, a classification paradigm designed to exploit the rich statistical structure of the data is proposed. The proposed framework employs local Fisher's discriminant analysis to reduce the dimensionality of the data while preserving its multimodal structure, while a subsequent Gaussian mixture model or support vector machine provides effective classification of the reduced-dimension multimodal data. Experimental results on several different multiple-class hyperspectral-classification tasks demonstrate that the proposed approach significantly outperforms several traditional alternatives.

Hyperspectral imagery comprises high-dimensional reflectance vectors representing the spectral response over a wide range of wavelengths per pixel in the image. The resulting high-dimensional feature spaces often result in statistically ill-conditioned class-conditional distributions. Conventional methods for alleviating this problem typically employ dimensionality reduction such as linear discriminant analysis along with single-classifier systems, yet these methods are suboptimal and lack noise robustness. In contrast, a divide-and-conquer approach is proposed to address the high dimensionality of hyperspectral data for effective and noise-robust classification. Central to the proposed framework is a redundant wavelet transform for representing the data in a feature space amenable to noise-robust multiscale analysis as well as a multiclassifier and decision-fusion system for classification and target recognition in high-dimensional spaces under small-sample-size conditions. The proposed partitioning of this feature space assigns a collection of all coefficients across all scales at a particular spectral wavelength to a dedicated classifier. It is demonstrated that such a partitioning of the feature space for a multiclassifier system yields superior noise performance for classification tasks. Additionally, validation studies with experimental hyperspectral data show that the proposed system significantly outperforms conventional denoising and classification approaches.

Remotely sensed data (such as hyperspectral imagery) is typically associated with a large number of features, which makes classification challenging. Feature subset selection is an effective approach to alleviate the curse of dimensionality when the number of features contained in datasets is huge. Considering the merits of genetic algorithms (GA) in solving combinatorial problems, GA is becoming an increasingly popular tool for feature subset selection. Most algorithms presented in the literature using GA for feature subset selection use the training classification accuracy of a specific algorithm as the fitness function to optimize over the space of possible feature subsets. Such algorithms require a large amount of time to search for an optimal feature subset. In this paper, we will present a new approach called Genetic Algorithm based Linear Discriminant Analysis (GA-LDA) to extract features in which feature selection and feature extraction are performed simultaneously to alleviate over-dimensionality and result in a useful and robust feature space. Experimental results with classification tasks involving both hyperspectral imagery and SAR data indicate that GA-LDA can result in very low-dimensional feature subspaces yielding high classification accuracies.

Ground cover classification using remotely sensed hyperspectral data is a challenging pattern recognition problem. The small (and expensive to collect) training sample sizes exacerbate the curse-of-dimensionality problem that already exists with such high dimensional feature spaces. However, Support Vector Machine (SVM) classifiers have been demonstrated to be better at handling such situations compared to other statistical classifiers. Recently, multi-classifier systems and a uniform random feature selection have proved to be very effective for hyperspectral image classification. In this paper, a support vector machines based multi-classifier system with non-uniform (spectrally-constrained) random feature selection is presented. We propose two approaches to perform such a non-uniform random-feature selection. Experimental results with the AVIRIS Indian Pines hyperspectral data demonstrate that the proposed approach outperforms regular random feature selection based on a uniform distribution.

Feature selection (FS) is a classical combinatorial problem in pattern recognition and data mining. It finds major importance in classification and regression scenarios. In this paper, a hybrid approach that combines branch-and-bound (BB) search with Bhattacharya distance based feature selection is presented for classifying hyperspectral data using Support Vector Machine (SVM) classifiers. The performance of this hybrid approach is compared to another hybrid approach that uses genetic algorithm (GA) based feature selection in place of BB. It is also compared to baseline SVMs with no feature reduction. Experimental results using hyperspectral data show that under small sample size situations, BB approach performs better than GA and SVM with no feature selection.

Linear discriminant analysis (LDA) has been widely applied for hyperspectral image (HSI) analysis as a popular method for feature extraction and dimensionality reduction. Linear methods such as LDA work well for unimodal Gaussian class-conditional distributions. However, when data samples between classes are nonlinearly separated in the input space, linear methods such as LDA are expected to fail. The kernel discriminant analysis (KDA) attempts to address this issue by mapping data in the input space onto a subspace such that Fisher's ratio in an intermediate (higher-dimensional) kernel-induced space is maximized. In recent studies with HSI data, KDA has been shown to outperform LDA, particularly when the data distributions are non-Gaussian and multimodal, such as when pixels represent target classes severely mixed with background classes. In this letter, a modified KDA algorithm, i.e., kernel local Fisher discriminant analysis (KLFDA), is studied for HSI analysis. Unlike KDA, KLFDA imposes an additional constraint on the mapping-it ensures that neighboring points in the input space stay close-by in the projected subspace and vice versa. Classification experiments with a challenging HSI task demonstrate that this approach outperforms current state-of-the-art HSI-classification methods.

The per-pixel spectral information present in hyperspectral imagery (HSI) is typically of very high dimensionality due to the presence of hundreds of continuous narrow spectral bands. Although such high dimensional data has the potential of providing useful information for land-cover classification and mapping tasks, it is often also likely to result in ill-conditioned statistical formulations and reduced performance due to over-dimensionality problems. Much of the research in HSI analysis attempts to find appropriate dimensionality reduction and classification techniques that best exploit this high dimensional imagery. Conventional approaches to dimensionality reduction and classification look at the HSI holistically and attempt to find projections and decision rules that optimize a global criterion, such as the overall accuracy, Fisher's ratio over all classes etc. In this paper, we propose an adaptation strategy that adapts conventional classifiers to re-focus on hard to recognize classes. After an appropriate “holistic” feature selection, the proposed adaptation helps identify additional features that best separate the most “confused” class pairs in the dataset. We demonstrate this data-dependent adaptation of conventional feature selection and classification methods results in improved classification performance.

Imaging spectrometers can acquire spectrally resolved images over a wide range of the electromagnetic spectrum. Availability of this rich spectral information makes it possible to design classification systems that can perform very accurate ground cover classification and target recognition. The Hyperspectral Infrared Imager (HyspIRI) - a National Research Council (NRC) decadal survey mission is much anticipated by researchers to aid in answering a wide variety of global ecological and anthropological questions. In this work, we study the efficacy of HyspIRI observations in precision vegetation mapping applications under limited ground truth availability. In particular, sensitivity to mixed pixel conditions, temporal misalignments and the amount of training (ground-truth) data are studied for various state-of-the-art classification systems. This study will provide valuable insight on the relationship between the quality and quantity of available ground truth and performance of classification systems with these HyspIRI observations.

The developments in sensor technology have made the high-resolution hyperspectral remote sensing data available to the remote sensing analyst for ground-cover classification and target recognition tasks. The inherent high dimensionality of such data sets and the limited ground-truth data availability in many real-life operating scenarios necessitate such hyperspectral classification systems to employ the dimensionality reduction algorithms. Previously, it has been shown that the addition of the spectral derivatives into the feature space improves the performance of the hyperspectral image analysis systems. Although the spectral derivative features are expected to provide additional information for the classification task at hand, the conventional classification techniques are typically not suitable for such fusion since simply combining these features would result in very high dimensional feature spaces, exacerbating the over-dimensionality problem. In this paper, we propose an effective approach for the decision-level fusion of the spectral reflectance information with the spectral derivative information for robust land cover classification. This paper differs from previous work because we propose effective classification strategies to alleviate the increased over-dimensionality problem introduced by the addition of the spectral derivatives for hyperspectral classification. The studies reported in this paper are conducted within the context of both single and multiple classifier systems that are designed to handle the high-dimensional feature spaces. The experimental results are reported with handheld, airborne, and spaceborne hyperspectral data. The efficacy of the proposed approaches (using spectral derivatives and single or multiple classifiers) as quantified by the overall classification accuracy (expressed in percentage) is significantly greater than that of these systems when exploiting only the reflectance information.

An imaging spectrometer's dense recording of radiance values (and consequently derived reflectance values) over a wide region of the electromagnetic spectrum provides the potential to design classification systems that can perform highly accurate ground cover classification and target recognition. The Hyperspectral Infrared Imager (HyspIRI) - a National Research Council (NRC) decadal survey mission is much anticipated by researchers to aid in answering a wide variety of global ecological and anthropological questions. In order to tackle these research topics and effectively exploit the seasonal global imaging spectroscopy provided by HyspIRI, there will be a great need for reliable hyperspectral-based products for use by domain experts. In this work, we create simulated/proxy HyspIRI data from a database of hyperspectral signatures of various vegetation species. We then study the performance of current state-of-the-art pattern classification paradigms for classifying this proxy data. The outcome of this study will provide valuable insight into the potential efficacy of employing HyspIRI data for vegetation mapping and similar remotely sensed pattern classification tasks.

In this study, the authors investigate the combination of the wavelet packet decomposition (WPD) and multiclassifiers and decision fusion (MCDF) for a robust hyperspectral classification system. The authors investigate the use of the WPD multiresolution feature grouping and selection, forming groups of local and global spectral features, where each group is input to a classifier, resulting in local and global classifications. Then the labels are fused to form one class label. The classification system was applied to hyperspectral data for an agricultural application, namely the detection of different soybean rust infestation levels. The system was compared to current state-of-the-art hyperspectral analysis techniques to determine its comparative efficacy as compared to more conventional approaches, such as stepwise-linear discriminant analysis (LDA) or discriminant analysis feature extraction (DAFE) and current state-of-the art approaches, like spectral-domain multiclassifiers and decision fusion (MCDF). The proposed system had a classification accuracy which was approximately 40% higher than the SLDA approach and approximately 15% higher than MCDF.

In this study, the authors investigate the use of hyperspectral imaging for food crop monitoring and contamination detection and characterization. The authors investigate the use of a newly developed automated target recognition (ATR) system, that uses a combination of discrete wavelet transforms, multiclassifiers, and decision fusion, to effectively exploit the hyperspectral data to achieve high detection rates while maintaining low false alarm rates. The performance of the proposed hyperspectral ATR system is compared to ATR methods currently used in the remote sensing community, including those based on principal component analysis (PCA), discriminant analysis feature extraction (DAFE), and maximum-likelihood classifiers. The efficacy of both the proposed and conventional hyperspectral analysis methods are evaluated via an extensive 2-year field campaign, consisting of field-level experiments of corn and wheat exposed to highly controlled, varying levels of chemical contaminations. Both handheld and airborne hyperspectral data were collected at multiple times throughout the two growing seasons. The proposed ATR system provided very promising results, indicating the potential of hyperspectral remote sensing as an effective tool for detection and characterization of chemical contaminants in agricultural food crops.

Hyperspectral-based automatic target recognition (ATR) and classification systems often project the high-dimensional hyperspectral reflectance signatures onto a lower dimensional subspace using techniques such as principal component analysis, Fisher's linear discriminant analysis (LDA), and stepwise LDA. In a general classification framework, these projections are suboptimal and, in the absence of sufficient training data, are likely to be ill conditioned. In recent work, the authors proposed a divide-and-conquer approach that partitions the hyperspectral space into contiguous subspaces followed by a multiclassifier and decision-fusion (MCDF) framework. Although this technique alleviated the small-sample-size problem and provided a good recognition performance in light and moderate pixel mixing, the performance significantly decreased under severe mixing conditions, as it does with conventional ATR techniques. In this letter, the authors propose a kernel discriminant analysis-based projection in each subspace of the partition, followed by the MCDF framework to ensure robust recognition even in severe pixel-mixing conditions. The performance of the proposed system (as measured by overall recognition accuracies) is greatly superior to conventional dimensionality-reduction techniques as well as the more recently proposed LDA-based MCDF technique.

The 2008 Data Fusion Contest organized by the IEEE Geoscience and Remote Sensing Data Fusion Technical Committee deals with the classification of high-resolution hyperspectral data from an urban area. Unlike in the previous issues of the contest, the goal was not only to identify the best algorithm but also to provide a collaborative effort: The decision fusion of the best individual algorithms was aiming at further improving the classification performances, and the best algorithms were ranked according to their relative contribution to the decision fusion. This paper presents the five awarded algorithms and the conclusions of the contest, stressing the importance of decision fusion, dimension reduction, and supervised classification methods, such as neural networks and support vector machines.

Hyperspectral signatures provide a dense recording of reflectance values over a wide region of the spectrum. This potentially increases the class separation capacity of the data as compared to gray scale imagery (where most of the class specific information is extracted from spatial relations between pixels) or multi-spectral imagery (where reflectance values at a few spectral bands are recorded). Availability of this rich spectral information has made it possible to design classification systems that can perform ground cover classification and target recognition very accurately. However, this advantage of hyperspectral data is typically accompanied by the burden of requiring large training sets. Another ramification of having a high dimensional feature space is over-fitting of decision boundaries by classifiers, and consequently, poor generalization capacity. In this paper, we will analyze and quantify the classification performance (as represented by overall and target recognition accuracies and false alarm rates) using various popular dimensionality reduction techniques. In particular, we will study the efficacy of PCA, Regularized LDA and Stepwise LDA in a single-classifier framework, and the efficacy of LDA in a multi-classifier, decision fusion framework.

Hyperspectral image based automatic target recognition (ATR) systems often project the high dimensional hyperspectral reflectance signatures onto a lower dimensional subspace using techniques such as principal components analysis (PCA), Fisher's linear discriminant analysis (LDA) and stepwise LDA. In a general classification framework, these projections are sub-optimal, and in the absence of sufficient training data, are likely to be ill conditioned. In recent work, the authors proposed a divide and conquer approach that partitions the hyperspectral space into contiguous subspaces followed by multi-classifiers and decision fusion. Although this technique alleviated the small sample size problem and provided a good recognition performance in light and moderate pixel mixing, the performance significantly decreased under severe mixing conditions, as does with conventional ATR techniques. In this work, the authors propose a kernel discriminant analysis based projection in each subspace of the partition, followed by the multi-classifier, decision fusion framework to ensure robust recognition even in severe pixel mixing conditions. The performance of the proposed system (as measured by overall target recognition accuracies) is greatly superior to conventional dimensionality reduction techniques, as well as the more recently proposed divide-and-conquer technique.

Multi-source data fusion in the context of automatic target recognition (ATR) involves the fusion of multiple, independent observations of a phenomenon. If the collection of sources is diverse, the resulting classification system is expected to perform better than one based on any one source. In recent work, the authors have demonstrated the use of such decision fusion strategies in alleviating the over-dimensionality and small-sample-size problems associated with hyperspectral data. Multi-temporal hyperspectral recognition and classification tasks are even more prone to over-dimensionality of features and small training sample size problems. In this work, the authors will extend their previously proposed framework to multi-temporal, hyperspectral target recognition / classification problems. The performance of the proposed system will be compared against that of conventional hyperspectral feature extraction techniques. The efficacy of the proposed system is quantified by overall recognition accuracies.

In this study, the authors investigate the use of the Wavelet Packet Decomposition (WPD) as a preprocessing stage for a multiclassifiers and decision fusion system used for hyperspectral automated target recognition (ATR). The hyperspectral signature is transformed using WPD, and each set of wavelet detail and approximation coefficients (terminal nodes or leaves on the WPD tree) are considered as feature vectors Dimensionality reduction and feature optimization is performed via WPD tree pruning, using ATR-appropriate pruning metrics such as class separation. The wavelet coefficients in the terminal nodes of the pruned tree are then used for form feature vectors. Three methods are then investigated: (i) treating each terminal node as independent feature vector that is input to an independent classifier in a multiclassifier decision fusion (MCDF) system, (ii) applying intelligent feature grouping to the terminal nodes, where each resulting group is treated as an independent feature vector that is input to an independent classifier in a MCDF system, (iii) concatenating all terminal nodes and applying stepwise linear discriminant analysis (SLDA) along with a single classifier. The efficacy of the proposed methods are investigated using an experimental hyperspectral database for a remote sensing agricultural application, namely early detection of the disease known as soybean rust (Phakopsora pachyrhizi) in soybean crops.

Dimensionality reduction is a necessity in most hyperspectral imaging applications. Tradeoffs exist between unsupervised statistical methods, which are typically based on principal components analysis (PCA), and supervised ones, which are often based on Fisher's linear discriminant analysis (LDA), and proponents for each approach exist in the remote sensing community. Recently, a combined approach known as subspace LDA has been proposed, where PCA is employed to recondition ill-posed LDA formulations. The key idea behind this approach is to use a PCA transformation as a preprocessor to discard the null space of rank-deficient scatter matrices, so that LDA can be applied on this reconditioned space. Thus, in theory, the subspace LDA technique benefits from the advantages of both methods. In this letter, we present a theoretical analysis of the effects (often ill effects) of PCA on the discrimination power of the projected subspace. The theoretical analysis is presented from a general pattern classification perspective for two possible scenarios: (1) when PCA is used as a simple dimensionality reduction tool and (2) when it is used to recondition an ill-posed LDA formulation. We also provide experimental evidence of the ineffectiveness of both scenarios for hyperspectral target recognition applications.

Conventional hyperspectral image-based automatic target recognition (ATR) systems project high-dimensional reflectance signatures onto a lower dimensional subspace using techniques such as principal components analysis (PCA), Fisher's linear discriminant analysis (LDA), and stepwise LDA. Typically, these feature space projections are suboptimal. In a typical hyperspectral ATR setup, the number of training signatures (ground truth) is often less than the dimensionality of the signatures. Standard dimensionality reduction tools such as LDA and PCA cannot be applied in such situations. In this paper, we present a divide-and-conquer approach that addresses this problem for robust ATR. We partition the hyperspectral space into contiguous subspaces based on the optimization of a performance metric. We then make local classification decisions in every subspace using a multiclassifier system and employ a decision fusion system for making the final decision on the class label. In this work, we propose a metric that incorporates higher order statistical information for accurate partitioning of the hyperspectral space. We also propose an adaptive weight assignment method in the decision fusion process based on the strengths (as measured by the training accuracies) of individual classifiers that made the local decisions. The proposed methods are tested using hyperspectral data with known ground truth, such that the efficacy can be quantitatively measured in terms of target recognition accuracies. The proposed system was found to significantly outperform conventional approaches. For example, under moderate pixel mixing, the proposed approach resulted in classification accuracies around 90%, where traditional feature fusion resulted in accuracies around 65%.

We present a supervised hyperspectral segmentation procedure, consisting of best band analysis (BBA), an initial distance-based segmentation, and level set segmentation enhancement by forcing localized vicinities to be more homogeneous. BBA uses the spectral information divergence (SID) to reduce each pixel's high dimensional data to a scalar value, where the Bhattacharyya distance (BD) is maximized. The initial segmentation is based on feature vectors created from the SID metric. The level set segmentation then enhances areas that do not have spatially homogeneous ground cover. The proposed method is tested on a 72-band compact airborne spectrographic imager (CASI) image of a farm area in northern Mississippi, U.S.A. The proposed method is compared to a BBA-based maximum-likelihood (ML) method. Quantitative results are compared using segmentation and classification accuracies. Results show that both the initial classification using BBA features and the level set enhancement produced high-quality ground cover maps and outperformed the ML method, as well as previous studies by the authors. The ML method resulted in accuracies ges95.5%, whereas the level set segmentation approach resulted in accuracies as high as 99.7%.

Principal components analysis (PCA) is commonly used as a tool for feature space dimensionality reduction for various automatic target recognition (ATR) systems. Recently, PCA has also been employed in conjunction with linear discriminant analysis (LDA) to recondition ill posed LDA formulations. The key idea behind this approach is to use a PCA transformation to discard the null space of rank deficient scatter matrices so that LDA can be applied on this reconditioned space. This approach, called subspace LDA, has been employed with some level of success for face recognition algorithms. In this paper, we present a theoretical analysis of the effects of PCA on discrimination power of the projected subspace. We also provide experimental evidence of the ineffectiveness of a PCA projection for hyperspectral ATR applications using three different hyperspectral invasive species datasets.

Multi-source data fusion is being actively explored in the remote sensing community for robust automatic target recognition (ATR) and other similar applications. Such an approach exploits multiple, independent observations of a phenomenon and performs a feature level or a decision level fusion for ATR, scene classification, land cover mapping, etc. In this paper, we present a method that utilizes such fusion techniques to exploit hyperspectral data, which otherwise typically suffers from the small sample size problem, (i.e., there are typically not as many ground truth pixels as the dimensionality of the data). In this work, we study the efficacy of using higher order statistical information (using average mutual information) for a bottom up band grouping in a multi- classifier setup. The band grouping procedure is employed to partition the hyperspectral space into approximately independent subspaces. A classifier is assigned to each subspace in the partition. Final classification decisions are made by fusing local decisions from each subspace. The goal of this paper is to (1) perform subspace identification using the proposed mutual information based metric, (2) explore the effect of the design parameters on the fusion performance and, (3) compare the performance of decision level fusion with feature level fusion over the partitioned subspace.

In this paper, the discrete wavelet transform (DWT) is employed as a preprocessing stage for a multiclassifier and decision fusion system for feature extraction and dimensionality reduction of hyperspectral data. As a result, both global and local spectral features can be exploited. Feature grouping is conducted according to wavelet decomposition levels, or scales. Each DWT decomposition level's detail coefficients are classified independently, creating a multiclassifier system. The resulting classifications are then fused using a simple majority voting scheme. The proposed target recognition system was applied to hyperspectral data for an agricultural applications, namely detecting the presence of the often devastating disease known as soybean rust in soybean crops. The proposed approach was compared to well-known hyperspectral dimensionality reduction methods, such as stepwise linear discriminant anlaysis (LDA). When using the DWT multiclassifier system, the overall classification accuracies ranged from the high 80's to the mid 90's. When using the stepwise LDA technique the overall classification accuracies ranged from the mid 60s to the mid 90's.

Automatic speech recognizers perform poorly when training and test data are systematically different in terms of noise and channel characteristics. One manifestation of such differences is variations in the probability density functions (pdfs) between training and test features. Consequently, both automatic speech recognition and automatic speaker identification may be severely degraded. Previous attempts to minimize this problem include Cepstral Mean and Variance Normalization and transforming all speech features to a univariate Gaussian pdf. In this paper, we present a quantile based Cumulative Density Function (CDF) matching technique for data drawn from different distributions. This method can be used to compensate for the systematic marginal (i.e. each feature individually) differences between training and test features. We further propose a linear covariance normalization technique to compensate for differences in covariance properties between training and test data. Experimental results are given that illustrate these techniques for speech recognition and automatic speaker identification.