This paper presents a patch-set-based sparse representation for image set classification. Compared with image-based image set representation, our patch-set-based representation is alignment free and thus has an advantage for tasks like video-based face recognition, image-set-based object recognition, and video-based hand gesture recognition, where precious alignment is usually difficult or even impossible due to large variance in view angle or pose. Specifically, to bypass the alignment issue, we propose to adopt the patch-based image set representation by dividing each image within each set into patches, then we cluster all the training patches into multiple clusters and classify the test patches based on the cluster centers of training patches. The labels of test patches within each cluster are inferred from a patch-set-based sparse representation for classification, and the labels of all test patches from all the clusters are then aggregated to predict a single label for the test set. Experimental results on video-based face recognition data sets (CMU-MoBo and YouTube Celebrities), image-set-based object recognition data set (ETH-80), and video-based hand gesture recognition data set (Kinect Hand Gestures) demonstrate that our proposed method consistently outperforms all existing ones, and the improvement is very significant on the YouTube Celebrities and Kinect Hand Gesture data sets. Moreover, we also quantitatively show the robustness of our method to misalignment on the Mutli-PIE data set.

The dictionary-aided sparse regression (SR) approach has recently emerged as a promising alternative to hyperspectral unmixing (HU) in remote sensing. By using an available spectral library as a dictionary, the SR approach identifies the underlying materials in a given hyperspectral image by selecting a small subset of spectral samples in the dictionary to represent the whole image. A drawback with the current SR developments is that an actual spectral signature in the scene is often assumed to have zero mismatch with its corresponding dictionary sample, and such an assumption is considered too ideal in practice. In this paper, we tackle the spectral signature mismatch problem by proposing a dictionary-adjusted nonconvex sparsity-encouraging regression (DANSER) framework. The main idea is to incorporate dictionary correcting variables in an SR formulation. A simple and low per-iteration complexity algorithm is tailor-designed for practical realization of DANSER. Using the same dictionary correcting idea, we also propose a robust subspace solution for dictionary pruning. Extensive simulations and real-data experiments show that the proposed method is effective in mitigating the undesirable spectral signature mismatch effects.

Hyperspectral unmixing (HU) is an essential signal processing procedure for blindly extracting the hidden spectral signatures of materials (or endmembers) from observed hyperspectral imaging data. Craig's criterion, stating that the vertices of the minimum volume enclosing simplex (MVES) of the data cloud yield high-fidelity endmember estimates, has been widely used for designing endmember extraction algorithms (EEAs) especially in the scenario of no pure pixels. However, most Craig-criterion-based EEAs generally suffer from high computational complexity due to heavy simplex volume computations, and performance sensitivity to random initialization, etc. In this work, based on the idea that Craig's simplex with N vertices can be defined by N associated hyperplanes, we develop a fast and reproducible EEA by identifying these hyperplanes from N(N - 1) data pixels extracted via simple and effective linear algebraic formulations, together with endmember identifiability analysis. Some Monte Carlo simulations are provided to demonstrate the superior efficacy of the proposed EEA over state-of-the-art Craig-criterion-based EEAs in both computational efficiency and estimation accuracy.

In this paper, we propose a very simple deep learning network for image classification that is based on very basic data processing components: 1) cascaded principal component analysis (PCA); 2) binary hashing; and 3) blockwise histograms. In the proposed architecture, the PCA is employed to learn multistage filter banks. This is followed by simple binary hashing and block histograms for indexing and pooling. This architecture is thus called the PCA network (PCANet) and can be extremely easily and efficiently designed and learned. For comparison and to provide a better understanding, we also introduce and study two simple variations of PCANet: 1) RandNet and 2) LDANet. They share the same topology as PCANet, but their cascaded filters are either randomly selected or learned from linear discriminant analysis. We have extensively tested these basic networks on many benchmark visual data sets for different tasks, including Labeled Faces in the Wild (LFW) for face verification; the MultiPIE, Extended Yale B, AR, Facial Recognition Technology (FERET) data sets for face recognition; and MNIST for hand-written digit recognition. Surprisingly, for all tasks, such a seemingly naive PCANet model is on par with the state-of-the-art features either prefixed, highly hand-crafted, or carefully learned [by deep neural networks (DNNs)]. Even more surprisingly, the model sets new records for many classification tasks on the Extended Yale B, AR, and FERET data sets and on MNIST variations. Additional experiments on other public data sets also demonstrate the potential of PCANet to serve as a simple but highly competitive baseline for texture classification and object recognition.

Most of the previous work on video action recognition use complex hand-designed local features, such as SIFT, HOG and SURF, but these approaches are implemented sophisticatedly and difficult to be extended to other sensor modalities. Recent studies discover that there are no universally best hand-engineered features for all datasets, and learning features directly from the data may be more advantageous. One such endeavor is Slow Feature Analysis (SFA) proposed by Wiskott and Sejnowski [33]. SFA can learn the invariant and slowly varying features from input signals and has been proved to be valuable in human action recognition [34]. It is also observed that the multi-layer feature representation has succeeded remarkably in widespread machine learning applications. In this paper, we propose to combine SFA with deep learning techniques to learn hierarchical representations from the video data itself. Specifically, we use a two-layered SFA learning structure with 3D convolution and max pooling operations to scale up the method to large inputs and capture abstract and structural features from the video. Thus, the proposed method is suitable for action recognition. At the same time, sharing the same merits of deep learning, the proposed method is generic and fully automated. Our classification results on Hollywood2, KTH and UCF Sports are competitive with previously published results. To highlight some, on the KTH dataset, our recognition rate shows approximately 1% improvement in comparison to state-of-the-art methods even without supervision or dense sampling.

Bleed-through effect is one of the most common degradations in old documents, even in today's newspapers. This effect must be removed for hancing human and automatic readability. The two images scanned from the recto and verso pages of a document can be treated as a linear combination of clean text images from two sides. In this paper, we introduce a nonnegative least-correlation approach to demix bleed-through text images. Experiments have been conducted on both synthetic and real world images. In synthetic case, our method can recover the source text images exactly. Under real world conditions, our approach also performs well. In addition, our approach is computationally efficient and does not need any postprocessing task.

We study in this paper the problem of learning classifiers from ambiguously labeled images. For instance, in the collection of new images, each image contains some samples of interest (\emph{e.g.,} human faces), and its associated caption has labels with the true ones included, while the sample-label association is unknown. The task is to learn classifiers from these ambiguously labeled images and generalize to new images. An essential consideration here is how to make use of the information embedded in the relations between samples and labels, both within each image and across the image set. To this end, we propose a novel framework to address this problem. Our framework is motivated by the observation that samples from the same class repetitively appear in the collection of ambiguously labeled training images, while they are just ambiguously labeled in each image. If we can identify samples of the same class from each image and associate them across the image set, the matrix formed by the samples from the same class would be ideally low-rank. By leveraging such a low-rank assumption, we can simultaneously optimize a partial permutation matrix (PPM) for each image, which is formulated in order to exploit all information between samples and labels in a principled way. The obtained PPMs can be readily used to assign labels to samples in training images, and then a standard SVM classifier can be trained and used for unseen data. Experiments on benchmark datasets show the effectiveness of our proposed method.

Promoting the spatial resolution of off-the-shelf hyperspectral sensors is expected to improve typical computer vision tasks, such as target tracking and image classification. In this paper, we investigate the scenario in which two cameras, one with a conventional RGB sensor and the other with a hyperspectral sensor, capture the same scene, attempting to extract redundant and complementary information. We propose a non-negative sparse promoting framework to integrate the hyperspectral and RGB data into a high resolution hyperspectral set of data. The formulated problem is in the form of a sparse non-negative matrix factorization with prior knowledge on the spectral and spatial transform responses, and it can be handled by alternating optimization where each subproblem is solved by efficient convex optimization solvers; e.g., the alternating direction method of multipliers. Experiments on a public database show that our method achieves much lower average reconstruction errors than other state-of-the-art methods.

Hyperspectral endmember extraction is a process to estimate endmember signatures from the hyperspectral observations, in an attempt to study the underlying mineral composition of a landscape. However, estimating the number of endmembers, which is usually assumed to be known a priori in most endmember estimation algorithms (EEAs), still remains a challenging task. In this paper, assuming hyperspectral linear mixing model, we propose a hyperspectral data geometry-based approach for estimating the number of endmembers by utilizing successive endmember estimation strategy of an EEA. The approach is fulfilled by two novel algorithms, namely geometry-based estimation of number of endmembers—convex hull (GENE-CH) algorithm and affine hull (GENE-AH) algorithm. The GENE-CH and GENE-AH algorithms are based on the fact that all the observed pixel vectors lie in the convex hull and affine hull of the endmember signatures, respectively. The proposed GENE algorithms estimate the number of endmembers by using the Neyman–Pearson hypothesis testing over the endmember estimates provided by a successive EEA until the estimate of the number of endmembers is obtained. Since the estimation accuracies of the proposed GENE algorithms depend on the performance of the EEA used, a reliable, reproducible, and successive EEA, called$p$-norm-based pure pixel identification (TRI-P) algorithm is then proposed. The performance of the proposed TRI-P algorithm, and the estimation accuracies of the GENE algorithms are demonstrated through Monte Carlo simulations. Finally, the proposed GENE and TRI-P algorithms are applied to real AVIRIS hyperspectral data obtained over the Cuprite mining site, Nevada, and some conclusions and future directions are provided.

Hyperspectral endmember extraction is to estimate endmember signatures (or material spectra) from the hyperspectral data of an area for analyzing the materials and their composition therein. The presence of noise and outliers in the data poses a serious problem in endmember extraction. In this paper, we handle the noise- and outlier-contaminated data by a two-step approach. We first propose a robust-affine-set-fitting algorithm for joint dimension reduction and outlier removal. The idea is to find a contamination-free data-representative affine set from the corrupted data, while keeping the effects of outliers minimum, in the least squares error sense. Then, we devise two computationally efficient algorithms for extracting endmembers from the outlier-removed data. The two algorithms are established from a simplex volume max-min formulation which is recently proposed to cope with noisy scenarios. A robust algorithm, called worst case alternating volume maximization (WAVMAX), has been previously developed for the simplex volume max-min formulation but is computationally expensive to use. The two new algorithms employ a different kind of decoupled max-min partial optimizations, wherein the design emphasis is on low-complexity implementations. Some computer simulations and real data experiments demonstrate the efficacy, the computational efficiency, and the applicability of the proposed algorithms, in comparison with the WAVMAX algorithm and some benchmark endmember extraction algorithms.

Hyperspectral unmixing is a process of decomposing the hyperspectral data cube into endmember signatures and their corresponding abundance maps. For the unmixing results to be completely interpretable, the number of materials (or endmembers) present in that area should be known a priori, which however is unknown in practice. In this work, we use hyperspectral data geometry and successive endmember estimation strategy of an endmember extraction algorithm (EEA) to develop two novel algorithms for estimating the number of endmembers, namely geometry based estimation of number of endmembers - convex hull (GENE-CH) algorithm and affine hull (GENE-AH) algorithm. The proposed GENE algorithms estimate the number of endmembers by using Neyman-Pearson hypothesis testing over the endmembers sequentially estimated by an EEA until the estimate of the number of endmembers is obtained. Monte- Carlo simulations demonstrate the efficacy of the proposed GENE algorithms, compared to some existing benchmark methods for estimating number of endmembers.

Hyperspectral endmember extraction (EE) is to estimate endmember signatures (or material spectra) from the hyperspectral data of an unexplored area for analyzing the materials and their composition therein. However, the presence of noise in the data posts a serious problem for EE. Recently, robustness against noise has been taken into account in the design of EE algorithms. The robust maximum-volume simplex criterion [1] has been shown to yield performance improvement in the noisy scenario, but its real applicability is limited by its high implementation complexity. In this paper, we propose two fast algorithms to approximate this robust criterion [1], which turns out to deal with a set of partial max-min optimization problems in alternating manner and successive manner, respectively. Some Monte Carlo simulations demonstrate the superior computational efficiency and efficacy of the proposed robust algorithms in the noisy scenario over the robust algorithm in [1] and some benchmark EE algorithms.

Endmember extraction is of prime importance in the process of hyperspectral unmixing so as to study the mineral composition of a landscape from its hyperspectral observations. Though, a whole bunch of pure-pixel based endmember extraction algorithms exists, the quest for a reliable, repeatable, and computationally efficient endmember extraction algorithm still prevails. In this work, we propose two pure-pixel based endmember extraction algorithms called simplex estimation by projection (SIMPLE-Pro) algorithm and p-norm based pure pixel identification (TRI-P) algorithm. The end member identifiability of the proposed two algorithms is theoretically proved under the pure pixel assumption. Both algorithms never require any initializations and hence they are repeatable. Monte Carlo simulations are performed to demonstrate the superior efficacy and computational efficiency of the proposed two algorithms over some existing benchmark endmember extraction algorithms.

In this paper, we describe a continuous optimization perspective on Winter's simplex volume maximization belief for endmember ex traction in hyperspectral remote sensing. Winter's belief, proposed in the late 90's, is very insightful and has led to one of the most widely used class of endmember extraction algorithms nowadays- N-FINDR. Our endeavor to revisit this problem is to provide an al ternative, systematic, framework of formulating and understanding Winter's belief. Under the continuous optimization formulation of Winter's belief, we show a fundamental result that the existence of pure pixels is not only sufficient for the Winter problem to perfectly identify the ground-truth endmembers, but also necessary. Then, we derive two Winter-based algorithms based on two different optimization strategies. Interestingly, the resulting algorithms are found to be similar to an N-FINDR variant and the vertex component analysis (VCA) algorithm. Hence, the developed framework provides linkage and alternative interpretations to these existing algorithms. Simulation results are also presented to compare the derived Winter algorithms and several existing algorithms.

SUMMARY: In vivo dynamic contrast-enhanced imaging tools provide non-invasive methods for analyzing various functional changes associated with disease initiation, progression and responses to therapy. The quantitative application of these tools has been hindered by its inability to accurately resolve and characterize targeted tissues due to spatially mixed tissue heterogeneity. Convex Analysis of Mixtures - Compartment Modeling (CAM-CM) signal deconvolution tool has been developed to automatically identify pure-volume pixels located at the corners of the clustered pixel time series scatter simplex and subsequently estimate tissue-specific pharmacokinetic parameters. CAM-CM can dissect complex tissues into regions with differential tracer kinetics at pixel-wise resolution and provide a systems biology tool for defining imaging signatures predictive of phenotypes. AVAILABILITY: The MATLAB source code can be downloaded at the authors' website www.cbil.ece.vt.edu/software.htm CONTACT: [email protected] SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

Effective unmixing of hyperspectral data cube under a noisy scenario has been a challenging research problem in remote sensing arena. A branch of existing hyperspectral unmixing algorithms is based on Craig's criterion, which states that the vertices of the minimum-volume simplex enclosing the hyperspectral data should yield high fidelity estimates of the endmember signatures associated with the data cloud. Recently, we have developed a minimum-volume enclosing simplex (MVES) algorithm based on Craig's criterion and validated that the MVES algorithm is very useful to unmix highly mixed hyperspectral data. However, the presence of noise in the observations expands the actual data cloud, and as a consequence, the endmember estimates obtained by applying Craig-criterion-based algorithms to the noisy data may no longer be in close proximity to the true endmember signatures. In this paper, we propose a robust MVES (RMVES) algorithm that accounts for the noise effects in the observations by employing chance constraints. These chance constraints in turn control the volume of the resulting simplex. Under the Gaussian noise assumption, the chance-constrained MVES problem can be formulated into a deterministic nonlinear program. The problem can then be conveniently handled by alternating optimization, in which each subproblem involved is handled by using sequential quadratic programming solvers. The proposed RMVES is compared with several existing benchmark algorithms, including its predecessor, the MVES algorithm. Monte Carlo simulations and real hyperspectral data experiments are presented to demonstrate the efficacy of the proposed RMVES algorithm.

In the late 1990s, Winter proposed an endmember extraction belief that has much impact on endmember extraction techniques in hyperspectral remote sensing. The idea is to find a maximum-volume simplex whose vertices are drawn from the pixel vectors. Winter's belief has stimulated much interest, resulting in many different variations of pixel search algorithms, widely known as N-FINDR, being proposed. In this paper, we take a continuous optimization perspective to revisit Winter's belief, where the aim is to provide an alternative framework of formulating and understanding Winter's belief in a systematic manner. We first prove that, fundamentally, the existence of pure pixels is not only sufficient for the Winter problem to perfectly identify the ground-truth endmembers but also necessary. Then, under the umbrella of the Winter problem, we derive two methods using two different optimization strategies. One is by alternating optimization. The resulting algorithm turns out to be an N-FINDR variant, but, with the proposed formulation, we can pin down some of its convergence characteristics. Another is by successive optimization; interestingly, the resulting algorithm is found to exhibit some similarity to vertex component analysis. Hence, the framework provides linkage and alternative interpretations to these existing algorithms. Furthermore, we propose a robust worst case generalization of the Winter problem for accounting for perturbed pixel effects in the noisy scenario. An algorithm combining alternating optimization and projected subgradients is devised to deal with the problem. We use both simulations and real data experiments to demonstrate the viability and merits of the proposed algorithms.

Dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) provides a noninvasive method for evaluating tumor vasculature patterns based on contrast accumulation and washout. However, due to limited imaging resolution and tumor tissue heterogeneity, tracer concentrations at many pixels often represent a mixture of more than one distinct compartment. This pixel-wise partial volume effect (PVE) would have profound impact on the accuracy of pharmacokinetics studies using existing compartmental modeling (CM) methods. We, therefore, propose a convex analysis of mixtures (CAM) algorithm to explicitly mitigate PVE by expressing the kinetics in each pixel as a nonnegative combination of underlying compartments and subsequently identifying pure volume pixels at the corners of the clustered pixel time series scatter plot simplex. The algorithm is supported theoretically by a well-grounded mathematical framework and practically by plug-in noise filtering and normalization preprocessing. We demonstrate the principle and feasibility of the CAM-CM approach on realistic synthetic data involving two functional tissue compartments, and compare the accuracy of parameter estimates obtained with and without PVE elimination using CAM or other relevant techniques. Experimental results show that CAM-CM achieves a significant improvement in the accuracy of kinetic parameter estimation. We apply the algorithm to real DCE-MRI breast cancer data and observe improved pharmacokinetic parameter estimation, separating tumor tissue into regions with differential tracer kinetics on a pixel-by-pixel basis and revealing biologically plausible tumor tissue heterogeneity patterns. This method combines the advantages of multivariate clustering, convex geometry analysis, and compartmental modeling approaches. The open-source MATLAB software of CAM-CM is publicly available from the Web.

Hyperspectral unmixing is a process of extracting hidden spectral signatures (or endmembers) and the corresponding proportions (or abundances) of a scene, from its hyperspectral observations. Motivated by Craig's belief, we recently proposed an alternating linear programming based hyperspectral unmixing algorithm called minimum volume enclosing simplex (MVES) algorithm, which can yield good unmixing performance even for instances of highly mixed data. In this paper, we propose a robust MVES algorithm called RMVES algorithm, which involves probabilistic reformulation of the MVES algorithm, so as to account for the presence of noise in the observations. The problem formulation for RMVES algorithm is manifested as a chance constrained program, which can be suitably implemented using sequential quadratic programming (SQP) solvers in an alternating fashion. Monte Carlo simulations are presented to demonstrate the efficacy of the proposed RMVES algorithm over several existing benchmark hyperspectral unmixing methods, including the original MVES algorithm.

Although significant efforts have been made in developing nonnegative blind source separation techniques, accurate separation of positive yet dependent sources remains a challenging task. In this paper, a joint correlation function of multiple signals is proposed to reveal and confirm that the observations after nonnegative mixing would have higher joint correlation than the original unknown sources. Accordingly, a new nonnegative least-correlated component analysis (n/LCA) method is proposed to design the unmixing matrix by minimizing the joint correlation function among the estimated nonnegative sources. In addition to a closed-form solution for unmixing two mixtures of two sources, the general algorithm of n/LCA for the multisource case is developed based on an iterative volume maximization (IVM) principle and linear programming. The source identifiability and required conditions are discussed and proven. The proposed n/LCA algorithm, denoted by n/LCA-IVM, is evaluated with both simulation data and real biomedical data to demonstrate its superior performance over several existing benchmark methods.

Hyperspectral unmixing aims at identifying the hidden spectral signatures (or endmembers) and their corresponding proportions (or abundances) from an observed hyperspectral scene. Many existing approaches to hyperspectral unmixing rely on the pure-pixel assumption, which may be violated for highly mixed data. A heuristic unmixing criterion without requiring the pure-pixel assumption has been reported by Craig: The endmember estimates are determined by the vertices of a minimum-volume simplex enclosing all the observed pixels. In this paper, using convex analysis, we show that the hyperspectral unmixing by Craig's criterion can be formulated as an optimization problem of finding a minimum-volume enclosing simplex (MVES). An algorithm that cyclically solves the MVES problem via linear programs (LPs) is also proposed. Some Monte Carlo simulations are provided to demonstrate the efficacy of the proposed MVES algorithm.

We recently reported a criterion for blind separation of non-negative sources, using a new concept called convex analysis for mixtures of non-negative sources (CAMNS). Under some assumptions that are considered realistic for sparse or high-contrast signals, the criterion is that the true source signals can be perfectly recovered by finding the extreme points of some observation-constructed convex set. In our last work we also developed methods for fulfilling the CAMNS criterion, but only for two to three sources. In this paper we propose a systematic linear programming (LP) based method that is applicable to any number of sources. The proposed method has two advantages. First, its dependence on LP means that the method does not suffer from local minima. Second, the maturity of LP solvers enables efficient implementation of the proposed method in practice. Simulation results are provided to demonstrate the efficacy of the proposed method.

In this paper, we apply convex analysis to the problem of blind source separation (BSS) of non-negative signals. Under realistic assumptions applicable to many real-world problems such as multichannel biomedical imaging, we formulate a new BSS criterion that does not require statistical source independence, a fundamental assumption to many existing BSS approaches. The new criterion guarantees perfect separation (in the absence of noise), by constructing a convex set from the observations and then finding the extreme points of the convex set. Some experimental results are provided to demonstrate the efficacy of the proposed method.