As a popular approach to collaborative filtering, matrix factorization (MF) models the underlying rating matrix as a product of two factor matrices, one for users and one for items. The MF model can be learned by Alternating Least Squares (ALS), which updates the two factor matrices alternately, keeping one fixed while updating the other. Although ALS improves the learning objective aggressively in each iteration, it suffers from high computational cost due to the necessity of inverting a separate matrix for every user and item. The softImpute-ALS reduces the per-iteration computation significantly using a strategy that requires only two matrix inversions; however, the computation saving leads to shrinkage of objective improvement. In this paper, we introduce a new algorithm, termed Data Augmentation with Optimal Step-size (DAOS), which alleviates the drawback of softImpute-ALS while still maintaining its low cost of computation per iteration. The DAOS is presented in the context that factor matrices may include fixed columns or rows, with this allowing bias terms and/or linear models to be incorporated into the ML model. Experimental results on synthetic and MovieLens 1M Dataset demonstrate the benefits of DAOS over ALS and softImpute-ALS in terms of generalization performance and computational time.

Gaussian graphical models (GGMs) are widely used for statistical modeling, because of ease of inference and the ubiquitous use of the normal distribution in practical approximations. However, they are also known for their limited modeling abilities, due to the Gaussian assumption. In this paper, we introduce a novel variant of GGMs, which relaxes the Gaussian restriction and yet admits efficient inference. Specifically, we impose a bipartite structure on the GGM and govern the hidden variables by truncated normal distributions. The nonlinearity of the model is revealed by its connection to rectified linear unit (ReLU) neural networks. Meanwhile, thanks to the bipartite structure and appealing properties of truncated normals, we are able to train the models efficiently using contrastive divergence. We consider three output constructs, accounting for real-valued, binary and count data. We further extend the model to deep constructions and show that deep models can be used for unsupervised pre-training of rectifier neural networks. Extensive experimental results are provided to validate the proposed models and demonstrate their superiority over competing models.

We present a probabilistic framework for nonlinearities, based on doubly truncated Gaussian distributions. By setting the truncation points appropriately, we are able to generate various types of nonlinearities within a unified framework, including sigmoid, tanh and ReLU, the most commonly used nonlinearities in neural networks. The framework readily integrates into existing stochastic neural networks (with hidden units characterized as random variables), allowing one for the first time to learn the nonlinearities alongside model weights in these networks. Extensive experiments demonstrate the performance improvements brought about by the proposed framework when integrated with the restricted Boltzmann machine (RBM), temporal RBM and the truncated Gaussian graphical model (TGGM).

We utilize copulas to constitute a unified framework for constructing and optimizing variational proposals in hierarchical Bayesian models. For models with continuous and non-Gaussian hidden variables, we propose a semiparametric and automated variational Gaussian copula approach, in which the parametric Gaussian copula family is able to preserve multivariate posterior dependence, and the nonparametric transformations based on Bernstein polynomials provide ample flexibility in characterizing the univariate marginal posteriors.

We introduce the truncated Gaussian graphical model (TGGM) as a novel framework for designing statistical models for nonlinear learning. A TGGM is a Gaussian graphical model (GGM) with a subset of variables truncated to be nonnegative. The truncated variables are assumed latent and integrated out to induce a marginal model. We show that the variables in the marginal model are non-Gaussian distributed and their expected relations are nonlinear. We use expectation-maximization to break the inference of the nonlinear model into a sequence of TGGM inference problems, each of which is efficiently solved by using the properties and numerical methods of multivariate Gaussian distributions. We use the TGGM to design models for nonlinear regression and classification, with the performances of these models demonstrated on extensive benchmark datasets and compared to state-of-the-art competing results.

Feature construction is of vital importance in reinforcement learning, as the quality of a value function or policy is largely determined by the corresponding features. The recent successes of deep reinforcement learning (RL) only increase the importance of understanding feature construction. Typical deep RL approaches use a linear output layer, which means that deep RL can be interpreted as a feature construction/encoding network followed by linear value function approximation. This paper develops and evaluates a theory of linear feature encoding. We extend theoretical results on feature quality for linear value function approximation from the uncontrolled case to the controlled case. We then develop a supervised linear feature encoding method that is motivated by insights from linear value function approximation theory, as well as empirical successes from deep RL. The resulting encoder is a surprisingly effective method for linear value function approximation using raw images as inputs.

Compressive sensing of signals drawn from a Gaussian mixture model (GMM) admits closed-form minimum mean squared error reconstruction from incomplete linear measurements. An accurate GMM signal model is usually not available a priori, because it is difficult to obtain training signals that match the statistics of the signals being sensed. We propose to solve that problem by learning the signal model in situ, based directly on the compressive measurements of the signals, without resorting to other signals to train a model. A key feature of our method is that the signals being sensed are treated as random variables and are integrated out in the likelihood. We derive a maximum marginal likelihood estimator (MMLE) that maximizes the likelihood of the GMM of the underlying signals given only their linear compressive measurements. We extend the MMLE to a GMM with dominantly low-rank covariance matrices, to gain computational speedup. We report extensive experimental results on image inpainting, compressive sensing of high-speed video, and compressive hyperspectral imaging (the latter two based on real compressive cameras). The results demonstrate that the proposed methods outperform state-of-the-art methods by significant margins.

A simple and inexpensive (low-power and low-bandwidth) modification is made to a conventional off-the-shelf color video camera, from which we recover multiple color frames for each of the original measured frames, and each of the recovered frames can be focused at a different depth. The recovery of multiple frames for each measured frame is made possible via high-speed coding, manifested via translation of a single coded aperture, the inexpensive translation is constituted by mounting the binary code on a piezoelectric device. To simultaneously recover depth information, a liquid lens is modulated at high speed, via a variable voltage. Consequently, during the aforementioned coding process, the liquid lens allows the camera to sweep the focus through multiple depths. In addition to designing and implementing the camera, fast recovery is achieved by an anytime algorithm exploiting the group-sparsity of wavelet/DCT coefficients.

We consider the group basis pursuit problem, which extends basis pursuit by replacing the $\ell_{1}$ norm with a weighted-$\ell_{2,1}$ norm. We provide an anytime algorithm, called generalized alternating projection (GAP), to solve this problem. The GAP algorithm extends classical alternating projection to the case in which projections are performed between convex sets that undergo a systematic sequence of changes. We prove that, under a set of group-restricted isometry property (group-RIP) conditions, the reconstruction error of GAP monotonically converges to zero. Thus the algorithm can be interrupted at any time to return a valid solution and can be resumed subsequently to improve the solution. This anytime convergence property saves iterations on retracting and correcting mistakes, which, along with an effective acceleration scheme, makes GAP converge fast. Moreover, the per-iteration computation is inexpensive, consisting of sorting of a linear array followed by groupwise thresholding and linear transform of vectors for which fast algorithms often exist. We evaluate the algorithmic performance through extensive experiments in which GAP is compared to other state-of-the-art algorithms and applied to compressive sensing of natural images and video.

A Gaussian mixture model (GMM)-based algorithm is proposed for video reconstruction from temporally compressed video measurements. The GMM is used to model spatio-temporal video patches, and the reconstruction can be efficiently computed based on analytic expressions. The GMM-based inversion method benefits from online adaptive learning and parallel computation. We demonstrate the efficacy of the proposed inversion method with videos reconstructed from simulated compressive video measurements, and from a real compressive video camera. We also use the GMM as a tool to investigate adaptive video compressive sensing, i.e., adaptive rate of temporal compression.

A Gaussian Mixture Model (GMM)-based algorithm is proposed for video reconstruction from temporal compressed measurements. The GMM is used to model spatio-temporal video patches, and the reconstruction can be efficiently computed based on analytic expressions. The developed GMM reconstruction method benefits from online adaptive learning and parallel computation. We demonstrate the efficacy of the proposed GMM with videos reconstructed from simulated compressive video measurements and from a real compressive video camera.

This paper introduces the concept of adaptive temporal compressive sensing (CS) for video. We propose a CS algorithm to adapt the compression ratio based on the scene's temporal complexity, computed from the compressed data, without compromising the quality of the reconstructed video. The temporal adaptivity is manifested by manipulating the integration time of the camera, opening the possibility to realtime implementation. The proposed algorithm is a generalized temporal CS approach that can be incorporated with a diverse set of existing hardware systems.

We present a non-factorized variational method for full posterior inference in Bayesian hierarchical models, with the goal of capturing the posterior variable dependencies via efficient and possibly parallel computation. Our approach unifies the integrated nested Laplace approximation (INLA) under the variational framework. The proposed method is applicable in more challenging scenarios than typically assumed by INLA, such as Bayesian Lasso, which is characterized by the non-differentiability of the $\ell_{1}$ norm arising from independent Laplace priors. We derive an upper bound for the Kullback-Leibler divergence, which yields a fast closed-form solution via decoupled optimization. Our method is a reliable analytic alternative to Markov chain Monte Carlo (MCMC), and it results in a tighter evidence lower bound than that of mean-field variational Bayes (VB) method.

We present a nonparametric Bayesian model for multi-task learning, with a focus on feature selection in binary classification. The model jointly identifies groups of similar tasks and selects the subset of features relevant to the tasks within each group. The model employs a Dirchlet process with a beta Bernoulli hierarchical base measure. The posterior inference is accomplished efficiently using a Gibbs sampler. Experimental results are presented on simulated as well as real data.

We present an algorithm for active learning (adaptive selection of training data) within the context of semi-supervised multi-task classifier design. The semi-supervised multi-task classifier exploits manifold information provided by the unlabeled data, while also leveraging relevant information across multiple data sets. The active-learning component defines which data would be most informative to classifier design if the associated labels are acquired. The framework is demonstrated through application to a real landmine detection problem.

A fundamental objective in reinforcement learning is the maintenance of a proper balance between exploration and exploitation. This problem becomes more challenging when the agent can only partially observe the states of its environment. In this paper we propose a dual-policy method for jointly learning the agent behavior and the balance between exploration exploitation, in partially observable environments. The method subsumes traditional exploration, in which the agent takes actions to gather information about the environment, and active learning, in which the agent queries an oracle for optimal actions (with an associated cost for employing the oracle). The form of the employed exploration is dictated by the specific problem. Theoretical guarantees are provided concerning the optimality of the balancing of exploration and exploitation. The effectiveness of the method is demonstrated by experimental results on benchmark problems.

Context plays an important role when performing classification, and in this paper we examine context from two perspectives. First, the classification of items within a single task is placed within the context of distinct concurrent or previous classification tasks (multiple distinct data collections). This is referred to as multi-task learning (MTL), and is implemented here in a statistical manner, using a simplified form of the Dirichlet process. In addition, when performing many classification tasks one has simultaneous access to all unlabeled data that must be classified, and therefore there is an opportunity to place the classification of any one feature vector within the context of all unlabeled feature vectors; this is referred to as semi-supervised learning. In this paper we integrate MTL and semi-supervised learning into a single framework, thereby exploiting two forms of contextual information. Example results are presented on a "toy" example, to demonstrate the concept, and the algorithm is also applied to three real data sets.

To achieve good generalization in supervised learning, the training and testing examples are usually required to be drawn from the same source distribution. In this paper, we propose a method to relax this requirement in the context of logistic regression. AssumingDpandDaare two sets of examples drawn from two different distributionsTandA(called concepts, borrowing a term from psychology), whereDaare fully labeled andDppartially labeled, our objective is to complete the labels ofDp. We introduce an auxiliary variable mu for each example inDato reflect its mismatch withDp. Under an appropriate constraint the mus are estimated as a byproduct, along with the classifier. We also present an active learning approach for selecting the labeled examples inDp. The proposed algorithm, calledmigratorylogisticregression, is demonstrated successfully on simulated data as well as on real measured data of interest for unexploded ordnance cleanup.

The purpose of this research is to develop a classifier capable of state-of-the-art performance in both computational efficiency and generalization ability while allowing the algorithm designer to choose arbitrary loss functions as appropriate for a give problem domain. This is critical in applications involving heavily imbalanced, noisy, or non-Gaussian distributed data. To achieve this goal, a kernel-matching pursuit (KMP) framework is formulated where the objective is margin maximization rather than the standard error minimization. This approach enables excellent performance and computational savings in the presence of large, imbalanced training data sets and facilitates the development of two general algorithms. These algorithms support the use of arbitrary loss functions allowing the algorithm designer to control the degree to which outliers are penalized and the manner in which non-Gaussian distributed data is handled. Example loss functions are provided and algorithm performance is illustrated in two groups of experimental results. The first group demonstrates that the proposed algorithms perform equivalent to several state-of-the-art machine learning algorithms on well-published, balanced data. The second group of results illustrates superior performance by the proposed algorithms on imbalanced, non-Gaussian data achieved by employing loss functions appropriate for the data characteristics and problem domain.

Semi supervised learning and active learning are considered for unexploded ordnance (UXO) detection. Semi supervised learning algorithms are designed using both labeled and unlabeled data, where here labeled data correspond to sensor signatures for which the identity of the buried item (UXO/non-UXO) is known; for unlabeled data, one only has access to the corresponding sensor data. Active learning is used to define which unlabeled signatures would be most informative to improve the classifier design if the associated label could be acquired (where for UXO sensing, the label is acquired by excavation). A graph-based semi supervised algorithm is applied, which employs the idea of a random Markov walk on a graph, thereby exploiting knowledge of the data manifold (where the manifold is defined by both the labeled and unlabeled data). The algorithm is used to infer labels for the unlabeled data, providing a probability that a given unlabeled signature corresponds to a buried UXO. An efficient active-learning procedure is developed for this algorithm, based on a mutual information measure. In this manner, one initially performs excavation with the purpose of acquiring labels to improve the classifier, and once this active-learning phase is completed, the resulting semi supervised classifier is then applied to the remaining unlabeled signatures to quantify the probability that each such item is a UXO. Example classification results are presented for an actual UXO site, based on electromagnetic induction and magnetometer data. Performance is assessed in comparison to other semi supervised approaches, as well as to supervised algorithms.

We present an algorithm for learning parametric classifiers on a partially labeled data manifold, based on a graph representation of the manifold. The unlabeled data are utilized by basing classifier learning on neighborhoods, formed via Markov random walks. The proposed algorithm yields superior performance on three benchmark data sets and the margin of improvements over existing semi-supervised algorithms is significant.

We introduce quadratically gated mixture of experts (QGME), a statistical model for multi-class nonlinear classification. The QGME is formulated in the setting of incomplete data, where the data values are partially observed. We show that the missing values entail joint estimation of the data manifold and the classifier, which allows adaptive imputation during classifier learning. The expectation maximization (EM) algorithm is derived for joint likelihood maximization, with adaptive imputation performed analytically in the E-step. The performance of QGME is evaluated on three benchmark data sets and the results show that the QGME yields significant improvements over competing methods.

A semi-supervised multitask learning (MTL) framework is presented, in which M parameterized semi-supervised classifiers, each associated with one of M par- tially labeled data manifolds, are learned jointly under the constraint of a soft- sharing prior imposed over the parameters of the classifiers. The unlabeled data are utilized by basing classifier learning on neighborhoods, induced by a Markov random walk over a graph representation of each manifold. Experimental results on real data sets demonstrate that semi-supervised MTL yields significant im- provements in generalization performance over either semi-supervised single-task learning (STL) or supervised MTL.

Consider the problem of learning logistic-regression models for multiple classification tasks, where the training data set for each task is not drawn from the same statistical distribution. In such a multi-task learning (MTL) scenario, it is necessary to identify groups of similar tasks that should be learned jointly. Relying on a Dirichlet process (DP) based statistical model to learn the extent of similarity between classification tasks, we develop computationally efficient algorithms for two different forms of the MTL problem. First, we consider a symmetric multi-task learning (SMTL) situation in which classifiers for multiple tasks are learned jointly using a variational Bayesian (VB) algorithm. Second, we consider an asymmetric multi-task learning (AMTL) formulation in which the posterior density function from the SMTL model parameters (from previous tasks) is used as a prior for a new task: this approach has the significant advantage of not requiring storage and use of all previous data from prior tasks. The AMTL formulation is solved with a simple Markov Chain Monte Carlo (MCMC) construction. Experimental results on two real life MTL problems indicate that the proposed algorithms: (a) automatically identify subgroups of related tasks whose training data appear to be drawn from similar distributions; and (b) are more accurate than simpler approaches such as single-task learning, pooling of data across all tasks, and simplified approximations to DP.

We address the incomplete-data problem in which feature vectors to be classified are missing data (features). A (supervised) logistic regression algorithm for the classification of incomplete data is developed. Single or multiple imputation for the missing data is avoided by performing analytic integration with an estimated conditional density function (conditioned on the observed data). Conditional density functions are estimated using a Gaussian mixture model (GMM), with parameter estimation performed using both Expectation-Maximization (EM) and Variational Bayesian EM (VB-EM). The proposed supervised algorithm is then extended to the semisupervised case by incorporating graph-based regularization. The semisupervised algorithm utilizes all available data-both incomplete and complete, as well as labeled and unlabeled. Experimental results of the proposed classification algorithms are shown.

We address the problem of unexploded ordnance (UXO) detection in which data to be classified is available from multiple sensor modalities and multiple resolutions. Specifically, features are extracted from measured magnetometer and electromagnetic induction data; multiple-resolution data are manifested when the sensors are separated from the buried targets of interest by different distances (e.g., different sensor-platform heights). The proposed classification algorithm explicitly emphasizes features extracted from fine-resolution imagery over those extracted from less reliable coarse-resolution data. When fine-resolution features are unavailable (due to undeployed sensors), the algorithm analytically integrates out the missing features via an estimated conditional density function, which is conditioned on the observed features (from deployed sensors). This density function exploits the statistical relationships that exist among features at different resolutions, as well as those among features from different sensors (in the multisensor case). Experimental classification results are shown for real UXO data, on which the proposed algorithm consistently achieves better classification performance than common alternative approaches.

We consider a classification problem wherein the class features are not given a priori. The classifier is responsible for selecting the features, to minimize the cost of observing features while also maximizing the classification performance. We propose a reward-directed Bayesian classifier (RDBC) to solve this problem. The RDBC features an internal state structure for preserving the feature dependence, and is formulated as a partially observable Markov decision process (POMDP). The results on a diabetes dataset show the RDBC with a moderate number of states significantly improves over the naive Bayes classifier, both in prediction accuracy and observation parsimony. It is also demonstrated that the RDBC performs better by using more states to increase its memory.

An approximate region-based value iteration (RBVI) algorithm is proposed to find the optimal policy for a partially observable Markov decision process (POMDP). The proposed RBVI approximates the true polyhedral partition of the belief simplex with an ellipsoidal partition, such that the optimal value function is linear in each of the ellipsoidal regions. The position and shape of each region, as well as the gradient (alpha-vector) of the optimal value function in the region, are parameterized explicitly, and are estimated via efficient expectation maximization (EM) and variational Bayesian EM (VBEM), based on a set of selected sample belief points. The RBVI maintains a much smaller number of alpha-vectors than point-based methods and yields a more parsimonious representation that approximates the true value function in the maximum likelihood (ML) sense. The results on benchmark problems show that the proposed RBVI is comparable in performance to state-of-the-art algorithms, despite of the small number of alpha-vectors that are used.

To achieve good generalization in supervised learning, the training and testing examples are usually required to be drawn from the same source distribution. In this paper we propose a method to relax this requirement in the context of logistic regression. Assuming Dp and Da are two sets of examples drawn from two mismatched distributions, where Da are fully labeled and Dp partially labeled, our objective is to complete the labels of Dp. We introduce an auxiliary variable μ for each example in Da to reflect its mismatch with Dp. Under an appropriate constraint the μ's are estimated as a byproduct, along with the classifier. We also present an active learning approach for selecting the labeled examples in Dp. The proposed algorithm, called "Migratory-Logit" or M-Logit, is demonstrated successfully on simulated as well as real data sets.

A logistic regression classification algorithm is developed for problems in which the feature vectors may be missing data (features). Single or multiple imputation for the missing data is avoided by performing analytic integration with an estimated conditional density function (conditioned on the non-missing data). Conditional density functions are estimated using a Gaussian mixture model (GMM), with parameter estimation performed using both expectation maximization (EM) and Variational Bayesian EM (VB-EM). Using widely available real data, we demonstrate the general advantage of the VB-EM GMM estimation for handling incomplete data, vis-à-vis the EM algorithm. Moreover, it is demonstrated that the approach proposed here is generally superior to standard imputation procedures.

We extend radial basis function (RBF) networks to the scenario in which multiple correlated tasks are learned simultaneously, and present the cor- responding learning algorithms. We develop the algorithms for learn- ing the network structure, in either a supervised or unsupervised manner. Training data may also be actively selected to improve the network’s gen- eralization to test data. Experimental results based on real data demon- strate the advantage of the proposed algorithms and support our conclu- sions.

We consider target classification and detection based on backscattered observations measured from a sequence of target-sensor orientations. The multi-aspect scattered waves from a given target are modeled with a hidden Markov model (HMM). The targets are assumed concealed and the absolute target-sensor orientation is assumed unknown; therefore, it is possible to control only the angular displacements (change in orientation) between consecutive measurements. The performance of the HMM classifiers/detectors is influenced by the choice of the angular displacements, the optimization of which motivates the developed adaptive search strategies, based on entropy-driven optimality criteria. The search proceeds in a sequential fashion. Based on the previous observations and their associated angular displacements, one determines the optimal next displacement to perform an associated observation. The search strategies are detailed and example results presented on adaptive classification and detection of underwater targets.

The kernel matching pursuit (KMP) algorithm is re-formulated in the framework of the theory of optimal experiments, using a weighted sum of squared errors as the loss function, and it is extended to the case of M-ary target classification and kernel optimization. The M-ary KMP classifier is applied to the multi-aspect classification of moving targets based on high-range resolution (HRR) radar signatures, for which the target-sensor orientations are assumed approximately known. A multi-aspect processing method is presented based on the use of the estimates of target-sensor orientation angles. The KMP classification results for ten MSTAR targets are presented, with a comparison to corresponding results using the relevance vector machine (RVM).

An information-theoretic approach is developed for target detection, with active training set selection, directly from the site-specific measured data. For the proposed kernel-based algorithm, a set of basis functions to characterize the signature distribution of the site are defined first; then we determine a parsimonious set of data, for which knowledge of the associated labels would be most informative to determine the weights for the basis functions. Both of them utilize the Fisher information criteria. The proposed framework is applied to subsurface target detection, with example results presented for an actual buried unexploded ordnance site.

A mobile electromagnetic-induction (EMI) sensor is considered for detection and characterization of buried conducting and/or ferrous targets. The sensor may be placed on a robot and, here, we consider design of an optimal adaptive-search strategy. A frequency-dependent magnetic-dipole model is used to characterize the target at EMI frequencies. The goal of the search is accurate characterization of the dipole-model parameters, denoted bythe vector theta; the target position and orientation are a subset of theta. The sensor position and operating frequency are denoted by the parameter vector p and a measurement is represented by the pair (p, O), where O denotes the observed data. The parametersp are fixed for a given measurement, but, in the context of a sequence of measurements p may be changed adaptively. In a locally optimal sequence of measurements, we desire the optimal sensor parameters, P(N+1) for estimation of theta, based on the previous measurements (p(n), On)n=1,N. The search strategy is based on the theory of optimal experiments, as discussed in detail and demonstrated via several numerical examples.

When sensing subsurface targets, such as landmines and unexploded ordnance (UXO), the target signatures are typically a strong function of environmental and historical circumstances. Consequently, it is difficult to constitute a universal training set for design of detection or classification algorithms. In this paper, we develop an efficient procedure by which information-theoretic concepts are used to design the basis functions and training set, directly from the site-specific measured data. Specifically, assume that measured data (e.g., induction and/or magnetometer) are available from a given site, unlabeled in the sense that it is not known a priori whether a given signature is associated with a target or clutter. For N signatures, the data may be expressed as {x/sub i/,y/sub i/}/sub i=1,N/, where x/sub i/ is the measured data for buried object i, and y/sub i/ is the associated unknown binary label (target/nontarget). Let the N x/sub i/ define the set X. The algorithm works in four steps: 1) the Fisher information matrix is used to select a set of basis functions for the kernel-based algorithm, this step defining a set of n signatures B/sub n//spl sube/X that are most informative in characterizing the signature distribution of the site; 2) the Fisher information matrix is used again to define a small subset X/sub s//spl sube/X, composed of those x/sub i/ for which knowledge of the associated labels y/sub i/ would be most informative in defining the weights for the basis functions in B/sub n/; 3) the buried objects associated with the signatures in X/sub s/ are excavated, yielding the associated labels y/sub i/, represented by the set Y/sub s/; and 4) using B/sub n/,X/sub s/, and Y/sub s/, a kernel-based classifier is designed for use in classifying all remaining buried objects. This framework is discussed in detail, with example results presented for an actual buried-UXO site.

The independent component analysis (ICA) with a single quadratic constraint on each source signal or column of the mixing matrix is extended to the case of multiple quadratic constraints. The criterion of Joint Approximate Diagonalization of Eigen-matrices (JADE) is used to measure the statistical independence. A new algorithm is derived to maximize the JADE criterion subject to the multiple quadratic constraints, using the augmented Lagrangian method. The extension offers the freedom to design various combinations of quadratic constraints. Examples include simultaneously constraining a source signal and the corresponding column of the mixing matrix, and two-sided constraints on the source signals or columns of the mixing matrix. Example results are provided to demonstrate the effectiveness of the algorithm.

The DNA microarray technique offers an ability to analyze the expression profile of a genome. The complex correlation between the large number of genes present in the genome undermines straightforward understanding of their functionality. In this paper, we have proposed a pair of modeling schemes to recognize the functional identities of the known genes. In Independent Component Analysis (ICA), each of the microarray signals is modeled as a linear combination of some underlying independent components having specific biological interpretation. The second algorithm, Partial Least Squares (PLS) is proposed to identify the latent functional units contributing to drug sensitivity from the microarray data. Applications of this research include prediction of drug responses based on gene expressions, and also to identify the function(s) of a new gene. We consider the Rosetta compendium data set with yeast gene profiles, and the NCI-60 data set of human gene expressions as a function of drug type (cancer drugs are considered).

An approach to identifying ground targets from sequential high-range-resolution (HRR) radar signatures is presented. A hidden Markov model (HMM) is employed to model the sequential information contained in multi-aspect target signatures. Dominant range-amplitude features are extracted via RELAX for dimension reduction. A new distance measure is incorporated into the HMM to allow a direct matching operation in the feature domain without requiring interpolation. The approach is applied to the dataset of ten MSTAR targets and is shown to yield an average identification rate of 90.3% using sequential information from 6 degree angular spans.

A new scheme of radar target recognition based on parameterized high resolution range profiles (PHRRP) is presented in this paper. A novel criterion called generalized-weighted-normalized correlation (GWNC) is proposed for measuring the similarity between PHRRP's. By properly choosing the parameter of the mainlobe width in GWNC, aspect sensitivity of PHRRP's can be reduced without sacrificing their discriminative power. Performance of the scheme is evaluated using a dataset of three scaled aircraft models. The experimental results show that by using GWNC, only a small number of most dominant scatterers can achieve the same recognition rates as HRRP's, thus leading to a significant data reduction for the recognition system.