An automated, rapid classification of transient events detected in the modern synoptic sky surveys is essential for their scientific utility and effective follow-up using scarce resources. This presents some unusual challenges: the data are sparse, heterogeneous and incomplete; evolving in time; and most of the relevant information comes not from the data stream itself, but from a variety of archival data and contextual information (spatial, temporal, and multi-wavelength). We are exploring a variety of novel techniques, mostly Bayesian, to respond to these challenges, using the ongoing CRTS sky survey as a testbed. The current surveys are already overwhelming our ability to effectively follow all of the potentially interesting events, and these challenges will grow by orders of magnitude over the next decade as the more ambitious sky surveys get under way. While we focus on an application in a specific domain (astrophysics), these challenges are more broadly relevant for event or anomaly detection and knowledge discovery in massive data streams.

In this paper we make several contributions towards accelerating approximate Bayesian structural inference for non-decomposable GGMs. Our first contribution is to show how to efficiently compute a BIC or Laplace approximation to the marginal likelihood of non-decomposable graphs using convex methods for precision matrix estimation. This optimization technique can be used as a fast scoring function inside standard Stochastic Local Search (SLS) for generating posterior samples. Our second contribution is a novel framework for efficiently generating large sets of high-quality graph topologies without performing local search. This graph proposal method, which we call Neighborhood Fusion" (NF), samples candidate Markov blankets at each node using sparse regression techniques. Our final contribution is a hybrid method combining the complementary strengths of NF and SLS. Experimental results in structural recovery and prediction tasks demonstrate that NF and hybrid NF/SLS out-perform state-of-the-art local search methods, on both synthetic and real-world datasets, when realistic computational limits are imposed."

We present a discrete spectral framework for the sparse or cardinality-constrained solution of a generalized Rayleigh quotient. This NP-hard combinatorial optimization problem is central to supervised learning tasks such as sparse LDA, feature selection and relevance ranking for classification. We derive a new generalized form of the Inclusion Principle for variational eigenvalue bounds, leading to exact and optimal sparse linear discriminants using branch-and-bound search. An efficient greedy (approximate) technique is also presented. The generalization performance of our sparse LDA algorithms is demonstrated with real-world UCI ML benchmarks and compared to a leading SVM-based gene selection algorithm for cancer classification.

We present a technique to generate an illumination subspace for arbitrary 3D faces based on the statistics of measured illuminations under variable lighting conditions from many subjects. A bilinear model based on the higher-order singular value decomposition is used to create a compact illumination subspace given arbitrary shape parameters from a parametric 3D face model. Using a fitting procedure based on minimizing the distance of the input image to the dynamically changing illumination subspace, we reconstruct a shape-specific illumination subspace from a single photograph. We use the reconstructed illumination subspace in various face recognition experiments with variable lighting conditions and obtain accuracies which are very competitive with previous methods that require specific training sessions or multiple images of the subject.

Sparse PCA seeks approximate sparse "eigenvectors" whose projections capture the maximal variance of data. As a cardinality-constrained and non-convex optimization problem, it is NP-hard and is encountered in a wide range of applied fields, from bio-informatics to finance. Recent progress has focused mainly on continuous approximation and convex relaxation of the hard cardinality constraint. In contrast, we consider an alternative discrete spectral formulation based on variational eigenvalue bounds and provide an effective greedy strategy as well as provably optimal solutions using branch-and-bound search. Moreover, the exact methodology used reveals a simple renormalization step that improves approximate solutions obtained by any continuous method. The resulting performance gain of discrete algorithms is demonstrated on real-world benchmark data and in extensive Monte Carlo evaluation trials.

We present an approach that incorporates appearance-adaptive models in a particle filter to realize robust visual tracking and recognition algorithms. Tracking needs modeling interframe motion and appearance changes, whereas recognition needs modeling appearance changes between frames and gallery images. In conventional tracking algorithms, the appearance model is either fixed or rapidly changing, and the motion model is simply a random walk with fixed noise variance. Also, the number of particles is typically fixed. All these factors make the visual tracker unstable. To stabilize the tracker, we propose the following modifications: an observation model arising from an adaptive appearance model, an adaptive velocity motion model with adaptive noise variance, and an adaptive number of particles. The adaptive-velocity model is derived using a first-order linear predictor based on the appearance difference between the incoming observation and the previous particle configuration. Occlusion analysis is implemented using robust statistics. Experimental results on tracking visual objects in long outdoor and indoor video sequences demonstrate the effectiveness and robustness of our tracking algorithm. We then perform simultaneous tracking and recognition by embedding them in a particle filter. For recognition purposes, we model the appearance changes between frames and gallery images by constructing the intra- and extrapersonal spaces. Accurate recognition is achieved when confronted by pose and view variations.

We propose a novel local appearance modeling method for object detection and recognition in cluttered scenes. The approach is based on the joint distribution of local feature vectors at multiple salient points and factorization with the independent component analysis (ICA). The resulting densities are simple multiplicative distributions modeled through adaptive Gaussian mixture models. This leads to computationally tractable joint probability densities, which can model high-order dependencies. Furthermore, different models are compared based on appearance, color and geometry information. Also, the combination of all of them results in a hybrid model, which obtains the best results using the COIL-100 object database. Our technique has been tested under different natural and cluttered scenes with different degrees of occlusions with promising results. Finally, a large statistical test with the MNIST digit database is used to demonstrate the improved performance obtained by explicit modeling of high-order dependencies.

A novel local appearance modeling method for object detection and recognition in cluttered scenes. The approach is based on the joint distribution of local feature vectors at multiple salient points and their factorization with independent component analysis (ICA). The resulting densities are simple multiplicative distributions modeled through adaptative Gaussian mixture models. This leads to computationally tractable joint probability densities which can model high-order dependencies. Our technique has been initially tested under different natural and cluttered scenes with different degrees of occlusions yielding promising results. In this work, we provide a large statistical test with the MNIST digit database in order to demonstrate the improved performance obtained by explicit modeling of higher-order dependencies.

This paper presents an improved method for simultaneous tracking and recognition of human faces from video, where a time series model is used to resolve the uncertainties in tracking and recognition. The improvements mainly arise from three aspects: (i) modeling the inter-frame appearance changes within the video sequence using an adaptive appearance model and an adaptive-velocity motion model; (ii) modeling the appearance changes between the video frames and gallery images by constructing intra- and extra-personal spaces; and (iii) utilization of the fact that the gallery images are in frontal views. By embedding them in a particle filter, we are able to achieve a stabilized tracker and an accurate recognizer when confronted by pose and illumination variations.

We present visualization and layout algorithms that can enhance informal story-telling using personal digital data such as photos in a face-to-face social setting. In order to build a more intuitive browser for retrieval, navigation and story-telling, we introduce a novel optimized layout technique for large image sets, which respects (context-sensitive) mutual similarities as visualized on a shared 2-D display (a table-top). The experimental results show a more perceptually intuitive and informative visualization of traditional CBIR-based retrievals, providing not only a better understanding of the query context but also aiding the user informing new queries. A framework for user-modeling is also introduced and tested. This allows the system to adapt to the user's preference and integrate relevance feedback.

We introduce a novel learning algorithm for binary classi(cid:12)cation with hyperplane discriminants based on pairs of training points from opposite classes (dyadic hypercuts). This algorithm is further extended to nonlinear discriminants using kernel functions satisfy- ing Mercer’s conditions. An ensemble of simple dyadic hypercuts is learned incrementally by means of a con(cid:12)dence-rated version of Ad- aBoost, which provides a sound strategy for searching through the (cid:12)nite set of hypercut hypotheses. In experiments with real-world datasets from the UCI repository, the generalization performance of the hypercut classi(cid:12)ers was found to be comparable to that of SVMs and k-NN classi(cid:12)ers. Furthermore, the computational cost of classi(cid:12)cation (at run time) was found to be similar to, or bet- ter than, that of SVM. Similarly to SVMs, boosted dyadic kernel discriminants tend to maximize the margin (via AdaBoost). In contrast to SVMs, however, we o(cid:11)er an on-line and incremental learning machine for building kernel discriminants whose complex- ity (number of kernel evaluations) can be directly controlled (traded o(cid:11) for accuracy).

Investigates the use of linear and nonlinear principal manifolds for learning low-dimensional representations for visual recognition. Several leading techniques - principal component analysis (PCA), independent component analysis (ICA) and nonlinear kernel PCA (KPCA) - are examined and tested in a visual recognition experiment using 1,800+ facial images from the "FERET" (FacE REcognition Technology) database. We compare the recognition performance of nearest-neighbor matching with each principal manifold representation to that of a maximum a-posteriori (MAP) matching rule using a Bayesian similarity measure derived from dual probabilistic subspaces. The experimental results demonstrate the simplicity, computational economy and performance superiority of the Bayesian subspace method over principal manifold techniques for visual matching.

Nonlinear support vector machines (SVMs) are investigated for appearance-based gender classification with low-resolution "thumbnail" faces processed from 1,755 images from the FERET (FacE REcognition Technology) face database. The performance of SVMs (3.4% error) is shown to be superior to traditional pattern classifiers (linear, quadratic, Fisher linear discriminant, nearest-neighbor) as well as more modern techniques, such as radial basis function (RBF) classifiers and large ensemble-RBF networks. Furthermore, the difference in classification performance with low-resolution "thumbnails" (21/spl times/12 pixels) and the corresponding higher-resolution images (84/spl times/48 pixels) was found to be only 1%, thus demonstrating robustness and stability with respect to scale and the degree of facial detail.

This paper proposes a novel image modeling scheme for object detection and localization. Object appearance is modeled by the joint distribution of k-tuple salient point feature vectors which are factorized component-wise after an independent component analysis (ICA). Also, we propose a distance-sensitive histograming technique for capturing spatial dependencies. The advantages over existing techniques include the ability to model non-rigid objects (at the expense of modeling accuracy) and the flexibility in modeling spatial relationships. Experiments show that ICA does improve modeling accuracy and detection performance. Experiments in object detection in cluttered scenes have demonstrated promising results.

In traditional content-based image retrieval (CBIR), the retrieved images are displayed in order of decreasing similarities from the query and can be considered as a 1-D display. In this paper a novel optimized technique is proposed to visualize the retrieved images not only in order of their decreasing similarities but also according to their mutual similarities visualized on a 2-D screen. Principle Component Analysis (PCA) is first performed on the retrieved images to project the images from the original high dimensional feature space to 2-D screen. The result of PCA analysis is denoted as a PCA Splat. To minimize the overlap between images, a constrained nonlinear optimization approach is used. The experimental results show a more perceptually intuitive and informative visualization of the retrieval results. The proposed technique not only provides a better understanding of the query results but also aids the user in forming a new query.

Support vector machines (SVM) are investigated for visual gender classification with low-resolution "thumbnail" faces (21-by-12 pixels) processed from 1755 images from the FERET face database. The performance of SVM (3.4% error) is shown to be superior to traditional pattern classifiers (linear, quadratic, Fisher linear discriminant, nearest-neighbor) as well as more modern techniques such as radial basis function (RBF) classifiers and large ensemble-RBF networks. SVM also out-performed human test subjects at the same task: in a perception study with 30 human test subjects, ranging in age from mid-20s to mid-40s, the average error rate was found to be 32% for the "thumbnails" and 6.7% with higher resolution images. The difference in performance between low- and high-resolution tests with SVM was only 1%, demonstrating robustness and relative scale invariance for visual classification.

To date most "content-based image retrieval" (CBIR) techniques rely on global attributes such as color or texture histograms which unfortunately ignore the spatial composition of the image. We present an alternative image retrieval system based on the principle that it is the user who is most qualified to specify the query "content" and not the computer. With our system, the user can select multiple "regions-of-interest" and can specify the relevance of their spatial layout in the retrieval process. We also derive similarity bounds on histogram distances for pruning the database search. This experimental system was found to be superior to global indexing techniques as measured by statistical sampling of multiple users' "satisfaction" ratings.

In this paper, support vector machines (SVMs) are investigated for visual gender classification with low-resolution "thumbnail" faces (21-by-12 pixels) processed from 1,755 images from the FERET face database. The performance of SVMs (3.4% error) is shown to be superior to traditional pattern classifiers (linear, quadratic, Fisher linear discriminant, nearest neighbor) as well as more modern techniques such as radial basis function (RBF) classifiers and large ensemble-RBF networks. SVMs have also been tested with high-resolution (80-by-40 pixels) images. The difference between low and high-resolution inputs with SVMs was only 1%, thus demonstrating a degree of robustness and relative scale invariance.

Support vector machines (SVM) are investigated for visual gender classification with low-resolution "thumbnail" faces (21-by-12 pixels) processed from 1755 images from the FERET face database. The performance of SVM (3.4% error) is shown to be superior to traditional pattern classifiers (linear, quadratic, Fisher linear discriminant, nearest-neighbor) as well as more modern techniques such as radial basis function (RBF) classifiers and large ensemble-RBF networks. Surprisingly, SVM also out-performed human test subjects at the same task: in an experimental study involving 30 human test subjects ranging in age from mid-20s to mid-40s, the average error rate was 32% for the same "thumbnails" and 6.7% with high-resolution images (still nearly twice the error rate of SVM). The difference between low and high-resolution inputs with SVM was only 1% thus demonstrating a degree of robustness and relative scale invariance.

Nonlinear Support Vector Machines (SVMs) are investigated for visual sex classification with low resolution "thumbnail" faces (21- by-12 pixels) processed from 1,755 images from the FE RET face database. The performance of SVMs is shown to be superior to traditional pattern classifiers (Linear, Quadratic, Fisher Linear Dis(cid:173) criminant, Nearest-Neighbor) as well as more modern techniques such as Radial Basis Function (RBF) classifiers and large ensemble(cid:173) RBF networks. Furthermore, the SVM performance (3.4% error) is currently the best result reported in the open literature.

We investigate the use of linear and nonlinear principal manifolds for learning low dimensional representations for visual recognition. Three techniques: principal component analysis (PCA), independent component analysis (ICA) and nonlinear PCA (NLPCA) are examined and tested in a visual recognition experiment using a large gallery of facial images from the "FERET" database. We compare the recognition performance of a nearest neighbour matching rule with each principal manifold representation to that of a maximum a posteriori (MAP) matching rule using a Bayesian similarity measure derived from probabilistic subspaces, and demonstrate the superiority of the latter.

Moghaddam et al. previously (1996, 1998) advanced a new technique for direct visual matching of images for the purposes of face recognition and image retrieval, using a probabilistic measure of similarity, based primarily on a Bayesian (MAP) analysis of image differences. The performance advantage of this probabilistic matching technique over standard Euclidean nearest-neighbor eigenspace matching was recently demonstrated using results from DARPA's 1996 "FERET" face recognition competition, in which our probabilistic matching algorithm was found to be the top performer. We have further developed a simple method of replacing the rather costly computation of nonlinear (online) Bayesian similarity measures by the relatively inexpensive computation of linear (off-line) subspace projections and simple Euclidean norms, thus resulting in a significant computational speed-up for implementation with very large image databases.

We present an unsupervised technique for visual learning, which is based on density estimation in high-dimensional spaces using an eigenspace decomposition. Two types of density estimates are derived for modeling the training data: a multivariate Gaussian (for unimodal distributions) and a mixture-of-Gaussians model (for multimodal distributions). Those probability densities are then used to formulate a maximum-likelihood estimation framework for visual search and target detection for automatic object recognition and coding. Our learning technique is applied to the probabilistic visual modeling, detection, recognition, and coding of human faces and nonrigid objects, such as hands.

We demonstrate real-time face tracking and pose estimation in an unconstrained office environment with an active foveated camera. Using vision routines previously implemented for an interactive environment, we determine the spatial location of a user's head and guide an active camera to obtain foveated images of the face. Faces are analyzed using a set of eigenspaces indexed over both pose and world location. Closed loop feedback from the estimated facial location is used to guide the camera when a face is present in the foveated view. Our system can detect the head pose of an unconstrained user in real-time as he or she moves about an open room.

We describe a novel technique for face recognition based on deformable intensity surfaces which incorporates both the shape and texture components of the 2D image. The intensity surface of the facial image is modeled as a deformable 3D mesh in (z, y, I(x, y)) space. Using an efficient technique for matching two surfaces (in terms of the analytic modes of vibration), we obtain a dense correspondence field (or 3D warp) between two images. The probability distributions of two classes of warps are then estimated from training data: interpersonal and extrapersonal variations. These densities are then used in a Bayesian framework for image matching and recognition. Experimental results with facial data from the US Army FERET database demonstrate an increased recognition rate over the previous best methods.

We propose a probabilistic similarity measure for direct image matching based on a Bayesian analysis of image deformations. We model two classes of variation in object appearance: intra-object and extra-object. The probability density functions for each class are then estimated from training data and used to compute a similarity measure based on the a posteriori probabilities. Furthermore, we use a novel representation for characterizing image differences using a deformable technique for obtaining pixel-wise correspondences. This representation, which is based on a deformable 3D mesh in XYI-space, is then experimentally compared with two simpler representations: intensity differences and optical flow. The performance advantage of our deformable matching technique is demonstrated using a typically hard test set drawn from the US Army's FERET face database.

We present a fully automatic system for 2D model-based image coding of human faces for potential applications such as video telephony, database image compression, and face recognition. The system operates by locating a face in the input image, normalizing its scale and geometry and representing it in terms of a compact parametric image model obtained with a Karhunen-Loeve basis. This leads to a compact representation of the face that can be used for both recognition as well as image compression. Good-quality facial images are automatically generated using approximately 100-bytes worth of encoded data. The system has been successfully tested on a database of nearly 2000 facial photographs.

We present an unsupervised technique for visual learning which is based on density estimation in high-dimensional spaces using an eigenspace decomposition. Two types of density estimates are derived for modeling the training data: a multivariate Gaussian (for a unimodal distributions) and a multivariate Mixture-of-Gaussians model (for multimodal distributions). These probability densities are then used to formulate a maximum-likelihood estimation framework for visual search and target detection for automatic object recognition. This learning technique is tested in experiments with modeling and subsequent detection of human faces and non-rigid objects such as hands.>

We present an unsupervised technique for visual target modeling which is based on density estimation in high-dimensional spaces using an eigenspace decomposition. A computationally efficient and optimal estimator for a multivariate Gaussian distribution is derived. This density estimate is then used to formulate a maximum likelihood estimation framework for visual search and target detection. Our learning technique is applied to the probabilistic visual modeling and subsequent detection of facial features and is shown to be superior to matched filtering.

We describe experiments with eigenfaces for recognition and interactive search in a large-scale face database. Accurate visual recognition is demonstrated using a database of O(10/sup 3/) faces. The problem of recognition under general viewing orientation is also examined. A view-based multiple-observer eigenspace technique is proposed for use in face recognition under variable pose. In addition, a modular eigenspace description technique is used which incorporates salient features such as the eyes, nose and mouth, in an eigenfeature layer. This modular representation yields higher recognition rates as well as a more robust framework for face recognition. An automatic feature extraction technique using feature eigentemplates is also demonstrated.>

The application of fractal random process models and their related scaling parameters as features in the analysis and segmentation of clutter in high-resolution, polarimetric synthetic aperture radar (SAR) imagery is demonstrated. Specifically, the fractal dimension of natural clutter sources, such as grass and trees, is computed and used as a texture feature for a Bayesian classifier. The SAR shadows are segmented in a separate manner using the original backscatter power as a discriminant. The proposed segmentation process yields a three-class segmentation map for the scenes considered in this study (with three clutter types: shadows, trees, and grass). The difficulty of computing texture metrics in high-speckle SAR imagery is addressed. In particular, a two-step preprocessing approach consisting of polarimetric minimum speckle filtering followed by noncoherent spatial averaging is used. The relevance of the resulting segmentation maps to constant-false-alarm-rate (CFAR) radar target detection techniques is discussed.>