The overall student enrollment has declined over the past eight years in US. The administrators in enrollment management are applying ever more sophisticated analytical techniques in efforts to determine the characteristics of students that are most probable to enroll upon offering them monetary incentives (i.e., financial aid and/or scholarship). In this paper we present a framework that reveals the characteristics of such students using different machine learning models. Particularly, we use a genetic algorithm optimization method along with three different classification models: logistic regression (LR), support vector machines (SVMs) and bayesian networks (BNs). The results show that students with particular gender, ethnicity, socioeconomic and academic backgrounds are most probable to enroll upon offering them institutional money. We validated our results using data of actual students from a large public Tier 1 research university.

We introduce a class of statistics for characterizing the periphery of a distribution, and show that these statistics are particularly valuable for problems in target detection. Because so many detection algorithms are rooted in Gaussian statistics, we concentrate on ellipsoidal models of high-dimensional data distributions (that is to say: covariance matrices), but we recommend several alternatives to the sample covariance matrix that more efficiently model the periphery of a distribution, and can more effectively detect anomalous data samples.

We establish a general oracle inequality for clipped approximate minimizers of regularized empirical risks and apply this inequality to support vector machine (SVM) type algorithms. We then show that for SVMs using Gaussian RBF kernels for classification this oracle inequality leads to learning rates that are faster than the ones established in [9]. Finally, we use our oracle inequality to show that a simple parameter selection approach based on a validation set can yield the same fast learning rates without knowing the noise exponents which were required to be known a-priori in [9].

We describe polynomial--time algorithms that produce approximate solutions with guaranteed accuracy for a class of QP problems that are used in the design of support vector machine classifiers. These algorithms employ a two--stage process where the first stage produces an approximate solution to a dual QP problem and the second stage maps this approximate dual solution to an approximate primal solution. For the second stage we describe an O(n log n) algorithm that maps an approximate dual solution with accuracy (2(2Km)1/2+8(λ)1/2)-2 λ εp2 to an approximate primal solution with accuracy εp where n is the number of data samples, Kn is the maximum kernel value over the data and λ > 0 is the SVM regularization parameter. For the first stage we present new results for decomposition algorithms and describe new decomposition algorithms with guaranteed accuracy and run time. In particular, for τ-rate certifying decomposition algorithms we establish the optimality of τ = 1/(n-1). In addition we extend the recent τ = 1/(n-1) algorithm of Simon (2004) to form two new composite algorithms that also achieve the τ = 1/(n-1) iteration bound of List and Simon (2005), but yield faster run times in practice. We also exploit the τ-rate certifying property of these algorithms to produce new stopping rules that are computationally efficient and that guarantee a specified accuracy for the approximate dual solution. Furthermore, for the dual QP problem corresponding to the standard classification problem we describe operational conditions for which the Simon and composite algorithms possess an upper bound of O(n) on the number of iterations. For this same problem we also describe general conditions for which a matching lower bound exists for any decomposition algorithm that uses working sets of size 2. For the Simon and composite algorithms we also establish an O(n2) bound on the overall run time for the first stage. Combining the first and second stages gives an overall run time of O(n2(ck + 1)) where ck is an upper bound on the computation to perform a kernel evaluation. Pseudocode is presented for a complete algorithm that inputs an accuracy εp and produces an approximate solution that satisfies this accuracy in low order polynomial time. Experiments are included to illustrate the new stopping rules and to compare the Simon and composite decomposition algorithms.

Although Gaussian radial basis function (RBF) kernels are one of the most often used kernels in modern machine learning methods such as support vector machines (SVMs), little is known about the structure of their reproducing kernel Hilbert spaces (RKHSs). In this work, two distinct explicit descriptions of the RKHSs corresponding to Gaussian RBF kernels are given and some consequences are discussed. Furthermore, an orthonormal basis for these spaces is presented. Finally, it is discussed how the results can be used for analyzing the learning performance of SVMs.

One way to describe anomalies is by saying that anomalies are not concentrated. This leads to the problem of finding level sets for the data generating density. We interpret this learning problem as a binary classification problem and compare the corresponding classification risk with the standard performance measure for the density level problem. In particular it turns out that the empirical classification risk can serve as an empirical performance measure for the anomaly detection problem. This allows us to compare different anomaly detection algorithms empirically, i.e. with the help of a test set. Furthermore, by the above interpretation we can give a strong justification for the well-known heuristic of artificially sampling "labeled" samples, provided that the sampling plan is well chosen. In particular this enables us to propose a support vector machine (SVM) for anomaly detection for which we can easily establish universal consistency. Finally, we report some experiments which compare our SVM to other commonly used methods including the standard one-class SVM.

In this paper we study Multiple Instance Learning, a variant of the standard classification problem. We demonstrate the utility of an empirical risk minimization approach allowing for a straightforward classification treatment of the problem. In addition we consider simple data dependent hypothesis classes that allow efficient minimization of the empirical loss function and the development of bounds on the estimation error. Our empirical results are competitive with those of the most successful previously published methods.

We show that anomaly detection can be interpreted as a binary classifi- cation problem. Using this interpretation we propose a support vector machine (SVM) for anomaly detection. We then present some theoret- ical results which include consistency and learning rates. Finally, we experimentally compare our SVM with the standard one-class SVM.

This paper introduces a learning problem related to the task of converting printed documents to ASCII text files. The goal of the learning procedure is to produce a function that maps documents to restoration techniques in such a way that on average the restored documents have minimum optical character recognition error. We derive a general form for the optimal function and use it to motivate the development of a nonparametric method based on nearest neighbors. We also develop a direct method of solution based on empirical error minimization for which we prove a finite sample bound on estimation error that is independent of distribution. We show that this empirical error minimization problem is an extension of the empirical optimization problem for traditional M-class classification with general loss function and prove computational hardness for this problem. We then derive a simple iterative algorithm called generalized multiclass ratchet (GMR) and prove that it produces an optimal function asymptotically (with probability 1). To obtain the GMR algorithm we introduce a new data map that extends Kesler's construction for the multiclass problem and then apply an algorithm called Ratchet to this mapped data, where Ratchet is a modification of the Pocket algorithm . Finally, we apply these methods to a collection of documents and report on the experimental results.

This article proves that the task of computing near-optimal weights for sigmoidal nodes under the L1 regression norm is NP-Hard. For the special case where the sigmoid is piecewise linear, we prove a slightly stronger result: that computing the optimal weights is NP-Hard. These results parallel that for the one-node pattern recognition problem—that determining the optimal weights for a threshold logic node is also intractable. Our results have important consequences for constructive algorithms that build a regression model one node at a time. It suggests that although such methods are (in principle) capable of producing efficient size representations (Barron, 1993; Jones, 1992), finding such representations may be computationally intractable. These results holds only in the deterministic sense; that is, they do not exclude the possibility that such representations may be found efficiently with high probability. In fact it motivates the use of heuristic or randomized algorithms for this problem.

This paper presents a computationally efficient algorithm for function approximation with piecewise linear sigmoidal nodes. A one hidden layer network is constructed one node at a time using the well-known method of fitting the residual. The task of fitting an individual node is accomplished using a new algorithm that searches for the best fit by solving a sequence of quadratic programming problems. This approach offers significant advantages over derivative-based search algorithms (e.g., backpropagation and its extensions). Unique characteristics of this algorithm include: finite step convergence, a simple stopping criterion, solutions that are independent of initial conditions, good scaling properties and a robust numerical implementation. Empirical results are included to illustrate these characteristics.

The authors have developed an interactive environment for the creation and maintenance of dynamic, active multimedia-based teaching mechanisms. The environment is designed to be user-friendly and to facilitate the creation of educational material. This tool has already been used to create courses in multidimensional signal processing (MDSP) by researchers working together while geographically separated.

Characteristics of error surfaces for the multilayer perceptron neural network that help explain why learning techniques that use hill-climbing methods are so slow in these networks and also provide insights into techniques to speed learning are examined. First, the surface has a stair-step appearance with many very flat and very steep regions. When the number of training samples is small there is often a one-to-one correspondence between individual training samples and the steps on the surface. As the number of samples increases, the surface becomes smoother. In addition the surface has flat regions that extend to infinity in all directions, making it dangerous to apply learning algorithms that perform line searches. The magnitude of the gradients on the surface strongly supports the need for floating-point representations during learning. The consequences of various weight initialization techniques are also discussed.>

A statistical analysis of a change detector for motion detection based on image modeling of a difference picture is presented. The approach is founded upon the fact that complete silhouettes of moving objects can be segmented at each frame time using a change detector based on image modeling of the difference picture. The change detection problem can be treated as a signal detection problem in which very little is known about the signal to be detected. The exact modeling of the background event is necessary for a robust change detector which can adapt to changing environments. The governing statistics for change measures, which have a form of sum of squares of the difference picture, can be inferred from the assumed Gaussian distribution of the stationary background in the difference picture and are used for calculation of the adaptive decision threshold for the change detector. It is shown that the adaptive decision threshold derived from the change measure minimizes the total probability of error as measured in false alarms and missed detections. Experimental results that support the underlying statistical analysis are presented.>

P-type nodes are capable of computing the exact a posteriori probability for the multiclass Gaussian problem. A theory that extends to problems where some classes are given by a sum of multiple Gaussian kernels is presented. The corresponding weight solution can actually be achieved through learning since this weight solution is a minima of the mean squared error criterion function. A special type of problem, called the 1/ΣMproblem, which demonstrates the capabilities of P-type nodes is presented. The authors show that the P-type node can solve a number of interesting problems, including theI-4i-16Iproblem, the XOR problem, a multimodal non-Gaussian problem, and a one-class classifier problem

Describes a family of adaptive change detectors for detecting and segmenting moving objects in video scenes. These change detectors employ several techniques to allow them to adjust to changing image conditions. One of these is a dynamic reference frame which allows the system to adapt to changing backgrounds and produces an unambiguous target silhouette for classification. Another is a scheme to estimate system noise and adjust detection thresholds accordingly to satisfy a chosen performance criterion. Finally, adaptive normalization is used to maintain constant image contrast.>

This paper presents an adaptive scheme for the detection and identification of the sinusoidal componments in a received signal. It uses a recursive ALE (Adaptive Line Enhancer) which is implemented as the sum of constrained second-order bandpass filters. Although not as robust as other non-recursive schemes for detection and identification, it is far more efficient with respect to the number of computations required.

The purpose of this paper is to present a simple adaptive algorithm for detecting and tracking a sinusoid in broadband noise, while at the same time improving its signal-to-noise ratio (SNR).