Deep learning methods, which include feature extraction in the training process, are achieving success in pattern recognition and machine learning fields but require huge parameter setting, and need the selection from various methods. On the contrary, Support Vector Machines (SVMs) have been popularly used in these fields in light of the simple algorithm and solid reasons based on the learning theory. However, it is difficult to improve recognition performance in SVMs beyond a certain level of capacity, in that higher dimensional feature space can only assure the linear separability of data as opposed to separation of the data manifolds themselves. We propose a new framework of kernel machine that generates essentially linearly separable kernel features. Our method utilizes pretraining process based on a kernel generative model and the mean field Fisher score with a higher-order autocorrelation kernel. Thus derived features are to be separated by a liner SVM, which exhibits far better generalization performance than any kernel-based SVMs. We show the experiments on the face detection using the appearance based approach, and that our method can attain comparable results with the state-of-the-art face detection methods based on AdaBoost, SURF, and cascade despite of smaller data size and no preprocessing.

The support vector machine (SVM) problem is a convex quadratic programming problem which scales with the training data size. If the training size is large, the problem cannot be solved by straighforward methods. The large-scale SVM problems are tackled by applying chunking (decomposition) technique. The quadratic programming problem involves a square matrix which is called kernel matrix is positive semi-definite. That is, the rank of the kernel matrix is less than or equal to its size. In this paper we discuss a method that can exploit the low-rank of the kernel matrix, and an interior-point method (IPM) is efficiently applied to the global (large-sized) problem. The method is based on the technique of second-order cone programming (SOCP). This method reformulates the SVM's quadratic programming problem into the second-order cone programming problem. The SOCP method is much faster than efficient softwares SVMlightand SVMTorch if the rank of the kernel matrix is small enough compared to the training set size or if the kernel matrix can be approximated by a low-rank positive semidefinite matrix.

We propose a data dependent learning method for the support vector machine. This method is based on the technique of second order cone programming. We reformulate the SVM quadratic problem into the second order cone problem. The proposed method requires decomposing the kernel matrix of SVM optimization problem. In this paper we apply Cholesky decomposition method. Since the kernel matrix is positive semi definite, some columns of the decomposed matrix diminish. The performance of the proposed method depends on the reduction of dimensionality of the decomposed matrix. Computational results show that when the columns of decomposed matrix are small enough, the proposed method is much faster than the quadratic programming solver LOQO.

Learning curves exhibit a diversity of behaviors such as phase transition. However, the understanding of learning curves is still extremely limited, and existing theories can give the impression that without empirical studies (e.g., cross validation), one can probably do nothing more than qualitative interpretations. In this note, we propose a theory of learning curves based on the idea of reducing learning problems to hypothesis-testing ones. This theory provides a simple approach that is potentially useful for predicting and interpreting (a diversity of) learning curve behaviors qualitatively and quantitatively, and it applies to finite training sample size and finite learning machine and for learning situations not necessarily within the Bayesian framework. We illustrate the results by examining some exponential learning curve behaviors observed in Cohn and Tesauro (1992)'s experiment.

In this paper, we motivate the need for estimating bounds on learning curves of average-case learning algorithms when they perform the worst on training samples. We then apply the method of reducing learning problems to hypothesis testing ones to investigate the learning curves of a so-called ill-disposed learning algorithm in terms of a system complexity, the Boolean interpolation dimension. Since the ill-disposed algorithm behaves worse than ordinal ones, and the Boolean interpolation dimension is generally bounded by the number of system weights, the results can apply to interpreting or to bounding the worst-case learning curve in real learning situations. This study leads to a new understanding of the worst-case generalization in real learning situations, which differs significantly from that in the uniform learnable setting via Vapnik-Chervonenkis (VC) dimension analysis. We illustrate the results with some numerical simulations.

A tight bound on the generalization performance of concept learning is shown by a novel approach. Unlike the existing theories, the new approach uses no assumption on large sample size as in Bayesian approach and does not consider the uniform learnability as in the VC dimension analysis. We analyze the generalization performance of some particular learning algorithm that is not necessarily well behaved, in the hope that once learning curves or sample complexity of this algorithm is obtained, it is applicable to real learning situations. The result is expressed in a dimension called Boolean interpolation dimension, and is tight in the sense that it meets the lower bound requirement of Baum and Haussler. The Boolean interpolation dimension is not greater than the number of modifiable system parameters, and definable for almost all the real-world networks such as backpropagaton networks and linear threshold multilayer networks. It is shown that the generalization error follows from a beta distribution of parameters m, the number of training examples, and d, the Boolean interpolation dimension. This implies that for large d, the learning results tend to the average-case result, known as the self-averaging property of the learning. The bound is shown to be applicable to the practical learning algorithms that can be modeled by Gibbs algorithm with a uniform prior. The result is also extended to the case of inconsistent learning.

In this paper, we describe a method which enables us to study the average generalization performance of learning directly via hypothesis testing inequalities. The resulting theory provides a unified viewpoint of average-case learning curves of concept learning and regression in realistic learning problems not necessarily within the Bayesian framework. The advantages of the theory are that it alleviates the practical pessimism frequently claimed for the results of the Vapnik-Chervonenkis (VC) theory and its alike, and provides general insights into generalization. Besides, the bounds on learning curves are directly related to the number of adjustable system weights. Although the theory is based on an approximation assumption, and cannot apply to the worst-case learning setting, the precondition of the assumption is mild, and the approximation itself is only a sufficient condition for the validity of the theory. We illustrate the results with numerical simulations, and apply the theory to examining the generalization ability of combination of neural networks.

Telecommunications networks have been designed to maintain predetermined service levels for time-varying traffic loads. This inevitably causes redundant investments, and thus, there are possibilities for further economization by adaptively controlling the systems. In order to establish the optimum strategy of controlling telecommunications networks, it is essential to study time dependent traffic characteristics. This paper analyzes a system in which a time-dependent Poisson input is offered to a finite primary trunk group, and the overflow traffic is carried by an infinite secondary trunk group. Methods for calculating the mean and variance of the overflow traffic (the number of calls existing in the secondary group) are proposed, including an approximate evaluation for which accuracy is examined. Calculated examples are given for various time dependent inputs, and properties of the overflow traffic are discussed. The methods and results presented will be useful for analysis and optimum design of the network systems with time dependent traffic.

A simple approximate formula for estimating individual call losses in overflow systems is derived using the interrupted Poisson process. The formula presents good accuracy in the single-overflow model, and it is also applicable to the multioverflow model. It is expected that the present formula is useful for the optimization of systems under given service criteria.

This paper reveals the general structure of the asymptotic expansion for the Erlang loss function and its derivative. A recursive formula for the coefficients in the expansion is given and some of numerical examples are presented. The truncation error of the expansion is estimated and charts for calculation with the error less than 0.1 percent are shown. The present formula facilitates the calculation by saving the computation time, and it is useful for the optimization problems of switching systems and other applications.