In machine learning, one of the methods to solve multiclass classification problems is a framework called Error-Correcting Output Codes (ECOC), which constructs a multiclass classifier by combining a lot of binary classifiers. ECOC assigns binary codewords to each category, and the multiclass classification performance varies depending on the code. In this study, we treat each element of the codeword as a random variable and evaluate the average performance of ECOC. As a result, for$M$class classification if the number of binary classifiers is$O(\log M)$, then the average error probability of various codes approaches that of MAP estimation. We show that the important points are the ratio between the number of binary classifiers and$\log M$and the difference between the maximum posterior probability and the second highest posterior probability for the categories.

Error-correcting output coding (ECOC) is a method for constructing a multi-valued classifier using a combination of given binary classifiers. ECOC can estimate the correct category by other binary classifiers even if the output of some binary classifiers is incorrect based on the framework of the coding theory. The code word table representing the combination of these binary classifiers is important in ECOC. ECOC is known to perform well experimentally on real data. However, the complexity of the classification problem makes it difficult to analyze the classification performance in detail. For this reason, theoretical analysis of ECOC has not been conducted. In this study, if a binary classifier outputs the estimated posterior probability with errors, then this binary classifier is said to be noisy. In contrast, if a binary classifier outputs the true posterior probability, then this binary classifier is said to be noiseless. For a theoretical analysis of ECOC, we discuss the optimality for the code word table with noiseless binary classifiers and the error rate for one with noisy binary classifiers. This evaluation result shows that the Hamming distance of the code word table is an important indicator.

Consider M-valued (M$\geq$3) classification systems realized by combination of N(N$\geq\lceil\log_{2}$M$\rceil$) binary classifiers. Such a construction method is called an Error Correcting Output Code (ECOC). First, focusing on a Reed-Muller (RM) code, we derive a modified RM (mRM) code to make it suitable for the ECOC. Using the mRM code and the Hadamard matrix, we introduce a simplex code which is one of the powerful equidistant codes. Next, from the viewpoint of system evaluation model, we evaluate the ECOC by using constructive coding described above. We show that they have desirable properties such as Flexible, Elastic, and Effective Elastic as M becomes large, by employing analytical formulas and experiments.

Error-Correcting Output Coding (ECOC) is a method for constructing a multi-valued classifier using a combination of binary classifiers. The effectiveness of ECOC for multivalued classification problems has been demonstrated by many experimental evaluations. Therefore, classification performance have strongly depended on the data under consideration, and it is not clear what kind of combinations of binary classifiers have good performance. Motivated by this fact, the authors have clarified the best combination of binary classifiers that makes ECOC, assuming a situation in which each binary classifier can estimate the true posterior probability. They also have proposed a total framework for analytical evaluation when a binary classifier outputs an estimated posterior probability that approximates the true posterior probability. These studies established a framework for evaluating the theoretical performance of ECOC.Based on these findings, this study discusses the theoretical performance of ECOC from the upper bound perspective. The results showed that increasing the Hamming distance between code words can blackuce the error rate. We then evaluate various combinations of binary classifiers based on analytical evaluation.

With the development of information technology, a wide variety of data have been accumulated, and there are many methods for analyzing such data. In this study, we model the input data and the metrics between the data based on the assumption that each metric is generated from a continuous latent variable. Specifically, we assume that the input data are generated using low-dimensional latent variables and their projection matrices. We describe a method for estimating the latent variables. Because the generative model defined in this study cannot obtain the Q function analytically, we use the Monte Carlo EM algorithm to approximate the Q function and investigate an efficient parameter estimation method. Experiments using artificial data and the 20 newsgroups dataset demonstrate the effectiveness of the proposed method.

We consider a distributed learning problem in which the computation is carried out on a system consisting of a master node and multiple worker nodes. In such systems, the existence of slow-running machines called stragglers will cause a significant decrease in performance. Recently, coding theoretic framework, which is named Gradient Coding (GC), for mitigating stragglers in distributed learning has been established by Tandon et al. Most studies on GC are aiming at recovering the gradient information completely assuming that the Gradient Descent (GD) algorithm is used as a learning algorithm. On the other hand, if the Stochastic Gradient Descent (SGD) algorithm is used, it is not necessary to completely recover the gradient information, and its unbiased estimator is sufficient for the learning. In this paper, we propose a distributed SGD scheme using Low Density Generator Matrix (LDGM) codes. In the proposed system, it may take longer time than existing GC methods to recover the gradient information completely, however, it enables the master node to obtain a high-quality unbiased estimator of the gradient at low computational cost and it leads to overall performance improvement.

To solve multiple classification problems with $M (\geq$ 3) categories, many studies have been devoted using $N (\geq\ \lceil\log_{2}M\rceil)$ binary $(\{0,1\})$ classifiers, where these systems are known as binary Error-Correcting Output Codes (binary ECOC). As an extended version of the binary ECOC, the ternary $(\{0,\ *,\ 1\})$ ECOC have also been discussed, where ternary classifiers classify data into positive examples when the element is 1, into negative examples when the element is 0, and no classification when the element is $*$. In this paper, we discuss the ternary ECOC system from the view point of the system evaluation model based on rate-distortion function. First, we discuss a table of M code words with length N which is given by a ternary matrix W of M rows and N columns. Next, by leveraging the benchmark data for multiclass document classification which is widely used in Japan, the relationships between the probability of classification error Peand the number of the ternary classifiers N for a given M are experimentally investigated. In addition, by assuming the M-dimensional Normal distribution for a classification data model, the relationship between Peand N for a given M is also examined. Finally, we show by the system evaluation model that the ternary ECOC systems have desirable properties such as “Flexible”, “Elastic”, and “Effective Elastic”, when M becomes large.

In this manuscript, we investigate a collaborative filtering method to characterize consumption behavior of customers and services with various attributes for marketing. We assume that each customer and service have the invisible attribute which is called latent class. Assuming a combination of attribute values of a customer and service is classified to a latent class, furthermore, we propose a new Bayesian statistical model that consumption behavior is probabilistically arise based on a latent class combination of a customer, service and attribute values. Then, we show the method to estimate parameters of a statistical model based on the variational Bayes method and the mean field approximation. Consequently, we show the effectiveness of the proposed model and the estimation method by simulation.

In this manuscript, we investigate a collaborative filtering method to characterize consumption behavior (or evaluation) of customers (or users) and services (or items) for marketing. Assuming that each customer and service have the invisible attribute, which is called latent class, we propose a new Bayesian statistical model that consumption behavior is probabilistically arise based on a latent class combination of a customer and service. Then, we show the method to estimate parameters of a statistical model based on the variational Bayes method and the mean field approximation. Consequently, we show the effectiveness of the proposed model and the estimation method by simulation and analyzing actual data.

In this paper, we focus on pattern recognition based on the vector space model. As one of the methods, distance metric learning is known for the learning metric matrix under the arbitrary constraint. Generally, it uses iterative optimization procedure in order to gain suitable distance structure by considering the statistical characteristics of training data. Most of the distance metric learning methods estimate suitable metric matrix from all pairs of training data. However, the computational cost is considerable if the number of training data increases in this setting. To avoid this problem, we propose the way of learning distance metric by using the each category centroid. To verify the effectiveness of proposed method, we conduct the simulation experiment by using benchmark data.

This paper proposes a new latent class model to represent user segments in a marketing model of electric commerce sites. The aspect model proposed by T. Hofmann is well known and is also called the probabilistic latent semantic indexing (PLSI) model. Although the aspect model is one of effective models for information retrieval, it is difficult to interpret the meaning of the probability of latent class in terms of marketing models. It is desirable that the probability of latent class means the size of customer segment for the purpose of marketing research. Through this formulation, the simulation analysis to dissect the several situations become possible by using the estimated model. The impact of the strategy that we contact to the specific customer segment and make effort to increase the number of customers belonging to this segment can be predicted by using the model demonstrating the size of customer segment. This paper proposes a new model whose probability parameter of latent variable means the rate of users with the same preference in market. By applying the proposed model to the data of an internet portal site for job hunting, the effectiveness of our proposal is verified.

F. P. Preparata et al. proposed a fault diagnosis model (PMC model) to find all fault units in the multicomputer system by using outcomes that each unit tests some other units. T. Kohda proposed a highly structured(HS) system and the syndrome analysis method(SAM) to diagnose from local testing results. In this paper, we introduce the maximum a posteriori probability algorithm(MAPDA) for the HS system in the probabilistic fault model. Analyzing the MAPDA, we show that the SAM is closer to the MAPDA as the fault probability becomes smaller. Finally, we show the robustness of the SAM in the HS system.

In this paper, we focus on pattern recognition based on the vector space model with the high dimensional and sparse data. One of the pattern recognition methods is metric learning which learns a metric matrix by using the iterative optimization procedure. However most of the metric learning methods tend to cause overfitting and increasing computational time for high dimensional and sparse settings. To avoid these problems, we propose the method of l1regularized metric learning by using the algorithm of alternating direction method of multiplier (ADMM) in the supervised setting. The effectiveness of our proposed method is clarified by classification experiments by using the Japanese newspaper article and UCI machine learning repository. And we show proposed method is the special case of the statistical sparse covariance selection.

In this paper, a new attack model in which the number of colluders are distributed according to a certain probability distribution is introduced. Two classes of collusion attacks which include well-known collusion attacks in the context of multimedia fingerprinting are provided. For these two attack classes, achievable rates for the unknown size of the actual colluders are derived. Based on the derived achievable rates, achieve rates for some particular attacks are investigated. For the AND attack, the bound derived in this paper coincides with the previous known bound, although the attack model in this paper does not assume that the decoder knows the actual number of colluders. Moreover, for the averaging attack, it is clarified that derived achievable rate is larger than previously known bound with random linear codes.

F.P.Preparata et al. have proposed a fault diagnosis model to find all faulty units in the multicomputer system by using outcomes which each unit tests some other units. In this paper, for probabilistic diagnosis models, we show an efficient diagnosis algorithm to obtain a posteriori probability that each of units is faulty given the test outcomes. Furthermore, we propose a method to analyze the diagnostic error probability of this algorithm.

An iterative decoding algorithm of rate-compatible punctured low-density parity-check (RCP-LDPC) codes of high coding rates is developed. This algorithm performs a predetermined recovering process of punctured bits sums at the beginning of each iteration of the standard belief-propagation (BP) decoding algorithm. By propagating messages of two punctured bits sum, this algorithm can recover much more punctured bits than the standard BP decoding algorithm. It is shown that the proposed algorithm is applicable for RCP-LDPC codes of higher coding rates with little increase of decoding complexity.