This paper investigates mixture of multilayer perceptron (MLP) regressions. Although mixture of MLP regressions (MoMR) can be a strong fitting model for noisy data, the research on it has been rare. We employ soft mixture approach and use the Expectation-Maximization (EM) algorithm as a basic learning method. Our learning method goes in a double-looped manner; the outer loop is controlled by the EM and the inner loop by MLP learning method. Given data, we will have many models; thus, we need a criterion to select the best. Bayesian Information Criterion (BIC) is used here because it works nicely for MLP model selection. Our experiments showed that the proposed MoMR method found the expected MoMR model as the best for artificial data and selected the MoMR model having smaller error than any linear models for real noisy data.

There are two ways to learn radial basis function (RBF) networks: one-stage and two-stage learnings. Recently a very powerful one-stage learning method called RBF-SSF has been proposed, which can stably find a series of excellent solutions, making good use of singular regions, and can monotonically decrease training error along with the increase of hidden units. RBF-SSF was built by applying the SSF (singularity stairs following) paradigm to RBF networks; the SSF paradigm was originally and successfully proposed for multilayer perceptrons. Although RBF-SSF has the strong capability to find excellent solutions, it required a lot of time mainly because it computes the Hessian. This paper proposes a faster version of RBF-SSF called RBF-SSF(pH) by introducing partial calculation of the Hessian. The experiments using two datasets showed RBF-SSF(pH) ran as fast as usual one-stage learning methods while keeping the excellent solution quality.

Equivalence structure (ES) extraction can allow for finding correspondence relations between different sequential datasets. A K -dimensional ES is a set of K -tuples to specify K -dimensional sequences that are considered equivalent. Whether or not two K -dimensional sequences are equivalent is decided based on comparisons of all of their subsequences. ES extraction can be used for preprocessing for transfer learning or imitation learning, as well as an analysis of multidimensional sequences. A recently proposed method called incremental search (IS) was much faster than brute-force search. However, IS can still take a long time to obtain ESs, because ESs obtained by IS can be subsets of other ESs and such subsets must be removed in the process. In this paper, we propose a new fast method called pairwise incremental search (PIS). In the process of PIS, the aforementioned problem about subsets of ESs does not exist, because the elements of ESs are searched pairwise. As shown by results of two experiments we conducted, PIS was 48 times faster than IS in an experiment using synthetic datasets and 171 times faster in an experiment using motion capture datasets.

An equivalence structure (ES) is a set of K-tuples composed of IDs that indicate K D sequences considered similar. The similarity is calculated by comparing continuous subsequences taken from KD sequences. When comparing one of such continuous subsequences with that of a different tuple, the starting point of the former subsequence can differ from the latter subsequence. The extraction of ES's can be applied for various purposes, such as feature extraction, the analysis of neural network behavior, and imitation learning. However, the extraction requires comparisons of all possible KD sequences taken from N sequences; that is, the number of the possible sequences is the number of K-permutations of N. Therefore, the comparisons cause combinatorial explosion with K. In this paper, we propose a new method to extract ES's increasing K, and show that, in our experiments, our method allowed us to extract ES's much faster than the brute-force search, which is the only existing method.

A complex-valued multilayer perceptron has the capability to represent complicated periodicity. We employ a very powerful learning method called C-SSF for learning a complex-valued multilayer perceptron. C-SSF finds a series of excellent solutions through successive learning. In deterministic chaos, long-term prediction is considered impossible. We apply C-SSF to two kinds of deterministic chaos and evaluate the learning and prediction performance of C-SSF.

In the search space of a complex-valued multilayer perceptron (C-MLP) there exist flat areas called singular regions. Although singular regions cause serious stagnation of learning, there exist descending paths from the regions. Based on this observation, a completely new learning method for C-MLP, called C-SSF1.0, was proposed, making good use of singular regions to stably find excellent solutions of successive C-MLPs. However, the method takes longer time than an existing method. This paper proposes a faster version of C-SSF called C-SSF1.1 by introducing search pruning. Our experiments showed the proposed method ran a few times faster than C-SSF1.0 without losing excellent solutions quality.