We study the Bayesian process to estimate the features of the environment. We focus on two aspects of the Bayesian process: how estimation error depends on the prior distribution of features and how the prior distribution can be learned from experience. The accuracy of the perception is underestimated when each feature of the environment is considered independently because many different features of the environment are usually highly correlated and the estimation error greatly depends on the correlations. The self-consistent learning process renews the prior distribution of correlated features jointly with the estimation of the environment. Here, maximum a posteriori probability (MAP) estimation decreases the effective dimensions of the feature vector. There are critical noise levels in self-consistent learning with MAP estimation, that cause hysteresis behaviors in learning. The self-consistent learning process with stochastic Bayesian estimation (SBE) makes the presumed distribution of environmental features converge to the true distribution for any level of channel noise. However, SBE is less accurate than MAP estimation. We also discuss another stochastic method of estimation, SBE2, which has a smaller estimation error than SBE without hysteresis.

We study the discrimination capability of spike time sequences using the Chernoff distance as a metric. We assume that spike sequences are generated by renewal processes and study how the Chernoff distance depends on the shape of interspike interval (ISI) distribution. First, we consider a lower bound to the Chernoff distance because it has a simple closed form. Then we consider specific models of ISI distributions such as the gamma, inverse gaussian (IG), exponential with refractory period (ER), and that of the leaky integrate-and-fire (LIF) neuron. We found that the discrimination capability of spike times strongly depends on high-order moments of ISI and that it is higher when the spike time sequence has a larger skewness and a smaller kurtosis. High variability in terms of coefficient of variation (CV) does not necessarily mean that the spike times have less discrimination capability. Spike sequences generated by the gamma distribution have the minimum discrimination capability for a given mean and variance of ISI. We used series expansions to calculate the mean and variance of ISIs for LIF neurons as a function of the mean input level and the input noise variance. Spike sequences from an LIF neuron are more capable of discrimination than those of IG and gamma distributions when the stationary voltage level is close to the neuron's threshold value of the neuron.

We study learning from example~by a population of neural networks.A group of single layer perceptions with discrete weights learn from a two-layer neural network.Each member is trained independently from the same or independent example sets.They vote for an answer of new problems.We calculate the generalization performance of the group decision by majority vote.The generalization error decreases to a minimum at a certain number of examples and increases again.

We study the learning from examples in a partially connected single layer perceptron and a two-layer network. Partially connected student networks learn from fully connected teacher networks. We study the generalization in the annealed approximation. We consider a single layer perceptron with binary weights. When a student is weakly diluted, there is a first order phase transition from the poor learning to the good learning state similar to that of fully connected perceptron. With a strong dilution, the first order phase transition disappears and the generalization error decreases continuously. We also study learning of a two-layer committee machine with binary weights. Contrary to the perceptron learning, there always exist a first order transition irrespective of dilution. The permutation symmetry is broken at the transition point and the generalization error is reduced to a non-zero minimum value.