As a fundamental mathematical problem in the field of machine learning, the linear separability test still lacks a theoretically complete and computationally efficient method. This paper proposes and proves a sufficient and necessary condition for linear separability test based on a sphere model. The advantage of this test method is two-fold: (1) it provides not only a qualitative test of linear separability but also a quantitative analysis of the separability of linear separable instances; (2) it has low time cost and is more efficient than existing test methods. The proposed method is validated through a large number of experiments on benchmark datasets and artificial datasets, demonstrating both its correctness and efficiency.

The purpose of supervised learning with temporal encoding for spiking neurons is to make the neurons emit a specific spike train encoded by the precise firing times of spikes. If only running time is considered, the supervised learning for a spiking neuron is equivalent to distinguishing the times of desired output spikes and the other time during the running process of the neuron through adjusting synaptic weights, which can be regarded as a classification problem. Based on this idea, this letter proposes a new supervised learning method for spiking neurons with temporal encoding; it first transforms the supervised learning into a classification problem and then solves the problem by using the perceptron learning rule. The experiment results show that the proposed method has higher learning accuracy and efficiency over the existing learning methods, so it is more powerful for solving complex and real-time problems.

This paper, with an aim at improving neural networks' generalization performance, proposes an effective neural network ensemble approach with two novel ideas. One is to apply neural networks' output sensitivity as a measure to evaluate neural networks' output diversity at the inputs near training samples so as to be able to select diverse individuals from a pool of well-trained neural networks; the other is to employ a learning mechanism to assign complementary weights for the combination of the selected individuals. Experimental results show that the proposed approach could construct a neural network ensemble with better generalization performance than that of each individual in the ensemble combining with all the other individuals, and than that of the ensembles with simply averaged weights.

This paper proposes a set of adaptive learning rules for binary feedforward neural networks (BFNNs) by means of the sensitivity measure that is established to investigate the effect of a BFNN's weight variation on its output. The rules are based on three basic adaptive learning principles: the benefit principle, the minimal disturbance principle, and the burden-sharing principle. In order to follow the benefit principle and the minimal disturbance principle, a neuron selection rule and a weight adaptation rule are developed. Besides, a learning control rule is developed to follow the burden-sharing principle. The advantage of the rules is that they can effectively guide the BFNN's learning to conduct constructive adaptations and avoid destructive ones. With these rules, a sensitivity-based adaptive learning (SBALR) algorithm for BFNNs is presented. Experimental results on a number of benchmark data demonstrate that the SBALR algorithm has better learning performance than the Madaline rule II and backpropagation algorithms.

The sensitivity of a neural network's output to its parameter variation is an important issue in both theoretical researches and practical applications of neural networks. This paper proposes a quantified sensitivity measure of the Radial Basis Function Neural Networks (RBFNNs) to input variation. The sensitivity is defined as the mathematical expectation of squared output deviations caused by input variations. In order to quantify the sensitivity, the input is treated as a statistical variable and a numerical integral technique is employed to approximately compute the expectation. Experimental verifications are run and the results show a very good agreement between the proposed sensitivity computation and computer simulation. The quantified sensitivity measure could be helpful as a general tool for evaluating RBFNNs' performance.

How to design proper architectures of neural networks for solving given problems is an important issue in neural network research. Nowadays, the existing training algorithms of neural networks only focus on adjusting neural networks' weights to improve training accuracy, and few of them adaptively adjust the networks' architecture. However, the architecture is indeed very critical for training neural networks to have high performance and needs to be coped with in the training process. In this paper, we present a new training algorithm of Madalines, which takes not only weight but also architecture adjusting into consideration. The algorithm can thus train Madalines with smaller architecture and higher generalization ability. Experimental results have demonstrated that our algorithm is effective.