Neural network ensemble is a learning paradigm where a collection of neural networks is trained for the same task. In this paper, the relationship between the generalization ability of the neural network ensemble and the correlation of the individual neural networks constituting the ensemble is analyzed in the context of combining neural regression estimators, which reveals that ensembling a selective subset of trained networks is superior to ensembling all the trained networks in some cases. Based on such recognition, an approach named GASEN is proposed. GASEN trains a number of individual neural networks at first. Then it assigns random weights to the individual networks and employs a genetic algorithm to evolve those weights so that they can characterize to some extent the importance of the individual networks in constituting an ensemble. Finally it selects an optimum subset of individual networks based on the evolved weights to make up the ensemble. Experimental results show that, comparing with a popular ensemble approach, i.e., averaging all, and a theoretically optimum selective ensemble approach, i.e. enumerating, GASEN has preferable performance in generating ensembles with strong generalization ability in relatively small computational cost. This paper also analyzes the working mechanism of GASEN from the view of error-ambiguity decomposition, which reveals that GASEN improves generalization ability mainly through reducing the average generalization error of the individual neural networks constituting the ensemble.

In this paper, a statistics based approach named STARE (statistics-based rule extraction) that is designed to extract symbolic rules from trained neural networks is proposed. STARE deals with continuous attributes in a unique way so that not only different attributes could be discretized to different number of clusters but also unnecessary discretization could be avoided. STARE introduces statistics to the generation and evaluation of priority rules that have concise appearance. Since it is independent of the network architectures and training algorithms, STARE could be applied to diversified neural classifiers. Experimental results show that rules extracted via STARE are comprehensible, compact and accurate.

Neural network technology has not been fully utilised in data mining. There are two reasons for this. Firstly, most neural algorithms need long-term training, and cannot perform incremental learning. Secondly, the knowledge learned by neural networks is concealed within a large quantity of connections. We have develop a neural network method to mine typhoon knowledge which overcomes those two obstacles. Experimental results show that our method shows considerable promise.