In training two networks with shared weights on tasks that are analogous, the shared weights tend to encode the high level similarity between the two tasks. In this work, knowledge transfer was studies by investigating the structural mapping with the proposed Identical Elements Neural Network which features shared hidden layers. First, two networks were trained simultaneously on structurally analogous tasks. After it converged, cross computation was performed. The result shows that structural mapping between the two tasks can be observed from the activated outputs.

Classifier ensembles have been shown to enhance classification as the expected error rate of a set of voting classifiers is known to be less than the expected error rate (r) of one of the individual members of the set, assuming r is uniform across the set and that r < 0.5. This effect is diminished by error redundancies among the individual classifiers; thus, diversity in error patterns across items among classifiers promotes performance. Here we introduce a second classification task to neural network classifiers designed to promote diversity among classifiers while minimizing performance on the main task. The result is an improvement in ensemble performance even if there is a small decrease in individual performance.

Algorithms for reducing variance in neural network prediction using committee and resampling techniques bootstrap and cross-validation are presented. Their effectiveness is tested on data sets with different levels of noise and on medical diagnosis data sets. The methods are most effective when the noise level in the data is high or the size of the learning set is small, which consequently produces high variance. The algorithms will not be of much help in cases where the error of prediction is mainly due to bias. An increase in bias is observed due to smaller effective learning size in the bootstrap and crossvalidation committee. The impact of increased bias on the performance ranges from negligible to completely undermining the benefit of reducing the variance.