In transfer learning we aim to solve new problems using fewer examples using information gained from solving related problems. Transfer learning has been successful in practice, and extensive PAC analysis of these methods has been de- veloped. However it is not yet clear how to define relatedness between tasks. This is considered as a major problem as it is conceptually troubling and it makes it unclear how much information to transfer and when and how to transfer it. In this paper we propose to measure the amount of information one task contains about another using conditional Kolmogorov complexity between the tasks. We show how existing theory neatly solves the problem of measuring relatedness and transferring the ‘right’ amount of information in sequential transfer learning in a Bayesian setting. The theory also suggests that, in a very formal and precise sense, no other reasonable transfer method can do much better than our Kolmogorov Complexity theoretic transfer method, and that sequential transfer is always justi- fied. We also develop a practical approximation to the method and use it to transfer information between 8 arbitrarily chosen databases from the UCI ML repository.

The recent detection of mirror neurons in monkeys suggests that brains encode parts of an observed action in a similar way they encode own actions. This paper models such a mirror system by means of adaptive resonance theory (ART) artificial neural networks coupling them in a manner similar to ARTMAP systems. Particularly, a bidirectional ARTMAP system (BiARTMAP) is created. The system associates executed actions with consequent action-effects. The associative structure gives the system mirror capabilities: on the one hand, perceived environmental changes cause an action association. On the other hand, activated action patterns cause the expectation of resulting environmental change. We also show that many other proposed cognitive processes relate to the BiARTMAP architecture. Future work includes the incorporation of situational dependencies, the combination of BiARTMAP with vector associative maps (VAMs), and the integration of BiARTMAP in a behavioral module enabling anticipatory behavior. Application wise, BiARTMAP can be applied as a general classifier and/or associative network.

The deep layers of the superior colliculus (SC) integrate information from multiple senses to initiate orienting movements in vertebrate animals. A probabilistic model of the SC based on an interpretation of the neuroscientific data has been proposed by Anastasio et. al. (2000). By incorporating this SC model, in the form of an artificial neural network, as the decision mechanism for a system with two senses, hearing and vision, we have constructed and tested a self-aiming camera (SAC). SAC senses and directs its lens toward the best "target" currently in the environment at any moment. Experiments were performed with SAC using several algorithms for combining the multisensory data as a comparison against the SC model. Generally, the SC model is superior in dealing with low amplitude signals and at least equal to any ad hoc model for the full range of unimodal and bimodal targets.

This paper proposes a new scheme for unsupervised multi-temporal sequence pattern extraction. The main idea of the scheme is iterative coarse to fine data examination. We decompose a pattern into ambiguous subpatterns and distinguishable sub-patterns (DSP). In each iteration, we coarsely examine the training temporal signal sequence by training an Elman neural network. The trained Elman network is used to select the DSP candidate set. Then, we look at the training signals around the DSPs and use maximum likelihood criteria to expand them into whole patterns. We cut out the new found patterns from the training signal sequence and repeat the whole procedure until no more new patterns are found. The experimental result shows this method promising.

We present a new hierarchical network architecture that integrates the outputs of recurrent ANN. The purpose of this architecture is to apply decomposition of time-series learning tasks (using self-organization on multi-channel input). Our approach yields the variance-reducing benefits of techniques such as stacked generalization, but exploits the ability of abstract targets to be factored based upon preprocessing, feature extraction, or multimodal sensor constraints. This research demonstrates how prior information can be applied to learn factorial structure from time series, to build a mixture of recurrent ANN.

The purpose of the paper is to propose a map-like artificial neural network for temporal sequence pattern clustering. The map construction in our presentation is related to the self-organizing map (SOM) idea. The SOM idea was originally designed for static pattern learning and recognition. It has been found efficient for organizing high dimensional data sets. One of the biggest limitations of the traditional SOM technique is caused by its static characteristics. We propose a new neural network construction model and its corresponding training algorithm based on traditional SOM training technology and backpropagation training technology. It overcomes the static limitation of traditional SOM and tries to reach a new stage for dynamic pattern clustering, and recognition. At the end of the paper, we give some experimental results for testing this proposed method on real speech data.

In this paper, we investigate a form of modular neural network for classification with (a) pre-separated input vectors entering its specialist (expert) networks, (b) specialist networks which are self-organized (radial-basis function or self-targeted feedforward type) and (c) which fuses (or integrates) the specialists with a single-layer net. When the modular architecture is applied to spatiotemporal sequences, the Specialist Nets are recurrent; specifically, we use the Input Recurrent type. The Specialist Networks (SNs) learn to divide their input space into a number of equivalence classes defined by self-organized clustering and learning using the statistical properties of the input domain. Once the specialists have settled in their training, the Fusion Network is trained by any supervised method to map to the semantic classes. We discuss the fact that this architecture and its training is quite distinct from the hierarchical mixture of experts (HME) type as well as from stacked generalization. Because the equivalence classes to which the SNs map the input vectors are determined by the natural clustering of the input data, the SNs learn rapidly and accurately. The fusion network also trains rapidly by reason of its simplicity. We argue, on theoretical grounds, that the accuracy of the system should be positively correlated to the product of the number of equivalence classes for all of the SNs. This network was applied, as an empirical test case, to the classification of melodies presented as direct audio events (temporal sequences) played by a human and subject, therefore, to biological variations. The audio input was divided into two modes: (a) frequency (or pitch) variation and (b) rhythm, both as functions of time. The results and observations show the technique to be very robust and support the theoretical deductions concerning accuracy.