Identification of module structure in brain functional networks is a promising way to obtain novel insights into neural information processing, as modules correspond to delineated brain regions in which interactions are strongly increased. Tracking of network modules in time-varying brain functional networks is not yet commonly considered in neuroscience despite its potential for gaining an understanding of the time evolution of functional interaction patterns and associated changing degrees of functional segregation and integration. We introduce a general computational framework for extracting consensus partitions from defined time windows in sequences of weighted directed edge-complete networks and show how the temporal reorganization of the module structure can be tracked and visualized. Part of the framework is a new approach for computing edge weight thresholds for individual networks based on multiobjective optimization of module structure quality criteria as well as an approach for matching modules across time steps. By testing our framework using synthetic network sequences and applying it to brain functional networks computed from electroencephalographic recordings of healthy subjects that were exposed to a major balance perturbation, we demonstrate the framework's potential for gaining meaningful insights into dynamic brain function in the form of evolving network modules. The precise chronology of the neural processing inferred with our framework and its interpretation helps to improve the currently incomplete understanding of the cortical contribution for the compensation of such balance perturbations.

In science and engineering mathematical modeling serves as a tool for the understanding of processes and systems and as a testing bed for several hypotheses, e.g., concerning the testing (prediction) of functional limits by simulations. A brief overview of current modeling strategies in brain research is given, spatial scales ranging from single neuron to large scale activity of and between brain regions are considered. The models are mainly time-invariant. Three time-variant modeling strategies, which enable a model-based signal analysis, are described and applied to large scale signals. The first is derived from adaptive filter theory and covers linear system and linear as well as nonlinear process models. The second is based on modeled brain source signals, i.e., the inverse problem must be solved. The third strategy consists of a generalization of Dynamic Causal Modeling (DCM); DCM is frequently used for analysis of directed interactions between brain structures. Examples are derived from neonatal electroencephalography (EEG) monitoring of preterm and fullterm newborns. A further example is based on high-density recordings of event-related potentials (ERPs) and shows the combination of a time-variant ERP-based source model, as a part of a realistic head model, with a multivariate process model to analyze the time evolution of interactions between source processes before and during the execution of a complex motoric task. In two other examples hemodynamic signals (functional magnetic resonance imaging-fMRI) are utilized for analysis of interactions between brain regions, where nonlinear, multivariate models are used.

This work addresses the problem of improving the generalization capabilities of continuous recurrent neural networks. The learning task is transformed into an optimal control framework in which the weights and the initial network state are treated as unknown controls. A new learning algorithm based on a variational formulation of Pontrayagin's maximum principle is proposed. Under reasonable assumptions, its convergence is discussed. Numerical examples are given that demonstrate an essential improvement of generalization capabilities after the learning process of a dynamic network.

This paper addresses the problem of training trajectories by means of continuous recurrent neural networks whose feedforward parts are multilayer perceptrons. Such networks can approximate a general nonlinear dynamic system with arbitrary accuracy. The learning process is transformed into an optimal control framework where the weights are the controls to be determined. A training algorithm based upon a variational formulation of Pontryagin's maximum principle is proposed for such networks. Computer examples demonstrating the efficiency of the given approach are also presented.

The problem of learning multiple continuous trajectories by means of recurrent neural networks with (in general) time-varying weights is addressed. The learning process is transformed into an optimal control framework where both the weights and the initial network state to be found are treated as controls. For such a task, a learning algorithm is proposed which is based on a variational formulation of Pontryagin's maximum principle. The convergence of this algorithm, under reasonable assumptions, is also investigated. Numerical examples of learning nontrivial two-class problems are presented which demonstrate the efficiency of the approach proposed.

This study addresses a problem of learning multiple continuous trajectories by means of recurrent neural networks with (in general) time-varying weights. The learning task is transformed into an optimal control problem where both the weights and initial network state to be found are treated as controls. Based on a variational formulation of Pontryagin's maximum principle, a new learning algorithm is proposed which generalizes the one given given previously (1999). Under reasonable assumptions, its convergence is also discussed. A numerical example of learning a two-class problem is presented which demonstrates the efficiency of the approach proposed.

This paper is concerned with a general learning (with arbitrary criterion and state-dependent constraints) of continuous trajectories by means of recurrent neural networks with time-varying weights. The learning process is transformed into an optimal control framework, where the weights to be found are treated as controls. A new learning algorithm based on a variational formulation of Pontryagin's maximum principle is proposed. This algorithm is shown to converge, under reasonable conditions, to an optimal solution. The neural networks with time-dependent weights make it possible to efficiently find an admissible solution (i.e., initial weights satisfying state constraints) which then serves as an initial guess to carry out a proper minimization of a given criterion. The proposed methodology may be directly applicable to both classification of temporal sequences and to optimal tracking of nonlinear dynamic systems. Numerical examples are also given which demonstrate the efficiency of the approach presented.

A method for the construction of optimal structures for feedforward neural networks is introduced. On the basis of a construction of a graph of network structures and an evaluation value which is assigned to each of them, an heuristic search algorithm can be installed on this graph. The application of the A*-algorithm ensures, in theory, both the optimality of the solution and the optimality of the search. For several examples, a comparison between the new strategy and the well-known cascade-correlation procedure is carried out with respect to the performance of the resulting structures.

Under idealized assumptions (infinitely large training sets, ideal training algorithms that avoid local minima and sufficient neural network (NN) structures) trained NNs realize Bayes-optimal classifiers (BOCs) with identical costs as long as the training set is representative. Training sets with relative class frequencies different from the a priori class probabilities implement nonidentical costs, but result in an identical receiver-operator-characteristic (ROC). However under real-world conditions, the equivalence of NNs trained with different learn sets does not hold. Some effects of limited sample size and insufficient network structures are analyzed. For a simulated example the performances of NNs trained with different learn sets are compared. The results make clear that one has to find a trade-off between the exhaustive use of available information and the risk of getting stuck in local minima in order to choose optimal learn sets.

Our earlier algorithm (1996) on neural clustering networks (proved to provide globally optimal cluster solutions) is improved herein for considerably faster convergence. The neural network approach and the sort of clustering (vector quantization) are explained. Then the new method is introduced. Computational results are given which demonstrate that the globally optimal solution may be reliable, obtained by the fast algorithm whose convergence rate is of the same order as that of the K-means clustering algorithm. In specific cases the new algorithm may be even faster than K-means. Latent risks of a poor performance of K-means are visualized and consequences for the potential use of vector quantization are discussed.