We propose an analysis scheme which addresses the Parzen-window and mixture model methods for estimating the probability density function of data points in feature space. Both methods construct the estimate as a sum of kernel functions (usually Gaussians). By adding an entropy-like function we prove that the probability distribution is a product of a weight function and a shape distribution. This Weight-Shape decomposition leads to new interpretations of established clustering algorithms. Furthermore, it suggests the construction of three different clustering schemes, which are based on gradient-ascent flow of replica points in feature space. Two of these are Quantum Clustering and the Mean-Shift algorithm. The third algorithm is based on maximal-entropy. In our terminology they become Maximal Shape Clustering, Maximal Probability Clustering and Maximal Weight Clustering, correspondingly. We demonstrate the different methods and compare them to each other on one artificial example and two natural data sets. We also apply the Weight-Shape decomposition to image analysis. The shape distribution acts as an edge detector. It serves to generate contours, as demonstrated on face images. Furthermore, it allows for defining a convolutional Shape Filter.

We present a novel analysis of compositional order (CO) based on the occurrence of Frequent amino-acid Triplets (FTs) that appear much more than random in protein sequences. The method captures all types of proteomic compositional order including single amino-acid runs, tandem repeats, periodic structure of motifs and otherwise low complexity amino-acid regions. We introduce new order measures, distinguishing between 'regularity', 'periodicity' and 'vocabulary', to quantify these phenomena and to facilitate the identification of evolutionary effects. Detailed analysis of representative species across the tree-of-life demonstrates that CO proteins exhibit numerous functional enrichments, including a wide repertoire of particular patterns of dependencies on regularity and periodicity. Comparison between human and mouse proteomes further reveals the interplay of CO with evolutionary trends, such as faster substitution rate in mouse leading to decrease of periodicity, while innovation along the human lineage leads to larger regularity. Large-scale analysis of 94 proteomes leads to systematic ordering of all major taxonomic groups according to FT-vocabulary size. This is measured by the count of Different Frequent Triplets (DFT) in proteomes. The latter provides a clear hierarchical delineation of vertebrates, invertebrates, plants, fungi and prokaryotes, with thermophiles showing the lowest level of FT-vocabulary. Among eukaryotes, this ordering correlates with phylogenetic proximity. Interestingly, in all kingdoms CO accumulation in the proteome has universal characteristics. We suggest that CO is a genomic-information correlate of both macroevolution and various protein functions. The results indicate a mechanism of genomic 'innovation' at the peptide level, involved in protein elongation, shaped in a universal manner by mutational and selective forces.

BACKGROUND: Feature selection is an important pre-processing task in the analysis of complex data. Selecting an appropriate subset of features can improve classification or clustering and lead to better understanding of the data. An important example is that of finding an informative group of genes out of thousands that appear in gene-expression analysis. Numerous supervised methods have been suggested but only a few unsupervised ones exist. Unsupervised Feature Filtering (UFF) is such a method, based on an entropy measure of Singular Value Decomposition (SVD), ranking features and selecting a group of preferred ones. RESULTS: We analyze the statistical properties of UFF and present an efficient approximation for the calculation of its entropy measure. This allows us to develop a web-tool that implements the UFF algorithm. We propose novel criteria to indicate whether a considered dataset is amenable to feature selection by UFF. Relying on formalism similar to UFF we propose also an Unsupervised Detection of Outliers (UDO) method, providing a novel definition of outliers and producing a measure to rank the "outlier-degree" of an instance.Our methods are demonstrated on gene and microRNA expression datasets, covering viral infection disease and cancer. We apply UFFizi to select genes from these datasets and discuss their biological and medical relevance. CONCLUSIONS: Statistical properties extracted from the UFF algorithm can distinguish selected features from others. UFFizi is a framework that is based on the UFF algorithm and it is applicable for a wide range of diseases. The framework is also implemented as a web-tool.The web-tool is available at: http://adios.tau.ac.il/UFFizi.

BACKGROUND: We propose a method for deriving enzymatic signatures from short read metagenomic data of unknown species. The short read data are converted to six pseudo-peptide candidates. We search for occurrences of Specific Peptides (SPs) on the latter. SPs are peptides that are indicative of enzymatic function as defined by the Enzyme Commission (EC) nomenclature. The number of SP hits on an ensemble of short reads is counted and then converted to estimates of numbers of enzymatic genes associated with different EC categories in the studied metagenome. Relative amounts of different EC categories define the enzymatic spectrum, without the need to perform genomic assemblies of short reads. RESULTS: The method is developed and tested on 22 bacteria for which there exist many EC annotations in Uniprot. Enzymatic signatures are derived for 3 metagenomes, and their functional profiles are explored.We extend the SP methodology to taxon-specific SPs (TSPs), allowing us to estimate taxonomic features of metagenomic data from short reads. Using recent Swiss-Prot data we obtain TSPs for different phyla of bacteria, and different classes of proteobacteria. These allow us to analyze the major taxonomic content of 4 different metagenomic data-sets. CONCLUSIONS: The SP methodology can be successfully extended to applications on short read genomic and metagenomic data. This leads to direct derivation of enzymatic signatures from raw short reads. Furthermore, by employing TSPs, one obtains valuable taxonomic information.

BACKGROUND: Predicting the function of a protein from its sequence is a long-standing challenge of bioinformatic research, typically addressed using either sequence-similarity or sequence-motifs. We employ the novel motif method that consists of Specific Peptides (SPs) that are unique to specific branches of the Enzyme Commission (EC) functional classification. We devise the Data Mining of Enzymes (DME) methodology that allows for searching SPs on arbitrary proteins, determining from its sequence whether a protein is an enzyme and what the enzyme's EC classification is. RESULTS: We extract novel SP sets from Swiss-Prot enzyme data. Using a training set of July 2006, and test sets of July 2008, we find that the predictive power of SPs, both for true-positives (enzymes) and true-negatives (non-enzymes), depends on the coverage length of all SP matches (the number of amino-acids matched on the protein sequence). DME is quite different from BLAST. Comparing the two on an enzyme test set of July 2008, we find that DME has lower recall. On the other hand, DME can provide predictions for proteins regarded by BLAST as having low homologies with known enzymes, thus supplying complementary information. We test our method on a set of proteins belonging to 10 bacteria, dated July 2008, establishing the usefulness of the coverage-length cutoff to determine true-negatives. Moreover, sifting through our predictions we find that some of them have been substantiated by Swiss-Prot annotations by July 2009. Finally we extract, for production purposes, a novel SP set trained on all Swiss-Prot enzymes as of July 2009. This new set increases considerably the recall of DME. The new SP set is being applied to three metagenomes: Sargasso Sea with over 1,000,000 proteins, producing predictions of over 220,000 enzymes, and two human gut metagenomes. The outcome of these analyses can be characterized by the enzymatic profile of the metagenomes, describing the relative numbers of enzymes observed for different EC categories. CONCLUSIONS: Employing SPs for predicting enzymatic activity of proteins works well once one utilizes coverage-length criteria. In our analysis, L >or= 7 has led to highly accurate results.

MOTIVATION: Feature selection methods aim to reduce the complexity of data and to uncover the most relevant biological variables. In reality, information in biological datasets is often incomplete as a result of untrustworthy samples and missing values. The reliability of selection methods may therefore be questioned. METHOD: Information loss is incorporated into a perturbation scheme, testing which features are stable under it. This method is applied to data analysis by unsupervised feature filtering (UFF). The latter has been shown to be a very successful method in analysis of gene-expression data. RESULTS: We find that the UFF quality degrades smoothly with information loss. It remains successful even under substantial damage. Our method allows for selection of a best imputation method on a dataset treated by UFF. More importantly, scoring features according to their stability under information loss is shown to be correlated with biological importance in cancer studies. This scoring may lead to novel biological insights.

Predicting the function of a protein from its sequence is a long-standing goal of bioinformatic research. While sequence similarity is the most popular tool used for this purpose, sequence motifs may also subserve this goal. Here we develop a motif-based method consisting of applying an unsupervised motif extraction algorithm (MEX) to all enzyme sequences, and filtering the results by the four-level classification hierarchy of the Enzyme Commission (EC). The resulting motifs serve as specific peptides (SPs), appearing on single branches of the EC. In contrast to previous motif-based methods, the new method does not require any preprocessing by multiple sequence alignment, nor does it rely on over-representation of motifs within EC branches. The SPs obtained comprise on average 8.4 +/- 4.5 amino acids, and specify the functions of 93% of all enzymes, which is much higher than the coverage of 63% provided by ProSite motifs. The SP classification thus compares favorably with previous function annotation methods and successfully demonstrates an added value in extreme cases where sequence similarity fails. Interestingly, SPs cover most of the annotated active and binding site amino acids, and occur in active-site neighboring 3-D pockets in a highly statistically significant manner. The latter are assumed to have strong biological relevance to the activity of the enzyme. Further filtering of SPs by biological functional annotations results in reduced small subsets of SPs that possess very large enzyme coverage. Overall, SPs both form a very useful tool for enzyme functional classification and bear responsibility for the catalytic biological function carried out by enzymes.

We study the problem of memory capacity in balanced networks of spiking neurons. Associative memories are represented by either synfire chains (SFC) or Hebbian cell assemblies (HCA). Both can be embedded in these balanced networks by a proper choice of the architecture of the network. The size w(E) of a pool in an SFC or of an HCA is limited from below and from above by dynamical considerations. Proper scaling of w(E) by radicalK, where K is the total excitatory synaptic connectivity, allows us to obtain a uniform description of our system for any given K. Using combinatorial arguments, we derive an upper limit on memory capacity. The capacity allowed by the dynamics of the system, alpha(c), is measured by simulations. For HCA, we obtain alpha(c) of order 0.1, and for SFC, we find values of order 0.065. The capacity can be improved by introducing shadow patterns, inhibitory cell assemblies that are fed by the excitatory assemblies in both memory models. This leads to a doubly balanced network, where, in addition to the usual global balancing of excitation and inhibition, there exists specific balance between the effects of both types of assemblies on the background activity of the network. For each of the memory models and for each network architecture, we obtain an allowed region (phase space) for w(E)/ radicalK in which the model is viable.

Cultured in vitro neuronal networks are known to exhibit synchronized bursting events (SBE), during which most of the neurons in the system spike within a time window of approximately 100 msec. Such phenomena can be obtained in model networks based on Markram-Tsodyks frequency-dependent synapses. In order to account correctly for the detailed behavior of SBEs, several modifications have to be implemented in such models. Random input currents have to be introduced to account for the rising profile of SBEs. Dynamic thresholds and inhomogeneity in the distribution of neuronal resistances enable us to describe the profile of activity within the SBE and the heavy-tailed distributions of interspike intervals and interevent intervals. Thus, we can account for the interesting appearance of Levy distributions in the data.

Recurrent networks can generate spatio-temporal neural sequences of very large cycles, having an apparent random behavior. Nonetheless a proximity measure between these sequences may be defined through comparison of the synaptic weight matrices that generate them. Following the dynamic neural filter (DNF) formalism we demonstrate this concept by comparing teacher and student recurrent networks of binary neurons. We show that large sequences, providing a training set well exceeding the Cover limit, allow for good determination of the synaptic matrices. Alternatively, assuming the matrices to be known, very fast determination of the biases can be achieved. Thus, a spatio-temporal sequence may be regarded as spatio-temporal encoding of the bias vector. We introduce a linear support vector machine (SVM) variant of the DNF in order to specify an optimal weight matrix. This approach allows us to deal with noise. Spatio-temporal sequences generated by different DNFs with the same number of neurons may be compared by calculating correlations of the synaptic matrices of the reconstructed DNFs. Other types of spatio-temporal sequences need the introduction of hidden neurons, and/or the use of a kernel variant of the SVM approach. The latter is being defined as a recurrent support vector network (RSVN).

A balanced network leads to contradictory constraints on memory models, as exemplified in previous work on accommodation of synfire chains. Here we show that these constraints can be overcome by introducing a 'shadow' inhibitory pattern for each excitatory pattern of the model. This is interpreted as a double- balance principle, whereby there exists both global balance between average excitatory and inhibitory currents and local balance between the currents carrying coherent activity at any given time frame. This principle can be applied to networks with Hebbian cell assemblies, leading to a high capacity of the associative memory. The number of possible patterns is limited by a combinatorial constraint that turns out to be P=0.06N within the specific model that we employ. This limit is reached by the Hebbian cell assembly network. To the best of our knowledge this is the first time that such high memory capacities are demonstrated in the asynchronous state of models of spiking neurons.

We describe a pattern acquisition algorithm that learns, in an unsuper- vised fashion, a streamlined representation of linguistic structures from a plain natural-language corpus. This paper addresses the issues of learn- ing structured knowledge from a large-scale natural language data set, and of generalization to unseen text. The implemented algorithm repre- sents sentences as paths on a graph whose vertices are words (or parts of words). Significant patterns, determined by recursive context-sensitive statistical inference, form new vertices. Linguistic constructions are rep- resented by trees composed of significant patterns and their associated equivalence classes. An input module allows the algorithm to be sub- jected to a standard test of English as a Second Language (ESL) profi- ciency. The results are encouraging: the model attains a level of per- formance considered to be “intermediate” for 9th-grade students, de- spite having been trained on a corpus (CHILDES) containing transcribed speech of parents directed to small children.

MOTIVATION: This paper introduces the application of a novel clustering method to microarray expression data. Its first stage involves compression of dimensions that can be achieved by applying SVD to the gene-sample matrix in microarray problems. Thus the data (samples or genes) can be represented by vectors in a truncated space of low dimensionality, 4 and 5 in the examples studied here. We find it preferable to project all vectors onto the unit sphere before applying a clustering algorithm. The clustering algorithm used here is the quantum clustering method that has one free scale parameter. Although the method is not hierarchical, it can be modified to allow hierarchy in terms of this scale parameter. RESULTS: We apply our method to three data sets. The results are very promising. On cancer cell data we obtain a dendrogram that reflects correct groupings of cells. In an AML/ALL data set we obtain very good clustering of samples into four classes of the data. Finally, in clustering of genes in yeast cell cycle data we obtain four groups in a problem that is estimated to contain five families. AVAILABILITY: Software is available as Matlab programs at http://neuron.tau.ac.il/~horn/QC.htm.

We investigate the formation of synfire waves in a balanced network of integrate-and-fire neurons. The synaptic connectivity of this network embodies synfire chains within a sparse random connectivity. This network can exhibit global oscillations but can also operate in an asynchronous activity mode. We analyze the correlations of two neurons in a pool as convenient indicators for the state of the network. We find, using different models, that these indicators depend on a scaling variable. Beyond a critical point, strong correlations and large network oscillations are obtained. We looked for the conditions under which a synfire wave could be propagated on top of an otherwise asynchronous state of the network. This condition was found to be highly restrictive, requiring a large number of neurons for its implementation in our network. The results are based on analytic derivations and simulations.

We describe and discuss the properties of a binary neural network that can serve as a dynamic neural filter (DNF), which maps regions of input space into spatiotemporal sequences of neuronal activity. Both deterministic and stochastic dynamics are studied, allowing the investigation of the stability of spatiotemporal sequences under noisy conditions. We define a measure of the coding capacity of a DNF and develop an algorithm for constructing a DNF that can serve as a source of given codes. On the basis of this algorithm, we suggest using a minimal DNF capable of generating observed sequences as a measure of complexity of spatiotemporal data. This measure is applied to experimental observations in the locust olfactory system, whose reverberating local field potential provides a natural temporal scale allowing the use of a binary DNF. For random synaptic matrices, a DNF can generate very large cycles, thus becoming an efficient tool for producing spatiotemporal codes. The latter can be stabilized by applying to the parameters of the DNF a learning algorithm with suitable margins.

The distributional principle according to which morphemes that occur in identical contexts belong, in some sense, to the same category [1] has been advanced as a means for extracting syntactic structures from corpus data. We extend this principle by applying it recursively, and by us- ing mutual information for estimating category coherence. The resulting model learns, in an unsupervised fashion, highly structured, distributed representations of syntactic knowledge from corpora. It also exhibits promising behavior in tasks usually thought to require representations anchored in a grammar, such as systematicity.

We propose a novel clustering method that is an extension of ideas inher- ent to scale-space clustering and support-vector clustering. Like the lat- ter, it associates every data point with a vector in Hilbert space, and like the former it puts emphasis on their total sum, that is equal to the scale- space probability function. The novelty of our approach is the study of an operator in Hilbert space, represented by the Schr¨odinger equation of which the probability function is a solution. This Schr¨odinger equation contains a potential function that can be derived analytically from the probability function. We associate minima of the potential with cluster centers. The method has one variable parameter, the scale of its Gaussian kernel. We demonstrate its applicability on known data sets. By limiting the evaluation of the Schr¨odinger potential to the locations of data points, we can apply this method to problems in high dimensions.

We present a novel kernel method for data clustering using a description of the data by support vectors. The kernel reflects a projection of the data points from data space to a high dimensional feature space. Cluster boundaries are defined as spheres in feature space, which represent complex geometric shapes in data space. We utilize this geometric representation of the data to construct a simple clustering algorithm.

We present a novel method for clustering using the support vector ma(cid:173) chine approach. Data points are mapped to a high dimensional feature space, where support vectors are used to define a sphere enclosing them. The boundary of the sphere forms in data space a set of closed contours containing the data. Data points enclosed by each contour are defined as a cluster. As the width parameter of the Gaussian kernel is decreased, these contours fit the data more tightly and splitting of contours occurs. The algorithm works by separating clusters according to valleys in the un(cid:173) derlying probability distribution, and thus clusters can take on arbitrary geometrical shapes. As in other SV algorithms, outliers can be dealt with by introducing a soft margin constant leading to smoother cluster bound(cid:173) aries. The structure of the data is explored by varying the two parame(cid:173) ters. We investigate the dependence of our method on these parameters and apply it to several data sets.

Recent imaging studies suggest that object knowledge is stored in the brain as a distributed network of many cortical areas. Motivated by these observations, we study a multimodular associative memory network, whose functional goal is to store patterns with different coding levels--patterns that vary in the number of modules in which they are encoded. We show that in order to accomplish this task, synaptic inputs should be segregated into intramodular projections and intermodular projections, with the latter undergoing additional nonlinear dendritic processing. This segregation makes sense anatomically if the intermodular projections represent distal synaptic connections on apical dendrites. It is then straightforward to show that memories encoded in more modules are more resilient to focal afferent damage. Further hierarchical segregation of intermodular connections on the dendritic tree improves this resilience, allowing memory retrieval from input to just one of the modules in which it is encoded.

We propose a method for estimating probability density functions (pdf) and conditional density functions (cdf) by training on data produced by such distributions. The algorithm employs new stochastic variables that amount to coding of the input, using a principle of entropy maximization. It is shown to be closely related to the maximum likelihood approach. The encoding step of the algorithm provides an estimate of the probability distribution. The decoding step serves as a generative mode, producing an ensemble of data with the desired distribution. The algorithm is readily implemented by neural networks, using stochastic gradient ascent to achieve entropy maximization.

We propose a simple theoretical structure of interacting integrate-and-fire neurons that can handle fast information processing and may account for the fact that only a few neuronal spikes suffice to transmit information in the brain. Using integrate-and-fire neurons that are subjected to individual noise and to a common external input, we calculate their first passage time (FPT), or interspike interval. We suggest using a population average for evaluating the FPT that represents the desired information. Instantaneous lateral excitation among these neurons helps the analysis. By employing a second layer of neurons with variable connections to the first layer, we represent the strength of the input by the number of output neurons that fire, thus decoding the temporal information. Such a model can easily lead to a logarithmic relation as in Weber's law. The latter follows naturally from information maximization if the input strength is statistically distributed according to an approximate inverse law.

Since their conception half a century ago, Hebbian cell assemblies have become a basic term in the neurosciences, and the idea that learning takes place through synaptic modifications has been accepted as a fundamental paradigm. As synapses undergo continuous metabolic turnover, adopting the stance that memories are engraved in the synaptic matrix raises a fundamental problem: How can memories be maintained for very long time periods? We present a novel solution to this long-standing question, based on biological evidence of neuronal regulation mechanisms that act to maintain neuronal activity. Our mechanism is developed within the framework of a neural model of associative memory. It is operative in conjunction with random activation of the memory system and is able to counterbalance degradation of synaptic weights and normalize the basins of attraction of all memories. Over long time periods, when the variance of the degradation process becomes important, the memory system stabilizes if its synapses are appropriately bounded. Thus, the remnant memory system is obtained by a dynamic process of synaptic selection and growth driven by neuronal regulatory mechanisms. Our model is a specific realization of dynamic stabilization of neural circuitry, which is often assumed to take place during sleep.

We propose a method for estimating probability density functions and conditional density functions by training on data produced by such distributions. The algorithm employs new stochastic variables that amount to coding of the input, using a principle of entropy maximization. It is shown to be closely related to the maximum likelihood approach. The encoding step of the algorithm provides an estimate of the probability distribution. The decoding step serves as a generative mode, producing an ensemble of data with the desired distribution. The algorithm is readily implemented by neural networks, using stochastic gradient ascent to achieve entropy maximization.

Arrays of interacting identical neurons can develop coherent firing patterns, such as moving stripes that have been suggested as possible explanations of hallucinatory phenomena. Other known formations include rotating spirals and expanding concentric rings. We obtain all of them using a novel two-variable description of integrate-and-fire neurons that allows for a continuum formulation of neural fields. One of these variables distinguishes between the two different states of refractoriness and depolarization and acquires topological meaning when it is turned into a field. Hence, it leads to a topologic characterization of the ensuing solitary waves, or excitons. They are limited to pointlike excitations on a line and linear excitations, including all the examples noted above, on a two-dimensional surface. A moving patch of firing activity is not an allowed solitary wave on our neural surface. Only the presence of strong inhomogeneity that destroys the neural field continuity allows for the appearance of patchy incoherent firing patterns driven by excitatory interactions.

In the framework of an associative memory model, we study the interplay between synaptic deletion and compensation, and memory deterioration, a clinical hallmark of Alzheimer's disease. Our study is motivated by experimental evidence that there are regulatory mechanisms that take part in the homeostasis of neuronal activity and act on the neuronal level. We show that following synaptic deletion, synaptic compensation can be carried out efficiently by a local, dynamic mechanism, where each neuron maintains the profile of its incoming post-synaptic current. Our results open up the possibility that the primary factor in the pathogenesis of cognitive deficiencies in Alzheimer's disease (AD) is the failure of local neuronal regulatory mechanisms. Allowing for neuronal death, we observe two pathological routes in AD, leading to different correlations between the levels of structural damage and functional decline.

Oscillatory attractor neural networks can perform temporal segmentation, i.e., separate the joint inputs they receive, through the formation of staggered oscillations. This property, which may be basic to many perceptual functions, is investigated here in the context of a symmetric dynamic system. The fully segmented mode is one type of limit cycle that this system can develop. It can be sustained for only a limited number n of oscillators. This limitation to a small number of segments is a basic phenomenon in such systems. Within our model we can explain it in terms of the limited range of narrow subharmonic solutions of the single nonlinear oscillator. Moreover, this point of view allows us to understand the dominance of three leading amplitudes in solutions of partial segmentation, which are obtained for high n. The latter are also abundant when we replace the common input with a graded one, allowing for different inputs to different oscillators. Switching to an input with fluctuating components, we obtain segmentation dominance for small systems and quite irregular waveforms for large systems.

We investigate the effect of synaptic compensation on the dynamic behavior of an attractor neural network receiving its input stimuli as external fields projecting on the network. It is shown how, in the face of weakened inputs, memory performance may be preserved by strengthening internal synaptic connections and increasing the noise level. Yet, these compensatory changes necessarily have adverse side effects, leading to spontaneous, stimulus-independent retrieval of stored patterns. These results can support Stevens' recent hypothesis that the onset of schizophrenia is associated with frontal synaptic regeneration, occurring subsequent to the degeneration of temporal neurons projecting on these areas.

We study the effect of removing temporal structure in the prediction error. We observe that networks which are not optimally trained, exhibit strong temporal structure in their prediction error, which can be eliminated using linear regression. This elimination improves performance significantly, but does not lead to the best performance which is achieved by training networks until they do not exhibit any such temporal structure. The improvement in performance of ensemble net averaging does not affect possible temporal structure of the error, thus averaging can be performed before or after temporal structure removal. We demonstrate these findings on the sunspot data set.

We implement and study a computational model of Stevens' [19921 theory of the pathogenesis of schizophrenia. This theory hypoth(cid:173) esizes that the onset of schizophrenia is associated with reactive synaptic regeneration occurring in brain regions receiving degener(cid:173) ating temporal lobe projections. Concentrating on one such area, the frontal cortex, we model a frontal module as an associative memory neural network whose input synapses represent incoming temporal projections. We analyze how, in the face of weakened external input projections, compensatory strengthening of internal synaptic connections and increased noise levels can maintain mem(cid:173) ory capacities (which are generally preserved in schizophrenia) . However, These compensatory changes adversely lead to sponta(cid:173) neous, biased retrieval of stored memories, which corresponds to the occurrence of schizophrenic delusions and hallucinations with(cid:173) out any apparent external trigger, and for their tendency to con(cid:173) centrate on just few central themes. Our results explain why these symptoms tend to wane as schizophrenia progresses, and why de(cid:173) layed therapeutical intervention leads to a much slower response. 150 Eytan Ruppin, James A. Reggia, David Hom

The clinical course of Alzheimer's disease (AD) is generally characterized by progressive gradual deterioration, although large clinical variability exists. Motivated by the recent quantitative reports of synaptic changes in AD, we use a neural network model to investigate how the interplay between synaptic deletion and compensation determines the pattern of memory deterioration, a clinical hallmark of AD. Within the model we show that the deterioration of memory retrieval due to synaptic deletion can be much delayed by multiplying all the remaining synaptic weights by a common factor, which keeps the average input to each neuron at the same level. This parallels the experimental observation that the total synaptic area per unit volume (TSA) is initially preserved when synaptic deletion occurs. By using different dependencies of the compensatory factor on the amount of synaptic deletion one can define various compensation strategies, which can account for the observed variation in the severity and progression rate of AD.

Neural networks can be trained to predict the next value of a time series on the basis of its preceding values. We try to find out how well such a network approximates the rule which underlies the series. For this purpose, we study the net-sequence, which is a long time series generated iteratively by the network. We introduce a new measure: the difference between the distributions of function values in the data and the net-sequence. We demonstrate its usefulness on the problem of the chaotic quadratic map. Adding random noise to the series we find, by using this tool, that the networks can approximate well the correct rule only if the noise amplitude is very small. Applying the new measure as a weak constraint in the problem of sunspot data, we see that it correlates well with the ability of the network to predict several time steps into the future.

Models of neural networks which include dynamical thresholds can display motion in pattern space, the space of all memories. We investigate this motion in a particular model which is based on a feedback network of excitatory and inhibitory neurons. We find that small variations in the parameters of the model can lead to big qualitative changes of its behavior. We display results of closed loops and chaotic motion which turn from one to the other through intermittency. We show that the basin of attraction of a closed orbit has a fractal shape, and find that the dimension of the chaotic motion is slightly bigger than 2. The general character of the dynamics of this model is convergence to centers of attraction on short time scales and divergence on long ones.

We investigate binding within the framework of a model of excitatory and inhibitory cell assemblies that form an oscillating neural network. Our model is composed of two such networks that are connected through their inhibitory neurons. The excitatory cell assemblies represent memory patterns. The latter have different meanings in the two networks, representing two different attributes of an object, such as shape and color. The networks segment an input that contains mixtures of such pairs into staggered oscillations of the relevant activities. Moreover, the phases of the oscillating activities representing the two attributes in each pair lock with each other to demonstrate binding. The system works very well for two inputs, but displays faulty correlations when the number of objects is larger than two. In other words, the network conjoins attributes of different objects, thus showing the phenomenon of "illusory conjunctions," as in human vision.

We describe a feedback neural network whose elements possess dynamic thresholds. This network has an oscillatory mode that we investigate by measuring the activities of memory patterns as functions of time. We observe spontaneous and induced transitions between the different oscillating memories. Moreover, the network exhibits pattern segmentation, by oscillating between different memories that are included as a mixture in a constant input. The efficiency of pattern segmentation decreases strongly as the number of the input memories is increased. Using oscillatory inputs we observe resonance behavior.

We investigate feedback networks containing excitatory and inhibitory neurons. The couplings between the neurons follow a Hebbian rule in which the memory patterns are encoded as cell assemblies of the excitatory neurons. Using disjoint patterns, we study the attractors of this model and point out the importance of mixed states. The latter become dominant at temperatures above 0.25. We use both numerical simulations and an analytic approach for our investigation. The latter is based on differential equations for the activity of the different memory patterns in the network configuration. Allowing the excitatory thresholds to develop dynamic features which correspond to fatigue of individual neurons, we obtain motion in pattern space, the space of all memories. The attractors turn into transients leading to chaotic motion for appropriate values of the dynamical parameters. The motion can be guided by overlaps between patterns, resembling a process of free associative thinking in the absence of any input.