Neural circuit development requires that synapses be formed between appropriate neurons. In addition, for a hierarchical network, successful development involves a sequencing of developmental events. It has been suggested that one mechanism that helps speed up development of proper connections is an early overproduction of synapses. Using a computational model of synapse development, such as adaptive synaptogenesis, it is possible to study such overproduction and its role in speeding up development; it is also possible to study other outcomes of synapse overproduction that are seemingly new to the literature. With a fixed number of neurons, adaptive synaptogenesis can control the speed of synaptic development in two ways: by altering the rate constants of the adaptive processes or by altering the initial number of rapidly but non-selectively accrued synapses. Using either mechanism, the simulations reveal that synapse overproduction appears as an unavoidable concomitant of rapid adaptive synaptogenesis. However, the shortest development times, which always produces the greatest amount of synapse overproduction, reduce adult performance by three measures: energy use, discrimination error rates, and proportional neuron allocation. Thus, the results here lead to the hypothesis that the observed speed of neural network development represents a particular inter-generational compromise: quick development benefits parental fecundity while slow development benefits offspring fecundity.

Intelligent organisms face a variety of tasks requiring the acquisition of expertise within a specific domain, including the ability to discriminate between a large number of similar patterns. From an energy-efficiency perspective, effective discrimination requires a prudent allocation of neural resources with more frequent patterns and their variants being represented with greater precision. In this work, we demonstrate a biologically plausible means of constructing a single-layer neural network that adaptively (i.e., without supervision) meets this criterion. Specifically, the adaptive algorithm includes synaptogenesis, synaptic shedding, and bi-directional synaptic weight modification to produce a network with outputs (i.e. neural codes) that represent input patterns proportional to the frequency of related patterns. In addition to pattern frequency, the correlational structure of the input environment also affects allocation of neural resources. The combined synaptic modification mechanisms provide an explanation of neuron allocation in the case of self-taught experts.

Neuronal information processing is energetically costly. Energy supply restrictions on information processing have resulted in the evolution of brains to compute and communicate information with remarkable energy efficiency. Indeed, energy minimization subject to functional constraints is a unifying principle. Toward better comprehension of neuronal information processing and communication from an information-energy standpoint, we consider a continuous time, continuous state-space neuron model with a generalized inverse Gaussian (GIG) conditional density. This GIG model arises from a Lévy diffusion that contains both homogeneous Poisson processes and Wiener processes with drift as special cases. We show that, when the energy constraints consist of a tripartite family apropos of the GIG model, the distribution of input excitation, A, that maximizes bits per Joule (bpJ) generates an output interspike interval duration T that possesses a related GIG marginal distribution. Most importantly, we obtain the exact expression for the bpJ-maximizing distribution of A in terms of Bessel functions that are readily evaluated numerically. This permits display of the strictly concave curves of bpJ-maximized mutual information I(A; T). Because bpJ equals bits per second/watt, these curves serve as the Shannon-theory capacity v. power curves for GIG neuron models. We show that the noise in GIG channels is essentially multiplicative, so our results may have significance for broader classes of fading channels.

Given enough physical constraints, the format of optimal computation may resolve into a rather small set of options, which we call design specifications. Our interest centers on computational problems that are so intensive, relative to the time and energy available, that they can be solved only in a probabilistic fashion. Here we consider just information and energy in one particular computational format, called neural-like (NL), and characterized as massively parallel, analog computation. Within this format, we consider only the design of a single NL element and the nature of its inputs. Importantly, we provide a specific mathematical format of a simple NL element. We consider this format to be minimal and generic and, therefore, extendable to structures composed of several NL compartments. Secondly, the information and energy constraints are linked, via Shannon's entropy, to classical results from mathematical statistics yielding design specifications that go beyond our initial description of a NL element and its inputs. Critically, for a NL element to preserve all of its relevant input-information at minimal energetic cost, it must transform its inputs so as to create and communicate a minimal sufficient statistic. Then, the assumptions associated with producing such a statistic become new design specifications for NL computing.

A neuroscience-based mathematical model of how a neuron stochastically processes data and communicates information is introduced and analyzed. Call the neuron in question 'neuron j", or just "j". The information j transmits approximately describes the time-varying intensity of the excitation j is continuously experiencing from neural spike trains delivered to its synapses by thousands of other neurons. Neuron j "encodes" this excitation history into a sequence of time instants at which it generates neural spikes of its own. By propagating these spikes along its axon, j acts as a multiaccess, partially degraded broadcast channel with thousands of input and output terminals that employs a time-continuous version of pulse position modulation. The mathematical model features three parameters, m, ¿, and b, which largely characterize j as an engine of computation and communication. Each set of values of these parameters corresponds to a long term maximization of the bits j conveys to its targets per joule it expends doing so, which is achieved by distributing the random duration between successive spikes j generates according to a gamma pdf with parameters ¿ and b and distributing b/A according to a beta probability density with parameters ¿ and m-¿, whereAis the random intensity of the effectively Poisson process of spikes that arrive to the union of all ofj's synapses at a randomly chosen time instant.

Quantifying the performance of a cognitive-behavioral model on a temporal paradigm requires mapping time onto the computational cycles of the simulation. We present a family of four minimal models of the hippocampus CA-3 simulated at different time resolutions. Behavioral results from the hippocampally-dependent trace classical conditioning paradigm show that rabbits can learn to properly anticipate US presentation for a specific range of trace interval time periods. Therefore, our hippocampal model should successfully anticipate US presentation for the same specific range of trace interval durations. Each model attempts to learn two different trace interval lengths. The results reinforce prior findings where we map time into the computational cycles of a minimal model. Further, our results support the the following idea : as the time resolution of a simulation increases, an increasing number of biological processes must be explicitly modeled to maintain behavioral performance for a temporal paradigm.

We introduce and analyze a biologically motivated mathematical model of a neuron in primary cortex that we call neuron j, or just j. Our goal is to ascertain how j maximizes the ratio of the information it conveys to the energy it expends.

Discovery of optimizations in biology provides a way to infer evolved function. Such optimizations are being sought at the level of neurons and synapses. Constraints being considered include information rates, metabolic costs, and time. Here we point out a classic optimization: neural computation should be Bayesian with a known prior distribution. A particular form of Bayesian inference is biologically feasible and necessarily linear. However, the linear requirement engenders metabolic costs, which are illustrated here

Summary form only given. In our simulations of hippocampal region CA3, we have observed a variety of dynamical behaviors over the course of learning. Such dynamics can either help or hurt the model as it is being trained to learn a cognitive task that depends on normal hippocampal function. The parameterization of the time spanning, associative synaptic modification rule has been investigated. In particular there is an interaction between the time constant of the NMDA-receptor and input longevity that has a direct effect on the dynamics of the neuronal codes that form. Inhibition potently affects the dynamics of cell firing, and it is easy to lose activity control when the excitatory synapses between primary, excitatory neurons alone are being modified. Synaptic modification of either the input or output synapses of an inhibitory neuron can, to a greater or lesser extent, compensate for synaptic modifications between primary neurons.

The rat hippocampus has been shown to mediate a large set of spatial navigation tasks such as the simple T-maze. We investigated the performance of a minimal computational, but biologically based, model of CA3 on this task. For successful performance, the model needs to generate and maintain neuronal codes for each of the two arms of the T-maze. Moreover, each code must be distinctively recalled in a goal-dependent manner. The development of such neuronal codes is aided by the appearance of repetitively firing recurrent neurons - known as local context units, analogous to hippocampal place cells which promote spatiotemporal association within the T-maze training sequences. The number, longevity, and connectivity of local context units exclusively coding for each arm of the maze grow with training. Although with too much training, the coding for one arm uncontrollably dominates over the other code, and goal appropriate choice-behavior is lost. That is, successful network codes can easily deteriorate with overtraining. The amount of training that produces this deterioration in performance depends on other network parameters. Rather than a failure of the model, we believe these results tell us something important about the biology of the hippocampal system. That is, this result provides support for the hypothesis of a hippocampal afferent system which down-regulates LTP once a task has been successfully learned. Modulatory Systems (e.g., dopaminergic, generally Dl/D5r) exist which are candidates for this functional role.

In recurrent neural networks, excessive activity oscillations can be very disruptive to successful learning performance. Inhibitory feedback can be used to offset a dominating positive feedback; however, synaptic modification at excitatory synapses would seem to require synaptic modification at inhibitory synapses if activity is to be controlled during training. Here, we present a novel synaptic modification rule that governs the synaptic strength of afferents (inputs) to activity controlling inhibitory interneurons. A hippocampal CA3 model incorporating this rule can avoid certain performance destroying activity oscillations. In the minimal model used here, this new rule for synaptic modification implements an error-correcting-like procedure at each excitatory input to a global feedback inhibitory interneuron. Simulations that include this novel modification rule demonstrate robust sequence learning as well as the elimination of major activity fluctuations that are outside the biologically plausible range. Importantly, simulations using this rule are able to adapt quickly and selectively to large, discontinuous jumps in the training sequences.

This article investigates the synaptic weight distribution of a self-supervised, sparse, and randomly connected recurrent network inspired by hippocampal region CA3. This network solves nontrivial sequence prediction problems by creating, on a neuron-by-neuron basis, special patterns of cell firing called local context units. These specialized patterns of cell firing--possibly an analog of hippocampal place cells--allow accurate prediction of the statistical distribution of synaptic weights, and this distribution is not at all gaussian. Aside from the majority of synapses that are, at least functionally, lost due to synaptic depression, the distribution is approximately uniform. Unexpectedly, this result is relatively independent of the input environment, and the uniform distribution of synaptic weights can be approximately parameterized based solely on the average activity level. Next, the results are generalized to other cell firing types (frequency codes and stochastic firing) and place cell-like firing distributions. Finally, we note that our predictions concerning the synaptic strength distribution can be extended to the distribution of correlated cell firings. Recent published neurophysiological results are consistent with this extension.

Reconstructing a time-varying stimulus estimate from a spike train (Bialek's "decoding" of a spike train) has become an important way to study neural information processing. In this paper, we describe a simple method for reconstructing a time-varying current injection signal from the simulated spike train it produces. This technique extracts most of the information from the spike train, provided that the input signal is appropriately matched to the spike generator. To conceptualize this matching, we consider spikes as instantaneous "samples" of the somatic current. The Sampling Theorem is then applicable, and it suggests that the bandwidth of the injected signal not exceed half the spike generator's average firing rate. The average firing rate, in turn, depends on the amplitude range and DC bias of the injected signal. We hypothesize that nature faces similar problems and constraints when transmitting a time-varying waveform from the soma of one neuron to the dendrite of the postsynaptic cell.

In 1969 Barlow introduced the phrase "economy of impulses" to express the tendency for successive neural systems to use lower and lower levels of cell firings to produce equivalent encodings. From this viewpoint, the ultimate economy of impulses is a neural code of minimal redundancy. The hypothesis motivating our research is that energy expenditures, e.g., the metabolic cost of recovering from an action potential relative to the cost of inactivity, should also be factored into the economy of impulses. In fact, coding schemes with the largest representational capacity are not, in general, optimal when energy expenditures are taken into account. We show that for both binary and analog neurons, increased energy expenditure per neuron implies a decrease in average firing rate if energy efficient information transmission is to be maintained.

We investigate the dynamics of a class of recurrent random networks with sparse, asymmetric excitatory connectivity and global shunting inhibition mediated by a single interneuron. Using probabilistic arguments and a hyperbolic tangent approximation to the gaussian, we develop a simple method for setting the average level of firing activity in these networks. We demonstrate through simulations that our technique works well and extends to networks with more complicated inhibitory schemes. We are interested primarily in the CA3 region of the mammalian hippocampus, and the random networks investigated here are seen as modeling the a priori dynamics of activity in this region. In the presence of external stimuli, a suitable synaptic modification rule could shape this dynamics to perform temporal information processing tasks such as sequence completion and prediction.

This paper presents several formulations for aggregating opinions about the outcome of a future event-e.g., opinions in the form of probabilities are aggregated into a single probability. The approach used to derive these aggregation formulas is maximum entropy inference, with the assumption that the opinions and the event being predicted are joint random variables. As part of the presentation, the novel, axiomatically developed aggregation formula due to Bordley is used as a foil for contrasting and comparing the results developed here. It is shown that Bordley's result is overly restrictive when looked at in terms of maximum entropy derivations that are clearly using more information. Initially, we suppose that the opinions given on the future event are in the form of odds; then we consider the case where these opinions are in the form of probabilities; and finally, we reach a general formulation in which they could be odds, probabilities or even binary, yes/no, predictions of a future event with interactions between experts. In regard to this last point, the use of Darroch and Ratcliff's method in maximum entropy aggregation is sketched.>