The maintenance of short-term memories is critical for survival in a dynamically changing world. Previous studies suggest that this memory can be stored in the form of persistent neural activity or using a synaptic mechanism, such as with short-term plasticity. Here, we compare the predictions of these two mechanisms to neural and behavioral measurements in a visual change detection task. Mice were trained to respond to changes in a repeated sequence of natural images while neural activity was recorded using two-photon calcium imaging. We also trained two types of artificial neural networks on the same change detection task as the mice. Following fixed pre-processing using a pretrained convolutional neural network, either a recurrent neural network (RNN) or a feedforward neural network with short-term synaptic depression (STPNet) was trained to the same level of performance as the mice. While both networks are able to learn the task, the STPNet model contains units whose activity are more similar to the in vivo data and produces errors which are more similar to the mice. When images are omitted, an unexpected perturbation which was absent during training, mice often do not respond to the omission but are more likely to respond to the subsequent image. Unlike the RNN model, STPNet produces a similar pattern of behavior. These results suggest that simple neural adaptation mechanisms may serve as an important bottom-up memory signal in this task, which can be used by downstream areas in the decision-making process.

Experimental studies in neuroscience are producing data at a rapidly increasing rate, providing exciting opportunities and formidable challenges to existing theoretical and modeling approaches. To turn massive datasets into predictive quantitative frameworks, the field needs software solutions for systematic integration of data into realistic, multiscale models. Here we describe the Brain Modeling ToolKit (BMTK), a software suite for building models and performing simulations at multiple levels of resolution, from biophysically detailed multi-compartmental, to point-neuron, to population-statistical approaches. Leveraging the SONATA file format and existing software such as NEURON, NEST, and others, BMTK offers a consistent user experience across multiple levels of resolution. It permits highly sophisticated simulations to be set up with little coding required, thus lowering entry barriers to new users. We illustrate successful applications of BMTK to large-scale simulations of a cortical area. BMTK is an open-source package provided as a resource supporting modeling-based discovery in the community.

Despite advances in experimental techniques and accumulation of large datasets concerning the composition and properties of the cortex, quantitative modeling of cortical circuits under in-vivo-like conditions remains challenging. Here we report and publicly release a biophysically detailed circuit model of layer 4 in the mouse primary visual cortex, receiving thalamo-cortical visual inputs. The 45,000-neuron model was subjected to a battery of visual stimuli, and results were compared to published work and new in vivo experiments. Simulations reproduced a variety of observations, including effects of optogenetic perturbations. Critical to the agreement between responses in silico and in vivo were the rules of functional synaptic connectivity between neurons. Interestingly, after extreme simplification the model still performed satisfactorily on many measurements, although quantitative agreement with experiments suffered. These results emphasize the importance of functional rules of cortical wiring and enable a next generation of data-driven models of in vivo neural activity and computations.

The mammalian neocortex has a repetitious, laminar structure and performs functions integral to higher cognitive processes, including sensory perception, memory, and coordinated motor output. What computations does this circuitry subserve that link these unique structural elements to their function? Potjans and Diesmann (2014) parameterized a four-layer, two cell type (i.e. excitatory and inhibitory) model of a cortical column with homogeneous populations and cell type dependent connection probabilities. We implement a version of their model using a displacement integro-partial differential equation (DiPDE) population density model. This approach, exact in the limit of large homogeneous populations, provides a fast numerical method to solve equations describing the full probability density distribution of neuronal membrane potentials. It lends itself to quickly analyzing the mean response properties of population-scale firing rate dynamics. We use this strategy to examine the input-output relationship of the Potjans and Diesmann cortical column model to understand its computational properties. When inputs are constrained to jointly and equally target excitatory and inhibitory neurons, we find a large linear regime where the effect of a multi-layer input signal can be reduced to a linear combination of component signals. One of these, a simple subtractive operation, can act as an error signal passed between hierarchical processing stages.

Single-neuron models are useful not only for studying the emergent properties of neural circuits in large-scale simulations, but also for extracting and summarizing in a principled way the information contained in electrophysiological recordings. Here we demonstrate that, using a convex optimization procedure we previously introduced, a Generalized Integrate-and-Fire model can be accurately fitted with a limited amount of data. The model is capable of predicting both the spiking activity and the subthreshold dynamics of different cell types, and can be used for online characterization of neuronal properties. A protocol is proposed that, combined with emergent technologies for automatic patch-clamp recordings, permits automated, in vitro high-throughput characterization of single neurons.

Unraveling the interplay between connectivity and spatio-temporal dynamics in neuronal networks is a key step to advance our understanding of neuronal information processing. Here we investigate how particular features of network connectivity underpin the propensity of neural networks to generate slow-switching assembly (SSA) dynamics, i.e., sustained epochs of increased firing within assemblies of neurons which transition slowly between different assemblies throughout the network. We show that the emergence of SSA activity is linked to spectral properties of the asymmetric synaptic weight matrix. In particular, the leading eigenvalues that dictate the slow dynamics exhibit a gap with respect to the bulk of the spectrum, and the associated Schur vectors exhibit a measure of block-localization on groups of neurons, thus resulting in coherent dynamical activity on those groups. Through simple rate models, we gain analytical understanding of the origin and importance of the spectral gap, and use these insights to develop new network topologies with alternative connectivity paradigms which also display SSA activity. Specifically, SSA dynamics involving excitatory and inhibitory neurons can be achieved by modifying the connectivity patterns between both types of neurons. We also show that SSA activity can occur at multiple timescales reflecting a hierarchy in the connectivity, and demonstrate the emergence of SSA in small-world like networks. Our work provides a step towards understanding how network structure (uncovered through advancements in neuroanatomy and connectomics) can impact on spatio-temporal neural activity and constrain the resulting dynamics.

L5 pyramidal neurons are the only neocortical cell type with dendrites reaching all six layers of cortex, casting them as one of the main integrators in the cortical column. What is the nature and mode of computation performed in mouse primary visual cortex (V1) given the physiology of L5 pyramidal neurons? First, we experimentally establish active properties of the dendrites of L5 pyramidal neurons of mouse V1 using patch-clamp recordings. Using a detailed multi-compartmental model, we show this physiological setup to be well suited for coincidence detection between basal and apical tuft inputs by controlling the frequency of spike output. We further show how direct inhibition of calcium channels in the dendrites modulates such coincidence detection. To establish the singe-cell computation that this biophysics supports, we show that the combination of frequency-modulation of somatic output by tuft input and (simulated) calcium-channel blockage functionally acts as a composite sigmoidal function. Finally, we explore how this computation provides a mechanism whereby dendritic spiking contributes to orientation tuning in pyramidal neurons.

In this paper we provide an extensive evaluation of fixation prediction and salient object segmentation algorithms as well as statistics of major datasets. Our analysis identifies serious design flaws of existing salient object benchmarks, called the dataset design bias, by over emphasising the stereotypical concepts of saliency. The dataset design bias does not only create the discomforting disconnection between fixations and salient object segmentation, but also misleads the algorithm designing. Based on our analysis, we propose a new high quality dataset that offers both fixation and salient object segmentation ground-truth. With fixations and salient object being presented simultaneously, we are able to bridge the gap between fixations and salient objects, and propose a novel method for salient object segmentation. Finally, we report significant benchmark progress on 3 existing datasets of segmenting salient objects.

Natural selection favors the evolution of brains that can capture fitness-relevant features of the environment's causal structure. We investigated the evolution of small, adaptive logic-gate networks ("animats") in task environments where falling blocks of different sizes have to be caught or avoided in a 'Tetris-like' game. Solving these tasks requires the integration of sensor inputs and memory. Evolved networks were evaluated using measures of information integration, including the number of evolved concepts and the total amount of integrated conceptual information. The results show that, over the course of the animats' adaptation, i) the number of concepts grows; ii) integrated conceptual information increases; iii) this increase depends on the complexity of the environment, especially on the requirement for sequential memory. These results suggest that the need to capture the causal structure of a rich environment, given limited sensors and internal mechanisms, is an important driving force for organisms to develop highly integrated networks ("brains") with many concepts, leading to an increase in their internal complexity.

For an ill-posed problem like boundary detection, human labeled datasets play a critical role. Compared with the active research on finding a better boundary detector to refresh the performance record, there is surprisingly little discussion on the boundary detection benchmark itself. The goal of this paper is to identify the potential pitfalls of today's most popular boundary benchmark, BSDS 300. In the paper, we first introduce a psychophysical experiment to show that many of the "weak" boundary labels are unreliable and may contaminate the benchmark. Then we analyze the computation of f-measure and point out that the current benchmarking protocol encourages an algorithm to bias towards those problematic "weak" boundary labels. With this evidence, we focus on a new problem of detecting strong boundaries as one alternative. Finally, we assess the performances of 9 major algorithms on different ways of utilizing the dataset, suggesting new directions for improvements.

The manner in which different distributions of synaptic weights onto cortical neurons shape their spiking activity remains open. To characterize a homogeneous neuronal population, we use the master equation for generalized leaky integrate-and-fire neurons with shot-noise synapses. We develop fast semi-analytic numerical methods to solve this equation for either current or conductance synapses, with and without synaptic depression. We show that its solutions match simulations of equivalent neuronal networks better than those of the Fokker-Planck equation and we compute bounds on the network response to non-instantaneous synapses. We apply these methods to study different synaptic weight distributions in feed-forward networks. We characterize the synaptic amplitude distributions using a set of measures, called tail weight numbers, designed to quantify the preponderance of very strong synapses. Even if synaptic amplitude distributions are equated for both the total current and average synaptic weight, distributions with sparse but strong synapses produce higher responses for small inputs, leading to a larger operating range. Furthermore, despite their small number, such synapses enable the network to respond faster and with more stability in the face of external fluctuations.

What is the relationship between the complexity and the fitness of evolved organisms, whether natural or artificial? It has been asserted, primarily based on empirical data, that the complexity of plants and animals increases as their fitness within a particular environment increases via evolution by natural selection. We simulate the evolution of the brains of simple organisms living in a planar maze that they have to traverse as rapidly as possible. Their connectome evolves over 10,000s of generations. We evaluate their circuit complexity, using four information-theoretical measures, including one that emphasizes the extent to which any network is an irreducible entity. We find that their minimal complexity increases with their fitness.

What is the relationship between the complexity and the fitness of evolved organisms, whether natural or artificial? It has been asserted, primarily based on empirical evidence, that the complexity of plants and animals increases as their fitness within a particular environment increases via evolution by natural selection (Bonner, 1988; McShea, 1996; Adami et al., 2000). We here derive an analytical relationship between these two quantities within an information-theoretical framework, showing that under certain conditions, complexity is a monotonically increasing function of fitness. We also simulate the adaptation of brains of digital organisms living in mazes and whose connectome evolves over 10,000s of generations in a stationary environment. We compute their circuit complexity, using an entropy-based measure (Balduzzi and Tononi, 2008). We find that their minimal complexity increases with their fitness, in line with our analytical derivation.

The ability to predict action content from neural signals in real time before the ac- tion occurs has been long sought in the neuroscientific study of decision-making, agency and volition. On-line real-time (ORT) prediction is important for under- standing the relation between neural correlates of decision-making and conscious, voluntary action as well as for brain-machine interfaces. Here, epilepsy patients, implanted with intracranial depth microelectrodes or subdural grid electrodes for clinical purposes, participated in a “matching-pennies” game against an opponent. In each trial, subjects were given a 5 s countdown, after which they had to raise their left or right hand immediately as the “go” signal appeared on a computer screen. They won a fixed amount of money if they raised a different hand than their opponent and lost that amount otherwise. The question we here studied was the extent to which neural precursors of the subjects’ decisions can be detected in intracranial local field potentials (LFP) prior to the onset of the action. We found that combined low-frequency (0.1–5 Hz) LFP signals from 10 electrodes were predictive of the intended left-/right-hand movements before the onset of the go signal. Our ORT system predicted which hand the patient would raise 0.5 s before the go signal with 68±3% accuracy in two patients. Based on these results, we constructed an ORT system that tracked up to 30 electrodes simultaneously, and tested it on retrospective data from 7 patients. On average, we could predict the correct hand choice in 83% of the trials, which rose to 92% if we let the system drop 3/10 of the trials on which it was less confident. Our system demonstrates— for the first time—the feasibility of accurately predicting a binary action on single trials in real time for patients with intracranial recordings, well before the action occurs.

We introduce a simple image descriptor referred to as the image signature. We show, within the theoretical framework of sparse signal mixing, that this quantity spatially approximates the foreground of an image. We experimentally investigate whether this approximate foreground overlaps with visually conspicuous image locations by developing a saliency algorithm based on the image signature. This saliency algorithm predicts human fixation points best among competitors on the Bruce and Tsotsos [1] benchmark data set and does so in much shorter running time. In a related experiment, we demonstrate with a change blindness data set that the distance between images induced by the image signature is closer to human perceptual distance than can be achieved using other saliency algorithms, pixel-wise, or GIST [2] descriptor methods.

Detecting text regions in natural scenes is an important part of computer vision. We propose a novel text detection algorithm that extracts six different classes features of text, and uses Modest AdaBoost with multi-scale sequential search. Experiments show that our algorithm can detect text regions with a f= 0.70, from the ICDAR 2003 datasets which include images with text of various fonts, sizes, colors, alphabets and scripts.

One of the hallmarks of biological organisms is their ability to integrate disparate information sources to optimize their behavior in complex environments. How this capability can be quantified and related to the functional complexity of an organism remains a challenging problem, in particular since organismal functional complexity is not well-defined. We present here several candidate measures that quantify information and integration, and study their dependence on fitness as an artificial agent ("animat") evolves over thousands of generations to solve a navigation task in a simple, simulated environment. We compare the ability of these measures to predict high fitness with more conventional information-theoretic processing measures. As the animat adapts by increasing its "fit" to the world, information integration and processing increase commensurately along the evolutionary line of descent. We suggest that the correlation of fitness with information integration and with processing measures implies that high fitness requires both information processing as well as integration, but that information integration may be a better measure when the task requires memory. A correlation of measures of information integration (but also information processing) and fitness strongly suggests that these measures reflect the functional complexity of the animat, and that such measures can be used to quantify functional complexity even in the absence of fitness data.

Under natural viewing conditions, human observers use shifts in gaze to allocate processing resources to subsets of the visual input. There are many computational models that try to predict these shifts in eye movement and attention. Although the important role of high level stimulus properties (e.g., semantic information) stands undisputed, most models are based solely on low-level image properties. We here demonstrate that a combined model of high-level object detection and low-level saliency significantly outperforms a low-level saliency model in predicting locations humans fixate on. The data is based on eye-movement recordings of humans observing photographs of natural scenes, which contained one of the following high-level stimuli: faces, text, scrambled text or cell phones. We show that observers - even when not instructed to look for anything particular, fixate on a face with a probability of over 80% within their first two fixations, on text and scrambled text with a probability of over 65.1% and 57.9% respectively, and on cell phones with probability of 8.3%. This suggests that content with meaningful semantic information is significantly more likely to be seen earlier. Adding regions of interest (ROI), which depict the locations of the high-level meaningful features, significantly improves the prediction of a saliency model for stimuli with high semantic importance, while it has little effect for an object with no semantic meaning.

Based on the membrane currents generated by an action potential in a biologically realistic model of a pyramidal, hippocampal cell within rat CA1, we perform a moment expansion of the extracellular field potential. We decompose the potential into both inverse and classical moments and show that this method is a rapid and efficient way to calculate the extracellular field both near and far from the cell body. The action potential gives rise to a large quadrupole moment that contributes to the extracellular field up to distances of almost 1 cm. This method will serve as a starting point in connecting the microscopic generation of electric fields at the level of neurons to macroscopic observables such as the local field potential.

Motivated by the existence of highly selective, sparsely firing cells observed in the human medial temporal lobe (MTL), we present an unsupervised method for learning and recognizing object categories from unlabeled images. In our model, a network of nonlinear neurons learns a sparse representation of its inputs through an unsupervised expectation-maximization process. We show that the application of this strategy to an invariant feature-based description of natural images leads to the development of units displaying sparse, invariant selectivity for particular individuals or image categories much like those observed in the MTL data.

We present an unsupervised method for learning and recognizing object categories from unlabeled images. Motivated by the existence of highly selective, sparsely firing cells observed in the human medial temporal lobe (MTL), we apply a sparse generative model to the outputs of a biologically faithful model of the primate ventral visual system. In our model, a network of nonlinear neurons learns a sparse representation of its inputs through an unsupervised expectation-maximization process. In recognition, this model is used in a maximum-likelihood manner to classify unseen images, and we find units emerging from learning that respond selectively to specific image categories. A significant advantage of this approach is that there is no need to specify the number of categories present in the training set. We present classification accuracy using three different evaluation metrics. 1

Under natural viewing conditions, human observers shift their gaze to allocate processing resources to subsets of the visual input. Many computational models have aimed at predicting such voluntary attentional shifts. Although the importance of high level stimulus properties (higher order statistics, semantics) stands undisputed, most models are based on low-level features of the input alone. In this study we recorded eye-movements of human observers while they viewed photographs of natural scenes. About two thirds of the stimuli contained at least one person. We demonstrate that a combined model of face detection and low-level saliency clearly outperforms a low-level model in predicting locations humans fixate. This is reflected in our finding fact that observes, even when not instructed to look for anything particular, fixate on a face with a probability of over 80% within their first two fixations (500ms). Remarkably, the model's predictive performance in images that do not contain faces is not impaired by spurious face detector responses, which is suggestive of a bottom-up mechanism for face detection. In summary, we provide a novel computational approach which combines high level object knowledge (in our case: face locations) with low-level features to successfully predict the allocation of attentional resources.

A new bottom-up visual saliency model, Graph-Based Visual Saliency (GBVS), is proposed. It consists of two steps: rst forming activation maps on certain feature channels, and then normalizing them in a way which highlights conspicuity and admits combination with other maps. The model is simple, and biologically plausible insofar as it is naturally parallelized. This model powerfully predicts human xations on 749 variations of 108 natural images, achieving 98% of the ROC area of a human-based control, whereas the classical algorithms of Itti & Koch ([2], [3], [4]) achieve only 84%.

The basic requirement for direction selectivity is a nonlinear interaction between two different inputs in space-time. In some models, the interaction is hypothesized to occur between excitation and inhibition of the shunting type in the neuron's dendritic tree. How can the required spatial specificity be acquired in an unsupervised manner? We here propose an activity-based, local learning model that can account for direction selectivity in visual cortex based on such a local veto operation and that depends on synaptically induced changes in intracellular calcium concentration. Our biophysical simulations suggest that a model cell with our learning algorithm can develop direction selectivity organically after unsupervised training. The learning rule is also applicable to a neuron with multiple-direction-selective subunits and to a pair of cells with opposite-direction selectivities and is stable under different starting conditions, delays, and velocities.

Reverse-phi motion is the illusory reversal of perceived direction of movement when the stimulus contrast is reversed in successive frames. Livingstone, Tsao, and Conway (2000) showed that direction-selective cells in striate cortex of the alert macaque monkey showed reversed excitatory and inhibitory regions when two different contrast bars were flashed sequentially during a two-bar interaction analysis. While correlation or motion energy models predict the reverse-phi response, it is unclear how neurons can accomplish this. We carried out detailed biophysical simulations of a direction-selective cell model implementing a synaptic shunting scheme. Our results suggest that a simple synaptic-veto mechanism with strong direction selectivity for normal motion cannot account for the observed reverse-phi motion effect. Given the nature of reverse-phi motion, a direct interaction between the ON and OFF pathway, missing in the original shunting-inhibition model, it is essential to account for the reversal of response. We here propose a double synaptic-veto mechanism in which ON excitatory synapses are gated by both delayed ON inhibition at their null side and delayed OFF inhibition at their preferred side. The converse applies to OFF excitatory synapses. Mapping this scheme onto the dendrites of a direction-selective neuron permits the model to respond best to normal motion in its preferred direction and to reverse-phi motion in its null direction. Two-bar interaction maps showed reversed excitation and inhibition regions when two different contrast bars are presented.

It remains unclear whether the variability of neuronal spike trains in vivo arises due to biological noise sources or represents highly precise encoding of temporally varying synaptic input signals. Determining the variability of spike timing can provide fundamental insights into the nature of strategies used in the brain to represent and transmit information in the form of discrete spike trains. In this study, we employ a signal estimation paradigm to determine how variability in spike timing affects encoding of random time-varying signals. We assess this for two types of spiking models: an integrate-and-fire model with random threshold and a more biophysically realistic stochastic ion channel model. Using the coding fraction and mutual information as information-theoretic measures, we quantify the efficacy of optimal linear decoding of random inputs from the model outputs and study the relationship between efficacy and variability in the output spike train. Our findings suggest that variability does not necessarily hinder signal decoding for the biophysically plausible encoders examined and that the functional role of spiking variability depends intimately on the nature of the encoder and the signal processing task; variability can either enhance or impede decoding performance.

We present new simulation results, in which a computational model of interacting visual neurons simultaneously predicts the modula(cid:173) tion of spatial vision thresholds by focal visual attention, for five dual-task human psychophysics experiments. This new study com(cid:173) plements our previous findings that attention activates a winner(cid:173) take-all competition among early visual neurons within one cortical hypercolumn. This "intensified competition" hypothesis assumed that attention equally affects all neurons, and yielded two single(cid:173) unit predictions: an increase in gain and a sharpening of tuning with attention. While both effects have been separately observed in electrophysiology, no single-unit study has yet shown them si(cid:173) multaneously. Hence, we here explore whether our model could still predict our data if attention might only modulate neuronal gain, but do so non-uniformly across neurons and tasks. Specifically, we investigate whether modulating the gain of only the neurons that are loudest, best-tuned, or most informative about the stimulus, or of all neurons equally but in a task-dependent manner, may ac(cid:173) count for the data. We find that none of these hypotheses yields predictions as plausible as the intensified competition hypothesis, hence providing additional support for our original findings.

The temporal precision with which neurons respond to synaptic inputs has a direct bearing on the nature of the neural code. A characterization of the neuronal noise sources associated with different sub-cellular components (synapse, dendrite, soma, axon, and so on) is needed to understand the relationship between noise and information transfer. Here we study the effect of the unreliable, probabilistic nature of synaptic transmission on information transfer in the absence of interaction among presynaptic inputs. We derive theoretical lower bounds on the capacity of a simple model of a cortical synapse under two different paradigms. In signal estimation, the signal is assumed to be encoded in the mean firing rate of the presynaptic neuron, and the objective is to estimate the continuous input signal from the postsynaptic voltage. In signal detection, the input is binary, and the presence or absence of a presynaptic action potential is to be detected from the postsynaptic voltage. The efficacy of information transfer in synaptic transmission is characterized by deriving optimal strategies under these two paradigms. On the basis of parameter values derived from neocortex, we find that single cortical synapses cannot transmit information reliably, but redundancy obtained using a small number of multiple synapses leads to a significant improvement in the information capacity of synaptic transmission.

We describe an analogy between psychophysically measured effects in contrast masking, and the behavior of a simple integrate-and(cid:173) fire neuron that receives time-modulated inhibition. In the psy(cid:173) chophysical experiments, we tested observers ability to discriminate contrasts of peripheral Gabor patches in the presence of collinear Gabor flankers. The data reveal a complex interaction pattern that we account for by assuming that flankers provide divisive inhibi(cid:173) tion to the target unit for low target contrasts, but provide sub(cid:173) tractive inhibition to the target unit for higher target contrasts. A similar switch from divisive to subtractive inhibition is observed in an integrate-and-fire unit that receives inhibition modulated in time such that the cell spends part of the time in a high-inhibition state and part of the time in a low-inhibition state. The simi(cid:173) larity between the effects suggests that one may cause the other. The biophysical model makes testable predictions for physiological single-cell recordings.

Flies are capable of stabilizing their body during free flight by using visual motion information to estimate self-rotation. We have built a hardware model of this optomotor control system in a standard CMOS VLSI process. The result is a small, low-power chip that receives input directly from the real world through on-board photoreceptors and generates motor commands in real time. The chip was tested under closed-loop conditions typically used for insect studies. The silicon system exhibited stable control sufficiently analogous to the biological system to allow for quantitative comparisons.

Flies are capable of rapid, coordinated flight through unstructured environments. This flight is guided by visual motion information that is extracted from photoreceptors in a robust manner. One feature of the fly's visual processing that adds to this robustness is the saturation of wide-field motion-sensitive neuron responses with increasing pattern size. This makes the cell's responses less dependent on the sparseness of the optical flow field while retaining motion information. By implementing a compartmental neuronal model in silicon, we add this "gain control" to an existing analog VLSI model of fly vision. This results in enhanced performance in a compact, low-power CMOS motion sensor. Our silicon system also demonstrates that modern, biophysically-detailed models of neural sensory processing systems can be instantiated in VLSI hardware.

One way to understand a neurobiological system is by building a simulacrum that replicates its behavior in real time using similar constraints. Analog very large-scale integrated (VLSI) electronic circuit technology provides such an enabling technology. We here describe a neuromorphic system that is part of a long-term effort to understand the primate oculomotor system. It requires both fast sensory processing and fast motor control to interact with the world. A one-dimensional hardware model of the primate eye has been built that simulates the physical dynamics of the biological system. It is driven by two different analog VLSI chips, one mimicking cortical visual processing for target selection and tracking and another modeling brain stem circuits that drive the eye muscles. Our oculomotor plant demonstrates both smooth pursuit movements, driven by a retinal velocity error signal, and saccadic eye movements, controlled by retinal position error, and can reproduce several behavioral, stimulation, lesion, and adaptation experiments performed on primates.

In recent theoretical approaches addressing the problem of neural coding, tools from statistical estimation and information theory have been applied to quantify the ability of neurons to transmit information through their spike outputs. These techniques, though fairly general, ignore the specific nature of neuronal processing in terms of its known biophysical properties. However, a systematic study of processing at various stages in a biophysically faithful model of a single neuron can identify the role of each stage in information transfer. Toward this end, we carry out a theoretical analysis of the information loss of a synaptic signal propagating along a linear, one-dimensional, weakly active cable due to neuronal noise sources along the way, using both a signal reconstruction and a signal detection paradigm. Here we begin such an analysis by quantitatively characterizing three sources of membrane noise: (1) thermal noise due to the passive membrane resistance, (2) noise due to stochastic openings and closings of voltage-gated membrane channels (NA+ and K+), and (3) noise due to random, background synaptic activity. Using analytical expressions for the power spectral densities of these noise sources, we compare their magnitudes in the case of a patch of membrane from a cortical pyramidal cell and explore their dependence on different biophysical parameters.

This is the second in a series of articles that seek to recast classical single-neuron biophysics in information-theoretical terms. Classical cable theory focuses on analyzing the voltage or current attenuation of a synaptic signal as it propagates from its dendritic input location to the spike initiation zone. On the other hand, we are interested in analyzing the amount of information lost about the signal in this process due to the presence of various noise sources distributed throughout the neuronal membrane. We use a stochastic version of the linear one-dimensional cable equation to derive closed-form expressions for the second-order moments of the fluctuations of the membrane potential associated with different membrane current noise sources: thermal noise, noise due to the random opening and closing of sodium and potassium channels, and noise due to the presence of "spontaneous" synaptic input. We consider two different scenarios. In the signal estimation paradigm, the time course of the membrane potential at a location on the cable is used to reconstruct the detailed time course of a random, band-limited current injected some distance away. Estimation performance is characterized in terms of the coding fraction and the mutual information. In the signal detection paradigm, the membrane potential is used to determine whether a distant synaptic event occurred within a given observation interval. In the light of our analytical results, we speculate that the length of weakly active apical dendrites might be limited by the information loss due to the accumulated noise between distal synaptic input sites and the soma and that the presence of dendritic nonlinearities probably serves to increase dendritic information transfer.

It is generally assumed that nerve cells optimize their performance to reflect the statistics of their input. Electronic circuit analogs of neurons require similar methods of self-optimization for stable and autonomous operation. We here describe and demonstrate a biologically plausible adaptive algorithm that enables a neuron to adapt the current threshold and the slope (or gain) of its current-frequency relationship to match the man (or dc offset) and variance (or dynamic range or contrast) of the time-varying somatic input current. The adaptation algorithm estimates the somatic current signal from the spike train by way of the intracellular somatic calcium concentration, thereby continuously adjusting the neurons' firing dynamics. This principle is shown to work in an analog VLSI-designed silicon neuron.

We here propose an adaptive procedure that enables a spiking neuron, whether artificial or biological, to make optimal use of its dynamic range and gain.We discuss an analog electronic circuit implementation of this algorithm using a biologically realistic artificial "silicon" neuron. The adaptation procedure adapts the neuron's firing threshold and the sensitivity (or gain) of its current-frequency relationship to match the dc offset (or mean) and the dynamic range (or variance) of the time-varying somatic input current. The neuron extracts the minimum and maximum levels of the reconstructed somatic current signals from the cell's own spike trains. These are used to regulate the somatic leak conductance in order to shift the somatic current-frequency relation and to adjust a calcium-activated potassium conductance to change the dynamic range of the cell's somatic current-frequency relationship. We report experimental data from a test neuron--built using analog subthreshold CMOS VLSI technology--that shows the expected behavior.

A visual attention system, inspired by the behavior and the neuronal architecture of the early primate visual system, is presented. Multiscale image features are combined into a single topographical saliency map. A dynamical neural network then selects attended locations in order of decreasing saliency. The system breaks down the complex problem of scene understanding by rapidly selecting, in a computationally efficient manner, conspicuous locations to be analyzed in detail.

Shunting inhibition, a conductance increase with a reversal potential close to the resting potential of the cell, has been shown to have a divisive effect on subthreshold excitatory postsynaptic potential amplitudes. It has therefore been assumed to have the same divisive effect on firing rates. We show that shunting inhibition actually has a subtractive effect on the firing rate in most circumstances. Averaged over several interspike intervals, the spiking mechanism effectively clamps the somatic membrane potential to a value significantly above the resting potential, so that the current through the shunting conductance is approximately independent of the firing rate. This leads to a subtractive rather than a divisive effect. In addition, at distal synapses, shunting inhibition will also have an approximately subtractive effect if the excitatory conductance is not small compared to the inhibitory conductance. Therefore regulating a cell's passive membrane conductance-for instance, via massive feedback-is not an adequate mechanism for normalizing or scaling its output.

How reliably do action potentials in cortical neurons encode information about a visual stimulus? Most physiological studies do not weigh the occurrences of particular action potentials as significant but treat them only as reflections of average neuronal excitation. We report that single neurons recorded in a previous study by Newsome et al. (1989; see also Britten et al. 1992) from cortical area MT in the behaving monkey respond to dynamic and unpredictable motion stimuli with a markedly reproducible temporal modulation that is precise to a few milliseconds. This temporal modulation is stimulus dependent, being present for highly dynamic random motion but absent when the stimulus translates rigidly.

Recently, methods of statistical estimation theory have been applied by Bialek and collaborators (1991) to reconstruct time-varying velocity signals and to investigate the processing of visual information by a directionally selective motion detector in the fly's visual system, the H1 cell. We summarize here our theoretical results obtained by studying these reconstructions starting from a simple model of H1 based on experimental data. Under additional technical assumptions, we derive a closed expression for the Fourier transform of the optimal reconstruction filter in terms of the statistics of the stimulus and the characteristics of the model neuron, such as its firing rate. It is shown that linear reconstruction filters will change in a nontrivial way if the statistics of the signal or the mean firing rate of the cell changes. Analytical expressions are then derived for the mean square error in the reconstructions and the lower bound on the rate of information transmission that was estimated experimentally by Bialek et al. (1991). For plausible values of the parameters, the model is in qualitative agreement with experimental data. We show that the rate of information transmission and mean square error represent different measures of the reconstructions: in particular, satisfactory reconstructions in terms of the mean square error can be achieved only using stimuli that are matched to the properties of the recorded cell. Finally, it is shown that at least for the class of models presented here, reconstruction methods can be understood as a generalization of the more familiar reverse-correlation technique.

This paper discusses some of the fundamental issues in the design of highly parallel, dense, low-power motion sensors in analog VLSI. Since photoreceptor circuits are an integral part of all visual motion sensors, we discuss how the sizing of photosensitive areas can affect the performance of such systems. We review the classic gradient and correlation algorithms and give a survey of analog motion-sensing architectures inspired by them. We calculate how the measurable speed range scales with signal-to-noise ratio (SNR) for a classic Reichardt sensor with a fixed time constant. We show how this speed range may be improved using a nonlinear filter with an adaptive time constant, constructed out of a diode and a capacitor, and present data from a velocity sensor based on such a filter. Finally, we describe how arrays of such velocity sensors call be employed to compute the heading direction of a moving subject and to estimate the time-to-contact between the sensor and a moving object.

An integrated circuit that computes the velocity vector of a visual stimulus in one dimension is presented. The circuit combines optical sensors and associated electronics on a single silicon chip, processed with standard CMOS technology. The velocity is inferred from the time delay of the appearance of an image feature at two fixed locations on the chip. The circuit operates quite robustly for high-contrast stimuli over considerable irradiance and velocity ranges. With lower-contrast stimuli the output signal for a given velocity tends to decrease, while the direction selectivity is still maintained. The individual motion-sensing cells are compact, and they are therefore suited for use in dense 1D or 2D imaging arrays.

Almost all models of orientation and direction selectivity in visual cortex are based on feedforward connection schemes, where genicu(cid:173) late input provides all excitation to both pyramidal and inhibitory neurons. The latter neurons then suppress the response of the for(cid:173) mer for non-optimal stimuli. However, anatomical studies show that up to 90 % of the excitatory synaptic input onto any corti(cid:173) cal cell is provided by other cortical cells. The massive excitatory feedback nature of cortical circuits is embedded in the canonical microcircuit of Douglas &. Martin (1991). We here investigate ana(cid:173) lytically and through biologically realistic simulations the function(cid:173) ing of a detailed model of this circuitry, operating in a hysteretic mode. In the model, weak geniculate input is dramatically ampli(cid:173) fied by intracortical excitation, while inhibition has a dual role: (i) to prevent the early geniculate-induced excitation in the null di(cid:173) rection and (ii) to restrain excitation and ensure that the neurons fire only when the stimulus is in their receptive-field. Among the 4 Humbert Suarez, Christo! Koch, Rodney Douglas insights gained are the possibility that hysteresis underlies visual cortical function, paralleling proposals for short-term memory, and strong limitations on linearity tests that use gratings. Properties of visual cortical neurons are compared in detail to this model and to a classical model of direction selectivity that does not include excitatory corti co-cortical connections. The model explain a num(cid:173) ber of puzzling features of direction-selective simple cells, includ(cid:173) ing the small somatic input conductance changes that have been measured experimentally during stimulation in the null direction. The model also allows us to understand why the velocity-response curve of area 17 neurons is different from that of their LG N affer(cid:173) ents, and the origin of expansive and compressive nonlinearities in the contrast-response curve of striate cortical neurons.

It is commonly assumed that temporal synchronization of excitatory synaptic inputs onto a single neuron increases its firing rate. We investigate here the role of synaptic synchronization for the leaky integrate-and-fire neuron as well as for a biophysically and anatomically detailed compartmental model of a cortical pyramidal cell. We find that if the number of excitatory inputs, N, is on the same order as the number of fully synchronized inputs necessary to trigger a single action potential, Nt, synchronization always increases the firing rate (for both constant and Poisson-distributed input). However, for large values of N compared to Nt, “overcrowding” occurs and temporal synchronization is detrimental to firing frequency. This behavior is caused by the conflicting influence of the low-pass nature of the passive dendritic membrane on the one hand and the refractory period on the other. If both temporal synchronization as well as the fraction of synchronized inputs (Murthy and Fetz 1993) is varied, synchronization is only advantageous if either N or the average input frequency, fin, are small enough.

We investigate a model for neural activity in a two-dimensional sheet of leaky integrate-and-fire neurons with feedback connectivity consisting of local excitation and surround inhibition. Each neuron receives stochastic input from an external source, independent in space and time. As recently suggested by Softky and Koch (1992, 1993), independent stochastic input alone cannot explain the high interspike interval variability exhibited by cortical neurons in behaving monkeys. We show that high variability can be obtained due to the amplification of correlated fluctuations in a recurrent network. Furthermore, the cross-correlation functions have a dual structure, with a sharp peak on top of a much broader hill. This is due to the inhibitory and excitatory feedback connections, which cause “hotspots” of neural activity to form within the network. These localized patterns of excitation appear as clusters or stripes that coalesce, disintegrate, or fluctuate in size while simultaneously moving in a random walk constrained by the interaction with other clusters. The synaptic current impinging upon a single neuron shows large fluctuations at many time scales, leading to a large coefficient of variation (CV) for the interspike interval statistics. The power spectrum associated with single units shows a 1/f decay for small frequencies and is flat at higher frequencies, while the power spectrum of the spiking activity averaged over many cells—equivalent to the local field potential—shows no 1/f decay but a prominent peak around 40 Hz, in agreement with data recorded from cat and monkey cortex (Gray et al. 1990; Eckhorn et al. 1993). Firing rates exhibit self-similarity between 20 and 800 msec, resulting in 1/f-like noise, consistent with the fractal nature of neural spike trains (Teich 1992).

The standard theory of a leaky integrator with stochastic spike input predicts that cortical cells should fire regularly. We tested interspike-interval histograms from awake, behaving macaque visual cortex (V1 and MT), and found high levels of variability (CV>0.5) characteristic of a nearly random (Poisson) process. A simple integrate-and-fire model, using accepted biophysical parameters, fails by more than a factor of ten to account for the high CV. We also simulated a biophysically detailed compartment-model of an anatomically reconstructed and physiologically characterized layer V pyramidal cell; again, at high firing rate, CV values are low, in disagreement with the data. Only a few situations could account for this discrepency: very large EPSP’s, a very short membrane time constant (τm<0.3 msec), highly synchronized input, or spikes in excitable dendrites. Our analysis suggests that cortical computation may occur at a time-scale much faster than previously realized.

The dynamic behavior of a network model consisting of all-to-all excitatory coupled binary neurons with global inhibition is studied analytically and numerically. We prove that for random input signals, the output of the network consists of synchronized bursts with apparently random intermissions of noisy activity. We introduce the fraction of simultaneously firing neurons as a measure for synchrony and prove that its temporal correlation function displays, besides a delta peak at zero indicating random processes, strongly dampened oscillations. Our results suggest that synchronous bursts can be generated by a simple neuronal architecture that amplifies incoming coincident signals. This synchronization process is accompanied by dampened oscillations that, by themselves, however, do not play any constructive role in this and can therefore be considered to be an epiphenomenon.

When a typical nerve cell is injected with enough current, it fires a regular stream of action potentials. But cortical cells in vivo usually fire irregularly, reflecting synaptic input from presynaptic cells as well as intrinsic biophysical properties. We have applied the theory of stochastic \nprocesses to spike trains recorded from cortical neurons (Tuckwell 1989) and find a fundamental contradiction between the large interspike variability observed and the much lower values predicted by well-accepted biophysical models of single cells.

To what extent do the mechanisms generating different receptive field properties of neurons depend on each other? We investigated this question theoretically within the context of orientation and direction tuning of simple cells in the mammalian visual cortex. In our model a cortical cell of the "simple" type receives its orientation tuning by afferent convergence of aligned receptive fields of the lateral geniculate nucleus (Hubel and Wiesel 1962). We sharpen this orientation bias by postulating a special type of radially symmetric long-range lateral inhibition called circular inhibition. Surprisingly, this isotropic mechanism leads to the emergence of a strong bias for the direction of motion of a bar. We show that this directional anisotropy is neither caused by the probabilistic nature of the connections nor is it a consequence of the specific columnar structure chosen but that it is an inherent feature of the architecture of visual cortex.

Vision is simple. We open our eyes and, instantly, the world surrounding us is perceived in all its splendor. Yet Artificial Intelligence has been trying with very limited success for over 20 years to endow machines with similar abilities. A large van, filled with computers and driving unguided at a mile per hour across gently sloping hills in Colorado and using a laser-range system to “see” is the most we have accomplished so far. On the other hand, computers can play a decent game of chess or prove simple mathematical theorems. It is ironic that we are unable to reproduce perceptual abilities which we share with most animals while some of the features distinguishing us from even our closest cousins, chimpanzees, can be carried out by machines. Vision is difficult.

Behavioral experiments on insects (Hassenstein and Reichardt 1956; Poggio and Reichardt 1976) as well as psychophysical evidence from human studies (Van Santen and Sperling 1985; Adelson and Bergen 1985; Watson and Ahumada 1985) support the notion that short-range motion perception is mediated by a system with a quadratic type of nonlinearity, as in correlation (Hassenstein and Reichardt 1956), multiplication (Torre and Poggio 1978), or squaring (Adelson and Bergen 1985). However, there is little physiological evidence for quadratic nonlinearities in directionally selective cells. For instance, the response of cortical simple cells to a moving sine grating is half-wave instead of full-wave rectified as it should be for a quadratic nonlinearity (Movshon ef al. 1978; Holub and Morton-Gibson 1981) and is linear for low contrast (Holub and Morton-Gibson 1981). Complex cells have full-wave rectified responses, but are also linear in contrast. Moreover, a detailed theoretical analysis of possible biophysical mechanisms underlying direction selectivity concludes that most do not have quadratic properties except under very limited conditions (Grzywacz and Koch 1987). Thus, it is presently mysterious how a system can show quadratic properties while its individual components do not. We briefly discuss here a simple population encoding scheme offering a possible solution to this problem.

Computing motion on the basis of the time-varying image intensity is a difficult problem for both artificial and biological vision systems. We show how gradient models, a well-known class of motion algorithms, can be implemented within the magnocellular pathway of the primate's visual system. Our cooperative algorithm computes optical flow in two steps. In the first stage, assumed to be located in primary visual cortex, local motion is measured while spatial integration occurs in the second stage, assumed to be located in the middle temporal area (MT). The final optical flow is extracted in this second stage using population coding, such that the velocity is represented by the vector sum of neurons coding for motion in different directions. Our theory, relating the single-cell to the perceptual level, accounts for a number of psychophysical and electrophysiological observations and illusions.