Sensory information about the outside world is encoded by neurons in sequences of discrete, identical pulses termed action potentials or spikes. There is persistent controversy about the extent to which the precise timing of these spikes is relevant to the function of the brain. We revisit this issue, using the motion-sensitive neurons of the fly visual system as a test case. Our experimental methods allow us to deliver more nearly natural visual stimuli, comparable to those which flies encounter in free, acrobatic flight. New mathematical methods allow us to draw more reliable conclusions about the information content of neural responses even when the set of possible responses is very large. We find that significant amounts of visual information are represented by details of the spike train at millisecond and sub-millisecond precision, even though the sensory input has a correlation time of approximately 55 ms; different patterns of spike timing represent distinct motion trajectories, and the absolute timing of spikes points to particular features of these trajectories with high precision. Finally, the efficiency of our entropy estimator makes it possible to uncover features of neural coding relevant for natural visual stimuli: first, the system's information transmission rate varies with natural fluctuations in light intensity, resulting from varying cloud cover, such that marginal increases in information rate thus occur even when the individual photoreceptors are counting on the order of one million photons per second. Secondly, we see that the system exploits the relatively slow dynamics of the stimulus to remove coding redundancy and so generate a more efficient neural code.

Does the brain construct an efficient representation of the sensory world? We review progress on this question, focusing on a series of experiments in the last decade which use fly vision as a model system in which theory and experiment can confront each other. Although the idea of efficient representation has been productive, clearly it is incomplete since it doesn't tell us which bits of sensory information are most valuable to the organism. We argue that, in fact, an organism which maximizes the (biologically meaningful) adaptive value of its actions given fixed resources must have internal representations of the outside world that are optimal in a very specific information theoretic sense: they maximize the information about the future of sensory inputs at a fixed value of the information about their past. This principle contains as special cases computations which the brain seems to carry out, and it should be possible to test this optimization directly. We return to the fly visual system and report the results of preliminary experiments that are in very suggestive agreement with theory

Many neural systems extend their dynamic range by adaptation. We ex(cid:173) amine the timescales of adaptation in the context of dynamically mod(cid:173) ulated rapidly-varying stimuli, and demonstrate in the fly visual system that adaptation to the statistical ensemble of the stimulus dynamically maximizes information transmission about the time-dependent stimulus. Further, while the rate response has long transients, the adaptation takes place on timescales consistent with optimal variance estimation.

The problem of neural coding is to understand how sequences of action potentials (spikes) are related to sensory stimuli, motor out(cid:173) puts, or (ultimately) thoughts and intentions. One clear question is whether the same coding rules are used by different neurons, or by corresponding neurons in different individuals. We present a quantitative formulation of this problem using ideas from informa(cid:173) tion theory, and apply this approach to the analysis of experiments in the fly visual system. We find significant individual differences in the structure of the code, particularly in the way that tempo(cid:173) ral patterns of spikes are used to convey information beyond that available from variations in spike rate. On the other hand, all the flies in our ensemble exhibit a high coding efficiency, so that every spike carries the same amount of information in all the individuals. Thus the neural code has a quantifiable mixture of individuality and universality.

We show that the information carried by compound events in neural spike trains-patterns of spikes across time or across a population of cells-can be measured, independent of assumptions about what these patterns might represent. By comparing the information carried by a compound pattern with the information carried independently by its parts, we directly measure the synergy among these parts. We illustrate the use of these methods by applying them to experiments on the motion-sensitive neuron H1 of the fly's visual system, where we confirm that two spikes close together in time carry far more than twice the information carried by a single spike. We analyze the sources of this synergy and provide evidence that pairs of spikes close together in time may be especially important patterns in the code of H1.

In the fly visual system each optic ganglion (lamina, medulla, lobula, lobula plate) constitutes a retinotopically ordered set of neural columns in which each single column corresponds to one single optical axis in the visual field of the compound eye. This very precise mapping of the visual surrounding of the animal through the retinal input mosaic on to all four ganglia is one of the features which make the fly visual system extremely suitable for the systematic study of the basic principles and strategies which underlie the detection and processing of movement. In earlier work we proved the applicability of the well-known correlation model for the perception of movement in the description and prediction of the behavior of the wide-field movement detectors which integrate the activity of all neural columns at the level of the highest order ganglion, the lobula plate. Such a wide-field element as a natural continuation of our extracellular electrode is used, in a way, to study the temporal resolution properties of the highest order ganglion (without damaging any part of it) and to compare these properties with those at the level of the retina. It is shown that the temporal resolution at the level of the retina is an almost completely “stiff” one, whereas at the level of the lobula plate this processing is very flexible and that its time constants are tuned by the time course of the stimulus over a wide range. The consequences of this stimulus-dependent temporal behavior for the processing of images at various levels in this visual system are discussed against the background of adaptive modeling techniques.