Mixed signal analog/digital neuromorphic circuits offer an ideal computational substrate for testing and validating hypotheses about models of sensory processing, as they are affected by low resolution, variability, and other limitations that affect in a similar way real neural circuits. In addition, their real-time response properties allow to test these models in closed-loop sensory-processing hardware setups and to get an immediate feedback on the effect of different parameter settings. Within this context we developed a recurrent neural network architecture based on a model of the retinocortical visual pathway to obtain neurons highly tuned to oriented visual stimuli along a specific direction and with a specific spatial frequency, with Gabor-like receptive fields. The computation performed by the retina is emulated by a Dynamic Vision Sensor (DVS) while the following feed-forward and recurrent processing stages are implemented by a Dynamic Neuromorphic Asynchronous Processor (DYNAP) chip that comprises adaptive integrate-and fire neurons and dynamic synapses. We show how the network implemented on this device gives rise to neurons tuned to specific orientations and spatial frequencies, independent of the temporal frequency of the visual stimulus. Compared to alternative feedforward schemes, the model proposed produces highly structured receptive fields with a limited number of synaptic connections, thus optimizing hardware resources. We validate the model and approach proposed with experimental results using both synthetic and natural images.

Similar to biological retinas, neuromorphic Dynamic Vision Sensor (DVS) devices only respond to changes in the visual scene. It has been observed that in biological systems there is a causal relationship between fixational eye movements and target visibility during fixation, which plays a central role in vision. Based on these findings we implemented an active vision system comprising of a DVS mounted on a pan-tilt unit to introduce microscopic and erratic camera movements as a pivot for artificial vision of static scenes. The key principle is that moving the sensor over an image shifts the low temporal frequency power of a static scene into a range that an event-based retina can properly signal and encode it as highly synchronous activity. By characterizing the signal provided by the active vision system we evidenced (1) an amplification of its response to high spatial frequencies; (2) a whitening effect when scaling stimulus contrast to match the structure of natural images; and (3) an equalized response to all possible orientations of static stimuli related to the isotropic statistics of the random-like motion. The design of a further proper anisotropic spatial summation of events with opponent contrast polarity in a biologically-realistic spiking neural network allowed the detection of information relative to the local orientation of stimuli in a fully bio-inspired fashion. We validate the system proposed with experimental results using synthetic control stimuli.

In stereoscopic vision, the ability of perceiving the three-dimensional structure of the surrounding environment is subordinated to a precise and effective motor control for the binocular coordination of the eyes/cameras. If, on the one side, the binocular coordination of camera movements is a complicating factor, on the other side, a proper vergence control, acting on the binocular disparity, facilitates the binocular fusion and the subsequent stereoscopic perception process. In real-world situations, an effective vergence control requires further features other than real time capabilities: real robot systems are indeed characterized by mechanical and geometrical imprecision that affect the binocular vision, and the illumination conditions are changeable and unpredictable. Moreover, in order to allow an effective visual exploration of the peripersonal space, it is necessary to cope with different gaze directions and provide a large working space. The proposed control strategy resorts to a neuromimetic approach that provides a distributed representation of disparity information. The vergence posture is obtained by an open-loop and a closed-loop control, which directly interacts with saccadic control. Before saccade, the open-loop component is computed in correspondence of the saccade target region, to obtain a vergence correction to be applied simultaneously with the saccade. At fixation, the closed-loop component drives the binocular disparity to zero in a foveal region. The obtained vergence servos are able to actively drive both the horizontal and the vertical alignment of the optical axes on the object of interest, thus ensuring a correct vergence posture. Experimental tests were purposely designed to measure the performance of the control in the peripersonal space, and were performed on three different robot platforms. The results demonstrated that the proposed approach yields real-time and effective vergence camera movements on a visual stimulus in a wide working range, regardless of the illumination in the environment and the geometry of the system.

We show how a cortical model of early disparity detectors is able to autonomously learn effective control signals in order to drive the vergence eye movements of a binocular active vision system. The proposed approach employs early binocular mechanisms of vision and basic learning processes such as synaptic plasticity and reward modulation. The computational substrate consists of a population of modeled V1 complex cells, that provides a distributed representation of binocular disparity information. The population response also provides a global signal to describe the state of the system and thus its deviation from the desired vergence position. The proposed network, by taking into account the modification of its internal state as a consequence of the action performed, evolves following a differential Hebbian rule. Furthermore, the weights update is driven by an intrinsic signal derived by the overall activity of the population. Exploiting this signal implies a maximization of the population activity itself, thus providing an highly effective reward for the developing of a stable and accurate vergence behaviour. The efficacy of the proposed intrinsic reward signal is comparatively assessed against the ground-truth signal (the actual disparity) providing equivalent results, and thus validating the approach. Experimental tests in a simulated environment demonstrate that the proposed network is able to cope with vergent geometry and thus to learn effective vergence movements for static and moving visual targets in realistic situations.

We present two neural models for vergence angle control of a robotic head, a simplified and a more complex one. Both models work in a closed-loop manner and do not rely on explicitly computed disparity, but extract the desired vergence angle from the post-processed response of a population of disparity tuned complex cells, the actual gaze direction and the actual vergence angle. The first model assumes that the gaze direction of the robotic head is orthogonal to its baseline and the stimulus is a frontoparallel plane orthogonal to the gaze direction. The second model goes beyond these assumptions, and operates reliably in the general case where all restrictions on the orientation of the gaze, as well as the stimulus position, type and orientation, are dropped.

A method to analyze first-order spatial properties of optical flow is proposed. The approach is based on the use of a set of linear models that dynamically adjust their properties on the basis of context information. These models are generated by a recursive network that takes into account spatial interaction between neighbors. By checking the presence of these models in the optic flow using a multiple model Kalman Filter it is possible to recover information about the coefficients of the affine description and the image motion invariants: divergence, curl and deformation. Reliable estimates of these quantities could help in the analysis of real world complex motion sequences. Experimental results on egomotion estimation and 3D surface reconstruction validate the approach. 1

A perceptual human-machine interface based on visual appearance of hand movements is presented. Gestures are defined as the temporal evolution of 3D poses the of user's hand. Each gesture is described by five temporal sequences corresponding to the time evolution of the stereo depth information at the finger-tips. To this end feature processing directly exploits binocular disparity information considered in its full spatio-temporal dimension. Hidden Markov models are used to represent the statistical properties of the gesture. The high recognition rate obtained by experimental testing on a five gesture alphabet validates the approach.

A cortical model for motion-in-depth selectivity of complex cells in the visual cortex is proposed. The model is based on a time ex(cid:173) tension of the phase-based techniques for disparity estimation. We consider the computation of the total temporal derivative of the time-varying disparity through the combination of the responses of disparity energy units. To take into account the physiological plau(cid:173) sibility, the model is based on the combinations of binocular cells characterized by different ocular dominance indices. The resulting cortical units of the model show a sharp selectivity for motion-in(cid:173) depth that has been compared with that reported in the literature for real cortical cells.

A variety of computational tasks in early vision can be formulated through lattice networks. The cooperative action of these networks depends on the topology of interconnections, both feedforward and recurrent ones. This paper shows that it is possible to consider a distinct general architectural solution for all recurrent computations of any given order. The Gabor-like impulse response of a second-order network is analyzed in detail, pointing out how a near-optimal filtering behavior in space and frequency domains can be achieved through excitatory/inhibitory interactions without impairing the stability of the system. These architectures can be mapped, very efficiently at transistor level, on very large scale integration (VLSI) structures operating as analog perceptual engines. The problem of hardware implementation of early vision tasks can, indeed, be tackled by combining these perceptual agents through suitable weighted sums. A 17-node analog current-mode VLSI circuit has been implemented on a CMOS 2 microm, NWELL, single-poly, and double-metal technology, to demonstrate the feasibility of the approach. Applications of the perceptual engine to various machine vision algorithms are proposed.

The spatial organization of cortical axon and dendritic fields could be an interesting structural paradigm to obtain a functional specificity without postulating highly specific feedforward connections. In this article, we investigate the functional implications of recurrent intracortical inhibition when it occurs through clustered medium-range interconnection schemes (Wörgötter & Koch, 1991; Somogyi, 1989; Kritzer, Cowey, & Somogyi, 1992). Moreover, the interaction between the inhibitory schemes and visual orientation maps is explored. Assuming linearity, we show that clustered inhibitory mechanisms can trigger a propagation process that allows the development of extra (i.e., induced) interactions among the cortical sites involved in the recurrent loops. In addition, we point out how these interactions functionally modify the response of cortical simple cells and yield to highly structured Gabor-like receptive fields. This study should be considered not as a realistic biological model of the primary visual cortex but as an attempt to explain possible computational principles related to intracortical connectivity and to the underlying single-cell properties.

We consider the design process of VLSI systems dedicated to the real-time implementation of cooperative algorithms whose functionalities can be characterized by multilayer ensembles of simple elements which interact locally. These algorithms are related, even though not exclusively, to the implementation of various tasks in low-level machine vision. The starting point in the design process is the formulation of the sequential algorithm that computes the behavior of the system. Algorithmic transformations are performed to expose the parallelism originally present in the task. Given the description in terms of parallel loops, we partition the system and organize it as a set of processing units. The architectural structure of these units takes properly into account the algorithmic constraints on precision both in data representation and computation. The program flow implemented by our programmable architectural solution (ASIP) is an iterative sequence of multiply-and-accumulate operations performed in parallel. The programmability concerns both the structure/coefficients of the algorithm-depending on the specific application-and its computational parameters. The architecture's main blocks are described in VHDL and synthesized as a semi-custom chip, using standard tools. Following this procedure, we designed an ASIP core for performing real-time texture-based image segregation.