Autonomous robots and humans need to create a coherent 3D representation of their peripersonal space in order to interact with nearby objects. Recent studies in visual neuroscience suggest that the small coordinated head/eye movements that humans continually perform during fixation provides useful depth information. In this work, we mimic such a behavior on a humanoid robot and propose a computational model that extracts depth information without requiring the kinematic model of the robot. First, we show that, during fixational head/eye movements, proprioceptive cues and optic flow lie on a low dimensional subspace that is a function of the depth of the target. Then, we use the generative adaptive subspace self-organizing map (GASSOM) to learn these depth-dependent subspaces. The depth of the target is eventually decoded using a winner-take-all strategy. The proposed model is validated on a simulated model of the iCub robot.

Autonomous robots need to make sense of their surroundings, recognize objects, detect and, possibly, identify the people around them. Visual perception would be ideally suited for these tasks. Yet, visual perception remains one of the limiting factors of modern robotics: artificial vision systems tend to perform poorly in normal environments, where the scene and the illumination conditions are unpredictable. Evolution has faced similar problems, leading to surprisingly accurate visual capabilities, even in species with very small brains. We argue that the success of biological perception systems relies on three fundamental computational principles: (a) the continual coupling of perception and behavior; (b) the resulting emergence of multimodal cues; (c) and their efficient integration. Building on these computational principles, we describe a humanoid robot that emulates the dynamic strategy by which humans examine a visual scene. The proprioceptive and visual depth cues resulting from this strategy are integrated in statistically optimal manner into a unified representation. We show that this approach yields accurate and robust 3D representations of the observed scene.

While looking at a point in the scene, humans continually perform smooth eye movements to compensate for involuntary head rotations. Since the optical nodal points of the eyes do not lie on the head rotation axes, this behavior yields useful 3-D information in the form of visual parallax. Here, we describe the replication of this behavior in a humanoid robot. We have developed a method for egocentric distance estimation based on the parallax that emerges during compensatory head/eye movements. This method was tested in a robotic platform equipped with an anthropomorphic neck and two binocular pan-tilt units specifically designed to reproduce the visual input signals experienced by humans. We show that this approach yields accurate and robust estimation of egocentric distance within the space nearby the agent. These results provide a further demonstration of how behavior facilitates the solution of complex perceptual problems.

In the eyes of many species, the optical nodal points of the cornea and lens do not lie on the axes of rotation of the eye. During eye movements, this lack of alignment produces depth information in the form of an oculomotor parallax. That is, a redirection of gaze shifts the projection of an object on the retina by an amount that depends not only on the amplitude of the rotation of the eye, but also on the distance of the object with respect to the observer. Species as diverse as the chameleon and the sandlance critically rely on this depth cue to estimate distance. An oculomotor parallax is present also in the human eye and, during natural eye movements, it produces retinal shifts that are well within the range of sensitivity of the human visual system. We have developed an anthropomorphic robot equipped with a pan/tilt head specifically designed to reproduce the oculomotor parallax present in the human eye. We show that replication of sequences of human eye movements with this robot produces accurate estimation of distance. In robotic vision it is often debated whether the dynamic analysis of a visual scene by means of a mobile camera presents advantages with respect to the static analysis provided by a stationary camera with a wide field of view. This study shows that by generating depth information, replication of the dynamic strategy by which humans scan a visual scene greatly facilitates the processes of figure/ground segregation and image segmentation, two of the hardest tasks of machine vision

Under natural viewing conditions, the physiological instability of visual fixation keeps the projection of the stimulus on the retina in constant motion. After eye opening, chronic exposure to a constantly moving retinal image might influence the experience-dependent refinement of cell response characteristics. The results of previous modeling studies have suggested a contribution of fixational instability to the Hebbian maturation of the receptive fields of V1 simple cells (Rucci, Edelman, & Wray, 2000; Rucci & Casile, 2004). This letter examines the origins of such a contribution. Using quasilinear models of lateral geniculate nucleus units and V1 simple cells, we derive analytical expressions for the second- order statistics of thalamocortical activity before and after eye opening. We show that in the presence of natural stimulation, fixational instability introduces a spatially uncorrelated signal in the retinal input, which strongly influences the structure of correlated activity in the model. This input signal produces a regime of thalamocortical activity similar to that present before eye opening and compatible with the Hebbian maturation of cortical receptive fields.

Under natural viewing conditions, small movements of the eye and body prevent the maintenance of a steady direction of gaze. It is known that stimuli tend to fade when they are stabilized on the retina for several sec- onds. However, it is unclear whether the physiological self-motion of the retinal image serves a visual purpose during the brief periods of natural visual fixation. This study examines the impact of fixational instability on the statistics of visual input to the retina and on the structure of neural activity in the early visual system. Fixational instability introduces fluc- tuations in the retinal input signals that, in the presence of natural images, lack spatial correlations. These input fluctuations strongly influence neu- ral activity in a model of the LGN. They decorrelate cell responses, even if the contrast sensitivity functions of simulated cells are not perfectly tuned to counter-balance the power-law spectrum of natural images. A decorrelation of neural activity has been proposed to be beneficial for discarding statistical redundancies in the input signals. Fixational insta- bility might, therefore, contribute to establishing efficient representations of natural stimuli.

Neural activity appears to be a crucial component for shaping the recep- tive fields of cortical simple cells into adjacent, oriented subregions alter- nately receiving ON- and OFF-center excitatory geniculate inputs. It is known that the orientation selective responses of V1 neurons are refined by visual experience. After eye opening, the spatiotemporal structure of neural activity in the early stages of the visual pathway depends both on the visual environment and on how the environment is scanned. We have used computational modeling to investigate how eye movements might affect the refinement of the orientation tuning of simple cells in the pres- ence of a Hebbian scheme of synaptic plasticity. Levels of correlation be- tween the activity of simulated cells were examined while natural scenes were scanned so as to model sequences of saccades and fixational eye movements, such as microsaccades, tremor and ocular drift. The specific patterns of activity required for a quantitatively accurate development of simple cell receptive fields with segregated ON and OFF subregions were observed during fixational eye movements, but not in the presence of saccades or with static presentation of natural visual input. These re- sults suggest an important role for the eye movements occurring during visual fixation in the refinement of orientation selectivity.

Autonomous robotic systems need to adjust their sensorimotor coordinations so as to maintain good performance in the presence of changes in their sensory and motor characteristics. Biological systems are able to adapt to large variations in their physical and functional properties. In the last decade, the adjustment of orienting behavior has been carefully investigated in the barn owl, a nocturnal predator with highly developed auditory capabilities. We have previously proposed that the development and maintenance of the barn owl's accurate orienting behavior can be explained through a process of learning based on the saliency of sensorimotor events. In this paper we consider the application of a detailed computer model of the principal neural structures involved in the process of spatial localization in the barn owl to the control of the orienting behavior of a robotic system, in the presence of auditory and visual stimulation. The system is composed of a robotic head equipped with two lateral microphones and a camera. We show that the model produces accurate orienting behavior toward both auditory and visual targets and is able to quickly recover good performance after alterations of the sensory inputs and motor outputs. The results illustrate that an architecture specifically designed to account for biological phenomena can produce flexible and robust motor control of a robotic system operating in the real world.

Autonomous robotic systems need to adjust their sensorimotor coordinations so as to maintain good performance in the presence of changes in their sensory and motor characteristics. Biological systems are able to adapt to large variations in their physical and functional properties. The adjustment of orienting behavior has been carefully investigated in the barn owl, a nocturnal predator with highly developed auditory capabilities. In the optic tectum of the barn owl, an area well-known to be involved in the production of orienting behavior, neural maps of space in the visual, auditory, and motor modalities are found in close alignment with each other. As a neurophysiological correlate of the adjustment of motor responses, neural maps in the tectum tend to realign if the sensory inputs are manipulated. We have recently proposed that the development and maintenance of such map alignment can be explained through a process of learning, in which plasticity is mediated by the activation of diffuse-projecting neuromodulatory systems which respond to innate or acquired salient cues. This proposal was tested using a detailed computer model of the principal neural structures involved in the process of spatial localisation in the barn owl. Here we consider the application of this model to the control of the orienting behavior of a robotic system in the presence of auditory and visual stimulation. The system we consider is composed of a robotic head equipped with two lateral microphones and a camera. We show that the model produces accurate orienting behavior toward both auditory and visual stimuli during normal visual experience, after alteration of the visual inputs, and after the reestablishment of normal visual conditions. The results illustrate that an architecture specifically designed to account for biological phenomena can produce flexible and robust control of a robotic system.

As it occurs in humans, robotic systems should be able to respond to unexpected tactile events by orienting their visual attention toward the location of the stimuli. This implies two basic problems: 1) it is necessary to develop a general method for integrating attentive processes which belong to different sensory modalities according to the attended task; 2) for the specific case of touch-driven shift of gaze, a sensory motor transformation needs to be identified, which links the stimulation of tactile receptors to the spatial position of the camera, via the current posture of the system. In this paper we describe a general framework for integrating multimodal attentive mechanisms, and we show how the visuo-tactile coordination can be autonomously learnt on the basis of sensory consistency and feedback. After the general presentation of the method, we consider the case of a robotic system composed of a 2-DOF arm and a 2-DOF head. Experiments with this system show that it discovers its own functional model without any external intervention and adapts it continuously during normal operation. The approach gives good results while presenting the advantages of autonomy and adaptability.

The application of tools by artificial systems depends on the ability to recognize an object as having a particular function as well as on the ability to express the functional interaction. This paper focuses on the operation of piercing as means of investigating the issue of functional representation. The operation is expressed using a formalism, based on discrete event system (DES) theory and on the paradigm of active perception. The visual and tactile data acquired during piercing motor actions performed with a robotic manipulator is discussed.

The traditional approach to automation and robotics has focused so far mainly on assembly problems, whereas the managing of manufactured products at the end of their life cycle has been almost entirely neglected. However, disassembly and recycling are becoming important factors as the ecological and economical implications of manufacturing raise increasing concerns. As an initial investigation of the very general problem of disassembly, in this paper the authors outline first the motivations and the potentially very important perspectives of this approach for robotics and automation research and for industrial application. Then the authors present a robotic system for the extraction, recognition and sorting of individual objects from an "agglomerate". Such operations are of crucial importance in several disassembly tasks, such as recycling and raw materials recovering. The system is based on the integration of different sensory modalities with motor actions. An example of application of the system is described and experimental results are discussed.>

The development of autonomous systems capable of executing intelligent exploratory procedures requires the understanding of the interactions between touch and motor control modalities. In robotic tactile perception this is the basic step toward the execution of highly sophisticated motor procedures such as grasping and manipulation. In this paper, the problem of autonomous learning of tactile-motor coordination is investigated in the case of a robotic system composed of a multi-functional tactile probe mounted on a robotic manipulator. A neural network architecture linking changes in the sensed tactile pattern with the motor action performed is described, and experimental results are analyzed. The generation of motor control procedures for actively estimating surface curvature is considered as an example of application of the proposed approach.>

Disassembly and recycling are becoming increasingly important in our society, especially for their ecological implications. In this paper the authors present an approach to disassembly problems, which is essentially based on the concepts of multisensory integration and fusion and on the use of purposive actions to simplify perceptual tasks. The authors present a robotic system for the recognition and sorting of individual objects from a group, operations that are extremely important in most disassembly tasks. An example application of the system is described and experimental results are discussed. The effectiveness of the system in operating in a partially structured environment, shows how problems which are difficult to manage by using a single sensory modality can be solved by integrating multisensory data.>

The authors present motivations for the long term research program on the various aspects of disassembly undertaken in their laboratory, and outline a theoretical framework for approaching disassembly problems. An application is described which is derived from such a framework in the context of a specific case-study of disassembly.

A system for visual recognition derived from a previously developed theoretical framework on the overall organization of the human visual system is proposed. The system operates dynamically by analyzing different parts of the input scene at variable levels of resolution through an attentional spotlight. A constant amount of information is gathered from the scene and a fixed dimension icon is produced, so that a trade-off occurs between the extension of the examined area and the level of resolution at which data are analyzed. The position of the spotlight and its dimensions are determined on the basis of the evolution of the recognition process. The icon is processed by a bottom-up path composed of a five-layer artificial neural network. The results of this net are analyzed by a planning module which determines if recognition has been achieved, or which action to undertake next. A top-down path, including a set of nets trained by the backpropagation algorithm, evaluates the parameters of the next sampling of information. The application of the system to object recognition with varying viewpoint and range from the camera is investigated.