The fusion of tactile and visual modalities is crucial for understanding objects and learning how to manipulate them. A common modus operandi in robotics is to deal with each of these modalities separately. We propose an integrated approach that associates to local visual features of an object, tactile feedback of the effector when touching that part of the object. Thus the agent learns to predict from a visual scene the shape/curvature properties of the object. The associated curvature properties are directly linked to grasp possibilities (as in approaches like [1] and [2]) but can also provide the agent with object categorization regarding the distribution of curvature classes.

We introduce affordance gradients (AGs), continuous sensorimotor structures that allow to predict the consequences of the agent's actions on the state of the environment. AGs allow to generalize among never performed actions and compress all possible consequences of the action state space. AGs also provide a way of estimating the world state after several interactions of the agent with objects. We validate the notion of AGs using benchmarks designed for mobile robotics that we solve using E-puck robot simulations: learn the affordances of several objects, push an object along a predefined trajectory and place an object in at a target position and orientation. We are interested in the neurophysiological basis of affordances and how they can be inserted in a sensorimotor loop with memory structures like the one proposed by the DAC architecture. We show that AGs provide a generalization of the perception-action couplets stored in memory and learned by they adaptive layer of DAC.

Sequence learning tasks depend on the ability to acquire and control the order of actions and their proper timing. Several studies have shown that in sequence learning different areas of the brain are involved when recalling the order of actions and their proper interval. One hypothesis proposes that two separate areas of the brain interact with each other, one computes order while the other would compute the interval. A second hypothesis proposes that one area computes both, order and interval. To better understand how this computation of order and interval might be realized by the brain, we developed a robot based architecture and investigated the behavioral and architectural implications of these two hypothesis: one or two neuronal areas computing order and interval. Using a sequence learning foraging task we show that performance is enhanced in case of distributed processes. However, we show that as a drawback, explicit interval information can not be reconstructed.

We present a novel biomimetic approach to mapless autonomous navigation based on insect neuroethology. We implemented and tested a real-time neuronal model based on the Distributed Adaptive Control framework. The model unifies different aspects of insect navigation and foraging including landmark recognition, chemical search, path integration and optimal memory usage. Consistent with recent findings the model supports navigation using heading direction information, thus precluding the use of global information. We tested our model using a mobile robot performing a foraging task. While foraging for chemical sources in a wind tunnel, the robot memorizes the followed trajectories, using information from landmarks and heading direction accumulators. After foraging, landmark navigation is tested with the odor source turned off. Our results show stability against robot kidnapping and generalization of homing behavior to stable mapless landmark navigation. This demonstrates that allocentric and efficient goal-oriented navigation strategies can be generated by relying on purely local information.