This paper introduces the Approximate Ethical Embedding Process, an algorithm for automating the design of ethical environments for learning agents. Our algorithm helps build environments wherein multiple agents learn policies that align with an ethical (moral) value while simultaneously pursuing their individual objectives. Therefore, we contribute to endowing environment designers with algorithmic tools for building ethical environments. We demonstrate the ethical design process for two different settings of an environment where agents have to adhere to beneficence to promote the collective survival of the population. Our experiments show that our approximate embedding process successfully generates environments that incentivise the learning of value-aligned policies.

The fusion of tactile and visual modalities is crucial for understanding objects and learning how to manipulate them. A common modus operandi in robotics is to deal with each of these modalities separately. We propose an integrated approach that associates to local visual features of an object, tactile feedback of the effector when touching that part of the object. Thus the agent learns to predict from a visual scene the shape/curvature properties of the object. The associated curvature properties are directly linked to grasp possibilities (as in approaches like [1] and [2]) but can also provide the agent with object categorization regarding the distribution of curvature classes.

The cerebellum is a brain structure necessary for skilled motor behaviour and has a well understood and repetitive architecture. Such an architecture inspired the Marr-Albus-Ito theory of cerebellar learning, that provides an explanation for the acquisition of motor skills by the cerebellum. Numerous computational models inspired in such a theory have already been employed in robotic tasks. Here we look into one of the suggested roles of the cerebellum, the replacement of reflexes by anticipatory actions and we apply it to a robot navigation task. The acquisition of anticipatory actions has been thoroughly studied in the field of classical conditioning. Of particular interest is the so-called CS-intensity effect, an effect that links the rapidity of execution of an anticipatory protective action, the Conditioned Response (CR), to the intensity of a predictive signal, the Conditioning Stimulus (CS). We propose that the CS-intensity effect implements a built-in sensory-motor contingency that allows to carry over a skill learned in a safe and easy context, e.g., turning at slow velocity, to a more difficult one, e.g., a turning at a faster speed. We demonstrate this hypothesis in a series of experiments where a robot has to navigate a track that has a turn. We show that after being trained at a slow velocity, by means of the CS-intensity effect, the cerebellar controller modulates the turning such that its onset anticipates as the robot speed increases. Ultimately, through incremental learning, this generalization allows the robot to learn to navigate the track at its maximum speed.

We introduce affordance gradients (AGs), continuous sensorimotor structures that allow to predict the consequences of the agent's actions on the state of the environment. AGs allow to generalize among never performed actions and compress all possible consequences of the action state space. AGs also provide a way of estimating the world state after several interactions of the agent with objects. We validate the notion of AGs using benchmarks designed for mobile robotics that we solve using E-puck robot simulations: learn the affordances of several objects, push an object along a predefined trajectory and place an object in at a target position and orientation. We are interested in the neurophysiological basis of affordances and how they can be inserted in a sensorimotor loop with memory structures like the one proposed by the DAC architecture. We show that AGs provide a generalization of the perception-action couplets stored in memory and learned by they adaptive layer of DAC.

Sequence learning tasks depend on the ability to acquire and control the order of actions and their proper timing. Several studies have shown that in sequence learning different areas of the brain are involved when recalling the order of actions and their proper interval. One hypothesis proposes that two separate areas of the brain interact with each other, one computes order while the other would compute the interval. A second hypothesis proposes that one area computes both, order and interval. To better understand how this computation of order and interval might be realized by the brain, we developed a robot based architecture and investigated the behavioral and architectural implications of these two hypothesis: one or two neuronal areas computing order and interval. Using a sequence learning foraging task we show that performance is enhanced in case of distributed processes. However, we show that as a drawback, explicit interval information can not be reconstructed.

Rodents are optimal real-world foragers that can smoothly regulate behaviors like homing and exploration combined with more elaborate abilities as food source localization. Here we investigate a robot based model that implements the self-regulatory processes that underly optimal foraging of rodents in unknown environments and is also able to combine it with goal directed behaviors. Behavior is decomposed into minimal homeostatic subsystems that regulate themselves through the local perception/detection of a gradient. On a higher level, the allostatic control orchestrates the interaction of the different homeostatic modules allowing it to dynamically manage the interactions between the desired values of each subsystem to achieve stability on a meta behavioral level. In this case, we show that overall behavioral stability can be achieved. We validate our model by comparing the behavior of both simulated and real robots with that of rodents. Our next step is then to justify gradients as a valid biological assumption by giving a biologically plausible process for generating them from a cognitive map, in this case, a set of approximated hippocampal place cells. We finally formulate path planning (used for goal reaching, e.g. food source localization) in the same context of a gradient map generation that can be then inserted as an additional subsystem of the higher meta level allostatic control.