Action inhibition is essential for cognitive control, enabling individuals to prioritize relevant information over internal urges in response to changing demands. While current artificial agents excel in repetitive tasks, real-world scenarios often require the handling of unexpected constraints, such as the suppression of unwanted actions. Meta-learning, acting as an outer loop that regulates the reinforcement learning scheme in the inner loop of learning, facilitates adaptation in dynamic environments. Building upon our previous work, 1 where we implemented a brain-inspired meta-reinforcement learning framework for conflictual inhibition decision-making encompassing brain regions of the prefrontal cortex and the basal ganglia circuit and tested it within the NoGo and Stop-Signal Paradigms, this study introduces the following novelties. We explored the effects of changes in concentration and efficacy of the [Formula: see text]-mesocorticolimbic and [Formula: see text]-nigrostriatal pathways externally modulated by serotonin release on meta-reinforcement learning rules, thus the extent to which they affect behavioral performance during action cancellation within the Stop-Signal Paradigm. Our findings suggest that external serotoninergic modulation on these pathways asymmetrically affects behavioral performance, revealing that inhibitory behavior is primarily mediated by serotonin acting on [Formula: see text] dopamine receptors and is therefore asymmetrically influenced by changes in [Formula: see text] and [Formula: see text] efficacy. These pathways exhibit synergistic effects in response inhibition, with a predominant role for reductions in [Formula: see text] efficacy. Furthermore, we extended the meta-reinforcement learning framework by designing brain-inspired meta-learning rules that replicate the serotonin–dopamine dynamic interactions during action inhibition in a closed-loop fashion by using the Wilson–Cowan formalism 2 enabling a dynamic regulation of the exploration/exploitation rate [Formula: see text] meta-parameter. Our framework generates new predictive hypotheses and provides insights about the dynamic interaction between serotonin and dopamine [Formula: see text] and [Formula: see text], understanding their impact in response inhibition and, consequently, how they might be involved in impulsive behaviors. This knowledge suggests potential neural mechanisms underlying cognitive control in the brain and, at the same time, could contribute to the development of more flexible and robust artificial systems capable of adapting in real-world applications.

The human brain consolidates episodic memories into more semantic representations during sleep, enabling the continuos integration of new knowledge. This transformation is supported by hippocampal replays, which triggers a reactivation and reshaping of synaptic connections in the neocortex. In this study, we developed a plastic spiking neural network of Leaky Integrate-and-Fire (LIF) neurons to simulate the interaction between the hippocampus, perceptual neocortical areas, and neocortical regions responsible for processing semantic and contextual information. The model operates in two learning phases: first, the network encodes new experiences as episodic memories; then, during a sleep-like phase, hippocampal reactivation propagates to the neocortex. Crucially, the model leverages the apical mechanisms recently proposed by the experimentally grounded Dendritic Integration Theory (DIT) to modulate the activity of neural assemblies during both learning and replay. We evaluated the model on continual learning tasks, including split and rotational MNIST, and further demonstrated its practical applicability by training and testing it on sensory data from a soft pneumatic gripper in a dynamic robotic environment. • We present a spiking neural network that models a system memory consolidation through the interaction between the hippocampus and neocortex. • We implement a learning rule that leverages apical amplification mechanisms and the presence of neuromodulators and compare it with the classical STDP rule. • We evaluate the proposed model in four CL scenarios: Split MNIST, Rotated MNIST, CIFAR10, CIFAR100 and compared it with a PNN. • To demonstrate the practical applications of the proposed system memory consolidation model, we test it in a dynamic environment, specifically on a soft pneumatic gripper equipped with two force sensors and two flex sensors for object classification.

Catastrophic forgetting is a phenomenon in which a neural network, upon learning a new task, struggles to maintain its performance on previously learned tasks. It is a common challenge in the realm of continual learning (CL) through neural networks. The mammalian brain addresses catastrophic forgetting by consolidating memories in different parts of the brain, involving the hippocampus and the neocortex. Taking inspiration from this brain strategy, we present a CL framework that combines a plastic model simulating the fast learning capabilities of the hippocampus and a stable model representing the slow consolidation nature of the neocortex. To supplement this, we introduce a variational autoencoder (VAE)-based pseudo memory for rehearsal purposes. In addition by applying lateral inhibition masks on the gradients of the convolutional layer, we aim at damping the activity of adjacent neurons and introduce a sleep phase to reorganize the learned representations. Empirical evaluation demonstrates the positive impact of such additions on the performance of our proposed framework; we evaluate the proposed model on several class-incremental and domain-incremental datasets and compare it with the standard benchmark algorithms, showing significant improvements. With the aim to showcase practical applicability, we implement the algorithm in a physical environment for object classification using a soft pneumatic gripper. The algorithm learns new classes incrementally in real time and also exhibits significant backward knowledge transfer (KT).

High precision control of soft robots is challenging due to their stochastic behavior and material-dependence nature. While RL has been applied in soft robotics, achieving precision in task execution is still a long way off. Traditionally, RL requires substantial data for convergence, often obtained from a training environment. Yet, despite exhibiting high accuracy in the training environment, RL-policies often fall short in reality due to the training-to-reality gap, and the performance is exacerbated by the stochastic nature of soft robots. This study paves the way for the implementation of RL for soft robot control to achieve high precision in task execution. Two sample-efficient adaptive control strategies are proposed, that leverage the RL-policy. The schemes can overcome stochasticity, bridge the training-to-reality gap, and attain desired accuracy even in challenging tasks such as obstacle avoidance. Additionally, deliberate and reversible damage is induced to the pneumatic actuation chamber, altering the soft robot's behavior to test the adaptability of our solutions. Despite the damage, desired accuracy was achieved in most scenarios without needing to retrain the RL-policy.

Controlling a soft robot poses a challenge due to its mechanical characteristics.Although the manufacturing process is well-established, there are still shortcomings in their control, which often limits them to static tasks.In this study, we aim to address some of these limitations by introducing a neural network-based controller specifically designed for the throwing task using a soft robotic arm.Drawing inspiration from previous research, we have devised a method for controlling the movement of the soft robotic arm during the ballistic task.By employing a feed-forward neural network, we approximate the relationship between the actuation pattern and the resulting landing position.This enables us to predict the input sequence that needs to be transmitted to the robot's actuators based on the desired landing coordinates.To validate our approach, we conducted experiments using a 2-module soft robotic arm, which was utilized to throw four different objects towards ten target boxes positioned beneath the robot.We considered two actuation modalities, depending on whether the distal module was activated.The results indicate a success rate, defined as the proportion of successful trials out of the total number of throws, of up to 68% when a single module was actuated.These findings demonstrate the potential of our proposed controller in achieving successful performance of the throwing task using a soft robotic arm.

In this paper, we propose a novel dynamic gait controller for the repetitive behavior of soft robot manipulators performing routine tasks. Compliance with soft robots is advantageous when the robot interacts with living organisms and other fragile objects. However, predicting and controlling repetitive behavior is challenging because of hysteresis and non-linear dynamics governing the interactions. Existing priorfree methods track the dynamic state using recurrent neural networks or rely on known generalized coordinates describing the robot's state. We propose to model the interaction induced by the repetitive behavior as gait dynamics and represent the dynamic state with Central Pattern Generator (CPG) tracking the motion phase and thus reduce the complexity of the robot's forward model. The proposed method bootstraps an ensemble of the forward models exploring multiple dynamic contexts that are expanded as it searches for repetitive motion producing the target repetitive behavior. The proposed approach is experimentally validated on a pneumatically actuated soft robot arm I-Support, where the method infers gaits for different targets.

Conflictual cues and unexpected changes in human real-case scenarios may be detrimental to the execution of tasks by artificial agents, thus affecting their performance. Meta-learning applied to reinforcement learning may enhance the design of control algorithms, where an outer learning system progressively adjusts the operation of an inner learning system, leading to practical benefits for the learning schema. Here, we developed a brain-inspired meta-learning framework for inhibition cognitive control that i) exploits the meta-learning principles in the neuromodulation theory proposed by Doya, ii) relies on a well-established neural architecture that contains distributed learning systems in the human brain, and iii) proposes optimization rules of meta-learning hyperparameters that mimic the dynamics of the major neurotransmitters in the brain. We tested an artificial agent in inhibiting the action command in two well-known tasks described in the literature: NoGo and Stop-Signal Paradigms. After a short learning phase, the artificial agent learned to react to the hold signal, and hence to successfully inhibit the motor command in both tasks, via the continuous adjustment of the learning hyperparameters. We found a significant increase in global accuracy, right inhibition, and a reduction in the latency time required to cancel the action process, i.e., the Stop-signal reaction time. We also performed a sensitivity analysis to evaluate the behavioral effects of the meta-parameters, focusing on the serotoninergic modulation of the dopamine release. We demonstrated that brain-inspired principles can be integrated into artificial agents to achieve more flexible behavior when conflictual inhibitory signals are present in the environment.

Saccadic eye-movements play a crucial role in visuo-motor control by allowing rapid foveation onto new targets. However, the neural processes governing saccades adaptation are not fully understood. Saccades, due to the short-time of execution (20-100 ms) and the absence of sensory information for online feedback control, must be controlled in a ballistic manner. Incomplete measurements of the movement trajectory, such as the visual endpoint error, are supposedly used to form internal predictions about the movement kinematics resulting in predictive control. In order to characterize the synaptic and neural circuit mechanisms underlying predictive saccadic control, we have reconstructed the saccadic system in a digital controller embedding a spiking neural network of the cerebellum with spike timing-dependent plasticity (STDP) rules driving parallel fiber-Purkinje cell long-term potentiation and depression (LTP and LTD). This model implements a control policy based on a dual plasticity mechanism, resulting in the identification of the roles of LTP and LTD in regulating the overall quality of saccade kinematics: it turns out that LTD increases the accuracy by decreasing visual error and LTP increases the peak speed. The control policy also required cerebellar PCs to be divided into two subpopulations, characterized by burst or pause responses. To our knowledge, this is the first model that explains in mechanistic terms the visual error and peak speed regulation of ballistic eye movements in forward mode exploiting spike-timing to regulate firing in different populations of the neuronal network. This elementary model of saccades could be extended and applied to other more complex cases in which single jerks are concatenated to compose articulated and coordinated movements.

Multimodal information can significantly increase the perceptual capabilities of robotic agents, at the cost of a more complex sensory processing. This complexity can be reduced by employing machine learning techniques, provided that there is enough meaningful data to train on. This paper reports on creating novel datasets constructed by employing the iCub robot equipped with an additional depth sensor and color camera. We used the robot to acquire color and depth information for 210 objects in different acquisition scenarios. At the end, the results were large scale datasets that can be used for robot and computer vision applications: multisensory object representation, action recognition, rotation and distance invariant object recognition.

Soft robotic grippers are shown to be high effective for grasping unstructured objects with simple sensing and control strategies. However, they are still limited by their speed, sensing capabilities and actuation mechanism. Hence, their usage have been restricted in highly dynamic grasping tasks. This paper presents a soft robotic gripper with tunable bistable properties for sensor-less dynamic grasping. The bistable mechanism allows us to store arbitrarily large strain energy in the soft system which is then released upon contact. The mechanism also provides flexibility on the type of actuation mechanism as the grasping and sensing phase is completely passive. Theoretical background behind the mechanism is presented with finite element analysis to provide insights into design parameters. Finally, we experimentally demonstrate sensor-less dynamic grasping of an unknown object within 0.02 seconds, including the time to sense and actuate.

The cerebellum, which is responsible for motor control and learning, has been suggested to act as a Smith predictor for compensation of time-delays by means of internal forward models. However, insights about how forward model predictions are integrated in the Smith predictor have not yet been unveiled. To fill this gap, a novel bio-inspired modular control architecture that merges a recurrent cerebellar-like loop for adaptive control and a Smith predictor controller is proposed. The goal is to provide accurate anticipatory corrections to the generation of the motor commands in spite of sensory delays and to validate the robustness of the proposed control method to input and physical dynamic changes. The outcome of the proposed architecture with other two control schemes that do not include the Smith control strategy or the cerebellar-like corrections are compared. The results obtained on four sets of experiments confirm that the cerebellum-like circuit provides more effective corrections when only the Smith strategy is adopted and that minor tuning in the parameters, fast adaptation and reproducible configuration are enabled.

Dynamic control of soft robotic manipulators is an open problem yet to be well explored and analyzed. Most of the current applications of soft robotic manipulators utilize static or quasi-dynamic controllers based on kinematic models or linearity in the joint space. However, such approaches are not truly exploiting the rich dynamics of a soft-bodied system. In this paper, we present a model-based policy learning algorithm for closed-loop predictive control of a soft robotic manipulator. The forward dynamic model is represented using a recurrent neural network. The closed-loop policy is derived using trajectory optimization and supervised learning. The approach is verified first on a simulated piecewise constant strain model of a cable driven under-actuated soft manipulator. Furthermore, we experimentally demonstrate on a soft pneumatically actuated manipulator how closed-loop control policies can be derived that can accommodate variable frequency control and unmodeled external loads.

International audience

This paper describes a walking stabilization control on a soft ground based on gait analysis for a humanoid robot. There are many researches on gait analysis on a hard ground, but few scientists analyze the walking ability of human beings on a soft ground. Therefore, we conducted anthropometric measurement using a motion capture system on a soft ground. By analyzing experimental data, we obtained two findings. The first finding is that although there are no significant differences in step width and step length, step height tends to increase to avoid the collision between the feet and a soft ground. The second finding is that there are no significant differences in the lateral CoM trajectories but the vertical CoM amplitude increases when walking on a soft ground. Based on these findings, we developed a walking stabilization control to stabilize the CoM motion in the lateral direction on a soft ground. Verification of the proposed control is conducted through experiments with a human-sized humanoid robot WABIAN-2R. The experimental videos are supplemented.

The results of the neuroscientific research show that humans tend to stabilize the head orientation during locomotion. In this paper we describe the implementation of inverse kinematics based head stabilization controller on the humanoid platform. The controller uses the IMU feedback and controls neck joints in order to align the head orientation with the global orientation reference. Thanks to the method, we can decouple the orientational motion of the head from the rest of the body. This way stabilized head becomes better platform for proprioceptive sensory apparatus, such as cameras or IMU. In the paper we present three experiments which prove that the method has good performance in damping both, high and low frequency motion of the head. We also prove that the proposed controller improves the stability of the tracked goal point on the image of in-built camera.

In this work we propose an adaptive model for the head stabilization based on a feedback error learning (FEL). This model is capable to overcome the delays caused by the head motor system and adapts itself to the dynamics of the head motion. It has been designed to track an arbitrary reference orientation for the head in space and reject the disturbance caused by trunk motion. For efficient error learning we use the recursive least square algorithm (RLS), a Newton-like method which guarantees very fast convergence. Moreover, we implement a neural network to compute the rotational part of the head inverse kinematics. Verification of the proposed control is conducted through experiments with Matlab SIMULINK and a humanoid robot SABIAN.

The maintenance of a stable and coherent representation of the surrounding environment is an essential capability in cognitive robotic systems. Most systems employ some form of 3D perception to create internal representations of space (maps) to support tasks such as navigation, manipulation and interaction. The creation and update of such representations may represent a significant effort in the overall computation performed by the robot. In this paper we propose an architecture based on the concept of Expected Perception that allows lightweight map updates whenever the course of action happens according to the robot's expectations. It is only when the robot's predictions and the real world outcomes differ, that corrections must be done at its full extent. We performed experiments and show results in a real robotic platform with stereo (3D) perception where map corrections are proposed by simple image level (2D) comparisons.

The purpose of this work is to investigate the applicability of a visual tracking model on humanoid robots in order to achieve a human-like predictive behavior. In humans, in case of moving targets the oculomotor system uses a combination of the smooth pursuit eye movement and saccadic movements, namely “catch up” saccades to fixate the object of interest. This work aims to validate the "catch up" saccade model in order to obtain a human-like tracking system able to correctly switch from a zero-lag predictive smooth pursuit to a fast orienting saccade for the position error compensation. Experimental results on the iCub simulator show several correspondences with the human behavior.