We present a framework for learning a single policy capable of producing all quadruped gaits and transitions. The framework consists of a policy trained with deep reinforcement learning (DRL) to modulate the parameters of a system of abstract oscillators (i.e. Central Pattern Generator), whose output is mapped to joint commands through a pattern formation layer that sets the gait style, i.e. body height, swing foot ground clearance height, and foot offset. Different gaits are formed by changing the coupling between different oscillators, which can be instantaneously selected at any velocity by a user. With this framework, we systematically investigate which gait should be used at which velocity, and when gait transitions should occur from a Cost of Transport (COT), i.e. energy-efficiency, point of view. Additionally, we note how gait style changes as a function of locomotion speed for each gait to keep the most energy-efficient locomotion. While the currently most popular gait (trot) does not result in the lowest COT, we find that considering different co-dependent metrics such as mean base angular velocity and joint acceleration result in different 'optimal' gaits than those that minimize COT. We deploy our controller in various hardware experiments, focusing on 9 quadruped animal gaits, and demonstrate generalizability to novel and unseen gaits during training, and robustness to leg failures.

Hip exoskeletons are increasing in popularity due to their effectiveness across various scenarios and their ability to adapt to different users. However, personalizing the assistance often requires lengthy tuning procedures and computationally intensive algorithms, and most existing methods do not incorporate user feedback. In this work, we propose a novel approach for rapidly learning users' preferences for hip exoskeleton assistance. We perform pairwise comparisons of distinct randomly generated assistive profiles, and collect participants preferences through active querying. Users' feed-back is integrated into a preference-learning algorithm that updates its belief, learns a user-dependent reward function, and changes the assistive torque profiles accordingly. Results from eight healthy subjects display distinct preferred torque profiles, and users' choices remain consistent when compared to a perturbed profile. A comprehensive evaluation of users' preferences reveals a close relationship with individual walking strategies. The tested torque profiles do not disrupt kinematic joint synergies, and participants favor assistive torques that are synchronized with their movements, resulting in lower negative power from the device. This straightforward approach enables the rapid learning of users preferences and rewards, grounding future studies on reward-based human-exoskeleton interaction.

Maintaining balance during walking is a critical yet under-addressed challenge in the control of lower-limb exoskeletons, especially for users with progressive neurological conditions such as multiple sclerosis and muscular dystrophy. While assist-as-needed strategies have enabled flexible support in the sagittal plane, most exoskeletons lack active control in the frontal plane, limiting their ability to support mediolateral (ML) balance. In this study, we introduce a 3D Path Control strategy that enables coordinated assistance across hip abduction/adduction, hip flexion/extension, and knee flexion/extension. The controller is designed to provide partial gait assistance while preserving user autonomy and the ability to modulate step width, an essential mechanism for maintaining ML balance. Two experiments with healthy participants were conducted to evaluate the approach. The first experiment showed that increasing ML assistance improved alignment with a nominal coordination pattern and allowed modulation of hip abduction/adduction range of motion, and consequently, lateral foot placement. The second experiment demonstrated that even with constraining controller settings, users could still deviate from the desired path and adopt different step widths. These results suggest that 3D Path Control can simultaneously assist gait and support balance by combining structured inter-joint coordination while still providing flexibility in foot placement to the subjects.

Humans can perform movements in various physical environments and positions (corresponding to different experienced gravity), requiring the interaction of the musculoskeletal system, the neural system and the external environment. The neural system is itself comprised of several interactive components, from the brain mainly conducting motor planning, to the spinal cord (SC) implementing its own motor control centres through sensory reflexes. Nevertheless, it remains unclear whether similar movements in various environmental dynamics necessitate adapting modulation at the brain level, correcting modulation at the spinal level, or both. Here, we addressed this question by focusing on upper limb motor control in various gravity conditions (magnitudes and directions) and using neuromusculoskeletal simulation tools. We integrated supraspinal sinusoidal commands with a modular SC model controlling a musculoskeletal model to reproduce various recorded arm trajectories (kinematics and EMGs) in different contexts. We first studied the role of various spinal pathways (such as stretch reflexes) in movement smoothness and robustness against perturbation. Then, we optimised the supraspinal sinusoidal commands without and with a fixed SC model including stretch reflexes to reproduce a target trajectory in various gravity conditions. Inversely, we fixed the supraspinal commands and optimised the spinal synaptic strengths in the different environments. In the first optimisation context, the presence of SC resulted in easier optimisation of the supraspinal commands (faster convergence, better performance). The main supraspinal commands modulation was found in the flexor sinusoid's amplitude, resp. frequency, to adapt to different gravity magnitudes, resp. directions. In the second optimisation context, the modulation of the spinal synaptic strengths also remarkably reproduced the target trajectory for the mild gravity changes. We highlighted that both strategies of modulation of the supraspinal commands or spinal stretch pathways can be used to control movements in different gravity environments. Our results thus support that the SC can assist gravity compensation.

This study introduces a novel neuromechanical model employing a detailed spiking neural network to explore the role of axial proprioceptive sensory feedback, namely stretch feedback, in salamander locomotion. Unlike previous studies that often oversimplified the dynamics of the locomotor networks, our model includes detailed simulations of the classes of neurons that are considered responsible for generating movement patterns. The locomotor circuits, modeled as a spiking neural network of adaptive leaky integrate-and-fire neurons, are coupled to a three-dimensional mechanical model of a salamander with realistic physical parameters and simulated muscles. In open-loop simulations (i.e., without sensory feedback), the model replicates locomotor patterns observed in-vitro and in-vivo for swimming and trotting gaits. Additionally, a modular descending reticulospinal drive to the central pattern generation network allows to accurately control the activation, frequency and phase relationship of the different sections of the limb and axial circuits. In closed-loop swimming simulations (i.e. including axial stretch feedback), systematic evaluations reveal that intermediate values of feedback strength increase the tail beat frequency and reduce the intersegmental phase lag, contributing to a more coordinated, faster and energy-efficient locomotion. Interestingly, the result is conserved across different feedback topologies (ascending or descending, excitatory or inhibitory), suggesting that it may be an inherent property of axial proprioception. Moreover, intermediate feedback strengths expand the stability region of the network, enhancing its tolerance to a wider range of descending drives, internal parameters' modifications and noise levels. Conversely, high values of feedback strength lead to a loss of controllability of the network and a degradation of its locomotor performance. Overall, this study highlights the beneficial role of proprioception in generating, modulating and stabilizing locomotion patterns, provided that it does not excessively override centrally-generated locomotor rhythms. This work also underscores the critical role of detailed, biologically-realistic neural networks to improve our understanding of vertebrate locomotion.

We present a framework for learning visually-guided quadruped locomotion by integrating exteroceptive sensing and central pattern generators (CPGs), i.e. systems of coupled oscillators, into the deep reinforcement learning (DRL) framework. Through both exteroceptive and proprioceptive sensing, the agent learns to coordinate rhythmic behavior among different oscillators to track velocity commands, while at the same time override these commands to avoid collisions with the environment. We investigate several open robotics and neuroscience questions: 1) What is the role of explicit interoscillator couplings between oscillators, and can such coupling improve sim-to-real transfer for navigation robustness? 2) What are the effects of using a memory-enabled vs. a memory-free policy network with respect to robustness, energy-efficiency, and tracking performance in sim-to-real navigation tasks? 3) How do animals manage to tolerate high sensorimotor delays, yet still produce smooth and robust gaits? To answer these questions, we train our perceptive locomotion policies in simulation and perform sim-to-real transfers to the Unitree Go1 quadruped, where we observe robust navigation in a variety of scenarios. Our results show that the CPG, explicit interoscillator couplings, and memory-enabled policy representations are all beneficial for energy efficiency, robustness to noise and sensory delays of 90 ms, and tracking performance for successful sim-to-real transfer for navigation tasks.

Assistive devices, such as exoskeletons and prostheses, have revolutionized the field of rehabilitation and mobility assistance. Efficiently detecting transitions between different activities, such as walking, stair ascending and descending, and sitting, is crucial for ensuring adaptive control and enhancing user experience. We present an approach for real-time transition detection, aimed at optimizing the processing-time performance. By establishing activity-specific threshold values through trained machine learning models, we effectively distinguish motion patterns and we identify transition moments between locomotion modes. This threshold-based method improves real-time embedded processing time performance by up to 11 times compared to machine learning approaches. The efficacy of the developed finite-state machine is validated using data collected from three different measurement systems. Moreover, experiments with healthy participants were conducted on an active pelvis orthosis to validate the robustness and reliability of our approach. The proposed algorithm achieved high accuracy in detecting transitions between activities. These promising results show the robustness and reliability of the method, reinforcing its potential for integration into practical applications.

Balance loss is a significant challenge in lower-limb exoskeleton applications, as it can lead to potential falls, thereby impacting user safety and confidence. We introduce a control framework for omnidirectional recovery step planning by online optimization of step duration and position in response to external forces. We map the step duration and position to a human-like foot trajectory, which is then translated into joint trajectories using inverse kinematics. These trajectories are executed via an impedance controller, promoting cooperation between the exoskeleton and the user. Moreover, our framework is based on the concept of the divergent component of motion, also known as the Extrapolated Center of Mass, which has been established as a consistent dynamic for describing human movement. This real-time online optimization framework enhances the adaptability of exoskeleton users under unforeseen forces thereby improving the overall user stability and safety. To validate the effectiveness of our approach, simulations, and experiments were conducted. Our push recovery experiments employing the exoskeleton in zero-torque mode (without assistance) exhibit an alignment with the exoskeleton’s recovery assistance mode, that shows the consistency of the control framework with human intention. To the best of our knowledge, this is the first cooperative push recovery framework for the lower-limb human exoskeleton that relies on the simultaneous adaptation of intra-stride parameters in both frontal and sagittal directions. The proposed control scheme has been validated with human subject experiments.

Learning a locomotion policy for quadruped robots has traditionally been constrained to a specific robot morphology, mass, and size. The learning process must usually be repeated for every new robot, where hyperparameters and reward function weights must be re-tuned to maximize performance for each new system. Alternatively, attempting to train a single policy to accommodate different robot sizes, while maintaining the same degrees of freedom (DoF) and morphology, requires either complex learning frameworks, or mass, inertia, and dimension randomization, which leads to prolonged training periods. In our study, we show that drawing inspiration from animal motor control allows us to effectively train a single locomotion policy capable of controlling a diverse range of quadruped robots. The robot differences encompass: a variable number of DoFs, (i.e. 12 or 16 joints), three distinct morphologies, a broad mass range spanning from 2 kg to 200 kg, and nominal standing heights ranging from 18 cm to 100 cm. Our policy modulates a representation of the Central Pattern Generator (CPG) in the spinal cord, effectively coordinating both frequencies and amplitudes of the CPG to produce rhythmic output (Rhythm Generation), which is then mapped to a Pattern Formation (PF) layer. Across different robots, the only varying component is the PF layer, which adjusts the scaling parameters for the stride height and length. Subsequently, we evaluate the sim-to-real transfer by testing the single policy on both the Unitree Go1 and A1 robots. Remarkably, we observe robust performance, even when adding a 15 kg load, equivalent to 125% of the A1 robot’s nominal mass.

This paper presents a framework for dynamic object catching using a quadruped robot’s front legs while it stands on its rear legs. The system integrates computer vision, trajectory prediction, and leg control to enable the quadruped to visually detect, track, and successfully catch a thrown object using an onboard camera. Leveraging a fine-tuned YOLOv8 model for object detection and a regression-based trajectory prediction module, the quadruped adapts its front leg positions iteratively to anticipate and intercept the object. The catching maneuver involves identifying the optimal catching position, controlling the front legs with Cartesian PD control, and closing the legs together at the right moment. We propose and validate three different methods for selecting the optimal catching position: 1) intersecting the predicted trajectory with a vertical plane, 2) selecting the point on the predicted trajectory with the minimal distance to the center of the robot’s legs in their nominal position, and 3) selecting the point on the predicted trajectory with the highest likelihood on a Gaussian Mixture Model (GMM) modelling the robot’s reachable space. Experimental results demonstrate robust catching capabilities across various scenarios, with the GMM method achieving the best performance, leading to an 80% catching success rate.

Animals possess a remarkable ability to navigate challenging terrains, achieved through the interplay of various pathways between the brain, central pattern generators (CPGs) in the spinal cord, and musculoskeletal system. Traditional bioinspired control frameworks often rely on a singular control policy that models both higher (supraspinal) and spinal cord functions. In this work, we build upon our previous research by introducing two distinct neural networks: one tasked with modulating the frequency and amplitude of CPGs to generate the basic locomotor rhythm (referred to as the spinal policy), and the other responsible for receiving environmental perception data and directly modulating the rhythmic output from the spinal policy to execute precise movements on challenging terrains (referred to as the descending modulation policy). This division of labor more closely mimics the hierarchical locomotor control systems observed in legged animals, thereby enhancing the robot’s ability to navigate various uneven surfaces, including steps, high obstacles, and terrains with gaps. Additionally, we investigate the impact of sensorimotor delays within our framework, validating several biological assumptions about animal locomotion systems. Specifically, we demonstrate that spinal circuits play a crucial role in generating the basic locomotor rhythm, while descending pathways are essential for enabling appropriate gait modifications to accommodate uneven terrain. Notably, our findings also reveal that the multi-layered control inherent in animals exhibits remarkable robustness against sensorimotor delays. These findings advance our understanding of the fundamental principles governing the interplay between spinal and supraspinal mechanisms in biological locomotion. Moreover, they inform the design of bioinspired locomotion controllers that emulate these biological structures, facilitating natural movement in complex and realistic environments.

Typical legged locomotion controllers are designed or trained offline. This is in contrast to many animals, which are able to locomote at birth, and rapidly improve their locomotion skills with few real-world interactions. Such motor control is possible through oscillatory neural networks located in the spinal cord of vertebrates, known as Central Pattern Generators (CPGs). Models of the CPG have been widely used to generate locomotion skills in robotics, but can require extensive hand-tuning or offline optimization of inter-connected parameters with genetic algorithms. In this paper, we present a framework for the online optimization of the CPG parameters through Bayesian Optimization. We show that our framework can rapidly optimize and adapt to varying velocity commands and changes in the terrain, for example to varying coefficients of friction, terrain slope angles, and added mass payloads placed on the robot. We study the effects of sensory feedback on the CPG, and find that both force feedback in the phase equations, as well as posture control (Virtual Model Control) are both beneficial for robot stability and energy efficiency. In hardware experiments on the Unitree Go1, we show rapid optimization (in under 3 minutes) and adaptation of energy-efficient gaits to varying target velocities in a variety of scenarios: varying coefficients of friction, added payloads up to 15 kg, and variable slopes up to 10 degrees.

Quadruped robots are showing impressive abilities to navigate the real world. If they are to become more integrated into society, social trust in interactions with humans will become increasingly important. Additionally, robots will need to be adaptable to different humans based on individual preferences. In this work, we study the social interaction task of learning optimal handshakes for quadruped robots based on user preferences. While maintaining balance on three legs, we parameterize handshakes with a Central Pattern Generator consisting of an amplitude, frequency, stiffness, and duration. Through 10 binary choices between handshakes, we learn a belief model to fit individual preferences for 25 different subjects. Our results show that this is an effective strategy, with 76% of users feeling happy with their identified optimal handshake parameters, and 20% feeling neutral. Moreover, compared with random and test handshakes, the optimized handshakes have significantly decreased errors in amplitude and frequency, lower Dynamic Time Warping scores, and improved energy efficiency, all of which indicate robot synchronization to the user’s preferences.

Complex interactions between brain regions and the spinal cord (SC) govern body motion, which is ultimately driven by muscle activation. Motor planning or learning are mainly conducted at higher brain regions, whilst the SC acts as a brain-muscle gateway and as a motor control centre providing fast reflexes and muscle activity regulation. Thus, higher brain areas need to cope with the SC as an inherent and evolutionary older part of the body dynamics. Here, we address the question of how SC dynamics affects motor learning within the cerebellum; in particular, does the SC facilitate cerebellar motor learning or constitute a biological constraint? We provide an exploratory framework by integrating biologically plausible cerebellar and SC computational models in a musculoskeletal upper limb control loop. The cerebellar model, equipped with the main form of cerebellar plasticity, provides motor adaptation; whilst the SC model implements stretch reflex and reciprocal inhibition between antagonist muscles. The resulting spino-cerebellar model is tested performing a set of upper limb motor tasks, including external perturbation studies. A cerebellar model, lacking the implemented SC model and directly controlling the simulated muscles, was also tested in the same. The performances of the spino-cerebellar and cerebellar models were then compared, thus allowing directly addressing the SC influence on cerebellar motor adaptation and learning, and on handling external motor perturbations. Performance was assessed in both joint and muscle space, and compared with kinematic and EMG recordings from healthy participants. The differences in cerebellar synaptic adaptation between both models were also studied. We conclude that the SC facilitates cerebellar motor learning; when the SC circuits are in the loop, faster convergence in motor learning is achieved with simpler cerebellar synaptic weight distributions. The SC is also found to improve robustness against external perturbations, by better reproducing and modulating muscle cocontraction patterns.

Quadruped animal locomotion emerges from the interactions between the spinal central pattern generator (CPG), sensory feedback, and supraspinal drive signals from the brain. Computational models of CPGs have been widely used for investigating the spinal cord contribution to animal locomotion control in computational neuroscience and in bio-inspired robotics. However, the contribution of supraspinal drive to anticipatory behavior, i.e. motor behavior that involves planning ahead of time (e.g. of footstep placements), is not yet properly understood. In particular, it is not clear whether the brain modulates CPG activity and/or directly modulates muscle activity (hence bypassing the CPG) for accurate foot placements. In this paper, we investigate the interaction of supraspinal drive and a CPG in an anticipatory locomotion scenario that involves stepping over gaps. By employing deep reinforcement learning (DRL), we train a neural network policy that replicates the supraspinal drive behavior. This policy can either modulate the CPG dynamics, or directly change actuation signals to bypass the CPG dynamics. Our results indicate that the direct supraspinal contribution to the actuation signal is a key component for a high gap crossing success rate. However, the CPG dynamics in the spinal cord are beneficial for gait smoothness and energy efficiency. Moreover, our investigation shows that sensing the front feet distances to the gap is the most important and sufficient sensory information for learning gap crossing. Our results support the biological hypothesis that cats and horses mainly control the front legs for obstacle avoidance, and that hind limbs follow an internal memory based on the front limbs' information. Our method enables the quadruped robot to cross gaps of up to 20 cm (50% of body-length) without any explicit dynamics modeling or Model Predictive Control (MPC).

Efficient and safe multi-agent swarm coordination in environments where humans operate, such as warehouses, assistive living rooms, or automated hospitals, is crucial for adopting automation. In this paper, we augment the obstacle avoidance algorithm based on dynamical system modulation for a swarm of heterogeneous holonomic mobile agents. A smooth prioritization is proposed to change the reactivity of the swarm towards the specific agents. Further, a soft decoupling of the initial agent's kinematics is used to design an independent rotation control to ensure the agent reaches the desired position and orientation simultaneously. This decoupling allowed the introduction of a novel heuristic, the virtual drag. It minimizes the disturbance influence an agent has when moving through its surrounding. Additionally, the safety module adapts the velocity commands from the dynamical system modulation to avoid colliding trajectories between agents. The evaluation was performed in simulated assisted living and hospital environments. The prioritization successfully increased the minimum distance relative to a moving agent. The safety module is observed to create collision-free dynamics where alternative methods fail. Additionally, the repulsive nature of the safety module augments the convergence rate, thus making the proposed method better applicable to dense real-world scenarios.

Collaborative object manipulation and transport with self-reconfigurable modular robots can take a major role in improving modularity and adaptability of smart-home and factory-like environments. Controlling modules to achieve efficient behaviours is challenging due to the high number of degrees of freedom in the system and the physical constraints. We present an end-to-end planner that discovers collaborative behaviours for modules to manipulate and transport objects to bring them to a human defined place. Our approach is based on a centralized planner using stochastic best-first search with a custom heuristic and pruning strategy. We use Quadratic Programming to define multi-robot controller to evaluate action feasibility for transitions between the search tree nodes with respect to important constraints of the system (collisions, joint and torque limits). The controller can be design to be aware of human reachable space for object handover and use it as a measure to asses closeness to the goal node. Results show that the proposed method can effectively coordinate the actions of multiple robots, leading to an emerging efficient manipulation and transport of objects with variable shapes and weight to within human reachable space. This work brings self-reconfigurable modular robots one step closer to assistive human-robot interaction applications or smart logistics.

Animal locomotion results from a combination of power modulation and cyclic appendage trajectories, but combining these two properties in small-sized robots is difficult. Here, we introduce and characterize a new elastic actuation system based on an inverted cam that is capable of generating cyclic locomotion with controlled elastic energy charge and release for small-sized robots. We designed a leg linkage and attached to the inverted cam to develop a single legged hopping platform with one actuated degree of freedom. The hopping platform was able to continuously hop forward at 1.82 Hz. The average horizontal hopping distance was 18.7 cm, and the average forward speed was 0.34 m/s. This speed was corresponding to a Froude number of 0.14. The energy consumed for one hop was 2.09 J, and the corresponding energetic cost of transport was 6.43. The combination of inverted cam and cyclic trajectory generation has the potential to be used in other robotic applications, such as flapping wings in the air and tail fin waving in water.

Assessing the traversability of rugged terrain is a difficult challenge for legged robots, especially when they implement multiple, distinct gaits. We tackle this problem on the k-rock2 amphibious, sprawling gait robot by training a gait-dependent traversability estimator. We verify that the estimator, trained solely on procedurally-generated simulated data, approaches the outcomes of real-world experiments conducted in an indoor motion capture arena using two distinct terrestrial gaits to cross various indoor obstacles. In simulation experiments on a large-scale outdoor heightmap representing real-world data, we quantify the performance gain using the estimator outputs for gait selection. Further, we apply the method to heightmaps of outdoor data to illustrate how the approach could readily be applied to field scenarios.

Physical simulation is an indispensable component of robotics simulation platforms that serves as the basis for a plethora of research directions. Looking strictly at robotics, the common characteristic of the most popular physics engines, such as ODE, DART, MuJoCo, bullet, SimBody, PhysX or RaiSim, is that they focus on the solution of articulated rigid bodies with collisions and contacts problems, while paying less attention to other physical phenomena. This restriction limits the range of addressable simulation problems, rendering applications such as soft robotics, cloth simulation, simulation of viscoelastic materials, and fluid dynamics, especially surface swimming, infeasible. In this work, we present Gazebo Fluids, an open-source extension of the popular Gazebo robotics simulator that enables the interaction of articulated rigid body dynamics with particle-based fluid and deformable solid simulation. We implement fluid dynamics and highly viscous and elastic material simulation capabilities based on the Smoothed Particle Hydrodynamics method. We demonstrate the practical impact of this extension for previously infeasible application scenarios in a series of experiments, showcasing one of the first self-propelled robot swimming simulations with SPH in a robotics simulator.

In order to facilitate and assist the indoor mobility of people with special needs, the classically static objects in the environment, such as furniture, can be rendered mobile. The need for efficient and safe autonomous coordination of a mobile furniture swarm arises. We present a closed-form approach for mobile furniture obstacle avoidance and navigation within an indoor environment. The approach shows that each mobile furniture agent, defined by a polygonal surface, does not collide with any static or mobile obstacle (e.g., a person is moving around). All controllable mobile furniture converges towards a defined goal position and orientation. We showcase the application of this algorithm in simulation on mobile furniture for smart environments. Results demonstrate that the proposed method can coordinate a swarm of mobile furniture to get out of the way of a mobile agent representing a person with limited mobility passing through the room while avoiding obstacles and converging towards a predefined target pose.

This article examines the importance of integrating locomotion and cognitive information for achieving dynamic locomotion from a viewpoint combining biology and ecological psychology. We present a mammalian neuromusculoskeletal model from external sensory information processing to muscle activation, which includes: 1) a visual-attention control mechanism for controlling attention to external inputs; 2) object recognition representing the primary motor cortex; 3) a motor control model that determines motor commands traveling down the corticospinal and reticulospinal tracts; 4) a central pattern generation model representing pattern generation in the spinal cord; and 5) a muscle reflex model representing the muscle model and its reflex mechanism. The proposed model is able to generate the locomotion of a quadruped robot in flat and natural terrain. The experiment also shows the importance of a postural reflex mechanism when experiencing a sudden obstacle. We show the reflex mechanism when a sudden obstacle is separately detected from both external (retina) and internal (touching afferent) sensory information. We present the biological rationale for supporting the proposed model. Finally, we discuss future contributions, trends, and the importance of the proposed research.

A bio-inspired way to model locomotion is using a network of coupled phase oscillators to create a Central Pattern Generator (CPG). The recently developed feedback control method tegotae includes exteroceptive force feedback into the governing phase update equations, leading to gait limit cycles. However, the oscillator coupling weights are often determined empirically. Here, we first investigate how the coupling coefficients influence the limit cycle convergence behavior on a 2- and 3-limbed structure in simulation. We find that the convergence with tegotae can be improved by introducing appropriate cross-couplings. This results in a smoother convergence and steady-state behavior where each individual oscillator drives the full network to a common convergence state in comparison to competing convergence states with ill-chosen cross-couplings. We then validate the findings in hardware and hypothesize how the appropriate couplings could be derived directly from the morphology, potentially eliminating the empiric determination.

The central nervous system of humans and other animals modulates spinal cord activity to achieve several locomotion behaviors. Previous neuromechanical models investigated the modulation of human gait changing selected parameters belonging to CPGs (Central Pattern Generators) feedforward oscillatory structures or to feedback reflex circuits. CPG-based models could replicate slow and fast walking by changing only the oscillation's properties. On the other hand, reflex-based models could achieve different behaviors through optimizations of large dimensional parameter spaces. However, they could not effectively identify individual key reflex parameters responsible for gait characteristics' modulation. This study investigates which reflex parameters modulate the gait characteristics through neuromechanical simulations. A recently developed reflex-based model is used to perform optimizations with different target behaviors on speed, step length, and step duration to analyze the correlation between reflex parameters and their influence on these gait characteristics. We identified nine key parameters that may affect the target speed ranging from slow to fast walking (0.48 and 1.71 m/s) as well as a large range of step lengths (0.43 and 0.88 m) and step duration (0.51, 0.98 s). The findings show that specific reflexes during stance significantly affect step length regulation, mainly given by positive force feedback of the ankle plantarflexors' group. On the other hand, stretch reflexes active during swing of iliopsoas and gluteus maximus regulate all the gait characteristics under analysis. Additionally, the results show that the hamstrings' group's stretch reflex during the landing phase is responsible for modulating the step length and step duration. Additional validation studies in simulations demonstrated that the modulation of identified reflexes is sufficient to regulate the investigated gait characteristics. Thus, this study provides an overview of possible reflexes involved in modulating speed, step length, and step duration of human gaits.

Human and animal locomotion are controlled by complex neural circuits, which can also serve as inspiration for designing locomotion controllers for dynamic locomotion in legged robots. We develop a locomotion controller model including a central pattern generator (CPGs) and a muscle reflex based on the forelimb and hindlimb structures of a cat. In this paper, we focus on modeling the muscle reflex and its optimization. This muscle reflex model regulates ground force afferents in each limb. There are two phases in each step performed by this model, the swing and stance phases. The muscle during swing phase is activated by a pattern formation signal from the CPG. During stance phase, the muscle is automatically controlled by the moving speed. We utilize a multi-objective evolutionary algorithm to optimize parameters of the model. We use the proposed model to control a cat-like robot in simulations using Open Dynamics Engine. Results show that the simulated robot is able to move at different speeds by modulating simple stimulation signals to the CPG without needing to modify muscle and reflex parameters.

Gait emergence and adaptation in animals is unmatched in robotic systems. Animals can create and recover locomotive functions "on-the-fly" after an injury whereas locomotion controllers for robots lack robustness to morphological changes. In this work, we extend previous research on emergent interlimb coordination of legged robots based on coupled phase oscillators with force feedback terms. We investigate how the coupling weights between these phase oscillators can be extracted from the morphology with a fast and computationally lightweight method based on a combination of twitching and Hebbian learning to form sensor-motor maps. The coefficients of these maps create naturally scaled weights, which not only lead to robust gait limit cycles, but can also adapt to morphological modifications such as sensor loss and limb injuries within a few gait cycles. We demonstrate the approach on a robotic quadruped and hexapod.

In the field of quadruped locomotion, dynamic locomotion behavior, and rich integration with sensory feedback represents a significant development. In this paper, we present an efficient neural model, which includes CPG and its sensorimotor coordination, and demonstrate its implementation in a quadruped robot to show how efficient integration of motor and sensory feedback can generate dynamic behavior and how sensorimotor coordination reconstructs the sensory network for leg malfunction compensation. Additionally, we delineate a network optimization strategy and suggest sensorimotor coordination as a strategy for controlling speed and regulating internal and external adaptation. The rhythm generation representing the leg injury was inactive, stimulating the sensorimotor system to reconstruct the network between CPG and feet force afferent without any commanding parameter. The performances of the simulated and real, cat-like robot on both flat and rough terrains and the leg malfunction tests demonstrated the effectiveness of the proposed model, indicating that a smooth gait-pattern transition could be generated during sudden leg malfunction.

We present a new framework to generate human-like lower-limb trajectories in periodic and non-periodic walking conditions. In our method, walking dynamics is encoded in 3LP, a linear simplified model composed of three pendulums to model falling, swing and torso balancing dynamics. To stabilize the motion, we use an optimal time-projecting controller which suggests new footstep locations. On top of gait generation and stabilization in the simplified space, we introduce a kinematic conversion method that synthesizes more human-like trajectories by combining geometric variables of the 3LP model adaptively. Without any tuning, numerical optimization or off-line data, our walking gaits are scalable with respect to body properties and gait parameters. We can change various parameters such as body mass and height, walking direction, speed, frequency, double support time, torso style, ground clearance and terrain inclination. We can also simulate the effect of constant external dragging forces or momentary perturbations. The proposed framework offers closed-form solutions in all the three stages which enable simulation speeds orders of magnitude faster than real time. This can be used for video games and animations on portable electronic devices with a limited power. It also gives insights for generation of more human-like walking gaits with humanoid robots.

In this paper, we present a novel and practical approach for benchmarking agility. We focus on terrestrial, multilegged locomotion in the field of bio-inspired robotics. We define agility as the ability to perform a set of different but specific tasks executed in a fast and efficient manner. This definition is inspired by the analysis of natural role models, such as dogs and horses as well as robotic systems. An evaluation of existing benchmarks in robotics is done and taken into account in our proposed benchmark. After the general definition, the actual normalized benchmarking values are defined, and measuring methods, as well as an online database for agility score collection and distribution, are presented. To provide a baseline for agile locomotion, various videos of dog-agility competitions were analyzed and agility scores calculated wherever applicable. Finally, validation and implementation of the benchmark are done with different robots directly available to the authors. In conclusion, our benchmark will enable researchers not only to compare existing robots and find out strengths and weaknesses in different design approaches, but also give a tool to define new fitness functions for optimization, learning processes, and future robots developments, intensifying the links between biology and technology even further.

Jamming of granular media has been shown to possess the property of stiffness variation, transitioning from a soft to a quasi-solid state depending on the packing density of the granules. Recently, a gradual stiffness change for bending has been reported by using compliant, cubic shaped granules. Here we demonstrate that the same method and material also exhibits a gradual stiffness change for compression. As a potential application of “compliant jamming”, a bio-inspired robotic platform with jamming membranes as end effectors is designed and tasked with climbing straight vertical shafts. First, the benefit of varying the bending stiffness is investigated in the climbing task by measuring the friction force that the end effectors are applying to the shaft walls, especially if the walls are irregularly shaped. Then the role of compressive stiffness variation is explored by analyzing the performance of the robot in the climbing task, showing multi-modal properties of jamming membranes: (i) enabling a pressure sensor to detect the shaft walls, acting (ii) as grippers that actively use the irregularities of the walls to climb up by state-switching the granular material and (iii) as force dissipators that can dissipate internal forces caused by closed kinematic chains.

One of the main strengths of self-reconfigurable modular robots (SRMR) is their ability to shape-shift and dynamically change their morphology. In the case of our SRMR system “Roombots”, these shapes can be quite arbitrary for a great variety of tasks while the major utility is envisioned to be self-reconfigurable furniture. As such, the ideas and inspirations from users quickly need to be translated into the final Roombots shape. This involves a multitude of separate processes and - most importantly - requires an intuitive user interface. Our current approach led to the development of a tangible user interface (TUI) which involves 3D-scanning of a shape formed by modeling clay and the necessary steps to prepare the digitized model to be formed by Roombots. The system is able to generate a solution in less than two minutes for our target use as demonstrated with various examples.

Autonomous trajectory generation through generalization requires a database of motion, which can be difficult and time consuming to obtain. In this paper, we propose a method for autonomous expansion of a database for the generation of compliant and accurate motion, achieved through the framework of compliant movement primitives (CMPs). These combine task-specific kinematic and corresponding feed-forward dynamic trajectories. The framework allows for generalization and modulation of dynamic behavior. Inspired by human sensorimotor learning abilities, we propose a novel method that can autonomously learn task-specific torque primitives (TPs) associated to given kinematic trajectories, encoded as dynamic movement primitives. The proposed algorithm is completely autonomous, and can be used to rapidly generate and expand the CMP database. Since CMPs are parameterized, statistical generalization can be used to obtain an initial TP estimate of a new CMP. Thereby, the learning rate of new CMPs can be significantly improved. The evaluation of the proposed approach on a Kuka LWR-4 robot performing a peg-in-hole task shows fast TP acquisition and accurate generalization estimates in real-world scenarios.

Model Predictive Control is becoming more and more present in robotic applications. It has been successfully used in control of humanoid robots to adjust positions of the footsteps in order to satisfy stability constraints. In this paper we show how to adapt such scheme for a quadruped robot utilizing a static walking. The MPC is used to provide a center of mass projection reference, keeping it within support polygons to ensure stability. User is given freedom in choosing the desired dynamical behavior of the reference. The proposed control framework is tested in simulation and on a real quadruped robot. An emphasize is put on generality of such approach which is independent of gait parameters.

We discuss challenges a sprawling posture legged robot faces when moving in a complex environment. These kind of robots can be used for search and rescue applications when proper sensors are used and placed correctly. Finding an adequate position for placing sensors on legged robots with a segmented spine to maximize the information retrieval is not trivial. In this paper, we talk about equipping the salamanderlike robot Pleurobot with necessary sensors to understand its challenges. Our setup gathers visual and inertial information for 3D mapping and localization. To do so, we have come up with three different experimental terrains to analyze the sensing behavior. The results are examples to understanding the behavior of quadrupedal legged robots similar to Pleurobot in search and rescue scenarios.

Animal locomotion exhibits all the features of complex non linear systems such as multi-stability, critical fluctuation, limit cycle behavior and chaos. Studying these aspects on real robots has been proved difficult and therefore results mostly rely on the use of computer simulation. Simple control approaches - based on phase oscillators - have been proposed and exhibit several of these features. In this work, we compare two types of controllers: (a) an open loop control approach based on phase oscillators and (b) the Tegotae-based closed loop extension of this controller. The first controller has been shown to exhibit synchronization features between the body and the controller when applied to a quadruped robot with compliant leg structures. In this contribution, we apply both controllers to the locomotion of a stiff quadruped structure. We show that the Tegotae-controller exhibits self-organizing behavior, such as spontaneous gait transition and critical fluctuation. Moreover, it exhibits features such as the ability to stabilize both asymmetric and symmetric morphological changes, despite the lack of compliance in the leg.

Self-reconfigurable modular robots (SRMR) offer high flexibility in task space by adopting different morphologies for different tasks. Using the same simple module, complex and more capable morphologies can be built. However, increasing the number of modules increases the degrees of freedom (DOF) of the system. Thus, controlling the system as a whole becomes harder. Indeed, even a 10 DOFs system is difficult to consider and manipulate. Intuitive and easy to use interfaces are needed, particularly when modular robots need to interact with humans. In this study we present an interface to assemble desired structures and placement of such structures, with a focus on the assembly process. Roombots modules, a particular SRMR design, are used for the demonstration of the proposed interface. Two non-conventional input/output devices - a head mounted display and hand tracking system - are added to the system to enhance the user experience. Finally, a user study was conducted to evaluate the interface. The results show that most users enjoyed their experience. However, they were not necessarily convinced by the gesture control, most likely for technical reasons.

This paper is aimed at developing optimal motion planning for a single autonomous surface vehicle (ASV) equipped with an on-board pollutant sensor that will maximize the sensor-related information available for source seeking. The ASV uses a nonlinear diffusion model of the pollutant source to estimate the intensity/level of the pollution at the present ASV location. The rate of detection of particles depends on the relative distance between the ASV and the source. First, we use a probabilistic map of the source location built through the sensor information for a dynamic motion planning of source seeking based on an entropy reduction formulation, where an appropriately defined Fisher information matrix (FIM) is used for entropy reduction or information gain. We derive the FIM for the set-up and investigate optimal trajectories. Next, we present an online nonlinear Monte Carlo algorithm that uses the obtained sensor information about pollutant at different vehicle locations to update a probabilistic uncertainty map of pollutant source location. As the mission unfolds the ASV motion is computed by considering a moving-horizon interval of decision, which will allow for the inclusion of new information available for optimal motion planning. The proposed motion planning approach is extended to take into account external disturbances and it is able to minimize the uncertainty in the pollutant source. Finally, we provide two case studies to demonstrate efficacy of the proposed motion planning algorithm.

In this paper, we propose a novel architecture to estimate external forces applied to a compliantly controlled balancing robot in simulations. We use similar dynamics equations used in the controller to find mismatches in the available sensory data and associate them to an unknown external force. Then by decomposing Jacobians, we search over the surface of all body links in the robot to find the force application point. By approximating link geometries with ellipsoids, we can derive analytic solutions to solve the search problem very fast in real time. The proposed approach is tested on a complex humanoid robot in simulations where it outperforms static estimators over fast dynamic motions. We foresee a lot of applications for this method especially in human-robot interactions where it can serve as a whole body virtual suit of tactile sensors. It can also be very useful in identifying the inertial properties of objects being manipulated or mounted on the robot like a backpack.

Roombots (RB) are self-reconfigurable modular robots designed to explore physical structure change by robotic reconfiguration and adaptive locomotion on structured grid environments or unstructured environments. The primary goal of RB is to create adaptive furniture. In this study, we propose a novel and user-friendly interface to control position and intensity of a mobile desk light using RB modules. In the proposed method, the user interacts with the RB with only hand/arm gestures. The user's arm is tracked with a single Kinect having bird's eye view. We demonstrate the effectiveness of the proposed interface in real hardware setup and discuss contributions of it.

We present Lynx-robot, a quadruped, modular, compliant machine. It alternately features a directly actuated, single-joint spine design, or an actively supported, passive compliant, multi-joint spine configuration. Both spine configurations bend in the sagittal plane. This study aims at characterizing these two, largely different spine concepts, for a bounding gait of a robot with a three segmented, pantograph leg design. An earlier, similar-sized, bounding, quadruped robot named Bobcat with a two-segment leg design and a directly actuated, single-joint spine design serves as a comparison robot, to study and compare the effect of the leg design on speed, while keeping the spine design fixed. Both proposed spine designs (single rotatory and active and multi-joint compliant) reach moderate, self-stable speeds.

Controllers based on neuromuscular models hold the promise of energy-efficient and human-like walkers. However, most of them rely on optimizations or cumbersome hand-tuning to find controller parameters which, in turn, are usually working for a specific gait or forward speed only. Consequently, designing neuromuscular controllers for a large variety of gaits is usually challenging and highly sensitive. In this contribution, we propose a neuromuscular controller combining reflexes and a central pattern generator able to generate gaits across a large range of speeds, within a single optimization. Applying this controller to the model of COMAN, a 95 cm tall humanoid robot, we were able to get energy-efficient gaits ranging from 0.4 m/s to 0.9 m/s. This covers normal human walking speeds once scaled to the robot height. In the proposed controller, the robot speed could be continuously commanded within this range by changing three high-level parameters as linear functions of the target speed. This allowed large speed transitions with no additional tuning. By combining reflexes and a central pattern generator, this approach can also predict when the next strike will occur and modulate the step length to step over a hole.

In this paper we present a novel whole-body compliance framework. It is based on the Multi Spring Model, a set of virtual springs interconnecting the limb extremities and the trunk as well. More specifically, six virtual springs connect the trunk of the robot to the feet and the hands and other six springs interconnect the limb extremities. By selecting the stiffness values of each individual spring, complex behaviors can be generated, opening the way for more challenging tasks. The framework is implemented in a multi-layer architecture that distributes the computation on the different control units available on the robot. The proposed method was tested in two different tasks using the COmpliant huMANoid (COMAN), a full-body torque-controlled humanoid robot. For the first task, the robot was able to grasp a box, lift it up and firmly hold it. In the second task, the robot balanced itself while having the hands in contact with two walls. COMAN was able to successfully perform both tasks while perturbed by external disturbances and environmental uncertainties.

Although considering dynamics in the control of humanoid robots can improve tracking and compliance in agile tasks, it requires local and global states of the system, precise torque control and proper modeling. In this paper we discuss practical issues to implement inverse dynamics on a torque controlled robot. By modeling electrical actuators offline, inverting such model and estimating the friction on-line, a high bandwidth torque controller is implemented. In addition, a cascade of optimization problems to fuse all the sensory data coming from IMU, joint encoders and contact force sensors estimate the robot's global state robustly. Our estimation builds the kinematic chain of the legs from the center of pressure which is more robust in case of slight slippage, tilting or rolling of the feet. Thanks to precise and fast torque control, robust state estimation and optimization-based whole body inverse dynamics, the real robot can keep balance with very small stiffness and damping in Cartesian space. It can also recover from strong pushes and perform dexterous tasks. The highly compliant and stable performance is based on pure torque control, without any joint damping or position/velocity tracking.

Search and rescue (SAR) missions are being carried out by several types of robots. They include ground, marine and air vehicles depending on the terrain and mission to be tackled. A particular niche for SAR activities are shallow waters. They present high difficultly for conventional ground or marine robots because of the mix of water and ground. Such an environment is difficult to be accessed for a robot without some built-in amphibious capabilities. Our lab has experience in the design of amphibious salamander-like robots. In order to consider whether these robots would be suited for SAR missions in shallow waters, a key requirement is the ability to tackle rough terrains. In this paper we present a control framework for a highly redundant salamander-like robot. It involves bio-inspired spine control, inverse kinematics-based limb control, proper limb-spine coordination, reflex mechanisms and attitude control. The framework is validated in a simulation and on the real robot. In both cases, the robot is used in two different configurations: with and without its tail, in order to investigate how the tail (which is necessary for swimming) affects ground locomotion. With this exploration, we aim to set the precedent for improving the problem of dynamic locomotion of salamander-like robots over unperceived rough terrain. Our results confirm that the design of reflexes like stumbling and extension, combined with an attitude controller, allows for the improving of the performance of the robot in a generic rough terrain which includes stairs, holes and bumps with several levels of complexity adjusted according to the robot dimensions.

Bipedal walking with humanoid robots requires efficient real-time control. Nowadays, most bipedal robots require to ensure local stability at every instant in time, preventing them from achieving the impressive human walking skills. At the same time, bio-inspired walking controllers are emerging, though they are still mostly explored in simulation studies. However, porting these controllers to real hardware is needed to validate their use on real robots, as well as adapting them to face the world non-idealities. Here, we implemented one of them on a real humanoid robot, namely the COMAN, by conducting dynamic walking experiments. More precisely, we used a muscle-reflex model producing efficient and humanlike gaits. Starting from an off-line optimization performed in simulation, we present the controller implementation, focussing on the additional steps required to port it to real hardware. In our experimental results, we highlight some discrepancies between simulation and reality, together with possible controller extensions to fix them. Despite these differences, the real robot still managed to perform dynamic walking. On top of that, its gait exhibited stretched legs and foot roll at some points of the gait, two human walking features hard to achieve with most robot gaits. We present this on a 50 steps walk where the robot was free to move in the sagittal plane while lateral balance was provided by a human operator.

This paper presents the results of a study on the effect of in-series compliance on the locomotion of a simulated 8-DoF Lola-OP™ Modular Snake Robot with added compliant elements. We explore whether there is an optimal stiffness for gait, terrain type, or several gaits and several terrains (i.e. a good “general-purpose” stiffness). Compliance was simulated using ball joints with eight different levels of stiffness. Two snake locomotion gaits (rolling and sidewinding) were tested over flat ground and three different types of rough terrains. We performed grid search and Particle Swarm Optimization to identify the locomotion parameters leading to fast locomotion and analyzed the best candidates in terms of locomotion speed and energy efficiency (cost of transport). Contrary to our expectations, we did not observe a clear trend that would favor the use of compliant elements over rigid structures. For sidewinding, compliant and stiff elements lead to comparable performances. For rolling gait, the general rule seems to be “the stiffer, the better”.

Quadruped locomotion in irregular and unknown terrains is still a problem to solve. The concept of reflexes is used in this work to contribute for the continuous search of answers about this theme. Biological researches show that spinal reflexes are crucial for a successful locomotion in the most varied terrains, so robotics investigation in this area could be a great advance in the robot's locomotion. In this work, we present a sensory driven reflex controller, capable of generating locomotion in a quadruped compliant robot. This controller is totally dependent on sensory information, so the robot's movements are the result of the robot interactions with the environment. Results show that the proposed controller is capable of generating movements in a flat terrain and is resilient to unexpected perturbations such as a small ramp.

In this paper, we propose a novel walking method for torque controlled robots. The method is able to produce a wide range of speeds without requiring off-line optimizations and re-tuning of parameters. We use a quadratic whole-body optimization method running online which generates joint torques, given desired Cartesian accelerations of center of mass and feet. Using a dynamics model of the robot inside this optimizer, we ensure both compliance and tracking, required for fast locomotion. We have designed a foot-step planner that uses a linear inverted pendulum as simplified robot internal model. This planner is formulated as a quadratic convex problem which optimizes future steps of the robot. Fast libraries help us performing these calculations online. With very few parameters to tune and no perception, our method shows notable robustness against strong external pushes, relatively large terrain variations, internal noises, model errors and also delayed communication.

Just as their discrete counterparts, periodic or rhythmic dynamic motion primitives allow easily modulated and robust motion generation, but for periodic tasks. In this paper we present an approach for modulating periodic dynamic movement primitives based on force feedback, allowing for rich motor behavior and skills. We propose and evaluate the combination of feedback and learned feed-forward terms to fully adapt the motions of a robot in order to achieve a desired force interaction with the environment. For the learning we employ the notion of repetitive control, which can effectively minimize the error of behavior towards a given reference. To demonstrate the approach, we show results of simulated and real world experiments on a compliant humanoid robot COMAN. We show the initial results of utilizing the approach to control a pedal-racer, a demanding balance toy best described as a hybrid between a skateboard and a bicycle.

We present a data-driven method for designing parallel compliance. Designing such compliance helps the system to improve energy efficiency, mainly by reducing negative work. The core idea is to design a controller first and then find springs working in parallel with each actuator such that force-displacement graph is lined up around displacement axis. By doing so, we simply shape the natural dynamics for performing the task efficiently. Maximum torque reduction for actuators is a byproduct of this design method. The method can be used in different cyclic robotic application, especially in legged locomotion systems. In this paper, we design a spinal compliance for a bounding quadruped robot in Webots. The results show that the power consumption and the maximum torque are reduced significantly.

Roombots (RB) are self-reconfigurable modular robots designed to study robotic reconfiguration on a structured grid and adaptive locomotion off grid. One of the main goals of this platform is to create adaptive furniture inside living spaces such as homes or offices. To ease the control of RB modules in these environments, we propose a novel and more natural way of interaction with the RB modules on a RB grid, called the Natural Roombots User Interface. In our method, the user commands the RB modules using pointing gestures. The user's body is tracked using multiple Kinects. The user is also given real-time visual feedback of their physical actions and the state of the system via LED illumination electronics installed on both RB modules and the grid. We demonstrate how our interface can be used to efficiently control RB modules on simple point-to-point grid locomotion and conclude by discussing future extensions.

This special issue focuses on elucidating computational neuroscience: an interdisciplinary field of scientific research in which one of the primary goals is to understand how electronic activity in brain cells and networks enables biological intelligence. The objective is to provide a selection of papers that expose and review research efforts in aspects of computational neuroscience that demonstrate its rapidly growing intersection with electrical, electronic and computer engineering, and the prospects for interaction in the near and long-term future.

The framework of dynamic movement primitives (DMPs) contains many favorable properties for the execution of robotic trajectories, such as indirect dependence on time, response to perturbations, and the ability to easily modulate the given trajectories, but the framework in its original form remains constrained to the kinematic aspect of the movement. In this paper, we bridge the gap to dynamic behavior by extending the framework with force/torque feedback. We propose and evaluate a modulation approach that allows interaction with objects and the environment. Through the proposed coupling of originally independent robotic trajectories, the approach also enables the execution of bimanual and tightly coupled cooperative tasks. We apply an iterative learning control algorithm to learn a coupling term, which is applied to the original trajectory in a feed-forward fashion and, thus, modifies the trajectory in accordance to the desired positions or external forces. A stability analysis and results of simulated and real-world experiments using two KUKA LWR arms for bimanual tasks and interaction with the environment are presented. By expanding on the framework of DMPs, we keep all the favorable properties, which is demonstrated with temporal modulation and in a two-agent obstacle avoidance task.

We present a modular controller for quadruped locomotion over unperceived rough terrain. Our approach is based on a computational Central Pattern Generator (CPG) model implemented as coupled nonlinear oscillators. Stumbling correction reflex is implemented as a sensory feedback mechanism affecting the CPG. We augment the outputs of the CPG with virtual model control torques responsible for posture control. The control strategy is validated on a 3D forward dynamics simulated quadruped robot platform of about the size and weight of a cat. To demonstrate the capabilities of the proposed approach, we perform locomotion over unperceived uneven terrain and slopes, as well as situations facing external pushes.

In this paper, a method is proposed for controlling a hopping monopod. It takes dynamics of the robot into account to have better nominal tracking of desired trajectories and more compliant environmental interactions at the same time. We have incorporated also natural dynamics of the robot into the system by using off-line gaits extracted from optimizations on energy. The main control loop consists of the projected inverse dynamics that generates actuator torques given desired trajectories and also a feedback loop designed and tuned specifically for the structure of the robot. A trajectory generator uses known optimal trajectories together with some stabilizing control laws that modify these trajectories to have better robustness in different situations. The average speed of the robot is also regulated by means of a self-organizing controller. We apply soft transitions in trajectories from phase to phase to avoid sharp actuator input profiles. Our method is successfully tested on a monopod hopper robot in simulation. It can handle slightly rough or sloped terrains while maintaining a given average speed. Simulation results suggest that our method is a promising candidate to control a real robot under construction.

We studied the effect of the control of an active spine versus a fixed spine, on a quadruped robot running in bound gait. Active spine supported actuation led to faster locomotion, with less foot sliding on the ground, and a higher stability to go straight forward. However, we did no observe an improvement of cost of transport of the spine-actuated, faster robot system compared to the rigid spine.

We present a general approach to design modular controllers for limit cycle locomotion over unperceived rough terrain. The control strategy uses a Central Pattern Generator (CPG) model implemented as coupled nonlinear oscillators as basis. Stumbling correction and leg extension reflexes are implemented as feedbacks for fast corrections, and model-based posture control mechanisms define feedbacks for continuous corrections. The control strategy is validated on a detailed physics-based simulated model of a compliant quadruped robot, the Oncilla robot. We demonstrate dynamic locomotion with a speed of more than 1.5 BodyLength/s over unperceived uneven terrains, steps, and slopes.

Manipulation and transport of objects using mobile robotic platforms is a well studied field with several successful approaches. The main difficulty while using such platforms is the lack of adaptation capabilities to changes in the environment and the restriction to flat working areas. In this paper, we present a novel manipulation and transport framework using the self-reconfigurable modular robots Roombots to collaboratively carry arbitrarily shaped passive elements in a non-regular 3D environment equipped with passive connectors. A hierarchical planner based on the notion of virtual kinematic chain is used to generate collision-free and hardware-friendly paths as well as sequences of collaborative manipulations. To the best of our knowledge, this is the first example of manipulation of fully passive elements in an arbitrary 3D environment using mobile self-reconfigurable robots. The simulated results show that the planner is robust to arbitrary complex environments with randomly distributed connectors. In addition to simulation results, a proof of concept of the manipulation of one passive element with two real Roombots meta-modules is described.

The framework of dynamic movement primitives allows the generation of discrete and periodic trajectories, which can be modulated in various aspects. We propose and evaluate a novel modulation approach that includes force feedback and thus allows physical interaction with objects and the environment. The proposed approach also enables the coupling of independently executed robotic trajectories, simplifying the execution of bimanual and tightly coupled cooperative tasks. We apply an iterative learning control algorithm to learn a coupling term, which is applied to the original trajectory in a feed-forward fashion. The coupling term modifies the trajectory in accordance to either the desired position or external force. The strengths of the approach are shown in bimanual or two-agent obstacle avoidance tasks, where no higher level cognitive reasoning or planning are required. Results of simulated and real-world experiments on the ARMAR-III humanoid robot in interaction and object lifting tasks, and on two KUKA LWR robots in a bimanual setting are presented.

In this paper we present a framework to learn a model-free feedback controller for locomotion and balance control of a compliant quadruped robot walking on rough terrain. Having designed an open-loop gait encoded in a Central Pattern Generator (CPG), we use a neural network to represent sensory feedback inside the CPG dynamics. This neural network accepts sensory inputs from a gyroscope or a camera, and its weights are learned using Particle Swarm Optimization (unsupervised learning). We show with a simulated compliant quadruped robot that our controller can perform significantly better than the open-loop one on slopes and randomized height maps.

The design of efficient locomotion gaits for robots with many degrees of freedom is challenging and time consuming even if optimization techniques are applied. Control parameters can be found through optimization in two ways: (i) through online optimization where the performance of a robot is measured while trying different control parameters on the actual hardware and (ii) through offline optimization by simulating the robot's behavior with the help of models of the robot and its environment. In this paper, we present a hybrid optimization method that combines the best properties of online and offline optimization to efficiently find locomotion gaits for arbitrary structures. In comparison to pure online optimization, both the number of experiments using robotic hardware as well as the total time required for finding efficient locomotion gaits get highly reduced by running the major part of the optimization process in simulation using a cluster of processors. The presented example shows that even for robots with a low number of degrees of freedom the time required for optimization can be reduced by a factor of 2.5 to 30, at least, depending on how extensive the search for optimized control parameters should be. Time for hardware experiments becomes minimal. More importantly, gaits that can possibly damage the robotic hardware can be filtered before being tried in hardware. Yet in contrast to pure offline optimization, we reach well matched behavior that allows a direct transfer of locomotion gaits from simulation to hardware. This is because through a meta-optimization we adapt not only the locomotion parameters but also the parameters for simulation models of the robot and environment allowing for a good matching of the robot behavior in simulation and hardware. We validate the proposed hybrid optimization method on a structure composed of two Roombots modules with a total number of six degrees of freedom. Roombots are self-reconfigurable modular robots that can form arbitrary structures with many degrees of freedom through an integrated active connection mechanism.

This paper presents the results of a study on the exploitation of compliance in structures made of self-reconfigurable modular robots - Roombots. This research was driven by the following three hypotheses: (1) compliance can improve locomotion performance; (2) different types of compliance will result in diverse locomotion behaviors; (3) control parameters optimized for a medium level of compliance will perform better for other values of compliance than parameters optimized for extremal compliance. Two types of in-series compliant elements were tested, with five different stiffness values for each of them, on a structure made of two Roombots modules. We ran dedicated on-line locomotion parameter optimizations for six different configurations and evaluated their performance for different stiffness values. Hypothesis 1 was confirmed for both types of compliant elements, with a peak of performance for an optimal level of compliance. The variety of locomotion strategies obtained for the different structures confirms hypothesis 2. Hypothesis 3 was only partially confirmed.

Nonlinear dynamical systems have been used in many disciplines to model complex behaviors, including biological motor control, robotics, perception, economics, traffic prediction, and neuroscience. While often the unexpected emergent behavior of nonlinear systems is the focus of investigations, it is of equal importance to create goal-directed behavior (e.g., stable locomotion from a system of coupled oscillators under perceptual guidance). Modeling goal-directed behavior with nonlinear systems is, however, rather difficult due to the parameter sensitivity of these systems, their complex phase transitions in response to subtle parameter changes, and the difficulty of analyzing and predicting their long-term behavior; intuition and time-consuming parameter tuning play a major role. This letter presents and reviews dynamical movement primitives, a line of research for modeling attractor behaviors of autonomous nonlinear dynamical systems with the help of statistical learning techniques. The essence of our approach is to start with a simple dynamical system, such as a set of linear differential equations, and transform those into a weakly nonlinear system with prescribed attractor dynamics by means of a learnable autonomous forcing term. Both point attractors and limit cycle attractors of almost arbitrary complexity can be generated. We explain the design principle of our approach and evaluate its properties in several example applications in motor control and robotics.

In this paper, we present Salamandra robotica II: an amphibious salamander robot that is able to walk and swim. The robot has four legs and an actuated spine that allow it to perform anguilliform swimming in water and walking on the ground. The paper first presents the new robot hardware design, which is an improved version of Salamandra robotica I. We then address several questions related to body-limb coordination in robots and animals that have a sprawling posture like salamanders and lizards, as opposed to the erect posture of mammals (e.g., in cats and dogs). In particular, we investigate how the speed of locomotion and curvature of turning motions depend on various gait parameters such as the body-limb coordination, the type of body undulation (offset, amplitude, and phase lag of body oscillations), and the frequency. Comparisons with animal data are presented, and our results show striking similarities with the gaits observed with real salamanders, in particular concerning the timing of the body's and limbs' movements and the relative speed of locomotion.

Estimation of the temporal derivatives of a noisy position signal is a ubiquitous problem in industrial and robotics engineering. Here, we propose a new approach to get velocity and acceleration estimates of cyclical/periodic signals near to steady-state regime, by using adaptive oscillators. Our method combines the advantages of introducing no delay, and filtering out the high-frequency noise. We expect this method to be useful in control applications requiring undelayed but smooth estimates of velocity and acceleration (e.g., velocity control and inverse dynamics) of quasi-periodic tasks (e.g., active vibration compensation, robot locomotion, and lower-limb movement assistance).

In this paper we present an approach to the problem of stabilizating the gaze of legged robots using Adaptive Frequency Oscillators to learn the frequency, phase and amplitude of the optical flow and generate compensatory commands during robot locomotion. Assuming periodic and nearly sine shaped motion of the head of the robot, the system successfully stabilizes the gaze of the robot, whether the robot itself is moving, or an external object is moving relative to the robot. We present experiments in simulation and, for object tracking, with a real robotics setup, the Hoap 3, showing that the system can be successfully applied to gaze stabilization during locomotion, even when the feedback loop is very slow and noisy.

In the context of underwater robotics, positioning and coordination of mobile agents can prove a challenging problem. To address this issue, we propose the use of electric sensing, with a technique inspired by weakly electric fishes. In particular, the approach relies on one or several of the agents applying an electric field to their environment. Using electric measures, others agents are able to reconstruct their relative position with respect to the emitter, over a range that is function of the geometry of the emitting agent and of the power applied to the environment. Efficacy of the technique is illustrated using a number of numerical examples. The approach is shown to allow coordination of unmanned underwater vehicles, including that of bio-inspired swimming robotic platforms.

Inferring temporal derivatives (like velocity and acceleration) from a noisy position signal is a well-known challenge in control engineering, due to the intrinsic trade-off between noise filtering and estimation bandwidth. To tackle this problem, in this paper we propose a new approach specifically designed for periodic movements. This approach uses an adaptive oscillator as fundamental building block. It is a tool capable of synchronizing to a periodic input while learning its features (frequency, amplitude, ...) in dedicated state variables. Since the oscillator's input and output are perfectly synchronized during steady-state regime, a non-delayed estimate of the input temporal derivatives can be obtained simply by deriving the output analytical form. Pending a (quasi-)periodic input signal, these temporal derivatives are thus synchronized with the actual kinematics, while the signal bandwidth can be arbitrarily tuned by the intrinsic dynamics of the oscillator. We further validate this approach by developing an impedance-based strategy for assisting human walking in the LOPES lower-limb exoskeleton. Preliminary results with a single participant give rise to three main conclusions. First, our method indeed provides velocity and acceleration estimates of the participant's joint kinematics which are smoother and less delayed with respect to the actual kinematics than using a standard Kalman filter. Second, closing the human-robot loop with a high-gain impedance field depending on the acceleration is not possible with a Kalman filter approach, due to unstable dynamics. In contrast, our approach tolerates high gains (up to 70% of the nominal walking torque), showing its intrinsic stability. Finally, no clear benefit of the acceleration-dependent field with respect to a simpler position-dependent field is visible regarding the reduction of metabolic cost. This last result illustrates the challenge of designing sound assistive strategies for complex tasks like walking.

We present the design and evaluation of an iPad application that will be used to operate the modular robots “Roombots”. Roombots are the building blocks for adaptive pieces of furniture. The application allows a user to arrange adaptive furniture within a room. We conducted a user study with 24 participants to evaluate our approach and to freely explore people's interaction. Data suggests that the ability to move with the tablet leads to a better precision of the furniture arrangement. No significant difference has been observed between using the application through a virtual representation of the room in contrast to an augmented reality environment, even if participants mentioned in a post-study questionnaire their preference for the augmented condition. Users described the interface as intuitive and easy to use.

The paper deals with the modeling of a fish-like robot equipped with the electric sense, suited to study sensorimotor loops. The proposed multi-physics model merges a swimming dynamic model of a fish-like robot with an electric model of an embedded electrolocation sensor. Based on a TCP-IP and threaded framework, the resulting simulator works in real time. After presenting the modeling aspects of this work, this article focuses on two numerical studies. In the first, the interactions between body deformations and perception variables are studied and a current correction process is proposed. In the second study, an electric exteroceptive feedback loop based on a direct current measurement method is designed and tested for obstacle avoidance.

In this paper we present our work on integrating a locomotion controller based on central pattern generator (CPG) and a motion planning algorithm using artificial potential fields for a non-holonomic crawling humanoid robot, the iCub. We also integrated a vision tracker and an inverse kinematics solver to perform reaching tasks. We study the influence of the various parameters of the potential field equations on the performance of the system and prove the efficiency of our framework by testing it on a physics-based robotics simulator and partially on the real iCub.

Modular robots offer the possibility to quickly design robots with a high diversity of shapes and functionalities. This nice feature also brings an important challenge: namely how to design efficient locomotion gaits for arbitrary robot structures with many degrees of freedom. In this paper, we present a framework that allows one to explore and identify highly different gaits for a given arbitrary-shaped modular robot. We use simulated robots made of several Roombots modules that have three degrees of freedom each. These modules have the interesting feature that they can produce both oscillatory movements (i.e. periodic movements around a rest position) and rotational movements (i.e. with continuously increasing angle), leading to rich locomotion patterns. Here we ask ourselves which types of movements - purely oscillatory, purely rotational, or a combination of both- lead to the fastest gaits. To address this question we designed a control architecture based on a distributed system of coupled phase oscillators that can produce synchronized rotations and oscillations in many degrees of freedom. We also designed a specific optimization algorithm that can automatically design hybrid controllers, i.e. controllers that use oscillations in some joints and rotations in others. The proposed framework is verified by multiple simulations for several robot morphologies. The results show that (i) the question whether it is better to oscillate or to rotate depends on the morphology of the robot, and that in general it is best to do both, (ii) the optimization framework can successfully generate hybrid controllers that outperform purely oscillatory and purely rotational ones, and (iii) the resulting gaits are fast, innovative, and would have been hard to design by hand.

This paper presents our work towards a decentralized reconfiguration strategy for self-reconfiguring modular robots, assembling furniture-like structures from Roombots (RB) metamodules. We explore how reconfiguration by locomotion from a configuration A to a configuration B can be controlled in a distributed fashion. This is done using Roombots metamodules-two Roombots modules connected serially-that use broadcast signals, lookup tables of their movement space, assumptions about their neighborhood, and connections to a structured surface to collectively build desired structures without the need of a centralized planner.

We aim at merging technologies from information technology, roomware, and robotics in order to design adaptive and intelligent furniture. This paper presents design principles for our modular robots, called Roombots, as future building blocks for furniture that moves and self-reconfigures. The reconfiguration is done using dynamic connection and disconnection of modules and rotations of the degrees of freedom. We are furthermore interested in applying Roombots towards adaptive behaviour, such as online learning of locomotion patterns. To create coordinated and efficient gait patterns, we use a Central Pattern Generator (CPG) approach, which can easily be optimized by any gradient-free optimization algorithm. To provide a hardware framework we present the mechanical design of the Roombots modules and an active connection mechanism based on physical latches. Further we discuss the application of our Roombots modules as pieces of a homogenic or heterogenic mix of building blocks for static structures.

In our previous works we had developed a framework for self-reconfiguration planning based on graph signature and graph edit-distance. The graph signature is a fast isomorphism test between different configurations and the graph edit-distance is a similarity metric. But the algorithm is not suitable for modules with symmetry. In this paper we improve the algorithm in order to deal with symmetric modules. Also, we present a new heuristic function to guide the search strategy by penalizing the solutions with more number of actions. The simulation results show the new algorithm not only deals with symmetric modules successfully but also finds better solutions in a shorter time.

Salamanders propel themselves by proper coordination of limb movements and body undulations. This type of locomotion is interesting for robotics to design robots capable of locomotion on water and land. In this work we identify the control and structural parameters that contribute to forward terrestrial locomotion. We introduce a kinematic model of Salamandra robotica II, a new salamander robot, to explore how the stride length varies with different limb sizes and different types of body oscillations. We also perform systematic tests using a dynamic model built in a physics-based simulator to analyze the locomotion performance in terms of forward speed and power consumption. The results show that it is beneficial to use body undulations with variable curvature along the body, and that the tail can serve as a fifth limb to provide thrust on ground. Experiments using the real robot validate the simulation results and the contribution of the proposed control strategies.

Central pattern generators (CPGs) are becoming a popular model for the control of locomotion of legged robots. Biological CPGs are neural networks responsible for the generation of rhythmic movements, especially locomotion. In robotics, a systematic way of designing such CPGs as artificial neural networks or systems of coupled oscillators with sensory feedback inclusion is still missing. In this contribution, we present a way of designing CPGs with coupled oscillators in which we can independently control the ascending and descending phases of the oscillations (i.e. the swing and stance phases of the limbs). Using insights from dynamical system theory, we construct generic networks of oscillators able to generate several gaits under simple parameter changes. Then we introduce a systematic way of adding sensory feedback from touch sensors in the CPG such that the controller is strongly coupled with the mechanical system it controls. Finally we control three different simulated robots (iCub, Aibo and Ghostdog) using the same controller to show the effectiveness of the approach. Our simulations prove the importance of independent control of swing and stance duration. The strong mutual coupling between the CPG and the robot allows for more robust locomotion, even under non precise parameters and non-flat environment.

This article presents a robust and heavy duty physical latching connection mechanism, which can be actuated with DC motors to actively connect and disconnect modular robot units. The special requirements include a lightweight and simple construction providing an active, strong, hermaphrodite, completely retractable connection mechanism with a 90 degree symmetry1and a no-energy consumption in the locked state. The mechanism volume is kept small to fit multiple copies into a single modular robot unit and to be used on as many faces of the robot unit as possible. This way several different lattice like modular robot structures are possible. The large selection for dock-able connection positions will likely simplify self-reconfiguration strategies. Tests with the implemented mechanism demonstrate its applicative potential for self-reconfiguring modular robots.

This project incorporates modular robots as building blocks for furniture that moves and self-reconfigures. The reconfiguration is done using dynamic connection / disconnection of modules and rotations of the degrees of freedom. This paper introduces a new approach to self-reconfiguration planning for modular robots based on the graph signature and the graph edit-distance. The method has been tested in simulation on two type of modules: YaMoR and M-TRAN. The simulation results shows interesting features of the approach, namely rapidly finding a near-optimal solution.

In this contribution we present a CPG (central pattern generator) controller based on coupled Rossler systems. It is able to generate both limit cycle and chaotic behaviors through bifurcation. We develop an experimental test bench to measure quantitatively the performance of different controllers on unknown terrains of increasing difficulty. First, we show that for flat terrains, open loop limit cycle systems are the most efficient (in terms of speed of locomotion) but that they are quite sensitive to environmental changes. Second, we show that sensory feedback is a crucial addition for unknown terrains. Third, we show that the chaotic controller with sensory feedback outperforms the other controllers in very difficult terrains and actually promotes the emergence of short synchronized movement patterns. All that is done using an unified framework for the generation of limit cycle and chaotic behaviors, where a simple parameter change can switch from one behavior to the other through bifurcation. Such flexibility would allow the automatic adaptation of the robot locomotion strategy to the terrain uncertainty.

The objective of the work presented here is to develop a low cost active knee prosthetic devices as real time embedded system which utilizes the available biological motor control circuit properly integrated with a central pattern generator (CPG) aided control scheme. The approach is completely different from the existing Active Prosthetic devices, designed primarily as stand alone systems utilizing multiple sensors and embedded rigid control schemes. First we analyzed a fuzzy logic based methodology for offering suitable gait for an amputee, followed by formulating a suitable algorithm for designing a CPG, based on Rayleighpsilas oscillator. Using the oscillator we presented a number of simulation results which showed the behavior of knee angles and hip angles and determined the stable limit cycles of the network, and compared them with the captured gaits of an individual. Subsequently, we presented a methodology about how to use CPG outputs for calculating the damping profile for controlling a prosthetic device called AMAL (adaptive modular active leg).

An important problem in the control of locomotion of robots with multiple degrees of freedom (e.g., biomimetic robots) is to adapt the locomotor patterns to the properties of the environment. This article addresses this problem for the locomotion of an amphibious snake robot, and aims at identifying fast swimming and crawling gaits for a variety of environments. Our approach uses a locomotion controller based on the biological concept of central pattern generators (CPGs) together with a gradient-free optimization method, Powell's method. A key aspect of our approach is that the gaits are optimized online, i.e., while moving, rather than as an off-line optimization process. We present various experiments with the real robot and in simulation: swimming, crawling on horizontal ground, and crawling on slopes. For each of these different situations, the optimized gaits are compared with the results of systematic explorations of the parameter space. The main outcomes of the experiments are: 1) optimal gaits are significantly different from one medium to the other; 2) the optimums are usually peaked, i.e., speed rapidly becomes suboptimal when the parameters are moved away from the optimal values; 3) our approach finds optimal gaits in much fewer iterations than the systematic search; and 4) the CPG has no problem dealing with the abrupt parameter changes during the optimization process. The relevance for robotic locomotion control is discussed.

This article presents a control architecture for controlling the locomotion of an amphibious snake/lamprey robot capable of swimming and serpentine locomotion. The control architecture is based on a central pattern generator (CPG) model inspired from the neural circuits controlling locomotion in the lamprey's spinal cord. The CPG model is implemented as a system of coupled nonlinear oscillators on board of the robot. The CPG generates coordinated travelling waves in real time while being interactively modulated by a human-operator. Interesting aspects of the CPG model include (1) that it exhibits limit cycle behavior (i.e. it produces stable rhythmic patterns that are robust against perturbations), (2) that the limit cycle behavior has a closed-form solution which provides explicit control over relevant characteristics such as frequency, amplitude and wavelength of the travelling waves, and (3) that the control parameters of the CPG can be continuously and interactively modulated by a human operator to offer high maneuverability. We demonstrate how the CPG allows one to easily adjust the speed and direction of locomotion both in water and on ground while ensuring that continuous and smooth setpoints are sent to the robot's actuated joints.

Locomotion on an irregular surface is a challenging task in robotics. Among different problems to solve to obtain robust locomotion, visually guided locomotion and accurate foot placement are of crucial importance. Robust controllers able to adapt to sensory-motor feedbacks, in particular to properly place feet on specific locations, are thus needed. Dynamical systems are well suited for this task as any online modification of the parameters leads to a smooth adaptation of the trajectories, allowing a safe integration of sensory-motor feedback. In this contribution, as a first step in the direction of locomotion on irregular surfaces, we present a controller that allows hand placement during crawling in a simulated humanoid robot. The goal of the controller is to superimpose rhythmic movements for crawling with discrete (i.e. short-term) modulations of the hand placements to reach specific marks on the ground.

We present a Bluetooth scatternet protocol (SNP) that provides the user with a serial link to all connected members in a transparent wireless Bluetooth (BT) network. By using only local decision making we can reduce the overhead of our scatternet protocol dramatically. We show how our SNP software layer simplifies a variety of tasks like the synchronization of central pattern generator controllers for actuators, collecting sensory data and building modular robot structures. The whole BT software stack including our new scatternet layer is implemented on a single Bluetooth and memory chip. To verify and characterize the SNP we provide data from experiments using real hardware instead of software simulation. This gives a realistic overview of the scatternet performance showing higher order effects that are difficult to be simulated correctly and guarantees the correct function of the SNP in real world applications.

Nowadays, the understanding of the human cognition and it application to robotic systems forms a great challenge of research. The iCub is a robotic platform that was developed within the RobotCub European project to provide the cognition research community with an open baby- humanoid platform for understanding and development of cognitive systems. In this paper we present the design requirements and mechanical realization of the lower body developed for the "iCub". In particular the leg and the waist mechanisms adopted for lower body to match the size and physical abilities of a 2 frac12 year old human baby are introduced.

We present a system of coupled nonlinear oscillators to be used as programmable central pattern generators, and apply it to control the locomotion of a humanoid robot. Central pattern generators are biological neural networks that can produce coordinated multidimensional rhythmic signals, under the control of simple input signals. They are found both in vertebrate and invertebrate animals for the control of locomotion. In this article, we present a novel system composed of coupled adaptive nonlinear oscillators that can learn arbitrary rhythmic signals in a supervised learning framework. Using adaptive rules implemented as differential equations, parameters such as intrinsic frequencies, amplitudes, and coupling weights are automatically adjusted to replicate a teaching signal. Once the teaching signal is removed, the trajectories remain embedded as the limit cycle of the dynamical system. An interesting aspect of this approach is that the learning is completely embedded into the dynamical system, and does not require external optimization algorithms. We use our system to encapsulate rhythmic trajectories for biped locomotion with a simulated humanoid robot, and demonstrate how it can be used to do online trajectory generation. The system can modulate the speed of locomotion, and even allow the reversal of direction (i.e. walking backwards). The integration of sensory feedback allows the online modulation of trajectories such as to increase the basin of stability of the gaits, and therefore the range of speeds that can be produced

There is much to gain from providing walking machines with passive dynamics, e.g. by including compliant elements in the structure. These elements can offer interesting properties such as self-stabilization, energy efficiency and simplified control. However, there is still no general design strategy for such robots and their controllers. In particular, the calibration of control parameters is often complicated because of the highly nonlinear behavior of the interactions between passive components and the environment. In this article, we propose an approach in which the calibration of a key parameter of a walking controller, namely its intrinsic frequency, is done automatically. The approach uses adaptive frequency oscillators to automatically tune the intrinsic frequency of the oscillators to the resonant frequency of a compliant quadruped robot. The tuning goes beyond simple synchronization and the learned frequency stays in the controller when the robot is put to halt. The controller is model free, robust and simple. Results are presented illustrating how the controller can robustly tune itself to the robot, as well as readapt when the mass of the robot is changed. We also provide an analysis of the convergence of the frequency adaptation for a linearized plant, and show how that analysis is useful for determining which type of sensory feedback must be used for stable convergence. This approach is expected to explain some aspects of developmental processes in biological and artificial adaptive systems that "develop" through the embodied system-environment interactions

We present AmphiBot I, an amphibious snake robot capable of crawling and swimming. Experiments have been carried out to characterize how the speed of locomotion depends on the frequencies, amplitudes, and phase lags of undulatory gaits, both in water and on ground. Using this characterization, we can identify the fastest gaits for a given medium. Results show that the fastest gaits are different from one medium to the other, with larger optimal regions in parameter space for the crawling gaits. Swimming gaits are faster than crawling gaits for the same frequencies. For both media, the fastest locomotion is obtained with total phase lags that are smaller than one. These results are compared with data from fishes and from amphibian snakes.

Presents an approach to movement planning, on-line trajectory modification, and imitation learning by representing movement plans based on a set of nonlinear differential equations with well-defined attractor dynamics. The resultant movement plan remains an autonomous set of nonlinear differential equations that forms a control policy (CP) which is robust to strong external perturbations and that can be modified on-line by additional perceptual variables. We evaluate the system with a humanoid robot simulation and an actual humanoid robot. Experiments are presented for the imitation of three types of movements: reaching movements with one arm, drawing movements of 2-D patterns, and tennis swings. Our results demonstrate (a) that multi-joint human movements can be encoded successfully by the CPs, (b) that a learned movement policy can readily be reused to produce robust trajectories towards different targets, (c) that a policy fitted for one particular target provides a good predictor of human reaching movements towards neighboring targets, and (d) that the parameter space which encodes a policy is suitable for measuring to which extent two trajectories are qualitatively similar.

BIOROB

Many control problems take place in continuous state-action spaces, e.g., as in manipulator robotics, where the control objective is of- ten deflned as flnding a desired trajectory that reaches a particular goal state. While reinforcement learning ofiers a theoretical frame- work to learn such control policies from scratch, its applicability to higher dimensional continuous state-action spaces remains rather limited to date. Instead of learning from scratch, in this paper we suggest to learn a desired complex control policy by transforming an existing simple canonical control policy. For this purpose, we represent canonical policies in terms of difierential equations with well-deflned attractor properties. By nonlinearly transforming the canonical attractor dynamics using techniques from nonparametric regression, almost arbitrary new nonlinear policies can be gener- ated without losing the stability properties of the canonical sys- tem. We demonstrate our techniques in the context of learning a set of movement skills for a humanoid robot from demonstrations of a human teacher. Policies are acquired rapidly, and, due to the properties of well formulated difierential equations, can be re-used and modifled on-line under dynamic changes of the environment. The linear parameterization of nonparametric regression moreover lends itself to recognize and classify previously learned movement skills. Evaluations in simulations and on an actual 30 degree-of- freedom humanoid robot exemplify the feasibility and robustness of our approach.

Explores an approach to learning by imitation and trajectory formation by representing movements as mixtures of nonlinear differential equations with well-defined attractor dynamics. An observed movement is approximated by finding a best fit of the mixture model to its data by a recursive least squares regression technique. In contrast to non-autonomous movement representations like splines, the resultant movement plan remains an autonomous set of nonlinear differential equations that forms a control policy which is robust to strong external perturbations and that can be modified by additional perceptual variables. This movement policy remains the same for a given target, regardless of the initial conditions, and can easily be re-used for new targets. We evaluate the trajectory formation system in the context of a humanoid robot simulation that is part of the Virtual Trainer project, which aims at supervising rehabilitation exercises in stroke-patients. A typical rehabilitation exercise was collected with a Sarcos Sensuit, a device to record joint angular movement from human subjects, and approximated and reproduced with our imitation techniques. Our results demonstrate that multijoint human movements can be encoded successfully, and that this system allows robust modifications of the,movement policy through external variables.

In this article, we present a macroscopic analytical model of collaboration in a group of reactive robots. The model consists of a series of coupled differential equations that describe the dynamics of group behavior. After presenting the general model, we analyze in detail a case study of collaboration, the stick-pulling experiment, studied experimentally and in simulation by Ijspeert et al. [Autonomous Robots, 11, 149-171]. The robots' task is to pull sticks out of their holes, and it can be successfully achieved only through the collaboration of two robots. There is no explicit communication or coordination between the robots. Unlike microscopic simulations (sensor-based or using a probabilistic numerical model), in which computational time scales with the robot group size, the macroscopic model is computationally efficient, because its solutions are independent of robot group size. Analysis reproduces several qualitative conclusions of Ijspeert et al.: namely, the different dynamical regimes for different values of the ratio of robots to sticks, the existence of optimal control parameters that maximize system performance as a function of group size, and the transition from superlinear to sublinear performance as the number of robots is increased.

This paper presents biologically inspired neural controllers for generating motor patterns in a quadruped robot. Sets of artificial neural networks are presented which provide 1) pattern generation and gait control, allowing continuous passage from walking to trotting to galloping, 2) control of sitting and lying down behaviors, and 3) control of scratching. The neural controllers consist of sets of oscillators composed of leaky-integrator neurons, which control pairs of flexor-extensor muscles attached to each joint. The networks receive sensory feedback proportional to the contraction of simulated muscles and to joint flexion. Similarly to what is observed in cats, locomotion can be initiated by either applying tonic (i.e. non-oscillating) input to the locomotion network or by sensory feedback from extending the legs. The networks are implemented in a quadruped robot. It is shown that computation can be carried out in real time and that the networks can generate the above mentioned motor behaviors.

This article describes the design of neural control architectures for locomotion using an evolutionary approach. Inspired by the central pattern generators found in animals, we develop neural controllers that can produce the patterns of oscillations necessary for the swimming of a simulated lamprey. This work is inspired by Ekeberg's neuronal and mechanical model of a lamprey [11] and follows experiments in which swimming controllers were evolved using a simple encoding scheme [25, 26]. Here, controllers are developed using an evolutionary algorithm based on the SGOCE encoding [31, 32] in which a genetic programming approach is used to evolve developmental programs that encode the growing of a dynamical neural network. The developmental programs determine how neurons located on a two-dimensional substrate produce new cells through cellular division and how they form efferent or afferent interconnections. Swimming controllers are generated when the growing networks eventually create connections to the muscles located on both sides of the rectangular substrate. These muscles are part of a two-dimensional mechanical simulation of the body of the lamprey in interaction with water. The motivation of this article is to develop a method for the design of control mechanisms for animal-like locomotion. Such a locomotion is characterized by a large number of actuators, a rhythmic activity, and the fact that efficient motion is only obtained when the actuators are well coordinated. The task of the control mechanism is therefore to transform commands concerning the speed and direction of motion into the signals sent to the multiple actuators. We define a fitness function, based on several simulations of the controller with different commands settings, that rewards the capacity of modulating the speed and the direction of swimming in response to simple, varying input signals. Central pattern generators are thus evolved capable of producing the relatively complex patterns of oscillations necessary for swimming. The best solutions generate traveling waves of neural activity, and propagate, similarly to the swimming of a real lamprey, undulations of the body from head to tail propelling the lamprey forward through water. By simply varying the amplitude of two input signals, the speed and the direction of swimming can be modulated.

This paper presents an experiment in collective robotics which investigates the influence of communication, of learning and of the number of robots in a specific task, namely learning the topography of an environment whose features change frequently. We propose a theoretical framework based on probabilistic modeling to describe the system's dynamics. The adaptive multi-robot system and its dynamic environment are modeled through a set of probabilistic equations which give an explicit description of the influence of the different variables of the system on the data-collecting performance of the group. Further, we implement the multi-robot system in experiments with a group of Khepera robots and in simulation using Webots, a three-dimensional simulator of Khepera robots. The robots are controlled by a distributed architecture with an associative-memory type of learning algorithm. Results show that the algorithm allows a group of robots to keep an up-to-date account of the environmental state when this changes regularly. Finally, the results of the simulated and physical experiments are compared with the predictions of the probabilistic model. It is found that the model shows both a good qualitative and a good quantitative correspondence to these results. This suggests that a probabilistic model can be a good first approximation of a multi-robot system.