Animal musculoskeletal systems are renowned for their ability to dynamically regulate stiffness and achieve energy-efficient motion. Being inspired by the biological control structure, this study presents a hybrid control framework that utilizes two-stage learning processes for body movement planning and muscle force computation. This methodology simplifies the learning process under joint redundancy and muscle redundancy. Then it enhances the interpretability of the resultant generated behaviors. The framework incorporates a reinforcement learning (RL)-trained joint controller to optimize joint torques, in conjunction with an LSTM-based muscle controller that translates these torques into muscle activations. Two control variants are proposed: One is prioritizing energy efficiency and the other is enhancing adaptability to environmental perturbations through co-contraction control. Validation with MuJoCo physics simulations demonstrates the framework’s capacity to autonomously learn and refine different gait modes without dependence on external motion datasets. The second variant demonstrates superior robustness and energy efficiency compared to conventional motor-driven models. This framework contributes to the enhancement of adaptability in complex scenarios dealing with the redundancy problem of musculoskeletal system coordination and holds potential for the development of bio-inspired locomotion control through the optimization of muscle activity composition.

Humans can generate and sustain a wide range of walking velocities while optimizing their energy efficiency. Understanding the intricate mechanisms governing human walking will contribute to the engineering applications such as energy-efficient biped robots and walking assistive devices. Reflex-based control mechanisms, which generate motor patterns in response to sensory feedback, have shown promise in generating human-like walking in musculoskeletal models. However, the precise regulation of velocity remains a major challenge. This limitation makes it difficult to identify the essential reflex circuits for energy-efficient walking. To explore the reflex control mechanism and gain a better understanding of its energy-efficient maintenance mechanism, we extend the reflex-based control system to enable controlled walking velocities based on target speeds. We developed a novel performance-weighted least squares (PWLS) method to design a parameter modulator that optimizes walking efficiency while maintaining target velocity for the reflex-based bipedal system. We have successfully generated walking gaits from 0.7 to 1.6 m/s in a two-dimensional musculoskeletal model based on an input target velocity in the simulation environment. Our detailed analysis of the parameter modulator in a reflex-based system revealed two key reflex circuits that have a significant impact on energy efficiency. Furthermore, this finding was confirmed to be not influenced by setting parameters, i.e., leg length, sensory time delay, and weight coefficients in the objective cost function. These findings provide a powerful tool for exploring the neural bases of locomotion control while shedding light on the intricate mechanisms underlying human walking and hold significant potential for practical engineering applications.

Central Pattern generators (CPG) are a biologically inspired, decentralized control architecture that enables model-free, but yet adaptively stable and computational lightweight locomotion capabilities on complex robots. Nevertheless, no unified design guidelines for closed-loop CPG controllers are available in the literature. Therefore, we propose a task-distributed, end-to-end trainable, closed-loop CPG control policy by generalizing and extending Tegotae control. The Tegotae approach modulates CPG activity by quantifying the discrepancy between internal belief states and environmental reactions. Spontaneous and adaptive gait formation towards situationally efficient locomotion patterns are intrinsic properties of Tegotae control. The Tegotae control policy is trained and benchmarked in simulation on a 1D hopping robot. We found that our approach can learn efficient and adaptive locomotion on minimal feedback information, while out-performing unstructured, classic reinforcement learning policies of equal complexity. To the best of our knowledge, this is the first study to fully generalize the Tegotae approach and construct unimpeded, end-to-end trainable Tegotae control policies.

Insects exhibit adaptive walking behavior in an unstructured environment, despite having only an extremely small number of neurons (105to 106). This suggests that not only the brain nervous system but also properties of the physical body, such as the morphological characteristics, play an essential role in generating such adaptive behavior. Our study aims at investigating the effect of body morphological characteristics on the walking performance in a robot model, which is designed to mimic an insect. To this end, we constructed an insect-like hexapod model in a simulation environment that implements a reflex-based intra-limb coordination control. Herein, for a set of walking parameters, which were optimized to maximize the energy efficiency at the target speed, we investigated the effects of changes in the standard posture of the two leg joints on the walking success rate for various initial conditions and cost of transport (CoT) as an index of energy efficiency. Simulation results indicated that robots with specific morphological characteristics similar to those of insects exhibited high gait stability and energetic efficiency. Because only the reflex-based control was employed, the inter-leg coordination occurred spontaneously, suggesting that our approach would lead to a useful design methodology from the perspective of computational cost in generating the walking locomotion.

The state-of-the-art reinforcement learning (RL) techniques have made innumerable advancements in robot control, especially in combination with deep neural networks (DNNs), known as deep reinforcement learning (DRL). In this article, instead of reviewing the theoretical studies on RL, which were almost fully completed several decades ago, we summarize some state-of-the-art techniques added to commonly used RL frameworks for robot control. We mainly review bioinspired robots (BIRs) because they can learn to locomote or produce natural behaviors similar to animals and humans. With the ultimate goal of practical applications in real world, we further narrow our review scope to techniques that could aid in sim-to-real transfer. We categorized these techniques into four groups: 1) use of accurate simulators; 2) use of kinematic and dynamic models; 3) use of hierarchical and distributed controllers; and 4) use of demonstrations. The purposes of these four groups of techniques are to supply general and accurate environments for RL training, improve sampling efficiency, divide and conquer complex motion tasks and redundant robot structures, and acquire natural skills. We found that, by synthetically using these techniques, it is possible to deploy RL on physical BIRs in actuality.

Versatile gait patterns are observed in quadrupeds according to the locomotion speed, environmental conditions, and animal species. These gait patterns are generated via inter- and intra-limb coordination mechanisms, both of which are controlled in part by an intraspinal neural network called the central pattern generator (CPG). Previous CPG-based models mainly focused on the inter-limb coordination mechanisms and not on the intra-limb coordination mechanisms, although both of them should play a pivotal role in generating various gait patterns. In this study, we present an autonomous decentralized control scheme for quadruped locomotion wherein inter- and intra-limb coordination mechanisms are well coupled. Simulation results show that the quadruped exhibits transitioning between walking and running and the ability to adapt to changes in body properties by appropriately modifying the phase relationship among body points through well-balanced coupling of the inter- and intra-limb coordination mechanisms. We also present a physical robot that we are currently developing.

In this paper, we investigate an adaptive bipedal walking control that exploits sensory information stemming from “soft deformable” feet. To this end, we developed a bipedal robot with soft deformable feet and proposed an unconventional CPG (central pattern generator)-based control that exploits the local force feedback generated from such deformation. Through experiments with the constructed robot, we have found that the robot exhibits a remarkably adaptive walking ability in response to a change in walking velocity and external perturbations. These results support the conclusion that the “deformation” of a robot's body plays a pivotal role in the emergence of “sensory-motor coordination”, which is the key for generating adaptive locomotion in a robotic system.

Versatile gait patterns that depend on the locomo- tion speed, environmental conditions, and animal species are observed in quadrupeds. Locomotor patterns are generated via the interlimb coordination, which is partially controlled by an intraspinal neural network called the “central pattern generator” (CPG). However, there is currently no clear understanding of the adaptive interlimb coordination mechanism. We hypothesize that the interlimb coordination should rely more on the “physical” interaction between leg movements through the body rather than the interlimb neural connection. To understand the coordination mechanism, we developed a simple-structured quadruped robot and proposed an unconventional CPG model that consists of four decoupled oscillators with only local force feedback in each leg. Experimental results show that our CPG model allows the robot to exhibit steady gait patterns, adaptability to changes in body properties, and adaptive gait transition between walking and trotting. Our robot mimics locomotor patterns of real quadrupeds following which it can capture the basic mechanism underlying the adaptive interlimb coordination.

This is the first study of a real physical kneed bipedal robot that exhibits passive-dynamic running (PDR), i.e., a bipedal gait with a flight phase in a device without an actuator. By carefully designing the properties of the elastic elements implemented into the hip joints and the stance legs in this device, we achieved a stable PDR consisting of 36 steps. The main contribution of this paper is the demonstration of PDR in the real world, which fully exploits the elastic mechanical properties.

This is the first study of a real physical kneed bipedal robot that exhibits passive dynamic running (PDR). Passive dynamic walking (PDW), which has its roots in the pioneering research of McGeer, intrinsically offers not only nonlinear phenomena such as the pull-in effect and period-doubling bifurcation, but also offers an extremely interesting phenomenon that facilitates the engineering of a highly efficient walking robot. In recent years, a wide variety of verification experiments in PDW were performed using actual devices. In contrast, however, very few studies addressed PDR. In the present study, we developed a two-dimensional real physical passive dynamic running biped with knees. The device stands 400 mm tall and weights 4.8 kg. By carefully designing the properties of the elastic elements implemented into the hip joints and the stance legs in the present device, we achieved stable passive dynamic running of 36 steps. The device runs at about 0.83 m/s down a 0.22 rad slope. To the best of our knowledge, this is a first report of such a performance. This result is expected to prove useful not only for designing human-like natural and efficient bipedal robots, but also for understanding the principles underlying bipedal locomotion.

Despite its appeal, a systematic design of an autonomous decentralized control system is yet to be realized. To bridge this gap, we have so far employed a “back-to-basics” approach inspired by true slime mold, a primitive living creature whose behavior is purely controlled by coupled biochemical oscillators similar to central pattern generators (CPGs). Based on this natural phenomenon, we have successfully developed a design scheme for local sensory feedback control leading to system-wide adaptive behavior. This design scheme is based on a “discrepancy function” that extracts the discrepancies among the mechanical system (i:e:, body), control system (i:e:, brain-nervous system) and the environment. The aim of this study is to intensively investigate the validity of this design scheme by applying it to the control of a quadruped locomotion. Simulation results show that the quadruped robot exhibits remarkably adaptive behavior in response to environmental changes and changes in body properties. Our results shed a new light on design methodologies for CPG-based decentralized control of various types of locomotion.

In this paper, we propose an idea which can solve the complexity of the overlapping situation observed in control system of living things. We introduce an another element between controlled object and control law. This newly introduced element is named as Implicit Control Law and decided by interaction of the controlled object, the control law and the field. Furthermore, the Implicit Control Law does not only solve the indivisibility problem but also produces a start point for understanding of realtime environmental adaptation function of living thing with tiny brain. That is, the Implicit Control Law is a core principle of Mobiligence.

In this study, we discuss the common stabilization mechanism underlying passive dynamic walking (PDW) and passive dynamic running (PDR), focusing on the feedback structures in analytical Poincare¿ maps. To this end, we have derived linearized analytical Poincare¿ maps for PDW and PDR, and analyzed these stabilities on two models, namely models with elastic elements and with stiff legs. Through our theoretical analysis, we have found an ¿implicit two-delay feedback structure¿, which can be seen as a certain type of two-delay input digital feedback control developed as an artificial control structure in the field of control theory, is an inherent stabilization mechanism in PDR appearing from the model with elastic elements, and two-period and four-period PDW appearing from with stiff legs. This mechanism is the key to adaptive function underlying phase transition phenomenon between PDW and PDR and period-doubling bifurcation phenomenon in PDW. To the best of our knowledge, this has not yet to be addressed and studied so far. Our results shed new light on the common underlying principle of passive dynamic locomotion, including biped PDW and PDR.

The control and mechanical systems of an embodied agent should be tightly coupled so as to emerge useful functionalities such as adaptivity. This indicates that the mechanical system as well as the control system should be responsible for a certain amount of “computation” for generating the behavior. However, there still leaves much to be understood about to what extent “computational offloading” from the control system to the mechanical system should be achieved. In order to effectively consider this, here we particularly focus on a passive dynamic running biped whose behavior is generated purely from its mechanical system, and investigate how the body’s properties influence the resulting behavior. Through the numerical simulations, we have found that two elastic parameters of its body, leg spring constant and hip coil spring constant, play a crucial role, and depending on which various kinds of stable gait patterns are generated. To the best of our knowledge, this has not been explicitly addressed so far. The results obtained are expected to shed a new light on to what extent the mechanical system should be responsible for generating the behavior.

Recently, it has been widely recognized that control and mechanical systems cannot be designed separately due to their tight interdependency. However, there still leaves much to be understood about how well-balanced coupling between control and mechanical systems can be achieved. Therefore, as an initial step toward this goal, this study intensively discusses the effect of the intrinsic dynamics of a robot's body on the resulting behavior, in the hope that mechanical systems appropriately designed will allow us to significantly reduce the complexity of control algorithm required. More precisely, we focus on the property of leg elasticity of a passive dynamic running biped, and investigate how this influences the stability of running. As a result, we have found that a certain type of nonlinearity in the leg elasticity plays a crucial role to enhance the stability of passive dynamic running. To the best of our knowledge, this has never been explicitly considered so far