Data-driven methods based on neural networks are becoming more widespread for predicting human lower-limb motion. Until now, however, actual examples have focused on only a handful, steady locomotion behaviors. Here we explore if neural network predictors can simultaneously cover many more behaviors including transient ones. Training four common types of predictor networks on a large data set of human gait, we find that they all accommodate these behaviors similarly well, maintaining prediction errors of a few centimeters (lower-limb joint positions) and degrees (joint angles) when tested on data of previously seen subjects. We further observe that although the prediction quality drops for data of unseen subjects, overall, the predicted and actual lower-limb motions remain well aligned. While the predictors demonstrated here cover the largest range of locomotion behaviors reported to date, we achieve this improvement not by better network design but simply by training on more data. This outcome clearly supports the notion that the fastest route to obtain truly general network predictors of lower-limb motion is by focusing time and effort on the rapid growth and sharing of data sets of locomotion behaviors encountered in daily life.

Avoiding obstacles poses a significant challenge for amputees using mechanically-passive transfemoral prosthetic limbs due to their lack of direct knee control. In contrast, powered prostheses can potentially improve obstacle avoidance via their ability to add energy to the system. In past work, researchers have proposed stumble recovery systems for powered prosthetic limbs that provide assistance in the event of a trip. However, these systems only aid recovery after an obstacle has disrupted the user's gait and do not proactively help the amputee avoid obstacles. To address this problem, we designed an adaptive system that learns online to use kinematic data from the prosthetic limb to detect the user's obstacle avoidance intent in early swing. When the system detects an obstacle, it alters the planned swing trajectory to help avoid trips. Additionally, the system uses a regression model to predict the required knee flexion angle for the trip response. We validated the system by comparing obstacle avoidance success rates with and without the obstacle avoidance system. For a non-amputee subject wearing the prosthesis through an adapter, the trip avoidance system improved the obstacle negotiation success rate from 37% to 89%, while an amputee subject improved his success rate from 35% to 71% when compared to utilizing minimum jerk trajectories for the knee and ankle joints.

Learning controllers for bipedal robots is a challenging problem, often requiring expert knowledge and extensive tuning of parameters that vary in different situations. Recently, deep reinforcement learning has shown promise at automatically learning controllers for complex systems in simulation. This has been followed by a push towards learning controllers that can be transferred between simulation and hardware, primarily with the use of domain randomization. However, domain randomization can make the problem of finding stable controllers even more challenging, especially for under actuated bipedal robots. In this work, we explore whether policies learned in simulation can be transferred to hardware with the use of high-fidelity simulators and structured controllers. We learn a neural network policy which is a part of a more structured controller. While the neural network is learned in simulation, the rest of the controller stays fixed, and can be tuned by the expert as needed. We show that using this approach can greatly speed up the rate of learning in simulation, as well as enable transfer of policies between simulation and hardware. We present our results on an ATRIAS robot and explore the effect of action spaces and cost functions on the rate of transfer between simulation and hardware. Our results show that structured policies can indeed be learned in simulation and implemented on hardware successfully. This has several advantages, as the structure preserves the intuitive nature of the policy, and the neural network improves the performance of the hand-designed policy. In this way, we propose a way of using neural networks to improve expert designed controllers, while maintaining ease of understanding.

Robotics controllers often consist of expert-designed heuristics, which can be hard to tune in higher dimensions. Simulation can aid in optimizing these controllers if parameters learned in simulation transfer to hardware. Unfortunately, this is often not the case in legged locomotion, necessitating learning directly on hardware. This motivates using data-efficient learning techniques like Bayesian Optimization (BO) to minimize collecting expensive data samples. BO is a black-box data-efficient optimization scheme, though its performance typically degrades in higher dimensions. We aim to overcome this problem by incorporating domain knowledge, with a focus on bipedal locomotion. In our previous work, we proposed a feature transformation that projected a 16-dimensional locomotion controller to a 1-dimensional space using knowledge of human walking. When optimizing a human-inspired neuromuscular controller in simulation, this feature transformation enhanced sample efficiency of BO over traditional BO with a Squared Exponential kernel. In this paper, we present a generalized feature transform applicable to non-humanoid robot morphologies and evaluate it on the ATRIAS bipedal robot, in both simulation and hardware. We present three different walking controllers and two are evaluated on the real robot. Our results show that this feature transform captures important aspects of walking and accelerates learning on hardware and simulation, as compared to traditional BO.

We propose a method for optimizing control policies for assistive lower-limb devices. The method frames parameter selection as a dueling bandits problem in which a user indicates his or her qualitative preferences between pairs of parameter sets chosen from a library. We generate the library through an offline optimization procedure that seeks to reproduce the varied gaits of healthy human subjects. By separating the parameter selection process into online and offline portions, the method can handle high-dimensional parameter spaces and produces policies that can generalize to different gait scenarios such as speed variation. We evaluate the method on five subjects walking on a powered knee and ankle prosthesis governed by a neuromuscular control policy that has 43 parameters. We find the five subjects preferred four different parameter sets from the library and that the resulting optima resemble intact subject gait data. This result suggests the offline portion of the optimization method indeed produces control parameters that can adapt to different gaits. Moreover, we find that for three out of the four parameter sets we tested, the procedure also generates parameters that improve the ability of the prosthesis to adapt to increasing gait speed by increasing ankle net work production. The results encourage further research and exploration in clinical settings toward advanced prosthesis controls that employ online learning.

The analysis of the conceptual spring mass model for running reveals swing-leg placement policies that generate very robust locomotion in unobserved terrain with large changes in ground height. However, while this theoretical result suggests a potential for large improvements on the robustness of running machines, it has so far not been demonstrated on a physical robot. Here we address this implementation and verification step for a human-sized bipedal robot platform confined to a boom. We detail challenges and solutions for the implementation of the control approach and show that it leads to very robust running (ground changes ±20% of leg length) over unobserved ground in a high fidelity simulation of the robot platform. We also present initial tests on the actual robot hardware, which indicate the feasibility of the approach. If it can be generalized to 3D running, it could trigger running machines with largely improved robustness.

In this paper we propose the fastest converging control policy (also known as deadbeat control) for walking with the bipedal spring-mass model, which serves as an abstraction of a robot on compliant legs. To fully leverage the passive dynamics of the system, the touchdown angle of the swing-leg is assigned as the only control input of the system. We show that two steps (or one stride) are necessary and sufficient to converge to target walking gaits. We first analyze the dynamics of the system to identify the limit cycles as well as the limitations of the control authority within the definition of walking. Then, we present the two-step deadbeat control policy that guarantees stability with the fastest possible convergence rate for the system. For each equilibrium gait, the basin of attraction in which this two-step control exists is a measure of the robustness of the system. The simulation results show that human-like walking gaits (double hump ground reaction force profile) have relatively large basins of attraction. Finally, we extend the policy to various energy levels to accommodate walking on uneven ground that has height changes. We show in simulation that the system indeed rejects various disturbances and converges to the desired equilibrium gait in two steps.

Series elastic actuators primarily use linear springs in their drivetrains, which introduces a design tradeoff: soft springs provide higher torque resolution at the cost of system bandwidth, whereas stiff springs provide a fast response but lower torque resolution. Nonlinear springs (NLSs) potentially incorporate the benefits of both soft and stiff springs, but such springs are often large. An NLS design was recently proposed that combines a variable radius cam with a rubber elastic element, enabling a compact spring design. However, the rubber introduces hysteresis, which can lead to poor torque tracking if not accounted for in the controller. To overcome this limitation, we here propose a state observer that captures hysteretic effects exhibited by the rubber to provide an accurate estimate of actuator torque. We perform torque-control experiments with this observer on an actuator testbed and compare the performance of the NLS to both soft and stiff linear metal springs. Experiments show that the NLS exhibits improved output impedance compared to both linear springs, and comparable bandwidth to the stiff linear spring up to 1.5 Hz. However, the hysteresis in the urethane rubber introduces instability in higher-frequency conditions, suggesting that future NLS designs can be improved by use of a different rubber as the elastic element.

Walking controllers for bipedal robots have not yet reached human levels of robustness in locomotion. Imitating the human motor control might be an alternative strategy for generating robust locomotion in robots. We seek to control bipedal robots with a specific neuromuscular human walking model proposed previously. Here, we present a virtual neuromuscular controller, VNMC, that emulates this neuromuscular model to generate desired motor torques for a bipedal robot. We test the VNMC on a high-fidelity simulation of the ATRIAS bipedal robot constrained to the sagittal plane. We optimize the control parameters to tolerate maximum ground-height changes, which resulted in ATRIAS walking on a terrain with up to ±7 cm height changes. We further evaluate the robustness of the optimized controller to external and internal disturbances. The optimized VNMC adapts to 90% of random terrains with ground-height changes up to ±2 cm. It endures 95% of ±30 Ns horizontal pushes on the trunk, and 90% of 8 Ns backward and 4 Ns forward impulses on the swing foot throughout the gait cycle. Furthermore, the VNMC is resilient to modeling errors and sensor noise much larger than the equivalent uncertainties in the real robot. The results suggest VNMC as a potential alternative to generate robust locomotion in bipedal robots.

Animals and robots balance dynamically by placing their feet into proper ground targets. While foot placement controls exist for both fully robotic systems and powered prostheses, none enable the dynamism and reactiveness of able-bodied humans. A control approach was recently developed for an ideal double pendulum dynamical system that places feet into ground targets for a wide range of initial conditions and in the presence of significant locomotion disturbances. While its performance in simulation make it an attractive candidate to control legged robotic systems, it is unclear if the approach can be used on real-world systems. In this paper we transfer and evaluate the approach on robotic hardware. Our results show that the controller can be transferred to robotic systems and allow them to achieve comparable foot placement accuracies to an ideal double pendulum simulation, both when motion is undisturbed, as well as when obstacles are encountered in early, mid, and late swing. These results suggest that the proposed approach is a potential alternative control method for legged robots and powered prostheses, enabling recovery from sudden swing-leg disturbances such as trips and unexpected obstacle encounters. The results also point out the need for additional considerations when tuning the controller in order to generate human-like swing trajectories and durations.

Series elastic actuators often use linear metal springs in their drivetrains, which requires design compromises between torque resolution and actuation bandwidth. Nonlinear springs (NLSs), with variable stiffness, overcome this limitation, enabling both high torque resolution and high bandwidth. Current NLS designs combine variable cam structures with off-the-shelf linear springs, which increases the overall size of these torque transmitting elements. NLS size could be reduced by using other materials as an elastic element. We present an optimization-based synthesis method for NLSs that are compact and encode a user-defined torque-deflection profile using elastic elements with an arbitrary stiffness profile. We experimentally validate the proposed method by creating a NLS prototype and testing it on an actuator testbed. The prototype uses rubber as the elastic element, resulting in a compact design that generates the desired torque profile, although hysteresis of the rubber material partially compromises performance. The results suggest that the proposed method successfully generates compact NLS designs, but that rubber elements need to be carefully chosen to mitigate hysteresis.

Transfemoral amputees often suffer from falls and a fear of falling that leads to a decreased quality of life. Existing control strategies for powered knee-ankle prostheses demonstrate only limited ability to react to disturbances that induce falls such as trips, slips, and obstacles. In contrast, prior work on neuromuscular modeling of human locomotion suggests that control strategies based on local reflexes exhibit robustness to unobserved terrain such as slopes and steps. Therefore, we propose that a powered knee-ankle prosthesis governed by reflexive local controls will more competently adapt to unperceived disturbances. To test this hypothesis, we simulate a neuromuscular model of a transfemoral amputee walking over rough ground with a powered knee-ankle prosthesis governed by the proposed reflexive controller. We show that the proposed control allows the amputee to walk farther over rough ground than does the state-of-the-art control. The proposed controller also more readily rejects deviations from nominal walking gaits such as those encountered during a trip. These results suggest that applying the proposed control to a powered knee-ankle prosthesis will substantially improve amputee gait stability.

Control design of running robots is often based on mapping the behavior of lower order models onto the robotic systems, and the robustness of running is largely determined by the robustness of these underlying models. However, existing implementations do not take full advantage of the stability that the low order models can provide. In particular, analysis of the theoretical spring mass model suggests leg placement policies that generate near deadbeat rejection of large, unobserved changes in ground height. Here we show in simulation that this blind robustness to rough terrain can be carried over to bipedal robots. We design a control that stably embeds the spring mass model's behavior in a planar robot model and show that resulting system rejects ground disturbances of up to 25% leg length, adapts to persistent ground slopes, and tolerates sensor noise, signal delays, and modeling errors. The results indicate that transferring control derived within the spring mass model is an effective technique for realizing highly robust running in robotic systems.

Swing leg placement is vital to dynamic stability in legged robots and animals. The most common approaches to generating swing leg motions in robotics use either position or impedance tracking of defined joint trajectories. While these approaches suffice in humanoids, they severely limit swing leg placement under large disturbances in prosthetic limbs, for which stabilizing reactions cannot be planned centrally. We previously identified a control for a conceptual double-pendulum swing leg that does not require predefined trajectories to robustly place the leg into target points on the ground in the presence of large disturbances. We here develop a more detailed neuromuscular model of the human swing leg and interpret the identified control with local muscle reflexes. The resulting reflex control robustly places the swing leg into a wide range of landing points observed in human walking and running, and it generates similar patterns of joint torques and muscle activations. While we plan to take advantage of the comparison between the neuromuscular model and humans to further improve robust swing leg placement, the current results suggest an alternative to existing swing leg controls in humanoid and rehabilitation robotics which does not require central processing.

Over the past three decades, the spring-mass model has developed into the basic behavior model to study running in animals and robots. In the planar version, this model has helped to reveal and understand the passive stabilization of running in the horizontal and sagittal planes, and to derive from this knowledge control strategies for running robots. However, only few attempts have been made to transfer the knowledge to 3-D locomotion. Here, we show that the 3-D spring-mass model reveals a deadbeat control that does not require feedback about the actual ground level to produce highly robust running and steering in uncertain environments. The control naturally extends the time-based control derived for the planar version of this model and allows it to navigate rough terrain, while stabilizing running and steering. Using this control strategy, we demonstrate in simulation that a human-like system running at 5 ms-1tolerates frequent ground disturbances up to 30% of the leg length. Moreover, we find that the control outperforms a classical leg-placement strategy in terms of turning rate and disturbance rejection if the relative errors in system energy and the other model parameters stay small ( 10%). Our results suggest that the time-based control can be a powerful alternative for leg-placement strategies in highly maneuverable running robots.

Although current humanoid controllers can rely on inverse kinematics or dynamics of the full humanoid system, powered prosthetic legs or assistive devices cannot, because they do not have access to the full states of the human system. This limitation creates the need for alternative control strategies. One strategy is to embed fundamental knowledge about legged dynamics and control in local feedback. In a previous paper, we have developed a control model of human locomotion which relies mostly on local feedback. The model can robustly walk at normal walking speeds. Here we extend this model to adapt to a wide range of walking speeds and to generate corresponding speed transitions. We use optimization of the model's control parameters and find key parameters responsible for steady walking between 0.8ms-1and 1.8ms-1, covering the range of speed at which humans normally walk. Using these parameters, we demonstrate speed transitions between slow and fast walking. In addition, we discuss how the speed-dependent changes of the identified control parameters connect to biped walking dynamics, and suggest how these changes can be integrated in local feedback control.

Dynamic balance depends on proper foot placement in legged locomotion and corresponding placement strategies have mainly been developed using the linear inverted pendulum model as theoretical framework. While this model can identify single leg strategies for balance control, it does not consider the double support that is common to bipedal locomotion, indicating that current strategies do not fully exploit the theoretical potential of balance control in bipedal systems. Here we extend the linear inverted pendulum model to a bipedal system which includes double support dynamics, and derive a reactive balance controller based on foot placement and double stance length. We show that this controller enables the model to stand and walk at user-defined target speeds, to transition between these behaviors by acceleration and deceleration, and to react to intermittent disturbances and compensate for permanent ones, as long as they are compatible with the swing leg dynamics placing the feet. Finally, we discuss how the versatility of this balance controller depends on including double support and suggest further steps to improve dynamic balance control in bipedal systems.