General-purpose navigation in challenging environments remains a significant problem in robotics, with current state-of-the-art approaches facing myriad limitations. Classical approaches struggle with cluttered settings and require extensive tuning, while learning-based methods face difficulties generalizing to out-of-distribution environments. This paper introduces X-Mobility, an end-to-end generalizable navigation model that overcomes existing challenges by leveraging three key ideas. First, X-Mobility employs an auto-regressive world modeling architecture with a latent state space to capture world dynamics. Second, a diverse set of multi-head decoders enables the model to learn a rich state representation that correlates strongly with effective navigation skills. Third, by decoupling world modeling from action policy, our architecture can train effectively on a variety of data sources, both with and without expert policies-off-policy data allows the model to learn world dynamics, while on-policy data with supervisory control enables optimal action policy learning. Through extensive experiments, we demonstrate that X-Mobility not only generalizes effectively but also surpasses current state-of-the-art navigation approaches. Additionally, X-Mobility also achieves zero-shot Sim2Real transferability and shows strong potential for crossembodiment generalization. Project page: https://nvlabs.github.io/X-MOBILITY.

This paper presents and experimentally implements three different adaptive and robust adaptive controllers as the first steps toward using model-based controllers for transfemoral prostheses. The goal of this paper is to translate these control methods to the robotic domain, from bipedal robotic walking to prosthesis walking, including a rigorous stability analysis. The human/prosthesis system is first modeled as a two-domain hybrid asymmetric system. An optimization problem is formulated to obtain a stable human-like gait. The proposed controllers are then developed for the combined human/prosthesis model and the optimized reference gait. The stability of all three controllers is proven using the Lyapunov stability theorem, ensuring convergence to the desired gait. The proposed controllers are first verified on a bipedal walking robot as a hybrid human/prosthesis model in simulation. They are then experimentally tested on a treadmill with an able-bodied subject using third iteration of AMBER Prosthetic (AMPRO3), a custom self-contained powered transfemoral prosthesis. Finally, outdoor tests are carried out using AMPRO3 with three test subjects walking on level ground, uphill slopes, and downhill slopes at slope angles of 3° and 8°, to demonstrate walking in different real-world environments.

This work presents the preliminary experimental validation of a systematic prosthetic control strategy on a custom compliant transfemoral prosthesis with the end result being energy efficient 3-dimension (3D) multi-contact prosthetic walking. In particular, with the goal of capturing essential components of realistic amputee-prosthesis system, a 3D asymmetric hybrid system model is presented - this forms the foundation for formal gait design and control construction. Based on this model, a two-step direct collocation optimization method is utilized to design an energy efficient multi-contact prosthetic gait in 3D. The designed gaits are also subject to various practical constraints such as human-likeness constraints and comfortability constraints. For experimental validation, a 3D capable powered transfemoral prosthetic device is custom built so as to be amendable to realizing the designed 3D prosthetic gaits. Differentiating this device from existing powered prosthesis, compliant components are added to the three joints (two pitch joints and one roll joint) for the purpose of energy saving and human-like behaviors. Combining the presented control methodology and the novel hardware design, the end result is experimentally realized 3D multi-contact prosthetic walking with improved energy efficiency compared to other devices and control methods.

Lower-limb prostheses provide a prime example of cyber-physical systems (CPSs) requiring the synergistic development of sensing, algorithms, and controllers. With a view towards better understanding CPSs of this form, this paper presents a systematic methodology using multidomain hybrid system models and optimization-based controllers to achieve human-like multicontact prosthetic walking on a custom-built prosthesis: AMPRO. To achieve this goal, unimpaired human locomotion data is collected and the nominal multicontact human gait is studied. Inspired by previous work which realized multicontact locomotion on the bipedal robot AMBER2, a hybrid system-based optimization problem utilizing the collected reference human gait as reference is utilized to formally design stable multicontact prosthetic gaits that can be implemented on the prosthesis directly. Leveraging control methods that stabilize bipedal walking robots–control Lyapunov function-based quadratic programs coupled with variable impedance control–an online optimization-based controller is formulated to realize the designed gait in both simulation and experimentally on AMPRO. Improved tracking and energy efficiency are seen when this methodology is implemented experimentally. Importantly, the resulting multicontact prosthetic walking captures the essentials of natural human walking both kinematically and kinetically.

This demonstration presents an unimpaired subject walking with a custom built self-contained powered transfemoral prosthesis: AMPRO, which is controlled by a novel nonlinear real-time optimization based controller. To achieve the behaviors that will be demonstrated, controllers that have been successfully implemented on bipedal walking robots are translated to the prosthesis with the goal of achieving natural human-like walking while minimizing power consumption. To achieve this goal, we begin by collecting reference human locomotion data via Inertial measurement Units (IMUs). This data forms the basis for an optimization problem that generates virtual constraints for the prosthesis that provably yields walking in simulation. Utilizing methods that have proven successful in generating stable robotic locomotion, control Lyapunov function (CLF) based Quadratic Programs (QPs) are utilized to optimally track the resulting desired trajectories. The parameterization of the trajectories is determined through a combination of on-board sensing on the prosthesis together with IMU data, thereby coupling the actions of the user with the controller. Finally, impedance control is integrated into the QP yielding an optimization based control law that displays remarkable tracking and robustness, outperforming traditional PD and impedance control strategies.

This paper describes a torque control scheme unifying feedback PD control and feed-forward impedance control to realize human-inspired walking on a novel planar footed bipedal robot: AMBER2. It starts with high fidelity modeling of the robot including nonlinear dynamics, motor model, and impact dynamics. Human data is then used by an optimization algorithm to produce a human-like gait that can be implemented on the robot. To realize the bipedal walking, first a PD controller is utilized to track the optimized trajectory. Next, impedance control parameters are estimated from the experimental data. Finally, the unified PD, impedance torque control law is experimentally realized on the bipedal robot AMBER2. Through the evidence of sustainable and unsupported walking on AMBER2 showing high consistency with the simulated gait, the feasibility of AMBER2 walking scheme will be verified.

This paper presents a novel optimal control strategy combining control Lyapunov function (CLF) based quadratic programs with impedance control, with the goal of improving both tracking performance and the stability of controllers implemented on transfemoral prosthesis. CLF based quadratic programs have the inherent capacity to optimally track a desired trajectory. This property is used in congruence with impedance control - implemented as a feedforward term - to realize significantly small tracking errors, while simultaneously yielding bipedal walking that is both stable and robust to disturbances. Moreover, instead of experimentally validating this on human subjects, a virtual prosthesis is attached to a robotic testbed, AMBER. The authors claim that the walking of AMBER is human like and therefore form a suitable substitute to human subjects on which a prosthetic control can be tested. Based on this idea, the proposed controller was first verified in simulation, then tested on the physical robot AMBER. The results indicate improved tracking performance, stability, and robustness to unknown disturbances.

Impedance control is a common framework for control of lower-limb prosthetic devices. This approach requires choosing many impedance controller parameters. In this paper, we show how to learn these parameters for lower-limb prostheses by observation of unimpaired human walkers. We validate our approach in simulation of a transfemoral amputee, and we demonstrate the performance of the learned parameters in a preliminary experiment with a lower-limb prosthetic device.

This paper presents an approach to the development of bipedal robotic control techniques for multiple locomotion behaviors. Insight into the fundamental behaviors of human locomotion is obtained through the examination of experimental human data for walking on flat ground, upstairs and downstairs. Specifically, it is shown that certain outputs of the human, independent of locomotion terrain, can be characterized by a single function, termed the extended canonical human function. Optimized functions of this form are tracked via feedback linearization in simulations of a planar robotic biped walking on flat ground, upstairs and downstairs - these three modes of locomotion are termed “motion primitives.” A second optimization is presented, which yields controllers that evolve the robot from one motion primitive to another - these modes of locomotion are termed “motion transitions.” A final simulation is given, which shows the controlled evolution of a robotic biped as it transitions through each mode of locomotion over a pyramidal staircase.

This paper presents a method for achieving stable “human-like” running in simulation by using human-inspired control. Data from human running experiments are processed, analyzed and split into the two domains: stance phase and flight phase. By examining this data, we present a set of outputs, i.e., functions of the kinematics, which appear to represent human running; moreover, we show that this output data can be described by the time-solution to a linear spring-mass-damper—termed the canonical locomotion function. This observation motivates the construction of a human-inspired optimization that determines the parameters of the canonical locomotion function that provide the best fit of the human data while simultaneously enforcing partial hybrid zero dynamics, i.e., that the human outputs track the canonical locomotion functions even through impacts. The main result is a method for numerically solving this optimization problem that provably results in stable robotic running. Simulation results are presented that demonstrate the “human-like” robotic running obtained through this procedure.

A method is proposed which enables testing of prosthetic devices in simulation. A hybrid model is used to represent human walking—the combination of continuous and discrete dynamics motivates the use of hybrid systems. A human walking experiment is analyzed and mathematical functions on the kinematics of the collected data are found which capture some of the fundamental behaviors associated with human walking. One model is considered in which these behaviors are fully tracked using feedback linearization; the intent of this is to simulate healthy human walking. Then, a second model is considered: this model is assumed to be a human with a transfemoral prosthesis; PD control is used on the prosthesis. All models considered demonstrate locally exponentially stable periodic orbits when simulated for four separate test subjects, or, in other words, the models exhibit stable walking even with a prosthetic lower extremity. The methods used in this paper are a stepping stone toward a process capable of rapidly prototyping potential prosthesis designs and controllers.