We describe a mobile manipulation hardware and software system capable of autonomously performing complex human-level tasks in real homes, after being taught the task with a single demonstration from a person in virtual reality. This is enabled by a highly capable mobile manipulation robot, whole-body task space hybrid position/force control, teaching of parameterized primitives linked to a robust learned dense visual embeddings representation of the scene, and a task graph of the taught behaviors. We demonstrate the robustness of the approach by presenting results for performing a variety of tasks, under different environmental conditions, in multiple real homes. Our approach achieves 85% overall success rate on three tasks that consist of an average of 45 behaviors each. The video is available at: https://youtu.be/HSyAGMGikLk.

A wide variety of strategies have been developed for assisting human locomotion using powered exoskeletons. Although these strategies differ in their aims as well as the control methods employed, they have the implicit property of causing a virtual modification of the dynamic response of the human limb. We use this property of the exoskeletons action to formulate a unified control design framework called Integral Admittance Shaping, which designs exoskeleton controllers capable of producing the desired dynamic response for the assisted limb. In this framework, a virtual increase in the admittance of the limb is produced by coupling it to an exoskeleton that exhibits active behavior. Specifically, our framework shapes the magnitude profile of the integral admittance (i.e. torque-to-angle relationship) of the coupled human-exoskeleton system, such that the desired assistance is achieved. This framework also ensures that the coupled stability and passivity are guaranteed. This paper presents a formulation of Integral Admittance Shaping for single degree-of-freedom (1-DOF) exoskeleton devices. We also present experimental results on a modified version of Honda's Stride Management Assist (SMA) device that successfully demonstrate motion amplification of the assisted hip joint during walking.

This paper presents a novel universal balancing controller that successfully stabilizes a planar bipedal robot in dynamic, unstable environments like seesaw and bongoboards, and also in static environments like curved and flat floors. These different dynamic systems have state spaces with different dimensions, and hence instead of using full state feedback, the universal controller is derived as a single output feedback controller that stabilizes them. This paper analyzes the robustness of the derived universal controller to disturbances and parameter uncertainties, and demonstrates its universality and superiority to similarly derived LQR and H∞controllers. This paper also presents nonlinear simulation results of the universal controller successfully stabilizing a family of bongoboard, curved floor, seesaw, tilting and rocking floor models.

This paper studies the balancing of simple planar bipedal robot models in dynamic, unstable environments such as seesaw, bongoboard, and board on a curved floor. This paper derives output feedback controllers that successfully stabilize seesaw, bongoboard, and curved floor models using only global robot information and with no direct feedback of the dynamic environment and, hence, demonstrates that direct feedback of environment information is not essential for successfully stabilizing the models considered in this paper. This paper presents an optimization to derive stabilizing output feedback controllers that are robust to disturbances on the board. It analyzes the robustness of the derived output feedback controllers to disturbances and parameter uncertainties and compares their performance with similarly derived robust linear quadratic regulator controllers. This paper also presents nonlinear simulation results of the output feedback controllers' successful stabilization of bongoboard, seesaw, and curved floor models.

Simple inverted pendulum models and their variants are often used to control humanoid robots in order to simplify the control design process. These simple models have significantly fewer degrees of freedom than the full robot model. The design and choice of these simple models are based on the designer's intuition, and the reduced state mapping and the control input mapping are manually chosen. This paper presents an automatic model reduction procedure for humanoid robots, which is task-specific. It also presents an optimization framework that uses the auto-generated task-specific reduced models to control humanoid robots. Successful simulation results of balancing, fast arm swing, and hip rock and roll motion tasks are demonstrated.

The ballbot with arms is an underactuated balancing mobile robot that moves on a single ball. Achieving desired motions in position space is a challenging task for such systems due to their unstable zero dynamics. This paper presents a novel approach that uses the dynamic constraint equations to plan shape trajectories, which when tracked will result in optimal tracking of desired position trajectories. The ballbot with arms has shape space of higher dimension than its position space and therefore, the procedure uses a user-defined weight matrix to choose between the infinite number of possible combinations of shape trajectories to achieve a particular desired trajectory in position space. Experimental results are shown on the real robot where different motions in position space are achieved by tracking motions of either the body lean angles, or the arm angles or combinations of both.

This paper presents controllers called motion policies that achieve fast, graceful motions in small, collision-free domains of the position space for balancing mobile robots like the ballbot. The motion policies are designed such that their valid compositions will produce overall graceful motions. An automatic instantiation procedure deploys motion policies on a 2D map of the environment to form a library and the validity of their composition is given by a gracefully prepares graph. Dijsktra's algorithm is used to plan in the space of these motion policies to achieve the desired navigation task. A hybrid controller is used to switch between the motion policies. The results of successful experimental testing of two navigation tasks, namely, point-point and surveillance motions on the ballbot platform are presented.

Humanoid robots are expected to share human environments in the future and it is important to ensure safety of their operation. A serious threat to safety is the fall of a humanoid robot, which can seriously damage both the robot and objects in its surrounding. This paper proposes a strategy for planning and control of fall. The controller's objective is to prevent the robot from hitting surrounding objects during a fall by modifying its default fall direction. We have earlier presented such a direction-changing fall controller in. However, the controller was applicable only when the robot's surrounding contained a single object. In this paper we introduce a generalized approach to humanoid fall-direction control among multiple objects. This new framework algorithmically establishes a desired fall direction through assigned scores, considers a number of control options, and selects and executes the best strategy. The fall planner is also able to select “No Action” as the best strategy, if appropriate. The controller is interactive and is applicable for fall occurring during upright standing or walking. The fall performance is continuously tracked and can be improved in real-time. The planning and control algorithms are demonstrated in simulation on an ASIMO-like humanoid robot.

Developed by Prof. Ralph Hollis in the Microdynamic Systems Laboratory at Carnegie Mellon University, Ballbot is a dynamically stable mobile robot moving on a single spherical wheel providing omni-directional motion. Unlike statically stable mobile robots, dynamically stable mobile robots can be tall and skinny with high center of gravity and small base. The ball drive mechanism is a four motor inverse mouse-ball setup. An Inertial Measuring Unit (IMU) and encoders on the motors provide all information needed for full-state feedback. Ballbot has three legs that provide static stability when powered down and is capable of auto-transitioning from the statically stable state to the dynamically stable state and vice versa. It is also capable of yaw rotation about its vertical axis. An absolute encoder provides the relative angle between the IMU and the ball drive unit.

Human-Robot Physical Interaction is an important attribute for robots operating in human environments. The authors illustrate some basic physically interactive behaviors with dynamically stable mobile robots using the ballbot as an example. The ballbot is a dynamically stable mobile robot moving on a single spherical wheel. The dynamic stability and robust controllers enable the ballbot to be physically moved with ease. The authors also demonstrate other behaviors like human intent detection and learn-repeat behavior on the real robot.

The ballbot is a dynamically stable mobile robot that moves on a single spherical wheel and is capable of omnidirectional movement. The ballbot is an underactuated system with nonholonomic dynamic constraints. The authors propose an offline trajectory planning algorithm that provides a class of parametric trajectories to the unactuated joint in order to reach desired static configurations of the system with regard to the dynamic constraint. The parameters of the trajectories are obtained using optimization techniques. A feedback controller is proposed that ensures accurate trajectory tracking. The trajectory planning algorithm and tracking controller are validated experimentally. The authors also extend the offline trajectory planning algorithm to a generalized case of motion between non-static configurations.

Unlike statically stable wheeled mobile robots, dynamically stable mobile robots can have higher centers of gravity, smaller bases of support and can be tall and thin resembling the shape of an adult human. This paper concerns the ballbot mobile robot, which balances dynamically on a single spherical wheel. The ballbot is omni-directional and can also rotate about its vertical axis (yaw motion). It uses a triad of legs to remain statically stable when powered off. This paper presents the evolved design with a four-motor inverse mouse-ball drive, yaw drive, leg drive, control system, and results including dynamic balancing, station keeping, yaw motion while balancing, and automatic transition between statically stable and dynamically stable states.