We address the optimal control problem of robotic systems with variable stiffness actuation (VSA) including switching dynamics and discontinuous state transitions. Our focus in this paper is to consider tasks that have multiple phases of movement, contacts and impacts with the environment. By modelling such tasks as an approximate hybrid dynamical system with time-based switching, we develop a systematic methodology to simultaneously optimize control commands, stiffness profiles and temporal aspect of the movement such as switching instances and total movement duration. Numerical evaluations on a simple switching system, a realistic brachiating robot model with VSA, and a hopper with variable stiffness springs demonstrate the effectiveness of the proposed approach.

Nonlinear dynamical systems have been used in many disciplines to model complex behaviors, including biological motor control, robotics, perception, economics, traffic prediction, and neuroscience. While often the unexpected emergent behavior of nonlinear systems is the focus of investigations, it is of equal importance to create goal-directed behavior (e.g., stable locomotion from a system of coupled oscillators under perceptual guidance). Modeling goal-directed behavior with nonlinear systems is, however, rather difficult due to the parameter sensitivity of these systems, their complex phase transitions in response to subtle parameter changes, and the difficulty of analyzing and predicting their long-term behavior; intuition and time-consuming parameter tuning play a major role. This letter presents and reviews dynamical movement primitives, a line of research for modeling attractor behaviors of autonomous nonlinear dynamical systems with the help of statistical learning techniques. The essence of our approach is to start with a simple dynamical system, such as a set of linear differential equations, and transform those into a weakly nonlinear system with prescribed attractor dynamics by means of a learnable autonomous forcing term. Both point attractors and limit cycle attractors of almost arbitrary complexity can be generated. We explain the design principle of our approach and evaluate its properties in several example applications in motor control and robotics.

We present a novel framework for stiffness and temporal optimization of periodic movements, with an emphasis on exploiting the intrinsic passive dynamics to realize efficient actuation and control. We use a dynamical systems based representation tuned to the requirements of rhythmic movements and propose a systematic methodology to optimize for control commands, temporal aspect of movements and timevarying stiffness profiles from first principles of optimality. Evaluations on a single pendulum and underactuated two-link robot simulation highlight the benefits, achieving remarkable actuation efficiency on complicated, highly dynamic tasks such as swing-up and brachiation.

In this video presentation we introduce a 50 degrees of freedom humanoid robot, CB -ComputationalBrain[1]. CB is a humanoid robot created for exploring the underlying processing of the human brain while dealing with the real world. We place our investigations within real world contexts, as humans do. In so doing, we focus on utilising a system that is closer to humans - in sensing, kinematics configuration and performance. We present a full-body compliance controller that was developed for the motion control of our humanoid robot [2]. Our initial experimentation on our system includes: 1) full-body compliant control - physical interactions/balancing/motion control; 2) the integrated visual ocular-motor responses; 3) perception and control - reaching, foveation, and active object recognition; 4) our studies of Central Pattern Generator for walking.

Biological systems seem to have a simpler but more robust locomotion strategy than that of the existing biped walking controllers for humanoid robots. We show that a humanoid robot can step and walk using simple sinusoidal desired joint trajectories with their phase adjusted by a coupled oscillator model. We use the center-of-pressure location and velocity to detect the phase of the lateral robot dynamics. This phase information is used to modulate the desired joint trajectories. We do not explicitly use dynamical parameters of the humanoid robot. We hypothesize that a similar mechanism may exist in biological systems. We applied the proposed biologically inspired control strategy to our newly developed human-sized humanoid robot computational brain (CB) and a small size humanoid robot, enabling them to generate successful stepping and walking patterns.

This paper presents a novel approach to acquire dynamic whole-body movements on humanoid robots focused on learning a control policy for the center of mass. A policy-gradient method is used to acquire a CoM movement as a control policy for achieving a desired dynamic task. A CoM-Jacobian-based redundancy resolution is then used to compute angular velocities for all joints in order to achieve a whole-body movement consistent with the CoM movement acquired through learning. To demonstrate the effectiveness of our method, we apply it in simulation to the learning of a strong punching movement on the Fujitsu humanoid robot, Hoap-2.

In this paper, we address the issues of compliant control of a robot under contact constraints with a goal of using joint space based pattern generators as movement primitives, as often considered in the studies of legged locomotion and biological motor control. For this purpose, we explore inverse dynamics control of constrained dynamical systems. When the system is overconstrained, it is not straightforward to formulate an inverse dynamics control law since the problem becomes an ill-posed one, where infinitely many combinations of joint torques are possible to achieve the desired joint accelerations. The goal of this paper is to develop a general and computationally efficient inverse dynamics algorithm for a robot with a free floating base and constraints. We suggest an approximate way of computing inverse dynamics algorithm by treating constraint forces computed with a Lagrange multiplier method as simply external forces based on Featherstone's floating base formulation of inverse dynamics. We present how all the necessary quantities to compute our controller can be efficiently extracted from Featherstone's spatial notation of robot dynamics. We evaluate the effectiveness of the suggested approach on a simulated biped robot model

This paper addresses locomotion with active balancing, via task space control with prioritization. The center of gravity (COG) and foot of the swing leg are treated as task space control points. Floating base inverse kinematics with constraints is employed, thereby allowing for a mobile platform suitable for locomotion. Different techniques of task prioritization are discussed and we clarify differences and similarities of previous suggested work. Varying levels of prioritization for control are examined with emphasis on singularity robustness and the negative effects of constraint switching. A novel controller for task space control of balance and locomotion is developed which attempts to address singularity robustness, while minimizing discontinuities created by constraint switching. Controllers are evaluated using a quadruped robot simulator engaging in a locomotion task.

Compliant control will be a prerequisite for humanoid robotics if these robots are supposed to work safely and robustly in human and/or dynamic environments. One view of compliant control is that a robot should control a minimal number of degrees-of-freedom (DOFs) directly, i.e., those relevant DOFs for the task, and keep the remaining DOFs maximally compliant, usually in the null space of the task. This view naturally leads to task space control. However, surprisingly few implementations of task space control can be found in actual humanoid robots. This paper makes a first step towards assessing the usefulness of task space controllers for humanoids by investigating which choices of controllers are available and what inherent control characteristics they have - this treatment will concern position and orientation control, where the latter is based on a quaternion formulation. Empirical evaluations on an anthropomorphic Sarcos master arm illustrate the robustness of the different controllers as well as the ease of implementing and tuning them. Our extensive empirical results demonstrate that simpler task space controllers, e.g., classical resolved motion rate control or resolved acceleration control can be quite advantageous in face of inevitable modeling errors in model- based control, and that well chosen formulations are easy to implement and quite robust, such that they are useful for humanoids.

We show that a humanoid robot can step and walk using simple sinusoidal desired joint trajectories with their phase adjusted by a coupled oscillator model. We use the center of pressure location and velocity to detect the phase of the lateral robot dynamics. This phase information is used to modulate the desired joint trajectories. We applied the proposed control approach to our newly developed human sized humanoid robot and a small size humanoid robot developed by Sony, enabling them to generate successful stepping and walking patterns

Recently, there has been a growing interest in biologically inspired biped locomotion control with Central Pattern Generator (CPG). However, few experimental attempts on real hardware 3D humanoid robots have yet been made. Our goal in this paper is to present our achievement of 3D biped locomotion using a neural oscillator applied to a humanoid robot, QRIO. We employ reduced number of neural oscillators as the CPG model, along with a task space Cartesian coordinate system and utilizing entrainment property to establish stable walking gait. We verify robustness against lateral perturbation, through numerical simulation of stepping motion in place along the lateral plane. We then implemented it on the QRIO. It could successfully cope with unknown 3mm bump by autonomously adjusting its stepping period. Sagittal motion produced by a neural oscillator is introduced, and then overlapped with the lateral motion generator in realizing 3D biped locomotion on a QRIO humanoid robot.

This paper proposes a learning framework for a CPG-based biped locomotion controller using a policy gradient method. Our goal in this study is to develop an efficient learning algorithm by reducing the dimensionality of the state space used for learning. We demonstrate that an appropriate feedback controller in the CPG-based controller can be acquired using the proposed method within a few thousand trials by numerical simulations. Furthermore, we implement the learned controller on the physical biped robot to experimentally show that the learned controller successfully works in the real environment.

We propose a model-based reinforcement learning algorithm for biped walking in which the robot learns to appropriately modulate an observed walking pattern. Via-points are detected from the observed walking trajectories using the minimum jerk criterion. The learning algorithm modulates the via-points as control actions to improve walking trajectories. This decision is based on a learned model of the Poincaré map of the periodic walking pattern. The model maps from a state in the single support phase and the control actions to a state in the next single support phase. We applied this approach to both a simulated robot model and an actual biped robot. We show that successful walking policies are acquired.

Understanding the principles of motor coordination with redundant degrees of freedom still remains a challenging problem, particularly for new research in highly redundant robots like humanoids. Even after more than a decade of research, task space control with redundacy resolution still remains an incompletely understood theoretical topic, and also lacks a larger body of thorough experimental investigation on complex robotic systems. This paper presents our first steps towards the development of a working redundancy resolution algorithm which is robust against modeling errors and unforeseen disturbances arising from contact forces. To gain a better understanding of the pros and cons of different approaches to redundancy resolution, we focus on a comparative empirical evaluation. First, we review several redundancy resolution schemes at the velocity, acceleration and torque levels presented in the literature in a common notational framework and also introduce some new variants of these previous approaches. Second, we present experimental comparisons of these approaches on a seven-degree-of-freedom anthropomorphic robot arm. Surprisingly, one of our simplest algorithms empirically demonstrates the best performance, despite, from a theoretical point, the algorithm does not share the same beauty as some of the other methods. Finally, we discuss practical properties of these control algorithms, particularly in light of inevitable modeling errors of the robot dynamics.

Recently, R. E. Udwadia (2003) suggested to derive tracking controllers for mechanical systems using a generalization of Gauss' principle of least constraint. This method allows us to reformulate control problems as a special class of optimal control. We take this line of reasoning one step further and demonstrate that well-known and also several novel nonlinear robot control laws can be derived from this generic methodology. We show experimental verifications on a Sarcos Master Arm robot for some of the derived controllers. We believe that the suggested approach offers a promising unification and simplification of nonlinear control law design for robots obeying rigid body dynamics equations, both with or without external constraints, with over-actuation or underactuation, as well as open-chain and closed-chain kinematics.

Humanoid research has made remarkable progress during the past 10 years. However, currently most humanoids use the target ZMP (zero moment point) control algorithm for bipedal locomotion, which requires precise modeling and actuation with high control gains. On the contrary, humans do not rely on such precise modeling and actuation. Our aim is to examine biologically related algorithms for bipedal locomotion that resemble human-like locomotion. This paper describes an empirical study of a neural oscillator for the control of biped locomotion. We propose a new neural oscillator arrangement applied to a compass-like biped robot. Dynamic simulations and experiments with a real biped robot were carried out and the controller performs steady walking for over 50 steps. Gait variations resulting in energy efficiency was made possible through the adjustment of only a single neural activity parameter. Aspects of adaptability and robustness of our approach are shown by allowing the robot to walk over terrains with varying surfaces with different frictional properties. Initial results suggesting optimal amplitude for dealing with perturbation are also presented.

An adaptive composite control technique was suggested before for nonlinear adaptive control with statistical learning methods. While this original work was restricted to a simple class of nonlinear SISO systems that were linear in the inputs, in this paper we present a more general treatment of learning a composite controller for the class of nonlinear systems characterized by the form x/spl dot/ = f(x) + g(x)u. We would first examine such systems in the first order SISO framework, and present a stability proof including a parameter projection method that is needed to avoid potential singularities during adaptation. Second, we generalize our adaptive controller to higher order SISO systems, and discuss the application to MIMO problems. We evaluate our theoretical control framework in numerical simulations to illustrate the effectiveness of the proposed learning adaptive controller for rapid convergence and high accuracy of control.

We propose a framework for learning biped locomotion using dynamical movement primitives based on nonlinear oscillators. In our previous work, we suggested dynamical movement primitives as a central pattern generator (CPG) to learn biped locomotion from demonstration. We introduced an adaptation algorithm for the frequency of the oscillators based on phase resetting at the instance of heel strike and entrainment between the phase oscillator and mechanical system using feedback from the environment. In this paper, we empirically explore the role of phase resetting in the proposed algorithm for robust biped locomotion. We demonstrate that phase resetting contributes to robustness against external perturbations and environmental changes by numerical simulations and experiments with a physical biped robot.

Presents an approach to movement planning, on-line trajectory modification, and imitation learning by representing movement plans based on a set of nonlinear differential equations with well-defined attractor dynamics. The resultant movement plan remains an autonomous set of nonlinear differential equations that forms a control policy (CP) which is robust to strong external perturbations and that can be modified on-line by additional perceptual variables. We evaluate the system with a humanoid robot simulation and an actual humanoid robot. Experiments are presented for the imitation of three types of movements: reaching movements with one arm, drawing movements of 2-D patterns, and tennis swings. Our results demonstrate (a) that multi-joint human movements can be encoded successfully by the CPs, (b) that a learned movement policy can readily be reused to produce robust trajectories towards different targets, (c) that a policy fitted for one particular target provides a good predictor of human reaching movements towards neighboring targets, and (d) that the parameter space which encodes a policy is suitable for measuring to which extent two trajectories are qualitatively similar.

BIOROB

This paper introduces a provably stable adaptive learning controller which employs nonlinear function approximation with automatic growth of the learning network according to the nonlinearities and working domain of the control system. The unknown function in the dynamical system is approximated by piecewise linear models using the nonparametric regression technique. Local models are allocated as necessary and their parameters are optimized online. Inspired by composite adaptive control methods, the proposed learning adaptive control algorithm uses both the tracking error and the estimation error to update the parameters. We provide Lyapunov analyses that demonstrate the stability properties of the learning controller. Numerical simulations illustrate the rapid convergence of the tracking error and the automatic structure adaptation capability of the function approximator.

Many control problems take place in continuous state-action spaces, e.g., as in manipulator robotics, where the control objective is of- ten deflned as flnding a desired trajectory that reaches a particular goal state. While reinforcement learning ofiers a theoretical frame- work to learn such control policies from scratch, its applicability to higher dimensional continuous state-action spaces remains rather limited to date. Instead of learning from scratch, in this paper we suggest to learn a desired complex control policy by transforming an existing simple canonical control policy. For this purpose, we represent canonical policies in terms of difierential equations with well-deflned attractor properties. By nonlinearly transforming the canonical attractor dynamics using techniques from nonparametric regression, almost arbitrary new nonlinear policies can be gener- ated without losing the stability properties of the canonical sys- tem. We demonstrate our techniques in the context of learning a set of movement skills for a humanoid robot from demonstrations of a human teacher. Policies are acquired rapidly, and, due to the properties of well formulated difierential equations, can be re-used and modifled on-line under dynamic changes of the environment. The linear parameterization of nonparametric regression moreover lends itself to recognize and classify previously learned movement skills. Evaluations in simulations and on an actual 30 degree-of- freedom humanoid robot exemplify the feasibility and robustness of our approach.

Explores an approach to learning by imitation and trajectory formation by representing movements as mixtures of nonlinear differential equations with well-defined attractor dynamics. An observed movement is approximated by finding a best fit of the mixture model to its data by a recursive least squares regression technique. In contrast to non-autonomous movement representations like splines, the resultant movement plan remains an autonomous set of nonlinear differential equations that forms a control policy which is robust to strong external perturbations and that can be modified by additional perceptual variables. This movement policy remains the same for a given target, regardless of the initial conditions, and can easily be re-used for new targets. We evaluate the trajectory formation system in the context of a humanoid robot simulation that is part of the Virtual Trainer project, which aims at supervising rehabilitation exercises in stroke-patients. A typical rehabilitation exercise was collected with a Sarcos Sensuit, a device to record joint angular movement from human subjects, and approximated and reproduced with our imitation techniques. Our results demonstrate that multijoint human movements can be encoded successfully, and that this system allows robust modifications of the,movement policy through external variables.

This paper proposes a self-organizing control strategy of a signal network for urban traffic flows. The signal network is described using a system of nonlinear oscillators with the nearest neighborhood coupling. The natural frequency, split and coupling constants of the signals are adjusted dynamically according to local information of traffic flows so that the desired network offset patterns are self-organized through mutual entrainment. We provide a preliminary analysis of the coupled oscillator system. Numerical simulations demonstrate the effectiveness of the proposed method under dynamical environmental changes.

We report on a control strategy for the "leap" problem arising from an ape's fast brachiation when the next branch is far out of reach and the task involves a significant component of free flight as well as swing motion. In the swing phase, good regulation of velocity and angular momentum for lift-off to aim at the target bar is required. In the flight phase, control of the posture and rotation of the body is important to catch the bar. We propose a control strategy to achieve such a dynamic maneuver for a two degrees of freedom brachiating robot. Numerical simulations suggest that the proposed strategy achieves the desired leaping maneuver.

We report on our empirical studies of a new controller for a two-link brachiating robot. Motivated by the pendulum-like motion of an ape's brachiation, we encode this task as the output of a "target dynamical system". Numerical simulations indicate that the resulting controller solves a number of brachiation problems that we term the "ladder", "swing-up", and "rope" problems. Preliminary analysis provides some explanation for this success. The proposed controller is implemented on a physical system in our laboratory. The robot achieves behaviors including "swing locomotion" and "swing up" and is capable of continuous locomotion over several rungs of a ladder. We discuss a number of formal questions whose answers will be required to gain a full understanding of the strengths and weaknesses of this approach.

We have previously developed a brachiation controller that allows a two degree of freedom robot to swing from hand-hold to hand-hold on a horizontal ladder with evenly space rungs as well as swing up from a suspended posture using a "target dynamics" controller. In this paper, we extend this class of algorithms to handle the much more natural problem of locomotion over irregularly spaced hand-holds. Numerical simulations and laboratory experiments illustrate the effectiveness of this generalization.

We report on our recent empirical success in the study of a two-link brachiating robot. The "target dynamics" controller developed in our previous work (1997) is implemented on a physical system in our laboratory. The swing locomotion and swing-up behavior of the robot as well as continuous locomotion have been successfully attained. The experimental results illustrate the effectiveness of our control strategy.

We report on our preliminary studies of a new controller for a two-link brachiating robot. Motivated by the pendulum-like motion of an ape's brachiation, we encode this task as the output of a "target dynamical system". Numerical simulations indicate that the resulting controller solves a number of brachiating problems that we term the "ladder", "swing up" and "rope" problems. Preliminary analysis provides some explanation for this success. We discuss a number of formal questions whose answers will be required to gain a full understanding of the strengths and weaknesses of this approach.