Existing preference optimization methods often assume scenarios where paired preference feedback (preferred/positive vs. dis-preferred/negative examples) is available. This requirement limits their applicability in scenarios where only unpaired feedback—for example, either positive or negative— is available. To address this, we introduce a novel approach that decouples learning from positive and negative feedback. This decoupling enables control over the influence of each feedback type and, importantly, allows learning even when only one feedback type is present. A key contribution is demonstrating stable learning from negative feedback alone, a capability not well-addressed by current methods. Our approach builds upon the probabilistic framework introduced in (Dayan and Hinton, 1997), which uses expectation-maximization (EM) to directly optimize the probability of positive outcomes (as opposed to classic expected reward maximization). We address a key limitation in current EM-based methods: they solely maximize the likelihood of positive examples, while neglecting negative ones. We show how to extend EM algorithms to explicitly incorporate negative examples, leading to a theoretically grounded algorithm that offers an intuitive and versatile way to learn from both positive and negative feedback. We evaluate our approach for training language models based on human feedback as well as training policies for sequential decision-making problems, where learned value functions are available.

This paper presents a novel nonlinear disturbance rejection approach for high precision model-based control of hydraulic robots. While most disturbance rejection approaches make use of observers, we propose a novel adaptive Unscented Kalman Filter to estimate the disturbances in an unbiased minimum-variance sense. The filter is made adaptive such that there is no need to tune the covariance matrix for the disturbance estimation. Furthermore, whereas most model-based control approaches require the linearization of the system dynamics, our method is nonlinear which means that no linearization is required. Through extensive simulations as well as real hardware experiments, we demonstrate that our proposed approach can achieve high precision tracking and can be readily applied to most robotic systems even in the presence of uncertainties and external disturbances. The proposed approach is also compared to existing approaches which demonstrates its superior tracking performance.

In this paper we present a system for the in situ33In the context of building construction, “in situ” means that fabrication takes place at the structure's final location directly on the building site. fabrication of a full-scale, load-bearing, and doubly-curved steel reinforced concrete wall. Two complementary vision-based sensing systems provide the feedback necessary to build a 12 meter long steel wire mesh as part of a novel digital building process. The sensing systems provide estimates of the robot pose, referenced to the CAD model of the building site, as well as feedback on the accuracy of the built structure over the course of construction. This second piece of information is used to adapt the building plan to compensate for system inaccuracies and material deformations which occur during buildup. In this way, the structure was successfully built with 98% of the total geometry within 2 centimeters of the designed position. To the best of our knowledge, this is the largest structure which has been built by a mobile robot using solely vision-based sensing.

This paper introduces a family of iterative algorithms for unconstrained nonlinear optimal control. We generalize the well-known iLQR algorithm to different multiple shooting variants, combining advantages like straightforward initialization and a closed-loop forward integration. All algorithms have similar computational complexity, i.e. linear complexity in the time horizon, and can be derived in the same computational framework. We compare the full-step variants of our algorithms and present several simulation examples, including a high-dimensional underactuated robot subject to contact switches. Simulation results show that our multiple shooting algorithms can achieve faster convergence, better local contraction rates and much shorter runtimes than classical iLQR, which makes them a superior choice for nonlinear model predictive control applications.

In this paper, we present an efficient Dynamic Programing framework for optimal planning and control of legged robots. First we formulate this problem as an optimal control problem for switched systems. Then we propose a multi-level optimization approach to find the optimal switching times and the optimal continuous control inputs. Through this scheme, the decomposed optimization can potentially be done more efficiently than the combined approach. Finally, we present a continuous-time constrained LQR algorithm which simultaneously optimizes the feedforward and feedback controller with O(n) time-complexity. In order to validate our approach, we show the performance of our framework on a quadrupedal robot. We choose the Center of Mass dynamics and the full kinematic formulation as the switched system model where the switching times as well as the contact forces and the joint velocities are optimized for different locomotion tasks such as gap crossing, walking and trotting.

This work addresses the problem of kinematic trajectory planning for mobile manipulators with non-holonomic constraints, and holonomic operational-space tracking constraints. We obtain whole-body trajectories and time-varying kinematic feedback controllers by solving a Constrained Sequential Linear Quadratic Optimal Control problem. The employed algorithm features high efficiency through a continuous-time formulation that benefits from adaptive step-size integrators and through linear complexity in the number of integration steps. In a first application example, we solve kinematic trajectory planning problems for a 26 DoF wheeled robot. In a second example, we apply Constrained SLQ to a real-world mobile manipulator in a receding-horizon optimal control fashion, where we obtain optimal controllers and plans at rates up to 100 Hz.

Despite the recent advances in, and the adoption of robotic technologies in the construction industry, the architectural processes which demand a high degree of geometric freedom still remain largely labour intensive and manual. This is due to the inherent difficulties in robotizing the current implementation of such processes coupled with the lack of alternate robotic technologies. A specific example, which is also the focus of this paper, is that of building a steel reinforced concrete structure, with varying curvature or cross-section. This process still remains rather manual and requires extensive support of customized form-work. In this paper, first we describe an alternate novel robotic fabrication process for building steel wire meshes which act as both reinforcement and formwork. The robotization of such a process is discussed with the use of a previously developed mobile robotic system. Based on the specifications derived from the process, design of a novel custom designed robotic end-effector, enabling this process, is detailed. Automation of the full robotic system comprising the mobile robotic system and the robotic end-effector is discussed from simulation to control. Through experimental evaluation of the robotic system, we demonstrate the ability to fully automate the construction of non-standard steel reinforced steel meshes of varying curvature and cell sizes.

We present a trajectory optimization framework for legged locomotion on rough terrain. We jointly optimize the center of mass motion and the foothold locations, while considering terrain conditions. We use a terrain costmap to quantify the desirability of a foothold location. We increase the gait's adaptability to the terrain by optimizing the step phase duration and modulating the trunk attitude, resulting in motions with guaranteed stability. We show that the combination of parametric models, stochastic-based exploration and receding horizon planning allows us to handle the many local minima associated with different terrain conditions and walking patterns. This combination delivers robust motion plans without the need for warm-starting. Moreover, we use soft-constraints to allow for increased flexibility when searching in the cost landscape of our problem. We showcase the performance of our trajectory optimization framework on multiple terrain conditions and validate our method in realistic simulation scenarios and experimental trials on a hydraulic, torque controlled quadruped robot.

We present an algorithm that generates walking motions for quadruped robots without the use of an explicit footstep planner by simultaneously optimizing over both the Center of Mass (CoM) trajectory and the footholds. Feasibility is achieved by imposing stability constraints on the CoM related to the Zero Moment Point and explicitly enforcing kinematic constraints between the footholds and the CoM position. Given a desired goal state, the problem is solved online by a Nonlinear Programming solver to generate the walking motion. Experimental trials show that the algorithm is able to generate walking gaits for multiple steps in milliseconds that can be executed on a real quadruped robot.

We introduce a robust control architecture for the whole-body motion control of torque controlled robots with arms and legs. The method is based on the robust control of contact forces in order to track a planned Center of Mass trajectory. Its appeal lies in the ability to guarantee robust stability and performance despite rigid body model mismatch, actuator dynamics, delays, contact surface stiffness, and unobserved ground profiles. Furthermore, we introduce a task space decomposition approach which removes the coupling effects between contact force controller and the other non-contact controllers. Finally, we verify our control performance on a quadruped robot and compare its performance to a standard inverse dynamics approach on hardware.

In this work we present a co-located task space sensing and control system designed to control the end-effector motion of a mobile manipulator in the presence of dynamic and unknown base motion. We present a method for generating end-effector motion estimates at 1 kilohertz for use in real time control through visual-inertial sensor fusion. We show that a Moving Horizon Estimator outperforms Kalman filter-based methods in generating accurate predictive estimates for use in real time. We use this estimator to close a task space control loop directly at the end-effector, assuming no prior knowledge of the base pose and motion. We demonstrate the performance of this system on a hydraulically actuated arm which performs task-space tracking tasks in the presence of significant unknown base motion.

This paper presents a framework for real-time, full-state feedback, unconstrained, nonlinear model predictive control that combines trajectory optimization and tracking control in a single, unified approach. The proposed method uses an iterative optimal control algorithm, namely Sequential Linear Quadratic (SLQ), in a Model Predictive Control (MPC) setting to solve the underlying nonlinear control problem and simultaneously derive the optimal feedforward and feedback terms. Our customized solver can generate trajectories of multiple seconds within only a few milliseconds. The performance of the approach is validated on two different hardware platforms, an AscTec Firefly hexacopter and the ball balancing robot Rezero. In contrast to similar approaches, we perform experiments that require leveraging the full system dynamics.

Despite the prevalent use of robotic technologies in industrial manufacturing, their use on building construction sites is still very limited. This is mainly due to the unstructured nature of construction sites and the fact that the structures built must be larger than the machines which build them. This paper addresses these difficulties by presenting a repositioning and localization system which, using purely on-board sensing, allows a mobile robot to maintain a high degree of end-effector positioning accuracy while moving among numerous building positions during a building task. We introduce a newly developed machine, called the In situ Fabricator (IF), whose goal is to bring digital fabrication to the construction site. The capabilities of the repositioning and localization system is demonstrated through the construction of a vertical stack of bricks, in which the IF repositions itself after placing each brick. With this experiment, we demonstrate that the IF can build with sub-centimeter accuracy over long building sequences.

Sums-of-Squares optimization represents an important tool for the direct computation of a local Lyapunov function for a nonlinear dynamic system. Specifically, it can certify a sub-levelset of the cost-to-go from an optimal feedback controller like the Linear Quadratic Regulator (LQR), geometrically an ellipsoid in the state space, as Region of Attraction (ROA) of the closed-loop system. More complex robotic tasks however require switching control to first take the system into the ROA before invoking the LQR stabilizer. In this paper, we propose computationally efficient measures of the ROA distance based on quadratic and conic optimization to effectively supervise such a trajectory as it approaches the ROA. As a one-dimensional condensate of the multi-dimensional state trajectory, monitoring the ROA distance evolution allows us to early detect deviations, e.g. due to input saturation or time delay, in order to quickly take corrective action such as replanning. Importantly, computing the ROA distance adds only a small overhead on top of the ROA calculation itself and can be done concurrently.

To render a wearable robot imperceptible to a user is a very challenging control task. The constant and intrinsic interaction between robot and human, and person-dependent behaviours are the main difficulties when designing such cooperative control. In this contribution we introduce and discuss a novel and promising transparency control framework. The foundation of the framework is to measure the acceleration of the human limbs and to exploit this measurement to generate feedforward control commands by using a rigid body model of the robot. The framework includes also an acceleration feedback controller and a state estimator to enhance the overall performance. We present a simplified stability analysis with different feedback controllers and preliminary experimental data that demonstrate the potential of the proposed method in reducing interaction forces and mimicking human motions.

Differential flatness is a property of certain systems that greatly simplifies the generation of optimal and dynamically feasible trajectories. Using a differentially flat model, there is no need to integrate the system dynamics to retrieve the states and the constraints of the optimization problem are simpler. Recently, the concept of partial differential flatness has been introduced covering a broader class of systems. In particular, it allows to reduce the need for integration by limiting it to a subset of the states. However, finding an analytical expression for the (partial) differential flatness requires the manipulation of the equations of motion in a very specific manner such that a series of properties are fulfilled. In general, finding such analytical model is not straightforward nor compatible with algorithmic models. In order to tackle this problem, in this paper we present a numerical method to find a (partially) differentially flat model of a system around a collection of states and inputs trajectories. We present results on three underactuated nonlinear systems (cart-pole, planar ballbot and a 3D quadrotor). As use case examples, we show online trajectory re-planning tasks. The validity of the trajectories obtained with the locally flat models is verified by forward integrating the original equations of motion together with an optimal stabilizer.

Learning from demonstration (LfD) provides an easy and intuitive way to program robot behaviours, potentially reducing development time and costs tremendously. This is especially appealing for manufacturers interested in using industrial manipulators for high-mix production, since this technique enables fast and flexible modifications to the robot behaviours and is thus suitable to teach the robot to perform a wide range of tasks regularly. We define a set of criteria to assess the applicability of state-of-the-art LfD frameworks in the industry. A three-stage LfD method is then proposed, which incorporates human-in-the-loop adaptation to iteratively correct a batch-learned policy to improve accuracy and precision. The system will then transit to open-loop execution of the task to enhance production speed, by removing the human teacher from the feedback loop. The proposed LfD framework addresses all criteria set in this work.

In recent years impressive results have been presented illustrating the potential of quadrotors to solve challenging tasks. Generally, the derivation of the controllers involve complex analytical manipulation of the dynamics and are very specific to the task at hand. In addition, most approaches construct a trajectory and then design a stabilizing controller in a separate step, whereas a fully optimal solution requires finding both simultaneously. In this paper, a generalized approach is presented using an iterative optimal control algorithm. A series of complex tasks are thus solved using the same algorithm without the need for manual manipulation of the system dynamics, heuristic simplifications, or manual trajectory generation. First, aggressive maneuvers are performed by requiring the quadrotor to pass with a slung load through a window not high enough for the load to pass while hanging straight down. Second, go-to-goal tasks with single and double rotor failure are demonstrated. The adaptability and applicability of this unified approach to such diverse tasks with a nonlinear, underactuated, constrained, and in the case of the slung load, hybrid quadrotor systems is thus shown.

An event-based communication framework for remote operation of a robot via a bandwidth-limited network is proposed. The robot sends state and environment estimation data to the operator, and the operator transmits updated control commands or policies to the robot. Event-based communication protocols are designed to ensure that data is transmitted only when required: the robot sends new estimation data only if this yields a significant information gain at the operator, and the operator transmits an updated control policy only if this comes with a significant improvement in control performance. The developed framework is modular and can be used with any standard estimation and control algorithms. Simulation results of a robotic arm highlight its potential for an efficient use of limited communication resources, for example, in disaster-response scenarios such as the DARPA Robotics Challenge.

Increasingly, robots are designed to interact with the environment, including humans and tools. Legged robots, in particular, have to deal with environmental contacts every time they take a step. To handle these interactions properly, it is desirable to be able to set the robot's dynamic behavior, i.e., its impedance. In this contribution, we investigate the most relevant theoretical and practical aspects in impedance control using hydraulic actuators, ranging from the force dynamics analysis and model-based controller design to the overall stability and performance assessment. We present results with one leg of the quadruped robot HyQ and also highlight the influence of hardware parameters, such as valve bandwidth and inertia, in the impedance and force tracking. In addition, we demonstrate the capabilities of HyQ's actively compliant leg by experimentally comparing it with a passively compliant version of the same leg. With such a broad spectrum of analyses and discussions, this paper aims to serve as a practical and comprehensive guide for implementing high-performance impedance control on highly dynamic hydraulic robots.

Learning motion control as a unified process of designing the reference trajectory and the controller is one of the most challenging problems in robotics. The complexity of the problem prevents most of the existing optimization algorithms from giving satisfactory results. While model-based algorithms like iterative linear-quadratic-Gaussian (iLQG) can be used to design a suitable controller for the motion control, their performance is strongly limited by the model accuracy. An inaccurate model may lead to degraded performance of the controller on the physical system. Although using machine learning approaches to learn the motion control on real systems have been proven to be effective, their performance depends on good initialization. To address these issues, this paper introduces a two-step algorithm which combines the proven performance of a model-based controller with a model-free method for compensating for model inaccuracy. The first step optimizes the problem using iLQG. Then, in the second step this controller is used to initialize the policy for our PI2-01 reinforcement learning algorithm. This algorithm is a derivation of the PI2algorithm enabling more stable and faster convergence. The performance of this method is demonstrated both in simulation and experimental results.

We propose a reactive controller framework for robust quadrupedal locomotion, designed to cope with terrain irregularities, trajectory tracking errors and poor state estimation. The framework comprises two main modules: One related to the generation of elliptic trajectories for the feet and the other for control of the stability of the whole robot. We propose a task space CPG-based trajectory generation that can be modulated according to terrain irregularities and the posture of the robot trunk. To improve the robot's stability, we implemented a null space based attitude control for the trunk and a push recovery algorithm based on the concept of capture points. Simulations and experimental results on the hydraulically actuated quadruped robot HyQ will be presented to demonstrate the effectiveness of our framework.

A legged robot has to deal with environmental contacts every time it takes a step. To properly handle these interactions, it is desirable to be able to set the foot compliance. For an actively-compliant legged robot, in order to ensure a stable contact with the environment the robot leg has to be passive at the contact point. In this work, we asses some passivity and stability issues of the actively-compliant leg of the quadruped robot HyQ, which employs a highperformance cascade compliance controller. We demonstrate that both the nested torque loop performance as well as the actuator bandwidth have a strong influence in the range of virtual impedances that can be passively rendered by the robot leg. Based on the stability analyses and experimental results, we propose a procedure for designing cascade compliance controllers. Furthermore, we experimentally demonstrate that the HyQ's actively-compliant leg is able to reproduce the compliant behavior presented by an identical but passively-compliant version of the same leg.

Legged robots have the potential to serve as versatile and useful autonomous robotic platforms for use in unstructured environments such as disaster sites. They need to be both capable of fast dynamic locomotion and precise movements. However, there is a lack of platforms with suitable mechanical properties and adequate controllers to advance the research in this direction. In this paper we are presenting results on the novel research platform HyQ, a torque controlled hydraulic quadruped robot. We identify the requirements for versatile robotic legged locomotion and show that HyQ is fulfilling most of these specifications. We show that HyQ is able to do both static and dynamic movements and is able to cope with the mechanical requirements of dynamic movements and locomotion, such as jumping and trotting. The required control, both on hydraulic level (force/torque control) and whole body level (rigid model based control) is discussed.

Robots are frequently modeled as rigid body systems, having torques as input to their dynamics. A high-performance low-level torque source allows us to control the robot/environment interaction and to straightforwardly take advantage of many model-based control techniques. In this paper, we define a general 1-DOF framework, using basic physical principles, to show that there exists an intrinsic velocity feedback in the generalized force dynamics, independently of the actuation technology. We illustrate this phenomena using three different systems: a generic spring-mass system, a hydraulic actuator, and an electric motor. This analogy helps to clarify important common aspects regarding torque/force control that can be useful when designing and controlling a robot. We demonstrate, using simulations and experimental data, that it is possible to compensate for the load motion influence and to increase the torque tracking capabilities.

Controllers for articulated robots such as an arm or a humanoid commonly need to continuously calculate complex algebraic quantities, such as the joint space inertia matrix or Jacobians. An effective and fast implementation of the calculation of these quantities is crucial to achieve complex, yet robust controllers and thus enable sophisticated behaviors in robots. Although the nature of these algebraic quantities is very well known in robotics, they do not lend themselves easily to manual implementation, because of ambiguities and the complexity in their development and use. We propose an approach that addresses this issue by relying on automatic code generation, thus relieving the user from hand crafted development. Our approach also addresses efficiency and speed, in order to satisfy the strict requirements of real time robot controllers, yet it is easy to use. We show the effectiveness of our method by means of some preliminary comparisons.

Inverse dynamics controllers and operational space controllers have proved to be very efficient for compliant control of fully actuated robots such as fixed base manipulators. However legged robots such as humanoids are inherently different as they are underactuated and subject to switching external contact constraints. Recently several methods have been proposed to create inverse dynamics controllers and operational space controllers for these robots. In an attempt to compare these different approaches, we develop a general framework for inverse dynamics control and show that these methods lead to very similar controllers. We are then able to greatly simplify recent whole-body controllers based on operational space approaches using kinematic projections, bringing them closer to efficient practical implementations. We also generalize these controllers such that they can be optimal under an arbitrary quadratic cost in the commands.

We present an approach that enables robots to learn motion primitives that are robust towards state estimation uncertainties. During reaching and preshaping, the robot learns to use line manipulation strategies to maneuver the object into a pose at which closing the hand to perform the grasp is more likely to succeed. In contrast, common assumptions in grasp planning and motion planning for reaching are that these tasks can be performed independently, and that the robot has perfect knowledge of the pose of the objects in the environment. We implement our approach using Dynamic Movement Primitives and the probabilistic model-free reinforcement learning algorithm Policy Improvement with Path Integrals (PI2). The cost function that PI2optimizes is a simple boolean that penalizes failed grasps. The key to acquiring robust motion primitives is to sample the actual pose of the object from a distribution that represents the state estimation uncertainty. During learning, the robot will thus optimize the chance of grasping an object from this distribution, rather than at one specific pose. In our empirical evaluation, we demonstrate how the motion primitives become more robust when grasping simple cylindrical objects, as well as more complex, non-convex objects. We also investigate how well the learned motion primitives generalize towards new object positions and other state estimation uncertainty distributions.

We present a control architecture for fast quadruped locomotion over rough terrain. We approach the problem by decomposing it into many sub-systems, in which we apply state-of-the-art learning, planning, optimization and control techniques to achieve robust, fast locomotion. Unique features of our control strategy include: (1) a system that learns optimal foothold choices from expert demonstration using terrain templates, (2) a body trajectory optimizer based on the Zero-Moment Point (ZMP) stability criterion, and (3) a floating-base inverse dynamics controller that, in conjunction with force control, allows for robust, compliant locomotion over unperceived obstacles. We evaluate the performance of our controller by testing it on the LittleDog quadruped robot, over a wide variety of rough terrain of varying difficulty levels. We demonstrate the generalization ability of this controller by presenting test results from an independent external test team on terrains that have never been shown to us.

Model-based control methods can be used to enable fast, dexterous, and compliant motion of robots without sacrificing control accuracy. However, implementing such techniques on floating base robots, e.g., humanoids and legged systems, is non-trivial due to under-actuation, dynamically changing constraints from the environment, and potentially closed loop kinematics. In this paper, we show how to compute the analytically correct inverse dynamics torques for model-based control of sufficiently constrained floating base rigid-body systems, such as humanoid robots with one or two feet in contact with the environment. While our previous inverse dynamics approach relied on an estimation of contact forces to compute an approximate inverse dynamics solution, here we present an analytically correct solution by using an orthogonal decomposition to project the robot dynamics onto a reduced dimensional space, independent of contact forces. We demonstrate the feasibility and robustness of our approach on a simulated floating base bipedal humanoid robot and an actual robot dog locomoting over rough terrain.

Reinforcement learning (RL) is one of the most general approaches to learning control. Its applicability to complex motor systems, however, has been largely impossible so far due to the computational difficulties that reinforcement learning encounters in high dimensional continuous state-action spaces. In this paper, we derive a novel approach to RL for parameterized control policies based on the framework of stochastic optimal control with path integrals. While solidly grounded in optimal control theory and estimation theory, the update equations for learning are surprisingly simple and have no danger of numerical instabilities as neither matrix inversions nor gradient learning rates are required. Empirical evaluations demonstrate significant performance improvements over gradient-based policy learning and scalability to high-dimensional control problems. Finally, a learning experiment on a robot dog illustrates the functionality of our algorithm in a real-world scenario. We believe that our new algorithm, Policy Improvement with Path Integrals (PI2), offers currently one of the most efficient, numerically robust, and easy to implement algorithms for RL in robotics.

Recent advances on path integral stochastic optimal control [1],[2] provide new insights in the optimal control of nonlinear stochastic systems which are linear in the controls, with state independent and time invariant control transition matrix. Under these assumptions, the Hamilton-Jacobi-Bellman (HJB) equation is formulated and linearized with the use of the logarithmic transformation of the optimal value function. The resulting HJB is a linear second order partial differential equation which is solved by an approximation based on the Feynman-Kac formula [3]. In this work we review the theory of path integral control and derive the linearized HJB equation for systems with state dependent control transition matrix. In addition we derive the path integral formulation for the general class of systems with state dimensionality that is higher than the dimensionality of the controls. Furthermore, by means of a modified inverse dynamics controller, we apply path integral stochastic optimal control over the new control space. Simulations illustrate the theoretical results. Future developments and extensions are discussed.

Many critical elements for statically stable walking for legged robots have been known for a long time, including stability criteria based on support polygons, good foothold selection, recovery strategies to name a few. All these criteria have to be accounted for in the planning as well as the control phase. Most legged robots usually employ high gain position control, which means that it is crucially important that the planned reference trajectories are a good match for the actual terrain, and that tracking is accurate. Such an approach leads to conservative controllers, i.e. relatively low speed, ground speed matching, etc. Not surprisingly such controllers are not very robust - they are not suited for the real world use outside of the laboratory where the knowledge of the world is limited and error prone. Thus, to achieve robust robotic locomotion in the archetypical domain of legged systems, namely complex rough terrain, where the size of the obstacles are in the order of leg length, additional elements are required. A possible solution to improve the robustness of legged locomotion is to maximize the compliance of the controller. While compliance is trivially achieved by reduced feedback gains, for terrain requiring precise foot placement (e.g. climbing rocks, walking over pegs or cracks) compliance cannot be introduced at the cost of inferior tracking. Thus, model-based control and - in contrast to passive dynamic walkers - active balance control is required. To achieve these objectives, in this paper we add two crucial elements to legged locomotion, i.e., floating-base inverse dynamics control and predictive force control, and we show that these elements increase robustness in face of unknown and unanticipated perturbations (e.g. obstacles). Furthermore, we introduce a novel line-based COG trajectory planner, which yields a simpler algorithm than traditional polygon based methods and creates the appropriate input to our control system.We show results from both simulation and real world of a robotic dog walking over non-perceived obstacles and rocky terrain. The results prove the effectivity of the inverse dynamics/force controller. The presented results show that we have all elements needed for robust all-terrain locomotion, which should also generalize to other legged systems, e.g., humanoid robots.

We address the problem of foothold selection in robotic legged locomotion over very rough terrain. The difficulty of the problem we address here is comparable to that of human rock-climbing, where foot/hand-hold selection is one of the most critical aspects. Previous work in this domain typically involves defining a reward function over footholds as a weighted linear combination of terrain features. However, a significant amount of effort needs to be spent in designing these features in order to model more complex decision functions, and hand-tuning their weights is not a trivial task. We propose the use of terrain templates, which are discretized height maps of the terrain under a foothold on different length scales, as an alternative to manually designed features. We describe an algorithm that can simultaneously learn a small set of templates and a foothold ranking function using these templates, from expert-demonstrated footholds. Using the LittleDog quadruped robot, we experimentally show that the use of terrain templates can produce complex ranking functions with higher performance than standard terrain features, and improved generalization to unseen terrain.

There is much to gain from providing walking machines with passive dynamics, e.g. by including compliant elements in the structure. These elements can offer interesting properties such as self-stabilization, energy efficiency and simplified control. However, there is still no general design strategy for such robots and their controllers. In particular, the calibration of control parameters is often complicated because of the highly nonlinear behavior of the interactions between passive components and the environment. In this article, we propose an approach in which the calibration of a key parameter of a walking controller, namely its intrinsic frequency, is done automatically. The approach uses adaptive frequency oscillators to automatically tune the intrinsic frequency of the oscillators to the resonant frequency of a compliant quadruped robot. The tuning goes beyond simple synchronization and the learned frequency stays in the controller when the robot is put to halt. The controller is model free, robust and simple. Results are presented illustrating how the controller can robustly tune itself to the robot, as well as readapt when the mass of the robot is changed. We also provide an analysis of the convergence of the frequency adaptation for a linearized plant, and show how that analysis is useful for determining which type of sensory feedback must be used for stable convergence. This approach is expected to explain some aspects of developmental processes in biological and artificial adaptive systems that "develop" through the embodied system-environment interactions