Deep Reinforcement Learning (RL) has demonstrated impressive results in solving complex robotic tasks such as quadruped locomotion. Yet, current solvers fail to produce efficient policies respecting hard constraints. In this work, we advocate for integrating constraints into robot learning and present Constraints as Terminations (CaT), a novel constrained RL algorithm. Departing from classical constrained RL formulations, we reformulate constraints through stochastic terminations during policy learning: any violation of a constraint triggers a probability of terminating potential future rewards the RL agent could attain. We propose an algorithmic approach to this formulation, by minimally modifying widely used off-the-shelf RL algorithms in robot learning (such as Proximal Policy Optimization). Our approach leads to excellent constraint adherence without introducing undue complexity and computational overhead, thus mitigating barriers to broader adoption. Through empirical evaluation on the real quadruped robot Solo crossing challenging obstacles, we demonstrate that CaT provides a compelling solution for incorporating constraints into RL frameworks. Videos and code are available at constraints-as-terminations.github.io.

Estimating the centroidal dynamics of legged robots is crucial in the context of multi-contact locomotion of legged robots. In this paper, we formulate the estimation of centroidal dynamics as a maximum a posteriori problem and we use a differential dynamic programming approach for solving it. The soundness of the proposed approach is first validated on a simulated humanoid robot, where ground truth data is available, enabling error analysis, and then compared to other alternatives of the state of the art, namely an extend Kalman filter and a recursive complementary filter. The results demonstrate that, compared to other approaches, the proposed method reduces the estimation error on the centroidal state in addition to ensuring the dynamics consistency of the state trajectory. Finally, the effectiveness of the proposed method is illustrated on real measurements, obtained from walking experiments with the HRP-2 humanoid robot.

As locomotion decisions must be taken by considering the future, most existing quadruped controllers are based on a model predictive controller (MPC) with a reduced model of the dynamics to generate the motion and a whole- body controller to execute it. Yet the simplifying assumptions of the MPC are often chosen ad-hoc or by intuition. In this article, we focus on a set of MPCs and analyze the effect of chosen model reductions on the behavior of the robot. Based on existing formulations, we present additional controllers to better understand the influence of model reductions on the controller capabilities. Finally, we propose a robust predictive controller capable of optimizing the foot placements, gait period, center- of-mass trajectory and ground reaction forces. The behavior of these controllers is statistically evaluated in simulation. This empirical study aims to assess the relative importance of the components of the optimal control problem (variables, costs, dynamics) to be able to take reasoned decisions instead of arbitrarily emphasizing or neglecting some of them. We also provide a qualitative study in simulation and on the real robot Solo-12.

This paper presents a computational framework for the design of high-performance legged robotic systems. The framework relies on the concurrent optimization of hardware parameters and control trajectories to find the best robot design for a given task. In particular, we focus on energy efficiency, presenting novel electro-mechanical models to account for the losses of the actuators due to friction and Joule effects. Thanks to a bi-level optimization scheme, featuring a genetic algorithm in the outer loop, our framework can also optimize for the duration of the motion, the actuators, and the size of the robot. We present a novel approach to scale both the actuators and the robot structure in a way that ensures structural integrity by maintaining constant the normalized deflection of the links. We validated our approach by designing a two-joint monoped robot to execute a jumping task. Our simulation results show that our framework can lead to remarkable energy savings (up to 60%) thanks to the concurrent optimization of robot size, motion duration, and actuators.

This paper aims at showing the dynamic performance and reliability of the low-cost, open-access quadruped robot Solo-12, which is developed within the framework of Open Dynamic Robot Initiative. It presents the implementation of a state-of-the-art control pipeline, close to the one that was previously implemented on Mini Cheetah, which implements a model predictive controller based on the centroidal dynamics to compute desired contact forces in order to track a reference velocity. Different contributions are proposed to speed up the computation process, notably at the level of the state estimation and the whole body controller. Experimental results demonstrate that the robot closely follow the reference velocity while being highly reactive and able to recover from perturbations.

Pneumatic actuators have inherent compliance and hence they are very interesting for applications involving interaction with environment or human. But controlling such kind of actuators is not trivial. The paper presents an implementation of iterative Linear Quadratic regulator (iLQR) based optimal control framework to control an anthropomorphic arm with each joint actuated by an agonist-antagonistic pair of Mckibben artificial muscles. The method is applied to positioning tasks and generation of explosive movements by maximizing the link speed. It is then compared to traditional control strategies to justify that optimal control is effective in controlling the position in highly non-linear pneumatic systems. Also the importance of varying compliance is highlighted by repeating the tasks at different compliance level. The algorithm validation is reported here by several simulations and hardware experiments in which the shoulder and elbow flexion are controlled simultaneously.

This paper presents a geometric approach to the problem of designing visual feedback control laws to steer a nonholonomic vehicle, equipped with a fixed monocular camera, through a gate. The originality of our approach is to introduce and exploit the natural geometry induced by the presence of a gate in the environment, e.g. bundle of hyperbolae, ellipses and circles, providing stabilizing feedback control laws that steer the vehicle through the middle of the gate. Moreover, using visual servoing we prove that this geometry can be measured directly in the camera image plane. As a consequence, we provide an image-based control scheme, avoiding the use of a state observer. Simulations in a realistic scenario and experiments are provided to show the effectiveness of our feedback control laws.

This paper presents a geometric approach to the problem of steering a robot subject to nonholonomic constraints through a door by using only visual measurements coming from a single fixed on-board monocular camera. The door is represented by two landmarks located on its vertical supports. After exploring the geometric structure that naturally emerges from the problem statement, e.g., bundle of hyperbolae, ellipses, and circles, we exploit this planar geometry to provide stabilizing feedback control laws to drive the vehicle through the middle of the door. Using visual servoing, we prove that this geometry can be directly measured in the camera image plane. Hence, we provide an image-based control scheme, avoiding the use of a state observer. Simulations in a realistic scenario and experiments are provided to show the effectiveness of the feedback control laws.

In this paper, contact dynamics of robot massage is described by the port-Hamiltonian modelling approach. In order to capture accurately the inherent characteristics of the human body in lumped-parameter manners, the conventional linear Kelvin-Voigt models are replaced by the nonlinear Hunt-Crossley models. As an application of the contact dynamics, coupled stability of compliant robotic arm with impedance control is theoretically analyzed from energetic viewpoints. Experiments are done to verify the massage stability. The proposed contact dynamics evidently has great potential on performance improvement of robot massage, which will be our research subject.

The most widely used technique for generating whole-body motions on a humanoid robot accounting for various tasks and constraints is inverse kinematics. Based on the task-function approach, this class of methods enables the coordination of robot movements to execute several tasks in parallel and account for the sensor feedback in real time, thanks to the low computation cost. To some extent, it also enables us to deal with some of the robot constraints (e.g., joint limits or visibility) and manage the quasi-static balance of the robot. In order to fully use the whole range of possible motions, this paper proposes extending the task-function approach to handle the full dynamics of the robot multibody along with any constraint written as equality or inequality of the state and control variables. The definition of multiple objectives is made possible by ordering them inside a strict hierarchy. Several models of contact with the environment can be implemented in the framework. We propose a reduced formulation of the multiple rigid planar contact that keeps a low computation cost. The efficiency of this approach is illustrated by presenting several multicontact dynamic motions in simulation and on the real HRP-2 robot.

Concurrent advancements in mechanical design and motion planning algorithms allow state-of-the-art humanoid robots to exhibit complex and realistic behavior. In face of this added complexity and the need for humanlike behavior, research has begun to look toward studies in human neuroscience to better organize and guide humanoid robot motion. In this paper, we present one such method of generating anthropomorphic motion by building the “invariants” of human movements and applying them as kinematic tasks. Whole-body motion of 14 healthy participants was recorded during a walking and grasping task. The recorded data were statistically analyzed to extract invariants which best described the observed motion. These invariants were expressed as a set of rules that were used to synthesize the stereotypy in human motion. We propose an algorithm that reproduces the key parameters of motion, taking into account the knowledge from human movement and the limitations of the target anthropomorph. The results are then generalized such that we can generate motion for targets which were not originally recorded. The algorithmic output is applied in a task-based prioritized inverse kinematics solver to generate dynamically stable and realistic anthropomorphic motion. We illustrate our results on the humanoid HRP-2 by making it walk to and grasp objects at various positions. Our approach complements classical optimization or motion-planning-based methods and provides interesting perspectives toward the use of human movements for deducing effective cost functions in optimization techniques or heuristics for planning algorithms.

In this paper, we propose a solution to compute full-dynamic motions for a humanoid robot, accounting for various kinds of constraints such as dynamic balance or joint limits. As a first step, we propose a unification of task-based control schemes, in inverse kinematics or inverse dynamics. Based on this unification, we generalize the cascade of quadratic programs that were developed for inverse kinematics only. Then, we apply the solution to generate, in simulation, whole-body motions for a humanoid robot in unilateral contact with the ground, while ensuring the dynamic balance on a non horizontal surface.

Control methods based on a hierarchy of tasks provide a fast, easily-modifiable, and accurate way of generating a motion. In this paper, we propose to extend this hierarchical approach by using a cascade of quadratic programs to handle simultaneously the robot dynamics, inequality and equality constraints, and multiple non-coplanar unilateral contacts. First, we detail the proposed generic inverse-dynamics solver. Then, we prove that the model used to handle contacts encompasses the classical zero-moment-point balance condition. Finally, as an example of the capabilities of the method, we generate a complex motion where the humanoid robot HRP2 sits down on an armchair, using the armrests as additional contacts, while ensuring joint position and velocity limits.

This paper deals with the problem of generating realistic human-like reaching movements from a small set of movement primitives. Two kinds of movement databases are used as reference. The first one is obtained numerically, by applying biological principles of motor control on the dynamic model of the robot arm. The second one is obtained by recording reaching movements of human subjects. From these databases, primitives are extracted and analyzed by using Principal Component Analysis. An original generalization method is then proposed for generating movements that did not belong to the initial database. We show that twenty primitives allow to produce new movements, having characteristics similar to that of humans. Experiments on the humanoid robot HRP-2 are presented to illustrate the result.

Efficient methods have so far been proposed for planning dynamically stable walking pattern for humanoid robots. However, to guarantee that the reference joint trajectory will produce a safe movement despite modeling errors and perturbations, a stabilizer needs to be implemented on the robot. Though this stabilizer constitutes an essential part of the control strategy of most advanced humanoid platform, it is usually not open-source and dedicated to the own robot characteristics. The goal of this paper is to propose a general and practical strategy for designing a stabilizer for joint-position controlled humanoid robots. The proposed method is based on a double inverted pendulum model and a decoupling approach thanks to which the position of the ZMP and the center of gravity can be controlled independently through the regulation of the ankle and hip joints. The stabilizer generates the expected stabilizing torques from the admissible joint position input. The resulting control algorithm is fast and can be easily executed on the robot. This algorithm was successfully implemented as real-time plugins for the OpenHRP simulator of the HRP2. Simulations showing the efficiency of the method are presented and discussed.

We present a novel method of guiding a humanoid robot, including stepping, by allowing a user to move its head. The motivation behind this approach comes from research in the field of human neuroscience. In human locomotion it has been found that the head plays a very important role in guiding and planning motion. We use this idea to generate humanoid whole-body motion derived purely as a result of moving the head joint. The input to move the head joint is provided by a user via a 6D mouse. The algorithm presented in this study judges when further head movement leads to instability, and then generates stepping motions to stabilize the robot. By providing the software with autonomy to decide when and where to step, the user is allowed to simply steer the robot head (via visual feedback) without worrying about stability. We illustrate our results by presenting experiments conducted in simulation, as well as on our robot, HRP2.

This paper addresses the problem of determining time-optimal trajectories, between two specified configurations, for a nonholonomic bidirectional steered robot. It presents an original geometric reasoning that is grounded on Pontryagin's maximum principle, which provides analytical solutions of this problem in a visually clear way and allows for an effective algorithm to compute the exact optimal trajectories between two arbitrarily specified configurations. The proposed geometric reasoning is based on the analysis of the switching functions of the optimal controller and the definition of a switching vector from which it is able to determine a unit vector rotating along a unit circle of an appropriate coordinate system. It is shown that simple geometric rules are sufficient to determine all possible rotations of this unit vector, from which the time-optimal trajectories can be uniquely determined. The proposed algorithm, which is based on this geometric reasoning, is guaranteed to be complete and has a low computational cost. Moreover, the proposed geometric representation provides an interesting insight into the structure of this class of nonholonomic systems, thereby offering a model for further studies.

This paper describes a control strategy to stabilize the position of a vertical takeoff and landing (VTOL) unmanned aerial vehicle (UAV) in wind gusts. The proposed approach takes advantage of the cascade structure of the system to design a hierarchical controller. The idea is to separate the controller in a high level controller devoted to position control and a low level controller devoted to stabilization and attitude control. Both controllers are designed by means of backstepping techniques that allow the stabilization of the vehicle's position while on-line estimation of the unknown aerodynamic forces. The global stability of the connected system is proven, and simulations as well as experimental results are presented

This paper describes a broadband beampattern synthesis method for sound source localization in the nearfield or in the farfield of a mobile robot, with a small-size linear array. The method is based on the theory of modal analysis and involves an original convex optimization procedure which benefits from the Parseval relation. The optimized beampattern is obtained by numerically minimizing the worst-case error between the modal coefficients of the array response and those of the reference beampattern, up to a finite rank of the series expansion, over a frequency grid. Simulations illustrate the analytical development

This paper deals with the autonomous navigation of a ducted fan vertical take off and landing (VTOL) unmanned aerial vehicle (UAV) in constrained environment. The proposed strategy considers the translational dynamics of the vehicle to design a control of the thrust vector that ensures safe navigation in presence of encountered obstacles. Adaptive backstepping techniques are used to design a nonlinear feedback control that allows to navigate amidst obstacles despite wind perturbations. Simulations results and a first experiment are presented to illustrate the concept

This paper shows how to compute the nonholonomic distance between a polygonal car-like robot and polygonal obstacles. The solution extends previous work of Reeds and Shepp by finding the shortest path to a manifold (rather than to a point) in configuration space. Based on optimal control theory, the proposed approach yields an analytic solution to the problem

The work presented in this paper comes as a part of a project which aims at developing an auditory system for a mobile robot. It presents a sound source localization strategy which enables the sensing of signals within a direction of arrival and frequency domain of interest while rejecting other data. A rapid prototyping method is proposed to design filter-sum beamformers on the basis of convex optimization. This method is well-suited to robotics applications as it copes with real-time constraints and allows the localization of broadband signals such as human voice. Numerous simulation results are used to illustrate the reasoning.

This paper addresses the problem of practically implementing an original sound source localization strategy for mobile robots applications. The proposed method is based on a convex optimization solution to beamforming. It allows the sensing of signals within a direction of arrival and frequency domain of interest while rejecting other data. A precise description of the acquisition chain is proposed and a careful mathematical modeling is given in order to bridge the gap between theory and practical implementation. Simulation results and comparisons with classical filter-sum beamformer techniques are provided at the end of the paper to illustrate the performance of the sensor.

In this paper, a nonlinear complimentary filter (x-estimator) is presented to estimate the attitude of a UAV (unmanned aerial vehicle). The measurements are taken from a low-cost SMU (inertial measurement unit) which consists of 3-axis accelerometers and 3-axis gyroscopes. The gyro bias are estimated online. A second nonlinear complimentary filter (z-estimator) is also designed, it combines 3-axis gyroscope readings with 3-axis magnetometer measurements. From the proposed estimators, the full rotation matrix R will be retrieved. Both estimators use the fact that the orientation matrix, evolving on SO(3), is not locally parameterized and thus could be used to describe any kind of 3D motion. Convergence of the two observers is theoretically proved and simulations as well as experiments are conducted on a real platform in hovering flight conditions.

This paper presents a multicriteria image-based controller and describes an application of this result to the task sequencing problem. The method allows to stabilize the camera and determine the associated region of stability in spite of unknown value of the target points depth, bounds on admissible visual feature errors which guarantee visibility, and limits on the camera velocity and acceleration. The proposed formulation, based on a mixed polytopic and norm-bounded representation of uncertainties, allows to consider LMI-based optimization schemes to maximize the size of the region of stability associated to the closed-loop system. Through this result we show the interest of the approach for designing control strategies that allow to link dynamically a sequence of sensor-based tasks. An application of the result to a problem of task sequencing is simulated in the last section.

This paper presents recent automatic control developments of the airship robot Karma of LAAS-CNRS. More precisely, it focuses on the lateral control strategy which is based on the decoupled dynamics. Using an external control loop to stabilize the longitudinal dynamics, a path following controller is proposed for driving the robot within the horizontal plane. The proposed strategy allows to consider the lateral slippage inherent in the blimp's dynamics.

This paper deals with the autonomous airship control in a case of very low perturbations. A flight decomposition allowing one to define canonical navigation phases from take-off to landing is proposed. For each phase a reduced model is determined and a controller is designed on the base of backstepping techniques. This approach allows one to consider the kinematic and dynamic requirement separately. Due to the decoupling properties, an equilibrium state is reached at the end of each flight phase, allowing one to model easily the transition between them. Simulations of the different controllers are presented for a realistic model of blimp including aerostatic, dynamic and aerodynamic effects.

This paper presents a path planning algorithm for a tractor to execute specific farmwork tasks requiring the complete coverage of a field while minimizing the overlapping between successive passages. The proposed method is based on the determination of a set of characteristic points from which a graph is defined. The covering trajectory is determined by means of a greedy algorithm associated to a heuristic and a cost function minimization which is used to determine the best Hamiltonian solution in the graph.

We consider the kinematic model of a vehicle moving forward with a lower bounded turning radius. This model can be used to describe the kinematics of road vehicles as well as aircraft cruising at constant altitude, or sea vessels. We consider the problem of minimizing the length travelled by the vehicle starting from a generic configuration to connect to a specified route. A feedback law is proposed, such that straight routes can be approached optimally, while system is asymptotically stabilized. Experimental results are reported showing real-time feasibility of the approach.

Presents two control strategies based on the data provided by a camera and a 2D laser range sensor for driving a mobile robot towards a target amidst obstacles. The methods are based on the task function formalism and use the robot redundancy with respect to the tracking task to control both the base and the camera during the avoidance phase. Simulation results are given at the end of the paper.

Presents a controller for driving a mobile robot towards a target in a cluttered environment. The proposed method combines visual servoing techniques allowing the target to be tracked, with an obstacle avoidance strategy based on the information provided by a 2D laser range sensor. Simulation results are presented at the end of the paper.

We present a new method for determining a path following controller for a nonholonomic mobile robot, which is robust with respect to position and orientation errors. We consider the kinematic model of a unicycle and dynamic extension. The control design is based on a change of variables allowing to define a set of n decoupled auxiliary variables. The exponential convergence of the state variables is deduced from the convergence of the auxiliary variables. We prove the control robustness by showing that, when the state variables measurement is noisy, the representative point of the system converges towards a compact attractive domain centred at the equilibrium point. As the auxiliary variables are decoupled, the computation of this attractive domain is done in a very simple and accurate way. The determination of such a domain can be used to determine a security margin to avoid obstacles during the path following process.

Classically, closed-loop path-following controller are used to perform the displacement of mobile robots along pre-planned paths. However, when the initial configuration of the robot is far from the reference path, the stabilization process may involve an important transition phase before the robot moves along the path. Though it is possible to guarantee that the robot will not collide with obstacles as it moves closely along the path, it is hard to insure the motion to be safe during the transition phase. The paper presents a path following controller for mobile robots, allowing it to avoid obstacles during the transition phase. The control design is based on sliding mode techniques. We define a time-varying sliding manifold and we use this degree of freedom to modify the shape of the robot's trajectory, during the path-following process. Our technique is applied to the model of a unicycle, making the simplifying hypothesis that the obstacles encountered are strictly convex.

The techniques of filtering and merging data coming from several sensors allow to localize a mobile robot in its environment with a precision which can be evaluated. However, as the localization error cannot be neglected, the design of robust closed-loop controller for wheeled robots constitutes a difficult problem. We present here a path following feedback controller robust with respect to localization error. The model is a dynamic extension of the usual kinematic model of a car, in the sense that the path curvature error is considered as a new state variable. The control inputs are respectively the linear velocity and the derivative of the curvature. We determine a variable structure control with sliding mode to stabilize the vehicle's motion around the reference path in the nominal case. Then, we prove that the system remains stable when the state feedback is computed from the estimated values instead of the exact ones. We show that the regulation error is contained in a compact attractive domain when the system has reached its steady state. From this domain, one can easily compute a security margin to guarantee obstacle avoidance during the path following process. Experimental results are presented at the end of the paper.

Clothoids are very useful for smoothing the motion of a mobile robot moving along a trajectory. This paper addresses the problem of smoothing mobile robot motions when cusps, i.e., changes of motion direction along the trajectory, are imposed. We pinpoint some special curves (that we call "anticlothoids") and we discuss how they can be used together with clothoids in order to smooth a predefined trajectory.>

We calculate the partition of the configuration space R/sup 2//spl times/S/sup 1/ of a car-like robot, only moving forwards, with respect to the type of the length optimal paths. This kind of robot is subject to kinematic constraints on its path curvature and its orientation. Starting from the results on shortest paths, we give new optimality conditions on these paths, and compute the partition for any horizontal plane of the configuration space.>

Deals with the computation of the shortest path metric for a nonholonomic car-like robot. Bounds on the length d/sub RS/(o,c) of the shortest path linking the configuration o=(0,0,0) to any configuration c=(x,y,/spl theta/) .

Shows how to compute the region of accessibility for a model of a car that moves forwards and backwards with a lower bounded turning radius and a constant velocity. This problem deals with the computation of the shortest paths when the starting configuration i.e., position and direction, is completely specified and the goal is only defined by a position letting free the direction. The authors first exhibit the shortest path for any goal position and show its uniqueness. Then they compute the boundary of the region reachable by such paths. Such regions are useful in motion planning for a nonholonomic mobile robot.