Achieving efficient remote teleoperation is particularly challenging in unknown environments, as the teleoperator must rapidly build an understanding of the site’s layout. Online 3D mapping is a proven strategy to tackle this challenge, as it enables the teleoperator to progressively explore the site from multiple perspectives. However, traditional online map-based teleoperation systems struggle to generate visually accurate 3D maps in real-time due to the high computational cost involved, leading to poor teleoperation performances. In this work, we propose a solution to improve teleoperation efficiency in unknown environments. Our approach proposes a novel, modular and efficient GPU-based integration between recent advancement in gaussian splatting SLAM and existing online map-based teleoperation systems. We compare the proposed solution against state-of-the-art teleoperation systems and validate its performances through real-world experiments using an aerial vehicle. The results show significant improvements in decision-making speed and more accurate interaction with the environment, leading to greater teleoperation efficiency. In doing so, our system enhances remote teleoperation by seamlessly integrating photorealistic mapping generation with real-time performances, enabling effective teleoperation in unfamiliar environments.Video: https://www.youtube.com/watch?v=-Md49rKkV8I(Code: https://github.com/ian-pge/GS_SLAM_teleoperation.git

A biped robot walking in a crowd must avoid falls and collisions at the same time. The latter is usually addressed through Passive Safety (PS), which guarantees that the robot is at rest when a collision is inevitable. Since PS may limit the robot's mobility, the purpose of this work is to introduce and explore the novel concept of Time To Danger (TTD) as an alternative. For a given robot motion, TTD is the time where the robot enters the region that a person can potentially occupy in the future. After having studied the properties of TTD, a novel locomotion strategy is proposed, which computes an optimal locomotion plan that guarantees balance preservation and TTD maximization, following a receding horizon Model Predictive Control scheme. Controlled experiments in a challenging simulated crowd scenario demonstrate how the novel locomotion strategy outperforms a Passive Safety-based locomotion strategy from a collision avoidance point of view.

We propose a combination of Model Predictive Control and Lexicographic Programming to address complex scenarios with conflicting goals related to various aspects of comfort and safety of passengers in a transit bus, generating different deceleration profiles depending on the speed of the bus and distance to obstacles, validated in experiments with a standard transit bus equipped with self-driving capabilities.

With new, safer manipulator robots, the probability of serious injury due to collisions with humans remains low (5%), even at speeds as high as 2 m.s-1. Collisions would better be avoided nevertheless, because they disrupt the tasks of both the robot and the human. We propose in this paper to equip robots with exteroceptive sensors and online motion generation so that the robot is able to perceive and react to the motion of the human in order to reduce the occurrence of collisions. It's impossible to guarantee that no collision will ever take place in a partially unknown dynamic environment such as a shared workspace, but we can guarantee instead that, if a collision takes place, the robot is at rest at the time of collision, so that it doesn't inject its own kinetic energy in the collision. To do so, we adapt a Model Predictive Control scheme which has been demonstrated previously with two industrial manipulator robots avoiding collisions while sharing their workspace. The proposed control scheme is validated in simulation.

We propose a linear MPC scheme for online computation of reactive walking motions, necessary for fast interactions such as physical collaboration with humans or collision avoidance in crowds. Unlike other existing schemes, it provides fully adaptable height, adaptable step placement and complete kinematic and dynamic feasibility guarantees, making it possible to walk perfectly safely on a piecewise horizontal ground such as stairs. A linear formulation is proposed, based on efficiently bounding the nonlinear term introduced by vertical motion, considering two linear constraints instead of one nonlinear constraint. Balance and Passive Safety guarantees are secured by enforcing a 3D capturability constraint. Based on a comparison between CoM and CoP trajectories involving exponentials instead of polynomials, this capturability constraint involves a CoM motion stopping along a segment of line, always maintaining complete kinematic and dynamic feasibility.

We control a biped robot moving in a crowd with a Model Predictive Control (MPC) scheme that generates stable walking motions, with automatic footstep placement. Most walking strategies propose to re-plan the walking motion to adapt to changing environments only once at every footstep. This is because a footstep is planted on the ground, it usually stays there at a constant position until the next footstep is initiated, what naturally constrains the capacity for the robot to react and adapt its motion in between footsteps. The objective of this paper is to measure if re-planning the walking motion more often than once at every footstep can lead to an improvement in collision avoidance when navigating in a crowd. Our result is that re-planning twice (or more) during each footstep leads to a significant reduction of the number of collisions when walking in a crowd, but depends on the density of the crowd.

This paper contributes to the field of physical human-robot collaboration. We present a complete control framework, which aims at making humanoid robots capable of carrying objects together with humans. First, we design a template identifying the primitive subtasks necessary for collaborative carrying. Then, these subtasks are formulated as constrained optimization problems for controlling the whole-body motion of a humanoid robot. The subtasks include two walking pattern generators that account for physical collaboration, as well as posture and grasping controllers. Finally, we validate our framework in a variety of collaborative carrying experiments, using the HRP-4 humanoid robot.

We want to enable the robot to reorient its feet in order to face its direction of motion. Model Predictive Control schemes for biped walking usually assume fixed feet rotation since adapting them online leads to a nonlinear problem. Nonlinear solvers do not guarantee the satisfaction of nonlinear constraints at every iterate and this can be problematic for the real-time operation of robots. We propose to define safe linear constraints that are always inside the intersection of the nonlinear constraints. We make simulations of the robot walking on a crowd and compare the performance of the proposed method with respect to the original nonlinear problem solved as a Sequential Quadratic Program.

We propose in this paper a new active perception scheme based on Model Predictive Control under constraints for generating a sequence of visual servoing tasks. The proposed control scheme is used to compute the motion of a camera whose task is to successively observe a set of robots for measuring their position and improving the accuracy of their localization. This method is based on the prediction of an uncertainty model (due to actuation and measurement noise) for determining which robot has to be observed by the camera. Simulation results are presented for validating the approach.

This paper is about the design of humanoid walking pattern generators to be used for physical collaboration. A particular use case is a humanoid robot helping a human to carry large and/or heavy objects. To do this, we construct a reduced model which takes into account physical interaction. This is used in a model predictive control framework to generate separate behaviors for being a follower or a leader. The approach is then validated both on simulation and on the HRP-4 humanoid robot.

A novel approach to minimum-time control is presented. It is stated in terms of a hierarchical optimization problem, which is standard in the field of robotics. This is advantageous as already existing tools can be used to approach its solution. Our formulation is applied to the online generation of trajectories for industrial robots performing pick and place operations in the presence of obstacles. Model predictive control is used in order to achieve reactive system behavior and to obtain accurate local approximations of the collision avoidance constraints (which are nonconvex). Our approach has the capacity to suppress high frequency chattering in the control signal in the presence of noise: a common drawback of aggressive control strategies. Experiment using two SCARA robots that share the same working environment is used to evaluate the presented approach.

One way to deal with occlusions or loss of tracking of the visual features used for visual servoing tasks is to predict the feature behavior in the image plane when the measurements are missing. Different prediction and correction methods have already been proposed in the literature. The purpose of this paper is to compare and experimentally validate some of these methods for eye-in-hand and eye-to-hand configurations. In particular, we show that a correction based both on the image and the camera/target pose provides the best results.

The goal of this paper is to present a real-time controller for an omnidirectional wheeled humanoid robot which can be strongly disturbed and tilt around its wheels. It is based on two linear model predictive controllers, managed by a tilt supervisor, which detects changes of the dynamic model caused by the tilt of the robot. Experimental results are proposed on the robot Pepper made by Aldebaran, showing good performance in term of stability an robustness.

Predicting the behavior of visual features on the image plane over a future time horizon is an important possibility in many different control problems. For example when dealing with occlusions (or other constraints such as joint limits) in a classical visual servoing loop, or also in the more advanced model predictive control schemes recently proposed in the literature. Several possibilities have been proposed to perform the initial correction step for then propagating the visual features by exploiting the measurements currently available by the camera. But the predictions proposed so far are inaccurate in situations where the depths of the tracked points are not correctly estimated. We then propose in this paper a new correction strategy which tries to directly correct the relative pose between the camera and the target instead of only adjusting the error on the image plane. This correction is then analysed and compared by evaluating the corresponding improvements in the feature prediction phase.

The article presents an improved formulation of an existing model predictive control scheme used to generate online "stable" walking motions for a humanoid robot. We introduce: (i) a change of variable that simplifies the optimiza tion problem to be solved; (ii) a simply bounded formulation in the case when the positions of the feet are predetermined; (iii) a formulation allowing foot repositioning (when the system is perturbed) based on ℓ1- and ℓ∞-norm minimization; (iv) a formulation that accounts for (approximate) double support constraints when foot repositioning occurs.

This article presents a comparison between dense and sparse model predictive control (MPC) formulations, in the context of walking motion generation for humanoid robots. The former formulation leads to smaller, the latter one to larger but more structured optimization problem. We put an accent on the sparse formulation and point out a number of advantages that it presents. In particular, motion generation with variable center of mass (CoM) height, as well as variable discretization of the preview window, come at a negligible additional computational cost. We present a sparse formulation that comprises a diagonal Hessian matrix and has only simple bounds (while still retaining the possibility to generate motions for an omnidirectional walk). Finally, we present the results from a customized code used to solve the underlying quadratic program (QP).

This paper examines the control continuity in hierarchical task-space controllers. While the continuity is ensured for any a priori fixed number of tasks -even in ill-conditioned configurations-, the control resulting from a hierarchical stack-of-task computation may not be continuous under some discrete events. In particular, we study how the continuity of the stack-of-task control computation is affected under discreet scheduling operations such as on-the-fly priority switching between tasks, or tasks insertion and removal, which changes the number of tasks in the stack controller. Different ways to formulate a hierarchy of tasks are presented together with their continuity properties, which is thoroughly analyzed under such discreet scheduling operations.

Redundant mechanical systems like humanoid robots are designed to fulfill multiple tasks at a time. A task, in velocity-resolved inverse kinematics, is a desired value for a function of the robot configuration that can be regulated with an ordinary differential equation (ODE). When facing simultaneous tasks, the corresponding equations can be grouped in a single system or, better, sorted in priority and solved each in the solutions set of higher priority tasks. This elegant framework for hierarchical task regulation has been implemented as a sequence of least-squares problems. Its limitation lies in the handling of inequality constraints, which are usually transformed into more restrictive equality constraints through potential fields. In this paper, we propose a new prioritized task-regulation framework based on a sequence of quadratic programs (QP) that removes the limitation. At the basis of the proposed algorithm, there is a study of the optimal sets resulting from the sequence of QPs. The algorithm is implemented and illustrated in simulation on the humanoid robot HRP-2.

Classically, the inverse kinematics is performed by computing the singular value decomposition of the matrix to invert. This enables a very simple writing of the algorithm. However, the computation cost is high, especially when applied to complex robots and complex sets of constraints (typically around 5ms for 50 degrees of freedom - DOF). In this paper, we propose a dedicated adaptation of quadratic programming that enables fast computations of the hierarchical inverse kinematics (around 0.1ms for 50 DOF). We then extend this algorithm to deal with unilateral constraints, obtaining sufficiently high performances for reactive control.

This paper introduces a visual servoing scheme for humanoid walking. Though most of the existing approaches follow a perception-decision-action scheme, we close the loop so that the control is robust to model error. Our approach is based on a new reactive pattern generator which modifies, at the control level, the footsteps, the center of mass and the center of pressure trajectories for the center of mass to track a reference velocity. And, in this paper, the reference velocity is directly given by a visual servoing control law. Since, the HRP-2 walk induces a sway motion that disturbs the regulation of the visual control law, we introduce a control law allowing convergence in the image space and taking into account this sway motion.

We demonstrate in this paper our motion generation sheme for the generation of stable bipedal walking motions and we expand it to enhance its flexibility and independency. An algorithm for the control of appropriate orientations of the feet and the trunk permits the robot to turn in a natural and safe way. Polygonal constraints on the positions of the computed feet positions serve to improve its reliability. A logic for the succession of the support phases and an algorithm for the automatic control of their orientations bridge the gap to more autonomy and to more practicability.

This article addresses the fast solution of a Quadratic Program underlying a Linear Model Predictive Control scheme that generates walking motions. We introduce an algorithm which is tailored to the particular requirements of this problem, and therefore able to solve it efficiently. Different aspects of the algorithm are examined, its computational complexity is presented, and a numerical comparison with an existing state of the art solver is made. The approach presented here, extends to other general problems in a straightforward way.

A novel approach is presented which aims at building autonomously visual models of unknown objects, using a humanoid robot. Previous methods have been proposed for the specific problem of the next-best-view during the modeling and the recognition process. However our approach differs as it takes advantage of humanoid specificities in terms of embedded vision sensor and redundant motion capabilities. In a previous work, another approach to this specific problem was presented which relies on a derivable formulation of the visual evaluation in order to integrate it with our posture generation method. However to get rid of some limitations we propose a new method, formulated using two steps: (i) an optimization algorithm without derivatives is used to find a camera pose which maximizes the amount of unknown data visible, and (ii) a whole robot posture is generated by using a different optimization method where the computed camera pose is set as a constraint on the robot head.

We present a novel method for prioritizing both linear equality and inequality systems and provide one algorithm for its resolution. This algorithm can be summarized as a sequence of optimal resolutions for each linear system following their priority order. We propose an optimality criterion that is adapted to linear inequality systems and characterize the resulting optimal sets at every priority level. We have successfully applied our method to plan local motions for the humanoid robot HPR-2. We will demonstrate the validity of the method using an original scenario where linear inequality constraints are solved at lower priority than equality constraints.

This article addresses the real-time implementation issues of a model predictive control based walking pattern generation for a humanoid robot. We approximate the multibody dynamic model with a linear discrete time system, and at each step solve a quadratic program in order to keep the output within a predefined set of constraints. The focus is on creating an efficient framework for forming and solving the underlying optimization problem. For that purpose we develop: a) a reliable guess for the active constraints at optimality; b) a fast way of generating an initial feasible point with respect to the set of constraints for each preview interval; c) a variable discretization sampling time. A simple implementation of a standard primal active set algorithm which exploits a "hot start" is used to demonstrate the advantages of the first point, while the latter one is verified using an existing dual solver.

Building on previous propositions to generate walking gaits online through the use of linear model predictive control, the goal of this paper is to show that it is possible to allow on top of that a continuous adaptation of the positions of the foot steps, allowing the generation of stable walking gaits even in the presence of strong perturbations, and that this additional adaptation requires only a minimal modification of the previous schemes, especially maintaining the same linear model predictive form. Simulation results are proposed then on the HRP-2 humanoid robot, showing a significant improvement over the previous schemes.

The problem of safe locomotion of legged and wheeled robots, when trying to avoid falling, tipping over or hitting obstacles, appears to be a problem of viability and not of Lyapunov stability. Theoretically speaking, viability and model predictive control are unquestionably related, but both can quickly lead to untractable numerical problems. We present here a promising approach for the problem of avoiding to fall in the case of legged locomotion that elegantly solves this difficulty. We propose then a brief discussion about what makes this approach successful with respect to the approaches proposed for the other problems where viability is at stake. This paper should be considered therefore mostly as a prospective reflection on the general problem of safe robotic locomotion.

We propose here to deal with the optimization of velocity profiles of manipulator robots with a minimum time criterion subject to thermal constraints. This paper deals with the real impact of thermal limitations on optimal velocity profiles and the methods to calculate the corresponding optimal trajectories. We first calculate analytically the optimal solution in a simple case in order to verify the validity of the numerical algorithm and also to present a general methodology to calculate optimal trajectories in robotics using results from the theory of calculus of variations and not from the theory of optimal control. We derive then a numerical algorithm based on the discretization of the time law through an interpolation with non uniform cubic splines. This algorithm shows robust and efficient convergence properties and the trajectories thus generated were executed successfully on a Staubli Rtimes90

This paper deals with optimizing the task cycle time of industrial robots integrated in complex robot cells. Trajectory optimizers are usually based on models and can't properly deal with uncertainties due to interactions between the robot and its environment. We propose here a trajectory optimizer with hardware in the loop which can take into account constraints such as maximum authorized temperature and maximum authorized torque. Our approach is based on unconstrained optimization algorithms without derivatives and penalty methods. Experiments on real industrial applications showed good robustness properties of this algorithm even with a high number of parameters and with changes of the robot task

The new generation of implanted neuroprostheses allows muscles to be controlled with fine accuracy, high selectivity and the repeatability of the muscle's response can be achieved. Thus, the closed loop control of such systems becomes possible. The SUAW project succeeded in the implantation of an advanced neuroprosthetic device on two patients, but the movement generation remains open loop and is tuned empirically. Nevertheless, the good results obtained give us the opportunity to envisage the system evolves towards closed loop control and automatic synthesis of the stimulation patterns generating the desired movement. To achieve this goal, some preliminary researches have to be carried out, beginning with a specific modelling that can be used in the context of functional electrical stimulation (FES). The main issues concern muscle modelling including the interaction with the skeleton, fatigue, FES parameters as inputs, and the identification of dynamic parameters, and afterwards, the motion synthesis and the closed loop control based on this model. Besides, the scientific approach is the same as in robotics so that the theoretical tools used in the control theory are the same and directly applicable. This paper describes the results obtained in the previous project SUAW and how we attempt, through the new project DEMAR, to enhance the global performances of the system.

This paper deals with an approach to carry out the transitions of a walking robot. A set of elementary transformations is proposed to modify the locomotion parameters allowing one to obtain various kinds of walks. Some simulation results are given.