Shared autonomy is the future of teleoperation as it reduces the teleoperator’s burden, enhances capabilities, and improves embodiment by offering seamless control of the robot. However, it remains rarely used, particularly with humanoid robots, as it faces numerous challenges. In this work, we introduce an innovative shared autonomy framework suitable for a wide range of robots, which we tested on a humanoid robot. This framework leverages Bayesian filtering over a Hidden Markov Model (HMM) to perform goal recognition, employing a landmark-based heuristic that minimizes computational demands while computing observation likelihoods without prior knowledge or a cost function. Once the teleoperator’s goal is identified, the robot assists according to its confidence level in the goal prediction. Assistance is provided by guiding the robot’s end-effector to reach a specified target position and orientation. In experiments with a diverse group of 10 teleoperators, conducted with video transmission delay, we achieved high accuracy in goal prediction and demonstrated significantly faster teleoperation time with shared autonomy.

We propose a novel walking control scheme based on the dynamics of the Linear Inverted Pendulum (LIP) model. The pattern generation incorporates a model of contact forces, enabling closed-loop control of the humanoid robot's state, including the Center of Mass (CoM) position, velocity, and Zero Moment Point (ZMP). No additional control policies are required to maintain static and dynamic balance. Our approach also includes dynamic re-planning of step locations and timings, thus preserving the LIP's boundedness condition. We validated this controller on five different humanoid robots, testing its robustness through various disturbances, including sudden pushes during walking and static phases. Additionally, our controller demonstrated effective locomotion over uneven and compliant terrain. Both simulation and experimental results confirm the effectiveness and robustness of this controller.

Anticipating a future scenario where the robot initiates its own actions and behaves voluntarily when collaborating with humans, our research focuses on human understanding and perception of unanticipated robot actions during physical human-robot interaction. While the current literature searches for key factors that make the human-robot collaboration successful, the question of how people experience the robot’s unanticipated action as cooperative or uncooperative seems to remain open. We designed a game-based experiment (N = 35) where the participant played a “catch-falling-coins” game by moving a robotic arm. Our experiment introduced unanticipated robot actions in an “active session” where the robot targeted higher-valued coins without first informing the participants. Through semi-structured interviews and statistical analysis of questionnaires (Big Five Personality Test, SAM, NARS and CH33), we examined the participants’ understanding of the robot’s “intention” and their positive or negative perception of the robot as cooperative or uncooperative. Among the participants who understood that the robot’s “intention” was to catch the higher-valued coins, the majority of them reported a positive perception of the robot (cooperative or helpful) while this was not the case among those who did not understand the robot’s intention. We also observed relevant relationships between some personality traits and a person’s understanding of the robot’s intention. Qualitative analysis of the interviews allowed us to structure the process of perception change during the game into three phases: confusion, investigation, and adaptation. We believe that our research contributes to the study of human perception, and particularly to the relationship between a human’s understanding of unanticipated robot actions and their positive or negative perception of the robot.

Unanticipated physical actions from the robot on humans [active physical human–robot interaction (pHRI)] may be inevitable with the deployment of robots in human-populated environments. However, it is still unclear how humans would perceive such actions and how the robot should execute them in a physically and psychologically safe manner. The objective of this article is to explore the possibility of quantifying the humans’ physical and mental state during an active physical interaction with a robot, by means of a laboratory experiment. We hypothesize that the active robot actions could cause measurable alterations in users’ data, which could be related to their perceptions and personalities. In the experiment, the user plays a visual game using the robot, which has a hidden task that results in active physical actions on the user. We collect data from physical and physiological sensors, and the perceptions and personalities via questionnaires and a semi-structured interview. Statistical analysis and clustering of the data collected from a total of 35 participants showed the relationships between participants’ physical and physiological data and their age, gender, perception, and personalities. Further developments based on these exploratory outcomes can be used to implement an active pHRI controller that can account for both the physical and the mental state of users.

Deep Convolutional Neural Networks (CNNs) have been successfully deployed on robots for 6-DoF object pose estimation through visual perception. However, obtaining labeled data on a scale required for the supervised training of CNNs is a difficult task - exacerbated if the object is novel and a 3D model is unavailable. To this end, this work presents an approach for rapidly generating real-world, pose-annotated RGB-D data for unknown objects. Our method not only circumvents the need for a prior 3D object model (textured or otherwise) but also bypasses complicated setups of fiducial markers, turntables, and sensors. With the help of a human user, we first source minimalistic labelings of an ordered set of arbitrarily chosen keypoints over a set of RGB-D videos. Then, by solving an optimization problem, we combine these labels under a world frame to recover a sparse, keypoint-based representation of the object. The sparse representation leads to the development of a dense model and the pose labels for each image frame in the set of scenes. We show that the sparse model can also be efficiently used for scaling to a large number of new scenes. We demonstrate the practicality of the generated labeled dataset by training a CNN based 6-DoF object pose estimator.

In this paper we report the safety-oriented framework for controlling the torque in the case of robots with high reduction gears and having no joint torque feedback. This kind of robots suffer from high joint friction and low backdrivability, requiring high gains and integral feedback, which can be dangerous. Our optimization-based framework includes feasibility and safety features borrowed from position control, and we introduce novel ones. We show how we limit the integral terms using a QP-based anti-windup which produces the optimal torque that maintains the best performances under safety limits. We show also a new controller for null-space compliance, providing strong guarantees of convergence in the task-space and ignoring the corresponding null-space where the robot can be moved freely. We validate these features with experiments on one 9 DoF arm of the robot HRP-5P performing a Cartesian task, and then a dual Cartesian / admittance task.

The simulation of static friction, and especially the effect of stiction, is cumbersome to perform in discrete-time due to its discontinuity at zero velocity and its switching behavior. However, it is essential to achieve reliable simulations of friction to develop compliant torque control algorithms, as they are much disturbed by this phenomenon. This paper takes as a base an elastoplastic model approach for friction, which is free from chattering and drift. It proposes two closed-form solutions that can be used to reliably simulate the effect of stiction consistently with the physics-based Stribeck model. These solutions consider the nonlinearity and velocity dependency, which are main characteristics of lubricated joints. One is directly inspired by the Stribeck nonlinear terms, and the other is a simplified rational approximation. The reliability of this simulation method is shown in simulation, where the consistency and stability are assessed. We also demonstrate the accuracy of these methods by comparing them to experimental data obtained from a robot joint equipped with a high gear reduction harmonic drive.

This paper focuses on a stabilization control for multi-contact motion which enables a humanoid robot to locomote by realizing dynamic contact transitions on non-flat environment. In the stabilization process of the multi-contact motion, the desired Zero-Moment Point (ZMP) is modified by the position of the Divergent Component of Motion (DCM) error with respect to the 3D Center of Mass (CoM) motion generated from the force distribution ratio. The contact wrench of each end-effector is determined by quadratic optimization considering the centroidal dynamics and contact friction constraints so as to satisfy the modified ZMP. Each end-effector is controlled by optimized force reference through a projection of null space by force distribution ratio. We propose a multi-contact stabilization framework which can be designed not only to generate 3D CoM motion but also the CoM position estimation and the optimal force distribution around the reference ZMP in a unified manner from a balance controller, by using the force distribution ratio. The effectiveness of proposed method is validated by a quadruped locomotion leaning against a vertical wall using the joint position controlled humanoid HRP-5P in a dynamic simulator.

The dynamics of a humanoid robot cannot be correctly described independently from the external forces acting on it. These forces have to be reconstructed to enable the robot to control them or to compensate for them. Force sensors are usually used to measure these forces, but because of their cost, they are often put only on the ankle/feet and possibly the wrists. This paper addresses the issue of the estimation of external forces and moments that apply at any part of a robot without direct force measurements and without torque measurements. The sensors used are the regular force sensors and the IMUs of the robot. The method relies on a model-based estimator able to make the fusion between these sensors and the whole body dynamics. The estimator reconstructs a single state vector containing the floating-base kinematics, a filtered measurement of contact force and an additional estimation external force that we evaluate in this paper. Validation is performed on HRP-2 in a multi-contact motion.

We propose a control framework for torque controlled humanoid robots that efficiently minimizes the tracking error in a Quadratic Programming (QP)formulated as multiobjective weighted tasks with constraints. It results in an optimal dynamically-feasible reference that can be tracked robustly, with exponential convergence, without joint torque feedback, in the presence of non modelled torque bias and low-frequency bounded disturbances. This is achieved by introducing integral gains in a Lyapunov-stable torque control, which exploit the passivity properties of the dynamical model of the robot and their effect on the dynamic constraints of the QP solver. The robustness of this framework is demonstrated in simulation by commanding our robot, the HRP-5P, to achieve simultaneously several objectives in the configuration and the Cartesian spaces, in the presence of non-modeled static and kinetic joint friction, as well as an uncertain torque scale.

To guarantee its balance, a humanoid robot has to respect some contact force constraints. Therefore, traditional controllers generate motions complying with these constraints, but they usually consider the robot as stiff and the joint position perfectly known. However, several robots contain compliant parts in their structure. This flexibility modifies the forces at contacts and endangers balance. However, most solutions to stabilize the robot rely on force sensors. But several humanoid robots aren't equipped with these sensors. This paper has two aims. The first one is to develop a compliance stabilizer using the center of mass position and upper-body orientation through a viscoelastic reaction mass pendulum model. The second objective is to show the performances of such a stabilizer when relying only on an IMU-based state observer. Experimental results on HRP-2 robot show that the stabilization successfully rejects perturbations with high gains using only these IMU signals. Moreover, the actuation of the upper-body orientation provides redundancy, robustness and finally improved performances to the stabilizer.

This paper discusses the problem of estimating the position of the center of mass for a polyarticulated system (e.g., a humanoid robot or a human body), which makes contact with its environment. The only sensors providing measurements on this point are either interaction force sensors or kinematic reconstruction applied to a dynamic model of the system. We first study the observability of the center-of-mass position using these sensors and we show that the accuracy domain of each measurement can be easily described through a spectral analysis. We finally introduce an original approach based on the theory of complementary filtering to efficiently merge these input measurements and obtain an estimation of the center-of-mass position. This approach is extensively validated in simulations by using a model of a humanoid robot through which we confirm the spectral analysis of the signal errors and show that the complementary filter offers a lower average reconstruction error than the classical Kalman filter. Some experimental applications of this filter on real signals are also presented.

A humanoid robot is underactuated and only relies on contacts with environment to move in the space. The ability to measure contact forces and torques enables then to predict the robot dynamics including balance. In classical cases, a humanoid robot is considered as a multi-body system with rigid limbs and joints and interactions with the environment are modeled as stiff contacts. Forces and torques at contacts are generally estimated with sensors which are expensive and sensitive to calibration errors. However, a robot is not perfectly rigid and contacts may have flexibilities. Therefore, external forces create geometric deformations of the body or its environment. These deformations may modify the robot dynamics and produce unwanted and unbalanced motions. Nonetheless, if we have a model of contact stiffness and are able to reconstruct reliably the geometric deformation, we can reconstruct forces and torques at contact. This study aims at estimating contact forces and torques and to observe the body kinematics of the robot with only an Inertial Measurements Unit (IMU). We show that we are able to reconstruct efficiently the position of the Center of Pressure (CoP) of the robot with only the IMU and proprioceptive data from the robot.

We propose a new bounding volume that achieves a tunable strict convexity of a given convex hull. This geometric operator is named sphere-tori-patches bounding volume (STP-BV), which is the acronym for the bounding volume made of patches of spheres and tori. The strict convexity of STP-BV guarantees a unique pair of witness points and at least C1continuity of the distance function resulting from a proximity query with another convex shape. Subsequently, the gradient of the distance function is continuous. This is useful for integrating distance as a constraint in robotic motion planners or controllers using smooth optimization techniques. For the sake of completeness, we compare performance in smooth and nonsmooth optimization with examples of growing complexity when involving distance queries between pairs of convex shapes.

Passivity-based walkers represent a model for human walking and a solution for low-energy locomotion for humanoid robots. The presence of an upper-body and even a head in this kind of systems is necessary as a better model for humans and to improve their usability. The benefits of these additions have never been studied, and no work experimented the addition of a head limb to a passivity-based walker. So, we aim, in this paper, to study the effects of the addition of these modifications on walkers with fully passive lower limbs. By comparing three systems (a passive compass, an upper-body stabilizing walker and a head stabilizing walker) simulations show that: (ii) upper-body stabilization improves the stability of the walking limit cycle; (ii) in return, the stabilization of the upper body requires a noticeable amount of the kinetic energy of the walker, and a significant energy supply (steeper slopes for the passive case) is necessary to guarantee the stability of the gait, especially for the case of head stabilization; and (iii) in a dynamical context, such as steep slopes, the upper-body and head stabilization have close performances for absorbing perturbations and smoothing the impacts, but with a slight advantage for the latter.

STP-BV is a bounding volume made of patches of spheres and toruses. These patches are assembled so that a convex polyhedral hull is bulged, in a tunable way, into a strictly convex form. Strict convexity ensures at least C1property of the distance function -and hence, its gradient continuity. STP-BV were introduced in our previous work [1], but proximity distance queries were limited to pairs of STP-BV covered objects. In this work we present an alternative to achieve fast proximity distance queries between a STP-BV object and any other convex shape. This is simply made by proposing a support mapping for STP-BV to be used with GJK algorithm [2] and its EPA extension to penetration cases [3]. Implementation and experiments of the proposed method and its performance are demonstrated with potential applications to robotics and computer graphics.