Most prior works on communication in multi-agent reinforcement learning have focused on emergent communication, which often results in inefficient and non-interpretable systems. Inspired by the role of language in natural intelligence, we investigate how grounding agents in a human-defined language can improve the learning and coordination of embodied agents. We propose a framework in which agents are trained not only to act but also to produce and interpret natural language descriptions of their observations. This language-augmented learning serves a dual role: enabling efficient and interpretable communication between agents, and guiding representation learning. We demonstrate that language-augmented agents outperform emergent communication baselines across various tasks. Our analysis reveals that language grounding leads to more informative internal representations, better generalization to new partners, and improved capability for human-agent interaction. These findings demonstrate the effectiveness of integrating structured language into multi-agent learning and open avenues for more interpretable and capable multi-agent systems.1 1 Appendices and code are available at github.com/MToquebiau/LAMARL

Recent advances in AI have led to significant results in robotic learning, but skills like grasping remain partially solved. Many recent works exploit synthetic grasping datasets to learn to grasp unknown objects. However, those datasets were generated using simple grasp sampling methods using priors. Recently, Quality-Diversity (QD) algorithms have been proven to make grasp sampling significantly more efficient. In this work, we extend QDG-6DoF, a QD framework for generating object-centric grasps, to scale up the production of synthetic grasping datasets. We propose a data augmentation method that combines the transformation of object meshes with transfer learning from previous grasping repertoires. The conducted experiments show that this approach reduces the number of required evaluations per discovered robust grasp by up to 20 %. We used this approach to generate QDGset, a dataset of 6 DoF grasp poses that contains about 3.5 and 4.5 times more grasps and objects, respectively, than the previous state-of-the-art. Our method allows anyone to easily generate data, eventually contributing to a large-scale collaborative dataset of synthetic grasps.

Fine dexterous manipulation requires reactive control based on rich sensing of manipulator-object interactions. Tactile sensing arrays provide rich contact information across the manipulator’s surface. However their implementation faces two main challenges: accurate force estimation across complex surfaces like robotic hands, and integration of these estimates into reactive control loops. We present a data-efficient calibration method that enables rapid, full-array force estimation across varying geometries, providing online feedback that accounts for non-linearities and deformation effects. Our force estimation model serves as feedback in an online closed-loop control system for interaction force tracking. The accuracy of our estimates is independently validated against measurements from a calibrated force-torque sensor. Using the Allegro Hand equipped with Xela uSkin sensors, we demonstrate precise force application through an admittance control loop running at 100Hz, achieving up to 0.12±0.08 [N] error margin—results that show promising potential for dexterous manipulation.

Robotic grasping refers to making a robotic system pick an object by applying forces and torques on its surface. Many recent studies use data-driven approaches to address grasping, but the sparse reward nature of this task made the learning process challenging to bootstrap. To avoid constraining the operational space, an increasing number of works propose grasping datasets to learn from. But most of them are limited to simulations. The present paper investigates how automatically generated grasps can be exploited in the real world. More than 7000 reach-and-grasp trajectories have been generated with Quality-Diversity (QD) methods on 3 different arms and grippers, including parallel fingers and a dexterous hand, and tested in the real world. Conducted analysis on the collected measure shows correlations between several Domain Randomization-based quality criteria and sim-to-real transferability. Key challenges regarding the reality gap for grasping have been identified, stressing matters on which researchers on grasping should focus in the future. A QD approach has finally been proposed for making grasps more robust to domain randomization, resulting in a transfer ratio of 84% on the Franka Research 3 arm.

Recent advances in AI have led to significant results in robotic learning, including natural language-conditioned planning and efficient optimization of controllers using generative models. However, the interaction data remains the bottleneck for generalization. Getting data for grasping is a critical challenge, as this skill is required to complete many manipulation tasks. Quality-Diversity (QD) algorithms optimize a set of solutions to get diverse, high-performing solutions to a given problem. This paper investigates how QD can be combined with priors to speed up the generation of diverse grasps poses in simulation compared to standard 6-DoF grasp sampling schemes. Experiments conducted on 4 grippers with 2-to-5 fingers on standard objects show that QD outperforms commonly used methods by a large margin. Further experiments show that QD optimization automatically finds some efficient priors that are usually hard coded. The deployment of generated grasps on a 2-finger gripper and an Allegro hand shows that the diversity produced maintains sim-to-real transferability. We believe these results to be a significant step toward the generation of large datasets that can lead to robust and generalizing robotic grasping policies.

In this paper, we propose a meaningful definition of rotational centroidal orientation which is somewhat missed in the state-of-the-art centroidal momentum and dynamics theory for locomotion robots with one floating base. This centroidal instantaneous orientation rotates as the robot runs, and it is extracted from the total system angular inertia. The new centroidal frame is proposed to be parallel with the principal axes of the centroidal angular inertia, which can describe the whole-robot rotational motion. To avoid high fluctuations of centroidal frame orientation parameters between adjacent control loops, we develop one algorithm to enable the centroidal instantaneous frame to be smooth. The relationship between the centroidal angle rate and the centroidal angular velocity is derived, as well as the relationship in the acceleration level, which can be used for whole-body torque control. The new centroidal orientation or Euler angle is verified by two-scenario simulations, and another scenario is used to track and control the centroidal angular motion in the first-order kinematics level. The idea has considerable potential for system design, motion generation, and torque control in robotics communities with different research topics and theoretical backgrounds.

This paper focuses on local path planning for obstacle avoidance tasks dedicated to off-road mobile robots. This approach calculates a new local path for the vehicle using a set of cubic Bezier curves once the safety distance is not respected; otherwise, the vehicle follows the global reference path which is defined off-line. Two basic steps are used to determine this new path. Firstly, some significant points that should belong to the planned path are extracted on-line according to the obstacle's sizes and the current state of the vehicle, these points are approved as waypoints. Secondly, on-line cubic Bezier curves are computed to create a smooth path for these points such that the safety and lateral stability of the vehicle are ensured (i.e., preventing huge curvatures and wide-variation in steering angles). This path will be used as a reference to be performed by the vehicle using a constrained model predictive control. The validation of our navigation strategy is performed via numerical simulations and experiments using a fast double-steering rover.

Path tracking controllers for an autonomous vehicle are often designed by using either a dynamic model or a kinematic one and some models are related to wheel-ground contact, that makes the efficiency of the controller highly dependent on the ground parameters estimation, especially for off-road mobile robots intended to navigate in open environments. This paper proposes a new nonlinear observer designed to estimate the front and rear contact cornering stiffnesses in real time, that are related both on tire and soil proprieties. The latter is estimated using steering angles as well as yaw rate and lateral velocity, which are provided by a preliminary Kalman-Bucy observer. The performance of the proposed nonlinear observer combined with the LQR controller is evaluated by both advanced simulations and experiments in real conditions at different speeds.

This paper develops a new constrained model predictive control for a dynamic path tracking of an off-road mobile robot with a double steering axle. The controller is based on a dynamic model that includes wheel-ground lateral slippage and terrain geometry parameters. It is formulated as an optimization problem that computes at each time-step the optimal front and rear steering angles required to perform a desired path, with respect to multiple constraints, essentially the steering joint limits and the tire adhesion area bounds (i.e., pseudo-sliding zone limits). The capabilities of such a path tracking controller are shown and discussed through numerical simulations and experiments on a real off-road mobile robot at different speeds.

Generic control of wheel-on-leg robots on arbitrary uneven terrains is a challenging task due to the complexity of the robot dynamics, surface interactions, and environmental structures. This paper deals with the control of a wheel-on-leg robot with passive and active internal compliance that enables estimation of wheel-ground interaction forces. The proposed method is based on a continuous state space Q-learning approach that uses the contact forces estimates to learn, through trial and error, the appropriate control policy from a set of predefined behaviors. Without any prior knowledge of the ground geometry, the robot is able to react to unanticipated obstacles. The learned policy proves to be generic and allows the robot to negotiate complex obstacles that had not been considered during learning phase.

Swing-up and throwing tasks for underactuated manipulators are examples of dynamic motions that exhibit highly nonlinear coupling dynamics. One of the key ingredients for such complex behaviors is motion coordination to exploit their passive dynamics. Despite the existence of powerful tools such as nonlinear trajectory optimization, they are usually treated as blackboxes that provide local optimal trajectories. We introduce the Dynamical Coupling Map (DCM), a novel graphical technique, to help gain insight into the output trajectory of the optimization and analyze the capability of underactuated robots. The DCM analysis is demonstrated on the swing up motion of a simplified model of a gymnast on high bar. The DCM shows in a graphical and intuitive way the pivotal role of exploiting the nonlinear inertial forces to reach the unstable equilibrium configuration while taking into account the torque bounds constraints. In this paper, we present the DCM as a posteriori analysis of a local optimal trajectory, found by employing the direct collocation trajectory optimization framework.

In this paper, we propose the use of compliant elements in the actuation of a wheel-legged robot in order to improve its locomotion properties on unknown and irregular terrains. Detection of the obstacles is achieved by a synergistic use of the structural compliances. The robot's capabilities to surmount steep obstacles is thus improved thanks to the inertia of the chassis and flexibility in postural control. In the proposed robot's kinematics, the four wheels are attached to the main body through vertical series elastic actuators (SEA) and with a passive horizontal compliant mechanism subject to a specific wheel speed control. The overall control relies on postural servoing and a local reactive loop which adapts the vertical forces applied by the SEA on each wheel according to the detected obstacle and the stability margin. The resulting system is evaluated with physical simulations for two case studies: a canonical steep obstacle on one wheel at a time and multiple random rough terrains.

Off-road operational conditions require large sus- pension displacements and a significant clearance between the ground and the main frame, yielding to an elevated position of the vehicle mass center. Consequently, this makes the vehicle more likely to turn over when cornering fast. This paper proposes a new design, and its associated control, of an active device which improves the stability of fast rover moving up to 10 m/s. The proposed design can be equipped on any off-road chassis which has independent suspensions. We propose the using of an active anti-roll system allowing the control of the roll angle and thus improving the vehicle stability, especially when turning or when moving on slopping ground. The proposed system increases the controllability of the vehicle, by giving access to the roll angle which is usually uncontrollable. We develop a model based predictive controller for the roll dynamics, which minimizes the load transfer during cornering and the energy consumed by the actuators. The control model is based on a dynamic model of the rover and on a stability criteria defined by the lateral load transfer. Dynamic simulation, carried out for different rover trajectories with different speeds, show the benefit of the proposed active system and the validity of the control approach.

The paper provides a method to determine and compare the reconfigurability of lattice systems. First it shows the difference that exists between the reconfigurability and self-reconfigurability features of a lattice system. Then a method using displacement groups is introduced to characterize these features. Based on this method, these features are then compared for some existing lattice systems.

This paper is focused on the problem of accurate and reliable path tracking control of a 4-wheels car-like mobile robot moving off-road at high speed. Dynamic and extended kinematic models that take into account the effects of wheel skidding are presented. Based on the extended kinematic model, an adaptive and predictive controller for path tracking is derived. This control law is combined to a stabilization algorithm of yaw motion, based on the dynamic model and the modulation of driven wheel forces. The overall control architecture is experimentally evaluated on a slipping terrain. Results demonstrate enhanced performances as the robot succeed in following the path at high speed, accurately and without loss of control.

The paper develops a method for analyzing and improving by control obstacle clearance capacities of articulated multi-wheeled rovers. On uneven ground surface, load and traction force distributions through the wheel/ground contact system are highly coupled. They are both conditioned by the global equilibrium of the mechanical system and the contact stability constraints. The optimal traction force distribution problem is formulated here as a convex optimization problem using Linear Matrix Inequalities (LMIs). Velocity and force transmissions in articulated multi-wheeled mobile robots are introduced under a generic form decomposed in task, joint and contact levels. A tyre-model is used for the evaluation of the robustness of the solution with respect to slippage phenomena. Simulation results show that the traction distribution forces which is so determined lead to a significant increase in obstacle clearance capacities compared to an usual velocity control technique.

We identify three fundamental properties of lattice robots such as (1) discreteness (2) translational symmetry and (3) composition, and explain the analogy of lattice robots kinematics and crystal symmetry described by space groups. Then we give the possible connectors symmetries and orientations compatible with space groups, and the possible sliding and hinge joints locations and orientations compatible with the displacements in such groups. We present a framework for the design of lattice robots by assembling compatible joints and connectors into a space group and give a 3D example.

This paper concerns the control of an autonomous high mobility wheel-legged rover crossing uneven terrains. A new control strategy, using active redundancies of the robot, leads to elaborate a posture control based on the potential field approach of the stability measurement. Then a decoupled posture and trajectory control algorithm based on the velocity model of the robot is proposed. Last, simulation results showing performance of the control algorithm are presented.

This paper presents a generic kinematic modeling approach for articulated multi-monocycle mobile robots. The formulation proposed to deduce the input/output velocity equations for such kinematic structures is an extension of the reciprocal screw based method of asymmetrical and constrained parallel mechanisms. The efficiency of this methodology for setting up the differential kinematic model is illustrated through application: the RobuRoc mobile robot. Its complex kinematic structure is first transformed into a spatial parallel mechanism which encapsulates the differential driving wheels system. Then, the analytical form of the reciprocal screw system which corresponds to the actively controlled wrenches applied on the controlled body is established. Reciprocally, it describes the way the wheel velocities are transferred to the output body. It also provides a geometrical information for an exhaustive singularity analysis and traction distribution optimization during the evolution of the system on highly irregular surfaces. From the differential kinematic model, the concept of traction ellipsoid is introduced for evaluating quantitatively the obstacle clearance capabilities when the configuration of the system and the contact conditions are highly variable

This work presents some general considerations on self-reconfigurable robots design, and proposes an original design of mechatronic modules. Geometrical and kinematical features of these modules, offer the ability to be used as well as wheels to produce rolling motion, and as joints for building kinematic chains as legs, arms or snakes.

This paper addresses the control of a hybrid wheel-legged system evolving on rough terrain. First, the posture and trajectory parameters are introduced. Then, a decoupled posture and trajectory control algorithm based on the velocity model of the robot is proposed. Last, the performance and feasibility of the control algorithm are evaluated through simulations and experiments with the Hylos robot.

In order to study the human being locomotion system, a multi-degrees of freedom (DOFs) biped prototype equipped with flexible feet, which is named ROBIAN, is developed. In this paper, the conceptual design of the biped prototype ROBIAN is discussed. The features of ROBIAN include: (1) parallel mechanism at the hip and the ankle, (2) modular design, and (3) 1-DOF active/passive joint between the heel and the toe. ROBIAN has 18-DOFs in total: 6-DOFs for each leg, 1-DOF passive or active joint for each foot and 4-DOFs for the upper limb. One of the major application of ROBIAN is the efficient development of a real testbed of active/passive prosthesis for the disabled.

In aim to optimize the control for an off-road robot and specially a robot with a crawling motion mode named peristaltical motion, it is essential to understand robot-soil interaction. We present an efficient method for simulating this interaction. This method can simulate the dynamic behavior of a 6-wheeled/3-axles Marsokhod type robot on different kinds of soils. With this simulation, we have defined the necessary conditions for using the peristaltical motion on sand or loose soil.

This paper is devoted to mechanical and control analysis of steering systems of fast wheeled robots. Steering performances are analysed by means of a formulation of a path-tracking control problem. The mechanical model takes into account the wheel-sliding and wheel-elasticity which are relevant to the steering behaviour. When the path is a straight line or a circular arc, the tracking offsets are small and the assigned velocity is constant, the proposed tracking scheme leads to a controller using direct state feedback and linear quadratic regulation approach. Four different steering ways are examined, they combine steer and drive actions on a four wheeled robots.

The authors address the navigation of wheeled vehicles over the rough terrains. They present a system for simulating vehicle motions over 3-D terrain which considers the kinematics of the locomotion mechanism, the physics of the interaction between the wheels and the ground being obtained by integration of some basic aspects of terramechanics. This system is exploited to search for paths which guarantee that the vehicle remains maneuverable and stable and rolls without sliding. Paths are mapped by deforming a nominal path joining a set of subgoal points. The choice of the best path can based on different performance criteria, e.g., covered distance, energy consumption, and risk factor.