As robotics gains popularity, interaction control becomes crucial for ensuring force tracking in manipulator-based tasks. Typically, traditional interaction controllers either require extensive tuning, or demand expert knowledge of the environment, which is often impractical in real-world applications. This work proposes a novel control strategy leveraging Neural Networks (NNs) to enhance the force-tracking behavior of a Direct Force Controller (DFC). Unlike similar previous approaches, it accounts for the manipulator's tangential velocity, a critical factor in force exertion, especially during fast motions. The method employs an ensemble of feedforward NNs to predict contact forces, then exploits the prediction to solve an optimization problem and generate an optimal residual action, which is added to the DFC output and applied to an impedance controller. The proposed Velocity-augmented Artificial intelligence Interaction Controller for Ambiguous Models (VAICAM) is validated in the Gazebo simulator on a Franka Emika Panda robot. Against a vast set of trajectories, VAICAM achieves superior performance compared to two baseline controllers.

The rapid growth of the aging population in developed countries makes healthcare an important social challenge. In this context, service robots can play a key role. This work presents a software application for a service robot (TIAGo, PAL Robotics) implementing a motor-cognitive game. The activity combines cognitive and physical stimulation, a design that is relatively uncommon in literature. The embodied interaction design of the game distinguishes it from classical touchscreen-based games. In the game, the robot mimes letters with its arm, and the user has to recognize and then imitate them. The letter sequence increases in length each turn to train memory. User gestures are tracked using an ArUCo marker, and classified via a neural network. The application was tested on 10 young subjects and 4 community-dwelling older adults (82.3 ± 3.5 years). Recognition accuracy reached 92.2% and 80.5%, respectively, for young and older adults. Post-Session questionnaires highlighted high engagement and perceived usefulness, especially among older users who appreciated the memory and physical training aspects. This pilot project demonstrates the potential of integrating service robots into eldercare to support both patients and caregivers.

The landscape of Deep Learning has experienced a major shift with the pervasive adoption of Transformerbased architectures, particularly in Natural Language Processing (NLP). Novel avenues for physical applications, such as solving Partial Differential Equations and Image Vision, have been explored. However, in challenging domains like robotics, where high non-linearity poses significant challenges, Transformer-based applications are scarce. While Transformers have been used to provide robots with knowledge about high-level tasks, few efforts have been made to perform system identification. This paper proposes a novel methodology to learn a meta-dynamical model of a high-dimensional physical system, such as the Franka robotic arm, using a Transformer-based architecture without prior knowledge of the system’s physical parameters. The objective is to predict quantities of interest (end-effector pose and joint positions) given the torque signals for each joint. This prediction can be useful as a component for Deep Model Predictive Control frameworks in robotics. The meta-model establishes the correlation between torques and positions and predicts the output for the complete trajectory. This work provides empirical evidence of the efficacy of the in-context learning paradigm, suggesting future improvements in learning the dynamics of robotic systems without explicit knowledge of physical parameters. Code, videos, and supplementary materials can be found at project website.

The past few years have seen immense progress on two fronts that are critical to safe, widespread mobile robot deployment: predicting uncertain motion of multiple agents, and planning robot motion under uncertainty. However, the numerical methods required on each front have resulted in a mismatch of representation for prediction and planning. In prediction, numerical tractability is usually achieved by coarsely discretizing time, and by representing multimodal multi-agent interactions as distributions with infinite support. On the other hand, safe planning typically requires very fine time discretization, paired with distributions with compact support, to reduce conservativeness and ensure numerical tractability. The result is, when existing predictors are coupled with planning and control, one may often find unsafe motion plans. This paper proposes ZAPP (Zonotope Agreement of Prediction and Planning) to resolve the representation mismatch. ZAPP unites a prediction-friendly coarse time discretization and a planning-friendly zonotope uncertainty representation; the method also enables differentiating through a zonotope collision check, allowing one to integrate prediction and planning within a gradient-based optimization framework. Numerical examples show how ZAPP can produce safer trajectories compared to baselines in interactive scenes.

In industrial settings, robots are typically employed to accurately track a reference force to exert on the surrounding environment to complete interaction tasks. Interaction controllers are typically used to achieve this goal. Still, they either require manual tuning, which demands a significant amount of time, or exact modeling of the environment the robot will interact with, thus possibly failing during the actual application. A significant advancement in this area would be a high-performance force controller that does not need operator calibration and is quick to be deployed in any scenario. With this aim, this paper proposes an Actor-Critic Model Predictive Force Controller (ACMPFC), which outputs the optimal setpoint to follow in order to guarantee force tracking, computed by continuously trained neural networks. This strategy is an extension of a reinforcement learning-based one, born in the context of human-robot collaboration, suitably adapted to robot-environment interaction. We validate the ACMPFC in a real-case scenario featuring a Franka Emika Panda robot. Compared with a base force controller and a learning-based approach, the proposed controller yields a reduction of the force tracking MSE, attaining fast convergence: with respect to the base force controller, ACMPFC reduces the MSE by a factor of 4.35.

Physical human-robot collaboration is increasingly required in many contexts. To implement an effective collaboration, the robot should be able to recognize the human’s intentions and guarantee safe and adaptive behavior along the intended motion directions. The robot-control strategies with such attributes are particularly demanded in the industrial field. Indeed, with this aim, this work proposes a Q-Learning-based Model Predictive Variable Impedance Control (Q-LMPVIC) to assist the operators in physical human-robot collaboration (pHRC) tasks. A Cartesian impedance control loop is designed to implement decoupled compliant robot dynamics. The impedance control parameters (i.e., setpoint and damping parameters) are then optimized online in order to maximize the performance of the pHRC. For this purpose, an ensemble of neural networks is designed to learn the modeling of the human-robot interaction dynamics while capturing the associated uncertainties. The derived modeling is then exploited by the model predictive controller (MPC), enhanced with stability guarantees by means of Lyapunov constraints. The MPC is solved by making use of a Q-Learning method that, in its online implementation, uses an actor-critic algorithm to approximate the exact solution. Indeed, the Q-learning method provides an accurate and highly efficient solution (in terms of computational time and resources). The proposed approach has been validated through experimental tests, in which a Franka EMIKA panda robot has been used as a test platform.

This work addresses human intention identification during physical Human-Robot Interaction (pHRI) tasks to include this information in an assistive controller. To this purpose, human intention is defined as the desired trajectory that the human wants to follow over a finite rolling prediction horizon so that the robot can assist in pursuing it. This work investigates a Recurrent Neural Network (RNN), specifically, Long-Short Term Memory (LSTM) cascaded with a Fully Connected layer. In particular, we propose an iterative training procedure to adapt the model. Such an iterative procedure is powerful in reducing the prediction error. Still, it has the drawback that it is time-consuming and does not generalize to different users or different co-manipulated objects. To overcome this issue, Transfer Learning (TL) adapts the pre-trained model to new trajectories, users, and co-manipulated objects by freezing the LSTM layer and fine-tuning the last FC layer, which makes the procedure faster. Experiments show that the iterative procedure adapts the model and reduces prediction error. Experiments also show that TL adapts to different users and to the co-manipulation of a large object. Finally, to check the utility of adopting the proposed method, we compare the proposed controller enhanced by the intention prediction with the other two standard controllers of pHRI.

Multi-modal sensor fusion constitutes an essential ingredient for safe autonomous navigation. In the last years, many works have improved the accuracy of Deep-Learning-based odometry estimators. However, the robustness of these algorithms to sensor failure or measurement degradation, which are very likely to happen during navigation, has been studied less extensively. Furthermore, works studying the robustness of the fusion modules are developed without modeling the correlation between sensor features, which is crucial to filter out features derived from noisy measurements and in sensor faults scenarios. To bridge this gap, in this paper, we propose a fault-resistant odometry estimator, which produces robust estimates even when the sensors completely fail, or measurements progressively degrade. Our framework models the correlation between the sensor embedding using Message Passing Neural Network (MPNN), a particular type of Graph Neural Network (GNN). A mask is then computed from the updated node features of the graph to weigh the multi-modal features computed from different sensors. We evaluate the proposed fusion strategy on the modified raw KITTI dataset with sensor degradation scenarios. Finally, we compare against state-of-the-art baselines based on trivial features concatenation and soft-fusion to demonstrate our method’s superiority in terms of accuracy and robustness to sensor degradation and failures.

This paper presents a modeling and control frame-work for multibody flying robots subject to non-negligible aero-dynamic forces acting on the centroidal dynamics. First, aero-dynamic forces are calculated during robot flight in different operating conditions by means of Computational Fluid Dynamics (CFD) analysis. Then, analytical models of the aerodynamics coefficients are generated from the dataset collected with CFD analysis. The obtained simplified aerodynamic model is also used to improve the flying robot control design. We present two control strategies: compensating for the aerodynamic effects via feedback linearization and enforcing the controller robustness with gain-scheduling. Simulation results on the jet-powered humanoid robot iRonCub validate the proposed approach.

Jet-powered vertical takeoff and landing (VTOL) drones require precise thrust estimation to ensure adequate stability margins and robust maneuvering. Small-scale turbojets have become good candidates for powering heavy aerial drones. However, due to limited instrumentation available in these turbojets, estimating the precise thrust using classical techniques is not straightforward. In this paper, we present a methodology to accurately estimate the online thrust for the small-scale turbojets used on the iRonCub - an aerial humanoid robot. We use a grey-box method to capture the turbojet system dynamics with a nonlinear state-space model based on the data acquired from a custom engine test bench. This model is then used to design an extended Kalman filter that estimates the turbojet thrust only from the angular speed measurements. We exploited the parameter estimation algorithm to ensure that the EKF gives smooth and accurate estimates even at engine failures. The designed EKF was validated on the test bench where the mean absolute error in estimated thrust was found to be within 2% of rated peak thrust.

Physical human-robot collaboration is increasingly required in many contexts (such as industrial and rehabilitation applications). The robot needs to interact with the human to perform the target task while relieving the user from the workload. To do that, the robot should be able to recognize the human's intentions and guarantee safe and adaptive behavior along the intended motion directions. The robot-control strategies with such attributes are particularly demanded in the industrial field, where the operator guides the robot manually to manipulate heavy parts (e.g., while teaching a specific task). With this aim, this work proposes a Q-Learning-based Model Predictive Variable Impedance Control (Q-LMPVIC) to assist the operators in a physical human-robot collaboration (pHRC) tasks. A Cartesian impedance control loop is designed to implement a decoupled compliant robot dynamics. The impedance control parameters (i.e., setpoint and damping parameters) are then optimized online in order to maximize the performance of the pHRC. For this purpose, an ensemble of neural networks is designed to learn the modeling of the human-robot interaction dynamics while capturing the associated uncertainties. The derived modeling is then exploited by the model predictive controller (MPC), enhanced with the stability guarantees by means of Lyapunov constraints. The MPC is solved by making use of a Q-Learning method that, in its online implementation, uses an actor-critic algorithm to approximate the exact solution. Indeed, the Q-learning method provides an accurate and highly efficient solution (in terms of computational time and resources). The proposed approach has been validated through experimental tests, in which a Franka EMIKA panda robot has been used as a test platform. Each user was asked to interact with the robot along the controlled vertical z Cartesian direction. The proposed controller has been compared with a model-based reinforcement learning variable impedance controller (MBRLC) previously developed by some of the authors in order to evaluate the performance. As highlighted in the achieved results, the proposed controller is able to improve the pHRC performance. Additionally, two industrial tasks (a collaborative assembly and a collaborative deposition task) have been demonstrated to prove the applicability of the proposed solution in real industrial scenarios.

Nowadays, micro-mobility is one of the major global trends in cities for the innovation of the transport system. In this context a breakthrough introduction of electric kick scooters (e-scooters) has taken place. Unfortunately, these mobility systems cause several accidents mainly for two reasons: wrong use and inadequate safety requirements. Since e-scooters are quite a new type of vehicle, generally accepted mathematical models are yet to be developed. These models can be useful in understanding the dynamical properties of this type of vehicle thus improving its design to reduce riding accidents. The present paper presents a model for the simulation of the vertical dynamic behavior of e-scooters that accounts also for the mechanical impedance of the driver, thus allowing to estimate the overall driver’s comfort and road holding capabilities providing information on possible speed limitations in case of bad road conditions. Furthermore, the paper shows experimental envelope curves for lumped obstacles obtained with a dedicated test bench on which the e-scooter is fixed and tested under different conditions of vertical load and tire inflation pressure.

In this article, we describe an optimal control strategy for shaping a large-scale swarm of particles using boundary global actuation. This problem arises as a key challenge in many swarm robotics applications, especially when the robots are passive particles that need to be guided by external control fields. The system is large-scale and underactuated, making the control strategy at the microscopic particle level infeasible. We consider the Kolmogorov forward equation associated to the stochastic process of the single particle to encode the macroscopic behavior of the particles swarm. The control inputs shape the velocity field of the density dynamics according to the physical model of the actuators. We find the optimal actuation considering an optimal control problem whose state dynamics is governed by a linear parabolic advection–diffusion equation where the control induces a transport field. From a theoretical standpoint, we show the existence of a solution to the resulting nonlinear optimal control problem. From a numerical standpoint, we employ the discrete adjoint method to accurately compute the reduced gradient and we show how it commutes with the optimize-then-discretize approach. Finally, numerical simulations show the effectiveness of the control strategy in driving the density sufficiently close to the target.

Robots are nowadays increasingly required to deal with (partially) unknown tasks and situations. The robot has, therefore, to adapt its behavior to the specific working conditions. Classical control methods in robotics require manually programming all actions of a robot. While very effective in fixed conditions, such model-based approaches cannot handle variations, demanding tedious tuning of parameters for every new task. Reinforcement learning (RL) holds the promise of autonomously learning new control policies through trial-and-error. However, RL approaches are prone to learning with high samples, particularly for continuous control problems. In this paper, a learning-based method is presented that leverages simulation data to learn an object manipulation task through RL. The control policy is parameterized by a neural network and learned using modern Proximal Policy Optimization (PPO) algorithm. A dense reward function has been designed for the task to enable efficient learning of an agent. The proposed approach is trained entirely in simulation (exploiting the MuJoCo environment) from scratch without any demonstrations of the task. A grasping task involving a Franka Emika Panda manipulator has been considered as the reference task to be learned. The task requires the robot to reach the part, grasp it, and lift it off the contact surface. The proposed approach has been demonstrated to be generalizable across multiple object geometries and initial robot/parts configurations, having the robot able to learn and re-execute the target task.

With the implementation of reinforcement learning (RL) algorithms, current state-of-art autonomous vehicle technology have the potential to get closer to full automation. However, most of the applications have been limited to game domains or discrete action space which are far from the real world driving. Moreover, it is very tough to tune the parameters of reward mechanism since the driving styles vary a lot among the different users. For instance, an aggressive driver may prefer driving with high acceleration whereas some conservative drivers prefer a safer driving style. Therefore, we propose an apprenticeship learning in combination with deep reinforcement learning approach that allows the agent to learn the driving and stopping behaviors with continuous actions. We use gradient inverse reinforcement learning (GIRL) algorithm to recover the unknown reward function and employ REINFORCE as well as Deep Deterministic Policy Gradient algorithm (DDPG) to learn the optimal policy. The performance of our method is evaluated in simulation-based scenario and the results demonstrate that the agent performs human like driving and even better in some aspects after training.

The paper focuses on industrial interaction robotics tasks, investigating a control approach involving multiples learning levels for training the manipulator to execute a repetitive (partially) changeable task, accurately controlling the interaction. Based on compliance control, the proposed approach consists of two main control levels: 1) iterative friction learning compensation controller with reinforcement and 2) iterative force-tracking learning controller with reinforcement. The learning algorithms rely on the iterative learning and reinforcement learning procedures to automatize the controllers parameters tuning. The proposed procedure has been applied to an automotive industrial assembly task. A standard industrial UR 10 Universal Robot has been used, equipped by a compliant pneumatic gripper and a force/torque sensor at the robot end-effector.

The impedance shaping control with robot base dynamics compensation is presented in this paper. The method has been conceived to avoid force overshoots in applications where the coupled dynamics of the global system (compliant robot base - controlled robot - interacting compliant environment) affects the force tracking task. Force tracking performance are obtained tuning on-line both the position set-point and the stiffness and damping parameters, based on the force error, the estimated stiffness of the interacting environment (an Extended Kalman Filter is used) and the estimated robot base position (a Kalman Filter is used). The stability of the presented strategy has been studied through Lyapunov. To validate the performance of the control an assembly task is taken into account, considering the geometrical and mechanical properties of the (partially) unknown environment. Results are compared with constant stiffness and damping impedance controllers, which show force overshoots and instabilities.

The impedance shaping control is presented in this paper, providing an extension of standard impedance controller. The method has been conceived to avoid force overshoots in applications where there is the need to track a force reference. Force tracking performance are obtained tuning on-line both the position setpoint and the stiffness and damping parameters, based on the force error and on the estimated stiffness of the interacting environment (an Extended Kalman Filter is used). The stability of the presented strategy has been studied through Lyapunov. To validate the performance of the control an assembly task is taken into account, considering the geometrical and mechanical properties of the environment (partially) unknown. Results are compared with constant stiffness and damping impedance controllers, which show force overshoots and instabilities.