Sorting of parcels is a critical process in intralogistics for the proper processing and dispatching of packages. Commonly, such a process is manually executed by operators along the plant, without any added value, and might result in musculoskeletal injuries due to the non-ergonomic working conditions. Automation solutions are also present in the market and scientific literature. However, available solutions are usually implemented with pre-defined, simplified sorting rules/finite state machines capable of managing only a limited number of parcel types/sorting scenarios. To generalize and fully automate the sorting process in intralogistics, we propose to employ Reinforcement Learning (RL) for the derivation of sorting policies in combination with machine vision for the online tracking of the parcels, used as the state of the RL. More in detail, the on-policy Proximal Policy Optimization (PPO) algorithm is used for RL, and Yolo is chosen as the machine vision algorithm for parcel recognition and tracking. Based on the AMS sorting module of the SAIET Engineering company, a modular kinematic model (with parcels collision modeling) of the sorting system (an n by m AMS – i.e., 2-action actuators – matrix) is derived, and used as the environment for the PPO. Offline sorting policy training is performed by randomizing the parcel number, size, and entry positions. The trained policy is then deployed to the sorting module, which is equipped with cameras for machine vision implementation and performance evaluation. In-distribution and out-of-distribution (i.e., with parcel types not considered in the off-line training) tests achieved the target performance of 96.5% and 94% sorting accuracy, respectively.

As robotics gains popularity, interaction control becomes crucial for ensuring force tracking in manipulator-based tasks. Typically, traditional interaction controllers either require extensive tuning, or demand expert knowledge of the environment, which is often impractical in real-world applications. This work proposes a novel control strategy leveraging Neural Networks (NNs) to enhance the force-tracking behavior of a Direct Force Controller (DFC). Unlike similar previous approaches, it accounts for the manipulator's tangential velocity, a critical factor in force exertion, especially during fast motions. The method employs an ensemble of feedforward NNs to predict contact forces, then exploits the prediction to solve an optimization problem and generate an optimal residual action, which is added to the DFC output and applied to an impedance controller. The proposed Velocity-augmented Artificial intelligence Interaction Controller for Ambiguous Models (VAICAM) is validated in the Gazebo simulator on a Franka Emika Panda robot. Against a vast set of trajectories, VAICAM achieves superior performance compared to two baseline controllers.

Recent progress in robot autonomy and safety has significantly improved human-robot interactions, enabling robots to work alongside humans on various tasks. However, complex assembly tasks still present significant challenges due to inherent task variability and the need for precise operations. This work explores deploying robots in an assistive role for such tasks, where the robot assists by fetching parts while the skilled worker provides high-level guidance and performs the assembly. We introduce GEAR, a gaze-enabled system designed to enhance human-robot collaboration by allowing robots to respond to the user’s gaze. We evaluate GEAR against a touch-based interface where users interact with the robot through a touchscreen. The experimental study involved 30 participants working on two distinct assembly scenarios of varying complexity. Results demonstrated that GEAR enabled participants to accomplish the assembly with reduced physical demand and effort compared to the touchscreen interface, especially for complex tasks, maintaining great performance, and receiving objects effectively. Participants also reported enhanced user experience while performing assembly tasks. Project page:sites.google.com/view/gear-hri

Black-box optimization involves solving optimization problems where the objective function and/or constraints are unknown, inaccessible, or do not explicitly exist. In many applications, particularly those involving human interaction, the optimization problem can only be accessed through physical experiments, with the available outcomes based on the preference of one candidate over one or more others. Accordingly, algorithms for active preference learning have been developed to exploit this specific information in constructing a surrogate of the objective function. This surrogate is then used to define an acquisition function that suggests new decision vectors to search for the optimal solution iteratively. Based on this idea, our approach aims to extend active preference learning algorithms to leverage further information effectively, which can be obtained in reality, such as: a five-point Likert-type scale for the outcomes of the preference query (i.e., the preference can be described not only as “this is better than that” but also as “this is much better than that”), or multiple outcomes for a single preference query with possible additive information on how certain the outcomes are. The validation of the proposed algorithm is done through some standard benchmark functions, and, in practice, through tuning parameters for robot sealing and human–robot collaboration experiments, showing a promising improvement with respect to the state-of-the-art algorithm in the same context.

The landscape of Deep Learning has experienced a major shift with the pervasive adoption of Transformerbased architectures, particularly in Natural Language Processing (NLP). Novel avenues for physical applications, such as solving Partial Differential Equations and Image Vision, have been explored. However, in challenging domains like robotics, where high non-linearity poses significant challenges, Transformer-based applications are scarce. While Transformers have been used to provide robots with knowledge about high-level tasks, few efforts have been made to perform system identification. This paper proposes a novel methodology to learn a meta-dynamical model of a high-dimensional physical system, such as the Franka robotic arm, using a Transformer-based architecture without prior knowledge of the system’s physical parameters. The objective is to predict quantities of interest (end-effector pose and joint positions) given the torque signals for each joint. This prediction can be useful as a component for Deep Model Predictive Control frameworks in robotics. The meta-model establishes the correlation between torques and positions and predicts the output for the complete trajectory. This work provides empirical evidence of the efficacy of the in-context learning paradigm, suggesting future improvements in learning the dynamics of robotic systems without explicit knowledge of physical parameters. Code, videos, and supplementary materials can be found at project website.

Control techniques like MPC can realize contact-rich manipulation which exploits dynamic information, maintaining friction limits and safety constraints. However, contact geometry and dynamics are required to be known. This information is often extracted from CAD, limiting scalability and the ability to handle tasks with varying geometry. To reduce the need for a priori models, we propose a framework for estimating contact models online based on torque and position measurements. To do this, compliant contact models are used, connected in parallel to model multi-point contact and constraints such as a hinge. They are parameterized to be differentiable with respect to all of their parameters (rest position, stiffness, contact location), allowing the coupled robot/environment dynamics to be linearized or efficiently used in gradient-based optimization. These models are then applied for: offline gradient-based parameter fitting, online estimation via an extended Kalman filter, and online gradient-based MPC. The proposed approach is validated on two robots, showing the efficacy of sensorless contact estimation and the effects of online estimation on MPC performance. Video results can be seen at https://youtu.be/CuCTcmn3H-o.

Planning for contact-rich manipulation involves discontinuous dynamics, which presents challenges to planning methods. Sampling-based planners have higher sample complexity in high-dimensional problems and cannot efficiently handle state constraints such as force limits. Gradient-based solvers can suffer from local optima and their convergence rate is often worse on non-smooth problems. We propose a planning method that is both sampling- and gradient-based, using the Cross-entropy Method to initialize a gradient-based solver, providing better initialization to the gradient-based method and allowing explicit handling of state constraints. The sampling-based planner also allows direct integration of a particle filter, which is here used for online contact mode estimation. The approach is shown to improve performance in MuJoCo environments and the effects of problem stiffness and planing horizon are investigated. The estimator and planner are then applied to an impedance-controlled robot, showing a reduction in solve time in contact transitions to only gradient-based.

In industrial settings, robots are typically employed to accurately track a reference force to exert on the surrounding environment to complete interaction tasks. Interaction controllers are typically used to achieve this goal. Still, they either require manual tuning, which demands a significant amount of time, or exact modeling of the environment the robot will interact with, thus possibly failing during the actual application. A significant advancement in this area would be a high-performance force controller that does not need operator calibration and is quick to be deployed in any scenario. With this aim, this paper proposes an Actor-Critic Model Predictive Force Controller (ACMPFC), which outputs the optimal setpoint to follow in order to guarantee force tracking, computed by continuously trained neural networks. This strategy is an extension of a reinforcement learning-based one, born in the context of human-robot collaboration, suitably adapted to robot-environment interaction. We validate the ACMPFC in a real-case scenario featuring a Franka Emika Panda robot. Compared with a base force controller and a learning-based approach, the proposed controller yields a reduction of the force tracking MSE, attaining fast convergence: with respect to the base force controller, ACMPFC reduces the MSE by a factor of 4.35.

Physical human-robot collaboration is increasingly required in many contexts. To implement an effective collaboration, the robot should be able to recognize the human’s intentions and guarantee safe and adaptive behavior along the intended motion directions. The robot-control strategies with such attributes are particularly demanded in the industrial field. Indeed, with this aim, this work proposes a Q-Learning-based Model Predictive Variable Impedance Control (Q-LMPVIC) to assist the operators in physical human-robot collaboration (pHRC) tasks. A Cartesian impedance control loop is designed to implement decoupled compliant robot dynamics. The impedance control parameters (i.e., setpoint and damping parameters) are then optimized online in order to maximize the performance of the pHRC. For this purpose, an ensemble of neural networks is designed to learn the modeling of the human-robot interaction dynamics while capturing the associated uncertainties. The derived modeling is then exploited by the model predictive controller (MPC), enhanced with stability guarantees by means of Lyapunov constraints. The MPC is solved by making use of a Q-Learning method that, in its online implementation, uses an actor-critic algorithm to approximate the exact solution. Indeed, the Q-learning method provides an accurate and highly efficient solution (in terms of computational time and resources). The proposed approach has been validated through experimental tests, in which a Franka EMIKA panda robot has been used as a test platform.

This work addresses human intention identification during physical Human-Robot Interaction (pHRI) tasks to include this information in an assistive controller. To this purpose, human intention is defined as the desired trajectory that the human wants to follow over a finite rolling prediction horizon so that the robot can assist in pursuing it. This work investigates a Recurrent Neural Network (RNN), specifically, Long-Short Term Memory (LSTM) cascaded with a Fully Connected layer. In particular, we propose an iterative training procedure to adapt the model. Such an iterative procedure is powerful in reducing the prediction error. Still, it has the drawback that it is time-consuming and does not generalize to different users or different co-manipulated objects. To overcome this issue, Transfer Learning (TL) adapts the pre-trained model to new trajectories, users, and co-manipulated objects by freezing the LSTM layer and fine-tuning the last FC layer, which makes the procedure faster. Experiments show that the iterative procedure adapts the model and reduces prediction error. Experiments also show that TL adapts to different users and to the co-manipulation of a large object. Finally, to check the utility of adopting the proposed method, we compare the proposed controller enhanced by the intention prediction with the other two standard controllers of pHRI.

Physical human-robot collaboration is increasingly required in many contexts (such as industrial and rehabilitation applications). The robot needs to interact with the human to perform the target task while relieving the user from the workload. To do that, the robot should be able to recognize the human's intentions and guarantee safe and adaptive behavior along the intended motion directions. The robot-control strategies with such attributes are particularly demanded in the industrial field, where the operator guides the robot manually to manipulate heavy parts (e.g., while teaching a specific task). With this aim, this work proposes a Q-Learning-based Model Predictive Variable Impedance Control (Q-LMPVIC) to assist the operators in a physical human-robot collaboration (pHRC) tasks. A Cartesian impedance control loop is designed to implement a decoupled compliant robot dynamics. The impedance control parameters (i.e., setpoint and damping parameters) are then optimized online in order to maximize the performance of the pHRC. For this purpose, an ensemble of neural networks is designed to learn the modeling of the human-robot interaction dynamics while capturing the associated uncertainties. The derived modeling is then exploited by the model predictive controller (MPC), enhanced with the stability guarantees by means of Lyapunov constraints. The MPC is solved by making use of a Q-Learning method that, in its online implementation, uses an actor-critic algorithm to approximate the exact solution. Indeed, the Q-learning method provides an accurate and highly efficient solution (in terms of computational time and resources). The proposed approach has been validated through experimental tests, in which a Franka EMIKA panda robot has been used as a test platform. Each user was asked to interact with the robot along the controlled vertical z Cartesian direction. The proposed controller has been compared with a model-based reinforcement learning variable impedance controller (MBRLC) previously developed by some of the authors in order to evaluate the performance. As highlighted in the achieved results, the proposed controller is able to improve the pHRC performance. Additionally, two industrial tasks (a collaborative assembly and a collaborative deposition task) have been demonstrated to prove the applicability of the proposed solution in real industrial scenarios.

Production plants are being re-designed to implement human-centered solutions. Especially considering high added-value operations, robots are required to optimize their behavior to achieve a task quality at least comparable to the one obtained by the skilled operators. A manual programming and tuning of the manipulator is not an efficient solution, requiring to adopt towards automated strategies. Adding external sensors (e.g., cameras) increases the robotic cell complexity and it doesn’t solve the issue since it is usually difficult to build explicit reward functions measuring the robot performance, while it is easier for the user to define a qualitative comparison between two experiments. According to these needs, in this paper, the recently-developed preferences-based optimization approach GLISp is employed and adapted to tune the novel developed path-based velocity planner. The implemented solution defines an intuitive human-centered procedure, capable of transferring (through pairwise preferences between experiments) the task knowledge from the operator to the manipulator. A Franka EMIKA panda robot has been employed as a test platform to perform a robotic sealing task (i.e., material deposition task), validating the proposed methodology. The proposed approach has been compared with a programming by demonstration approach, and with the manual tuning of the path-based velocity planner. Achieved results demonstrate the improved deposition quality obtained with the proposed optimized path-based velocity planner methodology in a limited number of experimental trials (20).

Within the Industry 4.0 context, industrial robots need to show increasing autonomy. The manipulator has to be able to react to uncertainties/changes in the working environment, displaying a robust behavior. In this paper, a control framework is proposed to perform industrial interaction tasks in uncertain working scenes. The proposed methodology relies on two components: i) a 6D pose estimation algorithm aiming to recognize large and featureless parts; ii) a variable damping impedance controller (inner loop) enhanced by an adaptive saturation PI (outer loop) for high accuracy force control (i.e., zero steady-state force error and force overshoots avoidance). The proposed methodology allows to be robust w.r.t. task uncertainties (i.e. , positioning errors and interaction dynamics). The proposed approach has been evaluated in an assembly task of a side-wall panel to be installed inside the aircraft cabin. As a test platform, the KUKA iiwa 14 R820 has been used together with the Microsoft Kinect 2.0 as RGB-D sensor. Experiments show the reliability in the 6D pose estimation and the high-performance in the force-tracking task, avoiding force overshoots while achieving the tracking of the reference force.

Robots are increasingly exploited in production plants, with the need to learn and to adapt themselves to new tasks. This paper focuses on the investigation of machine learning techniques to make a sensorless robot able to learn and optimize an industrial assembly task. Relying on sensorless Cartesian impedance control, a task-trajectory learning algorithm exploiting a limited number of human's demonstrations (based on Hidden Markov Model), and an autonomous optimization procedure (based on Bayesian Optimization) are proposed to learn and optimize the assembly task. To validate the proposed methodology, an assembly task of a gear into its square-section shaft has been considered. A Franka EMIKA Panda manipulator has been used as a test platform. The experiments show the effectiveness of the proposed strategy, making the robot able to learn and optimize its behaviour to accomplish the assembly task, even in the presence of uncertainties.

Robots are nowadays increasingly required to deal with (partially) unknown tasks and situations. The robot has, therefore, to adapt its behavior to the specific working conditions. Classical control methods in robotics require manually programming all actions of a robot. While very effective in fixed conditions, such model-based approaches cannot handle variations, demanding tedious tuning of parameters for every new task. Reinforcement learning (RL) holds the promise of autonomously learning new control policies through trial-and-error. However, RL approaches are prone to learning with high samples, particularly for continuous control problems. In this paper, a learning-based method is presented that leverages simulation data to learn an object manipulation task through RL. The control policy is parameterized by a neural network and learned using modern Proximal Policy Optimization (PPO) algorithm. A dense reward function has been designed for the task to enable efficient learning of an agent. The proposed approach is trained entirely in simulation (exploiting the MuJoCo environment) from scratch without any demonstrations of the task. A grasping task involving a Franka Emika Panda manipulator has been considered as the reference task to be learned. The task requires the robot to reach the part, grasp it, and lift it off the contact surface. The proposed approach has been demonstrated to be generalizable across multiple object geometries and initial robot/parts configurations, having the robot able to learn and re-execute the target task.

The paper describes a human-robot cooperative installation methodology of heavy and bulky components based on marker-based visual servoing, force control, and human-robot cooperation. The main advance in the human-robot cooperation is achieved by a shared-control of the interaction during the installation task, relieving the human operator by the manipulated load and giving to the robot a partially autonomous behaviour in the force-tracking direction. Experimental results are shown in the context of the H2020 CleanSky 2 EURECA project in which a side-wall panel is installed in a 1:1 scale mock-up scenario of an A320 plane fuselage environment.

The paper focuses on industrial interaction robotics tasks, investigating a control approach involving multiples learning levels for training the manipulator to execute a repetitive (partially) changeable task, accurately controlling the interaction. Based on compliance control, the proposed approach consists of two main control levels: 1) iterative friction learning compensation controller with reinforcement and 2) iterative force-tracking learning controller with reinforcement. The learning algorithms rely on the iterative learning and reinforcement learning procedures to automatize the controllers parameters tuning. The proposed procedure has been applied to an automotive industrial assembly task. A standard industrial UR 10 Universal Robot has been used, equipped by a compliant pneumatic gripper and a force/torque sensor at the robot end-effector.

In the presented paper an analytically force overshoots-free approach is described for the execution of robotics interaction tasks involving a compliant (of unknown geometrical and mechanical properties) environment. Based on the impedance control, the aim of the work is to perform force-tracking applications avoiding force overshoots that may result in task failures. The developed algorithm shapes the equivalent stiffness and damping of the closed-loop manipulator to regulate the interaction dynamics deforming the impedance control set-point. The force-tracking performance are obtained defining the control gains analytically based on the estimation of the interacting environment stiffness (performed using an Extended Kalman Filter). The method has been validated in a probing task, showing the avoidance of force overshoots and the achieved target dynamic performance.

The impedance shaping control with robot base dynamics compensation is presented in this paper. The method has been conceived to avoid force overshoots in applications where the coupled dynamics of the global system (compliant robot base - controlled robot - interacting compliant environment) affects the force tracking task. Force tracking performance are obtained tuning on-line both the position set-point and the stiffness and damping parameters, based on the force error, the estimated stiffness of the interacting environment (an Extended Kalman Filter is used) and the estimated robot base position (a Kalman Filter is used). The stability of the presented strategy has been studied through Lyapunov. To validate the performance of the control an assembly task is taken into account, considering the geometrical and mechanical properties of the (partially) unknown environment. Results are compared with constant stiffness and damping impedance controllers, which show force overshoots and instabilities.

The impedance shaping control is presented in this paper, providing an extension of standard impedance controller. The method has been conceived to avoid force overshoots in applications where there is the need to track a force reference. Force tracking performance are obtained tuning on-line both the position setpoint and the stiffness and damping parameters, based on the force error and on the estimated stiffness of the interacting environment (an Extended Kalman Filter is used). The stability of the presented strategy has been studied through Lyapunov. To validate the performance of the control an assembly task is taken into account, considering the geometrical and mechanical properties of the environment (partially) unknown. Results are compared with constant stiffness and damping impedance controllers, which show force overshoots and instabilities.

The paper presents a control law for interaction tasks with environments of unknown geometrical and mechanical properties by manipulators mounted on compliant bases. Based on force-tracking impedance controls, the control strategy allows the execution of such class of tasks using the estimation of base position as a feedback in the control loop, requiring at the same time the on-line estimation of the environment stiffness. The properties of the control using non co-located sensors and the dynamic configuration of the coupled baserobot-environment system are studied. An Extended Kalman Filter is used for the estimation of the environment because of measurement uncertainties and errors in compound interaction model. The base is modelled as a second-order physical system with known parameters (offline identification before the task execution) and the base position is estimated from the measure of interaction forces. The grounding position estimation and the defined control law are validated in simulation and with experiments, especially dedicated to an insertion-assembly task. Control laws with and without the base compensation in the feedback loop are compared, verifying the effectiveness of the developed control law.

A deformation-tracking impedance control strategy is discussed for applications where a manipulator interacts with environments of unknown geometrical and mechanical properties, especially with stiffness comparable to a controlled robot stiffness. Based on force-tracking impedance controls, the deformation-tracking strategy allows the control of a desired deformation of the target environment, requiring the on-line estimation of the environment stiffness. An Extended Kalman Filter is used for the estimation of the environment because of measurement uncertainties and errors in compound interaction model. The tasks presented involve full body spatial interactions with a time-varying environment stiffness. The Extended Kalman Filter and the deformation-tracking impedance control are validated in simulation and with experiments. In particular, a cooperative assembly task is also performed with a human operator acting as varying environment, i.e. unpredictably changing the handling arm stiffness.