Collaborative robots (cobots) are democratizing industrial automation with their user-friendly programming approaches. Nevertheless, the Blockly-like interfaces typically available on cobots still require the user to define the program logic flow. Recent advancements in robotics research provide the robotic system with the reasoning capabilities given by symbolic artificial intelligence. This way, the cobot can acquire a new skill from a user demonstration, understand its semantics, and use symbolic planning for grounding and sequencing. Such methodologies rely on a symbolic description of the scene that should adequately represent how the cobot’s actions modify the environment. The symbols employed in the literature, however, either lack descriptive accuracy or are too specific for the targeted task, resulting in the application of the proposed teaching methodologies only to simple scenarios. This paper addresses these issues by introducing a methodology for symbolically describing general-purpose spatial relations between entities in a workspace, enhancing the flexibility and the range of application of cobots symbolic reasoning for complex manipulation tasks. The proposed approach involves defining a tunable set of predicates for relative positions and orientations, enabling precise symbolic representations, necessary for real-world tasks. The adoption of these symbols into a Programming by Demonstration framework empowers non-expert users to teach skills and deploy cobots in complex industrial tasks without coding. Experimental results demonstrate the effectiveness of this method, showing that first-time users can deploy cobots for a complex machine tending task comprising parts reorientations. • Application of semantic-informed Programming by Demonstration for industrial tasks. • Definition of tunable, general-use quantitative symbols for relative positions. • Development of a strategy for representing relative orientations symbolically. • Validation through a user study on a machine-tending task with part reorientations.

Object manipulation without relying on complex fixtures remains a largely unresolved issue in industrial robotics, being generally limited to pick-and-place operations of easy to grasp objects. This work presents an adaptable manipulation planning algorithm for dual-arm robots, aiming to reorient an object from an initial position to a specified goal configuration without the need of external fixtures. Our approach integrates a precomputed regrasp graph with an online optimal handover planner that transforms the high-level sequence searched from the graph into executable grasp and handover poses. This approach reduces the overall graph complexity and enhances planning efficiency by merging high-level optimal sequence planning with the execution of predetermined motion primitives. The proposed algorithm is validated using different types of objects and a collaborative dual-arm robot. We also propose a comparison between our method and two benchmark approaches on a bin picking use case, to demonstrate how our pipeline improves the task efficiency.

In Human-Robot Collaboration, safety mechanisms such as Speed and Separation Monitoring and Power and Force Limitation dynamically adjust the robot’s speed based on human proximity. While essential for risk reduction, these mechanisms introduce slowdowns that makes cycle time estimation a hard task and impact job scheduling efficiency. Existing methods for estimating cycle times or designing schedulers often rely on predefined safety models, which may not accurately reflect real-world safety implementations, as these depend on case-specific risk assessments. In this paper, we propose a deep learning approach to predict the robot’s safety scaling factor directly from process execution data. We analyze multiple neural network architectures and demonstrate that a simple feed-forward network effectively estimates the robot’s slowdown. This capability is crucial for improving cycle time predictions and designing more effective scheduling algorithms in collaborative robotic environments.

This paper addresses the optimization of human-robot collaborative work-cells before their physical deployment. Most of the times, such environments are designed based on the experience of the system integrators, often leading to sub-optimal solutions. Accurate simulators of the robotic cell, accounting for the presence of the human as well, are available today and can be used in the pre-deployment. We propose an iterative optimization scheme where a digital model of the work-cell is updated based on a genetic algorithm. The methodology focuses on the layout optimization and task allocation, encoding both the problems simultaneously in the design variables handled by the genetic algorithm, while the task scheduling problem depends on the result of the upper-level one. The final solution balances conflicting objectives in the fitness function and is validated to show the impact of the objectives with respect to a baseline, which represents possible initial choices selected based on the human judgment.

Physics-based simulations and learning-based models are vital for complex robotics tasks like deformable object manipulation and liquid handling. However, these models often struggle with accuracy due to epistemic uncertainty or the sim-to-real gap. For instance, accurately pouring liquid from one container to another poses challenges, particularly when models are trained on limited demonstrations and may perform poorly in novel situations. This paper proposes an uncertainty-aware Monte Carlo Tree Search (MCTS) algorithm designed to mitigate these inaccuracies. By incorporating estimates of model uncertainty, the proposed MCTS strategy biases the search to-wards actions with lower predicted uncertainty. This approach enhances the reliability of planning under uncertain conditions. Applied to a liquid pouring task, our method demonstrates improved success rates even with models trained on minimal data, outperforming traditional methods and showcasing its potential for robust decision-making in robotics.

The collaboration between humans and robots re-quires a paradigm shift not only in robot perception, reasoning, and action, but also in the design of the robotic cell. This paper proposes an optimization framework for designing collaborative robotics cells using a digital twin during the pre-deployment phase. This approach mitigates the limitations of experience-based sub-optimal designs by means of Bayesian optimization to find the optimal layout after a certain number of iterations. By integrating production KPIs into a black-box optimization frame-work, the digital twin supports data-driven decision-making, reduces the need for costly prototypes, and ensures continuous improvement thanks to the learning nature of the algorithm. The paper presents a case study with preliminary results that show how this methodology can be applied to obtain safer, more efficient, and adaptable human-robot collaborative environments.

This work demonstrates the utility of dual-arm robots with dual-wrist force-torque sensors in manipulating a Deformable Linear Object (DLO) within an unknown environment that imposes constraints on the DLO’s movement through contacts and fixtures. We propose a strategy to estimate the pose of unknown environmental contacts encountered during the manipulation of a DLO, classifying the induced constraints as unilateral, bilateral and fully constrained, exploiting the redundancy of force sensors. A semantic approach to define environmental constraints is introduced and incorporated into a graph-based model of the DLO. This model remains accurate as long as the DLO is under tension and is dynamically updated throughout the manipulation process, built by sequencing a set of primitives. The estimation strategy is validated through simulations and real-world experiments, demonstrating its potential in handling DLOs under various, possibly uncertain, constraints.

This paper tackles the complex task of routing elastic deformable linear objects (DLOs) characterized by considerable stiffness, such as cables or hoses, which are already constrained at both ends. Specifically, a single arm robot is controlled to slide along the unknown contour of the cable, performing collision-free contour following, and to insert specific DLO segments into intermediate known clips. The contour following motion is executed avoiding both collisions with static obstacles and excessive deformation of the manipulated DLO. In particular, the path is defined considering an artificial potential field that is updated after each sliding motion along the DLO. This field accounts for static obstacles, the local cable shape (reconstructed using tactile sensors on the gripper fingertips) and the estimation of the global DLO shape obtained from a dynamic model of the DLO, accounting for the constraints imposed by the clips and the gripper. The proposed method is experimentally validated on an industrial robot executing cable routing in several DLO configurations.

Today, intelligent robotic manufacturing systems are reshaping the production industry. Using robots as actuators, multi-source sensors for perception, and Artificial Intelligence (AI) as decision-making systems, they can perform routine manufacturing tasks, surpassing the capabilities of traditional hard-programmed Computer Numerical Control (CNC) machinery. One specific challenge in footwear manufacturing is sole deburring, traditionally done manually by skilled workers. This paper focuses on developing a robust path-planning pipeline, comprising vision-based and Learning from Demonstrations (LfD) modules for autonomous deburring of soles. The vision-based module exploits Deep Learning (DL) techniques to handle key challenges such as precise segmentation of different soles types across diverse scenarios despite potential occlusions. Additionally, a novel method for burrs identification has been developed leveraging image processing and optimization techniques. Determining the optimal cutting tool orientation during sole deburring relies on human experience. The LfD module aims to impart this knowledge to the robot from videos of expert demonstrations, requiring adaptability to every new incoming sole that needs deburring. Experimental results showcase the method’s performance and flexibility, underlining the potential to advance the field of the proposed approach.

This paper proposes a set of tactile based skills to perform robotic cable routing operations for deformable linear objects (DLOs) characterized by considerable stiffness and constrained at both ends. In particular, tactile data are exploited to reconstruct the shape of the grasped portion of the DLO and to estimate the future local one. This information is exploited to obtain a grasping configuration aligned to the local shape of the DLO, starting from a rough initial grasping pose, and to follow the DLO's contour in the three-dimensional space. Taking into account the distance travelled along the arc length of the DLO, the robot can detect the cable segments that must be firmly grasped and inserted in intermediate clips, continuing then to slide along the contour until the next DLO's portion, that has to be clipped, is reached. The proposed skills are experimentally validated with an industrial robot on different DLOs in several configurations and on a cable routing use case.

This paper presents an effective approach to enable performance improvement in human-robot collaboration scenarios. The problem is tackled from the perspective of speed and separation monitoring principle, which stems from the recently instituted safety standard. The proposed approach attempts to seek for performance gains, measured by the speed-up of the production cycle, without compromising the safety constraints consistent with the standard. The approach is based on the notion of braking surface - an abstraction of the swept volume described by the manipulator during braking motion. We address two types of braking behavior: general and path-consistent. In both cases, the braking surface can be evaluated in a receding horizon manner. The robot velocity is continuously scaled such that, in case of a controlled stop, the corresponding volume spanned by the robot (braking surface) does not interfere with the surrounding obstacles. The approach is entirely kinematic and does not require the knowledge of the robot's dynamic model. Simulation study indicates that the pro-posed approach offers performance improvements compared to other state of the art methods. Moreover, the experiments demonstrate the real-time applicability of the method with the real robot in human-shared environment.

This paper deals with the challenging task of picking semi-deformable linear objects (SDLOs) from a bin. SDLOs are deformable elements, such as cables, joined to a rigid part as a connector. We propose a vision-based strategy to detect, classify and estimate the pose and the state (free or occluded) of connectors belonging to an unspecified number of SDLOs, arranged in an unknown configuration in the bin. The connectors can then be grasped and manipulated by a dual-arm robot through a set of manipulation primitives. In this way, a single SDLO can be extracted from the bin and laid on the worktable. A subsequent association between the connectors and the extracted SDLOs is performed, allowing to firmly grasp a SDLO at its ends to further manipulate it. The procedure is tested in bin picking operations with several kinds of SDLOs and is applied to a use case involving a collaborative wire harnesses assembly task.

This paper deals with safe human-robot collaboration in the context of speed and separation monitoring paradigm. The core of the approach is to continuously track the separation distance between the robot and the human. The robot speed is then adjusted according to the perceived distance so that it will be able to stop before eventually come into contact with the human. We present an approach that aims at maximizing the productivity of the robot, i.e., its speed, while keeping the prescribed safety requirements satisfied. The method is based on explicit representation of danger zones – regions around the robot, where safety requirements are violated. The motion is then generated such that the robot moves as fast as possible, while its danger zone still does not collide with human operators. The approach is validated within an experimental study. Note to Practitioners—This article was motivated by the problem of maximizing productivity of the robotic manipulator while ensuring the safety of human collaborator. The increase in productivity is achieved by a faster traversal of predefined paths without compromising the safety of the human, which is specifically defined by industrial standard. The approach requires limited knowledge on robot’s dynamical properties. More precisely, we only need the braking time as a “lumped” representation of robot’s inertia. The underlying optimization problem is conveniently resolved by introducing danger zones that allow for intuitive visualization and geometrical representation of the regions around the robot that must be avoided. On the other hand, the method assumes the representation of humans via typical geometric primitives, which can be obtained using of-the-shelf depth perception systems. The solution to the problem reduces to a repeated collision checking between danger zones and the human. Such an approach turns out to be suitable for real-time implementation due to availability of fast and efficient collision checking algorithms/libraries.

The perception and autonomous manipulation of clothes by robots is an ongoing research topic that is attracting a lot of contributions. We consider the application of handling garments for laundry in this work. A framework for loading a washing machine with clothes placed initially inside a box is presented. Our framework is created in a modular way to account for the sub-problems associated with the full process. We extend our grasping point estimation algorithm by finding multiple grasping points and defining a score to select one. Active contours segmentation is added to the algorithm as well for more robust clustering of the image. Model of the washing machine is used to create a motion plan for the robot to place the clothes inside the drum. A new module is added for detection of items fallen outside the drum so to plan corresponding corrective action. We use ROS, depth and 2D cameras and the Doosan A0509 robot for experiments.

This paper presents a novel probabilistic approach to deep robot learning from demonstrations (LfD). Deep move-ment primitives (DMPs) are deterministic LfD model that maps visual information directly into a robot trajectory. This paper extends DMPs and presents a deep probabilistic model that maps the visual information into a distribution of effective robot trajectories. The architecture that leads to the highest level of trajectory accuracy is presented and compared with the existing methods. Moreover, this paper introduces a novel training method for learning domain-specific latent features. We show the superiority of the proposed probabilistic approach and novel latent space learning in the real-robot task of strawberry harvesting in the lab. The experimental results demonstrate that latent space learning can significantly improve model prediction performances. The proposed approach allows to sample trajectories from distribution and optimises the robot trajectory to meet a secondary objective, e.g. collision avoidance.

The advent of collaborative robots has revolutionised the work concept for Small Medium Enterprises (SME), introducing quick line configuration changes and allowing a human operator to work together with a robot. Several problems are still present regarding the need to reprogram the robotic collaborator. This paper exploits the concept of Virtual Reality to instruct the robotic co-worker, allowing a non-skilled operator to teach a task effortlessly and straightforwardly. Each Skill composing the task is classified and characterised through its pre and postconditions to let the robot stop and notify the operator in case of inability to complete the scheduled action. Moreover, the system allows the operator to interact with the robot and be proactive during the task execution.

In this work, we address two crucial issues that arise in the design of a human-robot collaborative station for the assembly of products: the optimal task allocation and the scheduling problem. We propose an offline method to solve in series the two mentioned issues, considering a static allocation and taking into account several features such as the minimization of postural discomfort, operation processing times, idle times and hence the total cycle time. Our methodology consists of a mixed approach that combines a capability-based method, where the agents' capabilities are tested against a list of predefined criteria, with optimization. In particular, we formulate a modified version of the Hungarian Algorithm to solve also unbalanced assignment problems, where the number of tasks is different from the number of agents. The scheduling policy is obtained by means of a Mixed Integer Linear Programming (MILP) formulation, with a multi-objective optimization. Moreover, the concepts of operation, assembly tree and precedence graph are formalized, since they represent the inputs to our method, together with the information on the workstation layout and on the selected kind of robot. Finally, the proposed solution is applied to a case study to define the optimal task allocation and scheduling for two different workstation layouts: the results are compared and the best layout is accordingly selected.

Dynamic Movement Primitives (DMPs) are a well-known tool for encoding robotic motions. Their popularity stems from invariance properties in time and space, the ability to describe complex coordinated motions in multiple degrees of freedom with a relatively small number of parameters, and the linearity in the parameters that describe the motion. The latter allows easily fitting a DMP to motions e.g. demonstrated by a human. DMPs are at their core second order autonomous differential equations. However, feedforward controls of robots with flexible joints are known to require reference trajectories up to the fourth derivative of position. Consequently, classical DMPs are mechanically not compatible with flexible joint robots. In this paper, we propose an extension of DMPs by introducing FlexDMPs. This concept retains the structural properties and benefits of classical DMPs but generates trajectories up to the fourth derivative that can theoretically be tracked ideally (i.e. with zero tracking error) by flexible joint robots. The concept is demonstrated on a high fidelity simulation model of an industrial robot and in experimental results on a collaborative manipulator.

Production plants are being re-designed to implement human-centered solutions. Especially considering high added-value operations, robots are required to optimize their behavior to achieve a task quality at least comparable to the one obtained by the skilled operators. A manual programming and tuning of the manipulator is not an efficient solution, requiring to adopt towards automated strategies. Adding external sensors (e.g., cameras) increases the robotic cell complexity and it doesn’t solve the issue since it is usually difficult to build explicit reward functions measuring the robot performance, while it is easier for the user to define a qualitative comparison between two experiments. According to these needs, in this paper, the recently-developed preferences-based optimization approach GLISp is employed and adapted to tune the novel developed path-based velocity planner. The implemented solution defines an intuitive human-centered procedure, capable of transferring (through pairwise preferences between experiments) the task knowledge from the operator to the manipulator. A Franka EMIKA panda robot has been employed as a test platform to perform a robotic sealing task (i.e., material deposition task), validating the proposed methodology. The proposed approach has been compared with a programming by demonstration approach, and with the manual tuning of the path-based velocity planner. Achieved results demonstrate the improved deposition quality obtained with the proposed optimized path-based velocity planner methodology in a limited number of experimental trials (20).

The novel paradigm of collaborative automation, with machines and industrial robots that synergically share the same workspace with human workers, requires to rethink how activities are prioritized in order to account for possible variabilities in their durations. This article proposes a scheduling method for collaborative assembly tasks that allows to optimally plan assembly activities based on the knowledge acquired during runtime and so adapts to variations along the life cycle of a manufacturing process. The scheduler is based on time Petri nets and the output plan is optimized by minimizing the idle time of each agent. The experimental validation carried out on a realistic industrial use-case consisting of a small assembly line with two robots and a human operator confirms the effectiveness of the approach.

In manufacturing, kitting is the process of grouping separate items together to be supplied as one unit to the assembly line. This is a key logistic task, which is usually performed manually by human operators. However, picking objects from the warehouse implies a great repetitiveness in arm motion. Moreover, the weight and position of items may increase the physical strain and induce the development of work-related musculoskeletal disorders. The inclusion of a collaborative robot in the process may help to reduce the operator's effort and increase productivity. This paper introduces an online scheduling algorithm to guide the picking operations of the human and the robot. The proposed approach has been experimentally evaluated and compared with an offline scheduler, as well as with the baseline case of manual kitting.

In this paper we address variable admittance control for human-robot physical interaction in manual guidance applications. In the proposed solution, the parameters of the admittance filter can change not only as a function of the current state of motion (i.e. whether the human guiding the robot ia accelerating or decelerating) but also with reference to a predefined goal position. The human is in fact gently guided towards the goal along some curved paths, where the damping is conveniently scaled in order to accommodate the motion towards the goal position. The algorithm also allows the human to reach goals that he/she cannot directly see because for example the transported object is bulky and obstructs the worker view. The performance of the proposed controller are evaluated by means of point to point cooperative motions with multiple volunteers using an ABB IRB140 robot.

Prediction of the human behaviour is essential for allowing an efficient human-robot collaboration. This was confirmed recently showing how scheduling approaches can significantly increase the productivity of a robotic cell by planning the robotic actions in a way as much as possible compliant with the human predicted behaviour. This work proposes an innovative approach for human activity prediction, exploiting both a-priori information and knowledge revealed during operation. The resulting approach is proved to achieve good performance through both off-line simulated sequences and in a realistic co-assembly involving a human operator and a dual arm collaborative robot.

A crucial problem in human-robot collaboration is to achieve seamless coordination among the agents. Robots have to adapt to human behaviour, which is highly uncertain. In fact, humans can perform each task in many ways and with different speeds, occasional errors and short pauses. This paper offers a robust method to monitor the advancement of the current human activity in real-time in order to predict its duration. The algorithm learns online templates of new variants of the task and uses them as references for a Dynamic Time Warping-based algorithm. The proposed strategy has been tested within a realistic assembly task. Results show its ability to give accurate predictions also in case of peculiar variants, such as those associated with errors.

In a collaborative scenario, robots working side by side with humans might rely on vision sensors to monitor the activity of the other agent. When occlusions of the human body occur, both the safety of the cooperation and the performance of the team can be penalized, since the robot could receive incorrect information about the ongoing cooperation. In this work, we propose a novel particle filter algorithm that, by merging the data acquired through a RGB-D camera and a MR headset, estimates online the human wrist position. This algorithm allows to significantly reduce the uncertainty of the human pose estimation, in case of both static and dynamic occlusions. To this purpose, the proposed particle filter is integrated with a detailed virtual model of the real workspace. Moreover, additional constraints describing the boundaries of the motion of the human upper body are included in a virtualized framework. The results showed that the proposed technique entails significant improvements, determining a relevant reduction of the estimation error and of the uncertainty of the estimate.

In this paper, we approach the problem of ensuring safety requirements within human-robot collaborative scenarios. The safety requirements considered herein are consistent with the paradigm of speed and separation monitoring. In such a setup, safety guarantees for human operators usually imply limited robot velocities and/or significant distance margins, which in turn may have adverse effects regarding the productivity of the robot. In this paper, we propose a novel approach that minimally affects the productivity while being consistent with such a safety prescription. A comprehensive simulation study shows that our method outperforms the current state of the art algorithm.

In this paper, the process of modelling and identification of a hydraulic actuator is discussed. In this framework a simple model based on the classical theory has been derived and a first experimental campaign has been performed on a test bench. These tests highlighted the presence of unmodeled phenomena (e.g. dead-zone, hysteresis, etc.), therefore a second and more extensive experimental campaign has been done. With the acquired knowledge an improved model has been developed and its parameters identified. Finally several experimental tests have been performed in order to validate the model.

The coexistence of humans and robots in the future production plants is one of the pillars of Industry 4.0. Humans and robots will collaborate to accomplish common tasks in order to mutually compensate their deficiencies. In recent years, many efforts have been spent to develop safe motion planning strategies, designed to prevent robots from injuring humans. Most of the previous techniques are classifiable as reactive, since the considered motion controllers impose some local corrective actions in order to dodge the space occupied by the human. In this paper, a proactive approach is adopted, optimizing robotic paths according to a prediction of the volume occupied by the human when collaborating with the robot. The validity of the approach is shown in a realistic use-case involving the collaboration of a human operator with a 7 degrees robotic arm, the ABB YuMi.

Letting humans and robots share a common space for collaboration is considered a consolidated practice. The trajectories followed by the robot must be safe for the human mate, especially when the robot holds dangerous tools or parts. At the same time, the productivity must be preserved, without imposing too restrictive limitations on the robot's movements. This article proposes the use of Gaussian Processes to predict the motion of an operator in a robotic cell, with the aim of controlling the robot speed and avoid collisions. An adaptive approach is proposed and the model for the human motion is persistently re-updated. The resulting approach will be demonstrated to be less conservative than previous ones, while at the same time to preserve the safety of the operator. Real experiments have been conducted on the 7 d.o.f. ABB YuMi robot.

Rapidly Random exploring Trees are popular algorithms in the field of motion planning. A feasible path connecting two different poses is found by incrementally building a tree data structure. They are powerful and flexible, but also computationally intense, requiring thousands of iterations before their termination. The aim of this article is to show the capabilities of MT-RRT, a general purpose library which exploits four different multithreading strategies to speed up the planning process of Rapidly Random exploring Trees. MT-RRT will be proved to significantly reduce the computation time on various benchmarks.

In collaborative robotics applications, human behaviour is a major source of uncertainty. Predicting the evolution of the current human activity might be beneficial to the effectiveness of task planning, as it enables a higher level of coordination of robot and human activities. This paper addresses the problem of monitoring the advancement of human tasks in real-time giving an estimate of their expected duration. The proposed method relies on dynamic time warping to align the current activity with a reference template. No training phase is required, as the prototypical execution is learnt online from previous instances of the same activity. The applicability and performance of the method within an industrial context have been verified on a realistic assembly task.

Collaborative robotics refers to the cooperation between humans and machines and aims at improving productivity and at facilitating the worker in demanding tasks. The advantage of collaboration is to combine the superior cognitive and motor skills of the operator with the physical capabilities of the robots. This work presents a control strategy for the robotic manipulator to minimise the muscular fatigue of the human operator during the manipulation of bulky objects. The robot moves the workpiece so that the human is always operating close to his/her most natural and ergonomic posture. This way the risk of developing postures and movements inducing musculoskeletal disorders is minimised. A substantial reduction of the amplitude of the operator movements, without degrading the precision in fulfilling the task, has been registered in the experimental campaign.

It is widely agreed that future manufacturing environments will be populated by humans and robots sharing the same workspace. However, the real collaboration can be sporadic, especially in the case of assembly tasks, which might involve autonomous operations to be executed by either the robot or the human worker. In this scenario, it might be beneficial to predict the actions of the human in order to control the robot both safely and efficiently. In this paper, we propose a method to predict human activity patterns in order to early infer when a specific collaborative operation will be requested by the human and to allow the robot to perform alternative autonomous tasks in the meanwhile. The prediction algorithm is based on higher-order Markov chains and is experimentally verified in a realistic scenario involving a dual-arm robot employed in a small part collaborative assembly task.

Handling liquids with spilling avoidance is a topic of interest for a broad range of fields, both in industry and in service robotic applications. In this paper we present a new control architecture for motion planning of industrial robots, able to tackle the problem of liquid transfer with sloshing control. We do not focus on a complete sloshing suppression, but we show how to enforce an anti spilling constraint. This less conservative approach allows to impose higher accelerations, reducing motion time. A constraint-based approach, amenable to an Online implementation, has been developed. The proposed controller generates trajectories in real time, in order to follow a reference path, while being compliant to the spilling avoidance constraint. The approach has been validated on a 6 degree of freedom industrial ABB robot.

Collaborative robotics over the last few years has gained increasing interest in the industrial scenario. Co-bots can be equipped with vision sensors and cognitive software layers, allowing the robot to figure out human intentions. To make this level of perception possible, human pose estimation algorithms are required. Several techniques have been already proposed to tackle this problem, which however present some weaknesses in particular when occlusions occur. This work proposes an algorithm for human pose estimation in the situations of partial occlusion, based on particle filter techniques. We have proved its validity in a realistic human-robot coexistence scenario, where a human and a dual arm robot have to perform tasks in a shared workspace.

In human-robot collaboration, the robot is required to provide assistance to the user by facilitating task execution. However, due to stability requirements, a well-damped admittance behavior of the robot is necessary during interaction, thus inducing fatigue in the operator. While available schemes involve variable impedance controllers to mitigate this effect, here we propose an alternative approach entailing a proactive robot behavior that assists in the cooperative execution of trajectories towards desired goals, by estimating the user intention. To this end, we make use of Recurrent Neural Networks (RNNs) to predict and classify cooperative motions, on the basis of a set of predefined goals in the workspace and model-based generated data of human movements. Manual guidance validation experiments are conducted on a 6 d.o.f. ABB IRB140 industrial robot equipped with a force sensor.

The present paper exploits set invariance theory to address the problem of robot implicit force control in presence of stiffness uncertainty in the interaction model. A numerical approach is introduced to compute the invariance function for constraints with arbitrary relative degree. The method is then applied to robot force control, enhancing force regulation performance, in terms of steady state error and convergence speed, despite model mismatch and measurement noise. Its effectiveness is experimentally validated and compared to previous results of set invariance control on a hybrid force/position task performed with a 6 degrees of freedom industrial robot equipped with a force/torque sensor.

In this work a constraint-based control approach is proposed in order to perform a cap rotation task with a dual-arm robot. The method relies on the introduction of a robust specification for the constraint on the interaction force arising during the task, accounting for robot-environment contact model uncertainties, in addition to force measurement noise and surface uncertainties. Experiments have been performed on an ABB dual-arm prototype robot to validate the proposed approach in a cap assembly task, employing a model-based sensorless observer of the interaction forces.

Allowing a cobot to predict what the human operator is about to do can definitely enhance the effectiveness of human-robot collaboration. This paper addresses the problem of inferring the most likely reaching target of the human hand. The method allows the robot to promptly recognise the intention of the human to reach a certain position within the scene and can be thus used by the controller of the robot to take the optimal decision on what to do. A novel method based on Bayesian statistics has been developed in this work and its applicability in a realistic context has been verified within an industrial use case, consisting of a commercial collaborative robot and a human operator performing a collaborative assembly task.

Industrial robots, automated manufacturing, and efficient logistics processes are at the heart of the upcoming fourth industrial revolution. While there are seminal studies on the vulnerabilities of cyber-physical systems in the industry, as of today there has been no systematic analysis of the security of industrial robot controllers. We examine the standard architecture of an industrial robot and analyze a concrete deployment from a systems security standpoint. Then, we propose an attacker model and confront it with the minimal set of requirements that industrial robots should honor: precision in sensing the environment, correctness in execution of control logic, and safety for human operators. Following an experimental and practical approach, we then show how our modeled attacker can subvert such requirements through the exploitation of software vulnerabilities, leading to severe consequences that are unique to the robotics domain. We conclude by discussing safety standards and security challenges in industrial robotics.

Fast and sensorless peg-in-hole insertion is a challenging task for a robotic manipulator. In order to deal with the peg-in-hole insertion problem without any need of an external force/torque sensor, this paper proposes to actively accomplish compliance in the insertion task relying on an admittance based control. This is combined with a real-time trajectory generator, by means of constraint based optimization, where a model-based sensorless observer of interaction forces is exploited. Experiments have been performed on an ABB dual-arm 7-DOF lightweight prototype robot to validate the proposed approach, with an insertion speed comparable to human manual execution and in presence of geometric uncertainty.

This paper addresses time-optimal path-constrained trajectory planning. Given a geometric path for a manipulator, this paper focuses on the selection of the time law along the path. This law minimizes the time required to complete the path and at the same time is consistent with constraints, both at kinematic and dynamic levels. To obtain the optimal law a decision algorithm for the acceleration along the path has been developed. Remarkably, the algorithm is amenable to online implementation, thus allowing for path replanning. An experimental validation on an ABB IRB140 robot is shown.

Classical control approaches to robot force control have been extensively addressed by research in the last decades and are now considered a paradigm when dealing with force control for industrial robots. With this respect, the present paper exploits the capability of state-of-the-art Quadratic Programming (QP) solvers to specify a simple and intuitive constraint-based optimization strategy aiming at improving closed-loop performance of a classical force controller, such as the implicit force control with pure integral action for a position-controlled manipulator in contact with a compliant environment. The effectiveness of the proposed control strategy is experimentally validated on an industrial robot equipped with a force sensor.

Despite the availability in the literature of several constraint-based motion generation algorithms, modest attention has been paid to their robustness with respect to noise, and more in general, to unstructured uncertainties. Especially in the case of sensor-related constraints, the envisaged robustness properties are clearly crucial to enforce the correct and expected behaviour of these algorithms. This paper contributes with a method to explicitly account for different sources of uncertainty. We also suggest a computational efficient way to consistently modify the constraint specification in order to obtain such robustness. An experimental verification on a visual aided grasping task, where visibility of the object is to be maintained, enlightens the benefits of the proposed approach in terms of achieving the desired robustness.

New paradigms in industrial robotics no longer require physical separation between robotic manipulators and humans. Moreover, in order to optimize production, humans and robots are expected to collaborate to some extent. In this scenario, involving a shared environment between humans and robots, common motion generation algorithms might turn out to be inadequate for this purpose. This paper proposes a kinematic control strategy which enforces safety, while maintaining the maximum level of productivity of the robot. The resulting motion of the (possibly redundant) robot is obtained as an output of an optimization-based real-time algorithm in which safety is regarded as a hard constraint to be satisfied. The methodology is experimentally validated on a dual-arm concept robot with 7-DOF per arm performing a manipulation task.

In this work, we propose a constraint-based algorithm for combined trajectory generation and kinematic control for robotic manipulators. The main feature of the algorithm is to ease robot programming, shifting from an imperative paradigm, in which task constraints are semantically and uniquely mapped into a suitable end-effector velocity profile, towards a declarative motion programming, where such constraints are turned by the controller into motion commands only at run-time: The system embeds the capability of handling real-time events, such as updated sensor readings, with reduced pre-programmed control logics. An experimental case study based on a 7-DOF robot demonstrates the effectiveness of the approach.

In this paper, a controller based on constrained optimization for tracking problems in mobile manipulation is presented. A Model Predictive Control problem is set and solved online, allowing to deal with dynamic scenarios and unforeseen events. Besides acceleration, velocity and position constraints, collision avoidance constraints for the mobile base and the arm and Field-of-View constraints have been enforced and extended over the prediction horizon. Navigation performance has been improved by including an additional goal, derived from the classical vortex field approach, to the MPC problem. An experimental validation on a KUKA youBot mobile manipulator has been carried out, showing the online applicability of the presented approach.

In order to make humans feeling comfortable when working with robots, it is necessary for robots to be as much as possible “human-like” in both their appearance and movements. In redundant manipulators, it is possible to use the additional degrees of freedom in order to make the robot motion more human-like, thus increasing the quality of the human-robot interaction. In this work, a redundancy resolution method to address this issue is presented. Such a method considers the human musculoskeletal system and a biomechanical model of the human upper limbs in order to define a strategy to solve the redundancy for a dual-arm anthropomorphic manipulator as a human would do, then making the robot able to both perform the prescribed task and to assume human-like postures.

Enabling human-robot collaboration raises new challenges in safety-oriented robot design and control. Indices that quantitatively describe human injury due to a human-robot collision are needed to propose suitable pre-collision control strategies. This paper presents a novel model-based injury index built on the concept of dissipated kinetic energy in a potential inelastic impact. This quantity represents the fracture energy lost when a human-robot collision occurs, modeling both clamped and unclamped cases. It depends on the robot reflected mass and velocity in the impact direction. The proposed index is expressed in analytical form suitable to be integrated in a constraint-based pre-collision control strategy. The exploited control architecture allows to perform a given robot task while simultaneously bounding our injury assessment and minimizing the reflected mass in the direction of the impact. Experiments have been performed on a lightweight robot ABB FRIDA to validate the proposed injury index as well as the pre-collision control strategy.

In the past few years the need for more flexibility in industrial production has implied a growing attention towards scenarios where humans work directly in touch with robots. In order to allow safe human-robot interaction, a methodology to evaluate the severity of an impact between a human worker and an industrial robot, based on related work on injury knowledge in human-robot contacts and relying on information coming from different exteroceptive sensors, has been developed in this paper. On the basis of this severity evaluation, the robot controller enforces a suitable safety-oriented strategy, ranging from on-path speed reduction to task-consistent evasive motion and protective stop. The safety evaluation methodology has been implemented in a dedicated software component, integrated with a video surveillance system and with the real time robot controller to obtain a complete HW/SW architecture named “Safety Controller”. The system has been validated on an ABB IRB140 robot.

This paper addresses the problem of collision avoidance in human-robot interaction. To this end, we introduce the concept of kinetostatic safety field, a novel safety assessment about the risk in the vicinity of a rigid body (including a robot link or a human body part). The safety field depends on the position and velocity of the body but it is also influenced by its real shape and size. Since all the computation can be performed in closed form, the safety field is suitable for real-time applications. Moreover, we present a safety-oriented control strategy for redundant manipulators, based on safety field and developed entirely on the kinematic level, where the kinematic redundancy is exploited for simultaneous task performance and collision avoidance, such as self-collision avoidance and human-robot coexistence. The proposed control strategy is validated through experiments performed on ABB's FRIDA dual arm robot.

Industrial robots are today separated from human workers by means of safety barriers, that protect humans from the risk of collisions. This separation has a clear negative influence on diffusion of robotic technology in shopfloors. On the other hand the removal of protective barriers gives rise to safety issues, that can be addressed with a combination of approaches, including sensor based reactive control. In this paper a distributed proximity sensor, to be mounted on the links of the manipulator, is presented. The optimal placement of the spots of such sensor is discussed, taking into account detection capabilities and safety enhancement. Experiments developed on an ABB IRB 140 robot using off-the-shelf infrared distance sensors as spots are presented.

Tight human-robot interaction and collaboration will characterize future robot tasks. Robot working environments will be increasingly unstructured, as safety barriers will be removed to allow a continuous cooperation of robotic and human workers. Such a working scenario calls for novel safety systems capable of combining productivity with workers' safety. In this paper, a method for the definition of a task-consistent collision avoidance safety strategy is presented. A classification of task constraints based on relevance for task completion is introduced. Control of task constraints enforcement is performed through a state machine. A template for such state machine is proposed. Experimental validation of the proposed safety system on a dual-arm industrial robot prototype is presented.

In order to improve production flexibility, it is widely agreed that future working environments will be populated by both humans and robot manipulators, sharing the same workspace. This scenario introduces a series of safety issues which are uncommon in industrial settings where physical separation of robot areas is typically enforced. While several approaches for safe human-robot interaction exist, none of them can be easily integrated with production constraints. This paper discusses the composition of safety constraints with production ones. An algorithm is derived in order to maximize productivity, while guaranteeing a safe separation distance of the robot from the human. Experimental results showing the effectiveness of the approach in a typical industrial setting are also discussed.

This paper presents a synergistic approach to danger assessment and safety-oriented control of articulated robots that are based on a quantity called danger field. This quantity captures the state of the robot as a whole and indicates how dangerous the current posture and velocity of the robot are to the objects in the environment. The field itself is invariant with respect to objects around the robot and can be computed in any given point of the robot's workspace using measurements from the proprioceptive sensors. Furthermore, the danger field can be expressed in the closed form, which enables its fast computation. Apart from being a pure safety assessment, the danger field provides a natural prelude to safety-oriented control strategy. Namely, the information about the danger field can easily be fed back to shape standard control schemes in order to make the motion of the robot safer to the environment. The proposed method is validated through simulations and experiments.

Dual-arm robotic systems have been intensively studied in the literature. However, in industrial robotics, the resolution of the kinematic redundancy allowed by the coordinated manipulation task is still an open issue. In fact, typical proprietary industrial robotic controllers do not allow the programmer to modify the inverse kinematics algorithm, and thus to solve redundancy following any specified criterion. In this paper a method to enforce an arbitrary redundancy resolution criterion on top of an industrial robot controller is discussed and applied to the execution of a coordinated manipulation task. The extra degrees of freedom are used to maximize the dynamic manipulability measure in order to reduce the needed torque. Simulations and experimental results achieved on an ABB IRC 5 industrial robot controller are presented.

This paper presents a new control law for robotic manipulators in unstructured environments which guarantees the achievement of the goal position without incurring in local minima. The passivity of the closed-loop system renders this control scheme well-suited for human-robot coexistence, especially when the robot is supposed to share its workspace with humans. The given control law has been implemented and experimentally tested in a realistic scenario, demonstrating the effectiveness in driving the robot to a given configuration in a cluttered environment without any offline planning phase.

Redundant robotic manipulators under kinematic control may exhibit unpredictable behaviors at the joint level. When the end effector describes a closed trajectory, the joint angles may not return to their initial values. Likewise, final configuration in the joint space may depend on the trajectory that is followed by the end effector. In this paper, a complete parameterization of holonomic redundancy resolution techniques that avoid these problems is proposed. The flexibility of the proposed approach is discussed. In particular, it is shown that the selection of the redundancy resolution criterion is totally decoupled from the implementation of a closed-loop inverse kinematics (CLIK) algorithm. Any user-defined redundancy resolution criterion can, thus, be enforced. Potentialities of this new methodology are experimentally verified on an industrial robot in a case study where functional redundancy occurs and is applied in simulation on a 7-degree-of-freedom (7-DOF) anthropomorphic manipulator.

Research in the field of human kinematic analysis has gained interest in recent years and has fostered new ideas and expectations. Next generation manipulators are expected to resemble a human-like behaviour at kinematic level, in order to avoid any unease or discomfort (like fear or shock) to the nearby humans. In this work, a kinematic experimental approach to study and synthesize the motion of the human arm is presented. In particular, the proposed scenario will be used to study how humans exploit the kinematic redundancy of their arms, for a future use in a robotic controller. A simple, yet accurate, method for human-like redundancy resolution in robotic manipulators is developed and verified.

A drilling task requires a mechanism with five degrees of freedom, in order to achieve the correct position and orientation of the drilling tool. When performed with a standard 6-axes industrial robot, this task leaves an extra degree of freedom that can be exploited in order to achieve any additional criterion. Unfortunately, typical industrial robotic control architectures do not allow the user to modify the inverse kinematics algorithm, and thus to solve task redundancy following any specified criterion. In this paper a method to enforce an arbitrary redundancy resolution criterion on top of an industrial robot controller is discussed and applied to the execution of a drilling task. The extra degree of freedom is used to perform a torque-effective drilling. Experimental results achieved on the ABB IRB 140 industrial robot are presented.

Redundant robotic manipulators under kinematic control may exhibit unpredictable behaviours at joint level. When the end-effector describes a closed trajectory, the joint angles may not return to their initial values and final configuration in the joint space may depend on the trajectory followed by the end-effector. In this paper, a complete parameterization of holonomic local control strategies that avoid these problems is proposed. Only a basis of the null-space of the Jacobian matrix is required in order to design all the possible holonomic control strategies. The effectiveness of the proposed approach is verified on a simple case study and on a real industrial manipulator.