Deploying perception modules for human-robot handovers is challenging because they require a high degree of reactivity, generalizability, and robustness to work reliably for a diversity of cases. Further complications arise as each object can be handed over in a variety of ways, causing occlusions and viewpoint changes. On legged robots, deployment is particularly challenging because of the limited computational resources and the image-space noise resulting from locomotion. In this paper, we introduce an efficient and object-agnostic real-time tracking framework, specifically designed for human-to-robot handover tasks with a legged manipulator. The proposed method combines optical flow with Siamese-network-based tracking and depth segmentation in an adaptive Kalman Filter framework. We show that we outperform the state-of-the-art for tracking during human-to-robot handovers with our legged manipulator. We demonstrate the generalizability, reactivity, and robustness of our system through experiments in different scenarios and by carrying out a user study. Additionally, as timing is proven to be more important than spatial accuracy for human-robot handovers, we show that we reach close to human timing performance during the approaching phase, both in terms of objective metrics and subjective feedback from the participants of our user study.

Model Predictive Control (MPC) schemes have proven their efficiency in controlling high degree-of-freedom (DoF) complex robotic systems. However, they come at a high computational cost and an update rate of about tens of hertz. This relatively slow update rate hinders the possibility of stable haptic teleoperation of such systems since the slow feedback loops can cause instabilities and loss of transparency to the operator. This work presents a novel framework for transparent teleoperation of MPC-controlled complex robotic systems. In particular, we employ a feedback MPC approach [1] and exploit its structure to account for the operator input at a fast rate which is independent of the update rate of the MPC loop itself. We demonstrate our framework on a mobile manipulator platform and show that it significantly improves haptic teleoperation's transparency and stability. We also highlight that the proposed feedback structure is constraint satisfactory and does not violate any constraints defined in the optimal control problem. To the best of our knowledge, this work is the first realization of the bilateral teleoperation of a legged manipulator using a whole-body MPC framework.

To safely deploy legged robots in the real world it is necessary to provide them with the ability to reliably detect unexpected contacts and accurately estimate the corresponding contact force. In this paper, we propose a collision detection and identification pipeline for a quadrupedal manipulator. We first introduce an approach to estimate the collision time span based on band-pass filtering and show that this information is key for obtaining accurate collision force estimates. We then improve the accuracy of the identified force magnitude by compensating for model inaccuracies, unmodeled loads, and any other potential source of quasi-static disturbances acting on the robot. We validate our framework with extensive hardware experiments in various scenarios, including trotting and additional unmodeled load on the robot.

Robot-assisted cutting is considered an important task in several fields, such as robotic surgery, nuclear decommissioning, waste management, and manufacturing. Despite the complex dexterity requirements of cutting tasks, very simple mechanically-linked master-slave manipulators still dominate many of the above fields (e.g., nuclear robotics). Moreover, even when more dexterous manipulators are available (e.g., in robot-assisted surgery), the employed systems show little or no autonomy, delegating all control to the experience of the human operator. To ameliorate this situation, we present two haptic shared-control approaches for robotic cutting. They are designed to assist the human operator by enforcing different nonholonomic-like constraints representative of the cutting kinematics. To validate our approach, we carried out a human-subject experiment in a real cutting scenario. We compared our shared-control techniques with each other and with a standard haptic teleoperation scheme. Results show the usefulness of assisted control schemes in complex applications such as cutting. However, they also show a discrepancy between objective and subjective metrics.

Robotic telemanipulators are already widely used in nuclear decommissioning sites for handling radioactive waste. However, currently employed systems are still extremely primitive, making the handling of these materials prohibitively slow and ineffective. As the estimated cost for the decommissioning and clean-up of nuclear sites keeps rising, it is clear that one would need faster and more effective approaches. Towards this goal, in this paper we present the user evaluation of a recently proposed haptic-enabled shared-control architecture for telemanipulation. An autonomous algorithm regulates a subset of the slave manipulator degrees of freedom (DoF) in order to help the human operator in grasping an object of interest. The human operator can then steer the manipulator along the remaining null-space directions with respect to the main task by acting on a grounded haptic interface. The haptic cues provided to the operator are designed in order to inform about the feasibility of the user's commands with respect to possible constraints of the robotic system. In this paper we compared this shared-control architecture against a classical 6-DOF teleoperation approach in a real scenario by running experiments with 10 subjects. The results clearly show that the proposed shared-control approach is a viable and effective solution for improving currently-available teleoperation systems in remote telemanipulation tasks.

Robotic teleoperation is a key technology for a wide variety of fields. Teleoperating a humanoid in particular is essential as it allows the user to act remotely on an interface designed especially for humans, e.g., in a space station, or operating tools and machinery in disaster scenarios. This paper presents a ‘task-relevant’ haptic interface for humanoid teleoperation, which bridges the gap between the task at hand and the balance of the robot. The operator is given command over the humanoid's hands and is informed through haptic cues about the impact of her/his potential actions on the robot’ stability. Moreover, a null-space autonomous controller acts in the operator's null-space to provide her/him with a wider workspace and help in the successful execution of the task. The architecture is designed to top an existing compliance controller for a torque-controlled humanoid robot. Experiments on the humanoid robot TORO are reported to demonstrate the feasibility and effectiveness of the approach.

Shared control is a key technology for various robotic applications in which a robotic system and a human operator are meant to collaborate efficiently. In order to achieve efficient task execution in shared control, it is essential to predict the desired behavior for a given situation or context in order to simplify the control task for the human operator. This prediction is obtained by exploiting Learning from Demonstration (LfD), which is a popular approach for transferring human skills to robots. We encode the demonstrated behavior as trajectory distributions and generalize the learned distributions to new situations. The goal of this paper is to present a shared control framework that uses learned expert distributions to gain more autonomy. Our approach controls the balance between the controller's autonomy and the human preference based on the distributions of the demonstrated trajectories. Moreover, the learned distributions are autonomously refined from collaborative task executions, resulting in a master-slave system with increasing autonomy that requires less user input with an increasing number of task executions. We experimentally validated that our shared control approach enables efficient task executions. Moreover, the conducted experiments demonstrated that the developed system improves its performances through interactive task executions with our shared control.

Nowadays, one of the largest environmental challenges that European countries must face consists in dealing with the past half century of nuclear waste. In order to optimize maintenance costs, nuclear waste must be sorted, segregated and stored according to its radiation level. Towards this end, in [1] we have recently proposed a visual-based shared control architecture meant to facilitate a human operator in controlling two remote robotic arms (one equipped with a gripper and another with a camera) during remote manipulation tasks of nuclear waste via a master device. The operator could then receive force cues informative of the feasibility of her/his motion commands during the task execution. The strategy presented in [1], albeit effective, suffers however from a locality issue since the operator can only provide instantaneous velocity commands (in a suitable task space), and receive instantaneous force feedback cues. On the other hand, the ability to `steer' a whole future trajectory in task space, and to receive a corresponding integral force feedback along the whole planned trajectory (because of any constraint of the considered system), could significantly enhance the operator's performance, especially when dealing with complex manipulation tasks. The aim of this work is to then extend [1] towards a planning-based shared control architecture able to take into account the mentioned requirements. A human/hardware-in-the-loop experiment with simulated slave robots and a real master device is reported for demonstrating the feasibility and effectiveness of the proposed approach.

This paper addresses the problem of mixed initiative, shared control for master-slave grasping and manipulation. We propose a novel system, in which an autonomous agent assists a human in teleoperating a remote slave arm/gripper, using a haptic master device. Our system is designed to exploit the human operator's expertise in selecting stable grasps (still an open research topic in autonomous robotics). Meanwhile, a-priori knowledge of: i) the slave robot kinematics, and ii) the desired post-grasp manipulative trajectory, are fed to an autonomous agent which transmits force cues to the human, to encourage maximally manipulable grasp pose selections. Specifically, the autonomous agent provides force cues to the human, during the reach-to-grasp phase, which encourage the human to select grasp poses which maximise manipulation capability during the post-grasp object manipulation phase. We introduce a task-oriented velocity manipulability cost function (TOV), which is used to identify the maximum kinematic capability of a manipulator during post-grasp motions, and feed this back as force cues to the human during the pre-grasp phase. We show that grasps which minimise TOV result in significantly reduced control effort of the manipulator, compared to other feasible grasps. We demonstrate the effectiveness of our approach by experiments with both real and simulated robots.

Cleaning up the past half century of nuclear waste represents the largest environmental remediation project in the whole Europe. Nuclear waste must be sorted, segregated and stored according to its radiation level in order to optimize maintenance costs. The objective of this work is to develop a shared control framework for remote manipulation of objects using visual information. In the presented scenario, the human operator must control a system composed of two robotic arms, one equipped with a gripper and the other one with a camera. In order to facilitate the operator's task, a subset of the gripper motion are assumed to be regulated by an autonomous algorithm exploiting the camera view of the scene. At the same time, the operator has control over the remaining null-space motions w.r.t. the primary (autonomous) task by acting on a force feedback device. A novel force feedback algorithm is also proposed with the aim of informing the user about possible constraints of the robotic system such as, for instance, joint limits. Human/hardware-in-the-loop experiments with simulated slave robots and a real master device are finally reported for demonstrating the feasibility and effectiveness of the approach.