The teleoperation of complex, kinematically redundant robots with loco-manipulation capabilities represents a challenge for human operators, who have to learn how to operate the many degrees of freedom of the robot to accomplish a desired task. In this context, developing an easy-to-learn and easy-to-use human-robot interface is paramount. Recent works introduced a novel teleoperation concept, which relies on a virtual physical interaction interface between the human operator and the remote robot equivalent to a "Marionette" control, but whose feedback was limited to only visual feedback on the human side. In this paper, we propose extending the "Marionette" interface by adding a wearable haptic interface to cope with the limitations given by the previous works. Leveraging the additional haptic feedback modality, the human operator gains full sensorimotor control over the robot, and the awareness about the robot’s response and interactions with the environment is greatly improved. We evaluated the proposed interface and the related teleoperation framework with naive users, assessing the teleoperation performance and the user experience with and without haptic feedback. The conducted experiments consisted in a loco-manipulation mission with the CENTAURO robot, a hybrid leg-wheel quadruped with a humanoid dual-arm upper body.

This work presents the computational design and validation of the Multi-Arm Relocatable Manipulator (MARM), a three-limb robot for space applications, with particular reference to the MIRROR (i.e., the Multi-arm Installation Robot for Readying ORUs and Reflectors) use-case scenario as proposed by the European Space Agency. A holistic computational design and validation pipeline is proposed, with the aim of comparing different limb designs, as well as ensuring that valid limb candidates enable MARM to perform the complex loco-manipulation tasks required. Moti-vated by the task complexity in terms of kinematic reachability, (self)-collision avoidance, contact wrench limits, and motor torque limits affecting Earth experiments, this work leverages on multiple state-of-art planning and control approaches to aid the robot design and validation. These include sampling-based planning on manifolds, non-linear trajectory optimization, and quadratic programs for inverse dynamics computations with constraints. Finally, we present the attained MARM design and conduct preliminary tests for hardware validation through a set of lab experiments.

Wheeled-legged robots have the potential to navigate in cluttered and irregular scenarios by altering the locomotion modes to adapt to the terrain challenges and effectively reach targeted locations in unstructured spaces. To achieve this functionality, a hybrid locomotion planner is necessary. In this work we present a search-based planner, which explores a set of motion primitives and a 2.5D traversability map extracted from the environment to generate navigation plans for the hybrid mobility robot CENTAURO. The planner explores the map from the current robot position to the goal location requested by the user, considering the most appropriate composition and tuning of locomotion primitives to build up a feasible plan, which is then executed by the robot. The available primitives are prioritized and can be easily modified, added or removed through a configuration file. Our approach was evaluated both in simulation and on the real wheeled-legged robot CENTAURO, demonstrating traversing capabilities in cluttered environments with various obstacles.

The teleoperation of mobile manipulators may pose significant challenges, demanding complex interfaces and causing a substantial burden to the human operator due to the need to switch continuously from the manipulation of the arm to the control of the mobile platform. Hence, several works have considered to exploit shared control techniques to overcome this issue and, in general, to facilitate the task execution. This work proposes a manipulability-aware shared locoma-nipulation motion generation method to facilitate the execution of telemanipulation tasks with mobile manipulators. The method uses the manipulability level of the end-effector to control the generation of the mobile base and manipulator motions, facilitating their simultaneous control by the operator while executing telemanipulation tasks. Therefore, the operator can exclusively control the end -effector, while the underlying ar-chitecture generates the mobile platform commands depending on the end-effector manipulability level. The effectiveness of this approach is demonstrated with a number of experiments in which the CENTAURO robot, a hybrid leg-wheel platform with an anthropomorphic upper body, is teleoperated to execute a set of telemanipulation tasks.

This work introduces a framework for the Cartesian control of multi-legged, highly redundant robots. The proposed framework allows the untrained user to perform complex motion tasks with robotics platforms by leveraging a simple, auto-generated ROS-based interface. Contrary to other motion control frameworks (e.g. ROS MoveIt!), we focus on the execution of Cartesian trajectories that are specified online, rather than planned in advance, as it is the case, for instance, in tele-operation and locomotion tasks. Moreover, we address the problem of generating such motions within a hard real-time (RT) control loop. Finally, we demonstrate the capabilities of our framework both on the COMAN + humanoid robot, and on the hybrid wheeled-legged quadruped CENTAURO.

In this paper, we look at the challenge of human robot interaction in locomotion. We consider a hand-in-hand interaction scenario where a human compliantly interacts with the upper-body of an impedance controlled humanoid. By exploring the velocity transmission of the robot arms, and the interaction in terms of robot arms manipulation quality evaluated through the monitoring of their manipulability the proposed method derives suitable reactive steps in appropriate directions to ensure that the robot manipulation ability is maintained with the robot arms providing high capacity of motion along the different directions. The proposed approach can be combined with different walking pattern generators and is not tailored to a specific one used in this work. The results of the proposed method are experimentally validated on the COMAN + humanoid robot showing the efficacy of the method to generate reactive stepping driven by the interaction and manipulation motion of the human operator. Besides, the work also provides a real-time software architecture to control humanoid COMAN+, but it is also flexible to be used for the control of other robot platforms.

Human Robot interaction is a fundamental perquisite for any robot performing a physical task in collaboration with a human. The presence of disturbances arising from the partially known tasks payloads, the unexpected interaction forces in general, and the uncertainty in the interpretation of the human intention in terms of motions and forces can pose significant challenges and eventually compromise the execution of the collaborative task. This work presents a novel, intrinsically adaptable multimodal (force, motion and verbal) interaction framework for human-robot collaboration (HRC) that leverages on an online self-tuning stiffness regulation principle to provide adaptation to interaction/payload forces and reject disturbances arising by unexpected interaction loads. Besides the presented method, it enables the rejection of unnecessary motion commands (e.g. oscillations generated by the human operator) to reach the robot co-worker through the filtering of the human generated motions, that are outside the range (in terms of speed and acceleration) of the envisioned manipulation manoeuvres. Finally, a verbal interaction channel allows the operator to convey securely his high level intentions and to control the states of the task execution. We evaluated and demonstrated the effectiveness of the proposed multimodal interaction framework in a high weight carrying human-robot collaboration task using the humanoid robot COMAN +.

This work presents a new attitude balancing strategy implemented and validated on a quadrupedal robot equipped with a custom hybrid wheel-legged mobility system. The proposed method uses an inverse kinematics solution scheme based on Quadratic Programming optimization to generate full body motions that ensure the desired balancing performances. The strategy generates a compliant behaviour to cope with the applied external forces resulting in a stable and smooth reaction response. Furthermore, the method takes advantage of the robot hybrid wheeled-legged mobility system to provide new motion capabilities and balancing reactions as it will be shown through the paper. Extensive simulation studies on the Centauro robot are presented. Results show the efficiency of the propose method demonstrating significant contribution in the rejection of the applied external disturbances.

In this work we introduced a prioritized Cartesian impedance control under the framework of the Quadratic Programming (QP) optimization. In particular, we present a formulation which is simpler than full inverse dynamics, avoids any matrix pseudo-inversion, inverse kinematics computation and considers strict priorities among tasks. Our formulation is based on QP optimization permitting to take into account also explicit inequality constraints. We compare in simulation the tracking results obtained with a classical algebraic implementation against those derived from the proposed QP implementation taking into account joint torque limits. We consider the classical Cartesian impedance controller and a simplified version, also known as Virtual Model Control. Finally the proposed method was implemented and validated on a humanoid upper-body torque controlled robot. Experimental trials involving various physical interaction conditions were executed to demonstrate the performance of the proposed method.

Robotics teleoperation has been extensively studied and considered in the past in several task scenarios where direct human intervention is not possible due to the hazardous environments. In such applications, both communication degradation and reduced perception of the remote environment are practical issues that can challenge the human operator while controlling the robot and attempting to physically interact within the remote workspace. To address this challenge, we introduce a novel shared-autonomy Tele-Interaction control approach that blends the motion commands from the pilot (master side) with locally (slave side) executed autonomous motion and impedance modulators. This enables a remote robot to handle and autonomously avoid physical obstacles during manoeuvring, reduce interaction forces during contacts, and finally accommodate different payload conditions while at the same time operating with a “default” low impedance setting. We implemented and experimentally validated the proposed method both on simulation and on a real robot platform called CENTAURO. A series of tasks, such as maneuvering through the physical constraints of the remote environment in an autonomous manner, pushing and lifting heavy objects with autonomous impedance regulation and colliding with the rigid geometry of the remote environment were executed. The obtained results demonstrate the effectiveness of the shared-autonomy control principles that eventually aim to reduce the level of attention and stress of human pilot while manoeuvring the slave robot, and at the same time to enhance the robustness of the robot during physical interactions even if accidentally occurred.

We present a new method to translate videos to commands for robotic manipulation using Deep Recurrent Neural Networks (RNN). Our framework first extracts deep features from the input video frames with a deep Convolutional Neural Networks (CNN). Two RNN layers with an encoder-decoder architecture are then used to encode the visual features and sequentially generate the output words as the command. We demonstrate that the translation accuracy can be improved by allowing a smooth transaction between two RNN layers and using the state-of-the-art feature extractor. The experimental results on our new challenging dataset show that our approach outperforms recent methods by a fair margin. Furthermore, we combine the proposed translation module with the vision and planning system to let a robot perform various manipulation tasks. Finally, we demonstrate the effectiveness of our framework on a full-size humanoid robot WALK-MAN.

Complex interactions with unstructured environments require the application of appropriate restoring forces in response to the imposed displacements. Impedance control techniques provide effective solutions to achieve this, however, their quasi-static performance is highly dependent on the choice of parameters, i.e. stiffness and damping. In most cases, such parameters are previously selected by robot programmers to achieve a desired response, which limits the adaptation capability of robots to varying task conditions. To improve the generality of interaction planning through task-dependent regulation of the parameters, this paper introduces a novel self-regulating impedance controller. The regulation of the parameters is achieved based on the robot's local sensory data, and on an interaction expectancy value. This value combines the interaction values from the robot state machine and visual feedback, to authorize the autonomous tuning of the impedance parameters in selective Cartesian axes. The effectiveness of the proposed method is validated experimentally in a debris removal task.

Limitations with the on-board computational resources installed on untethered robots such as humanoids and mobile robots in general affects significantly the performance and capabilities of these machines. An approach to address this issue is to make use of the cloud robotics concept and take advantage of the extensive computational resources of the cloud. XBotCloud is a recently developed component of the XBot framework. It tackles the above challenges by introducing the tools and mechanisms to enable users and robots to exploit the computational resources of the cloud allowing the execution of services with low, soft or hard Real-Time execution/communication performance. The latter is ensured thanks to the functionality provided by the XBotCore Real-Time cross-robot software component of the XBot framework. XBotCloud addresses also one of the main challenges related with cloud robotics: security. To avoid remote attacks it takes advantage of the Amazon Web Services (AWS)Cloud Security and it uses an internal VPN Network to handle the connectivity between the robot and the cloud server. The full implementation of the framework is presented and its functionality is demonstrated in realistic tasks involving pipelines that mix the execution of cloud services with moderate execution time constraints and Real-Time modules running on the robot local control unit. XBotCloud performances and cross-robot flexibility are experimentally validated on two different robotic platforms, the WALK-MAN humanoid and the CENTAURO upper body/full-body.

Developing a high physical performance robotic manipulation platform with considerable power density, strength and resilience is not a trivial task and frequently leads to heavy and bulky systems unable to meet the application requirements, i.e. such robots should have human body size compatibility to work in infrastructures designed for humans. In this work we present a new high performance human size and weight compatible bi-manual manipulation platform that demonstrates notable physical strength and power capabilities. To attain this performance, design features including custom high performance elastic drives and robust light weight structure principles were considered resulting in large payload to robot mass ratio that is greater than 1.5 for short time heavy payloads. The design principles and mechanics of the upper body bi-manual robot are presented providing details on the solutions adopted for the various mechatronics components. The performance of the system actuation and the strength capacity of the overall platform is verified through the execution of heavy payload motion and impact experiments.