Neural Style Transfer (NST) refers to a class of algorithms able to manipulate an element, most often images, to adopt the appearance or style of another one. Each element is defined as a combination of Content and Style: the Content can be conceptually defined as the “what” and the Style as the “how” of said element. In this context, we propose a custom NST framework for transferring a set of styles to the motion of a robotic manipulator, e.g., the same robotic task can be carried out in an “angry”, “happy”, “calm”, or “sad” way. An autoencoder architecture extracts and defines the Content and the Style of the target robot motions. A Twin Delayed Deep Deterministic Policy Gradient (TD3) network generates the robot control policy using the loss defined by the autoencoder. The proposed Neural Policy Style Transfer TD3 NPST3 alters the robot motion by introducing the trained style. Such an approach can be implemented either offline, for carrying out autonomous robot motions in dynamic environments, or online, for adapting at runtime the style of a teleoperated robot. The considered styles can be learned online from human demonstrations. We carried out an evaluation with human subjects enrolling 73 volunteers, asking them to recognize the style behind some representative robotic motions. Results show a good recognition rate, proving that it is possible to convey different styles to a robot using this approach.

We present a decentralized connectivity-maintenance control framework for a heterogeneous human–robot team. The algorithm is able to manage a team composed of an arbitrary number of mobile robots (drones and ground robots in our case) and humans, for collaboratively achieving exploration and patrolling tasks. Differently from other works on the subject, here the human user physically becomes part of the team, moving in the same environment of the robots and receiving information about the team connectivity through wearable haptics or audio feedback. Although human explores the environment, robots move so as to keep the team connected via a connectivity-maintenance algorithm; at the same time, each robot can also be assigned with a specific target to visit. We carried out three human subject experiments, both in virtual and real environments. Results show that the proposed approach is effective in a wide range of scenarios. Moreover, providing either haptic or audio feedback for conveying information about the team connectivity significantly improves the performance of the considered tasks, although users significantly preferred receiving haptic stimuli w.r.t. the audio ones. Note to Practitioners—Exploration, patrolling, and search-and-rescue are highly dynamic and unstructured scenarios. When considering the operative conditions of such environments, the benefits of multirobot systems are evident. Most tasks can be carried out faster and more robustly by a team of robots with respect to a single unit. There are also situations explicitly requiring the presence of a multirobot team, e.g., using one drone for surveillance of the ground team and one ground mobile robot for carrying supplies. Of course, if the operator(s) in charge of the operation could share the same environment of the robots (i.e., be together with the robots in the field), they would be provided with a level of situational awareness that no teleoperation technology can match as of today. This work presents a framework for controlling heterogeneous teams composed of one human operator and an arbitrary number of aerial and ground mobile robots. The operator moves together with the robotic team and, at the same time, he or she receives meaningful information about the status of the formation. The algorithm only uses the relative position of the drones and humans with respect to each other, and all computations are designed in a decentralized fashion. Decentralization avoids relying on any absolute positioning system (e.g., GPS) or centralized command centers. These features make the proposed framework ready for deployment in different high-impact applications, such as in surveillance, search-and-rescue, and disaster response scenarios.

Virtual reality (VR) is a valuable experimental tool for studying human movement, including the analysis of interactions during locomotion tasks for developing crowd simulation algorithms. However, these studies are generally limited to distant interactions in crowds, due to the difficulty of rendering realistic sensations of collisions in VR. In this article, we explore the use of wearable haptics to render contacts during virtual crowd navigation. We focus on the behavioral changes occurring with or without haptic rendering during a navigation task in a dense crowd, as well as on potential after-effects introduced by the use haptic rendering. Our objective is to provide recommendations for designing VR setup to study crowd navigation behavior. To the end, we designed an experiment (N=23) where participants navigated in a crowded virtual train station without, then with, and then again without haptic feedback of their collisions with virtual characters. Results show that providing haptic feedback improved the overall realism of the interaction, as participants more actively avoided collisions. We also noticed a significant after-effect in the users' behavior when haptic rendering was once again disabled in the third part of the experiment. Nonetheless, haptic feedback did not have any significant impact on the users' sense of presence and embodiment.

This article presents the design of a decentralized connectivity-maintenance algorithm for the teleoperation of a team of multiple UAVs, together with an extensive human subject evaluation in virtual and real environments. The proposed connectivity-maintenance algorithm enhances earlier works by improving their applicability, safety, effectiveness, and ease of use, by including: 1) an airflow-avoidance behavior that avoids stack downwash phenomena in rotor-based aerial robots; 2) a consensus-based action for enabling fast displacements with minimal topology changes by having all follower robots moving at the leader’s velocity; 3) an automatic decrease of the minimum degree of connectivity, enabling an intuitive and dynamic expansion/compression of the formation; and 4) an automatic detection and resolution of deadlock configurations, i.e., when the robot leader cannot move due to counterbalancing connectivity- and external-related inputs. We also devised and evaluated different interfaces for teleoperating the team as well as different ways of receiving information about the connectivity force acting on the leader. The results of two human subject experiments show that the proposed algorithm is effective in various situations. Moreover, using haptic feedback to provide information about the team connectivity outperforms providing both no feedback at all and sensory substitution via visual feedback.Note to Practitioners—The control of one drone is usually performed with a remote controller (similar to a joypad) that uses radio-wave signals. When controlling more than one drone, even a simple task, such as moving the whole team around, becomes very challenging. Developing an easy, yet efficient way to impart commands to a formation of drones is necessary to achieve any complex task. This article proposes a framework to control a fleet of drones (quadrotors) in an intuitive way while receiving meaningful and effective information on the state of the formation. The proposed technique does not rely on any absolute positioning system (e.g., GPS) or centralized command center. Instead, it only uses the relative position of the drones with respect to each other, and all computations are designed in a decentralized fashion. These features make the proposed framework ready for deployment in different high-impact applications, such as in surveillance, search-and-rescue, and disaster response scenarios.

Robotic teleoperation in cluttered environments is attracting increasing attention for its potential in hazardous scenarios, disaster response, and telemaintenance. Although haptic feedback has been proven effective in such applications, commercially-available grounded haptic interfaces still show significant limitations in terms of workspace, safety, transparency, and encumbrance. For this reason, we present a novel robotic teleoperation system with wearable haptic feedback for telemanipulation in cluttered environments. The slave system is composed of a soft robotic hand attached to a 6-axis force sensor, which is fixed to a 6-degrees-of-freedom robotic arm. The master system is composed of two wearable vibrotactile armbands and a Leap Motion. The armbands are worn on the upper arm and forearm, and convey information about collisions on the robotic arm and hand, respectively. The position of the manipulator and the grasping configuration of the robotic hand are controlled by the user's hand pose as tracked by the Leap Motion. To validate our approach, we carried out a human-subject telemanipulation experiment in a cluttered scenario. Twelve participants were asked to teleoperate the robot to grasp an object hidden between debris of various shapes and stiffnesses. Haptic feedback provided by our wearable devices significantly improved the performance of the considered telemanipulation tasks. All subjects but one preferred conditions with wearable haptic feedback.

In this paper, we present a novel cooperative navigation control for human-robot teams. Assuming that a human wants to reach a final location in a large environment with the help of a mobile robot, the robot must steer the human from the initial to the target position. The challenges posed by cooperative human-robot navigation are typically addressed by using haptic feedback via physical interaction. In contrast with that, in this paper, we describe a different approach, in which the human-robot interaction is achieved via wearable vibrotactile armbands. In the proposed work, the subject is free to decide her/his own pace. A warning vibrational signal is generated by the haptic armbands when a large deviation with respect to the desired pose is detected by the robot. The proposed method has been evaluated in a large indoor environment, where 15 blindfolded human subjects were asked to follow the haptic cues provided by the robot. The participants had to reach a target area, while avoiding static and dynamic obstacles. Experimental results revealed that the blindfolded subjects were able to avoid the obstacles and safely reach the target in all of the performed trials. A comparison is provided between the results obtained with blindfolded users and experiments performed with sighted people.

Human-Robot teams can efficiently operate in several scenarios including Urban Search and Rescue (USAR). Robots can access areas too small or deep for a person, can begin surveying larger areas that people are not permitted to enter and can carry sensors and instruments. One important aspect in this cooperative framework is the way robots and humans can communicate during rescue operation. Vision and audio modalities may result not efficient in case of reduced visibility or high noise. A promising way to guarantee effective communications between robot and human in a team is the exploitation of haptic signals. In this work, we present a possible solution to let a robot guide the position of a human operator's hand by using vibrations. We demonstrate that an armband embedding four vibrating motors is enough to guide the wrist of an operator along a predefined path or in a target location. The results proposed can be exploited in human-robot teams. For instance, when the robot detects the position of a sensible target, it can guide the wrist of the operator in such position following an optimal path.

In this paper, we present a haptic guidance policy to steer the user along predefined paths, and we evaluate a predictive approach to compensate actuation delays that humans have when they are guided along a given trajectory via sensory stimuli. The proposed navigation policy exploits the nonholonomic nature of human locomotion in goal directed paths, which leads to a very simple guidance mechanism. The proposed method has been evaluated in a real scenario where seven human subjects were asked to walk along a set of predefined paths, and were guided via vibrotactile cues. Their poses as well as the related distances from the path have been recorded using an accurate optical tracking system. Results revealed that an average error of 0.24 m is achieved by using the proposed haptic policy, and that the predictive approach does not bring significant improvements to the path following problem for what concerns the distance error. On the contrary, the predictive approach achieved a definitely lower activation time of the haptic interfaces.

This paper proposes a novel use of haptic feedback for human navigation with a mobile robot. Assuming that a path-planner has provided a mobile robot with an obstacle-free trajectory, the vehicle must steer the human from an initial to a desired target position by only interacting with him/her via a custom-designed vibro-tactile bracelet. The subject is free to decide his/her own pace and a warning vibrational signal is generated by the bracelet only when a large deviation with respect to the planned trajectory is detected by the vision sensor on-board the robot. This leads to a cooperative navigation system that is less intrusive, more flexible and easy-to-use than the ones existing in literature. The effectiveness of the proposed system is demonstrated via extensive real-world experiments.

Compliance in multifingered hand improves grasp stability and effectiveness of the manipulation tasks. Compliance of robotic hands depends mainly on the joint control parameters, on the mechanical design of the hand, as joint passive springs, and on the contact properties. In object grasping the primary task of the robotic hand is the control of internal forces which allows to satisfy the contact constraints and consequently to guarantee a stable grasp of the object. When compliance is an essential element of the multifingered hand, and the control of the internal forces is not designed to be decoupled from the object motion, it happens that a change in the internal forces causes the object trajectory to deviate from the planned path with consequent performance degradation. This paper studies the structural conditions to design an internal force controller decoupled from object motions. The analysis is constructive and a controller of internal forces is proposed. We will refer to this controller as object motion-decoupled control of internal forces. The force controller has been successfully tested on a realistic model of the DLR Hand II. This controller provides a trajectory interface allowing to vary the internal forces (and to specify object motions) of an underactuated hand, which can be used by higher-level modules, e.g. planning tools.