In this article, we present a planning framework that uses a combination of implicit (robot motion) and explicit (visual/audio/haptic feedback) communication during mobile robot navigation. First, we developed a model that approximates both continuous movements and discrete behavior modes in human navigation, considering the effects of implicit and explicit communication on human decision-making. The model approximates the human as an optimal agent, with a reward function obtained through inverse reinforcement learning. Second, a planner uses this model to generate communicative actions that maximize the robot's transparency and efficiency. We implemented the planner on a mobile robot, using a wearable haptic device for explicit communication. In a user study of an indoor human-robot pair orthogonal crossing situation, the robot is able to actively communicate its intent to users in order to avoid collisions and facilitate efficient trajectories. Results show that the planner generated plans that are easier to understand, reduce users` effort, and increase users' trust of the robot, compared to simply performing collision avoidance. The key contribution of this article is the integration and analysis of explicit communication (together with implicit communication) for social navigation.

Fully autonomous navigation in populated environments is still a challenging problem for mobile robots. This paper explores the idea of using active human-robot communication to facilitate navigation tasks. We propose to convey a robot's intent to human users via a wearable haptic interface. The interface can display distinct haptic cues by modulating vibration amplitudes and patterns. We applied the concept to a single human/single robot orthogonal encounter scenario, where one of the two parties has to yield the right of way to avoid collision. Under certain conditions, the robot's intent (to yield to the human or not) is revealed to the human via the haptic interface prior to the interaction. We conducted an experiment with 10 users, in which the robot was teleoperated as a substitute for autonomy. Results show that, when given priority, users become more risk-accepting and use different strategies to navigate the collision scenario than when the robot takes priority or there is no haptic communication channel. In addition, we propose a social-force based model to predict human movement during navigation. The effect of communication can be explained as a shift in the user's safety buffer and expectation of the robot's future velocity.

In this article, we present a bi-directional communication scheme that facilitates interaction between a person and a mobile robot that follows the person. A person-following robot can assist people in many applications including load carrying, elder care, and emotional support. However, commercially available personal robot systems usually have limited sensing and actuation capabilities. They are not expected to function perfectly in complex environments, and human intervention is required when the robot fails. We propose to use a holdable mechatronic device to reduce the user’s effort in communication and enable natural interaction during the intervention. Our design of the holdable device consists of two parts: a haptic interface that displays touch cues to convey the robot’s failure status via asymmetric vibrations, and a command interface for teleoperating the robot follower with hand gestures. We experimentally evaluated the device and the communication strategy in two sets of user studies with a controlled environment and a physical robot follower. Results show that with the proposed method, users are able to perform their tasks better, respond to robot failure events faster, and adjust walking speed according to the robot’s limitations. We also demonstrate that users can successfully teleoperate the robot to avoid obstacles when navigating in challenging environments.

In a teleoperated system, misalignment between the master and slave manipulators can result from clutching, errors in the kinematic model, and/or sensor errors. This study examines the effects of type and magnitude of misalignment on the performance of the teleoperator. We first characterized the magnitude and direction of orientation misalignment created when clutching and unclutching during use of two surgical robots: the Raven II and the da Vinci Research Kit. We then purposely generated typical misalignments in order to measure the impact of such misalignment on user performance of a peg transfer task with the Raven II. Users were able to compensate for misalignment angles up to approximately 20 degrees in both tool orientation and camera viewpoint misalignment. These results can be used to guide the design and control of teleoperated systems for a variety of applications.

During robotic teleoperation, the dynamics of the master manipulators and the control of remote-side instruments impose challenges on the motor system of the human operator, and may impact performance and learning. In teleoperated robot-assisted minimally invasive surgery, there is a clear correlation between patient outcomes and the surgeon's case experience. However, the effect of the teleoperator on human motor skills and the relationship between these motor skills and patient outcomes are unknown. We used the da Vinci Research Kit, a custom research version of the da Vinci Surgical System, to compare teleoperated and open needle-driving movements of experienced da Vinci surgeons and novices. The experimental protocol consisted of structured but unconstrained needle driving trials repeated 80 times to allow for computational modeling of movement coordination and learning. Kinematic analysis showed that teleoperation increases trial time but reduces path length, that the trial times and path lengths of experienced surgeons are smaller than those of novices. In addition, there are significant differences in learning between experienced surgeons and novice users. Modeling of the movements and learning processes of experienced and novice surgeons may be used in the design of novel controllers that will expand robotic surgery capabilities and improve robot-assisted surgical skill acquisition.

Automation of repetitive tasks can improve laparoscopic surgical procedures by unloading surgeons and reducing duration, trauma, and expense. However, surgical procedures involve delicate manipulation of deformable tissues in a very dynamic environment, suggesting that automated execution of surgical tasks should be carried out under the supervision of the surgeon. We propose a teleoperated architecture that allows a surgeon to employ and supervise agents that can autonomously perform or assist with surgical tasks. The architecture is independent of the automation method. It includes a dominance factor that allows the surgeon to take control over the slave robot at any time, and an aggressiveness factor that sets the performance pace of the autonomous agent. We tested the architecture during execution of a multilateral tension-and-cut task, where a human operator and an autonomous agent are responsible for tensioning or cutting of a tissue. The architecture allowed for supervised and paced automation of the task. We found that collaboration of the human operator and autonomous agent can lead to shorter completion time compared to performance of only a human.