Robot-human trust is an important concern as robots become integrated into human spaces. We tested a method grounded in psychological theory to increase human-robot trust—synchronous motion. Human participants completed a goal-oriented ball-moving task with a robotic arm to sound cues that were synchronous or asynchronous with the robot’s pacing. Participants were instructed to follow sound cues without information about synchrony. We found that participants in the synchrony condition trusted the robot to complete a new task that was comparable to the task they completed, significantly more than the asynchrony condition. However, this effect did not extend to harder tasks. The participants in the synchrony condition also believed that the robot had more influence on the outcomes of the new task compared to the asynchrony condition. On average, participants’ trust increased with the robotic arm after completing the task, regardless of condition. We report findings from a thematic analysis that demonstrate that participants in the synchrony condition found synchrony to be beneficial, while participants in the asynchrony condition found it cognitively taxing to be out-of-sync. Results from this work may be used to improve human-robot interactions in various contexts.

Many industrial tasks-such as sanding, installing fasteners, and wire harnessing-are difficult to automate due to task complexity and variability. We instead investigate deploying robots in an assistive role for these tasks, where the robot assumes the physical task burden and the skilled worker provides both the high-level task planning and low-level feedback necessary to effectively complete the task. In this article, we describe the development of a system for flexible human-robot teaming that combines state-of-the-art methods in end-user programming and shared autonomy and its implementation in sanding applications. We demonstrate the use of the system in two types of sanding tasks, situated in aircraft manufacturing, that highlight two potential workflows within the human-robot teaming setup. We conclude by discussing challenges and opportunities in human-robot teaming identified during the development, application, and demonstration of our system.

Remotely programming robots to execute tasks often relies on registering objects of interest in the robot's environment. Frequently, these tasks involve articulating objects such as opening or closing a valve. However, existing human-in-the-loop methods for registering objects do not consider articulations and the corresponding impact to the geometry of the object, which can cause the methods to fail. In this work, we present an approach where the registration system attempts to automatically determine the object model, pose, and articulation for user-selected points using nonlinear fitting and the iterative closest point algorithm. When the fitting is incorrect, the operator can iteratively intervene with corrections after which the system will refit the object. We present an implementation of our fitting procedure for one degree-of-freedom (DOF) objects with revolute joints and evaluate it with a user study that shows that it can improve user performance, in measures of time on task and task load, ease of use, and usefulness compared to a manual registration approach. We also present a situated example that integrates our method into an end-to-end system for articulating a remote valve.

Drones can provide a minimally-constrained adapting camera view to support robot telemanipulation. Furthermore, the drone view can be automated to reduce the burden on the operator during teleoperation. However, existing approaches do not focus on two important aspects of using a drone as an automated view provider. The first is how the drone should select from a range of quality viewpoints within the workspace (e.g., opposite sides of an object). The second is how to compensate for unavoidable drone pose uncertainty in determining the viewpoint. In this paper, we provide a nonlinear optimization method that yields effective and adaptive drone viewpoints for telemanipulation with an articulated manipulator. Our first key idea is to use sparse human-in-the-loop input to toggle between multiple automatically-generated drone viewpoints. Our second key idea is to introduce optimization objectives that maintain a view of the manipulator while considering drone uncertainty and the impact on viewpoint occlusion and environment collisions. We provide an instantiation of our drone viewpoint method within a drone-manipulator remote teleoperation system. Finally, we provide an initial validation of our method in tasks where we complete common household and industrial manipulations.

Manipulations of a constrained object often use a non-rigid grasp that allows the object to rotate relative to the end effector. This orientation slip strategy is often present in natural human demonstrations, yet it is generally overlooked in methods to identify constraints from such demonstrations. In this paper, we present a method to model and recognize prehensile orientation slip in human demonstrations of constrained interactions. Using only observations of an end effector, we can detect the type of constraint, parameters of the constraint, and orientation slip properties. Our method uses a novel hierarchical model selection method that is informed by multiple origins of physics-based evidence. A study with eight participants shows that orientation slip occurs in natural demonstrations and confirms that it can be detected by our method.

We present situated live programming for human-robot collaboration, an approach that enables users with limited programming experience to program collaborative applications for human-robot interaction. Allowing end users, such as shop floor workers, to program collaborative robots themselves would make it easy to “retask” robots from one process to another, facilitating their adoption by small and medium enterprises. Our approach builds on the paradigm of trigger-action programming (TAP) by allowing end users to create rich interactions through simple trigger-action pairings. It enables end users to iteratively create, edit, and refine a reactive robot program while executing partial programs. This live programming approach enables the user to utilize the task space and objects by incrementally specifying situated trigger-action pairs, substantially lowering the barrier to entry for programming or reprogramming robots for collaboration. We instantiate situated live programming in an authoring system where users can create trigger-action programs by annotating an augmented video feed from the robot’s perspective and assign robot actions to trigger conditions. We evaluated this system in a study where participants (n = 10) developed robot programs for solving collaborative light-manufacturing tasks. Results showed that users with little programming experience were able to program HRC tasks in an interactive fashion and our situated live programming approach further supported individualized strategies and workflows. We conclude by discussing opportunities and limitations of the proposed approach, our system implementation, and our study and discuss a roadmap for expanding this approach to a broader range of tasks and applications.

In this paper we provide a method for identifying and temporally localizing tactile force actions from measured force signals. Our key idea is to use the continuous wavelet transform (CWT) with the Complex Morlet wavelet to transform force signals into feature vectors amenable to machine learning algorithms. Our method uses these feature vectors to train a classifier that recognizes different actions. We demonstrate our approach in a system that records human activities with an instrumented set of tongs. Our system successfully identifies a wide range of actions based on a small set of labeled examples.

Localization of steerable catheters in minimally invasive surgery is critical with respect to patient safety, surgeon manipulation, and procedural efficacy. While there are many potential benefits to patients including shorter recovery times, less tissue trauma, and lower infection rates than traditional surgeries, localization of surgical tools is still an area of much research. Current technology offers several sensory modalities. However, each system has drawbacks which do not provide a clear best practice. This research focuses on incorporating redundant commonplace surgical sensing technologies to reduce the likelihood of errors, failures, or inherent sensor characteristics causing harm to the patient and/or surgeon while providing accurate localization. Dual particle filters are implemented using both a fluoroscopic-like stereo imaging system and an electromagnetic pose sensor for measurement updates in a prototype catheter testbed. A previously developed catheter model is modified to increase accuracy in the particle filter outputs which are combined using a weighted average based on each filter's particle statistics. Experimental results implementing the combined particle filter multi-modality algorithm in feedback control validates the algorithms ability to provide accurate localization in a surgical setting while overcoming sensor limitations and possible failure modes.

A new manipulation approach, referred to as interleaved continuum-rigid manipulation, which combines inherently safe, flexible actuated segments with more precise embedded rigid-link joints has recently been introduced [1], [2]. The redundantly actuated manipulator possesses the safety characteristics inherent in flexible segment devices while gaining some of the performance attributes of rigid-link joint systems. In this paper, we describe a general controller developed for an interleaved manipulator. The controller is implemented on a clinically-relevant prototype, the results of which demonstrate the advantages of an interleaved manipulator. We also consider kinematic drivers of the interleaved manipulator workspace, showing that careful kinematic considerations can substantially improve manipulator workspace and task accuracy.

Tendon driven continuum robots have attracted a significant amount of attention in the last decade. As a result several discrete and continuous modeling approaches have been explored in order to understand their behavior. The following article proposes an analytical model describing the influence of frictional effects between the tendon and its conduit in tendon driven continuum robotic manipulators. Existing constant curvature models ignore internal device friction completely and treat it as a negligible effect. This results in an assumption that the tendon tension is always constant along the length of the continuum manipulator. While this assumption is reasonable for small articulation angles, internal device friction can significantly change internal tension distribution deteriorating performance of constant curvature models at high articulation angles. The proposed model departs from the constant tension assumption and predicts a tension distribution along the tendon. From the tension distribution a curvature distribution is determined. It is observed that the proposed nonlinear friction model performs better than the existing linear elastic constant curvature model. Following this, a consequence of internal device friction: hysteresis effects are predicted. An algorithm that estimates tendon tension distribution given applied tendon tension history is proposed.

A new manipulation approach, referred to as interleaved continuum-rigid manipulation, which combines inherently safe, flexible actuated segments with more precise embedded rigid-link joints is described. The redundantly actuated manipulator possesses the safety characteristics inherent in flexible segment devices while gaining some of the performance gains possible with rigid-link joint systems. A demonstration prototype was developed, the purpose of which was to explore the design space as well as demonstrate the feasibility of the approach in a clinically-relevant form. The overall design is described along with performance data evaluating its functionality.

In recent years, minimally-invasive surgical systems based on flexible robotic manipulators have met with success. One major advantage of the flexible manipulator approach is its superior safety characteristics as compared to rigid manipulators. However, their soft compliant structure, in combination with internal friction, results in poor position and force regulation and has limited their use to simpler surgical procedures. In this paper, we discuss a new approach to continuum robotic manipulation that combines flexible, actively actuated continuum segments with small, limited stroke rigid-link actuators. The small rigid-link joints are interleaved between successive continuum segments and provide a redundant motion and error correction capability. The authors refer to this approach as interleaved continuum-rigid manipulation. In this paper, we describe the overall approach and investigate its performance using a one degree-of-freedom testbed and two degree-of-freedom planar simulation.

Continuum manipulators are gaining widespread acceptance in commercial robotics, particularly in the medical field, where their compliance allows a large benefit for patient safety. However, this compliance also makes precise position control of these manipulators quite difficult. This paper presents two closed-loop control implementations applied to a small scale continuum manipulator. These implementations are both based on manipulator tip position feedback from an electromagnetic sensor. The command tracking and disturbance rejection properties of the two control implementations are shown to be approximately equivalent, and provide improved position control when compared to open-loop control, without sacrificing system stability.

Robotic catheters are gaining widespread use in the medical community for cardiac, neurological and other surgical interventions. However, many of the catheters used in these procedures exhibit non-linear behavior, and thus present many difficulties in implementing effective open-loop control. Systems such as this have been shown to benefit from closed-loop control, however very little investigation has been done into 3D closed loop control of this class of manipulators. Initial investigations by the authors have shown greatly improved positioning accuracy and response with closed-loop control based on feedback from an electromagnetic localization sensor. This paper describes the control approach and experimental results, and provides a robotic catheter system overview.

In recent years, flexible robotic catheter systems have been developed for minimally invasive cardiac surgery. These systems commonly include a passive flexible catheter, which is biocompatible, and an active servo system which provides the actuation for the flexible catheter. To relate control actuation motion of the servo system to catheter motion, models of continuous catheters have been developed. A typical approach is to assume a kinematic model in which the catheter maintains constant curvature along its length when articulated. However, this approach cannot explain the nonlinear behavior of the catheter when the effect of internal friction is considered. In this paper we present a lumped-parameter modeling approach that allows for the inclusion of nonlinear effects, including friction. The proposed approach has been used to model a prototype catheter where internal friction is modeled using a modified Dahl friction model. To evaluate the efficacy of the modeling approach, the simulation results were compared to an experimental catheter prototype. Results show that the proposed approach significantly improves simulation accuracy as compared to the case where friction is not considered. In addition, the effects of friction on catheter performance were investigated using the developed modeling approach.

Continuum robotic manipulators articulate due to their inherent compliance. Tendon actuation leads to compression of the manipulator, extension of the actuators, and is limited by the practical constraint that tendons cannot support compression. In light of these observations, we present a new linear model for transforming desired beam configuration to tendon displacements andviceversa. We begin from first principles in solid mechanics by analyzing the effects of geometrically nonlinear tendon loads. These loads act both distally at the termination point and proximally along the conduit contact interface. The resulting model simplifies to a linear system including only the bending and axial modes of the manipulator as well as the actuator compliance. The model is then manipulated to form a concise mapping from beam configuration-space parameters tonredundant tendon displacements via the internal loads and strains experienced by the system. We demonstrate the utility of this model by implementing an optimal feasible controller. The controller regulates axial strain to a constant value while guaranteeing positive tendon forces and minimizing their magnitudes over a range of articulations. The mechanics-based model from this study provides insight as well as performance gains for this increasingly ubiquitous class of manipulators.

Many successful robotic manipulator designs have been introduced. However, there remains the challenge of designing a manipulator that possesses the inherent safety characteristics necessary for human-centered robotics. In this paper, we present a new actuation approach that has the requisite characteristics for inherent safety while maintaining the performance expected of modern designs. By drastically reducing the effective impedance of the manipulator while maintaining high frequency torque capability, we show that the competing design requirements of performance and safety can be successfully integrated into a single manipulation system.