This paper presents the results of a within-subjects user study with 27 participants over the age of 60, comparing the use of two different user interfaces for an assistive robot scooter. The graphical user interface (GUI) shows a representation of the environment on a 10-inch touchscreen. The tangible user interface (TUI) consists of a joystick, a box of buttons, and a projector - designed to keep the user's attention in the real world. Trends suggest that the TUI could help mitigate difficulty caused by highly cluttered environments, as well as differences in individual spatial reasoning ability, but additional studies are needed.

As robots tackle increasingly complex tasks, the need for explanations becomes essential for gaining trust and acceptance. Explainable robotic systems should not only elucidate failures when they occur but also predict and preemptively explain potential issues. This paper compares explanations from Reactive Systems, which detect and explain failures after they occur, to Proactive Systems, which predict and explain issues in advance. Our study reveals that the Proactive System fosters higher perceived intelligence and trust and its explanations were rated more understandable and timely. Our findings aim to advance the design of effective robot explanation systems, allowing people to diagnose and provide assistance for problems that may prevent a robot from finishing its task.

The need for robots to explain their failures grows as the variety and number of robots deployed in public, homes, and work environments increases. This paper extends our prior work utilizing explanation templates by comparing those Templated explanations to Generative explanations created by a Large Language Model. Our study surprisingly reveals that Templated explanations result in similar or higher perceived intelligence and trust while also being more understandable. Through our findings, we aim to provide considerations for effective robot explanation systems, ultimately enabling people to be able to understand and provide assistance to robots that have encountered unforeseen circumstances.

Human-in-the-loop robot teleoperation interfaces enable operators to control robots to complete complex tasks, as seen by the success of teams in the DARPA Robotics Challenge (DRC). In this work, we compare two human-in-the-loop planning interfaces, a 2D keyboard and mouse (KBM) interface modeled after those used in the DRC and a 3D virtual reality (VR) interface, for teleoperating a robot to perform navigation and manipulation tasks. In our study, we investigated operator performance, and cognitive workload while using the interface, as well as the perceived usability of each. We found that participants had better performance in both task types when using the KBM interface, however they experienced fewer collisions between the robot and the world in the VR interface. Given these findings, we recommend utilizing a KBM interface in low-risk situations where task performance is the primary factor. In high-risk scenarios, where collisions can be detrimental, we recommend using VR. With this work we aim to contribute to building effective and intuitive interfaces for human-in-the-loop planning to allow robots to complete complex tasks in challenging environments.

Robots need to explain their behavior to gain trust. Existing research has focused on explaining a robot’s current behavior, yet it remains unknown yet challenging how to provide explanations of past actions in an environment that might change after a robot’s actions, leading to critical missing causal information due to moved objects. We conducted an experiment (N = 665) investigating how a robot could help participants infer the missing causal information by replaying the past behavior physically, using verbal explanations, and projecting visual information onto the environment. Participants watched videos of the robot replaying its completion of an integrated mobile kitting task. During the replay, the objects are already gone, so participants needed to infer where an object was picked, where a ground obstacle had been, and where the object was placed. Based on the results, we recommend combining physical replay with speech and projection indicators (Replay-Project-Say) to help infer all the missing causal information (picking, navigation, and placement) from the robot’s past actions. This condition had the best outcome in both task-based—effectiveness, efficiency, and confidence—and team-based metrics—workload and trust. If one’s focus is efficiency, then we recommend projection markers for navigation inferences and verbal markers for placing inferences.

For mobile robots, mobile manipulators, and autonomous vehicles to safely navigate around populous places such as streets and warehouses, human observers must be able to understand their navigation intent. One way to enable such understanding is by visualizing this intent through projections onto the surrounding environment. But despite the demonstrated effectiveness of such projections, no open codebase with an integrated hardware setup exists. In this work, we detail the empirical evidence for the effectiveness of such directional projections, and share a robot-agnostic implementation of such projections, coded in C++ using the widely-used Robot Operating System (ROS) and rviz. Additionally, we demonstrate a hardware configuration for deploying this software, using a Fetch robot, and briefly summarize a full-scale user study that motivates this configuration. The code, configuration files (roslaunch and rviz files), and documentation are freely available on GitHub at https://github.com/umhan35/arrow_projection.

As development of robots with the ability to self-assess their proficiency for accomplishing tasks continues to grow, metrics are needed to evaluate the characteristics and performance of these robot systems and their interactions with humans. This proficiency-based human-robot interaction (HRI) use case can occur before, during, or after the performance of a task. This article presents a set of metrics for this use case, driven by a four-stage cyclical interaction flow: (1) robot self-assessment of proficiency (RSA), (2) robot communication of proficiency to the human (RCP), (3) human understanding of proficiency (HUP), and (4) robot perception of the human’s intentions, values, and assessments (RPH). This effort leverages work from related fields including explainability, transparency, and introspection, by repurposing metrics under the context of proficiency self-assessment. Considerations for temporal level (a priori, in situ, and post hoc) on the metrics are reviewed, as are the connections between metrics within or across stages in the proficiency-based interaction flow. This article provides a common framework and language for metrics to enhance the development and measurement of HRI in the field of proficiency self-assessment.

Robot interfaces often only use the visual channel. Inspired by Wickens’ Multiple Resource Theory, we investigated if the addition of audio elements would reduce cognitive workload and improve performance. Specifically, we designed a search and threat-defusal task (primary) with a memory test task (secondary). Eleven participants – predominantly first responders – were recruited to control a robot to clear all threats in a combination of four conditions of primary and secondary tasks in visual and auditory channels. While we did not find any statistically significant differences in performance or workload across subjects, making it questionable that Multiple Resource Theory could shorten longer-term task completion time and reduce workload. Our results suggest that considering individual differences for splitting interface modalities across multiple channels requires further investigation.

As autonomous robots continue to be deployed near people, robots need to be able to explain their actions. In this article, we focus on organizing and representing complex tasks in a way that makes them readily explainable. Many actions consist of sub-actions, each of which may have several sub-actions of their own, and the robot must be able to represent these complex actions before it can explain them. To generate explanations for robot behavior, we propose using Behavior Trees (BTs), which are a powerful and rich tool for robot task specification and execution. However, for BTs to be used for robot explanations, their free-form, static structure must be adapted. In this work, we add structure to previously free-form BTs by framing them as a set of semantic sets {goal, subgoals, steps, actions} and subsequently build explanation generation algorithms that answer questions seeking causal information about robot behavior. We make BTs less static with an algorithm that inserts a subgoal that satisfies all dependencies. We evaluate our BTs for robot explanation generation in two domains: a kitting task to assemble a gearbox, and a taxi simulation. Code for the behavior trees (in XML) and all the algorithms is available at github.com/uml-robotics/robot-explanation-BTs.

Although non-verbal cues such as arm movement and eye gaze can convey robot intention, they alone may not provide enough information for a human to fully understand a robot’s behavior. To better understand how to convey robot intention, we conducted an experiment ( N = 366 ) investigating the need for robots to explain , and the content and properties of a desired explanation such as timing , engagement importance , similarity to human explanations, and summarization . Participants watched a video where the robot was commanded to hand an almost-reachable cup and one of six reactions intended to show the unreachability : doing nothing (No Cue), turning its head to the cup (Look), or turning its head to the cup with the addition of repeated arm movement pointed towards the cup (Look & Point), and each of these with or without a Headshake. The results indicated that participants agreed robot behavior should be explained across all conditions, in situ , in a similar manner as what human explain, and provide concise summaries and respond to only a few follow-up questions by participants. Additionally, we replicated the study again with N = 366 participants after a 15-month span and all major conclusions still held.

Human–robot collaboration is becoming increasingly common in factories around the world; accordingly, we need to improve the interaction experiences between humans and robots working in these spaces. In this article, we report on a user study that investigated methods for providing information to a person about a robot’s intent to move when working together in a shared workspace through signals provided by the robot. In this case, the workspace was the surface of a tabletop. Our study tested the effectiveness of three motion-based and three light-based intent signals as well as the overall level of comfort participants felt while working with the robot to sort colored blocks on the tabletop. Although not significant, our findings suggest that the light signal located closest to the workspace—an LED bracelet located closest to the robot’s end effector—was the most noticeable and least confusing to participants. These findings can be leveraged to support human–robot collaborations in shared spaces.

The FetchIt! Mobile Manipulation Challenge, held at the IEEE International Conference on Robots and Automation (ICRA) in May 2019, offered an environment with complex and integrated task sets, irregular objects, confined space, and machining, introducing new challenges in the mobile manipulation domain. Here we describe our efforts to address these challenges by demonstrating the assembly of a kit of mechanical parts in a caddy. In addition to implementation details, we examine the issues in this task set extensively, and we discuss our software architecture in the hope of providing a base for other researchers. To evaluate performance and consistency, we conducted 20 full runs, then examined failure cases with possible solutions. We conclude by identifying future research directions to address the open challenges.

It has been claimed that a main advantage of cognitive architectures (compared to other types of specialized robotic architectures) is that they are task-general and can thus learn to perform any task as long as they have the right perceptual and action primitives. In this paper, we provide empirical evidence for this claim by directly comparing a high-performing custom robotic architecture developed for the standardized robotic "FetchIt!" challenge task to a hybrid cognitive robotic architecture that allows for online one-shot task learning and task modifications through natural language instructions. The results show that there is no disadvantage of running the hybrid architecture (i.e., no significant difference in overall performance or computational overhead compared to the custom architecture) while adding the flexibility of online one-shot task instruction and modification not available in the custom architecture.

Most research on human-robot handovers focuses on how the robot should approach human receivers and notify them of the readiness to take an object; few studies have investigated the effects of different release behaviors. Not releasing an object when a person desires to take it breaks handover fluency and creates a bad handover experience. In this paper, we investigate the effects of different release behaviors. Specifically, we study the benefits of a proactive release, during which the robot actively detects a human grasp effort pattern. In a 36-participant user study11The study is ready to reproduce with a Baxter robot. The code and environment setup is available at https://github.com/umhan35/handover_moveit, results suggest proactive release is more efficient than rigid release (which only releases when the robot is fully stopped) and passive release (the robot detects pulling by checking if a threshold value is reached). Subjectively, the overall handover experience is improved: the proactive release is significantly better in terms of handover fluency and ease-of-taking.

Automated grasping has a long history of research that is increasing due to interest from industry. One grand challenge for robotics is Universal Picking: the ability to robustly grasp a broad variety of objects in diverse environments for applications from warehouses to assembly lines to homes. Although many researchers now openly share code and data, it is challenging to compare and/or reproduce experimental results to identify which aspects of which approaches work best due to variations in assumptions and experimental protocols, e.g., sensors, lighting, robot arms, grippers, and objects.

Many people with motor disabilities are unable to complete activities of daily living (ADLs) without assistance. This paper describes a complete robotic system developed to provide mobile grasping assistance for ADLs. The system is comprised of a robot arm from a Rethink Robotics Baxter robot mounted to an assistive mobility device, a control system for that arm, and a user interface with a variety of access methods for selecting desired objects. The system uses grasp detection to allow previously unseen objects to be picked up by the system. The grasp detection algorithms also allow for objects to be grasped in cluttered environments. We evaluate our system in a number of experiments on a large variety of objects. Overall, we achieve an object selection success rate of 88% and a grasp detection success rate of 90% in a non-mobile scenario, and success rates of 89% and 72% in a mobile scenario.

We have designed an augmented reality (AR) game for children with cerebral palsy (CP) to perform home-based neurorehabilitation. A Myo armband detects electromyographic (EMG) signals and accelerometer data from the arm, and a trained classifier determines whether the neuromotor performance of the arm satisfies the expectation of the exercise. The user can move a virtual object only through therapist-prescribed motor movement. The user completes the exercise by moving the virtual object to some targets displayed in the glass.

Human-robot interaction involving the failure of autonomous robots is not yet well understood. We conducted two online surveys with a total of 1200 participants in which people assessed situations where an autonomous robot experienced different kinds of failure. This information was used to construct a measurement scale of people's reaction to failure where positive values correspond with increasingly positive reactions and negative values with negative reactions. We then used this scale to compare different kinds of failure situations, including the severity of the failures, the context risk involved, and the effectiveness of different kinds of recovery strategies. We found evidence that the effectiveness of recovery strategies depends on the task, context, and severity of failure.

A significant amount of robotics research over the past decade has shown that many children with autism spectrum disorders (ASD) have a strong interest in robots and robot toys, concluding that robots are potential tools for the therapy of individuals with ASD. However, clinicians, who have the authority to approve robots in ASD therapy, are not convinced about the potential of robots. One major reason is that the research in this domain does not have a strong focus on the efficacy of robots. Robots in ASD therapy are end-user oriented technologies, the success of which depends on their demonstrated efficacy in real settings. This paper focuses on measuring the efficacy of robots in ASD therapy and, based on the data from a feasibility study, shows that the human-robot interaction (HRI) metrics commonly used in this research domain might not be sufficient.

A significant amount of research has been conducted regarding the technical aspects of haptic feedback. However, the design of effective haptic feedback behaviors for controlling ground-based mobile robots is not yet well understood from a human-robot interaction perspective. Past research of haptic feedback behaviors for mobile robots has sometimes made use of control paradigms that do not appropriately map to teleoperation or supervision tasks. Furthermore, evaluation of haptic behaviors has not been systematic and often only demonstrates feasibility. As a result, comparing various techniques is difficult. In this article, we focus on how haptic control research could be improved in the domain of teleoperation and supervision of ground-based mobile robots through the introduction of a haptic evaluation toolkit.

This workshop introduces participants to the Artbotics program, which combines art and robotics to teach students about computer science while creating kinetic, interactive sculptures. The material covered will be provided in introductory fashion, requiring no prior experience with computer science, art, or robotics. The Lego Mindstorms NXT platform will be used to create two projects during the workshop: a spirograph-like drawing produced by programming a car holding a marker to drive using a sequence of motor movements (teaching the need for looping in programming) and an interactive, kinetic sculpture that reacts to sensor input (teaching the need for decisions in programming and building simple mechanisms). Examples of both projects can be seen at youtube.com/artbotics. The workshop will end with a short discussion of lessons learned and best practices, using examples from previous Artbotics programs for a variety of ages. Topics will include appropriate time frames, how to best use limited resources, and appropriate levels of depth for each age group. The workshop administrators will be providing laptops with the proper Lego Mindstorms NXT software, Lego Mindstorms NXT kits, and all needed building materials.

This video shows a demonstration of a fully autonomous robot, an iRobot ATRV-JR, which can be given commands using natural language. Users type commands to the robot on a tablet computer, which are then parsed and processed using semantic analysis. This information is used to build a plan representing the high level autonomous behaviors the robot should perform [2][1]. The robot can be given commands to be executed immediately (e.g., "Search the floor for hostages.") as well as standing orders for use over the entire run (e.g., "Let me know if you see any bombs.").

No abstract available.

Prior work in human-autonomy interaction has focused on plant systems that operate in highly structured environments. In contrast, many human-robot interaction (HRI) tasks are dynamic and unstructured, occurring in the open world. It is our belief that methods developed for the measurement and modeling of trust in traditional automation need alteration in order to be useful for HRI. Therefore, it is important to characterize the factors in HRI that influence trust. This study focused on the influence of changing autonomy reliability. Participants experienced a set of challenging robot handling scenarios that forced autonomy use and kept them focused on autonomy performance. The counterbalanced experiment included scenarios with different low reliability windows so that we could examine how drops in reliability altered trust and use of autonomy. Drops in reliability were shown to affect trust, the frequency and timing of autonomy mode switching, as well as participants' self-assessments of performance. A regression analysis on a number of robot, personal, and scenario factors revealed that participants tie trust more strongly to their own actions rather than robot performance.

It is challenging to quantitatively measure a user's trust in a robot system using traditional survey methods due to their invasiveness and tendency to disrupt the flow of operation. Therefore, we analyzed data from an existing experiment to identify measures which (1) have face validity for measuring trust and (2) align with the collected post-run trust measures. Two measures are promising as real-time indications of a drop in trust. The first is the time between the most recent warning and when the participant reduces the robot's autonomy level. The second is the number of warnings prior to the reduction of the autonomy level.

This video demonstrates three users sharing control of eight simulated robots with a Microsoft Surface and two Apple iPads using our Multi-user Multi-touch Multi-robot Command and Control Interface.

Telepresence robots can be thought of as embodied video conferencing on wheels. Companies producing these robots imagine them being used in a wide variety of situations (e.g., ad-hoc conversations at the office, inspections and troubleshooting at factories, and patient rounds at medical facilities). In July and August 2010, we examined office-related use cases in a series of studies using two prototype robots (Anybots' QB and VGo Communications' VGo). In this paper, we present two studies: conference room meetings (n=6) and moving hallway conversations (n=24). We discuss who might benefit from using telepresence robots, in what scenarios, and the features that telepresence robots must incorporate for use in ad-hoc interactions.

Controlling the movements of mobile robots, including driving the robot through the world and panning the robot's cameras, typically requires many physical joysticks, buttons, and switches. Operators will often employ a technique called "chording" to cope with this situation. Much like a piano player, the operator will simultaneously actuate multiple joysticks and switches with his or her hands to create a combination of complimentary movements. However, these controls are in fixed locations and unable to be reprogrammed easily. Using a Microsoft Surface multi-touch table, we have designed an interface that allows chording and simultaneous multi-handed interaction anywhere that the user wishes to place his or her hands. Taking inspiration from the biomechanics of the human hand, we have created a dynamically resizing, ergonomic, and multi-touch controller (the DREAM Controller). This paper presents the design and testing of this controller with an iRobot ATRV-JR robot.

No abstract available.

Recent developments in multi-touch technologies have exposed fertile ground for research in enriched human-robot interaction. Although multi-touch technologies have been used for virtual 3D applications, to the authors' knowledge, ours is the first study to explore the use of a multi-touch table with a physical robot agent. This baseline study explores the control of a single agent with a multi-touch table using an adapted, previously studied, joystick-based interface. We performed a detailed analysis of users' interaction styles with two complex functions of the multi-touch interface and isolated mismatches between user expectations and interaction functionality.

Accessibility is a challenge for people with disabilities. Differences in cognitive ability, sensory impairments, motor dexterity, behavioral skills, and social skills must be taken into account when designing interfaces for assistive devices. Flexible interfaces tuned for individuals, instead of custom-built solutions, may benefit a larger number of people. The development and evaluation of a flexible interface for controlling a wheelchair mounted robotic arm is described in this paper. There are four versions of the interface based on input device (touch screen or joystick) and a moving or stationary shoulder camera. We describe results from an eight week experiment conducted with representative end users who range in physical and cognitive ability.

Good situation awareness (SA) is especially necessary when robots and their operators are not collocated, such as in urban search and rescue (USAR). This paper compares how SA is attained in two systems: one that has an emphasis on video and another that has an emphasis on a three-dimensional map. We performed a within-subjects study with eight USAR domain experts. To analyze the utterances made by the participants, we developed a SA analysis technique, called LASSO, which includes five awareness categories: location, activities, surroundings, status, and overall mission. Using our analysis technique, we show that a map-centric interface is more effective in providing good location and status awareness while a video-centric interface is more effective in providing good surroundings and activities awareness.

A renewed interest in multi-touch tabletop display technologies has research groups investigating alternate human computer interaction techniques. The UMass Lowell Robotics Lab recently began investigating the use of the Mitsubishi DiamondTouch display, which supports multiple points of contact, multiple users, and rich gesture recognition. These capabilities naturally lend themselves to disaster response command and control. This video describes two prototype urban search and rescue command and control interfaces that have been investigated for use in tabletop configurations.

Polymorphic, or shape-shifting, robots can normally tackle more types of tasks than non-polymorphic robots due to their flexible morphology. Their versatility adds to the challenge of designing a human interface, however. To investigate the utility of providing awareness information about the robot's physical configuration (or "pose"), we performed a within-subjects experiment with presence or absence of pose information being the independent variable. We found that participants were more likely to tip the robot or have it ride up on obstacles when they used the display that lacked pose information and also more likely to move the robot to the highest position to become oriented. There was no significant difference in the number of times that participants bumped into obstacles, however, indicating that having more awareness of the robot's state does not affect awareness of the robots' immediate surroundings. Participants thought the display with pose information was easier to use, helped their performance and was more enjoyable than having no pose information. Future research directions point toward providing recommendations to robot operators for which pose they should change to given the terrain to be traversed.

This tutorial presents the current status of research in interactions with robots, including adaptive robots/interfaces, speech, gestures, virtual reality, and social interactions. Different user interface designs will be shown and discussed during the tutorial. Human-robot interaction (HRI) guidelines, evaluation methodologies and metrics currently used by the community will be presented. Research needs will also be discussed. Participants will work in small groups to design a robotic application as well as an evaluation plan.

In this paper we report on the analysis of critical incidents during an urban search and rescue robot competition where critical incidents are defined as a situation where the robot could potentially cause damage to itself, the victim, or the environment. We look at the features present in the human-robot interface that contributed to success in different tasks needed in urban search and rescue and present guidelines for human-robot interaction design.

In this article we describe a programming framework called Pyro, which provides a set of abstractions that allows students to write platform-independent robot programs. This project is unique because of its focus on the pedagogical implications of teaching mobile robotics via a top-down approach. We describe the background of the project, its novel abstractions, its library of objects, and the many learning modules that have been created from which curricula for different types of courses can be drawn. Finally, we explore Pyro from the students' perspective in a case study.

This paper provides a set of definitions that form a framework for describing the types of awareness that humans have of robot activities and the knowledge that robots have of the commands given them by humans. As a case study, we applied this human-robot interaction (HRI) awareness framework to our analysis of the HRI approaches used at an urban search and rescue competition. We determined that most of the critical incidents (e.g., damage done by robots to the test arena) were directly attributable to lack of one or more kinds of HRI awareness.

In this article we describe a programming framework called Pyro, which provides a set of abstractions that allows students to write platform-independent robot programs. This project is unique because of its focus on the pedagogical implications of teaching mobile robotics via a top-down approach. We describe the background of the project, its novel abstractions, its library of objects, and the many learning modules that have been created from which curricula for different types of courses can be drawn. Finally, we explore Pyro from the students' perspective in a case study.

April 01 2000 Artificial Intelligence and Mobile Robots: Case Studies of Successful Robot Systems Artificial Intelligence and Mobile Robots: Case Studies of Successful Robot Systems. Edited by D.Kortenkamp, R. P.Bonasso, and R.Murphy, MIT Press, 1998 Holly A. Yanco Holly A. Yanco MIT Artificial Intelligence Laboratory, 545 Technology Square, Room 713, Cambridge, MA 02139 [email protected] Search for other works by this author on: This Site Google Scholar Author and Article Information Holly A. Yanco MIT Artificial Intelligence Laboratory, 545 Technology Square, Room 713, Cambridge, MA 02139 [email protected] Online Issn: 1530-9185 Print Issn: 1064-5462 © 2000 Massachusetts Institute of Technology2000 Artificial Life (2000) 6 (2): 181–183. https://doi.org/10.1162/106454600568393 Cite Icon Cite Permissions Share Icon Share Twitter LinkedIn Views Icon Views Article contents Figures & tables Video Audio Supplementary Data Peer Review Search Site Citation Holly A. Yanco; Artificial Intelligence and Mobile Robots: Case Studies of Successful Robot Systems. Artif Life 2000; 6 (2): 181–183. doi: https://doi.org/10.1162/106454600568393 Download citation file: Ris (Zotero) Reference Manager EasyBib Bookends Mendeley Papers EndNote RefWorks BibTex toolbar search Search Dropdown Menu toolbar search search input Search input auto suggest filter your search All ContentAll JournalsArtificial Life Search Advanced Search This content is only available as a PDF. © 2000 Massachusetts Institute of Technology2000 Article PDF first page preview Close Modal You do not currently have access to this content.