Non-verbal visual symbols and displays play an important role in communication when humans and robots work collaboratively. However, few studies have investigated how different types of non-verbal cues affect objective task performance, especially in a dynamic environment that requires real-time decision-making. In this work, we designed a collaborative navigation task where the user and the robot only had partial information about the map on each end and thus the users were forced to communicate with a robot to complete the task. We conducted our study in a public space and recruited 37 participants who randomly passed by our setup. Each participant collaborated with a robot utilizing either animated anthropomorphic eyes and animated icons, or static anthropomorphic eyes and static icons. We found that participants that interacted with a robot with animated displays reported the greatest level of trust and satisfaction; that participants interpreted static icons the best; and that participants with a robot with static eyes had the highest completion success. These results suggest that while animation can foster trust with robots, human-robot communication can be optimized by the addition of familiar static icons that may be easier for users to interpret. We published our code, designed symbols, and collected results online at: https://github.com/mattufts/huamn_Cozmo_interaction.

Learning from Demonstration (LfD) has become a popular approach for robots to learn new skills, despite most LfD methods suffering from imperfections in human demonstrations. Prior work in LfD often characterizes the sub-optimalities in human demonstrations as random noise. In this paper, we explored non-optimal behaviors in non-expert demonstrations and showed that these behaviors are not random and have systematic patterns: they form systematic demonstration sidetracks. We used a public space study dataset from our previous work with 40 participants and a long-horizon robot task. We recreated the experimental setup in a simulation and annotated all the demonstrations. We identified four types of demonstration sidetracks, Exploration, Mistake, Alignment, and Pause, and one control pattern one-dimension control. We found that instead of being random and rare, demonstration sidetracks frequently appear in non-expert demonstrations across all participants, and the distribution of demonstration sidetracks is associated with the robot task temporarily and spatially. Moreover, we found that users’ control patterns are affected by the control interface. Our findings highlight the need for better models of sub-optimal demonstrations, offering insights to improve LfD algorithms and reduce gaps between lab-based training and real-world applications. All the demonstrations, infrastructures, and annotations are available at https://github.com/AABL-Lab/Human-Demonstration-Sidetracks.

Reinforcement Learning from Human Feedback has recently achieved significant success in various fields, and its performance is highly related to feedback quality. While much prior work acknowledged that human teachers’ characteristics would affect human feedback patterns, there is little work that has closely investigated the actual effects. In this work, we designed an exploratory study investigating how human feedback patterns are associated with human characteristics. We conducted a public space study with two long-horizon tasks and 46 participants. We found that feedback patterns are not only correlated with task statistics, such as rewards, but also correlated with participants’ characteristics, especially robot experience and educational background. Additionally, we demonstrated that human feedback value can be more accurately predicted with human characteristics compared to only using task statistics. All human feedback and characteristics we collected, and codes for our data collection and predicting more accurate human feedback are available at https://github.com/AABL-Lab/CHARM.

Human-in-the-loop learning is gaining popularity, particularly in the field of robotics, because it leverages human knowledge about real-world tasks to facilitate agent learning. When people instruct robots, they naturally adapt their teaching behavior in response to changes in robot performance. While current research predominantly focuses on integrating human teaching dynamics from an algorithmic perspective, understanding these dynamics from a human-centered standpoint is an under-explored, yet fundamental problem. Addressing this issue will enhance both robot learning and user experience. Therefore, this paper explores one potential factor contributing to the dynamic nature of human teaching: robot errors. We conducted a user study to investigate how the presence and severity of robot errors affect three dimensions of human teaching dynamics: feedback granularity, feedback richness, and teaching time, in both forced-choice and open-ended teaching contexts. The results show that people tend to spend more time teaching robots with errors, provide more detailed feedback over specific segments of a robot’s trajectory, and that robot error can influence a teacher’s choice of feedback modality. Our findings offer valuable insights for designing effective interfaces for interactive learning and optimizing algorithms to better understand human intentions.

Shared control approaches to robot assistance, which predict a user's goal based on their control input and provide autonomous assistance towards the predicted goal, typically assume that user behavior remains the same despite the presence of the assistance and rely on this assumption to infer user goals. However, people operating assisted systems continuously observe the robot behaving differently from their expectations, which may lead them to adapt their control behavior to better achieve their desired outcomes. In this paper, we show that users both change their control behavior when assistance is added and describe these changes as responses to the new system dynamics. In a computer-based bubble popping study, participants report changing their strategies with different levels of assistance, and analysis of their actual control input validates this change. In an in-the-wild robot study, participants teleoperated a robot to pick up a cup despite the presence of "assistance'' that drives the system away from the true goals of the task. Participants can overcome the "assistance" and reach the goal, which requires them to correct for the novel system dynamics. These results motivate further research in user-centered design and evaluation of assistive systems that treat the user as intentional.

To expedite the development process of interactive reinforcement learning (IntRL) algorithms, prior work often uses perfect oracles as simulated human teachers to furnish feedback signals. These oracles typically derive from ground-truth knowledge or optimal policies, providing dense and error-free feedback to a robot learner without delay. However, this machine-like feedback behavior fails to accurately represent the diverse patterns observed in human feedback, which may lead to unstable or unexpected algorithm performance in real-world human-robot interaction. To alleviate this limitation of oracles in oversimplifying user behavior, we propose a method for modeling variation in human feedback that can be applied to a standard oracle. We present a model with 5 dimensions of feedback variation identified in prior work. This model enables the modification of feedback outputs from perfect oracles to introduce more human-like features. We demonstrate how each model attribute can impact on the learning performance of an IntRL algorithm through a simulation experiment. We also conduct a proof-of-concept study to illustrate how our model can be populated from people in two ways. The modeling results intuitively present the feedback variation among participants and help to explain the mismatch between oracles and human teachers. Overall, our method is a promising step towards refining simulated oracles by incorporating insights from real users.

Reinforcement Learning (RL) is an effective method for robots to learn tasks. However, in typical RL, end-users have little to no control over how the robot does the task after the robot has been deployed. To address this, we introduce the idea of online behavior modification, a paradigm in which users have control over behavior features of a robot in real-time as it autonomously completes a task using an RL-trained policy. To show the value of this user-centered formulation for human-robot interaction, we present a behavior-diversity--based algorithm, Adjustable Control Of RL Dynamics (ACORD), and demonstrate its applicability to online behavior modification in simulation and a user study. In the study (n =23), users adjust the style of paintings as a robot traces a shape autonomously. We compare \algoshort to RL and Shared Autonomy (SA), and show \algoshort affords user-preferred levels of control and expression, comparable to SA, but with the potential for autonomous execution and robustness of RL. The code for this paper is available at https://github.com/AABL-Lab/HRI2024_ACORD

People effortlessly manipulate fluids due to their learned understanding of fluid dynamics, while robots struggle with complex fluid dynamic calculations, particularly in tasks like pouring. To enhance assistive robots in such tasks, we propose involving users in correcting and providing feedback by visualizing the planned pouring trajectories before they are executed. This paper investigates whether people can predict robotic pouring outcomes and make adjustments to minimize spills, using visualization devices like augmented reality. In a human-participant study, participants evaluated and adjusted robot pouring behaviors of unique configurations for various source containers. Results highlight the effectiveness of visualization tools such as augmented reality headsets, as well as traditional 2D display, especially with specific pouring parameters, and users noted their benefits in open-ended responses. This research illuminates the potential for human-robot collaboration in fluid manipulation tasks, with visualization tools reducing spills in robot-controlled pours.

Prior research has suggested that equalizing participation may benefit group performance and group cohesion. Robot-moderated groups have largely focused on improving member participation by focusing on the least performing member and do not consider the frequency of interaction or type of interaction. We introduce a robot moderator that varies its frequency and interaction types to observe the impact on groups in terms of performance and group cohesion. We investigate this in user studies across four conditions for equalizing participation. Leveraging Bayesian statistical methods that can evaluate evidence both for and against the null hypothesis, we find evidence that neither personalizing robot actions nor balancing the target of the robot’s assistance affected user experience in the group (as measured by performance, group cohesion, and variance of participation). We find a lack of evidence for equalization of participation impacting performance and group cohesion. Additionally, we also find positive evidence against the correlation of equalized participation and group cohesion in our task and weak evidence against equalized participation correlating with performance. In addition to guiding future researchers regarding robot behaviors that may not be effective in affecting groups, this work is an important negative result suggesting that equalizing participation may not be adequate to improve group performance and cohesion in all tasks.

It is crucial that users are empowered to take advantage of the functionality of a robot and use their understanding of that functionality to perform novel and creative tasks. Given a robot trained with Reinforcement Learning (RL), a user may wish to leverage that autonomy along with their familiarity of how they expect the robot to behave to collaborate with the robot. One technique is for the user to take control of some of the robot’s action space through teleoperation, allowing the RL policy to simultaneously control the rest. We formalize this type of shared control as Partitioned Control (PC). However, this may not be possible using an out-of-the-box RL policy. For example, a user’s control may bring the robot into a failure state from the policy’s perspective, causing it to act unexpectedly and hindering the success of the user’s desired task. In this work, we formalize this problem and present Imaginary Out-of-Distribution Actions, IODA, an initial algorithm which empowers users to leverage their expectations of a robot’s behavior to accomplish new tasks. We deploy IODA in a user study with a real robot and find that IODA leads to both better task performance and a higher degree of alignment between robot behavior and user expectation. We also show that in PC, there is a strong and significant correlation between task performance and the robot’s ability to meet user expectations, highlighting the need for approaches like IODA. Code is available at https://github.com/AABL-Lab/ioda_roman_2024

Enhancing the expressiveness of human teaching is vital for both improving robots’ learning from humans and the human-teaching-robot experience. In this work, we characterize and test a little-used teaching signal: progress, designed to represent the completion percentage of a task. We conducted two online studies with 76 crowd-sourced participants and one public space study with 40 non-expert participants to validate the capability of this progress signal. We find that progress indicates whether the task is successfully performed, reflects the degree of task completion, identifies unproductive but harmless behaviors, and is likely to be more consistent across participants. Furthermore, our results show that giving progress does not require extra workload and time. An additional contribution of our work is a dataset of 40 non-expert demonstrations from the public space study through an ice cream topping-adding task, which we observe to be multi-policy and sub-optimal, with sub-optimality not only from teleoperation errors but also from exploratory actions and attempts. The dataset is available at https://github.com/TeachingwithProgress/Non-Expert_Demonstrations.

Motivated by the philosophical overlap between makerspace culture and the needs of assistive technology users, we investigated the ways that makerspaces can support the development of new technologies by and for disabled makers. Using eleven semi-structured interviews with makerspace operators and disabled makerspace users, we identified five categories of barriers to makerspace participation: recruitment/outreach, physical access, financial, access to information, and belonging. Based on these interviews, we highlight ways makerspaces can better welcome makers with disabilities: enabling members to create adaptive technologies for the space (“makerspacing the makerspace”), making the physical space and information within the space accessible, and fostering belonging by building relationships with the disability community. Overall, our work contributes to our understanding of the possibilities and challenges of connecting the disabled community with the maker community and suggests new directions for collaboration, especially towards building hybrid makerspaces that provide multiple modalities for connection and creativity.

Learning from human feedback is an effective way to improve robotic learning in exploration-heavy tasks. Compared to the wide application of binary human feedback, scalar human feedback has been used less because it is believed to be noisy and unstable. In this paper, we compare scalar and binary feedback, and demonstrate that scalar feedback benefits learning when properly handled. We collected binary or scalar feedback respectively from two groups of crowdworkers on a robot task. We found that when considering how consistently a participant labeled the same data, scalar feedback led to less consistency than binary feedback; however, the difference vanishes if small mismatches are allowed. Additionally, scalar and binary feedback show no significant differences in their correlations with key Reinforcement Learning targets. We then introduce Stabilizing TEacher Assessment DYnamics (STEADY) to improve learning from scalar feedback. Based on the idea that scalar feedback is muti-distributional, STEADY reconstructs underlying positive and negative feedback distributions and re-scales scalar feedback based on feedback statistics. We show that models trained with scalar feedback + STEADY outperform baselines, including binary feedback and raw scalar feedback, in a robot reaching task with non-expert human feedback. Our results show that both binary feedback and scalar feedback are dynamic, and scalar feedback is a promising signal for use in interactive Reinforcement Learning.

Assistive robot manipulators help people with upper motor impairments perform tasks by themselves. However, teleoperating a robot to perform complex tasks is difficult. Shared control algorithms make this easier: these algorithms predict the user’s goal, autonomously generate a plan to accomplish the goal, and fuse that plan with the user’s input. To accurately predict the user’s goal, these algorithms typically use the user’s input command (e.g., joystick input) directly. We use another sensing modality: the user’s natural eye gaze behavior, which is highly task-relevant and informative early in the task. We develop an algorithm using hidden Markov models to infer goals from natural eye gaze behavior that appears while users are teleoperating a robot. We show that gaze-based predictions outperform goal prediction based on the control input and that our sequence model improves the prediction quality relative to gaze-based aggregate models.

Human-robot collaboration systems benefit from recognizing people's intentions. This capability is especially useful for collaborative manipulation applications, in which users operate robot arms to manipulate objects. For collaborative manipulation, systems can determine users' intentions by tracking eye gaze and identifying gaze fixations on particular objects in the scene (i.e., semantic gaze labeling). Translating 2D fixation locations (from eye trackers) into 3D fixation locations (in the real world) is a technical challenge. One approach is to assign each fixation to the object closest to it. However, calibration drift, head motion, and the extra dimension required for real-world interactions make this position matching approach inaccurate. In this work, we introduce velocity features that compare the relative motion between subsequent gaze fixations and a finite set of known points and assign fixation position to one of those known points. We validate our approach on synthetic data to demonstrate that classifying using velocity features is more robust than a position matching approach. In addition, we show that a classifier using velocity features improves semantic labeling on a real-world dataset of human-robot assistive manipulation interactions.

Threaded fasteners are prevalent throughout modern manufacturing. Thus, as the demand for automation in manufacturing increases, so does the demand for automated threaded fastening systems. However, many fundamental issues and engineering challenges still hinder robustness in automation, particularly for smaller screws and critical product finishing requirements. This paper surveys the state of the art in threaded fastening automation and discusses open questions for further research. This survey covers the following areas: 1) fundamentals of threaded fastening, including basic concepts and definitions; 2) analysis of the entire assembly process (consisting of part feeding and orientation, pickup, alignment, and driving), including discussions of tools, control strategies, and other considerations; 3) failure modes and techniques to mitigate them; 4) threaded fastening systems and electromechanical approaches; and 5) open challenges and suggestions for future development. Understanding the current state of automation in threaded fastening will provide a foundation for researchers to advance this field.

Shared autonomy systems enhance people's abilities to perform activities of daily living using robotic manipulators. Recent systems succeed by first identifying their operators' intentions, typically by analyzing the user's joystick input. To enhance this recognition, it is useful to characterize people's behavior while performing such a task. Furthermore, eye gaze is a rich source of information for understanding operator intention. The goal of this paper is to provide novel insights into the dynamics of control behavior and eye gaze in human-robot shared manipulation tasks. To achieve this goal, we conduct a data collection study that uses an eye tracker to record eye gaze during a human-robot shared manipulation activity, both with and without shared autonomy assistance. We process the gaze signals from the study to extract gaze features like saccades, fixations, smooth pursuits, and scan paths. We analyze those features to identify novel patterns of gaze behaviors and highlight where these patterns are similar to and different from previous findings about eye gaze in human-only manipulation tasks. The work described in this paper lays a foundation for a model of natural human eye gaze in human-robot shared manipulation.

Hundreds of billions of small screws are assembled in consumer electronics industry every year, yet reliably automating the screwdriving process remains one of the most challenging tasks. Two barriers to further adoption of robotic threaded fastening systems are system cost and technical challenges, especially for small screws. An affordable intelligent screwdriving system that can support online stage and result classification is the first step to bridge the gap. To this end, starting from a state transition graph of screwdriving processes and a labeled screwdriving dataset (1862 runs of M1.4 screws) on multiple sensor signals, we develop classification algorithms and perform sensor reduction. Fast and accurate result classifiers are developed using linear discriminant analysis, while a wrapper method for feature subset selection is used to identify the optimal feature subset and corresponding sensor signals to reduce cost. A stage classifier based on decision tree is developed using the optimal sensor subset. The stage classifier achieves high accuracy in realtime prediction of various stages when augmented with the state transition graph.