Elaine Short

dblp:15/7092 · also Elaine Schaertl Short · DBLP profile ↗
← Back
34ranked-venue papers
8as first author
17since 2021 · last 2026
0000-0002-4789-9979ORCID · verified

Domains — the database's venue-derived domains; a paper can count in several

Artificial intelligence and machine learning · 30 · 7 first-author · 14 since 2021Human-computer interaction and ubiquitous computing · 26 · 8 first-author · 14 since 2021Applied, interdisciplinary, general and emerging computing · 11 · 3 first-author · 7 since 2021Systems, architecture and hardware · 8 · 3 since 2021
YearPublicationVenuePosition
2026 Creating Space to Succeed: How AccessComputing Supports Disabled Students' Computing Pathways
Alyson Yin, Elizabeth Moore, Lyla Mae Crawford, Brianna Blaser, Maya Cakmak, Richard E. Ladner, Elaine Short, Raja S. Kushalnagar, Stacy M. Branham
ICER (1)7
2025 Haptic Communication in Human-Human and Human-Robot Co-Manipulation
abstract
When a human dyad jointly manipulates an object, they must communicate about their intended motion plans. Some of that collaboration is achieved through the motion of the manipulated object itself, which we call "haptic communication." In this work, we captured the motion of human-human dyads moving an object together with one participant leading a motion plan about which the follower is uninformed. We then captured the same human participants manipulating the same object with a robot collaborator. By tracking the motion of the shared object using a low-cost IMU, we can directly compare human-human shared manipulation to the motion of those same participants interacting with the robot. Intra-study and post-study questionnaires provided participant feedback on the collaborations, indicating that the human-human collaborations are significantly more fluent, and analysis of the IMU data indicates that it captures objective differences in the motion profiles of the conditions. The differences in objective and subjective measures of accuracy and fluency between the human-human and human-robot trials motivate future research into improving robot assistants for physical tasks by enabling them to send and receive anthropomorphic haptic signals.
Katherine H. Allen, Chris Rogers, Elaine Short
RO-MAN3
2025 See What I Mean? Expressiveness and Clarity in Robot Display Design
abstract
Non-verbal visual symbols and displays play an important role in communication when humans and robots work collaboratively. However, few studies have investigated how different types of non-verbal cues affect objective task performance, especially in a dynamic environment that requires real-time decision-making. In this work, we designed a collaborative navigation task where the user and the robot only had partial information about the map on each end and thus the users were forced to communicate with a robot to complete the task. We conducted our study in a public space and recruited 37 participants who randomly passed by our setup. Each participant collaborated with a robot utilizing either animated anthropomorphic eyes and animated icons, or static anthropomorphic eyes and static icons. We found that participants that interacted with a robot with animated displays reported the greatest level of trust and satisfaction; that participants interpreted static icons the best; and that participants with a robot with static eyes had the highest completion success. These results suggest that while animation can foster trust with robots, human-robot communication can be optimized by the addition of familiar static icons that may be easier for users to interpret. We published our code, designed symbols, and collected results online at: https://github.com/mattufts/huamn_Cozmo_interaction.
Matthew Ebisu, Reuben M. Aronson, Elaine Short
RO-MAN4
2025 Demonstration Sidetracks: Categorizing Systematic Non-Optimality in Human Demonstrations
abstract
Learning from Demonstration (LfD) has become a popular approach for robots to learn new skills, despite most LfD methods suffering from imperfections in human demonstrations. Prior work in LfD often characterizes the sub-optimalities in human demonstrations as random noise. In this paper, we explored non-optimal behaviors in non-expert demonstrations and showed that these behaviors are not random and have systematic patterns: they form systematic demonstration sidetracks. We used a public space study dataset from our previous work with 40 participants and a long-horizon robot task. We recreated the experimental setup in a simulation and annotated all the demonstrations. We identified four types of demonstration sidetracks, Exploration, Mistake, Alignment, and Pause, and one control pattern one-dimension control. We found that instead of being random and rare, demonstration sidetracks frequently appear in non-expert demonstrations across all participants, and the distribution of demonstration sidetracks is associated with the robot task temporarily and spatially. Moreover, we found that users’ control patterns are affected by the control interface. Our findings highlight the need for better models of sub-optimal demonstrations, offering insights to improve LfD algorithms and reduce gaps between lab-based training and real-world applications. All the demonstrations, infrastructures, and annotations are available at https://github.com/AABL-Lab/Human-Demonstration-Sidetracks.
Shijie Fang, Qidi Fang, Reuben M. Aronson, Elaine Short
RO-MAN5
2025 CHARM: Considering Human Attributes for Reinforcement Modeling
abstract
Reinforcement Learning from Human Feedback has recently achieved significant success in various fields, and its performance is highly related to feedback quality. While much prior work acknowledged that human teachers’ characteristics would affect human feedback patterns, there is little work that has closely investigated the actual effects. In this work, we designed an exploratory study investigating how human feedback patterns are associated with human characteristics. We conducted a public space study with two long-horizon tasks and 46 participants. We found that feedback patterns are not only correlated with task statistics, such as rewards, but also correlated with participants’ characteristics, especially robot experience and educational background. Additionally, we demonstrated that human feedback value can be more accurately predicted with human characteristics compared to only using task statistics. All human feedback and characteristics we collected, and codes for our data collection and predicting more accurate human feedback are available at https://github.com/AABL-Lab/CHARM.
Qidi Fang, Shijie Fang, Jindan Huang, Qiuyu Chen, Reuben M. Aronson, Elaine Short
RO-MAN7
2024 On the Effect of Robot Errors on Human Teaching Dynamics
abstract
Human-in-the-loop learning is gaining popularity, particularly in the field of robotics, because it leverages human knowledge about real-world tasks to facilitate agent learning. When people instruct robots, they naturally adapt their teaching behavior in response to changes in robot performance. While current research predominantly focuses on integrating human teaching dynamics from an algorithmic perspective, understanding these dynamics from a human-centered standpoint is an under-explored, yet fundamental problem. Addressing this issue will enhance both robot learning and user experience. Therefore, this paper explores one potential factor contributing to the dynamic nature of human teaching: robot errors. We conducted a user study to investigate how the presence and severity of robot errors affect three dimensions of human teaching dynamics: feedback granularity, feedback richness, and teaching time, in both forced-choice and open-ended teaching contexts. The results show that people tend to spend more time teaching robots with errors, provide more detailed feedback over specific segments of a robot’s trajectory, and that robot error can influence a teacher’s choice of feedback modality. Our findings offer valuable insights for designing effective interfaces for interactive learning and optimizing algorithms to better understand human intentions.
Jindan Huang, Isaac S. Sheidlower, Reuben M. Aronson, Elaine Short
HAI4
2024 Intentional User Adaptation to Shared Control Assistance
abstract
Shared control approaches to robot assistance, which predict a user's goal based on their control input and provide autonomous assistance towards the predicted goal, typically assume that user behavior remains the same despite the presence of the assistance and rely on this assumption to infer user goals. However, people operating assisted systems continuously observe the robot behaving differently from their expectations, which may lead them to adapt their control behavior to better achieve their desired outcomes. In this paper, we show that users both change their control behavior when assistance is added and describe these changes as responses to the new system dynamics. In a computer-based bubble popping study, participants report changing their strategies with different levels of assistance, and analysis of their actual control input validates this change. In an in-the-wild robot study, participants teleoperated a robot to pick up a cup despite the presence of "assistance'' that drives the system away from the true goals of the task. Participants can overcome the "assistance" and reach the goal, which requires them to correct for the novel system dynamics. These results motivate further research in user-centered design and evaluation of assistive systems that treat the user as intentional.
Reuben M. Aronson, Elaine Short
HRI2
2024 Modeling Variation in Human Feedback with User Inputs: An Exploratory Methodology
abstract
To expedite the development process of interactive reinforcement learning (IntRL) algorithms, prior work often uses perfect oracles as simulated human teachers to furnish feedback signals. These oracles typically derive from ground-truth knowledge or optimal policies, providing dense and error-free feedback to a robot learner without delay. However, this machine-like feedback behavior fails to accurately represent the diverse patterns observed in human feedback, which may lead to unstable or unexpected algorithm performance in real-world human-robot interaction. To alleviate this limitation of oracles in oversimplifying user behavior, we propose a method for modeling variation in human feedback that can be applied to a standard oracle. We present a model with 5 dimensions of feedback variation identified in prior work. This model enables the modification of feedback outputs from perfect oracles to introduce more human-like features. We demonstrate how each model attribute can impact on the learning performance of an IntRL algorithm through a simulation experiment. We also conduct a proof-of-concept study to illustrate how our model can be populated from people in two ways. The modeling results intuitively present the feedback variation among participants and help to explain the mismatch between oracles and human teachers. Overall, our method is a promising step towards refining simulated oracles by incorporating insights from real users.
Jindan Huang, Reuben M. Aronson, Elaine Short
HRI3
2024 Online Behavior Modification for Expressive User Control of RL-Trained Robots
abstract
Reinforcement Learning (RL) is an effective method for robots to learn tasks. However, in typical RL, end-users have little to no control over how the robot does the task after the robot has been deployed. To address this, we introduce the idea of online behavior modification, a paradigm in which users have control over behavior features of a robot in real-time as it autonomously completes a task using an RL-trained policy. To show the value of this user-centered formulation for human-robot interaction, we present a behavior-diversity--based algorithm, Adjustable Control Of RL Dynamics (ACORD), and demonstrate its applicability to online behavior modification in simulation and a user study. In the study (n =23), users adjust the style of paintings as a robot traces a shape autonomously. We compare \algoshort to RL and Shared Autonomy (SA), and show \algoshort affords user-preferred levels of control and expression, comparable to SA, but with the potential for autonomous execution and robustness of RL. The code for this paper is available at https://github.com/AABL-Lab/HRI2024_ACORD
Isaac S. Sheidlower, Mavis Murdock, Emma Bethel, Reuben M. Aronson, Elaine Short
HRI5
2024 The Limits of Robot Moderators: Evidence Against Robot Personalization and Participation Equalization in a Building Task
abstract
Prior research has suggested that equalizing participation may benefit group performance and group cohesion. Robot-moderated groups have largely focused on improving member participation by focusing on the least performing member and do not consider the frequency of interaction or type of interaction. We introduce a robot moderator that varies its frequency and interaction types to observe the impact on groups in terms of performance and group cohesion. We investigate this in user studies across four conditions for equalizing participation. Leveraging Bayesian statistical methods that can evaluate evidence both for and against the null hypothesis, we find evidence that neither personalizing robot actions nor balancing the target of the robot’s assistance affected user experience in the group (as measured by performance, group cohesion, and variance of participation). We find a lack of evidence for equalization of participation impacting performance and group cohesion. Additionally, we also find positive evidence against the correlation of equalized participation and group cohesion in our task and weak evidence against equalized participation correlating with performance. In addition to guiding future researchers regarding robot behaviors that may not be effective in affecting groups, this work is an important negative result suggesting that equalizing participation may not be adequate to improve group performance and cohesion in all tasks.
Hayley Owens, Reuben M. Aronson, Elaine Short
RO-MAN3
2024 Imagining In-distribution States: How Predictable Robot Behavior Can Enable User Control Over Learned Policies
abstract
It is crucial that users are empowered to take advantage of the functionality of a robot and use their understanding of that functionality to perform novel and creative tasks. Given a robot trained with Reinforcement Learning (RL), a user may wish to leverage that autonomy along with their familiarity of how they expect the robot to behave to collaborate with the robot. One technique is for the user to take control of some of the robot’s action space through teleoperation, allowing the RL policy to simultaneously control the rest. We formalize this type of shared control as Partitioned Control (PC). However, this may not be possible using an out-of-the-box RL policy. For example, a user’s control may bring the robot into a failure state from the policy’s perspective, causing it to act unexpectedly and hindering the success of the user’s desired task. In this work, we formalize this problem and present Imaginary Out-of-Distribution Actions, IODA, an initial algorithm which empowers users to leverage their expectations of a robot’s behavior to accomplish new tasks. We deploy IODA in a user study with a real robot and find that IODA leads to both better task performance and a higher degree of alignment between robot behavior and user expectation. We also show that in PC, there is a strong and significant correlation between task performance and the robot’s ability to meet user expectations, highlighting the need for approaches like IODA. Code is available at https://github.com/AABL-Lab/ioda_roman_2024
Isaac S. Sheidlower, Emma Bethel, Douglas Lilly, Reuben M. Aronson, Elaine Short
RO-MAN5
2024 How Much Progress Did I Make? An Unexplored Human Feedback Signal for Teaching Robots
abstract
Enhancing the expressiveness of human teaching is vital for both improving robots’ learning from humans and the human-teaching-robot experience. In this work, we characterize and test a little-used teaching signal: progress, designed to represent the completion percentage of a task. We conducted two online studies with 76 crowd-sourced participants and one public space study with 40 non-expert participants to validate the capability of this progress signal. We find that progress indicates whether the task is successfully performed, reflects the degree of task completion, identifies unproductive but harmless behaviors, and is likely to be more consistent across participants. Furthermore, our results show that giving progress does not require extra workload and time. An additional contribution of our work is a dataset of 40 non-expert demonstrations from the public space study through an ice cream topping-adding task, which we observe to be multi-policy and sub-optimal, with sub-optimality not only from teleoperation errors but also from exploratory actions and attempts. The dataset is available at https://github.com/TeachingwithProgress/Non-Expert_Demonstrations.
Qidi Fang, Shijie Fang, Reuben M. Aronson, Elaine Short
RO-MAN5
2023 Barriers and Benefits: The Path to Accessible Makerspaces
abstract
Motivated by the philosophical overlap between makerspace culture and the needs of assistive technology users, we investigated the ways that makerspaces can support the development of new technologies by and for disabled makers. Using eleven semi-structured interviews with makerspace operators and disabled makerspace users, we identified five categories of barriers to makerspace participation: recruitment/outreach, physical access, financial, access to information, and belonging. Based on these interviews, we highlight ways makerspaces can better welcome makers with disabilities: enabling members to create adaptive technologies for the space (“makerspacing the makerspace”), making the physical space and information within the space accessible, and fostering belonging by building relationships with the disability community. Overall, our work contributes to our understanding of the possibilities and challenges of connecting the disabled community with the maker community and suggests new directions for collaboration, especially towards building hybrid makerspaces that provide multiple modalities for connection and creativity.
Katherine H. Allen, Audrey K. Balaska, Reuben M. Aronson, Chris Rogers, Elaine Short
ASSETS5
2023 From "Thumbs Up" to "10 out of 10": Reconsidering Scalar Feedback in Interactive Reinforcement Learning
abstract
Learning from human feedback is an effective way to improve robotic learning in exploration-heavy tasks. Compared to the wide application of binary human feedback, scalar human feedback has been used less because it is believed to be noisy and unstable. In this paper, we compare scalar and binary feedback, and demonstrate that scalar feedback benefits learning when properly handled. We collected binary or scalar feedback respectively from two groups of crowdworkers on a robot task. We found that when considering how consistently a participant labeled the same data, scalar feedback led to less consistency than binary feedback; however, the difference vanishes if small mismatches are allowed. Additionally, scalar and binary feedback show no significant differences in their correlations with key Reinforcement Learning targets. We then introduce Stabilizing TEacher Assessment DYnamics (STEADY) to improve learning from scalar feedback. Based on the idea that scalar feedback is muti-distributional, STEADY reconstructs underlying positive and negative feedback distributions and re-scales scalar feedback based on feedback statistics. We show that models trained with scalar feedback + STEADY outperform baselines, including binary feedback and raw scalar feedback, in a robot reaching task with non-expert human feedback. Our results show that both binary feedback and scalar feedback are dynamic, and scalar feedback is a promising signal for use in interactive Reinforcement Learning.
Reuben M. Aronson, Katherine H. Allen, Elaine Short
IROS4
2022 Inclusive HRI: Equity and Diversity in Design, Application, Methods, and Community
abstract
Discrimination and bias are pressing issues of many AI and robotics applications. These outcomes may derive from limited datasets that do not fully represent society as a whole or from the AI scientific community's western-male configuration bias. Although being a pressing issue, understanding how robotic systems can replicate and amplify inequalities and injustice among underrepresented communities is still in its infancy among social science and technical communities. This workshop contributes to filling this gap by exploring the research question: What do diversity and inclusion mean in the context of Human-Robot Interaction (HRI)? Here, attention is directed to three different levels of HRI: the technical, the community, and the target user level. Overall, this workshop will focus on the idea that AI systems can be created to be more attuned to inclusive societal needs, respect fundamental rights, and represent contemporary values in modern societies by integrating diversity and inclusion considerations.
Maartje M. A. de Graaf, Giulia Perugia, Eduard Fosch-Villaronga, Angelica Lim, Frank Broz, Elaine Short, Mark A. Neerincx
HRI6
2022 Keeping Humans in the Loop: Teaching via Feedback in Continuous Action Space Environments
abstract
Interactive Reinforcement Learning (IntRL) allows human teachers to accelerate the learning process of Reinforcement Learning (RL) robots. However, IntRL has largely been limited to tasks with discrete-action spaces in which actions are relatively slow. This limits IntRL's application to more complicated and challenging robotic tasks, the very tasks that modern RL is particularly well-suited for. We seek to bridge this gap by presenting Continuous Action-space Interactive Reinforcement learning (CAIR): the first continuous action-space IntRL algorithm that is capable of using teacher feedback to out-perform state-of-the-art RL algorithms in those tasks. CAIR combines policies learned from the environment and the teacher into a single policy that proportionally weights the two policies based on their agreement. This allows a CAIR agent to learn a relatively stable policy despite potentially noisy or coarse teacher feedback. We validate our approach in two simulated robotics tasks with easy-to-design and - understand heuristic oracle teachers. Furthermore, we validate our approach in a human subjects study through Amazon Mechanical Turk and show CAIR out-performs the prior state-of-the-art in Interactive RL.
Isaac S. Sheidlower, Allison Moore, Elaine Short
IROS3
2021 Robust Planning with Emergent Human-like Behavior for Agents Traveling in Groups
abstract
To enable robots to smoothly interact with humans during their travels together as a group, robots need the ability to adapt their motions under environmental changes and ensure all group members’ routes are feasible. To achieve this ability, robots require knowledge of the final destination and the subgoals in between. In practice, such information is seldom shared explicitly among group members, and may be frequently updated. Under this uncertain setting, maintaining travel efficiency and behavior appropriateness becomes a challenge. Previous literature approached the problem by generating compliant coordinating motions inspired by human groups, with subgoal uncertainty remaining isolated from the plan evaluation process. We show that such coordination can lead the robot to "bad" transient states where inefficient planning and lost tracking may incur. We propose to resolve the problem by formulating the coordinating motion as a Bayesian stochastic game, to plan for the robot as a group member, in the meanwhile considering the long-term effect of uncertainty during path coordination. We show that the approach improves travel efficiency and partner tracking robustness, by preventing assertive decisions during the inference update process. Moreover, the approach presents "agency", in the sense that it can generate human-like motions, which can be applied and contribute to the pedestrian simulation literature; the approach also affords variants from the human-like motions to generate robot behaviors based on sensing capabilities, contributing to the methodology of robot behavior design.
Shih-Yun Lo, Elaine Short, Andrea Thomaz
ICRA2
2020 Planning with Partner Uncertainty Modeling for Efficient Information Revealing in Teamwork
abstract
Communication among team members is important for efficient teamwork, to coordinate behavior and ensure that all team members have the information they need to complete the task. To enable effective communication and thus efficient teamwork, we propose a multi-agent planning approach to revealing information based on its benefit to joint team performance. By explicitly modeling the partner's knowledge and behavior, our approach allows a robot in a team to reason about when information is useful, how the communication is effective, and to communicate through efficient actions. That is, the robot provides only the necessary information for task completion, provides the information at the time that it is needed, and through the action(s) that optimizes team performance. We validated this approach in a human study in which participants walk together with a robot to a destination that is known only to the robot. We compared to a legible motion generation approach, and showed that users perceived our approach as more natural, socially appropriate, and fluent to team with, while being both more predictable and intent-clear. The ratings of our approach are equal or higher than legible motion across all 18 survey items.
Shih-Yun Lo, Elaine Short, Andrea Thomaz
HRI2
2020 Interactive Reinforcement Learning with Inaccurate Feedback
abstract
Interactive Reinforcement Learning (RL) enables agents to learn from two sources: rewards taken from observations of the environment, and feedback or advice from a secondary critic source, such as human teachers or sensor feedback. The addition of information from a critic during the learning process allows the agents to learn more quickly than non-interactive RL. There are many methods that allow policy feedback or advice to be combined with RL. However, critics can often give imperfect information. In this work, we introduce a framework for characterizing Interactive RL methods with imperfect teachers and propose an algorithm, Revision Estimation from Partially Incorrect Resources (REPaIR), which can estimate corrections to imperfect feedback over time. We run experiments both in simulations and demonstrate performance on a physical robot, and find that when baseline algorithms do not have prior information on the exact quality of a feedback source, using REPaIR matches or improves the expected performance of these algorithms.
Taylor Kessler Faulkner, Elaine Short, Andrea Thomaz
ICRA2
2020 TASC: Teammate Algorithm for Shared Cooperation
abstract
For robots to be perceived as full-fledged team members, they must display intelligent behavior along multiple dimensions. One challenge is that even when the robot and human are on the same team, the interaction may not feel like teamwork to the human. We present a novel algorithm, Teammate Algorithm for Shared Cooperation (TASC). TASC is motivated by the concept of shared cooperative activity (SCA) for human-human teamwork, developed in prior work by Bratman. We focus on enabling the robot to prioritize certain SCA facets in its action selection depending on the task. We evaluated TASC in three experiments using different tasks with human users on Amazon Mechanical Turk. Our results show that TASC enabled participants to predict the robot's goal earlier by one robot move and with greater confidence. The robot also helped reduce participants' energy usage in a simulated block-moving task. Altogether, these results show that considering the SCA facets in the robot's action selection improves teamwork.
Mai Lee Chang, Taylor Kessler Faulkner, Thomas Benjamin Wei, Elaine Short, Gokul Anandaraman, Andrea Thomaz
IROS4
2020 Defining Fairness in Human-Robot Teams
abstract
We seek to understand the human teammate's perception of fairness during a human-robot physical collaborative task where certain subtasks leverage the robot's strengths and others leverage the human's. We conduct a user study (n=30) to investigate the effects of fluency (absent vs. present) and effort (absent vs. present) on participants' perception of fairness. Fluency controls if the robot minimizes the idle time between the human's action and robot's action. Effort controls if the robot performs tasks that it is least skilled at, i.e., most time-consuming tasks, as quickly as possible. We evaluated four human-robot teaming algorithms that consider different levels of fluency and effort. Our results show that effort and fluency help improve fairness without making a trade-off with efficiency. When the robot displays effort, this significantly increased participants' perceived fairness. Participants' perception of fairness is also influenced by team members' skill levels and task type. To that end, we propose three notions of fairness for effective human-robot teamwork: equality of workload, equality of capability, and equality of task type.
Mai Lee Chang, Zachary Pope, Elaine Short, Andrea Thomaz
RO-MAN3
2019 Learning from Corrective Demonstrations
abstract
Robots deployed in human environments will inevitably encounter unmodeled scenarios which are likely to result in execution failures. To address this issue, we would like to allow co-present naive users to correct and improve the robot's behavior as these edge cases are encountered over time.
Reymundo Gutierrez, Elaine Short, Scott Niekum, Andrea Thomaz
HRI2
2019 Enhancing Robot Learning with Human Social Cues
abstract
Imagine a learning scenario between two humans: a teacher demonstrating how to play a new musical instrument or a craftsman teaching a new skill like pottery or knitting to a novice. Even though learning a skill has a learning curve to get the nuances of the technique right, some basic social principles are followed between the teacher and the student to make the learning process eventually succeed. There are several assumptions or social priors in this communication for teaching: mutual eye contact to draw attention to instructions, following the gaze of the teacher to understand the skill, the teacher following the student's gaze during imitation to give feedback, the teacher demonstrating by pointing towards something she is going to approach or manipulate and verbal interruptions or corrections during the learning process [1], [2]. In prior research, verbal and non-verbal social cues such as eye gaze and gestures have been shown to make human-human interactions seamless and augment verbal, collaborative behavior [3], [4]. They serve as an indicator of engagement, interest and attention when people interact face-to-face with one another [5], [6].
Akanksha Saran, Elaine Short, Andrea Thomaz, Scott Niekum
HRI2
2019 SAIL: Simulation-Informed Active In-the-Wild Learning
abstract
Robots in real-world environments may need to adapt context-specific behaviors learned in one environment to new environments with new constraints. In many cases, copresent humans can provide the robot with information, but it may not be safe for them to provide hands-on demonstrations and there may not be a dedicated supervisor to provide constant feedback. In this work we present the SAIL (Simulation-Informed Active In-the-Wild Learning) algorithm for learning new approaches to manipulation skills starting from a single demonstration. In this three-step algorithm, the robot simulates task execution to choose new potential approaches; collects unsupervised data on task execution in the target environment; and finally, chooses informative actions to show to co-present humans and obtain labels. Our approach enables a robot to learn new ways of executing two different tasks by using success/failure labels obtained from naïve users in a public space, performing 496 manipulation actions and collecting 163 labels from users in the wild over six 45-minute to 1-hour deployments. We show that classifiers based low-level sensor data can be used to accurately distinguish between successful and unsuccessful motions in a multi-step task ( ), even when trained in the wild. We also show that using the sensor data to choose which actions to sample is more effective than choosing the least-sampled action.
Elaine Short, Adam Allevato, Andrea Thomaz
HRI1
2018 Detecting Contingency for HRI in Open-World Environments
abstract
This paper presents a novel algorithm for detecting contingent reactions to robot behavior in noisy real-world environments with naive users. Prior work has established that one way to detect contingency is by calculating a difference metric between sensor data before and after a robot probe of the environment. Our algorithm, CIRCLE (Contingency for Interactive Real-time CLassification of Engagement) provides a new approach to calculating this difference and detecting contingency, improving the running time for the difference calculation from 2.5 seconds to approximately 0.001 seconds on an 1100-sample vector, and effectively enabling real-time detection of contingent events. We show accuracy comparable to the best offline results for detecting contingency in this way (89.5% vs 91% in prior work), and demonstrate the utility of the real-time contingency detection in a field study of a survey-administering robot in a noisy open-world environment with naive users, showing that the robot can decrease the number of requests it makes (from 38 to 13) while more efficiently collecting survey responses (30% response rate rather than 26.3%).
Elaine Short, Mai Lee Chang, Andrea Thomaz
HRI1
2018 Effects of Integrated Intent Recognition and Communication on Human-Robot Collaboration
abstract
Human-robot interaction research to date has investigated intent recognition and communication separately. In this paper, we explore the effects of integrating both the robot's ability to generate intentional motion and predict the human's motion in a collaborative physical task. We implemented an intent recognition system to recognize the human partner's hand motion intent and a motion planner system to enable the robot to communicate its intent by using legible and predictable motion. We tested this bi-directional intent system in a 2-way within-subjects user study. Results suggest that an integrated intent recognition and communication system may facilitate more collaborative behavior among team members.
Mai Lee Chang, Reymundo Gutierrez, Priyanka Khante, Elaine Short, Andrea Thomaz
IROS4
2018 Policy Shaping with Supervisory Attention Driven Exploration
abstract
Robots deployed for long periods of time need to be able to explore and learn from their environment. One approach to this problem has been reinforcement learning (RL), in which robots receive rewards from the environment that allow them to choose optimal actions. To speed learning when human supervision is available, interactive reinforcement learning solicits feedback from a human teacher. However, this approach typically assumes that learning takes place under continuous supervision, which is unlikely to hold in long-term scenarios. We propose an extension to a method of interactive reinforcement learning, policy shaping, that takes into account human attention. Our approach enables better performance while unattended by favoring information-gathering actions when attended and actions that have received positive feedback when unattended. We test our approach in both simulation and on a robot, finding that our method learns faster than policy shaping and performs more safely than policy shaping while no one is paying attention to the robot.
Taylor Kessler Faulkner, Elaine Short, Andrea Thomaz
IROS2
2018 Human Gaze Following for Human-Robot Interaction
abstract
Gaze provides subtle informative cues to aid fluent interactions among people. Incorporating human gaze predictions can signify how engaged a person is while interacting with a robot and allow the robot to predict a human's intentions or goals. We propose a novel approach to predict human gaze fixations relevant for human-robot interaction tasks-both referential and mutual gaze-in real time on a robot. We use a deep learning approach which tracks a human's gaze from a robot's perspective in real time. The approach builds on prior work which uses a deep network to predict the referential gaze of a person from a single 2D image. Our work uses an interpretable part of the network, a gaze heat map, and incorporates contextual task knowledge such as location of relevant objects, to predict referential gaze. We find that the gaze heat map statistics also capture differences between mutual and referential gaze conditions, which we use to predict whether a person is facing the robot's camera or not. We highlight the challenges of following a person's gaze on a robot in real time and show improved performance for referential gaze and mutual gaze prediction.
Akanksha Saran, Srinjoy Majumdar, Elaine Short, Andrea Thomaz, Scott Niekum
IROS3
2017 Robot moderation of a collaborative game: Towards socially assistive robotics in group interactions
abstract
This paper presents an algorithm for enabling a robot to act as the moderator in a group interaction centered around a tablet-based assembly game. The algorithm uses one of two different objective functions: one intended to be “performance equalizing”, wherein the robot attempts to equalize scoring among users, and another intended to be “performance reinforcing”, wherein the robot attempts to help the group score as many points as possible. In an evaluation study with ten groups of three participants, we found that the “performance equalizing” algorithm improved task performance and reduced group cohesion, while the “performance reinforcing” algorithm improved group cohesion and reduced task performance.
Elaine Short, Maja J. Mataric
RO-MAN1
2017 Understanding social interactions with socially assistive robotics in intergenerational family groups
abstract
We present a pilot study of a socially assistive robot interacting with intergenerational groups. The system is designed to improve the social well-being of older adults by supporting interactions within families. Six intergenerational family groups interacted with the robot in four tablet-based games. Users' behavior during the sessions was used to compare the games and understand how members of different generations and different families interact with the robot. Interviews with users provide insight into users' priorities for in-home deployment of socially assistive robots, as well as preferences about the activities, appearance, and behavior of the robot.
Elaine Short, Katelyn Swift-Spong, Hyunju Shim, Kristi M. Wisniewski, Deanah Kim Zak, Shinyi Wu, Elizabeth M. Zelinski, Maja J. Mataric
RO-MAN1
2017 Understanding agency in interactions between children with autism and socially assistive robots
abstract
Socially assistive robotics (SAR) has increasingly been shown to have potential as a tool for social skills therapy for children with autism, a developmental disorder associated with atypical social development. This work presents the results of a study of robot agency on child-robot interactions involving children with autism. We describe the development of a SAR interaction scenario with both agent-like and object-like robot behaviors and present the results of a pilot study of six children with autism interacting with a humanoid robot with the full controller, as well as three types of control: a non-humanoid box robot with similar behavior (reduced morphological agency), a humanoid robot with random behavior (reduced behavioral agency), and a robotic toy (reduced morphological and behavioral agency). We find that the children can be divided into two groups depending on their reaction to the robot; for some children, the robot was an engaging object that elicited social behavior by providing novel and appealing sensory experiences (primarily bubbleblowing), while for other children, the robot was an agent and elicited social behavior through agentlike actions such as autonomous movement. We found that the first group had small differences between robot conditions and vocalized most with the bubble-blowing toy, while the second group vocalized most with the humanoid robots and looked less at the humanoid portion of the robot with reduced behavioral agency.
Elaine Short, Eric Deng, David Feil-Seifer, Maja J. Mataric
J. Hum. Robot Interact.1
2014 How to train your DragonBot: Socially assistive robots for teaching children about nutrition through play
abstract
This paper describes an extended (6-session) interaction between an ethnically and geographically diverse group of 26 first-grade children and the DragonBot robot in the context of learning about healthy food choices. We find that children demonstrate a high level of enjoyment when interacting with the robot, and a statistically significant increase in engagement with the system over the duration of the interaction. We also find evidence of relationship-building between the child and robot, and encouraging trends towards child learning. These results are promising for the use of socially assistive robotic technologies for long-term one-on-one educational interventions for younger children.
Elaine Short, Katelyn Swift-Spong, Jillian Greczek, Aditi Ramachandran, Alexandru Litoiu, Elena Corina Grigore, David Feil-Seifer, Samuel Shuster, Jin Joo Lee, Shaobo Huang, Svetlana Levonisova, Sarah Litz, Jamy Li, Gisele Ragusa, Donna Spruijt-Metz, Maja J. Mataric, Brian Scassellati
RO-MAN1
2011 A comparison of machine learning techniques for modeling human-robot interaction with children with autism
abstract
Several machine learning techniques are used to model the behavior of children with autism interacting with a humanoid robot, comparing a static model to a dynamic model using hand-coded features. Good accuracy (over 80%) is achieved in predicting child vocalizations; directions for future approaches to modeling the behavior of children with autism are suggested.
Elaine Short, David Feil-Seifer, Maja J. Mataric
HRI1
2010 No fair!!: an interaction with a cheating robot
abstract
Using a humanoid robot and a simple children's game, we examine the degree to which variations in behavior result in attributions of mental state and intentionality. Participants play the well-known children's game "rock-paper-scissors" against a robot that either plays fairly, or that cheats in one of two ways. In the "verbal cheat" condition, the robot announces the wrong outcome on several rounds which it loses, declaring itself the winner. In the "action cheat"' condition, the robot changes its gesture after seeing its opponent's play. We find that participants display a greater level of social engagement and make greater attributions of mental state when playing against the robot in the conditions in which it cheats.
Elaine Short, Justin W. Hart, Michelle Vu, Brian Scassellati
HRI1