When we design assistive robots, we are largely well-intentioned: we want to use our research and engineering skills to help others. However, to meet this end, as we move towards more disability- and social justice-oriented HRI research, it is important to critically examine the dominant social narratives of disability that shape how assistive robots for disabled people are conceptualized, designed, and deployed. In this paper, we introduce the concept of robot-mediated paternalism (RMP), drawing on a multidisciplinary body of literature, including critical HCI and crip technoscience. Then, we discuss how RMP may manifest when assistive robots provide unwanted assistance to disabled people and interfere with their decisions/actions based on assumptions about their autonomy and care needs. Finally, we reflect on and offer actionable suggestions for how HRI researchers might mitigate RMP by aligning their work with the principles of disability justice. Overall, our work advances disability-centered research on assistive robots and, in doing so, supports more equitable, inclusive futures for disabled communities.

Control devices are essential in shaping the user experience of telepresence robot operators. This is especially true for mobile telemanipulator robots (MTRs), which offer greater opportunities for social interaction compared to mobile or tabletop telepresence robots, but are more difficult to control. This increased difficulty can diminish key aspects of user experience, such as sense of control (SoC), which many users value over performance. However, the usability of telepresence control devices has been critically overlooked in HRI research. In a between-subjects study (n = 63), we investigate how three widely used control devices (mouse, gamepad, haptic controller) affect critical aspects of teleoperation: perceived SoC, safety, usability, and cognitive load; and task performance. Participants remotely controlled an MTR (Stretch) to wait on a customer in a social telepresence setting (cafe). We found that the type of control device significantly impacts both task performance and SoC in direct teleoperation, with the mouse having the best performance/SoC. There were notable tradeoffs in speeds, errors, and task completion times, and participants with higher SoC performed significantly better than those with lower SoC. We provide participant-informed suggestions for future control device design, such as allowing users to reconfigure its physical form, input sensitivity, and controls to align with video game conventions. By foregrounding the role of control devices, our work contributes to ongoing conversations in HRI around the tradeoffs between autonomy, usability, and user agency in social telepresence.

Patients with cancer (PwC) have a hard time getting prompt treatment in acute care settings, and feel unseen, unheard, and neglected. This is due to systemic problems: worldwide, Emergency Department (ED) healthcare workers (HCWs) are overworked and EDs are understaffed. Robots will not fix these problems; however, prior work suggests if well-designed and contextualized, they may support cancer care. Based on longstanding collaborations with PwC and ED HCWs, in this paper we report on an exploration of the design space of social robots for acute cancer care. Using a care ethics lens, we found robots can be uniquely positioned to amplify compassion within deeply human care relationships through their social presence, while performing routine tasks, such as patient monitoring. However, participants suggested the human experiences of pain and distress may remain elusive for robots to engage with meaningfully. Our work reveals HCWs and PwC saw robots as means to expand relational care in the ED, and explores how future HRI research may meaningfully support these care relationships.

When designing socially assistive robots, HRI researchers often focus on robot personality as a means of increasing a person’s engagement, enjoyment, and trust. In this work, we argue that using only trait-based personality models is often limited in its ability to capture the nuance that matches end users’ desires, experiences, and cultural backgrounds. To address this gap, we introduce the concept of a robot character , a holistic framing of robot personality that extends the trait-based approach to include external factors, such as shared interests between the user and robot, as sociocultural and environmental factors. We introduced and validated the Robot Role Character Creation (R2C2) tool, an accessible scaffolding tool to co-design robot characters with end users in order to support more nuanced and personalized robots. R2C2 highlights the voices of end users and enables them to easily ideate and communicate their unique robot characters, particularly for populations often underrepresented in robot design. Through a cross-cultural study (the U.S. and Mexico), we validated the R2C2 tool in eliciting rich design insights for robot characters from people with mild cognitive impairment (MCI) and dementia (PwD). We report our findings, enabled by the R2C2 tool, on the role participants envisioned for their desired robot characters, the multidimensionality and adaptability of these robot characters, and how participants’ socio-cultural backgrounds influenced their characters. Our findings demonstrate that R2C2 can facilitate the creation of nuanced and personalized robot characters that resonate with user experiences, needs, and preferences. We analyze how participants envisioned the roles, multidimensionality, and cultural influences shaping their ideal robot characters, highlighting R2C2’s ability to capture these diverse perspectives. This work will serve as a basis for HRI designers to create more effective robot interactions, enhance acceptance and trust, and promote engagement with robot characters while centering the wisdom and personhood of people with cognitive impairments.

Long Covid (LC) is a debilitating, multisystemic disease that has emerged as the largest mass-disabling event in recent history. Due to the episodic disability and stigma associated with the condition, people with LC (PwLC) often experience social isolation. Mobile telemanipulator robots (MTRs) have the potential to support remote social inclusion for PwLC. However, nuanced MTR design is necessary to accommodate PwLC’s fluctuating symptoms and avoid exacerbating them due to the complexities of teleoperation. In this work, we conducted participatory research with eight PwLC to explore how MTRs can be designed to support their needs. Through online, semi-structured interviews, we found that all participants recognized the potential of MTRs to enhance social inclusion across various settings. Our findings highlight the importance of providing PwLC with adaptive, autonomous support during teleoperation to meet their pacing needs and minimize exertion. Many PwLC preferred MTRs with adjustable autonomy, as they would offer greater agency over the robot’s actions in social spaces. Due to concerns about stigma, participants also wanted MTRs to provide flexible control over the visibility of their disability, allowing them to manage how others perceive them according to their preferences and context. Based on these findings, we present key design considerations, grounded in critical disability studies and critical access studies, for designing MTRs that support remote social inclusion for PwLC while safeguarding their well-being. This work serves as a basis for developing accessible MTR systems that promote inclusivity for PwLC and other chronic conditions.

Emergency departments (EDs) are fast-paced, dynamic, safety-critical spaces where clinicians are overworked and underpaid. To support clinicians, researchers are exploring the contextualization and development of clinically assistive robots (CARs) that can assume non-critical tasks to reduce clinician overload. In this article, we introduce Clinically Assistive Robot System for Emergency Medicine (CAR-EM), collaboratively developed with ED clinicians. CAR-EM includes an autonomous robot and a task specification interface. It completes tasks by leveraging control synthesis, a framework that automatically transforms high-level tasks into control while providing guarantees and feedback. We conducted a feasibility study across two different hospital EDs, where interprofessional clinicians tasked the robot to perform patient assessments and item deliveries. Clinicians found the system easy to use, and particularly helpful to offload busywork. This work demonstrates control synthesis as a feasible tool to develop autonomy for robots in safety-critical spaces, and identifies considerations for failure interventions. We also discuss ethical considerations for deploying robots in hospitals, including healthcare worker displacement and work disruption. Thus, our work: (1) highlights the unique requirements of situating robots in real world hospital EDs, and (2) demonstrates a novel approach leveraging guarantees and feedback from control synthesis methods to successfully implement context-specific CAR behaviors. Through this work, we aim to further research for safer and more reliable robots in real world, uncertain environments.

Many HRI researchers have engaged in participatory research to include users in robot design processes. However, to our knowledge, people with mild cognitive impairment (PwMCI) and early stage dementia have yet to be included in developing and programming robots, and the HRI community lacks tools to facilitate their inclusion. We bridge this gap by introducing PODER (PrOgramming framework to Develop Robot behaviors), which enables a lived technology experience for PwMCI via scaffolding, peer programming, and development tools to support them as key developers of social robots. We conducted a study where PwMCI and early stage dementia used PODER to program robot interactions, and found that participants were highly engaged and deeply enjoyed their experience, creating programs for robots that reflected their interests, experiences, and needs. Our results show the impact of including participants with MCI and early stage dementia in robot programming, including an increased understanding of technology, shifting their perceived role from technology users to programmers, and desire to be involved with the end-to-end process. By releasing PODER to the community, we hope this work can facilitate the intentional inclusion of people with cognitive impairments in further HRI research.

Gait rehabilitation is a critical aspect of post-stroke recovery, and emerging technologies such as virtual reality and wearables are playing a pivotal role in facilitating this process. However, despite the potential benefits, there is a significant gap in robot-based rehabilitative systems that facilitate repeated use by maintaining users' attention long-term. Our research aims to bridge this gap by creating a comprehensive system that utilizes different feedback types and robotic assistance to support users' gait rehabilitation outcomes. In this paper, we introduce GARRY (Gait Rehabilitation Robotic System), a new robotic system that provides interactive feedback during locomotor training. It promotes engagement by gamifying the rehabilitation process, offering a fun means for the user to meet their rehabilitation goals defined and set by physical therapists. GARRY also incorporates behavioral feedback to introduce a sense of companionship during a session. We make GARRY open-source to other researchers in hopes of encouraging accessibility and to promote research in the field. Our code can be found here: https://github.com/UCSD-RHC-Lab/GARRY

Cognitively assistive robots (CARs) have great potential to extend the reach of clinical interventions to the home. Due to the wide variety of cognitive abilities and rehabilitation goals, these systems must be flexible to support rapid and accurate implementation of intervention content that is grounded in existing clinical practice. To this end, we detail the system architecture of CARMEN (Cognitively Assistive Robot for Motivation and Neurorehabilitation), a flexible robot system we developed in collaboration with our key stakeholders: clinicians and people with mild cognitive impairment (PwMCI). We implemented a well-validated compensatory cognitive training (CCT) intervention on CARMEN, which it autonomously delivers to PwMCI. We deployed CARMEN in the homes of these stakeholders to evaluate and gain initial feedback on the system. We found that CARMEN gave participants confidence to use cognitive strategies in their everyday life, and participants saw opportunities for CARMEN to exhibit greater levels of autonomy or be used for other applications. Furthermore, elements of CARMEN are open source to support flexible home-deployed robots. Thus, CARMEN will enable the HRI community to deploy quality interventions to robots, ultimately increasing their accessibility and extensibility.

Robotics researchers are increasingly exploring how robots can support human groups and teams. Action teams experience high workload, must work quickly, and must make decisions under uncertainty. Robots in these teams must be designed and contextualized to not contribute to errors or interrupt human team workflow. We conducted a study where human-robot action teams collaborated with a mobile manipulator (Stretch) in an escape room paradigm to better understand: 1) how a robot’s actions influenced intra-team dynamics, and 2) how attitudes towards the robot affected human-robot teaming. Our behavioral analysis highlights the effect of a robot’s functional and social behaviors on its acceptance within teams, how human teams adapt to perceived robot capabilities, and the significance of the robot’s nonverbal cues in shaping human expectations. These insights offer valuable implications for designing effective human-robot interactions in dynamic environments where team success can save lives.

Many robot-delivered health interventions aim to support people longitudinally at home to complement or replace in-clinic treatments. However, there is little guidance on how robots can support collaborative goal setting (CGS). CGS is the process in which a person works with a clinician to set and modify their goals for care; it can improve treatment adherence and efficacy. However, for home-deployed robots, clinicians will have limited availability to help set and modify goals over time, which necessitates that robots support CGS on their own. In this work, we explore how robots can facilitate CGS in the context of our robot CARMEN (Cognitively Assistive Robot for Motivation and Neurorehabilitation), which delivers neurorehabilitation to people with mild cognitive impairment (PwMCI). We co-designed robot behaviors for supporting CGS with clinical neuropsychologists and PwMCI, and prototyped them on CARMEN. We present feedback on how PwMCI envision these behaviors supporting goal progress and motivation during an intervention. We report insights on how to support this process with home-deployed robots and propose a framework to support HRI researchers interested in exploring this both in the context of cognitively assistive robots and beyond. This work supports designing and implementing CGS on robots, which will ultimately extend the efficacy of robot-delivered health interventions.

Emergency department (ED) healthcare workers (HCWs) are interrupted as often as once every six minutes, increasing the risk of errors and preventable patient harm. As more robots enter hospitals, and the ED, they must support HCWs in managing interruptions, and ideally mitigate their harmful effects, without disrupting ED communication. However, interruption-mitigation strategies, particularly for mobile telemanipulator robots (MTRs), are not well understood. In this work, we explore interruption-mitigation and reorientation methods for MTRs in the ED. We conducted a study where ED HCWs teleoperated an MTR in a realistic hospital simulation environment. Our findings revealed insights on how MTRs might support multitasking in environments with frequent task switching, and the place of autonomy in safety-critical spaces. Conflicting opinions about the appropriateness of different MTR behaviors highlighted challenges and ethical dilemmas that influence the integration of MTRs in the ED. This work will support the implementation of interruption-mitigation strategies on MTRs, enabling them to better support people in fast-paced, interruption-driven environments thus reducing the risk of errors in these situations.

Tele-operated collaborative robots are used by many children for academic learning. However, as child-directed play is important for social-emotional learning, it is also important to understand how robots can facilitate play. In this article, we present findings from an analysis of a national, multi-year case study, where we explore how 53 children in grades K–12 ( n = 53) used robots for self-directed play activities. The contributions of this article are as follows. First, we present empirical data on novel play scenarios that remote children created using their tele-operated robots. These play scenarios emerged in five categories of play: physical, verbal, visual, extracurricular, and wished-for play. Second, we identify two unique themes that emerged from the data—robot-mediated play as a foundational support of general friendships and as a foundational support of self-expression and identity. Third, our work found that robot-mediated play provided benefits similar to in-person play. Findings from our work will inform novel robot and HRI design for tele-operated and social robots that facilitate self-directed play. Findings will also inform future interdisciplinary studies on robot-mediated play.

Although the concept of wisdom is ancient, empirical research on it has only recently received attention in gerontology. This coincides with a critical turn away from a deficit model of aging, viewing aging as a series of losses, toward a more supportive and developmental model. This article draws on this recent work to consider how wisdom can be a critical lens for human-robot interaction (HRI) researchers and other technology design researchers to pay more attention to the coping strategies that older adults accumulated throughout their lives. We engaged in a 6-month collaborative design process with community-dwelling older adults. The contributions of this article are twofold. First, we found that wisdom as a design concept helps researchers to critically examine how they define knowledge. Wisdom as an accumulation of experiential knowledge of older adults helps researchers rethink the definition of knowledge—valuing computational and technological knowledge—in the field of HRI. Second, wisdom leads researchers to the past experiences of older adults. Although past experiences are as important as current experiences, they are not actively considered in robot design studies for older adults. We hope wisdom as a critical lens could allow researchers to integrate the invisible aspects of older adults’ aging experiences into the existing practices of designing robots for aging users.

Much research in healthcare robotics explores extending rehabilitative interventions to the home. However, for adults, little guidance exists on how to translate human-delivered, clinic-based interventions into robot-delivered, home-based ones to support longitudinal interaction. This is particularly problematic for neurorehabilitation, where adults with cognitive impairments require unique styles of interaction to avoid frustration or overstimulation. In this paper, we address this gap by exploring the design of robot-delivered neurorehabilitation interventions for people with mild cognitive impairment (PwMCI). Through a multi-year collaboration with clinical neuropsychologists and PwMCI, we developed robot prototypes which deliver cognitive training at home. We used these prototypes as design probes to understand how participants envision long-term deployment of the intervention, and how it can be contextualized to the lives of PwMCI. We report our findings and specify design patterns and considerations for translating neurorehabilitation interventions to robots. This work will serve as a basis for future endeavors to translate cognitive training and other clinical interventions onto a robot, support longitudinal engagement with home-deployed robots, and ultimately extend the accessibility of longitudinal health interventions for people with cognitive impairments.

To facilitate HRI's transition from dyadic to group interaction, new methods are needed for robots to sense and understand team behavior. We introduce the Robot-Centric Group Detection and Tracking System (REGROUP), a new method that enables robots to detect and track groups of people from an ego-centric perspective using a crowd-aware, tracking-by-detection approach. Our system employs a novel technique that leverages person re-identification deep learning features to address the group data association problem. REGROUP is robust to real-world vision challenges such as occlusion, camera egomotion, shadow, and varying lighting illuminations. Also, it runs in real-time on real-world data. We show that REGROUP outperformed three group detection methods by up to 40% in terms of precision and up to 18 % in terms of recall. Also, we show that REGROUP's group tracking method outperformed three state-of-the-art methods by up to 66% in terms of tracking accuracy and 20% in terms of tracking precision. We plan to publicly release our system to support HRI teaming research and development. We hope this work will enable the development of robots that can more effectively locate and perceive their teammates, particularly in uncertain, unstructured environments.

Clinical educators have used robotic and virtual patient simulator systems (RPS) for dozens of years, to help clinical learners (CL) gain key skills to help avoid future patient harm. These systems can simulate human physiological traits; however, they have static faces and lack the realistic depiction of facial cues, which limits CL engagement and immersion. In this article, we provide a detailed review of existing systems in use, as well as describe the possibilities for new technologies from the human–robot interaction and intelligent virtual agents communities to push forward the state of the art. We also discuss our own work in this area, including new approaches for facial recognition and synthesis on RPS systems, including the ability to realistically display patient facial cues such as pain and stroke. Finally, we discuss future research directions for the field.

The Emergency Department (ED) is a stressful, safety-critical environment, which is often overcrowded, noisy, chaotic, and understaffed. The built environment plays a key role in patient outcomes, experiences, and the mental health of healthcare workers (HCWs). However, once a space is built, it is difficult to change it; so the modularity and adaptability of new technologies such as robots could potentially help stakeholders mitigate some of these challenges; yet, there is a lack of research in this area, particularly in the ED. In this paper, we address this gap by engaging HCWs in a research-through-design process, utilizing design fiction, to envision a future resilient ED. Here, robots scurry along the ceiling, provide help at the bedside, and smart furniture and walls provide spaces for privacy and calm. We co-created design prototypes of future intelligent systems that can modify the built environment to support resilience, which we then used to co-create a Design Catalog with HCWs, which contains a collection of future technology prototypes contextualized within the ED. We found that HCWs envisioned many ways for intelligent systems to help them reimagine the built environment, including ways to enhance HCW-patient communication, improve patient experience, support both HCW and patient safety, and use reconfigurable spaces to support privacy. We hope our work inspires further exploration into using new technologies to reimagine and reconfigure the built environment to support resilient hospitals.

Proximate human-robot teaming (pxHRT) is a complex subspace within human-robot interaction. Studies in this space involve a range of equipment and methods, including the ability to sense people and robots precisely. Research in this area draws from a wide variety of other fields, from human-human interaction to control theory, making the study design complex, particularly for those outside the field of HRI. In this paper, we introduce a framework that helps researchers consider tradeoffs across various task contexts, platforms, sensors, and analysis methods; metrics frequently used in the field; and common challenges researchers may face. We demonstrate the use of the framework via a case study which employs an autonomous mobile manipulator continuously engaging in shared workspace, handover, and co-manipulation tasks with people, and explores the effect of cognitive workload on pxHRT dynamics. We also demonstrate the utility of the framework in a case study with two groups of researchers new to pxHRT. With this framework, we hope to enable researchers, especially those outside HRI, to more thoroughly consider these complex components within their studies, more easily design experiments, and more fully explore research questions within the space of pxHRT.

In many real-world applications, multiple agents seek to learn how to perform highly related yet slightly different tasks in an online bandit learning protocol. We formulate this problem as the $\epsilon$-multi-player multi-armed bandit problem, in which a set of players concurrently interact with a set of arms, and for each arm, the reward distributions for all players are similar but not necessarily identical. We develop an upper confidence bound-based algorithm, RobustAgg($\epsilon$), that adaptively aggregates rewards collected by different players. In the setting where an upper bound on the pairwise dissimilarities of reward distributions between players is known, we achieve instance-dependent regret guarantees that depend on the amenability of information sharing across players. We complement these upper bounds with nearly matching lower bounds. In the setting where pairwise dissimilarities are unknown, we provide a lower bound, as well as an algorithm that trades off minimax regret guarantees for adaptivity to unknown similarity structure.

Dementia affects >50 million worldwide, causing progressive cognitive and physical disabilities. Its caregiving burden falls largely onto informal caregivers, who experience their own health problems, and face tremendous stress with little support–all exacerbated during COVID-19. In this paper, we present a new caregiver support perspective, where the lenses of health equity and community health can shape future technology design. Through a 1.5 year long, in-depth research process with dementia community health workers, we learned how caregiving support technology can reflect key concepts in dementia community health practice. This paper makes two contributions: 1) We propose employing embodied cueing, such as imitation or action mimicry, as a communication modality that can align technology with community caregiving approaches, promote agency in people with dementia, and relieve caregiver burden, and 2) We suggest new avenues for HCI research to advance health equity in the context of dementia technology design.

As robots work with human teams, they will be expected to fluently coordinate with them. While people are adept at coordination and real-time adaptation, robots still lack this skill. In this paper, we introduce TANDEM: Temporal Anticipation and Adaptation for Machines, a series of neurobiologically-inspired algorithms that enable robots to fluently coordinate with people. TANDEM leverages a humanlike understanding of external and internal temporal changes to facilitate coordination. We experimentally validated the approach via a human-robot collaborative drumming task across tempo-changing rhythmic conditions. We found that an adaptation process alone enables a robot to achieve human-level performance. Moreover, by combining anticipatory knowledge along with an adaptation process, robots can potentially perform such tasks better than people. We hope this work will enable researchers to create robots more sensitive to changes in team dynamics.

The emergency department (ED) is a safety-critical environment in which healthcare workers (HCWs) are overburdened, overworked, and have limited resources, especially during the COVID-19 pandemic. One way to address this problem is to explore the use of robots that can support clinical teams, e.g., to deliver materials or restock supplies. However, due to EDs being overcrowded, and the cognitive overload HCWs experience, robots need to understand various levels of patient acuity so they avoid disrupting care delivery. In this paper, we introduce the Safety-Critical Deep Q-Network (SafeDQN) system, a new acuity-aware navigation system for mobile robots. SafeDQN is based on two insights about care in EDs: high-acuity patients tend to have more HCWs in attendance and those HCWs tend to move more quickly. We compared SafeDQN to three classic navigation methods, and show that it generates the safest, quickest path for mobile robots when navigating in a simulated ED environment. We hope this work encourages future exploration of social robots that work in safety-critical, human-centered environments, and ultimately help to improve patient outcomes and save lives.

JESSIE is a robotic system that enables novice programmers to program social robots by expressing high-level specifications. We employ control synthesis with a tangible front-end to allow users to define complex behavior for which we automatically generate control code. We demonstrate JESSIE in the context of enabling clinicians to create personalized treatments for people with mild cognitive impairment (MCI) on a Kuri robot, in little time and without error. We evaluated JESSIE with neuropsychologists who reported high usability and learnability. They gave suggestions for improvement, including increased support for personalization, multi-party programming, collaborative goal setting, and re-tasking robot role post-deployment, which each raise technical and sociotechnical issues in HRI. We exhibit JESSIE's reproducibility by replicating a clinician-created program on a TurtleBot~2. As an open-source means of accessing control synthesis, JESSIE supports reproducibility, scalability, and accessibility of personalized robots for HRI.

The robotics community continually strives to create robots that are deployable in real-world environments. Often, robots are expected to interact with human groups. To achieve this goal, we introduce a new method, the Robot-Centric Group Estimation Model (RoboGEM), which enables robots to detect groups of people. Much of the work reported in the literature focuses on dyadic interactions, leaving a gap in our understanding of how to build robots that can effectively team with larger groups of people. Moreover, many current methods rely on exocentric vision, where cameras and sensors are placed externally in the environment, rather than onboard the robot. Consequently, these methods are impractical for robots in unstructured, human-centric environments, which are novel and unpredictable. Furthermore, the majority of work on group perception is supervised, which can inhibit performance in real-world settings. RoboGEM addresses these gaps by being able to predict social groups solely from an egocentric perspective using color and depth (RGB-D) data. To achieve group predictions, RoboGEM leverages joint motion and proximity estimations. We evaluated RoboGEM against a challenging, egocentric, real-world dataset where both pedestrians and the robot are in motion simultaneously, and show RoboGEM outperformed two state-of-the-art supervised methods in detection accuracy by up to 30%, with a lower miss rate. Our work will be helpful to the robotics community, and serve as a milestone to building unsupervised systems that will enable robots to work with human groups in real-world environments.

Advancements within human–robot interaction generate increasing opportunities for proximate, goal-directed joint action (GDJA). However, robot errors are common and researchers must determine how to mitigate them. In this article, we examine how expectations for robot functionality affect people’s perceptions of robot reliability and trust for a robot that makes errors. Here 35 participants ( n = 35) performed a collaborative banner-hanging task with an autonomous mobile manipulator (Toyota HSR). Each participant received either a low- or high-functionality framing for the robot. We then measured how participants perceived the robot’s reliability and trust prior to, during, and after interaction. Functionality framing changed how robot errors affected participant experiences of robot behavior. People with low expectations experienced positive changes in reliability and trust after interacting with the robot, while those with high expectations experienced a negative change in reliability and no change in trust. The low-expectation group also showed greater trust recovery following the robot’s first error compared to the high group. Our findings inform human–robot teaming through: (1) identifying robot presentation factors that can be employed to facilitate trust calibration and (2) establishing the effects of framing, functionality, and the interactions between them to improve dynamic models of human–robot teaming.

Over 22 million people worldwide are affected by Parkinson's disease, stroke, and Bell's palsy (BP), which can cause facial paralysis (FP). People with FP have trouble having their expressions understood: both laypersons and clinicians have difficulty understanding them and often misinterpret them, which can result in poor social interactions and poor care delivery. One way to address this problem is through better education and training, of which computational tools may prove invaluable. Thus, in this paper, we explore how to build systems that can recognize and synthesize asymmetrical facial expressions. We introduce a novel computational model of asymmetric facial expressions for BP, which we can synthesize on either virtual and robotic patient simulators. We explore this within the context of clinical education, and built a patient simulator with synthesized FP in order to help clinicians perceive facial paralysis in patients. We conducted both computational and human-focused evaluations of the model, including the feedback from clinical experts. Our results suggest that our BP model is realistic, and comparable to the expressions of people with BP. Thus, this work has the potential to provide a practical training tool for clinical learners to better understand the expressions of people with BP. Our work can also help researchers in the facial recognition community to explore new methods for asymmetric facial expression analysis and synthesis.

Many new technologies are being built to support people with dementia. However, they largely focus on the people with dementia; consequently, informal caregivers, one of the most important stakeholders in dementia care, remain invisible within the technology design space. In this paper, we present a six-month long, community-based design research process where we collaborated with dementia caregiver support groups to design robots for dementia caregiving. The contributions of this paper are threefold. First, we broaden the context of dementia robot design to give a more prominent role to informal family care-givers in the co-design process. Second, we provide new design guidelines that contextualize robots within the family caregiving paradigm, which suggest new roles and behaviors of robots. These include lessening emotional labor by communicating information caregivees do not want to hear (e.g., regarding diet or medication) or providing redirection during emotionally difficult times, as well as facilitating positive shared moments. Third, our work found connections between certain robot attributes and their relationship to the stage of dementia a caregivee is experiencing. For example, caregivers wanted their robots to facilitate interaction with their caregivees in early stages of dementia, yet be in the background. However, for later stages of dementia, they wanted robots to replace caregiver-caregivee interaction to lessen their emotional burden, and be foregrounded. These connections provide important insights in to how we think about adaptability and long-term interaction in HRI. We hope our work provides new avenues for HRI researchers to studying robots for dementia caregivers by engaging in community-based design.

Worldwide, manufacturers are reimagining the future of their workforce and its connection to technology. Rather than replacing humans, Industry 5.0 explores how humans and robots can best complement one another's unique strengths. However, realizing this vision requires an in-depth understanding of how workers view the positive and negative attributes of their jobs, and the place of robots within it. In this paper, we explore the relationship between work attributes and automation goals by engaging in field research at a manufacturing plant. We conducted 50 face-to-face interviews with assembly-line workers (n=50), which we analyzed using discourse analysis and social constructivist methods. We found that the work attributes deemed most positive by participants include social interaction, movement and exercise, (human) autonomy, problem solving, task variety, and building with their hands. The main negative work attributes included health and safety issues, feeling rushed, and repetitive work. We identified several ways robots could help reduce negative work attributes and enhance positive ones, such as reducing work interruptions and cultivating physical and psychological well-being. Based on our findings, we created a set of integration considerations for organizations planning to deploy robotics technology, and discuss how the manufacturing and HRI communities can explore these ideas in the future.

In safety-critical environments, robots need to reliably recognize human activity to be effective and trust-worthy partners. Since most human activity recognition (HAR) approaches rely on unimodal sensor data (e.g. motion capture or wearable sensors), it is unclear how the relationship between the sensor modality and motion granularity (e.g. gross or fine) of the activities impacts classification accuracy. To our knowledge, we are the first to investigate the efficacy of using motion capture as compared to wearable sensor data for recognizing human motion in manufacturing settings. We introduce the UCSD-MIT Human Motion dataset, composed of two assembly tasks that entail either gross or fine-grained motion. For both tasks, we compared the accuracy of a Vicon motion capture system to a Myo armband using three widely used HAR algorithms. We found that motion capture yielded higher accuracy than the wearable sensor for gross motion recognition (up to 36.95%), while the wearable sensor yielded higher accuracy for fine-grained motion (up to 28.06%). These results suggest that these sensor modalities are complementary, and that robots may benefit from systems that utilize multiple modalities to simultaneously, but independently, detect gross and fine-grained motion. Our findings will help guide researchers in numerous fields of robotics including learning from demonstration and grasping to effectively choose sensor modalities that are most suitable for their applications.

The recent emergence of object proposal algorithms in the computer vision community shows great promise to addressing difficult problems in robotic such as object discovery and salient object detection. However, it is difficult to determine how these algorithms actually perform for real-world robot vision applications, because the standard evaluation protocol uses datasets which do not adequately account for real-world noise (motion blur, occlusion, etc.). We evaluated several state-of-the-art object proposal algorithms using naturalistic datasets from the robotics community, and found a substantial performance drop across all algorithms. This suggests that many object proposal algorithms are not as generalizable as the computer vision literature purports, which can have a significant impact on how they are applied to robotics. We also conducted a study on how each algorithm is influenced by specific kinds of real-world robot vision challenges, including variable brightness, gamma correction, Gaussian blur, and Gaussian noise. Our results provide insight into certain weaknesses of object proposal algorithms, which can be used to gauge how they might be suitable for different robotics applications. It is our intent that this work will motivate future research about how to design more flexible and robust object proposal algorithms for the robotics community.

In this work, we present a novel non-visual HAR system that achieves state-of-the-art performance on realistic SCE tasks via a single wearable sensor. We leverage surface electromyography and inertial data from a low-profile wearable sensor to attain performant robot perception while remaining unobtrusive and user-friendly. By capturing both convolutional and temporal features with a hybrid CNN-LSTM classifier, our system is able to robustly and effectively classify complex, full-body human activities with only this single sensor. We perform a rigorous analysis of our method on two datasets representative of SCE tasks, and compare performance with several prominent HAR algorithms. Results show our system substantially outperforms rival algorithms in identifying complex human tasks from minimal sensing hardware, achieving F1-scores up to 84% over 31 strenuous activity classes. To our knowledge, we are the first to robustly identify complex full-body tasks using a single, unobtrusive sensor feasible for real-world use in SCEs. Using our approach, robots will be able to more reliably understand human activity, enabling them to safely navigate sensitive, crowded spaces.

Patient safety errors account for over 400,000 preventable deaths annually in US in hospitals alone, 70% of which are caused by team communication breakdowns, stemming from hierarchical structures and asymmetrical power dynamics between physicians, nurses, patients, and others. Nurses are uniquely positioned to identify and prevent these errors, but they are often penalized for speaking up, particularly when physicians are responsible. Nevertheless, empowering nurses and building strong interdisciplinary teams can lead to improved patient safety and outcomes. Thus, our group has been developing a series of intelligent systems that support teaming in safety critical settings, Robot-Centric Team Support System (RoboTSS), and recently developed a group detection and tracking system for collaborative robots. In this paper, we explore how RoboTSS can be used to empower nurses in interprofessional team settings, through a three month long, collaborative design process with nurses across five US-based hospitals. The main findings and contributions of this paper are as follows. First, we found that participants envisioned using a robotic crash cart to guide resuscitation procedures to improve efficiency and reduce errors. Second, nurses discussed how RoboTSS can generate choreography for efficient spatial reconfigurations in co-located clinical teams, which is particularly important in time-sensitive situations such as resuscitation. Third, we found that nurses want to use RoboTSS to "stop the line," and disrupt power dynamics by policing unsafe physician behavior, such as avoiding safety protocols using a robotic crash cart. Fourth, nurses envisioned using our system to support real-time error identification, such as breaking the sterile field, and then communicating those errors to physicians, to relieve them of responsibility. Finally, based on our findings, we propose robot design implications that capture how nurses envision utilizing RoboTSS. We hope this work promotes further exploration in how to design technology to challenge authority in asymmetrical power relationships, particularly in healthcare, as strong teams save lives.

We are living in an exciting time, as people are living longer, more active lives. This is reshaping how we think about aging. Rather than viewing aging as a problem to be fixed (i.e., a deficit model of aging), many aging researchers are viewing aging as a developmental stage of life to be celebrated and supported, that is, “successful aging.” In this article, we embrace this approach and consider it in the context of assistive robot design in an aim to steer the conversation away from deficit models that have limited robot design possibilities. To explore an alternative design approach to the study of aging in human-robot interaction (HRI), we invited five aging researchers (three geriatricians, one gerontologist, and one epidemiologist) and nine older adults to participate in our research. In the study, participants illustrated their interpretations of aging and suggested potential assistive robots. We found that while all participants perceived the importance of potential disabilities due to aging, they considered potential disabilities as only one aspect of the experience of aging. They highlighted other key themes to consider in designing robots to support successful aging, such as older adults’ autonomy and resilience. We discuss these findings for the HRI community and call for “robots for successful aging.”

In this paper, we introduce a novel application of social robotics in healthcare: high fidelity, facially expressive, robotic patient simulators (RPSs), and explore their usage within a clinical experimental context. Current commercially-available RPSs, the most commonly used humanoid robots worldwide, are substantially limited in their usability and fidelity due to the fact that they lack one of the most important clinical interaction and diagnostic tools: an expressive face. Using autonomous facial synthesis techniques, we synthesized pain both on a humanoid robot and comparable virtual avatar. We conducted an experiment with 51 clinicians and 51 laypersons (n = 102), to explore differences in pain perception across the two groups, and also to explore the effects of embodiment (robot or avatar) on pain perception. Our results suggest that clinicians have lower overall accuracy in detecting synthesized pain in comparison to lay participants. We also found that all participants are overall less accurate detecting pain from a humanoid robot in comparison to a comparable virtual avatar, lending support to other recent findings in the HRI community. This research ultimately reveals new insights into the use of RPSs as a training tool for calibrating clinicians' pain detection skills.

This paper introduces Salient Depth Partitioning (SDP), a depth-based region cropping algorithm devised to be easily adapted to existing detection algorithms. SDP is designed to give robots a better sense of visual attention, and to reduce the processing time of pedestrian detectors. In contrast to proposal generators, our algorithm generates sparse regions, to combat image degradation caused by robot motion, making them more suitable for real-world operation. Furthermore, SDP is able achieve real-time performance (77 frames per second) on a single processor without a GPU. Our algorithm requires no training, and is designed to work with any pedestrian detection algorithm, provided that the input is in the form of a calibrated RGB-D image. We tested our algorithm with four state-of-the-art pedestrian detectors (HOG and SVM [1], Aggregate Channel Features [2], Checkerboards [3], and R-CNN [4]), and show that it improves computation time by up to 30%, with no discernible change in accuracy.

Robots with different behaviors will be a part of human-robot teams in the future and will impact the overall interaction patterns of teams. In this paper, we investigate how the presence of robots affect the coordination of human-robot teams when a single robot or multiple robots with the same or different behavior are the part of that team. We compare two different event anticipation methods for robots, and then extend those findings to assess its effects on the group coordination. Our results indicate that humans are significantly more synchronous as a group when they danced alone than with the robots. We also find that an addition of a robot with a different anticipation algorithm to a single robot team significantly reduces the group synchrony. This work will prove useful for the robotics community to build more fluent human-robot interactions in the future.

As robots enter human environments, they will be expected to accomplish a tremendous range of tasks. It is not feasible for robot designers to pre-program these behaviors or know them in advance, so one way to address this is through end-user programming, such as learning from demonstration (LfD). While significant work has been done on the mechanics of enabling robot learning from human teachers, one unexplored aspect is enabling mutual feedback between both the human teacher and robot during the learning process, i.e., implicit learning. In this paper, we explore one aspect of this mutual understanding, grounding sequences, where both a human and robot provide non-verbal feedback to signify their mutual understanding during interaction. We conducted a study where people taught an autonomous humanoid robot a dance, and performed gesture analysis to measure people's responses to the robot during correct and incorrect demonstrations.

Group interaction is an important aspect of human social behavior. During some group events, the activities performed by each group member continually influence the activities of others. This process of influence can lead to synchronized group activity, or the entrainment of the group. Understanding entrainment is important, because it can be a critical behavioral indicator of group cohesiveness, and can provide context for accurately understanding a group's affective behavior. In this paper, we present a novel method to automatically detect group psychomotor entrainment, which takes multiple types of discrete, task-level events into consideration. We experimentally validated the method on two synchronous rhythmic activities, “the cup game” and a marching task. We also compared its accuracy against two alternate synchrony detection methods in the literature. The results suggest our method can successfully measure group psychomotor entrainment, and is more accurate compared to other methods. This method will be useful to researchers interested in quantitatively and automatically measuring entrainment, and can also provide insight into understanding how groups interact socially.

In order to be effective teammates, robots need to be able to understand high-level human behavior to recognize, anticipate, and adapt to human motion. We have designed a new approach to enable robots to perceive human group motion in real time to anticipate future actions and synthesize their own motion accordingly. We explore this within the context of joint action, in which humans and robots move together synchronously. In this paper we present an anticipation method, which takes high-level group behavior into account. We validate the method within a human-robot interaction scenario, in which an autonomous mobile robot observes a team of human dancers and then successfully and contingently coordinates its movements to “join the dance.” We compared the results of our anticipation method to move the robot with another method that did not rely on high-level group behavior and found that our method performed better both in terms of more closely synchronizing the robot's motion to the team and exhibiting more contingent and fluent motion. These findings suggest that the robot performs better when it has an understanding of high-level group behavior than when it does not. This study will help enable others in the robotics community to build more fluent and adaptable robots in the future.

To become capable teammates to people, robots need the ability to interpret human activities and appropriately adjust their actions in real time. The goal of our research is to build robots that can work fluently and contingently with human teams. To this end, we have designed novel nonlinear dynamical methods to automatically model and detect synchronous joint action (SJA) in human teams. We also have extended this work to enable robots to move jointly with human teammates in real time. In this paper, we describe our work to date, and discuss our future research plans to further explore this research space. The results of this work are expected to benefit researchers in social signal processing, human-machine interaction, and robotics.

As robots enter human spaces, unique perception challenges are emerging. Sensing human activity, adapting to highly dynamic environments, and acting coherently and contingently is challenging when robots transition from structured environments to human-centric ones. We approach this problem by employing context-based perception, a biologically-inspired, low-cost approach to sensing that leverages noisy, global features. Across several months, our mobile robot collected real-world, multimodal data from multi-use locations; where the same space might be used for many different activities. We then ran a series of unimodal and multimodal classification experiments. We successfully classified several aspects of situational context from noisy data, and, to our knowledge are the first group to do so. This work represents an important step toward enabling robots that can readily leverage context to solve perceptual tasks.

To be effective team members, it is important for robots to understand the high-level behaviors of collocated humans. This is a challenging perceptual task when both the robots and people are in motion. In this paper, we describe an event-based model for multiple robots to automatically measure synchronous joint action of a group while both the robots and co-present humans are moving. We validated our model through an experiment where two people marched both synchronously and asynchronously, while being followed by two mobile robots. Our results suggest that our model accurately identifies synchronous motion, which can enable more adept human-robot collaboration.

Contextual cues strongly in influence the behavior of people in social environments, and people are very adept at interpreting and responding to these cues. While robots are becoming increasingly present in these spaces, they do not yet share humans' essential sense of contextually-bounded social propriety. However, it is essential for robots to be able to modify their behavior depending on context so that they operate in an appropriate manner across a variety of situations. In our work, we are building models of context for social robots, that operate on real-world, naturalistic, noisy data, across multi-context and multi-person settings. In this paper, we discuss one aspect of this work, which concerns teaching a robot an appropriateness function for interrupting a person in a public space. We trained a support-vector machine (SVM) to learn an association between contextual cues and the reaction of people being interrupted by a robot across three different contexts. Overall, our results are promising, and further work on integrating context models into social robots could lead to interesting and impactful findings across the HRI community.

This workshop covers real-time computational techniques for the recognition and interpretation of human affective and social behaviour, and techniques for synthesis of believable social behaviour supporting real-time adaptive human-agent and human-robot interaction in real-world environments.

Currently in medical education, clinical students learn how to interact with real patients via simulated patients, which are inexpressive, teleoperated robot mannequins. We obtained five simulations that used such a robot to explore verbal communication between clinical students and the robot patient, specifically if the students sought approval before performing haptic-actions. We found that in our sample, student clinicians frequently acted without seeking approval or providing information to the robot patient. We hope to further our studies in order to identify if either current training of clinical students in communication is ineffective, or if the robot patients are too nonhuman-like and inexpressive to engender appropriate communication.

There is much discussion in the robotics community concerning the nature of people's impressions of robots. This pilot study employed the use of mobile robots coupled with artistic elements to create an environment conducive to human participation. PhotoBot took photos of participants (n = 16) in a gallery space and provided them with a physical copy of their image, while ProjectorBot displayed 3D Kinect imagery for participants to view. Participants completed a self-report measure of rapport (Bernieri's Rapport Criterion); the results of which suggest that they experienced a high degree of positive interaction with the robots.

Many researchers use Wizard of Oz (WoZ) as an experimental technique, but there are methodological concerns over its use, and no comprehensive criteria on how to best employ it. We systematically review 54 WoZ experiments published in the primary HRI publication venues from 2001 -- 2011. Using criteria proposed by Fraser and Gilbert (1991), Green et al. (2004), Steinfeld et al. (2009), and Kelley (1984), we analyzed how researchers conducted HRI WoZ experiments. Researchers mainly used WoZ for verbal (72.2%) and non-verbal (48.1%) processing. Most constrained wizard production (90.7%), but few constrained wizard recognition (11%). Few reported measuring wizard error (3.7%), and few reported pre-experiment wizard training (5.4%). Few reported using WoZ in an iterative manner (24.1%). Based on these results we propose new reporting guidelines to aid future research.

Affect-sensitive systems such as social robots and virtual agents are increasingly being investigated in real-world settings. In order to work effectively in natural environments, these systems require the ability to infer the affective and mental states of humans and to provide appropriate timely output that helps to sustain long-term interactions. This special issue, which appears in two parts, includes articles on the design of socio-emotional behaviors and expressions in robots and virtual agents and on computational approaches for the automatic recognition of social signals and affective states.

Cooperative gestures are a key aspect of human-human pro-social interaction. Thus, it is reasonable to expect that endowing humanoid robots with the ability to use such gestures when interacting with humans would be useful. However, while people are used to responding to such gestures expressed by other humans, it is unclear how they might react to a robot making them. To explore this topic, we conducted a within-subjects, video based laboratory experiment, measuring time to cooperate with a humanoid robot making interactional gestures. We manipulated the gesture type (beckon, give, shake hands), the gesture style (smooth, abrupt), and the gesture orientation (front, side). We also employed two measures of individual differences: negative attitudes toward robots (NARS) and human gesture decoding ability (DANVA2-POS). Our results show that people cooperate with abrupt gestures more quickly than smooth ones and front-oriented gestures more quickly than those made to the side, people's speed at decoding robot gestures is correlated with their ability to decode human gestures, and negative attitudes toward robots is strongly correlated with a decreased ability in decoding human gestures.

The 3rd International Workshop on Affective Interaction in Natural Environments, AFFINE, follows a number of successful AFFINE workshops and events commencing in 2008.A key aim of AFFINE is the identification and investigation of significant open issues in real-time, affect-aware applications 'in the wild' and especially in embodied interaction, for example, with robots or virtual agents. AFFINE seeks to bring together researchers working on the real-time interpretation of user behaviour with those who are concerned with social robot and virtual agent interaction frameworks.

A long-standing question within the robotics community is about the degree of human-likeness robots ought to have when interacting with humans. We explore an unexamined aspect of this problem: how people empathize with robots along the anthropomorphic spectrum. We conducted an experiment that measured how people empathized with robots shown to be experiencing mistreatment by humans. Our results indicate that people empathize more strongly with more human-looking robots and less with mechanicallooking robots.

This paper presents a set of candidate requirements and survey questions for physical avatar systems as derived from the literature. These requirements will be applied to analyze a fictional, yet well-envisioned, physical avatar system depicted in the film Hinokio. It is hoped that these requirements and survey questions can be used by other researchers as a guide when performing formal engineering tradeoff analysis during the design phase of new physical avatar systems, or during evaluation of existing systems.

This paper presents a fine-grained decomposition of situation awareness (SA) as it pertains to the use of unmanned aerial vehicles (UAVs), and uses this decomposition to understand the types of SA attained by operators of the Desert Hawk UAV. Since UAVs are airborne robots, we adapt a definition previously developed for human-robot awareness after learning about the SA needs of operators through observations and interviews. We describe the applicability of UAV-related SA for people in three roles: UAV operators, air traffic controllers, and pilots of manned aircraft in the vicinity of UAVs. Using our decomposition, UAV interaction designers can specify SA needs and analysts can evaluate a UAV interface's SA support with greater precision and specificity than can be attained using other SA definitions.