While resuscitation training is critical, healthcare workers (HCWs) with high workload have limited chance to get trained and re-trained due to time and resource constraints. To address this gap, we engaged in a co-design process of robots that facilitate and prepare HCWs for resuscitation procedures (i.e., codes). First, we investigated what resuscitation training consists of, including challenges faced by trainees and trainers. Second, we collaboratively explored how a crash cart robot, that guides users to medical supplies and equipment, could assist trainers and trainees synchronously–during team-based clinical simulations and asynchronously–during one-on-one training. We found that robots could 1) serve as a learning assistant by providing real-time feedback and supporting personalized training needs; and 2) an evaluating assistant by monitoring multiple trainees and tracking critical timing of interventions in the training. Through this new training paradigm, we hope to demonstrate opportunities for crash cart robots to aid HCWs for their sustainable training and reskilling. We discuss the role of robots in training beyond cognitive knowledge, situating them within two underexplored contexts: practical skill training and team-based training.

Patients with cancer (PwC) have a hard time getting prompt treatment in acute care settings, and feel unseen, unheard, and neglected. This is due to systemic problems: worldwide, Emergency Department (ED) healthcare workers (HCWs) are overworked and EDs are understaffed. Robots will not fix these problems; however, prior work suggests if well-designed and contextualized, they may support cancer care. Based on longstanding collaborations with PwC and ED HCWs, in this paper we report on an exploration of the design space of social robots for acute cancer care. Using a care ethics lens, we found robots can be uniquely positioned to amplify compassion within deeply human care relationships through their social presence, while performing routine tasks, such as patient monitoring. However, participants suggested the human experiences of pain and distress may remain elusive for robots to engage with meaningfully. Our work reveals HCWs and PwC saw robots as means to expand relational care in the ED, and explores how future HRI research may meaningfully support these care relationships.

Designing robots to support high-stakes teamwork in emergency settings presents unique challenges, including seamless integration into fast-paced environments, facilitating effective communication among team members, and adapting to rapidly changing situations. While teleoperated robots have been successfully used in high-stakes domains such as firefighting and space exploration, autonomous robots that aid high-stakes teamwork remain underexplored. To address this gap, we conducted a rapid prototyping process to develop a series of seemingly autonomous robots designed to assist clinical teams in the Emergency Room. We transformed a standard crash cart-which stores medical equipment and emergency supplies into a medical robotic crash cart (MCCR). The MCCR was evaluated through field deployments to assess its impact on team workload and usability, identified taxonomies of failure, and refined the MCCR in collaboration with healthcare professionals. Our work advances the understanding of robot design for high-stakes, time-sensitive settings, providing insights into useful MCCR capabilities and considerations for effective human-robot collaboration. By publicly disseminating our MCCR tutorial, we hope to encourage HRI researchers to explore the design of robots for high-stakes teamwork.

Healthcare workers (HCWs) encounter challenges in hospitals, such as retrieving medical supplies quickly from crash carts, which could potentially result in medical errors and delays in patient care. Robotic crash carts (RCCs) have shown promise in assisting healthcare teams during medical tasks through guided object searches and task reminders. Limited exploration has been done to determine what communication modalities are most effective and least disruptive to patient care in real-world settings. To address this gap, we conducted a between-subjects experiment comparing the RCC’s verbal and non-verbal communication of object search with a standard crash cart in resuscitation scenarios to understand the impact of robot communication on workload and attitudes toward using robots in the workplace. Our findings indicate that verbal communication significantly reduced mental demand and effort compared to visual cues and with a traditional crash cart. Although, frustration levels were slightly higher during collaborations with the robot compared to a traditional cart. These research insights provide valuable implications for human-robot teamwork in high-stakes environments.

The human-robot interaction (HRI) field has recognized the importance of enabling robots to interact with teams. Human teams rely on effective communication for successful collaboration in time-sensitive environments. Robots can play a role in enhancing team coordination through real-time assistance. Despite significant progress in human-robot teaming research, there remains an essential gap in how robots can effectively communicate with action teams using multimodal interaction cues in time-sensitive environments. This study addresses this knowledge gap in an experimental in-lab study to investigate how multimodal robot communication in action teams affects workload and human perception of robots. We explore team collaboration in a medical training scenario where a robotic crash cart (RCC) provides verbal and non-verbal cues to help users remember to perform iterative tasks and search for supplies. Our findings show that verbal cues for object search tasks and visual cues for task reminders reduce team workload and increase perceived ease of use and perceived usefulness more effectively than a robot with no feedback. Our work contributes to multimodal interaction research in the HRI field, highlighting the need for more human-robot teaming research to understand best practices for integrating collaborative robots in time-sensitive environments such as hospitals, search and rescue, and manufacturing applications.

Healthcare workers (HCWs) face many challenges during bedside care that impede team collaboration and often lead to poor patient outcomes. Robots have the potential to support medical decision-making, help identify medical errors, and deliver supplies to clinical teams in a timely manner. However, there is a lack of knowledge about using robots to support clinical team dynamics despite being used in surgery, healthcare operations, and other applications. To address this gap, we engaged in a co-design process of robots that support clinical teamwork. We collaboratively explore how robots can support clinical teamwork with HCWs. This collaborative process includes understanding the challenges they face during bedside care and envisioning robots that can help mitigate these issues. Our study shows that robots can act as a shared mental model for clinical teams, help close communication gaps, and provide procedural steps to assist HCWs with limited in-hospital experience. This research highlights new ways HRI researchers can deploy robots in acute care settings, as well as define appropriate levels of autonomy to maintain human control in safety-critical settings.

Our bodies are constantly in motion—from the bending of arms and legs to the less conscious movement of breathing, our precise shape and location change constantly. This can make subtler developments (e.g., the growth of hair, or the healing of a wound) difficult to observe. Our work focuses on helping users record and visualize this type of subtle, longer-term change. We present a mobile tool that combines custom 3D tracking with interactive visual feedback and computational imaging to capture personal time-lapse, which approximates longer-term video of the subject (typically, part of the capturing user’s body) under a fixed viewpoint, body pose, and lighting condition. These personal time-lapses offer a powerful and detailed way to track visual changes of the subject over time. We begin with a formative study that examines what makes personal time-lapse so difficult to capture. Building on our findings, we motivate the design of our capture tool, evaluate this design with users, and demonstrate its effectiveness in a variety of challenging examples.

To facilitate HRI's transition from dyadic to group interaction, new methods are needed for robots to sense and understand team behavior. We introduce the Robot-Centric Group Detection and Tracking System (REGROUP), a new method that enables robots to detect and track groups of people from an ego-centric perspective using a crowd-aware, tracking-by-detection approach. Our system employs a novel technique that leverages person re-identification deep learning features to address the group data association problem. REGROUP is robust to real-world vision challenges such as occlusion, camera egomotion, shadow, and varying lighting illuminations. Also, it runs in real-time on real-world data. We show that REGROUP outperformed three group detection methods by up to 40% in terms of precision and up to 18 % in terms of recall. Also, we show that REGROUP's group tracking method outperformed three state-of-the-art methods by up to 66% in terms of tracking accuracy and 20% in terms of tracking precision. We plan to publicly release our system to support HRI teaming research and development. We hope this work will enable the development of robots that can more effectively locate and perceive their teammates, particularly in uncertain, unstructured environments.

The Emergency Department (ED) is a stressful, safety-critical environment, which is often overcrowded, noisy, chaotic, and understaffed. The built environment plays a key role in patient outcomes, experiences, and the mental health of healthcare workers (HCWs). However, once a space is built, it is difficult to change it; so the modularity and adaptability of new technologies such as robots could potentially help stakeholders mitigate some of these challenges; yet, there is a lack of research in this area, particularly in the ED. In this paper, we address this gap by engaging HCWs in a research-through-design process, utilizing design fiction, to envision a future resilient ED. Here, robots scurry along the ceiling, provide help at the bedside, and smart furniture and walls provide spaces for privacy and calm. We co-created design prototypes of future intelligent systems that can modify the built environment to support resilience, which we then used to co-create a Design Catalog with HCWs, which contains a collection of future technology prototypes contextualized within the ED. We found that HCWs envisioned many ways for intelligent systems to help them reimagine the built environment, including ways to enhance HCW-patient communication, improve patient experience, support both HCW and patient safety, and use reconfigurable spaces to support privacy. We hope our work inspires further exploration into using new technologies to reimagine and reconfigure the built environment to support resilient hospitals.

The emergency department (ED) is a safety-critical environment in which healthcare workers (HCWs) are overburdened, overworked, and have limited resources, especially during the COVID-19 pandemic. One way to address this problem is to explore the use of robots that can support clinical teams, e.g., to deliver materials or restock supplies. However, due to EDs being overcrowded, and the cognitive overload HCWs experience, robots need to understand various levels of patient acuity so they avoid disrupting care delivery. In this paper, we introduce the Safety-Critical Deep Q-Network (SafeDQN) system, a new acuity-aware navigation system for mobile robots. SafeDQN is based on two insights about care in EDs: high-acuity patients tend to have more HCWs in attendance and those HCWs tend to move more quickly. We compared SafeDQN to three classic navigation methods, and show that it generates the safest, quickest path for mobile robots when navigating in a simulated ED environment. We hope this work encourages future exploration of social robots that work in safety-critical, human-centered environments, and ultimately help to improve patient outcomes and save lives.

The robotics community continually strives to create robots that are deployable in real-world environments. Often, robots are expected to interact with human groups. To achieve this goal, we introduce a new method, the Robot-Centric Group Estimation Model (RoboGEM), which enables robots to detect groups of people. Much of the work reported in the literature focuses on dyadic interactions, leaving a gap in our understanding of how to build robots that can effectively team with larger groups of people. Moreover, many current methods rely on exocentric vision, where cameras and sensors are placed externally in the environment, rather than onboard the robot. Consequently, these methods are impractical for robots in unstructured, human-centric environments, which are novel and unpredictable. Furthermore, the majority of work on group perception is supervised, which can inhibit performance in real-world settings. RoboGEM addresses these gaps by being able to predict social groups solely from an egocentric perspective using color and depth (RGB-D) data. To achieve group predictions, RoboGEM leverages joint motion and proximity estimations. We evaluated RoboGEM against a challenging, egocentric, real-world dataset where both pedestrians and the robot are in motion simultaneously, and show RoboGEM outperformed two state-of-the-art supervised methods in detection accuracy by up to 30%, with a lower miss rate. Our work will be helpful to the robotics community, and serve as a milestone to building unsupervised systems that will enable robots to work with human groups in real-world environments.

Patient safety errors account for over 400,000 preventable deaths annually in US in hospitals alone, 70% of which are caused by team communication breakdowns, stemming from hierarchical structures and asymmetrical power dynamics between physicians, nurses, patients, and others. Nurses are uniquely positioned to identify and prevent these errors, but they are often penalized for speaking up, particularly when physicians are responsible. Nevertheless, empowering nurses and building strong interdisciplinary teams can lead to improved patient safety and outcomes. Thus, our group has been developing a series of intelligent systems that support teaming in safety critical settings, Robot-Centric Team Support System (RoboTSS), and recently developed a group detection and tracking system for collaborative robots. In this paper, we explore how RoboTSS can be used to empower nurses in interprofessional team settings, through a three month long, collaborative design process with nurses across five US-based hospitals. The main findings and contributions of this paper are as follows. First, we found that participants envisioned using a robotic crash cart to guide resuscitation procedures to improve efficiency and reduce errors. Second, nurses discussed how RoboTSS can generate choreography for efficient spatial reconfigurations in co-located clinical teams, which is particularly important in time-sensitive situations such as resuscitation. Third, we found that nurses want to use RoboTSS to "stop the line," and disrupt power dynamics by policing unsafe physician behavior, such as avoiding safety protocols using a robotic crash cart. Fourth, nurses envisioned using our system to support real-time error identification, such as breaking the sterile field, and then communicating those errors to physicians, to relieve them of responsibility. Finally, based on our findings, we propose robot design implications that capture how nurses envision utilizing RoboTSS. We hope this work promotes further exploration in how to design technology to challenge authority in asymmetrical power relationships, particularly in healthcare, as strong teams save lives.

This paper introduces Salient Depth Partitioning (SDP), a depth-based region cropping algorithm devised to be easily adapted to existing detection algorithms. SDP is designed to give robots a better sense of visual attention, and to reduce the processing time of pedestrian detectors. In contrast to proposal generators, our algorithm generates sparse regions, to combat image degradation caused by robot motion, making them more suitable for real-world operation. Furthermore, SDP is able achieve real-time performance (77 frames per second) on a single processor without a GPU. Our algorithm requires no training, and is designed to work with any pedestrian detection algorithm, provided that the input is in the form of a calibrated RGB-D image. We tested our algorithm with four state-of-the-art pedestrian detectors (HOG and SVM [1], Aggregate Channel Features [2], Checkerboards [3], and R-CNN [4]), and show that it improves computation time by up to 30%, with no discernible change in accuracy.

In the recent years, we have witnessed social networks blossom. Social networking reshaped worldwide communication significantly increased the speed of news spread, and connected the world stronger than ever. Although social networking has been such a revolutionary invention for the society, and many researchers have turned towards social media to explore trending topics, mainstream media still remains as the origin of the majority of the news discussed in social networking sites. Social stream mining to make video recommendations based on the trending topics has been an active direction in the research community. Understanding the trending topics and its impact on video sharing sites is very interesting for network traffic engineers. Quality of service can be significantly improved if we can predict what kind of video content will generate large traffic. The focus of this paper is to study which type of media, mainstream or social, can contribute better towards identifying trending topics. We present the experimental study of the story development process in mainstream and social media based on the real-world data. The study helps us properly identify which media source is more appropriate for the video recommendation and network traffic prediction systems. Through our findings, we discovered mainstream media could significantly improve the trend detection.