VLDB 2026 Research / reviewers in the wild / expert
Aleksandra Sarcevic
dblp:14/1957
· DBLP profile ↗
52ranked-venue papers
11as first author
22since 2021 · last 2026
0000-0003-2126-8527ORCID · verified
Domains — the database's venue-derived domains; a paper can count in several
Human-computer interaction and ubiquitous computing · 37 · 9 first-author · 15 since 2021Applied, interdisciplinary, general and emerging computing · 12 · 2 first-author · 6 since 2021Computer networks · 2Artificial intelligence and machine learning · 1 · 1 since 2021
| Year | Publication | Venue | Position |
|---|---|---|---|
| 2026 | SAFE: A Smart Adherence Detection Framework for Monitoring Personal Protective Equipment in Healthcare SettingsabstractPersonal protective equipment (PPE) is critical for infection control in healthcare, protecting workers and patients from infection risks. The COVID-19 pandemic further highlighted the importance of correct PPE use, yet adherence to U.S. Centers for Disease Control and Prevention guidelines remains inconsistent. Continuous human monitoring of PPE adherence is impractical because it is labor-intensive and may expose observers to infection risk. Automated monitoring is a promising alternative, but reliable PPE assessment in clinical videos remains difficult due to occlusion and subtle differences between adherence levels. To address these challenges, we propose SAFE - Smart Adherence detection Framework for PPE, a cascaded computer vision system for real-time monitoring of PPE wearing status, including complete, incomplete, and absent cases, with a focus on gowns and masks. SAFE uses a two-stage design: Stage 1 detects gown status and localizes head regions, and Stage 2 classifies mask status from head crops. We evaluate SAFE on R2PPE, a ceiling-view trauma-room simulation dataset with dense PPE annotations and complex scenes. SAFE improves overall average precision from 0.48 to 0.67 and increases mask-class average precision by 0.33 compared to a baseline one-stage detector. We further validate SAFE across modern detector backbones, including transformer-based detectors, and on real-case trauma-room data using class-level and alarm-level criteria, improving class-level mask accuracy from 0.59 to 0.65 while maintaining a high alarm-level recall of 0.98. SAFE could enhance PPE monitoring with minimal human intervention, providing a scalable solution for improving infection control in healthcare settings. Wanzhao Yang, Beomseok Park, Mary S. Kim, Aleksandra Sarcevic, Syed Muhammad Anwar, Marius G. Linguraru, Ivan Marsic, Randall S. Burd |
IEEE J. Biomed. Health Informatics | 4 |
| 2025 | Understanding Personal Protective Equipment Use in Interdisciplinary Medical Settings: Design Explorations for Just-in-Time Compliance Alerts: Improving PPE Practices in Medical Settings Through Alert DesignabstractWe examine the use of personal protective equipment (PPE) in two interdisciplinary medical settings to inform the design of just-in-time alerts and reminders for correcting PPE noncompliance. We reviewed videos of 26 pediatric resuscitations occurring over the course of the COVID-19 pandemic at an urban pediatric teaching hospital. Through video review, we identified causes for PPE noncompliance, activities that were frequently performed without PPE, instances in which PPE was intentionally removed, and mechanisms by which healthcare providers corrected PPE noncompliance. We also interviewed 18 registered nurses working in the hospital's emergency department (ED) and intensive care unit (ICU) to better understand observed PPE behaviors and practices. Our results suggest that alert design will require considering the urgency of correcting PPE noncompliance against the urgency of tasks being performed. We discuss our findings through the lens of the COM-B framework and conclude by exploring design opportunities for just-in-time alerts and reminders for prompting PPE noncompliance corrections in dynamic medical work. Aleksandra Sarcevic, Eleanor Wood, Katherine Ann Zellner, Christine Dodeye Ikponmwonba, Mary S. Kim, Ivan Marsic, Randall S. Burd |
Conference on Designing Interactive Systems | 1 |
| 2025 | Comparative analysis of personal protective equipment nonadherence detection: computer vision versus human observersabstractOBJECTIVES: Human monitoring of personal protective equipment (PPE) adherence among healthcare providers has several limitations, including the need for additional personnel during staff shortages and decreased vigilance during prolonged tasks. To address these challenges, we developed an automated computer vision system for monitoring PPE adherence in healthcare settings. We assessed the system performance against human observers detecting nonadherence in a video surveillance experiment. MATERIALS AND METHODS: The automated system was trained to detect 15 classes of eyewear, masks, gloves, and gowns using an object detector and tracker. To assess how the system performs compared to human observers in detecting nonadherence, we designed a video surveillance experiment under 2 conditions: variations in video durations (20, 40, and 60 seconds) and the number of individuals in the videos (3 versus 6). Twelve nurses participated as human observers. Performance was assessed based on the number of detections of nonadherence. RESULTS: Human observers detected fewer instances of nonadherence than the system (parameter estimate -0.3, 95% CI -0.4 to -0.2, P < .001). Human observers detected more nonadherence during longer video durations (parameter estimate 0.7, 95% CI 0.4-1.0, P < .001). The system achieved a sensitivity of 0.86, specificity of 1, and Matthew's correlation coefficient of 0.82 for detecting PPE nonadherence. DISCUSSION: An automated system simultaneously tracks multiple objects and individuals. The system performance is also independent of observation duration, an improvement over human monitoring. CONCLUSION: The automated system presents a potential solution for scalable monitoring of hospital-wide infection control practices and improving PPE usage in healthcare settings. Mary S. Kim, Beomseok Park, Genevieve J. Sippel, Aaron H. Mun, Wanzhao Yang, Kathleen H. McCarthy, Emely Fernandez, Marius George Linguraru, Aleksandra Sarcevic, Ivan Marsic, Randall S. Burd |
J. Am. Medical Informatics Assoc. | 9 |
| 2025 | Human intention recognition for trauma resuscitation: An interpretable deep learning approach for medical process data
Mary S. Kim, Sen Yang 0002, Genevieve J. Sippel, Aleksandra Sarcevic, Randall S. Burd, Ivan Marsic |
J. Biomed. Informatics | 6 |
| 2025 | Beyond Decision Making: Considering Collaboration and Agency in the Design of AI-Based Decision-Support Systems for Fast-Response Medical TeamsabstractIn addition to aiding decision making, AI-based clinical decision-support systems may need to consider and support provider agency and collaboration between medical providers. We analyzed collaboration and agency within fast-response teams, identifying implications for designing decision-support systems that not only facilitate decision making, but also collaboration and agency. Using an Actor-Network Theory approach, we reviewed videos of 12 pediatric trauma resuscitations and conducted a secondary analysis of 27 interviews with trauma team members. We identified actants in trauma resuscitation, shifts in agency that can occur within fast-response teams during medical emergencies, and factors considered by providers when envisioning the design of decision-support systems. From our analysis, we propose implications for existing human-AI interaction guidelines when designing AI systems for fast-response medical teams. We also highlight parallels between the introduction of clinical practice guidelines and the introduction of AI-based decision-support systems, suggesting that these systems may influence the training and ''clinical gaze'' of providers. Angela Mastrianni, Paige Kmetz-Cutrone, Kathryn Chang, Jonathan Y. Stein, Aleksandra Sarcevic |
Proc. ACM Hum. Comput. Interact. | 5 |
| 2025 | To Recommend or Not to Recommend: Designing and Evaluating AI-Enabled Decision Support for Time-Critical Medical EventsabstractAI-enabled decision-support systems aim to help medical providers rapidly make decisions with limited information during medical emergencies. A critical challenge in developing these systems is supporting providers in interpreting the system output to make optimal treatment decisions. In this study, we designed and evaluated an AI-enabled decision-support system to aid providers in treating patients with traumatic injuries. We first conducted user research with physicians to identify and design information types and AI outputs for a decision-support display. We then conducted an online experiment with 35 medical providers from six health systems to evaluate two human-AI interaction strategies: (1) AI information synthesis and (2) AI information and recommendations. We found that providers were more likely to make correct decisions when AI information and recommendations were provided compared to receiving no AI support. We also identified two socio-technical barriers to providing AI recommendations during time-critical medical events: (1) an accuracy-time trade-off in providing recommendations and (2) polarizing perceptions of recommendations between providers. We discuss three implications for developing AI-enabled decision support used in time-critical events, contributing to the limited research on human-AI interaction in this context. Angela Mastrianni, Mary S. Kim, Travis M. Sullivan, Genevieve J. Sippel, Randall S. Burd, Krzysztof Z. Gajos, Aleksandra Sarcevic |
Proc. ACM Hum. Comput. Interact. | 7 |
| 2025 | Addressing Teamwork Delays during Life-Saving Interventions through an Activity Theory-Informed AnalysisabstractHemorrhage, or severe blood loss due to injury, is a leading cause of preventable deaths after injury. This study uses and extends activity theory to understand the dynamics of team-based hemorrhage control during trauma resuscitation and to explore potential computerized mechanisms to support this time- and safety-critical process. We reviewed videos of 25 resuscitation cases and analyzed hemorrhage control activities using nine activity theory prompts, including a new prompt-speech intention-a critical but underexplored dimension of teamwork in prior activity theory analyses. Through this process, we identified the most common delay-causing activities and developed routine and non-routine activity models for each. A comparison of these models showed that variations from the routine models emerged due to changes in the division of labor, instruments, community, and speech intentions. We contribute to research on designing socio-technical systems by (1) identifying needs and opportunities for computerized support that address delays in complex medical teamwork and (2) examining how an intervention changes an activity model. We also show how adding detailed speech data aids in identifying contradictions between elements in an activity model. Katherine Ann Zellner, Aleksandra Sarcevic, Maja Barnouw, Megan A. Krentsa, Travis M. Sullivan, Mary S. Kim, Randall S. Burd |
Proc. ACM Hum. Comput. Interact. | 2 |
| 2025 | AI-Enhanced Sensemaking: Exploring the Design of a Generative AI-Based Assistant to Support Genetic ProfessionalsabstractGenerative AI has the potential to transform knowledge work, but further research is needed to understand how knowledge workers envision using and interacting with generative AI. We investigate the development of generative AI tools to support domain experts in knowledge work, examining task delegation and the design of human–AI interactions. Our research focused on designing a generative AI assistant to aid genetic professionals in analyzing whole genome sequences (WGS) and other clinical data for rare disease diagnosis. Through interviews with 17 genetics professionals, we identified current challenges in WGS analysis. We then conducted co-design sessions with six genetics professionals to determine tasks that could be supported by an AI assistant and considerations for designing interactions with the AI assistant. From our findings, we identified sensemaking as both a current challenge in WGS analysis and a process that could be supported by AI. We contribute an understanding of how domain experts envision interacting with generative AI in their knowledge work, a detailed empirical study of WGS analysis, and three design considerations for using generative AI to support domain experts in sensemaking during knowledge work. Angela Mastrianni, Hope Twede, Aleksandra Sarcevic, Jeremiah Wander, Christina Austin-Tse, T. Scott Saponas, Heidi L. Rehm, Ashley Mae Conard, Amanda K. Hall |
ACM Trans. Interact. Intell. Syst. | 3 |
| 2023 | Supporting Awareness of Dynamic Data: Approaches to Designing and Capturing Data within Interactive Clinical ChecklistsabstractAutomatically integrating data within interactive clinical checklists allows for enhanced dynamic displays, while also providing information needed for checklist adaptation to the context of the medical event. In this mixed-methods study, we used user-centered design sessions with clinicians to design a checklist interface that automatically captures and displays dynamic patient data. We compared the manual and automatic checklist versions during video-guided simulation sessions, evaluating the effects of automatic capture on clinicians' interactions with dynamic data and their situation awareness. Despite clinicians' concerns that automatic data capture would affect situation awareness, we found no significant difference in awareness scores. Participants preferred the automatic version, highlighting its improved accuracy and completeness. From our findings, we propose a framework for capturing dynamic data and designing dynamic data interfaces within interactive checklists. We conclude by discussing barriers and design opportunities for supporting awareness of data trends through checklists. Angela Mastrianni, Aleksandra Sarcevic, Megan A. Krentsa, Travis M. Sullivan, Issa Zakeri, Ivan Marsic, Randall S. Burd |
Conference on Designing Interactive Systems | 2 |
| 2023 | Discovering interpretable medical process models: A case study in trauma resuscitation
Ivan Marsic, Aleksandra Sarcevic, Sen Yang 0002, Travis M. Sullivan, Peyton E. Tempel, Zachary P. Milestone, Karen J. O'Connell, Randall S. Burd |
J. Biomed. Informatics | 3 |
| 2023 | Understanding Delay Awareness and Mitigation Mechanisms through an Iterative Design and Evaluation of a Prototype Alert System for Complex TeamworkabstractAlmost half of the preventable deaths in emergency care can be associated with a medical delay. Understanding how clinicians experience delays can lead to improved alert designs to increase delay awareness and mitigation. In this paper, we present the findings from an iterative user-centered design process involving 48 clinicians to develop a prototype alert system for supporting delay awareness in complex medical teamwork such as trauma resuscitation. We used semi-structured interviews and card-sorting workshops to identify the most common delays and elicit design requirements for the prototype alert system. We then conducted a survey to refine the alert designs, followed by near-live, video-guided simulations to investigate clinicians' reactions to the alerts. We contribute to CSCW by designing a prototype alert system to support delay awareness in time-critical, complex teamwork and identifying four mechanisms through which teams mitigate delays. Katherine Ann Zellner, Aleksandra Sarcevic, Megan A. Krentsa, Travis M. Sullivan, Randall S. Burd |
Proc. ACM Hum. Comput. Interact. | 2 |
| 2023 | Transitioning Cognitive Aids into Decision Support Platforms: Requirements and Design GuidelinesabstractDigital cognitive aids have the potential to serve as clinical decision support platforms, triggering alerts about process delays and recommending interventions. In this mixed-methods study, we examined how a digital checklist for pediatric trauma resuscitation could trigger decision support alerts and recommendations. We identified two criteria that cognitive aids must satisfy to support these alerts: (1) context information must be entered in a timely, accurate, and standardized manner, and (2) task status must be accurately documented. Using co-design sessions and near-live simulations, we created two checklist features to satisfy these criteria: a form for entering the pre-hospital information and a progress slider for documenting the progression of a multi-step task. We evaluated these two features in the wild, contributing guidelines for designing these features on cognitive aids to support alerts and recommendations in time- and safety-critical scenarios. Angela Mastrianni, Aleksandra Sarcevic, Allison Hu, Lynn Almengor, Peyton E. Tempel, Sarah Gao, Randall S. Burd |
ACM Trans. Comput. Hum. Interact. | 2 |
| 2022 | An Analysis of Speech during Life Saving Interventions to Inform the Design of a Computerized System for Delay Detection
Katherine Ann Zellner, Louis Jiorgio Villegas, Charles Neff, Waverly Gestrich-Thompson, Randall S. Burd, Ivan Marsic, Aleksandra Sarcevic |
AMIA | 7 |
| 2022 | PACMHCI V6, CSCW1, April 2022 EditorialabstractNo abstract available. Shaowen Bardzell, Siân E. Lindley, Aleksandra Sarcevic, Hideaki Kuzuoka, Katharina Reinecke, Hao-Chuan Wang, Naomi Yamashita |
Proc. ACM Hum. Comput. Interact. | 3 |
| 2022 | A Speech-Based Model for Tracking the Progression of Activities in Extreme Action TeamworkabstractDesigning computerized approaches to support complex teamwork requires an understanding of how activity-related information is relayed among team members. In this paper, we focus on verbal communication and describe a speech-based model that we developed for tracking activity progression during time-critical teamwork. We situated our study in the emergency medical domain of trauma resuscitation and transcribed speech from 104 audio recordings of actual resuscitations. Using the transcripts, we first studied the nature of speech during 34 clinically relevant activities. From this analysis, we identified 11 communicative events across three different stages of activity performance-before, during, and after. For each activity, we created sequential ordering of the communicative events using the concept of narrative schemas. The final speech-based model emerged by extracting and aggregating generalized aspects of the 34 schemas. We evaluated the model performance by using 17 new transcripts and found that the model reliably recognized an activity stage in 98% of activity-related conversation instances. We conclude by discussing these results, their implications for designing computerized approaches that support complex teamwork, and their generalizability to other safety-critical domains. Swathi Jagannath, Neha Kamireddi, Katherine Ann Zellner, Randall S. Burd, Ivan Marsic, Aleksandra Sarcevic |
Proc. ACM Hum. Comput. Interact. | 6 |
| 2022 | PACMHCI V6, CSCW2, November 2022 EditorialabstractWe are delighted to present this issue of the Proceedings of the ACM on Human-Computer Interaction, which contains scholarship from the Computer-Supported Cooperative Work and Social Computing (CSCW) community. This issue has 293 papers, 94 that were accepted from the April 2021 cycle, 61 that were accepted from the July 2021 cycle, and 138 that were accepted from the January 2022 cycle. It reflects great efforts and contributions from external reviewers, Associate Chairs and Editors, who together have conducted a rigorous review process. As Papers Chairs, we are grateful for the community's collective efforts to continue shaping and sharing CSCW's tradition of high-quality scholarship during an ongoing global pandemic. Hideaki Kuzuoka, Katharina Reinecke, Hao-Chuan Wang, Naomi Yamashita, Shaowen Bardzell, Siân E. Lindley, Aleksandra Sarcevic |
Proc. ACM Hum. Comput. Interact. | 7 |
| 2022 | Alerts as Coordination Mechanisms: Implications for Designing Alerts for Multidisciplinary and Shared Decision MakingabstractIn this study, we explore how clinical decision support features can be designed to aid teams in caring for patients during time-critical medical emergencies. We interviewed 12 clinicians with experience in leading pediatric trauma resuscitations to elicit design requirements for decision support alerts and how these alerts should be designed for teams with shared leadership. Based on the interview data, we identified three types of decision support alerts: reminders to perform tasks, alerts to changes in patient status, and suggestions for interventions. We also found that clinicians perceived alerts in this setting as coordination mechanisms and that some alert preferences were associated with leader experience levels. From these findings, we contribute three perspectives on how alerts can aid coordination and discuss implications for designing decision support alerts for shared leadership in time-critical medical processes. Angela Mastrianni, Lynn Almengor, Aleksandra Sarcevic |
Proc. ACM Hum. Comput. Interact. | 3 |
| 2021 | Designing Interactive Alerts to Improve Recognition of Critical Events in Medical EmergenciesabstractVital sign values during medical emergencies can help clinicians recognize and treat patients with life-threatening injuries. Identifying abnormal vital signs, however, is frequently delayed and the values may not be documented at all. In this mixed-methods study, we designed and evaluated a two-phased visual alert approach for a digital checklist in trauma resuscitation that informs users about undocumented vital signs. Using an interrupted time series analysis, we compared documentation in the periods before (two years) and after (four months) the introduction of the alerts. We found that introducing alerts led to an increase in documentation throughout the post-intervention period, with clinicians documenting vital signs earlier. Interviews with users and video review of cases showed that alerts were ineffective when clinicians engaged less with the checklist or set the checklist down to perform another activity. From these findings, we discuss approaches to designing alerts for dynamic team-based settings. Angela Mastrianni, Aleksandra Sarcevic, Lauren Chung, Issa Zakeri, Emily Alberto, Zachary P. Milestone, Ivan Marsic, Randall S. Burd |
Conference on Designing Interactive Systems | 2 |
| 2021 | User Needs and Challenges in Information Sharing between Pre-Hospital and Hospital Emergency Care Providers
Zhan Zhang 0008, Aleksandra Sarcevic, Karen Joy, Mustafa Ozkaynak, Kathleen Adelgais |
AMIA | 2 |
| 2021 | PACMHCI V5 CSCW1 April 2021 EditorialabstractWe are delighted to present this issue of the Proceedings of the ACM on Human-Computer Interaction, which contains scholarship from the Computer-Supported Cooperative Work and Social Computing (CSCW) community. This issue has 190 papers, 177 submitted in June 2020 and 13 submitted in October 2020. It represents contributions from two Program Committees, including external reviewers, Associate Chairs, and Editors, who together have conducted a rigorous review process. As Papers Chairs, we are grateful for the community's collective efforts to continue shaping and sharing CSCW's tradition of high-quality scholarship during a global pandemic. Shaowen Bardzell, Juho Kim 0001, Siân E. Lindley, Aleksandra Sarcevic, Sarita Yardi Schoenebeck |
Proc. ACM Hum. Comput. Interact. | 4 |
| 2021 | PACMHCI V5 CSCW2 October 2021 Editorial
Shaowen Bardzell, Siân E. Lindley, Aleksandra Sarcevic |
Proc. ACM Hum. Comput. Interact. | 3 |
| 2021 | Towards Dynamic Checklists: Understanding Contexts of Use and Deriving Requirements for Context-Driven AdaptationabstractThe goal of this in-the-wild study was to understand how different patient, provider, and environment contexts affected the use of a tablet-based checklist in a dynamic medical setting. Fifteen team leaders used the digital checklist in 187 actual trauma resuscitations. The measures of checklist interactions included the number of unchecked items and the number of notes written on the checklist. Of the 10 contexts we studied, team leaders’ arrival after the patient and patients with penetrating injuries were both associated with more unchecked items. We also found that the care of patients with external injuries contributed to more notes written on the checklist. Finally, our results showed that more experienced leaders took significantly more notes overall and more numerical notes than less experienced leaders. We conclude by discussing design implications and steps that can be achieved with context-aware computing towards adaptive checklists that meet the needs of dynamic use contexts. Leah Kulp, Aleksandra Sarcevic, Megan Cheng, Randall S. Burd |
ACM Trans. Comput. Hum. Interact. | 2 |
| 2020 | Checklist Design Reconsidered: Understanding Checklist Compliance and Timing of InteractionsabstractWe examine the association between user interactions with a checklist and task performance in a time-critical medical setting. By comparing 98 logs from a digital checklist for trauma resuscitation with activity logs generated by video review, we identified three non-compliant checklist use behaviors: failure to check items for completed tasks, falsely checking items when tasks were not performed, and inaccurately checking items for incomplete tasks. Using video review, we found that user perceptions of task completion were often misaligned with clinical practices that guided activity coding, thereby contributing to non-compliant check-offs. Our analysis of associations between different contexts and the timing of check-offs showed longer delays when (1) checklist users were absent during patient arrival, (2) patients had penetrating injuries, and (3) resuscitations were assigned to the highest acuity. We discuss opportunities for reconsidering checklist designs to reduce non-compliant checklist use. Leah Kulp, Aleksandra Sarcevic, Yinan Zheng, Megan Cheng, Emily Alberto, Randall S. Burd |
CHI | 2 |
| 2020 | Envisioning a Shared Checklist Display to Support Teamwork During Emergency Medical CareabstractInformation technology and systems in healthcare are becoming increasingly complex and dynamic, requiring design approaches that consider not only providers’ information needs and work practices, but also their perceptions and expectations about technology. In this paper, we describe an exploratory study conducted to assess the feasibility and impact of a shared checklist display for supporting teamwork during trauma resuscitations. Using the concept of technological frames, we explore how members of multidisciplinary emergency medical teams perceive current technology and what they expect from a future shared checklist display. Our results showed discrepancies in team members’ perceptions of technology and its roles. Some care providers envisioned a shared checklist display as a tool for achieving a global view of the process, improving communication and maintaining situation awareness. Others perceived it as a point of distraction, where team members may be checking each other’s work to ensure accuracy of documentation. These requirements and limitations extend beyond the capacity of a simple checklist, while also shifting the privacy of work from individuals to the entire team. We conclude by discussing design implications for a future shared display and how shifting to a multi-display environment within a safety-critical context may transform work practices. Leah Kulp, Aleksandra Sarcevic |
ECSCW | 2 |
| 2019 | Temporal Rhythms and Patterns of Electronic Documentation in Time-Critical Medical WorkabstractWe examine nursing documentation on a newly implemented electronic flowsheet in medical resuscitations to identify the temporal patterns of documentation and how the recorded information supported time-critical teamwork. To determine when the information was documented, we compared timestamps from 58 flowsheet logs to those of verbal communications derived from video review. We also drew on observations of 95 resuscitations to understand the behaviors of nurse documenters. We found that only 8% of the verbal reports were documented in near real-time (one minute within the verbal report), while 42% of reports were not documented in the electronic flowsheet. In addition, 38% were documented early (before the verbal report) and 12% were documented with a delay, ranging from one to 58 minutes after the report. Our study showed that the electronic flowsheet design posed many challenges for real-time documentation, leading to paper-based workarounds and the use of free-text fields on the flowsheet to visualize and keep track of time, and to communicate temporal information to the team. These findings suggest that documenters shape the temporal rhythms of not only their own work but also the rhythms of the electronic record and medical process. We discuss the implications of these rhythms for EHR redesign to support real-time documentation in high-risk, safety-critical settings. Swathi Jagannath, Aleksandra Sarcevic, Victoria Young, Sage Myers |
CHI | 2 |
| 2019 | Comparing the Effects of Paper and Digital Checklists on Team Performance in Time-Critical WorkabstractThis mixed-methods study examines the effects of a tablet-based checklist system on team performance during a dynamic and safety-critical process of trauma resuscitation. We compared team performance from 47 resuscitations that used a paper checklist to that from 47 cases with a digital checklist to determine if digitizing a checklist led to improvements in task completion rates and in how fast the tasks were initiated for 18 most critical assessment and treatment tasks. We also compared if the checklist compliance increased with the digital design. We found that using the digital checklist led to more frequent completions of the initial airway assessment task but fewer completions of ear and lower extremities exams. We did not observe any significant differences in time to task performance, but found increased compliance with the checklist. Although improvements in team performance with the digital checklist were minor, our findings are important because they showed no adverse effects as a result of the digital checklist introduction. We conclude by discussing the takeaways and implications of these results for effective digitization of medical work. Leah Kulp, Aleksandra Sarcevic, Megan Cheng, Yinan Zheng, Randall S. Burd |
CHI | 2 |
| 2018 | An approach to automatic process deviation detection in a time-critical clinical process
Sen Yang 0002, Aleksandra Sarcevic, Richard A. Farneth, Shuhong Chen, Omar Z. Ahmed, Ivan Marsic, Randall S. Burd |
J. Biomed. Informatics | 2 |
| 2018 | Coordination Mechanisms for Self-Organized Work in an Emergency Communication CenterabstractWe describe an observational study of work coordination in an emergency communication center, where a collocated team of communication specialists engages in complex activities of communicating with pre-hospital medical teams, and coordinating patient care and transport. Unlike teams with clearly defined work roles and team structures that were introduced to increase work efficiency and minimize redundancy, the team we studied lacks the role differentiation. To better understand how complex work is accomplished under these conditions, we conducted in-situ observations in the center's control room and interviewed communication specialists. We found that communication specialists self-organized by using a mix of material and immaterial coordination mechanisms, including work schedules, computer systems, and tacit agreements to coordinate tasks. Using these findings, we then identified three features of self-organized, collocated and time-critical teamwork that require technology support: awareness of task ownership, task self-assignment, and informal team hierarchy. We conclude by discussing technology requirements to support these teamwork features. Zhan Zhang 0008, Aleksandra Sarcevic |
Proc. ACM Hum. Comput. Interact. | 2 |
| 2017 | Exploring Design Opportunities for a Context-Adaptive Medical Checklist Through Technology Probe ApproachabstractThis paper explores the workflow and use of an interactive medical checklist for trauma resuscitation-an emerging technology developed for trauma team leaders to support decision making and task coordination among team members. We used a technology probe approach and ethnographic methods, including video review, interviews, and content analysis of checklist logs, to examine how team leaders use the checklist probe during live resuscitations. We found that team leaders of various experience levels use the technology differently. Some leaders frequently glance at the checklist and take notes during task performance, while others place the checklist on a stand and only interact with the checklist when checking items. We compared checklist timestamps to task activities and found that most items are checked off after tasks are performed. We conclude by discussing design implications and new design opportunities for a future dynamic, adaptive checklist. Leah Kulp, Aleksandra Sarcevic, Richard A. Farneth, Omar Z. Ahmed, Dung Mai, Ivan Marsic, Randall S. Burd |
Conference on Designing Interactive Systems | 2 |
| 2017 | Constructing Common Information Spaces across Distributed Emergency Medical TeamsabstractThis paper investigates coordination and real-time information sharing across four emergency medical teams in a high-risk and distributed setting as they provide care to critically injured patients within the first hour after injury. Through multiple field studies we explored how common understanding of critical patient data is established across these heterogeneous teams and what coordination mechanisms are being used to support information sharing and interpretation. To interpret the data, we drew on the concept of Common Information Spaces (CIS). Our results showed that teams faced many challenges in achieving efficient information sharing and coordination, including difficulties in locating and assembling team members, communicating and interpreting information from the field, and reconciling differences in team perspectives and information needs, all while having minimal technology support. We reflect on these challenges to suggest an extension of the classic CSCW time-space matrix, as well as future development of CIS as an analytical framework. The paper concludes with design opportunities for supporting highly distributed and heterogeneous teamwork in time-critical work environments. Zhan Zhang 0008, Aleksandra Sarcevic, Claus Bossen |
CSCW | 2 |
| 2017 | On the Use of Electronic Documentation Systems in Fast-Paced, Time-Critical Medical SettingsabstractThis study examines the use of an electronic flowsheet developed for capturing patient and process information during trauma resuscitation—a high-risk, fast-paced process of treating severely injured patients. Unlike medical records in most hospital units, the resuscitation record necessitates real-time completion because of the ephemeral nature of information and its contemporaneous use for patient care. We draw on in situ observations of 15 resuscitation events to understand how scribes interact with the system in such an environment. Our results showed frequent tab switching and problems navigating the interface, resulting in delayed data capture. We attribute these inefficiencies to the current layout of the electronic flowsheet, where information grouping follows logical rather than context-driven structure. We conclude by discussing design opportunities for future systems to improve both documentation and teamwork processes in fast-paced, time-critical medical settings. Aleksandra Sarcevic, Nicole Ferraro |
Interact. Comput. | 1 |
| 2016 | Checklist as a Memory Externalization Tool during a Critical Care Process
Aleksandra Sarcevic, Zhan Zhang 0008, Ivan Marsic, Randall S. Burd, Leah Kulp |
AMIA | 1 |
| 2016 | Deep Learning for RFID-Based Activity RecognitionabstractWe present a system for activity recognition from passive RFID data using a deep convolutional neural network. We directly feed the RFID data into a deep convolutional neural network for activity recognition instead of selecting features and using a cascade structure that first detects object use from RFID data followed by predicting the activity. Because our system treats activity recognition as a multi-class classification problem, it is scalable for applications with large number of activity classes. We tested our system using RFID data collected in a trauma room, including 14 hours of RFID data from 16 actual trauma resuscitations. Our system outperformed existing systems developed for activity recognition and achieved similar performance with process-phase detection as systems that require wearable sensors or manually-generated input. We also analyzed the strengths and limitations of our current deep learning architecture for activity recognition from RFID data. Xinyu Li 0003, Yanyi Zhang, Ivan Marsic, Aleksandra Sarcevic, Randall S. Burd |
SenSys | 4 |
| 2016 | Passive RFID for Object and Use Detection during Trauma ResuscitationabstractWe evaluated passive radio-frequency identification (RFID) technology for detecting the use of objects and related activities during trauma resuscitation. Our system consists of RFID tags and antennas, optimally placed for object detection, as well as algorithms for processing RFID data to infer object use. To evaluate our approach, we tagged 81 objects in the resuscitation room and recorded RFID signal strength during 32 simulated resuscitations performed by trauma teams. We then analyzed RFID data to identify cues for recognizing resuscitation activities. Using these cues, we extracted descriptive features and applied machine-learning techniques to monitor interactions with objects. Our results show that an instance of a used object can be detected with accuracy rates greater than 90 percent in a crowded and fast-paced medical setting using off-the-shelf RFID equipment, and the time and duration of use can be identified with up to 83 percent accuracy. We conclude with insights into the limitations of passive RFID and areas in which RFID needs to be complemented with other sensing technologies. Siddika Parlak, Ivan Marsic, Aleksandra Sarcevic, Waheed U. Bajwa, Lauren J. Waterhouse, Randall S. Burd |
IEEE Trans. Mob. Comput. | 3 |
| 2015 | Designing for Temporal Awareness: The Role of Temporality in Time-Critical Medical TeamworkabstractThis paper describes the role of temporal information in emergency medical teamwork and how time-based features can be designed to support the temporal awareness of clinicians in this fast-paced and dynamic environment. Engagement in iterative design activities with clinicians over the course of two years revealed a strong need for time-based features and mechanisms, including timestamps for tasks based on absolute time and automatic stopclocks measuring time by counting up since task performance. We describe in detail the aspects of temporal awareness central to clinicians' awareness needs and then provide examples of how we addressed these needs through the design of a shared information display. As an outcome of this process, we define four types of time representation techniques to facilitate the design of time-based features: (1) timestamps based on absolute time, (2) timestamps relative to the process start time, (3) time since task performance, and (4) time until the next required task. Diana S. Kusunoki, Aleksandra Sarcevic |
CSCW | 2 |
| 2015 | Constructing Awareness Through Speech, Gesture, Gaze and Movement During a Time-Critical Medical Task
Zhan Zhang 0008, Aleksandra Sarcevic |
ECSCW | 2 |
| 2015 | Sketching Awareness: A Participatory Study to Elicit Designs for Supporting Ad Hoc Emergency Medical Teamwork
Diana S. Kusunoki, Aleksandra Sarcevic, Zhan Zhang 0008, Maria Yala |
Comput. Support. Cooperative Work. | 2 |
| 2014 | Going Digital: Transforming Medical Checklists for Improved Patient Care
Bradford Winters, Randall S. Burd, Jesse Cirimele, Leslie Wu, Aleksandra Sarcevic |
AMIA | 5 |
| 2014 | Balancing design tensions: iterative display design to support ad hoc and multidisciplinary medical teamworkabstractIn this paper, we describe how we developed an information display prototype for trauma resuscitation teams based on design ideas and feedback from clinicians. Our approach is grounded in participatory design, emphasizing the importance of gaining long-term commitment from clinicians in system development. Through a series of participatory design workshops, heuristic evaluation, and simulated resuscitation sessions, we identified the main information features to include on our display. Our results focus on how we balanced the design tensions that emerged when addressing the ad hoc, hierarchical, and multidisciplinary nature of trauma teamwork. We discuss the implications of balancing role-based differences for each information feature, as well as two major design tensions: process-based vs. state-based designs and role-based vs. team-based displays. Diana S. Kusunoki, Aleksandra Sarcevic, Nadir Weibel, Ivan Marsic, Zhan Zhang 0008, Genevieve Tuveson, Randall S. Burd |
CHI | 2 |
| 2014 | Informing Digital Cognitive Aids Design for Emergency Medical Work by Understanding Paper Checklist UseabstractWe examine the use of a paper-based checklist during 48 simulated trauma resuscitations to inform the design of digital cognitive aids for safety-critical medical teamwork. Our analysis focused on team communication and interaction behaviors as physician leaders led resuscitations and administered the checklist. We found that the checklist increased the amount of communication between the leader and the team, but did not compromise the leader's interactions with the environment. In addition, we observed several changes in team dynamics: the checklist facilitated collaborative decision making and process reflections, but it also made some team members reactive rather than proactive. As the push toward digitizing medical work continues, we expect that paper checklists will soon be replaced by their digital counterparts. Designing interactive cognitive aids for medical domains, however, poses many challenges. Our results offer directions for how these tools could be designed to support medical work in increasingly digital environments. Zhan Zhang 0008, Aleksandra Sarcevic, Maria Yala, Randall S. Burd |
GROUP | 2 |
| 2013 | Supporting Information Use and Retention of Pre-Hospital Information during Trauma Resuscitation: A Qualitative Study of Pre-Hospital Communications and Information Needs
Zhan Zhang 0008, Aleksandra Sarcevic, Randall S. Burd |
AMIA | 2 |
| 2013 | Understanding visual attention of teams in dynamic medical settings through vital signs monitor useabstractThe purpose of this study was to understand how vital signs monitors support teamwork during trauma resuscitation -- the fast-paced and information-rich process of stabilizing critically injured patients. We analyzed 12 videos of simulated resuscitations to characterize trauma team monitor use. To structure our observations, we adopted the feedback loop concept. Our results showed that the monitor was used frequently, especially by team leaders and anesthesiologists. We identified three patterns of monitor use: (i) periods with a low frequency of short looks (glances) to maintain overall process awareness; (ii) periods with a medium frequency of long looks (scrutiny) to monitor trends in patient status; and (iii) peaks with a high frequency of glances to maintain attention on both the patient and monitor during critical tasks. Approximately 75% of looks were 3 seconds or shorter, but many looks (25%) ranged between 3 and 26 seconds. Our results have implications for improving displays by presenting the status of the patient's physiological systems and team activities. Diana S. Kusunoki, Aleksandra Sarcevic, Zhan Zhang 0008, Randall S. Burd |
CSCW | 2 |
| 2012 | "Beacons of hope" in decentralized coordination: learning from on-the-ground medical twitterers during the 2010 Haiti earthquakeabstractWe examine the public, social media communications of 110 emergency medical response teams and organizations in the immediate aftermath of the January 12, 2010 Haiti earthquake. We found the teams through an inductive analysis of Twitter communications acquired over the three-week emergency period from 89,114 Twitterers. We then analyzed the teams' Twitter streams, as well as all digital media they generated and pointed to in their streams - blog posts, photographs, videos, status updates and field reports - to understand the medical coordination challenges they faced from pre-deployment readiness to on-the-ground action. Here we identify opportunities for improving coordination in a decentralized and distributed environment where staffing, disease trajectories, and other circumstances rapidly change. We extrapolate from these findings to theorize about how "beaconing" behavior is a sign of latent potential for coordination upon which mechanisms of coordination can capitalize. Aleksandra Sarcevic, Leysia Palen, Joanne I. White, Kate Starbird, Mossaab Bagdouri, Kenneth M. Anderson |
CSCW | 1 |
| 2012 | Decision making tasks in time-critical medical settingsabstractWe examine decision-making tasks and information sources during fast-paced, high-risk medical events, such as trauma resuscitation. Interviews with surgical team leaders and ED physicians reveal several environmental aspects that make decision making difficult, including diagnostic tradeoffs, missing and unreliable information, and managing multiple patients simultaneously. We discuss the implications of these findings for the design of wall displays to support decision making in time-critical medical settings. Aleksandra Sarcevic, Zhan Zhang 0008, Diana S. Kusunoki |
GROUP | 1 |
| 2012 | Introducing RFID technology in dynamic and time-critical medical settings: Requirements and challenges
Siddika Parlak, Aleksandra Sarcevic, Ivan Marsic, Randall S. Burd |
J. Biomed. Informatics | 2 |
| 2012 | Teamwork Errors in Trauma ResuscitationabstractHuman errors in trauma resuscitation can have cascading effects leading to poor patient outcomes. To determine the nature of teamwork errors, we conducted an observational study in a trauma center over a two-year period. While eventually successful in treating the patients, trauma teams had problems tracking and integrating information in a longitudinal trajectory, which resulted in inefficiencies and near-miss errors. As an initial step in system design to support trauma teams, we proposed a model of teamwork and a novel classification of team errors. Four types of team errors emerged from our analysis: communication errors, vigilance errors, interpretation errors, and management errors. Based on these findings, we identified key information structures to support team cognition and decision making. We believe that displaying these information structures will support distributed cognition of trauma teams. Our findings have broader applicability to other collaborative and dynamic work settings that are prone to human error. Aleksandra Sarcevic, Ivan Marsic, Randall S. Burd |
ACM Trans. Comput. Hum. Interact. | 1 |
| 2011 | Coordinating time-critical work with role-taggingabstractA Level-1 US trauma center introduced role-tags in their trauma resuscitation rooms to help team members identify respective medical functions, and to limit the number of people in the rooms to required staff only. We use this in situ experiment with a paper prototype to investigate the role-driven nature of coordination and to identify system requirements for computerized support of role-based coordination in time-critical work. While role information is useful in coordinating time-critical work, our findings show that the current low-tech solution did not provide significant improvement in team coordination. The situations that were most in need of role-identification were the least likely to achieve it because role-tags required work by trauma team members. Similarly, because role-tags allowed workarounds and misuse, they proved ineffective in controlling the number of people in the room. We suggest technological ways of identifying roles to help coordination in the trauma bay. Aleksandra Sarcevic, Leysia Palen, Randall S. Burd |
CSCW | 1 |
| 2010 | "Who's scribing?": documenting patient encounter during trauma resuscitationabstractWith healthcare moving towards electronic health records, it is important to understand existing work practices to design effective systems. We conducted an observational study in a Level I trauma center to examine the documentation process and the role of the nurse recorder in trauma resuscitation. We identified several difficulties with current recording practices, including the late arrival of the nurse recorder, parallel activities of the trauma team, and multitasking by the recorder. Our observations showed that the recorder's role extends beyond archival responsibilities. The recorder, with the help of a paper record, manages the resuscitation process, rather than passively documenting it. Our findings highlighted the complexity of the recorder's role and the need to consider documentation in the broader context of trauma teamwork. We proposed a set of design challenges that emphasize important aspects of trauma care to be considered when designing technologies to support the documentation process. Aleksandra Sarcevic |
CHI | 1 |
| 2009 | Information handover in time-critical workabstractInformation transfer under time pressure and stress often leads to information loss. This paper studies the characteristics and problems of information handover from the emergency medical services (EMS) crew to the trauma team when a critically injured patient arrives to the trauma bay. We consider the characteristics of the handover process and the subsequent use of transferred information. Our goal is to support the design of technology for information transfer by identifying specific challenges faced by EMS crews and trauma teams during handover. Data were drawn from observation and video recording of 18 trauma resuscitations. The study shows how EMS crews report information from the field and the types of information that they include in their reports. Particular problems occur when reports lack structure, continuity, and complete descriptions of treatments given en route. We also found that trauma team members have problems retaining reported information. They pay attention to the items needed for immediately treating the patient and inquire about other items when needed during the resuscitation. The paper identifies a set of design challenges that arise during information transfer under time pressure and stress, and discusses characteristics of potential technological solutions. Aleksandra Sarcevic, Randall S. Burd |
GROUP | 1 |
| 2008 | "What's the Story?" Information Needs of Trauma Teams
Aleksandra Sarcevic, Randall S. Burd |
AMIA | 1 |
| 2008 | Transactive memory in trauma resuscitationabstractThis paper describes an ethnographic study conducted to explore the possibilities for future design and development of technological support for trauma teams. We videotaped 10 trauma resuscitations and transcribed each event. Using a framework that we developed, we coded each transcript to allow qualitative and quantitative analysis of the trauma teams' collaborative processes. We analyzed teams' tasks, interactions, and communication patterns that support information acquisition and sharing. Our results showed the importance of team transactive memory, but also pointed to inefficiencies in communication processes, which enable the functioning of this collective memory system. Based on quantitative and qualitative observations of trauma teamwork, we present opportunities for technological solutions that may reduce the cognitive effort needed for maintaining the working memory of trauma teams. Aleksandra Sarcevic, Ivan Marsic, Michael E. Lesk, Randall S. Burd |
CSCW | 1 |
| 2004 | "Who's in charge here?" communicating across unequal computer platformsabstractPeople use personal data assistants in the field to collect data and to communicate with others both in the field and office. The individual in the office invariably has a laptop or a high-end personal workstation and thus, significantly more computing power, more screen real estate, and higher volume input devices, such as a mouse and keyboard. These differences give the high-end user the ability to represent and manipulate collaborative tasks more effectively. It is therefore useful to know what impact these differences have on work performance and work communications. Four different platform combinations involving a PC and a PDA were used to examine the effect of communicating via heterogeneous computer platforms. The PC platform used a mouse, a keyboard, and a 3-dimensional screen display. The PDA platform used a stylus, soft buttons, and a 2-dimensional screen display. A variation of the Tetris wall-building game called Slow Tetris was used as the subjects' collaborative task. A second factor in the experiment was role asymmetry. One subject was arbitrarily put in charge of the task solution in all of the combinations. An analysis of the solution times found that subjects with mixed platforms worked slower than their homogeneous counterparts, that is, a person in charge with a PC worked faster if his partner had a PC. An in-depth analysis of the communication patterns found significant differences in the exchanges between heterogeneous and homogenous combinations. The PC-to-PDA combination (with the person on the PC in charge of the solution) took significantly more time than the PC-to-PC combination. This extra time appears to come from the disadvantage of having a partner on the PDA who is unable to help in solving the problems. The PDA-to-PC combination took approximately the same amount of time as the PDA-to-PDA combination despite having one team member with a better representation. This member was, unfortunately, not in charge of the solution. The PDA-to-PC heterogeneous combination exhibited more direction giving, less one-sided collaboration, and more takeover attempts than any of the other combinations. Overall, roles were maintained in the partnerships except for the person with the PDA directing the person with the PC. Maria C. Velez, Marilyn Tremaine, Aleksandra Sarcevic, Bogdan Dorohonceanu, Allan Meng Krebs, Ivan Marsic |
ACM Trans. Comput. Hum. Interact. | 3 |