VLDB 2026 Research / reviewers in the wild / expert
Randall S. Burd
dblp:86/1444
· DBLP profile ↗
41ranked-venue papers
0as first author
18since 2021 · last 2026
0000-0003-4465-9117ORCID · verified
Domains — the database's venue-derived domains; a paper can count in several
Human-computer interaction and ubiquitous computing · 19 · 9 since 2021Applied, interdisciplinary, general and emerging computing · 12 · 6 since 2021Computer networks · 4Artificial intelligence and machine learning · 3 · 2 since 2021Databases, data management, data science and information retrieval · 2 · 1 since 2021Graphics, computer vision, multimedia, augmented reality and games · 2
| Year | Publication | Venue | Position |
|---|---|---|---|
| 2026 | Multi-view real-time detection of patient arrival in trauma resuscitation using computer visionabstractEvery minute delay in life-saving intervention increases mortality risk in injured patients. Given this relationship, quality measurement of trauma resuscitation includes the timing of provider decisions and interventions. The current approach of manual recording of patient arrival and other timestamps may be inaccurate due to providers underestimating elapsed time during resuscitation. We introduce a frame-based computer vision system to automatically detect and classify two specific phases of patient arrival: (1) when the patient enters the trauma resuscitation room, and (2) when the patient is moved onto the bed. The proposed system consists of two stages. The first stage uses a pattern-based method to detect when a patient enters the room. The second stage uses both side-view and top-view video feeds to determine when the patient has been moved to the bed, addressing occlusion issues and enhancing robustness and precision. To minimize labeling effort and accelerate detection, we introduce PA-YOLO, a frame-based classification model, in both stages of the system. We evaluated our system using 5-fold cross-validation on 60 trauma resuscitation cases. Our results show that this approach achieves an accuracy of 0.92 ± 0.01 for patient-at-door detection and 0.93 ± 0.03 for patient-on-bed detection, with an average detection delay of 2.15 ± 1.24 s. The proposed system outperforms SlowFast by up to 5 percentage points and our previous I3D-based system by up to 7 percentage points in detection accuracy, while reducing detection delay from over 5 s to about 2 s. Compared to baseline model YOLO11, our PA-YOLO improves detection performance by 2.0% while reducing floating point operations per second (FLOPS) by 51.5%. • Use of computer vision methods to detect different phases of patient arrival. • Modification of the YOLO model for fast scene classification and reconstruction of activities from the video stream. • Mitigation of occlusion problems by switching between different views based on occlusion detection. • Evaluation of the effectiveness of the system in actual resuscitations and comparison with existing methods. Sifan Yuan, Mary S. Kim, Aaron H. Mun, Rebecca Cunningham, Ivan Marsic, Randall S. Burd |
Comput. Vis. Image Underst. | 6 |
| 2026 | SAFE: A Smart Adherence Detection Framework for Monitoring Personal Protective Equipment in Healthcare SettingsabstractPersonal protective equipment (PPE) is critical for infection control in healthcare, protecting workers and patients from infection risks. The COVID-19 pandemic further highlighted the importance of correct PPE use, yet adherence to U.S. Centers for Disease Control and Prevention guidelines remains inconsistent. Continuous human monitoring of PPE adherence is impractical because it is labor-intensive and may expose observers to infection risk. Automated monitoring is a promising alternative, but reliable PPE assessment in clinical videos remains difficult due to occlusion and subtle differences between adherence levels. To address these challenges, we propose SAFE - Smart Adherence detection Framework for PPE, a cascaded computer vision system for real-time monitoring of PPE wearing status, including complete, incomplete, and absent cases, with a focus on gowns and masks. SAFE uses a two-stage design: Stage 1 detects gown status and localizes head regions, and Stage 2 classifies mask status from head crops. We evaluate SAFE on R2PPE, a ceiling-view trauma-room simulation dataset with dense PPE annotations and complex scenes. SAFE improves overall average precision from 0.48 to 0.67 and increases mask-class average precision by 0.33 compared to a baseline one-stage detector. We further validate SAFE across modern detector backbones, including transformer-based detectors, and on real-case trauma-room data using class-level and alarm-level criteria, improving class-level mask accuracy from 0.59 to 0.65 while maintaining a high alarm-level recall of 0.98. SAFE could enhance PPE monitoring with minimal human intervention, providing a scalable solution for improving infection control in healthcare settings. Wanzhao Yang, Beomseok Park, Mary S. Kim, Aleksandra Sarcevic, Syed Muhammad Anwar, Marius G. Linguraru, Ivan Marsic, Randall S. Burd |
IEEE J. Biomed. Health Informatics | 8 |
| 2025 | Understanding Personal Protective Equipment Use in Interdisciplinary Medical Settings: Design Explorations for Just-in-Time Compliance Alerts: Improving PPE Practices in Medical Settings Through Alert DesignabstractWe examine the use of personal protective equipment (PPE) in two interdisciplinary medical settings to inform the design of just-in-time alerts and reminders for correcting PPE noncompliance. We reviewed videos of 26 pediatric resuscitations occurring over the course of the COVID-19 pandemic at an urban pediatric teaching hospital. Through video review, we identified causes for PPE noncompliance, activities that were frequently performed without PPE, instances in which PPE was intentionally removed, and mechanisms by which healthcare providers corrected PPE noncompliance. We also interviewed 18 registered nurses working in the hospital's emergency department (ED) and intensive care unit (ICU) to better understand observed PPE behaviors and practices. Our results suggest that alert design will require considering the urgency of correcting PPE noncompliance against the urgency of tasks being performed. We discuss our findings through the lens of the COM-B framework and conclude by exploring design opportunities for just-in-time alerts and reminders for prompting PPE noncompliance corrections in dynamic medical work. Aleksandra Sarcevic, Eleanor Wood, Katherine Ann Zellner, Christine Dodeye Ikponmwonba, Mary S. Kim, Ivan Marsic, Randall S. Burd |
Conference on Designing Interactive Systems | 7 |
| 2025 | ASELMAR: Active and semi-supervised learning-based framework to reduce multi-labeling efforts for activity recognitionabstractManual annotation of unlabeled data for model training is expensive and time-consuming, especially for visual datasets requiring domain-specific experience for multi-labeling, such as video records generated in hospital settings. There is a need to build frameworks to reduce human labeling efforts while improving training performance. Semi-supervised learning is widely used to generate predictions for unlabeled samples in a partially labeled datasets. Active learning can be used with semi-supervised learning to annotate unlabeled samples to reduce the sampling bias due to the label predictions. We developed the aselmar framework based on active and semi-supervised learning techniques to reduce the time and effort associated with multi-labeling of unlabeled samples for activity recognition. aselmar (i) categorizes the predictions for unlabeled data based on the confidence level in predictions using fixed and adaptive threshold settings, (ii) applies a label verification procedure for the samples with the ambiguous prediction, and (iii) retrains the model iteratively using samples with their high-confidence predictions or manual annotations. We also designed a software tool to guide domain experts in verifying ambiguous predictions. We applied aselmar to recognize eight selected activities from our trauma resuscitation video dataset and evaluated their performance based on the label verification time and the mean ap score metric. The label verification required by aselmar was 12.1% of the manual annotation effort for the unlabeled video records. The improvement in the mean ap score was 5.7% for the first iteration and 8.3% for the second iteration with the fixed threshold-based method compared to the baseline model . The p-values were below 0.05 for the target activities. Using an adaptive-threshold method, aselmar achieved a decrease in ap score deviation, implying an improvement in model robustness. For a speech-based case study , the word error rate decreased by 6.2%, and the average transcription factor increased 2.6 times, supporting the broad applicability of ASELMAR in reducing labeling efforts from domain experts. Aydin Saribudak, Sifan Yuan, Chenyang Gao, Waverly Gestrich-Thompson, Zachary P. Milestone, Randall S. Burd, Ivan Marsic |
Comput. Vis. Image Underst. | 6 |
| 2025 | Comparative analysis of personal protective equipment nonadherence detection: computer vision versus human observersabstractOBJECTIVES: Human monitoring of personal protective equipment (PPE) adherence among healthcare providers has several limitations, including the need for additional personnel during staff shortages and decreased vigilance during prolonged tasks. To address these challenges, we developed an automated computer vision system for monitoring PPE adherence in healthcare settings. We assessed the system performance against human observers detecting nonadherence in a video surveillance experiment. MATERIALS AND METHODS: The automated system was trained to detect 15 classes of eyewear, masks, gloves, and gowns using an object detector and tracker. To assess how the system performs compared to human observers in detecting nonadherence, we designed a video surveillance experiment under 2 conditions: variations in video durations (20, 40, and 60 seconds) and the number of individuals in the videos (3 versus 6). Twelve nurses participated as human observers. Performance was assessed based on the number of detections of nonadherence. RESULTS: Human observers detected fewer instances of nonadherence than the system (parameter estimate -0.3, 95% CI -0.4 to -0.2, P < .001). Human observers detected more nonadherence during longer video durations (parameter estimate 0.7, 95% CI 0.4-1.0, P < .001). The system achieved a sensitivity of 0.86, specificity of 1, and Matthew's correlation coefficient of 0.82 for detecting PPE nonadherence. DISCUSSION: An automated system simultaneously tracks multiple objects and individuals. The system performance is also independent of observation duration, an improvement over human monitoring. CONCLUSION: The automated system presents a potential solution for scalable monitoring of hospital-wide infection control practices and improving PPE usage in healthcare settings. Mary S. Kim, Beomseok Park, Genevieve J. Sippel, Aaron H. Mun, Wanzhao Yang, Kathleen H. McCarthy, Emely Fernandez, Marius George Linguraru, Aleksandra Sarcevic, Ivan Marsic, Randall S. Burd |
J. Am. Medical Informatics Assoc. | 11 |
| 2025 | Human intention recognition for trauma resuscitation: An interpretable deep learning approach for medical process data
Mary S. Kim, Sen Yang 0002, Genevieve J. Sippel, Aleksandra Sarcevic, Randall S. Burd, Ivan Marsic |
J. Biomed. Informatics | 7 |
| 2025 | To Recommend or Not to Recommend: Designing and Evaluating AI-Enabled Decision Support for Time-Critical Medical EventsabstractAI-enabled decision-support systems aim to help medical providers rapidly make decisions with limited information during medical emergencies. A critical challenge in developing these systems is supporting providers in interpreting the system output to make optimal treatment decisions. In this study, we designed and evaluated an AI-enabled decision-support system to aid providers in treating patients with traumatic injuries. We first conducted user research with physicians to identify and design information types and AI outputs for a decision-support display. We then conducted an online experiment with 35 medical providers from six health systems to evaluate two human-AI interaction strategies: (1) AI information synthesis and (2) AI information and recommendations. We found that providers were more likely to make correct decisions when AI information and recommendations were provided compared to receiving no AI support. We also identified two socio-technical barriers to providing AI recommendations during time-critical medical events: (1) an accuracy-time trade-off in providing recommendations and (2) polarizing perceptions of recommendations between providers. We discuss three implications for developing AI-enabled decision support used in time-critical events, contributing to the limited research on human-AI interaction in this context. Angela Mastrianni, Mary S. Kim, Travis M. Sullivan, Genevieve J. Sippel, Randall S. Burd, Krzysztof Z. Gajos, Aleksandra Sarcevic |
Proc. ACM Hum. Comput. Interact. | 5 |
| 2025 | Addressing Teamwork Delays during Life-Saving Interventions through an Activity Theory-Informed AnalysisabstractHemorrhage, or severe blood loss due to injury, is a leading cause of preventable deaths after injury. This study uses and extends activity theory to understand the dynamics of team-based hemorrhage control during trauma resuscitation and to explore potential computerized mechanisms to support this time- and safety-critical process. We reviewed videos of 25 resuscitation cases and analyzed hemorrhage control activities using nine activity theory prompts, including a new prompt-speech intention-a critical but underexplored dimension of teamwork in prior activity theory analyses. Through this process, we identified the most common delay-causing activities and developed routine and non-routine activity models for each. A comparison of these models showed that variations from the routine models emerged due to changes in the division of labor, instruments, community, and speech intentions. We contribute to research on designing socio-technical systems by (1) identifying needs and opportunities for computerized support that address delays in complex medical teamwork and (2) examining how an intervention changes an activity model. We also show how adding detailed speech data aids in identifying contradictions between elements in an activity model. Katherine Ann Zellner, Aleksandra Sarcevic, Maja Barnouw, Megan A. Krentsa, Travis M. Sullivan, Mary S. Kim, Randall S. Burd |
Proc. ACM Hum. Comput. Interact. | 7 |
| 2024 | ProcessGAN: Generating Privacy-Preserving Time-Aware Process Data with Conditional Generative Adversarial NetsabstractProcess data constructed from event logs provides valuable insights into procedural dynamics over time. The confidential information in process data, together with the data's intricate nature, makes the datasets not sharable and challenging to collect. Consequently, research is limited using process data and analytics in the process mining domain. In this study, we introduced a synthetic process data generation task to address the limitation of sharable process data. We introduced a generative adversarial network, called ProcessGAN, to generate process data with activity sequences and corresponding timestamps. ProcessGAN consists of a transformer-based network as the generator, and a time-aware self-attention network as the discriminator. It can generate privacy-preserving process data from random noise. ProcessGAN considers the duration of the process and time intervals between activities to generate realistic activity sequences with timestamps. We evaluated ProcessGAN on five real-world datasets, two that are public and three collected in medical domains that are private. To evaluate the synthetic data, in addition to statistical metrics, we trained a supervised model to score the synthetic processes. We also used process mining to discover workflows for synthetic medical processes and had domain experts evaluate the clinical applicability of the synthetic workflows. ProcessGAN outperformed the existing generative models in generating complex processes with valid parallel pathways. The synthetic process data generated by ProcessGAN better represented the long-range dependencies between activities, a feature relevant to complicated medical and other processes. The timestamps generated by the ProcessGAN model showed similar distributions with the authentic timestamps. In addition, we trained a transformer-based network to generate synthetic contexts (e.g., patient demographics) that were associated with the synthetic processes. The synthetic contexts generated by our model outperformed the baseline models, with the distributions similar to the authentic contexts. We conclude that ProcessGAN can generate sharable synthetic process data indistinguishable from authentic data. Our source code is available in https://github.com/raaachli/ProcessGAN. Sen Yang 0002, Travis M. Sullivan, Randall S. Burd, Ivan Marsic |
ACM Trans. Knowl. Discov. Data | 4 |
| 2023 | Supporting Awareness of Dynamic Data: Approaches to Designing and Capturing Data within Interactive Clinical ChecklistsabstractAutomatically integrating data within interactive clinical checklists allows for enhanced dynamic displays, while also providing information needed for checklist adaptation to the context of the medical event. In this mixed-methods study, we used user-centered design sessions with clinicians to design a checklist interface that automatically captures and displays dynamic patient data. We compared the manual and automatic checklist versions during video-guided simulation sessions, evaluating the effects of automatic capture on clinicians' interactions with dynamic data and their situation awareness. Despite clinicians' concerns that automatic data capture would affect situation awareness, we found no significant difference in awareness scores. Participants preferred the automatic version, highlighting its improved accuracy and completeness. From our findings, we propose a framework for capturing dynamic data and designing dynamic data interfaces within interactive checklists. We conclude by discussing barriers and design opportunities for supporting awareness of data trends through checklists. Angela Mastrianni, Aleksandra Sarcevic, Megan A. Krentsa, Travis M. Sullivan, Issa Zakeri, Ivan Marsic, Randall S. Burd |
Conference on Designing Interactive Systems | 8 |
| 2023 | Discovering interpretable medical process models: A case study in trauma resuscitation
Ivan Marsic, Aleksandra Sarcevic, Sen Yang 0002, Travis M. Sullivan, Peyton E. Tempel, Zachary P. Milestone, Karen J. O'Connell, Randall S. Burd |
J. Biomed. Informatics | 9 |
| 2023 | Understanding Delay Awareness and Mitigation Mechanisms through an Iterative Design and Evaluation of a Prototype Alert System for Complex TeamworkabstractAlmost half of the preventable deaths in emergency care can be associated with a medical delay. Understanding how clinicians experience delays can lead to improved alert designs to increase delay awareness and mitigation. In this paper, we present the findings from an iterative user-centered design process involving 48 clinicians to develop a prototype alert system for supporting delay awareness in complex medical teamwork such as trauma resuscitation. We used semi-structured interviews and card-sorting workshops to identify the most common delays and elicit design requirements for the prototype alert system. We then conducted a survey to refine the alert designs, followed by near-live, video-guided simulations to investigate clinicians' reactions to the alerts. We contribute to CSCW by designing a prototype alert system to support delay awareness in time-critical, complex teamwork and identifying four mechanisms through which teams mitigate delays. Katherine Ann Zellner, Aleksandra Sarcevic, Megan A. Krentsa, Travis M. Sullivan, Randall S. Burd |
Proc. ACM Hum. Comput. Interact. | 5 |
| 2023 | Transitioning Cognitive Aids into Decision Support Platforms: Requirements and Design GuidelinesabstractDigital cognitive aids have the potential to serve as clinical decision support platforms, triggering alerts about process delays and recommending interventions. In this mixed-methods study, we examined how a digital checklist for pediatric trauma resuscitation could trigger decision support alerts and recommendations. We identified two criteria that cognitive aids must satisfy to support these alerts: (1) context information must be entered in a timely, accurate, and standardized manner, and (2) task status must be accurately documented. Using co-design sessions and near-live simulations, we created two checklist features to satisfy these criteria: a form for entering the pre-hospital information and a progress slider for documenting the progression of a multi-step task. We evaluated these two features in the wild, contributing guidelines for designing these features on cognitive aids to support alerts and recommendations in time- and safety-critical scenarios. Angela Mastrianni, Aleksandra Sarcevic, Allison Hu, Lynn Almengor, Peyton E. Tempel, Sarah Gao, Randall S. Burd |
ACM Trans. Comput. Hum. Interact. | 7 |
| 2022 | An Analysis of Speech during Life Saving Interventions to Inform the Design of a Computerized System for Delay Detection
Katherine Ann Zellner, Louis Jiorgio Villegas, Charles Neff, Waverly Gestrich-Thompson, Randall S. Burd, Ivan Marsic, Aleksandra Sarcevic |
AMIA | 5 |
| 2022 | A Speech-Based Model for Tracking the Progression of Activities in Extreme Action TeamworkabstractDesigning computerized approaches to support complex teamwork requires an understanding of how activity-related information is relayed among team members. In this paper, we focus on verbal communication and describe a speech-based model that we developed for tracking activity progression during time-critical teamwork. We situated our study in the emergency medical domain of trauma resuscitation and transcribed speech from 104 audio recordings of actual resuscitations. Using the transcripts, we first studied the nature of speech during 34 clinically relevant activities. From this analysis, we identified 11 communicative events across three different stages of activity performance-before, during, and after. For each activity, we created sequential ordering of the communicative events using the concept of narrative schemas. The final speech-based model emerged by extracting and aggregating generalized aspects of the 34 schemas. We evaluated the model performance by using 17 new transcripts and found that the model reliably recognized an activity stage in 98% of activity-related conversation instances. We conclude by discussing these results, their implications for designing computerized approaches that support complex teamwork, and their generalizability to other safety-critical domains. Swathi Jagannath, Neha Kamireddi, Katherine Ann Zellner, Randall S. Burd, Ivan Marsic, Aleksandra Sarcevic |
Proc. ACM Hum. Comput. Interact. | 4 |
| 2021 | Designing Interactive Alerts to Improve Recognition of Critical Events in Medical EmergenciesabstractVital sign values during medical emergencies can help clinicians recognize and treat patients with life-threatening injuries. Identifying abnormal vital signs, however, is frequently delayed and the values may not be documented at all. In this mixed-methods study, we designed and evaluated a two-phased visual alert approach for a digital checklist in trauma resuscitation that informs users about undocumented vital signs. Using an interrupted time series analysis, we compared documentation in the periods before (two years) and after (four months) the introduction of the alerts. We found that introducing alerts led to an increase in documentation throughout the post-intervention period, with clinicians documenting vital signs earlier. Interviews with users and video review of cases showed that alerts were ineffective when clinicians engaged less with the checklist or set the checklist down to perform another activity. From these findings, we discuss approaches to designing alerts for dynamic team-based settings. Angela Mastrianni, Aleksandra Sarcevic, Lauren Chung, Issa Zakeri, Emily Alberto, Zachary P. Milestone, Ivan Marsic, Randall S. Burd |
Conference on Designing Interactive Systems | 8 |
| 2021 | Real-time medical phase recognition using long-term video understanding and progress gate method
Yanyi Zhang, Ivan Marsic, Randall S. Burd |
Medical Image Anal. | 3 |
| 2021 | Towards Dynamic Checklists: Understanding Contexts of Use and Deriving Requirements for Context-Driven AdaptationabstractThe goal of this in-the-wild study was to understand how different patient, provider, and environment contexts affected the use of a tablet-based checklist in a dynamic medical setting. Fifteen team leaders used the digital checklist in 187 actual trauma resuscitations. The measures of checklist interactions included the number of unchecked items and the number of notes written on the checklist. Of the 10 contexts we studied, team leaders’ arrival after the patient and patients with penetrating injuries were both associated with more unchecked items. We also found that the care of patients with external injuries contributed to more notes written on the checklist. Finally, our results showed that more experienced leaders took significantly more notes overall and more numerical notes than less experienced leaders. We conclude by discussing design implications and steps that can be achieved with context-aware computing towards adaptive checklists that meet the needs of dynamic use contexts. Leah Kulp, Aleksandra Sarcevic, Megan Cheng, Randall S. Burd |
ACM Trans. Comput. Hum. Interact. | 4 |
| 2020 | Checklist Design Reconsidered: Understanding Checklist Compliance and Timing of InteractionsabstractWe examine the association between user interactions with a checklist and task performance in a time-critical medical setting. By comparing 98 logs from a digital checklist for trauma resuscitation with activity logs generated by video review, we identified three non-compliant checklist use behaviors: failure to check items for completed tasks, falsely checking items when tasks were not performed, and inaccurately checking items for incomplete tasks. Using video review, we found that user perceptions of task completion were often misaligned with clinical practices that guided activity coding, thereby contributing to non-compliant check-offs. Our analysis of associations between different contexts and the timing of check-offs showed longer delays when (1) checklist users were absent during patient arrival, (2) patients had penetrating injuries, and (3) resuscitations were assigned to the highest acuity. We discuss opportunities for reconsidering checklist designs to reduce non-compliant checklist use. Leah Kulp, Aleksandra Sarcevic, Yinan Zheng, Megan Cheng, Emily Alberto, Randall S. Burd |
CHI | 6 |
| 2019 | Comparing the Effects of Paper and Digital Checklists on Team Performance in Time-Critical WorkabstractThis mixed-methods study examines the effects of a tablet-based checklist system on team performance during a dynamic and safety-critical process of trauma resuscitation. We compared team performance from 47 resuscitations that used a paper checklist to that from 47 cases with a digital checklist to determine if digitizing a checklist led to improvements in task completion rates and in how fast the tasks were initiated for 18 most critical assessment and treatment tasks. We also compared if the checklist compliance increased with the digital design. We found that using the digital checklist led to more frequent completions of the initial airway assessment task but fewer completions of ear and lower extremities exams. We did not observe any significant differences in time to task performance, but found increased compliance with the checklist. Although improvements in team performance with the digital checklist were minor, our findings are important because they showed no adverse effects as a result of the digital checklist introduction. We conclude by discussing the takeaways and implications of these results for effective digitization of medical work. Leah Kulp, Aleksandra Sarcevic, Megan Cheng, Yinan Zheng, Randall S. Burd |
CHI | 5 |
| 2018 | An approach to automatic process deviation detection in a time-critical clinical process
Sen Yang 0002, Aleksandra Sarcevic, Richard A. Farneth, Shuhong Chen, Omar Z. Ahmed, Ivan Marsic, Randall S. Burd |
J. Biomed. Informatics | 7 |
| 2017 | Exploring Design Opportunities for a Context-Adaptive Medical Checklist Through Technology Probe ApproachabstractThis paper explores the workflow and use of an interactive medical checklist for trauma resuscitation-an emerging technology developed for trauma team leaders to support decision making and task coordination among team members. We used a technology probe approach and ethnographic methods, including video review, interviews, and content analysis of checklist logs, to examine how team leaders use the checklist probe during live resuscitations. We found that team leaders of various experience levels use the technology differently. Some leaders frequently glance at the checklist and take notes during task performance, while others place the checklist on a stand and only interact with the checklist when checking items. We compared checklist timestamps to task activities and found that most items are checked off after tasks are performed. We conclude by discussing design implications and new design opportunities for a future dynamic, adaptive checklist. Leah Kulp, Aleksandra Sarcevic, Richard A. Farneth, Omar Z. Ahmed, Dung Mai, Ivan Marsic, Randall S. Burd |
Conference on Designing Interactive Systems | 7 |
| 2017 | 3D activity localization with multiple sensors: poster abstractabstractWe present a deep learning framework for fast 3D activity localization and tracking in a dynamic and crowded real world setting. Our training approach reverses the traditional activity localization approach, which first estimates the possible location of activities and then predicts their occurrence. Instead, we first trained a deep convolutional neural network for activity recognition using depth video and RFID data as input, and then used the activation maps of the network to locate the recognized activity in the 3D space. Our system achieved around 20cm average localization error (in a 4m × 5m room) which is comparable to Kinect's body skeleton tracking error (10--20cm), but our system tracks activities instead of Kinect's location of people. Xinyu Li 0003, Yanyi Zhang, Shuhong Chen, Richard A. Farneth, Ivan Marsic, Randall S. Burd |
IPSN | 8 |
| 2017 | CAR - a deep learning structure for concurrent activity recognition: poster abstractabstractWe introduce the Concurrent Activity Recognizer (CAR) - an efficient deep learning structure that recognizes complex concurrent teamwork activities from multimodal data. We implemented the system in a challenging medical setting, where it recognizes 35 different activities using Kinect depth video and data from passive RFID tags on 25 types of medical objects. Our preliminary results showed our system achieved an 84% average accuracy with 0.20 F1-Score. Yanyi Zhang, Xinyu Li 0003, Shuhong Chen, Moliang Zhou, Richard A. Farneth, Ivan Marsic, Randall S. Burd |
IPSN | 8 |
| 2017 | A Data-driven Process Recommender FrameworkabstractWe present an approach for improving the performance of complex knowledge-based processes by providing data-driven step-by-step recommendations. Our framework uses the associations between similar historic process performances and contextual information to determine the prototypical way of enacting the process. We introduce a novel similarity metric for grouping traces into clusters that incorporates temporal information about activity performance and handles concurrent activities. Our data-driven recommender system selects the appropriate prototype performance of the process based on user-provided context attributes. Our approach for determining the prototypes discovers the commonly performed activities and their temporal relationships. We tested our system on data from three real-world medical processes and achieved recommendation accuracy up to an F1 score of 0.77 (compared to an F1 score of 0.37 using ZeroR) with 63.2% of recommended enactments being within the first five neighbors of the actual historic enactments in a set of 87 cases. Our framework works as an interactive visual analytic tool for process mining. This work shows the feasibility of data-driven decision support system for complex knowledge-based processes. Sen Yang 0002, Xin Dong 0010, Leilei Sun, Richard A. Farneth, Hui Xiong 0001, Randall S. Burd, Ivan Marsic |
KDD | 7 |
| 2017 | Region-based Activity Recognition Using Conditional GANabstractWe present a method for activity recognition that first estimates the activity performer's location and uses it with input data for activity recognition. Existing approaches directly take video frames or entire video for feature extraction and recognition, and treat the classifier as a black box. Our method first locates the activities in each input video frame by generating an activity mask using a conditional generative adversarial network (cGAN). The generated mask is appended to color channels of input images and fed into a VGG-LSTM network for activity recognition. To test our system, we produced two datasets with manually created masks, one containing Olympic sports activities and the other containing trauma resuscitation activities. Our system makes activity prediction for each video frame and achieves performance comparable to the state-of-the-art systems while simultaneously outlining the location of the activity. We show how the generated masks facilitate the learning of features that are representative of the activity rather than accidental surrounding information. Xinyu Li 0003, Yanyi Zhang, Yueyang Chen, Huangcan Li, Ivan Marsic, Randall S. Burd |
ACM Multimedia | 7 |
| 2016 | Checklist as a Memory Externalization Tool during a Critical Care Process
Aleksandra Sarcevic, Zhan Zhang 0008, Ivan Marsic, Randall S. Burd, Leah Kulp |
AMIA | 4 |
| 2016 | Privacy Preserving Dynamic Room Layout Mapping
Xinyu Li 0003, Yanyi Zhang, Ivan Marsic, Randall S. Burd |
ICISP | 4 |
| 2016 | Deep Learning for RFID-Based Activity RecognitionabstractWe present a system for activity recognition from passive RFID data using a deep convolutional neural network. We directly feed the RFID data into a deep convolutional neural network for activity recognition instead of selecting features and using a cascade structure that first detects object use from RFID data followed by predicting the activity. Because our system treats activity recognition as a multi-class classification problem, it is scalable for applications with large number of activity classes. We tested our system using RFID data collected in a trauma room, including 14 hours of RFID data from 16 actual trauma resuscitations. Our system outperformed existing systems developed for activity recognition and achieved similar performance with process-phase detection as systems that require wearable sensors or manually-generated input. We also analyzed the strengths and limitations of our current deep learning architecture for activity recognition from RFID data. Xinyu Li 0003, Yanyi Zhang, Ivan Marsic, Aleksandra Sarcevic, Randall S. Burd |
SenSys | 5 |
| 2016 | Passive RFID for Object and Use Detection during Trauma ResuscitationabstractWe evaluated passive radio-frequency identification (RFID) technology for detecting the use of objects and related activities during trauma resuscitation. Our system consists of RFID tags and antennas, optimally placed for object detection, as well as algorithms for processing RFID data to infer object use. To evaluate our approach, we tagged 81 objects in the resuscitation room and recorded RFID signal strength during 32 simulated resuscitations performed by trauma teams. We then analyzed RFID data to identify cues for recognizing resuscitation activities. Using these cues, we extracted descriptive features and applied machine-learning techniques to monitor interactions with objects. Our results show that an instance of a used object can be detected with accuracy rates greater than 90 percent in a crowded and fast-paced medical setting using off-the-shelf RFID equipment, and the time and duration of use can be identified with up to 83 percent accuracy. We conclude with insights into the limitations of passive RFID and areas in which RFID needs to be complemented with other sensing technologies. Siddika Parlak, Ivan Marsic, Aleksandra Sarcevic, Waheed U. Bajwa, Lauren J. Waterhouse, Randall S. Burd |
IEEE Trans. Mob. Comput. | 6 |
| 2014 | Going Digital: Transforming Medical Checklists for Improved Patient Care
Bradford Winters, Randall S. Burd, Jesse Cirimele, Leslie Wu, Aleksandra Sarcevic |
AMIA | 2 |
| 2014 | Balancing design tensions: iterative display design to support ad hoc and multidisciplinary medical teamworkabstractIn this paper, we describe how we developed an information display prototype for trauma resuscitation teams based on design ideas and feedback from clinicians. Our approach is grounded in participatory design, emphasizing the importance of gaining long-term commitment from clinicians in system development. Through a series of participatory design workshops, heuristic evaluation, and simulated resuscitation sessions, we identified the main information features to include on our display. Our results focus on how we balanced the design tensions that emerged when addressing the ad hoc, hierarchical, and multidisciplinary nature of trauma teamwork. We discuss the implications of balancing role-based differences for each information feature, as well as two major design tensions: process-based vs. state-based designs and role-based vs. team-based displays. Diana S. Kusunoki, Aleksandra Sarcevic, Nadir Weibel, Ivan Marsic, Zhan Zhang 0008, Genevieve Tuveson, Randall S. Burd |
CHI | 7 |
| 2014 | Informing Digital Cognitive Aids Design for Emergency Medical Work by Understanding Paper Checklist UseabstractWe examine the use of a paper-based checklist during 48 simulated trauma resuscitations to inform the design of digital cognitive aids for safety-critical medical teamwork. Our analysis focused on team communication and interaction behaviors as physician leaders led resuscitations and administered the checklist. We found that the checklist increased the amount of communication between the leader and the team, but did not compromise the leader's interactions with the environment. In addition, we observed several changes in team dynamics: the checklist facilitated collaborative decision making and process reflections, but it also made some team members reactive rather than proactive. As the push toward digitizing medical work continues, we expect that paper checklists will soon be replaced by their digital counterparts. Designing interactive cognitive aids for medical domains, however, poses many challenges. Our results offer directions for how these tools could be designed to support medical work in increasingly digital environments. Zhan Zhang 0008, Aleksandra Sarcevic, Maria Yala, Randall S. Burd |
GROUP | 4 |
| 2013 | Supporting Information Use and Retention of Pre-Hospital Information during Trauma Resuscitation: A Qualitative Study of Pre-Hospital Communications and Information Needs
Zhan Zhang 0008, Aleksandra Sarcevic, Randall S. Burd |
AMIA | 3 |
| 2013 | Understanding visual attention of teams in dynamic medical settings through vital signs monitor useabstractThe purpose of this study was to understand how vital signs monitors support teamwork during trauma resuscitation -- the fast-paced and information-rich process of stabilizing critically injured patients. We analyzed 12 videos of simulated resuscitations to characterize trauma team monitor use. To structure our observations, we adopted the feedback loop concept. Our results showed that the monitor was used frequently, especially by team leaders and anesthesiologists. We identified three patterns of monitor use: (i) periods with a low frequency of short looks (glances) to maintain overall process awareness; (ii) periods with a medium frequency of long looks (scrutiny) to monitor trends in patient status; and (iii) peaks with a high frequency of glances to maintain attention on both the patient and monitor during critical tasks. Approximately 75% of looks were 3 seconds or shorter, but many looks (25%) ranged between 3 and 26 seconds. Our results have implications for improving displays by presenting the status of the patient's physiological systems and team activities. Diana S. Kusunoki, Aleksandra Sarcevic, Zhan Zhang 0008, Randall S. Burd |
CSCW | 4 |
| 2012 | Introducing RFID technology in dynamic and time-critical medical settings: Requirements and challenges
Siddika Parlak, Aleksandra Sarcevic, Ivan Marsic, Randall S. Burd |
J. Biomed. Informatics | 4 |
| 2012 | Teamwork Errors in Trauma ResuscitationabstractHuman errors in trauma resuscitation can have cascading effects leading to poor patient outcomes. To determine the nature of teamwork errors, we conducted an observational study in a trauma center over a two-year period. While eventually successful in treating the patients, trauma teams had problems tracking and integrating information in a longitudinal trajectory, which resulted in inefficiencies and near-miss errors. As an initial step in system design to support trauma teams, we proposed a model of teamwork and a novel classification of team errors. Four types of team errors emerged from our analysis: communication errors, vigilance errors, interpretation errors, and management errors. Based on these findings, we identified key information structures to support team cognition and decision making. We believe that displaying these information structures will support distributed cognition of trauma teams. Our findings have broader applicability to other collaborative and dynamic work settings that are prone to human error. Aleksandra Sarcevic, Ivan Marsic, Randall S. Burd |
ACM Trans. Comput. Hum. Interact. | 3 |
| 2011 | Coordinating time-critical work with role-taggingabstractA Level-1 US trauma center introduced role-tags in their trauma resuscitation rooms to help team members identify respective medical functions, and to limit the number of people in the rooms to required staff only. We use this in situ experiment with a paper prototype to investigate the role-driven nature of coordination and to identify system requirements for computerized support of role-based coordination in time-critical work. While role information is useful in coordinating time-critical work, our findings show that the current low-tech solution did not provide significant improvement in team coordination. The situations that were most in need of role-identification were the least likely to achieve it because role-tags required work by trauma team members. Similarly, because role-tags allowed workarounds and misuse, they proved ineffective in controlling the number of people in the room. We suggest technological ways of identifying roles to help coordination in the trauma bay. Aleksandra Sarcevic, Leysia Palen, Randall S. Burd |
CSCW | 3 |
| 2009 | Information handover in time-critical workabstractInformation transfer under time pressure and stress often leads to information loss. This paper studies the characteristics and problems of information handover from the emergency medical services (EMS) crew to the trauma team when a critically injured patient arrives to the trauma bay. We consider the characteristics of the handover process and the subsequent use of transferred information. Our goal is to support the design of technology for information transfer by identifying specific challenges faced by EMS crews and trauma teams during handover. Data were drawn from observation and video recording of 18 trauma resuscitations. The study shows how EMS crews report information from the field and the types of information that they include in their reports. Particular problems occur when reports lack structure, continuity, and complete descriptions of treatments given en route. We also found that trauma team members have problems retaining reported information. They pay attention to the items needed for immediately treating the patient and inquire about other items when needed during the resuscitation. The paper identifies a set of design challenges that arise during information transfer under time pressure and stress, and discusses characteristics of potential technological solutions. Aleksandra Sarcevic, Randall S. Burd |
GROUP | 2 |
| 2008 | "What's the Story?" Information Needs of Trauma Teams
Aleksandra Sarcevic, Randall S. Burd |
AMIA | 2 |
| 2008 | Transactive memory in trauma resuscitationabstractThis paper describes an ethnographic study conducted to explore the possibilities for future design and development of technological support for trauma teams. We videotaped 10 trauma resuscitations and transcribed each event. Using a framework that we developed, we coded each transcript to allow qualitative and quantitative analysis of the trauma teams' collaborative processes. We analyzed teams' tasks, interactions, and communication patterns that support information acquisition and sharing. Our results showed the importance of team transactive memory, but also pointed to inefficiencies in communication processes, which enable the functioning of this collective memory system. Based on quantitative and qualitative observations of trauma teamwork, we present opportunities for technological solutions that may reduce the cognitive effort needed for maintaining the working memory of trauma teams. Aleksandra Sarcevic, Ivan Marsic, Michael E. Lesk, Randall S. Burd |
CSCW | 4 |