OBJECTIVE: This study aimed to evaluate how differences in case complexity affect information-processing burden, as measured by eye-tracking, as well as the efficiency of electronic health record (EHR) use among healthcare providers in the intensive care unit. MATERIALS AND METHODS: This cross-sectional study recruited providers from 4 U.S. medical centers that use 2 prominent EHR systems (Epic and Oracle). After reporting demographic information, participants reviewed 2 complex cases and 2 standard cases in their institution's EHR system and then responded to 5 questions about each case, yielding a performance score. Information-processing burden was assessed by measuring the number of eye fixations via eye-tracking software. The efficiency of EHR use was assessed by measuring the task completion time, number of mouse clicks per minute, number of EHR screens viewed, and performance score. RESULTS: Eighty-one providers were included for analysis. Providers exhibited significantly more eye fixations (P < .001) and longer task completion times (P < .001) for complex cases than for standard cases. There were also significantly fewer mouse clicks per minute during complex cases (P < .001). Reviewing a complex case first led to significantly more eye fixations (P = .015) and longer task completion times (P < .01) than when a standard case was presented first. Case complexity did not significantly affect performance scores or the number of EHR screens viewed. DISCUSSION: Higher case complexity was shown to be associated with greater information-processing burden and less efficient EHR use. These findings have implications for enhancing the efficiency of EHR use, thereby leading to improved clinical decision-making and patient safety. Furthermore, reviewing complex cases first led to a greater information-processing burden, suggesting that providers could benefit from "warming up" with standard cases before reviewing complex cases.

OBJECTIVES: The objective of this work is to demonstrate the value of simulation testing for rapidly evaluating artificial intelligence (AI) products. MATERIALS AND METHODS: Researcher-physician teams simulated the use of 2 Ambient Digital Scribe (ADS) products by reading scripts of outpatient encounters while using both products, yielding a total of 44 draft notes. Time to edit, perceived amount of effort and editing, and errors in the AI-generated draft notes were analyzed. RESULTS: Ambient Digital Scribe Product A draft notes took significantly longer to edit, had fewer omissions, and more additions and irrelevant or misplaced text errors than ADS Product B. Ambient Digital Scribe Product A was rated as performing better for most encounters. DISCUSSION: Artificial intelligence-enabled products are being rapidly developed and implemented into practice, outpacing safety concerns. Simulation testing can efficiently identify safety issues. CONCLUSION: Simulation testing is a crucial first step to take when evaluating AI-enabled technologies.

OBJECTIVE: To conduct a meta-ethnographic synthesis summarizing the overarching themes of the qualitative literature on nurse interaction with medication administration technologies (MAT) comprising electronic medication administration record (eMAR) and bar-coded medication administration (BCMA). MATERIALS AND METHODS: We searched scientific databases from their inception until September 23, 2024, resulting in 2270 unique articles, and extracted data from 27 articles. Scientific rigor was assessed by the Standards for Reporting Qualitative Research (SRQR) checklist. Noblit and Hare's methodology was used to conduct a meta-ethnography to identify and interpret emergent themes. RESULTS: SRQR revealed low-to-medium methodological quality and theoretical framing of the literature. We found 6 overarching themes connecting MAT with nursing work: (1) View of the MAT system as mechanistic and invariant vs living and co-evolving with its users drives the research approach; (2) MAT limits nurse autonomy; (3) MAT unnaturally splits the medication administration workflow; (4) Nurses must manage MAT challenges at the sharp end; (5) MAT does not align with social dependencies of nursing work; and (6) MAT increases perceived safety but can also result in new types of errors. DISCUSSION: MAT does not support key cognitive work that nurses must perform to maintain safety. Additionally, MAT can impede problem solving during medication management and limit nursing autonomy that is essential for safe medication administration. CONCLUSION: Recommendations based in human factors engineering recognizing how MAT design impacts nursing work and workload are essential in improving the fit of MAT to nurse cognitive workflows.

OBJECTIVES: Physician burnout in the US has reached crisis levels, with one source identified as extensive after-hours documentation work in the electronic health record (EHR). Evidence has illustrated that physician preferences for after-hours work vary, such that after-hours work may not be universally burdensome. Our objectives were to analyze variation in preferences for after-hours documentation and assess if preferences mediate the relationship between after-hours documentation time and burnout. MATERIALS AND METHODS: We combined EHR active use data capturing physicians' hourly documentation work with survey data capturing documentation preferences and burnout. Our sample included 318 ambulatory physicians at MedStar Health. We conducted a mediation analysis to estimate if and how preferences mediated the relationship between after-hours documentation time and burnout. Our primary outcome was physician-reported burnout. We measured preferences for after-hours documentation work via a novel survey instrument (Burden Scenarios Assessment). We measured after-hours documentation time in the EHR as the total active time respondents spent documenting between 7 pm and 3 am. RESULTS: Physician preferences varied, with completing clinical documentation after clinic hours while at home the scenario rated most burdensome (52.8% of physicians), followed by dealing with prior authorization (49.5% of physicians). In mediation analyses, preferences partially mediated the relationship between after-hours documentation time and burnout. DISCUSSION: Physician preferences regarding EHR-based work play an important role in the relationship between after-hours documentation time and burnout. CONCLUSION: Studies of EHR work and burnout should incorporate preferences, and operational leaders should assess preferences to better target interventions aimed at EHR-based contributors to burnout.

OBJECTIVE: To determine whether the Office of the National Coordinator's policy change restricting the use of "gag clauses" in contracts between electronic health record (EHR) vendors and healthcare facilities increased the prevalence of screenshots in peer-reviewed literature. MATERIALS AND METHODS: We reviewed EHR usability and safety-related peer-reviewed journal articles from 2015 to 2023 and quantified the number of articles containing screenshots. For those that did not contain screenshots, we identified whether they would have benefited from screenshots. RESULTS: When gag clauses were permitted 6 of 79 (7.6%) of articles contained screenshots and 8 (10.1%) would have benefited from screenshots. When gag clauses were restricted 3 of 40 (7.5%) contained screenshots and 8 (20%) would have benefited from screenshots. DISCUSSION: The policy change does not appear to have an impact on the prevalence of screenshots in peer-reviewed literature. CONCLUSIONS: Additional steps are necessary to promote the use of screenshots in peer-reviewed literature.

We appreciate the thoughtful response to our brief article entitled “Electronic health record ‘gag clauses and the prevalence of screenshots in peer-reviewed literature.” Dr. Koppel raised several important points that we wish to acknowledge and respond to. Dr. Koppel reminds us that there were 2 Institute of Medicine Reports and an American Medical Informatics Association task force that described electronic health record (EHR) gag clauses and the problems associated with these clauses. There have also been several other stakeholders that have criticized these gag clauses as well as EHR vendors that have argued the clauses are necessary to protect intellectual property. It has taken far too long for a policy change to address the gag clause issue that was identified in 2009 by Dr. Koppel and others. In the over 10 years that it took for a policy change numerous harm events associated with EHRs have occurred that could have potentially been prevented by screenshot sharing to identify safety issues and resulting redesign, improvement, and mitigation. The Office of the National Coordinator for Health Information Technology (ONC) policy change is a major advancement, however, our data suggest it may not be having the desired effect of increasing the sharing of screenshots.

OBJECTIVES: (1) Characterize persistent hazards and inefficiencies in inpatient medication administration; (2) Explore cognitive attributes of medication administration tasks; and (3) Discuss strategies to reduce medication administration technology-related hazards. MATERIALS AND METHODS: Interviews were conducted with 32 nurses practicing at 2 urban, eastern and western US health systems. Qualitative analysis using inductive and deductive coding included consensus discussion, iterative review, and coding structure revision. We abstracted hazards and inefficiencies through the lens of risks to patient safety and the cognitive perception-action cycle (PAC). RESULTS: Persistent safety hazards and inefficiencies related to MAT organized around the PAC cycle included: (1) Compatibility constraints create information silos; (2) Missing action cues; (3) Intermittent communication flow between safety monitoring systems and nurses; (4) Occlusion of important alerts by other, less helpful alerts; (5) Dispersed information: Information required for tasks is not collocated; (6) Inconsistent data organization: Mismatch of the display and the user's mental model; (7) Hidden medication administration technologies (MAT) limitations: Inaccurate beliefs about MAT functionality contribute to overreliance on the technology; (8) Software rigidity causes workarounds; (9) Cumbersome dependencies between technology and the physical environment; and (10) Technology breakdowns require adaptive actions. DISCUSSION: Errors might persist in medication administration despite successful Bar Code Medication Administration and Electronic Medication Administration Record deployment for reducing errors. Opportunities to improve MAT require a deeper understanding of high-level reasoning in medication administration, including control over the information space, collaboration tools, and decision support. CONCLUSION: Future medication administration technology should consider a deeper understanding of nursing knowledge work for medication administration.

Despite basic federal requirements promoting a user-centered design approach to electronic health record (EHR) development and usability testing there have been usability and safety risks with EHR technology. Four EHR vendors were asked to provide written descriptions of their usability practices, and we reviewed these descriptions to identify areas where there has been advancement and areas for improvement. All 4 vendors described user-centered design processes and usability testing methods that demonstrate advancement from previous studies of vendor practices. Importantly, vendors are also beginning to address aspects of EHR implementation that play a critical role in shaping EHR usability. There are important areas for improvement in vendor practices including a greater focus on safety and on measurement and benchmarking. Vendors sharing their current usability practices demonstrates a step toward greater transparency which has typically been lacking.

OBJECTIVE: To derive 7 proposed core electronic health record (EHR) use metrics across 2 healthcare systems with different EHR vendor product installations and examine factors associated with EHR time. MATERIALS AND METHODS: A cross-sectional analysis of ambulatory physicians EHR use across the Yale-New Haven and MedStar Health systems was performed for August 2019 using 7 proposed core EHR use metrics normalized to 8 hours of patient scheduled time. RESULTS: Five out of 7 proposed metrics could be measured in a population of nonteaching, exclusively ambulatory physicians. Among 573 physicians (Yale-New Haven N = 290, MedStar N = 283) in the analysis, median EHR-Time8 was 5.23 hours. Gender, additional clinical hours scheduled, and certain medical specialties were associated with EHR-Time8 after adjusting for age and health system on multivariable analysis. For every 8 hours of scheduled patient time, the model predicted these differences in EHR time (P < .001, unless otherwise indicated): female physicians +0.58 hours; each additional clinical hour scheduled per month -0.01 hours; practicing cardiology -1.30 hours; medical subspecialties -0.89 hours (except gastroenterology, P = .002); neurology/psychiatry -2.60 hours; obstetrics/gynecology -1.88 hours; pediatrics -1.05 hours (P = .001); sports/physical medicine and rehabilitation -3.25 hours; and surgical specialties -3.65 hours. CONCLUSIONS: For every 8 hours of scheduled patient time, ambulatory physicians spend more than 5 hours on the EHR. Physician gender, specialty, and number of clinical hours practicing are associated with differences in EHR time. While audit logs remain a powerful tool for understanding physician EHR use, additional transparency, granularity, and standardization of vendor-derived EHR use data definitions are still necessary to standardize EHR use measurement.

OBJECTIVE: Despite a proliferation of applications (apps) to conveniently collect patient-reported outcomes (PROs) from patients, PRO data are yet to be seamlessly integrated with electronic health records (EHRs) in a way that improves interoperability and scalability. We applied the newly created PRO standards from the Office of the National Coordinator for Health Information Technology to facilitate the collection and integration of standardized PRO data. A novel multitiered architecture was created to enable seamless integration of PRO data via Substitutable Medical Apps and Reusable Technologies on Fast Healthcare Interoperability Resources apps and scaled to different EHR platforms in multiple ambulatory settings. MATERIALS AND METHODS: We used a standards-based approach to deploy 2 apps that source and surface PRO data in real-time for provider use within the EHR and which rely on PRO assessments from an external center to streamline app and EHR integration. RESULTS: The apps were developed to enable patients to answer validated assessments (eg, a Patient-Reported Outcomes Measurement Information System including using a Computer Adaptive Test format). Both apps were developed to populate the EHR in real time using the Health Level Seven FHIR standard allowing providers to view patients' data during the clinical encounter. The process of implementing this architecture with 2 different apps across 18 ambulatory care sites and 3 different EHR platforms is described. CONCLUSION: Our approach and solution proved feasible, secure, and time- and resource-efficient. We offer actionable guidance for this technology to be scaled and adapted to promote adoption in diverse ambulatory care settings and across different EHRs.

The COVID-19 pandemic has led to the rapid expansion of telehealth services as healthcare organizations aim to mitigate community transmission while providing safe patient care. As technology adoption rapidly increases, operational telehealth teams must maintain awareness of critical information, such as patient volumes and wait times, patient and provider experience, and telehealth platform performance. Using a model of situation awareness as a conceptual foundation and a user-centered design approach we describe our process for rapidly developing and disseminating dashboard visualizations to support telehealth operations. We used a 5-step process to gain domain knowledge, identify user needs, identify data sources, design and develop visualizations, and iteratively refine these visualizations. Through this process we identified 3 distinct stakeholder groups and designed and developed visualization dashboards to meet their needs. Feedback from users demonstrated the dashboard's support situation awareness and informed important operational decisions. Lessons learned are shared to provide other organizations with insights from our process.

OBJECTIVE: We sought to determine rates of computerized provider order entry (CPOE) patient identity verification and when and where in the ordering process verification occurred. MATERIALS AND METHODS: Fifty-five physicians from 4 healthcare systems completed simulated patient scenarios using their respective CPOE system (Epic or Cerner). Eye movements were recorded and analyzed. RESULTS: Across all participants patient id was verified significantly more often than not (62.4% vs 37.6%). Vendor A had significantly higher verification rates than not; vendor B had no difference. Participants using vendor A verified information significantly more often before signing the order than after (88.4% vs 11.6%); there was no difference in vendor B. The banner bar was the most frequent verification location. DISCUSSION: Factors such as CPOE design, physician training, and the use of a simulated methodology may be impacting verification rates. CONCLUSIONS: Verification rates vary by CPOE product, and this can have patient safety consequences.

Healthcare information technologies are now a routine component of patient-clinician interactions. Originally designed for operational functions including billing and regulatory compliance, these systems have had unintended consequences including increased exam room documentation, divided attention during the visit, and use of scribes to alleviate documentation burdens. In an age in which technology is ubiquitous in everyday life, we must re-envision healthcare technology to support both clinical operations and, above all, the patient-clinician relationship. We present 6 habits for designing user-centered health technologies: (1) put patient care first, (2) assemble a team with the right skills, (3) relentlessly ask WHY, (4) keep it simple, (5) be Darwinian, and (6) don't lose the forest for the trees. These habits should open dialogues between developers, implementers, end users, and stakeholders, as well as outline a path for better, more usable technology that puts patients and their clinicians back at the center of care.

Objective: We sought to understand the types of clinical processes, such as image and medication ordering, that are disrupted during electronic health record (EHR) downtime periods by analyzing the narratives of patient safety event report data. Materials and Methods: From a database of 80 381 event reports, 76 reports were identified as explicitly describing a safety event associated with an EHR downtime period. These reports were analyzed and categorized based on a developed code book to identify the clinical processes that were impacted by downtime. We also examined whether downtime procedures were in place and followed. Results: The reports were coded into categories related to their reported clinical process: Laboratory, Medication, Imaging, Registration, Patient Handoff, Documentation, History Viewing, Delay of Procedure, and General. A majority of reports (48.7%, n = 37) were associated with lab orders and results, followed by medication ordering and administration (14.5%, n = 11). Incidents commonly involved patient identification and communication of clinical information. A majority of reports (46%, n = 35) indicated that downtime procedures either were not followed or were not in place. Only 27.6% of incidents (n = 21) indicated that downtime procedures were successfully executed. Discussion: Patient safety report data offer a lens into EHR downtime-related safety hazards. Important areas of risk during EHR downtime periods were patient identification and communication of clinical information; these should be a focus of downtime procedure planning to reduce safety hazards. Conclusion: EHR downtime events pose patient safety hazards, and we highlight critical areas for downtime procedure improvement.

Objective: Provider acceptance and associated patient outcomes are widely discussed in the evaluation of clinical decision support systems (CDSSs), but critical design criteria for tools have generally been overlooked. The objective of this work is to inform electronic health record alert optimization and clinical practice workflow by identifying, compiling, and reporting design recommendations for CDSS to support the efficient, effective, and timely delivery of high-quality care. Material and Methods: A narrative review was conducted from 2000 to 2016 in PubMed and The Journal of Human Factors and Ergonomics Society to identify papers that discussed/recommended design features of CDSSs that are associated with the success of these systems. Results: Fourteen papers were included as meeting the criteria and were found to have a total of 42 unique recommendations; 11 were classified as interface features, 10 as information features, and 21 as interaction features. Discussion: Features are defined and described, providing actionable guidance that can be applied to CDSS development and policy. To our knowledge, no reviews have been completed that discuss/recommend design features of CDSS at this scale, and thus we found that this was important for the body of literature. The recommendations identified in this narrative review will help to optimize design, organization, management, presentation, and utilization of information through presentation, content, and function. The designation of 3 categories (interface, information, and interaction) should be further evaluated to determine the critical importance of the categories. Future work will determine how to prioritize them with limited resources for designers and developers in order to maximize the clinical utility of CDSS. Conclusion: This review will expand the field of knowledge and provide a novel organization structure to identify key recommendations for CDSS.

To characterize the variability in usability and safety of EHRs from two vendors across four healthcare systems (2 Epic and 2 Cerner). Twelve to 15 emergency medicine physicians participated from each site and completed six clinical scenarios. Keystroke, mouse click, and video data were collected. From the six scenarios, two diagnostic imaging, laboratory, and medication tasks were analyzed. There was wide variability in task completion time, clicks, and error rates. For certain tasks, there were an average of a nine-fold difference in time and eight-fold difference in clicks. Error rates varied by task (X-ray 16.7% to 25%, MRI: 0 to 10%, Lactate: 0% to 14.3%, Tylenol: 0 to 30%; Taper: 16.7% to 50%). The variability in time, clicks, and error rates highlights the need for improved implementation optimization. EHR implementation, in addition to vendor design and development, is critical to usable and safe products.

Interruptions in the emergency department (ED) can have serious patient safety consequences, and few solutions exist to mitigate the disruptiveness of interruptions. We developed a theoretically motivated model to predict the likelihood of emergency physicians returning to an interrupted task. Eighteen emergency physicians were observed individually for two-hour blocks of time, resulting in a total of 2160 minutes of observation and 231 interruptions. We used a mixed effects logistic regression model to predict the likelihood of primary task resumption after interruptions. The likelihood of primary task resumption was predicted by memory decay, measured by the duration of the interruption, workload, measured by the patient volume during the shift, and whether shift was day or night. With a better understanding of these interruptions, we can help design interventions to manage interruptions, minimize medical errors, and improve patient safety.

Despite the widespread adoption of electronic health records (EHRs), usability of many EHRs continues to be suboptimal, with some vendors failing to meet usability standards, resulting in clinician frustration and patient safety hazards. In an effort to increase EHR vendor competition on usability, recommendations have been made and legislation drafted to develop comparison tools that would allow purchasers to better understand the usability of EHR products prior to purchase. Usability comparison can be based on EHR vendor design and development processes, vendor usability testing as part of the Office of the National Coordinator for Health Information Technology certification program, and usability of implemented products. Barriers exist within the current certified health technology program that prevent effective comparison of usability during each of these stages. We describe the importance of providing purchasers with improved information about EHR usability, barriers to making usability comparisons, and solutions to overcome these barriers.

OBJECTIVE: Currently, there are few resources for electronic health record (EHR) purchasers and end users to understand the usability processes employed by EHR vendors during product design and development. We developed a framework, based on human factors literature and industry standards, to systematically evaluate the user-centered design processes and usability testing methods used by EHR vendors. MATERIALS AND METHODS: We reviewed current usability certification requirements and the human factors literature to develop a 15-point framework for evaluating EHR products. The framework is based on 3 dimensions: user-centered design process, summative testing methodology, and summative testing results. Two vendor usability reports were retrieved from the Office of the National Coordinator's Certified Health IT Product List and were evaluated using the framework. RESULTS: One vendor scored low on the framework (5 pts) while the other vendor scored high on the framework (15 pts). The 2 scored vendor reports demonstrate the framework's ability to discriminate between the variabilities in vendor processes and to determine which vendors are meeting best practices. DISCUSSION: The framework provides a method to more easily comprehend EHR vendors' usability processes and serves to highlight where EHR vendors may be falling short in terms of best practices. The framework provides a greater level of transparency for both purchasers and end users of EHRs. CONCLUSION: The framework highlights the need for clearer certification requirements and suggests that the authorized certification bodies that examine vendor usability reports may need to be provided with clearer guidance.

OBJECTIVE: Despite federal policies put in place by the Office of the National Coordinator (ONC) to promote safe and usable electronic health record (EHR) products, the usability of EHRs continues to frustrate providers and have patient safety implications. This study sought to compare government policies on usability and safety, and methods of examining compliance to those policies, across 3 federal agencies: the ONC and EHRs, the Federal Aviation Administration (FAA) and avionics, and the Food and Drug Administration (FDA) and medical devices. Our goal was to identify whether differences in policies exist and, if they do exist, how policies and enforcement mechanisms from other industries might be applied to optimize EHR usability. METHOD: We performed a qualitative study using publicly available governing documents to examine similarities and differences in usability and safety policies across agencies. RESULTS: The policy review and analysis revealed several consistencies within each agency's usability policies. Critical differences emerged in the usability standards and policy enforcement mechanisms utilized by the 3 agencies. The FAA and FDA look at evidence of usability processes and are more prescriptive when it comes to testing final products as compared to the ONC, which relies on attestation and is less prescriptive. DISCUSSION: A comparison of usability policies across industries illustrates key differences between the ONC and other federal agencies. These differences could be contributing to the usability challenges associated with EHRs. CONCLUSION: Our analysis highlights important areas of usability and safety policy from other industries that can better inform ONC policies on EHRs.

An increasing number of healthcare providers are adopting patient safety event reporting systems, yet leveraging these data to improve safety remains a challenge, particularly with large datasets composed of thousands of event reports. A MedStar Health research team, with expertise in data analytics and human factors, developed intuitive visualization dashboards to facilitate data exploration and trend analysis. Dashboards were developed using an iterative design and development process that was end-user focused. A system level dashboard, representing data from multiple hospitals, and a hospital level dashboard were developed. The dashboards allowed users to directly manipulate the data, provided coordinated displays in different formats, and allowed users to quickly zoom in on specific variables of interest. Overall feedback was incredibly positive with nearly all users wanting to adopt the system immediately. Several improvements were suggested and are discussed. The success of this approach highlights the need for more intuitive data analysis tools.

The usability of electronic health records (EHRs) continues to be a point of dissatisfaction for providers, despite certification requirements from the Office of the National Coordinator that require EHR vendors to employ a user-centered design (UCD) process. To better understand factors that contribute to poor usability, a research team visited 11 different EHR vendors in order to analyze their UCD processes and discover the specific challenges that vendors faced as they sought to integrate UCD with their EHR development. Our analysis demonstrates a diverse range of vendors' UCD practices that fall into 3 categories: well-developed UCD, basic UCD, and misconceptions of UCD. Specific challenges to practicing UCD include conducting contextually rich studies of clinical workflow, recruiting participants for usability studies, and having support from leadership within the vendor organization. The results of the study provide novel insights for how to improve usability practices of EHR vendors.

Many interfaces have been designed to prevent or reduce errors. These interfaces may, in fact, reduce the error rate of specific error classes, but may also have unintended consequences. In this paper, we show a series of studies where a better interface did not reduce the number of errors but instead shifted errors from one error class (omissions) to another error class (perseverations). We also show that having access to progress tracking (a progress bar) does not reduce the number of errors. We propose and demonstrate a solution -- a predictive error system -- that reduces errors based on the error class, not on the type of interface.

Procedural errors occur despite the user having the correct knowledge of how to perform a particular task. Previous research has mostly focused on preventing these errors by redesigning tasks to eliminate error prone steps. A differentmethod of preventing errors, specifically postcompletion errors (e.g., forgetting to retrieve the original document from a photocopier), has been proposed by Ratwani, McCurry, and Trafton (2008), which uses theoretically motivated eye movement measures to predict when a user will make an error. The predictive value of the eye-movement-based model was examined and validated on two different tasks using a receiver-operating characteristic analysis. A real-time eye-tracking postcompletion error pre-diction system was then developed and tested; results demonstrate that the real-time system successfully predicts and prevents postcompletion errors before a user commits the error. 1.

For a single operator to effectively control multiple robots, operator situation awareness is a critical component of the human-robot system. There are three levels of situation awareness: perception, comprehension, and projection into the future [1]. We focus on the perception level to develop a theoretic model of the perceptual-cognitive processes underlying situation awareness. Eye movement measures were developed as indicators of cognitive processing and these measures were used to account for operator situation awareness on a supervisory control task. The eye movement based model emphasizes the importance of visual scanning and attention allocation as the cognitive processes that lead to operator situation awareness and the model lays the groundwork for real-time prediction of operator situation awareness.

A postcompletion error is a distinct type of procedural error where one fails to complete the final step of a task. While redesigning interfaces and providing explicit cues have been shown to be effective in reducing the postcompletion error rate, these methods are not always feasible or well liked. This paper demonstrates how specific eye movement measures can be used to predict when a user will make a postcompletion error. We describe a real-time eye gaze system that provides cues to the user if and only if there is a high probability of the user making a postcompletion error.

We describe a computational cognitive architecture for robots which we call ACT-R/E (ACT-R/Embodied). ACT-R/E is based on ACT-R [1, 2] but uses different visual, auditory, and movement modules. We describe a model that uses ACT-R/E to integrate visual and auditory information to perform conversation tracking in a dynamic environment. We also performed an empirical evaluation study which shows that people see our conversational tracking system as extremely natural.