OBJECTIVE: Healthcare decisions are increasingly made with the assistance of machine learning (ML). ML has been known to have unfairness-inconsistent outcomes across subpopulations. Clinicians interacting with these systems can perpetuate such unfairness by overreliance. Recent work exploring ML suppression-silencing predictions based on auditing the ML-shows promise in mitigating performance issues originating from overreliance. This study aims to evaluate the impact of suppression on collaboration fairness and evaluate ML uncertainty as desiderata to audit the ML. MATERIALS AND METHODS: We used data from the Vanderbilt University Medical Center electronic health record (n = 58 817) and the MIMIC-IV-ED dataset (n = 363 145) to predict likelihood of death or intensive care unit transfer and likelihood of 30-day readmission using gradient-boosted trees and an artificially high-performing oracle model. We derived clinician decisions directly from the dataset and simulated clinician acceptance of ML predictions based on previous empirical work on acceptance of clinical decision support alerts. We measured performance as area under the receiver operating characteristic curve and algorithmic fairness using absolute averaged odds difference. RESULTS: When the ML outperforms humans, suppression outperforms the human alone (P < 8.2 × 10-6) and at least does not degrade fairness. When the human outperforms the ML, the human is either fairer than suppression (P < 8.2 × 10-4) or there is no statistically significant difference in fairness. Incorporating uncertainty quantification into suppression approaches can improve performance. CONCLUSION: Suppression of poor-quality ML predictions through an auditor model shows promise in improving collaborative human-AI performance and fairness.

OBJECTIVES: Research on artificial intelligence (AI)-based clinical decision-support (AI-CDS) systems has returned mixed results. Sometimes providing AI-CDS to a clinician will improve decision-making performance, sometimes it will not, and it is not always clear why. This scoping review seeks to clarify existing evidence by identifying clinician-level and technology design factors that impact the effectiveness of AI-assisted decision-making in medicine. MATERIALS AND METHODS: We searched MEDLINE, Web of Science, and Embase for peer-reviewed papers that studied factors impacting the effectiveness of AI-CDS. We identified the factors studied and their impact on 3 outcomes: clinicians' attitudes toward AI, their decisions (eg, acceptance rate of AI recommendations), and their performance when utilizing AI-CDS. RESULTS: We retrieved 5850 articles and included 45. Four clinician-level and technology design factors were commonly studied. Expert clinicians may benefit less from AI-CDS than nonexperts, with some mixed results. Explainable AI increased clinicians' trust, but could also increase trust in incorrect AI recommendations, potentially harming human-AI collaborative performance. Clinicians' baseline attitudes toward AI predict their acceptance rates of AI recommendations. Of the 3 outcomes of interest, human-AI collaborative performance was most commonly assessed. DISCUSSION AND CONCLUSION: Few factors have been studied for their impact on the effectiveness of AI-CDS. Due to conflicting outcomes between studies, we recommend future work should leverage the concept of "appropriate trust" to facilitate more robust research on AI-CDS, aiming not to increase overall trust in or acceptance of AI but to ensure that clinicians accept AI recommendations only when trust in AI is warranted.

OBJECTIVE: Returning aggregate study results is an important ethical responsibility to promote trust and inform decision making, but the practice of providing results to a lay audience is not widely adopted. Barriers include significant cost and time required to develop lay summaries and scarce infrastructure necessary for returning them to the public. Our study aims to generate, evaluate, and implement ChatGPT 4 lay summaries of scientific abstracts on a national clinical study recruitment platform, ResearchMatch, to facilitate timely and cost-effective return of study results at scale. MATERIALS AND METHODS: We engineered prompts to summarize abstracts at a literacy level accessible to the public, prioritizing succinctness, clarity, and practical relevance. Researchers and volunteers assessed ChatGPT-generated lay summaries across five dimensions: accuracy, relevance, accessibility, transparency, and harmfulness. We used precision analysis and adaptive random sampling to determine the optimal number of summaries for evaluation, ensuring high statistical precision. RESULTS: ChatGPT achieved 95.9% (95% CI, 92.1-97.9) accuracy and 96.2% (92.4-98.1) relevance across 192 summary sentences from 33 abstracts based on researcher review. 85.3% (69.9-93.6) of 34 volunteers perceived ChatGPT-generated summaries as more accessible and 73.5% (56.9-85.4) more transparent than the original abstract. None of the summaries were deemed harmful. We expanded ResearchMatch's technical infrastructure to automatically generate and display lay summaries for over 750 published studies that resulted from the platform's recruitment mechanism. DISCUSSION AND CONCLUSION: Implementing AI-generated lay summaries on ResearchMatch demonstrates the potential of a scalable framework generalizable to broader platforms for enhancing research accessibility and transparency.

Use of artificial intelligence in healthcare, such as machine learning-based predictive algorithms, holds promise for advancing outcomes, but few systems are used in routine clinical practice. Trust has been cited as an important challenge to meaningful use of artificial intelligence in clinical practice. Artificial intelligence systems often involve automating cognitively challenging tasks. Therefore, previous literature on trust in automation may hold important lessons for artificial intelligence applications in healthcare. In this perspective, we argue that informatics should take lessons from literature on trust in automation such that the goal should be to foster appropriate trust in artificial intelligence based on the purpose of the tool, its process for making recommendations, and its performance in the given context. We adapt a conceptual model to support this argument and present recommendations for future work.

OBJECTIVE: The characteristics of clinician activities while interacting with electronic health record (EHR) systems can influence the time spent in EHRs and workload. This study aims to characterize EHR activities as tasks and define novel, data-driven metrics. MATERIALS AND METHODS: We leveraged unsupervised learning approaches to learn tasks from sequences of events in EHR audit logs. We developed metrics characterizing the prevalence of unique events and event repetition and applied them to categorize tasks into 4 complexity profiles. Between these profiles, Mann-Whitney U tests were applied to measure the differences in performance time, event type, and clinician prevalence, or the number of unique clinicians who were observed performing these tasks. In addition, we apply process mining frameworks paired with clinical annotations to support the validity of a sample of our identified tasks. We apply our approaches to learn tasks performed by nurses in the Vanderbilt University Medical Center neonatal intensive care unit. RESULTS: We examined EHR audit logs generated by 33 neonatal intensive care unit nurses resulting in 57 234 sessions and 81 tasks. Our results indicated significant differences in performance time for each observed task complexity profile. There were no significant differences in clinician prevalence or in the frequency of viewing and modifying event types between tasks of different complexities. We presented a sample of expert-reviewed, annotated task workflows supporting the interpretation of their clinical meaningfulness. CONCLUSIONS: The use of the audit log provides an opportunity to assist hospitals in further investigating clinician activities to optimize EHR workflows.

OBJECTIVE: Using the case of barcode medication administration (BCMA), our objective is to describe the challenges nurses face when informatics tools are not designed to accommodate the full complexity of their work. MATERIALS AND METHODS: Autonomy is associated with nurse satisfaction and quality of care. BCMA organizes patient information and verifies medication administration. However, it presents challenges to nurse autonomy. Qualitative fieldwork, including observations of everyday work and interviews, was conducted during the implementation of BCMA in a large academic medical center. Fieldnotes and interview transcripts were coded and analyzed to describe nurses' perspectives on medication safety. RESULTS: Nurses adopt orienting frames to structure work routines and require autonomy to ensure safe task completion. Nurses exerted agency by trusting their own judgment over system information when the system did not consider workload complexity. Our results indicate that the system's rigidity clashed with adaptive needs embodied by nurses' orienting frames. DISCUSSION: Despite the fact that the concept of nurse as knowledge worker is foundational to informatics, nurses may be perceived as doers, rather than knowledge workers. In practice, nurses not only make decisions, but also engage in highly complex task-related work that is not well supported by process-oriented information technology tools. CONCLUSIONS: Information technology developers and healthcare organization managers should engage and better understand nursing work in order to develop technological and social systems to support it.

OBJECTIVE: The goals of this study are to describe the value and impact of Project HealthDesign (PHD), a program of the Robert Wood Johnson Foundation that applied design thinking to personal health records, and to explore the applicability of the PHD model to another challenging translational informatics problem: the integration of AI into the healthcare system. MATERIALS AND METHODS: We assessed PHD's impact and value in 2 ways. First, we analyzed publication impact by calculating a PHD h-index and characterizing the professional domains of citing journals. Next, we surveyed and interviewed PHD grantees, expert consultants, and codirectors to assess the program's components and the potential future application of design thinking to artificial intelligence (AI) integration into healthcare. RESULTS: There was a total of 1171 unique citations to PHD-funded work (collective h-index of 25). Studies citing PHD span medical, legal, and computational journals. Participants stated that this project transformed their thinking, altered their career trajectory, and resulted in technology transfer into the commercial sector. Participants felt, in general, that the approach would be valuable in solving contemporary challenges integrating AI in healthcare including complex social questions, integrating knowledge from multiple domains, implementation, and governance. CONCLUSION: Design thinking is a systematic approach to problem-solving characterized by cooperation and collaboration. PHD generated significant impacts as measured by citations, reach, and overall effect on participants. PHD's design thinking methods are potentially useful to other work on cyber-physical systems, such as the use of AI in healthcare, to propose structural or policy-related changes that may affect adoption, value, and improvement of the care delivery system.

OBJECTIVE: Successful technological implementations frequently involve individuals who serve as mediators between end users, management, and technology developers. The goal for this project was to evaluate the structure and activities of such mediators in a large-scale electronic health record implementation. MATERIALS AND METHODS: Field notes from observations taken during implementation beginning in November 2017 were analyzed qualitatively using a thematic analysis framework to examine the relationship between specific types of mediators and the type and level of support to end users. RESULTS: We found that support personnel possessing both contextual knowledge of the institution's workflow and training in the new technology were the most successful in mediation of adoption and use. Those that lacked context of either technology or institutional workflow often displayed barriers in communication, trust, and active problem solving. CONCLUSIONS: These findings suggest that institutional investment in technology training and explicit programs to foster skills in mediation, including roles for professionals with career development opportunities, prior to implementation can be beneficial in easing the pain of system transition.

Objective: Sickle cell disease (SCD) is a chronic condition affecting over 100 000 individuals in the United States, predominantly from vulnerable populations. Clinical practice guidelines, written for providers, have low adherence. This study explored knowledge about guidelines; desire for guidelines; and how technology could support guideline awareness and adherence, examining current technology uses, and user preferences to inform design of a patient-centered guidelines application in a chronic disease. Methods: This cross-sectional mixed-methods study involved semi-structured interviews, surveys, and focus groups of adolescents and adults with SCD. We evaluated interest, preferences, and anticipated benefits or barriers of a patient-centered adaptation of SCD practice guidelines; prospective technology uses for health; and barriers to technology utilization. Results: Forty-seven individuals completed surveys and interviews, and 39 participated in three separate focus groups. Most participants (91%) were unaware of SCD guidelines, but almost all (96%) expressed interest in a guidelines application, identifying benefits (knowledge, activation, individualization, and rewards), and barriers (poor information, low motivation, and resource limitations). Current technology health uses included information access, care coordination, and reminders about health-related actions. Prospective technology uses included informational messaging and timely alerts. Barriers to technology use included lack of interest, lack of utility, and preference for direct communication. Conclusions: This study's findings can inform the design of clinical practice guideline applications, suggesting a promising role for technology to engage patients, facilitate care decisions and actions, and improve outcomes.

OBJECTIVE: To describe nurses' preferences for the design of a probability-based clinical decision support (PB-CDS) tool for in-hospital clinical deterioration. METHODS: A convenience sample of bedside nurses, charge nurses, and rapid response nurses (n = 20) from adult and pediatric hospitals completed participatory design sessions with researchers in a simulation laboratory to elicit preferred design considerations for a PB-CDS tool. Following theme-based content analysis, we shared findings with user interface designers and created a low-fidelity prototype. RESULTS: Three major themes and several considerations for design elements of a PB-CDS tool surfaced from end users. Themes focused on "painting a picture" of the patient condition over time, promoting empowerment, and aligning probability information with what a nurse already believes about the patient. The most notable design element consideration included visualizing a temporal trend of the predicted probability of the outcome along with user-selected overlapping depictions of vital signs, laboratory values, and outcome-related treatments and interventions. Participants expressed that the prototype adequately operationalized requests from the design sessions. CONCLUSIONS: Participatory design served as a valuable method in taking the first step toward developing PB-CDS tools for nurses. This information about preferred design elements of tools that support, rather than interrupt, nurses' cognitive workflows can benefit future studies in this field as well as nurses' practice.

Designing patient-centered consumer health informatics (CHI) applications requires understanding and creating alignment with patients' and their family members' health-related activities, referred to here as 'patient work'. A patient work approach to CHI draws on medical social science and human factors engineering models and simultaneously attends to patients, their family members, activities, and context. A patient work approach extends existing approaches to CHI design that are responsive to patients' biomedical realities and personal skills and behaviors. It focuses on the embeddedness of patients' health management in larger processes and contexts and prioritizes patients' perspectives on illness management. Future research is required to advance (1) theories of patient work, (2) methods for assessing patient work, and (3) techniques for translating knowledge of patient work into CHI application design. Advancing a patient work approach within CHI is integral to developing and deploying consumer-facing technologies that are integrated with patients' everyday lives.

In their response to our original paper, “Transforming Consumer Health Informatics through a Patient Work Framework: Connecting Patients to Context,” Marceglia and colleagues propose an architecture that integrates the patient work framework into a higher-order framework linking consumer health informatics (CHI) applications and professional health information systems (designated by the authors as the health-Information Technology (IT) ecosystem).1 The purpose of our letter is threefold. First, we detail how an expanded understanding of the patient work framework already conceptually encompasses the larger contexts in which CHI use must occur. Second, we assert that meaningful application of the patient work perspective yields implications not only for integration with professional health information systems but also with the larger information infrastructures within the community. Third, we propose modifications to Marceglia and colleagues’ architecture to explicitly represent a “shared space” between CHI applications and professional health information systems; this space contains collaborative work and collaborative informatics. Our original patient work framework was intended to serve as a foundation for CHI design by enabling the understanding of people, their daily contexts, and their daily activities. As such, we limited the scope of our discussion to the immediate home and community environments of the patient. We agree, however, with Marceglia and colleagues that a deeper understanding of the macrostructures encompassing patient work is required. This understanding is necessary not only for specifying constraints on the design outcome, but also for generating creative design alternatives. Larger macrostructures are included in the human factor engineering and social science theories that form the core of the patient work framework; consequently, they are already conceptually embedded.2–4 These macrostructures include not only the larger technological infrastructure (including, but not limited to, the health-IT ecosystem specified by Marceglia and colleagues), but also the economic, regulatory, and policy landscapes. For example, awareness of insurance policies may lead designers to plan for a future in which CHI applications are covered entities. However, designers of applications for people with low socioeconomic status and less generous health insurance plans may need to pursue low-cost alternatives. Designing for populations that use older hardware or operating systems will similarly require technological challenges to be understood and addressed, such as the backward compatibility of mHealth applications and Short Message Service (SMS)-based alternatives. In the phase of conceptual design, the second step in our user-centered design process, integrating knowledge of these broader contexts is particularly salient. Although we agree that the patient work framework can produce valuable insights into connecting CHI applications with professional health information systems, we assert that its technical implications are, in fact, much broader. As an investigational framework, patient work also illuminates numerous areas of connection to a more broadly conceptualized health-IT ecosystem that includes the information infrastructures of patients’ homes and communities. To illustrate, we offer a few examples of design alternatives that build upon a recognition of this larger health-IT ecosystem. In the category “physical environment,” neighborhood walkability scores and safety information, along with patient health and social network data, may inform the development of strategies for integrating therapeutic physical activity into patients’ daily lives. Moreover, establishing a connection between patients and relevant environmental data may facilitate health management planning (e.g., providing a pollen count warning for patients with asthma triggered by seasonal allergies). Smart reminders may also help patients and their caregivers anticipate medication refills when medication usage is seasonally higher. Similarly, the analysis of data from the category “articulation work” may reveal transportation challenges, indicating that linking bus routes, schedules, and location data with patient reminders may assist with appointment-keeping. Long-term solutions may involve linking patients with data regarding insurers that provide transportation assistance or social media–facilitated informal transportation networks. This integrated approach moves beyond the vision of seamless exchange of patient health information to the seamless exchange of all information that, from the perspective of social determinants of health, shapes patient outcomes.5 Thus, technical architectures to support patient work may be infinitely more complex than those that facilitate data exchange between CHI applications and professional health information systems, despite the importance of efforts in that direction. The architecture proposed by Marceglia and colleagues creates a clear distinction between CHI applications and professional health information systems. However, we assert that patient and health care provider work are often jointly constructed and performed (e.g., patient-clinician communication/secure messaging, medication reconciliation, family-centered pediatric rounds, and cancer treatment planning). The clinical encounter is often grounded in patient narratives that relate the experience of self-management in the home. Similarly, patients’ self-management practices in the home are often influenced by discussions with providers in clinical settings. Patients’ health management practices in the home may also involve the presence of health professionals such as physical therapists, social workers, and home care nurses. Consequently, we contend that the proposed architecture should be expanded to contain a middle space for collaborative professional-patient work2 and informatics solutions (see Figure 1). Collaborative informatics solutions include tethered personal health records, which enable the sharing of both clinic-generated data with patients and patient-generated data with health care providers. We believe that this amendment to Marceglia and colleagues’ approach may yield promising directions for system design, focusing on the interconnectedness of and interactions between both sets of work systems. Informatics tools based on such an understanding would promote interoperability not only at the technological and semantic levels but also at the level of work processes. Three forms of work activity performed by professionals, patients, and families, and corresponding technologies (adapted from Holden et al. 20132 and Marceglia et al. 20141). Achieving the full vision of informatics-supported patient engagement will require understanding and designing solutions that integrate with the numerous macrostructures within which patient work is performed. It will require creating CHI applications that communicate not only with professional health information systems but also with the larger information infrastructures of the community. Finally, it will further necessitate creating informatics solutions that recognize the collaborative nature of work that ultimately maintains and improves health.

Public interest in the quality and safety of health care has spurred examination of specific organizational routines believed to yield risk in health care work. Medication administration routines, in particular, have been the subject of numerous improvement projects involving information technology development, and other forms of research and regulation. This study draws from ethnographic observation to examine how the common routine of medication administration intersects with other organizational routines, and why understanding such intersections is important. We present three cases describing intersections between medication administration and other routines, including a pharmacy routine, medication administration on the next shift and management reporting. We found that each intersection had ostensive and performative dimensions; and furthermore, that IT-enabled changes to one routine led to unintended consequences in its intersection with others, resulting in misalignment of ostensive and performative aspects of the intersection. Our analysis focused on the activities of a group of nurses who provide technology use mediation (TUM) before and after the rollout of a new health IT system. This research offers new insights on the intersection of organizational routines, demonstrates the value of analyzing TUM activities to better understand the relationship between IT introduction and changes in routines, and has practical implications for the implementation of technology in complex practice settings.

OBJECTIVE: Without careful attention to the work of users, implementation of health IT can produce new risks and inefficiencies in care. This paper uses the technology use mediation framework to examine the work of a group of nurses who serve as mediators of the adoption and use of a barcode medication administration (BCMA) system in an inpatient setting. MATERIALS AND METHODS: The study uses ethnographic methods to explore the mediators' work. Data included field notes from observations, documents, and email communications. This variety of sources enabled triangulation of findings between activities observed, discussed in meetings, and reported in emails. RESULTS: Mediation work integrated the BCMA tool with nursing practice, anticipating and solving implementation problems. Three themes of mediation work include: resolving challenges related to coordination, integrating the physical aspects of BCMA into everyday practice, and advocacy work. DISCUSSION: Previous work suggests the following factors impact mediation effectiveness: proximity to the context of use, understanding of users' practices and norms, credibility with users, and knowledge of the technology and users' technical abilities. We describe three additional factors observed in this case: 'influence on system developers,' 'influence on institutional authorities,' and 'understanding the network of organizational relationships that shape the users' work.' CONCLUSION: Institutionally supported clinicians who facilitate adoption and use of health IT systems can improve the safety and effectiveness of implementation through the management of unintended consequences. Additional research on technology use mediation can advance the science of implementation by providing decision-makers with theoretically durable, empirically grounded evidence for designing implementations.

The objective of this review was to describe methods used to study and model workflow. The authors included studies set in a variety of industries using qualitative, quantitative and mixed methods. Of the 6221 matching abstracts, 127 articles were included in the final corpus. The authors collected data from each article on researcher perspective, study type, methods type, specific methods, approaches to evaluating quality of results, definition of workflow and dependent variables. Ethnographic observation and interviews were the most frequently used methods. Long study durations revealed the large time commitment required for descriptive workflow research. The most frequently discussed technique for evaluating quality of study results was triangulation. The definition of the term "workflow" and choice of methods for studying workflow varied widely across research areas and researcher perspectives. The authors developed a conceptual framework of workflow-related terminology for use in future research and present this model for use by other researchers.

As health care organizations dramatically increase investment in information technology (IT) and the scope of their IT projects, implementation failures become critical events. Implementation failures cause stress on clinical units, increase risk to patients, and result in massive costs that are often not recoverable. At an estimated 28% success rate, the current level of investment defies management logic. This paper asserts that there are "chasms" in IT implementations that represent risky stages in the process. Contributors to the chasms are classified into four categories: design, management, organization, and assessment. The American College of Medical Informatics symposium participants recommend bold action to better understand problems and challenges in implementation and to improve the ability of organizations to bridge these implementation chasms. The bold action includes the creation of a Team Science for Implementation strategy that allows for participation from multiple institutions to address the long standing and costly implementation issues. The outcomes of this endeavor will include a new focus on interdisciplinary research and an inter-organizational knowledge base of strategies and methods to optimize implementations and subsequent achievement of organizational objectives.