OBJECTIVE: The use of ambient AI documentation tools is rapidly growing in US hospitals and clinics. Such tools generate the first draft of clinical notes from scribed patient-provider conversations, which clinicians can then review and edit before signing into electronic health records (EHR). Understanding how and why clinicians make modifications to AI-generated drafts is critical to improving AI design and clinical efficiency, yet it has been under-studied. This study aims to address this gap. MATERIALS AND METHODS: We conducted semistructured interviews with 30 clinicians from the University of California, Irvine Health who used a commercial ambient AI tool in routine outpatient care. We invited them to describe how and why they edited AI drafts based on both their personal experience and review of some real-world examples identified from our previous studies. RESULTS: Modifications to AI drafts were primarily made to improve clinical accuracy and specialty-specific precision, reduce medico-legal and liability risk, and meet billing, coding, and documentation standards. Such editing was necessary due to reasons such as transcription errors, speaker attribution mistakes, overconfident statements without evidence, missing key clinical details, and AI's lack of information about the patient context. CONCLUSION AND DISCUSSION: Improving ambient AI documentation will require coordinated effort from vendors, institutions, and clinicians. Key targets include core model reliability (eg, transcription accuracy), specialty- and encounter-level customization, clinician-level personalization, more effective EHR integration, and institutional support (eg, training, governance, and standardized review guidance), complemented by clinicians' adaptive communication strategies that strengthen human-AI collaboration.

OBJECTIVE: Ambient artificial intelligence (AI) documentation is increasingly used to draft clinical notes from patient-provider conversations, but how clinicians revise and finalize these drafts is not well understood. This qualitative content analysis study characterizes real-world edits to AI-generated drafts and identifies opportunities for improvement of AI design and the implementation process. MATERIALS AND METHODS: Eight coders analyzed clinical documentation generated by ambient AI from 200 clinical encounters. We developed an inductive coding framework with 11 codes across 3 categories: clinical content, terminology, and language style. Interrater reliability was assessed using Cohen's kappa. We then applied thematic analysis to synthesize patterns across the coded edits. RESULTS: The most frequently edited content pertained to clinical facts including orders (eg, procedures, lab tests) (40.0%), symptoms (30.3%), medication prescriptions (27.3%), and diagnosis descriptions (25.9%). In comparison, edits related to terminology use (11.6%) and language style (7.2%) were less frequent. The results of our thematic analysis show that most edits can be categorized into one of the following 5 types: to revise factual discrepancies, to add medical specialty-specific details, to express diagnostic certainties, to convert patient expressions into objective assessments recorded in medical terms, and to reorganize or condense content. CONCLUSION AND DISCUSSION: Clinicians routinely revise ambient AI drafts to modify factual details and clinical specificity. Future work on AI development and clinical implementation should emphasize specialty customization and support personalized documentation practices, alongside clinician education that promotes robust and consistent review routines to ensure documentation quality.

BACKGROUND AND SIGNIFICANCE: Ambient listening tools powered by generative artificial intelligence (GenAI) offer real-time, scribe-like support that reduce documentation burden and may help alleviate burnout. This study assesses physician-perceived benefits and challenges of ambient AI implementation through surveys and evaluates its effectiveness in clinical workflows using automatically recorded electronic health record (EHR) time-efficiency metrics. METHOD AND MATERIALS: A quality improvement pilot has been underway at UCI Health since December 2023. Epic EHR Signal metrics were analyzed to assess changes in note length, documentation time, and same-day encounter closure rates. Matched pre- and post-implementation surveys evaluated physician-perceived changes in documentation burden, clinical efficiency, and care quality. We also examined open-ended survey responses using thematic analysis to supplement quantitative findings. RESULTS: Analysis on EHR usage data from 167 physicians showed significant reductions in note-writing time, despite an increase in note length. Survey responses (n = 65) also indicated statistically significant improvements across multiple domains. Physicians reported reduced cognitive demand (P = .031) and documentation effort (P = .014), alongside perceptions of enhanced clinical efficiency, patient-centered care, and EHR system usability. Thematic analysis confirmed these quantitative findings and identified opportunities for improvement, including specialty-specific customization and expanded AI functionality. DISCUSSION: Ambient AI tools demonstrated improved documentation efficiency, perceived care quality, and reduced cognitive workload. These benefits suggest potential to alleviate key burdens in clinical documentation. CONCLUSION: Future development should prioritize customization for specialty-specific and individual physician needs, ensure the reliability and accuracy of AI-generated content, and integrate ethical and legal considerations to facilitate safe and scalable implementation in patient-centered care contexts.

OBJECTIVES: We report on findings from a meeting convened by the American College of Medical Informatics (ACMI) to characterize aspects of the patient experience that could be improved using informatics. MATERIALS AND METHODS: The American College of Medical Informatics fellows were invited to share their experiences as patients and suggest informatics approaches that may improve the patient experience. RESULTS: We identified 4 themes: (1) getting the right care, (2) data sharing and data interoperability, (3) guiding low-cost evaluations, and (4) predictive analytics. DISCUSSION: Despite widespread adoption of health IT, patient experiences remain far from optimal. CONCLUSION: The American College of Medical Informatics fellows identified informatics approaches, applications, and research areas that have the potential to improve patient experiences with health care systems.

OBJECTIVES: To detail patient challenges, and how technology support addressed them, in a remote patient activation intervention for hemodialysis patients (n = 93) from trained patient mentors (n = 26). MATERIALS AND METHODS: Using digital divide theory-derived codes, content analysis of: technology support program delivery data, hemodialysis clinic staff interviews, and support staff reflection papers. Descriptive statistics from postintervention mentee/mentor surveys. RESULTS: All mentees and 46.2% of mentors received support. Motivational access was targeted with explanations, rapport, and support availability. Study-provided, data-capable tablets enhanced material access, but internet access barriers persisted. Skills access was addressed by training; password-related challenges initially dominated. For usage access, on-demand technology support was balanced by engagement support: proactive prementoring session calls and login monitoring. DISCUSSION: Interventionists should examine internet coverage in targeted areas, potentially using multiple carriers. A balance between password usability and security is required. Engagement support may be needed. CONCLUSION: Technology support can close patient digital divides.

Patient-controlled data-sharing systems are increasingly promoted as a way to empower patients with greater autonomy over their health data. Yet it remains unclear how different stakeholders, especially patients and health system leaders, perceive the benefits and challenges of enabling granular control over the sharing of de-identified medical data for research. To address this gap, we developed a high-fidelity prototype of a patient-controlled, web-based consent platform and conducted a two-phase mixed-methods study: semi-structured interviews with 16 health system leaders and a survey with 523 patient participants. While both groups appreciated the potential of such a platform to enhance transparency and autonomy, their views diverged in meaningful ways. Leaders viewed transparency and granular control through the lens of informed consent and institutional ethics, whereas patients interpreted these factors as safeguards against potential risks and uncertainties. Our findings underscore critical tensions such as individual control and research integrity. We offer design implications for building trustworthy, context-aware systems that support flexible granularity, provide ongoing benefit‑centered transparency, and adapt to diverse literacy and privacy needs.

OBJECTIVE: We aimed to develop a distributed, immutable, and highly available cross-cloud blockchain system to facilitate federated data analysis activities among multiple institutions. MATERIALS AND METHODS: We preprocessed 9166 COVID-19 Structured Query Language (SQL) code, summary statistics, and user activity logs, from the GitHub repository of the Reliable Response Data Discovery for COVID-19 (R2D2) Consortium. The repository collected local summary statistics from participating institutions and aggregated the global result to a COVID-19-related clinical query, previously posted by clinicians on a website. We developed both on-chain and off-chain components to store/query these activity logs and their associated queries/results on a blockchain for immutability, transparency, and high availability of research communication. We measured run-time efficiency of contract deployment, network transactions, and confirmed the accuracy of recorded logs compared to a centralized baseline solution. RESULTS: The smart contract deployment took 4.5 s on an average. The time to record an activity log on blockchain was slightly over 2 s, versus 5-9 s for baseline. For querying, each query took on an average less than 0.4 s on blockchain, versus around 2.1 s for baseline. DISCUSSION: The low deployment, recording, and querying times confirm the feasibility of our cross-cloud, blockchain-based federated data analysis system. We have yet to evaluate the system on a larger network with multiple nodes per cloud, to consider how to accommodate a surge in activities, and to investigate methods to lower querying time as the blockchain grows. CONCLUSION: Blockchain technology can be used to support federated data analysis among multiple institutions.

OBJECTIVES: Ambient clinical documentation technology uses automatic speech recognition (ASR) and natural language processing (NLP) to turn patient-clinician conversations into clinical documentation. It is a promising approach to reducing clinician burden and improving documentation quality. However, the performance of current-generation ASR remains inadequately validated. In this study, we investigated the impact of non-lexical conversational sounds (NLCS) on ASR performance. NLCS, such as Mm-hm and Uh-uh, are commonly used to convey important information in clinical conversations, for example, Mm-hm as a "yes" response from the patient to the clinician question "are you allergic to antibiotics?" MATERIALS AND METHODS: In this study, we evaluated 2 contemporary ASR engines, Google Speech-to-Text Clinical Conversation ("Google ASR"), and Amazon Transcribe Medical ("Amazon ASR"), both of which have their language models specifically tailored to clinical conversations. The empirical data used were from 36 primary care encounters. We conducted a series of quantitative and qualitative analyses to examine the word error rate (WER) and the potential impact of misrecognized NLCS on the quality of clinical documentation. RESULTS: Out of a total of 135 647 spoken words contained in the evaluation data, 3284 (2.4%) were NLCS. Among these NLCS, 76 (0.06% of total words, 2.3% of all NLCS) were used to convey clinically relevant information. The overall WER, of all spoken words, was 11.8% for Google ASR and 12.8% for Amazon ASR. However, both ASR engines demonstrated poor performance in recognizing NLCS: the WERs across frequently used NLCS were 40.8% (Google) and 57.2% (Amazon), respectively; and among the NLCS that conveyed clinically relevant information, 94.7% and 98.7%, respectively. DISCUSSION AND CONCLUSION: Current ASR solutions are not capable of properly recognizing NLCS, particularly those that convey clinically relevant information. Although the volume of NLCS in our evaluation data was very small (2.4% of the total corpus; and for NLCS that conveyed clinically relevant information: 0.06%), incorrect recognition of them could result in inaccuracies in clinical documentation and introduce new patient safety risks.

While easy patient access to laboratory test results is necessary for patient engagement in their healthcare, it is not sufficient to enable patients to thrive in this role ? they must be able to understand and act upon these data. In this study, we analyzed posts to an online health community (OHC) that contained a patient's laboratory test results. The objective was to understand the nature of patients' questions related to these data to gain insights into how to better support patients as they individually and collaboratively make sense of their data. We found that patients seek help on the OHC to understand and use their laboratory test results at multiple points in the medical care trajectory. Specifically, in the diagnosis phase, patients tend to be focused on comprehending their data, to be receiving emotionally charged results and, of course, to be engaging the OHC in naming their medical issues. In the treatment phase, patients are often using their laboratory test results to ask more focused questions to identify treatment options, to seek treatment guidance from peers, and to predict the likely course of their disease. Throughout both phases, individuals are highly engaged in the medical process and put in substantial effort to proactively prepare for their care and interactions with doctors. They enlist the OHC in these efforts for many reasons such as a lack of confidence in their doctor. We discuss how gaps in the provision of healthcare services lead to the significant work involved in managing the complex and dynamic interplay between OHCs and the healthcare system. We offer design recommendations both for technologies that provide patients with access to their medical records and for OHCs that will likely continue to play an important role in filling gaps in healthcare services.

Patient-generated health data (PGHD) is crucial for healthcare providers' decision making, as it complements clinical data by providing a more holistic view of patients' daily conditions. We interviewed 20 healthcare providers in asthma care to envision future technologies to support their PGHD use. We found that healthcare providers want future artificial intelligence (AI) systems to enhance their ability to treat patients by analyzing PGHD for profiling risk and predicting deterioration. Despite the potential benefits of AI, providers perceived various challenges of AI use with PGHD, including AI-driven data inequity, added burden, lack of trust toward AI, and fear of being replaced by AI. Clinicians wished for a future of co-dependent human-AI collaboration, where AI will help them to improve their clinical practice. In turn, healthcare providers can improve AI systems by making AI outputs more trustworthy and humane. Through the lens of data feminism, we discuss the importance of considering context and aligning the complex human infrastructure before designing or deploying PGHD-based AI systems in clinical settings. We highlight the opportunity to design for human-AI enrichment, where humans and AI not only partner with each other for improved performance, but also enrich each other to enhance each other's work overtime.

OBJECTIVE: Facial masks are an essential personal protective measure to fight the COVID-19 (coronavirus disease) pandemic. However, the mask adoption rate in the United States is still less than optimal. This study aims to understand the beliefs held by individuals who oppose the use of facial masks, and the evidence that they use to support these beliefs, to inform the development of targeted public health communication strategies. MATERIALS AND METHODS: We analyzed a total of 771 268 U.S.-based tweets between January to October 2020. We developed machine learning classifiers to identify and categorize relevant tweets, followed by a qualitative content analysis of a subset of the tweets to understand the rationale of those opposed mask wearing. RESULTS: We identified 267 152 tweets that contained personal opinions about wearing facial masks to prevent the spread of COVID-19. While the majority of the tweets supported mask wearing, the proportion of anti-mask tweets stayed constant at about a 10% level throughout the study period. Common reasons for opposition included physical discomfort and negative effects, lack of effectiveness, and being unnecessary or inappropriate for certain people or under certain circumstances. The opposing tweets were significantly less likely to cite external sources of information such as public health agencies' websites to support the arguments. CONCLUSIONS: Combining machine learning and qualitative content analysis is an effective strategy for identifying public attitudes toward mask wearing and the reasons for opposition. The results may inform better communication strategies to improve the public perception of wearing masks and, in particular, to specifically address common anti-mask beliefs.

OBJECTIVE: Sentiment analysis is a popular tool for analyzing health-related social media content. However, existing studies exhibit numerous methodological issues and inconsistencies with respect to research design and results reporting, which could lead to biased data, imprecise or incorrect conclusions, or incomparable results across studies. This article reports a systematic analysis of the literature with respect to such issues. The objective was to develop a standardized protocol for improving the research validity and comparability of results in future relevant studies. MATERIALS AND METHODS: We developed the Protocol of Analysis of senTiment in Health (PATH) based on a systematic review that analyzed common research design choices and how such choices were made, or reported, among eligible studies published 2010-2019. RESULTS: Of 409 articles screened, 89 met the inclusion criteria. A total of 16 distinctive research design choices were identified, 9 of which have significant methodological or reporting inconsistencies among the articles reviewed, ranging from how relevance of study data was determined to how the sentiment analysis tool selected was validated. Based on this result, we developed the PATH protocol that encompasses all these distinctive design choices and highlights the ones for which careful consideration and detailed reporting are particularly warranted. CONCLUSIONS: A substantial degree of methodological and reporting inconsistencies exist in the extant literature that applied sentiment analysis to analyzing health-related social media data. The PATH protocol developed through this research may contribute to mitigating such issues in future relevant studies.

OBJECTIVE: To utilize, in an individual and institutional privacy-preserving manner, electronic health record (EHR) data from 202 hospitals by analyzing answers to COVID-19-related questions and posting these answers online. MATERIALS AND METHODS: We developed a distributed, federated network of 12 health systems that harmonized their EHRs and submitted aggregate answers to consortia questions posted at https://www.covid19questions.org. Our consortium developed processes and implemented distributed algorithms to produce answers to a variety of questions. We were able to generate counts, descriptive statistics, and build a multivariate, iterative regression model without centralizing individual-level data. RESULTS: Our public website contains answers to various clinical questions, a web form for users to ask questions in natural language, and a list of items that are currently pending responses. The results show, for example, that patients who were taking angiotensin-converting enzyme inhibitors and angiotensin II receptor blockers, within the year before admission, had lower unadjusted in-hospital mortality rates. We also showed that, when adjusted for, age, sex, and ethnicity were not significantly associated with mortality. We demonstrated that it is possible to answer questions about COVID-19 using EHR data from systems that have different policies and must follow various regulations, without moving data out of their health systems. DISCUSSION AND CONCLUSIONS: We present an alternative or a complement to centralized COVID-19 registries of EHR data. We can use multivariate distributed logistic regression on observations recorded in the process of care to generate results without transferring individual-level data outside the health systems.

Burnout among healthcare providers has been increasingly recognized as a significant problem.1 The National Academy of Medicine has defined burnout as “a syndrome characterized by high emotional exhaustion, high depersonalization (ie, cynicism), and a low sense of personal accomplishment from work.”2 The Agency for Healthcare Research and Quality similarly defines Burnout as a long-term stress reaction marked by emotional exhaustion, depersonalization, and a lack of sense of personal accomplishment.3 Clinician burnout is both costly and has been associated with reduced job satisfaction, quality and safety of care, and patient health outcomes.4 Burnout is common—affecting between 35% and 54% of U.S. nurses and physicians and between 45% and 60% for medical students and residents.2 Research to date has identified a number of contributing factors as associated with burnout.5 Among these, health information technologies (HITs) are often implicated. Electronic health record (EHR) systems, for example, are often seen as cumbersome to use, failing to fulfill the promise of improved healthcare delivery, and little more than a means of meeting regulatory and billing requirements.6 However, there remains considerable debate in the informatics community as to the actual role health information technologies play in the problem of clinician burnout.7 Existing research suggests that technologies may be confounded with other important causes, including regulatory mandates, clinical volumes, increasing hyperspecialization among healthcare providers, and a mismatch between the incentives driving system designers and purchasers and those driving providers. Regardless of the role information technology plays in clinician burnout, innovative solutions to prevent or mitigate burnout are urgently needed. In this special focus issue of Journal of the American Medical Informatics Association, we target articles evaluating the role that health information technologies have in causing and mitigating burnout, identify confounding factors, and consider informatics and policy-based solutions. This special focus issue is an outgrowth of the 2020 American College of Medical Informatics Symposium, “Clinician Burnout: Is it Informatics’ Fault, and What Can We Do About It?” In parallel with this special issue, the American College of Medical Informatics (ACMI) Symposium also led to the 25x5 initiative,8 to reduce the burden imposed by clinical documentation on healthcare providers in the United States to 25% of its current level within 5 years. The 25x5 initiative in turn resulted in a National Library of Medicine–funded 6-week symposium that concluded in February 2021. Follow-on work will lay out concrete steps that can be taken to reduce burden, create a community of like-minded stakeholders, and will work with key organizations and associations to guide this change. Taken together, the 2020 ACMI symposium, the 25x5 initiative, and this special issue comprise concrete steps the informatics community is taking to address the problem of clinician burnout. This special issue includes 24 articles across the variety of JAMIA formats: Research and Applications (n = 6),9–14 Brief Communications (n = 4),7,15–17 Reviews (n = 5),18–22 and Perspectives (n = 9).23–31 In the following paragraphs, we summarize selected papers reflecting 3 key themes: (1) understanding the relationship between HIT and clinician burnout, (2) emerging HIT approaches to mitigate clinician burnout, and (3) future directions. Several articles in this special focus issue anchor our understanding of the relationship between HIT and clinician burnout. Two review articles, one by Yan et al21 and another by Nguyen et al19 both identified documentation burden, high inbox message volumes, and negative perceptions of EHR functionality and usability as key EHR-related factors most consistently associated with objective measures of provider burnout in the extant literature. Both review articles also identified time spent on EHR after work hours—often called “pajama time”—but not total time spent on EHR, as being associated with burnout. Findings from these review articles point to opportunities for HIT systems to identify clinicians at increased risk of burnout. Baxter et al16 demonstrated that 3 leading EHR vendors currently provide “off-the-shelf” metrics to measure provider activity on the EHR through log data. While further work is needed to harmonize the metrics’ definitions so that meaningful cross-vendor comparisons can be made, these measures are now routinely available to healthcare organizations and informatics researchers. Two articles in this special issue offer practical insights on how these metrics could be used to identify the subset of clinicians at elevated risk of burnout and to target burnout mitigation interventions. Eschenroeder et al15 analyzed data from the KLAS Arch Collaborative and found that physicians who spend 6 or more hours per week performing after-hours charting were more likely to report burnout. Similarly, Peccoralo et al14 found using survey data from a single institution that faculty members who used EHR for more than 90 minutes a day after hours or who spent more than 60 minutes a day performing clerical tasks were more likely to report burnout. Taken together, these findings suggest that risk of burnout for full-time clinicians may rise significantly if they spend more than 60 to 90 minutes per day on the EHR after hours. Articles in this special issue also highlight opportunities to leverage HIT to address the pervasive problem of clinician burnout. Several contributions build on the evidence base for approaches that healthcare organizations could adopt. Lourie et al12 reported that personalized customization and training sessions across 14 specialty and 31 primary care ambulatory care practices led to improved self-reported efficiency and burnout perception. Simpson et al17 found that a 2-week EHR optimization sprint consisting of EHR changes and one-on-one training sessions led to an improvement in clinicians’ satisfaction toward the EHR in a single-specialty practice but did not impact measures of emotional burnout. A qualitative study conducted by Tran et al13 found that medical scribes are commonly used to offload 7 categories of clinical or clerical tasks as a way to a alleviate burnout attributable to the use of HIT. While these approaches require dedicated resources, these articles should help healthcare organizations build the case for these investments. As suboptimal EHR usability has often been cited as a significant contributor to clinician burnout in the United States, the editors of this special focus issue invited key EHR vendors and usability experts to elucidate current approaches to and opportunities for vendors to improve EHR usability. Leading EHR vendors were invited to respond to a semi-structured written survey on how they meet or exceed the 2015 EHR usability (or user-centered design) requirements issued by the ONC (https://www.healthit.gov/test-method/safety-enhanced-design#ccg). Anonymized responses from 4 major vendors (Supplementary Appendix) were sent to usability experts to comment on the strengths and weaknesses adopted by the EHR industry, and to suggest improvement opportunities. When compared to research from 2015 on the usability of EHR products, Hettinger et al23 noted that vendors have increased their adoption and maturity of user-centered design practices. This observation highlighted the ongoing efforts from U.S. federal policy makers, as summarized by Gettinger et al25 to promote HIT usability by implementing usability standards and funding research to examine the efficacy of these policies. However, much work remains. Hettinger et al highlighted the usability reality gap between EHR as designed by vendors and EHR as implemented by each healthcare organization, citing the paucity of the workforce trained to optimally configure and usability or safety test local configurations as a key driver of this gap. Carayon and Salwei24 further pointed out that the path for reducing clinician burnout through improving EHR usability requires a continuous approach, as vendors and their clients need to work together to turn their focus away from technology embedded in work-as-imagined toward sociotechnical systems supporting work-as-done. EHR vendors should also recognize that they can and should partner with informatics innovators to advance EHR usability and mitigate clinician burnout. In a block-randomized study, Semanik et al11 found that problem-oriented summaries of clinical data, built directly into a vendor EHR, allowed clinicians across three academic medical centers to retrieve data faster and with fewer errors. With the use of this tool, clinicians also reported a reduced cognitive load and increased satisfaction. This article by Semanik et al demonstrates how EHR vendors could support efforts to mitigate burnout by spreading and sustaining usability innovations coming from an individual customer across their customer base. So where does the topic of informatics and clinician burnout go from here? Perspectives articles from several ACMI members offer new lenses through which to view, understand, and address HIT-associated clinician burnout. Williams30 contended that moral injury associated with EHR, as defined by EHR use that leads clinicians to transgress deeply held moral beliefs and expectations, may be a hidden contributor to clinician burnout. Weir et al31 further postulated that burnout may be linked to drivers of intrinsic motivation, and that goal-based decision making, sense making, and agency or autonomy should be considered in the design of future technological interventions to mitigate clinician burnout. From a methodological perspective, significant opportunities remain. Moy et al20 pointed out in their scoping review that standard and validated measures of documentation burden are still lacking, which in turn forms a barrier to the rigorous study of documentation burden. Moy et al further called for efforts to operationalize the concept of documentation burden and develop best practices for measurement. Kannampallil et al26 proposed a conceptual framework that would allow the informatics community to build on EHR activity measures evaluated by Baxter et al16 and use technology to assess holistically clinicians’ workload, cognitive burden, and well-being. The editors of this JAMIA special issue recognize that this body of work is but a snapshot of a rapidly growing and evolving topic. New technologies such as ambient voice speech to text, internet of things, natural language processing and machine learning–driven data visualization, and Fast Healthcare Interoperability Resources, as highlighted by Dymek et al28 and Gettinger and Zayas-Cabán,25 may yet open up opportunities to support more meaningful clinician-patient interactions and more efficient workflows. The policy landscape is also constantly changing, as evidenced by the recent simplification in documentation requirements initiated by the Centers for Medicare and Medicaid Services Burden Reduction efforts intended to place “Patients Over Paperwork.”32 As a target for multidisciplinary scientific inquiry, the subject of HIT-associated clinician burnout must continue to evolve through future empirical studies. Its current evidence base remains modest, at best. We therefore encourage readers of this special issue to participate in and accelerate the ongoing work in this area. STR, KZ, and EGP presented the special issue proposal to JAMIA; STR, KZ, and EGP fulfilled Associate Editor duties; EGP fulfilled EIC duties; EGP contributed to instrument design, data collection from vendors and commissioning of invited perspectives articles; and EGP, STR, and KZ contributed to drafting and finalization of the editorial. Supplementary material is available at Journal of the American Medical Informatics Association online. The authors have no relevant conflicts of interest to declare.

OBJECTIVES: To understand how medical scribes' work may contribute to alleviating clinician burnout attributable directly or indirectly to the use of health IT. MATERIALS AND METHODS: Qualitative analysis of semistructured interviews with 32 participants who had scribing experience in a variety of clinical settings. RESULTS: We identified 7 categories of clinical tasks that clinicians commonly choose to offload to medical scribes, many of which involve delegated use of health IT. These range from notes-taking and computerized data entry to foraging, assembling, and tracking information scattered across multiple clinical information systems. Some common characteristics shared among these tasks include: (1) time-consuming to perform; (2) difficult to remember or keep track of; (3) disruptive to clinical workflow, clinicians' cognitive processes, or patient-provider interactions; (4) perceived to be low-skill "clerical" work; and (5) deemed as adding no value to direct patient care. DISCUSSION: The fact that clinicians opt to "outsource" certain clinical tasks to medical scribes is a strong indication that performing these tasks is not perceived to be the best use of their time. Given that a vast majority of healthcare practices in the US do not have the luxury of affording medical scribes, the burden would inevitably fall onto clinicians' shoulders, which could be a major source for clinician burnout. CONCLUSIONS: Medical scribes help to offload a substantial amount of burden from clinicians-particularly with tasks that involve onerous interactions with health IT. Developing a better understanding of medical scribes' work provides useful insights into the sources of clinician burnout and potential solutions to it.

OBJECTIVE: Hemodialysis patients frequently experience dialysis therapy sessions complicated by intradialytic hypotension (IDH), a major patient safety concern. We investigate user-centered design requirements for a theory-informed, peer mentoring-based, informatics intervention to activate patients toward IDH prevention. METHODS: We conducted observations (156 hours) and interviews (n = 28) with patients in 3 hemodialysis clinics, followed by 9 focus groups (including participatory design activities) with patients (n = 17). Inductive and deductive analyses resulted in themes and design principles linked to constructs from social, cognitive, and self-determination theories. RESULTS: Hemodialysis patients want an informatics intervention for IDH prevention that collapses distance between patients, peers, and family; harnesses patients' strength of character and resolve in all parts of their life; respects and supports patients' individual needs, preferences, and choices; and links "feeling better on dialysis" to becoming more involved in IDH prevention. Related design principles included designing for: depth of interpersonal connections; positivity; individual choice and initiative; and comprehension of connections and possible actions. DISCUSSION: Findings advance the design of informatics interventions by presenting design requirements for outpatient safety and addressing key design opportunities for informatics to support patient involvement; these include incorporation of behavior change theories. Results also demonstrate the meaning of design choices for hemodialysis patients in the context of their experiences; this may have applicability to other populations with serious illnesses. CONCLUSION: The resulting patient-facing informatics intervention will be evaluated in a pragmatic cluster-randomized controlled trial in 28 hemodialysis facilities in 4 US regions.

Journal of the American Medical Informatics Association, ocab033, https://doi.org/10.1093/jamia/ocab033 In the originally publication of this article, co-author Leah Hein was omitted from the authorship list. This should read: “Matthew A Willis, Leah Brand Hein, Zhaoxian Hu, Rajiv Saran, Marissa Argentina, Jennifer Bragg-Gresham, Sarah L Krein, Brenda Gillespie, Kai Zheng, Tiffany C Veinot” instead of “Matthew A Willis, Zhaoxian Hu, Rajiv Saran, Marissa Argentina, Jennifer Bragg-Gresham, Sarah L Krein, Brenda Gillespie, Kai Zheng, Tiffany C Veinot.” The AUTHOR CONTRIBUTIONS statement also lacked details. This should read: “TV supervised the project and study design, TV designed data collection instruments for the observations and individual interviews, and LH designed focus group data collection instruments. TV, LH, and MA collected data, and LH conducted first-cycle data analysis. ZH designed ideas for the application and developed the application. MAW and TV conducted the literature review, analyzed data, developed the theoretical framework, and drafted the manuscript and revisions. RS, MA, JBG, SLK, BG, and KZ provided input into design of the study, substantive review, and final approval.” instead of “TV supervised the project and study design. MAW and TV conducted literature review, drafted the manuscript and revisions, analyzed data, and developed the theoretical framework. ZH designed ideas for the application and developed the application. TV and MA collected data. RS, MA, JBG, SLK, BG, and KZ contributed input into design of the study, substantive review, and final approval.” These errors have now been corrected.

Abstract Personal informatics tools can help users self-reflect on their experiences. When reflective thought occurs, it sometimes leads to negative thought and emotion cycles. To help explain these cycles, we draw from Psychology to introduce the concept of rumination—anxious, perseverative cognition focused on negative aspects of the self—as a result of engaging with personal data. Rumination is an important concept for the Human Computer Interaction community because it can negatively affect users’ well-being and lead to maladaptive use. Thus, preventing and mitigating rumination is beneficial. In this conceptual paper, we differentiate reflection from rumination. We also explain how self-tracking technologies may inadvertently lead to rumination and the implications this has for design. Our goal is to expand self-tracking research by discussing these negative cycles and encourage researchers to consider rumination when studying, designing, and promoting tools to prevent adverse unintended consequences among users.

OBJECTIVE: Use of medical scribes reduces clinician burnout by sharing the burden of clinical documentation. However, medical scribes are cost-prohibitive for most settings, prompting a growing interest in developing ambient, speech-based technologies capable of automatically generating clinical documentation based on patient-provider conversation. Through a systematic review, we aimed to develop a thorough understanding of the work performed by medical scribes in order to inform the design of such technologies. MATERIALS AND METHODS: Relevant articles retrieved by searching in multiple literature databases. We conducted the screening process following the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) in guidelines, and then analyzed the data using qualitative methods to identify recurring themes. RESULTS: The literature search returned 854 results, 65 of which met the inclusion criteria. We found that there is significant variation in scribe expectations and responsibilities across healthcare organizations; scribes also frequently adapt their work based on the provider's style and preferences. Further, scribes' job extends far beyond capturing conversation in the exam room; they also actively interact with patients and the care team and integrate data from other sources such as prior charts and lab test results. DISCUSSION: The results of this study provide several implications for designing technologies that can generate clinical documentation based on naturalistic conversations taking place in the exam room. First, a one-size-fits-all solution will be unlikely to work because of the significant variation in scribe work. Second, technology designers need to be aware of the limited role that their solution can fulfill. Third, to produce comprehensive clinical documentation, such technologies will likely have to incorporate information beyond the exam room conversation. Finally, issues of patient consent and privacy have yet to be adequately addressed, which could become paramount barriers to implementing such technologies in realistic clinical settings. CONCLUSIONS: Medical scribes perform complex and delicate work. Further research is needed to better understand their roles in a clinical setting in order to inform the development of speech-based clinical documentation technologies.

Hemodialysis is life-saving therapy for end-stage renal disease; yet, 20% of hemodialysis sessions are complicated by intradialytic hypotension ("IDH"). There is a need for approaches to preventing IDH that account for their implementation contexts. Using Activity Theory, we outline the design of a digital diagnostic checklist to identify patients at risk of IDH. Checklists were chosen a priori as an outcome due to prior evidence of effectiveness. Drawing on individual interviews with 20 clinicians and three focus groups with 17 patients, we describe four activity systems within hemodialysis care. We then outline a novel design process that includes co-design activities with clinicians, and four rapid-cycle iterations that progressively incorporated activity system elements into checklist design. We contribute a new type of checklist design to HCI: one that supports diagnostic thinking rather than consistent task completion. We further broaden checklist design by including a formal role for patients in checklist completion.

OBJECTIVE: Alert fatigue limits the effectiveness of medication safety alerts, a type of computerized clinical decision support (CDS). Researchers have suggested alternative interactive designs, as well as tailoring alerts to clinical roles. As examples, alerts may be tiered to convey risk, and certain alerts may be sent to pharmacists. We aimed to evaluate which variants elicit less alert fatigue. MATERIALS AND METHODS: We searched for articles published between 2007 and 2017 using the PubMed, Embase, CINAHL, and Cochrane databases. We included articles documenting peer-reviewed empirical research that described the interactive design of a CDS system, to which clinical role it was presented, and how often prescribers accepted the resultant advice. Next, we compared the acceptance rates of conventional CDS-presenting prescribers with interruptive modal dialogs (ie, "pop-ups")-with alternative designs, such as role-tailored alerts. RESULTS: Of 1011 articles returned by the search, we included 39. We found different methods for measuring acceptance rates; these produced incomparable results. The most common type of CDS-in which modals interrupted prescribers-was accepted the least often. Tiering by risk, providing shortcuts for common corrections, requiring a reason to override, and tailoring CDS to match the roles of pharmacists and prescribers were the most common alternatives. Only 1 alternative appeared to increase prescriber acceptance: role tailoring. Possible reasons include the importance of etiquette in delivering advice, the cognitive benefits of delegation, and the difficulties of computing "relevance." CONCLUSIONS: Alert fatigue may be mitigated by redesigning the interactive behavior of CDS and tailoring CDS to clinical roles. Further research is needed to develop alternative designs, and to standardize measurement methods to enable meta-analyses.

Objective: Medical word sense disambiguation (WSD) is challenging and often requires significant training with data labeled by domain experts. This work aims to develop an interactive learning algorithm that makes efficient use of expert's domain knowledge in building high-quality medical WSD models with minimal human effort. Methods: We developed an interactive learning algorithm with expert labeling instances and features. An expert can provide supervision in 3 ways: labeling instances, specifying indicative words of a sense, and highlighting supporting evidence in a labeled instance. The algorithm learns from these labels and iteratively selects the most informative instances to ask for future labels. Our evaluation used 3 WSD corpora: 198 ambiguous terms from Medical Subject Headings (MSH) as MEDLINE indexing terms, 74 ambiguous abbreviations in clinical notes from the University of Minnesota (UMN), and 24 ambiguous abbreviations in clinical notes from Vanderbilt University Hospital (VUH). For each ambiguous term and each learning algorithm, a learning curve that plots the accuracy on the test set against the number of labeled instances was generated. The area under the learning curve was used as the primary evaluation metric. Results: Our interactive learning algorithm significantly outperformed active learning, the previous fastest learning algorithm for medical WSD. Compared to active learning, it achieved 90% accuracy for the MSH corpus with 42% less labeling effort, 35% less labeling effort for the UMN corpus, and 16% less labeling effort for the VUH corpus. Conclusions: High-quality WSD models can be efficiently trained with minimal supervision by inviting experts to label informative instances and provide domain knowledge through labeling/highlighting contextual features.

Timely access to professional medical advice is crucial for patient health outcomes. Traditional offline, one-on-one patient-provider interactions are time consuming and costly. As a result, online "Ask the doctor" (AtD) services have become increasingly popular, where patients and caregivers can obtain advice from medical professionals at a lower information and transaction cost. In this paper, we present an empirical study of Fenda, an innovative AtD platform recently introduced in China where patients and caregivers can consult a wide variety of healthcare professionals for a small fee. Using qualitative research methods, we analyzed how patients and caregivers interact with medical professionals on this platform, focusing on the nature of the questions asked and user strategies to optimize the usage of the platform. We further derived implications for designing better online AtD services connecting patients and caregivers to medical professionals.

Effectively making sense of short texts is a critical task for many real world applications such as search engines, social media services, and recommender systems. The task is particularly challenging as a short text contains very sparse information, often too sparse for a machine learning algorithm to pick up useful signals. A common practice for analyzing short text is to first expand it with external information, which is usually harvested from a large collection of longer texts. In literature, short text expansion has been done with all kinds of heuristics. We propose an end-to-end solution that automatically learns how to expand short text to optimize a given learning task. A novel deep memory network is proposed to automatically find relevant information from a collection of longer documents and reformulate the short text through a gating mechanism. Using short text classification as a demonstrating task, we show that the deep memory network significantly outperforms classical text expansion methods with comprehensive experiments on real world data sets.

This report describes a 2-year prospective, longitudinal survey of attending physicians in 3 clinical areas (family medicine, general pediatrics, internal medicine) who experienced a transition from a homegrown electronic health record (EHR) to a vendor EHR. Participants were already highly familiar with using EHRs. Data were collected 1 month before and 3, 6, 13, and 25 months post implementation. Our primary goal was to determine if perceptions followed a J-curve pattern in which they initially dropped but eventually surpassed baseline measures. A J-curve was not found for any measures, including workflow, safety, communication, and satisfaction. Only the reminders and alerts measure dropped and then returned to baseline (U-curve); a few remained flatlined. Most dropped and remained below baseline (L-curve). The only measure that remained above baseline was documenting in the exam room with the patient. This study adds to the literature about current controversies surrounding EHR adoption and physician satisfaction.

OBJECTIVE: The introduction of health information technology systems, electronic health records in particular, is changing the nature of how clinicians interact with patients. Lack of knowledge remains on how best to integrate such systems in the exam room. The purpose of this systematic review was to (1) distill "best" behavioral and communication practices recommended in the literature for clinicians when interacting with patients in the presence of computerized systems during a clinical encounter, (2) weigh the evidence of each recommendation, and (3) rank evidence-based recommendations for electronic health record communication training initiatives for clinicians. METHODS: We conducted a literature search of 6 databases, resulting in 52 articles included in the analysis. We extracted information such as study setting, research design, sample, findings, and implications. Recommendations were distilled based on consistent support for behavioral and communication practices across studies. RESULTS: Eight behavioral and communication practices received strong support of evidence in the literature and included specific aspects of using computerized systems to facilitate conversation and transparency in the exam room, such as spatial (re)organization of the exam room, maintaining nonverbal communication, and specific techniques that integrate the computerized system into the visit and engage the patient. Four practices, although patient-centered, have received insufficient evidence to date. DISCUSSION AND CONCLUSION: We developed an evidence base of best practices for clinicians to maintain patient-centered communications in the presence of computerized systems in the exam room. Further work includes development and empirical evaluation of evidence-based guidelines to better integrate computerized systems into clinical care.

Infertility is a global health concern that affects countless couples trying to conceive a child. Effective fertility treatment requires continuous monitoring of a wide range of health indicators through self-tracking. The process of collecting and interpreting data and information about fertility is complex, and much of the burden falls on women. In this study, we analyzed patient-generated content in a popular online health community dedicated to fertility issues. The objective was to understand the process in which women engage in tracking relevant information, and the challenges they face. Leveraging the Personal Informatics Model, we describe women's self-tracking experiences during their fertility cycles. We discuss how a complex and highly personalized context leads to responsibility, pressure, and emotional burden on women performing self-tracking activities, as well as the role of collaboration in creating individualized solutions. Finally, we provide implications for technologies aiming to support women with fertility care needs.

OBJECTIVE: ClinicalTrials.gov serves critical functions of disseminating trial information to the public and helping the trials recruit participants. This study assessed the readability of trial descriptions at ClinicalTrials.gov using multiple quantitative measures. MATERIALS AND METHODS: The analysis included all 165,988 trials registered at ClinicalTrials.gov as of April 30, 2014. To obtain benchmarks, the authors also analyzed 2 other medical corpora: (1) all 955 Health Topics articles from MedlinePlus and (2) a random sample of 100,000 clinician notes retrieved from an electronic health records system intended for conveying internal communication among medical professionals. The authors characterized each of the corpora using 4 surface metrics, and then applied 5 different scoring algorithms to assess their readability. The authors hypothesized that clinician notes would be most difficult to read, followed by trial descriptions and MedlinePlus Health Topics articles. RESULTS: Trial descriptions have the longest average sentence length (26.1 words) across all corpora; 65% of their words used are not covered by a basic medical English dictionary. In comparison, average sentence length of MedlinePlus Health Topics articles is 61% shorter, vocabulary size is 95% smaller, and dictionary coverage is 46% higher. All 5 scoring algorithms consistently rated CliniclTrials.gov trial descriptions the most difficult corpus to read, even harder than clinician notes. On average, it requires 18 years of education to properly understand these trial descriptions according to the results generated by the readability assessment algorithms. DISCUSSION AND CONCLUSION: Trial descriptions at CliniclTrials.gov are extremely difficult to read. Significant work is warranted to improve their readability in order to achieve CliniclTrials.gov's goal of facilitating information dissemination and subject recruitment.

OBJECTIVE: Electronic health records (EHRs) have great potential to improve quality of care. However, their use may diminish "patient-centeredness" in exam rooms by distracting the healthcare provider from focusing on direct patient interaction. The authors conducted a qualitative interview study to understand the magnitude of this issue, and the strategies that primary care providers devised to mitigate the unintended adverse effect associated with EHR use. METHODS AND MATERIALS: Semi-structured interviews were conducted with 21 healthcare providers at 4 Veterans Affairs (VAs) outpatient primary care clinics in San Diego County. Data analysis was performed using the grounded theory approach. RESULTS: The results show that providers face demands from both patients and the EHR system. To cope with these demands, and to provide patient-centered care, providers attempt to perform EHR work outside of patient encounters and create templates to streamline documentation work. Providers also attempt to use the EHR to engage patients, establish patient buy-in for EHR use, and multitask between communicating with patients and using the EHR. DISCUSSION AND CONCLUSION: This study has uncovered the challenges that primary care providers face in integrating the EHR into their work practice, and the strategies they use to overcome these challenges in order to maintain patient-centered care. These findings illuminate the importance of developing "best" practices to improve patient-centered care in today's highly "wired" health environment. These findings also show that more user-centered EHR design is needed to improve system usability.

The recent study by Taft and colleagues, which explores communication differences in paper versus electronic health records (EHRs), was both interesting and timely.1 EHRs are becoming a focal point for healthcare delivery in the US, yet the impact of EHRs on the patient-provider relationship remains poorly understood. Communication is at the heart of this relationship, and providers are concerned about the potential for EHRs to reduce the quality of their communications with patients.2,3 We would like to provide additional thoughts on Taft et al.’s reported findings and put them into a broader context. First, it is interesting to note that the authors found that EHRs fostered better communications with patients across nearly all measures. However, we wonder what might explain why a physician would greet a patient more warmly when walking into an exam room with a laptop computer vs. a paper chart. This suggests that the effect of EHRs vs. paper records must either be very strong and rapid or that there may be other factors at play. With regards to the overall premise of comparing paper to electronic charts – EHRs are capable of much more than paper records, but that capability is partly the reason why clinicians may perceive that patient communications have been impaired by EHRs. Clinicians in the exam room are taking on more tasks and interacting with the EHR in ways that were not possible with paper records. For example, tasks associated with an ambulatory EHR that do not have comparable actions in paper records include acknowledgment of medical assistant- or nurse-entered data via button clicks; e-prescribing, which enforces more conformity than paper prescriptions and may display numerous alerts that require review and confirmation; coding the encounter for billing purposes, which might previously have been handled by clerical staff; responding to reminders about immunizations and overdue tests (and subsequent order entry); documenting the encounter using dropdown menus, checkboxes, free-text entry, and many other modalities; and other documentation requirements that have resulted from new healthcare regulations, such as meaningful use. The Methods section of the Taft et al. article stated that their mock EHR was “styled after the Department of Veterans’ Affairs’ computerized patient record system,” but it is unclear how many of the aforementioned tasks were handled by the residents using the system in the study or even if the study’s mock EHR supported these complex functions. Practicing clinicians will readily recognize the challenges of handling these tasks, especially entering data through structured data entry forms and responding to computer-generated alerts, while interacting with patients.4 These additional tasks are not necessarily bad, and some only exist because EHRs enable them and because they are considered beneficial (eg, drug safety alerts), but they do present different challenges than paper records. Clinicians are certainly not required to perform all of the tasks required by the EHR while the patient is in the exam room, but time pressures often encourage them to do so. In summary, if residents in the Taft et al. experiment did not complete many of these additional tasks while interacting with patients, then the study scenarios may not represent realistic and typical EHR use. Further, while the authors clearly took great care to reduce potential biases, including not informing the residents or patient actors about the purpose of the study, it is not clear if the raters were also blinded to the study’s objectives. It would be interesting to see if the measures of residents’ communication skills would change if the raters were only provided with audio recordings of the physician-patient interactions, without the accompanying video footage. Furthermore, in this context, patients’ perceptions may be more relevant. While patient actors in the study were given a copy of the communication tool “so they could provide feedback to the residents’ supervisors at the end of the study,” their impressions about the residents’ communication skills were not reported. We look forward to future work on how EHR use impacts physician-patient communications, including comparisons of experienced clinicians using different EHRs in real practice settings. Some studies have noted communication differences between providers who engage in extensive in-room EHR use and those who use the EHR minimally during patient encounters.5 There is no standard etiquette about what does or does not constitute appropriate EHR use during an ambulatory patient encounter, but further study of EHR use and doctor-patient communication could help inform better EHR usage guidelines.

OBJECTIVE: This paper describes the University of Michigan's nine-year experience in developing and using a full-text search engine designed to facilitate information retrieval (IR) from narrative documents stored in electronic health records (EHRs). The system, called the Electronic Medical Record Search Engine (EMERSE), functions similar to Google but is equipped with special functionalities for handling challenges unique to retrieving information from medical text. MATERIALS AND METHODS: Key features that distinguish EMERSE from general-purpose search engines are discussed, with an emphasis on functions crucial to (1) improving medical IR performance and (2) assuring search quality and results consistency regardless of users' medical background, stage of training, or level of technical expertise. RESULTS: Since its initial deployment, EMERSE has been enthusiastically embraced by clinicians, administrators, and clinical and translational researchers. To date, the system has been used in supporting more than 750 research projects yielding 80 peer-reviewed publications. In several evaluation studies, EMERSE demonstrated very high levels of sensitivity and specificity in addition to greatly improved chart review efficiency. DISCUSSION: Increased availability of electronic data in healthcare does not automatically warrant increased availability of information. The success of EMERSE at our institution illustrates that free-text EHR search engines can be a valuable tool to help practitioners and researchers retrieve information from EHRs more effectively and efficiently, enabling critical tasks such as patient case synthesis and research data abstraction. CONCLUSION: EMERSE, available free of charge for academic use, represents a state-of-the-art medical IR tool with proven effectiveness and user acceptance.

OBJECTIVE: In recognition of potential barriers that may inhibit the widespread adoption of biomedical software, the 2014 i2b2 Challenge introduced a special track, Track 3 - Software Usability Assessment, in order to develop a better understanding of the adoption issues that might be associated with the state-of-the-art clinical NLP systems. This paper reports the ease of adoption assessment methods we developed for this track, and the results of evaluating five clinical NLP system submissions. MATERIALS AND METHODS: A team of human evaluators performed a series of scripted adoptability test tasks with each of the participating systems. The evaluation team consisted of four "expert evaluators" with training in computer science, and eight "end user evaluators" with mixed backgrounds in medicine, nursing, pharmacy, and health informatics. We assessed how easy it is to adopt the submitted systems along the following three dimensions: communication effectiveness (i.e., how effective a system is in communicating its designed objectives to intended audience), effort required to install, and effort required to use. We used a formal software usability testing tool, TURF, to record the evaluators' interactions with the systems and 'think-aloud' data revealing their thought processes when installing and using the systems and when resolving unexpected issues. RESULTS: Overall, the ease of adoption ratings that the five systems received are unsatisfactory. Installation of some of the systems proved to be rather difficult, and some systems failed to adequately communicate their designed objectives to intended adopters. Further, the average ratings provided by the end user evaluators on ease of use and ease of interpreting output are -0.35 and -0.53, respectively, indicating that this group of users generally deemed the systems extremely difficult to work with. While the ratings provided by the expert evaluators are higher, 0.6 and 0.45, respectively, these ratings are still low indicating that they also experienced considerable struggles. DISCUSSION: The results of the Track 3 evaluation show that the adoptability of the five participating clinical NLP systems has a great margin for improvement. Remedy strategies suggested by the evaluators included (1) more detailed and operation system specific use instructions; (2) provision of more pertinent onscreen feedback for easier diagnosis of problems; (3) including screen walk-throughs in use instructions so users know what to expect and what might have gone wrong; (4) avoiding jargon and acronyms in materials intended for end users; and (5) packaging prerequisites required within software distributions so that prospective adopters of the software do not have to obtain each of the third-party components on their own.

BACKGROUND AND OBJECTIVE: Providing patients access to their medical records offers many potential benefits including identification and correction of errors. The process by which patients ask for changes to be made to their records is called an 'amendment request'. Little is known about the nature of such amendment requests and whether they result in modifications to the chart. METHODS: We conducted a qualitative content analysis of all patient-initiated amendment requests that our institution received over a 7-year period. Recurring themes were identified along three analytic dimensions: (1) clinical/documentation area, (2) patient motivation for making the request, and (3) outcome of the request. RESULTS: The dataset consisted of 818 distinct requests submitted by 181 patients. The majority of these requests (n=636, 77.8%) were made to rectify incorrect information and 49.7% of all requests were ultimately approved. In 6.6% of the requests, patients wanted valid information removed from their record, 27.8% of which were approved. Among all of the patients requesting a copy of their chart, only a very small percentage (approximately 0.2%) submitted an amendment request. CONCLUSIONS: The low number of amendment requests may be due to inadequate awareness by patients about how to make changes to their records. To make this approach effective, it will be important to inform patients of their right to view and amend records and about the process for doing so. Increasing patient access to medical records could encourage patient participation in improving the accuracy of medical records; however, caution should be used.

OBJECTIVE: We describe experiments designed to determine the feasibility of distinguishing known from novel associations based on a clinical dataset comprised of International Classification of Disease, V.9 (ICD-9) codes from 1.6 million patients by comparing them to associations of ICD-9 codes derived from 20.5 million Medline citations processed using MetaMap. Associations appearing only in the clinical dataset, but not in Medline citations, are potentially novel. METHODS: Pairwise associations of ICD-9 codes were independently identified in both the clinical and Medline datasets, which were then compared to quantify their degree of overlap. We also performed a manual review of a subset of the associations to validate how well MetaMap performed in identifying diagnoses mentioned in Medline citations that formed the basis of the Medline associations. RESULTS: The overlap of associations based on ICD-9 codes in the clinical and Medline datasets was low: only 6.6% of the 3.1 million associations found in the clinical dataset were also present in the Medline dataset. Further, a manual review of a subset of the associations that appeared in both datasets revealed that co-occurring diagnoses from Medline citations do not always represent clinically meaningful associations. DISCUSSION: Identifying novel associations derived from large clinical datasets remains challenging. Medline as a sole data source for existing knowledge may not be adequate to filter out widely known associations. CONCLUSIONS: In this study, novel associations were not readily identified. Further improvements in accuracy and relevance for tools such as MetaMap are needed to realize their expected utility.

OBJECTIVES: This study was to assess whether active learning strategies can be integrated with supervised word sense disambiguation (WSD) methods, thus reducing the number of annotated samples, while keeping or improving the quality of disambiguation models. METHODS: We developed support vector machine (SVM) classifiers to disambiguate 197 ambiguous terms and abbreviations in the MSH WSD collection. Three different uncertainty sampling-based active learning algorithms were implemented with the SVM classifiers and were compared with a passive learner (PL) based on random sampling. For each ambiguous term and each learning algorithm, a learning curve that plots the accuracy computed from the test set as a function of the number of annotated samples used in the model was generated. The area under the learning curve (ALC) was used as the primary metric for evaluation. RESULTS: Our experiments demonstrated that active learners (ALs) significantly outperformed the PL, showing better performance for 177 out of 197 (89.8%) WSD tasks. Further analysis showed that to achieve an average accuracy of 90%, the PL needed 38 annotated samples, while the ALs needed only 24, a 37% reduction in annotation effort. Moreover, we analyzed cases where active learning algorithms did not achieve superior performance and identified three causes: (1) poor models in the early learning stage; (2) easy WSD cases; and (3) difficult WSD cases, which provide useful insight for future improvements. CONCLUSIONS: This study demonstrated that integrating active learning strategies with supervised WSD methods could effectively reduce annotation cost and improve the disambiguation models.

OBJECTIVE: To examine how clinicians on the receiving end of admission handoffs use electronic health records (EHRs) in preparation for those handoffs and to identify the kinds of impacts such usage may have. MATERIALS AND METHODS: This analysis is part of a two-year ethnographic study of emergency department (ED) to internal medicine admission handoffs at a tertiary teaching and referral hospital. Qualitative data were gathered and analyzed iteratively, following a grounded theory methodology. Data collection methods included semi-structured interviews (N = 48), observations (349 hours), and recording of handoff conversations (N = 48). Data analyses involved coding, memo writing, and member checking. RESULTS: The use of EHRs has enabled an emerging practice that we refer to as pre-handoff "chart biopsy": the activity of selectively examining portions of a patient's health record to gather specific data or information about that patient or to get a broader sense of the patient and the care that patient has received. Three functions of chart biopsy are identified: getting an overview of the patient; preparing for handoff and subsequent care; and defending against potential biases. Chart biopsies appear to impact important clinical and organizational processes. Among these are the nature and quality of handoff interactions, and the quality of care, including the appropriateness of dispositioning of patients. CONCLUSIONS: Chart biopsy has the potential to enrich collaboration and to enable the hospital to act safely, efficiently, and effectively. Implications for handoff research and for the design and evaluation of EHRs are also discussed.

The patient Problem List (PL) is a mandated documentation component of electronic health records supporting the longitudinal summarization of patient information in addition to facilitating the coordination of care by multidisciplinary medical teams. In this paper, we report an ethnographic study that examined the institutionalization of the PL. Specifically, we explored: (1) how different groups (primary care providers, inpatient hospitalists, specialists, and emergency doctors) perceived the purposes of the PL differently; (2) how these deviated perceptions might affect their use of the PL; and (3) how the technical design of the PL facilitated or hindered the clinical practices of these groups. We found significant ambiguity regarding the definition, benefits, and use of the PL across different groups. We also found that certain groups (e.g. primary care providers) had developed effective cooperative strategies regarding the use of the PL; however, suboptimal usage was common among other user types, which could have a profound impact on quality of care and safety. Based on these findings, we provide suggestions to improve the design of the PL, particularly on strengthening its support on longitudinal and cooperative clinical practices.

OBJECTIVE: To understand the nature of emerging electronic documentation practices, disconnects between documentation workflows and computing systems designed to support them, and ways to improve the design of electronic documentation systems. MATERIALS AND METHODS: Time-and-motion study of resident physicians' note-writing practices using a commercial electronic health record system that includes an electronic documentation module. The study was conducted in the general medicine unit of a large academic hospital. RESULTS: During the study, 96 note-writing sessions by 11 resident physicians, resulting in close to 100 h of observations were seen. Seven of the 10 most common transitions between activities during note composition were between documenting, and gathering and reviewing patient data, and updating the plan of care. DISCUSSION: The high frequency of transitions seen in the study suggested that clinical documentation is fundamentally a synthesis activity, in which clinicians review available patient data and summarize their impressions and judgments. At the same time, most electronic health record systems are optimized to support documentation as uninterrupted composition. This mismatch leads to fragmentation in clinical work, and results in inefficiencies and workarounds. In contrast, we propose that documentation can be best supported with tools that facilitate data exploration and search for relevant information, selective reading and annotation, and composition of a note as a temporal structure. CONCLUSIONS: Time-and-motion study of clinicians' electronic documentation practices revealed a high level of fragmentation of documentation activities and frequent task transitions. Treating documentation as synthesis rather than composition suggests new possibilities for supporting it more effectively with electronic systems.

We conducted an ethnographically based study at a large teaching hospital to examine clinician workarounds engendered by the adoption of a Computerized Prescribe Order Entry (CPOE) system. Specifically, we investigated how adoption of computerized systems may alter medical practice, order management in particular, as manifested through the working-around behavior developed by doctors and nurses to accommodate the changes in their day-to-day work environment. In this paper, we focus on clinicians' workarounds, including those workarounds that gradually disappeared and those that have become routinized. Further, we extend the CSCW concept of boundary object (to "assemblage") in order to understand the workarounds created with CPOE system use and the changing nature of clinical practices that are increasingly computerized.

OBJECTIVE: To present a partnership-based and community-oriented approach designed to ease provider anxiety and facilitate the implementation of electronic health records (EHR) in resource-limited primary care settings. MATERIALS AND METHODS: The approach, referred to as partnership model, was developed and iteratively refined through the research team's previous work on implementing health information technology (HIT) in over 30 safety net practices. This paper uses two case studies to illustrate how the model was applied to help two nurse-managed health centers (NMHC), a particularly vulnerable primary care setting, implement EHR and get prepared to meet the meaningful use criteria. RESULTS: The strong focus of the model on continuous quality improvement led to eventual implementation success at both sites, despite difficulties encountered during the initial stages of the project. DISCUSSION: There has been a lack of research, particularly in resource-limited primary care settings, on strategies for abating provider anxiety and preparing them to manage complex changes associated with EHR uptake. The partnership model described in this paper may provide useful insights into the work shepherded by HIT regional extension centers dedicated to supporting resource-limited communities disproportionally affected by EHR adoption barriers. CONCLUSION: NMHC, similar to other primary care settings, are often poorly resourced, understaffed, and lack the necessary expertise to deploy EHR and integrate its use into their day-to-day practice. This study demonstrates that implementation of EHR, a prerequisite to meaningful use, can be successfully achieved in this setting, and partnership efforts extending far beyond the initial software deployment stage may be the key.

OBJECTIVE: To develop a theoretically informed and empirically validated survey instrument for assessing prescribers' perception of computerized drug-drug interaction (DDI) alerts. MATERIALS AND METHODS: The survey is grounded in the unified theory of acceptance and use of technology and an adapted accident causation model. Development of the instrument was also informed by a review of the extant literature on prescribers' attitude toward computerized medication safety alerts and common prescriber-provided reasons for overriding. To refine and validate the survey, we conducted a two-stage empirical validation study consisting of a pretest with a panel of domain experts followed by a field test among all eligible prescribers at our institution. RESULTS: The resulting survey instrument contains 28 questionnaire items assessing six theoretical dimensions: performance expectancy, effort expectancy, social influence, facilitating conditions, perceived fatigue, and perceived use behavior. Satisfactory results were obtained from the field validation; however, a few potential issues were also identified. We analyzed these issues accordingly and the results led to the final survey instrument as well as usage recommendations. DISCUSSION: High override rates of computerized medication safety alerts have been a prevalent problem. They are usually caused by, or manifested in, issues of poor end user acceptance. However, standardized research tools for assessing and understanding end users' perception are currently lacking, which inhibits knowledge accumulation and consequently forgoes improvement opportunities. The survey instrument presented in this paper may help fill this methodological gap. CONCLUSION: We developed and empirically validated a survey instrument that may be useful for future research on DDI alerts and other types of computerized medication safety alerts more generally.

OBJECTIVE: To identify ways for improving the consistency of design, conduct, and results reporting of time and motion (T&M) research in health informatics. MATERIALS AND METHODS: We analyzed the commonalities and divergences of empirical studies published 1990-2010 that have applied the T&M approach to examine the impact of health IT implementation on clinical work processes and workflow. The analysis led to the development of a suggested 'checklist' intended to help future T&M research produce compatible and comparable results. We call this checklist STAMP (Suggested Time And Motion Procedures). RESULTS: STAMP outlines a minimum set of 29 data/ information elements organized into eight key areas, plus three supplemental elements contained in an 'Ancillary Data' area, that researchers may consider collecting and reporting in their future T&M endeavors. DISCUSSION: T&M is generally regarded as the most reliable approach for assessing the impact of health IT implementation on clinical work. However, there exist considerable inconsistencies in how previous T&M studies were conducted and/or how their results were reported, many of which do not seem necessary yet can have a significant impact on quality of research and generalisability of results. Therefore, we deem it is time to call for standards that can help improve the consistency of T&M research in health informatics. This study represents an initial attempt. CONCLUSION: We developed a suggested checklist to improve the methodological and results reporting consistency of T&M research, so that meaningful insights can be derived from across-study synthesis and health informatics, as a field, will be able to accumulate knowledge from these studies.

Unpredictable yet frequently occurring exception situations pervade clinical care. Handling them properly often requires aberrant actions temporarily departing from normal practice. In this study, the authors investigated several exception-handling procedures provided in an electronic health records system for facilitating clinical documentation, which the authors refer to as 'data entry exit strategies.' Through a longitudinal analysis of computer-recorded usage data, the authors found that (1) utilization of the exit strategies was not affected by postimplementation system maturity or patient visit volume, suggesting clinicians' needs to 'exit' unwanted situations are persistent; and (2) clinician type and gender are strong predictors of exit-strategy usage. Drilldown analyses further revealed that the exit strategies were judiciously used and enabled actions that would be otherwise difficult or impossible. However, many data entries recorded via them could have been 'properly' documented, yet were not, and a considerable proportion containing temporary or incomplete information was never subsequently amended. These findings may have significant implications for the design of safer and more user-friendly point-of-care information systems for healthcare.

OBJECTIVE: A full-text search engine can be a useful tool for augmenting the reuse value of unstructured narrative data stored in electronic health records (EHR). A prominent barrier to the effective utilization of such tools originates from users' lack of search expertise and/or medical-domain knowledge. To mitigate the issue, the authors experimented with a 'collaborative search' feature through a homegrown EHR search engine that allows users to preserve their search knowledge and share it with others. This feature was inspired by the success of many social information-foraging techniques used on the web that leverage users' collective wisdom to improve the quality and efficiency of information retrieval. DESIGN: The authors conducted an empirical evaluation study over a 4-year period. The user sample consisted of 451 academic researchers, medical practitioners, and hospital administrators. The data were analyzed using a social-network analysis to delineate the structure of the user collaboration networks that mediated the diffusion of knowledge of search. RESULTS: The users embraced the concept with considerable enthusiasm. About half of the EHR searches processed by the system (0.44 million) were based on stored search knowledge; 0.16 million utilized shared knowledge made available by other users. The social-network analysis results also suggest that the user-collaboration networks engendered by the collaborative search feature played an instrumental role in enabling the transfer of search knowledge across people and domains. CONCLUSION: Applying collaborative search, a social information-foraging technique popularly used on the web, may provide the potential to improve the quality and efficiency of information retrieval in healthcare.

We conducted a field-based study at a large teaching hospital to examine doctors' use and documentation of patient care information, with a special focus on a patient's psychosocial information. We were particularly interested in the gaps between the medical work and any representations of the patient. The paper describes how doctors record this information for immediate and long-term use. We found that doctors documented a considerable amount of psychosocial information in their electronic health records (EHR) system. Yet, we also observed that such information was recorded selectively, and a medicalized view-point is a key contributing factor. Our study shows how missing or problematic representations of a patient affect work activities and patient care. We accordingly suggest that EHR systems could be made more usable and useful in the long run, by supporting both representations of medical processes and of patients.

In this paper, we report a qualitative study of translational researchers in health sciences in a large public Research I university in the United States. This paper first identifies challenges faced by translational researchers in mobilizing necessary resources such as data and personnel to effectively conduct research. Then we use the theoretical constructs underlying those challenges to identify social and technical infrastructural needs that facilitate translational research. The major challenges emerged from the data include difficulty in identifying compatible collaborators, barriers within the existing research culture, insufficient organizational and technical support for complex data management tasks, and the lack of project management skills and tools. Using institutional infrastructures to support translational research is suggested.

Health IT implementations often introduce radical changes to clinical work processes and workflow. Prior research investigating this effect has shown conflicting results. Recent time and motion studies have consistently found that this impact is negligible; whereas qualitative studies have repeatedly revealed negative end-user perceptions suggesting decreased efficiency and disrupted workflow. We speculate that this discrepancy may be due in part to the design of the time and motion studies, which is focused on measuring clinicians' 'time expenditures' among different clinical activities rather than inspecting clinical 'workflow' from the true 'flow of the work' perspective. In this paper, we present a set of new analytical methods consisting of workflow fragmentation assessments, pattern recognition, and data visualization, which are accordingly designed to uncover hidden regularities embedded in the flow of the work. Through an empirical study, we demonstrate the potential value of these new methods in enriching workflow analysis in clinical settings.

OBJECTIVE: To study how social interactions influence physician adoption of an electronic health records (EHR) system. DESIGN: A social network survey was used to delineate the structure of social interactions among 40 residents and 15 attending physicians in an ambulatory primary care practice. Social network analysis was then applied to relate the interaction structures to individual physicians' utilization rates of an EHR system. MEASUREMENTS: The social network survey assessed three distinct types of interaction structures: professional network based on consultation on patient care-related matters; friendship network based on personal intimacy; and perceived influence network based on a person's perception of how other people have affected her intention to adopt the EHR system. EHR utilization rates were measured as the proportion of patient visits in which sentinel use events consisting of patient data documentation or retrieval activities were recorded. The usage data were collected over a time period of 14 months from computer-recorded audit trail logs. RESULTS: Neither the professional nor the perceived influence network is correlated with EHR usage. The structure of the friendship network significantly influenced individual physicians' adoption of the EHR system. Residents who occupied similar social positions in the friendship network shared similar EHR utilization rates (p<0.05). In other words, residents who had personal friends in common tended to develop comparable levels of EHR adoption. This effect is particularly prominent when the mutual personal friends of these 'socially similar' residents were attending physicians (p<0.001). CONCLUSIONS: Social influence affecting physician adoption of EHR seems to be predominantly conveyed through interactions with personal friends rather than interactions in professional settings.

We conducted a field-based study examining informal nursing information. We examined the use of this information before and after the adoption of a CPOE (Computerized Provider Order Entry) system in an inpatient unit of a large teaching hospital. Before CPOE adoption, nurses used paper working documents to detail psycho-social information about patients; after the CPOE adoption, they did not use paper or digital notes as was planned. The paper describes this process and analyses how several interlocked reasons contributed to the loss of this information in written form. We found that a change in physical location, sufficient convenience, visibility of the information, and permanency of information account for some, but not all, of the outcome. As well, we found that computerization of the nursing data led to a shift in the politics of the information itself - the nurses no longer had a cohesive agreement about the kinds of data to enter into the system. The findings address the requirements of healthcare computerization to support both formal and informal work practices, respecting the nature of nursing work and the politics of information inherent in complex medical work.

No abstract available.

OBJECTIVES: This study sought to investigate user interactions with an electronic health records (EHR) system by uncovering hidden navigational patterns in the EHR usage data automatically recorded as clinicians navigated through the system's software user interface (UI) to perform different clinical tasks. DESIGN: A homegrown EHR was adapted to allow real-time capture of comprehensive UI interaction events. These events, constituting time-stamped event sequences, were used to replay how the EHR was used in actual patient care settings. The study site is an ambulatory primary care clinic at an urban teaching hospital. Internal medicine residents were the primary EHR users. MEASUREMENTS: Computer-recorded event sequences reflecting the order in which different EHR features were sequentially accessed. METHODS: We apply sequential pattern analysis (SPA) and a first-order Markov chain model to uncover recurring UI navigational patterns. RESULTS: Of 17 main EHR features provided in the system, SPA identified 3 bundled features: "Assessment and Plan" and "Diagnosis," "Order" and "Medication," and "Order" and "Laboratory Test." Clinicians often accessed these paired features in a bundle together in a continuous sequence. The Markov chain analysis revealed a global navigational pathway, suggesting an overall sequential order of EHR feature accesses. "History of Present Illness" followed by "Social History" and then "Assessment and Plan" was identified as an example of such global navigational pathways commonly traversed by the EHR users. CONCLUSION: Users showed consistent UI navigational patterns, some of which were not anticipated by system designers or the clinic management. Awareness of such unanticipated patterns may help identify undesirable user behavior as well as reengineering opportunities for improving the system's usability.