OBJECTIVE: Overdiagnosis occurs when abnormalities meeting diagnostic criteria would remain asymptomatic if undiagnosed. Cases initially identified through digital diagnostic tools but later recognised as overdiagnosis are referred to as 'digital overdiagnosis'. Data-driven frameworks to quantify and mitigate overdiagnosis remain limited. This study introduces a framework that integrates clinical trajectories to train a machine learning (ML)-based disease classifier, enabling the quantification and mitigation of digital overdiagnosis, using Heparin-Induced Thrombocytopenia (HIT) as a case study. METHODS: A pre-existing HIT classifier identified HIT-positive and HIT-negative cases, with ground truth based on HIT diagnostic criteria. Clinical trajectories for True Positive (TP) and True Negative (TN) patients were clustered using a novel process-models-based approach. Overdiagnosis was detected when TP cases clustered with predominantly TN cases. The classifier was then retrained with an 'updated label' integrating both HIT criteria and the concordant trajectory, to reduce overdiagnosis while maintaining accuracy. RESULTS: 7.2% of TP cases were identified as overdiagnosed. Retraining with the updated labels successfully reclassified 89.5% of overdiagnosed cases as TN, with only a minimal reduction in performance (MCC decreased by 0.03, positive likelihood ratio decreased by 0.49, and negative likelihood ratio increased by 0.05). Clinical outcomes-length of stay, thrombotic events, and mortality-differed significantly between non-overdiagnosed and overdiagnosed cases, and between non-overdiagnosed and TN cases, but not between overdiagnosed and TN cases, confirming that overdiagnosed patients resemble TN patients. CONCLUSION: Incorporating clinical trajectories into ML-based diagnosis enables the quantification of digital overdiagnosis. This approach could refine ML algorithms by prompting a reassessment of criteria-based disease labels in supervised learning.

IMPORTANCE: Understanding factors that contribute to clinical variability in patient care is critical, as unwarranted variability can lead to increased adverse events and prolonged hospital stays. Determining when this variability becomes excessive can be a step in optimizing patient outcomes and healthcare efficiency. OBJECTIVE: Explore the association between clinical variation and clinical outcomes. This study aims to identify the point in time when the relationship between clinical variation and length of stay (LOS) becomes significant. METHODS: This cohort study uses MIMIC-IV, a dataset collecting electronic health records of the Beth Israel Deaconess Medical Center in the United States. We focused on adult patients who underwent elective coronary bypass surgery, generating 847 patient observations. Demographic factors such as age, race, insurance type, and the Charlson Comorbidity Index (CCI) were recorded. We performed a variability analysis where patients' clinical processes are represented as sequences of events. The data was segmented based on the initial day of recorded activity to establish observation windows. Using a regression analysis, we identified the temporal window where variability's impact on LOS becomes independently significant. RESULT: Regression analysis revealed that patients in the top 20 % of the variability distance group experienced an 81 % increase in LOS (95 % CI: 1.72 to 1.91, p < 0.001). Insurance types, such as Medicare and Other, were associated with 18 % (95 % CI: 0.73 to 0.92, p < 0.001) and 21 % (95 % CI: 0.71 to 0.88, p < 0.001) decreases in LOS, respectively. Neither age nor race significantly affected LOS, but a higher CCI was associated with a 3.3 % increase in LOS (95 % CI: 1.02 to 1.05, p < 0.001). These findings indicate that higher variability and CCI significantly influence LOS, with insurance type also playing a crucial role. CONCLUSION: In the studied cohort, patient journeys with greater variability were associated with longer LOS with a dose-response relationship: the higher the variability, the longer LOS. This study presents a standardized way to measure and visualize variability in clinical processes and measure its impact on patient-relevant outcomes.

Clinical predictive analysis is a crucial task with numerous applications and has been extensively studied using machine learning approaches. Clinical notes, a vital data source, have been employed to develop natural language processing (NLP) models for risk prediction in healthcare with robust performance. However, clinical notes vary considerably in text composition-written by diverse healthcare providers for different purposes-and the impact of these variations on NLP modeling is also underexplored. It also remains uncertain whether the recent Large Language Models (LLMs) with instruction-following capabilities can effectively handle the risk prediction task out-of-the-box, especially when using routinely collected clinical notes instead of polished text. We address these two important research questions in the context of in-hospital mortality prediction within the critical care setting. Specifically, we propose a supervised hierarchical network with note-specific modules to account for variations across different note categories, and provide a detailed comparison with strong supervised baselines and LLMs. We benchmark 34 instruction-following LLMs based on zero-shot, few-shot, and chain-of-thought prompting with diverse prompt templates. Our results demonstrate that the note-specific network delivers improved risk prediction performance compared to established supervised baselines from both measurement-based and text-based modeling. In contrast, LLMs consistently underperform on this critical task, despite their remarkable performances in other domains. This highlights important limitations and raises caution regarding the use of LLMs for risk assessment in the critical setting. Additionally, we show that the proposed model can be leveraged to select informative clinical notes to enhance the training of other models.

OBJECTIVES: Hospital costs continue to rise unsustainably. Up to 20% of care is wasteful including low value care (LVC). This study aimed to understand whether electronic medical record (EMR) alerts are effective at reducing pediatric LVC and measure the impact on hospital costs. MATERIALS AND METHODS: Using EMR data over a 76-month period, we evaluated changes in 4 LVC practices following the implementation of EMR alerts, using time series analysis to control for underlying time-based trends, in a large pediatric hospital in Australia. The main outcome measure was the change in rate of each LVC practice. Balancing measures included the rate of alert adherence as a proxy measure for risk of alert fatigue. Hospital costs were calculated by the volume of LVC avoided multiplied by the unit costs. Costs of the intervention were calculated from clinician and analyst time required. RESULTS: All 4 LVC practices showed a statistically significant reduction following alert implementation. Two LVC practices (blood tests) showed an abrupt change, associated with high rates of alert adherence. The other 2 LVC practices (bronchodilator use in bronchiolitis and electrocardiogram ordering for sleeping bradycardia) showed an accelerated rate of improvement compared to baseline trends with lower rates of alert adherence. Hospital savings were $325 to $180 000 per alert. DISCUSSION AND CONCLUSION: EMR alerts are effective in reducing pediatric LVC practices and offer a cost-saving opportunity to the hospital. Further efforts to leverage EMR alerts in pediatric settings to reduce LVC are likely to support future sustainable healthcare delivery.

Over the years, patient satisfaction has become a key factor when evaluating the quality of healthcare. There is a constant desire to further analyze patients' needs and expectations, with the aim of improving their healthcare experience. With tools such as customer journey maps (CJM), two objectives are addressed: a) identify how patients interact through the phases of the care cycle and b) execute multiple analyses to deliver a wide variety of outcomes focused on improving patient experience, through the combination of these tools with emerging disciplines such as process mining. We propose a new method based on a predefined framework to optimize the creation of healthcare indicators through process mining. Our method focuses on identifying touchpoints, defining, calculating, validating, and visualizing Key Performance Indicators (KPIs) in a clinical process. The proposed method was applied to analyze an emergency room process as a case study. Results demonstrate the usefulness of the method to discover the main process interaction points, to create key indicators, and generate dashboards that support the goal of understanding and optimizing patient care.

OBJECTIVE: With the increasing amount and growing variety of healthcare data, multimodal machine learning supporting integrated modeling of structured and unstructured data is an increasingly important tool for clinical machine learning tasks. However, it is non-trivial to manage the differences in dimensionality, volume, and temporal characteristics of data modalities in the context of a shared target task. Furthermore, patients can have substantial variations in the availability of data, while existing multimodal modeling methods typically assume data completeness and lack a mechanism to handle missing modalities. METHODS: We propose a Transformer-based fusion model with modality-specific tokens that summarize the corresponding modalities to achieve effective cross-modal interaction accommodating missing modalities in the clinical context. The model is further refined by inter-modal, inter-sample contrastive learning to improve the representations for better predictive performance. We denote the model as Attention-based cRoss-MOdal fUsion with contRast (ARMOUR). We evaluate ARMOUR using two input modalities (structured measurements and unstructured text), six clinical prediction tasks, and two evaluation regimes, either including or excluding samples with missing modalities. RESULTS: Our model shows improved performances over unimodal or multimodal baselines in both evaluation regimes, including or excluding patients with missing modalities in the input. The contrastive learning improves the representation power and is shown to be essential for better results. The simple setup of modality-specific tokens enables ARMOUR to handle patients with missing modalities and allows comparison with existing unimodal benchmark results. CONCLUSION: We propose a multimodal model for robust clinical prediction to achieve improved performance while accommodating patients with missing modalities. This work could inspire future research to study the effective incorporation of multiple, more complex modalities of clinical data into a single model.

INTRODUCTION: Adequate methods to promptly translate digital health innovations for improved patient care are essential. Advances in Artificial Intelligence (AI) and Machine Learning (ML) have been sources of digital innovation and hold the promise to revolutionize the way we treat, manage and diagnose patients. Understanding the benefits but also the potential adverse effects of digital health innovations, particularly when these are made available or applied on healthier segments of the population is essential. One of such adverse effects is overdiagnosis. OBJECTIVE: to comprehensively analyze quantification strategies and data-driven definitions for overdiagnosis reported in the literature. METHODS: we conducted a scoping systematic review of manuscripts describing quantitative methods to estimate the proportion of overdiagnosed patients. RESULTS: we identified 46 studies that met our inclusion criteria. They covered a variety of clinical conditions, primarily breast and prostate cancer. Methods to quantify overdiagnosis included both prospective and retrospective methods including randomized clinical trials, and simulations. CONCLUSION: a variety of methods to quantify overdiagnosis have been published, producing widely diverging results. A standard method to quantify overdiagnosis is needed to allow its mitigation during the rapidly increasing development of new digital diagnostic tools.

One unintended consequence of the Electronic Health Records (EHR) implementation is the overuse of content-importing technology, such as copy-and-paste, that creates "bloated" notes containing large amounts of textual redundancy. Despite the rising interest in applying machine learning models to learn from real-patient data, it is unclear how the phenomenon of note bloat might affect the Natural Language Processing (NLP) models derived from these notes. Therefore, in this work we examine the impact of redundancy on deep learning-based NLP models, considering four clinical prediction tasks using a publicly available EHR database. We applied two deduplication methods to the hospital notes, identifying large quantities of redundancy, and found that removing the redundancy usually has little negative impact on downstream performances, and can in certain circumstances assist models to achieve significantly better results. We also showed it is possible to attack model predictions by simply adding note duplicates, causing changes of correct predictions made by trained models into wrong predictions. In conclusion, we demonstrated that EHR text redundancy substantively affects NLP models for clinical prediction tasks, showing that the awareness of clinical contexts and robust modeling methods are important to create effective and reliable NLP systems in healthcare contexts.

Process mining techniques can be used to analyse business processes using the data logged during their execution. These techniques are leveraged in a wide range of domains, including healthcare, where it focuses mainly on the analysis of diagnostic, treatment, and organisational processes. Despite the huge amount of data generated in hospitals by staff and machinery involved in healthcare processes, there is no evidence of a systematic uptake of process mining beyond targeted case studies in a research context. When developing and using process mining in healthcare, distinguishing characteristics of healthcare processes such as their variability and patient-centred focus require targeted attention. Against this background, the Process-Oriented Data Science in Healthcare Alliance has been established to propagate the research and application of techniques targeting the data-driven improvement of healthcare processes. This paper, an initiative of the alliance, presents the distinguishing characteristics of the healthcare domain that need to be considered to successfully use process mining, as well as open challenges that need to be addressed by the community in the future.

OBJECTIVE: Data quality (DQ) must be consistently defined in context. The attributes, metadata, and context of longitudinal real-world data (RWD) have not been formalized for quality improvement across the data production and curation life cycle. We sought to complete a literature review on DQ assessment frameworks, indicators and tools for research, public health, service, and quality improvement across the data life cycle. MATERIALS AND METHODS: The review followed PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. Databases from health, physical and social sciences were used: Cinahl, Embase, Scopus, ProQuest, Emcare, PsycINFO, Compendex, and Inspec. Embase was used instead of PubMed (an interface to search MEDLINE) because it includes all MeSH (Medical Subject Headings) terms used and journals in MEDLINE as well as additional unique journals and conference abstracts. A combined data life cycle and quality framework guided the search of published and gray literature for DQ frameworks, indicators, and tools. At least 2 authors independently identified articles for inclusion and extracted and categorized DQ concepts and constructs. All authors discussed findings iteratively until consensus was reached. RESULTS: The 120 included articles yielded concepts related to contextual (data source, custodian, and user) and technical (interoperability) factors across the data life cycle. Contextual DQ subcategories included relevance, usability, accessibility, timeliness, and trust. Well-tested computable DQ indicators and assessment tools were also found. CONCLUSIONS: A DQ assessment framework that covers intrinsic, technical, and contextual categories across the data life cycle enables assessment and management of RWD repositories to ensure fitness for purpose. Balancing security, privacy, and FAIR principles requires trust and reciprocity, transparent governance, and organizational cultures that value good documentation.