OBJECTIVES: We report on findings from a meeting convened by the American College of Medical Informatics (ACMI) to characterize aspects of the patient experience that could be improved using informatics. MATERIALS AND METHODS: The American College of Medical Informatics fellows were invited to share their experiences as patients and suggest informatics approaches that may improve the patient experience. RESULTS: We identified 4 themes: (1) getting the right care, (2) data sharing and data interoperability, (3) guiding low-cost evaluations, and (4) predictive analytics. DISCUSSION: Despite widespread adoption of health IT, patient experiences remain far from optimal. CONCLUSION: The American College of Medical Informatics fellows identified informatics approaches, applications, and research areas that have the potential to improve patient experiences with health care systems.

Micheal Abaho, Danushka Bollegala, Gary Leeming, Dan Joyce, Iain Buchan. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2024.

Depression is a common mental health condition that often occurs in association with other chronic illnesses, and varies considerably in severity. Electronic Health Records (EHRs) contain rich information about a patient's medical history and can be used to train, test and maintain predictive models to support and improve patient care. This work evaluated the feasibility of implementing an environment for predicting mental health crisis among people living with depression based on both structured and unstructured EHRs. A large EHR from a mental health provider, Mersey Care, was pseudonymised and ingested into the Natural Language Processing (NLP) platform CogStack, allowing text content in binary clinical notes to be extracted. All unstructured clinical notes and summaries were semantically annotated by MedCAT and BioYODIE NLP services. Cases of crisis in patients with depression were then identified. Random forest models, gradient boosting trees, and Long Short-Term Memory (LSTM) networks, with varying feature arrangement, were trained to predict the occurrence of crisis. The results showed that all the prediction models can use a combination of structured and unstructured EHR information to predict crisis in patients with depression with good and useful accuracy. The LSTM network that was trained on a modified dataset with only 1000 most-important features from the random forest model with temporality showed the best performance with a mean AUC of 0.901 and a standard deviation of 0.006 using a training dataset and a mean AUC of 0.810 and 0.01 using a hold-out test dataset. Comparing the results from the technical evaluation with the views of psychiatrists shows that there are now opportunities to refine and integrate such prediction models into pragmatic point-of-care clinical decision support tools for supporting mental healthcare delivery.

OBJECTIVES: (1) Systematically review the literature on computerized audit and feedback (e-A&F) systems in healthcare. (2) Compare features of current systems against e-A&F best practices. (3) Generate hypotheses on how e-A&F systems may impact patient care and outcomes. METHODS: We searched MEDLINE (Ovid), EMBASE (Ovid), and CINAHL (Ebsco) databases to December 31, 2020. Two reviewers independently performed selection, extraction, and quality appraisal (Mixed Methods Appraisal Tool). System features were compared with 18 best practices derived from Clinical Performance Feedback Intervention Theory. We then used realist concepts to generate hypotheses on mechanisms of e-A&F impact. Results are reported in accordance with the PRISMA statement. RESULTS: Our search yielded 4301 unique articles. We included 88 studies evaluating 65 e-A&F systems, spanning a diverse range of clinical areas, including medical, surgical, general practice, etc. Systems adopted a median of 8 best practices (interquartile range 6-10), with 32 systems providing near real-time feedback data and 20 systems incorporating action planning. High-confidence hypotheses suggested that favorable e-A&F systems prompted specific actions, particularly enabled by timely and role-specific feedback (including patient lists and individual performance data) and embedded action plans, in order to improve system usage, care quality, and patient outcomes. CONCLUSIONS: e-A&F systems continue to be developed for many clinical applications. Yet, several systems still lack basic features recommended by best practice, such as timely feedback and action planning. Systems should focus on actionability, by providing real-time data for feedback that is specific to user roles, with embedded action plans. PROTOCOL REGISTRATION: PROSPERO CRD42016048695.

INTRODUCTION: Electronic audit and feedback (e-A&F) systems are used worldwide for care quality improvement. They measure health professionals' performance against clinical guidelines, and some systems suggest improvement actions. However, little is known about optimal interface designs for e-A&F, in particular how to present suggested actions for improvement. We developed a novel theory-informed system for primary care (the Performance Improvement plaN GeneratoR; PINGR) that covers the four principal interface components: clinical performance summaries; patient lists; detailed patient-level information; and suggested actions. As far as we are aware, this is the first report of an e-A&F system with all four interface components. OBJECTIVES: (1) Use a combination of quantitative and qualitative methods to evaluate the usability of PINGR with target end-users; (2) refine existing design recommendations for e-A&F systems; (3) determine the implications of these recommendations for patient safety. METHODS: We recruited seven primary care physicians to perform seven tasks with PINGR, during which we measured on-screen behaviour and eye movements. Participants subsequently completed usability questionnaires, and were interviewed in-depth. Data were integrated to: gain a more complete understanding of usability issues; enhance and explain each other's findings; and triangulate results to increase validity. RESULTS: Participants committed a median of 10 errors (range 8-21) when using PINGR's interface, and completed a median of five out of seven tasks (range 4-7). Errors violated six usability heuristics: clear response options; perceptual grouping and data relationships; representational formats; unambiguous description; visually distinct screens for confusable items; and workflow integration. Eye movement analysis revealed the integration of components largely supported effective user workflow, although the modular design of clinical performance summaries unnecessarily increased cognitive load. Interviews and questionnaires revealed PINGR is user-friendly, and that improved information prioritisation could further promote useful user action. CONCLUSIONS: Comparing our results with the wider usability literature we refine a previously published set of interface design recommendations for e-A&F. The implications for patient safety are significant regarding: user engagement; actionability; and information prioritisation. Our results also support adopting multi-method approaches in usability studies to maximise issue discovery and the credibility of findings.

INTRODUCTION: The construction of reliable, reusable clinical code sets is essential when re-using Electronic Health Record (EHR) data for research. Yet code set definitions are rarely transparent and their sharing is almost non-existent. There is a lack of methodological standards for the management (construction, sharing, revision and reuse) of clinical code sets which needs to be addressed to ensure the reliability and credibility of studies which use code sets. OBJECTIVE: To review methodological literature on the management of sets of clinical codes used in research on clinical databases and to provide a list of best practice recommendations for future studies and software tools. METHODS: We performed an exhaustive search for methodological papers about clinical code set engineering for re-using EHR data in research. This was supplemented with papers identified by snowball sampling. In addition, a list of e-phenotyping systems was constructed by merging references from several systematic reviews on this topic, and the processes adopted by those systems for code set management was reviewed. RESULTS: Thirty methodological papers were reviewed. Common approaches included: creating an initial list of synonyms for the condition of interest (n=20); making use of the hierarchical nature of coding terminologies during searching (n=23); reviewing sets with clinician input (n=20); and reusing and updating an existing code set (n=20). Several open source software tools (n=3) were discovered. DISCUSSION: There is a need for software tools that enable users to easily and quickly create, revise, extend, review and share code sets and we provide a list of recommendations for their design and implementation. CONCLUSION: Research re-using EHR data could be improved through the further development, more widespread use and routine reporting of the methods by which clinical codes were selected.

Next-generation sequencing (NGS) technologies have superseded traditional Sanger sequencing approach in many experimental settings, given their tremendous yield and affordable cost. Nowadays it is possible to sequence any microbial organism or meta-genomic sample within hours, and to obtain a whole human genome in weeks. Nonetheless, NGS technologies are error-prone. Correcting errors is a challenge due to multiple factors, including the data sizes, the machine-specific and non-at-random characteristics of errors, and the error distributions. Errors in NGS experiments can hamper the subsequent data analysis and inference. This work proposes an error correction method based on the de Bruijn graph that permits its execution on Gigabyte-sized data sets using normal desktop/laptop computers, ideal for genome sizes in the Megabase range, e.g. bacteria. The implementation makes extensive use of hashing techniques, and implements an A* algorithm for optimal error correction, minimizing the distance between an erroneous read and its possible replacement with the Needleman-Wunsch score. Our approach outperforms other popular methods both in terms of random access memory usage and computing times.

Local health policies are not as evidence based as they could be if the public health impacts of policies were easier to simulate. Here we address the inaccessibility of high quality models of public health and policy - presenting the concepts of a new simulation framework, IMPACT, built on Semantic Web principles. Model and simulation data are persisted with rich semantics and context to support sharing and interpretation. For this purpose, graph storage systems are explored alongside a new framework for mapping clinical data objects to graphical models. The computation employs functional programming for the parallelised simulation of locally representative populations/cohorts changing over time. The input data, model information and simulation results are mapped to social networks of policy making using the Work/Research Object and e-Lab paradigm that is emerging in E-Science.

Over the last four decades, the UK has made large investments in healthcare information technology. The authors conducted interviews and reviewed published and unpublished documents to describe national-scale clinical information exchange in England, how it was achieved, and the problems experienced that the USA might avoid. Clinical information exchange in the UK was accomplished by establishing a foundation of policy, infrastructure, and systems of care, by creating and acquiring clinical computing applications and with strong use of financial and clinical incentives. Many software and hardware vendors played a part in this effort; they participated in a national framework created by the NHS in which standards for exchange are specified and their applications designed to make clinical information exchange part of normal practice. Great potential exists for cost reduction, increased safety, and greater patient involvement as a result of clinical information exchange.

Scientific data stands to represent a significant portion of the linked open data cloud and science itself stands to benefit from the data fusion capability that this will afford. However, simply publishing linked data into the cloud does not necessarily meet the requirements of reuse. Publishing has requirements of provenance, quality, credit, attribution, methods in order to provide the \emph{reproducibility} that allows validation of results. In this paper we make the case for a scientific data publication model on top of linked data and introduce the notion of \emph{Research Objects} as first class citizens for sharing and publishing.

Epidemiology requires large-scale, high-resolution, representative population data sets; data extracted from electronic health record systems meets these criteria. However, within the UK, there is no single electronic health record, and the record of a patient's healthcare is fragmented over multiple systems and multiple organizations. In the SHORE project we have developed a proof of concept system that addresses these problems by retaining control of patient data at a local level, where it can be effectively interpreted and governed, and by overlaying on these data sources privacy-preserving record linkage providing a unified view of the health and care of the population.

The study of the genetics of diseases has been revolutionised by the advent of genome-wide genotyping technologies. Increasingly, genome-wide association studies are being used to identify positions within the human genome that have a link with a disease condition. These new data sets require the use of distributed resources, both for the statistical analysis and for the interpretation of the analysis results. Aiding the latter will be be crucial for the statistical analysis process to be successful. In this paper we report our experiences in developing a user-friendly High Performance Computing (HPC) statistical genetics analysis platform for use by clinical researchers. Specifically, we report work on supporting the interpretation process through the automatic annotation of the statistical analysis results with relevant biological information. Retrieval of the biological annotation is performed by high-volume invocation of multiple Web-services orchestrated via pre-existing scientific workflows. We also report work on developing tools to aid the capture and replay of the processes performed by a user when exploring analysis results.

The study of the genetic causes of disease is entering a new era. Variations in DNA sequence between individuals at a single position (locus) within the human genome are termed single nucleotide polymorphisms (SNPs), and may lead to a frank disease state or a variation in normal physiology. By comparing and contrasting the genomes of people who have a disease with the genomes of people who don't, we can begin to identify those genetic locii which potentially play a role in the disease. Modern biotechnology allows for the genotyping of individuals at hundreds of thousands of genetic locii. Whilst metrics to quantify the statistical importance of a single locus are essentially of low complexity, for example calculation of a x2statistic, within a genome-wide association study this process is repeated at every locus. In addition, the entire computational process is often repeated with a number of randomised data sets, necessary for estimation of the statistical significance. The large number of locii, number of randomized data sets, and rapid combinatorial increase when analysing multiple SNPs, naturally dictates that a high performance computing (HPC) solution be developed. On a single core machine analysis of significant numbers of SNP pairs would take many years. Once statistical analysis of the data has been performed results must be annotated with relevant information to aid biological interpretation and hypothesis generation - this is a standard, but not in substantial bioinformatic task.

We describe the experience of using a combination of requirements engineering techniques (scenarios, storyboards, observation and workshops) in an e-science application to develop a geographical analysis tool for epidemiologists. Problems encountered were: eliciting tacit knowledge; and creating new visions and working practices for our users. The combination of techniques worked well, although observation of working practice was not so effective in this scientific domain, where activity is mainly cognitive.

The process of hypothesis-driven epidemiological research has three phases - the establishment and characterisation of a large, representative cohort from a geographically distributed population; the integration of the cohort data with other data sources to provide additional characterisation; the formulation of a hypothesis and generation of the corresponding predictions. Grid-computing technologies make possible secure, distributed collaboration, and the ability to share data sources, computational resources and storage resources across administrative boundaries. PsyGrid is an e-Science project established to apply grid-computing technologies to each of the three phases, with the aim of eliminating the obstacles that hinder epidemiological research. We describe a system for distributed cohort characterisation, and the first application to the study of First Episode Psychosis.