Abstract Incident report triage is a common endeavor in many industry sectors, often coupled with serious public safety implications. For example, at the US Food and Drug Administration (FDA), analysts triage an influx of incident reports to identify previously undiscovered drug safety problems. However, these analysts currently conduct this critical yet error‐prone incident report triage using a generic table‐based interface, with no formal support. Visualization design, task‐characterization methodologies, and evaluation models offer several possibilities for better supporting triage workflows, including those dealing with drug safety and beyond. In this work, we aim to elevate the work of triage through a task‐abstraction activity with FDA analysts. Second, we design an alternative gist‐based summary of text documents used in triage (SumRe). Third, we conduct a crowdsourced evaluation of SumRe with medical experts. Results of the crowdsourced study with medical experts (n = 20) suggest that SumRe better supports accuracy in understanding the gist of a given report, and in identifying important reports for followup activities. We discuss implications of these results, including design considerations for triage workflows beyond the drug domain, as well as methodologies for comparing visualization‐enabled text summaries.

Clinical narratives that describe complex medical events are often accompanied by meta-information such as a patient’s demographics, diagnoses and medications. This structured information implicitly relates to the logical and semantic structure of the entire narrative, and thus affects vocabulary choices for the narrative composition. To leverage this meta-information, we propose a supervised topic compositional neural language model, called MeTRNN, that integrates the strength of supervised topic modeling in capturing global semantics with the capacity of contextual recurrent neural networks (RNN) in modeling local word dependencies. MeTRNN generates interpretable topics from global meta-information and uses them to facilitate contextual RNNs in modeling local dependencies of text. For efficient training of MeTRNN, we develop an autoencoding variational Bayes inference method. We evaluate MeTRNN on the word prediction tasks using public text datasets. MeTRNN consistently outperforms all baselines across all datasets in perplexity ranging from 5% to 40%. Our case studies on real world electronic health records (EHR) data show that MeTRNN can learn and benefit from meaningful topics.

Abstract Adverse reactions caused by drug‐drug interactions are a major public health concern. Currently, adverse reaction signals are detected through a tedious manual process in which drug safety analysts review a large number of reports collected through post‐marketing drug surveillance. While computational techniques in support of this signal analysis are necessary, alone they are not sufficient. In particular, when machine learning techniques are applied to extract candidate signals from reports, the resulting set is (1) too large in size, i.e., exponential to the number of unique drugs and reactions in reports, (2) disconnected from the underlying reports that serve as evidence and context, and (3) ultimately requires human intervention to be validated in the domain context as a true signal warranting action. In this work, we address these challenges though a visual analytics system, DIVA, designed to align with the drug safety analysis workflow by supporting the detection, screening, and verification of candidate drug interaction signals. DTVA's abstractions and encodings are informed by formative interviews with drug safety analysts. DIVA's coordinated visualizations realize a proposed novel augmented interaction data model (AIM) which links signals generated by machine learning techniques with domain‐specific metadata critical for signal analysis. DIVA's alignment with the drug review process allows an analyst to interactively screen for important signals, triage signals for in‐depth investigation, and validate signals by reviewing the underlying reports that serve as evidence. The evaluation of DIVA encompasses case‐studies and interviews by drug analysts at the US Food and Drug Administration ‐ both of which confirm that DIVA indeed is effective in supporting analysts in the critical task of exploring and verifying dangerous drug‐drug interactions.

Drug-drug interaction related adverse events (DIAE) signals are a major public health issue. Drug safety analysts must sift through thousands of adverse event reports submitted daily to U.S. Food and Drug Administration (FDA) to discover unexpected DIAE signals, which if addressed can lead to life-saving actions. To facilitate the DIAE discovery from these massive data sets, we design several technological innovations that together are integrated into an interactive visual analytics system called DEVES 1. First, our state- of-the-art DIAE mining algorithm efficiently infers potential DIAE signals from these reports, and then ranks them based on their significance score. For interpretability of these inferred DIAE signals, domain knowledge of adverse events and already known drug interactions is extracted from external authoritative data sources and then seamlessly integrated with the inferred signal set. Guided by this augmented signal model, DEVES supports advanced signal analytics - empowering the analyst to interact with linked visualizations offering complementary perspectives into the signal set and its supporting evidence in the form of reports. Our demonstration showcases the technological innovations of DEVES using real-world FDA datasets, demonstrating that DEVES effectively supports the core regulatory tasks from signal screening, signal prioritization to signal validation.

Adverse drug reactions (ADRs) caused by drug-drug interactions (DDI) are a major cause of morbidity and mortality worldwide. There is a growing need for computing-supported methods that facilitate the automated signaling of DDI related ADRs (DIARs) that otherwise would remain undiscovered in millions of ADR reports. In this demonstration, we showcase our MeDIAR technology - an end-to-end DIAR signal generation, exploration and validation solution for pharmaceutical regulatory agencies to detect true DIAR signals from a drug surveillance database. MeDIAR's innovations include an efficient rule-driven learning algorithm for deriving DIAR signals from ADR reports, an innovative scoring methodology based on the proposed contextual association cluster model to rank the generated signals by their importance. Further, these ranked signals are augmented with meta information such as their significance level and their severity, along with links to their supporting ADR reports. Lastly, MeDIAR features an interactive visual analytics interface to support drug safety evaluators in reviewing and discovering unknown severe DIARs.

There is a growing need for computing-supported methods that facilitate the automated signaling of Adverse Drug Reactions (ADRs) otherwise left undiscovered from the exploding amount of ADR reports filed by patients, medical professionals and drug manufacturers. In this research, we design a Multi-Drug Adverse Reaction Analytics Strategy, called MARAS, to signal severe unknown ADRs triggered by the usage of a combination of drugs, also known as Multi-Drug Adverse Reactions (MDAR). First, MARAS features an efficient signal generation algorithm based on association rule learning that extracts non-spurious MDAR associations. Second, MARAS incorporates contextual information to detect drug combinations that are strongly associated with a set of ADRs. It groups related associations into Contextual Association Clusters (CACs) that then avail contextual information to evaluate the significance of the discovered MDAR Associations. Lastly, we use this contextual significance to rank discoveries by their notion of interestingness to signal the most compelling MDARs. To demonstrate the utility of MARAS, it is compared with state-of-the-art techniques and evaluated via case studies on datasets collected by U.S. Food and Drug Administration Adverse Event Reporting System (FAERS).