Disease named entity recognition (NER) is an important enabling technology to develop various downstream biomedical natural language processing applications. This is a challenging task, which requires addressing potential ambiguities due to variable contextual usage of the disease name mentions in clinical texts. In particular, clinical trial texts have unique complexities compared to patient-focused clinical reports or information-rich biomedical research articles, as they typically define drug testing eligibility requirements for patient cohorts via compound contextual and logical relationships. In this paper, we propose a novel disease NER model for clinical trial texts by using deep contextual embeddings with relevant domain-specific features, word embeddings, and character embeddings in a bidirectional long short-term memory network-conditional random field (BiLSTM-CRF) framework. Experiments and analyses on a clinical trial dataset and the benchmark NCBI scientific article dataset show the effectiveness of the proposed model.

Biomedical Entity Recognition tasks have gained significant importance in the clinical research domain. There has been a lot of prior work on improving entity recognition of biomedical concepts from using rule-based to more context-dependent deep learning-based approaches. However, due to its high domain dependency and distinctive vocabularies, appropriate utilization of contextual knowledge becomes a challenge even for context dependent deep models. To this end, we propose a novel performance-boosting improvement module “BoostER” that can be applied to any existing entity recognition system to boost its performance in terms of precision and F1 scores. The proposed module has been developed on top of a pre-trained BERT model and fine-tuned to give more weights to contextual learning compared to word-specific information. We tested our system with Chemical and Disease Entity Recognition tasks using the BioCreative CDR dataset to demonstrate its effectiveness compared to existing state-of-the-art models.

Reza Ghaeini, Sadid A. Hasan, Vivek Datla, Joey Liu, Kathy Lee, Ashequl Qadir, Yuan Ling, Aaditya Prakash, Xiaoli Fern, Oladimeji Farri. Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers). 2018.

Diagnosis of a clinical condition is a challenging task, which often requires significant medical investigation. Previous work related to diagnostic inferencing problems mostly consider multivariate observational data (e.g. physiological signals, lab tests etc.). In contrast, we explore the problem using free-text medical notes recorded in an electronic health record (EHR). Complex tasks like these can benefit from structured knowledge bases, but those are not scalable. We instead exploit raw text from Wikipedia as a knowledge source. Memory networks have been demonstrated to be effective in tasks which require comprehension of free-form text. They use the final iteration of the learned representation to predict probable classes. We introduce condensed memory neural networks (C-MemNNs), a novel model with iterative condensation of memory representations that preserves the hierarchy of features in the memory. Experiments on the MIMIC-III dataset show that the proposed model outperforms other variants of memory networks to predict the most probable diagnoses given a complex clinical scenario.

Clinical diagnosis is a critical aspect of patient care that is typically driven by expert medical knowledge and intuition. An automated system for clinical diagnosis could reduce the cognitive burden of clinicians during patient care and medical education. In this paper, we describe a Knowledge Graph (KG)-based clinical diagnosis system that leverages publicly available knowledge sources to infer possible diagnoses from free-text clinical narratives. We experiment with the content in various sections of a clinical document within the electronic health record (EHR) to investigate the contribution of each section to the performance of automated diagnosis systems. Evaluation on MIMIC-III dataset demonstrates that the content of “history of present illness” and “past medical history” sections can play a greater role for clinical diagnosis inference than other sections and all sections combined. Comparison with a state-of-the-art deep learning-based clinical diagnosis system confirms the effectiveness of our system.

Clinical diagnosis is a critical and non-trivial aspect of patient care which often requires significant medical research and investigation based on an underlying clinical scenario. This paper proposes a novel approach by formulating clinical diagnosis as a reinforcement learning problem. During training, the reinforcement learning agent mimics the clinician’s cognitive process and learns the optimal policy to obtain the most appropriate diagnoses for a clinical narrative. This is achieved through an iterative search for candidate diagnoses from external knowledge sources via a sentence-by-sentence analysis of the inherent clinical context. A deep Q-network architecture is trained to optimize a reward function that measures the accuracy of the candidate diagnoses. Experiments on the TREC CDS datasets demonstrate the effectiveness of our system over various non-reinforcement learning-based systems.

Current Adverse Drug Events (ADE) surveillance systems are often associated with a sizable time lag before such events are published. Online social media such as Twitter could describe adverse drug events in real-time, prior to official reporting. Deep learning has significantly improved text classification performance in recent years and can potentially enhance ADE classification in tweets. However, these models typically require large corpora with human expert-derived labels, and such resources are very expensive to generate and are hardly available. Semi-supervised deep learning models, which offer a plausible alternative to fully supervised models, involve the use of a small set of labeled data and a relatively larger collection of unlabeled data for training. Traditionally, these models are trained on labeled and unlabeled data from similar topics or domains. In reality, millions of tweets generated daily often focus on disparate topics, and this could present a challenge for building deep learning models for ADE classification with random Twitter stream as unlabeled training data. In this work, we build several semi-supervised convolutional neural network (CNN) models for ADE classification in tweets, specifically leveraging different types of unlabeled data in developing the models to address the problem. We demonstrate that, with the selective use of a variety of unlabeled data, our semi-supervised CNN models outperform a strong state-of-the-art supervised classification model by +9.9% F1-score. We evaluated our models on the Twitter data set used in the PSB 2016 Social Media Shared Task. Our results present the new state-of-the-art for this data set.

In this paper, we propose a novel neural approach for paraphrase generation. Conventional paraphrase generation methods either leverage hand-written rules and thesauri-based alignments, or use statistical machine learning principles. To the best of our knowledge, this work is the first to explore deep learning models for paraphrase generation. Our primary contribution is a stacked residual LSTM network, where we add residual connections between LSTM layers. This allows for efficient training of deep LSTMs. We evaluate our model and other state-of-the-art deep learning models on three different datasets: PPDB, WikiAnswers, and MSCOCO. Evaluation results demonstrate that our model outperforms sequence to sequence, attention-based, and bi-directional LSTM models on BLEU, METEOR, TER, and an embedding-based sentence similarity metric.

The computing modules in distributed scientific workflows must be mapped to computer nodes in shared network environments for optimal workflow performance. Finding a good workflow mapping scheme critically depends on an accurate prediction of the execution time of each individual computational module in the workflow. The time prediction of a scientific computation does not have a silver bullet as it is determined collectively by several dynamic system factors including concurrent loads, memory size, CPU speed, and also by the complexity of the computational program itself. This paper investigates the problem of modeling scientific computations and predicting their execution time based on a combination of both hardware and software properties. We employ statistical learning techniques to estimate the effective computational power of a given computer node at any point of time and estimate the total number of CPU cycles needed for executing a given computational program on any input data size. We analytically derive an upper bound of the estimation error for execution time prediction given the hardware and software properties. The proposed statistical analysis-based solution to performance modeling and prediction is validated and justified by experimental results measured on the computing nodes that vary significantly in terms of the hardware specifications.