Neural abstractive summarization models are able to generate summaries which have high overlap with human references.However, existing models are not optimized for factual correctness, a critical metric in real-world applications.In this work, we develop a general framework where we evaluate the factual correctness of a generated summary by factchecking it automatically against its reference using an information extraction module.We further propose a training strategy which optimizes a neural summarization model with a factual correctness reward via reinforcement learning.We apply the proposed method to the summarization of radiology reports, where factual correctness is a key requirement.On two separate datasets collected from hospitals, we show via both automatic and human evaluation that the proposed approach substantially improves the factual correctness and overall quality of outputs over a competitive neural summarization system, producing radiology summaries that approach the quality of humanauthored ones.Background: radiographic examination of the chest.clinical history: 80 years of age, male ... Findings: frontal radiograph of the chest demonstrates repositioning of the right atrial lead possibly into the ivc.... a right apical pneumothorax can be seen from the image.moderate right and small left pleural effusions continue.no pulmonary edema is observed.heart size is upper limits of normal. Human Summary: pneumothorax is seen. bilateral pleural effusions continue.Summary A (ROUGE-L = 0.77): no pneumothorax is observed.bilateral pleural effusions continue.Summary B (ROUGE-L = 0.44): pneumothorax is observed on radiograph.bilateral pleural effusions continue to be seen.

Objective was to assess the relationship between task demand, mental effort, task difficulty, and performance during physicians’ interaction with electronic health records (EHRs). Seventeen physicians performed three EHR-based scenarios with varying task demands. Mental effort was measured using eye tracking measures via task evoked pupillary responses (TEPR), blink frequency, and gaze speed; task difficulty (or user behavior) was measured using frequent mouse click patterns and task flow; user performance was quantified using two types of omission errors: (i) omission errors with no evidence of trying to complete the task and (ii) omission error with evidence of trying but unable to complete the task. The results indicated that task demand significantly increased mental effort, but not task difficulty. Task demand, mental effort, and task difficulty all predicted performance. Specifically, there was a significant relationship between (i) task demand, TEPR and omission errors with no evidence of trying to complete the task, and (ii) blink frequency, repeated search clicks and omission error with evidence of trying but unable to complete the task. In concert, results suggest that physicians’ performance during EHR interaction was negatively affected by task demands and increase in mental effort. This highlight the need for implementation of appropriate quality assurance (QA) measures, in addition to EHR usability improvement, to minimize omission errors and improve physician’s performance. Additionally, the lack of relationship between task demand and task difficulty highlights a need for further methodological and empirical studies to advance our understanding from theory to application during physician–EHR interaction.