Abstract Coreference resolution is the task of identifying and clustering mentions that refer to the same entity in a document. Based on state-of-the-art deep learning approaches, end-to-end coreference resolution considers all spans as candidate mentions and tackles mention detection and coreference resolution simultaneously. Recently, researchers have attempted to incorporate document-level context using higher-order inference (HOI) to improve end-to-end coreference resolution. However, HOI methods have been shown to have marginal or even negative impact on coreference resolution. In this paper, we reveal the reasons for the negative impact of HOI coreference resolution. Contextualized representations (e.g., those produced by BERT) for building span embeddings have been shown to be highly anisotropic. We show that HOI actually increases and thus worsens the anisotropy of span embeddings and makes it difficult to distinguish between related but distinct entities (e.g., pilots and flight attendants ). Instead of using HOI, we propose two methods, Less-Anisotropic Internal Representations (LAIR) and Data Augmentation with Document Synthesis and Mention Swap (DSMS), to learn less-anisotropic span embeddings for coreference resolution. LAIR uses a linear aggregation of the first layer and the topmost layer of contextualized embeddings. DSMS generates more diversified examples of related but distinct entities by synthesizing documents and by mention swapping. Our experiments show that less-anisotropic span embeddings improve the performance significantly (+2.8 F1 gain on the OntoNotes benchmark) reaching new state-of-the-art performance on the GAP dataset.

Fine-grained entity typing (FGET) is an important natural language processing task. It is to assign fine-grained semantic types of a type taxonomy (e.g., Person/artist/actor) to entity mentions. Fine-grained entity semantic types have been successfully applied in many natural language processing (NLP) applications, such as relation extraction, entity linking and question answering. The key challenge for FGET is how to deal with label noises that disperse in the corpora since the corpora are normally automatically annotated. Various type taxonomies, typing methods and representation learning approaches for FGET have been proposed and developed in the past two decades. This paper systematically categorizes and reviews these various typing methods and representation learning approaches to provide a reference for future studies on FGET. We identify the current trends in FGET research: (i) Learning embedded feature representations to address the challenges posed by label noises, tail types and new entities; (ii) Tackling FGET jointly with other entity analysis sub-tasks (e.g., entity linking and coreference resolution) is also a promising direction. We also present a comprehensive review of type taxonomies, resources, applications for FGET and methods for automatically generating FGET training corpora.