Identifying named entities in written text is an essential component of the text processing pipeline used in applications such as text editors to gain a better understanding of the semantics of the text.However, the typical experimental setup for evaluating Named Entity Recognition (NER) systems is not directly applicable to systems that process text in real time as the text is being typed.Evaluation is performed on a sentence level assuming the end-user is willing to wait until the entire sentence is typed for entities to be identified and further linked to identifiers or coreferenced.We introduce a novel experimental setup for NER systems for applications where decisions about named entity boundaries need to be performed in an online fashion.We study how state-of-the-art methods perform under this setup in multiple languages and propose adaptations to these models to suit this new experimental setup.Experimental results show that the best systems that are evaluated on each token after its typed, reach performance within 1-5 F 1 points of systems that are evaluated at the end of the sentence.These show that entity recognition can be performed in this setup and open up the development of other NLP tools in a similar setup.

Named entity recognition (NER) is the backbone of many NLP solutions.F 1 score, the harmonic mean of precision and recall, is often used to select/evaluate the best models.However, when precision needs to be prioritized over recall, a state-of-the-art model might not be the best choice.There is little in the literature that directly addresses training-time modifications to achieve higher precision information extraction.In this paper, we propose a neural semi-Markov structured support vector machine model that controls the precisionrecall trade-off by assigning weights to different types of errors in the loss-augmented inference during training.The semi-Markov property provides more accurate phrase-level predictions, thereby improving performance.We empirically demonstrate the advantage of our model when high precision is required by comparing against strong baselines based on CRF.In our experiments with the CoNLL 2003 dataset, our model achieves a better precisionrecall trade-off at various precision levels.ORG PER LOC MISC ALL CRF P. 89.5 96.3 91.8 81.1 91.06 R. 87.7 95.4 93.8 81.3 90.88 F1 88.6 95.8 92.8 81.2 90.97 SSVM P. 90.0 95.7 91.0 80.4 90.75 R. 87.7 95.5 93.7 80.5 90.79 F1 88.8 95.6 92.4 80.4 90.77Semi.SSVM P. 89.3 96.0 92.3 80.1 90.92 R. 87.2 95.2 93.2 81.9 90.60 F1 88.2 95.6 92.8 81.0 90.76

Supervised machine learning assumes the availability of fully-labeled data, but in many cases, such as low-resource languages, the only data available is partially annotated.We study the problem of Named Entity Recognition (NER) with partially annotated training data in which a fraction of the named entities are labeled, and all other tokens, entities or otherwise, are labeled as non-entity by default.In order to train on this noisy dataset, we need to distinguish between the true and false negatives.To this end, we introduce a constraintdriven iterative algorithm that learns to detect false negatives in the noisy set and downweigh them, resulting in a weighted training set.With this set, we train a weighted NER model.We evaluate our algorithm with weighted variants of neural and non-neural NER models on data in 8 languages from several language and script families, showing strong ability to learn from partial data.Finally, to show real-world efficacy, we evaluate on a Bengali NER corpus annotated by non-speakers, outperforming the prior state-of-the-art by over 5 points F1.

A notable challenge in cross-lingual wikification is the problem of retrieving English Wikipedia title candidates given a non-English mention, a step that requires translating names written in a foreign language into English. Creating training data for name translation requires significant amount of human efforts. In order to cover as many languages as possible, we propose a probabilistic model that leverages indirect supervision signals in a knowledge base. More specifically, the model learns name translation from title pairs obtained from the inter-language links in Wikipedia. The model jointly considers word alignment and word transliteration. Comparing to 6 other approaches on 9 languages, we show that the proposed model outperforms others not only on the transliteration metric, but also on the ability to generate target English titles for a cross-lingual wikifier. Consequently, as we show, it improves the end-to-end performance of a cross-lingual wikifier on the TAC 2016 EDL dataset.

The problem of entity-typing has been studied predominantly in supervised learning fashion, mostly with task-specific annotations (for coarse types) and sometimes with distant supervision (for fine types).While such approaches have strong performance within datasets, they often lack the flexibility to transfer across text genres and to generalize to new type taxonomies.In this work we propose a zero-shot entity typing approach that requires no annotated data and can flexibly identify newly defined types.Given a type taxonomy defined as Boolean functions of FREEBASE "types", we ground a given mention to a set of type-compatible Wikipedia entries and then infer the target mention's types using an inference algorithm that makes use of the types of these entries.We evaluate our system on a broad range of datasets, including standard fine-grained and coarse-grained entity typing datasets, and also a dataset in the biological domain.Our system is shown to be competitive with state-of-theart supervised NER systems and outperforms them on out-of-domain datasets.We also show that our system significantly outperforms other zero-shot fine typing systems.

Recent work in NLP has attempted to deal with low-resource languages but still assumed a resource level that is not present for most languages, e.g., the availability of Wikipedia in the target language. We propose a simple method for cross-lingual named entity recognition (NER) that works well in settings with very minimal resources. Our approach makes use of a lexicon to "translate" annotated data available in one or several high resource language(s) into the target language, and learns a standard monolingual NER model there. Further, when Wikipedia is available in the target language, our method can enhance Wikipedia based methods to yield state-of-the-art NER results; we evaluate on 7 diverse languages, improving the state-of-the-art by an average of 5.5% F1 points. With the minimal resources required, this is an extremely portable cross-lingual NER approach, as illustrated using a truly low-resource language, Uyghur.

Named Entity Recognition (NER) models for language L are typically trained using annotated data in that language.We study cross-lingual NER, where a model for NER in L is trained on another, source, language (or multiple source languages).We introduce a language independent method for NER, building on cross-lingual wikification, a technique that grounds words and phrases in non-English text into English Wikipedia entries.Thus, mentions in any language can be described using a set of categories and FreeBase types, yielding, as we show, strong language-independent features.With this insight, we propose an NER model that can be applied to all languages in Wikipedia.When trained on English, our model outperforms comparable approaches on the standard CoNLL datasets (Spanish, German, and Dutch) and also performs very well on lowresource languages (e.g., Turkish, Tagalog, Yoruba, Bengali, and Tamil) that have significantly smaller Wikipedia.Moreover, our method allows us to train on multiple source languages, typically improving NER results on the target languages.Finally, we show that our languageindependent features can be used also to enhance monolingual NER systems, yielding improved results for all 9 languages.

Cross-lingual Wikification is the task of grounding mentions written in non-English documents to entries in the English Wikipedia.This task involves the problem of comparing textual clues across languages, which requires developing a notion of similarity between text snippets across languages.In this paper, we address this problem by jointly training multilingual embeddings for words and Wikipedia titles.The proposed method can be applied to all languages represented in Wikipedia, including those for which no machine translation technology is available.We create a challenging dataset in 12 languages and show that our proposed approach outperforms various baselines.Moreover, our model compares favorably with the best systems on the TAC KBP2015 Entity Linking task including those that relied on the availability of translation from the target language to English.

We consider the problem of disambiguating concept mentions appearing in documents and grounding them in multiple knowledge bases, where each knowledge base addresses some aspects of the domain. This problem poses a few additional challenges beyond those addressed in the popular Wikification problem. Key among them is that most knowledge bases do not contain the rich textual and structural information Wikipedia does; consequently, the main supervision signal used to train Wikification rankers does not exist anymore. In this work we develop an algorithmic approach that, by carefully examining the relations between various related knowledge bases, generates an indirect supervision signal it uses to train a ranking model that accurately chooses knowledge base entries for a given mention; moreover, it also induces prior knowledge that can be used to support a global coherent mapping of all the concepts in a given document to the knowledge bases. Using the biomedical domain as our application, we show that our indirectly supervised ranking model outperforms other unsupervised baselines and that the quality of this indirect supervision scheme is very close to a supervised model. We also show that considering multiple knowledge bases together has an advantage over grounding concepts to each knowledge base individually.

Most recent question answering (QA) systems query large-scale knowledge bases (KBs) to answer a question, after parsing and transforming natural language questions to KBs-executable forms (e.g., logical forms). As a well-known fact, KBs are far from complete, so that information required to answer questions may not always exist in KBs. In this paper, we develop a new QA system that mines answers directly from the Web, and meanwhile employs KBs as a significant auxiliary to further boost the QA performance. Specifically, to the best of our knowledge, we make the first attempt to link answer candidates to entities in Freebase, during answer candidate generation. Several remarkable advantages follow: (1) Redundancy among answer candidates is automatically reduced. (2) The types of an answer candidate can be effortlessly determined by those of its corresponding entity in Freebase. (3) Capitalizing on the rich information about entities in Freebase, we can develop semantic features for each answer candidate after linking them to Freebase. Particularly, we construct answer-type related features with two novel probabilistic models, which directly evaluate the appropriateness of an answer candidate's types under a given question. Overall, such semantic features turn out to play significant roles in determining the true answers from the large answer candidate pool. The experimental results show that across two testing datasets, our QA system achieves an 18%~54% improvement under F_1 metric, compared with various existing QA systems.

This paper studies the importance of identifying and categorizing scientific concepts as a way to achieve a deeper understanding of the research literature of a scientific community. To reach this goal, we propose an unsupervised bootstrapping algorithm for identifying and categorizing mentions of concepts. We then propose a new clustering algorithm that uses citations' context as a way to cluster the extracted mentions into coherent concepts. Our evaluation of the algorithms against gold standards shows significant improvement over state-of-the-art results. More importantly, we analyze the computational linguistic literature using the proposed algorithms and show four different ways to summarize and understand the research community which are difficult to obtain using existing techniques.