The Latin script is often used to informally write languages with non-Latin native scripts.In many cases (e.g., most languages in India), the lack of conventional spelling in the Latin script results in high spelling variability.Such romanization renders languages that are normally easily distinguished due to being written in different scripts -Hindi and Urdu, for example -highly confusable.In this work, we increase language identification (LID) accuracy for romanized text by improving the methods used to synthesize training sets.We find that training on synthetic samples which incorporate natural spelling variation yields higher LID system accuracy than including available naturally occurring examples in the training set, or even training higher capacity models.We demonstrate new state-of-the-art LID performance on romanized text from 20 Indic languages in the Bhasha-Abhijnaanam evaluation set (Madhani et al., 2023a), improving test F1 from the reported 74.7% (using a pretrained neural model) to 85.4% using a linear classifier trained solely on synthetic data and 88.2% when also training on available harvested text.

Autoregressive language models have achieved remarkable advancements, yet their potential is often limited by the slow inference speeds associated with sequential token generation. Blockwise parallel decoding (BPD) was proposed by Stern et al. [42] as a method to improve inference speed of language models by simultaneously predicting multiple future tokens, termed block drafts, which are subsequently verified by the autoregressive model. This paper advances the understanding and improvement of block drafts in two ways. First, we analyze token distributions generated across multiple prediction heads. Second, leveraging these insights, we propose algorithms to improve BPD inference speed by refining the block drafts using task-independent \ngram and neural language models as lightweight rescorers. Experiments demonstrate that by refining block drafts of open-sourced Vicuna and Medusa LLMs, the mean accepted token length are increased by 5-25% relative. This results in over a 3x speedup in wall clock time compared to standard autoregressive decoding in open-source 7B and 13B LLMs.

We propose the task of updated headline generation, in which a system generates a headline for an updated article, considering both the previous article and headline.The system must identify the novel information in the article update, and modify the existing headline accordingly.We create data for this task using the NewsEdits corpus (Spangher and May, 2021) by automatically identifying contiguous article versions that are likely to require a substantive headline update.We find that models conditioned on the prior headline and body revisions produce headlines judged by humans to be as factual as gold headlines while making fewer unnecessary edits compared to a standard headline generation model.Our experiments establish benchmarks for this new contextual summarization task.

Part of speech (POS) tagging is a familiar NLP task.State of the art taggers routinely achieve token-level accuracies of over 97% on news body text, evidence that the problem is wellunderstood.However, the register of English news headlines, "headlinese", is very different from the register of long-form text, causing POS tagging models to underperform on headlines.In this work, we automatically annotate news headlines with POS tags by projecting predicted tags from corresponding sentences in news bodies.We train a multi-domain POS tagger on both long-form and headline text and show that joint training on both registers improves over training on just one or naïvely concatenating training sets.We evaluate on a newly-annotated corpus of over 5,248 English news headlines from the Google sentence compression corpus, and show that our model yields a 23% relative error reduction per token and 19% per headline.In addition, we demonstrate that better headline POS tags can improve the performance of a syntax-based open information extraction system.We make POSH, the POS-tagged Headline corpus, available to encourage research in improved NLP models for news headlines.

In recent years, few-shot models have been applied successfully to a variety of NLP tasks.Han et al. (2018) introduced a few-shot learning framework for relation classification, and since then, several models have surpassed human performance on this task, leading to the impression that few-shot relation classification is solved.In this paper we take a deeper look at the efficacy of strong few-shot classification models in the more common relation extraction setting, and show that typical few-shot evaluation metrics obscure a wide variability in performance across relations.In particular, we find that state of the art few-shot relation classification models overly rely on entity type information, and propose modifications to the training routine to encourage models to better discriminate between relations involving similar entity types.

Tianze Shi, Adrian Benton, Igor Malioutov, Ozan İrsoy. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2021.

Pallavi Patil, Kriti Myer, Ronak Zala, Arpit Singh, Sheshera Mysore, Andrew McCallum, Adrian Benton, Amanda Stent. Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL). 2019.

Adrian Benton, Mark Dredze. Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers). 2018.

We introduce initial groundwork for estimating suicide risk and mental health in a deep learning framework.By modeling multiple conditions, the system learns to make predictions about suicide risk and mental health at a low false positive rate.Conditions are modeled as tasks in a multitask learning (MTL) framework, with gender prediction as an additional auxiliary task.We demonstrate the effectiveness of multi-task learning by comparison to a well-tuned single-task baseline with the same number of parameters.Our best MTL model predicts potential suicide attempt, as well as the presence of atypical mental health, with AUC > 0.8.We also find additional large improvements using multi-task learning on mental health tasks with limited training data.* Now at Google Research. 1 https://www.nami.org/Learn-More/Mental-Health-Conditions/Related-Conditions/Suicide#sthash.dMAhrKTU.dpuf 2 Communication with clinicians at the 2016 JSALT workshop (Hollingshead, 2016).

This paper considers survey prediction from social media. We use topic models to correlate social media messages with survey outcomes and to provide an interpretable representation of the data. Rather than rely on fully unsupervised topic models, we use existing aggregated survey data to inform the inferred topics, a class of topic model supervision referred to as collective supervision. We introduce and explore a variety of topic model variants and provide an empirical analysis, with conclusions of the most effective models for this task.

Research on entity linking has considered a broad range of text, including newswire, blogs and web documents in multiple languages.However, the problem of entity linking for spoken language remains unexplored.Spoken language obtained from automatic speech recognition systems poses different types of challenges for entity linking; transcription errors can distort the context, and named entities tend to have high error rates.We propose features to mitigate these errors and evaluate the impact of ASR errors on entity linking using a new corpus of entity linked broadcast news transcripts.

SUMMARY: We have developed medpie, a software package for preparing medical message board corpora and extracting patient mentions and statistics for drugs, herbs and adverse effects experienced from them. The package is divided into web-crawling, HTML-cleaning, de-identification and information extraction modules. It also includes a sample controlled vocabulary of drugs, herbs and adverse effect terms. AVAILABILITY: http://www.cis.upenn.edu/~ungar/medpie.zip. DEPENDENCIES: Python 2.6 or 2.7.

There are millions of public posts to medical message boards by users seeking support and information on a wide range of medical conditions. It has been shown that these posts can be used to gain a greater understanding of patients' experiences and concerns. As investigators continue to explore large corpora of medical discussion board data for research purposes, protecting the privacy of the members of these online communities becomes an important challenge that needs to be met. Extant entity recognition methods used for more structured text are not sufficient because message posts present additional challenges: the posts contain many typographical errors, larger variety of possible names, terms and abbreviations specific to Internet posts or a particular message board, and mentions of the authors' personal lives. The main contribution of this paper is a system to de-identify the authors of message board posts automatically, taking into account the aforementioned challenges. We demonstrate our system on two different message board corpora, one on breast cancer and another on arthritis. We show that our approach significantly outperforms other publicly available named entity recognition and de-identification systems, which have been tuned for more structured text like operative reports, pathology reports, discharge summaries, or newswire.

There are millions of public posts to medical message boards by users seeking support and information on a wide range of medical conditions. It has been shown that these posts can be used to gain a greater understanding of patients' experiences and concerns. As investigators continue to explore large corpora of medical discussion board data for research purposes, protecting the privacy of the members of these online communities becomes an important challenge that needs to be met. Extant entity recognition methods used for more structured text are not sufficient because message posts present additional challenges: the posts contain many typographical errors, larger variety of possible names, terms and abbreviations specific to Internet posts or a particular message board, and mentions of the authors' personal lives. The main contribution of this paper is a system to de-identify the authors of discussion board posts automatically, taking into account the aforementioned challenges. We demonstrate our system on two different message board corpora, one on breast cancer and another on arthritis. We show that our approach significantly outperforms other publicly available de-identification systems, which have been tuned for more structured text like operative reports, pathology reports and discharge summaries. Our software will be available for download as open source code in the near future.