WARNING: This paper contains content that maybe upsetting or offensive to some readers.Dog whistles are coded expressions with dual meanings: one intended for the general public (outgroup) and another that conveys a specific message to an intended audience (ingroup).Often, these expressions are used to convey controversial political opinions while maintaining plausible deniability and slip by content moderation filters.Identification of dog whistles relies on curated lexicons, which have trouble keeping up to date.We introduce FETCH!, a task for finding emergent dog whistles in massive social media corpora.We find that stateof-the-art systems fail to achieve meaningful results across three distinct social media case studies.We present EarShot, a strong baseline system that combines the strengths of vector databases and Large Language Models (LLMs) to efficiently and effectively identify new dog whistles.1 Scenario Filtering Model Keyword Extraction Model Threshold Max n-grams Prec DPR F 0.5 Balanced RoBERTA R4 KeyBERT 50 1 14.81 1.65 5.70 LLaMa 13B KeyBERT 100 1 11.43 1.65 5.22 Synthetic RoBERTA R4 KeyBERT 800 1 14.86 13.45 14.55 LLaMA 8B KeyBERT 400 1 19.13 7.14 14.32

Plain Language Summarization (PLS) aims to distill complex documents into accessible summaries for non-expert audiences.In this paper, we conduct a thorough survey of PLS literature, and identify that the current standard practice for readability evaluation is to use traditional readability metrics, such as Flesch-Kincaid Grade Level (FKGL).However, despite proven utility in other fields, these metrics have not been compared to human readability judgments in PLS.We evaluate 8 readability metrics and show that most correlate poorly with human judgments, including the most popular metric, FKGL.We then show that Language Models (LMs) are better judges of readability, with the best-performing model achieving a Pearson correlation of 0.56 with human judgments.Extending our analysis to PLS datasets, which contain summaries aimed at non-expert audiences, we find that LMs better capture deeper measures of readability, such as required background knowledge, and lead to different conclusions than the traditional metrics.Based on these findings, we offer recommendations for best practices in the evaluation of plain language summaries.We release our analysis code and survey data.

Kelvin Luu, Xinyi Wu, Rik Koncel-Kedziorski, Kyle Lo, Isabel Cachola, Noah A. Smith. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.

We present SciA11y, a system that renders inaccessible scientific paper PDFs into HTML. SciA11y uses machine learning models to extract and understand the content of scientific PDFs, and reorganizes the resulting paper components into a form that better supports skimming and scanning for blind and low vision (BLV) readers. SciA11y adds navigation features such as tagged headings, a table of contents, and bidirectional links between inline citations and references, which allow readers to resolve citations without losing their context. A set of 1.5 million open access papers are processed and available at https://scia11y.org/. This system is a first step in addressing scientific PDF accessibility, and may significantly improve the experience of paper reading for BLV users.

While machine learning models can produce accurate predictions of complex real-world phenomena, domain experts may be unwilling to trust such a prediction without an explanation of the model’s behavior. This concern has motivated widespread research and produced many methods for interpreting black-box models. Many such methods explain predictions one-by-one, which can be slow and inconsistent across a large dataset, and ill-suited for time series applications. We introduce a proxy model approach that is fast to train, faithful to the original model, and globally consistent in its explanations. We compare our approach to several previous methods and find both that methods disagree with one another and that our approach improves over existing methods in an application to political event forecasting.

Vulgarity is a common linguistic expression and is used to perform several linguistic functions. Understanding their usage can aid both linguistic and psychological phenomena as well as benefit downstream natural language processing applications such as sentiment analysis. This study performs a large-scale, data-driven empirical analysis of vulgar words using social media data. We analyze the socio-cultural and pragmatic aspects of vulgarity using tweets from users with known demographics. Further, we collect sentiment ratings for vulgar tweets to study the relationship between the use of vulgar words and perceived sentiment and show that explicitly modeling vulgar words can boost sentiment analysis performance.

Vulgar words are employed in language use for several different functions, ranging from expressing aggression to signaling group identity or the informality of the communication.This versatility of usage of a restricted set of words is challenging for downstream applications and has yet to be studied quantitatively or using natural language processing techniques.We introduce a novel data set of 7,800 tweets from users with known demographic traits where all instances of vulgar words are annotated with one of the six categories of vulgar word use.Using this data set, we present the first analysis of the pragmatic aspects of vulgarity and how they relate to social factors.We build a model able to predict the category of a vulgar word based on the immediate context it appears in with 67.4 macro F1 across six classes.Finally, we demonstrate the utility of modeling the type of vulgar word use in context by using this information to achieve state-of-the-art performance in hate speech detection on a benchmark data set.