WARNING: This paper contains content that maybe upsetting or offensive to some readers.Dog whistles are coded expressions with dual meanings: one intended for the general public (outgroup) and another that conveys a specific message to an intended audience (ingroup).Often, these expressions are used to convey controversial political opinions while maintaining plausible deniability and slip by content moderation filters.Identification of dog whistles relies on curated lexicons, which have trouble keeping up to date.We introduce FETCH!, a task for finding emergent dog whistles in massive social media corpora.We find that stateof-the-art systems fail to achieve meaningful results across three distinct social media case studies.We present EarShot, a strong baseline system that combines the strengths of vector databases and Large Language Models (LLMs) to efficiently and effectively identify new dog whistles.1 Scenario Filtering Model Keyword Extraction Model Threshold Max n-grams Prec DPR F 0.5 Balanced RoBERTA R4 KeyBERT 50 1 14.81 1.65 5.70 LLaMa 13B KeyBERT 100 1 11.43 1.65 5.22 Synthetic RoBERTA R4 KeyBERT 800 1 14.86 13.45 14.55 LLaMA 8B KeyBERT 400 1 19.13 7.14 14.32

The language of Twitter differs significantly from that of other domains commonly included in large language model training.While tweets are typically multilingual and contain informal language, including emoji and hashtags, most pre-trained language models for Twitter are either monolingual, adapted from other domains rather than trained exclusively on Twitter, or are trained on a limited amount of in-domain Twitter data.We introduce Bernice, the first multilingual RoBERTa language model trained from scratch on 2.5 billion tweets with a custom tweet-focused tokenizer.We evaluate on a variety of monolingual and multilingual Twitter benchmarks, finding that our model consistently exceeds or matches the performance of a variety of models adapted to social media data as well as strong multilingual baselines, despite being trained on less data overall.We posit that it is more efficient compute-and data-wise to train completely on in-domain data with a specialized domain-specific tokenizer.

Multiple studies have demonstrated that behavior on internet-based social media platforms can be indicative of an individual's mental health status.The widespread availability of such data has spurred interest in mental health research from a computational lens.While previous research has raised concerns about possible biases in models produced from this data, no study has quantified how these biases actually manifest themselves with respect to different demographic groups, such as gender and racial/ethnic groups.Here, we analyze the fairness of depression classifiers trained on Twitter data with respect to gender and racial demographic groups.We find that model performance systematically differs for underrepresented groups and that these discrepancies cannot be fully explained by trivial data representation issues.Our study concludes with recommendations on how to avoid these biases in future research.