The capacity of large language models (LLMs) to understand and distinguish socially unacceptable texts enables them to play a promising role in abusive language detection. However, various factors can affect their sensitivity. In this work, we test whether LLMs have an unintended bias in abusive language detection, i.e., whether they predict more or less of a given abusive class than expected in zero-shot settings. Our results show that instruction-tuned LLMs tend to under-predict positive classes, since datasets used for tuning are dominated by the negative class. On the contrary, models fine-tuned with human feedback tend to be overly sensitive. In an exploratory approach to mitigate these issues, we show that label frequency in the prompt helps with the significant over-prediction.

Iterative evaluation of LLMs during training is essential to ensure expected capability development, but can be time-and compute-intensive.While NLU tasks, where the model selects from fixed answer choices, are cheap to evaluate, essential capabilities like reasoning and code generation rely on the more time-consuming NLG (token-by-token generation) format.In this work, our aim is to decrease the computational burden of NLG benchmarks in order to enable monitoring crucial LLM capabilities during model training.We reformulate generative tasks into computationally cheaper NLU alternatives.We test the performance correlation between the original and reformulated tasks using 8 LMs of various sizes and 4 capabilities: mathematical reasoning, code generation, factual knowledge and reading comprehension.Our results show a strong correlation between task formats, supporting capability assessment via cheaper alternatives and achieving over 35× average reduction in evaluation time.

Faeze Ghorbanpour, Viktor Hangya, Alexander Fraser. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2025.

Due to the broad range of social media platforms, the requirements of abusive language detection systems are varied and ever-changing. Already a large set of annotated corpora with different properties and label sets were created, such as hate or misogyny detection, but the form and targets of abusive speech are constantly evolving. Since, the annotation of new corpora is expensive, in this work we leverage datasets we already have, covering a wide range of tasks related to abusive language detection. Our goal is to build models cheaply for a new target label set and/or language, using only a few training examples of the target domain. We propose a two-step approach: first we train our model in a multitask fashion. We then carry out few-shot adaptation to the target requirements. Our experiments show that using already existing datasets and only a few-shots of the target task the performance of models improve both monolingually and across languages. Our analysis also shows that our models acquire a general understanding of abusive language, since they improve the prediction of labels which are present only in the target dataset and can benefit from knowledge about labels which are not directly used for the target task.

Text style transfer (TST) aims to modify the style of a text without altering its original meaning.Large language models (LLMs) demonstrate superior performance across multiple tasks, including TST.However, in zero-shot setups, they tend to directly copy a significant portion of the input text to the output without effectively changing its style.To enhance the stylistic variety and fluency of the text, we present sNeuron-TST, a novel approach for steering LLMs using style-specific neurons in TST.Specifically, we identify neurons associated with the source and target styles and deactivate source-style-only neurons to give target-style words a higher probability, aiming to enhance the stylistic diversity of the generated text.However, we find that this deactivation negatively impacts the fluency of the generated text, which we address by proposing an improved contrastive decoding method that accounts for rapid token probability shifts across layers caused by deactivated source-style neurons.Empirical experiments demonstrate the effectiveness of the proposed method on six benchmarks, encompassing formality, toxicity, politics, politeness, authorship, and sentiment 1 .

For subjective tasks such as hate detection, where people perceive hate differently, the Large Language Model's (LLM) ability to represent diverse groups is unclear.By including additional context in prompts, we comprehensively analyze LLM's sensitivity to geographical priming, persona attributes, and numerical information to assess how well the needs of various groups are reflected.Our findings on two LLMs, five languages, and six datasets reveal that mimicking persona-based attributes leads to annotation variability.Meanwhile, incorporating geographical signals leads to better regional alignment.We also find that the LLMs are sensitive to numerical anchors, indicating the ability to leverage community-based flagging efforts and exposure to adversaries.Our work provides preliminary guidelines and highlights the nuances of applying LLMs in culturally sensitive cases. 1

Pre-trained multilingual language models are the foundation of many NLP approaches, including cross-lingual transfer solutions.However, languages with small available monolingual corpora are often not well-supported by these models leading to poor performance.We propose an unsupervised approach to improve the cross-lingual representations of lowresource languages by bootstrapping word translation pairs from monolingual corpora and using them to improve language alignment in pretrained language models.We perform experiments on nine languages, using contextual word retrieval and zero-shot named entity recognition to measure both intrinsic cross-lingual word representation quality and downstream task performance, showing improvements on both tasks.Our results show that it is possible to improve pre-trained multilingual language models by relying only on non-parallel resources.

Bilingual dictionary induction (BDI) is the task of accurately translating words to the target language.It is of great importance in many low-resource scenarios where cross-lingual training data is not available.To perform BDI, bilingual word embeddings (BWEs) are often used due to their low bilingual training signal requirements.They achieve high performance, but problematic cases still remain, such as the translation of rare words or named entities, which often need to be transliterated.In this paper, we enrich BWE-based BDI with transliteration information by using Bilingual Orthography Embeddings (BOEs).BOEs represent source and target language transliteration word pairs with similar vectors.A key problem in our BDI setup is to decide which information source -BWEs (or semantics) vs. BOEs (or orthography) -is more reliable for a particular word pair.We propose a novel classification-based BDI system that uses BWEs, BOEs and a number of other features to make this decision.We test our system on English-Russian BDI and show improved performance.In addition, we show the effectiveness of our BOEs by successfully using them for transliteration mining based on cosine similarity.

This paper investigates the use of bilingual word embeddings for mining Hiligaynon translations of English words. There is very little research on Hiligaynon, an extremely low-resource language of Malayo-Polynesian origin with over 9 million speakers in the Philippines (we found just one paper). We use a publicly available Hiligaynon corpus with only 300K words, and match it with a comparable corpus in English. As there are no bilingual resources available, we manually develop a English-Hiligaynon lexicon and use this to train bilingual word embeddings. But we fail to mine accurate translations due to the small amount of data. To find out if the same holds true for a related language pair, we simulate the same low-resource setup on English to German and arrive at similar results. We then vary the size of the comparable English and German corpora to determine the minimum corpus size necessary to achieve competitive results. Further, we investigate the role of the seed lexicon. We show that with the same corpus size but with a smaller seed lexicon, performance can surpass results of previous studies. We release the lexicon of 1,200 English-Hiligaynon word pairs we created to encourage further investigation.

Mining parallel sentences from comparable corpora is important.Most previous work relies on supervised systems, which are trained on parallel data, thus their applicability is problematic in low-resource scenarios.Recent developments in building unsupervised bilingual word embeddings made it possible to mine parallel sentences based on cosine similarities of source and target language words.We show that relying only on this information is not enough, since sentences often have similar words but different meanings.We detect continuous parallel segments in sentence pair candidates and rely on them when mining parallel sentences.We show better mining accuracy on three language pairs in a standard shared task on artificial data.We also provide the first experiments showing that parallel sentences mined from real life sources improve unsupervised MT.Our code is available, we hope it will be used to support low-resource MT research.

Unseen words, also called out-of-vocabulary words (OOVs), are difficult for machine translation.In neural machine translation, byte-pair encoding can be used to represent OOVs, but they are still often incorrectly translated.We improve the translation of OOVs in NMT using easy-to-obtain monolingual data.We look for OOVs in the text to be translated and translate them using simple-to-construct bilingual word embeddings (BWEs).In our MT experiments we take the 5-best candidates, which is motivated by intrinsic mining experiments.Using all five of the proposed target language words as queries we mine target-language sentences.We then back-translate, forcing the back-translation of each of the five proposed target-language OOV-translation-candidates to be the original source-language OOV.We show that by using this synthetic data to finetune our system the translation of OOVs can be dramatically improved.In our experiments we use a system trained on Europarl and mine sentences containing medical terms from monolingual data.

Bilingual tasks, such as bilingual lexicon induction and cross-lingual classification, are crucial for overcoming data sparsity in the target language.Resources required for such tasks are often out-of-domain, thus domain adaptation is an important problem here.We make two contributions.First, we test a delightfully simple method for domain adaptation of bilingual word embeddings.We evaluate these embeddings on two bilingual tasks involving different domains: cross-lingual twitter sentiment classification and medical bilingual lexicon induction.Second, we tailor a broadly applicable semi-supervised classification method from computer vision to these tasks.We show that this method also helps in low-resource setups.Using both methods together we achieve large improvements over our baselines, by using only additional unlabeled data.