Özge Alacam, Sanne Hoeken, Andreas Säuberli, Hannes Gröner, Diego Frassinelli, Sina Zarrieß, Barbara Plank. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025.

Hate speech is a complex and subjective phenomenon.In this paper, we present a dataset (GAZE4HATE) that provides gaze data collected in a hate speech annotation experiment.We study whether the gaze of an annotator provides predictors of their subjective hatefulness rating, and how gaze features can improve Hate Speech Detection (HSD).We conduct experiments on statistical modeling of subjective hate ratings and gaze and analyze to what extent rationales derived from hate speech models correspond to human gaze and explanations in our data.Finally, we introduce MEANION, a first gaze-integrated HSD model.Our experiments show that particular gaze features like dwell time or fixation counts systematically correlate with annotators' subjective hate rating, and improve predictions of text-only hate speech models.

Hate speech detection is a prevalent research field, yet it remains underexplored at the level of word meaning.This is significant, as terms used to convey hate often involve non-standard or novel usages which might be overlooked by commonly leveraged LMs trained on general language use.In this paper, we introduce the Hateful Word in Context Classification (HateWiC) task and present a dataset of ∼4000 WiC-instances, each labeled by three annotators.Our analyses and computational exploration focus on the interplay between the subjective nature (context-dependent connotations) and the descriptive nature (as described in dictionary definitions) of hateful word senses.HateWiC annotations confirm that hatefulness of a word in context does not always derive from the sense definition alone.We explore the prediction of both majority and individual annotator labels, and we experiment with modeling context-and sense-based inputs.Our findings indicate that including definitions proves effective overall, yet not in cases where hateful connotations vary.Conversely, including annotator demographics becomes more important for mitigating performance drop in subjective hate prediction.