Prosodic features such as pitch, timing, and intonation are central to spoken communication, conveying emotion, intent, and discourse structure.In text-based settings, where these cues are absent, emojis act as visual surrogates that add affective and pragmatic nuance.This study examines how emojis influence prosodic realisation in speech and how listeners interpret prosodic cues to recover emoji meanings.Unlike previous work, we directly link prosody and emojis by analysing human speech data collected through a controlled elicited production task 1 .Using Bayesian multilevel modelling, we show that speakers systematically adapt their prosody based on emoji cues, and that listeners can recover intended meanings significantly above chance.Furthermore, our results reveal a clear hierarchy in prosodic shifts: greater semantic differences between emojis correspond to increased prosodic divergence.These findings suggest that emojis are meaningful carriers of prosodic intent that bridge the gap between digital text and spoken production.

Over the past decade, the use of emojis in social media has seen a rapid increase.Despite their popularity and image-grounded nature, previous studies have found that people interpret emojis inconsistently when presented in context and in isolation.In this work, we explore whether emoji semantics differ across languages and how semantics interacts with sentiment in emoji use across languages.To do so, we developed a corpus containing the literal meanings for a set of emojis, as defined by L1 speakers in English, Portuguese and Chinese.We then use these definitions to assess whether speakers of different languages agree on whether an emoji is being used literally or figuratively in the context where they are grounded in, as well as whether this literal and figurative use correlates with the sentiment of the context itself.We found that there were varying levels of disagreement on the definition for each emoji but that these stayed fairly consistent across languages.We also demonstrated a correlation between the sentiment of a tweet and the figurative use of an emoji, providing theoretical underpinnings for empirical results in NLP tasks, particularly offering insights that can benefit sentiment analysis models.

Personalized recommender systems suffuse modern life, shaping what media we read and what products we consume. Algorithms powering such systems tend to consist of supervised-learning-based heuristics, such as latent factor models with a variety of heuristically chosen prediction targets. Meanwhile, theoretical treatments of recommendation frequently address the decision-theoretic nature of the problem, including the need to balance exploration and exploitation, via the multi-armed bandits (MABs) framework. However, MAB-based approaches rely heavily on assumptions about human preferences. These preference assumptions are seldom tested using human subject studies, partly due to the lack of publicly available toolkits to conduct such studies. In this work, we conduct a study with crowdworkers in a comics recommendation MABs setting. Each arm represents a comic category, and users provide feedback after each recommendation. We check the validity of core MABs assumptions—that human preferences (reward distributions) are fixed over time—and find that they do not hold. This finding suggests that any MAB algorithm used for recommender systems should account for human preference dynamics. While answering these questions, we provide a flexible experimental framework for understanding human preference dynamics and testing MABs algorithms with human users. The code for our experimental framework and the collected data can be found at https://github.com/HumainLab/human-bandit-evaluation.

Giulio Zhou, Gerasimos Lampouras. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.

Large-scale document retrieval systems often utilize two styles of neural network models which live at two different ends of the joint computation vs. accuracy spectrum.The first style is dual encoder (or two-tower) models, where the query and document representations are computed completely independently and combined with a simple dot product operation.The second style is cross-attention models, where the query and document features are concatenated in the input layer and all computation is based on the joint querydocument representation.Dual encoder models are typically used for retrieval and deep re-ranking, while cross-attention models are typically used for shallow re-ranking.In this paper, we present a lightweight architecture that explores this joint cost vs.accuracy trade-off based on multi-vector attention (MVA).We thoroughly evaluate our method on the MS-MARCO passage retrieval dataset and show how to efficiently trade off retrieval accuracy with joint computation and offline document storage cost.We show that a highly compressed document representation and inexpensive joint computation can be achieved through a combination of learned pooling tokens and aggressive downprojection.Our code and model checkpoints are available on GitHub.