Peter Zeng, Weiling Li, Amie J. Paige, Zhengxiang Wang, Panagiotis Kaliosis, Dimitris Samaras, Gregory J. Zelinsky, Susan Brennan, Owen Rambow. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.

During conversation, speakers collaborate on spontaneous referring expressions, which they can then re-use in subsequent conversation with the same partner.Understanding such referring expressions is an important ability for an embodied agent so that it can carry out tasks in the real world.This requires integrating and understanding language, vision, and conversational interaction.We study the capabilities of seven state-of-the-art Large Vision Language Models (LVLMs) as overhearers to a corpus of spontaneous conversations between pairs of human discourse participants engaged in a collaborative object-matching task.We find that such a task remains challenging for current LVLMs, which fail to show a consistent performance improvement as they overhear more conversations from the same discourse participants repeating the same task for multiple rounds.We release our corpus and code 1 for reproducibility and to facilitate future research.

Hedges allow speakers to mark utterances as provisional, whether to signal nonprototypicality or "fuzziness", to indicate a lack of commitment to an utterance, to attribute responsibility for a statement to someone else, to invite input from a partner, or to soften critical feedback in the service of face-management needs.Here we focus on hedges in an experimentally parameterized corpus of 63 Roadrunner cartoon narratives spontaneously produced from memory by 21 speakers for copresent addressees, transcribed to text (Galati and Brennan, 2010).We created a gold standard of hedges annotated by human coders (the Roadrunner-Hedge corpus) and compared three LLM-based approaches for hedge detection: fine-tuning BERT, and zero and fewshot prompting with GPT-4o and LLaMA-3.The best-performing approach was a fine-tuned BERT model, followed by few-shot GPT-4o.After an error analysis on the top performing approaches, we used an LLM-in-the-Loop approach to improve the gold standard coding, as well as to highlight cases in which hedges are ambiguous in linguistically interesting ways that will guide future research.This is the first step in our research program to train LLMs to interpret and generate collateral signals appropriately and meaningfully in conversation.

What is dialogue, anyway-language produced in alternating turns by two or more speakers?A way to collaboratively accomplish a task or transaction with an agent, whether human or computer?An interactive process by which two people entrain and coordinate their behaviors and mental states?A corpus that can be analyzed to answer a research question?The ways in which researchers conceptualize dialogue affect the assumptions and decisions they make about how to design an experiment, collect or code a corpus, or build a system.Often such assumptions are not explicit.Researchers may decide to characterize, stage, control, or entirely ignore such potentially key factors as the task two people are charged with, their identities, their common ground, or the medium in which dialogue is conducted.Such decisions, especially when left implicit, can affect the products and processes of dialogue in substantial but unanticipated ways; in fact, they can change the results of an experiment.As one example, spoken dialogue experiments often use a simulated partner or confederate in the role of speaker or addressee; just how the confederate is deployed reflects the researcher's explicit theory and implicit assumptions about the nature of dialogue.As another example, sometimes experiments place people in infelicitous situations; this can change the kind of language game people think they're playing.I will cover some implicit assumptions about the nature of dialogue that affect the risks researchers take, and highlight pairs of studies that have found different results, perhaps due to these assumptions.

Expertise in computer programming can often be difficult to transfer verbally. Moreover, technical training and communication occur more and more between people who are located at a distance. We tested the hypothesis that seeing one person's visual focus of attention (represented as an eyegaze cursor) while debugging software (displayed as text on a screen) can be helpful to another person doing the same task. In an experiment, a group of professional programmers searched for bugs in small Java programs while wearing an unobtrusive head-mounted eye tracker. Later, a second set of programmers searched for bugs in the same programs. For half of the bugs, the second set of programmers first viewed a recording of an eyegaze cursor from one of the first programmers displayed over the (indistinct) screen of code, and for the other half they did not. The second set of programmers found the bugs more quickly after viewing the eye gaze of the first programmers, suggesting that another person's eye gaze, produced instrumentally (as opposed to intentionally, like pointing with a mouse), can be a useful cue in problem solving. This finding supports the potential of eye gaze as a valuable cue for collaborative interaction in a visuo-spatial task conducted at a distance.

Experimental studies of interactive language use have shed light on the cognitive and interpersonal processes that shape conversation; corpora are the emergent products of these processes. I will survey studies that focus on under-modelled aspects of interactive language use, including the processing of spontaneous speech and disfluencies; metalinguistic displays such as hedges; interactive processes that affect choices of referring expressions; and how communication media shape conversations. The findings suggest some agendas for computational linguistics.

Article Free Access Share on Anthropomorphism: from Eliza to Terminator 2 Authors: Abbe Don View Profile , Susan Brennan View Profile , Brenda Laurel View Profile , Ben Shneiderman View Profile Authors Info & Claims CHI '92: Proceedings of the SIGCHI Conference on Human Factors in Computing SystemsJune 1992 Pages 67–70https://doi.org/10.1145/142750.142760Online:01 June 1992Publication History 25citation1,190DownloadsMetricsTotal Citations25Total Downloads1,190Last 12 Months78Last 6 weeks7 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteeReaderPDF

No abstract available.

In this paper we present a formalization of the centering approach to modeling attentional structure in discourse and use it as the basis for an algorithm to track discourse context and bind pronouns. As described in [GJW86], the process of centering attention on entities in the discourse gives rise to the intersentential transitional states of continuing, retaining and shifting. We propose an extension to these states which handles some additional cases of multiple ambiguous pronouns. The algorithm has been implemented in an HPSG natural language system which serves as the interface to a database query application.