Natural language semantics has recently sought to combine the complementary strengths of formal and distributional approaches to meaning. However, given the fundamentally different ‘representational currency’ underlying these approaches—models of the world versus linguistic co-occurrence—their unification has proven extremely difficult. Here, we define Distributional Formal Semantics, which integrates distributionality into a formal semantic system on the level of formal models. This approach offers probabilistic, distributed meaning representations that are inherently compositional, and that naturally capture fundamental semantic notions such as quantification and entailment. Furthermore, we show how the probabilistic nature of these representations allows for probabilistic inference, and how the information-theoretic notion of “information” (measured in Entropy and Surprisal) naturally follows from it. Finally, we illustrate how meaning representations can be derived incrementally from linguistic input using a recurrent neural network model, and how the resultant incremental semantic construction procedure intuitively captures key semantic phenomena, including negation, presupposition, and anaphoricity.

Psycholinguistic studies of situated language processing have revealed that gaze in the visual environment is tightly coupled with both spoken language comprehension and production. It has also been established that interlocutors monitor the gaze of their partners, a phenomenon called "joint attention", as a further means for facilitating mutual understanding. We hypothesise that human-robot interaction will benefit when the robot's language-related gaze behaviour is similar to that of people, potentially providing the user with valuable non-verbal information concerning the robot's intended message or the robot's successful understanding. We report findings from two eye-tracking experiments demonstrating (1) that human gaze is modulated by both the robot speech and gaze, and (2) that human comprehension of robot speech is improved when the robot's real-time gaze behaviour is similar to that of humans.

In the area of Psycholinguistics, eye-tracking has been a successful and valuable tool for the investigation of on-line processes of language comprehension and language production (e.g., [Griffin and Bock 2000], [Tanenhaus et al. 1995]). However, the application of eye-tracking to the investigation of mechanisms underlying more naturalistic language use, e.g. dialogue, has so far been limited to the examination of eye-movements of either the speaker or the listener in isolation (e.g., [Brown-Schmidt et al. 2005]; [Richardson and Dale 2004]). Even offline dialogue experiments investigating, e.g. priming effects, usually involve only one "real" participant while their interlocutor is a confederate of the experimenter. In order to test predictions coming from dialogue models (e.g., [Pickering and Garrod 2004]) and in order to provide the kinds of evidence necessary for their further development, experimental methods that directly examine behaviour of participants actually engaged in a conversation are needed. Additionally, eye-tracking measures established in psycholinguistic monologue research need to be compared with their dialogue processing counterparts. Furthermore, new measures describing the relation between speaker and listener eye-movements in communication are needed, as they can give rise to the language mechanisms underlying conversational interaction.

Narrow focus refers to accent placement that forces one interpretation\nof a sentence, which is then often perceived contrastively.\nNarrow focus is formalised in terms of alternative\nsets, i.e. contextually or situationally salient alternatives. In\nthis paper, we investigate whether this model is valid also in\nhuman utterance processing. We present an eye-tracking experiment\nto study listeners’ expectations (i.e. eye-movements) with\nrespect to upcoming referents. Some of the objects contrast in\ncolour with objects that were previously referred to, others do\nnot; the objects are referred to with either a narrow focus on the\ncolour adjective or with broad focus on the noun. Results show\nthat narrow focus on the adjective increases early fixations to\ncontrastive referents. Narrow focus hence activates alternative\nreferents in human utterance processing