We consider the problem of Vision-and-Language Navigation (VLN). The majority of current methods for VLN are trained end-to-end using either unstructured memory such as LSTM, or using cross-modal attention over the egocentric observations of the agent. In contrast to other works, our key insight is that the association between language and vision is stronger when it occurs in explicit spatial representations. In this work, we propose a cross-modal map learning model for vision-and-language navigation that first learns to predict the top-down semantics on an egocentric map for both observed and unobserved regions, and then predicts a path towards the goal as a set of way-points. In both cases, the prediction is informed by the language through cross-modal attention mechanisms. We experimentally test the basic hypothesis that language-driven navigation can be solved given a map, and then show competitive results on the full VLN-CE benchmark.

Reno Kriz, Eleni Miltsakaki, Marianna Apidianaki, Chris Callison-Burch. Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers). 2018.

Existing software systems for automated essay scoring can provide NLP researchers with opportunities to test certain theoretical hypotheses, including some derived from Centering Theory. In this study we employ the Educational Testing Service's e-rater essay scoring system to examine whether local discourse coherence, as defined by a measure of Centering Theory's Rough-Shift transitions, might be a significant contributor to the evaluation of essays. Rough-Shifts within students' paragraphs often occur when topics are short-lived and unconnected, and are therefore indicative of poor topic development. We show that adding the Rough-Shift based metric to the system improves its performance significantly, better approximating human scores and providing the capability of valuable instructional feedback to the student. These results indicate that Rough-Shifts do indeed capture a source of incoherence, one that has not been closely examined in the Centering literature. They not only justify Rough-Shifts as a valid transition type, but they also support the original formulation of Centering as a measure of discourse continuity even in pronominal-free text. Finally, our study design, which used a combination of automated and manual NLP techniques, highlights specific areas of NLP research and development needed for engineering practical applications.

The problem of proposing referents for anaphoric expressions has been extensively researched in the literature and significant insights have been gained through the various approaches. However, no single model is capable of handling all the cases. We argue that this is due to a failure of the models to identify two distinct processes. Drawing on current insights and empirical data from various languages we propose an aposynthetic 1 model of discourse in which topic continuity, computed across units, and focusing preferences internal to these units are subject to different mechanisms. The observed focusing preferences across the units (i.e., intersententially) are best modeled structurally, along the lines suggested in centering theory. The focusing mechanism within the unit is subject to preferences projected by the semantics of the verbs and the connectives in the unit as suggested in semantic/pragmatic focusing accounts. We show that this distinction not only overcomes important problems in anaphora resolution but also reconciles seemingly contradictory experimental results reported in the literature. We specify a model of anaphora resolution that interleaves the two mechanisms. We test the central hypotheses of the proposed model with an experimental study in English and a corpus-based study in Greek. 1 “Aposynthesis” is a Greek word that means “decomposition,” that is, pulling apart the components that constitute what appears to be a uniform entity.

Existing software systems for automated essay scoring can provide NLP researchers with opportunities to test certain theoretical hypotheses, including some derived from Centering Theory. In this study we employ ETS's e-rater essay scoring system to examine whether local discourse coherence, as defined by a measure of Rough-Shift transitions, might be a significant contributor to the evaluation of essays. Our positive results indicate that Rough-Shifts do indeed capture a source of incoherence, one that has not been closely examined in the Centering literature. These results not only justify Rough-Shifts as a valid transition type, but they also support the original formulation of Centering as a measure of discourse continuity even in pronominal-free text.