We present a modern formulation of Embodied Question Answering (EQA) as the task of understanding an environment well enough to answer questions about it in natural language. An agent can achieve such an understanding by either drawing upon episodic memory, exemplified by agents on smart glasses, or by actively exploring the environment, as in the case of mobile robots. We accompany our formulation with OpenEQA - the first open-vocabulary benchmark dataset for EQA supporting both episodic memory and active exploration use cases. OpenEQA contains over 1600 high-quality human generated questions drawn from over 180 real-world environments. In addition to the dataset, we also provide an automatic LLM-powered evaluation protocol that has excellent correlation with human judgement. Using this dataset and evaluation protocol, we evaluate several state-of-the-art foundation models including GPT-4V, and find that they significantly lag behind human-level performance. Consequently, OpenEQA stands out as a straightforward, measurable, and practically rele-vant benchmark that poses a considerable challenge to current generation offoundation models. We hope this inspires and stimulates future research at the intersection of Embod-ied AI, conversational agents, and world models.

Museums are always looking for new ways to attract and engage their visitors. Virtual reality (VR) is becoming an increasingly viable option for doing this and enables the recreation of historical sites/structures for visitors to explore. This paper describes a VR application being developed in partnership with the Bawdsey Radar Museum to recreate the radar transmitter towers that stood on the site and were pivotal in winning the Battle of Britain. This paper describes the application’s development and discusses user trial results to determine whether VR can be effectively used within a museum setting to assist in historical learning.