Effective communication is critical to the success of many types of human-robot interaction. A key capability for enabling effective communication is accurate modeling of the cognitive status that entities hold (e.g., modeling what objects interlocutors are currently thinking about, or are generally aware of). However, existing models of cognitive status estimation are not well suited for situated and embodied interactions, as they do not account for nonverbal cues, which are a key way in which humans moderate cognitive status. To address this gap, we make three primary contributions. First, we introduce the BOWTIE corpus of dialogues from a multi-modal open-world referential task, annotated with cognitive status, gesture type, linguistic roles, and grammatical roles of entities across utterances. Second, we introduce RECS, the first LSTM-based model of cognitive status, which we train on the BOWTIE corpus. Third, we present empirical evidence for the success of RECS. These contributions stand to accelerate the future use of cognitively informed algorithms for robot language understanding and generation.

As autonomous robots become integrated into society, they must socially navigate around humans. We propose that effective social robot navigation relies on three key principles: social norms, perceived safety, and legibility. Our framework, Overlapping Social Navigation Principles, suggests that the strength of each principle is influenced by the presence of other principles. To test our framework, we implemented SRN behaviors on an autonomous robot in a passing scenario and conducted an online study where participants ranked videos of different SRN behavior combinations. Our findings show that incorporating all three principles enhances SRN, with social norms having the greatest impact.

As robots are deployed into large-scale human environments, they will need to engage in task-oriented dialogues about objects and locations beyond those that can currently be seen. In these contexts, speakers use a wide range of referring gestures beyond those used in the small-scale interaction contexts that HRI research typically investigates. In this work, we thus seek to understand how robots can better generate gestures to accompany their referring language in large-scale interaction contexts. In service of this goal, we present the results of two human-subject studies: (1) a human-human study exploring how human gestures change in large-scale interaction contexts, and to identify human-like gestures suitable to such contexts yet readily implemented on robot hardware; and (2) a human-robot study conducted in a tightly controlled Virtual Reality environment, to evaluate robots' use of those identified gestures. Our results show that robot use of Precise Deictic and Abstract Pointing gestures afford different types of benefits when used to refer to visible vs. non-visible referents, leading us to formulate three concrete design guidelines. These results highlight both the opportunities for robot use of more humanlike gestures in large-scale interaction contexts, as well as the need for future work exploring their use as part of multi-modal communication.

To critically analyze and adapt to the risks and benefits of social robotics, future user communities will require technology and AI literacy: the ability to use new robotic technologies, understand their strengths and limitations, and critically evaluate the implications of their use. Research shows that collaborative, creative, and informal learning experiences can support AI literacy among non-technologists. Therefore, we designed Degrees of Freedom, a multiplayer interactive storytelling game that supports technology literacy about social robots. Degrees of Freedom supports technology literacy competencies by encouraging players to explore how values are encoded in robot designs, compelling players to consider the risks and limitations of robots, and encouraging them to make connections to their own lives and values. We present both the design of Degrees of Freedom and the results of game playtesting. Our results show that the narrative, collaborative nature of the game supported players in critical thinking about the role robots can or should have in their communities.