As artificial agents proliferate, there will be more and more situations in which they must communicate their capabilities to humans, including what they can “see.” Artificial agents have existed for decades in the form of computer-controlled agents in videogames. We analyze videogames in order to not only inspire the design of better agents, but to stop agent designers from replicating research that has already been theorized, designed, and tested in-depth. We present a qualitative thematic analysis of sight cues in videogames and develop a framework to support human-agent interaction design. The framework identifies the different locations and stimulus types – both visualizations and sonifications – available to designers and the types of information they can convey as sight cues. Insights from several other cue properties are also presented. We close with suggestions for implementing such cues with existing technologies to improve the safety, privacy, and efficiency of human-agent interactions.

As artificial agents proliferate into society, there will be more and more situations in which they need to communicate their capabilities to humans, including what they can “see.” Humans do this with each other by using mental models of human capabilities coupled with social cues that enable situation awareness. In order to design better agents, we analyze video games, which have been communicating to humans about agent visual perception for decades. We present preliminary findings from a qualitative thematic analysis of sight cues in video games. We focus on three cue properties that warrant further study: whether the cue specifies the perceiver, whether the stimulus’ primary purpose is to be a sight cue, and the communication of non-binary sighting information. We close with an additional call for future work: on the effects of using multiple sight cues in combination.