Difficulty in social interaction is a key factor in the diagnostic criteria for autism. Although not fully understood, fluid human social interaction demands a complex exchange of verbal and nonverbal signals, which is disrupted in autistic individuals. Differences in gaze behavior, gross motor movement, and physiological responses related to arousal and attention have been observed repeatedly in autistic individuals, potentially impacting social interaction. Our prior work (Simmons et al., 2023) uses a fully immersive virtual reality video game custom-designed to examine the role of social presence through solo and cooperative versions of the game. We predicted that the inclusion of a virtual presence would impact the temporal execution of gaze behaviors and gross motor movements, as well as modulate physiological arousal and blink rate in autistic individuals differently than nonautistic controls. We found that the cooperative condition produced a larger number of differences in gaze behaviors and gross motor movements for autistic individuals. In addition, arousal and blink rate displayed differences during the cooperative condition. These findings demonstrate the specific effects of a virtual social presence on fundamental behaviors that comprise social interaction and can be measured and eventually manipulated in virtual reality.

Spatial awareness is the ability to perceive the spatial relationships among objects, including other people and one’s own body, in the surrounding environment. A strong sense of spatial awareness is an essential component of athletic performance, allowing players to move and react effectively in a fast-paced, dynamic environment. Characterizing spatial awareness with athletes on, for example, a soccer pitch presents major challenges, however. The eSports game, Rocket League, shares some feature of fast-paced, dynamic ball sports including an expectation of increased spatial awareness with increasing player skill. Because Rocket League is played on a screen, we sought to characterize player skill in terms of both objective game data and gaze behavior. We trained a YOLOv7 neural network to robustly identify the ball and players from each frame of gameplay to form areas of interest for gaze behavior characterization. We measured gameplay and gaze behavior from complete novices, players with some but limited gameplay exposure, and very experienced players. We see this pilot effort as a start to characterizing spatial awareness in Rocket League play as it develops with game skill, as well as an opportunity to examine how spatial awareness may transfer from the physical world to screen-based environments and vice-versa.

Fluid human social interaction demands a complex exchange of verbal and non-verbal signals. Effectively interpreting and responding to these signals requires timely visual attention to relevant stimuli and coordination of appropriate motor responses. Differences in social interaction are a key part of the criteria for autism, but such differences are also frequently found in conjunction with differences in gaze and gross motor behavior, as well as physiological arousal. This finding suggests these basic functions may contribute to atypical social interactions, especially in highly dynamic situations. Our study uses a virtual reality pattern completion video game with solo and cooperative conditions to measure gaze, motor, and physiological responses to social and non-social stimuli within the virtual environment. In a population of 11 autistic and 16 non-autistic young adults, we measured gaze behavior and pupil diameter as an index of physiological arousal and two different perception-action sequences that occur in the pattern completion task. Compared to non-autistic players, the autistic players had longer fixation duration times in cooperative condition during looks to the most socially dynamic region of the virtual environment. We also observed differences across group and condition in a perception action sequence that demanded visually localizing a block and placing it onto a specific location to complete the sequence. Pupil arousal responses were triggered on the same block placement event and revealed significantly larger arousal changes for autistic players in the cooperative condition. These findings demonstrate specific effects of social presence on fundamental behaviors that comprise social interaction and can be measured, and eventually manipulated, in virtual reality.