The article investigates cooperative interaction in Virtual Reality (VR) and Augmented Reality (AR) for individuals with neurodevelopmental disorders (NDDs) and compares the potential of these interaction paradigms to enhance the communication skills of this population. Two versions of a cooperative game called SMUP (Social Match UP) were created in AR and VR, respectively, involving NDD specialists in the co-design process. During a within-subjects empirical study involving 30 subjects with NDD, we measured both the usability and verbal interaction of the two versions, known as SMUP-AR and SMUP-VR. The findings suggest that both SMUP-AR and SMUP-VR exhibited a good degree of usability for individuals with NDD, with no statistically significant differences between the two versions. However, the evaluation of communication skills, based on the analysis of the verbal production of users under the two experimental conditions, indicates that SMUP-AR had a stronger effect in promoting dialogic interaction compared to SMUP-VR. In addition, the paper discusses insights from the study on the UX design of cooperative AR and VR technology for people with NDD.

Autism Spectrum Disorder is a neurodevelopmental condition that can affect autonomy and independence. Our research explores the integration of Cross-Reality and Conversational Agents for Autistic persons to improve ability and confidence in everyday life situations. We combine two technologies of the Virtual-Real continuum. User experiences unfold from the simulation of tasks in VR to the execution of similar tasks supported by AR in the real world. A speech-based Conversational Agent is integrated with both VR and AR. It provides contextualized help, promotes generalization, and stimulates users to apply what they learned in the virtual space. The paper presents the approach and describes an empirical study involving 17 young Autistic persons.

Communication can be defined as the understanding and exchanging of meaningful messages. The role of communication is central to the lives of human beings as, everyday, we use language to interact with the world around us. Linguistic skills play a fundamental role in this scenario and Language Disorders (LD) are impairments that limit the processing of linguistic information. Early and accurate identification of LD is thus essential to promote lifelong learning and well-being. From an evolutionary perspective, some human language constructs evolved from an ancestral motor system and share the same neural pathways in the Broca’s area of the brain. This suggests a correlation between action and language. If such a relationship is well established and reliable, it would be possible to use the former as a marker of the latter. The hypothesis of our work, in a nutshell, is that movement can be a predictor of language. To study this correlation, we developed C(H)o(L)ordination, a Mixed Reality (MR) application for HoloLens 2. The application offers several activities based on visual stimuli involving motor movements, which tap on the same skills needed to perform some language tasks. We performed an exploratory study with N=22 users to test the application usability and user experience. The results suggest that C(H)o(L)ordination is a usable and powerful tool to gather insights on the ongoing debate about language evolution and language disorders.

Cognitive rigidity and behavioral and communication problems are the most common symptoms that characterize Autism Spectrum Disorder (ASD). In fact, people with ASD face complications in everyday life, from interacting with other persons to being autonomous in the most ordinary daily tasks. To overcome these limitations and improve their quality of life, we developed 5A. 5A exploits cutting-edge technologies such as Virtual Reality (VR), Augmented Reality (AR), and Conversational Agents (CA), combining smartphones and wearable headsets to offer new forms of intervention to persons with ASD. Virtual reality experiences enable learning common tasks, such as using public transport, through simulation in a safe environment. When people are in similar real-life situations, AR can help them generalize the skills they have learned in VR by overlaying interactive media assets on top of the view of the physical world around them. A customizable conversational assistant provides individualized prompts and feedback in both VR and AR to provide maximum user support.

Cognitive impairment is a condition that results in a person’s inability to remember, learn, concentrate or make decisions that affect his/her everyday life. The assessment of these deficits is usually performed using standardized paper and pencil or computerized tests within a controlled clinical setting. Many traditionally designed tools show only low to moderate levels of ecological validity, limiting the reliability of the collected measures. The proposed system adapts existing screening tests within an immersive virtual reality environment with 360° video, recreating a familiar setting for the patient. This faithful reproduction of everyday environments and situations can enhance the ecological validity of the assessment procedure while maintaining a standardized stimuli delivery, all in a controlled and safe setting. As a computerized system, virtual reality technology allows an error-free computation of the test scores, here collected by means of accuracy for each task. The system involves many technologies aimed at capturing any kind of user input provided by the patient. Additionally, using a visor with integrated eye-tracker sensor, the system can register the visual exploration pattern adopted by the patient during the task execution, providing information concerning the attentional and visuo-spatial functioning which are not obtainable using traditional assessment procedures. Finally, the results of an exploratory study that was conducted with 11 users on the reliability and usability of the system are presented.