Machine learning is an essential aspect of modern life that many educational institutions incorporate into their curricula. Often, students struggle to grasp how neural networks learn. Teaching these concepts could be assisted with pedagogical agents and serious games, which both have proven helpful for complex topics like engineering. We present "Traversing the Pass," a serious game that utilizes a mentor-like agent to explain the underlying machine learning concepts and provides feedback. We optimized the agent's design in a pre-study before evaluating its effectiveness compared to a text-only user interface with experts and students. Participants performed better in a second assessment two weeks later if they played the game using the agent. Although criticized as repetitive, the game created an understanding of basic machine learning concepts and achieved high flow values. Our results indicate that agents could be used to enhance the beneficial effects of serious games with improved knowledge retention.

Multimodal Interfaces (MMIs) combining speech and spatial input have the potential to elicit minimal cognitive load. Low cognitive load increases effectiveness as well as user satisfaction and is regarded as an important aspect of intuitive use. While this potential has been extensively theorized in the research community, experiments that provide supporting observations based on functional interfaces are still scarce. In particular, there is a lack of studies comparing the commonly used Unimodal Interfaces (UMIs) with theoretically superior synergistic MMI alternatives. Yet, these studies are an essential prerequisite for generalizing results, developing practice-oriented guidelines, and ultimately exploiting the potential of MMIs in a broader range of applications. This work contributes a novel observation towards the resolution of this shortcoming in the context of the following combination of applied interaction techniques, tasks, application domain, and technology: We present a comprehensive evaluation of a synergistic speech & touch MMI and a touch-only menu-based UMI (interaction techniques) for selection and system control tasks in a digital tabletop game (application domain) on an interactive surface (technology). Cognitive load, user experience, and intuitive use are evaluated, with the former being assessed by means of the dual-task paradigm. Our experiment shows that the implemented MMI causes significantly less cognitive load and is perceived significantly more usable and intuitive than the UMI. Based on our results, we derive recommendations for the interface design of digital tabletop games on interactive surfaces. Further, we argue that our results and design recommendations are suitable to be generalized to other application domains on interactive surfaces for selection and system control tasks.

We present the gamified AR learning environment ARES. During the gameplay, learners guide the little dragon called “Ares” through dungeons. Using AR, ARES three-dimensionally integrates the dungeons in the real-world. Each dungeon represents a coordinate system with obstacles that ultimately create mathematical exercises, e.g. rotating a door to let the dragon fly through it. Overcoming such a challenge requires a learner to spatially analyze the exercise and apply the fundamental mathematical principles. ARES displays a debriefing screen at the end of a dungeon to further support the learning process. In a preliminary qualitative user study, pre-service and in-service teachers saw great potential in ARES for providing further practical and motivating exercises to deepen the knowledge in the classroom and at home.

This article presents an experiment exploring the possible impact of avatar's body part visibility on players' experience and performance using current Virtual Reality (VR) gaming platform capacities. In an action-based VR game, a player sees an avatar in first person perspective which is replicating his/her hand, head and body motion. In contrast to the expected outcome from non-game VR contexts, our results did not reveal significant differences with an avatar presenting an increasing number of visible body parts. The body ownership, immersion, emotional and cognitive involvements as well as the perceived control and difficulty were not improved with a more coherent virtual body. This tends to confirm the strong performance aspect of action-based games, whereby control efficiency and enemy awareness is paramount, and could overcome the perceptual, behavioural or emotional effects of avatar embodiment. Digital games are indeed prone to create an intense flow state typically reducing self-awareness, and focusing on the game completion and high performance achievements. However, further experiments with full-body tracking and different game types are necessary to confirm this trend. This research outcome motivates further analysis of the mutual influence of bottom-up and top-down factors of avatar embodiment causing psychophysical effects. In addition, it provides useful indications for VR game developers and researchers on possible effects and evaluation methods.