Abstract Game-based learning (GBL) environments are designed to foster emotional experiences conducive to learning; yet, there are mixed findings regarding their effectiveness. The inconsistent results may stem from challenges in measuring and modeling emotions as multi-dimensional constructs during GBL. Traditional approaches often use one data channel and conventional statistics to study emotions, which limit our understanding of the multi-componential interactions that underlie emotional states during GBL. In this study, we merged non-linear dynamical systems (NLDS) theory with the component process model of emotion to examine interactions and synchrony among two emotion signals during GBL, facial expressions and heart rate variability (HRV), and assessed its relation to knowledge and learning gain. Data were collected from 58 participants (n = 58) at a university in Central Finland while they learned about pathology with a tower defense game called Antidote COVID-19. Results showed a significant improvement in knowledge after GBL. A NLDS technique called cross-wavelet transformation showed there were varying degrees of synchrony between facial expressions and HRV. Neutral expressions showed the highest degree of synchrony with HRV, followed closely by happiness and anger with HRV. However, the synchrony between facial expressions and HRV did not affect knowledge and learning gain. This research contributes to the field by studying emotions as multidimensional systems during GLB.

Early prediction of children's literacy skills is crucial for successful literacy development. However, standardized screenings for pre-readers are mainly paper-based and designed for one-on-one sessions, demanding significant resources. We present the development and feasibility evaluation of a digital, game-based literacy screening for German pre-readers that supports group sessions. The screening comprises five tasks that do not rely on written language skills. We detail critical design decisions and guidelines for the effective implementation of this group-based screening. We evaluated the feasibility and user experience with 34 German second- and third-graders. Results revealed that the screening is suitable for use in group settings and that it was positively perceived by the children. Children found the tasks enganging and straightforward, often perceiving them as games. This study demonstrates that digital game-based screenings can be used effectively in group settings with young children with minimal adult guidance, offering a motivating and engaging assessment method.

Assessment of cognitive load is a major step towards adaptive interfaces. However, non-invasive assessment is rather subjective as well as task specific and generalizes poorly, mainly due to methodological limitations. Additionally, it heavily relies on performance data like game scores or test results. In this study, we present an eye-tracking approach that circumvents these shortcomings and allows for effective generalizing across participants and tasks. First, we established classifiers for predicting cognitive load individually for a typical working memory task (n-back), which we then applied to an emergency simulation game by considering the similar ones and weighting their predictions. Standardization steps helped achieve high levels of cross-task and cross-participant classification accuracy between 63.78 and 67.25 percent for the distinction between easy and hard levels of the emergency simulation game. These very promising results could pave the way for novel adaptive computer-human interaction across domains and particularly for gaming and learning environments.

Functional near-infrared spectroscopy (fNIRS) provides reliable results for determining cognitive load based on averaged cortical blood flow during multiple repetitions of short cognitive tasks. At the same time, it remains unclear how to use this technique for assessing cognitive load during prolonged single-trial activity. In this study, we used a computer-based emergency simulation game for inducing different levels of cognitive load. We propose a novel approach to measure cognitive load using specific time slots, determined based on simulation log-data interpreted in light of Barrouillet's time-based resource-sharing model. To validate this approach, we compared cortical activity in dorsolateral prefrontal cortex (DLPFC) and left inferior frontal gyrus (IFG) regions measured at four specific time slots during a simulation. We found significant associations between cognitive load and neuronal activity within the DLPFC depending on the chosen time slot, whereas no such dependencies were found for the IFG. These results illustrate how knowledge of task structure could be used advantageously for the identification of cognitive load. Although requiring further investigation in terms of reliability and generalizability, the presented approach can be considered promising evidence that fNIRS might be suitable for more general reliable assessments of cognitive load during prolonged single-trial activities and for real-time adaptations in simulation-based learning environments.

Users' emotional engagement in a task is important for performance and motivation. Non-intrusive, computerized process measures of engagement have the potential to provide fine-grained access to underlying affective states and processes. Thus, the current work brings together subjective measures (questionnaires) and objective process measures (facial expressions and head movements) of emotions to examine users' emotional engagement with respect to the absence or presence of game-elements. In particular, we randomly assigned 156 adult participants to either a spatial working memory task with or without game elements present, while their faces and head movements were recorded with a webcam during task execution. Positive and negative emotions were assessed before the task and twice during task execution using conventional questionnaires. We additionally examined whether perceived subjective effort, assumed to inherit a substantial affective component, manifests at a bodily expressive level alongside positive and negative emotions. Importantly, we explored the relationship between subjective and objective measures of emotions across the two tasks versions. We found a series of action units and head movements associated with the subjective experience of emotions as well as to subjective effort. Impacted by game elements, these associations often fit intuitively or lined up with findings from literature. As did a linear increase of blink (action unit 45) intensity relate to participants performing the task without game elements, presumably indicating disengagement in the more tedious task variant. On other occasions, associations between subjective and objective measures seemed indiscriminative or even contraindicated. Additionally, facial and bodily reactions and the resulting subjective-objective correspondences were rather consistent within, but not between the two task versions. Our work therefore both gains detailed access to automated emotion recognition and promotes its feasibility within research of game elements while highlighting the individuality and context dependency of emotional expressions.

There is a constantly growing number of initiatives asserting the relevance of programming already in primary education and offering respective interventions with the goal to foster interest in and positive attitudes toward programming. To evaluate to what extent this goal is achieved, assessing students' attitudes toward programming reliably is indispensable. However, there still is a need for validated instruments for assessing this in elementary school students. This seems particularly relevant as self-concept and attitudes toward a school subject were repeatedly observed to be significant predictors of learning motivation and achievement. The newly developed Self-Concept and Attitude toward Programming Assessment (SCAPA) is based on existing instruments for assessing students' self-concept and attitude toward mathematics. SCAPA measures aspects of students' self-concept and attitudes toward programming on seven scales: i) self-reported previous programming experience and understanding, ii) self-concept, iii) intrinsic value belief, iv) attainment value belief, v) utility value belief, vi) cost belief, and vii) compliance and persistence. We administered SCAPA to 197 elementary school students between seven and ten years of age in the context of an evaluation of a computational thinking intervention. Data were analyzed for reliability (i.e., internal consistency on item and scale level) and construct validity (by means of confirmatory factor analysis). Results indicated good reliability for all scales except for the self-reported previous programming experience and understanding scale. Overall, these results reflect SCAPA's suitability for assessing different aspects of elementary school students' self-concept and attitudes toward programming.

The reliable estimation of cognitive load is an integral step towards real-time adaptivity of learning or gaming environments. We introduce a novel and robust machine learning method for cognitive load assessment based on behavioral and physiological measures in a combined within- and cross-participant approach. 47 participants completed different scenarios of a commercially available emergency personnel simulation game realizing several levels of difficulty based on cognitive load. Using interaction metrics, pupil dilation, eye-fixation behavior, and heart rate data, we trained individual, participant-specific forests of extremely randomized trees differentiating between low and high cognitive load. We achieved an average classification accuracy of 72%. We then apply these participant-specific classifiers in a novel way, using similarity between participants, normalization, and relative importance of individual features to successfully achieve the same level of classification accuracy in cross-participant classification. These results indicate that a combination of behavioral and physiological indicators allows for reliable prediction of cognitive load in an emergency simulation game, opening up new avenues for adaptivity and interaction.