Programming skills are becoming increasingly important in our digital age. Here, block-based programming languages offer a great way for novices to learn programming, as their drag-and-drop mechanic makes the assembly of code very intuitive. In this study we wanted to answer the question whether fluid intelligence helps with learning to program and what influence it has on the amount of effort needed to succeed in block-programming tasks. We used eye tracking to look into physiological measures of mental effort, focusing on pupil diameter, fixation characteristics, and blinks. Our results suggest that the mental effort needed to successfully complete block-programming tasks is not linked to fluid intelligence and all learners have to invest similar amounts of effort in order to succeed. Additionally, we found that high fluid intelligence speeds up the completion of basic block-programming tasks, but does not influence the time it takes to solve more complex block-programming tasks.

Changes in mathematical representation are an essential part of many STEM fields, especially chemistry. In order to better understand these transitions between different representations, we investigated possible eye-tracking indicators of cognitive load during a task that had two experimental conditions: one was purely mathematical, the other one was similar, but had a chemical context. We used pupil diameter, fixation duration, and pupil fluctuations as measured through the IPA as indicators of cognitive load. Our preliminary results indicate that there may not be a significant difference in cognitive load between the two conditions, which in turn suggests that a chemical context does not impose additional cognitive load on participants. The present study may serve as a starting point to investigate this issue further and shed light on the potential influence of a chemical context on cognitive load.

Pupil diameter is a reliable indicator of mental effort, but it must be baseline corrected to account for its idiosyncratic nature. Established methods for measuring baselines cannot be applied in virtual reality (VR) experiments. To reliably measure a pupil diameter baseline in VR, we propose a short testing environment of visual arithmetic tasks. In an experiment with 66 university students, we analyzed external reliability and internal validity criteria for pupil diameter measures during counting and summation tasks. During the counting task, we found a high retest reliability between stimulus intervals. Acceptable retest reliability was found for task repetition at a second measuring time. Analyzing internal validity, we found that pupil diameter increased with task difficulty comparing both tasks. Further, a linear effect was found between the pupil diameter amplitude and luminance levels. Our findings highlight the potential of counting tasks as a pupil diameter baseline for VR experiments.

Assessment of cognitive load is a major step towards adaptive interfaces. However, non-invasive assessment is rather subjective as well as task specific and generalizes poorly, mainly due to methodological limitations. Additionally, it heavily relies on performance data like game scores or test results. In this study, we present an eye-tracking approach that circumvents these shortcomings and allows for effective generalizing across participants and tasks. First, we established classifiers for predicting cognitive load individually for a typical working memory task (n-back), which we then applied to an emergency simulation game by considering the similar ones and weighting their predictions. Standardization steps helped achieve high levels of cross-task and cross-participant classification accuracy between 63.78 and 67.25 percent for the distinction between easy and hard levels of the emergency simulation game. These very promising results could pave the way for novel adaptive computer-human interaction across domains and particularly for gaming and learning environments.

Fluid intelligence is considered to be the foundation to many aspects of human learning and performance. Individuals’ behavior while solving intelligence tests is therefore an important component in understanding problem-solving strategies and learning processes. We present preliminary results of a novel eye-tracking-based approach to predict participants’ decisions while solving a fluid intelligence test that utilizes semantic scanpath comparisons. Normalizing scanpaths and applying a knn classifier allows us to make individual predictions and combine them to predict final scores. We evaluated our proposed approach on the TüEyeQ dataset published by Kasneci et al. containing data of 315 university students, who worked on the Culture Fair Intelligence Test. Our approach was able to explain 39.207% of variance in the final score and predictions for participants’ final scores showed a correlation of τ = 0.65759 with participants’ actual scores. Overall, the proposed method has shown great potential that can be expanded on in future research.

The reliable estimation of cognitive load is an integral step towards real-time adaptivity of learning or gaming environments. We introduce a novel and robust machine learning method for cognitive load assessment based on behavioral and physiological measures in a combined within- and cross-participant approach. 47 participants completed different scenarios of a commercially available emergency personnel simulation game realizing several levels of difficulty based on cognitive load. Using interaction metrics, pupil dilation, eye-fixation behavior, and heart rate data, we trained individual, participant-specific forests of extremely randomized trees differentiating between low and high cognitive load. We achieved an average classification accuracy of 72%. We then apply these participant-specific classifiers in a novel way, using similarity between participants, normalization, and relative importance of individual features to successfully achieve the same level of classification accuracy in cross-participant classification. These results indicate that a combination of behavioral and physiological indicators allows for reliable prediction of cognitive load in an emergency simulation game, opening up new avenues for adaptivity and interaction.

Real-time evaluation of a person's cognitive load can be desirable in many situations. It can be employed to automatically assess or adjust the difficulty of a task, as a safety measure, or in psychological research. Eye-related measures, such as the pupil diameter or blink rate, provide a non-intrusive way to assess the cognitive load of a subject and have therefore been used in a variety of applications. Usually, workload classifiers trained on these measures are highly subject-dependent and transfer poorly to other subjects. We present a novel method to generalize from a set of trained classifiers to new and unknown subjects. We use normalized features and a similarity function to match a new subject with similar subjects, for which classifiers have been previously trained. These classifiers are then used in a weighted voting system to detect workload for an unknown subject. For real-time workload classification, our methods performs at 70.4% accuracy. Higher accuracy of 76.8% can be achieved in an offline classification setting.

Modern eye tracking systems rely on fast and robust pupil detection, and several algorithms have been proposed for eye tracking under real world conditions. In this work, we propose a novel binary feature selection approach that is trained by computing conditional distributions. These features are scalable and rotatable, allowing for distinct image resolutions, and consist of simple intensity comparisons, making the approach robust to different illumination conditions as well as rapid illumination changes. The proposed method was evaluated on multiple publicly available data sets, considerably outperforming state-of-the-art methods, and being real-time capable for very high frame rates. Moreover, our method is designed to be able to sustain pupil center estimation even when typical edge-detection-based approaches fail - e.g., when the pupil outline is not visible due to occlusions from reflections or eye lids / lashes. As a consequece, it does not attempt to provide an estimate for the pupil outline. Nevertheless, the pupil center suffices for gaze estimation - e.g., by regressing the relationship between pupil center and gaze point during calibration.

We introduce iHEARu-PLAY, a web-based multi-player game for crowdsourced database collection and — most important — labelling. Existing databases (with speech and video content) can be added to the game and labelling tasks can be defined via a web-interface. The primary purpose of iHEARu-PLAY is multi-label, holistic annotation of multi-modal affective speech databases. Players perform labelling (or prompted recording) tasks and are rewarded with scores and prizes, which are computed based on the “correctness” of their annotations, e.g., the agreement with a pre-defined gold standard or with the other players. iHEARu-PLAY is implemented with the open source high-level Python Web framework Django and can be installed on Unix and Windows platforms. Its modular architecture allows for easy integration of custom extensions: New gaming components can be added as plugins in order to support new databases and modalities. Label categories for each database are individually selectable and editable. Audio, image and video annotation are currently supported. iHEARu-PLAY will be available to the research community as a ready-to-use web-service. Researchers can add their own databases, optionally post rewards, and receive annotation results in the end. General users can register to play the game, have fun, compete with other players, and at the same time support science.