Game-Based Learning (GBL) is widely used to engage children through playful activities. However, it is unknown whether play elements or learning elements primarily drive children’s GBL performance. To bridge this gap, this paper presents a study involving 75 children aged 8-9 in GBL through two educational mathematical games. Using eye tracking, physiological sensing, motion capture, and gameplay logs, we captured children’s GBL experiences and examined how play and learning elements correlated with them. Our results demonstrated significant differences in children’s GBL experiences (as reflected in multimodal data) across play and learning elements. For example, we found learning elements, especially when combined with play elements, elicit higher engagement (as indicated by phasic electrodermal activity) but also increase cognitive load (as indicated by pupil dilation) in children. Whereas, play elements alone reduce cognitive load and broaden information processing. Furthermore, our study demonstrated that attention (as indicated by fixation duration), engagement, and information processing are positively associated with performance, whereas their cognitive load and stress (as indicated by heart-rate variability index) are negatively associated with performance.

Artificial Intelligence (AI) is playing a growing role in K-12 education.However, curricula often lack structure and proper assessment when paired with collaborative approaches like Design Thinking (DT).Here, behavioral and affective dynamics are overlooked, even though they are indicators of both performance and quality of the learning experience, warranting a more in-depth exploration through Multi-Modal Learning Analytics.Therefore, we engaged 63 students, divided into 29 groups (aged 11 to 15) in a DT workshop on AI, analyzing their performance across each stage of their experience, including their behavioral and affective (i.e., emotional) states, using data collected from physiological sensors, audio, and video recordings.Our results show that certain conditions (e.g., joint visual attention, boredom, and high stress) consistently predicted positive or negative performance across all stages of the workshop, while affective states such as confusion, frustration, high engagement, and low stress fluctuate with implications on the learning experience.

The democratic and emancipatory principles underpinning Participatory Design (PD) set PD methodology apart from other user-oriented design methodologies associated with Human-Computer Interaction. In turning PD principles into practice, PD facilitators play a vital role. However, at present, there is a lack of understanding regarding skills relevant to enacting the role. To address this issue, we present the results from an interview study involving fourteen respondents with considerable PD facilitation experience. The analysis of the interviews uncovered six facilitation skills of perceived relevance: openness, patience, empathy, attentiveness, responsiveness, and adaptiveness. The significance of each skill, as expressed by respondents, is accounted for. We further discuss the composition of skills in the derived skill set, possible implications of missing skills, and how the findings complement relevant existing work. Drawing on the findings, the paper offers an empirically based qualitative understanding of what constitutes skillful PD facilitation.

Design thinking (DT) is an approach used to address complex societal issues by engaging learners in hands-on problem-solving. Current educational guidelines seek to leverage the synergies between DT, digital literacy, and computational skills to enhance students' understanding and application in real-world matters, such as cyber-security and online awareness. However, tackling such learning experiences requires a shift in teaching approaches and the endorsement of educational technologies to foster digital agency. This empowers students to become proactive, informed participants in the digital future, starting from the classroom. To explore the feasibility of this approach from the students' perspectives, our study carried out a DT project with a block-based programming platform to emphasize the importance of cyber-security. Our study involved 113 students aged 10 to 12 across three K-12 schools. Of these, 52 agreed to participate in data collection via a semi-structured interview at the end of the project to share their experiences. We gained insights into students' perceptions regarding collaboration, technology interaction, and knowledge gained through thematic analysis of the interview data, considering both challenges and potential. For instance, we shed light on roadblocks arising from mismatched tool affordances and students' familiarity, as well as topic differentiation in elaboration based on participants' skills. At the same time, DT potential, when paired with digital tools, emerged to ease into everyday wicked problems in an engaging way and foster solution-oriented thinking.

Multimodal data enables us to capture the cognitive and affective states of students to provide a holistic understanding of learning processes in a wide variety of contexts. With the use of sensing technology, we can capture learners' states in near real-time and support learning. Moreover, multimodal data allows us to obtain early predictions of learning performance, and support learning in a timely manner. In this contribution, we utilise the notion of ‘carry forward effect’, an inferential and predictive modelling approach that utilises multimodal data measurements detrimental to learning performance to provide timely feedback suggestions. Carry forward effect can provide a way to prioritise conflicting feedback suggestions in a multimodal data-based scaffolding tool. We showcase the empirical proof of the carry forward effect with the use of three different learning scenarios: game-based learning, individual debugging, and collaborative debugging.

Inquiry-based science learning is an educational strategy to enable students to actively engage in science learning concepts through inquiry activities such as experiments and observations. Gamification demonstrates a promising potential to engage children in learning contexts. In this regard, this paper presents Experiverse as an exemplar along with its associated six key design considerations to illustrate how to develop an application based on the concept of gamification and inquiry-based science learning for children. This paper reports on our experience evaluating Experiverse with 25 children (aged 9-13) in an informal setting based on data collected from log data, surveys and interviews to explore the feasibility of engaging children in science learning outside their classroom. Results indicated that children’s motivation (MO) significantly correlates with their enjoyment (PE) and perceived learning outcome (LOA) from using Experiverse. While children’s perceived learning outcome is significantly positively correlated with the number of view visits on Experiverse (EVV), the number of experiment view visits (EVV) is also significantly positively correlated with children’s perceived easiness of the app. Finally, this paper discusses the key findings of this study and points out the design implications for future research, like combining in-app experience and hands-on experimentation in real-life situations.

Technology-enhanced embodied learning has gained traction in HCI, yet deeper insights into how children’s physical actions interplay with their cognitive and emotional states remain underexplored. This study investigates MOVES-NL, an embodied digital learning environment, as a medium for advancing understanding of the dynamic relationship between movement, engagement, and cognitive processes such as stress and learning. MOVES-NL combines movement and immediate formative feedback to foster arithmetic understanding of integers, offering a novel perspective on the integration between embodied interactions and conceptual development. Moving beyond traditional evaluations of learning impact or media comparisons, this work employs multimodal learning analytics (MMLA) integrating motion capture with physiological data to explore the nuanced dynamics of embodied learning. Through a mixed-methods approach integrating both qualitative and quantitative analyses, this study reveals how students’ physical movements relate to cognitive and emotional states, offering actionable insights to support engagement and learning processes. This research advances the understanding of how children’s physical movements relate to their cognitive processes and highlights key considerations for integrating embodied technologies into curricula to foster student engagement and deepen their conceptual understanding, adding value to ongoing conversations about the role of digital technology in children’s education and development.

Design thinking (DT) and computational activities foster children’s knowledge capital for 21st-century literacies. The analysis of these activities often overlooks affective and behavioural states despite their significance in providing insights into children’s learning processes. Typically, these states and their changes are self-reported, lacking real-time capturing. Moreover, inquiries via Multi-Modal Data (MMD) for more comprehensive views are underrepresented in the current literature. We, therefore, conducted a DT activity focusing on coding engaging 33 children (aged 10 to 12) and analysed measurements including learning gain (from knowledge tests) and behavioural and affective states (from physiological sensors, video and voice recordings). Our results show that engagement and confusion exhibit positive correlations between MMD measurements and learning gain, while stress, frustration and anger stand out as detrimental for it. By mapping transitions in states experienced by the children, we unravelled negative learning scenarios that should be limited, along with positive indicators of increased performance.

When the technology encompasses some form of intelligence or agency in the form of robots, virtual agents or artificial intelligence, understanding the reasoning behind their actions and decisions becomes an integral part of the interaction. This challenge extends beyond mere interaction to ensure these technological entities engage with children in an understandable and transparent manner. Given the current emergence of research in explainability and transparency within human-robot interaction, a noticeable gap emerges when the target population shifts to children. Several challenges have contributed to this gap, including the more difficult job of considering children’s unique cognitive and emotional needs or aligning the complexity of the technology and the developmental stages of young users. As we advance the field through generating more effective explanations or transparent behaviours in robots and agents, transitioning these advancements to more child-centric contexts demands a deeper understanding of how children perceive and comprehend technological behaviours. This workshop explores this gap and how we could tackle the critical role of developing technologies, e.g., robots, AI, and toys that are more transparent and express more explainable behaviours. We aim to initiate discussions on the importance of understanding children’s perception of different technologies and approaches to generate and evaluate explainability features that are tailored for child users interacting with autonomous agents and robots. Simultaneously, we address the challenges inherent in this context, including potential biases in explainability and the risks associated with deception in child-technology interaction.

Nowadays, learning activities have become more interactive and collaborative than ever before. However, it remains unclear what makes the group perform differently in such a learning context. With the empowerment of multimodal data (MMD), we conducted a field study involving 12 groups of children who collaborated during two-day-long classroom activities. This paper reports on a quantitative analysis and temporal explanation concerning the relation between children's performance and their group-level MMD measurements during a collaborative coding session in a design thinking activity. We computed each group's performance based on the created artefacts and compared the groups with better performance than the others. The results demonstrate that high-performing groups show more joint engagement, joint visual attention, and joint emotional intensity of delight, while low-performing groups show significantly more joint emotional intensity of frustration. In addition, the evolution over the four temporal phases showed different patterns between high and low-performing groups. Finally, this paper discusses design and theoretical implications for educators, researchers and practitioners.

Allowing children to engage in technologically enabled embodied interaction activities has the potential to enhance learning and play. This work leverages the capabilities provided by multisensory environments (MSEs) to address the underlying research question: What are the benefits, challenges, and trade-offs between the various interaction modalities in the context of educational MSEs for children? To answer this question, we designed and deployed MOVES, a MSE-enabler that goes beyond the previous "hardwired" technologies and affords different interaction modalities. We conducted an in-situ field study with 175 children aged 6–10, who engaged with MOVES and the five interaction modalities. We captured children’s experiences (perceptions and actual use) through action logs, data collected from the various sensors (e.g., physiological data from wristbands, skeletal data from motion sensors), and pictorial-based self-reports. The results provide the differences between the various interaction modalities and design considerations aimed at facilitating children’s learning experiences within an MSE.

Understanding what information is important for consumers when making a purchase-related decision has been a key question for researchers and practitioners ever since the advent of empirical research in commerce. Nevertheless, our knowledge of what information is important has been formed primarily through post-purchase conscious capturing approaches, such as surveys and questionnaires. To overcome these limitations, we ground this research on an exploratory study that captures eye-tracking data during a decision-making task of product selection. Grounded on the dynamic attention theory, we utilize different information types and formats present on a popular social commerce platform, to identify elements which are important when deciding about online product purchase decision. Specifically, we employ a series of prediction algorithms and use an ensemble learning setup to predict the aspects that contribute to product selection by consumers. Our analysis highlights the most important informational cues to accurately predict product selection among alternatives. In addition, the results showcase how such elements shift in importance during the temporal sequence of comparing different product alternatives. Our results provide insight into how we can understand the journey of decision-making for social commerce customers when navigating through information to select a product. In addition, it opens the discussion about the shifts that eye-tracking in combination with machine learning can create for researchers and marketers.

The role of fun in learning, and specifically in learning to code, is critical but not yet fully understood. Fun is typically measured by post session questionnaires, which are coarse-grained, evaluating activities that sometimes last an hour, a day or longer. Here we examine how fun impacts learning during a coding activity, combining continuous physiological response data from wristbands and facial expressions from facial camera videos, along with self-reported measures (i.e. knowledge test and reported fun). Data were collected from primary school students (N = 53) in a single-occasion, two-hours long coding workshop, with the BBC micro:bits. We found that a) sadness, anger and stress are negatively, and arousal is positively related to students’ relative learning gain (RLG), b) experienced fun is positively related to students' RLG and c) RLG and fun are related to certain physiological markers derived from the physiological response data.

Participatory design facilitators have a significant impact on participatory activities, processes, and outcomes. However, the facilitator role has not yet been thoroughly debated in existing design discourse, and support for role-related reflections is limited. As the first steps towards an enriched collective understanding of this specific role and its realization, we interviewed 14 respondents with an academic background in participatory design and extensive facilitation experience. Based on a content analysis of the interviews, we identified six facets of the role: (1) trust builder, (2) enabler, (3) inquirer, (4) direction setter, (5) value provider, and (6) users’ advocate. Each facet is presented as consisting of the respondents’ perceived associated responsibilities and corresponding strategies. Our results paint a complex picture of participatory design facilitation. We propose the multi-faceted understanding of the facilitator role emerging from this work as a basis for problematized reflection on the role and its realization.

The confluence of wearable technologies for sensing learners and the quantified-self provides a unique opportunity to understand learners’ experience in diverse learning contexts. We use data from learners using Empatica Wristbands and self-reported questionnaire. We compute stress, arousal, engagement and emotional regulation from physiological data; and perceived performance from the self-reported data. We use Fuzzy Set Qualitative Comparative Analysis (fsQCA) to find relations between the physiological measurements and the perceived learning performance. The results show how the presence or absence of arousal, engagement, emotional regulation, and stress, as well as their combinations, can be sufficient to explain high perceived learning performance

Identifying and supporting children’s play and problem solving behaviour is important for designing educational technologies. This can inform feedback mechanisms to scaffold learning (provide hints or progress information), and assist facilitators (teachers, parents) in supporting children. Traditionally, researchers manually code video to dissect children’s nuanced play and problem solving behaviour. Advancements in sensing technologies and their respective Multi-Modal Data (MMD), afford observation of invisible states (cognitive, affective, physiological), and provide opportunities to inspect internal processes experienced during learning and play. However, limited research combines traditional video annotations and MMD to understand children’s behaviour as they interact with educational technology. To address this concern, we collected data from webcam, wristband, eye-trackers, and Kinect, as 26 children, aged 10-12, played a Motion-Based Educational Games (MBEG). Results showed significant differences in children’s experience during play and problem solving episodes, and motivate design considerations aimed to facilitate children’s interactions with MBEG.

This paper investigates the relation between children’s joint gaze and emotions with the information flow of the screen from a causal point of view, in the context of collaborative coding. We employ Granger’s definition of causality to extend the knowledge we have about children’s collaborative activities from correlational methods. We organised a coding workshop with 50 children (10 dyads and 10 triads; 13-16 years old). While the children were coding collaboratively, their facial video and the screen were recorded. From the screen recording we computed the information flow; and from the facial video we computed children’s emotions (e.g., frustration and boredom) and estimated their gaze. The gaze estimation was used to compute the joint visual attention (JVA) of the team. Our results show that for high performing teams JVA drives the information flow; while for low performing teams we observe causal relation between emotions and information flow. In particular for the low performing teams, frustration and boredom drive the information flow and the information flow then drives children’s confusion. These results extend the understanding of the socio-cognitive processes underlying collaborative performance, which is primarily correlational in nature, with the causal relations between measurements. These novel results have the potential to guide the design of learning tools that scaffold children’s learning and collaboration.

In the context of learning systems, identifying causal relationships among information presented to the user, their behavior and cognitive effort required/exerted to understand and perform a task is key to building effective learning experiences, and to maintain engagement in learning processes. An unexplored question is whether our interaction with presented information affects our cognitive effort (and behaviour), or vice-versa. We investigate causal relationship between information presented and cognitive effort (and behaviour) in the context of two separate studies (N = 40, N = 98), and study the effect of instruction (active/passive task). We utilize screen-recordings and eye-tracking data to investigate the relationship among these variables. To investigate the causal relationships among the different measurements, we use Granger’s causality. Further, we propose a new method to combine two time-series from multiple participants for detecting causal relationships. Our results indicate that information presentation drives user focus size (behaviour), and that cognitive load (a measure of cognitive effort exerted) drives information presentation. This relationship is also moderated by instruction type and performance-level (high/low). We draw implications for design of educational material and learning technologies.

Motion-Based Touchless Games (MBTG) are being investigated as a promising interaction paradigm in children's learning experiences. Within these games, children's digital persona (i.e, avatar), enables them to efficiently communicate their motion-based interactivity. However, the role of children's Avatar Self-Representation (ASR) in educational MBTG is rather under-explored. We present an in-situ within subjects study where 46 children, aged 8--12, played three MBTG with different ASRs. Each avatar had varying visual similarity and movement congruity (synchronisation of movement in digital and physical spaces) to the child. We automatically and continuously monitored children's experiences using sensing technology (eye-trackers, facial video, wristband data, and Kinect skeleton data). This allowed us to understand how children experience the different ASRs, by providing insights into their affective and behavioural processes. The results showed that ASRs have an effect on children's stress, arousal, fatigue, movement, visual inspection (focus) and cognitive load. By exploring the relationship between children's degree of self-representation and their affective and behavioural states, our findings help shape the design of future educational MBTG for children, and emphasises the need for additional studies to investigate how ASRs impacts children's behavioural, interaction, cognitive and learning processes.

Multi-Modal Data (MMD) can help educational games researchers understand the synergistic relationship between player's movement and their learning experiences, and consequently uncover insights that may lead to improved design of movement-based game technologies for learning. Predicting player performance fosters opportunities to cultivate heightened educational experiences and outcomes. However, predicting player's performance utilising player-generated MMD during their interactions with educational Motion-Based Touchless Games (MBTG) is challenging. To bridge this gap, we implemented an in-situ study where 26 users, age 11, played 2 maths MBTGs in a single 20-30 minute session. We collected player's MMD (i.e., gaze data from eye-tracking glasses, physiological data from wristbands, and skeleton data from Kinect) produced during game-play. To investigate the potential of MMD for predicting player's academic performance, we used machine learning techniques and MMD derived from player's game-play. This allowed us to identify the MMD features that drive rapid highly accurate predictions of players' academic performance in educational MBTGs. This might allow us to provide real-time proactive feedback to the player to support them through their educational gaming experience. Our analysis compared two data lengths corresponding to half and full duration of the player's question solving time. We showed that all combinations of extracted features associated with gaze, physiological, and skeleton data, predicted student performance more accurately than the majority baseline. Additionally, the most accurate prediction of player's performance derived from the combination of gaze and physiological data for both full and half data lengths. Our findings emphasise the significance of using MMD for real-time performance prediction in educational MBTG and offer implications for practice.

Many factors influence learners' performance on an activity beyond the knowledge required. Learners' on-task effort has been acknowledged for strongly relating to their educational outcomes, reflecting how actively they are engaged in that activity. However, effort is not directly observable. Multimodal data can provide additional insights into the learning processes and may allow for effort estimation. This paper presents an approach for the classification of effort in an adaptive assessment context. Specifically, the behaviour of 32 students was captured during an adaptive self-assessment activity, using logs and physiological data (i.e., eye-tracking, EEG, wristband and facial expressions). We applied k-means to the multimodal data to cluster students' behavioural patterns. Next, we predicted students' effort to complete the upcoming task, based on the discovered behavioural patterns using a combination of Hidden Markov Models (HMMs) and the Viterbi algorithm. We also compared the results with other state-of-the-art classification algorithms (SVM, Random Forest). Our findings provide evidence that HMMs can encode the relationship between effort and behaviour (captured by the multimodal data) in a more efficient way than the other methods. Foremost, a practical implication of the approach is that the derived HMMs also pinpoint the moments to provide preventive/prescriptive feedback to the learners in real-time, by building-upon the relationship between behavioural patterns and the effort the learners are putting in.

Popular approaches in learner modeling explore response-time as observational data supplemental to response correctness, to enrich the predictive models of learner knowledge. It has been argued that the relationship between response-time and knowledge mastery is non-linear. Determining the degree of association (dependence structure) between those two observations is an open question. To address this objective, we propose an approach based on copulas, i.e., a statistical tool suitable for capturing dependence structure between two variables. All of the information about the dependence structures can be estimated by copula models separately, allowing for the construction of more flexible joint distributions than existing multivariate distributions. This paper puts into practice a two-step pipeline for building the analytical models. Specifically, we propose a flexible copula-based approach that describes the dependence structure between students' response-time and mastery, in learning and testing contexts, and apply the methodology on four datasets. The two datasets are coming from Intelligent Tutoring Systems and are shared via an online repository, and the other two were collected during the validation of an (adaptive) assessment system. The results reveal five generic patterns of associations across-datasets, for various types of activities, domains and learner characteristics (i.e., not across-contexts). We elaborate on those findings and on the implications of our approach for adaptive systems.

The assessment of learning during class activities mostly relies on standardized questionnaires to evaluate the efficacy of the learning design elements. However, standardized questionnaires pose additional strain on students, do not provide “temporal” information during the learning experience, require considerable effort and language competence, and sometimes are not appropriate. To overcome these challenges, we propose using wearable devices, which allow for continuous and unobtrusive monitoring of physiological parameters during learning. In this paper we set out to quantify how well we can infer students’ learning experience from wrist-worn devices capturing physiological data. We collected data from 31 students in 93 class sessions (3 class sessions per student), and our analysis shows that wrist data can predict the learning experience with 11% error. We also show that 6.25 min (SD = 3.1 min) of data are needed to achieve a reliable estimate (i.e., 13.8% error). Our work highlights the benefits and limitations of utilizing wearable devices to assess learning experiences. Our findings help shape the future of quantified-self technologies in learning by pointing out the substantial benefits of physiological sensing for self-monitoring, evaluation, and metacognitive reflection in learning.

This paper employs facial features to recognize emotions during a coding activity with 50 children. Extracting group-level emotional states via facial features, allows us to understand how emotions of a group affect collaboration. To do so, we captured joint emotional state using videos and collaborative experience using questionnaires, from collaborative coding sessions. We define groups' emotional state using a method inspired from dynamic systems, utilizing a measure called cross-recurrence. We also define a collaborative emotional profile using the different measurements from facial features of children. The results show that the emotional cross recurrence (coming from the videos) is positively related with the collaborative experience (coming from the surveys). We also show that the groups with better experience than the others showcase more positive and a consistent set of emotions during the coding activity. The results inform the design of an emotion-aware collaborative support system.

The interaction with the various learners in a Massive Open Online Course (MOOC) is often complex. Contemporary MOOC learning analytics relate with click-streams, keystrokes and other user-input variables. Such variables however, do not always capture learners' learning and behavior (e.g., passive video watching). In this paper, we present a study with 40 students who watched a MOOC lecture while their eye-movements were being recorded. We then proposed a method to define stimuli-based gaze variables that can be used for any kind of stimulus. The proposed stimuli-based gaze variables indicate students' attention (i.e., with-me-ness), at the perceptual (following teacher's deictic acts) and conceptual levels (following teacher discourse). In our experiment, we identified a significant mediation effect of the two levels of with-me-ness on the relation between students' motivation and their learning performance. Such variables enable common measurements for the different kind of stimuli present in distinct MOOCs. Our long-term goal is to create student profiles based on their performance and learning strategy using stimuli-based gaze variables and to provide students gaze-aware feedback to improve overall learning process.

In this article, we present a multi-level time scales framework for the analysis of human-robot interaction (HRI). Such a framework allows HRI scientists to model the inter-relation between measures and factors of an experiment. Our final goal with the introduction of this framework is to unify scientific practice in the HRI community for better reproducibility. Our new approach transposes Newell’s framework of human actions to model human-robot interaction. Measures from the interaction are sorted into categories (time scales) corresponding to the temporal constraints proposed by Newell. According to this sorting, a bottom-up or top-down analysis can then be performed to correlate variables which allows a better understanding and explanation of the interaction. The utilization of our method within two experimental use cases is then presented. The first one, a child-robot interaction, involves two robots and one child playing a memory game. The second is based on an analysis of the PInSoRo dataset, involving 30 child-robot pairs in a freeplay interaction. Finally, we introduce clear guidelines to re-use the framework.

Rehabilitation aims to ameliorate deficits in motor control via intensive practice with the affected limb. Current strategies, such as one-on-one therapy done in rehabilitation centers, have limitations such as treatment frequency and intensity, cost and requirement of mobility. Thus, a promising strategy is home-based therapy that includes task specific exercises. However, traditional rehabilitation tasks may frustrate the patient due to their repetitive nature and may result in lack of motivation and poor rehabilitation. In this article, we propose the design and verification of an effective upper extremity rehabilitation game with a tangible robotic platform named Cellulo as a novel solution to these issues. We first describe the process of determining the design rationales to tune speed, accuracy and challenge. Then we detail our iterative participatory design process and test sessions conducted with the help of stroke, brachial plexus and cerebral palsy patients (18 in total) and 7 therapists in 4 different therapy centers. We present the initial quantitative results, which support several aspects of our design rationales and conclude with our future study plans.

We investigate the potential of gaze as a predictor for the quality of dialogue and the level of understanding in collaborative problem-solving. We unveil a triangular relation among collaborators' dialogue, their gaze pattern, and their performance in the context of a pair programming task. Pairs of participants were asked to understand a JAVA program while their gaze was synchronously recorded. The performance was measured as the level of understanding attained by the pair at the end of the program comprehension task. Gaze patterns were analyzed based on probabilistic hit based areas of interest called gaze tokens. A novel dialogue coding scheme was developed to capture the program description as well as collaboration management aspect of pair program understanding. Both the areas of interest and the dialogue codes reflect top-down and bottom-up program comprehension strategies. Results show that it is possible to relate gaze to the level of abstraction in dialogue and to the level of understanding.

The presented study tries to tackle an intriguing question of how user-generated data from current technologies can be used to reinforce learners' reflections, improve teaching practices, and close the learning analytics loop. In particular, the aim of the study is to utilize users' gaze to examine the role of a mirroring tool (i.e. Exercise View in Eclipse) in orchestrating basic behavioral regulation of participants engaged in a debugging task. The results demonstrated that students who processed the information presented in the Exercise View and acted upon it, improved their performance and achieved higher level of success than those who failed to do it. The findings shed a light how to capture what constitute relevant data within a particular context using gaze patterns, that could guide collection of essential learner-centred analytics for the purpose of designing usable and modular learning environments based on data-driven approaches.

Computational thinking and coding is gradually becoming an important part of K-12 education. Most parents, policy makers, teachers, and industrial stakeholders want their children to attain computational thinking and coding competences, since learning how to code is emerging as an important skill for the 21st century. Currently, educators are leveraging a variety of technological tools and programming environments, which can provide challenging and dynamic coding experiences. Despite the growing research on the design of coding experiences for children, it is still difficult to say how children of different ages learn to code, and to cite differences in their task-based behaviour. This study uses eye-tracking data from 44 children (here divided into "kids" [age 8-12] and "teens" [age 13-17]) to understand the learning process of coding in a deeper way, and the role of gaze in the learning gain and the different age groups. The results show that kids are more interested in the appearance of the characters, while teens exhibit more hypothesis-testing behaviour in relation to the code. In terms of collaboration, teens spent more time overall performing the task than did kids (higher similarity gaze). Our results suggest that eye-tracking data can successfully reveal how children of different ages learn to code.

'Teaching analytics' is the application of learning analytics techniques to understand teaching and learning processes, and eventually enable supportive interventions. However, in the case of (often, half-improvised) teaching in face-to-face classrooms, such interventions would require first an understanding of what the teacher actually did, as the starting point for teacher reflection and inquiry. Currently, such teacher enactment characterization requires costly manual coding by researchers. This paper presents a case study exploring the potential of machine learning techniques to automatically extract teaching actions during classroom enactment, from five data sources collected using wearable sensors (eye-tracking, EEG, accelerometer, audio and video). Our results highlight the feasibility of this approach, with high levels of accuracy in determining the social plane of interaction (90%, κ=0.8). The reliable detection of concrete teaching activity (e.g., explanation vs. questioning) accurately still remains challenging (67%, κ=0.56), a fact that will prompt further research on multimodal features and models for teaching activity extraction, as well as the collection of a larger multimodal dataset to improve the accuracy and generalizability of these methods.

In the context of MOOCs, "With-me-ness" refers to the extent to which the learner succeeds in following the teacher, specifically in terms of looking at the area in the video that the teacher is explaining. In our previous works, we employed eye-tracking methods to quantify learners' With-me-ness and showed that it is positively correlated with their learning gains. In this contribution, we describe a tool that is designed to improve With-me-ness by providing a visual-aid superimposed on the video. The position of the visual-aid is suggested by the teachers' dialogue and deixis, and it is displayed when the learner's With-me-ness is under the average value, which is computed from the other students' gaze behavior. We report on a user-study that examines the effectiveness of the proposed tool. The results show that it significantly improves the learning gain and it significantly increases the extent to which the students follow the teacher. Finally, we demonstrate how With-me-ness can create a complete theoretical framework for conducting gaze-based learning analytics in the context of MOOCs.

Dynamic graphs are a powerful way to model an evolving set of objects and their ongoing interactions. A broad spectrum of systems, such as information, communication, and social, are naturally represented by dynamic graphs. Outlier (or anomaly) detection in dynamic graphs can provide unique insights into the relationships of objects and identify novel or emerging relationships. To date, outlier detection in dynamic graphs has been studied in the context of graph streams, focusing on the analysis and comparison of entire graph objects. However, the volume and velocity of data are necessitating a transition from outlier detection in the context of graph streams to outlier detection in the context of edge streams–where the stream consists of individual graph edges instead of entire graph objects. In this paper, we propose the first approach for outlier detection in edge streams. We first describe a high-level model for outlier detection based on global and local structural properties of a stream. We then propose a novel application of the Count-Min sketch for approximating these properties, and prove probabilistic error bounds on our edge outlier scoring functions. Our sketch-based implementation provides a scalable solution, having constant time updates and constant space requirements. Experiments on synthetic and real-world datasets demonstrate our method's scalability, effectiveness for discovering outliers, and the effects of approximation.

In this study, we investigated the way users memorize, analyze, collaborate, and learn new concepts on a Tangible User Interface (TUI). Twenty-seven pairs of apprentices in logistics ( N = 54) interacted with an interactive simulation of a warehouse. Their task was to discover efficient design principles for building storehouses. In a between-subjects experimental design, half of the participants used 3D physical shelves, whereas the other half used 2D paper shelves. This manipulation allowed us to control for the “representational effect” of 3D tangibles: the first group saw the warehouse as a small-scale model with realistic shelves, whereas the second group had access to a more abstract layout with rectangular pieces of paper. Both groups interacted with the system in the same way. We found that participants in the first group (i.e., who used 3D realistic shelves) better memorized a warehouse layout, built a more efficient model, and scored higher on a learning test. Additionally, students wore eye-tracking goggles while completing those tasks; preliminary results suggest that 3D interfaces increased joint visual attention, which was found to be a significant predictor for participants’ task performance and learning gains. Implications for designing TUIs in collaborative settings are discussed.

We present an eye-tracking study where we augment a Massive Open Online Course (MOOC) video with the gaze information of the teacher. We tracked the gaze of a teacher while he was recording the content for a MOOC lecture. Our working hypothesis is that displaying the gaze of the teacher will act as cues in crucial moments of dyadic conversation, the teacher-student dyad, such as reference disambiguation. We collected data about students’ video interaction behaviour within a MOOC. The results show that the showing the teacher’s gaze made the content easier to follow for the students even when complex visual stimulus present in the video lecture.