With generative artificial intelligence driving the growth of dialogic data in education, automated coding is a promising direction for learning analytics to improve efficiency. This surge highlights the need to understand the nuances of student-AI interactions, especially those rare yet crucial. However, automated coding may struggle to capture these rare codes due to imbalanced data, while human coding remains time-consuming and labour-intensive. The current study examined the potential of large language models (LLMs) to approximate or replace humans in deductive, theory-driven coding, while also exploring how human–AI collaboration might support such coding tasks at scale. We compared the coding performance of small transformer classifiers (e.g., BERT) and LLMs in two datasets, with particular attention to imbalanced head–tail distributions in dialogue codes. Our results showed that LLMs did not outperform BERT-based models and exhibited systematic errors and biases in deductive coding tasks. We designed and evaluated a human–AI collaborative workflow that improved coding efficiency while maintaining coding reliability. Our findings reveal both the limitations of LLMs – especially their difficulties with semantic similarity and theoretical interpretations – and the indispensable role of human judgment, while demonstrating the practical promise of human–AI collaborative workflows for coding.

While the capacity to self-regulate has been found to be crucial for secondary school students, prior studies often rely on self-report surveys and think-aloud protocols that present notable limitations in capturing self-regulated learning (SRL) processes. This study advances the understanding of SRL in secondary education by using trace data to examine SRL processes during multi-source writing tasks, with higher education participants included for comparison. We collected fine-grained trace data from 66 secondary school students and 59 university students working on the same writing tasks within a shared SRL-oriented learning environment. The data were labelled using Bannert's validated SRL coding scheme to reflect specific SRL processes, and we examined the relationship between these processes, essay performance, and educational levels. Using epistemic network analysis (ENA) to model and visualise the interconnected SRL processes in Bannert's coding scheme, we found that: (a) secondary school students predominantly engaged in three SRL processes - Orientation, Re-reading, and Elaboration/Organisation; (b) high-performing secondary students engaged more in Re-reading, while low-performing students showed more Orientation process; and (c) higher education students exhibited more diverse SRL processes such as Monitoring and Evaluation than their secondary education counterparts, who heavily relied on following task instructions and rubrics to guide their writing. These findings highlight the necessity of designing scaffolding tools and developing teacher training programs to enhance awareness and development of SRL skills for secondary school learners.

Learning analytics dashboards (LADs) simplify complex learner data into accessible visualisations, providing actionable insights for educators and students. However, their educational effectiveness has not always matched the sophistication of the technology behind them. Explanatory and interactive LADs, enhanced by generative AI (GenAI) chatbots, hold promise by enabling dynamic, dialogue-based interactions with data visualisations and offering personalised feedback through text. Yet, the effectiveness of these tools may be limited by learners' varying levels of GenAI literacy, a factor that remains underexplored in current research. This study investigates the role of GenAI literacy in learner interactions with conventional (reactive) versus scaffolding (proactive) chatbot-assisted LADs. Through a comparative analysis of 81 participants, we examine how GenAI literacy is associated with learners' ability to interpret complex visualisations and their cognitive processes during interactions with chatbot-assisted LADs. Results show that while both chatbots significantly improved learner comprehension, those with higher GenAI literacy benefited the most, particularly with conventional chatbots, demonstrating diverse prompting strategies. Findings highlight the importance of considering learners' GenAI literacy when integrating GenAI chatbots in LADs and educational technologies. Incorporating scaffolding techniques within GenAI chatbots can be an effective strategy, offering a more guided experience that reduces reliance on learners' GenAI literacy.

In computer-based learning environments (CBLEs), adopting effective self-regulated learning (SRL) strategies requires sophisticated coordination of multiple SRL processes. While various studies have proposed adaptive SRL scaffolds (i.e. real-time advice on adopting effective SRL processes) and embedded them in CBLEs to facilitate learners' effective use of SRL strategies, two key research gaps remain. First, there is a lack of research on SRL scaffolds that are based on continuous assessment of both learners' SRL processes and learning conditions (e.g., awareness of learning resources) to provide adaptive support. Second, current analytics-based scaffolding mechanisms lack the scalability needed to effectively address multiple learning conditions. Integration of analytics of SRL with generative artificial intelligence (GenAI) can provide scalable scaffolding for real-time SRL processes and evolving conditions. Yet, empirical studies implementing and evaluating effects of this integration remain scarce. To address these limitations, we conducted a randomized control trial, assigning participants to three groups (control, process only, and process with condition groups) to investigate the effects of using GenAI to turn insights from real-time analytics about students' SRL processes and conditions into adaptive scaffolds. The results demonstrate that integrating real-time analytics with GenAI in adaptive SRL scaffolds - addressing both SRL processes and dynamic conditions - promotes more metacognitive learning patterns compared to the control and process-only groups. In addition, the learners showed varying levels of compliance with analytics-based GenAI scaffolds, and this was also reflected in how the learners coordinated their SRL processes, particularly in the performance phase of SRL. This study contributes to the literature by designing, implementing, and evaluating the impact of adaptive scaffolds on learners' SRL processes using real-time analytics with GenAI.

Productively engaging in SRL is challenging for learners since it involves coordinating multiple motivational, affective, cognitive, and metacognitive processes. Researchers have investigated methods to adaptively scaffold learners' productive engagement using SRL processes automatically captured by SRL detectors. However, most previous studies relied solely on the frequency of SRL processes to drive adaptive scaffolds (e.g., feedback, hints), possibly missing the sequential characteristics inherent to self-regulation, a crucial dimension of productive SRL. To address this gap, this study analysed the impact of sequential transitions between multiple SRL processes on learners' performance on a reading-writing task with a hypermedia environment called Flora. A sample of 66 secondary-school learners completed the task and trace data were collected. Grounded in the COPES model of SRL, a rule-based SRL detector was employed to capture SRL processes from collected trace data. We employed a method combining logistic regression with ordered network analysis (ONA) to analyse the transitions between the detected SRL processes. This exploratory study revealed several influential transitions to learners' performance in different temporal learning blocks of self-regulation. The implications suggest the potential of using COPES SRL process transitions to drive adaptive scaffolds to facilitate engagement in productive SRL, benefiting performance outcomes in hypermedia environments.

Even though the engagement in self-regulated learning (SRL) has been shown to boost academic performance, SRL skills of many learners remain underdeveloped. They often struggle to productively navigate multiple cognitive, affective, metacognitive and motivational (CAMM) processes in SRL. To provide learners with the required SRL support, it is essential to understand how learners enact CAMM processes as they study. More research is needed to advance the measurement of affective and motivational processes within SRL, and investigate how these processes influence learners’ cognition and metacognition. With this in mind, we conducted a lab study involving 22 university students who worked on a 45-minute reading and writing task in digital learning environment. We used a wearable electroencephalogram device to record learner academic emotional and motivational states, and digital trace data to record learner cognitive and metacognitive processes. We harnessed time series prediction and explainable artificial intelligence methods to examine how learner’s emotional and motivational states influence their choice of cognitive and metacognitive processes. Our results indicate that emotional and motivational states can predict learners’ use of low cognitive, high cognitive and metacognitive processes with considerable classification accuracy (F1 > 0.73), and that higher values of interest, engagement and excitement promote cognitive processing.

Effective collaboration and team communication are critical across many sectors. However, the complex dynamics of collaboration in physical learning spaces, with overlapping dialogue segments and varying participant interactions, pose assessment challenges for educators and self-reflection difficulties for students. Epistemic network analysis (ENA) is a relatively novel technique that has been used in learning analytics (LA) to unpack salient aspects of group communication. Yet, most LA works based on ENA have primarily sought to advance research knowledge rather than directly aid teachers and students by closing the LA loop. We address this gap by conducting a study in which we i) engaged teachers in designing human-centred versions of epistemic networks; ii) formulated an NLP methodology to code physically distributed dialogue segments of students based on multimodal (audio and positioning) data, enabling automatic generation of epistemic networks; and iii) deployed the automatically generated epistemic networks in 28 authentic learning sessions and investigated how they can support teaching. The results indicate the viability of completing the analytics loop through the design of streamlined epistemic network representations that enable teachers to support students’ reflections.

Reverse Approximate Nearest Neighbor (RANN) query relaxes the RkNN definition of influence, where a user u can be influenced by not only its closest facility but also by every other facility that is almost as close to u as its closest facility is. In this paper, we study the continuous monitoring of RANN queries on road network. Existing continuous RANN algorithms on Euclidean space cannot be extended to continuously monitor RANN queries on road network. We propose two different methods to efficiently monitor RANN queries. We conduct an extensive experiment on different real data sets and demonstrate that our both proposed algorithms are significantly better than the competitor

Multimodal Learning Analytics (MMLA) innovations make use of rapidly evolving sensing and artificial intelligence algorithms to collect rich data about learning activities that unfold in physical spaces. The analysis of these data is opening exciting new avenues for both studying and supporting learning. Yet, practical and logistical challenges commonly appear while deploying MMLA innovations “in-the-wild”. These can span from technical issues related to enhancing the learning space with sensing capabilities, to the increased complexity of teachers’ tasks. These practicalities have been rarely investigated. This article addresses this gap by presenting a set of lessons learnt from a 2-year human-centred MMLA in-the-wild study conducted with 399 students and 17 educators in the context of nursing education. The lessons learnt were synthesised into topics related to (i) technological/physical aspects of the deployment; (ii) multimodal data and interfaces; (iii) the design process; (iv) participation, ethics and privacy; and (v) sustainability of the deployment.

Capturing data on socio-spatial behaviours is essential in obtaining meaningful educational insights into collaborative learning and teamwork in co-located learning contexts. Existing solutions, however, have limitations regarding scalability and practicality since they rely largely on costly location tracking systems, are labour-intensive, or are unsuitable for complex learning environments. To address these limitations, we propose an innovative computer-vision-based approach – Computer Vision for Position Estimation (CVPE) – for collecting socio-spatial data in complex learning settings where sophisticated collaborations occur. CVPE is scalable and practical with a fast processing time and only needs low-cost hardware (e.g., cameras and computers). The built-in privacy protection modules also minimise potential privacy and data security issues by masking individuals’ facial identities and provide options to automatically delete recordings after processing, making CVPE a suitable option for generating continuous multimodal/classroom analytics. The potential of CVPE was evaluated by applying it to analyse video data about teamwork in simulation-based learning. The results showed that CVPE extracted socio-spatial behaviours relatively reliably from video recordings compared to indoor positioning data. These socio-spatial behaviours extracted with CVPE uncovered valuable insights into teamwork when analysed with epistemic network analysis. The limitations of CVPE for effective use in learning analytics are also discussed.

Homophily is a fundamental sociological theory that describes the tendency of individuals to interact with others who share similar attributes. This theory has shown evident relevance for studying collaborative learning and classroom orchestration in learning analytics research from a social constructivist perspective. Emerging advancements in multimodal learning analytics have shown promising results in capturing interaction data and generating socio-spatial analytics in physical learning spaces through computer vision and wearable positioning technologies. Yet, there are limited ways for analysing homophily (e.g., social network analysis; SNA), especially for unpacking the temporal connections between different homophilic behaviours. This paper presents a novel analytic approach, Social-epistemic Network Analysis (SeNA), for analysing homophily by combining social network analysis with epistemic network analysis to infuse socio-spatial analytics with temporal insights. The additional insights SeNA may offer over traditional approaches (e.g., SNA) were illustrated through analysing the homophily of 98 students in open learning spaces. The findings showed that SeNA could reveal significant behavioural differences in homophily between comparison groups across different learning designs, which were not accessible to SNA alone. The implications and limitations of SeNA in supporting future learning analytics research regarding homophily in physical learning spaces are also discussed.

Embodied team learning is a form of group learning that occurs in co-located settings where students need to interact with others while actively using resources in the physical learning space to achieve a common goal. In such situations, communication dynamics can be complex as team discourse segments can happen in parallel at different locations of the physical space with varied team member configurations. This can make it hard for teachers to assess the effectiveness of teamwork and for students to reflect on their own experiences. To address this problem, we propose METS (Multimodal Embodied Teamwork Signature), a method to model team dialogue content in combination with spatial and temporal data to generate a signature of embodied teamwork. We present a study in the context of a highly dynamic healthcare team simulation space where students can freely move. We illustrate how signatures of embodied teamwork can help to identify key differences between high and low performing teams: i) across the whole learning session; ii) at different phases of learning sessions; and iii) at particular spaces of interest in the learning space.

In co-located situations, team members use a combination of verbal and visual signals to communicate effectively, among which positional forms play a key role. The spatial patterns adopted by team members in terms of where in the physical space they are standing, and who their body is oriented to, can be key in analysing and increasing the quality of interaction during such face-to-face situations. In this paper, we model the students’ communication based on spatial (positioning) and audio (voice detection) data captured from 92 students working in teams of four in the context of healthcare simulation. We extract non-verbal events (i.e., total speaking time, overlapped speech,and speech responses to team members and teachers) and investigate to what extent they can serve as meaningful indicators of students’ performance according to teachers’ learning intentions. The contribution of this paper to multimodal learning analytics includes: i) a generic method to semi-automatically model communication in a setting where students can freely move in the learning space; and ii) results from a mixed-methods analysis of non-verbal indicators of team communication with respect to teachers’ learning design.

Zijie Zeng, Xinyu Li, Dragan Gasevic, Guanliang Chen. Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2022.