Large language models (LLMs) present new opportunities for creating pedagogical agents that engage in meaningful dialogue to support student learning. However, current LLM systems used in classrooms often lack the solid theoretical foundations found in earlier intelligent tutoring systems. To bridge this gap, we propose a framework that combines Evidence-Centered Design with Social Cognitive Theory and Zone of Proximal Development for adaptive scaffolding in LLM-based agents focused on STEM+C learning. We instantiate this framework with Inquizzitor, an LLM-based formative assessment agent that integrates human-AI hybrid intelligence and provides feedback grounded in cognitive science principles. Our findings show that Inquizzitor delivers high-quality assessment and interaction aligned with core learning theories, offering effective guidance that students value. This research demonstrates the potential for theory-driven LLM integration in education, highlighting the ability of these systems to provide adaptive and principled instruction.

This paper presents an approach that uses Large Language Models (LLMs) as late-fusion interpreters to synthesize multimodal signals from embodied classroom activities and infer students’ metacognitive behaviors. Our multimodal pipeline analyzes students’ movements, gaze, gestures, and speech within a mixed-reality simulation displayed on a classroom screen to support enactment and learning. Vision- and speech-derived features are fused at the interpretive layer via zero-shot prompting, self-consistency reasoning, and targeted prompt engineering to derive planning, enacting, monitoring, reflecting, and interacting behaviors. We investigate whether LLMs can reliably integrate modality-specific analytics to produce accurate behavioral labeling and whether an LLM-as-a-Judge can validate them at scale. To address scalability and reduce human burden, we introduce an automated evaluation protocol employing LLM-as-a-Judge to assess classification quality, enabling rapid, iterative benchmarking of model variants and prompt strategies. Using a balanced corpus of human-validated segments and perturbed controls, we compare text-only language models (e.g., GPT-5) with visual–language models (e.g., Qwen2.5-VL) that incorporate direct visual processing. Results indicate late-fusion, text-based LLMs can outperform VLMs on behavior judgment without raw video, and precision- or recall-oriented prompts adjust decision boundaries for subtle or brief segments. These findings position LLMs as effective late-fusion mechanisms for multimodal learning analytics and demonstrate the viability of LLM-as-a-Judge for scalable, human-in-the-loop evaluation.

Computer-Based Open-Ended Learning Environments (OELEs) are designed to challenge learners to become proficient problem-solvers and develop the ability to independently solve complex problems. However, the traditional focus of OELE research has been on demonstrating overall learning gains, potentially overlooking students who struggle in these environments. To address this gap, we take a social justice-based approach by studying 99 sixth-grade students who participated in a week-long classroom study. We first assessed learning outcomes across all then identified 20 students who failed to do well. We qualitatively analyzed video recordings of their interactions with the OELE to understand why they struggled and to determine if interface issues inhibited their learning. Five themes emerged: (1) challenges in knowledge acquisition; (2) challenges in scaffolding learning; (3) disregarding system guidance, (4) not leveraging supporting tools; (5) and getting discouraged by incorrect answers. Based on our findings, we make design recommendations for OELEs to better support underserved learners, recognizing that failure is an important catalyst for motivating improvements in child-centered design.

Embodied learning represents a natural and immersive approach to education, where the physical engagement of learners plays a critical role in how they perceive and internalize concepts. This allows students to actively embody and explore knowledge through interaction with their environment, significantly enhancing retention and understanding of complex subjects. However, researchers face significant challenges in exploring children's learning in these physically interactive spaces, particularly due to the complexity of tracking multiple students' movements and dynamic interactions in real-time. To address these challenges, this paper introduces a Double Diamond design thinking process for developing an AI-enhanced timeline aimed at assisting researchers in visualizing and analyzing interactions within embodied learning environments. We outline key considerations, challenges, and lessons learned in this user-centered design process. Our goal is to create a timeline that employs state-of-the-art AI techniques to help researchers interpret complex datasets, such as children's movements, gaze directions, and affective states during learning activities, thereby simplifying their tasks and augmenting the process of interaction analysis.

This study introduces a methodology to investigate students’ collaborative behaviors as they work in pairs to build computational models of scientific processes. We expand the Self-Regulated Learning (SRL) framework—specifically, Planning, Enacting, and Reflection—proposed in the literature, applying it to examine students’ collaborative problem-solving (CPS) behaviors in a computational modeling task. We analyze these behaviors by employing a Markov Chain (MC) modeling approach that scrutinizes students’ model construction and model debugging behaviors during CPS. This involves interpreting their actions in the system collected through computer logs and analyzing their conversations using a Large Language Model (LLM) as they progress through their modeling task in segments. Our analytical framework assesses the behaviors of high- and low-performing students by evaluating their proficiency in completing the specified computational model for a kinematics problem. We employ a mixed-methods approach, combining Markov Chain analysis of student problem-solving transitions with qualitative interpretations of their conversation segments. The results highlight distinct differences in behaviors between high- and low-performing groups, suggesting potential for developing adaptive scaffolds in future work to enhance support for students in collaborative problem-solving.

Multimodal data analysis provides profound insights into behaviors and interactions within various settings. However, the collection and analysis of this data in real-world scenarios are intricate and resource-intensive. To streamline these processes, we introduce ChimeraPy: an open-source, distributed streaming platform optimized for high-throughput data transfer across processing nodes within a computer cluster. The utility and performance of ChimeraPy are showcased through two benchmark applications, highlighting its capability to handle complex data environments.

Eye-tracking technology has expanded our ability to quantitatively measure human perception. This rich data source has been widely used to characterize human behavior and cognition. However, eye-tracking analysis has been limited in its applicability, as contextualizing gaze to environmental artifacts is non-trivial. Moreover, the temporal evolution of gaze behavior through open-ended environments where learners are alternating between tasks often remains unclear. In this paper, we propose temporal fully-weighted scanpath graphs as a novel representation of gaze behavior and combine it with a clustering scheme to obtain high-level gaze summaries that can be mapped to cognitive tasks via network metrics and cluster mean graphs. In a case study with nurse simulation-based team training, our approach was able to explain changes in gaze behavior with respect to key events during the simulation. By identifying cognitive tasks via gaze behavior, learners’ strategies can be evaluated to create online performance metrics and personalized feedback.