This paper presents an approach that uses Large Language Models (LLMs) as late-fusion interpreters to synthesize multimodal signals from embodied classroom activities and infer students’ metacognitive behaviors. Our multimodal pipeline analyzes students’ movements, gaze, gestures, and speech within a mixed-reality simulation displayed on a classroom screen to support enactment and learning. Vision- and speech-derived features are fused at the interpretive layer via zero-shot prompting, self-consistency reasoning, and targeted prompt engineering to derive planning, enacting, monitoring, reflecting, and interacting behaviors. We investigate whether LLMs can reliably integrate modality-specific analytics to produce accurate behavioral labeling and whether an LLM-as-a-Judge can validate them at scale. To address scalability and reduce human burden, we introduce an automated evaluation protocol employing LLM-as-a-Judge to assess classification quality, enabling rapid, iterative benchmarking of model variants and prompt strategies. Using a balanced corpus of human-validated segments and perturbed controls, we compare text-only language models (e.g., GPT-5) with visual–language models (e.g., Qwen2.5-VL) that incorporate direct visual processing. Results indicate late-fusion, text-based LLMs can outperform VLMs on behavior judgment without raw video, and precision- or recall-oriented prompts adjust decision boundaries for subtle or brief segments. These findings position LLMs as effective late-fusion mechanisms for multimodal learning analytics and demonstrate the viability of LLM-as-a-Judge for scalable, human-in-the-loop evaluation.

The field of learning analytics has made notable strides in automating the detection of complex learning processes in multimodal data. However, most advancements have focused on individualized problem-solving instead of collaborative, open-ended problem-solving, which may offer both affordances (richer data) and challenges (low cohesion) to behavioral prediction. Here, we extend predictive models to automatically detect socially shared regulation of learning (SSRL) behaviors in collaborative computational modeling environments using embedding-based approaches. We leverage large language models (LLMs) as summarization tools to generate task-aware representations of student dialogue aligned with system logs. These summaries, combined with text-only embeddings, context-enriched embeddings, and log-derived features, were used to train predictive models. Results show that text-only embeddings often achieve stronger performance in detecting SSRL behaviors related to enactment or group dynamics (e.g., off-task behavior or requesting assistance). In contrast, contextual and multimodal features provide complementary benefits for constructs such as planning and reflection. Overall, our findings highlight the promise of embedding-based models for extending learning analytics by enabling scalable detection of SSRL behaviors, ultimately supporting real-time feedback and adaptive scaffolding in collaborative learning environments that teachers value.

Embodied learning represents a natural and immersive approach to education, where the physical engagement of learners plays a critical role in how they perceive and internalize concepts. This allows students to actively embody and explore knowledge through interaction with their environment, significantly enhancing retention and understanding of complex subjects. However, researchers face significant challenges in exploring children's learning in these physically interactive spaces, particularly due to the complexity of tracking multiple students' movements and dynamic interactions in real-time. To address these challenges, this paper introduces a Double Diamond design thinking process for developing an AI-enhanced timeline aimed at assisting researchers in visualizing and analyzing interactions within embodied learning environments. We outline key considerations, challenges, and lessons learned in this user-centered design process. Our goal is to create a timeline that employs state-of-the-art AI techniques to help researchers interpret complex datasets, such as children's movements, gaze directions, and affective states during learning activities, thereby simplifying their tasks and augmenting the process of interaction analysis.

The incorporation of technology into primary and secondary education has facilitated the creation of curricula that utilize computational tools for problem-solving. In Open-Ended Learning Environments (OELEs), students participate in learning-by- modeling activities that enhance their understanding of (Science, technology, engineering, and mathematics) STEM and computational concepts. This research presents an innovative multimodal emotion recognition approach that analyzes facial expressions and speech data to identify pertinent learning-centered emotions, such as engagement, delight, confusion, frustration, and boredom. Utilizing sophisticated machine learning algorithms, including High-Speed Face Emotion Recognition (HSEmotion) model for visual data and wav2vec 2.0 for auditory data, our method is refined with a modality verification step and a fusion layer for accurate emotion classification. The multimodal technique significantly increases emotion detection accuracy, with an overall accuracy of 87%, and an Fl -score of 84%. The study also correlates these emotions with model building strategies in collaborative settings, with statistical analyses indicating distinct emotional patterns associated with effective and ineffective strategy use for tasks model construction and debugging tasks. These findings underscore the role of adaptive learning environments in fostering students' emotional and cognitive development.

While understanding Self-Regulated Learning (SRL) in Open-Ended Learning Environments (OELEs), it is crucial to examine the interplay between students’ cognitive processes and affective states, especially learning centered emotions like delight, engagement, boredom, frustration and confusion. These affective states are particularly challenging to detect using facial expressions in middle school students, primarily due to the scarcity of relevant databases. This study introduces a novel approach that utilizes the EmoNet framework, enhanced with self-attention networks, to detect and analyze learning-centered emotions. We investigated the emotional and cognitive dynamics of 41 middle school students within an OELE. Our findings demonstrate distinct emotional patterns that significantly correlate with students’ performance levels across various cognitive processes. By creating and analyzing a dataset from ten students, the proposed model achieved a test accuracy of 85%, indicating a substantial improvement over existing state-of-the-art models. These results lay the groundwork for future educational tools capable of adapting to a combination of students’ affective and cognitive states thus enhancing their overall learning experiences that influence their educational outcomes.

Online learning and MOOCs have become increasingly popular in recent years, and the trend will continue, given the technology boom. There is a dire need to observe learners' behavior in these online courses, similar to what instructors do in a face-to-face classroom. Learners’ strategies and activities become crucial to understanding their behavior. One major challenge in online courses is predicting and preventing dropout behavior. While several studies have tried to perform such analysis, there is still a shortage of studies that employ different data streams to understand and predict the drop rates. Moreover, studies rarely use a fully online team-based collaborative environment as their context. Thus, the current study employs an online longitudinal problem-based learning (PBL) collaborative robotics competition as the testbed. Through methodological triangulation, the study aims to predict dropout behavior via the contributions of Discourse discussion forum ‘activities’ of participating teams, along with a self-reported Online Learning Strategies Questionnaire (OSLQ). The study also uses Qualitative interviews to enhance the ground truth and results. The OSLQ data is collected from more than 4000 participants. Furthermore, the study seeks to establish the reliability of OSLQ to advance research within online environments. Various Machine Learning algorithms are applied to analyze the data. The findings demonstrate the reliability of OSLQ with our substantial sample size and reveal promising results for predicting the dropout rate in online competition. Overall, the study contributes to online learning by addressing the need to understand and predict dropout behavior in online courses. The study’s methodological triangulation, involving qualitative interviews, provides insights into such contexts' unique dynamics and challenges by utilizing a fully online team-based collaborative environment.

Adaptive intelligent educational systems are gaining popularity, offering personalized learning experiences to students based on their individual needs and styles. One crucial feature of such systems is real-time personalized feedback. However, identifying real-time learning processes impacting student performance remains challenging due to data volume constraints. Current research often relies on labor-intensive human observation, which is time-consuming and not scalable. To efficiently collect real-time data, an observation tool is essential. Qualitative/Mixed Method research explores participant experiences in education, social science, and healthcare, utilizing methods like focus groups and observations. However, these methods can be labor-intensive, particularly in maintaining observation time intervals. Existing tools lack comprehensive support for education-focused focus groups and observations. To address these issues, this paper introduces the Data Logging and Organizational Tool (DLOT), a flexible tool designed for qualitative studies with human observers. DLOT offers customizable time intervals, cross-platform compatibility, and data saving and sharing options. The tool empowers observers to log timestamped data and is available on GitHub. The DLOT was validated through two studies. The first study predicted students' affective states using real-time annotations collected via DLOT, observing 30 students in each class. The second study created multimodal datasets in a computer-enabled learning environment, observing 38 students individually. A successful usability test was conducted, offering a potential solution to challenges in real-time learning process identification and labor-intensive qualitative research observation.

In today's world we need to maintain safety and security of the people around us. So we need to have a well connected surveillance system for keeping active information of various locations according to our needs. A real-time object detection is very important for many applications such as traffic monitoring, classroom monitoring, security & rescue, and parking system. From past decade, Convolutional Neural Networks is evolved as a powerful models for recognizing images and videos and it is widely used in the computer vision related work for the best and most used approach for different problem scenario related to object detection and localization. In this work, we have proposed a deep convolutional network architecture to automate the parking system in smart campus with modified Single-shot Multibox Detector (SSD) approach. Further, we created our dataset to train and test the proposed computer vision technique. The experimental results demonstrated an accuracy of 71.2% for the created dataset.

Blindness disables a person from self-navigating outside well-known environments. It affects their ability to perform several jobs, duties, and activities. They are dependent on external assistance which can be provided by humans, dogs or special electronic devices for better decision making. This motivated us to create a prototype called “Smart cane for assisting visually impaired people” to overcome the problems they face in their daily life. Our device is a low cost and lightweight system that processes signals and alerts the visually impaired over any obstacle, potholes or water puddles through different beeping patterns. It senses the light intensity of the environment and illuminates the LED accordingly. These are accomplished by incorporating two ultrasonic sensors, a moisture sensor and a LDR sensor along with an Arduino Nano micro-controller. These are placed at specific positions of the cane for efficient guidance. Moreover, a GSM module is also added to the system so that the visually impaired person can send a message to the emergency contact number in case of distress. The developed model showed 89 percent accuracy and 80 percent of the users were satisfied with the developed prototype.

One of the major challenges in a smart classroom environment is to develop a computer vision based unobtrusive classroom attendance management system. Traditional classroom environment follows a manual attendance marking system either by calling the student's names or by forwarding an attendance sheet; both interrupts the teaching-learning process and also consume a lot of time. Further, it can be erroneous due to factors such as students' proxy etc. In this paper, we propose an unobtrusive face recognition based smart classroom attendance management system using the high definition rotating camera for capturing the faces of students. The proposed system uses Max-Margin Face Detection (MMFD) technique for the face detection and the model is trained using the Inception-V3 CNN technique for the students' identification. The proposed smart classroom system was tested for a classroom with 20 students at National Institute of Technology Karnataka Surathkal, Mangalore, India and we got the experimental results demonstrate the train and test accuracy of 97.67% and 96.66%, respectively.

Nowadays, analysing the students' engagement using non-verbal cues is very popular and effective. There are several web camera based applications for predicting the students' engagement in an e-learning environment. But there are very limited works on analyzing the students' engagement using the video surveillance cameras in a teaching laboratory. In this paper, we propose a Convolutional Neural Networks based methodology for analysing the students' engagement using video surveillance cameras in a teaching laboratory. The proposed system is tested on five different courses of computer science and information technology with 243 students of NITK Surathkal, Mangalore, India. The experimental results demonstrate that there is a positive correlation between the students' engagement and learning, thus the proposed system outperforms the existing systems.

With the exponential growth in areas of machine intelligence, the world has witnessed promising solutions to the personalized content recommendation. The ability of interactive learning agents to take optimal decisions in dynamic environments has been very well conceptualized and proven by Reinforcement Learning (RL). The learning characteristics of Deep-Bidirectional Recurrent Neural Networks (DBRNN) in both positive and negative time directions has shown exceptional performance as generative models to generate sequential data in supervised learning tasks. In this paper, we harness the potential of the said two techniques and strive to create personalized video recommendation through emotional intelligence by presenting a novel context-aware collaborative filtering approach where intensity of users' spontaneous non-verbal emotional response towards recommended video is captured through system-interactions and facial expression analysis for decision-making and video corpus evolution with real-time data streams. We take into account a user's dynamic nature in the formulation of optimal policies, by framing up an RL-scenario with an off-policy (Q-Learning) algorithm for temporal-difference learning, which is used to train DBRNN to learn contextual patterns and generate new video sequences for the recommendation. Evaluation of our system with real users for a month shows that our approach outperforms state-of-the-art methods and models a user's emotional preferences very well with stable convergence.

In this paper, we have proposed a novel design for a vision based unistroke keyboard system for the disabled. The keyboard layout considers the commonly used character patterns, which makes it convenient for the user to type. In addition to this, Shift functionality is provided to accommodate a larger set of characters. A webcam is positioned so as to monitor the keyboard and the characters are identified based on the laser pointer which the user can control by minor head movements. Experimental results demonstrate that the design achieves very promising results, thus establishing a baseline for such models in this domain.

Ember is a smartphone web browser interface designed exclusively for the blind user. The Ember keypad enables blind users to type using their knowledge of Braille. The interface is intuitive to the blind user because the layout consists of a very few large targets and remains consistent throughout the application. The verbal command option provides another dimension for user-interface interaction. Twelve out of thirteen users found that Ember verbal command navigation was easier than using a traditional web browser. Ten out of thirteen users found it faster to use the Ember tactile method of navigation compared to a traditional web browser. The learning rate for both the tactile and verbal command methods was faster compared to the learning rate associated with a traditional web browser layout. Finally it was seen that five out of five users found it significantly faster to use the Ember keypad compared to the QWERTY keypad.

Visual Impairment makes the person depend on another person for all his works and daily chores. Through the application proposed in this paper, we aim to eliminate this dependency of a visually impaired person when travelling from one place to another. The main goal is to provide information regarding the current location, how much distance and time is required to reach the destination as well as provide the user with the directions and turns to be taken while travelling by providing continuous audio feedback in his understandable language.