In online mathematics education, self-explanation is increasingly recognized as a key tool for improving learning outcomes. Identifying learning impasses, which present significant educational challenges, is crucial. Typically, detecting these impasses demands considerable effort from educators to manually review and identify issues in students’ mathematical reasoning.This paper introduces a fully automated impasse detection system designed for online math learning that leverages self-explanations. The system collects high-quality data from students working on the same quizzes, generates example answers, and uses these as benchmarks to identify where students are struggling. The system architecture is described in detail, focusing on the methods used to gather and validate high-quality self-explanation data.Empirical analysis using text regression models shows promising results: the models predict self-explanation scores with an error rate of 0.585 for validation data and 0.655 for evaluation data. While there are variations in scoring accuracy across different mathematical topics, the findings suggest that the system has the potential to significantly improve mathematics education by automating the detection of learning impasses.

In the field of mathematics education, self-explanation is recognized as a critical facilitator for learners to articulate their understanding of complex mathematical concepts and problem-solving techniques. With the emergence of digital learning platforms, the potential to utilize such self-explanations for automated evaluation has expanded, yet significant challenges remain. This study introduces a method that integrates vision and language models to enhance the accuracy of automated evaluations of self-explanations in mathematics quizzes. By leveraging the CLIP encoder, we utilize features from both handwritten images and textual self-explanations, aiming to incorporate the characteristics of handwritten solutions that have been overlooked by text-only evaluations. Models were developed to include self-explanations alone (baseline) and those that integrate image features, using both the original and a fine-tuned CLIP encoder adapted to our dataset of self-explanations and handwritten images. Experimental results demonstrated that the model utilizing the fine-tuned CLIP significantly outperformed the baseline, showing a notable reduction in MAE. Conversely, the model employing the original CLIP encoder exhibited decreased performance compared to the baseline, revealing the complex interplay between integrating self-explanations and image features. These findings suggest that the benefits of embedding image features depend on the quality and appropriateness of the visual data incorporated.

Public educational data, including government-conducted national surveys and research cases, are widely available to the public and intended for use in municipal policymaking. However, some of this data has been published in PDF format and remains underutilized. Therefore, this study leverages new tools in the era of generative Al, such as Large Language Model (LLM) and Retrieval Augmented Generation (RAG), to process 705 public educational document PDF files in Japanese. This process involves extracting text, vectorizing it, and generating responses, thereby presenting a case study of methods for effectively utilizing public educational data. This study revealed that without using the RAG, the outputs from GPT-3.5 and GPT-4 were verbose, while the use of the RAG led to more specific answers based on the retrieval results. Furthermore, GPT-4 can be used to evaluate the quality of retrieval results. These results demonstrate that LLMs can be applied to local educational knowledge in countries with local languages, such as Japanese, and suggest that previously underutilized educational data can be leveraged to aid in formulating educational policies.

Educational explainable AI (XAI) applications are gaining research focus and have distinct needs in the domain of Education. This research presents Educational eXplainable AI Tool (EXAIT), a system for math quiz recommendations, along with an explanation. EXAIT was implemented in a Japanese public high school where students received the top 5 math problems based on Bayesian Knowledge Tracing (BKT) algorithm in a learning analytics dashboard. It aimed to help them complete their summer vacation assignments having 240 questions. On click, the students were redirected to an eBook platform to submit their accuracy and confidence level in each problem. We conducted a study with a quasi-experimental design and divided into 3 groups based on compliance of use. RecoExp group received and used explanations regarding why an item was recommended and how it aims to maximize learners' knowledge-gaining path. RecoCon was the control group that received just the recommendations and used it and RecoNone group did not use the system at all during the time period. We provide a framework to analyze learning logs from EXAIT and extract emerging self-directed learning patterns. Analyzing 222 students' EXAIT logs, we found learners who had checked explanations while selecting recommendations had significantly higher performance. Further differential process mining highlighted significant active daily engagement transitions of the RecoExp group in the self-directed activity.

Explainable recommendations, which provide explanations about why an item is recommended, help to improve the transparency, persuasiveness, and trustworthiness. However, few research in educational technology utilize explainable recommendations. We developed an explanation generator using the parameters from Bayesian knowledge tracing models. We used this educational explainable recommendation system to investigate the effects of explanation on the summer vacation assignment for high school students. Comparing the click counts of recommended quizzes with and without explanations, we found that the number of clicks was significantly higher for quizzes with explanations. Furthermore, system usage pattern mining revealed that students can be divided to three clusters— none, steady and late users. In the cluster of steady users, recommended quizzes with explanations were continuously used. These results suggest the effectiveness of an explainable recommendation system in the field of education.