Assessing handwritten mathematical solutions is essential for identifying students’ weaknesses and fostering personalized learning. However, scaling such assessment remains challenging for Learning Analytics, which has traditionally focused on digital or typed data. The current study investigated the potential of Large Language Models (LLMs) to automating the assessment of handwritten mathematical solutions and explore how they can be incorporated into large scale learning analytics pipelines. We curated 300 student solution images, annotated them using to a taxonomy of math error types, and compared open-source (Qwen2.5-7B and Gemma3 12B-IT) and closed-source (Gemini 2.0 Flash and GPT-4) LLMs. Two prompting strategies were tested: from adapted from related work and one tailed to the taxonomy of math error types using established prompt design principles. The results revealed that LLMs, particularly Gemini, achieved strong to moderate performance in diagnosing and classifying student errors, while exposing recurring model specific errors. These findings highlight both the promise and limitations of LLMs for integrating handwritten work in LA and recommend that learning analytics practitioners and researchers combine careful model selection, principled prompt design, and error-level analysis to develop AI-powered LA systems that are accurate, equitable, and pedagogically actionable.

This study explores the integration of Augmented Intelligence (AuI) in Intelligent Tutoring Systems (ITS) to address challenges in Artificial Intelligence in Education (AIED), including teacher involvement, AI reliability, and resource accessibility. We present MathAIde, an ITS that uses computer vision and AI to correct mathematics exercises from student work photos and provide feedback. The system was designed through a collaborative process involving brainstorming with teachers, high-fidelity prototyping, A/B testing, and a real-world case study. Findings emphasize the importance of a teacher-centered, user-driven approach, where AI suggests remediation alternatives while teachers retain decision-making. Results highlight efficiency, usability, and adoption potential in classroom contexts, particularly in resource-limited environments. The study contributes practical insights into designing ITSs that balance user needs and technological feasibility, while advancing AIED research by demonstrating the effectiveness of a mixed-methods, user-centered approach to implementing AuI in educational technologies.

The COVID-19 pandemic caused a rapid change in workplace dynamics with the forced adoption of remote/hybrid activities. However, best practices for managing distributed software development teams, especially in specialized contexts, are still evolving. This paper reports on the experience of conducting a 1.5-year software development project using a hybrid management approach with nationally distributed teams in Brazil to develop an unplugged intelligent mathematics tutoring system for low-technology educational environments. The study presents a discussion of practices and challenges observed during the project execution. The project delivered a mobile application for automated handwritten math problem correction and instructional material generation. This experience provides practical insights for managing complex, distributed software projects in similar contexts, contributing to deepening knowledge about educational inclusion from a management perspective and refining hybrid management approaches in remote work settings.

Contains fulltext : 317127.pdf (Publisher’s version ) (Open Access)

Assessing short answers in educational settings is challenging due to the need for scalability and accuracy, which led to the field of Automatic Short Answer Grading (ASAG). Traditional machine learning models, such as ensemble and embeddings, have been widely researched in ASAG, but they often suffer from generalizability issues. Recently, Large Language Models (LLMs) emerged as an alternative to optimize ASAG systems. However, previous research has failed to present a comprehensive analysis of LLMs' performance powered by prompt engineering strategies and compare its capabilities to traditional models. This study presents a comparative analysis between traditional machine learning models and GPT-4 in the context of ASAG. We investigated the effectiveness of different models and text representation techniques and explored prompt engineering strategies for LLMs. The results indicate that traditional machine learning models outperform LLMs. However, GPT-4 showed promising capabilities, especially when configured with optimized prompt components, such as few-shot examples and clear instructions. This study contributes to the literature by providing a detailed evaluation of LLM performance compared to traditional machine learning models in a multilingual ASAG context, offering insights for developing more efficient automatic grading systems.

Question-answering systems facilitate adaptive learning and respond to student queries, making education more responsive. Despite that, challenges such as natural language understanding and context management complicate their widespread adoption, where Large Language Models (LLMs) offer a promising solution. However, existing research is predominantly focused on English, proprietary models, and often limited to a single question type, subject, or skill, leaving a gap in understanding LLMs' performance in languages like Brazilian Portuguese and across questions of various characteristics. This study investigates how LLMs could be integrated in an educational question-answering system efficiently to answer different question types (multiple-choice, cloze, open-ended), subjects (mathematics and Portuguese language), and skills (summation/subtraction, multiplication, interpretation, and grammar), evaluating answers by GPT-4 - the main LLM at the time of writing - and Sabiá - the open-source Brazilian Portuguese LLM - based on grades assigned by two experienced teachers. Overall, both LLMs demonstrated strong overall performance, with mean scores close to 9.8 out of 10. However, specific challenges emerged, with distinct strengths and weaknesses observed for each model, such as GPT-4's error in a multiple-choice subtraction question and Sabiá's misinterpretation of a cloze question.

Students face several difficulties in introductory programming courses (CS1), often leading to high dropout rates, student demotivation, and lack of interest. The literature has indicated that the adequate use of gamification might improve learning in several domains, including CS1. However, the understanding of which (and how) factors influence gamification’s success, especially for CS1 education, is lacking. Thus, there is a clear need to shed light on pre-determinants of gamification’s impact. To tackle this gap, we investigate how user and contextual factors influence gamification’s effect on CS1 students through a quasi-experimental retrospective study ( \( N = 399 \) ), based on a between-subject design (conditions: gamified or non-gamified) in terms of final grade (academic achievement) and the number of programming assignments completed in an educational system (i.e., how much they practiced). Then, we evaluate whether and how user and contextual characteristics (e.g., age, gender, major, programming experience, working situation, internet access, and computer access/sharing) moderate that effect. Our findings indicate that gamification amplified to some extent the impact of practicing. Overall, students practicing in the gamified version presented higher academic achievement than those practicing the same amount in the non-gamified version. Intriguingly, those in the gamified version that practiced much more extensively than the average showed lower academic achievements than those who practiced comparable amounts in the non-gamified version. Furthermore, our results reveal gender as the only statistically significant moderator of gamification’s effect: in our data, it was positive for females but non-significant for males. These findings suggest which (and how) personal and contextual factors moderate gamification’s effects, indicate the need to further understand and examine context’s role, and show that gamification must be cautiously designed to prevent students from playing instead of learning.

Programming is a complex, not trivial to learn and teach task, which gamification can facilitate. However, how gamification affects learning and the influence of context-related aspects on that effect demand research to better understand how and to whom gamification enhances programming learning. Therefore, we conducted an experimental study analyzing how gamification worked and the role of context-related aspects in terms of intervention duration and learners' familiarity with programming (i.e., the task's topic). It was a six-week study with 19 undergraduate students from an Algorithms class that measured their learning gains, intrinsic motivation, and number of completed quizzes. Mainly, we found gamification affected learning via intrinsic motivation, effect that depended on intervention duration and learners' familiarity with programming. That is, intrinsic motivation strongly predicted learning gains and gamification's effect on intrinsic motivation changed over time, decreasing from positive to negative as learners had less familiarity with programming. Thus, showing gamification can positively impact programming learning by improving students' intrinsic motivation, although that effect changes over time depending on one's previous familiarity with programming.

Personalization of gamification is an alternative to overcome the shortcomings of the one-size-fits-all approach, but the few empirical studies analyzing its effects do not provide conclusive results. While many user and contextual information affect gamified experiences, prior personalized gamification research focused on a single user characteristic/dimension. Therefore, we hypothesize if a multidimensional approach for personalized gamification, considering multiple (user and contextual) information, can improve user motivation when compared to the traditional implementation of gamification. In this paper, we test that hypothesis through a mixed-methods sequential explanatory study. First, 26 participants completed two assessments using one of the two gamification designs and self-reported their motivations through the Situational Motivation Scale. Then, we conducted semi-structured interviews to understand learners' subjective experiences during these assessments. As result, the students using the personalized design were more motivated than those using the one-size-fits-all approach regarding intrinsic motivation and identified regulation. Furthermore, we found the personalized design featured game elements suitable to users' preferences, being perceived as motivating and need-supporting. Thus, informing i) practitioners on the use of a strategy for personalizing gamified educational systems that is likely to improve students' motivations, compared to OSFA gamification, and ii) researchers on the potential of multidimensional personalization to improve single-dimension strategies. For transparency, dataset and analysis procedures are available at https://osf.io/grzhp.

Abstract Gamification has been discussed as a standout approach to improve user experience, with different studies showing that users can have different preferences over game elements according to their user types. However, relatively less is known how different kinds of users may react to different types of gamification. Therefore, in this study ( $$N=331$$ N = 331 ) we investigate how user orientation (Achiever, Disruptor, Free Spirit, Philanthropist, Player, and Socializer) is associated with the preference for and perceived sense of accomplishment from different gamification designs. Beyond singular associations between the user orientation and the gamification designs, the findings indicate no comprehensive and consistent patterns of associations. From the six user orientations, five presented significant associations: Socializer orientation was positively associated with Social, Fictional, and Personal designs, while negatively associated with Performance design; Player orientation was positively associated with Social (Accomplishment), Personal, and Ecological designs, while negatively associated with the Social design (Preference); Disruptor orientation was positively associated with Social design; Achiever orientation was positively associated with Performance and Social designs; and Free Spirit orientation was negatively associated with Social design. Based on the results, we provide recommendations on how to personalize gamified systems and set further research trajectories on personalized gamification.

• Procedural Content Generation (PCG) is commonly used to improve video games development. • Whether PCG-created levels lead to player experiences consistent to human-created ones is unclear. • We conducted an A/B study (N = 507) to compare PCG- and human-created levels. • We found PCG can improve game development without major influences on player experiences. • We found user characteristics influenced on differences between conditions. Game development often requires a multidisciplinary team, demands substantial time and budget, and leads to a limited number of game contents (e.g., levels). Procedural Content Generation (PCG) can remedy some of these problems, aiding with the automatic creation of content such as levels and graphics, in both the development and playing time. However, little research has been performed in terms of how PCG influences players, especially on Digital Math Games (DMG). This article addresses this problem by investigating the interactions of players with a DMG that uses PCG, investigating the hypothesis that interacting with this intervention can provide experiences as good as human-designed content. To accomplish this goal, an A/B test was performed wherein the only difference was that one version ( static , N = 242) had human-designed levels, whereas the other ( dynamic , N = 265) provided procedurally generated levels. To validate the approach, a two-sample experiment was designed in which each sample played a single version and, thereafter, self-reported their experiences through questionnaires. We contribute by showing how the participants interactions with a DMG are reported in terms of (1) fun, (2) willingness to play the game again, and (3) curiosity, in addition to how they (4) describe their experiences. Our findings show that samples' experiences did not significantly differ on the four metrics, but did differ on in-game performance. We discuss possible factors that might have influenced players' experiences, in terms of the participants performances and their demographic attributes, and how our findings contribute to human interaction with computers.