Open-ended comments in teaching evaluation surveys offer rich insights into how teaching and learning processes unfold. However, large-scale analyses often rely on superficial labels, limiting their usefulness for designing effective interventions. Current Learning Analytics research has yet to fully leverage this data source to inform teaching development. In this study, we present the design and validation of a coding instrument for labeling effective and malfunctional teaching practices, based on 44 practices systematized from prior research. A panel of five experts from different parts of the world validated the content. For construct validation, two trained annotators applied the refined instrument to label 2,618 open-ended responses, and we used Exploratory Multidimensional Item Response Theory to iteratively prune the practices considered. The resulting instrument retained 21 practices linked to conceptually substantive dimensions of teaching, such as Teaching and assessment consistency, Clear content transmission, Breakdown of constructive alignment, and Lack of pedagogical support, among others. Our work contributes to LA by introducing a novel empirical data source—student-generated open-ended feedback—and a methodological framework for extracting actionable insights to support faculty development and improve teaching quality.

This study explores why faculty engage with curriculum analytics (CA) tools for learning outcomes (LOs) assessment and how such engagement generates actionable insights for continuous curriculum improvement. Using a two-case design across Latin American universities, we collected data through interviews, cognitive walkthroughs, and document analysis. Findings reveal that faculty engagement is not solely driven by tool adoption, but by an assessment process grounded in reflection, collaboration, and institutional support. Actionable insights emerged throughout the assessment cycle: during planning, as faculty aligned teaching with LOs; during implementation, through real-time pedagogical adjustments; and post-assessment, via visualizations generated by the CA tools. Faculty interpretation of these visualizations informed evidence-based decisions to improve course design and student learning. By documenting these dynamics across two distinct institutional contexts, the study provides empirical evidence and practical guidance for integrating CA into assessment processes, highlighting the importance of fostering a reflective culture to support sustainable engagement and continuous improvement. More broadly, the findings contribute to the LA field by moving beyond tool functionality to explain the organizational and cultural conditions required for impact, thereby addressing a persistent gap between technical innovation and demonstrated educational change.

International audience

Learning analytics (LA) emerged with the promise of improving student learning outcomes (LOs), however, its effectiveness in informing actionable insights remains a challenge. Curriculum analytics (CA), a subfield of LA, seeks to address this by using data to inform curriculum development. This study explores using CA to evaluate LOs through direct standardized measures at the subject level, examining how this process informs curriculum decision-making. Conducted at an engineering-focused higher education institution, the research involved 32 administrators and 153 faculty members, serving 9.906 students across nine programs. By utilizing the Integrative Learning Design Framework, we conducted three phases of this framework and present key results. Findings confirm the importance of stakeholder involvement throughout different design phases, highlighting the need for ongoing training and support. Among the actionable insights that emerged from LOs assessments, we identified faculty reflections regarding the need to incorporate active learning strategies, improve course planning, and acknowledge the need for education-specific training for faculty development. Although the study does not demonstrate whether these insights lead to improvements in LOs, this paper contributes to the CA field by offering a practical approach to evaluating LOs and translating these assessments into actionable improvements within an actual-world educational context.

In recent years, instructional design has become even more challenging for teaching staff members in higher education institutions. If instructional design causes student overload, it could lead to superficial learning and decreased student well-being. A strategy to avoid overload is reflecting upon the effectiveness of teaching practices in terms of time-on-task. This article presents a Work-In-Progress conducted to provide teachers with a dashboard to visualize student self-reports of time-on-task regarding subject activities. A questionnaire was applied to 15 instructors during a set trial period to evaluate the perceived usability and usefulness of the dashboard. Preliminary findings reveal that the dashboard helped instructors became aware about the number of hours spent outside of class time. Furthermore, data visualizations of time-on-task evidence enabled them to redesign subject activities. Currently, the dashboard has been adopted by 106 engineering instructors. Future work involves the development of a framework to incorporate user-based improvements.

This work in progress paper investigates if timely feedback increases learners' passing rate in a MOOC. An experiment conducted with 2,421 learners in the Coursera platform tests if weekly messages sent to groups of learners with the same probability of dropping out the course can improve retention. These messages can contain information about: (1) the average time spent in the course, or (2) the average time per learning session, or (3) the exercises performed, or (4) the video-lectures completed. Preliminary results show that the completion rate increased 12% with the intervention compared with data from 1,445 learners that participated in the same course in a previous session without the intervention. We discuss the limitations of these preliminary results and the future research derived from them.

In order to develop an entrepreneurial mindset in future engineers, entrepreneurial training has become a key aspect of engineering education. Following this trend, a large and prestigious engineering school in Chile designed and implemented a third-year compulsory course on technology-focused entrepreneurship. To understand how course teaching and assessment methods have benefited students in terms of self-efficacy and learning gains, a cross-sectional survey study has been conducted since 2015. Over the last four academic periods, we assessed Pre-Post self-efficacy gains and learning benefits in 1,335 students. These survey results show that students perceived positive benefits from all course teaching and assessment methods. Thus, we discuss how a core engineering course can develop an entrepreneurial mindset in a diverse population of students.

Curriculum analytics (CA) emerged as a sub-field of learning analytics, aiming to use evidence to drive curriculum decision-making and program improvement. However, its overall impact on program outcomes remains unknown. In this context, this paper presents work-in-progress of a large research project to understand how CA could support continuous improvement processes at a program-level. We followed an approach based on design-based research to develop a CA tool: The Integrative Learning Design Framework. This paper describes three out of four phases of this framework and its main results, including the evaluation of the local impact of this CA tool. This evaluation consisted of an instrumental case study to evaluate its use to support 124 teaching staff in a 3-year continuous improvement process in a Latin American university. Lessons learned indicate that the tool helped staff to collect information for curriculum discussions, facilitating the availability of evidence regarding student competency attainment. To generalize these lessons, future work will consist of evaluating the tool in different university settings.

This paper presents the results of a pilot study about students’ adoption and learning outcomes of 4 MOOCs proposed as a complementary resource for traditional remedial courses on calculus. While the MOOCs were not mandatory, traditional remedial courses were required for those freshmen failing a diagnostic exam. The effects on 589 freshmen students were investigated. The data analysis shows that up to 16 % of the students were active in the MOOCs under study, mostly during the days before taking the diagnostic exam that preceded the traditional face-to-face remedial courses. Trace data about learner actions within the platform were collected as well as the students’ scores. According to a statistical comparison of the students’ exam scores and their interaction behavior with the MOOCs, we observe that active students had more chances of passing the diagnostic exam and skipping the required remedial courses. However, we found no significant differences on the remedial course exam scores between the students that were active in the MOOCs and those that were not. These findings suggest that MOOCs are a good solution to strengthening skills and reviewing concepts, but that more guidance is needed when used as a complement to traditional f2f courses.