Abstract: Research in learning analytics (LA) has long held a strong interest in improving student self-regulated learning and measuring the impact of feedback on student outcomes. Despite more than a decade of work in this space very little is known around the contextual factors that influence the topics and diversity of feedback and assessment a student encounters during their full program of study. This paper presents research investigating the institutional adoption of a personalized feedback tool. The reported findings illustrate an association between the topics of feedback, student performance, year level of the course and discipline. The results highlight the need for LA research to capture feedback, assessment and learning outcomes over an entire program of study. Herein we propose a more integrated model drawing on contemporary understandings of feedback with current research findings. The goal is to push LA towards addressing more complex teaching and learning processes from a systems lens. The model posed in this paper begins to illustrate where and how LA can address noted deficits in education practice to better understand how feedback and assessment are enacted by instructors and interpreted by students.

This study explored two aspects of temporality in students’ SRL behaviours to understand the dynamics of SRL phase transitions. In the first aspect, which refers to the temporal order of Self-regulated learning (SRL) phases, we characterized four types of SRL processes based on phase transitions and the cyclical nature of SRL. The SRL types were mapped into the kinds of iterative behaviours over SRL phases which correspond to the theorized self-regulatory behaviours of students at different levels of SRL skills. We found a significant association between SRL types and the assignment grades that suggests the higher achieved learning outcomes, i.e., programming skills demonstrated in the assignments, being associated with more advanced SRL processes. This study also focused on the second aspect of temporality, which refers to the instance of time. We revealed the temporal dynamics between SRL phase transitions by analyzing time profiles for transitions in each SRL process type. Next, we showed that a two-day interval is a threshold by which most students iteratively transition from adapting to enactment phases, which provides a suitable time to intervene if the transition is not observed.

Research has emphasized that self-regulated learning (SRL) is critically important for learning. However, students have different capabilities of regulating their learning processes and individual needs. To help students improve their SRL capabilities, we need to identify students’ current behaviors. Specifically, we applied instructional design to create visible and meaningful markers of student learning at different points in time in LMS logs. We adopted knowledge engineering to develop a framework of proximal indicators representing SRL phases and evaluated them in a quasi-experiment in two different learning activities. A comparison of two sources of collected students’ SRL data, self-reported and trace data, revealed a relatively high agreement between our classifications (weighted kappa, κ = .74 and κ = .68). However, our indicators did not always discriminate adjacent SRL phases, particularly for enactment and adapting phases, compared with students’ real-time self-reported behaviors. Our behavioral indicators also were comparably successful at classifying SRL phases for different self-regulatory engagement levels. This study demonstrated how the triangulation of various sources of students’ self-regulatory data could help to unravel the complex nature of metacognitive processes.

The aim of student-facing dashboards is to support learning by providing students with actionable information and promoting self-regulated learning. We created a new dashboard design aligned with SRL theory, called MyLA, to better understand how students use a learning analytics tool. We conducted sequence analysis on students' interactions with three different visualizations in the dashboard, implemented in a LMS, for a large number of students (860) in ten courses representing different disciplines. To evaluate different students' experiences with the dashboard, we computed chi-squared tests of independence on dashboard users (52%) to find frequent patterns that discriminate students by their differences in academic achievement and self-regulated learning behaviors. The results revealed discriminating patterns in dashboard use among different levels of academic achievement and self-regulated learning, particularly for low achieving students and high self-regulated learners. Our findings highlight the importance of differences in students' experience with a student-facing dashboard, and emphasize that one size does not fit all in the design of learning analytics tools.