Self-regulated learning (SRL) is the process by which students set goals, monitor progress, and regulate behavior during learning. SRL is an acquired skill; by distinguishing students who struggle with content knowledge from those who struggle with SRL, suitable support can be provided to each. We present a novel behavioural measure of SRL which quantifies how consistently students perform during adaptive learning technology (ALT)-supported practice. Using parameter estimates of an item response theory model, we simulate students’ expected performance and compare it against their actual performance. Some students perform worse than expected; such inconsistent performance indicates they are struggling to engage in SRL. We apply our analysis to trace data from primary school students (n = 239) practicing math at three adaptive difficulty levels (easy, medium, hard), yielding ability and consistency scores for each student. We find that consistency is moderately correlated with ability (r =.34, p <.001); that consistency and ability significantly predict national standardised math exam scores (R2 =.54); and that students are significantly more consistent in the easy condition. We present consistency as a novel behavioural measure of SRL which complements established metrics, and can help practitioners and researchers to distinguish students who struggle developing content knowledge from those who struggle with SRL.

Contains fulltext : 316067.pdf (Publisher’s version ) (Open Access)

Contains fulltext : 316609.pdf (Publisher’s version ) (Open Access)

Writing from multiple sources is a commonly administered learning task across educational levels and disciplines. In this task, learners are instructed to comprehend information from source documents and integrate it into a coherent written composition to fulfil the assignment requirements. Even though educationally potent, multi-source writing tasks are considered challenging to many learners, in particular because many learners underuse monitoring and control, critical metacognitive processes for productive engagement in multi-source writing. To understand these processes, we conducted a laboratory study involving 44 university students. They engaged in multi-source writing task hosted in digital learning environment. Adding to previous research, we unobtrusively measured metacognitive processes using learners’ trace data collected via multiple data channels and in both writing and reading space of the multi-source writing task. We further investigated how these processes affect the quality of a written product, i.e., essay score. In the analysis, we utilised both automatically and human-generated essay score. The rating performance of the essay scoring algorithm was comparable to that of human raters. Our results largely support the theoretical assumptions that engagement in metacognitive monitoring and control benefits the quality of written product. Moreover, our results can inform the development of analytics-based tools that support student writing by making use of trace data and automated essay scoring.

Self-regulated learning (SRL) skills are essential for successful learning in a technology-enhanced learning environment. Learning Analytics techniques have shown a great potential in identifying and exploring SRL strategies from trace data in various learning environments. However, these strategies have been mainly identified through analysis of sequences of learning actions, and thus interpretation of the strategies is heavily task and context dependent. Further, little research has been done on the association of SRL strategies with different influencing factors or conditions. To address these gaps, we propose an analytic method for detecting SRL strategies from theoretically supported SRL processes and applied the method to a dataset collected from a multi-source writing task. The detected SRL strategies were explored in terms of their association with the learning outcome, internal conditions (prior-knowledge, metacognitive knowledge and motivation) and external conditions (scaffolding). The study results showed our analytic method successfully identified three theoretically meaningful SRL strategies. The study results revealed small effect size in the association between the internal conditions and the identified SRL strategies, but revealed a moderate effect size in the association between external conditions and the SRL strategy use.

Researchers have been struggling with the measurement of Self-Regulated Learning (SRL) for decades. Instrumentation tools have been proposed to help capture SRL processes that are difficult to capture. The aim of the present study was to improve measurement of SRL by embedding instrumentation tools in a learning environment and validating the measurement of SRL with these instrumentation tools using think aloud. Synchronizing log data and concurrent think aloud data helped identify which SRL processes were captured by particular instrumentation tools. One tool was associated with a single SRL process: the timer co-occurred with monitoring. Other tools co-occurred with a number of SRL processes, i.e., the highlighter and note taker captured superficial writing down, organizing, and monitoring, whereas the search and planner tools revealed planning and monitoring. When specific learner actions with the tool were analyzed, a clearer picture emerged of the relation between the highlighter and note taker and SRL processes. By aligning log data with think aloud data, we showed that instrumentation tool use indeed reflects SRL processes. The main contribution is that this paper is the first to show that SRL processes that are difficult to measure by trace data can indeed be captured by instrumentation tools such as high cognition and metacognition. Future challenges are to collect and process log data real time with learning analytic techniques to measure ongoing SRL processes and support learners during learning with personalized SRL scaffolds.

This paper describes the design and evaluation of personalized visualizations to support young learners' Self-Regulated Learning (SRL) in Adaptive Learning Technologies (ALTs). Our learning path app combines three Personalized Visualizations (PV) that are designed as an external reference to support learners' internal regulation process. The personalized visualizations are based on three pillars: grounding in SRL theory, the usage of trace data and the provision of clear actionable recommendations for learners to improve regulation. This quasi-experimental pre-posttest study finds that learners in the personalized visualization condition improved the regulation of their practice behavior, as indicated by higher accuracy and less complex moment-by-moment learning curves compared to learners in the control group. Learners in the PV condition showed better transfer on learning. Finally, students in the personalized visualizations condition were more likely to under-estimate instead of over-estimate their performance. Overall, these findings indicates that the personalized visualizations improved regulation of practice behavior, transfer of learning and changed the bias in relative monitoring accuracy.

This paper proposes a new approach to translate learner data into self-regulated learning support. Learning phases in blended classrooms place unique requirements on students' self-regulated learning (SRL). Learning path graphs merge moment-by-moment learning curves and learning phase data to understand student' SRL support needs. Results indicate 4 groups with different SRL support needs. Students in the self-regulated learning group are capable of learning without external regulation. In the teacher regulation group students need initial teacher regulation but rely on SRL thereafter. Students in the system regulation group require teacher and system regulation to learn. Finally, the advanced system support group is in need of support beyond the current level of system regulation. Based on these insights, the application of personalized dashboards and hybrid human-system regulation is further specified.

In this poster, we describe the aim and current activities of the EARLI-Centre for Innovative Research (E-CIR) "Measuring and Supporting Student's Self-Regulated Learning in Adaptive Educational Technologies" which is funded by the European Association for Research on Learning and Instruction (EARLI) from 2015 to 2019. The aim is to develop our understanding of multimodal data that unobtrusively capture cognitive, meta-cognitive, affective and motivational states of learners over time. This demands for a concerted interdisciplinary dialogue combining findings from psychology and educational sciences with advances in computer sciences and artificial intelligence. The participants in this E-CIR are leading international researchers who have articulated different emerging perspectives and methodologies to measure cognition, metacognition, motivation, and emotions during learning. The participants recognize the need for intensive collaboration to accelerate progress with new interdisciplinary methods including learning analytics to develop more powerful adaptive educational technologies.

Learning analytics empowered educational technologies (LA-ET) in primary classrooms allow for blended learning scenarios with teacher-lead instructions, class-paced and individually-paced practice. This quasi-experimental study investigates the effects of a LA-ET on the development of students' arithmetic skills over one schoolyear. Children learning in a traditional paper & pencil condition were compared to learners using a LA-ET on tablet computers in grade 4. The educational technology combined teacher dashboards (extracted analytics) and class and individually paced assignments (embedded analytics). The results indicated that children in the LA-ET condition made significantly more progress on arithmetic skills in one schoolyear compared to children in the paper & pencil condition.

Even though the recent influx of tablets in primary education goes together with the vision that educational technology empowered with learning analytics will revolutionize education, empirical results supporting this claim are scares. Adaptive educational technology Snappet combines extracted and embedded learning analytics daily in classrooms. While students make exercises on the tablet this technology displays real-time data of learner performance in a teacher dashboard (extracted analytics). At the same time, learner performance is used to adaptively adjust exercises to students' progress (embedded analytics). This quasiexperimental study compares the development of students' arithmetic skills over one schoolyear (grade 2 and 4) in a traditional paper based setting to learning with the adaptive educational technology Snappet. The results indicate that students in the Snappet condition make significantly more progress on arithmetic skills in grade 4. Moreover, in this grade students with a high ability level, benefit the most from working with this adaptive educational technology. Overall the development pattern of students with different abilities was more divergent in the AET condition compared to the control condition. These results indicate that adaptive educational technologies combining extracted and embedded learning analytics are indeed creating new education scenarios that contribute to personalized learning in primary education.

Past research shows that regulative activities (metacognitive or relational) can aid learning and that sequences of cognitive, metacognitive and relational activities affect subsequent cognition. Extending this research, this study examines whether sequences of socially regulated learning differ across low, medium or high performing groups. Scaffolded by a computer avatar, 54 primary school students (working in 18 groups of 3) discussed writing a report about a foreign country for 51,338 turns. Statistical discourse analysis (SDA) of these sequences of talk showed that in high performing groups, high cognition was preceded more often by high cognition and less often by denials or low cognition. In medium performing groups, high cognition was preceded more often by high cognition or planning. As these results indicate that different sequences among students' cognitive, metacognitive and relational activities are linked to levels of performance, they can inform a micro-temporal theory of socially shared regulation.

The study examined the effects of dynamic scaffolding of metacognition on learning outcomes of students in primary schools. In an experimental study two experimental groups and one control group were compared. The experimental groups differed in the form of scaffolding used; structuring scaffolds vs. problematizing scaffolds. We analyzed the effects of dynamic scaffolding and of different forms of scaffolding on learning outcomes. The results showed a small effect of scaffolding on the group performance, no effects on declarative knowledge and a significant effect on procedural knowledge. With respect to the effects of different forms of scaffolding, we found significant effects on group performance, the procedural knowledge acquired and on transfer of knowledge.