Collaborative Problem Solving (CPS) has emerged as a key competence for the 21st century. In support of this, valid assessments of CPS skills have become critical. However, limited research has designed and developed CPS assessments for young students. Based on multimodal learning analytics, we aim to develop and validate a game-based assessment of CPS for primary school students. In this study, evidence centered design approach was used to design and develop the game-based CPS assessment. Specifically, we designed and developed a mobile multiplayer online 3D role-playing game on CPS and a coding scheme for coding students’ gameplay data (i.e., game logs and voice chat) based on the ATC21S CPS framework. A total of 32 primary 5 students participated in this study to play the game in a group of four and complete a questionnaire of CPS skills. The gameplay data were coded based on our coding scheme. Correlation analysis between the coded results and the CPS questionnaire data supported the criterion validity of our game-based assessment measure. Additionally, the results of expert interview facilitated our understanding of assessment design and data use. This study will make methodological and practical contributions to the integration of MMLA into game-based CPS assessments.

Well-designed digital games can serve as the vehicle to assess and support young people’ collaborative problem solving (CPS) skills. However, there is limited research leveraging multimodal learning analytics (MmLA) to explore students’ game-based CPS processes and outcomes. Inspired by MmLA methods and approaches, this preliminary study aims to examine students’ demonstration of CPS skills through collecting and analyzing a dataset of combined game logs and verbal discourses from two groups of primary school students with contrasting performances. Based on the Assessment and Teaching of 21st Century Skills CPS framework, we iteratively coded the dataset. Results of descriptive statistics showed that the successful group exhibited cognitive skills more frequently while the unsuccessful group showcased social skills more. Results of epistemic network analysis (ENA) revealed that, in both social and cognitive dimensions, the successful group demonstrated more diverse and stronger associations among various subskills, whereas there were fewer associations in the unsuccessful group. Implications are drawn for MmLA and CPS research and teaching practices of CPS skills.

Existing learning analytics (LA) studies on self-regulated learning (SRL) have rarely focused on maker education that emphasizes student autonomy in their learning process. Towards using LA methods for generating evidence of SRL in maker-based courses, this study leverages logs of a learning management system (LMS) with its activity design aligned with the maker-based pedagogy. We explored frequencies and sequential patterns of students’ SRL behaviors as reflected in the LMS logs and their relations with learning performance. Adopting a mixed method approach, we collected and triangulated both quantitative (i.e., system logs, performance scores) and qualitative (i.e., student-written reflections) data sources from 104 students. Based on current LA-based SRL research, we developed an LMS log-based analytic framework to define the SRL phases and behaviors applicable to maker activities. Statistical, data mining, and qualitative analysis methods were conducted on 48,602 logged events and 131 excerpts extracted from student reflections. Results reveal that high-performing students demonstrated some SRL behaviors (e.g., Making Personal Plans, Evaluation) more frequently than their low-performing counterparts, yet the two groups showcased fairly similar sequences of SRL behaviors. Theoretical, methodological and pedagogical implications are drawn for LA-based SRL research and maker education.

This study aims to use PLS-SEM to investigate the influence of smart instant feedback with goal setting and task strategies functions on learners’ germane cognitive load and extraneous cognitive load in the contexts of online courses. Thirty-live graduate students were recruited to participate in our experiment and complete four units of digital learning materials and questionnaires. Results show that task strategies have significant mediation effects between behavioral engagement and extraneous cognitive load and between cognitive engagement and extraneous cognitive load. Finally, according to the research findings, we provided some suggestions to online learners, instructors, and researchers whose interests are online course design. Inappropriate use of task strategies such as worked examples results in increased extraneous cognitive load.