Writing-to-Learn (WTL) is an evidence-based instructional practice which can help students construct knowledge across many disciplines. Though it is known to be an effective practice, many instructors do not implement WTL in their courses due to time constraints and inability to provide students with personalized feedback. One way to address this is to include peer review, which allows students to receive feedback on their writing and benefits them as they act as reviewers. To further ease the implementation of peer review and provide instructors with feedback on their students’ work, we labeled students’ peer review comments across courses for type of feedback provided and trained a machine learning model to automatically classify those comments, improving upon models reported in prior work. We then created a dashboard which takes students’ comments, labels the comments using the model, and allows instructors to filter through their students’ comments based on how the model labels the comments. This dashboard can be used by instructors to monitor the peer review collaborations occurring in their courses. The dashboard will allow them to efficiently use information provided by peers to identify common issues in their students’ writing and better evaluate the quality of their students’ peer review.

Writing-to-learn (WTL) pedagogy supports the implementation of writing assignments in STEM courses to engage students in conceptual learning. Recent studies in the undergraduate STEM context demonstrate the value of implementing WTL, with findings that WTL can support meaningful learning and elicit students’ reasoning. However, the need for instructors to provide feedback on students’ writing poses a significant barrier to implementing WTL; this barrier is especially notable in the context of introductory organic chemistry courses at large universities, which often have large enrollments. This work describes one approach to overcome this barrier by presenting the development of an automated feedback tool for providing students with formative feedback on their responses to an organic chemistry WTL assignment. This approach leverages machine learning models to identify features of students’ mechanistic reasoning in response to WTL assignments in a second-semester, introductory organic chemistry laboratory course. The automated feedback tool development was guided by a framework for designing automated feedback, theories of self-regulated learning, and the components of effective WTL pedagogy. Herein, we describe the design of the automated feedback tool and report our initial evaluation of the tool through pilot interviews with organic chemistry students.

Writing-to-learn pedagogies are an evidence-based practice known to aid students in constructing knowledge. Barriers exist for the implementation of such assignments; namely, instructors feel they do not have time to provide each student with feedback. To ease implementation of writing-to-learn assignments at scale, we have incorporated automated peer review, which facilitates peer review without input from the instructor. Participating in peer review can positively impact students’ learning and allow students to receive feedback on their writing. Instructors may want to monitor these peer interactions and gain insight into their students’ understanding using the feedback generated by their peers. To facilitate instructors’ use of the content from students’ peer review comments, we pre-trained a transformer model called PeerBERT. PeerBERT was fine-tuned on several downstream tasks to categorize students’ peer review comments as praise, problem/solution, or verification/summary. The model exhibits high accuracy, even across different peer review prompts, assignments, and courses. Additional downstream tasks label problem/solution peer review comments as one or more types: writing/formatting, missing content/needs elaboration, and incorrect content. This approach can help instructors pinpoint common issues in student writing by parsing out which comments are problem/solution and which type of problem/solution students identify.

Providing students in STEM courses the opportunity to write about scientific content can be beneficial to the learning process. However, it is a logistical challenge to provide feedback to students’ written work in large-enrollment courses. Motivated by these reasons, the study presented herein considers a method to identify the depth of students’ scientific reasoning in their written work. A writing-to-learn (WTL) activity was implemented in a large undergraduate general chemistry class. An analytical framework of cognitive operations that characterizes students’ scientific reasoning evidenced in their writing was applied. Engagement in some of the more complex cognitive operations, such as causal reasoning and argumentation, was a sign that students were properly engaging in meaning making activities. This work considers a method to automate coaching of students in using more complex reasoning in their writing with the desired outcome that it may help students better engage with the science content. We consider a series of new natural language processing models to discern types of reasoning in student essays from the WTL activity.

We will present three similar studies that examine online peer evaluation of student-generated explanations for missed exam problems in introductory physics. In the first study, students created video solutions using YouTube and in the second two studies, they created written solutions using Google documents. All peer evaluations were performed using a tournament module as part of the interactive online coaching system called E2Coach[4] at the University of Michigan. With the theme of LAK 2014 being "intersection of learning analytics research, theory and practice", we think this poster will provide an accessible example that combines a classroom experiment with rigorous analysis to understand outcomes.