As civil discourse in America is becoming less substantive and respectful (Doherty et al., 2019), some educators have turned to educational games as a potential solution. However, due to the complexity the civic education space, games tend to focus on conceptual knowledge (e.g., the structure of government) rather than the practical skills students need to become productive and engaged citizens. Here we present Persuasion Invasion, an educational game that uses Value-Adaptive Instruction to help students learn to engage in productive civil discourse. Throughout the game, players learn about the values that underpin our beliefs and barriers to productive discourse (e.g., tribalism). Importantly, players also practice key civil discourse skills like identifying shared values across political lines and engaging in perspective taking. In this paper, we discuss the unique challenges of designing educational games in the civics domain, how we addressed those challenges, and how students responded to our design. Our results suggest that in-game practice was associated with improved learning outcomes on key discourse skill assessments. These results, coupled with qualitative student feedback, support the development of transformative and meaningful games in the civil discourse space.

As the media landscape is increasingly populated by less than reputable sources of information, educators have turned to argument evaluation training as a potential solution. Unfortunately, the bias literature suggests that our ability to objectively evaluate an argument is, to a large extent, determined by the relationship between our own beliefs and the beliefs latent in the argument we are evaluating. If the argument supports our worldview, we are much more likely to overlook logical errors. Teachers recognize this need to adapt argument evaluation instruction to the specific beliefs of students. For instance, a teacher might intentionally assign a student an argument that the student disagrees with. Unfortunately, this kind of value-adaptive instruction is infrequent due to its unscalability. We propose a novel method for data-driven value-adaptive instruction in instructional technologies. This method can be used to combat bias in real-world contexts and support human reasoning during media consumption.

We demonstrate that, by using a small set of hand-graded student work, we can automatically generate rubric criteria with a high degree of validity, and that a predictive model incorporating these rubric criteria is more accurate than a previously reported model. We present this method as one approach to addressing the often challenging problem of grading assignments in programming environments. A classic solution is creating unit-tests that the student-generated program must pass, but the rigid, structured nature of unit-tests is suboptimal for assessing the more open-ended assignments students encounter in introductory programming environments like Alice. Furthermore, the creation of unit-tests requires predicting the various ways a student might correctly solve a problem - a challenging and time-intensive process. The current study proposes an alternative, semi-automated method for generating rubric criteria using low-level data from the Alice programming environment.

Many introductory programming environments generate a large amount of log data, but making insights from these data accessible to instructors remains a challenge. This research demonstrates that student outcomes can be accurately predicted from student program states at various time points throughout the course, and integrates the resulting predictive models into an instructor dashboard. The effectiveness of the dashboard is evaluated by measuring how well the dashboard analytics correctly suggest that the instructor help students classified as most in need. Finally, we describe a method of matching low-performing students with high-performing peer tutors, and show that the inclusion of peer tutors not only increases the amount of help given, but the consistency of help availability as well.

K-12 classrooms use block-based programming environments (BBPEs) for teaching computer science and computational thinking (CT). To support assessment of student learning in BBPEs, we propose a learning analytics framework that combines hypothesis- and data-driven approaches to discern students' programming strategies from BBPE log data. We use a principled approach to design assessment tasks to elicit evidence of specific CT skills. Piloting these tasks in high school classrooms enabled us to analyze student programs and video recordings of students as they built their programs. We discuss a priori patterns derived from this analysis to support data-driven analysis of log data in order to better assess understanding and use of CT in BBPEs.

Systematic endeavors to take computer science (CS) and computational thinking (CT) to scale in middle and high school classrooms are underway with curricula that emphasize the enactment of authentic CT skills, especially in the context of programming in block-based programming environments. There is, therefore, a growing need to measure students’ learning of CT in the context of programming and also support all learners through this process of learning computational problem solving. The goal of this research is to explore hypothesis-driven approaches that can be combined with data-driven ones to better interpret student actions and processes in log data captured from block-based programming environments with the goal of measuring and assessing students’ CT skills. Informed by past literature and based on our empirical work examining a dataset from the use of the Fairy Assessment in the Alice programming environment in middle schools, we present a framework that formalizes a process where a hypothesis-driven approach informed by Evidence-Centered Design effectively complements data-driven learning analytics in interpreting students’ programming process and assessing CT in block-based programming environments. We apply the framework to the design of Alice tasks for high school CS to be used for measuring CT during programming.