In recent years, computer science (CS) education has undergone rapid change, first due to the shift to online learning during the COVID-19 pandemic, and more recently with the rise of generative AI (GenAI) tools. How these changes impact students' learning habits and problem-solving attitude remains unknown. This paper presents a long-term analysis of attitudinal survey data from a CS2 course across eight semesters. Drawing on the Computing Attitudes Survey (CAS v4) and the Computer Science Attitudes (CSA) survey, we measure changes in seven constructs, including confidence, mindset, strategies, and motivation. We observe stable or positive shifts in earlier semesters, but sharp negative trends in Fall 2023 and Spring 2024 semesters coinciding with the widespread availability of Generative AI (GenAI) tools. These results highlight the urgent need for instructors to consider how new technologies shape not just students' learning outcomes but their beliefs about their own ability to succeed in computing.

Office hours are a core feature of CS courses. They provide a crucial vector for personalized instruction and individual support. Despite their importance, we the impact of these personal events has not been well assessed each office hour interaction is often hidden from instructors. We do not know if students actually understand the solution or the advice given during the interaction. This lack of knowledge presents a challenge to efforts geared at improving instructional practices and student outcomes. In this poster we report on the results of a survey analysis of office hour participants. Our results show that students and instructional staff had divergent assessments of the interactions with teaching staff overestimating the benefits more than 18.9% of the time. This work highlights detailed analyses of these results along with implications for teaching strategies.

Figure 1: EclipseMonitor workflow diagram.It shows how the student's coding session data from Eclipse development environment has been collected through the EclipseMonitor.

Question-asking is a crucial learning and teaching approach. It reveals different levels of students' understanding, application, and potential misconceptions. Previous studies have categorized question types into higher and lower orders, finding positive and significant associations between higher-order questions and students' critical thinking ability and their learning outcomes in different learning contexts. However, the diversity of higher-order questions, especially in collaborative learning environments. has left open the question of how they may be different from other types of dialogue that emerge from students' conversations, To address these questions, our study utilized natural language processing techniques to build a model and investigate the characteristics of students' higher-order questions. We interpreted these questions using Bloom's taxonomy, and our results reveal three types of higher-order questions during collaborative problem-solving. Students often use "Why", "How" and "What If' questions to I) understand the reason and thought process behind their partners' actions: 2) explore and analyze the project by pinpointing the problem: and 3) propose and evaluate ideas or alternative solutions. In addition. we found dialogue labeled 'Social'. 'Question - other', 'Directed at Agent', and 'Confusion/Help Seeking' shows similar underlying patterns to higher-order questions, Our findings provide insight into the different scenarios driving students' higher-order questions and inform the design of adaptive systems to deliver personalized feedback based on students' questions.

Promptly and properly addressing students' help requests during office hours is a critical challenge for large CS courses. With a large number of help requests, the queue gets longer and students have to endure long wait times. To address this problem, we try to quantify students' patience in the queue through survival analysis. Our results show that half of the students are willing to stay in the queue after waiting for 142.5 minutes. Moreover, we find that female students, morning requests, returning students, and requests about test failures are more likely to stay in the queue for a longer time.

The emergence of publicly accessible large language models (LLMs) such as ChatGPT poses unprecedented risks of new types of plagiarism and cheating where students use LLMs to solve exercises for them. Detecting this behavior will be a necessary component in introductory computer science (CS1) courses, and educators should be well-equipped with detection tools when the need arises. However, ChatGPT generates code non-deterministically, and thus, traditional similarity detectors might not suffice to detect AI-created code. In this work, we explore the affordances of Machine Learning (ML) models for the detection task. We used an openly available dataset of student programs for CS1 assignments and had ChatGPT generate code for the same assignments, and then evaluated the performance of both traditional machine learning models and Abstract Syntax Tree-based (AST-based) deep learning models in detecting ChatGPT code from student code submissions. Our results suggest that both traditional machine learning models and AST-based deep learning models are effective in identifying ChatGPT-generated code with accuracy above 90%. Since the deployment of such models requires ML knowledge and resources that are not always accessible to instructors, we also explore the patterns detected by deep learning models that indicate possible ChatGPT code signatures, which instructors could possibly use to detect LLM-based cheating manually. We also explore whether explicitly asking ChatGPT to impersonate a novice programmer affects the code produced. We further discuss the potential applications of our proposed models for enhancing introductory computer science instruction.

To promote good coding practices, we need to understand what students do when they are on their own. In this research study, we explore students' testing behavior and response to persistent errors to better understand their coding patterns. We investigate how those patterns change when they struggle, and how help-seeking might influence their coding behaviors. We define struggle during coding as failing the same unit test case consecutively for more than four submission events, considering only unit test cases created by the instructors. To analyze the students' coding data, we use progress indicators, student test implementation indicators, and both student-generated and instructor-generated unit test results from each student submission event. In addition, we use office hours attendance records and amount of assignment-related posts created on the course forum. Results show that students tend not to follow test-driven development practices, even when explicitly directed to, and tend to create unit tests only to earn assignment credit rather than to guide their software development. Students also tend not to modify their own unit tests once they have earned the related credits, even when facing coding struggle; they tend to modify their unit tests only after they have been facing coding struggle for an extended number of submission events.

Use of technology-enhanced education and online learning systems has become more popular, especially after COVID-19. These systems capture a rich array of data as students interact with them. Predicting student performance is an essential part of technology-enhanced education systems to enable the generation of hints and provide recommendations to students. Typically, this is done through use of data on student interactions with questions without utilizing important data on the temporal ordering of students’ other interaction behavior, (e.g., reading, video watching). In this paper, we hypothesize that to predict students’ question performance, it is necessary to (i) consider other learning activities beyond question-answering and (ii) understand how these activities are related to question-solving behavior. We collected middle school physical science students’ data within a K12 reading platform, Actively Learn. This platform provides reading-support to students and collects trace data on their use of the system. We propose a transformer-based model to predict students' question scores utilizing question interaction and reading-related behaviors. Our findings show that integrating question attempts and reading-related behaviors results in better predictive power compared to using only question attempt features. The interpretable visualization of the transformer’s attention can be helpful for teachers to make tailored interventions in students’ learning.

Intelligent virtual agents have tremendous potential for facilitating collaborative learning by modeling and reinforcing desirable collaborative practices. Despite recent work in this area, the extent to which intelligent virtual agents can facilitate improvements in the collaborative behavior of children is largely unknown. This study employed a wizard-of-oz study design and investigated elementary children's collaborative behavior after interacting with virtual agents. These agents model exploratory talk for upper elementary school dyads, such as asking higher-order questions and listening to their partners. The findings uncover associations between elementary learner dyads' positive changes in collaboration after agent interventions, the dyads' affective reactions to interventions, and their attentiveness to the agents. Our results also reveal associations between positive changes in collaboration and the timing of interventions: for example, earlier interventions had a higher occurrence of positive changes, and positive changes in collaboration typically happened within five seconds of interventions. The results suggest ways in which intelligent virtual agents may be used to promote effective collaborative learning practices for children.

As enrollment in CS programs have risen, it has become increasingly difficult for teaching staff to provide timely and detailed guidance on student projects. To address this, instructors use automated assessment tools to evaluate students' code and processes as they work. Even with automation, understanding students' progress, and more importantly, if students are making the 'right' progress toward the solution is challenging at scale. To help students manage their time and learn good software engineering processes, instructors may create intermediate deadlines, or milestones, to support progress. However, student's adherence to these processes is opaque and may hinder student success and instructional support. Better understanding of how students follow process guidance in practice is needed to identify the right assignment structures to support development of high-quality process skills.

In Computer Science (CS) education, instructors use office hours for one-on-one help-seeking. Prior work has shown that traditional in-person office hours may be underutilized. In response many instructors are adding or transitioning to virtual office hours. Our research focuses on comparing in-person and online office hours to investigate differences between performance, interaction time, and the characteristics of the students who utilize in-person and virtual office hours. We analyze a rich dataset covering two semesters of a CS2 course which used in-person office hours in Fall 2019 and virtual office hours in Fall 2020. Our data covers students' use of office hours, the nature of their questions, and the time spent receiving help as well as demographic and attitude data. Our results show no relationship between student's attendance in office hours and class performance. However we found that female students attended office hours more frequently, as did students with a fixed mindset in computing, and those with weaker skills in transferring theory to practice. We also found that students with low confidence in or low enjoyment toward CS were more active in virtual office hours. Finally, we observed a significant correlation between students attending virtual office hours and an increased interest in CS study; while students attending in-person office hours tend to show an increase in their growth mindset.

CS courses often use a variety of learning activities to assist students while learning concepts. These activities' levels of engagement can be categorized through the ICAP framework as Interactive, Constructive, Active, and Passive respectfully. For this work, we categorize learning activities from an online professional development course and analyzed the probabilities of transitioning between ICAP modalities on a week-by-week basis. This poster presents our analysis on which ICAP modes students visited during each week of the course. We found the majority of students would review Passive materials before Interactive activities, then repeating this process. The second most common transition followed the ICAP Framework, selecting activities with increasing levels of engagement. Contrary to our assumptions, students primarily worked on 'new' materials, rather than 'review' previous activities.

Classroom dashboards are designed to help instructors effectively orchestrate classrooms by providing summary statistics, activity tracking, and other information. Existing dashboards are generally specific to an LMS or platform and they generally summarize individual work, not group behaviors. However, CS courses typically involve constellations of tools and mix on- and offline collaboration. Thus, cross-platform monitoring of individuals and teams is important to develop a full picture of the class. In this work, we describe our work on Concert, a data integration platform that collects data about student activities from several sources such as Piazza, My Digital Hand, and GitHub and uses it to support classroom monitoring through analysis and visualizations. We discuss team visualizations that we have developed to support effective group management and to help instructors identify teams in need of intervention.

The COVID-19 pandemic shifted many U.S. schools from in-person to remote instruction. While collaborative CS activities had become increasingly common in classrooms prior to the pandemic, the sudden shift to remote learning presented challenges for both teachers and students in implementing and supporting collaborative learning. Though some research on remote collaborative CS learning has been conducted with adult learners, less has been done with younger learners such as elementary school students. This experience report describes lessons learned from a remote after-school camp with 24 elementary school students who participated in a series of individual and paired learning activities over three weeks. We describe the design of the learning activities, participant recruitment, group formation, and data collection process. We also provide practical implications for implementation such as how to guide facilitators, pair students, and calibrate task difficulty to foster collaboration. This experience report contributes to the understanding of remote CS learning practices, particularly for elementary school students, and we hope it will provoke methodological advancement in this important area.

Pair programming is a popular strategy in computer science education to teach programming to novices. In this study, we examined the effect of three different pair programming conditions on upper elementary school students' CS conceptual understanding. The three conditions were one-computer with roles (1C with roles), two computers without roles (2C no roles), and two computers with roles (2C with roles). These students were engaged in four days of computer programming activities and took the CS concept assessment, CS attitudes, and collaboration perceptions before and after the activities. We used the validated E-CSCA (Elementary Computer Science Concepts Assessment) to measure elementary students' understanding of CS concepts. We tested the relationship of different pair programming conditions on the students' CS conceptual understanding and found that different conditions impacted students' CS conceptual understanding, wherein students in 2C roles demonstrated better CS learning than the other two conditions. The results also showed no changes in students' CS attitudes and perceptions of collaboration before and after the activities. Furthermore, the results indicated no significant impact of these attitudinal factors on students' learning CS concepts in pair programming settings. Our study highlights the importance of the roles and number of computers in pair programming settings, especially for elementary students.

In successful collaborative paradigms such as pair programming, students engage in productive dialogue and work to resolve conflicts as they arise. However, little is known about how elementary students engage in collaborative dialogue for computer science learning. Early findings indicate that these younger students may struggle to manage conflicts that arise during pair programming. To investigate collaborative dialogue that elementary learners use and the conflicts that they encounter, we analyzed videos of twelve pairs of fifth grade students completing pair programming activities. We developed a novel annotation scheme with a focus on collaborative dialogue and conflicts. We found that student pairs used best-practice dialogue moves such as self-explanation, question generation, uptake, and praise in less than 23% of their dialogue. High-conflict pairs antagonized their partner, whereas this behavior was not observed with low-conflict pairs. We also observed more praise (e.g., "We did it!") and uptake (e.g., "Yeah and...") in low-conflict pairs than high-conflict pairs. All pairs exhibited some conflicts about the task, but high-conflict pairs also engaged in conflicts about control of the computer and their partner's contributions. The results presented here provide insights into the collaborative process of young learners in CS problem solving, and also hold implications for educators as we move toward building learning environments that support students in this context.

Analytics about how students navigate online learning tools throughout the duration of an assignment is scarce. Knowledge about how students use online tools before a course's end could positively impact students' learning outcomes. We introduce PEDI (Piazza Explorer Dashboard for Intervention), a tool which analyzes and presents visualizations of forum activity on Piazza, a question and answer forum, to instructors. We outline the design principles and data-informed recommendations used to design PEDI. Our prior research revealed two critical periods in students' forum engagement over the duration of an assignment. Early engagement in the first half of an assignment duration positively correlates with class average performance. Whereas, extremely high engagement toward the deadline predicted lower class average performance. PEDI uses these findings to detect and flag troubling engagement levels and informs instructors through clear visualizations to promote data-informed interventions. By providing insights to instructors, PEDI may improve class performance and pave the way for a new generation of online tools.

Collaborative learning has demonstrated benefits for girls in computer science [7] and this may be a way to help address the gender gap in CS. Research indicates that while collaborating, boys often express more individualistic ideas whereas girls tend to be more supportive [1]. It is important for students to regulate their learning in collaborative learning environments because they need to negotiate group goals and diverse approaches to the task [3], and the use of open-ended tasks with multiple solution paths are common [4]. There is minimal research in CS education on regulation of learning (e.g., [5,6,]). Co-regulated learning is the process in which an other helps regulate the learning of a student [2] self such as by asking questions that prompt the student to monitor and evaluate (e.g., "What do you already know about 'if' blocks that would help here?''). In this way, thinking and reasoning through the problem is shared by the group members [2] self.

In this paper, we perform a predictive analysis of a curriculum-integrated math game, ST Math, to suggest a partial ordering for the game's curriculum sequence. We analyzed the sequence of ST Math objectives played by elementary school students in 5 U.S. districts and grouped each objective into difficult and easy categories according to how many retries were needed for students to master an objective. We observed that retries on some objectives were high in one district and low in another district where the objectives are played in a different order. Motivated by this observation, we investigated what makes an effective curriculum sequence. To infer a new partially-ordered sequence, we performed an expanded replication study of a novel predictive analysis by a prior study to find predictive relationships between 15 objectives played in different sequences by 3,328 students from 5 districts. Based on the predictive abilities of objectives in these districts, we found 17 suggested objective orderings. After deriving these orderings, we confirmed the validity of the order by evaluating the impact of the suggested sequence on changes in rates of retries and corresponding performance. We observed that when the objectives were played in the suggested sequence, we record a drastic reduction in retries, implying that these objectives are easier for students. This indicates that objectives that come earlier can provide prerequisite knowledge for later objectives. We believe that data-informed sequences, such as the ones we suggest, may improve efficiency of instruction and increase content learning and performance.

We present a data-driven analysis that provides generalized insights of how a curriculum integrated educational math game gets used as a routinized classroom activity throughout the year in authentic primary school classrooms. Our study relates observations from a field study on Spatial Temporal Math (ST Math) to our findings mined from ST Math students' sequential game play data. We identified features that vary across game play sessions and modeled their relationship with session performance. We also derived data-informed suggestions that may provide teachers with insights into how to design classroom game play sessions to facilitate more effective learning.

Rich models of students' learning and problem-solving behaviors can support tailored interventions by instructors and scaffolding of complex learning activities. Our goal in this paper is to identify students' reading behaviors as they engage with instructional texts in domain-specific activities. In this work, we apply theory and methodology from the learning sciences to a large-scale middle school dataset within a digital literacy platform, Actively Learn. We compare students' reading behaviors both within and across domains for 12,566 science and 16,240 social studies students. Our findings show that higher-performing students in science engaged in more metacognitively-rich reading activities, such as text annotation; whereas lower-performing students relied more on simple highlighting and took longer to respond to embedded questions. Higher-performing students in social studies, by contrast, engaged more with the vocabulary and took longer to read before attempting question responses. Our finding may be used as recommendations to help both teachers and students engage in and support more effective behaviors.

As computer science education opportunities for elementary students (grades K-5) are expanding, there is growing interest in using pair programming with these students. However, previous research findings do not fully support its use with younger learners, and some researchers have begun to examine whether introducing a second computer with a shared coding workspace can provide important benefits. This experience report describes a series of classroom activities in the 4th and 5th grades (ages 9-11 years old) with two different pair programming configurations: one-computer pair programming, in which both students share a keyboard, mouse, and monitor; and two-computer pair programming, in which each student has a separate computer but coding workspaces are synchronized over the web. In both cases the students sat next to each other and engaged in face-to-face conversation. We found that students largely preferred two-computer pair programming over one-computer pair programming. We conducted focus groups and transcribed collaborative dialogues to gain more insight into this preference. We learned that students felt more independence in two-computer pair programming, although they struggled with coordinating their edits with their partner. In one-computer pair programming, students reported not wanting to wait for their turn to drive, but feeling as though they communicated more with their partner. Both configurations can be productive for students, but the tradeoffs described in this experience report are important for CS educators and researchers to consider when determining which collaborative configuration to use in each K-5 classroom context.

The language and concepts used by curriculum designers are not always interpreted by children as designers intended. This can be problematic when researchers use self-reported survey instruments in concert with curricula, which often rely on the implicit belief that students’ understanding aligns with their own. We report on our refinement of a validated survey to measure upper elementary students’ attitudes and perspectives about computer science (CS), using an iterative, design-based research approach informed by educational and psychological cognitive interview processes. We interviewed six groups of students over three iterations of the instrument on their understanding of CS concepts and attitudes toward coding. Our findings indicated that students could not explain the terms computer programs nor computer science as expected. Furthermore, they struggled to understand how coding may support their learning in other domains. These results may guide the development of appropriate CS-related survey instruments and curricular materials for K–6 students.

Participatory design practices create informed designs by bringing stakeholders into the design process early and often. This approach is a powerful tool, especially when the designer and the intended user are very different. This paper reports on work in which researchers co-design pedagogical agents to support collaborative computer science learning with elementary school students using an iterative drawing methodology. In the open drawing phase, students drew what they believe good collaboration looked like. Next, researchers analyzed those drawings under the requirements of the broader project and created a drawing scaffold (similar to a coloring book page). In the scaffolded drawing phase, students ideated within the more focused context. This process resulted in actionable design guidelines for the appearance of pedagogical agents.

We present a new framework describing how teachers use ST Math, a curriculum-integrated, year-long educational game, in 3rd-4th grade classrooms. We combined authentic classroom observations with teacher interviews to identify teacher needs and practices. Our findings extended and contrasted with prior work on teachers' behaviors around classroom games, identifying differences likely arising from a digital platform and year-long curricular integration. We suggest practical ways that curriculum-integrated games can be designed to help teachers support effective classroom culture and practice.

A significant issue Computer Science students face are syntax errors. This poster presents two studies on the use of typing exercises. In a usability and interaction study, 14 students were asked to complete typing, fill in the blank, and self-explanation style exercises. Fill in the blank were similar to typing exercises with 1 line of code omitted. Self-explanation exercises were graded on whether the student could adequately describe how the source code of a program worked. Students with "Poor"-labeled self-explanations experienced more typing errors and took more time completing exercises. In a semester-long study, 99 students in a CS2 course completed 538 submissions of 66 weekly typing exercises. Students were divided into four categories: users that joined but never used the platform, users that barely used the platform, regular viewers of exercises, and regular completers of exercises. Regular completers earned a minimum final letter grade of a B, compared to 90% regular viewers, 76% from barely used, and 81% from never used. Regular completers were not simply high performing students that did additional work, as 40% of regular completers scored a C or lower on the course's first midterm exam. Based on these findings, students who used the system performed similarly or better than students who did not. While not the only source of practice, typing exercises (and other novel exercises) can serve as a viable tool for teaching Computer Science and boosting low-performing students' abilities.

Extensive prior research suggests that pair programming holds many benefits for novices. Pair programming has been well studied at the undergraduate level, and recently, the CS education research community has started to realize that younger learners may also benefit from pair programming. However, an important factor in pair programming success for young learners is the ability to resolve conflicts during the process. Little is known about what types of conflicts occur while elementary students pair program or how those conflicts are, or are not, resolved. To investigate this phenomenon, we analyzed the videos of six pairs of students completing a programming activity. We found that conflicts evolve in four general stages, which may not all be present in each conflict: initiation, escalation, de-escalation, and conclusion. Some conflicts are resolved when the students come to an agreement, others end passively. The analysis revealed that the pairs' conflicts began around disagreements about code, who should have control of the keyboard and mouse, and other interpersonal events. This research indicates that conflicts are a significant concern for young students, and supporting young learners in developing improved collaboration skills is a key direction for CS education research.

Demand for K-6 computer science (CS) curricula is growing considerably. Many of the existing curricula have been developed by domain experts who are comfortable with specific and technical terminology, which they expect students to master. However, children are not always comfortable with these terms nor do they understand general concepts like 'coding' in the way that the curriculum designers intend. This is a problem because many researchers use self-report and attitudinal survey instruments with the implicit belief that the students' understanding of the terms and concepts resemble their own. This mismatch may invalidate results. For this project, we report on our modification of a validated survey to measure upper elementary students' attitudes about and perspectives on CS by attempting to understand the appropriate language to use when querying children about these topics. We use an iterative, design-based research approach that is informed by educational and psychological cognitive interview processes. We interviewed two groups (N=64) of upper elementary students on their understanding of computer science concepts and attitudes toward coding. Our findings indicate that 4th and 5th grade students could not explain the terms computer programs nor computer science as we had expected and that they struggled to understand how coding may connect with or support their learning in other domains. These results will help to guide the development of appropriate survey instruments and course materials for K-6 students, which both match their use of broad domain concepts and therefore inform their understanding and improve their outcomes.

Collaboration plays an essential role in computer science. While there is growing recognition that learners of all ages can benefit from collaborative learning, little is known about how elementary-age children engage in collaborative problem solving in computer science. This paper reports on the analysis of a dataset of elementary students collaborating on a programming project. We found that children tend to make several different types of suggestions. In turn, their partners address those suggestions in different ways such as by implementing them directly in code or by replying through dialogue. We observe that students regularly accept or reject suggestions without explanation or explicit acknowledgement and that it is often unclear whether they understand the substance of the suggestion. These behaviors may inhibit the development of a shared understanding between the partners and limit the value of the collaborative process. These results can inform instructional practice and the development of new adaptive tools that facilitate productive collaborative problem solving in computer science.

Elementary computer science has gained increasing attention within the computer science education research community. We have only recently begun to explore the many unanswered questions about how young students learn computer science, how they interact with each other, and how their skill levels and backgrounds vary. One set of unanswered questions focuses on gender equality for young computer science learners. This paper examines how the gender composition of collaborative groups in elementary computer science relates to student achievement. We report on data collected from an in-school 5th grade computer science elective offered over four quarters in 2014-2015. We found a significant difference in the quality of artifacts produced by learner groups depending upon their gender composition, with groups of all female students performing significantly lower than other groups. Our analyses suggest important factors that are influential as these learners begin to solve computer science problems. This new evidence of gender disparities in computer science achievement as young as ten years of age highlights the importance of future study of these factors in order to provide effective, equitable computer science education to learners of all ages.

Argumentation is central to law. Written and oral argument structures, however, are often difficult to analyze and employ in instruction. Diagrammatic models of argument offer a potential solution to these problems. In this paper we report on the results of an empirical study into the diagnostic utility of argument diagrams in a legal writing context. The focus is on comparing experts' and student-produced argument diagrams and on the extent to which the latter can be used to predict students' performance on subsequent writing tasks. We present the results and draw some tentative conclusions.

This paper describes a study in which student-created diagrams about arguments in an ill-defined domain were manually graded by two independent human graders. Findings include that the graders overall agreed with each other on their grades, but their agreement was lower than one would expect in well-defined domains, and higher for solutions of extreme quality.

The development of graphical argument models is an active and growing area of research in Artificial Intelligence and Law. The aim is to develop models which may be readily used by legal professionals and novices to produce and parse arguments. If this goal is to be realized it is important to develop models that human reasoners can manipulate and assess consistently. We report on an ongoing study of graph agreement in the context of the LARGO system.

Diagrammatic models of argument are increasingly prominent in AI and Law. Unlike everyday language these models formalize many of the the components and relationships present in arguments and permit a more formal analysis of an arguments' structural weaknesses. Formalization, however, can raise problems of agreement. In order for argument diagramming to be widely accepted as a communications tool, individual authors and readers must be able to agree on the quality and meaning of a diagram as well as the role that key components play. This is especially problematic when arguers seek to map their diagrams to or from more conventional prose. In this paper we present results from a grader agreement study that we have conducted using LARGO diagrams. We then describe a detailed example of disagreement and highlight its implications for both our diagram model and modeling argument diagrams in general.

This paper presents a process model of arguing with hypotheticals and uses it to explain examples of oral arguments before the U.S. Supreme Court that are like those employed in Socratic law teaching. The process model has been partially implemented in the LARGO (Legal ARgument Graph Observer) intelligent tutoring system. The program supports students in diagramming oral argument examples; its feedback on students' diagrammatic reconstructions of the examples enforces the expectations of the process model. The paper presents empirical evidence that features of the argument diagrams made with LARGO are correlated with independent measures of argumentation ability. The examples and empirical results support the model's explanatory and diagnostic utility.

This paper describes an intelligent tutoring system, LARGO, that helps students learn skills of legal reasoning with hypotheticals by analyzing oral arguments before the US Supreme Court. The skills involve proposing a rule-like test for deciding a case, posing hypotheticals to challenge the rule, and responding by analogizing or distinguishing the hypotheticals and/or modifying the proposed test. Students diagram arguments in a special-purpose graphical language and receive feedback in the form of reflection questions.

The transcripts of oral arguments before the US Supreme Court provide interesting opportunities from the viewpoint of legal education. As the pinnacle of legal argumentation, they illustrate, often in dramatic fashion, a sophisticated process of concept formation and testing driven by skillful posing of hypotheticals. Yet it is not easy to get beginning law students to understand the arguments and the underlying processes of hypothesis formation and testing. We introduce a novel project with the dual aims of developing an AI model of concept formation and testing as well as an intelligent tutoring system for beginning law students. We describe a planned experiment in which we will evaluate to what extent law students' study of the Supreme Court oral arguments can be improved by providing detailed and specific self-explanation prompts. It is hypothesized that detailed prompts to explain connections between tests, rationales, dimensions, and hypotheticals will help students to induce adequate mental models of concept formation processes.