Automated Essay Scoring (AES) and Automatic Essay Feedback (AEF) systems aim to reduce the workload of human raters in educational assessment. However, most existing systems prioritize numeric scoring accuracy over feedback quality and are primarily evaluated on pre-secondary school level writing. This paper presents Multi-Agent Argumentation and Grammar Integrated Critiquer (MAGIC), a framework using five specialized agents to evaluate prompt adherence, persuasiveness, organization, vocabulary, and grammar for both holistic scoring and detailed feedback generation. To support evaluation at the college level, we collated a dataset of Graduate Record Examination (GRE) practice essays with expert-evaluated scores and feedback. MAGIC achieves substantial to near-perfect scoring agreement with humans on the GRE data, outperforming baseline LLM models while providing enhanced interpretability through its multi-agent approach. We also compare MAGIC's feedback generation capabilities against ground truth human feedback and baseline models, finding that MAGIC achieves strong feedback quality and naturalness.

With the growing use of Large Language Model (LLM)-based Question-Answering (QA) systems in education, it is critical to evaluate their performance across individual pipeline components. In this work, we introduce EduMod-LLM, a modular function-calling LLM pipeline, and present a comprehensive evaluation along three key axes: function calling strategies, retrieval methods, and generative language models. Our framework enables fine-grained analysis by isolating and assessing each component. We benchmark function-calling performance across LLMs, compare our novel structure-aware retrieval method to vector-based and LLM-scoring baselines, and evaluate various LLMs for response synthesis. This modular approach reveals specific failure modes and performance patterns, supporting the development of interpretable and effective educational QA systems. Our findings demonstrate the value of modular function calling in improving system transparency and pedagogical alignment.

This paper considers the development of an AI-based provably-correct mathematical proof tutor. While Large Language Models (LLMs) allow seamless communication in natural language, they are error prone. Theorem provers such as Lean allow for provable-correctness, but these are hard for students to learn. We present a proof-of-concept system (LeanTutor) by combining the complementary strengths of LLMs and theorem provers. LeanTutor is composed of three modules: (i) an autoformalizer/proof-checker, (ii) a next-step generator, and (iii) a natural language feedback generator. To evaluate the system, we introduce PeanoBench, a dataset of 371 Peano Arithmetic proofs in human-written natural language and formal language, derived from the Natural Numbers Game.

Adaptive programming practice often relies on fixed libraries of worked examples and practice problems, which require substantial authoring effort and may not correspond well to the logical errors and partial solutions students produce while writing code. As a result, students may receive learning content that does not directly address the concepts they are working to understand, while instructors must either invest additional effort in expanding content libraries or accept a coarse level of personalization. We present an approach for knowledge-component (KC) guided educational content generation using pattern-based KCs extracted from student code. Given a problem statement and student submissions, our pipeline extracts recurring structural KC patterns from students' code through AST-based analysis and uses them to condition a generative model. In this study, we apply this approach to worked example generation, and compare baseline and KC-conditioned outputs through expert evaluation. Results suggest that KC-conditioned generation improves topical focus and relevance to students' underlying logical errors, providing evidence that KC-based steering of generative models can support personalized learning at scale.

We aim to bring together colleagues who share a common interest in Computer Science and Engineering Bridge programs, which are designed to bridge the gap for less experienced and/or underserved students as they begin their university studies. We invite those interested in starting bridge programs at their institutions, as well as those knowledgeable about existing bridge programs, their best practices, and lessons learned. Together, we will discuss the most effective approaches to these programs and challenges that arise in their implementations, including exclusivity in the selection of admits, target audience, funding limitations, outreach, and more. We aim to have participants leave with resources, including information about other programs, points of contact for networking, a collaborative document from the session, as well as a website for hosting resources of existing programs. Our goal is to foster conversation and create a network of support so these programs can sustain despite general discouragement.

Large CS1 courses routinely need several versions of the same idea, such as practice items, make ups, and multi-form exams that target the same learning objective. he manual authoring of these isomorphic items is slow, and automatic generation of one shot often wanders off concept, changes difficulty, or produces code that does not run. We address this problem with a verification first, self-healing framework for generating CS1 exercise variants. The design is generator agnostic. In our framework, we use three separate role instances of a Large Language Model (LLM): a Generator that proposes candidate items, an Evaluator that checks them against constraints extracted from the base problem, and a Solver that produces a reference solution. Acceptance is determined only by executable docstring tests (doctests). When tests fail, a lightweight controller turns failure traces into targeted repairs and retries. In short, generation proposes and tests decide.

Recent advancements in large language models (LLMs) have paved the way for AI educational assistants in academic settings. However, AI assistants often respond differently than TAs, providing extensive explanations that may overwhelm students or inadvertently reveal more than intended. This study identifies the main differences between TA and LLM responses to students by using a four-class utterance classification system to compare the utterance distributions found in TA replies and in responses generated by Edison, a state-of-the-art AI educational assistant. Using this classification, striking distributional differences are observed: Edison produces far more Advance utterances, whereas TAs use many more React and Social Convention utterances. This research examines how differences in these distributions relate to response quality in student–TA interactions. Through prompt engineering, we align Edison's utterance distribution with TA patterns, producing responses that are more concise, directly address student questions, and avoid unnecessary elaboration.

Active learning in introductory programming depends on frequent, high-quality feedback, yet instructors often struggle to deliver consistent support at scale. In this poster, we introduce INSIGHT, an AI-driven classroom assistant designed to promote active learning in introductory programming (CS1) courses through scalable, personalized, and explainable feedback. The assistant combines the generative capabilities of large language models (LLMs) with instructor-in-the-loop authoring and an explainable code analysis engine to ensure pedagogically aligned support. Instructors can co-design problems with LLM assistance, provide exemplar solutions, define common student errors, and author targeted feedback. The AI engine analyzes student code submissions, identifies misconceptions, and maps them to instructor-verified feedback in real time. INSIGHT is designed to ensure that key educational concepts and common misconceptions are explicitly addressed by instructors, while also leveraging LLMs to provide reasonable feedback for novel or edge-case solutions that instructors may not have anticipated. By combining instructor expertise with the flexibility of generative AI, the assistant helps close feedback gaps and ensures more comprehensive coverage of student learning needs, especially in large or diverse classrooms with limited instructional support.

We study instructor perspectives on LLM-generated programming feedback in an introductory Python course. LLM tutors predominantly offered debugging help, while human instructors preferred more diverse feedback types, including conceptual reminders, revisiting the problem, and examples. Cases where LLM tutor feedback diverged from human instructors' intent required major edits with different feedback types, while cases with closer alignment needed only minor changes with similar feedback types. Findings highlight the need for LLM tutors to reflect on instructor intent to ensure pedagogically aligned feedback.

Large Language Models (LLMs) are increasingly used as programming tutors, but their feedback is often generic and prone to solution leakage. To address these issues, we present MisconceptionTutor, which grounds feedback in common student misconceptions. Through both pre-deployment analyses and a real-classroom deployment, we find that even simple prompting frameworks can meaningfully steer tutor behavior to be more pedagogically oriented and noticeably more satisfying to students.

We present a reproducible pipeline that aligns CS1 textbook sections with problems from a public dataset via a Knowledge Component (KC) -a single conceptual skill required for problem solving- ontology. It assigns KCs to sections and problems, respects the prerequisite order to avoid inserting problems too early, and generates tips for not-yet-taught concepts. We evaluate three KC assignment strategies: embedding-only, embedding with a Large Language Model (LLM) tie-breaker, and direct LLM assignment. We find direct assignment matches or exceeds human annotators. Our results show that constrained LLMs can enrich CS1 textbooks with curriculum-aware practice problems.

Artificial Intelligence (AI) has impacted the world tremendously in the last decade, causing an increased demand for accessible AI education globally. Students benefit from studying AI earlier in the curriculum; however, AI courses can require a range of prerequisites, which can be structured differently in various educational contexts. In this paper, we study the curriculum structure of AI, Machine Learning (ML), and Data Science (DS) courses in Canadian Universities and compare it with that of US Research-1 institutions. There are many similarities between AI, ML, and DS courses in Canada and the US. For example, DS courses tend to be more accessible earlier in the CS curriculum compared to AI and ML. However, there are key differences between the two countries, with Canadian AI, ML, and DS courses generally being a part of a longer prerequisites chain, and Canadian CS departments offering fewer DS courses. Still, both Canadian and US institutions find innovative ways to introduce AI earlier in the curriculum, including via interdisciplinary courses and specialized courses with few prerequisites. This study corroborates earlier work in recognizing diversity in curricular frameworks in North America and recommends curricular revisions and early academic advising to ensure access to AI courses.

Vision Transformers (ViTs) have shown remarkable performance and scalability across various computer vision tasks. To apply single-scale ViTs to image segmentation, existing methods adopt a convolutional adapter to generate multi-scale features, a pixel decoder to fuse these features, and a Transformer decoder that uses the fused features to make predictions. In this paper, we show that the inductive biases introduced by these task-specific components can instead be learned by the ViT itself, given sufficiently large models and extensive pre-training. Based on these findings, we introduce the Encoder-Only Mask Transformer (EoMT), which repurposes the plain ViT architecture to conduct image segmentation. With large-scale models and pre-training, EoMT obtains a segmentation accuracy similar to state-of-the-art models that use task-specific components. At the same time, EoMT is significantly faster than these methods due to its architectural simplicity, e.g., up to 4 × faster with ViT-L. Across a range of model sizes, EoMT demonstrates an optimal balance between segmentation accuracy and prediction speed, suggesting that compute resources are better spent on scaling the ViT itself rather than adding architectural complexity. Code: https://www.tue-mps.org/eomt/.

Mastering algorithms and graph theory requires students to understand both the theoretical concepts and the practical mechanics. While most current assessments focus on the practical aspects, a deeper understanding of the theoretical concepts is often more crucial for truly grasping the material. Visualizations aim to help bridge this gap by allowing students to interact with data structures to trace traversals and outputs dynamically. We introduce an interactive tool through an online assessment platform that will enable students to click on different nodes and/or edges to dynamically change a graph model. There are numerous use cases, from introductory data structures and traversals such as depth-first and breadth-first search to more complicated algorithms such as tracing hypercube node processing. Although there are currently decorative components that display graphs and can be supplemented with submission elements, we hypothesize that by combining both features into one, students' learning will be significantly more effective. Through such a tool, we plan to assess students' performance in regard to (a) their score, (b) completion time, and (c) student satisfaction with the interactive assessments. We plan to analyze the types of errors students make depending on whether they are in the control or experimental groups. Further, we aim to assess how abstracting interactive assessment tools can be applied to introductory computer science courses to bridge the gap between proficiency and mastery learning.

While Large Language Models (LLMs) have emerged as promising methods for automated student question-answering, guaranteeing consistent instructional effectiveness of the response remains a key challenge. Therefore, there is a need for fine-grained analysis of State-Of-The-Art (SOTA) LLM-powered educational assistants.

Large Language Models (LLMs) can be used to automate many aspects of the educational field. In this paper, we look into the benefits of automating responses to student questions in course discussion forums using our Retrieval-Augmented Generation (RAG)-based LLM pipeline (Edison). Our research questions are:

Identifying Knowledge Components (KCs) in computer science education improves curriculum design and teaching strategies. We introduce a framework using Large Language Models to identify KCs from programming assignments automatically. Our framework helps educators align assignments with course objectives. GPT-4 identifies relevant KCs well, though there's a low match with expert-generated KCs at the course level. At the problem level, performance is lower, but key KCs are reasonably identified.

Summer bridge programs are initiatives aimed at supporting students as they transition into university, helping them build confidence, skills, and a sense of belonging. Many existing programs focus on serving less experienced and/or historically underrepresented students, striving to broaden participation in STEM fields. We aim to bring together colleagues who share a common interest in Computer Science and Engineering Bridge programs. We invite both those interested in starting bridge programs at their institutions and those with knowledge of existing programs, including best practices and lessons learned. We will collaborate to discuss the most effective approaches to these programs and challenges that arise in their implementations, including exclusivity in the selection of admits, target audience, funding limitations, outreach, and more. We aim to have participants leave with resources, including information about other programs, points of contact for networking, and a collaborative document from the discussion, as well as a website for hosting resources of existing programs.

Gauging a student's understanding of course concepts, at an arbitrary point during a course, can be challenging. Standardized exams offer only a snapshot of performance rather than a deep understanding of progress. However, with Large Language Models (LLMs) now deployed at scale in CS1 courses, we can track multiple attempts from each student for every homework problem. This data provides insights into how students learn and deploy concepts over time, presenting a unique opportunity to rethink how we track changes in individual student knowledge. Traditional Knowledge Tracing (KT) methods often lack explainability and are computationally expensive. In contrast, our framework leverages an LLM to identify student progress on labeled, problem-level concepts from a student homework code submission. Our initial results show that the student's knowledge state can be dynamically updated. This knowledge state can then be used to provide more targeted, effective feedback and create tailored study materials.

LLM-based chatbots enable students to get immediate, interactive help on homework assignments, but even a thoughtfully-designed bot may not serve all pedagogical goals. We report here on the development and deployment of a GPT-4-based interactive homework assistant ("61A Bot'') for students in a large CS1 course; over 2000 students made over 100,000 requests of our Bot across two semesters. Our assistant offers one-shot, contextual feedback within the command-line "autograder'' students use to test their code. Our Bot wraps student code in a custom prompt that supports our pedagogical goals and avoids providing solutions directly. Analyzing student feedback, questions, and autograder data, we find reductions in homework-related question rates in our course forum, as well as reductions in homework completion time when our Bot is available. For students in the 50th -80th percentile, reductions can exceed 30 minutes per assignment, up to 50% less time than students at the same percentile rank in prior semesters. Finally, we discuss these observations, potential impacts on student learning, and other potential costs and benefits of AI assistance in CS1.

This paper focuses on using Large Language Models to support teaching assistants in answering questions on large student forums such as Piazza and EdSTEM. Since student questions on these forums are often closely tied to specific aspects of the institution, instructor, and course delivery, general-purpose LLMs do not directly do well on this task. We introduce RetLLM-E, a method that combines text-retrieval and prompting approaches to enable LLMs to provide precise and high-quality answers to student questions. When presented with a student question, our system initiates a two-step process. First, it retrieves relevant context from (i) a dataset of student questions addressed by course instructors (Q&A Retrieval) and (ii) relevant segments of course materials (Document Retrieval). RetLLM-E then prompts LLM using the retrieved text and an engineered prompt structure to yield an answer optimized for the student question. We present a set of quantitative and human evaluation experiments, comparing our method to ground truth answers to questions in a test set of actual student questions. Our results demonstrate that our approach provides higher-quality responses to course-related questions than an LLM operating without context or relying solely on retrieval-based context. RetLLM-E can easily be adopted in different courses, providing instructors and students with context-aware automatic responses.

This work presents Adaptive Local-then-Global Merging (ALGM), a token reduction method for semantic segmentation networks that use plain Vision Transformers. ALGM merges tokens in two stages: (1) In the first network layer, it merges similar tokens within a small local window and (2) halfway through the network, it merges similar tokens across the entire image. This is motivated by an analysis in which we found that, in those situations, tokens with a high cosine similarity can likely be merged without a drop in segmentation quality. With extensive experiments across multiple datasets and network configurations, we show that ALGM not only significantly improves the throughput by up to 100%, but can also enhance the mean IoU by up to + 1.1, thereby achieving a better trade-off between segmentation quality and efficiency than existing methods. Moreover, our approach is adaptive during inference, meaning that the same model can be used for optimal efficiency or accuracy, depending on the application. Code is available at https://tue-mps.github.io/ALGM.

This Work-In-Progress Innovative Practice paper describes a new flexible, at-scale assignment extension policy and its implementation in large undergraduate computing class-rooms. While prior work has studied flexible deadlines and their effect on student learning, such policies are still under-utilized in post-secondary classrooms due to practical constraints on administrative workload, from managing hundreds to thousands of requests to reducing excessive grading overhead, especially in large post-secondary classrooms. The Flextensions tool-an automated flexible extension assignment software-promotes equitable learning opportunities in higher education by providing sufficient accommodations to each student's unique learning needs and life circumstances. Part of Flextensions is a scalable software tool implementation that facilitates instructor management of extension requests across thousands of students and a variety of course policies. We present Flextensions and describe initial experiences with adapting the tool to computer science and data science undergraduate courses at an R1 institution in the United States. This work shares the open-source software that enables and streamlines the management of extension requests across different course policies. Additionally, we provide an initial analysis of flexible extensions in three specific large-scale (500–1500 students) undergraduate computing courses. Overall, students tended not to take advantage of the policy but rather used it only when needed, with many citing extenuating circumstances-personal or otherwise. By analzing survey results, the policy was well-received, with positive impressions on well-being, learning outcomes, and overall academic experience. One student-reported benefit was that many felt valued as individuals in the classroom. Despite some students still reporting stigma towards requesting an extension, Flextensions has the promising ability to improve the quality and responsiveness of creating accommodations in large-scale classrooms.

This full research paper investigates the significance of a summer bridge program on the non-academic metrics of incoming freshman computing students at an R1 institution in the United States. Summer bridge programs are designed to increase academic major preparedness, reduce attrition, and provide other opportunities such as faculty networking and community building. Our team designed and delivered a summer bridge program to serve students from historically marginalized groups in Computer Science and Engineering. We offered the program to cohorts of 35–55 students annually from 2020 through 2023. Due to COVID-19, the program was offered online in 2020 and 2021 and then moved to an in-person version in 2022 and 2023. To measure the impact of the summer program, we utilized a closed-ended Likert-scale survey both before and after the program. The survey aimed to evaluate 10 principal non-academic factors that have been proven to influence student retention in the major. These factors include students' mathematics and programming self-concept, science motivation, science identity, help-seeking and concealment attitude, growth mindset, peer community, sense of belonging, and awareness of and accessibility to resources. Survey results were consolidated and used for quantitative analyses to address three research questions: RQ1: In what way, if at all, does participation in the program affect students' non-academic indicators? RQ2: What is the difference in impact between online (2020, 2021) and in-person (2022, 2023) program modalities? RQ3: How did participants' perceptions of the program differ across the two modalities?

This Work-In-Progress Research paper details a quantitative analysis of the impact of a summer bridge program for incoming Freshmen majoring in Computer Science, mainly focusing on post-program academic outcomes. We designed and delivered a summer bridge program to serve students who are members of a historically underserved group in Computer Science. The program is designed to help participants build a foundation in mathematics and programming while fostering an environment for community and confidence building. In our study, we utilize data from a Likert-scale survey administered to students before and after the program to compare attitudes towards non-academic factors such as programming and mathematics self-concept, science motivation, and sense of belonging, alongside post-program academic performance indicators such as performance in the first engineering mathematics course, performance in the first programming course, academic standing, and major declaration and retention outcomes. This data, collected over four cohorts of participants from 2020 to 2023, includes two cohorts participating online due to the COVID-19 pandemic (2020 and 2021) and two in-person cohorts (2022 and 2023). We aim to address three primary research Questions: (RQ1) To what extent does the overall post-program academic performance vary between online (2020, 2021) and in-person (2022, 2023) modalities? (RQ2) Can a potential change in attitudes towards non-academic indicators influence or explain a student's post-program academic indicators? (RQ3) Can a student's non-academic indicators and math/programming self-concept influence or explain a student's persistence in STEM?

This Work-In-Progress Research paper examines the measurable impacts of a flexible extension policy on course learning objectives in large-scale computing courses. In higher education, flexible extension policies have become increasingly common, where students can individually request additional time on assignments to accommodate their unique learning needs and life circumstances. This paper analyzes the effects of extensions on cultivating student learning and academic performance using flexible extension data from an undergraduate Data Science course at a U.S. Rl institution. We study how extension policies impact different groups of students based on their usage. Using the policy, students are achieving high rates of assignment submissions. While prior experience has no bearing on how students use the extension policy, students who use the policy tend to have slightly lower final exam scores. This early work aims to inform educators about the efficacy of flexible extensions and how they impact student learning and academic outcomes. Ultimately, our goal is to contribute to the creation of a supportive learning environment where all students can succeed. In this paper, we intend to answer the following research questions: 1) How does a flexible extension policy improve students' learning experience in the course? 2) How does student extension usage across a course term correlate with academic learning goals?

This Research Full paper focuses on the challenges in artificial intelligence, machine learning, and data science education—referred to as “artificial intelligence” courses here-after-often characterized by extensive prerequisites that limit student access. We analyze the course structures and prerequisites of these courses in computing departments at 50 Research-1 institutions in the United States, recognized for their “Very High Research Activity.” Our methodology involves analyzing course syllabi to examine the structure and prerequisites of these courses, using open coding to develop a unified codebook to identify prerequisites and determine the earliest exposure levels for students. A clustering analysis was also conducted to identify common and differing curriculum approaches among institutions. Results show that data science courses require less initial exposure, while artificial intelligence and machine learning courses require more prerequisites. Standard requirements for artificial intelligence courses include basic data structure (Computer Science 2) and algorithms, with machine learning courses requiring more mathematics preparation. Moreover, public institutions offer advanced courses with more prerequisites compared to private institutions. Overall, this study recognizes considerable diversity in curricular frameworks across Research-1 institutions and encourages institutions to revise curricula to broaden access to artificial intelligence education and increase participation in research.

Generative AI (GenAI) has seen great advancements in the past two years and the conversation around adoption is increasing. Widely available GenAI tools are disrupting classroom practices as they can write and explain code with minimal student prompting. While most acknowledge that there is no way to stop students from using such tools, a consensus has yet to form on how students should use them if they choose to do so. At the same time, researchers have begun to introduce new pedagogical tools that integrate GenAI into computing curricula. These new tools offer students personalized help or attempt to teach prompting skills without undercutting code comprehension. This working group aims to detail the current landscape of education-focused GenAI tools and teaching approaches, present gaps where new tools or approaches could appear, identify good practice-examples, and provide a guide for instructors to utilize GenAI as they continue to adapt to this new era.

The panel comprises a diverse set of Computing educators working on AI in education. The panelists will address four areas of AI in Computing education: 1) AI for introductory CS classrooms, 2) Investigating opportunities presented by LLMs, 3) LLM-based tool development, and 4) Ethics and inclusion in AI curriculum. The panel will share experiences and discuss opportunities and challenges in AI education with the community.

Table transformations are a critical skill to master in order to fluently work with data. In introductory data science courses, however, students have found these transformations particularly challenging to learn. One complex transformation is the pivot transformation, which reorganizes a table based on aggregation and summarizing along selected columns and rows. Current assessments test student understanding in static scenarios. Thus, there is an opportunity to help students explicitly work through the steps and variables needed to express a pivot transformation in a randomizable manner. As such, we explore whether a dynamic digital assessment for the pivot transformation can effectively achieve mastery learning towards this skill. Our design is inspired by Parsons problems, in which answer components (pivot table output labels and values) can be composed into the output of a pivot transformation. A question can be derived from a small randomized dataset, and randomized Pandas code that operates upon the dataset and can be autograded. We plan to conduct a pilot study with data science students to investigate whether 1) using a programmable online platform to practice pivot tables helps improve performance on exams, and 2) randomization and instant feedback on the online platform contribute to improved student learning.

Identifying misconceptions in student programming solutions is an important step in evaluating their comprehension of fundamental programming concepts. While misconceptions are latent constructs that are hard to evaluate directly from student programs, logical errors can signal their existence in students' understanding. Tracing multiple occurrences of related logical bugs over different problems can provide strong evidence of students' misconceptions. This study presents preliminary results of utilizing an interpretable state-of-the-art Abstract Syntax Tree-based embedding neural network to identify logical mistakes in student code. In this study, we show a proof-of-concept of the errors identified in student programs by classifying correct versus incorrect programs. Our preliminary results show that our framework is able to automatically identify misconceptions without designing and applying a detailed rubric. This approach shows promise for improving the quality of instruction in introductory programming courses by providing educators with a powerful tool that offers personalized feedback while enabling accurate modeling of student misconceptions.

Today's rapidly evolving educational landscape prioritizes active student engagement. Classrooms at scale face particular challenges in fostering meaningful interactions between students and course content. In this study, we introduce EIT (Earnest Insight Toolkit), a tool designed to assess students' engagement within interactive lecture participation exercises-particularly in the context of large-scale hybrid classrooms. We use EIT to conduct a comprehensive assessment of student responses to interactive lecture poll questions. Our objective with EIT is to equip educators with valuable means of identifying at-risk students for enhancing intervention and support strategies and measuring student engagement with course content.

Recent advancements in instruction-tuned large language models offer new potential for enhancing students' experiences in large-scale classes. Deploying LLMs as student-facing assistants, however, presents challenges. Key issues include integrating class-specific content into responses and applying effective pedagogical techniques. This study addresses these challenges through retrieval and prompting techniques, focusing on mitigating hallucinations in LLM-generated responses, a crucial concern in education. Furthermore, practical deployment brings further challenges related to student data privacy and computational constraints. This research strives to enhance the quality and relevance of LLM responses while addressing practical deployment issues, with an emphasis on creating a versatile system for diverse domains and teaching styles.

This study utilizes large language models to extract foundational programming concepts in programming assignments in a CS1 course. We seek to answer the following research questions: RQ1. How effectively can large language models identify knowledge components in a CS1 course from programming assignments? RQ2. Can large language models be used to extract program-level knowledge components, and how can the information be used to identify students' misconceptions? Preliminary results demonstrated a high similarity between course-level knowledge components retrieved from a large language model and that of an expert-generated list.

In this work, we present the development of an automated extension tool that supports flexible extension policies. Students interact with a wide range of extension policies in similar ways; in particular, some students repeatedly request multi-day long extensions. When scaled to courses with hundreds or potentially thousands of students, course staff time is the limiting resource preventing adequate student support. We present a tool to help automate a range of extension processes. The use of this tool should reduce staff load while increasing individualized student support, through email communication and consequent recovery of student agency. Our early research questions are: Does the extension tool reduce barriers and stigma around asking for assistance? Does the tool lessen the wait time between requesting and receiving an extension, and how does the tool improve students' learning experience in the course? These questions will help inform us about how an automated tool for flexible extensions helps support growing course sizes and students who may not otherwise receive the support they need for their success and well-being in the course.

Equitable grading practices and flexible deadline policies have previously demonstrated positive student learning and well-being outcomes. In this poster, we contribute a framework for flexible extension policies that emphasize equitable grading. We then analyze extension requests and grades obtained by students in a Data Science course with a flexible extension policy. We present two research questions based on this data. RQ1: How does the length of an extension relate to student performance on the corresponding assignment? RQ2: How does student extension usage across the semester relate to students' learning of the content?

The Model AI Assignments session seeks to gather and disseminate the best assignment designs of the Artificial Intelligence (AI) Education community. Recognizing that assignments form the core of student learning experience, we here present abstracts of six AI assignments from the 2023 session that are easily adoptable, playfully engaging, and flexible for a variety of instructor needs. Assignment specifications and supporting resources may be found at http://modelai.gettysburg.edu .

Learning exercises that activate students’ additional cognitive understanding of course concepts facilitate contextualizing the content knowledge and developing higher-order thinking and problem-solving skills. Student-generated instructional materials such as course summaries and problem sets are amongst the instructional strategies that reflect active learning and constructivist philosophy. The contributions of this work are twofold: 1) We introduce a practical implementation of inside-outside learning strategy in an undergraduate deep learning course and will share our experiences in incorporating student-generated instructional materials learning strategy in course design, and 2) We develop a context-aware deep learning framework to draw insights from the student-generated materials for (i) Detecting anomalies in group activities and (ii) Predicting the median quiz performance of students in each group. This work opens up an avenue for effectively implementing a constructivism learning strategy in large-scale and online courses to build a sense of community between learners while providing an automated tool for instructors to identify at-risk groups.

In this paper, we study three focus areas: investigating the identity-related sense of belonging in a gateway computer science course, examining the dynamics of the sense of belonging between the beginning and the end of the course, and offering actions to improve the sense of belonging that addresses the needs of students from intersecting identity groups.

In this poster, we investigate the impact of online versus in-person summer equity programs. We designed and implemented a summer bridge program from 2020 through 2022 and collected pre- and post-survey data from students to measure the program's impact on students' self-concept, science motivation, growth mindset, help-seeking and concealment attitude, science identity, and sense of belonging.

Deep learning has been shown to be able to support the analysis of imaging and signal based techniques such as electroencephalograms (EEG). In this work we apply deep learning methods in order to classify individuals with intellectual developmental disorder (IDD) using EEG signals. The dataset used contains 7 IDD subjects and 7 control subjects. EEG recordings were taken using a dry electrode system while subjects were in a resting state and while listening to music. Scalograms of the recordings were then generated using a Morlet wavelet. These scalograms were fed into a deep learning model for classification. This project compared the performance of using several fully connected and convolutional neural network (CNN) models. The highest accuracy achieved was 98.9% and 99.8% while subjects were at rest and listening to music, respectively.

In this Research to Practice Full Paper, we intend to share our experience in design, implementation, and evaluation of a summer bridge program for a group of first-year first-generation and underrepresented students entering engineering majors at the University of California Santa Cruz. Our program is proven to prepare students for engineering programming and mathematics courses, develop a growth mindset, build self-efficacy, and bond with the cohort to build peer-mentoring opportunities and increase a sense of belonging to the campus community and our school of engineering. We describe 1) program design and goals, 2) program schedule and curriculum (mathematics and programming), 3) description of research-based program evaluation, and 4) outline of research findings (both quantitatively and qualitatively).

The Model AI Assignments session seeks to gather and disseminate the best assignment designs of the Artificial Intelligence (AI) Education community. Recognizing that assignments form the core of student learning experience, we here present abstracts of nine AI assignments from the 2020 session that are easily adoptable, playfully engaging, and flexible for a variety of instructor needs. Assignment specifications and supporting resources may be found at http://modelai.gettysburg.edu.

This paper describes an experience in teaching Machine Learning (ML) and Natural Language Processing (NLP) to a group of high school students over an intense one-month period. In this work, we provide an outline of an AI course curriculum we designed for high school students and then evaluate its effectiveness by analyzing student's feedback and student outcomes. After closely observing students, evaluating their responses to our surveys, and analyzing their contribution to the course project, we identified some possible impediments in teaching AI to high school students and propose some measures to avoid them. These measures include employing a combination of objectivist and constructivist pedagogies, reviewing/introducing basic programming concepts at the beginning of the course, and addressing gender discrepancies throughout the course.

We sought to explore the problem of teaching a reinforcement learning agent how to play Texas Hold `Em (THE), a popular poker game played with a standard 52-card deck. This is an interesting problem because THE, and poker in general, is an incomplete information game in which the best strategy must take into account a significant amount of uncertainty, and for which the input vector of relevant information could be potentially very large. The final product of our research is a simplistic but elegant application of reinforcement learning, with various approaches yielding promising results within the context of THE.

In this WIP Research to Practice paper, we explored the impact of integrating the university's learning management system and an automatic grading system in delivering a large-scale introduction to programming course in 2 consecutive quarters. Our initial approach utilizes an on-demand standalone automatic grading system and a separate assignment submission portal on Canvas. After evaluating our performance and specific student feedback, we integrated the assignment submission portal with the autograder system to provide a real-time objective assessment of assignments.The main improvement after enforcing assignment submission through the autograder (Stepik) was the noticeable improvement in the class average of assignment scores by 20.5% even though most of the test cases were hidden. Another interesting observation was the effect of our approach in decreasing the DFW rate to 12.5% from 46% and a considerable increase in the passing rate of female students, by 22%. We also noticed that in the second iteration of the course students who took the course as an elective were able to perform comparably and even better than students who took it as a requirement. It is also worth mentioning that using autograder helped students increase their code quality.

In this paper, we experimentally explore the impact of different teaching paradigms on teaching a large-scale iOS programming MOOC consisting of 30,162 students. Our initial approach utilizes methods from our in-person lecturing experience. After launching the initial version of the course, we analyzed our performance and student feedback based on which we recreated and re-launched the entire course with a particular focus on clarity, video-quality, and packaging the topics in 1-2 minute micro-modules. Based on feedback from 650 students, we observed that the overall lecture positive feedback increased from 65% before our course adjustment to 83% after the adjustment. In this paper we will provide a detailed overview of the lessons learned and the impact of our new teaching methods on each section of the course. We also noticed a slight increase in course completion rates, from 5.4% before the adjustment to 5.8% after the adjustment.