Motivation: Database courses focus on SQL querying (DQL) while treating schema definition (DDL) and data manipulation (DML) as side topics, even though real-world database work begins with understanding schema design and data updates. This misalignment leaves students underprepared for authentic data management practice. Method: We integrated scaffolded DDL and DML exercises as a core concept in a third-year data course across three offerings (2023-2025). Students completed structured weekly tasks in an LMS that provides immediate feedback and unlimited attempts, encouraging low-stakes, iterative practice. We analyzed student interaction data (number of attempts and performance) to examine learning patterns across DDL/DML and DQL. We analyzed 9,071 total exercise submissions from 669 students, examining both the number of LMS exercise attempts and assignment performance across DDL/DML and DQL. Results: Students required fewer attempts on DDL/DML tasks than on traditional DQL tasks, indicating strong receptiveness when these topics were properly scaffolded. Early performance on schema-definition tasks was moderately correlated with later SQL performance, suggesting that schema competence supports subsequent query learning. Implications: We encourage database educators to teach schema design and data manipulation as core topics to strengthen students' conceptual foundations, as our results suggest these skills are learnable with scaffolding and may support subsequent query learning.

Adaptive Experimentation is one of the most promising approaches to support complex decision-making in learning experience design and delivery. This paper reports on our experience with a real-world, multi-experimental evaluation of an adaptive experimentation platform within the XPRIZE Digital Learning Challenge framework, and summarizes data-driven lessons learned and best practices for Adaptive Experimentation in education. We outline key scenarios of the applicability of platform-supported experiments and reflect on lessons learned from this two-year project, focusing on implications relevant to platform developers, researchers, practitioners, and policy stakeholders to integrate Adaptive Experiments in real-world courses.

Digital mental health (DMH) interventions, such as text-message-based lessons and activities, offer immense potential for accessible mental health support. While these interventions can be effective, real-world experimental testing can further enhance their design and impact. Adaptive experimentation, utilizing algorithms like Thompson Sampling for (contextual) multi-armed bandit (MAB) problems, can lead to continuous improvement and personalization. However, it remains unclear when these algorithms can simultaneously increase user experience rewards and facilitate appropriate data collection for social-behavioral scientists to analyze with sufficient statistical confidence. Although a growing body of research addresses the practical and statistical aspects of MAB and other adaptive algorithms, further exploration is needed to assess their impact across diverse real-world contexts. This paper presents a software system developed over two years that allows text-messaging intervention components to be adapted using bandit and other algorithms while collecting data for side-by-side comparison with traditional uniform random non-adaptive experiments. We evaluate the system by deploying a text-message-based DMH intervention to 1100 users, recruited through a large mental health non-profit organization, and share the path forward for deploying this system at scale. This system not only enables applications in mental health but could also serve as a model testbed for adaptive experimentation algorithms in other domains.

Exploring alternative ideas by rewriting text is integral to the writing process. State-of-the-art Large Language Models (LLMs) can simplify writing variation generation. However, current interfaces pose challenges for simultaneous consideration of multiple variations: creating new variations without overwriting text can be difficult, and pasting them sequentially can clutter documents, increasing workload and disrupting writers’ flow. To tackle this, we present ABScribe, an interface that supports rapid, yet visually structured, exploration and organization of writing variations in human-AI co-writing tasks. With ABScribe, users can swiftly modify variations using LLM prompts, which are auto-converted into reusable buttons. Variations are stored adjacently within text fields for rapid in-place comparisons using mouse-over interactions on a popup toolbar. Our user study with 12 writers shows that ABScribe significantly reduces task workload (d = 1.20, p < 0.001), enhances user perceptions of the revision process (d = 2.41, p < 0.001) compared to a popular baseline workflow, and provides insights into how writers explore variations using LLMs.

Self-reflection on learning experiences constitutes a fundamental cognitive process, essential for consolidating knowledge and enhancing learning efficacy. However, traditional methods to facilitate reflection often face challenges in personalization, immediacy of feedback, engagement, and scalability. Integration of Large Language Models (LLMs) into the reflection process could mitigate these limitations. In this paper, we conducted two randomized field experiments in undergraduate computer science courses to investigate the potential of LLMs to help students engage in post-lesson reflection. In the first experiment (N=145), students completed a take-home assignment with the support of an LLM assistant; half of these students were then provided access to an LLM designed to facilitate self-reflection. The results indicated that the students assigned to LLM-guided reflection reported somewhat increased self-confidence compared to peers in a no-reflection control and a non-significant trend towards higher scores on a later assessment. Thematic analysis of students' interactions with the LLM showed that the LLM often affirmed the student's understanding, expanded on the student's reflection, and prompted additional reflection; these behaviors suggest ways LLM-interaction might facilitate reflection. In the second experiment (N=112), we evaluated the impact of LLM-guided self-reflection against other scalable reflection methods, such as questionnaire-based activities and review of key lecture slides, after assignment. Our findings suggest that the students in the questionnaire and LLM-based reflection groups performed equally well and better than those who were only exposed to lecture slides, according to their scores on a proctored exam two weeks later on the same subject matter. These results underscore the utility of LLM-guided reflection and questionnaire-based activities in improving learning outcomes. Our work highlights that focusing solely on the accuracy of LLMs can overlook their potential to enhance metacognitive skills through practices such as self-reflection. We discuss the implications of our research for the learning-at-scale community, highlighting the potential of LLMs to enhance learning experiences through personalized, engaging, and scalable reflection practices.

Personalized chatbot-based teaching assistants can be crucial in addressing increasing classroom sizes, especially where direct teacher presence is limited. Large language models (LLMs) offer a promising avenue, with increasing research exploring their educational utility. However, the challenge lies not only in establishing the efficacy of LLMs but also in discerning the nuances of interaction between learners and these models, which impact learners' engagement and results. We conducted a formative study in an undergraduate computer science classroom (N=145) and a controlled experiment on Prolific (N=356) to explore the impact of four pedagogically informed guidance strategies on the learners' performance, confidence and trust in LLMs. Direct LLM answers marginally improved performance, while refining student solutions fostered trust. Structured guidance reduced random queries as well as instances of students copy-pasting assignment questions to the LLM. Our work highlights the role that teachers can play in shaping LLM-supported learning environments.

In this lightning talk, I reflect on the ongoing rise of software infrastructures for educational experiments and their role in potential shifts in Computer Science Education research. I outline some emerging challenges and opportunities for scientific discovery and educational practice. In particular, I discuss overlapping interventions, experimental design optimization, and knowledge accumulation from multi-cohort multi-site studies.

Drawing inspiration from machine learning and experimentation in product development at leading technology companies, we explore how adaptive experimentation might help in continuous course improvement. In adaptive experiments, as different arms/conditions are deployed to students, data is analyzed and used to change the experience for future students. We discuss an example side-by-side comparison of traditional and adaptive experimentation of self-explanation prompts in online homework problems in a CS1 course. This provides the first step in exploring the future of how this approach can help bridge research and practice in continuous course improvement.

Introducing students to different perspectives and roles in the development process allows them to engage in the work of cross-disciplinary diverse teams and even can enable them to change roles in designer-developer interactions. Industry work often places recent graduates in preexisting polarized relationship dynamics between different participants in the design and development process. This paper describes a two-stage attempt at co-alignment of software engineering and user-centered design courses: from full alignment with topic intersections and joint project to partial alignment through separate activities. We discuss challenges of both ways including time or technical constraints, increased effort from the program developers and instructors, students' and instructors' frustrations. We finalize by describing benefits of providing students with early experience identifying trade-offs between design requirements and architecture and opportunities for diverse group with different background in computer science.

Digital experiments can be used in CSedu to test hypotheses about interventions and conditions' efficacy (or inefficacy). This workshop will discuss and deconstruct the design process and analysis for various experiments conducted in CS1. E.g., experiments testing which explanations students find helpful, which emails get them to start homework early, or which webpages effectively encourage and motivate students. This workshop teaches participants how to conduct, interpret, and analyze adaptive field experiments. These adaptive experiments employ machine learning algorithms to analyze experiments during deployment and dynamically shift the allocation of arms/conditions to give future students better conditions more rapidly. Adaptive field experiments can accelerate scientific discovery by enabling more complex experimental designs and increasing statistical power by phasing conditions in and out more efficiently. The workshop is supported by a 5-year NSF grant to build software tools and a digital community, gathering instructors, domain scientists and methodologists to teach them how to run adaptive experiments. The methodological focus includes understanding: (1) which algorithms are best for adaptive experiments that meet domain scientists' needs in specific experimental designs and data sets; (2) which hypothesis tests and Bayesian analyses to choose. Software companies use these innovative methodologies extensively to continuously improve product design. This workshop demonstrates how the same methods can be used in CSedu to improve research rigor and accelerate educational research implementation, ultimately improving student outcomes.

The introductory agent-based modeling course we are presenting aims to equip second-year sociology students with basic skills which are critical to transforming their research ideas into theories and computational models. Our course follows a general course on theory construction in social science. As one of the first interactions of undergraduate social science students with computer science concepts, the course partially serves as a CS0 course; however, the main focus is on the higher-level skills needed to model social phenomena. This requires the course to maintain multiple foci on computer science and domain skills. In this paper, we present a course design considerations and establish a foundation for the comparison of agent-based models and the computational thinking and CS0 skills required for undergraduate social scientists studying agent-based modeling.

In this work-in-progress report, we outline the first results from our study of collaboration networks in a two-year Data Science minor for non-STEM students at the Higher School of Economics, St. Petersburg, Russia.

Cosmetic items do not provide functional advantages in games, but, nevertheless, they play an important role in the overall player experience. Possessing predominantly socially-constructed dimensions of value, cosmetic items are chosen, discussed, assessed, and valuated in an ongoing iterative collaborative process by communities of players. In our study, we explore the case of Dota 2 and apply Topic Modeling to community-discussions data gathered from Reddit.com. We describe social experiences related to the valuation of cosmetic items in interaction and collision of various logics, including artificial scarcity, decomposition of visual effects, and connectedness to the game lore. Our findings connect the collective experience of players in the game and on online community platforms, suggesting that non-utility-based social value construction becomes an important part of game experience.