Learning to communicate with code-generating AI models is an emerging skill for novice programmers. One recent pedagogical approach, Prompt Problems, has students solve computational tasks by writing natural-language prompts for code-generating AI models. However, little is known about the specific prompt-level mistakes novice programmers make, the kinds of computational details they fail to communicate, and what strategies they use to recover when generated code is incorrect. In a CS1 course, we studied attempts by more than 900 students to solve dialogue-based Prompt Problems. We analyzed student reflections, unsuccessful prompts, and reported debugging strategies. Compared to traditional coding tasks, students generally found prompting easier, more enjoyable, and better targeted at developing problem-solving skills. The most common mistakes are related to the omission of key details, suggesting both a failure to acknowledge their importance and over-reliance on AI to infer them. When prompts failed, students focused more on clarifying their intent and reflecting on the provided problem details than on tracing generated code or examining test cases.

Modern programming tools use generative AI (GenAI) to suggest code to the user as they type, interrupting their problem-solving behavior and undermining the development of their programming critical thinking skills. In this paper, we present a scaffolded programming exercise designed to support student differentiation between good and bad GenAI code suggestions based on negative expertise–that identifying why an answer is wrong is part of developing conceptual knowledge. We compare a version of the tool that showed one suggestion (correct or not), to a version that showed three suggestions (one of which was correct). We present results on performance and error rates as well as qualitative findings centered on Pintrich and DeGroot’s theory of self-regulation. Students reported that the single suggestion version better aligned with industry tools and presented a lower cognitive load. Students also reported that the multiple suggestion version caused them to slow down and think critically about the line under consideration, the overall purpose of the code, and the benefits of planning.

AI code completion tools, such as Github Copilot, provide students with code suggestions to help them write programs. However, recent qualitative studies suggest that students fail to critically evaluate these suggestions. We present Clover, a code completion tool that logs students' interactions with code suggestions and additionally offers attention checks to probe reflective engagement during programming tasks. We also develop a taxonomy of behavioral interaction metrics for AI-assisted programming, informed by literature. We analyzed relationships between interaction patterns, engagement with attention checks, and task performance. We observed that higher rates of tab accept were associated with lower attention check performance, while increased dwell time was associated with higher attention check performance. We conclude by discussing how programming process data and attention checks might support reflective engagement in AI-assisted programming.

Video content, including lectures and tutorials on platforms like YouTube, has become an invaluable source of informal support for computing students. However, with the emergence of short-form videos on TikTok as a possible new resource, we conducted a content analysis of 300 TikTok videos about computing careers to investigate how creators present career-related content. Our analysis found that videos offered valuable insights about daily job experiences, educational resources, and advice about securing internships. However, the content often lacked concrete actionable guidance. Videos often reinforce stereotypes about computing careers, emphasizing high salaries and remote work over other benefits, such as meaningful and impactful work. Creators also performed expertise and identity by mentioning their credentials, showing their face, and speaking directly to the camera. These findings show how biases in career content, reinforced by creators' trust-building performances, may potentially influence students' expectations and values around computing careers.

Large Language Models (LLMs) have shown the potential to generate code explanations that surpass those of peers in quality, offering promising opportunities for computer science education. Inspired by this, we explore whether combining multiple diverse explanations, each emphasizing distinct aspects (e.g., function, concept, goal), can enhance students' understanding of programming exercises compared to generic explanations that do not emphasize distinct conceptual aspects. Insights from other fields, such as computational creativity, suggest that diverse ideas may be more beneficial than relying solely on a single, high-quality option. Variation Theory holds that learners grasp a concept when they see systematic variation that exposes its critical features, helping them distinguish it from related ideas. In creative domains, uniform or homogeneous exemplars can lead to design fixation, whereas varied inputs support more flexible reasoning. In a study with 971 first-year computing students, participants were randomly assigned either diverse or generic LLM-generated explanations for two programming exercises. Students completed multiple-choice (MCQ) and open-ended (OE) questions for each exercise to assess understanding, followed by Likert-scale questions and OE reflections to understand preferences and perception. Across participants, performance was consistently 7.7% higher when students received diverse explanations, and there was no difference in perceived cognitive load. Performance on the closed-form multiple-choice questions was similar for diverse and generic explanations.

Figure 1: The three screens that comprise the user interface of our prototype.(1) VSD programmers choose to upload or capture an image to use in a VSD.(2) The application automatically generates a set of potential hotspots for use in the VSD and the programmer can choose to edit, use, or delete these hotspots.They can also manually add their own hotspots.Once all of the hotspots are created, they can use the canvas to draw the hotspots on the image.(3) Once they have finished configuring the VSD, users can see a preview of what the VSD looks like and interact with the created hotspots.

Introductory programming courses often rely on small code-writing exercises that have clearly specified problem statements. This limits opportunities for students to practice how to clarify ambiguous requirements -- a critical skill in real-world programming. In addition, the emerging capabilities of large language models (LLMs) to produce code from well-defined specifications may harm student engagement with traditional programming exercises. This study explores the use of ``Probeable Problems'', automatically gradable tasks that have deliberately vague or incomplete specifications. Such problems require students to submit test inputs, or `probes', to clarify requirements before implementation. Through analysis of over 40,000 probes in an introductory course, we identify patterns linking probing behaviors to task success. Systematic strategies, such as thoroughly exploring expected behavior before coding, resulted in fewer incorrect code submissions and correlated with course success. Feedback from nearly 1,000 participants highlighted the challenges and real-world relevance of these tasks, as well as benefits to critical thinking and metacognitive skills. Probeable Problems are easy to set up and deploy at scale, and help students recognize and resolve uncertainties in programming problems.

The widespread adoption of generative AI is already impacting learning and help-seeking. While the benefits of generative AI are well-understood, recent studies have also raised concerns about increased potential for cheating and negative impacts on students' metacognition and critical thinking. However, the potential impacts on social interactions, peer learning, and classroom dynamics are not yet well understood. To investigate these aspects, we conducted 17 semi-structured interviews with undergraduate computing students across seven R1 universities in North America. Our findings suggest that help-seeking requests are now often mediated by generative AI. For example, students often redirected questions from their peers to generative AI instead of providing assistance themselves, undermining peer interaction. Students also reported feeling increasingly isolated and demotivated as the social support systems they rely on begin to break down. These findings are concerning given the important role that social interactions play in students' learning and sense of belonging.

Collaboration is a crucial part of computing education. The increase in AI capabilities over the last couple of years is bound to profoundly affect all aspects of systems and software engineering, including collaboration. In this position paper, we consider a scenario where AI agents would be able to take on any role in collaborative processes in computing education. We outline these roles, the activities and group dynamics that software development currently include, and discuss if and in what way AI could facilitate these roles and activities. The goal of our work is to envision and critically examine potential futures. We present scenarios suggesting how AI can be integrated into existing collaborations. These are contrasted by design fictions that help demonstrate the new possibilities and challenges for computing education in the AI era.

Publisher Copyright: © 2025 Copyright held by the owner/author(s).

Generative AI (GenAI) presents societal and ethical challenges related to equity, academic integrity, bias, and data provenance. This working group will consider the ethical and societal impacts of GenAI in higher computing education. In this paper, we outline the goals, methodology and expected deliverables of the working group. In particular, we will carry out a systematic literature review to address a wide set of issues and topics covering the rapidly emerging technology of GenAI from the perspective of its ethical and social impacts, we will provide an evaluation of university policies on the adoption and guidelines for use of GenAI for computing education and develop a framework to outline the ethical and societal impacts of GenAI in computing education. This work synthesizes existing research and considers the implications for educational and professional codes of ethics.

Ensuring equitable access to computing education for all students---including those with autism, dyslexia, or ADHD---is essential to developing a diverse and inclusive workforce. To understand the state of disability research in computing education, we conducted a systematic literature review of research on neurodiversity in computing education. Our search resulted in 1,943 total papers, which we filtered to 14 papers based on our inclusion criteria. Our mixed-methods approach analyzed research methods, participants, contribution types, and findings. The three main contribution types included empirical contributions based on user studies (57.1%), opinion contributions and position papers (50%), and survey contributions (21.4%). Interviews were the most common methodology (75% of empirical contributions). There were often inconsistencies in how research methods were described (e.g., number of participants and interview and survey materials). Our work shows that research on neurodivergence in computing education is still very preliminary. Most papers provided curricular recommendations that lacked empirical evidence to support those recommendations. Three areas of future work include investigating the impacts of active learning, increasing awareness and knowledge about neurodiverse students' experiences, and engaging neurodivergent students in the design of pedagogical materials and computing education research.

Many barriers exist when new members join a research community, including impostor syndrome. These barriers can be especially challenging for undergraduate students who are new to research. In our work, we explore how the use of social computing tools in the form of spontaneous online social networks (SOSNs) can be used in small research communities to improve sense of belonging, peripheral awareness, and feelings of togetherness within an existing CS research community. Inspired by SOSNs such as BeReal, we integrated a Wizard-of-Oz photo sharing bot into a computing research lab to foster community building among members. Through a small sample of lab members (N = 17) over the course of 2 weeks, we observed an increase in participants' sense of togetherness based on pre and post-study surveys. Our surveys and semi-structured interviews revealed that this approach has the potential to increase awareness of peers' personal lives, increase feelings of community, and reduce feelings of disconnectedness.

Help-seeking is a critical aspect of learning and problem-solving for computing students. Recent research has shown that many students are aware of generative AI (GenAI) tools; however, there are gaps in the extent and effectiveness of how students use them. With over two years of widespread GenAI usage, it is crucial to understand whether students' help-seeking behaviors with these tools have evolved and how. This paper presents findings from a repeated cross-sectional survey conducted among computing students across North American universities ( n=95 ). Our results indicate shifts in GenAI usage patterns. In 2023, 34.1% of students ( n=47 ) reported never using ChatGPT for help, ranking it fourth after online searches, peer support, and class forums. By 2024, this figure dropped sharply to 6.3% ( n=48 ), with ChatGPT nearly matching online search as the most commonly used help resource. Despite this growing prevalence, there has been a decline in students' hourly and daily usage of GenAI tools, which may be attributed to a common tendency to underestimate usage frequency. These findings offer new insights into the evolving role of GenAI in computing education, highlighting its increasing acceptance and solidifying its position as a key help resource.

Code reading and comprehension skills are essential for novices learning programming, and explain-in-plain-English tasks (EiPE) are a well-established approach for assessing these skills. However, manual grading of EiPE tasks is time-consuming and this has limited their use in practice. To address this, we explore an approach where students explain code samples to a large language model (LLM) which generates code based on their explanations. This generated code is then evaluated using test suites, and shown to students along with the test results. We are interested in understanding how automated formative feedback from an LLM guides students' subsequent prompts towards solving EiPE tasks. We analyzed 177 unique attempts on four EiPE exercises from 21 students, looking at what kinds of mistakes they made and how they fixed them. We found that when students made mistakes, they identified and corrected them using either a combination of the LLM-generated code and test case results, or they switched from describing the purpose of the code to describing the sample code line-by-line until the LLM-generated code exactly matched the obfuscated sample code. Our findings suggest both optimism and caution with the use of LLMs for unmonitored formative feedback. We identified false positive and negative cases, helpful variable naming, and clues of direct code recitation by students. For most students, this approach represents an efficient way to demonstrate and assess their code comprehension skills. However, we also found evidence of misconceptions being reinforced, suggesting the need for further work to identify and guide students more effectively.

While hackathons are often celebrated for their impact on participants, less attention has been given to the unique leadership development opportunities for the student organizers who create and run these events. Unlike traditional classroom settings, where leadership and collaboration skills are typically delayed until upper-level courses, hackathon organizers must tackle these challenges earlier on. In managing a large-scale, formative event, student organizers take on roles that require decision-making, teamwork, and project management. This poster explores the experiences of student organizers at Anonymous Hackathon, emphasizing how this informal learning opportunity complements gaps in the traditional computer science (CS) curriculum by fostering essential leadership and collaboration skills earlier in students' academic careers.

Autistic children face significant challenges in vocal communication and social interaction, often leading to social isolation. There is evidence that Augmentative and Alternative Communication (AAC) offers support to mitigate these challenges, enabling them to communicate with non-vocal means through forms of AAC, such as speech-generation devices (SGDs). However, the adoption and use of SGDs are hindered by several factors, including the large amount of practice required to learn to use SGDs and the limited options for highly engaging social learning contexts. Our study introduces the novel approach of using SGDs as game controller for digital and interactive games. With three design goals guiding our work, we conducted a Wizard-of-Oz formative case study with five participants aged 3-5 years, who were learning to use their SGD. We simulated a digital coloring game, integrating the speech-generated output of the participant's SGD to function as the game's controller. From this case study, we observed that all participants engaged with the game using their SGD for at least one turn, and two participants also engaged in emerging joint attention responses with the game and game's facilitator. This paper discusses these findings and contributes directions for future research, with suggestions for the design of future SGD-controlled games and exploration of social connection and collaboration between autistic children who use AAC and their caregivers, siblings, and peers.

Millions of people worldwide rely on alternative and augmentative communication devices to communicate. Visual scene displays (VSDs) can enhance communication for these individuals by embedding communication options within contextualized images. However, existing VSDs often present default images that may lack relevance or require manual configuration, placing a significant burden on communication partners. In this study, we assess the feasibility of leveraging large multimodal models (LMM), such as GPT-4V, to automatically create communication options for VSDs. Communication options were sourced from a LMM and speech-language pathologists (SLPs) and AAC researchers (N=13) for evaluation through an expert assessment conducted by the SLPs and AAC researchers. We present the study’s findings, supplemented by insights from semi-structured interviews (N=5) about SLP’s and AAC researchers’ opinions on the use of generative AI in augmentative and alternative communication devices. Our results indicate that the communication options generated by the LMM were contextually relevant and often resembled those created by humans. However, vital questions remain that must be addressed before LMMs can be confidently implemented in AAC devices.

Grid displays are the most common form of augmentative and alternative communication device recommended by speech-language pathologists for children. Grid displays present a large variety of vocabulary which can be beneficial for a users’ language development. However, the extensive navigation and cognitive overhead required of users of grid displays can negatively impact users’ ability to actively participate in social interactions, which is an important factor of their language development. We present a novel interaction technique for grid displays, Predictive Anchoring, based on user interaction theory and language development theory. Our design is informed by existing literature in AAC research, presented in the form of a set of design goals and a preliminary design sketch. Future work in user studies and interaction design are also discussed.

Web search engines have long served as indispensable tools for information retrieval; user behavior and query formulation strategies have been well studied. The introduction of search engines powered by large language models (LLMs) suggested more conversational search and new types of query strategies. In this paper, we compare traditional and LLM-based search for the task of image geolocation, i.e., determining the location where an image was captured. Our work examines user interactions, with a particular focus on query formulation strategies. In our study, 60 participants were assigned either traditional or LLM-based search engines as assistants for geolocation. Participants using traditional search more accurately predicted the location of the image compared to those using the LLM-based search. Distinct strategies emerged between users depending on the type of assistant. Participants using the LLM-based search issued longer, more natural language queries, but had shorter search sessions. When reformulating their search queries, traditional search participants tended to add more terms to their initial queries, whereas participants using the LLM-based search consistently rephrased their initial queries.

This work-in-progress research paper explores the effectiveness of tutorials in interdisciplinary learning environments, specifically focusing on bioinformatics. Tutorials are typically designed for a single audience, but our study aims to uncover how they function in contexts where learners have diverse backgrounds. With the rise of interdisciplinary learning, the importance of learning materials that accommodate diverse learner needs has become evident. We chose bioinformatics as our context because it involves at least two distinct user groups: those with computational backgrounds and those with biological backgrounds. The goal of our research is to better understand current bioinformatics software tutorial designs and assess them in the conceptual framework of interdisciplinarity. We conducted a content analysis of 22 representative bioinformatics software tutorials to identify design patterns and understand their strengths and limitations. We found common codes in the representative tutorials and synthesized them into ten themes. Our assessment shows degrees to which current bioinformatics software tutorials fulfill interdisciplinarity.

Novice programmers often struggle through programming problem solving due to a lack of metacognitive awareness and strategies. Previous research has shown that novices can encounter multiple metacognitive difficulties while programming, such as forming incorrect conceptual models of the problem or having a false sense of progress after testing their solution. Novices are typically unaware of how these difficulties are hindering their progress. Meanwhile, many novices are now programming with generative AI (GenAI), which can provide complete solutions to most introductory programming problems, code suggestions, hints for next steps when stuck, and explain cryptic error messages. Its impact on novice metacognition has only started to be explored. Here we replicate a previous study that examined novice programming problem solving behavior and extend it by incorporating GenAI tools. Through 21 lab sessions consisting of participant observation, interview, and eye tracking, we explore how novices are coding with GenAI tools. Although 20 of 21 students completed the assigned programming problem, our findings show an unfortunate divide in the use of GenAI tools between students who did and did not struggle. Some students who did not struggle were able to use GenAI to accelerate, creating code they already intended to make, and were able to ignore unhelpful or incorrect inline code suggestions. But for students who struggled, our findings indicate that previously known metacognitive difficulties persist, and that GenAI unfortunately can compound them and even introduce new metacognitive difficulties. Furthermore, struggling students often expressed cognitive dissonance about their problem solving ability, thought they performed better than they did, and finished with an illusion of competence. Based on our observations from both groups, we propose ways to scaffold the novice GenAI experience and make suggestions for future work.

Providing timely and personalized feedback to large numbers of students is a long-standing challenge in programming courses. Relying on human teaching assistants (TAs) has been extensively studied, revealing a number of potential shortcomings. These include inequitable access for students with low confidence when needing support, as well as situations where TAs provide direct solutions without helping students to develop their own problem-solving skills. With the advent of powerful large language models (LLMs), digital teaching assistants configured for programming contexts have emerged as an appealing and scalable way to provide instant, equitable, round-the-clock support. Although digital TAs can provide a variety of help for programming tasks, from high-level problem solving advice to direct solution generation, the effectiveness of such tools depends on their ability to promote meaningful learning experiences. If students find the guardrails implemented in digital TAs too constraining, or if other expectations are not met, they may seek assistance in ways that do not help them learn. Thus, it is essential to identify the features that students believe make digital teaching assistants valuable. We deployed an LLM-powered digital assistant in an introductory programming course and collected student feedback ($n=813$) on the characteristics of the tool they perceived to be most important. Our results highlight that students value such tools for their ability to provide instant, engaging support, particularly during peak times such as before assessment deadlines. They also expressed a strong preference for features that enable them to retain autonomy in their learning journey, such as scaffolding that helps to guide them through problem-solving steps rather than simply being shown direct solutions.

Grasping complex computing concepts often poses a challenge for students who struggle to anchor these new ideas to familiar experiences and understandings. To help with this, a good analogy can bridge the gap between unfamiliar concepts and familiar ones, providing an engaging way to aid understanding. However, creating effective educational analogies is difficult even for experienced instructors. We investigate to what extent large language models (LLMs), specifically ChatGPT, can provide access to personally relevant analogies on demand. Focusing on recursion, a challenging threshold concept, we conducted an investigation analyzing the analogies generated by more than 350 first-year computing students. They were provided with a code snippet and tasked to generate their own recursion-based analogies using ChatGPT, optionally including personally relevant topics in their prompts. We observed a great deal of diversity in the analogies produced with student-prescribed topics, in contrast to the otherwise generic analogies, highlighting the value of student creativity when working with LLMs. Not only did students enjoy the activity and report an improved understanding of recursion, but they described more easily remembering analogies that were personally and culturally relevant.

Introducing students to new concepts in computer science can often be challenging, as these concepts may differ significantly from their existing knowledge and conceptual understanding. To address this, we employed analogies to help students connect new concepts to familiar ideas. Specifically, we generated analogies using large language models (LLMs), namely ChatGPT, and used them to help students make the necessary connections. In this poster, we present the results of our survey, in which students were provided with two analogies relating to different computing concepts, and were asked to describe the extent to which they were accurate, interesting, and useful. This data was used to determine how effective LLM-generated analogies can be for teaching computer science concepts, as well as how responsive students are to this approach.

Generative AI (GenAI) has seen great advancements in the past two years and the conversation around adoption is increasing. Widely available GenAI tools are disrupting classroom practices as they can write and explain code with minimal student prompting. While most acknowledge that there is no way to stop students from using such tools, a consensus has yet to form on how students should use them if they choose to do so. At the same time, researchers have begun to introduce new pedagogical tools that integrate GenAI into computing curricula. These new tools offer students personalized help or attempt to teach prompting skills without undercutting code comprehension. This working group aims to detail the current landscape of education-focused GenAI tools and teaching approaches, present gaps where new tools or approaches could appear, identify good practice-examples, and provide a guide for instructors to utilize GenAI as they continue to adapt to this new era.

Annotated images are required for numerous computer vision tasks; however, the annotation process can be time-consuming for crowdworkers and experts. Previous work has investigated novel interaction techniques and task reformulation to speed up this process; however, there remains a gap in optimizing more complex annotation tasks, such as object matching. In this paper, we explore the impact of varying the amount of context provided to annotators. We hypothesize that reducing the context around the object being matched will improve speed without sacrificing the accuracy of the annotation task. To test this hypothesis, we developed a semi-automated annotation pipeline that pre-processes images to adjust the amount of context shown around an object of interest. We conducted two studies (n = 130, n = 10) to assess the effects of context quantitatively and qualitatively. We found that while the accuracy remained the same, the time spent on the task was significantly reduced when there was less context surrounding the object. However, our qualitative findings revealed multiple scenarios in which context served as a means of guiding the object matching task, and many others in which the distinctiveness of the object guided the matching task and additional context was not needed.

In the past year, large language models (LLMs) have taken the world by storm, demonstrating their potential as a transformative force in many domains including computing education. Computing education researchers have found that LLMs can solve most assessments in introductory programming courses, including both traditional code writing tasks and other popular tasks such as Parsons problems. As more and more students start to make use of LLMs, the question instructors might ask themselves is "what can I do?". We propose that one promising way forward is to integrate LLMs into teaching practice, providing all students with an equal opportunity to learn how to interact productively with LLMs as well as encounter and understand their limitations. In this workshop, we first present state-of-the-art research results on how to utilize LLMs in computing education practice, after which participants will take part in hands-on activities using LLMs. We end the workshop by brainstorming ideas with participants around adapting their classrooms to most effectively integrate LLMs while avoiding some common pitfalls.

In a previous Birds of a Feather discussion, we delved into the nascent applications of generative AI, contemplating its potential and speculating on future trajectories. Since then, the landscape has continued to evolve revealing the capabilities and limitations of these models. Despite this progress, the computing education research community still faces uncertainty around pivotal aspects such as (1) academic integrity and assessments, (2) curricular adaptations, (3) pedagogical strategies, and (4) the competencies students require to instill responsible use of these tools. The goal of this Birds of a Feather discussion is to unravel these pressing and persistent issues with computing educators and researchers, fostering a collaborative exploration of strategies to navigate the educational implications of advancing generative AI technologies. Aligned with this goal of building an inclusive learning community, our BoF is led by globally distributed leaders to facilitate multiple coordinated discussions that can lead to a broader conversation about the role of LLMs in CS education.

Cross-domain analogical reasoning is a core creative ability that can be challenging for humans. Recent work has shown some proofs-of-concept of Large language Models’ (LLMs) ability to generate cross-domain analogies. However, the reliability and potential usefulness of this capacity for augmenting human creative work has received little systematic exploration. In this paper, we systematically explore LLMs capacity to augment cross-domain analogical reasoning. Across three studies, we found: 1) LLM-generated cross-domain analogies were frequently judged as helpful in the context of a problem reformulation task (median 4 out of 5 helpfulness rating), and frequently (∼ 80% of cases) led to observable changes in problem formulations, and 2) there was an upper bound of ∼ 25% of outputs being rated as potentially harmful, with a majority due to potentially upsetting content, rather than biased or toxic content. These results demonstrate the potential utility — and risks — of LLMs for augmenting cross-domain analogical creativity.

Designers explore design spaces to iteratively define and refine their problem statements and solutions. Yet, designers often get fixated on specific problems, solutions, or trivial surface details without considering the bigger system and the intricate relationships within. Previous works demonstrated how suggesting relevant design examples can help designers break out of fixation. However, current systems recommend design ideas that are decontextualized from other relevant design information. As a result, designers need to manually maintain key relationships and context during the design process. DesignNet is a system to explicitly model the inter-relationships between design information with a knowledge graph. Future work will investigate whether and how a knowledge graph representation and associated graph algorithms can support computational creativity and guide design exploration.

Professional designers often construct and explore conceptual representations (e.g.: design spaces) to help them reason about complex design situations and consider potential design pitfalls. However, it is often challenging, even for professional designers, to exhaustively consider the many pitfalls that might result from design activity. We present CausalMapper, a mixed-initiative system, that leverages a large language model (LLM) and a causal map representation to teach design students how to reason about the relationships between problems and solutions. Where creativity support tools often focus on ideating creative solutions, our mixed-initiative approach focuses on ideating ecosystems of solutions that holistically address a set of related problems. By leveraging the generative creativity of LLMs, designers are inspired to consider solutions and potential consequences that emerge when solutions are adopted. At the same time, leveraging the designers’ domain knowledge to account for and correct the biases inherent in LLMs. Through a case study, we demonstrate the functionality of this mixed-initiative system. The goal of this demo is to present a creativity support tool that is intended to teach design students to think more systematically by generating ideas that challenge their thinking rather just augmenting their creative potential.

Online whiteboards are becoming a popular way to facilitate collaborative design work, providing a free-form environment to curate ideas. However, as templates are increasingly being used to scaffold contributions from non-experts designers, it is crucial to understand their impact on the creative process. In this paper, we present the results from a study with 114 students in a large introductory design course. Our results confirm prior findings that templates benefit students by providing a starting point, a shared process, and the ability to access their own work from previous steps. While prior research has criticized templates for being too rigid, we discovered that using templates within a free-form environment resulted in visual patterns of free-form curation where concepts were spatially organized, clustered, color-coded, and connected using arrows and lines. We introduce the concept of ‘Free-form Templates’ to illustrate how templates and free-form curation can be synergistic.

Generating high-quality multiple-choice questions (MCQs) is a time-consuming activity that has led practitioners and researchers to develop community question banks and reuse the same questions from semester to semester. This results in generic MCQs which are not relevant to every course. Template-based methods for generating MCQs require less effort but are similarly limited. At the same time, advances in natural language processing have resulted in large language models (LLMs) that are capable of doing tasks previously reserved for people, such as generating code, code explanations, and programming assignments. In this paper, we investigate whether these generative capabilities of LLMs can be used to craft high-quality M CQs more efficiently, thereby enabling instructors to focus on personalizing MCQs to each course and the associated learning goals. We used two LLMs, GPT-3 and GPT-4, to generate isomorphic MCQs based on MCQs from the Canterbury Question Bank and an Introductory to Low-level C Programming Course. We evaluated the resulting MCQs to assess their ability to generate correct answers based on the question stem, a task that was previously not possible. Finally, we investigate whether there is a correlation between model performance and the discrimination score of the associated MCQ to understand whether low discrimination questions required the model to do more inference and therefore perform poorly. GPT-4 correctly generated the answer for 78.5% of MCQs based only on the question stem. This suggests that instructors could use these models to quickly draft quizzes, such as during a live class, to identify misconceptions in real-time. We also replicate previous findings that GPT-3 performs poorly on answering, or in our case generating, correct answers to MCQs. We also present cases we observed where LLMs struggled to produce correct answers. Finally, we discuss implications for computing education.

Generative models are now capable of producing natural language text that is, in some cases, comparable in quality to the text produced by people. In the computing education context, these models are being used to generate code, code explanations, and programming exercises. The rapid adoption of these models has prompted multiple position papers and workshops which discuss the implications of these models for computing education, both positive and negative. This paper presents results from a series of semi-structured interviews with 12 students and 6 instructors about their awareness, experiences, and preferences regarding the use of tools powered by generative AI in computing classrooms. The results suggest that Generative AI (GAI) tools will play an increasingly significant role in computing education. However, students and instructors also raised numerous concerns about how these models should be integrated to best support the needs and learning goals of students. We also identified interesting tensions and alignments that emerged between how instructors and students prefer to engage with these models. We discuss these results and provide recommendations related to curriculum development, assessment methods, and pedagogical practice. As GAI tools become increasingly prevalent, it's important to understand educational stakeholders' preferences and values to ensure that these tools can be used for good and that potential harms can be mitigated.

Curating course material that aligns with students’ learning goals is a challenging and time-consuming task that instructors undergo when preparing their curricula. For instance, it is a challenge to find multiple-choice questions or example codes that demonstrate recursion in an unlabeled question bank or repository. Recently, Large Language Models (LLMs) have demonstrated the capability to generate high-quality learning materials at scale. In this poster, we use LLMs to identify programming concepts found within code snippets, allowing instructors to quickly curate their course materials. We compare programming concepts generated by LLMs with concepts generated by experts to see the extent to which they agree. The agreement was calculated using Cohen’s Kappa.

Reasoning about code and explaining its purpose are fundamental skills for computer scientists. There has been extensive research in the field of computing education on the relationship between a student's ability to explain code and other skills such as writing and tracing code. In particular, the ability to describe at a high-level of abstraction how code will behave over all possible inputs correlates strongly with code writing skills. However, developing the expertise to comprehend and explain code accurately and succinctly is a challenge for many students. Existing pedagogical approaches that scaffold the ability to explain code, such as producing exemplar code explanations on demand, do not currently scale well to large classrooms. The recent emergence of powerful large language models (LLMs) may offer a solution. In this paper, we explore the potential of LLMs in generating explanations that can serve as examples to scaffold students' ability to understand and explain code. To evaluate LLM-created explanations, we compare them with explanations created by students in a large course (n ≈ 1000) with respect to accuracy, understandability and length. We find that LLM-created explanations, which can be produced automatically on demand, are rated as being significantly easier to understand and more accurate summaries of code than student-created explanations. We discuss the significance of this finding, and suggest how such models can be incorporated into introductory programming education.

The recent advent of highly accurate and scalable large language models (LLMs) has taken the world by storm. From art to essays to computer code, LLMs are producing novel content that until recently was thought only humans could produce. Recent work in computing education has sought to understand the capabilities of LLMs for solving tasks such as writing code, explaining code, creating novel coding assignments, interpreting programming error messages, and more. However, these technologies continue to evolve at an astonishing rate leaving educators little time to adapt. This working group seeks to document the state-of-the-art for code generation LLMs, detail current opportunities and challenges related to their use, and present actionable approaches to integrating them into computing curricula.

The introduction of Large Language Models (LLMs) has generated a significant amount of excitement both in industry and among researchers. Recently, tools that leverage LLMs have made their way into the classroom where they help students generate code and help instructors generate learning materials. There are likely many more uses of these tools -- both beneficial to learning and possibly detrimental to learning. To help ensure that these tools are used to enhance learning, educators need to not only be familiar with these tools, but with their use and potential misuse. The goal of this BoF is to raise awareness about LLMs and to build a learning community around their use in computing education. Aligned with this goal of building an inclusive learning community, our BoF is led by globally distributed discussion leaders, including undergraduate researchers, to facilitate multiple coordinated discussions that can lead to a broader conversation about the role of LLMs in CS education.

Recent breakthroughs in Large Language Models (LLMs), such as GPT-3 and Codex, now enable software developers to generate code based on a natural language prompt. Within computer science education, researchers are exploring the potential for LLMs to generate code explanations and programming assignments using carefully crafted prompts. These advances may enable students to interact with code in new ways while helping instructors scale their learning materials. However, LLMs also introduce new implications for academic integrity, curriculum design, and software engineering careers. This workshop will demonstrate the capabilities of LLMs to help attendees evaluate whether and how LLMs might be integrated into their pedagogy and research. We will also engage attendees in brainstorming to consider how LLMs will impact our field.

Advances in natural language processing have resulted in large language models (LLMs) that can generate code and code explanations. In this paper, we report on our experiences generating multiple code explanation types using LLMs and integrating them into an interactive e-book on web software development. Three different types of explanations -- a line-by-line explanation, a list of important concepts, and a high-level summary of the code -- were created. Students could view explanations by clicking a button next to code snippets, which showed the explanation and asked about its utility. Our results show that all explanation types were viewed by students and that the majority of students perceived the code explanations as helpful to them. However, student engagement varied by code snippet complexity, explanation type, and code snippet length. Drawing on our experiences, we discuss future directions for integrating explanations generated by LLMs into CS classrooms.

Good explanations are essential to efficiently learning introductory programming concepts [10]. To provide high-quality explanations at scale, numerous systems automate the process by tracing the execution of code [8, 12], defining terms [9], giving hints [16], and providing error-specific feedback [10, 16]. However, these approaches often require manual effort to configure and only explain a single aspect of a given code segment. Large language models (LLMs) are also changing how students interact with code [7]. For example, Github's Copilot can generate code for programmers [4], leading researchers to raise concerns about cheating [7]. Instead, our work focuses on LLMs' potential to support learning by explaining numerous aspects of a given code snippet. This poster features a systematic analysis of the diverse natural language explanations that GPT-3 can generate automatically for a given code snippet. We present a subset of three use cases from our evolving design space of AI Explanations of Code.

Data was a topic previously reserved for expert computer scientists. However, the general public now uses data and visualizations to form opinions and make decisions that affect their lives, such as their health (e.g.: vaccines) and their political opinions (e.g.: gentrification). Unfortunately, few have access to the formal data literacy training required to analyze this data effectively [9]. Existing approaches to provide informal CS education include hackathons [8, 10], after school programs [1], and online tutorials [5]. However, these learning opportunities are often decontextualized from a learner's daily life and require them to go out of their way to learn. Our research investigates techniques to integrate informal learning opportunities seamlessly into people's everyday lives. Prior research in just-in-time learning shows how heuristics can be used in deterministic settings to provide explanations for code [3] and math equations [4]. However, this is challenging when there is no single correct way to interpret and explain content, such as in visualizations. We present Expert Goggles, a browser extension that automatically detects visualizations that people encounter on the web and provides just-in-time, contextually-relevant training modules to scaffold learning and interpretation.

Problem framing—the process of defining a problem—has been described by many researchers and designers as the crux of the design process. However, novice designers struggle with problem framing. To better understand this process and the potential for scaffolding, we conducted two studies. In the first study, we analyzed 41 problem statements from an introductory design course and found that novices often omit key information, like the primary stakeholder or the obstacles they face. To get novices to reflect on and include necessary design information, we created a tool called ProbLib that cues novices to explicitly reflect on aspects of the problem such as the stakeholders. To evaluate this approach, we conducted a between-subjects study (N=73) to compare ProbLib with an unstructured open text form. We found that participants using ProbLib wrote higher quality statements, included more information, and were more confident about specifying design needs. We observed creative behaviors such as brainstorming and analogical reasoning.

When exploring a new domain through web search, people often struggle to articulate queries because they lack domain-specific language and well-defined informational goals. Perhaps search tools rely too much on the query to understand what a searcher wants. Towards expanding this contextual understanding of a user during exploratory search, we introduce a novel system, CoNotate, which offers query suggestions based on analyzing the searcher’s notes and previous searches for patterns and gaps in information. To evaluate this approach, we conducted a within-subjects study where participants (n=38) conducted exploratory searches using a baseline system (standard web search) and the CoNotate system. The CoNotate approach helped searchers issue significantly more queries, and discover more terminology than standard web search. This work demonstrates how search can leverage user-generated content to help people get started when exploring complex, multi-faceted information spaces.

While research shows that web search plays a role throughout the creative process, less is known about about how people use web search to learn and frame their thinking about an open problem. People need web search to gather information about a problem area, but this can also influence the rest of the creative process. To understand how web search affects early-stage design, we collected and analyzed search log and self-report data from 34 students in a project-based design class. Participants reported struggling with scoping broad, ill-defined information goals into queries, learning domain-specific language, and assessing the usefulness of information. Analysis found that more active and diverse search behavior (i.e. issuing more frequent and diverse queries, and opening more webpages) related to more progress in early-stage design (i.e. gathering more facts, articulating more insights, and developing better problem frames). Based on these findings, we discuss implications for designing search tools to support peoples' creative processes.

Many organizations have adopted design processes that integrate community voices to discover the real problems that communities face. Online discussion forums offer a familiar and flexible technology that can help facilitate discussion around problems and potential solutions. However, we lack understanding about what information community members share, how that information is structured, and how social interactions affect design processes at scale. This paper presents a mixed-methods analysis of Canvas, a learning management system, which enables users to contribute to the design of the platform by sharing and deliberating on problems and solutions in a discussion forum. We collected and analyzed 1412 ideas and 18,335 associated comments shared on the Canvas discussion forum. We found that the distributed nature of design information, the presence of duplicate ideas, and contributors' gaming behaviors made it difficult for the community to make sense of the design discussion. These gaming behaviors also constitute a new concern for participatory design research. Finally, we reflect on how Canvas community members contribute information to a shared design space and how future systems could more effectively coordinate community design efforts.

This work-in-progress paper proposes a semiautomated method to analyze students' reflections. It is challenging to include reflection activities in computing classes because of the amount of time required from students to answer the reflection questions and the amount of effort required for instructors to review the students' responses. These challenges inspired us to adopt Digital Minute Paper (DMP) as a way to give students multiple, quick opportunities to stop and reflect on their experiences in class. In this way, students are given an opportunity to develop metacognitive skills and to potentially improve their performance in the class. In addition, we used these DMPs as formative feedback for the instructors to address students' problems in the class and to continuously improve the course design. Reading reflections is tedious, time-consuming, and does not scale to large classes. To extract insights from the DMPs, we created a semi-automated process for analyzing DMPs by applying natural language processing (NLP). Our process extracts unigrams and bigrams from the reflections and then visualizes related quotes from the reflections using a treemap visualization. We found that this semi-automatic analysis of the reflections is a good, low-effort way to capture student feedback in addition to helping students be more self-regulating learners.

This paper describes and compares two categories of pedagogical design patterns that have emerged from CS education practice: lecture-based design patterns and active learning design patterns. Pedagogical design patterns provide faculty with combinations of generalized descriptions of problems and solutions that occur in teaching and learning. The benefit of forming design patterns is the codification of successful practice that can be reused in multiple scenarios and draw on the creativity of the instructor for defining the details relevant to the course and the students. Design patterns have been represented in many formats since Alexander’s initial design pattern model highlighting different aspects of what is important in each domain in which the patterns are created and used. This paper analyzes design patterns emerging from recent developments in lecture-based pedagogy and active learning in CS education. Traditional lectures in computer science, engineering, and other STEM disciplines are being reconsidered due to research that shows that students are less likely to learn while listening and more likely to learn while actively engaged. Design patterns that address problems and provide potential solutions to traditional lectures in computer science education have been published that provide solutions to engage students during the lecture. The pedagogy of flipped classrooms and active learning have recently been adopted by many faculty in Computer Science leading to emerging design patterns for active learning. We compare how previously published lecture-based patterns and our active learning patterns address similar problems with different solutions to engaging students. We show how an object-based structure for pedagogical design patterns can provide additional information about the problems and the solutions addressed by the patterns that are more easily indexed and combined.

This research to practice full paper provides preliminary evidence that integrating reflections is a significant feature to identify at-risk students early in a semester as verified and validated by a sequence-based learning analytics model. We've devised an active learning classroom model which incorporates reflection at multiple points in students' learning experience. This active learning model adopts Kolb's Learning model to provide a coherent and connected set of activities before, during, and after the class. Unlike periodic assessment through testing, reflections can provide nearly-real-time information about student's experiences in class. We extract sentiment feature vectors to capture students' affect from written reflections. These features typically aren't assessed on tests or during in-class activities. These features were extracted automatically using LIWC (Linguistic Inquiry and Word Count) is a tool for applied natural language processing) which is less cumbersome to implement than manually reading the written reflections. We find that using these sentiment feature vectors extracted from the reflections in our learning model increased accuracy while decreasing time-to-detect at-risk students significantly.

This paper presents the evolution of a curricular design over four semesters with the intention of improving learning for all while increasing retention and equity in a fully flipped, team-based data structures class. Using peer instruction, lightweight teams, an automated grader, TA-mentorship, and pair programming labs, we create an inclusive, equity-driven peer learning environment. We describe our holistic and iterative course design, reports and analysis of student background and performance, results from anonymous student surveys, and future plans for further improving the inclusive success in computing courses.

We present an exploration into the use of dimensional reasoning and the creation of research design spaces. We hypothesized that researchers, engaged in open-ended creative problem solving, could adapt the methods of design thinking and design spaces to create dimensionalized research design spaces. To investigate how researchers might explicitly engage in such dimensionalization processes, we created and studied (n=5, n=5) an interactive web-based system for the creation of research design spaces. Our results showed that a 'dimensions-first' approach was difficult for researchers to work with. We then created and studied (n=11) a prototype 'examples-first' approach. Our results suggest that the ability of researchers to explicitly dimensionalize their research areas is quite varied and that significant scaffolding is required to help researchers reason in this way.

In industry, professionals often work with a variety of stakeholders and collaborators from multiple disciplines. This ability to work collaboratively can be as important to a project's success as their technical skills. Traditionally in STEM education, these collaborative skills are developed in a capstone course which mimics an industry experience. These experiences are invaluable in preparing students for the collaborative real-world nature of industry; however, these experiences can also be very stressful for students in dysfunctional teams with members who haven't developed necessary social, technical or teamwork skills. Although students may be exposed to some team-based activities in previous courses, it is not clear that this piecemeal exposure teaches students to work in teams effectively. Flipped classroom and active learning attempt to fill this gap by exposing students to peer learning earlier in the curriculum. However, these techniques are peppered throughout the curriculum and may not target all the skills necessary for teamwork. Design patterns in education formalize pedagogical approaches. But, applying design patterns without an intended progression or overarching goal may not lead students to successfully adopt these skills. Design patterns have the potential to scaffold students' development throughout the curriculum, but only if staged effectively and systematically. In this paper, we propose Spectrums and Dependency Graphs to ensure that students are prepared for each new design pattern as they experience it. Spectrums can plot design patterns along a continuum between introductory and capstone courses. Dependency graphs recursively specify patterns that prepare students for subsequent patterns. Each pattern will contain prerequisite skills or experiences that students have demonstrated in a previous pattern. In this way, students are systematically progressed from introductory to capstone courses. Through these two models, we attempt to get a better overview of the curriculum and create progressions through that curriculum that ensure students are prepared at each level, building on previous skills.

Reflection is a process of "critical review" of previous experiences to inform future action. Reflection has its origins in design and engineering but has gained traction in education as well. Reflective learning affords students the opportunity to reflect critically on their learning and develop metacognitive skills. Scaffolding is necessary as students adopt a reflective practice, but few tools support this process. Our prior work with teams suggests that students have difficulty estimating their turn-taking behaviors during peer learning activities and reflecting on such misconceptions might be detrimental to the development of social and metacognitive skills. I propose two tools that support data-driven student reflection: BloomMatrix and IneqDetect. BloomMatrix allows students to encode their perceived cognitive processes in an interactive version of Bloom's Taxonomy Matrix. This supports individual reflection, and an aggregated peer heatmap shows other students' perceptions. IneqDetect uses lapel microphones and signal processing to encode live conversations into turn-taking behaviors. In each case, students can reflect about themselves and also about others.

Active learning is becoming increasingly popular as a way to engage students and to provide social support. Providing social support is important because it has been shown to improve both student retention (Slavin 1990, Tinto 1987, Wenger 1999) and performance (Hsiung 2012, Kulkarni 2015, MacNeil 2015). Most forms of active learning use a flipped classroom model of teaching where students are expected to interact with materials, such as lecture videos, alone before class and then engage in active learning during class time (Lage 2000). In our previous work with team-based learning, students generally preferred learning in face-to-face teams (MacNeil 2016). To support co-located teams outside of class, we propose the design for a mobile app that is currently in development which uses students' location to form ad-hoc social learning opportunities for students that are located near each other. We take inspiration from flash teams, which create dynamic expert teams based on the availability of experts online (Retelny 2014). Our app will create teams based on the availability of students and their range of expertise, while leveraging additional context, such as student's location and intention. Students upload course topics that they are struggling with and a general geo-location. The app opportunistically pairs students and suggests that they arrange to meet based on either a geo-location match, a topic match, or preferably, both. In designing our prototype, we consider the possibility of suggesting exercises or assignments that students could work on together. Finally, we provide a brief discussion of the theoretical opportunities and challenges associated with using student's location for ad-hoc social learning.

In both flipped classroom settings and distance learning, educational content is typically delivered via video lectures that students watch alone. While flipped classrooms typically provide students with opportunities for social interaction that feature active learning, online learners are not typically afforded these opportunities. Cooperative learning techniques like Lightweight Teams provide social, collaborative learning opportunities to students in flipped classrooms but extending these techniques to distance learning settings is not straightforward. In this paper, we present our experiences with online, distributed Lightweight Teams. We present an in-the-wild study that compares learning outcomes and student preferences between co-located and distributed Lightweight Teams against the base case of individual learning. Our results show that while there are no significant learning differences between the two team conditions and the individual condition, students significantly prefer the team conditions.

Active learning is important in computer science education, where students often don't have enough opportunities for social learning and development of soft skills. Flipped classrooms can provide social interaction through approaches such as lightweight teams [23], where students collaborate during class in low-stakes peer learning. These teams scaffold positive interdependence [17] by removing high-stakes assignments that heavily impact student's grades.

Abstract We present a new model for creating composite visualizations of multidimensional data sets using simple visual representations such as point charts, scatterplots and parallel coordinates as components. Each visual representation is contained in a tile, and the tiles are arranged in a mosaic of views using a space‐filling slice‐and‐dice layout. Tiles can be created, resized, split or merged using a versatile set of interaction techniques, and the visual representation of individual tiles can also be dynamically changed to another representation. Because each tile is self‐contained and independent, it can be implemented in any programming language, on any platform and using any visual representation. We also propose a formalism for expressing visualization mosaics. A Web‐based implementation called MosaicJS supporting multidimensional visual exploration showcases the versatility of the concept and illustrates how it can be used to integrate visualization components provided by different toolkits.