Abstract Large language models (LLMs) are rapidly reshaping software development, but their impact across the full software development lifecycle is underexplored. Existing work tends to focus on isolated activities such as code generation or testing, leaving open questions about how LLMs affect developers, processes, products, and the broader software ecosystem. We address this gap through semi-structured interviews with sixteen early-adopter software professionals who integrated LLM-based tools into their day-to-day work in early to mid-2023. We treat these interviews as early empirical evidence and compare participants’ accounts with recent work on LLMs in software engineering, noting which early patterns persist or shift. Using thematic analysis, we organize our findings around four dimensions: people, process, product, and society. Developers reported substantial productivity gains from reducing mundane tasks, streamlining search, and accelerating debugging, but also described a productivity-quality paradox: they frequently discarded generated code and shifted effort from writing code to critically evaluating and integrating it. LLM use was highly phase-dependent, with strong uptake in implementation and debugging but limited influence on requirements gathering and collaborative work. Participants developed new competencies to use LLMs effectively, including prompt engineering strategies, multi-layered verification, and security-conscious integration to protect proprietary data. They also anticipated changes in hiring expectations, team practices, and computing education, while emphasizing that human judgment and foundational software engineering skills remain essential. Our findings, consistent with evidence from large-scale studies, offer actionable implications for developers, organizations, educators, and tool designers seeking to integrate LLMs responsibly into professional software practice.

Large Language Models (LLMs) and Generative AI (GenAI) have markedly changed the landscape of many fields, including education. While these tools have significant capabilities, they also require understanding to effectively and responsibly use them. Additionally, little work has been done to evaluate how these tools can best benefit education at the secondary level, with design insights from instructors. My work focuses on informing secondary instructors of these tools, receiving their input on how to make these tools work best for them, and finally using this input to create and evaluate an in-class Retrieval-Augmented Generation (RAG)-based chatbot for their students to use to improve learning outcomes. This work aims to bridge the gap between the latest in computing technology and secondary education classrooms.

Background and Context. This innovative practice full paper describes the development and implementation of a professional development (PD) opportunity for secondary teachers to learn about ChatGPT. Incorporating generative AI techniques from Large Language Models (LLMs) such as ChatGPT into educational environments offers unprecedented opportunities and challenges. Prior research has highlighted their potential to personalize feedback, assist in lesson planning, generate educational content, and reduce teachers' workload, alongside concerns such as academic integrity and student privacy. However, the rapid adoption of LLMs since ChatGPT's public release in late 2022 has left educators, particularly at the secondary level, with a lack of clear guidance on how LLMs work and can be effectively adopted. Objective. This study aims to introduce a comprehensive, free, and vetted ChatGPT course tailored for secondary teachers, with the objective of enhancing their technological competencies in LLMs and fostering innovative teaching practices. Method. We developed a five-session interactive course on ChatGPT capabilities, limitations, prompt-engineering techniques, ethical considerations, and strategies for incorporating ChatGPT into teaching. We introduced the course to six middle and high school teachers. Our curriculum emphasized active learning through peer discussions, hands-on activities, and project-based learning. We conducted pre- and post-course focus groups to determine the effectiveness of the course and the extent to which teachers' attitudes toward the use of LLMs in schools had changed. To identify trends in knowledge and attitudes, we asked teachers to complete feedback forms at the end of each of the five sessions. We performed a thematic analysis to classify teacher quotes from focus groups' transcripts as positive, negative, and neutral and calculated the ratio of positive to negative comments in the pre- and post-focus groups. We also analyzed their feedback on each individual session. Finally, we interviewed all participants five months after course completion to understand the longer-term impacts of the course. Findings. Our participants unanimously shared that all five of the sessions provided a deeper understanding of ChatGPT, featured enough opportunities for hands-on practice, and achieved their learning objectives. Our thematic analysis underlined that teachers gained a more positive and nuanced understanding of ChatGPT after the course. This change is evidenced quantitatively by the fact that quotes with positive connotations rose from 45% to 68% of the total number of positive and negative quotes. Participants shared that in the longer term, the course improved their professional development, understanding of ChatGPT, and teaching practices. Implications. This research underscores the effectiveness of active learning in professional development settings, particularly for technological innovations in computing like LLMs. Our findings suggest that introducing teachers to LLM tools through active learning can improve their work processes and give them a thorough and accurate understanding of how these tools work. By detailing our process and providing a model for similar initiatives, our work contributes to the broader discourse on teaching professional educators about computing and integrating emerging technologies in educational and professional development settings.

Background and Context. The importance of CS to 21st-century life and work has made it important to find ways to integrate learning CS and programming into the regular school day. However, learning CS is difficult, so teachers integrating programming need effective strategies to scaffold the learning. In this study, we analyze students’ log data and apply a novel technique to compare Parsons Problems with from-scratch programming in a middle school science class. Objectives. Our research questions aimed to investigate whether, how, and when Parsons Problems improve learning efficiency for a programming exercise within science, utilizing log data analysis and an automated progress detector (SPD). Method. We conducted a study on 199 students in a 6th-grade science course, divided into two groups: one engaged with Parsons problems, and the other, a control group, worked on the same programming task without scaffolding. Then, we analyzed differences in performance and coding characteristics between the groups. We also adopted an innovative application of SPD to gain a better understanding of how and when Parsons problems helped students make more progress on the coding task, with an objective measure of final student grades. Findings. The experimental group, with scaffolding through Parsons Problems, achieved significantly higher grades, spent significantly less time programming, and toggled less between block category tabs. Interestingly, they ran their code more frequently compared to the control group. The SPD analysis revealed that the experimental group made significantly higher progress in all four quartiles of their coding time. Implications. Our findings suggest that Parsons problems can improve learning efficiency by enhancing novices’ learning experience without negatively impacting their performance or grades, which is especially important when programming is integrated into K12 courses.

Affective state, referring to an individual's feeling, can impact students' confidence and retention in CS, particularly for novice programmers. However, little research has been conducted to examine how moments that occur during programming impact students' affective states in real-time. In this pilot study, seven undergraduate students in an introductory block-based programming course completed a programming assignment and were surveyed and interviewed about their experience and self-efficacy as programmers. While programming, students periodically recorded their affective states via a popup in the programming environment. We performed retrospective think-aloud interviews with students afterward, asking them to watch and reflect on recordings of their programming. We subsequently analyzed student interviews using thematic analysis to derive 206 codes. These codes were grouped into three areas that impacted affect: the environment, objective progress, and perceptions during programming. To explore why students responded as they did to moment occurrence, we further categorized students based on four dimensions: programming experience, assignment completion, confidence, and the impact of the programming session on self-efficacy. Our initial results suggest that while certain moments elicit similar affective states among students, the interaction of the aforementioned four dimensions may have a higher impact on novices' affective states during programming. We conclude with recommendations for educators to improve students' affective states during and after programming.

Many students struggle with decomposition and planning despite the necessity of these skills in computing education. Hence, more tools are needed to scaffold these processes. In this paper, we present Jigsaw, a standalone visual planning tool to help students practice decomposition and planning before writing code. Jigsaw allows students to compose a solution to a new problem based on previously seen “patterns,” such as the accumulator pattern for summing values or the filter pattern for conditional input selection. Students can connect these patterns together to see how data flows between them and define a solution plan. Jigsaw’s goal is to scaffold students’ planning processes by presenting relevant patterns for a given problem. Using a within-subjects design, we evaluated Jigsaw by observing 17 undergraduate students as they planned for and implemented two programming assignments. The experimental task included Jigsaw, and the control task did not. This design aimed to understand how the tool impacted students’ planning and programming process. Subsequently, we conducted interviews with these students regarding their planning and programming experiences with and without Jigsaw. Many students explicitly mentioned they would employ Jigsaw for planning and appreciated the scaffolding it provided. Students also admired the Jigsaw’s novelty in visualizing programming problems. We conclude with our design takeaways and recommendations for future work.

Many students struggle when they are first learning to program. Without help, these students can lose confidence and negatively assess their programming ability, which can ultimately lead to dropouts. However, detecting the exact moment of student struggle is still an open question in computing education. In this work, we conducted a think-aloud study with five high-school students to investigate the automatic detection of progressing and struggling moments using a detector algorithm (SPD). SPD classifies student trace logs into moments of struggle and progress based on their similarity to prior students' correct solutions. We explored the extent to which the SPD-identified moments of struggle aligned with expert-identified moments based on novices' verbalized thoughts and programming actions. Our analysis results suggest that SPD can catch students' struggling and progressing moments with a 72.5% F1-score, but room remains for improvement in detecting struggle. Moreover, we conducted an in-depth examination to discover why discrepancies arose between expert-identified and detector-identified struggle moments. We conclude with recommendations for future data-driven struggle detection systems.