As AI assistance becomes embedded in programming practice, researchers have increasingly examined how these systems help learners generate code and work more efficiently. However, these studies often position AI as a replacement for human collaboration and overlook the social and learning-oriented aspects that emerge in collaborative programming. Our work introduces human-human-AI (HHAI) triadic programming, where an AI agent serves as an additional collaborator rather than a substitute for a human partner. Through a within-subjects study with 20 participants, we show that triadic collaboration enhances collaborative learning and social presence compared to the dyadic human–AI (HAI) baseline. In the triadic HHAI conditions, participants relied significantly less on AI generated code in their work. This effect was strongest in the HHAI-shared condition, where participants had an increased sense of responsibility to understand AI suggestions before applying them. These findings demonstrate how triadic settings activate socially shared regulation of learning by making AI use visible and accountable to a human peer, suggesting that AI systems that augment rather than automate peer collaboration can better preserve the learning processes that collaborative programming relies on.

Code reviews are a critical aspect of open-source software (OSS) development, ensuring quality and fostering collaboration. This study examines perceptions, challenges, and biases in OSS code review processes, focusing on the perspectives of Contributors and Maintainers. Through surveys (n = 289), interviews (n = 23), and repository analysis (n = 81), we identify key areas of alignment and disparity. While both groups share common objectives, differences emerge in priorities, e.g, with Maintainers emphasizing alignment with project goals while Contributors overestimated the value of novelty. Bias, particularly familiarity bias, disproportionately affects underrepresented groups, discouraging participation and limiting community growth. Misinterpretation of approach differences as bias further complicates reviews. Our findings underscore the need for improved documentation, better tools, and automated solutions to address delays and enhance inclusivity. This work provides actionable strategies to promote fairness and sustain the long-term innovation of OSS ecosystems.

Background: Recent innovations in generative artificial intelligence (AI) have transformed how programmers develop and maintain software. The advanced capabilities of generative AI tools in supporting development tasks have led to a rise in their adoption within software engineering (SE) workflows. However, little is known about how AI tools perceive evidence-based practices supported by empirical SE research. Aim: To this end, we explore the”beliefs“of generative AI tools increasingly used to support software development in practice. Method: We conduct a preliminary evaluation conceptually replicating prior work to investigate 17 evidence-based claims across five generative AI tools. Results: Our findings demonstrate generative AI tools have ambiguous beliefs regarding research claims and lack credible evidence to support responses. Conclusions: Based on our results, we provide implications for practitioners integrating generative AI-based systems into development contexts and shed light on future research directions to enhance the reliability and trustworthiness of generative AI—aiming to increase awareness and adoption of evidence-based SE research findings in practice.

Continuous Integration and Continuous Delivery (CI/CD) processes are vital to meet the growing demands of open source software (OSS), providing a pipeline to enhance project quality and productivity. To ensure the user interfaces (UIs) of these systems work as intended, UI testing is crucial for verifying visual elements of software. Integrating UI tests in CI/CD pipelines should provide fast delivery and comprehensive test coverage. However, there is a gap in understanding how popular UI testing frameworks are adopted within CI/CD workflows-and the effects of this integration on OSS development. Aims: This study aims to explore developers' perceptions of the challenges, strategies, and impacts of incorporating UI testing into CI/CD environments. In particular, we focus on OSS developers utilizing popular web-based UI testing frameworks-such as Selenium, Cypress, and Playwright-and popular CI/CD platforms-including GitHub Actions, Travis CI, CircleCI, and Jenkins-on public GitHub repositories. Method: We conducted an online survey targeting OSS developers (n = 94) from GitHub with experience integrating UI testing frameworks into configuration files for CI/CD platforms. To augment our results, we conducted follow-up interviews ($n$= 18) to gain insights on the challenges, opportunities, and impacts of integrating UI testing into CI/CD pipelines. Results: Our results indicate adapting testing strategy, flakiness and longer executions are major challenges in integrating UI testing into CI pipelines-negatively impacting development practices. Alternatively, the benefits include support for realistic test cases and increased detection of issues. However, developers lack effective strategies to mitigate the challenges, relying on ad hoc trial-and-error based approaches, such as temporarily removing flaky tests from CI workflows until they are resolved. Conclusion: Our findings provide implications for OSS developers working on or considering including UI tests in CI/CD pipelines. We also motivate future directions for research and tooling to improve UI testing integration in CI/CD workflows.

Current hiring practices use technical & soft skills proxy keyword based resume matching via Automated Resume Parsers (ARPs). However, this process fails to extract the actual abilities of candidates, such as the quality of their written code, raising concerns regarding the effectiveness of current approaches. Thus, novel evidences accurately depicting candidates' skills are necessary to inform hiring decisions. We posit GitHub-supported resume matching as a solution, mining data from candidates' open source projects to provide evidence for their technical skills. We conducted a preliminary survey$(n=48)$to gain insights from candidates and recruiters on proxies from GitHub projects indicative of technical abilities. We found both groups preferred metrics regarding code quality and used technologies, and there was overwhelming willingness to incorporate this analysis in resume matching tasks. Based on these insights, we designed GitMeter - a tool to capture technical abilities (i.e., code quality) and soft skills of candidates by mining public GitHub repositories. GitMeter uses a novel heuristic-based approach to find the most accurate code quality approximation for candidate-written code (core code), minimizing the time and computational overhead. Finally, we evaluate effectiveness and potential impact of GitMeter through a user study$(n=20)$with developers and recruiters. Our findings provide implications for future tools and methods aiming to promote evidence-based hiring in software engineering (SE) contexts.

One challenge in technical interviews is the thinkaloud process, where candidates verbalize their thought processes while solving coding tasks. Despite its importance, opportunities for structured practice remain limited. Conversational AI offers potential assistance, but limited research explores user perceptions of its role in think-aloud practice. To address this gap, we conducted a study with 17 participants using an LLM-based technical interview practice tool. Participants valued AI’s role in simulation, feedback, and learning from generated examples. Key design recommendations include promoting social presence in conversational AI for technical interview simulation, providing feedback beyond verbal content analysis, and enabling crowdsourced think-aloud examples through humanAI collaboration. Beyond feature design, we examined broader considerations, including intersectional challenges and potential strategies to address them, how AI-driven interview preparation could promote equitable learning in computing careers, and the need to rethink AI’s role in interview practice by suggesting a research direction that integrates human-AI collaboration.

Debugging is one of the most difficult and tedious tasks for software engineers. While various tools and techniques have been introduced to assist debugging, most programmers use print statement debugging to find and fix errors in their code. That is, they manually add code to print values to verify if the code is executing as expected and make sure a certain section of the program is reached. However, this process can be time-consuming and error-prone, especially in large and complex programs. To that end, we introduce AutoPrint, a tool that automatically inserts and removes print statements to streamline print statement debugging in Java code. We conducted a judgment study with 23 participants—students and practitioners—to elicit feedback on AutoPrint and gain insights on its utility in practical debugging tasks. Our results show participants perceive AutoPrint saves debugging time and effort through a faster, simpler, and more usable tool compared to other approaches.

Human subjects research is fundamental to advancing human-computer interaction (HCI), as it helps researchers understand user behaviors, needs, and experiences to enhance the design of software products [3]. However, conducting such human-centric studies is often hindered by challenges in participant recruitment, including inefficiencies in finding eligible participants and administrative burdens [1]. Participants are hindered from joining research studies due to logistical, eligibility, communication, or personal barriers that make participation impractical, inaccessible, or unappealing. These barriers not only slow down research progress, but also limit the diversity and representativeness of study participants. To address these issues, we propose ParticipantGuide-a structured labelbased approach to enhance participant recruitment through providing key and interpretable information to potential participants. We discuss existing work, present a preliminary design, and provide implications for future research.

Studying abroad can be a life-changing experience that can help students develop skills, make friends, and gain a global perspective. However, Computer Science (CS) students rarely encounter opportunities to participate in a study abroad program. We interviewed students who participated in our institution’s first CS Study Abroad program-focused on software engineering-to understand participants’ experiences and how the program impacted the students’ personal and computing identities. We found that students faced unique challenges, such as working with teammates who have different cultural backgrounds and programming styles. Through overcoming those challenges, students were able to strengthen their computing identity and gain confidence that they could work in diverse software engineering teams. Based on the lessons that we learned, we provide guidelines to enhance future CS Study Abroad experiences.

Large Language Models (LLMs) are gaining traction in various hiring-related tasks for both candidates and employers. For instance, employers are increasingly using LLMs to rate applicants based on the match between their resume content and job description requirements. Meanwhile, candidates use LLMs to write cover letters and tailor resume content. However, research shows LLMs can impart biases for LLMgenerated content and against minorities, such as disabled candidates in hiring contexts. Thus, we conduct an observational study to investigate how widely used LLMs - OpenAI’s GPT-4, Google’s Gemini and Anthropic’s Claude - perform in supporting resume tasks for employers and candidates. Using a real-world dataset of job descriptions and resumes from disabled ($n=$ 209) and non-disabled (n = 209) candidates, we examine the capabilities of these models across resume rating and resume generation tasks in a zero-shot setting. Our main findings show moderate alignment across models for resume matching and no significant differences in ratings between disabled and nondisabled candidates. However, we did observe increased ratings for LLM-generated resumes for GPT-4 and Claude. We discuss the implications for both candidates and employers based on our findings, aiming to promote non-biased and equitable AIbased hiring processes and motivate human-AI collaboration in the design of future hiring systems.

Cognitive presence and social presence are crucial for a comprehensive learning experience. Despite the flexibility of asynchronous learning environments to accommodate individual schedules, the inherent constraints of asynchronous environments make augmenting cognitive and social presence particularly challenging. Students often face challenges such as a lack of timely feedback and support, an absence of non-verbal cues in communication, and a sense of isolation. To address this challenge, this paper introduces Generative Co-Learners, a system designed to leverage generative AI-powered agents, simulating co-learners supporting multimodal interactions, to improve cognitive and social presence in asynchronous learning environments. We conducted a study involving 12 student participants who used our system to engage with online programming tutorials to assess the system's effectiveness. The results show that by implementing features to support textual and visual communication and simulate an interactive learning environment with generative agents, our system enhances the cognitive and social presence in the asynchronous learning environment. These results suggest the potential to use generative AI to support student learning and transform asynchronous learning into a more inclusive, engaging, and efficacious educational approach.

Introduction : Digital products increasingly reshape industries, influencing human behavior and decision-making. However, the software development teams developing these systems often lack diversity, which may lead to designs that overlook the needs, equal treatment or safety of diverse user groups. These risks highlight the need for fostering diversity and inclusion in software development to create safer, more equitable technology. Method : This research is based on insights from an academic meeting in June 2023 involving 23 software engineering researchers and practitioners. We used the collaborative discussion method 1-2-4-ALL as a systematic research approach and identified six themes around the theme “challenges and opportunities to improve Software Developer Diversity and Inclusion (SDDI).” We identified benefits, harms, and future research directions for the four main themes. Then, we discuss the remaining two themes, AI & SDDI and AI & Computer Science education, which have a cross-cutting effect on the other themes. Results : This research explores the key challenges and research opportunities for promoting SDDI, providing a roadmap to guide both researchers and practitioners. We underline that research around SDDI requires a constant focus on maximizing benefits while minimizing harms, especially to vulnerable groups. As a research community, we must strike this balance in a responsible way.

Software systems are expected to be secure and robust. To verify and ensure software security, it is vital to include security activities, or development practices to detect and prevent security vulnerabilities, into the software development process. Agile software development is a popular software engineering (SE) process used by many organizations and development teams. However, while Agile aims to be a lightweight and responsive process, security activities are typically more cumbersome and involve more documentation and tools–violating the core principles of Agile. This work investigates the impact of security activities on various aspects of Agile development. To understand how software engineers perceive incorporating security practices into Agile methodologies, we distributed an online survey to collect data from software practitioners with experience working in Agile teams. Our results from 34 survey participants show most software practitioners believe security activities are beneficial to development overall but lack confidence in their impact on the security of software systems. Our findings provide insight into how security activities affect Agile development and provide implications to help SE teams better incorporate security activities into implementing Agile development processes.

Background: Governments worldwide are considering data privacy regulations. These laws, such as the European Union’s General Data Protection Regulation (GDPR), require software developers to meet privacy-related requirements when interacting with users’ data. Prior research describes the impact of such laws on software development, but only for commercial software. Although open-source software is commonly integrated into regulated software, and thus must be engineered or adapted for compliance, we do not know how such laws impact open-source software development.

Software engineering (SE) is constantly changing and evolving, increasing the gaps in knowledge and skills that students need to become productive software engineers in industry. To address the gaps in SE education, this paper introduces SE Gap Awareness--an online platform to increase awareness of gaps from industry professionals using gamification and provide resources to help users improve on SE skills. We conducted a preliminary evaluation by engaging with practitioners to identify gaps they perceive in SE education and perform a user study to students' self-assessment of gaps and usage of our system. Our findings show students rank their skills higher than practitioners in soft, hard, and coding skills, and found SE Gap Awareness useful for increasing awareness of gaps and exploring resources concerning deficient skills. Based on our initial tool and evaluation, we provide implications for future systems to mitigate gaps in SE education.

Supporting novice computer science students in learning the software development life cycle (SDLC) at scale is vital for ensuring the quality of future software systems. However, this presents unique challenges, including the need for effective interactive collaboration and access to diverse skill sets of members in the software development team. To address these problems, we present ''DevCoach'', an online system designed to support students learning the SDLC at scale by interacting with generative agents powered by large language models simulating members with different roles in a software development team. Our preliminary user study results reveal that DevCoach improves the experiences and outcomes for students, with regard to learning concepts in SDLC's ''Plan and Design'' and ''Develop'' phases. We aim to use our findings to enhance DevCoach to support the entire SDLC workflow by incorporating additional simulated roles and enabling students to choose their project topics. Future studies will be conducted in an online Software Engineering class at our institution, aiming to explore and inspire the development of intelligent systems that provide comprehensive SDLC learning experiences to students at scale.

Recruiting participants for human-centric computing (HCC) research studies is crucial for understanding user needs and behavior, evaluating usability, and providing real-world insights to design effective technology solutions. However, this process presents various challenges, from reaching target demographics to managing recruitment communication techniques. This paper investigates HCC research recruitment strategies, challenges, and solutions through 12 focus groups comprising of two different stakeholders in HCC research: researchers and research study participants, with a total of 26 participants. By examining the experiences and challenges faced by these groups, we aim to identify effective strategies to improve participant recruitment. The findings highlight common obstacles and offer recommendations for enhancing recruitment practices in HCC research, ultimately contributing to more robust experiments that promote user-centered technology development and solutions.

Recent developments in Artificial Intelligence (AI) have shifted the software development paradigm. Past studies demonstrated how effective AI can generate code for programming purposes. However, to our knowledge, no prior study has been done to evaluate the effectiveness of SQL queries generated by AI. We utilized nine AI assistants to generate SQL queries. Our results reveal that most AI assistants generate inaccurate SQL queries, and based on the results, we provide possible implications for SQL developers.

Dynamic Application Security Testing (DAST) tools test web application security by simulating attacks on its front end and evaluating it externally like a malicious attacker. DAST tools aim to identify vulnerabilities and provide recommendations for improving security. However, the security alerts generated by these tools are lengthy and contain numerous details that may not be relevant to a software practitioner seeking a quick overview of the results. To solve this challenge, we propose using Large Language Models (LLMs) to summarize the alerts generated by DAST tools. We generated security alerts using two popular DAST tools: Burp Suite and ZAP. Then, we generated various summaries of these alerts using five different LLMs. We surveyed 48 software practitioners to understand the challenges software practitioners face when specifically dealing with DAST reports and determine the effectiveness of the LLM-generated summaries in understanding the security issue. The results bring to light various challenges software practitioners face and indicate that the LLM-generated summaries are clearer and more comprehensible in understanding the security issue and, hence, more preferred. This approach can significantly improve the security of software products by making the security alerts more accessible to different stakeholders, making the software product more robust and resilient to cyber threats.

Debugging is one of the most expensive and time-consuming processes in software development. To support programmers, researchers, and developers have introduced a wide variety of debuggers or tools to automatically find errors in code, to make this process more efficient. However, there is a gap between industry developers and students regarding the skillful use of debuggers. We aim to understand this gap by studying barriers that hinder new programmers from using debuggers. We conducted a survey involving 73 students with various extents of programming experience and performed qualitative analysis. The goal was to extract insights into why students do not develop debugger usage skills. Our results suggest the general lack of academic course focus on debuggers is one of the primary reasons for avoidance. At the same time, complex user interfaces and a lack of visualization also seem intimidating for many students, making using a debugger unappealing. Based on the results, we provide guidelines to motivate future debugger designs and education materials to improve debugger usage. Our survey results summary is publicly available at https://github.com/minhyukko/vlhcc23

To streamline tech hiring processes, the use of talent management platforms has emerged as a new norm. AI-driven Automated Resume Parsers (ARPs) are aimed to simplify the application process for candidates and employers. However, ARP designs typically prioritize employers over candidates. Further, prior work also demonstrates these AI systems are not able to achieve the intended goals of inclusivity and fairness for candidates, negatively impacting minorities in the tech hiring pipeline. Thus, aspiring IT professionals on the job market often spend significant time and effort preparing applications, only to have their resume rejected by an AI model without receiving attention from a recruiter. This work aims to study candidates' perspectives of ARPs. We sent an survey, receiving responses from 103 undergraduate and graduate CS students, and analyze their perspectives through a prism of procedural justice, a measure of fairness. By introducing procedural justice and opting for a human-centered design approach, we believe AI models in the hiring pipeline can achieve the intended goals of inclusivity and fairness. The findings from this study will be beneficial for future designs of more transparent and fair ARPs.

The use of AI assistants, along with the challenges they present, has sparked significant debate within the community of computer science education. While these tools demonstrate the potential to support students' learning and instructors' teaching, they also raise concerns about enabling unethical uses by students. Previous research has suggested various strategies aimed at addressing these issues. However, they concentrate on introductory programming courses and focus on one specific type of problem. The present research evaluated the performance of ChatGPT, a state-of-the-art AI assistant, at solving 187 problems spanning three distinct types that were collected from six undergraduate computer science. The selected courses covered different topics and targeted different program levels. We then explored methods to modify these problems to adapt them to ChatGPT's capabilities to reduce potential misuse by students. Finally, we conducted semi-structured interviews with 11 computer science instructors. The aim was to gather their opinions on our problem modification methods, understand their perspectives on the impact of AI assistants on computer science education, and learn their strategies for adapting their courses to leverage these AI capabilities for educational improvement. The results revealed issues ranging from academic fairness to long-term impact on students' mental models. From our results, we derived design implications and recommended tools to help instructors design and create future course material that could more effectively adapt to AI assistants' capabilities.

Software engineers often face a critical test before landing a job—passing a technical interview. During these sessions, candidates must write code while thinking aloud as they work toward a solution to a problem under the watchful eye of an interviewer. While thinking aloud during technical interviews gives interviewers a picture of candidates’ problem-solving ability, surprisingly, these types of interviews often prevent candidates from communicating their thought process effectively. To understand if poor performance related to interviewer presence can be reduced while preserving communication and technical skills, we introduce asynchronous technical interviews—where candidates submit recordings of think-aloud and coding. We compare this approach to traditional whiteboard interviews and find that, by eliminating interviewer supervision, asynchronicity significantly improved the clarity of think-aloud via increased informativeness and reduced stress. Moreover, we discovered asynchronous technical interviews preserved, and in some cases even enhanced, technical problem-solving strategies and code quality. This work offers insight into asynchronous technical interviews as a design for supporting communication during interviews, and discusses trade-offs and guidelines for implementing this approach in software engineering hiring practices.

Understanding the nature of regular expression (regex) issues is important to tackle practical issues developers face in regular expression usage. Knowledge about the nature and frequency of various types of regular expression issues, such as those related to performance, API misuse, and code smells, can guide testing, inform documentation writers, and motivate refactoring efforts. However, beyond ReDoS (Regular expression Denial of Service), little is known about to what extent regular expression issues affect software development and how these issues are addressed in practice.

Recommendations between colleagues are effective for encouraging developers to adopt better practices. Research shows these peer interactions are useful for improving developer behaviors, or the adoption of activities to help software engineers complete programming tasks. However, in-person recommendations between developers in the workplace are declining. One form of online recommendations between developers are pull requests, which allow users to propose code changes and provide feedback on contributions. GitHub, a popular code hosting platform, recently introduced the suggested changes feature, which allows users to recommend improvements for pull requests. To better understand this feature and its impact on recommendations between developers, we report an empirical study of this system, measuring usage, effectiveness, and perception. Our results show that suggested changes support code review activities and significantly impact the timing and communication between developers on pull requests. This work provides insight into the suggested changes feature and implications for improving future systems for automated developer recommendations, such as providing situated, concise, and actionable feedback.

Temporal API specifications are useful for many software engineering tasks, such as test case generation. In practice, however, APIs are rarely formally specified, inspiring researchers to develop tools that infer or mine specifications automatically.Traditional specification miners infer likely temporal properties by statically analyzing the source code or by analyzing program runtime traces. These approaches are frequently confounded by the complexity of modern software and by the unavailability of representative and correct traces. Formally specifying software is traditionally an expert task. We hypothesize that human crowd intelligence provides a scalable and high-quality alternative to experts, without compromising on quality. In this work we present CrowdSpec, an approach to use collective intelligence of crowds to generate or improve automatically mined specifications. CrowdSpec uses the observation that APIs are often accompanied by natural language documentation, which is a more appropriate resource for humans to interpret and is a complementary source of information to what is used by most automated specification miners.

To help users gain awareness of tools and features available in applications, recommender systems can automatically suggest useful tools. Such systems aim to present recommendations just like users would recommend tools to one another, but little is known about the nature of these user-to-user recommendations. This paper explores user-to-user recommendations through a study of 13 pairs of software users performing data analysis tasks. We found that users were more likely to adopt tools when they were receptive to the recommendation, but did not find the recommendations were any more likely to be effective when they contained other characteristics such as politeness, persuasiveness, or referred to observable tools. These findings suggest that, for example, automated systems should avoid recommending obscure and unfamiliar tools, but making recommendations politely is not a critical design goal.

Program navigation is a critical task for software developers. State-of-the-art tools have been shown to support effective program navigation strategies, and do so by adding widgets, secondary views, and visualizations to the screen. In this work, we build on prior work by exploring what types of navigation can be supported with relatively few interface elements. To that end, we designed and implemented a prototype tool, named Flower, that supports structural program navigation while maintaining a minimalistic interface. Flower enables developers to simultaneously navigate control flow and data flow within the Eclipse Integrated Development Environment. Based on a preliminary evaluation with eight programmers, Flower succeeds when call graphs contained relatively few branches, but was strained by complex program structures.

In order to get students interested in computing, we teach middle school teachers of different disciplines programming with Alice and work with them on integrating computing into their discipline. Alice provides an interface for novices to create animations easily and quickly, which is attractive to and fun for students. We have been developing Alice curriculum materials for integrating computing into middle school disciplines for six years. Although our target audience is middle school, our materials are used by teachers from elementary school to introductory college level. This paper describes our newest curriculum materials for several disciplines developed by both us and our teachers. Our newest curriculum materials include tutorials, sample projects, and challenges, which are projects with missing pieces. We also discuss our recent outreach efforts with middle school students.

Our project is part of the Adventures in Alice Programming project at Duke University. In particular, our project is integrating computer science into middle school math using Alice. We show several ways for students to improve their math skills while engaging their interest in programming. First, we have created Alice worlds for students to interact with to practice math concepts. Second, we have created tutorials to guide students on building such worlds. Third, we have created short challenge problems for students to focus on the math and the programming statements to complete a mostly built world. To encourage the use of Alice with projects we have developed many sample math projects. To encourage teachers to use Alice with math and computer science, we have been mapping our free curriculum materials to both the Commmon Core Math standards and the CSTA CS standards. Our curriculum materials are available at www.cs.duke.edu/csed/alice/aliceInSchools

This interactive workshop will present the new features of Alice 2.3, and show how to integrate Alice 2.3 into multiple disciplines in middle school and high school. Participants will get hands-on experience with working with new Alice models and creating Alice projects. The workshop will also review curriculum materials and discuss mapping Alice to CSTA computer science standards. The curriculum materials presented could be used in middle school or high school in a variety of disciplines, or in college in a pre-CS 1 course. The target audience is middle school and high school teachers, and college faculty providing outreach to K-12 or teaching a pre-CS 1 course. Alice is available for free at www.alice.org. Curriculum materials are available at www.cs.duke.edu/csed/alice/aliceInSchools and at www.alice.org. Laptop required, and two-button mouse recommended.