Open source software (OSS) generates trillions of dollars in economic value and has become essential to the technical infrastructures that power organizations worldwide. As these systems increasingly depend on OSS, understanding the evolution of these projects is critical. While existing metrics provide insights into project health, one dimension remains understudied: project resilience, or the ability to return to normal operations after disturbances such as contributor departures,security vulnerabilities and bug report spikes. We hypothesize that stable commit patterns may serve as an indicator of underlying project characteristics such as mature governance, sustained contributors, and robust development processes, factors that existing research associates with resilience. Our findings reveal that only 2% of repositories exhibit daily stability, 29% achieve weekly stability, and 50\% demonstrate monthly stability, while the remaining half are unstable across all levels of granularity. Analysis of the 50 unstable repositories indicate that 86% of activity is concentrated among a few maintainers, with the top 3 contributors accounting for over 50% of commits in the past 5 years. In contrast, the 50 stable repositories distribute work more evenly, with the top 3 contributors representing less than 50% of commits. Our insights thus far indicate the fragile and multi-dimensional nature of OSS project stability, suggesting a need to go beyond commits to understand how our understanding of stability can be enriched with other considerations such as community engagement metrics and issue or pull request churn. Though our efforts only identified two repositories that achieved stability at all three temporal commit granularities, further investigation into their processes and policies can provide insights and foundations for stability-informed risk assessment in practice.

Current AI-powered programming assistants generate large code blocks through single-shot interactions, violating established human-computer interaction principles and increasing cognitive load while reducing developer understanding and control. We present ProDec, a system that automatically decomposes complex programming prompts into structured subtasks with explicit dependencies, enabling interactive visual exploration and modification of AI reasoning steps. By transforming AIassisted programming from black-box code generation to transparent collaborative problem-solving tool, our approach aims to restore developer agency.

The rapid integration of AI-assisted tools such as ChatGPT, GitHub Copilot, and Gemini into software development has reshaped how practitioners perform tasks, collaborate, and manage their workloads. While these tools offer productivity gains, they also introduce new job demands, such as cognitive overload, emotional stress, and blurred work-life boundaries that can affect practitioners’ well-being. We designed a research study to investigate the short and long-term well-being impacts of AIassisted tools on software practitioners, using the Job DemandsResources (JD-R) model as a guiding framework. Through a mixed-methods study combining survey and interview data, we aim to identify both beneficial and harmful patterns emerging from use of AI-assisted tools. Our findings thus far suggest both positive and negative impacts of AI-Assisted tools on software practitioner well-being. We also found significant differences in impact between practitioners with and without neurocognitive conditions, underscoring the need for inclusive tool design and thoughtful organizational policies. This research contributes theoretical insights into the evolving human-tool relationship in software development and aims to inform the ethical and sustainable integration of AI into developer workflows.

Fairness in AI models has become essential as our society increasingly becoming more dependent on AI. A biased model can have a harmful impact on marginalized communities. To address this issue, practitioners have developed fairness tools over time. To understand their practical implication, we designed an interview to curate experiences with fairness tools from academic practioners’. In this paper, we discuss insights from our first round of interviews with practitioners from academia. Although numerous fairness tools have been developed, only a few industry-developed (e.g., AIF360 and Fairlearn) are practically usable and commonly employed by practitioners due to regular maintenance & visibility. The existing toolkit landscape is primarily equipped to solely handle textual data and lacks sufficient resources for language models. Our findings thus far provide insights into one perspective on fairness tool engagement; our future efforts will investigate experiences and perspectives on fairness tool support beyond traditional models.

HCI research to date has only scratched the surface of the unique approaches racially minoritized communities take to building, designing, and using technology systems. While there has been an increase in understanding how people across racial groups create community across different platforms, there is still a lack of studies that explicitly center on how Black technologists design with and for their own communities. In this paper, we present findings from a series of semi-structured interviews with Black technologists who have used, created, or curated resources to support lived Black experiences. From their experiences, we find a multifaceted approach to design as a means of survival, to stay connected, for cultural significance, and to bask in celebratory joy. Further, we provide considerations that emphasize the need for centering lived Black experiences in design and share approaches that can empower the broader research community to conduct further inquiries into design focused on those in the margins.

Automated program repair, also known as APR, is an approach for automatically repairing software faults. There is a large amount of research on automated program repair, but very little offers in-depth insights into how practitioners think about and employ APR in practice. To learn more about practitioners' perspectives and experiences with current APR tools and techniques, we administered a survey, which received valid responses from 331 software practitioners. We analyzed survey responses to gain insights regarding factors that correlate with APR awareness, experience, and use. We established a strong correlation between APR awareness and tool use and attributes including job position, company size, total coding experience, and preferred language of software practitioners. We also found that practitioners are using other forms of support, such as co-workers and ChatGPT, more frequently than APR tools when fixing software defects. We learned about the drawbacks that practitioners encounter while utilizing existing APR tools and the impact that each drawback has on their practice. Our findings provide implications for research and practice centered on development, adoption, and use of APR.

Consistent, diverse, and quality contributions are essential to the sustainability of the open source community. Therefore, it is important that there is infrastructure for effectively onboarding and retaining diverse newcomers to open source software projects. Most often, open source projects rely on onboarding documentation to support newcomers in making their first contributions. Unfortunately, prior studies suggest that information overload from available documentation, along with the predominantly monolingual nature of repositories, can have negative effects on the newcomer experiences and onboarding process. This, coupled with the effort involved in creating and maintaining onboarding documentation, suggest a need for support in creating more accessible documentation. Large language models (LLMs) have shown great potential in providing text transformation support in other domains, and even shown promise in simplifying or generating other kinds of computing artifacts, such as source code and technical documentation. We contend that LLMs can also help make software onboarding documentation more accessible, thereby reducing the potential for information overload. Using ChatGPT (GPT-3.5 Turbo) and Gemini Pro as case studies, we assessed the effectiveness of LLMs for simplifying software onboarding documentation, one method for reducing information overload. We discuss a broader vision for using LLMs to support the creation of more accessible documentation and outline future research directions toward this vision.

Survey research, while common, has known challenges and limitations. In this paper, we discuss the challenges and concerns of conducting research using online survey in the age of widespread development and use of generative AI technologies. We base our discussion on our recent experiences conducting survey research to better understand software practitioners’ knowledge of and experience with automated program repair (APR). In our efforts, we encountered both anticipated and unexpected challenges, much of which stemmed from advancements in AI-assisted automation (e.g., generative AI). For example, we found that many of the open ended responses provided were likely generated by AI. Based on our experiences, we outline how we mitigated or handled these challenges and discuss opportunities for improving survey design and administration to ensure high quality research and outcomes in the age of advanced and widely available AI technologies.

Psychoactive substances, which influence the brain to alter perceptions and moods, have the potential to have positive and negative effects on critical software engineering tasks. They are widely used in software, but that use is not well understood. We present the results of the first qualitative investigation of the experiences of, and challenges faced by, psychoactive substance users in professional software communities. We conduct a the-matic analysis of hour-long interviews with 26 professional pro-grammers who use psychoactive substances at work. Our results provide insight into individual motivations and impacts, including mental health and the relationships between various substances and productivity. Our findings elaborate on socialization effects, including soft skills, stigma, and remote work. The analysis also highlights implications for organizational policy, including positive and negative impacts on recruitment and retention. By exploring individual usage motivations, social and cultural ramifications, and organizational policy, we demonstrate how substance use can permeate all levels of software development.

Biased machine learning (ML) models in real-world applications, such as healthcare and criminal justice, have led to significant societal harm. Several ML fairness tools promise to help create less biased ML models. However, little effort has been made to help practitioners or researchers reason about the growing space of available tools. In this work, we evaluated and categorized 14 existing fairness tools based on their features and usage workflows to develop a practical taxonomy of machine learning fairness tools. Our resulting taxonomy of fairness tools suggests the availability of an array of fairness tools, including tools that require little coding or allow for customization or extension. By structuring and organizing the landscape of fairness tools, we can identify gaps and explore options for supporting researchers and practitioners, such as automated tools for finding and selecting fairness tools.

Experience and knowledge are essential and adequate conditions to assess developer expertise. In practice, while various mechanisms are used for assessing developer expertise, research has shown that existing mechanisms may not support the holistic evaluation of developer expertise or contain the complexity behind the development of that expertise (e.g., experience with individual APIs and programming concepts). Prior research has shown that developer contributions on platforms like GitHub and StackOverflow can provide personal insights into broader developer expertise. This paper explores the potential for providing more granular assessments of developer expertise based on their interactions in these online programming communities by empirically analyzing developer contributions on StackOverflow and GitHub related to Python APIs. We found that techniques from prior work scale to support the automated ranking of developers based on their experience with a given Python API. Our findings suggest that we can use syntax patterns in written language and code to detect and rank developers' low-level technical expertise accurately.

As the field of computing has advanced over the years, how we engage and innovate as a society has as well. From computing practitioners that develop the technologies to end-users, the internet has become a central hub for our daily activities. There have been numerous efforts to ensure that the content in online social communities like Twitter are inclusive. However, despite the history of harmful terminology use in computing, little work explicitly attempts to provide support for detecting and correcting offensive language in online programming communities like GitHub. This paper introduces the Hate Detector, a web application for detecting and correcting harmful terminology in computing artifacts. The current implementation connects with GitHub and uses existing resources and corpora to support the analysis of repository markdown files. Our ongoing efforts aim to evaluate and scale the Hate Detector across and beyond GitHub projects to further promote inclusive practices in online programming communities.

Both research and practice are increasingly using data to drive innovation. While innovative technology has the potential to better our day to day lives, we too often see examples of innovative technology doing more harm than good. These elevate concerns around ethics and equity, and point to the need for interventions that support intersectional analyses that prevent harm to marginalized communities. Traditional development tools narrowly focus on solving technical challenges, without helping practitioners make explicit connections to the potential impact their contributions may have, which studies have shown is an important factor when changing behavior in software teams. In this showpiece, we introduce Quintessence, a novel proof-of-concept tool that supports reflexivity, as it pertains to intersectional identities, in data analysis and modeling. In the future, we plan to work with practitioners to improve Quintessence for practical use, including a deeper dive into effective reflexive support and the addition of more analytic support in the modeling phases.

The physical work environment of software engineers can have various effects on their satisfaction and the ability to get the work done. To better understand the factors of the environment that affect productivity and satisfaction of software engineers, we explored different work environments at Microsoft. We used a mixed-methods, multiple stage research design with a total of 1,159 participants: two surveys with 297 and 843 responses respectively and interviews with 19 employees. We found several factors that were considered as important for work environments: personalization, social norms and signals, room composition and atmosphere, work-related environment affordances, work area and furniture, and productivity strategies. We built statistical models for satisfaction with the work environment and perceived productivity of software engineers and compared them to models for employees in the Program Management, IT Operations, Marketing, and Business Program & Operations disciplines. In the satisfaction models, the ability to work privately with no interruptions and the ability to communicate with the team and leads were important factors among all disciplines. In the productivity models, the overall satisfaction with the work environment and the ability to work privately with no interruptions were important factors among all disciplines. For software engineers, another important factor for perceived productivity was the ability to communicate with the team and leads. We found that private offices were linked to higher perceived productivity across all disciplines.

Understanding the root cause of a defect is critical to isolating and repairing buggy behavior. We present Causal Testing, a new method of root-cause analysis that relies on the theory of counterfactual causality to identify a set of executions that likely hold key causal information necessary to understand and repair buggy behavior. Using the Defects4J benchmark, we find that Causal Testing could be applied to 71% of real-world defects, and for 77% of those, it can help developers identify the root cause of the defect. A controlled experiment with 37 developers shows that Causal Testing improves participants' ability to identify the cause of the defect from 80% of the time with standard testing tools to 86% of the time with Causal Testing. The participants report that Causal Testing provides useful information they cannot get using tools such as JUnit. Holmes, our prototype, open-source Eclipse plugin implementation of Causal Testing, is available at http://holmes.cs.umass.edu/.

While using security tools to resolve security defects, software developers must apply considerable effort. Success depends on a developer's ability to interact with tools, ask the right questions, and make strategic decisions. To build better security tools and subsequently help developers resolve defects more accurately and efficiently, we studied the defect resolution process-from the questions developers ask to their strategies for answering them. In this paper, we report on an exploratory study with novice and experienced software developers. We equipped them with Find Security Bugs, a security-oriented static analysis tool, and observed their interactions with security vulnerabilities in an open-source system that they had previously contributed to. We found that they asked questions not only about security vulnerabilities, associated attacks, and fixes, but also questions about the software itself, the social ecosystem that built the software, and related resources and tools. We describe the strategic successes and failures we observed and how future tools can leverage our findings to encourage better strategies.

Bias in decisions made by modern software is becoming a common and serious problem. We present Themis, an automated test suite generator to measure two types of discrimination, including causal relationships between sensitive inputs and program behavior. We explain how Themis can measure discrimination and aid its debugging, describe a set of optimizations Themis uses to reduce test suite size, and demonstrate Themis' effectiveness on open-source software. Themis is open-source and all our evaluation data are available at http://fairness.cs.umass.edu/. See a video of Themis in action: https://youtu.be/brB8wkaUesY

Peer code reviews are important for giving and receiving peer feedback, but the code review process is time consuming. Static analysis tools can help reduce reviewer effort by catching common mistakes prior to peer code review. Ideally, contributors would use static analysis tools prior to pull request submission so common mistakes could be addressed first, before invoking the reviewer. To explore the potential efficiency gains for peer reviewers, we explore the overlap between reviewer comments on pull requests and warnings from the PMD static analysis tool. In an empirical study of 274 comments from 92 pull requests on GitHub, we observed that PMD overlapped with nearly 16% of the reviewer comments, indicating a time benefit to the reviewer if static analyzers would have been used prior to pull request submission. Using the non-overlapping set of comments, we identify four additional rules that, if implemented, could further reduce reviewer effort.

Quick Fixes as implemented by IDEs today prioritize the speed of applying the fix as a primary criteria for success. In this paper, we argue that when tools over-optimize this criteria, such tools neglect other dimensions that are important to successfully applying a fix, such as being able to explore the design space of multiple fixes. This is especially true in cases where a fix only partially implements the intention of the developer. In this paper, we implement an extension to the FindBugs defect finding tool, called FixBugs, an interactive resolution approach within the Eclipse development environment that prioritizes other design criteria to the successful application of suggested fixes. Our empirical evaluation method of 12 developers suggests that FixBugs enables developers to explore alternative designs and balances the benefits of manual fixing with automated fixing, without having to compromise in either effectiveness or efficiency. Our analytic evaluation method with six usability experts identified trade-offs between FixBugs and Quick Fix, and suggests ways in which FixBugs and Quick Fix can offer complementary capabilities to better support developers.

Program analysis tools use notifications to communicate with developers, but previous research suggests that developers encounter challenges that impede this communication. This paper describes a qualitative study that identifies 10 kinds of challenges that cause notifications to miscommunicate with developers. Our resulting notification communication theory reveals that many challenges span multiple tools and multiple levels of developer experience. Our results suggest that, for example, future tools that model developer experience could improve communication and help developers build more accurate mental models.

Grounded theory is an important research method in empirical software engineering, but it is also time consuming, tedious, and complex. This makes it difficult for researchers to assess if threats, such as missing themes or sample bias, have inadvertently materialized. To better assess such threats, our new idea is that we can automatically extract knowledge from social news websites, such as Hacker News, to easily replicate existing grounded theory research --- and then compare the results. We conduct a replication study on static analysis tool adoption using Hacker News. We confirm that even a basic replication and analysis using social news websites can offer additional insights to existing themes in studies, while also identifying new themes. For example, we identified that security was not a theme discovered in the original study on tool adoption. As a long-term vision, we consider techniques from the discipline of knowledge discovery to make this replication process more automatic.

Even though different developers have varying levels of expertise, the tools in one developer's integrated development environment (IDE) behave the same as the tools in every other developers' IDE. In this paper, we propose the idea of automatically customizing development tools by modeling what a developer knows about software concepts. We then sketch three such ``bespoke'' tools and describe how development data can be used to infer what a developer knows about relevant concepts. Finally, we describe our ongoing efforts to make bespoke program analysis tools that customize their notifications to the developer using them.

Security tools can help developers answer questions about potential vulnerabilities in their code. A better understanding of the types of questions asked by developers may help toolsmiths design more effective tools. In this paper, we describe how we collected and categorized these questions by conducting an exploratory study with novice and experienced software developers. We equipped them with Find Security Bugs, a security-oriented static analysis tool, and observed their interactions with security vulnerabilities in an open-source system that they had previously contributed to. We found that they asked questions not only about security vulnerabilities, associated attacks, and fixes, but also questions about the software itself, the social ecosystem that built the software, and related resources and tools. For example, when participants asked questions about the source of tainted data, their tools forced them to make imperfect tradeoffs between systematic and ad hoc program navigation strategies.

There are a variety of tools available for developers to use in their IDEs. Research suggests, however, developers may not use these tools due to difficulty interpreting the output. My research explores the possibility of creating frameworks that enable more individualized program analysis tool notifications for increased usability. I propose creating models that represent what developers know about programming concepts using exisitng developer data. These models could then be used to inform tools on notifications the developer may or may not understand.

Program analysis tools can help developers produce high quality code by automating time-consuming tasks such as error-finding. Research has shown, however, that these tools are often not used by developers. Results from studies I have conducted provide insights into the difficulties programmers may encounter when using program analysis tools, leading to lower productivity and desire to use them. Based on these findings, my dissertation will research improving program analysis tools by enhancing tool intelligence with a programmer model that adapts notifications to programmers based on their experience with the concepts relevant to the notification.

Program analysis tools are available to make programmers' jobs easier by automating tasks that would otherwise be performed manually or not at all. To communicate with the programmer, these tools use notifications, which may be textual, visual, or a combination of both. Research has shown that these notifications need improvement in two areas: expressiveness and scalability. In the research described here, I begin an investigation into the expressiveness and scalability of existing program analysis tools and potential improvements in expressiveness and scalability in and across these tools.

Using static analysis tools for automating code inspections can be beneficial for software engineers. Such tools can make finding bugs, or software defects, faster and cheaper than manual inspections. Despite the benefits of using static analysis tools to find bugs, research suggests that these tools are underused. In this paper, we investigate why developers are not widely using static analysis tools and how current tools could potentially be improved. We conducted interviews with 20 developers and found that although all of our participants felt that use is beneficial, false positives and the way in which the warnings are presented, among other things, are barriers to use. We discuss several implications of these results, such as the need for an interactive mechanism to help developers fix defects.

Using static analysis tools for automating code inspections can be beneficial for software engineers. Despite the benefits of using static analysis tools, research suggests that these tools are underused. In this research, we propose to investigate why developers are not widely using static analysis tools and how current tools could potentially be improved to increase usage.