When a system failure is sufficiently severe, system owners may conduct a post-mortem analysis to learn from the failure and then propose ways to evolve the system in an effort to prevent similar failures in the future. Our interest in this paper is broadly in characterizing the system evolution that follows from post-mortem analysis. Specifically, we have three research questions: (1) what happened during the incident that motivated proposed system changes, (2) what parts of the system are targeted by the proposed changes, and (3) what are the intended effects of the proposed changes on system characteristics. To answer these questions, we have conducted an empirical study of 359 proposed changes, referred to as action items (AIs), from 75 public incident reports. From our analysis we have found that proposed changes are motivated by a wide variety of events experienced during the incident, including how the incident was triggered, ways the failure propagated, response related events, and the recovery of the system. We have also found that a wide variety of system parts are targeted by AIs, including system aspects that may not be considered for evolution in other contexts. Finally, we have found that AIs primarily propose to evolve performance efficiency, reliability and to a lesser degree flexibility and safety, and propose to do so by considering narrow scenarios related to the incident.

This paper presents the results of a research study related to software system failures, with the goal of understanding how we might better evolve, maintain and support software systems in production. We have qualitatively analyzed thirty incidents: fifteen collected through in depth interviews with engineers, and fifteen sampled from publicly published incident reports (generally produced as part of postmortem reviews). Our analysis focused on understanding and categorizing how failures occurred, and how they were detected, investigated and mitigated. We also captured analytic insights related to the current state of the practice and associated challenges in the form of 11 key observations. For example, we observed that failures can cascade through a system leading to major outages; and that often engineers do not understand the scaling limits of systems they are supporting until those limits are exceeded. We argue that the challenges we have identified can lead to improvements to how systems are engineered and supported.

Scientific and engineering research is heavily dependent on effective development and use of software artifacts. Many of these artifacts are produced by the scientists themselves, rather than by trained software engineers. To address the challenges in this area, a research community often referred to as "Development of Scientific Software" has emerged in the last few decades. As this research area has matured, there has been a sharp increase in the number of papers and results made available, and it has thus become important to summarize and provide an overview about those studies. Through a systematic mapping and bibliometrics study, we have reviewed 130 papers in this area. We present the results of our study in this paper. Also we have made the mapping data available on an online repository which is planned to be updated on a regular basis. The results of our study seem to suggest that many software engineering techniques and activities are being used in the development of scientific software. However, there is still a need for further exploration of the usefulness of specific software engineering techniques (e.g., regarding software maintenance, evolution, refactoring, re(v)-engineering, process and project management) in the scientific context. It is hoped that this article will help (new) researchers get an overview of the research space and help them to understand the trends in the area.

Version control branching allows an organization to parallelize its development efforts. Releasing a software system developed in this manner requires release managers, and other project stakeholders, to make decisions about how to integrate the branched work. This group decision-making process becomes very complex in the case of large-scale parallel development. To better understand the information needs of release managers in this context, we conducted an interview study at a large software company. Our analysis of the interviews provides a view into how release managers make integration decisions, organized around ten key factors. Based on these factors, we discuss specific information needs for release managers and how the needs can be met in future work.

Programmers learning how to use an API or a programming language often rely on code examples to support their learning activities. However, what makes for an effective ode example remains an open question. Finding the haracteristics of the effective examples is essential in improving the appropriateness of these learning aids. To help answer this question we have onducted a qualitative analysis of the questions and answers posted to a programming Q&A web site called StackOverflow. On StackOverflow answers can be voted on, indicating which answers were found helpful by users of the site. By analyzing these well-received answers we identified haracteristics of effective examples. We found that the explanations acompanying examples are as important as the examples themselves. Our findings have implications for the way the API documentation and example set should be developed and evolved as well as the design of the tools assisting the development of these materials.

It has been claimed that crosscutting concerns are pervasive and problematic, leading to difficulties in program comprehension, evolution, and long-term design degradation. To consider whether this theory bears out, we examine the patch history of the Mozilla project over a period of a decade to consider whether crosscutting concerns exist therein and whether we can see evidence of problems arising from them. Mozilla is an interesting case, due to its longevity; size; polylingual nature; and use of a patch review process, which maintains strong connections between issue reports and the patches that are intended to address each. We perform several statistical analyses of the over 200,000 patches submitted to address over 90,000 issues reported in this time period. We find that 90% of patches show little or no evidence of scattering, that the scattering of a patch tends to decrease slightly upon review on average, and that the system shows at worst a slow increase of average scattering over time.

For many software projects, bug tracking systems play a central role in supporting collaboration between the developers and the users of the software. To better understand this collaboration and how tool support can be improved, we have quantitatively and qualitatively analysed the questions asked in a sample of 600 bug reports from the MOZILLA and ECLIPSE projects. We categorised the questions and analysed response rates and times by category and project. Our results show that the role of users goes beyond simply reporting bugs: their active and ongoing participation is important for making progress on the bugs they report. Based on the results, we suggest four ways in which bug tracking systems can be improved.

Global and distributed software development increases the need to find and connect developers with relevant expertise. Existing recommendation systems typically model expertise based on file changes (implementation expertise). While these approaches have shown success, they require a substantial recorded history of development for a project. Previously, we have proposed the concept of usage expertise, i.e., expertise manifested through the act of calling (using) a method. In this paper, we assess the viability of this concept by evaluating expert recommendations for the ASPECTJ and ECLIPSE projects. We find that both usage and implementation expertise have comparable levels of accuracy, which suggests that usage expertise may be used as a substitute measure. We also find a notable overlap of method calls across both projects, which suggests that usage expertise can be leveraged to recommend experts from different projects and thus for projects with little or no history.

Source code search is an important activity for programmers working on a change task to a software system. As part of a larger project to improve tool support for finding information in source code, we conducted a formative study in which programmers were asked to perform corrective tasks to a system they were initially unfamiliar with. Our analysis focused specifically on how programmers decide what to search for, and how they decide which results are relevant to their task. Based on our analysis, we present five observations about our participant's approach to finding information and some of the challenges they faced. We also discuss the implications these observations have for the design of source code search tools.

Previous research has identified several challenges that programmers face in answering questions about a code base. To explore ways to overcome those challenges, we have developed a prototypical programming tool called the Code Set tool. The tool allows programmers to work with sets of source code entities in novel ways, and this support allows programmers to more directly answer a range of questions about a code base. The tool also provides important contextual information for understanding the answers to those questions. The main focus of this paper is on the design and implementation of the Code Set tool. We also report on a small user study that serves as a first step in evaluating the effectiveness of the Code Set tool.

Little is known about the specific kinds of questions programmers ask when evolving a code base and how well existing tools support those questions. To better support the activity of programming, answers are needed to three broad research questions: 1) What does a programmer need to know about a code base when evolving a software system? 2) How does a programmer go about finding that information? 3) How well do existing tools support programmers in answering those questions? We undertook two qualitative studies of programmers performing change tasks to provide answers to these questions. In this paper, we report on an analysis of the data from these two user studies. This paper makes three key contributions. The first contribution is a catalog of 44 types of questions programmers ask during software evolution tasks. The second contribution is a description of the observed behavior around answering those questions. The third contribution is a description of how existing deployed and proposed tools do, and do not, support answering programmers' questions.

Software maintenance is a highly collaborative activity whose social context is rarely addressed. To explore this context, we conducted an ethnographic study at a large technology company involving participant observation with software engineers. Thirty-six participants (nine managers and twenty-seven software engineers) at the company participated in semi-formal interviews, while six months of participant observation produced insights about the work practice. The paper presents nine key observations that demonstrate the social context of maintenance activities. These observations provide a description of how work was divided between groups, the social dependencies that exist between groups, challenges in managing branches, the role of small projects, issues of making cross-group changes to source code, how dependencies are identified, problems of confidence in testing, and the impacts of working across widely different time-zones. The paper also highlights implications these observations have for software engineering research and practice.

Though many tools are available to help programmers working on change tasks, and several studies have been conducted to understand how programmers comprehend systems, little is known about the specific kinds of questions programmers ask when evolving a code base. To fill this gap we conducted two qualitative studies of programmers performing change tasks to medium to large sized programs. One study involved newcomers working on assigned change tasks to a medium-sized code base. The other study involved industrial programmers working on their own change tasks on code with which they had experience. The focus of our analysis has been on what information a programmer needs to know about a code base while performing a change task and also on howthey go about discovering that information. Based on this analysis we catalog and categorize 44 different kinds of questions asked by our participants. We also describe important context for how those questions were answered by our participants, including their use of tools.