In large-scale software systems, the majority of defective files are architecturally connected, and the architecture connections usually exhibit design flaws, which are associated with higher change-proneness among files and higher maintenance costs. As software evolves with bug fixes, new features, or improvements, unresolved architecture design flaws can contribute to maintenance difficulties. The impact on effort due to architecture design flaws has been difficult to quantify and justify. In this paper, we conducted a case study where we identified flawed architecture relations and quantified their effects on maintenance activities. Using data from this project's source code and revision history, we identified file groups where files are architecturally connected and participated in flawed architecture designs, quantified the maintenance activities in the detected files, and assessed the penalty related to these files.

Commercial software development projects frequently build code on third-party components. However, depending on third-party code requires that projects keep current with the latest version of each component. When projects do not stay current, they begin to incur a form of technical debt where API calls that have been deprecated remain in the code base. At some point, projects must upgrade the third-party component to remain on a supported version of the component. Then the projects incur the cost of paying down the debt that was built up over time. The model described herein intends to estimate the cost of paying down the debt for aging third-party components.

In this paper, we report our experiences of applying three complementary automated software architecture analysis techniques, supported by a tool suite, called DV8, to 8 industrial projects within a large company. DV8 includes two state-of-the-art architecture-level maintainability metrics—Decoupling Level and Propagation Cost, an architecture flaw detection tool, and an architecture root detection tool. We collected development process data from the project teams as input to these tools, reported the results back to the practitioners, and followed up with telephone conferences and interviews. Our experiences revealed that the metrics scores, quantitative debt analysis, and architecture flaw visualization can effectively bridge the gap between management and development, help them decide if, when, and where to refactor. In particular, the metrics scores, compared against industrial benchmarks, faithfully reflected the practitioners’ intuitions about the maintainability of their projects, and enabled them to better understand the maintainability relative to other projects internal to their company, and to other industrial products. The automatically detected architecture flaws and roots enabled the practitioners to precisely pinpoint, visualize, and quantify the “hotspots" within the systems that are responsible for high maintenance costs. Except for the two smallest projects for which both architecture metrics indicated high maintainability, all other projects are planning or have already begun refactorings to address the problems detected by our analyses. We are working on further automating the tool chain, and transforming the analysis suite into deployable services accessible by all projects within the company.

Due to the high number and cost of interruptions at work, several approaches have been suggested to reduce this cost for knowledge workers. These approaches predominantly focus either on a manual and physical indicator, such as headphones or a closed office door, or on the automatic measure of a worker's interruptibilty in combination with a computer-based indicator. Little is known about the combination of a physical indicator with an automatic interruptibility measure and its long-term impact in the workplace. In our research, we developed the FlowLight, that combines a physical traffic-light like LED with an automatic interruptibility measure based on computer interaction data. In a large-scale and long-term field study with 449 participants from 12 countries, we found, amongst other results, that the FlowLight reduced the interruptions of participants by 46%, increased their awareness on the potential disruptiveness of interruptions and most participants never stopped using it.

It is a fact that a lot of software is written by people without a formal education in software engineering. As an example, material scientists often capture their knowledge in the form of simulation software that contains sophisticated algorithms representing complex physical concepts. Since software engineering is typically not a core skill of these scientists, there is a risk that their software becomes unmaintainable once it reaches a substantial size or structural complexity. This paper reports on a case study in which software engineers consulted magnetics researchers in refactoring their simulation software. This software had grown to 30 kloc of Java and was considered unmaintainable by the stakeholders of the research project. The case study describes the process of refactoring a system under the guidance of a software engineer with results supported by static analysis and software metrics. It shows how software engineers evaluated and selected refactorings to apply to the system using their expert judgment with input from static analysis tools and discusses the outcome of refactoring as evaluated by code owners and reported via static analysis metrics.

This paper describes a case study of using developer activity logs as indicators of a program comprehension effort by analyzing temporal sequences of developer actions (e.g., navigation and edit actions). We analyze developer activity data spanning 109,065 events and 69 hours of work on a medium-sized industrial application. We examine potential correlations between different measures of developer activity, code change metrics and code smells to gain insight into questions that could direct future technical debt interest estimation. To gain more insights into the data, we follow our analysis with commit message analysis and a developer interview. Our results indicate that developer activity as an estimate of program comprehension effort is correlated with both change proneness and static metrics for code smells.

Past studies show that developers who navigate code in a structural manner complete tasks faster and more correctly than those whose behavior is more opportunistic. The goal of this work is to move professional developers towards more effective program comprehension and maintenance habits by providing an approach that fosters structural code navigation. To this end, we created a Visual Studio plugin called Prodet that integrates an always-on navigable visualization of the most contextually relevant portions of the call graph. We evaluated the effectiveness of our approach by deploying it in a six week field study with professional software developers. The study results show a statistically significant increase in developers' use of structural navigation after installing Prodet. The results also show that developers continuously used the filtered and navigable call graph over the three week period in which it was deployed in production. These results indicate the maturity and value of our approach to increase developers' effectiveness in a practical and professional environment.

To complete a change task, software developers spend a substantial amount of time navigating code to understand the relevant parts. During this investigation phase, they implicitly build context models of the elements and relations that are relevant to the task. Through an exploratory study with twelve developers completing change tasks in three open source systems, we identified important characteristics of these context models and how they are created. In a second empirical analysis, we further examined our findings on data collected from eighty developers working on a variety of change tasks on open and closed source projects. Our studies uncovered, amongst other results, that code context models are highly connected, structurally and lexically, that developers start tasks using a combination of search and navigation and that code navigation varies substantially across developers. Based on these findings we identify and discuss design requirements to better support developers in the initial creation of code context models. We believe this work represents a substantial step in better understanding developers' code navigation and providing better tool support that will reduce time and effort needed for change tasks.

Software engineering researchers develop great techniques consisting of practices and tools that improve efficiency and quality of software development. Prior work evaluates developers' use of techniques such as Test-Driven-Development and refactoring by measuring actions in the development environment. What we still lack is a method to communicate effectively and motivate developers to adopt best practices and tools. This work proposes a game-like system to motivate adoption while continuously measuring developers' use of more efficient development techniques.

Commercial software development teams have limited time available to focus on improvements to their software. These teams need a way to quickly identify areas of the source code that would benefit from improvement, as well as quantifiable data to defend the selected improvements to management. Past research has shown that mining configuration management systems for change information can be useful in determining faulty areas of the code. We present a tool named Code Hot Spot, which mines change records out of Microsoft's TFS configuration management system and creates a report of hot spots. Hot spots are contiguous areas of the code that have higher values of metrics that are indicators of faulty code. We present a study where we use this tool to study projects at ABB to determine areas that need improvement. The resulting data have been used to prioritize areas for additional code reviews and unit testing, as well as identifying change prone areas in need of refactoring.

Corrective maintenance activities are a common cause of schedule delays in software development projects. Organizations frequently fail to properly plan the effort required to fix field defects. This study aims to provide relevant guidance to software development organizations on planning for these corrective maintenance activities by correlating metrics that are available prior to release with parameters of the selected software reliability model that has historically best fit the product's field defect data. Many organizations do not have adequate historical data, especially historical deployment and field usage information. The study identifies a set of metrics calculable from available data to approximate these missing predictor categories. Two key metrics estimable prior to release surfaced with potentially useful correlations, (1) the number of periods until the next release and (2) the peak deployment percentage. Finally, these metrics were used in a case study to plan corrective maintenance efforts on current development releases.

Software fails and fixing it is expensive. Research in failure prediction has been highly successful at modeling software failures. Few models, however, consider the key cause of failures in software: people. Understanding the structure of developer collaboration could explain a lot about the reliability of the final product. We examine this collaboration structure with the developer network derived from code churn information that can predict failures at the file level. We conducted a case study involving a mature Nortel networking product of over three million lines of code. Failure prediction models were developed using test and post-release failure data from two releases, then validated against a subsequent release. One model's prioritization revealed 58% of the failures in 20% of the files compared with the optimal prioritization that would have found 61% in 20% of the files, indicating that a significant correlation exists between file-based developer network metrics and failures.

No single software fault-detection technique is capable of addressing all fault-detection concerns. Similarly to software reviews and testing, static analysis tools (or automated static analysis) can be used to remove defects prior to release of a software product. To determine to what extent automated static analysis can help in the economic production of a high-quality product, we have analyzed static analysis faults and test and customer-reported failures for three large-scale industrial software systems developed at Nortel Networks. The data indicate that automated static analysis is an affordable means of software fault detection. Using the orthogonal defect classification scheme, we found that automated static analysis is effective at identifying assignment and checking faults, allowing the later software production phases to focus on more complex, functional, and algorithmic faults. A majority of the defects found by automated static analysis appear to be produced by a few key types of programmer errors and some of these types have the potential to cause security vulnerabilities. Statistical analysis results indicate the number of automated static analysis faults can be effective for identifying problem modules. Our results indicate static analysis tools are complementary to other fault-detection techniques for the economic production of a high-quality software product.

To maximize business value, commercial software organizations need to apply software reliability engineering (SRE) using a distributed model where key practices are performed by different roles in the organization. SRE practitioners need to understand the landscape of software development in order to define a program that is effective at moving the organization towards higher reliability

Software inspection has been shown to be an effective defect removal practice, leading to higher quality software with lower field failures. Automated software inspection tools are emerging for identifying a subset of defects in a less labor-intensive manner than manual inspection. This paper investigates the use of automated inspection for a large-scale industrial software system at Nortel Networks. We propose and utilize a defect classification scheme for enumerating the types of defects that can be identified by automated inspections. Additionally, we demonstrate that automated code inspection faults can be used as efficient predictors of field failures and are effective for identifying fault-prone modules.