The paper entitled “Qualitative Methods in Empirical Studies of Software Engineering” by Carolyn Seaman was published in TSE in 1999. It has been chosen as one of the most influential papers from the third decade of TSE's 50 years history. In this retrospective, the authors discuss the evolution of the use of qualitative methods in software engineering research, the impact it's had on research and practice, and reflections on what is coming and deserves attention.

Context . Technical debt (TD) prevention allows software practitioners to apply practices to avoid potential TD items in their projects. Aims . To uncover and prioritize, from the point of view of software practitioners, the practices that could be used to avoid TD items, the relations between these practices and the causes of TD, and the practice avoidance reasons (PARs) that could explain the failure to prevent TD. Method . We analyze data collected from six replications of a global industrial family of surveys on TD, totaling 653 answers. We also conducted a follow up survey to understand the importance level of analyzed data. Results . Most practitioners indicated that TD could be prevented, revealing 89 prevention practices and 23 PARs for explaining the failure to prevent TD. The article identifies statistically significant relationships between preventive practices and certain causes of TD. Further, it prioritizes the list of practices, PARs, and relationships regarding their level of importance for TD prevention based on the opinion of software practitioners. Conclusion . This work organizes TD prevention practices and PARs in a conceptual map and the relationships between practices and causes of TD in a Sankey diagram to help the visualization of the body of knowledge reported in this study.

Context: Task interdependence is a work design factor that expresses the mutual dependency between tasks that compose a whole work. In software development, task interdependencies are created by the technical dependencies between the components of the software system and by how the development tasks are allocated to individuals in a teamwork context. Despite its importance for individual and team effectiveness, we still do not have studies about how software engineers perceive task interdependence in practice. Goal: To understand the perceptions of software engineers about the interdependence in their work and how these perceptions interact with other human and technical factors in the development process. Method: We performed an exploratory qualitative case study of a single software development team in a Brazilian software company that developed solutions for the financial market. We interviewed all 10 team members and used standard coding techniques from qualitative research to code, categorize, and synthesize data. Results: Individuals are consistent in their understanding of task interdependence and how it happens in practice. However, there are asymmetries between the individual perceptions in an interdependence relationship, which seem to exacerbate expressed feelings of anxiety and dissatisfaction. Conclusion: Our results suggest that the perception of task interdependence in software development is often not symmetrical with potential negative effects on emotional states that are related to motivation and satisfaction in the workplace.

The advancement of artificial intelligence and the implementation of machine learning capabilities in programming languages such as Python, along with cloud services, allow researchers to apply methods to cluster and predict behaviors and patterns in software engineering data. On the other hand, these methods need a large amount of data in order to work with high accuracy in different contexts. This paper introduces Sonarlizer Xplorer: a tool that captures a large number of technical debt items and code metrics from public GitHub projects. Sonarlizer Xplorer is composed of two sub-tools. The first is Github Xplorer, responsible for mining public Github repositories from an initial project. The second is Sonarlizer, responsible for taking projects and analyzing them using SonarQube. We used the tool over four months, collecting technical debt items and code metrics on almost 46,000 public Java projects. In addition, we mined over 57 million repositories and 4 million users.

Background: The prioritization of technical debt is an essential task in managing software projects because, with current analysis tools, it is possible to find thousands of technical debt items in the software that would take months or even years to be fully paid. Aims: In this study, we aim to understand which criteria software developers use to prioritize code technical debt in real software projects. Methods: We performed a survey to collect data from open-source software projects in order to reach a large and diverse set of experiences. We analyzed the data using Straussian Grounded Theory techniques: open coding, axial coding, and selective coding. Results: We grouped the criteria into 15 categories and divided them into 2 super-categories related to paying off the technical debt and 3 related to not paying it. Conclusions: When participants decided to pay off technical debt, they wanted to do it soon. However, when they decided not to pay it, it is often because the debt occurred intentionally due to a project decision. Also, participants using similar criteria for their decisions tended to choose similar priority levels for those decisions. Finally, we observed that each software project needs to tailor the rules used to identify code technical debt to their project context.

Context: Technical debt (TD) is a challenge for companies who develop software on which their critical operations depend. To properly manage TD, it is necessary to make it visible to the different stakeholders involved to support informed decisions. Objective: To validate a TD visualization approach based on hybrid Sankey diagrams that makes the TD visible by showing (a) technical and business aspects, and (b) the flow of value and TD impacts. This approach regards visualizations as boundary objects. Method: We performed a multi-case study in a large multi-industry state-owned company. The objective was to validate the effectiveness of such visualizations and to explore their possible uses in TD management. We first used a retrospective case study on a TD decision-making scenario and, later, visualization usage scenarios using focus groups to evaluate its usefulness. Results: The results suggest that the proposed approach: (a) provides a structured process for systematic TD visualization to help the decision-making process; (b) enables the communication at knowledge boundaries between stakeholders to make informed decisions; (c) uses flow representations that are important for assessing the impact in multiple functional areas; and (d) enables documentation and reuse. Conclusion: The study results suggest that TD decision-making events can benefit from using our TD visualizations based on hybrid Sankey diagrams as boundary objects to portray the impact of TD in business, services, and technical aspects.

Context: Knowing the effects of technical debt (TD) can support software development teams in the prioritization of TD items to pay off. However, little is known about the relations between the effects of TD and TD payment practices. Having this knowledge can provide valuable information for decision making about which payment practice can be applied given the presence of specific effects of TD. Aims: To investigate, from the point of view of software practitioners, (i) which TD payment practices have been used when certain effects of the presence of debt are felt in software projects and (ii) the reasons for not paying debt items despite the effects they are causing to the project. Method: We analyze quantitatively and qualitatively data collected from a survey with 432 practitioners across four countries. Results: Among the identified relations, the practice "code refactoring" is commonly used to pay debt items off when the effects "delivery delay" and "rework" are felt in software projects. On the other hand, when practitioners face the TD effects "low external quality" and "delivery delay", ，they commonly justify the non- payment of the debt items indicating the need of "focusing on short term goals". Conclusion: We organize the relationship between TD effects, and payment practices and reasons for not eliminating debt items. All this information is structured in an alluvial diagram, which can facilitate the visualization of the identified relations.

Background: Little is known about the practices used for technical debt (TD) payment. The study of payment practices, as well as the reasons for not applying them, can help practitioners to control and manage TD items. Aims: To investigate, from the point of view of software practitioners, if TD items have been paid off in software projects, the practices that have been used to pay off TD and the reasons that hamper the implementation of these practices. Method: We analyzed - both quantitatively and qualitatively - a corpus of responses from a survey of 432 practitioners, from four countries, about the possibility of TD payment. Results: We found that, for most of the cases, TD items have not been eliminated from software projects. The main reasons for not paying off TD are lack of organizational interest, low priority on the debt, focus on short-term goals, cost, and lack of time. On the other hand, we identified that code refactoring, design refactoring, and update system documentation are the most used practices for TD payment. Practitioners also cited practices related to the prevention, prioritization, and creation of a favorable setting as part of TD payment initiatives. Conclusion: This paper summarizes the identified practices and reasons for not paying off debt items in a map. Our map reveals that the majority of payment practices are of a technical nature while the majority of reasons for not paying off debts are associated with non-technical issues.

Background: The concept of technical debt (TD) describes a phenomenon that impacts software projects and makes them difficult to manage. In recent years, various techniques and best practices in terms of TD management were proposed and although important on its own this knowledge must be complemented with a broader comprehension of what causes TD and what are the effects of TD. This paper presents a replication of the InsighTD survey-a globally distributed family of industrial surveys on causes and effects of TD-and thus amplifies the InsighTD reach and expands its knowledge base. Objective: The research presented in this paper gives insight on the state of practice and understanding of the TD concept alongside with data on causes and effects of TD in the Serbian IT industry. Method: A nation-wide survey, as a part of the InsighTD initiative, was conducted in Serbia in order to obtain feedback from software industry practitioners. Results: In total 93 practitioners from the Serbian IT industry filled out the survey. The results indicate that the concept of TD is broadly distributed, but at the same time it is not widely accepted for use (only 35% of participants had some sort of practical experiences with projects that were TD aware). The top cited causes were: deadlines, ineffective project management, lack of experience, test not performed and misconduct. On the other side the most common effects of TD were: low maintainability, increased effort, rework and low external quality. Conclusion: The research presented in this paper confirms the original study findings that deadlines are the top cited cause of TD. It also identifies new causes of TD in the context of InsighTD, with misconduct being one of the most cited ones. Regarding the effects of TD this research differs in most of the top 10 identified effects from the original study but confirms the occurrence of some effects most cited.

Context: Technical debt (TD) is a metaphor used to describe technical decisions that can give the company a benefit in the short term but possibly hurting the overall quality of the software in the long term. Objective: This study aims to characterize the current state of practices related to TD payment from the point of view of software practitioners. Method: We used a survey research method to collect and analyze - both quantitatively and qualitatively - a corpus of responses from a survey of 432 software practitioners from Colombia, Chile, Brazil, and the United States, as a part of the InsighTD project. Results: We were able to identify that refactoring (24.3%) was the main practice related to TD payment, along with improving testing (6.2%) and improve design (5.8%). Also, we identify that small-sized systems and big-sized systems, along with young systems (less than one year) tend to use more refactoring. As a part of these results, we also could identify that some practices do not eliminate the debt by itself, but support a favorable scenario for TD payment or prevention. Additionally, after comparing the three major TD types cited (code debt, test debt and design debt) we could discover an important similarity of TD payment practices between code debt and design debt. Lastly, we identified that no matter the cause leading to TD occurrence, refactoring remained the most common practice. Conclusion: Definition of practices related to TD payment is an essential activity for software development teams. Developing healthy software systems that can be maintained in the future requires that companies find the right approaches for TD payment.

Understanding TD causes can support development teams in defining actions that could be taken to prevent the occurrence of debt items. Understanding the effects of TD could aid in prioritization of TD items to pay off to minimize possible negative consequences for the project. Existing work has revealed 105 causes and 85 effects of TD, and this high number can make it difficult to make practical use of this information. Without a consolidated representation, we would need to rely on a set of tables and isolated pieces of data. In this work, we propose the use of cross-company probabilistic cause-effect diagrams to represent information about TD causes and effects. We hypothesize that such diagrams can be useful to support TD cause/effect analysis sessions and empirically investigate this issue. Results from a case study performed with 72 participants indicate that the diagrams are able to positively support the management of TD, making it easier to identify its causes and the effects of its presence. Most of the participants also agreed that, by using the proposed diagrams, they gain agility, productivity, performance, and effectiveness. Finally, 89% of the participants stated that the use of the diagrams helped them to identify causes and effects of TD that they would not have identified without their support.

Background: The presence of technical debt (TD) brings risks to a software project and makes it difficult to manage. Several TD management strategies have been proposed, but considering actions that could explicitly prevent the insertion of TD in the first place and monitor its effects is not yet a common practice. Thus, while TD management is an important topic, it is also worthwhile to understand the causes that could lead development teams to incur different types of debt as well as the effects of their presence on software projects. Aims: The objective of this work is twofold. First, we investigate the state of practice in the TD area including the status quo, the causes that lead to TD occurrence, and the effects of existing TD. Second, we present the design of InsighTD, a globally distributed family of industrial surveys on causes and effects of TD, and the results of its first execution. Method: We designed the InsighTD in joint collaboration with several TD researchers. It is designed to run as an incremental large scale study based on continuous and independent replications of the questionnaire in different countries. Results: This paper presents the first results of the first execution of the survey. In total, 107 practitioners from the Brazilian software industry answered the questionnaire. Results indicate that there is a broad familiarity with the concept of TD. Deadlines, inappropriate planning, lack of knowledge, and lack of a well-defined process are among the top 10 cited and most likely causes that lead to the occurrence of TD. On the other side, low quality, delivery delay, low maintainability, rework and financial loss are among the top 10 most commonly cited and impactful effects of TD. Conclusion: With InsighTD, we intend to reduce the problem of isolated investigations in TD that are not yet representative and, thus, build a continuous and generalizable empirical basis for understanding practical problems and challenges of TD.

Context: Several technical debt (TD) management strategies have been proposed, but considering actions that could prevent the insertion of TD in the first place is not yet a common practice. This is a point that deserves investigation because it is expected that TD prevention could be sometimes "cheaper" than TD repayment. Besides, TD prevention also helps other TD management activities, especially in catching inexperienced developers' 'not-so-good' solutions. Thus, while TD management is an important topic, it is also worthwhile to understand the motivations and factors that could lead development teams to incur different types of debt. Objective: To identify causes that lead to the occurrence of TD, investigate if these causes occur in isolation or in combination, understand if TD can be prevented, and investigate, in terms of effort, if it is better to prevent debt, or incur it and pay it off later. Method: An interview based case study was performed with software practitioners. The results were formulated based on a synthesis of the text fragments coded for each response. Results: We identified 57 causes that lead a development team to incur debt. For the majority of TD types, these causes occur in combination. It was also indicated that debt can be prevented, and it is better to work on prevention activities than to pay off debt later. Conclusion: Results allowed us to identify 57 causes that lead a development team to incur debt. These causes mostly occur in combination, not in isolation. It was also indicated that debt can be prevented, and it would be better to work on prevention activities than to pay off debt later.

In this paper, we propose a model that formalizes the role of software evolution in characterizing Technical Debt (TD) by defining a series of software product states, where each successive state represents an increased level of maintenance code churn, and thus presumably an increased level of change difficulty. We also propose a way to use these states to estimate TD principal and interest and use this information in decision making during release planning. In addition, we illustrate our model using bug report data from the Eclipse-Birt project.

The focus of this paper is an evaluation of our peer mentoring framework designed to encourage more students to seek cybersecurity career pathways through providing peer interactions. We present and compare results from two years (Spring 2016 and 2017) of interaction between students in an introductory Information Systems class (IS 300: Management of Information Systems) and an upper-level elective Cybersecurity course (IS 471: Data Analytics for Cybersecurity). Our results show a continuation of the general trend observed in the 2016 study. The students who receive peer mentoring show more interest in cybersecurity issues and careers and gain more overall knowledge throughout the semester, than those who don't. This is reflected by the results of an anonymous survey and overall grade improvements. These students show more variations regarding their choice of cybersecurity as a career compared to students who did not receive any mentoring, demonstrating that they are able to make more informed decisions. Female students exhibit more pronounced responses to peer mentoring in contrast to their male counterparts.

Background: Although there has been a significant amount of research focused on designing and conducting controlled experiments, few studies report how experienced experimental software engineering researchers actually design and conduct their studies. Aims: This study aimed to offer a practical perspective from their viewpoint regarding controlled experiment planning. Method: We collected data through semi-structured interviews from 11 researchers, and we used qualitative analysis methods from the grounded theory approach to analyze them. Result: Although the complete study presents four research questions, in this paper, we answer the first one. As a result, we present a preliminary result about what these experts actually do when they design experiments. Conclusions: This work contributes to a better understanding of the practical performance of experimental software engineering.

One criterion that can be used to compare programming languages is learnability, or the ease with which a programming language can be learned by a programmer. Learnability has many aspects as well, such as mastery, change in performance over time, and initial learnability. This work examines a proposed measure of initial learnability that is based on the amount of time that a programmer needs to transfer his/her knowledge of one programming language to the understanding of another. The study design involves presenting human participants (who are programmers) with a sample of programming code written in a language that they know, alongside a sample of programming code implementing the same operations but in a language with which they are not familiar. The participant is then asked to examine the two samples of code, and then indicate when they feel that they are ready to reflect in writing on the differences and similarities between the familiar and the unfamiliar language. The proposed measure of initial learnability is the time it takes the participant to indicate that they are ready to proceed with the comparison. Repeating this procedure many times produces a series of learnability measures, each of which is connected to a pair of programming languages, i.e. each measurement characterizes the learnability of one programming language, given a subject's familiarity with another programming language. Because this approach is novel and not similar to other approaches tried in the past, we wish to gain some assurance of its validity before a full experiment. To do this, we plan to run a pre-experiment using this study design but with natural languages instead of programming languages. The data from this pre-experiment, if the approach is valid, will show that certain natural languages are easier to learn than others, given prior familiarity with similar languages. Because the notion of “similar” natural languages is well defined and documented, we can validate our data to make sure that it makes sense and is consistent with current knowledge of natural languages.

Technical Debt is a metaphor that has, in recent years, helped developers to think about and to monitor software quality. The metaphor refers to flaws in software (usually caused by shortcuts to save time) that may affect future maintenance and evolution. We conducted an empirical study in an academic environment, with nine teams of graduate and undergraduate students during two offerings of a laboratory course on Extreme Programming (XP Lab). The teams had a comprehensive lecture about several alternative ways to identify and manage Technical Debt. We monitored the teams, performed interviews, did close observations and collected feedback. The results show that the awareness of Technical Debt influences team behavior. Team members report thinking and discussing more about software quality after becoming aware of Technical Debt in their projects.

In this paper we present a peer mentoring framework for students in an introductory Information Systems class (IS 300: Management of Information Systems) to interact with peers in an upper level elective course in cybersecurity (IS 471: Data Analytics for Cybersecurity). The purpose of this framework is to encourage more students to explore cybersecurity careers through peer led cybersecurity discussions. We discuss preliminary results from a peer interaction for the spring 2016 semester. Our initial results show an increased interest in cybersecurity for the IS 300 students. Moreover, we see that post the interactions both groups have a clearer interest and understanding of cybersecurity.

Context: Software maintenance is important to keep existing software systems functional for organizations or users that depend on that software. Goal: We aim to identify the factors, i.e., software characteristics such as code complexity, leading to maintenance problems. Method: We present a Mapping Study (MS) on controlled experiments that investigated software characteristics related to defects during maintenance. Results: The search strategy identified 78 papers, of which 9 have been included in our study, dated from 1985 to 2013, after applying our inclusion and exclusion criteria. We extracted data from these papers to identify the research methods, and the independent, dependent, blocked, and measured variables. Conclusions: Our MS results point to a weak evidence on software factors causing defects during maintenance. Stronger evidence can be developed via more controlled experiments that address multiple independent variables and hold the software objects constant.

End-user software engineers often lack the training, tools, methodologies or other supports common to professional developers. Further, how and where they go for help is unknown. This paper presents results from an interview study where end-user programmer scientists relate, via storytelling, their help seeking behaviors. Our goal is to explore ways that the software engineering community can aid this population. We conducted interviews with seven scientists and two professional developers using storytelling prompts with follow-up questions. Results show that scientists prefer their human help to be proximal both physically and socially. Our analysis suggests that providing human help with specific characteristics, or providing tools and environments that support math-centric perspectives are two potential approaches to increasing the research efficiency of end-user programmer scientists.

Building trustworthy knowledge in software engineering depends on the systematic synthesis of empirical evidence. In recent years the number of published syntheses has increased, but only a few showed high quality and scientific rigor. We performed an online survey of software engineering researchers to identify difficulties experienced when synthesizing evidence. The results confirm that the state of primary research and the low quality of reports are perceived as the most important difficulties. Respondents who were experienced in quantitative and qualitative synthesis methods claim a lack of support for selecting and applying synthesis methods. This indicates the need for identifying criteria for selecting synthesis methods, deriving recommendations, and developing more rigorous guidelines for applying them.

[Context] Software engineering teams must have a shared understanding of the system design in order to work independently but successfully integrate their code. Success also depends on the differentiated skill and experience of the team members. These issues of understanding are important to project success but difficult to investigate with current approaches. [Goal] To investigate this problem, we developed and evaluated a technique to measure the degree of shared understanding and identify areas of similarity and difference. Adapted from the Pathfinder technique for evaluating Team Mental Models, this is a quantitative analysis of paired comparisons of design concepts as understood by the team. [Method] We performed an empirical, mixed-methods pilot study of the technique with 5 student teams developing a semester long project. We used questionnaires and interviews to evaluate the effectiveness of the technique in measuring areas of similarity and difference. We also investigated the association between differences in understanding and problems during development. [Results] Our results support the ability of the technique to identify and measure areas of similarity and difference. There is limited support for the association between differences and poor project outcomes. [Conclusions] We find these pilot results encouraging. We will use them to refine the technique and plan to re-evaluate it with a professional software development team.

ABSTRACT This paper presents our research experiences studying process knowledge with qualitative and quantitative methods. Informed by related arguments, we present two hypotheses that are tested using a between subjects experimental design. The first hypothesis concerns the accuracy of software process elicitation, which we characterize as the human perception of error between description and performance of a process. The second hypothesis explores the detection of subtle differences in process articulation using latent semantic analysis, a computational technique for measuring patterns in written discourse. The results of our analyses are compared along with discussion of theoretical implications. We define a new type of process property, Coherence, which represents a measure of relatedness in process knowledge. Results also indicate that Coherence may be increased with the use of a conceptual model. Future research is guided by experimental and industrial application. Copyright © 2012 John Wiley & Sons, Ltd.

Context: The technical debt (TD) concept describes a tradeoff between short-term and long-term goals in software development. While it is highly useful as a metaphor, it has utility beyond the facilitation of discussion, to inspire a useful set of methods and tools that support the identification, measurement, monitoring, management, and payment of TD. Objective: This study focuses on the identification of TD. We evaluate human elicitation of TD and compare it to automated identification. Method: We asked a development team to identify TD items in artifacts from a software project on which they were working. We provided the participants with a TD template and a short questionnaire. In addition, we also collected the output of three tools to automatically identify TD and compared it to the results of human elicitation. Results: There is little overlap between the TD reported by different developers, so aggregation, rather than consensus, is an appropriate way to combine TD reported by multiple developers. The tools used are especially useful for identifying defect debt but cannot help in identifying many other types of debt, so involving humans in the identification process is necessary. Conclusion: We have conducted a case study that focuses on the practical identification of TD, one area that could be facilitated by tools and techniques. It contributes to the TD landscape, which depicts an understanding of relationships between different types of debt and how they are best discovered.

Background: Existing software defect classification schemes support multiple tasks, such as root cause analysis and process improvement guidance. However, existing schemes do not assist in assigning defects to a broad range of high level software goals, such as software quality characteristics like functionality, maintainability, and usability. Aim: We investigate whether a classification based on the ISO/IEC 9126 software product quality model is reliable and useful to link defects to quality aspects impacted. Method: Six different subjects, divided in two groups with respect to their expertise, classified 78 defects from an industrial web application using the ISO/IEC 9126 quality main characteristics and sub-characteristics, and a set of proposed extended guidelines. Results: The ISO/IEC 9126 model is reasonably reliable when used to classify defects, even using incomplete defect reports. Reliability and variability is better for the six high level main characteristics of the model than for the 22 subcharacteristics. Conclusions: The ISO/IEC 9126 software quality model provides a solid foundation for defect classification. We also recommend, based on the follow up qualitative analysis performed, to use more complete defect reports and tailor the quality model to the context of use.

Volatile software modules, for the purposes of this work, are defined as those that are significantly more change-prone than other modules in the same system or subsystem. There is significant literature investigating models for predicting which modules in a system will become volatile, and/or are defect-prone. Much of this work focuses on using source code-related characteristics (e.g., complexity metrics) and simple change metrics (e.g., number of past changes) as inputs to the predictive models. Our work attempts to broaden the array of factors considered in such prediction approaches. To this end, we collected data directly from development personnel about the factors they rely on to foresee what parts of a system are going to become volatile. In this paper, we describe a focus group study conducted with the development team of a small but active open source project, in which we asked this very question. The results of the focus group indicate, among other things, that a period of volatility in a particular area of the system is often predicted by a pattern characterized by inactivity in a certain area (resulting in that area becoming less mature than others), increased communication between developers regarding opportunities for improvement in that area, and then the emergence of a champion who takes the initiative to start working on those improvements. The initial changes lead to more changes (both to extend the improvements already made and to fix problems introduced), thus leading to volatility.

A significant body of knowledge concerning software inspection practice indicates that the value of inspections varies widely both within and across organizations. Inspection effectiveness and efficiency may be affected by a variety of factors such as inspection planning, the type of software, the developing organization, and many others. In the early 1990's, a governmental organization developing complex and highly critical software systems formulated heuristics for inspection planning based on best practices and their early inspection data. Since the development context at the organization has changed in some ways since the heuristics were proposed, it is important to assess whether the heuristics are still a suitable guideline to use. To investigate this question, we statistically evaluated the differences in effectiveness and efficiency between inspections that adhered to the heuristics and ones that did not. Our analysis revealed no significant difference in effectiveness or efficiency for most heuristics. We also learned that compliance with the heuristics is diminishing over time.

Uncontrollable costs, schedule overruns, and poor end product quality continue to plague the software engineering field. This research investigates software process improvement (SPI) through the application of knowledge management (KM) at the software project level. A pilot study was conducted to investigate what types of obstacles to knowledge flow exist within a software development project, as well as the potential influence on SPI of their mitigation or removal. The KM technique of “knowledge mapping” was used as a research technique to characterize knowledge flow. Results show that such mitigation or removal was acknowledged by project team members as having the potential for lowering project labor cost, improving schedule adherence, and enhancing final product quality.

Background: Automatic static analysis (ASA) tools examine source code to discover “issues”, i.e. code patterns that are symptoms of bad programming practices and that can lead to defective behavior. Studies in the literature have shown that these tools find defects earlier than other verification activities, but they produce a substantial number of false positive warnings. For this reason, an alternative approach is to use the set of ASA issues to identify defect prone files and components rather than focusing on the individual issues. Aim: We conducted an exploratory study to investigate whether ASA issues can be used as early indicators of faulty files and components and, for the first time, whether they point to a decay of specific software quality attributes, such as maintainability or functionality. Our aim is to understand the critical parameters and feasibility of such an approach to feed into future research on more specific quality and defect prediction models. Method: We analyzed an industrial C# web application using the Resharper ASA tool and explored if significant correlations exist in such a data set. Results: We found promising results when predicting defectprone files. A set of specific Resharper categories are better indicators of faulty files than common software metrics or the collection of issues of all issue categories, and these categories correlate to different software quality attributes. Conclusions: Our advice for future research is to perform analysis on file rather component level and to evaluate the generalizability of categories. We also recommend using larger datasets as we learned that data sparseness can lead to challenges in the proposed analysis process.

A pilot study was conducted within a software development domain using "knowledge mapping" (K-mapping) as a research technique to locate and indicate improvements to problematic software project-level knowledge flows (K-flows). The goal of the study is to show that the mitigation or removal of obstacles to project-level K-flow will result in software process improvement (SPI) by lowering project labor cost, improving schedule adherence, and/or enhancing end-product quality. Results show support for the efficacy of the study's K-mapping technique for the identification and management of software project K-flow obstacles. Additionally, the suggested alleviation of such obstacles was acknowledged by project team members as having the potential for a positive effect on SPI.

The technical debt metaphor is increasingly being used to describe the effect of delaying certain software maintenance tasks on software projects. Practitioners understand intuitively how technical debt can turn into a serious problem if it is left unattended. However, it remains unknown how serious the problem is and whether explicit measurement and management of technical debt is useful. In this paper, we explore the effect of technical debt by tracking a single delayed maintenance task in a real software project throughout its lifecycle and simulate how explicit technical debt management might have changed project outcomes. The results from this study demonstrate how and to what extent technical debt affects software projects. The study also sheds light on the research methodologies that can be used to investigate the technical debt management problem.

Process models are often the basis for demonstrating compliance and recommending improvement in software engineering organizations. A descriptive model is a type of process model describing the human activities in software development that actually occur. The purpose of a descriptive model is to provide a baseline for further process improvement and analysis. Ideally, a descriptive model provides an explicit representation. However, if the descriptive model does not represent how a process is actually performed, subsequent recommendations for improvement may be based upon information that is depicted in the model but that does not actually take place. Similarly, a descriptive model may omit important information that is centrally relevant for an organization's process improvement goals. The accuracy of software process elicitation is an important measure and is the degree a descriptive model reflects an actual process in the real world. This study, informed by a synthesis of arguments from related literature, characterizes the accuracy of software process elicitation as the perception of error for a descriptive model. We collected data from 48 users in professional training settings using a between subjects design. The results suggest that users in the treatment group perceived significantly higher error.

Identifying refactoring opportunities in software systems is an important activity in today's agile development environments. The concept of code smells has been proposed to characterize different types of design shortcomings in code. Additionally, metric-based detection algorithms claim to identify the "smelly" components automatically. This paper presents results for an empirical study performed in a commercial environment. The study investigates the way professional software developers detect god class code smells, then compares these results to automatic classification. The results show that, even though the subjects perceive detecting god classes as an easy task, the agreement for the classification is low. Misplaced methods are a strong driver for letting subjects identify god classes as such. Earlier proposed metric-based detection approaches performed well compared to the human classification. These results lead to the conclusion that an automated metric-based pre-selection decreases the effort spent on manual code inspections. Automatic detection accompanied by a manual review increases the overall confidence in the results of metric-based classifiers.

Code smells refer to commonly occurring patterns in source code that indicate poor programming practices or code decay. Detecting code smells helps developers find design problems that can cause trouble in future maintenance. Detection rules for code smells, based on software metrics, have been proposed, but they do not take domain-specific characteristics into consideration. In this study we investigate whether such generic heuristics can be tailored to include domain-specific factors. Input into these domain-specific heuristics comes from an iterative empirical field study in a software maintenance project. The results yield valuable insight into code smell detection.

As software engineering educators, it is important for us to realize the increasing domain-specificity of software, and incorporate these changes in our design of teaching material. Bioinformatics software is an example of immensely complex and critical scientific software and this domain provides an excellent illustration of the role of computing in the life sciences. To study bioinformatics from a software engineering standpoint, we conducted an exploratory survey of bioinformatics developers. The survey had a range of questions about people, processes and products. We learned that practices like extreme programming, requirements engineering and documentation. As software engineering educators, we realized that the survey results had important implications for the education of bioinformatics professionals. We also investigated the current status of software engineering education in bioinformatics, by examining the curricula of more than fifty bioinformatics programs and the contents of over fifteen textbooks. We observed that there was no mention of the role and importance of software engineering practices essential for creating dependable software systems. Based on our findings and existing literature we present a set of recommendations for improving software engineering education in bioinformatics.

This paper presents the results of exploratory research that investigated how people describe software processes in natural language. We conducted a small field study with four participants working at an IT help desk. We elicited and documented a trouble ticketing process using a template under conditions similar to that of many process improvement initiatives. This study included two treatments. In the first treatment, the process engineer elicited information and documented the process. In the second treatment, the participants used the template to document the process on their own. The resulting data, including the process representations, observation field notes, and interview transcripts, were analyzed using a grounded theory approach. The results suggest that there are distinct ways in which process users describe process. We construct a theory that posits that descriptions of process are dependent upon perspectives shaped by the elicitation and process context. Future research will focus on the evaluation of this theory relative to other elicitation approaches and contexts.

The objective of this study is to present a body of evidence that will assist software project managers to make informed choices about software development approaches for their projects. In particular, two broadly defined competing approaches, the traditional ldquowaterfallrdquo approach and iterative and incremental development (IID), are compared with regards to development cost and duration, and resulting product quality. The method used for this comparison is a systematic literature review. The small set of studies we located did not demonstrate any identifiable cost, duration, or quality trends, although there was some evidence suggesting the superiority of IID (in particular XP). The results of this review indicate that further empirical studies, both quantitative and qualitative, on this topic need to be undertaken. In order to effectively compare study results, the research community needs to reach a consensus on a set of comparable parameters that best assess cost, duration, and quality.

Metrics efforts are often impeded by factors such as poor data quality and developer resistance. To better understand and thus to address the developer perspective in a metrics program we undertook a case study at a large multi-national corporation. We identified six projects, and conducted surveys of both project managers and developers. These surveys were based on the metrics acceptance model (MAM) which is a framework (i.e. a model of relationships between factors, operationalized by a survey instrument) for gauging developer opinions toward software metrics. We noticed some interesting differences between developers' and managers' perceptions of metrics. While managers were on the same page as developers when it came to factors such as ease of use of the metrics tool, they over-estimated developers' confidence to report accurate measures. Managers under-estimated developers' beliefs about the usefulness of metrics and about their fear of adverse consequences. These findings suggest that the MAM could provide useful insights to project managers to train and motivate their developers. We also found that the MAM can be an effective diagnostic tool both at an organizational and project level to identify potential impediments in metrics programs.

Video is widely used as a data collection method in observational research. However, the process of analyzing video data can be overwhelming. In this study, we discuss several issues that we found during analyzing video data, such as the boundary of behaviors, priority for intertwined behaviors, and misunderstanding of behavior codes among analysts. We propose an iterative approach with reliability analysis to refine the coding scheme and solve these problems. This process involves two analyses, alternating between independent coding and coding in pairs.

Software project managers play an important role in selecting their software development process. In this study we conducted a survey of software project managers about software process change. The result of the survey revealed several factors affecting this type of decision making. It also revealed critical issues in software development projects. In particular, the findings point to the importance of a piloting strategy in technology transfer, as well as the importance of highlighting cost, quality, and schedule information in reporting evidence of a new technique's effectiveness. We expect that the findings of this study could facilitate research on technology transfer and adoption.

This paper describes our experience in aggregating a number of historical datasets containing inspection defect data using different categorization schemes. Our goal was to make use of the historical data by creating models to guide future development projects. We describe our approach to reconciling the different choices used in the historical datasets to categorize defects, and the challenges we faced. We also present a set of recommendations for others involved in classifying defects.

Software process improvement initiatives such as metrics programs have a high failure rate during their assimilation in a software organization. Social and organizational issues are some of the factors affecting the adoption and acceptance of metrics, and these issues have not been discussed in detail in existing metrics literature. We undertook an interview-based study with the purpose of studying factors that influence the buy-in of metrics. We interviewed 12 members of the metrics team of a large multi-national corporation, with a thriving metrics program. We found that there was some resistance to standardization of corporate metrics processes introduced by the metrics team. This resistance centered on the metrics data collection and reporting processes. One cause of resistance was the presence of sub-cultures and native data collection and reporting processes within organizational units that were independent businesses before they were acquired. Some of the pushback manifested itself through begrudging compliance, and avoidance activities like scripting and gaming of metrics. In this paper, we present the perspectives of developers, managers and upper-level management to emphasize that each stakeholder in the metrics initiative has a valid viewpoint that should be taken into account while implementing a metrics program and that each metrics effort is inextricably enmeshed with the organizational context. We provide actionable recommendations to understand the different perspectives and to adapt the metrics effort accordingly.

Penny Grubb and Armstrong Takang published a second edition of their text 'Software Maintenance: Concepts and Practice' in 2003.I reviewed the first edition, published in 1996, in this journal in 2001.This review, then, describes the new edition, in particular with respect to how it compares with the first edition.Since my first review, I have also had the opportunity to teach a semester of software maintenance at the university level, using the second edition, and so this review will reflect that experience as well.The new edition has much the same content as the previous one.Nothing has been deleted, but there are several notable additions.The most important, especially in terms of instructional aids, is the addition of many new case studies, including a series of vignettes taken from a single case study (at 'ACME Health Clinic').The inadequacy of the case studies in the first edition was a point raised in my 2001 review, and so I am particularly pleased to see that the volume and depth of the case studies in the new edition have significantly improved.These case studies, while not extensive in detail, are nicely chunked to illustrate particular concepts and provide a good basis for class discussions.Another addition is the grouping of the chapters into five 'parts'.The grouping of the chapters is logical, but the real contribution is the introductions to each of these parts.These introductions provide a very nice summing up of the various topics presented and the relationships between them.Further, each part introduction includes very useful discussion points, which are well suited for use in the classroom.

Millennial students (those born after 1982), particularly African Americans and women, have demonstrated a propensity toward collaborative activities. We conducted a collective case study at North Carolina State University and North Carolina A&T to ascertain the role of collaboration and social interaction in attracting and retaining students in information technology. Responses from semi-structured interviews with 11 representative African American students in these classes were coded and analyzed. The responses from these minority students were used to evolve a social interaction model. The conjectures generated from the model suggest that pair programming and agile software methodologies effectively create a collaborative environment that is desirable to Millennial students, male and female, and, with the new evidence, minority and majority. Additionally, the African American Millennial students enjoy learning from their peers and believe that a collaborative environment better prepares them for the "real world".

Abstract In 2005 Idea Group Publishing released another volume on issues in software maintenance. This follows their 2003 volume entitled Advances in Software Maintenance Management: Technologies and Solutions, which I also reviewed here (Journal of Software Maintenance and Evolution: Research and Practice 2003; 15(5): 375–377). A variety of readers, with a variety of goals, will find this 2005 book useful. While it is clearly not a textbook, particular chapters could be used in a classroom setting to expose students to current streams of research in software maintenance, as well as to different perspectives on basic maintenance issues. Practitioners will find practical information on particular maintenance problems they might be facing, including the migration of a large legacy system in general (Chapters 5, 13 and 8–11), or to the Web in particular (Chapters 6, 7 and 11), assessment of maintenance and organizational structure (Chapters 3, 4 and 14), and approaches for changing how an organization carries out software development and maintenance (Chapters 2, 12, 13 and 15), as well as on maintenance management (Chapter 1). While some of these chapters provide a level of detail sufficient to allow the reader to apply the techniques and methods described, most are not meant as recipes, or detailed how‐tos of techniques. Instead, the chapters provide the necessary overview, augmented by extensive lists of references. Copyright © 2006 John Wiley & Sons, Ltd.

Abstract Welcome to this special issue of the Journal of Software Maintenance and Evolution on process diversity. We, the guest editors, are very pleased to be able to include three fine papers on this subject, and we hope that our readers will find them stimulating and informative. Copyright © 2003 John Wiley & Sons, Ltd.

Abstract Here is a new offering, now available both as a hardcover and as an e‐book, in the much‐neglected category of comprehensive texts on software maintenance. It is especially welcome for those of us struggling to provide our students with a text for learning the basics of software maintenance, in preparation for their inevitable first maintenance assignments as junior software engineers. This new book, Advances in Software Maintenance Management: Technologies and Solutions, edited by Macario Polo, Mario Piattini, and Francisco Ruiz at the University of Castilla‐La Mancha in Spain, is not intended to be an academic textbook and so is lacking in some areas for that purpose. However, it has some important advantages over the few other choices in this area. It also is a useful volume for managers and others interested in getting an up‐to‐date understanding of some of the major issues in software maintenance. Copyright © 2003 John Wiley & Sons, Ltd.

Experience management refers to the capture, structuring, analysis, synthesis, and reuse of an organization's experience in the form of documents, plans, templates, processes, data, etc. The problem of managing experience effectively is not unique to software development, but the field of software engineering has had a high-level approach to this problem for some time. The Experience Factory is an organizational infrastructure whose goal is to produce, store, and reuse experiences gained in a software development organization. This paper describes The Q-Labs Experience Management System (Q-Labs EMS), which is based on the Experience Factory concept and was developed for use in a multinational software engineering consultancy. A critical aspect of the Q-Labs EMS project is its emphasis on empirical evaluation as a major driver of its development and evolution. The initial prototype requirements were grounded in the organizational needs and vision of Q-Labs, as were the goals and evaluation criteria later used to evaluate the prototype. However, the Q-Labs EMS architecture, data model, and user interface were designed to evolve, based on evolving user needs. This paper describes this approach, including the evaluation that was conducted of the initial prototype and its implications for the further development of systems to support software experience management.

In examining software maintenance processes for improvement opportunities, an obvious choice is information flow. Obtaining accurate, up-to-date, and useful information about a system being maintained is a major task. It is also a difficult task because the sources of this information are often limited, inaccessible, or unknown. Clearly this impacts maintenance productivity-simply because of the time it takes to find and use the appropriate information sources-as well as the quality of system changes, which depends on the quality of the system information available. This paper describes the results of a survey study that aims to discover the information gathering strategies that software maintainers employ. The survey was completed by 45 software professionals in two different organizations with varying degrees of experience in maintenance. Their responses, on the surface, simply show that maintainers overwhelmingly rely on source code, which is not surprising. However, a deeper analysis of the responses show that other sources of information, in particular human sources, some types of CASE support, and lessons learned recorded from previous projects are at least as valuable than source code under some conditions. The results of this ongoing survey study are meant to determine a set of hypotheses about information gathering strategies, which will then be empirically evaluated in future studies.

Abstract Software Maintenance: Concepts and Practice is an excellent paperback book that comprehensively covers a breadth of software maintenance issues. It is more than adequate as a textbook for a university‐level course about software maintenance, although it has a few drawbacks in this respect (in particular a lack of technical guidance on key maintenance concerns such as impact analysis). In addition, it can be a useful reference for software researchers and, to a lesser extent, practitioners. Copyright © 2001 John Wiley & Sons, Ltd.

The work described in this paper is an investigation of COTS-based software development within a particular NASA environment, with an emphasis on the processes used. Fifteen projects using a COTS-based approach were studied and their actual process was documented. This process is evaluated to identify essential differences in comparison to traditional software development. The main differences, and the activities for which projects require more guidance, are requirements definition and COTS selection, high level design, integration and testing.

While empirical studies in software engineering are beginning to gain recognition in the research community, this subarea is also entering a new level of maturity by beginning to address the human aspects of software development. This added focus has added a new layer of complexity to an already challenging area of research. Along with new research questions, new research methods are needed to study nontechnical aspects of software engineering. In many other disciplines, qualitative research methods have been developed and are commonly used to handle the complexity of issues involving human behaviour. The paper presents several qualitative methods for data collection and analysis and describes them in terms of how they might be incorporated into empirical studies of software engineering, in particular how they might be combined with quantitative methods. To illustrate this use of qualitative methods, examples from real software engineering studies are used throughout.

In this paper, we propose a qualitative, inductive method for characterizing and evaluating software maintenance processes, thereby identifying their specific problems and needs. This method encompasses a set of procedures which attempt to determine causal links between maintenance problems and flaws in the maintenance organization and process. This allows for a set of concrete steps to be taken for maintenance quality and productivity improvement, based on a tangible understanding of the relevant maintenance issues in a particular maintenance environment. Moreover, this understanding provides a solid basis on which to define relevant software maintenance models and measures. A case study of the application of this method, called Q-MOPP, is presented to further illustrate its feasibility and benefits. © 1998 John Wiley & Sons, Ltd.

This paper describes an empirical study that addresses the issue of communication among members of a software development organization. In particular, data was collected concerning code inspections in one software development project. The question of interest is whether or not organizational structure (the network of relationships between developers) has an effect on the amount of effort expended on communication between developers. The independent variables in this study are various attributes of the organizational structure in which the inspection participants work. The dependent variables are measures of the communication effort expended in various parts of the code inspection process, focusing on the inspection meeting. Both quantitative and qualitative methods were used, including participant observation, structured interviews, generation of hypotheses from field notes, statistical tests of relationships, and interpretation of results with qualitative anecdotes. The study results show that past and present working relationships between inspection participants affect the amount of meeting time spent in different types of discussion, thus affecting the overall inspection meeting length. Reporting relationships and physical proximity also have an effect. The contribution of the study is a set of well-supported hypotheses for further investigation.

This paper describes an empirical study which addresses the issue of communication among members of a software development organization.In particular, data was collected concerning code inspections in one software development project.The question of interest is whether or not organizational structure (the network of relationships between developers) has an effect on the amount of effort expended on communication between developers.Both quantitative and qualitative methods were used, including participant observation, structured interviews, generation of hypotheses from field notes, some simple statistical tests of relationships, and interpretation of results with qualitative anecdotes.The study results show that past and present working relationships between inspection participants affect the amount of meeting time spent in different types of discussion, thus affecting the overall meeting length.Reporting relationships and physical proximity also have an effect, as well as the point in the project that an inspection occurs.All but the last of these factors are organizational structure relationships.The contribution of the study is a set of well-supported hypotheses for further investigation.

One important component of a software process is the organizational context in which the process is enacted.This component is ofien missing or incomplete in current process modeling approaches.One technique for modeling this perspective is the Actor-Dependency (AD) Model.This paper reports on a case study which used this approach to analyze and assess a large software maintenance organization.Our goal was to identifi the approach's strengths and weaknesses while providing practical recommendations for improvement and research directions.The AD model was found to be very useful in capturing the important properties of the organizational context of the maintenance process, and aided in the understanding of the flaws found in this process.However, a number of opportunities for extending and improving the AD model were identified.Among others, there is a need to incorporate quantitative information to complement the qualitative model.