Readability models and tools have been proposed to measure the effort to read code. However, these models are not completely able to capture the quality improvements in code as perceived by developers. To investigate possible features for new readability models and production-ready tools, we aim to better understand the types of readability improvements performed by developers when actually improving code readability, and identify discrepancies between suggestions of automatic static tools and the actual improvements performed by developers. We collected 370 code readability improvements from 284 Merged Pull Requests (PRs) under 109 GitHub repositories and produce a catalog with 26 different types of code readability improvements, where in most of the scenarios, the developers improved the code readability to be more intuitive, modular, and less verbose. Surprisingly, SonarQube only detected 26 out of the 370 code readability improvements. This suggests that some of the catalog produced has not yet been addressed by SonarQube rules, highlighting the potential for improvement in Automatic static analysis tools (ASAT) code readability rules as they are perceived by developers.

Abstract Official documentation of software development technologies, for example, APIs, may not be sufficient for all developer needs, so searching on the Internet is a usual practice. Nonetheless, finding useful information may be challenging because the best solutions are not always among the first ranked pages. Developers need to read and discard irrelevant pages, that is, those without code examples or those that have content with little focus on the desired solution. This work aims at proposing an approach to mine relevant solutions for programming tasks from search engine results by removing irrelevant pages. The authors evaluated the top‐20 pages returned by the Google search engine, for 10 different queries, and observed that only 31% of the evaluated pages are relevant to developers. Then, the authors proposed and evaluated three different approaches to mine the relevant pages returned by the search engine. Google's search engine has been used as a baseline, and authors’ results have shown that it returns a reasonable number of irrelevant pages for developers, and the authors could establish an effective approach to remove irrelevant pages, suggesting that developers could benefit from a customised web search filter for development content.

Abstract Proper software modularization still poses challenges to developers. One of the symptoms of inappropriate modularization is the large size of object‐oriented classes. In that case, a possible solution would be class restructuring with refactorings, such asExtract Class,Extract Super‐Class, orMove Method. However, class refactoring is challenging because of the possible side effects of improper changes. In this context, more effective decision support systems on which classes are worthwhile for restructuring to improve modularity are still lacking. This work focuses on exploring possible alternatives for supporting decision on class restructuring. A prospective study was performed on selected kinds of restructuring, aiming at determining what types of strategies are typically adopted to restructure bloated classes and which classes developers decided to restructure. Then, we proposed and evaluated a predictive model for indicating which classes to restructure, aiming at delivering a restructuring guide on those classes. Finally, we conducted a qualitative study to evaluate the perception of developers on such guides based on predictions for real software. The results have shown situations in which the proposed predictions could help the restructuring process but also elucidated possible improvements and limitations.

Bad smells are sub-optimal code structures that may represent problems needing attention. We conduct an extensive literature review on bad smells relying on a large body of knowledge from 1990 to 2017. We show that some smells are much more studied in the literature than others, and also that some of them are intrinsically inter-related (which). We give a perspective on how the research has been driven across time (when). In particular, while the interest in duplicated code emerged before the reference publications by Fowler and Beck and by Brown et al., other types of bad smells only started to be studied after these seminal publications, with an increasing trend in the last decade. We analyzed aims, findings, and respective experimental settings, and observed that the variability of these elements may be responsible for some apparently contradictory findings on bad smells (what). Moreover, we could observe that, in general, papers tend to study different types of smells at once. However, only a small percentage of those papers actually investigate possible relations between the respective smells (co-studies), i.e., each smell tends to be studied in isolation. Despite of a few relations between some types of bad smells have been investigated, there are other possible relations for further investigation. We also report that authors have different levels of interest in the subject, some of them publishing sporadically and others continuously (who). We observed that scientific connections are ruled by a large “small world” connected graph among researchers and several small disconnected graphs. We also found that the communities studying duplicated code and other types of bad smells are largely separated. Finally, we observed that some venues are more likely to disseminate knowledge on Duplicate Code (which often is listed as a conference topic on its own), while others have a more balanced distribution among other smells (where). Finally, we provide a discussion on future directions for bad smell research.

A substantial effort, in general, is required for understanding APIs of application frameworks. High-quality API documentation may alleviate the effort, but the production of such documentation still poses a major challenge for modern frameworks. To facilitate the production of framework instantiation documentation, we hypothesize that the framework code itself and the code of existing instantiations provide useful information. However, given the size and complexity of existent code, automated approaches are required to assist the documentation production. Our goal is to assess an automated approach for constructing relevant documentation for framework instantiation based on source code analysis of the framework itself and of existing instantiations. The criterion for defining whether documentation is relevant would be to compare the documentation with an traditional framework documentation, considering the time spent and correctness during instantiation activities, information usefulness, complexity of the activity, navigation, satisfaction, information localization and clarity. We propose an automated approach for constructing relevant documentation for framework instantiation based on source code analysis of the framework itself and of existing instantiations. The proposed approach generates documentation in a cookbook style, where the recipes are programming activities using the necessary API elements driven by the framework features. We performed an empirical study, consisting of three experiments with 44 human subjects executing real framework instantiations aimed at comparing the use of the proposed cookbooks to traditional manual framework documentation (baseline). Our empirical assessment shows that the generated cookbooks performed better or, at least, with non-significant difference when compared to the traditional documentation, evidencing the effectiveness of the approach.

Developers often search for relevant code examples on the web for their programming tasks. Unfortunately, they face two major problems. First, the search is impaired due to a lexical gap between their query (task description) and the information associated with the solution. Second, the retrieved solution may not be comprehensive, i.e., the code segment might miss a succinct explanation. These problems make the developers browse dozens of documents in order to synthesize an appropriate solution. To address these two problems, we propose CROKAGE (Crowd Knowledge Answer Generator), a tool that takes the description of a programming task (the query) and provides a comprehensive solution for the task. Our solutions contain not only relevant code examples but also their succinct explanations. Our proposed approach expands the task description with relevant API classes from Stack Overflow Q&A threads and then mitigates the lexical gap problems. Furthermore, we perform natural language processing on the top quality answers and then return such programming solutions containing code examples and code explanations unlike earlier studies. We evaluate our approach using 97 programming queries, of which 50% was used for training and 50% was used for testing, and show that it outperforms six baselines including the state-of-art by a statistically significant margin. Furthermore, our evaluation with 29 developers using 24 tasks (queries) confirms the superiority of CROKAGE over the state-of-art tool in terms of relevance of the suggested code examples, benefit of the code explanations and the overall solution quality (code + explanation).

Benchmarks of bugs are essential to empirically evaluate automatic program repair tools. In this paper, we present BEARS, a project for collecting and storing bugs into an extensible bug benchmark for automatic repair studies in Java. The collection of bugs relies on commit building state from Continuous Integration (CI) to find potential pairs of buggy and patched program versions from open-source projects hosted on GitHub. Each pair of program versions passes through a pipeline where an attempt of reproducing a bug and its patch is performed. The core step of the reproduction pipeline is the execution of the test suite of the program on both program versions. If a test failure is found in the buggy program version candidate and no test failure is found in its patched program version candidate, a bug and its patch were successfully reproduced. The uniqueness of Bears is the usage of CI (builds) to identify buggy and patched program version candidates, which has been widely adopted in the last years in open-source projects. This approach allows us to collect bugs from a diversity of projects beyond mature projects that use bug tracking systems. Moreover, BEARS was designed to be publicly available and to be easily extensible by the research community through automatic creation of branches with bugs in a given GitHub repository, which can be used for pull requests in the BEARS repository. We present in this paper the approach employed by BEARS, and we deliver the version 1.0 of BEARS, which contains 251 reproducible bugs collected from 72 projects that use the Travis CI and Maven build environment.

Inadequate documentation of software design has been known to be a barrier for developers. Interestingly, several relevant object-oriented systems have their design documented using key classes, which are meant to represent key concepts of the systems. In order to fill the gap of under-documented design, we present Keecle, an approach for detecting a predefined number of key classes in a semi-automatic way. The main challenge is to reduce the space of potentially thousands of classes to just a few representatives of the main concepts of a system, while maintaining high precision. The approach is evaluated with 13 systems in order to assess its correctness. The ground-truth is obtained either from the original documentation, or from third-party, or from the respective developers. The results were analyzed in terms of precision and recall, and have shown to be superior compared to the state-of-the-art approach. In order to evaluate if key classes are more critical from the design point of view, we evaluated whether they are associated with cohesion and coupling metrics. We found that although key classes, in general, are critical from the point of view of design, there are other classes that are also critical, suggesting that being aware of key classes encompass information not available in structural metrics, and could be useful as a additional facet for design assessment.

Abstract Background: Automatic program repair aims to reduce costs associated with defect repair. The detection and characterization of common bug‐fix patterns in software repositories play an important role in advancing this field. Aim: In this paper, we characterize the occurrence of known bug‐fix patterns in Java repositories at an unprecedented large scale. Furthermore, we propose a novel automatic technique for unveiling frequent and isolated repair actions corresponding to realistic bug fixes in Java. Method: The study was conducted for Java GitHub projects organized in two distinct data sets. The first data set (Boa) contains more than 4 million bug‐fix commits from 101 471 projects. The second data set (Defects4J) contains 369 real bug fixes from five open‐source projects. Results: We characterized the prevalence of the five most common bug‐fix patterns (identified in the work of Pan et al) in those bug fixes. The combined results showed direct evidence that developers often forget to add IF preconditions in the code.Conclusion: We discover a total of 155 repair actions from Defects4J patches and discuss 10 pervasive repair actions that occur across all analyzed Java projects. Moreover, the overall Precision and Recall values for the clustering approach were 0.62 and 0.64, respectively.

Stack Overflow has become a fundamental element of developer toolset. Such influence increase has been accompanied by an effort from Stack Overflow community to keep the quality of its content. One of the problems which jeopardizes that quality is the continuous growth of duplicated questions. To solve this problem, prior works focused on automatically detecting duplicated questions. Two important solutions are DupPredictor and Dupe. Despite reporting significant results, both works do not provide their implementations publicly available, hindering subsequent works in scientific literature which rely on them. We executed an empirical study as a reproduction of DupPredictor and Dupe. Our results, not robust when attempted with different set of tools and data sets, show that the barriers to reproduce these approaches are high. Furthermore, when applied to more recent data, we observe a performance decay of our both reproductions in terms of recall-rate over time, as the number of questions increases. Our findings suggest that the subsequent works concerning detection of duplicated questions in Question and Answer communities require more investigation to assert their findings.

Well-designed and publicly available datasets of bugs are an invaluable asset to advance research fields such as fault localization and program repair as they allow directly and fairly comparison between competing techniques and also the replication of experiments. These datasets need to be deeply understood by researchers: the answer for questions like "which bugs can my technique handle?" and "for which bugs is my technique effective?" depends on the comprehension of properties related to bugs and their patches. However, such properties are usually not included in the datasets, and there is still no widely adopted methodology for characterizing bugs and patches. In this work, we deeply study 395 patches of the Defects4J dataset. Quantitative properties (patch size and spreading) were automatically extracted, whereas qualitative ones (repair actions and patterns) were manually extracted using a thematic analysis-based approach. We found that 1) the median size of Defects4J patches is four lines, and almost 30% of the patches contain only addition of lines; 2) 92% of the patches change only one file, and 38% has no spreading at all; 3) the top-3 most applied repair actions are addition of method calls, conditionals, and assignments, occurring in 77% of the patches; and 4) nine repair patterns were found for 95% of the patches, where the most prevalent, appearing in 43% of the patches, is on conditional blocks. These results are useful for researchers to perform advanced analysis on their techniques' results based on Defects4J. Moreover, our set of properties can be used to characterize and compare different bug datasets.

[Background]: There are more bugs in real-world programs than human programmers can realistically address. Several approaches have been proposed to aid debugging. A recent research direction that has been increasingly gaining interest to address the reduction of costs associated with defect repair is automatic program repair. Recent work has shown that some kind of bugs are more suitable for automatic repair techniques. [Aim]: The detection and characterization of common bug-fix patterns in software repositories play an important role in advancing the field of automatic program repair. In this paper, we aim to characterize the occurrence of known bug-fix patterns in Java repositories at an unprecedented large scale. [Method]: The study was conducted for Java GitHub projects organized in two distinct data sets: the first one (i.e., Boa data set) contains more than 4 million bug-fix commits from 101,471 projects and the second one (i.e., Defects4J data set) contains 369 real bug fixes from five open-source projects. We used a domain-specific programming language called Boa in the first data set and conducted a manual analysis on the second data set in order to confront the results. [Results]: We characterized the prevalence of the five most common bug-fix patterns (identified in the work of Pan et al.) in those bug fixes. The combined results showed direct evidence that developers often forget to add IF preconditions in the code. Moreover, 76% of bug-fix commits associated with the IF-APC bug-fix pattern are isolated from the other four bug-fix patterns analyzed. [Conclusion]: Targeting on bugs that miss preconditions is a feasible alternative in automatic repair techniques that would produce a relevant payback.

Several object-oriented systems have their respective designs documented by using only a few design-relevant classes, which we will refer to as key classes. In this paper, we automatically detect key classes, and investigate some of their properties, and evaluate their role for assessing design. We propose focusing on such classes to make design decisions during maintenance tasks as those classes of this type are, by definition, more relevant than non-key classes. First, we show that key classes are more prone to bad smells than non-key classes. Although, structural metrics of key classes tend to be, in general, higher than non-key classes, there are still a significant set of non-key classes with poor structural metrics, suggesting that prioritizing design anomaly assessment using key classes would likely to be more effective.

Continuous Integration (CI) implies that a whole developer team works together on the mainline of a software project. CI systems automate the builds of a software. Sometimes a developer checks in code, which breaks the build. A broken build might not be a problem by itself, but it has the potential to disrupt co-workers, hence it affects the performance of the team. In this study, we investigate the interplay between non-functional requirements (NFRs) and builds statuses from 1,283 software projects. We found significant differences among NFRs related-builds statuses. Thus, tools can be proposed to improve CI with focus on new ways to prevent failures into CI, specially for efficiency and usability related builds. Also, the time required to put a broken build back on track indicates a bimodal distribution along all NFRs, with higher peaks within a day and lower peaks in six weeks. Our results suggest that more planned schedule for maintainability for Ruby, and for functionality and reliability for Java would decrease delays related to broken builds.

From time to time, developers perform sequences of code transformations in a systematic and repetitive way. This may happen, for example, when introducing a design pattern in a legacy system: similar classes have to be introduced, containing similar methods that are called in a similar way. Automation of these sequences of transformations has been proposed in the literature to avoid errors due to their repetitive nature. However, developers still need support to identify all the relevant code locations that are candidate for transformation. Past research showed that these kinds of transformation can lag for years with forgotten instances popping out from time to time as other evolutions bring them into light. In this paper, we evaluate three distinct code search approaches (“structural”, based on Information Retrieval, and AST based algorithm) to find code locations that would require similar transformations. We validate the resulting candidate locations from these approaches on real cases identified previously in literature. The results show that looking for code with similar roles, e.g., classes in the same hierarchy, provides interesting results with an average recall of 87% and in some cases the precision up to 70%.

Abstract Stack Overflow (SO) is a question and answer service directed to issues related to software development. In SO, developers post questions related to a programming topic and other members of the site can provide answers to help them. The information available on this type of service is also known as ‘crowd knowledge’ and currently is one important trend in supporting activities related to software development. We present an approach that makes use of ‘crowd knowledge’ in SO to recommend information that can assist developer activities. This strategy recommends a ranked list of question‐answer pairs from SO based on a query. The criteria for ranking are based on three main aspects: the textual similarity of the pairs with respect to the query related to the developer's problem, the quality of the pairs, and a filtering mechanism that considers only ‘how‐to’ posts. We conducted an experiment considering programming problems on three different topics (Swing, Boost and LINQ) widely used by the software development community to evaluate the proposed recommendation strategy. The results have shown that for Lucene + Score + How-to approach, 77.14% of the assessed activities have at least one recommended pair proved to be useful concerning the target programming problem. Copyright © 2016 John Wiley & Sons, Ltd.

Co-change clusters are groups of classes that frequently change together. They are proposed as an alternative modular view, which can be used to assess the traditional decomposition of systems in packages. To investigate developer's perception of co-change clusters, we report in this paper a study with experts on six systems, implemented in two languages. We mine 102 co-change clusters from the version history of such systems, which are classified in three patterns regarding their projection to the package structure: Encapsulated, Crosscutting, and Octopus. We then collect the perception of expert developers on such clusters, aiming to ask two central questions: (a) what concerns and changes are captured by the extracted clusters? (b) do the extracted clusters reveal design anomalies? We conclude that Encapsulated Clusters are often viewed as healthy designs and that Crosscutting Clusters tend to be associated to design anomalies. Octopus Clusters are normally associated to expected class distributions, which are not easy to implement in an encapsulated way, according to the interviewed developers.

Reconstructing architectural components from existing software applications is an important task during the software maintenance cycle because either those elements do not exist or are outdated. Reverse engineering techniques are used to reduce the effort demanded during the reconstruction. Unfortunately, there is no widely accepted technique to retrieve software components from source code. Moreover, in several architectural descriptions of systems, a set of architecturally relevant classes are used to represent the set of architectural components. Based on this fact, we propose Keecle, a novel dynamic analysis approach for the detection of such classes from execution traces in a semi-automatic manner. Several mechanisms are applied to reduce the size of traces, and finally the reduced set of key classes is identified using Naïve Bayes classification. We evaluated the approach with two open source systems, in order to assess if the encountered classes map to the actual architectural classes defined in the documentation of those respective systems. The results were analyzed in terms of precision and recall, and suggest that the proposed approach is effective for revealing key classes that conceptualize architectural components, outperforming a state-of-the-art approach.

Software reuse is one of the major goals in software engineering. Frameworks promote the reuse of not only individual building blocks, but also of system design. However, framework instantiation requires a substantial understanding effort. High quality documentation is essential to minimize this effort. However, in most cases, appropriate documentation does not exist or is not updated. Our hypothesis is that the framework code itself and existing instantiations can serve as a guide for new instantiations. The challenge is that users still have to read large portions of code, which hinders the understanding process, thus our goal is to provide relevant information for framework instantiation with static and dynamic analysis of the framework and pre-existing instantiations. The final documentation is presented in a cookbook style, where recipes are composed of programming tasks and information about hotspots related to a feature instantiation. We conducted two preliminary experiments, the first to evaluate the recall of the approach and the second to study the practical usefulness of the recipe information for developers. Results reveal that our approach discloses accurate and relevant information about classes and methods used for framework instantiation.

Several companies use the amount of deliveries as a metric of performance evaluation of the developer.However, the productivity of a developer and his importance for the company is not only related to the amount of lines of code produced.There are a variety of factors that can contribute to the relevance of a developer for a team.This paper aims at mapping some of these factors, measuring those that are more important for companies and propose an evaluation model of developer importance that considers more than just deliveries.We have found that some factors are more important than others and that there are minor differences for different companies.We have also developed a high accuracy classifier that can indicate the importance of the developer based on a set of attributes.

Several companies use amount of deliveries as a metric for performance evaluation of developers. However, the productivity of a developer and his importance for the company is not only related to the amount of produced lines of code. There are a variety of factors that can contribute to the relevance of developers for their teams. This paper aims at mapping some of these factors, measuring those that are more important for companies and propose an evaluation model of developer importance that considers more than just deliveries. We have found that some factors are more important than others and that there are minor differences for different companies. We have also developed a high accuracy classifier that can indicate the importance of the developer based on a set of attributes.

StackOverflow.com (SO) is a Question and Answer service oriented to support collaboration among developers in order to help them solving their issues related to software development. In SO, developers post questions related to a programming topic and other members of the site can provide answers to help them. The information available on this type of service is also known as "crowd knowledge" and currently is one important trend in supporting activities related to software development and maintenance.

Software metrics is a valuable mechanism to assess the quality of software systems. Metrics can help the automated analysis of the growing data available in software repositories. Coupling metrics is a kind of software metrics that have been extensively used since the seventies to evaluate several software properties related to maintenance, evolution and reuse tasks. For example, several works have shown that we can use coupling metrics to assess the reusability of software artifacts available in repositories. However, thresholds for software metrics to indicate adequate coupling levels are still a matter of discussion. In this paper, we investigate the impact of software categories on the coupling level of software systems. We have found that different categories may have different levels of coupling, suggesting that we need special attention when comparing software systems in different categories and when using predefined thresholds already available in the literature.

Recently, a number of scalable stream sharing protocols have been proposed with the promise of great reductions in the server and network bandwidth required for delivering popular media content. Although the scalability of these protocols has been evaluated mostly for sequential user accesses, a high degree of interactivity has been observed in the accesses to several real media servers. Moreover, some studies have indicated that user interactivity can severely penalize the scalability of stream sharing protocols.This paper investigates alternative mechanisms for scalable streaming to interactive users. We first identify a set of workload aspects that are determinant to the scalability of classes of streaming protocols. Using real workloads and a new interactive media workload generator, we build a rich set of realistic synthetic workloads. We evaluate Bandwidth Skimming and Patching, two state-of-the-art streaming protocols, covering, with our workloads, a larger region of the design space than previous work. Finally, we propose and evaluate five optimizations to Bandwidth Skimming, the most scalable of the two protocols. Our best optimization reduces the average server bandwidth required for interactive workloads in up to 54%, for unlimited client buffers, and 29%, if buffers are constrained to 25% of media size.