As the innovation of smart devices and internet-of-things (IoT), smart homes have become prevalent. People tend to transform residences into smart homes by customizing off-the-shelf smart home platforms, instead of creating IoT systems from scratch. Among the alternatives, Home Assistant (HA) is one of the most popular platforms. It allows programmers (i.e., home residents or smart-home creators) to smartify homes by (S1) integrating selected devices into the system and (S2) programming YAML-based software to control those devices. Unfortunately, due to the diversity of devices and complexity of automatic configurations, many programmers have difficulty correctly creating YAML files. Consequently, their smart homes may not work as expected, causing frustration and concern in people. This article presents a novel study on issues of YAML-based automation configuration in smart homes (issues related to S2). We mined the online forum Home Assistant Community for discussion threads related to programming of automation configuration. By manually inspecting 190 threads, we revealed 3 categories of concerns: implementation, optimization, and debugging. Under each category, we classified discussions based on the issue locations and technical concepts involved. Among debugging discussions, we further classified discussions based on users’ resolution strategies; we also applied existing analysis tools to buggy YAML files, to assess the tool effectiveness. Our study reveals the common challenges faced by programmers and frequently applied resolution strategies. There are 129 (68%) examined issues concerning debugging, but existing tools can detect at most 14 of the issues and fix none. It implies that existing tools provide limited assistance in automation configuration. Our research sheds light on future directions in smart home programming.

As software is widely used in daily life, bugs can introduce catastrophic consequences. Researchers have conducted empirical studies to delve into bug characteristics, exploring topics such as buggy locations, symptoms, causes, and repair patterns. To attract users, many applications have implementations in different languages. If an implementation has a bug, other implementations can have similar bugs. In this article, we call such cross-language clone bugs mirror bugs. Understanding mirror bugs is crucial, as they offer insights into broader bug patterns. Still, no prior study has explored mirror bugs, leaving several research questions unanswered. For instance, can bug fixes in one language help detect and repair bugs in other languages? To address these questions, we conducted the first empirical study analyzing mirror bugs. Our investigation focused on 638 bugs from four projects, implemented in both Java and C#. Our study presents answers to five interesting research questions. For instance, some programmers actively fix mirror bugs even without tool support. Consequently, there is a timely need for tools that assist in detecting mirror bugs. Following this insight, we manually identified and implemented the patches of nine new mirror bugs. Among them, five patches are already accepted by programmers.

In software development, many documents (e.g., tutorials for tools and mobile application websites) contain screenshots of GUIs to illustrate functionalities. Although screenshots are critical in such documents, screenshots can become outdated, especially if document developers forget to update them. Outdated screenshots can mislead users and diminish the credibility of documentation. Identifying screenshots manually is tedious and error-prone, especially when documents are numerous. However, no existing tools are proposed to detect outdated screenshots in GUI documents. To mitigate manual efforts, we propose DOSUD , a novel approach for detecting outdated screenshots. It is challenging to identify outdated screenshots since the differences are subtle and only specific areas are useful to identify such screenshots. To address the challenges, DOSUD automatically extracts and labels screenshots and trains a classification model to identify outdated screenshots. As the first exploration, we focus on Android applications and the most popular IDE, VS Code. We evaluated DOSUD on a benchmark comprising 10 popular applications, achieving high F1-scores. When applied in the wild, DOSUD identified 20 outdated screenshots across 50 Android application websites and 17 outdated screenshots in VS Code documentation. VS Code developers have confirmed and fixed all our bug reports.

As a paradigm of end-user programming, lowcode development has attracted attention from academics and enterprises. Consulting firms estimate that this paradigm can have great business value and that many large enterprises can switch to low-code development. To understand the emerging paradigm, researchers conducted various empirical studies about low-code development. Still, most studies analyze bugs in lowcode development platforms. Although these studies are related, low-code development platforms are classical programs, not lowcode applications. Several other studies analyze how programmers perceive low-code applications, but do not provide direct analysis. As low-code applications are typically commercial, most low-code projects are not open source. It is difficult to collect the subjects, and many questions are still open. In this paper, we propose the first study on bugs in lowcode applications. To overcome the above-mentioned challenge, we collect 8 actively maintained low-code projects from our industrial partner. From the 8 projects, we collected 248 bugs from our industrial partner and analyzed their symptoms, causes, and repairs. Our study derives 7 interesting findings. For instance, we find that 78.23 % of low-code bugs are related to human-computer interactions. The result indicates that GUI testing can be critical in detecting bugs in low-code applications. As another example, we find that even if programs are developed on low-code platforms, modifying source files is necessary to fix 78.23 % of low-code bugs.

Compilers are critical in development, but compiler bugs can cause hidden and serious bugs in their compiled code. To deepen the understanding of compiler bugs, in the prior empirical studies, researchers read the bug reports and patches of compilers, and analyze their causes, locations, and patterns. Although they derive many interesting findings, their studies are limited. First, as bug reports seldom explain which projects encounter compiler bugs, it is infeasible to understand the outreaching impact. Second, before compiler bugs are fixed, programmers can bypass such bugs. The bug reports of compilers do not introduce such workarounds. Finally, the distribution of compiler bugs can be distorted, since researchers and compiler developers also file bug reports. In this paper, we propose a novel angle to analyze compiler bugs. Instead of compiler bug reports, we collect compiler bugs that are mentioned in real development. When programmers encounter compiler bugs in real development, they can leave traces in their commit messages. By searching such messages, we collected 644 unique commits whose messages explicitly mention the urls of compiler bugs. From this angle, in this paper, we conduct the first empirical study to analyze compiler bugs in the wild. We summarize our results into seven useful findings for users, compiler developers, and researchers. For example, for researchers, we find that some large workarounds of compiler bugs involve repetitive and systematic changes, which indicates a new research opportunity for code migration tools. Furthermore, we attempt to apply our findings in real development, and we obtain positive feedback.

Compilers are essential in daily development, but their bugs can introduce hidden bugs to many software projects. As compiler bugs are important, researchers have conducted various empirical studies to understand their characteristics. Most of the prior studies analyze only the bug reports and patches of compilers. From such sources, they analyze the root causes, locations, and repairs of compiler bugs. Their findings are useful for understanding compiler bugs themselves, but to the best of our knowledge, the impact of compiler bugs is rarely explored. For example, before compiler bugs are fixed, programmers would bypass such bugs, but no prior study analyzes the workarounds for compiler bugs. To complement the prior studies, in this paper, we analyze how the bugs of two mainstream compilers, gcc and llvm, affect real development. Besides the bug reports and patches of compilers, in this study, from 603 GitHub projects, we collect 806 commits whose messages mention compiler bugs. From the dataset, we analyze how compiler bugs affect real projects. Furthermore, we manually analyze 219 commits whose messages explicitly mention compiler bugs. Our results lead to eight useful findings for programmers, compiler developers, and researchers. For instance, based on our findings, we suggest that it is critical to analyze the long-term impact of compiler bugs. Our story about a gcc bug confirms that compiler bugs can affect the compilations of projects even after these compiler bugs have already been fixed.

Natural languages are “natural” in that texts in natural languages are repetitive and predictable. Recent research indicates that programming languages share similar characteristics (naturalness), with source code displaying patterns of repetition and predictability. Notably, studies have shown that buggy code deviates from these natural patterns in that buggy code is significantly less natural than bug-free one. In this article, we conduct a large-scale and extensive empirical study to investigate whether code defects lead to unnaturalness of source code. Different from existing studies, we leverage multiple large-scale and high-quality bug repositories where bug-irrelevant changes in bug-fixing commits have been explicitly excluded. The leveraged software applications cover different programming languages, and the empirical study involves real-world software defects as well as defects injected automatically with well-known mutation operators. On the one side, our evaluation results confirm existing studies in that buggy source code lines are often less natural than bug-free ones. On the other side, our evaluation reveals some interesting new findings. First, fixing bugs does not significantly improve the naturalness of code lines and the fixed lines on average are as unnatural as buggy ones. This finding may suggest that software defects are not the root causes of source code’s unnaturalness although there does existing statistically significant correlation between software defects and source code’s naturalness. Second, defects in different programming languages have similar effect on source code’s naturalness. The conclusions (i.e., buggy code is less natural but fixing the bugs cannot improve source code’s naturalness) hold regardless of the programming languages. Third, injecting defects automatically by well-known mutation operators does not significantly reduce the naturalness of involved source code lines. This suggests that automatically injected defects may have a similar impact on the naturalness of source code as real-world defects inadvertently introduced by developers. Fourth, the detects’ impact on source code’s naturalness varies slightly among different categories of software defects. Although fixing bugs on average does not significantly improve the naturalness of involved source code, fixing “checking” related bugs does significantly improve the naturalness of source code. Finally, locating buggy code lines according to naturalness alone is inaccurate, resulting in extremely low precision (less than one percent).

API developers evolve software libraries to fix bugs, add new features, or refactor code, but the evolution can introduce API-breaking changes (e.g., API renaming). To benefit from such evolution, the programmers of client projects have to repetitively upgrade the callsites of libraries, since API-breaking changes introduce many compilation errors. It is tedious and error-prone to resolve such errors, especially when programmers are often unfamiliar with the API usages of newer versions. To migrate client code, the prior approaches either mine API mappings or learn edit scripts, but both the research lines have inherent limitations. For example, mappings alone cannot handle complex cases, and there is no sufficient source (e.g., migration commits) for learning edit scripts.

Mobile edge computing (MEC) offers a promising technology that deploys computing resources closer to mobile devices for improving performance. Most of the existing studies support on-demand remote execution of the computing tasks in applications through program transformation, but they commonly assume that mobile devices merely resort a single server for computation offloading, which cannot make full use of the scattered and changeable computing resources. Thus, for object-oriented applications, we propose a novel approach, called FUNOff to support dynamic offloading of applications in MEC at the function granularity. First, we extract a call tree via code analysis and locate the function invocations that are suitable for offloading. Next, we refactor the code of related object functions according to a specific program structure. Finally, we make offloading decisions referring to the context at runtime and send function invocations to multiple remote servers for execution. We evaluate the proposed FUNOff on two real-world applications. The results show that, compared with other approaches, FUNOff better supports the computation offloading of object-oriented applications in MEC, which reduces the response time by 10.7%-58.2%.

Advances in Artificial Intelligence (AI), and in particular in Machine Learning (ML), are introducing profound changes to scholarly submissions across publication venues, affecting in particular the contributions that are being submitted to Software Engineering (SE) conferences and journals. In this context, it is not always clear whether manuscripts submitted to SE venues under the umbrella term SE for AI are indeed relevant to SE, in the sense that they explicitly contain contributions to the SE body of knowledge. This leads to recurring discussions on whether certain AI-related submissions are appropriate to SE venues, or should instead be submitted to other journals and conferences, including AI or ML-specific ones. In this editorial, we discuss the kinds of AI-related contributions that are a better fit-and a less good fit-for publication in the IEEE Transactions on Software Engineering.

Researchers have proposed various approaches to generate test programs. The state-of-the-art approaches can be roughly divided into random-based and mutation-based approaches: random-based approaches generate random programs and mutation-based approaches mutate programs to generate more test programs. Both lines of approaches mainly generate random code, but it is more beneficial to use real programs, since it is easier to learn the impacts of compiler bugs and it becomes reasonable to use both valid and invalid code. However, most real programs from code repositories are ineffective to trigger compiler bugs, partially because they are compiled before they are submitted.

Although the exception handling mechanism is critical for resolving runtime errors, bugs inside this process can have far-reaching impacts. Therefore, researchers have proposed various approaches to assist catching and handling such thrown exceptions and to detect corresponding bugs. If the thrown exceptions themselves are incorrect, their errors will never be correctly caught and handled. Like bugs in catching and handling exceptions, wrong thrown exceptions have caused real critical bugs. However, to the best of our knowledge, no approach has been proposed to recommend which exceptions shall be thrown. Exceptions are widely adopted in programs, often poorly documented, and sometimes ambiguous, making the rules of throwing correct exceptions rather complicated. A project team can leverage exceptions in a way totally different from other teams. As a result, even experienced programmers can have difficulties in determining which exception shall be thrown, although they have the skills to implement its surrounding code. In this paper, we propose the first approach, ThEx, to predict which exception(s) shall be thrown under a given programming context. The basic idea is to learn a classification model from existing thrown exceptions in source files. Here, the learning features are extracted from various code information surrounding the thrown exceptions, such as the thrown locations and related variable names. Then, given a new context, ThEx can predict its best exception(s).

Identifying code authors is important in many research topics, and various approaches have been proposed. Although these approaches achieve promising results on their datasets, their true effectiveness is still in question. To the best of our knowledge, only one large-scale study was conducted to explore the impacts of related factors (e.g., the temporal effect and the distribution of files per author). This study selected Google Code Jam programs as their subjects, but such programs are quite different from the source files that programmers write in daily development. To understand their effectiveness and challenges, we replicate their study and use their approach to analyze source files that are retrieved from real projects. The prior study claims that the temporal effect and the distribution of files per author have only minor impacts on their trained models. In the contrast, we find that in 85.48% pairs of training and testing sets, the accuracy of a trained model is less effective when the temporal effect is considered, and in total, the average accuracy decreases by 0.4298. In addition, when we use the real distribution of files as inputs, their approach can accurately identify only one or two core code authors, although a project can have more than ten authors. By revealing the limitations of the prior approach, our study sheds lights on where to make future improvements.

Texts in natural languages are highly repetitive and predictable because of the naturalness of natural languages. Recent research validated that source code in programming languages is also repetitive and predictable, and naturalness is an inherent property of source code. It was also reported that buggy code is significantly less natural than bug-free one, and bug fixing substantially improves the naturalness of the involved source code. In this paper, we revisit the naturalness of buggy code and investigate the effect of bug-fixing on the naturalness of source code. Different from the existing investigation, we leverage two large-scale and high-quality bug repositories where bug-irrelevant changes in bug-fixing commits have been explicitly excluded. Our evaluation results confirm that buggy lines are often less natural than bug-free ones. However, fixing bugs could not significantly improve the naturalness of involved code lines. Fixed lines on average are as unnatural as buggy ones. Consequently, bugs are not the root cause of the unnaturalness of source code, and it could be inaccurate to identify buggy code lines solely by the naturalness of source code. Our evaluation results suggest that the naturalness-based buggy line detection results in extremely low precision (less than one percentage).

In recent years, deep learning obtains amazing achievements in various fields, and has been used in safety-critical scenarios. In such scenarios, bugs in deep learning software can introduce disastrous consequences. To deepen the understanding on bugs in deep learning software, researchers have conducted several empirical studies on their bug characteristics. In the prior studies, researchers analyzed the source code, bug reports, pull requests, and fixes of deep learning bugs. Although these studies provide meaningful findings, to the best of our knowledge, no prior studies have explored the runtime behaviors of deep learning bugs, because it is rather expensive to collect runtime impacts of deep learning bugs. As a result, some fundamental questions along with deep learning bugs are still open. For example, do most such bugs introduce significant impacts on prediction accuracy? The answers to these open questions are useful to a wide range of audience. In this paper, we conducted the first empirical study to analyze the runtime impacts of deep learning bugs. Our basic idea is to inject deliberately designed bugs into a typical deep learning application and its libraries with a mutation tool, and to compare the runtime differences between clean and buggy versions. In this way, we constructed 1,832 buggy versions, and compared their execution results with corresponding clean versions. Based on our comparison, we summarize 9 findings, and present our answers to 3 research questions. For example, we find that more than half of buggy versions do not lead to any observable errors, and most of them introduce only insignificant differences on the accuracy of their trained models. We interpret the significance of our findings from the perspectives of application programmers, API developers, and researchers. For example, based on our findings, better results alone are insufficient to prove better parameters nor better treatments, and researchers shall build strong theories to explain their improvements.

A deep neural network (DNN) has become increasingly popular in industrial Internet of Things scenarios. Due to high demands on computational capability, it is hard for DNN-based applications to directly run on intelligent end devices with limited resources. Computation offloading technology offers a feasible solution by offloading some computation-intensive tasks to the cloud or edges. Supporting such capability is not easy due to two aspects:Adaptability:offloading should dynamically occur among computation nodes.Effectiveness:it needs to be determined which parts are worth offloading. This article proposes a novel approach, called DNNOff. For a given DNN-based application, DNNOff first rewrites the source code to implement a special program structure supporting on-demand offloading and, at runtime, automatically determines the offloading scheme. We evaluated DNNOff on a real-world intelligent application, with three DNN models. Our results show that, compared with other approaches, DNNOff saves response time by 12.4–66.6% on average.

Static tools like Findbugs allow their users to manually define bug patterns, so they can detect more types of bugs, but due to the complexity and variety of programs, it is difficult to manually enumerate all bug patterns, especially for those related to API usages or project-specific rules. Therefore, existing bug-detection tools (e.g., Findbugs) based on manual bug patterns are insufficient in detecting many bugs. Meanwhile, with the rapid development of software, many past bug fixes accumulate in software version histories. These bug fixes contain valuable samples of illegal coding practices. The gap between existing bug samples and well-defined bug patterns motivates our research. In the literature, researchers have explored techniques on learning bug signatures from existing bugs, and a bug signature is defined as a set of program elements explaining the cause/effect of the bug. However, due to various limitations, existing approaches cannot analyze past bug fixes in large scale, and to the best of our knowledge, no previously unknown bugs were ever reported by their work. The major challenge to automatically analyze past bug fixes is that, bug-inducing inputs are typically not recorded, and many bug fixes are partial programs that have compilation errors. As a result, for most bugs in the version history, it is infeasible to reproduce them for dynamic analysis or to feed buggy/fixed code directly into static analysis tools which mostly depend on compilable complete programs. In this paper, we propose an approach, calledDePa, that extracts bug signatures based on accurate partial-code analysis of bug fixes. With its support, we conduct the first large scale evaluation on 6,048 past bug fixes collected from four popular Apache projects. In particular, we useDePato infer bug signatures from these fixes, and to check the latest versions of the four projects with the inferred bug signatures. Our results show thatDePadetected 27 unique previously unknown bugs in total, including at least one bug from each project. These bugs are not detected by their developers nor other researchers. Among them, three of our reported bugs are already confirmed and repaired by their developers. Furthermore, our results show that the state-of-the-art tools detected only two of our found bugs, and our filtering techniques improve our precision from 25.5 to 51.5 percent.

Calling Application Programming Interfaces (APIs) shall follow various constraints (e.g., call orders). If these con-straints are violated, API misuses are introduced to code, and such misuses can cause severe bugs. To effectively detect API misuses, most prior approaches mine constraints from client code, and assume that the violations of constraints are potential misuses. However, as client code only illustrates a small portion of API usages, constraints mined from client code are typically incomplete. As a result, when mined constraints are used to detect bugs, many violations of constraints turn out to be false positives. In this paper, our research purpose is to find more misuses and to reduce false positives. As library code contains many details on APIs, we propose an approach that mines API constraints from both client and library code. From client code, our approach builds API usage graphs and uses a frequent subgraph mining algorithm to mine frequent usage patterns as API constraints. From library code, our approach derives various types of constraints with our predefined strategies. With constraints from both sources, our graph matching algorithm can detect API misuses. As a result, our approach takes advantage from both the comprehensiveness and informativeness of library-based constraints and the accuracy of client-based patterns. We compared our approach with MuDetect on the MuBench dataset. Our results show that it significantly improves the detection effectiveness of MuBench from 39.5% to 50.2% of the recall, and from 30.6% to 41.7% of the precision.

Virtual Reality (VR) is an emerging technique that provides immersive experience for users. Due to the high computation cost of rendering real-time animation twice (for both eyes) and the resource limitation of wearable devices, VR applications often face performance bottlenecks and performanceoptimization plays an important role in VR software develop-ment. Performance optimizations of VR applications can be very different from those in traditional software as VR involves more elements such as graphics rendering and real-time animation. In this paper, we present the first empirical study on 183 real-world performance optimizations from 45 VR software projects. In particular, we manually categorized the optimizations in to 11 categories, and applied static analysis to identify how they affect different life-cycle phases of VR applications. Furthermore, we studied the complexity and design / behavior effects of performance optimizations, and how optimizations are different between large organizational software projects and smaller personal software projects. Our major findings include: (1) graphics simplification (24.0%), rendering optimization (16.9%), language / API optimization (15.3%), heap avoidance (14.8%), and valuecaching (12.0%) are the most common categories of performance optimization in VR applications; (2) game logic updates (30.4%) and before-scene initialization (20.0%) are the most common life-cycle phases affected by performance issues; (3) 45.9% of the optimizations have behavior and design effects and 39.3% of the optimizations are systematic changes; (4) the distributionsof optimization classes are very different between organizational VR projects and personal VR projects.

In recent years, with the flourish of deep learning techniques, deep learning libraries have been used by many smart applications. As smart applications are used in critical scenarios, their bugs become a concern, and bugs in deep learning libraries have far-reaching impacts on their built-on applications. Although programmers write many test cases for deep learning libraries, to the best of our knowledge, no prior study has ever explored to what degree such test cases are sufficient. As a result, some fundamental questions about these test cases are still open. For example, to what degree can existing test cases detect bugs in deep libraries? How to improve such test cases? To help programmers improve their test cases and to shed light on the detection techniques of deep learning bugs, there is a strong need for a study on the test quality of deep learning libraries. To meet the strong need, in this paper, we conduct the first empirical study on this issue. Our basic idea is to inject bugs into deep learning libraries, and to check to what degree existing test cases can detect our injected bugs. With a mutation tool, we constructed 1,545 buggy versions (i.e., mutants). By comparing the testing results between clean and buggy versions, our study leads to 11 findings, and we summarize them into the answers to three research questions. For example, we find that although existing test cases detected 60% of our injected bugs, only 30% of such bugs were detected by the assertions of these test cases. As another example, we find that some exceptions were thrown only in specific learning phases. Furthermore, we interpret our results from the perspectives of researchers, library developers, and application programmers.

Developers change software programs for various purposes (e.g., bug fixes, feature additions, and code refactorings), but the intents of code changes are often not recorded or are poorly documented. To automatically infer the change intent of each program commit (i.e., a set of code changes), existing work classifies commits based on commit messages and/or the sheer counts of edited files, lines, or abstract syntax tree (AST) nodes. However, none of these tools reason about the syntactic or semantic dependencies between co-applied changes, neither do they adopt any deep learning method. To better characterize program commits, in this paper, we present CClassifier—a new approach that classifies commits by (1) using advanced static program analysis to comprehend relationship between co-applied edits, (2) representing edits and their relationship via graphs, and (3) applying convolutional neural networks (CNN) to classify those graphs. Compared with prior work, CClassifier extracts a richer set of features from program changes; it is the first to classify program commits using CNN. For evaluation, we prepared a benchmark that contains 7,414 code changes from 5 open-source Java projects. On this benchmark, we empirically compared CClassifier and the state-of-the-art approach with five-fold cross validation. On average, when predicting bug-fixing commits within the same projects, CClassifier improved the prediction accuracy from 70% to 72%. More importantly, prior work seldom identifies feature-addition commits; CClassifier can successfully identify such commits in a lot more scenarios. Our evaluation shows that CClassifier outperforms prior work due to its usage of advanced program analysis and CNN.

Although many programmers write their names in the comments of a source file, from such comments, it is unreliable to identify code authors, since the modifications of many programmers are not recorded. Even if they are recorded in a code repository, many authors are hidden in revision histories.The true authors of source files are important in many research topics. For example, when detecting plagiarism, if the authors of two source are overlapped, it becomes more challenging to determine plagiarism than the source files that are written by individual authors. As it is difficult to determine true authors of a source file, researchers typically use source files whose authors are already known (e.g., the source files from Google Code Jam), but such files are not many and less representative. Meanwhile, although some empirical studies touch code authors, to the best of our knowledge, no prior study has analyzed the characteristics of code authors that are hidden in revision histories. As a result, many research questions along with code authors are still open. For example, how many authors does a source file can have, and what are the proportions of contributions per source file, if they are written by more than one author?To answer the timely questions, in this paper, we conducted an empirical study on code authors that are hidden in revision histories. To support our study, we implemented a tool called CODA. By comparing the latest code lines with past commits, CODA identifies the true authors of all code lines. With its support, we analyzed 12,092 source files that were written by 506 programmers. Our study answers several interesting questions concerning code authors. For example, we find that 75.4% source files are written by multiple authors, and their contributions follow the famous 80/20 principle. These findings are useful to understand authors of source files in open source communities.

Exception-handling is a critical mechanism in many programming languages. Although it is beneficial in many programming contexts, the bugs in this mechanism can lead to disastrous consequences. In the literature, researchers have proposed various approaches to detect the bugs in handling exceptions. Meanwhile, the thrown exceptions themselves can also be buggy, but to the best of our knowledge, no prior approach has ever been proposed to detect bugs in thrown exceptions, because it is even difficult for programmers to determine whether a thrown exception is buggy or not. In this paper, we propose the first automatic criterion (meta-oracle) to determine whether a thrown exception contains a bug. Our meta-oracle says that if the messages of two thrown exceptions are quite similar, the type of the two exceptions shall be the same. Based on this meta-oracle, we implement exMiner that is able to detect bugs in thrown exceptions. It introduces sequence mining to identify similar exception messages, and groups thrown exceptions by the patterns that are mined from their messages. From each group of exception messages, it detects the ones whose types are different as bugs in thrown exceptions. For the first time, exMiner detected three bugs from the latest versions of three Apache projects. We reported this new type of bugs to their developers, and all the bugs were confirmed and fixed.

IR-based fault localization approaches achieves promising results when locating faulty files by comparing a bug report with source code. Unfortunately, they become less effective to locate faulty methods. We conduct a preliminary study to explore its challenges, and identify three problems: the semantic gap problem, the representation sparseness problem, and the single revision problem.To tackle these problems, we propose MRAM, a mixed RNN and attention model, which combines bug-fixing features and method structured features to explore both implicit and explicit relevance between methods and bug reports for method level fault localization task. The core ideas of our model are: (1) constructing code revision graphs from code, commits and past bug reports, which reveal the latent relations among methods to augment short methods and as well provide all revisions of code and past fixes to train more accurate models; (2) embedding three method structured features (token sequences, API invocation sequences, and comments) jointly with RNN and soft attention to represent source methods and obtain their implicit relevance with bug reports; and (3) integrating multi-revision bug-fixing features, which provide the explicit relevance between bug reports and methods, to improve the performance.We have implemented MRAM and conducted a controlled experiment on five open-source projects. Comparing with state-of-the-art approaches, our MRAM improves MRR values by 3.8-5.1% (3.7-5.4%) when the dataset contains (does not contain) localized bug reports. Our statistics test shows that our improvements are significant.

With the increase of software users, programmers use issue tracking systems to manage bug reports and researchers propose bug-triage approaches that assign bug reports to programmers. Programmers assign bug reports often according to their descriptions. Based on by this observation, prior approaches typically use classic natural language processing (NLP) to analyze bug reports. Although the technical choice is straightforward, the true effectiveness of this technical choice is largely unknown. Taking a state-of-the-art approach as an example, we explore the impact of textual features in bug triage. By enabling and disabling the textual features of this approach, we analyze their impacts on assigning thousands bug reports from six widely used open source projects. Our result shows that instead of improving it, textual features in fact reduce the effectiveness. In particular, after we turn off its textual features, the f-scores of the baseline approach are improved by 8%. After manual inspection, we find two reasons to explain our result: (1) classic NLP techniques are insufficient to analyze bug reports, because they are not pure natural language texts and contain other elements (e.g., code samples); and (2) some bug reports are poorly written. Our findings reveal a strong need and sufficient opportunities to explore more advanced techniques to handle these complicated elements in bug reports.

Code comments are vital for software development and maintenance. To supplement the code comments, researchers proposed various approach that generate code comments. The prior approaches take three sources: (1) programming experience, (2) code-comment pairs in source files, and (3) comments of similar code snippets. Most of their generated comments explain code functionalities, but programmers also need comments that explain why a code fragment was developed as it is. To meet the timely needs, in this paper, we introduce a new source, issue reports (e.g. maintenance types, symptoms, and purposes of modifications), to generate code comments. Issue reports contain rich information on how code was maintained. The valuable details of issue reports are useful to understand source code, especially when programmers learn why code was developed in a specific way. Towards this research direction, we propose the first approach, called ISSUECOMM, that builds the links between code fragments and issue reports. Our results show that it links more than 70% issue numbers that are written by programmers in code comments. Furthermore, the links built by our tool covers 4× bugs, and 10× other issues than the links written in manual comments. We present samples of our built links, and explain why our links are useful to describe the functionalities and the purpose of code.

Issue reports record valuable maintenance details. Developers write issue numbers into code comprehension and researchers mine knowledge from issue reports to assist various programming tasks. Although issue reports are useful, some of them can be obsolete, in that their corresponding commits are overwritten or rolled back, with the evolution of software. The obsolete issue reports can invalidate their references and descriptions, and can have far-reaching impacts on the approaches built on them. To explore their impacts, we conduct the first empirical study to analyze obsolete issue reports.To measure how an issue report becomes obsolete, we define an obsolete ratio of an issue report as its deleted lines over all its modified lines. To support our analysis, we build a tool, ICLINKER, that builds the links between an issue report and its commits, and calculates the obsolete ratio for each issue report. In our study, we analyze 70,180 commits and 46,257 issue reports that are collected from 5 Apache projects. We explore two research questions, which concern the distributions of obsolete issue reports and the obsolete references in code comments. Our findings to these research questions enrich the knowledge on obsolete issue reports, and some are even counterintuitive. For example, we find that obsolete issue reports are mixed with other issue reports. Even when recent issue reports are obsolete, some old issue reports keep up-to-date.

Developers build programs based on software libraries to reduce coding effort. If a program inappropriately sets an API parameter, the program may exhibit unexpected runtime behaviors. To help developers correctly use library APIs, researchers built tools to mine API parameter rules. However, it is still unknown (1) what types of parameter rules there are, and (2) how these rules distribute inside documents and source files. In this paper, we conducted an empirical study to investigate the above-mentioned questions. To analyze as many parameter rules as possible, we took a hybrid approach that combines automatic localization of constrained parameters with manual inspection. Our automatic approach---PaRu---locates parameters that have constraints either documented in Javadoc (i.e., document rules) or implied by source code (i.e., code rules). Our manual inspection (1) identifies and categorizes rules for the located parameters, and (2) establishes mapping between document and code rules. By applying PaRu to 9 widely used libraries, we located 5,334 parameters with either document or code rules. Interestingly, there are only 187 parameters that have both types of rules, and 79 pairs of these parameter rules are unmatched. Additionally, PaRu extracted 1,688 rule sentences from Javadoc and code. We manually classified these sentences into six categories, two of which are overlooked by prior approaches. We found that 86.2% of parameters have only code rules; 10.3% of parameters have only document rules; and only 3.5% of parameters have both document and code rules. Our research reveals the challenges for automating parameter rule extraction. Based on our findings, we discuss the potentials of prior approaches and present our insights for future tool design.

Given a bug report of a project, the task of locating the faults of the bug report is called fault localization. To help programmers in the fault localization process, many approaches have been proposed, and have achieved promising results to locate faulty files. However, it is still challenging to locate faulty methods, because many methods are short and do not have sufficient details to determine whether they are faulty. In this paper, we present BugPecker, a novel approach to locate faulty methods based on its deep learning on revision graphs. Its key idea includes (1) building revision graphs and capturing the details of past fixes as much as possible, and (2) discovering relations inside our revision graphs to expand the details for methods and calculating various features to assist our ranking. We have implemented BugPecker, and evaluated it on three open source projects. The early results show that BugPecker achieves a mean average precision (MAP) of 0.263 and mean reciprocal rank (MRR) of 0.291, which improve the prior approaches significantly. For example, BugPecker improves the MAP values of all three projects by five times, compared with two recent approaches such as DNNLoc-m and BLIA 1.5.

In software development, issue tracker systems are widely used to manage bug reports. In such a system, a bug report can be filed, diagnosed, assigned, and fixed. In the standard process, a bug can be resolved as fixed, invalid, duplicated or won't fix. Although the above resolutions are well-defined and easy to understand, a bug report can end with a less known resolution, i.e., workaround. Compared with other resolutions, the definition of workarounds is more ambiguous. Besides the problem that is reported in a bug report, the resolution of a workaround raises more questions. Some questions are important for users, especially those programmers who build their projects upon others (e.g., libraries). Although some early studies have been conducted to analyze API workarounds, many research questions on workarounds are still open. For example, which bugs are resolved as workarounds? Why is a bug report resolved as workarounds? What are the repairs of workarounds? In this experience paper, we conduct the first empirical study to explore the above research questions. In particular, we analyzed 221 real workarounds that were collected from Apache projects. Our results lead to some interesting and useful answers to all the above questions. For example, we find that most bug reports are resolved as workarounds, because their problems reside in libraries (24.43%), settings (18.55%), and clients (10.41%). Among them, many bugs are difficult to be fixed fully and perfectly. As a late breaking result, we can only briefly introduce our study, but we present a detailed plan to extend it to a full paper.

Nowadays, Apple iOS and Android are two most popular platforms for mobile applications. To attract more users, many software companies and organizations are migrating their applications from one platform to the other, and besides source files, they also need to migrate their GUI tests. The migration of GUI tests is tedious and difficult to be automated, since two platforms have subtle differences and there are often few or even no migrated GUI tests for learning. To address the problem, in this paper, we propose a novel approach, TestMig, that migrates GUI tests from iOS to Android, without any migrated code samples. Specifically, TestMig first executes the GUI tests of the iOS version, and records their GUI event sequences. Guided by the iOS GUI events, TestMig explores the Android version of the application to generate the corresponding Android event sequences. We conducted an evaluation on five well known mobile applications: 2048, SimpleNote, Wire, Wikipedia, and WordPress. The results show that, on average, TestMig correctly converts 80.2% of recorded iOS UI events to Android UI events and have them successfully executed, and our migrated Android test cases achieve similar statement coverage compared with the original iOS test cases (59.7% vs 60.4%).

Programming is typically a difficult and repetitive task. Programmers encounter endless problems during programming, and they often need to write similar code over and over again. To prevent programmers from reinventing wheels thus increase their productivity, we propose a context-aware code-to-code recommendation tool named Lancer. With the support of a Library-Sensitive Language Model (LSLM) and the BERT model, Lancer is able to automatically analyze the intention of the incomplete code and recommend relevant and reusable code samples in real-time. A video demonstration of Lancer can be found at https://youtu.be/tO9nhqZY35g. Lancer is open source and the code is available at https://github.com/sfzhou5678/Lancer.

API libraries provide thousands of APIs, and are essential in daily programming tasks. To understand their usages, it has long been a hot research topic to mine specifications that formally define legal usages for APIs. Furthermore, researchers are working on many other research topics on APIs. Although the research on APIs is intensively studied, many fundamental questions on APIs are still open. For example, the answers to open questions, such as which format can naturally define API usages and in which case, are still largely unknown. We notice that many such open questions are not concerned with concrete usages of specific APIs, but usages that describe how to use different types of APIs. To explore these questions, in this paper, we conduct an empirical study on API usages, with an emphasis on how different types of APIs are used. Our empirical results lead to nine findings on API usages. For example, we find that single-type usages are mostly strict orders, but multi-type usages are more complicated since they include both strict orders and partial orders. Based on these findings, for the research on APIs, we provide our suggestions on the four key aspects such as the challenges, the importance of different API elements, usage patterns, and pitfalls in designing evaluations. Furthermore, we interpret our findings, and present our insights on data sources, extraction techniques, mining techniques, and formats of specifications for the research of mining specifications.

Programming is typically a difficult and repetitive task. Programmers will encounter endless problems during programming, and they often need to write similar code over and over again. Over the years, many tools have been proposed to support programming. However, to the best of our knowledge, these approaches require high-quality queries or programming contexts, which are often difficult to be built or even unavailable. To address this challenge, we propose SLAMPA, a novel tool which takes advantage of statistical language model and clone detection techniques to recommend code snippets during programming. Given a piece of incomplete code, SLAMPA first infers its intention using a neural language model. Then it retrieves code snippets from codebase with the support of an efficient clone detection technology Hybrid-CD we proposed. Finally, it recommends the most similar code snippets to programmers. Our evaluation results demonstrate that Hybrid-CD precisely detects similar code snippets and it outperforms previous techniques. Our results also show that the snippets recommended by SLAMPA catch the intention of programmers and SLAMPA is capable of finding potential code reuse opportunities during programming.

Researchers have recently proposed various automatic program repair (APR) approaches that reuse past fixes to fix new bugs. However, some fundamental questions, such as how new fixes overlap with old fixes, have not been investigated. Intuitively, the overlap between old and new fixes decides how APR approaches can construct new fixes with old ones. Based on this intuition, we systematically designed six overlap metrics, and performed an empirical study on 5,735 bug fixes to investigate the usefulness of past fixes when composing new fixes. For each bug fix, we created delta dependency graphs (i.e., program dependency graphs for code changes), and identified how bug fixes overlapped with each other in terms of the content, code structure, and identifier names of fixes. Our results show that if an APR approach composes new fixes by fully or partially reusing the content of past fixes, only 2.1% and 3.2% new fixes can be created from single or multiple past fixes in the same project, compared with 0.9% and 1.2% fixes created from past fixes across projects. However, if an APR approach composes new fixes by fully or partially reusing the code structure of past fixes, up to 41.3% and 29.7% new fixes can be created.

Prior studies showed that developers applied repeated bug fixes-similar or identical code changes-to multiple locations. According to the observation, researchers built tools to automatically generate candidate patches from the repeated bug-fixing patterns. However, all such research focuses on the recurring change patterns within single methods. We are curious whether there are also repeated bug fixes that change multiple program entities (e.g., classes, methods, and fields); and if so, how we can leverage such recurring change patterns to further help developers fix bugs. In this paper, we present a comprehensive empirical study on multi-entity bug fixes in terms of their frequency, composition, and semantic meanings. Specifically for each bug fix, we first used our approach InterPart to perform static inter-procedural analysis on partial programs (i.e., the old and new versions of changed Java files), and to extract change dependency graphs (CDGs)-graphs that connect multiple changed entities based on their syntactic dependencies. By extracting common subgraphs from the CDGs of different fixes, we identified the recurring change patterns. Our study on Aries, Cassandra, Derby, and Mahout shows that (1) 52-58% of bug fixes involved multi-entity changes; (2) 6 recurring change patterns commonly exist in all projects; and (3) 19-210 entity pairs were repetitively co-changed mainly because the pairs invoked the same methods, accessed the same fields, or contained similar content. These results helped us better understand the gap between the fixes generated by existing automatic program repair (APR) approaches and the real fixes. Our observations will shed light on the follow-up research of automatic program comprehension and modification.

In recent years, automatic program repair has been a hot research topic in the software engineering community, and many approaches have been proposed. Although these approaches produce promising results, some researchers criticize that existing approaches are still limited in their repair capability, due to their limited repair templates. Indeed, it is quite difficult to design effective repair templates. An award-wining paper analyzes thousands of manual bug fixes, but summarizes only ten repair templates. Although more bugs are thus repaired, recent studies show such repair templates are still insufficient. We notice that programmers often refer to Stack Overflow, when they repair bugs. With years of accumulation, Stack Overflow has millions of posts that are potentially useful to repair many bugs. The observation motives our work towards mining repair templates from Stack Overflow. In this paper, we propose a novel approach, called SOFix, that extracts code samples from Stack Overflow, and mines repair patterns from extracted code samples. Based on our mined repair patterns, we derived 13 repair templates. We implemented these repair templates in SOFix, and conducted evaluations on the widely used benchmark, Defects4J. Our results show that SOFix repaired 23 bugs, which are more than existing approaches. After comparing repaired bugs and templates, we find that SOFix repaired more bugs, since it has more repair templates. In addition, our results also reveal the urgent need for better fault localization techniques.

Concurrent programming is pervasive in nowadays software development. Many programmers believe that concurrent programming is difficult, and maintaining concurrency code is error-prone. Although researchers have conducted empirical studies to understand concurrent programming, they still rarely study how programmers maintain concurrent code. To the best of our knowledge, only a recent study explored the modifications on critical sections, and many related questions are still open. In this paper, we conduct an empirical study to explore how programmers maintain concurrent code. We analyze more concurrency-related commits and explore more issues such as the change patterns of maintaining concurrent code than the previous study. We summarize five change patterns according to our analysis on 696 concurrency-related commits. We apply our change patterns to three open source projects, and synthesize three pull requests. Until now, two of them have been accepted. Our results can be useful for programmers to maintain concurrent code and for researchers to implement treating techniques.

With the exponentially rising of software development in the past decades, millions of software products have been created. Existing empirical studies show that many code snippets are similar. Although there exist many difficulties in maintaining these similar code snippets, we believe that it is feasible to leverage the similarity to enhance program repairs, as bugs may have already been repaired in many other similar code snippets.

To improve software quality, researchers and practitioners have proposed static analysis tools for various purposes (e.g., detecting bugs, anomalies, and vulnerabilities). Although many such tools are powerful, they typically need complete programs where all the code names (e.g., class names, method names) are resolved. In many scenarios, researchers have to analyze partial programs in bug fixes (the revised source files can be viewed as a partial program), tutorials, and code search results. As a partial program is a subset of a complete program, many code names in partial programs are unknown. As a result, despite their syntactical correctness, existing complete-code tools cannot analyze partial programs, and existing partial-code tools are limited in both their number and analysis capability. Instead of proposing another tool for analyzing partial programs, we propose a general approach, called GRAPA, that boosts existing tools for complete programs to analyze partial programs. Our major insight is that after unknown code names are resolved, tools for complete programs can analyze partial programs with minor modifications. In particular, GRAPA locates Java archive files to resolve unknown code names, and resolves the remaining unknown code names from resolved code names. To illustrate GRAPA, we implement a tool that leverages the state-of-the-art tool, WALA, to analyze Java partial programs. We thus implemented the first tool that is able to build system dependency graphs for partial programs, complementing existing tools. We conduct an evaluation on 8,198 partial-code commits from four popular open source projects. Our results show that GRAPA fully resolved unknown code names for 98.5% bug fixes, with an accuracy of 96.1% in total. Furthermore, our results show the significance of GRAPA's internal techniques, which provides insights on how to integrate with more complete-code tools to analyze partial programs.

An exception-related bug is a kind of program bug which causes exceptions. During software maintenance, when programmers repair exception-related bugs, they typically analyze thrown exceptions to understand the root causes of such bugs. When encountering unfamiliar thrown exceptions, programmers often refer to online forum threads (e.g. StackOverflow) to understand how to fix them. Although some general search engines are available and some research tools are proposed, they are insufficient to recommend threads for exception-related bugs from large-scale online resources. In this paper, we propose an approach, named EXPSOL, which recommends online threads as solutions for a newly reported exception-related bug with a model trained by support vector machines. We conduct two evaluations on thousands of threads from StackOverflow and fixed issues from GitHub. The results of our first evaluation show the significance of our internal features and highlight the importance of integrating different features. The results of our second evaluation show that, EXPSOL performs better in mean average precision, mean reciprocal rank and recall than those of the Google search engine, the internal search engine of StackOverflow, and other existing approaches.

Failures in data center networks sometimes can lead to user-perceived service interruptions. Automated failure detection is needed to maintain the reliability of data centers. However, researches rarely identify quality of service (QoS) multiplicity for failure detection in data center networks.

Multicast denotes an idea of sending data to numbers of receivers from one source in one transmission. It has been widely applied in group communication (e.g., media streaming, multi-point video conferencing). Multicast routing tree (MRT) is usually built to keep the right paths to transmit data, where data copies are created in parent nodes and then forwarded to child nodes. However, constructing an MRT is usually difficult for a given network topology; finding an optimal multicast routing tree with the minimal cost is a proven NP-complete problem. Moreover, multicast applications usually run in local or small networks due to the limitations in flexibility, scalability, and security.

Code clones are prevalent in software systems due to many factors in software development. Detecting code clones and managing consistency between them along code evolution can be very useful for reducing clone-related bugs and maintenance costs. Despite some early attempts at detecting code clones and managing the consistency between them, the state-of-the-art tool can only handle simple code clones whose structures are identical or quite similar. However, existing empirical studies show that clones can have quite different structures with their evolution, which can easily go beyond the capability of the state-of-the-art tool. In this paper, we propose CCSync, a novel, rule-directed approach, which paves the structure differences between the code clones and synchronizes them even when code clones become quite different in their structures. The key steps of this approach are, given two code clones, to (1) extract a synchronization rule from the relationship between the clones, and (2) once one code fragment is updated, propagate the modifications to the other following the synchronization rule. We have implemented a tool for CCSync and evaluated its effectiveness on five Java projects. Our results shows that there are many code clones suitable for synchronization, and our tool achieves precisions of up to 92% and recalls of up to 84%. In particular, more than 76% of our generated revisions are identical with manual revisions.

To attract more users on different platforms, many projects release their versions in multiple programming languages (e.g., Java and C#). They typically have many code snippets that implement similar functionalities, i.e., cross-language clones. Programmers often need to track and modify cross-language clones consistently to maintain similar functionalities across different language implementations. In literature, researchers have proposed approaches to detect cross- language clones, mostly for languages that share a common intermediate language (such as the .NET language family) so that techniques for detecting single-language clones can be applied. As a result, those approaches cannot detect cross-language clones for many projects that are not implemented in a .NET language. To overcome the limitation, in this paper, we propose a novel approach, CLCMiner, that detects cross-language clones automatically without the need of an intermediate language. Our approach mines such clones from revision histories, which reflect how programmers maintain cross-language clones in practice. We have implemented a prototype tool for our approach and conducted an evaluation on five open source projects that have versions in Java and C#. The results show that CLCMiner achieves high accuracy and point to promising future work.

Detecting lock-related defects has long been a hot research topic in software engineering. Many efforts have been spent on detecting such deadlocks in concurrent software systems. However, latent locks may be hidden in application programming interface (API) methods whose source code may not be accessible to developers. Many APIs have latent locks. For example, our study has shown that J2SE alone can have 2,000+ latent locks. As latent locks are less known by developers, they can cause deadlocks that are hard to perceive or diagnose. Meanwhile, the state-of-the-art tools mostly handle API methods as black boxes, and cannot detect deadlocks that involve such latent locks. In this paper, we propose a novel black-box testing approach, called LockPeeker, that reveals latent locks in Java APIs. The essential idea of LockPeeker is that latent locks of a given API method can be revealed by testing the method and summarizing the locking effects during testing execution. We have evaluated LockPeeker on ten real-world Java projects. Our evaluation results show that (1) LockPeeker detects 74.9% of latent locks in API methods, and (2) it enables state-of-the-art tools to detect deadlocks that otherwise cannot be detected.

Due to various considerations, programmers often need to backtrack their code. Furthermore, as the most recent edit may not be the wrong edit, programmers sometimes have to backtrack their code for arbitrary edits, which is referred as selective undo in this paper. To meet the needs, researchers have proposed various approaches to support selective undo. However, to the best of our knowledge, these approaches can support only simple edits, and cannot handle refactoring, although most code editors already provide various refactoring actions. Indeed, it is challenging to support selective undo for refactoring, since multiple code elements and complicated actions can be involved. In this paper, we present a novel approach that leverages Bidirectional Transformation (BX) to support selective undo for refactoring. We evaluate our approach on a recent refactoring tool that transfers enhanced for loops to lambda expressions. Our results show that our approach achieves an accuracy of up to 89%.

Software bugs can cause significant financial loss and even the loss of human lives. To reduce such loss, developers devote substantial efforts to fixing bugs, which generally requires much expertise and experience. Various approaches have been proposed to aid debugging. An interesting recent research direction is automatic program repair, which achieves promising results, and attracts much academic and industrial attention. However, people also cast doubt on the effectiveness and promise of this direction. A key criticism is to what extent such approaches can fix real bugs. As only research prototypes for these approaches are available, it is infeasible to address the criticism by evaluating them directly on real bugs. Instead, in this paper, we design and develop BUGSTAT, a tool that extracts and analyzes bug fixes. With BUGSTAT's support, we conduct an empirical study on more than 9,000 real-world bug fixes from six popular Java projects. Comparing the nature of manual fixes with automatic program repair, we distill 15 findings, which are further summarized into four insights on the two key ingredients of automatic program repair: fault localization and faulty code fix. In addition, we provide indirect evidence on the size of the search space to fix real bugs and find that bugs may also reside in non-source files. Our results provide useful guidance and insights for improving the state-of-the-art of automatic program repair.

Hosting over 10 million repositories, GitHub becomes the largest open source community in the world. Besides sharing code, Github is also a social network, in which developers can follow others or keep track of their interested projects. Considering the multi-roles of Github, integrating heterogenous data of each developer to identify experts is a challenging task. In this paper, we propose GEMiner, a novel approach to identify experts for some specific programming languages in Github. Different from previous approaches, GEMiner analyzes the social behaviors and programming behaviors of a developer to determine the expertise of the developer. When modeling social behaviors of developers, to integrate heterogenous social networks in Github, GEMiner implements a Multi-Sources PageRank algorithm. Also, GEMiner analyzes the behaviors of developers when they are programming (e.g., their commit activities and their preferred programming languages) to model programming behaviors of them. Based on our expertise models and our extracted programming languages data, GEMiner can then identify experts for some specific programming languages in Github. We conducted experiments on a real data set, and our results show that GEMiner identifies experts with 60% accuracy higher than the state-of-the-art algorithms.

Researchers have proposed various approaches to detect concurrency bugs and improve multi-threaded programs, but performing evaluations of the effectiveness of these approaches still remains a substantial challenge. We survey the existing evaluations and find out that they often use code or bugs not representative of real world. To improve representativeness, we have prepared JaConTeBe, a benchmark suite of 47 confirmed concurrency bugs from 8 popular open-source projects, supplemented with test cases for reproducing buggy behaviors. Running three approaches on JaConTeBe shows that our benchmark suite confirms some limitations of the three approaches. We submitted JaConTeBe to the SIR repository (a software-artifact repository for rigorous controlled experiments), and it was included as a part of SIR.

One of the key challenges in the software engineering lies in requirement engineering. As an important technique for modeling and analyzing requirements, software architecture has been intensively studied in recent years. Although various modeling tools have been proposed in both academy and industry, these tools typically provide limited support for analyzing non-functional requirements at architecture level. To address this problem, in this tool demo, we present a tool, called Breeze, that models, analyzes, and improves software architecture, with an emphasis on its non-functional requirements. In particular, Breeze has three key modules: (1) a modeling module that facilitates the modeling for software systems, (2) an analysis module that verifies non-functional requirements (e.g. safety, reliability and correctness) at the architecture level, and (3) a reconfiguration module that allows users to repair defects or to further improve architectures.

Crowdsourcing has attracted increasing attention from both industry and academia since it was proposed.Now a lot of work is finished by crowdsourcing, such as logo design, website promotion, industrial design, copywriting, software development, translation and image annotation.Although software crowdsourcing achieves positive results in practice, we still face a challenge of assigning suitable developers to specific tasks.In this paper, we propose a novel approach that recommends developers.In particular, our approach supports: comprehensively measuring the tasks and developers in software crowdsourcing, and recommending developers on the basis of the developer-task competence, task-task similarity, and soft power.

When programmers encounter an unfamiliar API library, they often need to refer to its documentations, tutorials, or discussions on development forums to learn its proper usage. These API documents contain valuable information, but may also mislead programmers as they may contain errors (e.g., broken code names and obsolete code samples). Although most API documents are actively maintained and updated, studies show that many new and latent errors do exist. It is tedious and error-prone to find such errors manually as API documents can be enormous with thousands of pages. Existing tools are ineffective in locating documentation errors because traditional natural language (NL) tools do not understand code names and code samples, and traditional code analysis tools do not understand NL sentences. In this paper, we propose the first approach, DOCREF, specifically designed and developed to detect API documentation errors. We formulate a class of inconsistencies to indicate potential documentation errors, and combine NL and code analysis techniques to detect and report such inconsistencies. We have implemented DOCREF and evaluated its effectiveness on the latest documentations of five widely-used API libraries. DOCREF has detected more than 1,000 new documentation errors, which we have reported to the authors. Many of the errors have already been confirmed and fixed, after we reported them.

BACKGROUND: Software engineering researchers have carried out many empirical studies on open source software (OSS) projects to understand the OSS phenomenon, and to develop better software engineering techniques. Many of these studies typically use only a few successful projects as study subjects. Recently, these studies have received criticisms and challenges on their representativeness on OSS projects. AIM: First, we aim to examine to what extent data extracted from successful projects are different from data extracted from the majority. If data extracted from successful projects are quite different from data extracted from the majority, approaches that are effective on successful projects may not be effective in general. Second, we aim to examine whether successful OSS projects are representative to the whole population of OSS. If they are not, conclusions that are drawn from only successful projects may reflect the OSS phenomenon partially. METHODOLOGY: We analyzed 11, 684 OSS projects that are hosted on Source Forge. When researchers select subjects, they typically select successful projects that are attractive to both users and developers. Considering this preference, we clustered these projects into four categories based their attractiveness to users and developers. Here, we use the K-means clustering technique to produce combined result. Furthermore, we selected eight indicators that are used in many existing studies (e.g., team sizes), and compared indicators that are extracted from different categories to investigate to what degree they are different. RESULT: For the first research aim, the result shows that 66.1% projects are under developing projects, 14.7% projects are user-preference projects, 14.2% projects are developer-preference projects, and only 5.0% projects are considered successful. For the second research aim, the result shows that all the eight analyzed indicators are highly unbalanced with the gamma distribution. Furthermore, the result reveals that users and developers of Source Forge have different perceptions on the development status defined by Source Forge. CONCLUSION: We conclude that successful projects are not representative to the whole population of OSS, and data extracted from successful projects are quite different from data extracted from the majority. The result implies that conclusions drawn from only a few successful projects may be challenged. This work is important to allow researchers to refine conclusions of existing studies, and to better understand and to carefully select OSS project subjects for their future empirical experiments.

Application Programming Interface (API) documents are a typical way of describing legal usage of reusable software libraries, thus facilitating software reuse. However, even with such documents, developers often overlook some documents and build software systems that are inconsistent with the legal usage of those libraries. Existing software verification tools require formal specifications (such as code contracts), and therefore cannot directly verify the legal usage described in natural language text in API documents against code using that library. However, in practice, most libraries do not come with formal specifications, thus hindering tool-based verification. To address this issue, we propose a novel approach to infer formal specifications from natural language text of API documents. Our evaluation results show that our approach achieves an average of 92% precision and 93% recall in identifying sentences that describe code contracts from more than 2500 sentences of API documents. Furthermore, our results show that our approach has an average 83% accuracy in inferring specifications from over 1600 sentences describing code contracts.

To address business requirements and to survive in competing markets, companies or open source organizations often have to release different versions of their projects in different languages. Manually migrating projects from one language to another (such as from Java to C#) is a tedious and error-prone task. To reduce manual effort or human errors, tools can be developed for automatic migration of projects from one language to another. However, these tools require the knowledge of how Application Programming Interfaces (APIs) of one language are mapped to APIs of the other language, referred to as API mapping relations. In this paper, we propose a novel approach, called MAM (Mining API Mapping), that mines API mapping relations from one language to another using API client code. MAM accepts a set of projects each with two versions in two languages and mines API mapping relations between those two languages based on how APIs are used by the two versions. These mined API mapping relations assist in migration of projects from one language to another. We implemented a tool and conducted two evaluations to show the effectiveness of MAM. The results show that our tool mines 25,805 unique mapping relations of APIs between Java and C# with more than 80% accuracy. The results also show that mined API mapping relations help reduce 54.4% compilation errors and 43.0% defects during migration of projects with an existing migration tool, called Java2CSharp. The reduction in compilation errors and defects is due to our new mined mapping relations that are not available with the existing migration tool.

Typically, software libraries provide API documentation, through which developers can learn how to use libraries correctly. However, developers may still write code inconsistent with API documentation and thus introduce bugs, as existing research shows that many developers are reluctant to carefully read API documentation. To find those bugs, researchers have proposed various detection approaches based on known specifications. To mine specifications, many approaches have been proposed, and most of them rely on existing client code. Consequently, these mining approaches would fail to mine specifications when client code is not available. In this paper, we propose an approach, called Doc2Spec, that infers resource specifications from API documentation. For our approach, we implemented a tool and conducted an evaluation on Javadocs of five libraries. The results show that our approach infers various specifications with relatively high precisions, recalls, and F-scores. We further evaluated the usefulness of inferred specifications through detecting bugs in open source projects. The results show that specifications inferred by Doc2Spec are useful to detect real bugs in existing projects.

Temporal specifications can describe the legal call sequences of API libraries. With these specifications, verification tools can find defects in existing clients automatically. However, temporal specifications are often not provided due to the high cost of writing them manually or being out-of-date due to the rapid evolution of software. As API clients contain many usages of libraries including temporal rules, various approaches have been proposed to automatically mine temporal specifications from these clients. Typically, only a small part of the mined specifications are real specifications because the generated traces from clients are quite large and polluted. In this paper, we analyze four types of unwanted method calls that are not useful for mining, and we refer to these method calls as polluting method calls. As these method calls are not useful for mining, it is desirable to filter out them as early as possible. To address the problem, we develop a tool, named mining accurate temporal specifications (MATS), that filters out most of the preceding polluting method calls before traces are generated. Our experiments show that with these filtering techniques, the specifications mined by MATS are more accurate than without these filtering techniques. Our experiments also show the detailed impacts of MATS¿s filtering techniques. The results provide further insight on how and why MATS improves existing specification mining.

API libraries are becoming increasingly popular in modern software industries because these libraries provide various methods and classes for reuse. However, as pointed out by researchers, libraries are typically difficult to use. It is desirable to infer some specifications for libraries so that programmers can learn the correct usages of these libraries. In this paper, we propose an approach to infer specifications from source code of API libraries. Our approach is based on the observation that rules in object-oriented programs can be traced from basic constraints such as memory usage, file usage, and network protocol. In addition, rules of one class spread to its dependent classes through the features of object-oriented programs such as derivation, invocation relationship, and field access among methods. Based on our approach, we implemented a prototype named Java Rule Finder (JRF) to infer specifications from source code of API libraries in Java. We conducted four case studies using JRF. The result shows that JRF infers some rules correctly. We further conducted an experiment on three open source API libraries. The results show that JRF scales well with real API libraries.

As a test suite usually contains redundancy, a subset of the test suite (representative set) may still satisfy all the test objectives. As the redundancy increases the cost of executing the test suite, many test suite reduction techniques have been brought out in spite of the NP-completeness of the general problem of finding the optimal representative set of the test suite. In the literature, some experimental studies of test suite reduction techniques have already been reported, but there are still shortcomings of the studies of these techniques. This paper presents an experimental comparison of the four typical test suite reduction techniques: heuristic H, heuristic GRE, genetic algorithm-based approach and ILP-based approach. The aim of the study is to provide a guideline for choosing the appropriate test suite reduction techniques.

In the process of software maintenance, it is usually a time-consuming task to track down bugs. To reduce the cost on debugging, several approaches have been proposed to localize the fault(s) to facilitate debugging. Intuitively, testing-based fault localization (TBFL), such as dicing and TRANTULA, is quite promising as it can take the advantage of a large set of execution traces at the same time. However, redundant test cases may bias the distribution of the test suite and harm this kind of approaches. Therefore, we suggest that the test suite, which is the input of TBFL, should be reduced before used in TBFL. To evaluate whether and to what extent TBFL can benefit from test suite reduction, we performed an experimental study on two source programs. The experimental results show that, for test suites containing unevenly distributed redundant test cases, performing test suite reduction before applying TBFL may be more advantageous.