Open source software (OSS) development widely adopts modern code review to identify issues and guarantee code quality. As reported repeatedly, maintainers are under heavy workloads when reviewing code changes. Meanwhile, we notice that some code reviews were overlooked by the authors of the code changes, i.e., neither causing code modification nor being replied to. These code reviews, if requiring responses but not receiving any, might represent a significant inefficiency, risk of overlooking critical issues, and problematic social exchange. Moreover, leaving code reviews publicly unanswered may cause a negative impression on both the corresponding OSS contributors and the OSS projects. Existing literature on code review mainly focuses on the usefulness of code reviews, reviewer recommendations, factors affecting PR acceptance, and review comment generation; the nature of overlooked reviews has not been explored. To this end, we focus on a widely-used modern code review mechanism, i.e., reviewing Pull Request (PR) code before merge, and conduct the first empirical study on 80 Java OSS projects to explore the prevalence, characteristics, rationales, and possible impact of the overlooked code reviews. We find that approximately 7.5% of PRs have at least one review comment being ignored. We further show that pull requests containing no-response comments are significantly associated with longer review lifecycles and lower acceptance rates, indicating measurable negative outcomes beyond their modest prevalence. Then, we categorize these no-response comments through thematic analysis and find two main categories with seven subcategories: Review inquiry and PR management. We also extract four subcategories in Review inquiry, e.g., Give suggestions about code implementation, Point out implementation issues, and Additional task requests. PR management consists of three subcategories, i.e., PR status checks, PR merge conflict notifications, and Reject PR with uncertain reasons. To better understand the existence of no-response comments, we surveyed developers and received 45 responses. We found that the reasons for the existence of no-response comments are diverse, such as prolonged review times and a lack of consensus on opinions. Developers also hold the consensus that ignored reviews will have negative effects on software projects. These findings emphasize the need for attention from both academia and industry to the responses to review comments and optimization of the reminder mechanism.

Large language models (LLMs) have demonstrated impressive performance in code generation, particularly when augmented with chain-of-thought (CoT) prompting techniques. They break down requirements into intermediate reasoning steps, which act as design rationales to guide LLMs in writing code like human programmers. Thus, the quality of these steps is crucial for ensuring the correctness and reliability of the generated code. However, the specific factors influencing the quality of CoT generated by LLMs remain largely unexplored. To what extent can we trust the thoughts generated by LLMs? How good are they? This paper empirically explores the external and internal factors of why LLMs generate unsatisfactory CoTs by analyzing 1,023 failed code samples on two widely used code generation benchmarks. We also evaluate their impact on code generation performance by analyzing 210 CoT-code pairs and refining the unsatisfied CoTs by prompting LLMs. Our study yields the following findings: 1) Among the factors affecting CoT quality, external factors account for 53.60%, primarily including unclear requirements and lack of contextual information. Internal factors make up 40.10%, mainly due to inconsistencies between CoT and prompts caused by LLMs’ misunderstanding of the instructions. 2) Despite CoT being correct, 18.5% of the generated code still contains errors. This is primarily due to LLMs failing to follow instructions, leading to inconsistencies between CoT and the code. Additionally, we found that even when the code is correct, there is an 11.90% chance that the CoT contains errors. 3) Our further research on refining the low-quality CoTs reveals that LLMs can improve CoT, especially when providing detailed CoT problem information. Our findings shed light on the underlying issues that hinder the effectiveness of CoT in LLM-based code generation, offering valuable insights for enhancing both the reasoning process and the overall reliability of code generation.

Large Language Models (LLMs) have demonstrated remarkable capabilities in code editing, substantially enhancing software development productivity. However, the inherent complexity of code editing tasks forces existing approaches to rely on LLMs’ autoregressive end-to-end generation, where decoding speed plays a critical role in efficiency. While inference acceleration techniques like speculative decoding are applied to improve the decoding efficiency, these methods fail to account for the unique characteristics of code editing tasks, where changes are typically localized and existing code segments are reused. To address this limitation, we propose EfficientEdit, a novel method that improves LLM-based code editing efficiency through two key mechanisms based on speculative decoding: (1) effective reuse of original code segments while identifying potential edit locations, and (2) efficient generation of edit content via high-quality drafts from edit-oriented draft models and a dynamic verification mechanism that balances quality and acceleration. Experimental results show that EfficientEdit can achieve up to 10.38× and 13.09× speedup compared to standard autoregressive decoding in CanItEdit and CodeIF-Bench, respectively, outperforming state-of-the-art inference acceleration approaches by up to 90.6%. The code and data are available at https://github.com/zhu-zhu-ding/EfficientEdit.

Code generation is a latency-sensitive task that demands high timeliness. However, with the growing interest and inherent difficulty in repository-level code generation, most existing code generation studies focus on improving the correctness of generated code while overlooking the inference efficiency, which is substantially affected by the overhead during LLM generation. Although there has been work on accelerating LLM inference, these approaches are not tailored to the specific characteristics of code generation; instead, they treat code the same as natural language sequences and ignore its unique syntax and semantic characteristics, which are also crucial for improving efficiency. Consequently, these approaches exhibit limited effectiveness in code generation tasks, particularly for repository-level scenarios with considerable complexity and difficulty. To alleviate this issue, following draft-verification paradigm, we propose FastCoder, a simple yet highly efficient inference acceleration approach specifically designed for code generation, without compromising the quality of the output. FastCoder constructs a multi-source datastore, providing access to both general and project-specific knowledge, facilitating the retrieval of high-quality draft sequences. Moreover, FastCoder reduces the retrieval cost by controlling retrieval timing, and enhances efficiency through parallel retrieval and a context- and LLM preference-aware cache. Experimental results show that FastCoder can reach up to 2.53× and 2.54× speedup compared to autoregressive decoding in repository-level and standalone code generation tasks, respectively, outperforming state-of-the-art inference acceleration approaches by up to 88%. FastCoder can also be integrated with existing correctness-focused code generation approaches to accelerate the LLM generation process, and reach a speedup exceeding 2.6×.

Abstract Researchers have recently achieved significant advances in deep learning techniques, which in turn has substantially advanced other research disciplines, such as natural language processing, image processing, speech recognition, and software engineering. Various deep learning techniques have been successfully employed to facilitate software engineering tasks, including code generation, software refactoring, and fault localization. Many studies have also been presented in top conferences and journals, demonstrating the applications of deep learning techniques in resolving various software engineering tasks. However, although several surveys have provided overall pictures of the application of deep learning techniques in software engineering, they focus more on learning techniques, that is, what kind of deep learning techniques are employed and how deep models are trained or fine-tuned for software engineering tasks. We still lack surveys explaining the advances of subareas in software engineering driven by deep learning techniques, as well as challenges and opportunities in each subarea. To this end, in this study, we present the first task-oriented survey on deep learning-based software engineering. It covers twelve major software engineering subareas significantly impacted by deep learning techniques. Such subareas spread out through the whole lifecycle of software development and maintenance, including requirements engineering, software development, testing, maintenance, and developer collaboration. As we believe that deep learning may provide an opportunity to revolutionize the whole discipline of software engineering, providing one survey covering as many subareas as possible in software engineering can help future research push forward the frontier of deep learning-based software engineering more systematically. For each of the selected subareas, we highlight the major advances achieved by applying deep learning techniques with pointers to the available datasets in such a subarea. We also discuss the challenges and opportunities concerning each of the surveyed software engineering subareas.

Automatic API recommendation can accelerate developers’ programming and has been studied for years. There are two orthogonal lines of approaches for this task, i.e., information retrieval-based (IR-based) approaches and sequence to sequence (seq2seq) model-based approaches. Although these approaches were reported to have remarkable performance, our observation finds two major drawbacks, i.e., IR-based approaches lack the consideration of relations among the recommended APIs, and seq2seq models do not model the API’s semantic meaning. To alleviate the above two problems, we propose APIGens, which is a retrieval-enhanced large language model (LLM)-based API recommendation approach to recommend an API sequence for a natural language query. The approach first retrieves similar programming questions in history based on the input natural language query, and then scores the results based on API documents via a scorer model. Finally, these results are used as samples for few-shot learning of LLM. To reduce the risk of encountering local optima, we also extract API seeds from the retrieved results to increase the search scope during the LLM generation process. The results show that our approach can achieve 48.41% ROUGE@10 on API sequence recommendation and the 82.61% MAP on API set recommendation, largely outperforming the state-of-the-art baselines.

Security patches are essential for enhancing the stability and robustness of projects in the open-source software community. While vulnerabilities are officially expected to be patched before being disclosed, patching vulnerabilities is complicated and remains a struggle for many organizations. To patch vulnerabilities, security practitioners typically track vulnerable issue reports (IRs), and analyze their relevant insecure code to generate potential patches. However, the relevant insecure code may not be explicitly specified and practitioners cannot track the insecure code in the repositories, thus limiting their ability to generate patches. In such cases, providing examples of insecure code and the corresponding patches would benefit the security developers to better locate and resolve the actual insecure code. In this paper, we propose PatUntrack, an automated approach to generating patch examples from IRs without tracked insecure code. PatUntrack utilizes auto-prompting to optimize the Large Language Model (LLM) to make it applicable for analyzing the vulnerabilities described in IRs and generating appropriate patch examples. Specifically, it first generates the completed description of the Vulnerability-Triggering Path (VTP) from vulnerable IRs. Then, it corrects potential hallucinations in the VTP description with external golden knowledge. Finally, it generates Top-K pairs of Insecure Code and Patch Example based on the corrected VTP description. To evaluate the performance of PatUntrack, we conducted experiments on 5,465 vulnerable IRs. The experimental results show that PatUntrack can obtain the highest performance and improve the traditional LLM baselines by +17.7% (MatchFix) and +14.6% (Fix@10) on average in patch example generation. Furthermore, PatUntrack was applied to generate patch examples for 76 newly disclosed vulnerable IRs. 27 out of 37 replies from the authors of these IRs confirmed the usefulness of the patch examples generated by PatUntrack, indicating that they can benefit from these examples for patching the vulnerabilities.

Detecting and refactoring code smells is challenging, laborious, and sustaining. Although large language models have demonstrated potential in identifying various types of code smells, they also have limitations such as input-output token restrictions, difficulty in accessing repository-level knowledge, and performing dynamic source code analysis. Existing learning-based methods or commercial expert toolsets have advantages in handling complex smells. They can analyze project structures and contextual information in-depth, access global code repositories, and utilize advanced code analysis techniques. However, these toolsets are often designed for specific types and patterns of code smells and can only address fixed smells, lacking flexibility and scalability. To resolve that problem, we propose iSMELL, an ensemble approach that employs various code smell detection toolsets via Mixture of Experts (MoE) architecture for comprehensive code smell detection, and enhances the LLMs with the detection results from expert toolsets for refactoring those identified code smells. First, we train a MoE model that, based on input code vectors, outputs the most suitable expert tool for identifying each type of smell. Then, we select the recommended toolsets for code smell detection and obtain their results. Finally, we equip the prompts with the detection results from the expert toolsets, thereby enhancing the refactoring capability of LLMs for code with existing smells, enabling them to provide different solutions based on the type of smell. We evaluate our approach on detecting and refactoring three classical and complex code smells, i.e., Refused Bequest, God Class, and Feature Envy. The results show that, by adopting seven expert code smell toolsets, iSMELL achieved an average F1 score of 75.17% on code smell detection, outperforming LLMs baselines by an increase of 35.05% in F1 score. We further evaluate the code refactored by the enhanced LLM. The quantitative and human evaluation results show that iSMELL could improve code quality metrics and conduct satisfactory refactoring toward the identified code smells. We believe that our proposed solution could provide new insights into better leveraging LLMs and existing approaches to resolving complex software tasks.

Machine learning (ML) has been increasingly used in a variety of domains, while solving ML programming tasks poses unique challenges due to the fundamental difference in the nature and the construct of general programming tasks, especially for developers who do not have ML backgrounds. Automatic code generation that produces a code snippet from a natural language description can be a promising technique to accelerate ML programming tasks. In recent years, although many deep learning-based neural code generation models have been proposed with high accuracy, the fact that most of them are mainly evaluated on general programming tasks calls into question their effectiveness and usefulness in ML programming tasks. In this article, we set out to investigate the effectiveness of existing neural code generation models on ML programming tasks. For our analysis, we select six state-of-the-art neural code generation models and evaluate their performance on four widely used ML libraries, with newly created 83K pairs of natural-language described ML programming tasks. Our empirical study reveals some good, bad, and missing aspects of neural code generation models on ML tasks, with a few major ones listed below. ( Good ) Neural code generation models perform significantly better on ML tasks than on non-ML tasks with an average difference of 10.6 points in BLEU-4 scores. ( Bad ) More than 80% of the generated code is semantically incorrect. ( Bad ) Code generation models do not have significance in improving developers’ completion time. ( Good ) The generated code can help developers write correct code by providing developers with clues for using correct APIs. ( Missing ) The observation from our user study reveals the missing aspects of code generation for ML tasks, e.g., decomposing code generation for divide-and-conquer into API sequence identification and API usage generation.

Recently, a growing body of research has realized that live chat via modern communication platforms plays an increasingly important role in OSS (Open Source Software) collaborative development. Among these platforms, Gitter has emerged as a popular choice since it is directed toward GitHub projects by account sharing and activity subscribing. But little is known about how Gitter affects the OSS development on GitHub. Who are the developers being active in both social and technical platforms? How important are they? In this paper, we perform a comprehensive cross-platform study on Gitter and GitHub, two representative platforms for live communication and distributed development, to explore the characteristics of cross-platform contributors (CPCs) and whether live chat can provoke open source development. This study yields interesting findings: 1) Despite CPCs being small in quantity yet account for a much bigger amount of communication and development; 2) Gitter continually attracts new contributors; 3) Communication on Gitter has a positive impact on the contributions of OSS developers; and 4) Inactive developers on GitHub still participate in discussions on Gitter. Based on our findings, we provide recommendations for OSS communities and developers and shed light on future research directions. We believe that the findings and insights will inspire the OSS communities, enable a broader view of the interplay between Gitter and GitHub, and enhance the sustainability of the OSS ecosystem.

Existing automatic code comment generators mainly focus on producing a general description of functionality for a given code snippet without considering developer intentions. However, in real-world practice, comments are complicated, which often contain information reflecting various intentions of developers, e.g., functionality summarization, design rationale, implementation details, code properties, etc. To bridge the gap between automatic code comment generation and real-world comment practice, we define Developer-Intent Driven Code Comment Generation, which can generate intent-aware comments for the same source code with different intents. To tackle this challenging task, we propose DOME, an approach that utilizes Intent-guided Selective Attention to explicitly select intent-relevant information from the source code, and produces various comments reflecting different intents. Our approach is evaluated on two real-world Java datasets, and the experimental results show that our approach outperforms the state-of-the-art baselines. A human evaluation also confirms the significant potential of applying DOME in practical usage, enabling developers to comment code effectively according to their own needs.

Secure coding practices (SCPs) have been proposed to guide software developers to write code securely to prevent potential security vulnerabilities. Yet, they are typically one-sentence principles without detailed specifications, e.g., “Properly free allocated memory upon the completion of functions and at all exit points.”, which makes them difficult to follow in practice, especially for software developers who are not yet experienced in secure programming. To address this problem, this paper proposes SCPatcher, an automated approach to enrich secure coding practices by mining crowd security discussions on online knowledge-sharing platforms, such as Stack Overflow. In particular, for each security post, SCPatcher first extracts the area of coding examples and coding explanations with a fix-prompt tuned Large Language Model (LLM) via Prompt Learning. Then, it hierarchically slices the lengthy code into coding examples and summarizes the coding explanations with the areas. Finally, SCPatcher matches the CWE and Public SCP, integrating them with extracted coding examples and explanations to form the SCP specifications, which are the wild SCPs with details, proposed by the developers. To evaluate the performance of SCPatcher, we conduct experiments on 3,907 security posts from Stack Overflow. The experimental results show that SCPatcher outperforms all baselines in extracting the coding examples with 2.73 % MLine on average, as well as coding explanations with 3.97 % F1 on average. Moreover, we apply SCPatcher on 447 new security posts to further evaluate its practicality, and the extracted SCP specifications enrich the public SCPs with 3,074 lines of code and 1,967 sentences.

In community-based software development, developers frequently rely on live-chatting to discuss emergent bugs/errors they encounter in daily development tasks. However, it remains a challenging task to accurately record such knowledge due to the noisy nature of interleaved dialogs in live chat data. In this paper, we first formulate the task of identifying and synthesizing bug reports from community live chats, and propose a novel approach, named BugListener, to address the challenges. Specifically, BugListener automates three sub-tasks: 1) Disentangle the dialogs from massive chat logs by using a Feed-Forward neural network; 2) Identify the bug-report dialogs from separated dialogs by leveraging the Graph neural network to learn the contextual information; 3) Synthesize the bug reports by utilizing Transfer Learning techniques to classify the sentences into: observed behaviors (OB), expected behaviors (EB), and steps to reproduce the bug (SR). BugListener is evaluated on six open source projects. The results show that: for bug report identification, BugListener achieves the average F1 of 77.74%, improving the best baseline by 12.96%; and for bug report synthesis task, BugListener could classify the OB, EB, and SR sentences with the F1 of 84.62%, 71.46%, and 73.13%, improving the best baselines by 9.32%, 12.21%, 10.91%, respectively. A human evaluation study also confirms the effectiveness of BugListener in generating relevant and accurate bug reports. These demonstrate the significant potential of applying BugListener in community-based software development, for promoting bug discovery and quality improvement.

User reviews of mobile apps provide a communication channel for developers to perceive user satisfaction. Many app features that users have problems with are usually expressed by key phrases such as "upload pictures", which could be buried in the review texts. The lack of fine-grained view about problematic features could obscure the developers' understanding of where the app is frustrating users, and postpone the improvement of the apps. Existing pattern-based approaches to extract target phrases suffer from low accuracy due to insufficient semantic understanding of the reviews, thus can only summarize the high-level topics/aspects of the reviews. This paper proposes a semantic-aware, fine-grained app review analysis approach (SIRA) to extract, cluster, and visualize the problematic features of apps. The main component of SIRA is a novel BERT+Attr-CRF model for fine-grained problematic feature extraction, which combines textual descriptions and review attributes to better model the semantics of reviews and boost the performance of the traditional BERT-CRF model. SIRA also clusters the extracted phrases based on their semantic relations and presents a visualization of the summaries. Our evaluation on 3,426 reviews from six apps confirms the effectiveness of SIRA in problematic feature extraction and clustering. We further conduct an empirical study with SIRA on 318,534 reviews of 18 popular apps to explore its potential application and examine its usefulness in real-world practice.

The main goal of dialogue disentanglement is to separate the mixed utterances from a chat slice into independent dialogues. Existing models often utilize either an utterance-to-utterance (U2U) prediction to determine whether two utterances that have the “reply-to” relationship belong to one dialogue, or an utterance-to-thread (U2T) prediction to determine which dialogue-thread a given utterance should belong to. Inspired by mutual leaning, we propose MuiDial, a novel dialogue disentanglement model, to exploit the intent of each utterance and feed the intent to a mutual learning U2U-U2T disentanglement model. Experimental results and in-depth analysis on several benchmark datasets demonstrate the effectiveness and generalizability of our approach.

Deliberation is a common and natural behavior in human daily life. For example, when writing papers or articles, we usually first write drafts, and then iteratively polish them until satisfied. In light of such a human cognitive process, we propose DECOM, which is a multi-pass deliberation framework for automatic comment generation. DECOM consists of multiple Deliberation Models and one Evaluation Model. Given a code snippet, we first extract keywords from the code and retrieve a similar code fragment from a pre-defined corpus. Then, we treat the comment of the retrieved code as the initial draft and input it with the code and keywords into DECOM to start the iterative deliberation process. At each deliberation, the deliberation model polishes the draft and generates a new comment. The evaluation model measures the quality of the newly generated comment to determine whether to end the iterative process or not. When the iterative process is terminated, the best-generated comment will be selected as the target comment. Our approach is evaluated on two real-world datasets in Java (87K) and Python (108K), and experiment results show that our approach outperforms the state-of-the-art baselines. A human evaluation study also confirms the comments generated by DECOM tend to be more readable, informative, and useful.

Code summarization, the task of generating useful comments given the code, has long been of interest. Most of the existing code summarization models are trained and validated on widely-used code comment benchmark datasets. However, little is known about the quality of the benchmark datasets built from real-world projects. Are the benchmark datasets as good as expected? To bridge the gap, we conduct a systematic research to assess and improve the quality of four benchmark datasets widely used for code summarization tasks. First, we propose an automated code-comment cleaning tool that can accurately detect noisy data caused by inappropriate data preprocessing operations from existing benchmark datasets. Then, we apply the tool to further assess the data quality of the four benchmark datasets, based on the detected noises. Finally, we conduct comparative experiments to investigate the impact of noisy data on the performance of code summarization models. The results show that these data preprocessing noises widely exist in all four benchmark datasets, and removing these noisy data leads to a significant improvement on the performance of code summarization. We believe that the findings and insights will enable a better understanding of data quality in code summarization tasks, and pave the way for relevant research and practice.

Traceability links between requirements and source code are beneficial to the maintenance and evolution activities. Compared with the proposed unsupervised solutions, supervised solutions are more effective to generate trace links automatically and gaining more attention. However, supervised solutions often need to spend a lot of effort on labeling data. To overcome this limitation, we propose a self-enhanced automatic traceability link recovery approach based on structure knowledge mining for small-scale labeled data, named K2Trace, which not only enhances the semantic representations of artifacts by mining context information but also self-enhances the size of training set by exploring transitive relationships. Evaluation results show that K2Trace can outperform the state-of-the-art baseline approach. K2Trace proves the usefulness of mining knowledge from the structure information of software artifacts, as well as provides a new way to substantially reduce the amount of training data needed for training efficient classification models, which may pave the way for generating accurate trace links.

Despite the valuable information contained in software chat messages, disentangling them into distinct conversations is an essential prerequisite for any in-depth analyses that utilize this information. To provide a better understanding of the current state-of-the-art, we evaluate five popular dialog disentanglement approaches on software-related chat. We find that existing approaches do not perform well on disentangling software-related dialogs that discuss technical and complex topics. Further investigation on how well the existing disentanglement measures reflect human satisfaction shows that existing measures cannot correctly indicate human satisfaction on disentanglement results. Therefore, in this paper, we introduce and evaluate a novel measure, named DLD. Using results of human satisfaction, we further summarize four most frequently appeared bad disentanglement cases on software-related chat to insight future improvements. These cases include (i) Ignoring Interaction Patterns, (ii) Ignoring Contextual Information, (iii) Mixing up Topics, and (iv) Ignoring User Relationships. We believe that our findings provide valuable insights on the effectiveness of existing dialog disentanglement approaches and these findings would promote a better application of dialog disentanglement in software engineering.

Collaborative live chats are gaining popularity as a development communication tool. In community live chatting, developers are likely to post issues they encountered (e.g., setup issues and compile issues), and other developers respond with possible solutions. Therefore, community live chats contain rich sets of information for reported issues and their corresponding solutions, which can be quite useful for knowledge sharing and future reuse if extracted and restored in time. However, it remains challenging to accurately mine such knowledge due to the noisy nature of interleaved dialogs in live chat data. In this paper, we first formulate the problem of issue-solution pair extraction from developer live chat data, and propose an automated approach, named ISPY, based on natural language processing and deep learning techniques with customized enhancements, to address the problem. Specifically, ISPY automates three tasks: 1) Disentangle live chat logs, employing a feedforward neural network to disentangle a conversation history into separate dialogs automatically; 2) Detect dialogs discussing issues, using a novel convolutional neural network (CNN), which consists of a BERT-based utterance embedding layer, a context-aware dialog embedding layer, and an output layer; 3) Extract appropriate utterances and combine them as corresponding solutions, based on the same CNN structure but with different feeding inputs. To evaluate ISPY, we compare it with six baselines, utilizing a dataset with 750 dialogs including 171 issue-solution pairs and evaluate ISPY from eight open source communities. The results show that, for issue-detection, our approach achieves the F1 of 76%, and outperforms all baselines by 30%. Our approach achieves the F1 of 63% for solution-extraction and outperforms the baselines by 20%. Furthermore, we apply ISPY on three new communities to extensively evaluate ISPY’s practical usage. Moreover, we publish over 30K issue-solution pairs extracted from 11 communities. We believe that ISPY can facilitate community-based software development by promoting knowledge sharing and shortening the issue-resolving process.

Background: When submitting patches for code review, a patch is expected as the patch that is accepted in the fewest reworks (rounds of revisions) possibly. However, in practice, not all submitted patches can be accepted and integrated into the codebase the first time, since their quality is unsatisfactory or even poor, which might not pass patch review or need multiple revisions. Aims: In order to improve the effectiveness and efficiency of modern code review, this paper aims to identify the factors that impact the the patch acceptance and the rounds of revisions. Method: We first extract 29 factors from four categories: author, patch, static code quality and review. Then, we apply statistical approach to examine which of and how these factors impact the patch acceptance and the rounds of revisions, respectively. Results: Through case studies of 16,950 patches of 18 projects from Android and Eclipse systems, the result shows that: 1) the number of accepted/reviewed patches by an author has a statistically significant effect on the patch acceptance; 2) the kinds of languages and the number of invited reviewers have a statistically significant effect on the rounds of the revisions; 3) static code quality of patches has little impact on both patch acceptance and the rounds of revisions, although complexity has impact on the number of invited reviewers. Besides, there are other factors that have impact on the patch acceptance and rounds of revisions for different systems. Conclusions: Based on the analysis result, this study offers some suggestions for researchers and practitioners to improve the quality of submitted patches and the process of code review.

Abstractions are significant domain terms that have assisted in requirements elicitation and modeling. To extend the assistance towards requirements validation, we present in this paper an automated approach to identifying the abstractions for supporting requirements-based testing. We select relevant Wikipedia pages to serve as a domain corpus that is independent from any specific software system. We further define five novel patterns based on part-of-speech tagging and dependency parsing, and frame our candidate abstractions in the form ofpairs for better testability. We evaluate our approach with six software systems in two application domains: Electronic health records and Web conferencing. The results show that our abstractions are more accurate than those generated by two of the state-of-the-art techniques. Initial findings also indicate our abstractions’ capabilities of revealing bugs and matching the environmental assumptions created manually.

Modern communication platforms such as Gitter and Slack play an increasingly critical role in supporting software teamwork, especially in open source development.Conversations on such platforms often contain intensive, valuable information that may be used for better understanding OSS developer communication and collaboration. However, little work has been done in this regard. To bridge the gap, this paper reports a first comprehensive empirical study on developers' live chat, investigating when they interact, what community structures look like, which topics are discussed, and how they interact. We manually analyze 749 dialogs in the first phase, followed by an automated analysis of over 173K dialogs in the second phase. We find that developers tend to converse more often on weekdays, especially on Wednesdays and Thursdays (UTC), that there are three common community structures observed, that developers tend to discuss topics such as API usages and errors, and that six dialog interaction patterns are identified in the live chat communities. Based on the findings, we provide recommendations for individual developers and OSS communities, highlight desired features for platform vendors, and shed light on future research directions. We believe that the findings and insights will enable a better understanding of developers' live chat, pave the way for other researchers, as well as a better utilization and mining of knowledge embedded in the massive chat history.

Online chatting is gaining popularity and plays an increasingly significant role in software development. When discussing functionalities, developers might reveal their desired features to other developers. Automated mining techniques towards retrieving feature requests from massive chat messages can benefit the requirements gathering process. But it is quite challenging to perform such techniques because detecting feature requests from dialogues requires a thorough understanding of the contextual information, and it is also extremely expensive on annotating feature-request dialogues for learning. To bridge that gap, we recast the traditional text classification task of mapping single dialog to its class into the task of determining whether two dialogues are similar or not by incorporating few-shot learning. We propose a novel approach, named FRMiner, which can detect feature-request dialogues from chat messages via deep Siamese network. We design a BiLSTM-based dialog model that can learn the contextual information of a dialog in both forward and reverse directions. Evaluation on the real-world projects shows that our approach achieves average precision, recall and F1-score of 88.52%, 88.50% and 88.51%, which confirms that our approach could effectively detect hidden feature requests from chat messages, thus can facilitate gathering comprehensive requirements from the crowd in an automated way.

Requirement entities, "explicit specification of concepts that define the primary function objects", play an important role in requirement analysis for software development and maintenance. It is a labor-intensive activity to extract requirement entities from textual requirements, which is typically done manually. A few existing studies propose automated methods to support key requirement concept extraction. However, they face two main challenges: lack of domain-specific natural language processing techniques and expensive labeling effort. To address the challenges, this study presents a novel approach named RENE, which employs LSTM-CRF model for requirement entity extraction and introduces the general knowledge to reduce the demands for labeled data. It consists of four phases: 1) Model construction, where RENE builds LSTM-CRF model and an isomorphic LSTM language model for transfer learning; 2) LSTM language model training, where RENE captures general knowledge and adapt to requirement context; 3) LSTM-CRF training, where RENE trains the LSTM-CRF model with the transferred layers; 4) Requirement entity extraction, where RENE applies the trained LSTM-CRF model to a new-coming requirement, and automatically extracts its requirement entities. RENE is evaluated using two methods: evaluation on historical dataset and user study. The evaluation on the historical dataset shows that RENE could achieve 79% precision, 81% recall, and 80% F1. The evaluation results from the user study also suggest that RENE could produce more accurate and comprehensive requirement entities, compared with those produced by engineers.

The ability in rapidly learning and adapting to evolving user needs is key to modern business successes. Existing methods are based on text mining and machine learning techniques to analyze user comments and feedback, and often constrained by heavy reliance on manually codified rules or insufficient training data. Multitask learning (MTL) is an effective approach with many successful applications, with the potential to address these limitations associated with requirements analysis tasks. In this paper, we propose a deep MTL-based approach, DEMAR, to address these limitations when discovering requirements from massive issue reports and annotating the sentences in support of automated requirements analysis. DEMAR consists of three main phases: (1) data augmentation phase, for data preparation and allowing data sharing beyond single task learning; (2) model construction phase, for constructing the MTL-based model for requirements discovery and requirements annotation tasks; and (3) model training phase, enabling eavesdropping by shared loss function between the two related tasks. Evaluation results from eight open-source projects show that, the proposed multitask learning approach outperforms two state-of-the-art approaches (CNC and FRA) and six common machine learning algorithms, with the precision of 91% and the recall of 83% for requirements discovery task, and the overall accuracy of 83% for requirements annotation task. The proposed approach provides a novel and effective way to jointly learn two related requirements analysis tasks. We believe that it also sheds light on further directions of exploring multitask learning in solving other software engineering problems.

Utilizing massive user feedback, e.g. feature requests from Bugzilla, JIRA, or GitHub, to motivate software evolution has become a new trend in RE community. However, manually understanding and analyzing feature requests from issue tracking systems is a time-consuming and labor-intensive task. In this paper, we present NERO (coNtent annotation and smElly Feature Requests detection), an automated tool to support analysts to understand the semantic meaning of feature requests and detect the smells in feature requests. It can also provide an overall score based on the smell detection results to help analysts quickly judge the quality of feature requests.

Requirements are usually written by different stakeholders with diverse backgrounds and skills and evolve continuously. Therefore inconsistency caused by specialized jargons and different domains, is inevitable. In particular, entity coreference in Requirement Engineering (RE) is that different linguistic expressions refer to the same real-world entity. It leads to misconception about technical terminologies, and impacts the readability and understandability of requirements negatively. Manual detection entity coreference is labor-intensive and time-consuming. In this paper, we propose a DEEP context-wise semantic method named DeepCoref to entity COREFerence detection. It consists of one fine-tuning BERT model for context representation and a Word2Vec-based network for entity representation. We use a multi-layer perception in the end to fuse and make a trade-off between two representations for obtaining a better representation of entities. The input of the network is requirement contextual text and related entities, and the output is the predictive label to infer whether two entities are coreferent. The evaluation on industry data shows that our approach significantly outperforms three baselines with average precision and recall of 96.10% and 96.06% respectively. We also compare DeepCoref with three variants to demonstrate the performance enhancement from different components.

In practice, it is very important to determine the size of a proposed software system yet to be built based on its requirements, i.e., early in the development life cycle. The most widely used approach for size estimation is Function Point Analysis (FPA). However, since FPA involves human judgment, the estimation results are some degree of subjective, and the process is labor and cost intensive. In this paper, we propose a novel approach to identify transaction functions from textual requirements automatically by leveraging a set of natural language processing techniques and machine learning models. We evaluate our approach on 1,864 requirements and 104,691 transaction functions taken from 36 financial projects from one banking industry. The results show that the contents of the suggested transaction functions by our approach are high in quality, with low perplexity value of 8.5 and high BLEU score of 34 on average. The types of suggested transaction functions can also be accurately classified, with overall accuracy of 0.99 on average. Our approach can provide reasonable suggestions that assist industrial practitioners to identify transaction functions faster and easier.

In open source communities, there are numerous duplicate issue reports, considered as useless and negligible by developers. Conversely, some researches argued that duplicates deliver complementary information that could benefit issue resolving. Considering all duplicates as value-neutral will result in either overestimation or underestimation of valuable information. It is necessary to be aware of whether all duplicates are redundant or beneficial. In this paper, we investigate whether duplicates have the same impacts on issue resolving and identification cost. We divide duplicates into three categories according to the statuses of master reports when duplicates are submitted. The results show duplicates in different categories play different roles in issue-resolving, and identification cost is also significantly different. Our study reveals duplicates are different, but almost are paid equal attentions. It is promising to propose new approaches and tools to resolve the problem.

Beyond the functional requirements of a system, software maintainability is essential for project success. While there exists a large knowledge base of software maintainability, this knowledge is rarely used in open source software due to the large number of developers and inefficiency in identifying quality issues. To effectively utilize the current knowledge base in practice requires a deeper understanding of how problems associated with the different qualities arise and change over time. In this paper, we sample over 6000 real bugs found from several Mozilla products to examine how maintainability is expressed with subgroups of repairability and modifiability. Furthermore, we manually study how these qualities evolve as the products mature, what the root causes of the bugs are for each quality and the impact and dependency of each quality. Our results inform which areas should be focused on to ensure maintainability at different stages of the development and maintenance process.

Kernel rootkits pose a serious threat to system security by tampering with the state of operating system inconspicuously. To ensure operating system kernel integrity, Virtual Machine Monitor (VMM) based approaches have been proposed. Most of these approaches use snapshot-based or event-triggered techniques. However, snapshot-based techniques have been suffering from missing transient attacks or significant performance overhead, while event-triggered methods are facing with heavy workload as integrity checking might be triggered by any suspicious actions. In this paper, we propose a novel solution which is a behavior-triggered integrity checking approach named BehaviorKI. By analyzing attacking processes, BehaviorKI can extract a set of behavior patterns which characterize malicious behaviors. BehaviorKI will trigger integrity checking with kernel invariants when a malicious behavior pattern detected. In this way, our approach can alleviate the performance burden by reducing the frequent kernel integrity checking. The experiment results show that Be-haviorKI outperforms existing snapshot-based and event-triggered approaches.

The development of High-level programming languages(HLPL) is a long process of evolution, which has gone through procedure-oriented languages, object-oriented languages, script languages and visual & database languages. Throughout the process of evolution, coding in an efficient and convenient way is the primary impetus. Hence, the productivity should vary across different languages. When evaluating the productivity of developers coding in different programming languages, such variations are usually ignored, which may not reflect their actual coding efficiencies and make the developers feel unfair. Especially, ignoring the variations will lead to inappropriate baselines of process performance, thus may potentially reduce the effectiveness of quantitative management. In this paper, we conducted an empirical study to revisit the productivity variations across different programming languages based on the data of International Software Benchmarking Standards Group (ISBSG) and a software organization. We found that, in most language categories, the productivity is all significantly different in ISBSG and the organization respectively. In the software industry represented by ISBSG, the relative productivity levels are gradually increasing with the evolution of HLPL. However, for the organization, it does not always keep the same increasing trend, but the average productivity within the four categories is almost stable. The finding could guide the software organization to establish an appropriate productivity baseline.

In open software development environment, a large number of feature requests with mixed quality are often posted by stakeholders and usually managed in issue tracking systems. Thoroughly understanding and analyzing the real intents that feature requests imply is a labor-intensive and challenging task. In this paper, we introduce an approach to understand feature requests automatically. We generate a set of fuzzy rules based on natural language processing techniques that classify each sentence in feature requests into a set of categories: Intent, Explanation, Benefit, Drawback, Example and Trivia. Consequently, the feature requests can be automatically structured based on the classification results. We conduct experiments on 2,112 sentences taken from 602 feature requests of nine popular open source projects. The results show that our method can reach a high performance on classifying sentences from feature requests. Moreover, when applying fuzzy rules on machine learning methods, the performance can be improved significantly.

Open Source Software (OSS) is distributed and maintained collaboratively by developers all over the world. However, frequent personnel turnover and lack of organizational management makes it difficult to capture the actual development effort. Various OSS maintenance effort estimation approaches have been developed to provide a way to understand and estimate development effort. The goal of this study is to identify the current state of art of the existing maintenance effort estimation approaches for OSS. We performed a systematic literature review on the relevant studies published in the period between 2000-2015 by both automatic and manual searches from different sources. We derived a set of keywords from the research questions and established selection criteria to carefully choose the papers to evaluate. 29 out of 3,312 papers were selected based on a well designed selection process. Our results show that the commonly used OSS maintenance effort estimation methods are actual effort estimation and maintenance activity time prediction, the most commonly used metrics and factors for actual effort estimation are source code measurements and people related metrics, the most commonly mentioned activity for maintenance activity time prediction is bug fixing. Accuracy measures and cross validation is used for validating the estimation models. Based on the above findings, we identified the issues in evaluation methods for actual maintenance effort estimations and the needs for quantitative OSS maintenance effort inference from size-related metrics. Meanwhile, we highlighted individual contribution and performance measurement as a novel and promising research area.

Open source projects rely on issue tracking systems such as JIRA or online forums to keep track of users' feedback, expectations and requested features. However, since users are not fully aware of existing features, when submitting new feature requests, redundant requests often appear in the new feature list. It is a waste of time and effort for project contributors to manually identify and reject them, especially in complex systems with many features. Our research is aiming to find a suitable solution to identify redundant feature requests in OSS projects. We have conducted a survey on a well-known Open Source community, Hibernate and gathered all of its feature requests up-to-date. Through studying and categorizing the characteristics of these feature requests, we have found that about 37% of the feature requests were rejected and the most common rejection reason was redundancy. Also we have found that it is very expensive to identify and resolve these redundant feature requests. In this paper, we have proposed our solution to automatically identify redundant feature requests through a Feature Tree Model along with a future research agenda.

The timely fixing of bugs is important to ensure software quality.In Open Source Software (OSS) development, behaviors of stakeholders impact the bug fixing process, especially the different stages respectively.However, most of the existing studies on impact factors of bug fixing time usually treat bug fixing process as a whole, while neglecting the particularity at its different stages.Ignoring the detail of different stages cannot let us understand why the fixing time is longer or shorter.In this paper, we aimed at investigating whether the factors have different impacts on the time of different stages and the whole process.Three stages of the whole fixing process were formalized, and twenty-four factors were defined and extracted from three aspects: bug reports, their associated source code and code changes.An empirical study based on two OSS projects, Eclipse JDT Core and Linux Kernel, was conducted for the investigation.The results of our study provide a very positive validation that the influence of factors on bug fixing time is stage related, rather than for the whole process.Our results can help developers better understand influences of factors on the bug fixing process, and thus provide opportunities to improve their process effectively.

Managing the costs and risks of evolution is a challenging problem in the RE community. The challenge lies in the difficulty of analyzing and assessing the proneness to requirement changes across multiple versions, especially when the scale of requirements is large. In this paper, we define a series of metrics to characterize historic evolution information, and propose a novel method for predicting requirements that are likely to evolve in the future based on the metrics. We apply the prediction method to analyze the product updates history through a case study. The empirical results show that this method can provide a tradeoff solution that narrows down the scope of change analysis to a small set of requirements, but it still can retrieve nearly half of the future changes. The results indicate that the defined metrics are sensitive to the history of requirements evolution, and the prediction method can reach a valuable outcome for requirement engineers to balance their workload and risks.