Part-of-Speech (POS) tags are natural attributes of words in natural languages, and they are fundamental for natural language analysis. Many automated approaches have been proposed to tag natural language texts. Identifiers in source code have POS tags as well, which are useful for various source code analysis tasks, like code search, code comment generation, and code completion. Currently, state-of-the-art POS taggers originally designed for natural languages are often employed to tag source code identifiers. However, identifiers in source code are significantly different from natural languages. Consequently, POS taggers designed for natural languages could be less accurate in source code identifiers. Recently, several identifier-specific taggers have been proposed within the field of software engineering, but their adoption in practical software engineering tasks remains limited. This raises the question of why these taggers have not been more widely utilized in such tasks. In this article, we investigate the performance of natural language POS taggers on source code identifiers, specifically method names, parameter names, and class names. To do so, we manually annotated identifiers from open source projects in Java, C, and Python, creating a large dataset IDData for evaluation. We then evaluated six widely used natural language POS taggers: NLTK, CoreNLP, OpenNLP, spaCy, Flair, and Stanza, alongside three identifier-specific taggers: SWUM, POSSE, and Ensemble Tagger. Our evaluation reveals that while natural language-oriented POS taggers outperform identifier-specific taggers, their performance on identifiers is still significantly lower compared to their performance on natural language sentences. To understand the underlying reasons for this, we conducted an in-depth analysis, examining factors such as identifier length, POS distribution, syntactic structures, and special tags, which differentiate identifiers from natural language sentences. To further improve POS tagging performance on identifiers, we created a large-scale method name dataset MNTrain with manually labeled tags and retrained the natural language taggers on this new dataset. The results show substantial improvements in method name POS tagging performance, with taggers achieving performance comparable to their results on natural language sentences. Finally, we discuss the significance and practical implications of our findings, offering insights for future research.

Manually creating concise and meaningful method names is often challenging and time-consuming for software engineers, especially for less experienced and non-native English speakers. They may find it easier to provide functional descriptions in their native languages; however, this practice results in inherent ambiguities (e.g., natural language, semantic gaps, cultural misinterpretations) and limited lexical overlap with English names. To address these challenges, this paper presentsLangInsightCraft, a novel context-rich prompt generation algorithm to guide LLMs in suggesting method names from cross-language functional descriptions. From the input description, it retrieves similar high-quality native examples and extracts key contextual elements (e.g., entities, actions, functionality scope) from descriptions of these examples, to overcome inherent ambiguities. It also generates a set of candidate method names for each example’s description, evaluates them against the example’s method name using edit-distance and a semantic similarity index, and ranks them by alignment score. Using these ranked names,LangInsightCraftgenerates an English textual explanation for each candidate name, which helps LLM handle the challenge of limited lexical overlap between English method names and non-English descriptions. Finally, it embeds these outputs, input and examples into a predefined prompt template to generate a context-rich prompt for LLM to generate the expected method name. Evaluated on 64,815 Java methods with English/Chinese descriptions,LangInsightCraftsubstantially outperformsRNN-att-Copy, improving exact-match by 54% and 122%, and decreasing edit-distance by 33% and 54%, on English and Chinese datasets respectively, while generalizing well across different LLMs.

Open source software (OSS) development widely adopts modern code review to identify issues and guarantee code quality. As reported repeatedly, maintainers are under heavy workloads when reviewing code changes. Meanwhile, we notice that some code reviews were overlooked by the authors of the code changes, i.e., neither causing code modification nor being replied to. These code reviews, if requiring responses but not receiving any, might represent a significant inefficiency, risk of overlooking critical issues, and problematic social exchange. Moreover, leaving code reviews publicly unanswered may cause a negative impression on both the corresponding OSS contributors and the OSS projects. Existing literature on code review mainly focuses on the usefulness of code reviews, reviewer recommendations, factors affecting PR acceptance, and review comment generation; the nature of overlooked reviews has not been explored. To this end, we focus on a widely-used modern code review mechanism, i.e., reviewing Pull Request (PR) code before merge, and conduct the first empirical study on 80 Java OSS projects to explore the prevalence, characteristics, rationales, and possible impact of the overlooked code reviews. We find that approximately 7.5% of PRs have at least one review comment being ignored. We further show that pull requests containing no-response comments are significantly associated with longer review lifecycles and lower acceptance rates, indicating measurable negative outcomes beyond their modest prevalence. Then, we categorize these no-response comments through thematic analysis and find two main categories with seven subcategories: Review inquiry and PR management. We also extract four subcategories in Review inquiry, e.g., Give suggestions about code implementation, Point out implementation issues, and Additional task requests. PR management consists of three subcategories, i.e., PR status checks, PR merge conflict notifications, and Reject PR with uncertain reasons. To better understand the existence of no-response comments, we surveyed developers and received 45 responses. We found that the reasons for the existence of no-response comments are diverse, such as prolonged review times and a lack of consensus on opinions. Developers also hold the consensus that ignored reviews will have negative effects on software projects. These findings emphasize the need for attention from both academia and industry to the responses to review comments and optimization of the reminder mechanism.

Software refactoring is widely employed to improve software quality. However, conducting refactorings manually is tedious, time-consuming, and error-prone. Consequently, automated and semi-automated tool support is highly desirable for software refactoring in the industry, and most of the main-stream IDEs provide powerful tool support for refactoring. However, complex refactoring engines are prone to errors, which in turn may result in imperfect and incorrect refactorings. To this end, in this paper, we propose a ChatGPT-based approach to testing refactoring engines. We first manually analyze bug reports and test cases associated with refactoring engines, and construct a feature library containing fine-grained features that may trigger defects in refactoring engines. The approach automatically generates prompts according to both predefined prompt templates and features randomly selected from the feature library, requesting ChatGPT to generate test programs with the requested features. Test programs generated by ChatGPT are then forwarded to multiple refactoring engines for differential testing. To the best of our knowledge, it is the first approach in testing refactoring engines that guides test program generation with features derived from existing bugs. It is also the first approach in this line that exploits LLMs in the generation of test programs. Our initial evaluation of four main-stream refactoring engines suggests that the proposed approach is effective. It identified a total of 115 previously unknown bugs besides 28 inconsistent refactoring behaviors among different engines. Among the 115 bugs, 78 have been manually confirmed by the original developers of the tested engines, i.e., IntelliJ IDEA, Eclipse, VScode-Java, and NetBeans.

Copy-paste-modify is a widespread and pragmatic practice in software development, where developers adapt reused code snippets, sourced from platforms such as Stack Overflow, GitHub, or LLM outputs, into their local codebase. A critical yet underexplored aspect of this adaptation is code wiring: the context-aware process of substituting unresolved variables in pasted code with suitable variables or expressions from the surrounding context. Existing solutions either rely on heuristic rules or historical templates, often failing to effectively utilize contextual information, despite studies showing that over half of adaptation cases are context-dependent. In this paper, we introduce WIRL, an LLM-based agent for code wiring framed as a Retrieval-Augmented Generation (RAG) infilling task. WIRL combines an LLM, a customized toolkit, and an orchestration module to identify unresolved variables, retrieve context, and perform context-aware substitutions. To balance efficiency and autonomy, the agent adopts a mixed strategy: deterministic rule-based steps for common patterns, and a state-machine-guided decision process for intelligent exploration. We evaluate WIRL on a carefully curated, high-quality dataset consisting of real-world code adaptation scenarios. Our approach achieves an exact match precision of 91.7% and a recall of 90.0%, outperforming advanced LLMs by 22.6 and 13.7 percentage points in precision and recall, respectively, and surpassing IntelliJ IDEA by 54.3 and 49.9 percentage points. These results underscore its practical utility, particularly in contexts with complex variable dependencies or multiple unresolved variables. We believe WIRL paves the way for more intelligent and context-aware developer assistance in modern IDEs.

Type errors in Python often lead to runtime failures, posing significant challenges to software reliability and developer productivity. Existing static analysis tools aim to detect such errors without execution but frequently suffer from high false positive rates. Recently, unit test generation techniques offer great promise in achieving high test coverage, but they often struggle to produce bug-revealing tests without tailored guidance. To address these limitations, we present rTED, a novel type-aware test generation technique for automatically detecting Python type errors. Specifically, rTED combines step-by-step type constraint analysis with reflective validation to guide the test generation process and effectively suppress false positives. We evaluated rTED on two widely-used benchmarks, BugsInPy and TypeBugs. Experimental results show that rTED can detect 22 ∼ 29 more benchmarked type errors than four state-of-the-art techniques. rTED is also capable of producing fewer false positives, achieving an improvement of 173.9%∼245.9% in precision. Furthermore, we applied rTED to six real-world open-source Python projects, and successfully discovered 12 previously unknown type errors, demonstrating rTED’s practical value.

Code review is critical for ensuring software quality and maintainability. With the rapid growth in software scale and complexity, code review has become a bottleneck in the development process because of its time-consuming and knowledge-intensive nature and the shortage of experienced developers willing to review code. Several approaches have been proposed for automatically generating code reviews based on retrieval, neural machine translation, pre-trained models, or large language models (LLMs). These approaches mainly leverage historical code changes and review comments. However, a large amount of crucial information for code review, such as the context of code changes and prior review knowledge, has been overlooked. This paper proposes an LLM-based review knowledge-augmented, context-aware framework for code review generation, named LAURA. The framework integrates review exemplar retrieval, context augmentation, and systematic guidance to enhance the performance of ChatGPT-4o and DeepSeek v3 in generating code review comments. Besides, given the extensive low-quality reviews in existing datasets, we also constructed a high-quality dataset. Experimental results show that for both models, LAURA generates review comments that are either completely correct or at least helpful to developers in 42.2% and 40.4% of cases, respectively, significantly outperforming SOTA baselines. Furthermore, our ablation studies demonstrate that all components of LAURA contribute positively to improving comment quality.

Abstract Researchers have recently achieved significant advances in deep learning techniques, which in turn has substantially advanced other research disciplines, such as natural language processing, image processing, speech recognition, and software engineering. Various deep learning techniques have been successfully employed to facilitate software engineering tasks, including code generation, software refactoring, and fault localization. Many studies have also been presented in top conferences and journals, demonstrating the applications of deep learning techniques in resolving various software engineering tasks. However, although several surveys have provided overall pictures of the application of deep learning techniques in software engineering, they focus more on learning techniques, that is, what kind of deep learning techniques are employed and how deep models are trained or fine-tuned for software engineering tasks. We still lack surveys explaining the advances of subareas in software engineering driven by deep learning techniques, as well as challenges and opportunities in each subarea. To this end, in this study, we present the first task-oriented survey on deep learning-based software engineering. It covers twelve major software engineering subareas significantly impacted by deep learning techniques. Such subareas spread out through the whole lifecycle of software development and maintenance, including requirements engineering, software development, testing, maintenance, and developer collaboration. As we believe that deep learning may provide an opportunity to revolutionize the whole discipline of software engineering, providing one survey covering as many subareas as possible in software engineering can help future research push forward the frontier of deep learning-based software engineering more systematically. For each of the selected subareas, we highlight the major advances achieved by applying deep learning techniques with pointers to the available datasets in such a subarea. We also discuss the challenges and opportunities concerning each of the surveyed software engineering subareas.

Mobile applications are widely used by billions of users in their daily work and life. Such GUI software is prone to bugs, potentially degrading user experience. Notably, many bugs in mobile apps are reported by end-users who cannot access the requirements of the app or test cases accompanied by explicitly specified test oracles. It may suggest that such bugs are not identified in the traditional way, i.e., by comparing the actual behaviors of the apps against their expected behaviors explicitly specified in the requirements or test cases. Instead, such bugs are often identified by comparing the actual behaviors against users’ common knowledge of apps, noted as common sense. We refer to such bugs as common sense-violating bugs. Although it is well-known that common sense-violating bugs are common in mobile apps, it remains unclear how popular they are and what kind of common sense principles are violated by them, let alone the relationship among the violated common sense principles. To this end, in this paper, we conduct the first large-scale empirical study on common sense-violating bugs in open-source mobile apps. We manually analyzed 2,808 real-world bug reports across 948 open-sourced mobile apps on GitHub. Our analysis results suggest that 1,006 (35.8%) out of the 2,808 bugs pertain to common sense-violating bugs. From those common sense-violating bugs, we identified a set of common sense principles violated by the buggy behaviors, and built a taxonomy for the common sense principles. Such principles fall into three categories: UI content-related common sense principles, UI layout-related common sense principles, and interaction-related common sense principles. By analyzing the frequency of the common sense principles being violated, we observed that a small set of common sense principles were frequently violated by the majority of common sense-violating bugs: 18 common sense principles, accounting for only 5% of the violated common sense principles, were violated by more than half of the common sense-violating bugs. These findings suggest that identifying the most frequent common sense-violating bugs could be achieved by using a small set of critical common sense principles, which may significantly reduce the cost of common sense-based bug detection. Finally, to demonstrate the feasibility of automated bug detection with common sense-based test oracles, we propose an automated approach to validating whether a given test run violates the most frequently violated common sense principle: No raw error message. Our evaluation results suggest that the automated approach is accurate, whose precision and recall are 91.3% and 91.6%, respectively.

Extracting local variable refactoring is frequently employed to replace one or more occurrences of a complex expression with simple accesses to a newly introduced variable. To facilitate refactoring, most IDEs can automate the extract local variable refactorings when the to-be-extracted expressions are selected by developers. However, refactoring tools usually replace all expressions that are lexically identical to the selected one without a comprehensive analysis of the safety of the refactoring. The automatically conducted refactorings may lead to serious software defects. Besides that, existing refactoring tools rely heavily on software developers to spot to-be-extracted expressions although it is often challenging for inexperienced developers and maintainers to make the selection. To this end, in this article, we propose an automated approach, called ValExtractor+ , to recommending extract local variable refactoring opportunities and to automatically and safely conduct the refactorings. ValExtractor+ is composed of two parts, i.e., solutionAdvisor and opportunityAdvisor . Given a to-be-extracted expression, solutionAdvisor leverages lightweight static source code analysis to validate potential side effects of the expression, and to identify expressions that could be extracted together with the selected expression as a single variable without changing the semantics of the program or introducing any new exceptions. The static code analysis significantly improves the safety of automated extraction of local variables. To free programmers from manually selecting to-be-extracted expressions, opportunityAdvisor leverages solutionAdvisor to automatically retrieve all expressions that could be extracted safely as well as their refactoring solutions. It then leverages a learning-based classifier to predict which of the retrieved expressions should be extracted. Evaluations on open-source applications suggest that solutionAdvisor successfully avoided all defects (more than two hundred) caused by extracting local variable refactorings conducted by Eclipse (243 defects) or IntelliJ IDEA (263 defects). Additionally, opportunityAdvisor was able to effectively recommend expressions for extraction, achieving 307 true positives (TP) and 21,121 true negatives (TN). Four pull requests from our work (PR IDs: 66, 333, 439, and 360) were successfully merged into the Eclipse community repository, showcasing the practical impact and robustness of our approach as recognized by the wider developer community.

Natural languages are “natural” in that texts in natural languages are repetitive and predictable. Recent research indicates that programming languages share similar characteristics (naturalness), with source code displaying patterns of repetition and predictability. Notably, studies have shown that buggy code deviates from these natural patterns in that buggy code is significantly less natural than bug-free one. In this article, we conduct a large-scale and extensive empirical study to investigate whether code defects lead to unnaturalness of source code. Different from existing studies, we leverage multiple large-scale and high-quality bug repositories where bug-irrelevant changes in bug-fixing commits have been explicitly excluded. The leveraged software applications cover different programming languages, and the empirical study involves real-world software defects as well as defects injected automatically with well-known mutation operators. On the one side, our evaluation results confirm existing studies in that buggy source code lines are often less natural than bug-free ones. On the other side, our evaluation reveals some interesting new findings. First, fixing bugs does not significantly improve the naturalness of code lines and the fixed lines on average are as unnatural as buggy ones. This finding may suggest that software defects are not the root causes of source code’s unnaturalness although there does existing statistically significant correlation between software defects and source code’s naturalness. Second, defects in different programming languages have similar effect on source code’s naturalness. The conclusions (i.e., buggy code is less natural but fixing the bugs cannot improve source code’s naturalness) hold regardless of the programming languages. Third, injecting defects automatically by well-known mutation operators does not significantly reduce the naturalness of involved source code lines. This suggests that automatically injected defects may have a similar impact on the naturalness of source code as real-world defects inadvertently introduced by developers. Fourth, the detects’ impact on source code’s naturalness varies slightly among different categories of software defects. Although fixing bugs on average does not significantly improve the naturalness of involved source code, fixing “checking” related bugs does significantly improve the naturalness of source code. Finally, locating buggy code lines according to naturalness alone is inaccurate, resulting in extremely low precision (less than one percent).

Extract local variable is one of the most popular refactorings. It is frequently employed to replace occurrences of a complex expression with simple accesses to a newly introduced variable that is initialized by the original complex expression. Consequently, most IDEs and refactoring tools provide automated support for this refactoring, e.g., to suggest names for the newly extracted variables. However, we find approximately 70% of the names recommended by these IDEs are different from what developers manually constructed, adding additional renaming burdens to developers and providing limited assistance. In this article, we introduce VarNamer , an automated approach designed to recommend variable names for extract local variable refactorings. Through a large-scale empirical study, we identify key contexts, such as variable initializations and homogeneous variables (variables whose initializations are identical to that of the newly extracted variable), that are useful for composing variable names. Leveraging these insights, we developed a set of heuristic rules through program static analysis techniques, e.g., lexical analysis, syntax analysis, control flow analysis, and data flow analysis, and employ data mining techniques, i.e., FP-growth algorithm, to recommend variable names effectively. Notably, some of our heuristic rules have been successfully integrated into Eclipse , where they are now distributed with the latest releases of the IDE. Evaluation of VarNamer on a dataset of 27,158 real-world extract local variable refactorings in Java applications demonstrates its superiority over state-of-the-art IDEs. Specifically, VarNamer significantly increases the chance of exact match by 52.6% compared to Eclipse and 40.7% compared to IntelliJ IDEA . We also evaluated the proposed approach with real-world extract local variable refactorings conducted in C \(++\) projects, and the results suggest that the approach can achieve comparable performance on programming languages besides Java. It may suggest the generalizability of VarNamer . Finally, we designed and conducted a user study to investigate the impact of VarNamer on developers’ productivity. The results of the user study suggest that our approach can speed up the refactoring by 27.8% and reduce 49.3% edits on the recommended variable names.

Source code identifiers often contain abbreviations. Such abbreviations may reduce the readability of the source code, which in turn hinders the maintenance of the software applications. To this end, accurate and automated approaches to expanding abbreviations in source code are desirable and abbreviation expansion has been intensively investigated. However, to the best of our knowledge, most existing approaches are heuristics, and none of them has even employed deep learning techniques, let alone the most advanced large language models (LLMs). LLMs have demonstrated cutting-edge performance in various software engineering tasks, and thus it has the potential to expand abbreviations automatically. To this end, in this paper, we present the first empirical study on prompt-based usage of LLMs for abbreviation expansion. Our evaluation results on a public benchmark suggest that GPT is substantially less accurate than the state-of-the-art approach, reducing precision and recall by 28.2% and 27.8%, respectively. We manually analyzed the failed cases, and discovered the root causes for the failures: 1) Lack of contexts and 2) Inability to recognize abbreviations. In response to the first cause, we investigated the effect of various contexts and found surrounding source code is the best selection. In response to the second cause, we designed an iterative approach that identifies and explicitly marks missed abbreviations in prompts. Finally, we propose a post-condition checking to exclude incorrect expansions that violate common sense. All such measures together make LLM-based abbreviation expansion comparable to the state of the art while avoiding expensive source code parsing and deep analysis that are indispensable for state-of-the-art approaches. Our evaluation results on open-source LLMs, i.e., DeepSeek-Coder and Llama, confirm that the post-condition checking works well with various LLMs.

LLMs demonstrate significant potential across various software engineering tasks. However, they still face challenges in generating correct code on the first attempt when addressing complex requirements. Introducing a discriminator to select reliable outputs from multiple generated results is an effective way to enhance their reliability and stability. Currently, these discriminators fall into two categories: execution-based discriminators and non-execution-based discriminators. Execution-based discriminators face flexibility challenges due to difficulties in obtaining test cases and security concerns, while non-execution-based discriminators, although more flexible, struggle to capture subtle differences in code details. To maintain flexibility while improving the model’s ability to capture fine-grained code details, this paper proposes Condor. We first design contrastive learning to optimize the code representations of the base model, enabling it to reflect differences in code details. Then, we leverage intermediate data from the code modification process to further enrich the discriminator’s training data, enhancing its ability to discern code details. Experimental results indicate that on the subtle code difference dataset (i.e., CodeNanoFix), Condor significantly outperforms other discriminators in discriminative performance: Condor (1.3B) improves the discriminative F1 score of DeepSeek-Coder (1.3B) from 67% to 73%. In discriminating LLM-generated outputs, Condor (1.3B) and Condor (110M) raise the Pass@1 score of Llama-3.1-Instruct (70B) on the CodeNanoFix dataset from 52.64% to 62.63% and 59.64%, respectively. Moreover, Condor demonstrates strong generalization capabilities on the APPS, MBPP, and LiveCodeBench datasets. For example, Condor (1.3B) improves the Pass@1 of Llama-3.1-Instruct (70B) on the APPS dataset by 147.05%.

Software developers and maintainers frequently conduct software refactorings to improve software quality. Identifying the conducted software refactorings may significantly facilitate the comprehension of software evolution, and thus facilitate software maintenance and evolution. Besides that, the identified refactorings are also valuable for data-driven approaches in software refactoring. To this end, researchers have proposed a few approaches to identifying software refactorings automatically. However, the performance (especially precision) of such approaches deserves substantial improvement. To this end, in this paper, we propose a novel refactoring detection approach, calledReExtractor+. At the heart ofReExtractor+is a reference-based entity matching algorithm that matches coarse-grained code entities (e.g., classes and methods) between two successive versions, and a context-aware statement matching algorithm that matches statements within a pair of matched methods. We evaluatedReExtractor+on a benchmark consisting of 400 commits from 20 real-world projects. The evaluation results suggested thatReExtractor+significantly outperformed the state of the art in refactoring detection, reducing the number of false positives by 57.4% and improving recall by 18.4%. We also evaluated the performance of the proposed matching algorithms that serve as the cornerstone of refactoring detection. The evaluation results suggested that the proposed algorithms excel in matching code entities, substantially reducing the number of mistakes (false positives plus false negatives) by 67% compared to the state-of-the-art approaches.

Software refactoring is widely conducted in the industry and well-studied in the academic community. There are dozens of software refactoring types, and each type of refactoring often requires its unique tool support and algorithms. Consequently, knowing which types of refactorings are popular in real-world practice and which are less supported by existing tools is highly valuable. To this end, in this paper, we present a large-scale empirical study on software refactorings in real-world open-source Java projects. We first retrieved by keywords 15,860 code commits from GitHub that might contain software refactorings. From the resulting commits, we manually analyzed 1,200 of them and successfully identified 100 types of refactorings from 420 commits. We built a taxonomy for the discovered refactorings, and compared them against the refactoring types supported by state-of-the-art refactoring engines and miners. The comparison results suggest that 61 out of the 100 refactoring types have not yet been explicitly supported by any of the refactoring engines, and any refactoring miners have not explicitly supported 62. The empirical study has identified and revealed 31 under-explored refactorings observed in Java real-world open-source applications but not yet supported by existing refactoring tools. These refactorings may have implications for the development of future tool support and enhancements in the refactoring ecosystem.

Developers often manually design a few unit tests for a given method under development. After passing such manually designed tests, however, they usually have to turn to automated test case generation tools like EvoSuite and Randoop for more thorough testing. Although the automatically generated tests may achieve a high coverage, they rarely identify hard-to-detect defects automatically because of the well-known test oracle problem: It is challenging to tell whether the output is correct or incorrect without explicit test oracle (expected output). Consequently, developers should manually select and verify a few suspicious test cases to identify hard-to-detect defects. To this end, in this paper, we propose a novel approach, calledOneMoreTest, to generating and selecting the most suspicious tests for manual verification. Based on a manually designed passed test,OneMoreTestautomatically generates millions of input-output pairs for the method under test (MUT) with mutationbased fuzzing. It then trains an automatically generated neural network to simulate the MUT’s behavior. For new tests automatically generated for the same MUT,OneMoreTestsuggests developers with the topkmost suspicious tests that have the greatest distances between their actual output and estimated output (i.e., network’s output). Our evaluation on real-world faulty methods suggests thatOneMoreTestis accurate. On 70.79% of the involved 178 real-world faulty methods, we can identify the defects by manually verifying only a SINGLE test for each of the methods according toOneMoreTest’s suggestions. Compared against the state of the art,OneMoreTestimproved the precision from 46.63% to 72.62%, and recall from 46.63% to 70.79%.

Renaming is one of the most popular software refactorings. Although developers may know what the new name should be when they conduct a renaming, it remains valuable for refactoring tools to recommend new names automatically so that developers can simply hit Enter and efficiently accept the recommendation to accomplish the refactoring. Consequently, most IDEs automatically recommend new names for renaming refactorings by default. However, the recommendation made by mainstream IDEs is often incorrect. For example, the precision of IntelliJ IDEA in recommending names for field renamings is as low as 6.3%. To improve the accuracy, in this paper, we propose a context-aware lightweight approach (called CARER) to recommend new names for Java field renamings. Different from mainstream IDEs that rely heavily on initializers and data types of the to-be-renamed fields, CARER exploits both dynamic and static contexts of the renamings as well as naming conventions. We evaluate CARER on 1.1K real-world field renamings discovered from open-source applications. Our evaluation results suggest that CARER can significantly improve the state of the practice in recommending new names for field renamings, improving the precision from 6.30% to 61.15%, and recall from 6.30% to 41.50%. Our evaluation results also suggest that CARER is as efficient as IntelliJ IDEA is, making it suitable to be integrated into IDEs.

God classes are widely recognized as code smells, significantly impairing the maintainability and readability of source code. However, resolving the identified God classes remains a formidable challenge, and we still lack automated and accurate tools to resolve God classes automatically. To this end, in this paper, we propose a novel approach (called ClassSplitter) to decompose God classes. The key observation behind the proposed approach is that software entities (i.e., methods and fields) that are physically adjacent often have strong semantic correlations and thus have a great chance of being classified into the same class during God class deposition. We validate this hypothesis by analyzing 54 God class decomposition refactorings actually conducted in the wild. According to the observation, we measure the similarity between software entities by exploiting not only traditional code metrics but also their relative physical positions. Based on the similarity, we customize a clustering algorithm to classify the methods within a given God class, and each of the resulting clusters is taken as a new class. Finally, ClassSplitter allocates the fields of the God class to the new classes according to the field-access-based coupling between fields and classes. We evaluate ClassSplitter using 133 real-world God classes from open-source applications. Our evaluation results suggest that ClassSplitter could substantially improve the state of the art in God class decomposition, improving the average MoJoFM by 47%. Manual evaluation also confirmed that in most cases (77%) the solutions suggested by ClassSplitter were preferred by developers to alternatives suggested by the state-of-the-art baseline approach.

Methods should be named to summarize their responsibilities meaningfully. When a method has a non-trivial responsibility, it may require a naming using multiple words. However, overlong method names are susceptible to typos and reduced readability (e.g., displaying a statement partially in standard screen width or splitting it into multiple lines). Programming naming conventions commonly adopt a maximal length (in characters) for identifiers. In practice, developers may not necessarily find a meaningful name that follows such naming conventions when coding a non-trivial method. This article presents the first automated technique (called NameCompressor ) to shorten overlong method names. Our inspiration is that many lengthy words/phrases in an overlong method name have known and unambiguous abbreviations. The use of these abbreviations for method names is common. To shorten an overlong method name, NameCompressor employs three compression techniques, i.e., context-aware compression, probability-based compression, and machine learning-based compression, to find appropriate abbreviations for the words/phrases in the method name. We evaluate NameCompressor on a dataset of 700 overlong method names. It correctly generates 613 short names identical to those specified by the developers of these methods.

Version control systems are widely used to manage the evolution of software applications. However, such version control systems take source code as lines of plain text, and thus they cannot present the evolution of software entities embedded in the source code. To this end, a few approaches have been proposed to match software entities before and after a given commit, known as software entity matching algorithms. However, the accuracy of such algorithms requires further improvement. In this paper, we propose an automated iterative algorithm (called ReMapper) to match software entities between two successive versions. The key insight of ReMapper is that the qualified name, the implementation, and the references of a software entity together can distinguish it from others. It matches software entities iteratively because the mapping depends on the reference-based similarity whereas the reference-based similarity depends on the mapping of entities as well. We evaluated ReMapper on a benchmark consisting of 215 commits from 21 real-world projects. Our evaluation results suggest that ReMapper substantially outperformed the state of the art, reducing the number of mistakes (false positives plus false negatives) substantially by 85.8%. We also evaluated to what extent it may improve the automated refactoring discovery (mining) that relies heavily on automated entity matching. Our evaluation results suggest that it substantially improved the state of the art in refactoring discovery, improving recall by 6.9% and reducing the number of false positives by 72.6%.

Extract local variable is a well-known and widely used refactoring. It is frequently employed to replace one or more occurrences of a complex expression with simple accesses to a newly added variable. Although most IDEs provide tool support for extract local variables, such tools without deep analysis of the refactorings may result in semantic errors. To this end, in this paper, we propose a novel and more reliable approach, called ValExtractor, to conduct extract variable refactorings automatically. The major challenge of automated extract local variable refactorings is how to efficiently and accurately identify the side effect of the extracted expressions and the potential interaction between the extracted expressions and their contexts without time-consuming dynamic execution of the involved programs. To resolve this challenge, ValExtractor leverages a lightweight static source code analysis to validate the side effect of the selected expression, and to identify which occurrences of the selected expression could be extracted together without changing the semantics of the program or introducing potential new exceptions. Our evaluation results on open-source Java applications suggest that Eclipse and IntelliJ IDEA, the state-of-the-practice refactoring engines, resulted in a large number of faulty extract variable refactorings whereas ValExtractor successfully avoided all such errors. The proposed approach has been merged into (and distributed with) Eclipse to improve the safety of extract local variable refactoring.

Feature envy is one of the well-recognized code smells that should be removed by software refactoring. A major challenge in feature envy detection is that traditional approaches are less accurate whereas deep learning-based approaches are suffering from the lack of high-quality large-scale training data. Although existing refactoring detection tools could be employed to discover real-world feature envy examples, the noise (i.e., false positives) within the resulting data could significantly influence the quality of the training data as well as the performance of the models trained on the data. To this end, in this paper, we propose a sequence of heuristic rules and a decision tree-based classifier to filter out false positives reported by state-of-the-art refactoring detection tools. The data after filtering serve as the positive items in the requested training data. From the same subject projects, we randomly select methods that are different from positive items as negative items. With the real-world examples (both positive and negative examples), we design and train a deep learning-based binary model to predict whether a given method should be moved to a potential target class. Different from existing models, it leverages additional features, i.e., coupling between methods and classes (CBMC) and the message passing coupling between methods and classes (MCMC) that have not yet been exploited by existing approaches. Our evaluation results on real-world open-source projects suggest that the proposed approach substantially outperforms the state of the art in feature envy detection, improving precision and recall by 38.5% and 20.8%, respectively.

Bug-related research, e.g., fault localization, program repair, and software testing, relies heavily on high-quality and large-scale software bug repositories. The importance of such repositories is twofold. On one side, real-world bugs and their associated patches may inspire novel approaches for finding, locating, and repairing software bugs. On the other side, the real-world bugs and their patches are indispensable for rigorous and meaningful evaluation of approaches to software testing, fault localization, and program repair. To this end, a number of software bug repositories, e.g., iBUGS and Defects4J, have been constructed recently by mining version control systems and bug tracking systems. However, fully automated construction of bug repositories by simply taking bug-fixing commits from version control systems often results in inaccurate patches that contain many bug-irrelevant changes. Although we may request experts or developers to manually exclude the bug-irrelevant changes (as the authors of Defects4J did), such extensive human intervention makes it difficult to build large-scale bug repositories. To this end, in this paper, we propose an automatic approach, calledBugBuilder, to construct bug repositories from version control systems. Different from existing approaches, it automatically extracts complete and concise bug-fixing patches and excludes bug-irrelevant changes. It first detects and excludes software refactorings involved in bug-fixing commits.BugBuilderthen enumerates all subsets of the remaining part, and discards invalid subsets by compilation and software testing. If exactly a single subset survives the validation, this subset is taken as the complete and concise bug-fixing patch for the associated bug. In case multiple subsets survive, BugBuilder employs a sequence of heuristics to select the most likely one. Evaluation results on 809 real-world bug-fixing commits in Defects4J suggest thatBugBuildersuccessfully extracted complete and concise bug-fixing patches from forty-three percent of the bug-fixing commits, and its precision (99%) was even higher than human experts. We also built a bug repository, called GrowingBugs, with the proposed approach. The resulting repository serves as evidence of the usefulness of the proposed approach, as well as a publicly available benchmark for bug-related research.

In programming courses, providing students with concise and constructive feedback on faulty submissions (programs) is highly desirable. However, providing feedback manually is often time-consuming and tedious. To release tutors from the manual construction of concise feedback, researchers have proposed approaches such asCLARAandRefactoryto construct feedback automatically. The key to such approaches is to fix a faulty program by making it equivalent to one of its correct reference programs whose overall structure is identical to that of the faulty submission. However, for a newly released assignment, it is likely that there are no correct reference programs at all, let alone correct reference programs sharing identical structure with the faulty submission. Therefore, in this paper, we proposeAssignmentMendergenerating concise patches for newly released assignments. The key insight ofAssignmentMenderis that a faulty submission can be repaired by reusing fine-grained code snippets from submissions (even when they are faulty) for the same assignment. It automatically locates suspicious code in the faulty program and leverages static analysis to retrieve reference code from existing submissions with a graph-based matching algorithm. Finally, it generates candidate patches by modifying the suspicious code based on the reference code. Different from existing approaches,AssignmentMenderexploits faulty submissions in addition to bug-free submissions to generate patches. Another advantage ofAssignmentMenderis that it can leverage submissions whose overall structures are different from those of the to-be-fixed submission. Evaluation results on 128 faulty submissions from 10 assignments show thatAssignmentMenderimproves the state-of-the-art in feedback generation for newly released assignments. A case study involving 40 students and 80 submissions further provides initial evidence showing that the proposed approach is useful in practice.

Texts in natural languages are highly repetitive and predictable because of the naturalness of natural languages. Recent research validated that source code in programming languages is also repetitive and predictable, and naturalness is an inherent property of source code. It was also reported that buggy code is significantly less natural than bug-free one, and bug fixing substantially improves the naturalness of the involved source code. In this paper, we revisit the naturalness of buggy code and investigate the effect of bug-fixing on the naturalness of source code. Different from the existing investigation, we leverage two large-scale and high-quality bug repositories where bug-irrelevant changes in bug-fixing commits have been explicitly excluded. Our evaluation results confirm that buggy lines are often less natural than bug-free ones. However, fixing bugs could not significantly improve the naturalness of involved code lines. Fixed lines on average are as unnatural as buggy ones. Consequently, bugs are not the root cause of the unnaturalness of source code, and it could be inaccurate to identify buggy code lines solely by the naturalness of source code. Our evaluation results suggest that the naturalness-based buggy line detection results in extremely low precision (less than one percentage).

Although the negative impact of abbreviations in source code is well-recognized, abbreviations are common for various reasons. To this end, a number of approaches have been proposed to expand abbreviations in identifiers. However, such approaches are either inaccurate or confined to specific identifiers. To this end, in this paper, we propose a generic and accurate approach to expand identifier abbreviations by leveraging both semantic relation and transfer expansion. One of the key insights of the approach is that abbreviations in the name of software entity$e$have a great chance to find their full terms in names of software entities that are semantically related to$e$. Consequently, the proposed approach builds a knowledge graph to represent such entities and their relationships with$e$and searches the graph for full terms. Another key insight is that literally identical abbreviations within the same application are likely (but not necessary) to have identical expansions, and thus the semantics-based expansion in one place may be transferred to other places. To investigate when abbreviation expansion could be transferred safely, we conduct a case study on three open-source applications. The results suggest that a significant part (75 percent) of expansions could be transferred among lexically identical abbreviations within the same application. However, the risk of transfer varies according to various factors, e.g., length of abbreviations, the physical distance between abbreviations, and semantic relations between abbreviations. Based on these findings, we design nine heuristics for transfer expansion and propose a learning-based approach to prioritize both transfer heuristics and semantic-based expansion heuristics. Evaluation results on nine open-source applications suggest that the proposed approach significantly improves the state of the art, improving recall from 29 to 89 percent and precision from 39 to 92 percent.

High-quality and large-scale repositories of real bugs and their concise patches collected from real-world applications are critical for research in software engineering community. In such a repository, each real bug is explicitly associated with its fix. Therefore, on one side, the real bugs and their fixes may inspire novel approaches for finding, locating, and repairing software bugs; on the other side, the real bugs and their fixes are indispensable for rigorous and meaningful evaluation of approaches for software testing, fault localization, and program repair. To this end, a number of such repositories, e.g., Defects4J, have been proposed. However, such repositories are rather small because their construction involves expensive human intervention. Although bug-fixing code commits as well as associated test cases could be retrieved from version control systems automatically, existing approaches could not yet automatically extract concise bug-fixing patches from bug-fixing commits because such commits often involve bug-irrelevant changes. In this paper, we propose an automatic approach, called BugBuilder, to extracting complete and concise bug-fixing patches from human-written patches in version control systems. It excludes refactorings by detecting refactorings involved in bug-fixing commits, and reapplying detected refactorings on the faulty version. It enumerates all subsets of the remaining part and validates them on test cases. If none of the subsets has the potential to be a complete bug-fixing patch, the remaining part as a whole is taken as a complete and concise bug-fixing patch. Evaluation results on 809 real bug-fixing commits in Defects4J suggest that BugBuilder successfully generated complete and concise bug-fixing patches for forty percent of the bug-fixing commits, and its precision (99%) was even higher than human experts.

Library developers can provide classes and methods with underdetermined specifications that allow flexibility in future implementations. Library users may write code that relies on a specific implementation rather than on the specification, e.g., assuming mistakenly that the order of elements cannot change in the future. Prior work proposed the NonDex approach that detects such wrong assumptions. We present a novel approach, called DexFix, to repair wrong assumptions on underdetermined specifications in an automated way. We run the NonDex tool on 200 open-source Java projects and detect 275 tests that fail due to wrong assumptions. The majority of failures are from iterating over HashMap/HashSet collections and the getDeclaredFields method. We provide several new repair strategies that can fix these violations in both the test code and the main code. DexFix proposes fixes for 119 tests from the detected 275 tests. We have already reported fixes for 102 tests as GitHub pull requests: 74 have been merged, with only 5 rejected, and the remaining pending.

Abbreviations are common in source code. Properly designed abbreviations may significantly facilitate typing, typesetting, and reading of lengthy source code. However, abbreviations, if used improperly, may also significantly reduce the readability and maintainability of source code. Although a few automated approaches have been proposed to suggest full terms for given abbreviations, to the best of our knowledge, there is no automated approaches to suggest whether abbreviations are used properly, i.e., whether they should be replaced with corresponding full terms. Notably, it is often challenging for inexperienced developers and maintainers to make such decisions. To this end, in this paper, we propose an automated approach to assisting developers and maintainers in making the decisions. The rationale of the approach is that abbreviations should not be expanded if the expansion would result in unacceptably lengthy identifiers or if developers/maintainers can easily figure out the meaning (full terms) of the abbreviations based on their domain knowledge or contexts of the abbreviations. From a corpus of programs, we leverage data mining techniques to discover common abbreviations that are frequently employed by various developers in similar contexts. The key of the data mining is to turn the problem of mining common abbreviations into the maximal clique problem that has been extensively studied. We suggest to not expand given abbreviation if it matches at least one of the discovered common abbreviations. From the same corpus, we also calculate the probability distribution for the length of different types of identifier, e.g., variable names and method names. The probability distribution specifies how likely an identifier of type T is composed of exactly n characters. Our heuristic is to not expand the abbreviation if the probability of its enclosing identifier would be reduced by the expansion. Finally, we also suggest to not expand the abbreviation if its full terms are contained in surrounding contexts of the abbreviation, i.e., tokens on the same source code line. Other abbreviations that do not receive suggestions from the proposed approach are expected to be replaced with their full terms. Our evaluation results on 1,818 abbreviations from five open-source applications suggest that the proposed approach is accurate with a high accuracy of 95%.

When users encounter problems with mobile apps, they may commit such problems to developers as bug reports. To facilitate the processing of bug reports, researchers proposed approaches to validate the reported issues automatically according to the steps to reproduce specified in bug reports. Although such approaches have achieved high success rate in reproducing the reported issues, they often rely on a predefined vocabulary to identify and classify actions in bug reports. However, such manually constructed vocabulary and classification have significant limitations. It is challenging for the vocabulary to cover all potential action words because users may describe the same action with different words. Besides that, classification of actions solely based on the action words could be inaccurate because the same action word, appearing in different contexts, may have different meaning and thus belongs to different action categories. To this end, in this paper we propose an automated approach, called MaCa, to identify and classify action words in Mobile apps’ bug reports. For a given bug report, it first identifies action words based on natural language processing. For each of the resulting action words, MaCa extracts its contexts, i.e., its enclosing segment, the associated UI target, and the type of its target element by both natural language processing and static analysis of the associated app. The action word and its contexts are then fed into a machine learning based classifier that predicts the category of the given action word in the given context. To train the classifier, we manually labelled 1,202 actions words from 525 bug reports that are associated with 207 apps. Our evaluation results on manually labelled data suggested that MaCa was accurate with high accuracy varying from 95% to 96.7%. We also investigated to what extent MaCa could further improve existing approaches (i.e., Yakusu and ReCDroid) in reproducing bug reports. Our evaluation results suggested that integrating MaCa into existing approaches significantly improved the success rates of ReCDroid and Yakusu by 22.7% = (69.2%-56.4%)/56.4% and 22.9%= (62.7%-51%)/51%, respectively.

Abbreviations are widely used in identifiers. However, they have severe negative impact on program comprehension and IR-based software maintenance activities, e.g., concept location, software clustering, and recovery of traceability links. Consequently, a number of efficient approaches have been proposed successfully to expand abbreviations in identifiers. Most of such approaches rely heavily on dictionaries, and rarely exploit the specific and fine-grained context of identifiers. As a result, such approaches are less accurate in expanding abbreviations (especially short ones) that may match multiple dictionary words. To this end, in this paper we propose an automatic approach to improve the accuracy of abbreviation expansion by exploiting the specific and fine-grained context. It focuses on a special but common category of abbreviations (abbreviations in parameter names), and thus it can exploit the specific and fine-grained context, i.e., the type of the enclosing parameter as well the corresponding formal (or actual) parameter name. The recent empirical study on parameters suggest that actual parameters are often lexically similar to their corresponding formal parameters. Consequently, it is likely that an abbreviation in a formal parameter can find its full terms in the corresponding actual parameter, and vice versa. Based on this assumption, a series of heuristics are proposed to look for full terms from the corresponding actual (or formal) parameter names. To the best of our knowledge, we are the first to expand abbreviations by exploiting the lexical similarity between actual and formal parameters. We also search for full terms in the data type of the enclosing parameter. Only if all such heuristics fail, the approach turns to the traditional abbreviation dictionaries. We evaluate the proposed approach on seven well known open-source projects. Evaluation results suggest that when only parameter abbreviations are involved, the proposed approach can improve the precision from 26 to 95 percent and recall from 26 to 65 percent compared against the state-of-the-art general purpose approach. Consequently, the proposed approach could be employed as a useful supplement to existing approaches to expand parameter abbreviations.

Presents corrections to author information in the above named paper.

Identifiers account for 70% of source code in terms of characters, and thus the quality of such identifiers is critical for program comprehension and software maintenance. For various reasons, however, many identifiers contain abbreviations, which reduces the readability and maintainability of source code. To this end, a number of approaches have been proposed to expand abbreviations in identifiers. However, such approaches are either inaccurate or confined to specific identifiers. To this end, in this paper we propose a generic and accurate approach to expand identifier abbreviations. The key insight of the approach is that abbreviations in the name of software entity e have great chance to find their full terms in names of software entities that are semantically related to e. Consequently, the proposed approach builds a knowledge graph to represent such entities and their relationships with e, and searches the graph for full terms. The optimal searching strategy for the graph could be learned automatically from a corpus of manually expanded abbreviations. We evaluate the proposed approach on nine well known open-source projects. Results of our k-fold evaluation suggest that the proposed approach improves the state of the art. It improves precision significantly from 29% to 85%, and recall from 29% to 77%. Evaluation results also suggest that the proposed generic approach is even better than the state-of-the-art parameter-specific approach in expanding parameter abbreviations, improving F1 score significantly from 75% to 87%.