Recent advancements in end-to-end autonomous driving systems (ADSs) underscore their potential for perception and planning capabilities. However, challenges remain. Complex driving scenarios contain rich semantic information, yet ambiguous or noisy semantics can compromise decision reliability, while interference between multiple driving tasks may hinder optimal planning. Furthermore, prolonged inference latency slows decision-making, increasing the risk of unsafe driving behaviors. To address these challenges, we propose ExpertAD, a novel framework that enhances the performance of ADS with Mixture of Experts (MoE) architecture. We introduce a Perception Adapter (PA) to amplify task-critical features, ensuring contextually relevant scene understanding, and a Mixture of Sparse Experts (MoSE) to minimize task interference during prediction, allowing for effective and efficient planning. Our experiments show that ExpertAD reduces average collision rates by up to 20% and inference latency by 25% compared to prior methods. We further evaluate its multi-skill planning capabilities in rare scenarios (e.g., accidents, yielding to emergency vehicles) and demonstrate strong generalization to unseen urban environments. Additionally, we present a case study that illustrates its decision-making process in complex driving scenarios.

Zhenhao Zhou, Zhuochen Huang, Yike He, Chong Wang, Jiajun Wang, Yijian Wu, Xin Peng, Yiling Lou. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.

Trace analysis is essential for understanding system behaviors, detecting anomalies, and diagnosing faults in complex microservice-based web applications. Existing trace analysis approaches face several challenges in industrial microservice-based systems, including high manual overhead, limited functionality, unfriendly interaction mechanisms, and difficulties in deployment and integration. The strong capabilities of large language models (LLMs) in natural language understanding, reasoning, and multi-task generalization provide new opportunities for a more intelligent and flexible trace analysis approach. However, the trace analysis capabilities of LLMs remain underexplored and underdeveloped. To bridge this gap, we conduct the first comprehensive evaluation on the trace analysis capabilities of LLMs. In particular, we construct the first instruction&response benchmark dataset for trace analysis, named TraceBench. It involves a wide range of trace analysis tasks, allowing us to systematically evaluate the capabilities of LLMs in this area. Experimental results show that LLMs have potential in handling trace analysis tasks, but there leaves room for improvement. To this end, we propose TraceLLM, an approach that significantly enhances the capabilities of LLMs via fine-tuning, outperforming the open-source LLMs by 34.77% on average in terms of accuracy, and outperforming the closed-source model by 21.66% in the best case. The generalization and robustness of TraceLLM are also confirmed in our experiments. To the best of our knowledge, TraceLLM is the first LLM which is specialized for handling various types of trace analysis tasks. This work provides a foundation for future research to further explore the trace analysis capabilities of LLMs.

Python's dynamic typing system offers flexibility and expressiveness but can lead to type-related errors, prompting the need for automated type inference to enhance type hinting. While existing learning-based approaches show promising inference accuracy, they struggle with practical challenges in comprehensively handling various types, including complex parameterized types and (unseen) user-defined types. In this paper, we introduce TIGER, a two-stage generating-then-ranking (GTR) framework, designed to effectively handle Python's diverse type categories. TIGER leverages fine-tuned pre-trained code models to train a generative model with a span masking objective and a similarity model with a contrastive training objective. This approach allows TIGER to generate a wide range of type candidates, including complex parameterized types in the generating stage, and accurately rank them with user-defined types in the ranking stage. Our evaluation on the ManyTypes4Py dataset shows TIGER's advantage over existing methods in various type categories, notably improving accuracy in inferring user-defined and unseen types by 11.2% and 20.1% respectively in Top-5 Exact Match. Moreover, the experimental results not only demonstrate TIGER's superior performance and efficiency, but also underscore the significance of its generating and ranking stages in enhancing automated type inference.

Automated library APIs testing is difficult as it requires exploring a vast space of parameter inputs that may involve objects with complex data types. Existing search based approaches, with limited knowledge of relations between object states and program branches, often suffer from the low efficiency issue, i.e., tending to generate invalid inputs. Symbolic execution based approaches can effectively identify such relations, but fail to scale to large programs. In this work, we present an LLM-based input space partitioning testing approach, LISP, for library APIs. The approach lever-ages LLMs to understand the code of a library API under test and perform input space partitioning based on its understanding and rich common knowledge. Specifically, we provide the signature and code of the API under test to LLMs, with the expectation of obtaining a text description of each input space partition of the API under test. Then, we generate inputs through employing the generated text description to sample inputs from each partition, ultimately resulting in test suites that systematically explore the program behavior of the API. We evaluate LISP on more than 2,205 library API meth-ods taken from 10 popular open-source Java libraries (e.g.,$a$p$a$che/commons-lang with 2.6k stars, guava with 48.8k stars on GitHub). Our experiment results show that LISP is effective in library API testing. It significantly outperforms state-of-the-art tool EvoSuite in terms of edge coverage. On average, LISP achieves 67.82 % branch coverage, surpassing EvoSuite by 1.21 times. In total, LISP triggers 404 exceptions or errors in the experiments, and discovers 13 previously unknown vulnerabilities during evaluation, which have been assigned CVE IDs.

Large language models (LLMs), pre-trained or fine-tuned on large code corpora, have shown effectiveness in generating code completions. However, in LLM-based code completion, LLMs may struggle to use correct and up-to-date Application Programming Interfaces (APIs) due to the rapid and continuous evolution of libraries. While existing studies have highlighted issues with predicting incorrect APIs, the specific problem of deprecated API usage in LLM-based code completion has not been thoroughly investigated. To address this gap, we conducted the first evaluation study on deprecated API usage in LLM-based code completion. This study involved seven advanced LLMs, 145 API mappings from eight popular Python libraries, and$\mathbf{2 8, 1 2 5}$completion prompts. The study results reveal the status quo (i.e., API usage plausibility and deprecated usage rate) of deprecated API and replacing API usage in LLM-based code completion from the perspectives of model, prompt, and library, and indicate the root causes behind. Based on these findings, we propose two lightweight fixing approaches, Replaceapi and InsertPrompt, which can serve as baseline approaches for future research on mitigating deprecated API usage in LLM-based completion. Additionally, we provide implications for future research on integrating library evolution with LLMdriven software development.

Resource leaks, caused by resources not being released after acquisition, often lead to performance issues and system crashes. Existing static detection techniques rely on mechanical matching of predefined resource acquisition/release APIs and null-checking conditions to find unreleased resources, suffering from both (1) false negatives caused by the incompleteness of predefined resource acquisition/release APIs and (2) false positives caused by the incompleteness of resource reachability validation identification. To overcome these challenges, we propose InferROI, a novel approach that leverages the exceptional code comprehension capability of large language models (LLMs) to directly infer resource-oriented intentions (acquisition, release, and reachability validation) in code. InferROI first prompts the LLM to infer involved intentions for a given code snippet, and then incorporates a two-stage static analysis approach to check control-flow paths for resource leak detection based on the inferred intentions. We evaluate the effectiveness of InferROI in both resource-oriented intention inference and resource leak detection. Experimental results on the DroidLeaks and JLeaks datasets demonstrate InferROI achieves promising bug detection rate (59.3% and 62.5%) and false alarm rate (18.6% and 19.5%). Compared to three industrial static detectors, InferROI detects 14~45 and 149~485 more bugs in DroidLeaks and JLeaks, respectively. When applied to real-world open-source projects, InferROI identifies 29 unknown resource leak bugs (verified by authors), with 7 of them being confirmed by developers. In addition, the results of an ablation study underscores the importance of combining LLM-based inference with static analysis. Finally, manual annotation indicated that InferROI achieved a precision of 74.6% and a recall of 81.8% in intention inference, covering more than 60% resource types involved in the datasets.

Web of Things (WoT) technology standardizes the integration of various IoT devices deployed in daily environments, promoting the capability of applications to automatically sense and regulate the physical environment.Meanwhile, the complex nature of such a ubiquitous software system, where heterogeneous applications, user activities, and environment states collectively influence device behaviors, poses risks of unexpected or even hazardous safety and security violations caused by improper device operations.Existing works on WoT violation identification primarily focus on the sole analysis of software applications, however, lacking consideration of the multi-source violations stemming from the human-cyberphysical ternary spaces, as well as the intricate interplay between environment and devices.Furthermore, the investigation into users' preferences for violation resolution remains unexplored.To address these limitations, we introduce EnvGuard, an environment-centric approach for customizing safety and security properties, identifying violations, and executing resolutions in the WoT environment.Our evaluation in two real-world WoT systems shows that Env-Guard outperforms previous state-of-the-art works, and confirms its usability, effectiveness, and runtime efficiency. CCS Concepts•

Automated test generation plays a crucial role in ensuring software security. It heavily relies on formal specifications to validate the correctness of the system behavior. However, the main approach to defining these formal specifications is through manual analysis of software documents, which requires a significant amount of engineering effort from experienced researchers and engineers. Meanwhile, system update further increases the human labor cost to maintain a corresponding formal specification, making the manual analysis approach a time-consuming and error-prone task. Recent advances in Large Language Models (LLMs) have demonstrated promising capabilities in natural language understanding. Yet, the feasibility of using LLMs to automate the extraction of formal specifications from software documents remains unexplored. We conduct an empirical study by constructing a comprehensive dataset comprising 603 specifications from 37 documents across three representative open-source software. We then evaluate the most recent LLMs' capabilities in extracting formal specifications from documents in an end-to-end fashion, including GPT-4o, Claude, and Llama. Our study demonstrates the application of LLMs in formal specification extraction tasks while identifying two major limitations: specification oversimplification and specification fabrication. We attribute these deficiencies to the LLMs' inherent limitations in processing and expressive capabilities, as well as their tendency to fabricate fictional information. Inspired by human cognitive processes, we propose a novel two-stage method, annotation-then-conversion, to address these challenges. Our method decomposes the task into sentence annotation and temporal logic conversion, reducing the demands on LLMs' processing and expressive capabilities for each subtask. Furthermore, by generating verifiable sentence-specification pairs, our method enables effective fact-checking, thereby mitigating hallucination effects. Our method demonstrates significant improvements over the end-to-end method, with a 29.2 % increase in the number of correctly extracted specifications and a 14.0 % improvement in average accuracy. In particular, our best-performing LLM achieves an accuracy of$\mathbf{7 1. 6 \%}$.

Open source software (OSS) has become the foundation of modern applications, but its transitive dependencies make it especially vulnerable to supply chain attacks. One common tactic is to inject malicious code into third-party packages. NPM, in particular, due to its widespread use and large volume of packages, has become the popular target of malicious code injection. While various detectors have been proposed, they suffer three limitations, i.e., inadequate behavior modeling of obfuscated code, ignoring object-centric features of JavaScript, and lack of synergy between static and dynamic analysis. These limitations lead to imprecise modeling of program behavior and hinder detection effectiveness.To address these limitations, we propose ProfMal to identify malicious NPM packages, which leverages the synergy between static and dynamic analysis to construct behavior graphs for each package. Specifically, our static analysis constructs the behavior graphs through object-sensitive analysis, while identifying sensitive API calls and locating statically unresolved calls. Our dynamic analysis augments the behavior graphs by resolving those statically unresolved calls. Based on these comprehensive behavior graphs, we train a graph-based classifier to identify maliciousness. Our evaluation has indicated that ProfMal achieves the highest F1-score of 92.4%, outperforming the state-of-the-arts by 6.2% to 48.8%. During a three-month real-world detection, ProfMal has detected 496 previously unknown malicious NPM packages, and all of them have been confirmed and removed from NPM.

Recent advancements in large language models (LLMs) have demonstrated impressive capabilities in code translation, typically evaluated using benchmarks like CodeTransOcean and RepoTransBench. However, dependency-free benchmarks fail to capture real-world complexities by focusing primarily on simple function-level translations and overlooking repository-level context (e.g., dependencies). Full-repository translation benchmarks significantly exceed the current capabilities of existing models, resulting in performance bottlenecks that fail to provide actionable insights for guiding model development. Furthermore, existing benchmarks do not account for the scenario of incrementally translating new or modified modules from the source to the target language, which demands careful handling of repository-level contexts such as dependencies, cross-module references, and architectural divergence. Moreover, LLMs’ effectiveness in translating to newer, low-resource languages like Rust remains largely underexplored.To address these gaps, we introduce RustRepoTrans, the first repository-level context code translation benchmark targeting incremental translation, comprising 375 tasks translating into Rust from C, Java, and Python. Using this benchmark, we evaluate seven representative LLMs, analyzing their errors to assess limitations in complex translation scenarios. Among them, DeepSeek-R1 performs best with 51.5% Pass@1, excelling in both basic functionality and additional translation abilities, such as noise robustness and syntactical difference identification. However, even DeepSeek-R1 experiences a 22.2% performance drop (Pass@1 from 73.7% to 51.5%) when handling repository-level context compared to previous benchmarks without such context. Meanwhile, we propose a set of more fine-grained evaluation metrics and an enhanced evaluation framework, enabling a more comprehensive analysis of LLMs’ performance in repository-level context code translation tasks to provide fine-grained insights that can effectively inform the development of code translation techniques.

End-to-end autonomous driving systems (ADSs), with their strong capabilities in environmental perception and generalizable driving decisions, are attracting growing attention from both academia and industry. However, once deployed on public roads, ADSs are inevitably exposed to diverse driving hazards that may compromise safety and degrade system performance. This raises a strong demand for resilience of ADSs, particularly the capability to continuously monitor driving hazards and adaptively respond to potential safety violations, which is crucial for maintaining robust driving behaviors in complex driving scenarios.To bridge this gap, we propose a resilience-oriented runtime framework, named Argus, to mitigate the driving hazards, thus preventing potential safety violations and improving the driving performance of an ADS. Argus continuously monitors the trajectories generated by the ADS for potential hazards and, whenever the EGO vehicle is deemed unsafe, seamlessly takes control via a hazard mitigator. We integrate Argus with three state-of-the-art end-to-end ADSs, i.e., TCP, UniAD and VAD. Our evaluation has demonstrated that Argus effectively and efficiently enhances the resilience of ADSs, improving the driving score of ADSs by 150.30% on average, and preventing 64.38% of the violations, with little additional time overhead.

Large-scale scenarios, such as drone-based search and rescue, often require seamless access to devices distributed across hierarchical edge clusters. However, existing multi-cluster communication solutions designed for cloud environments cannot be directly applied to edge environments due to resource constraints, network limitations, and privacy concerns. To address these challenges, this paper introduces EdgeConnector, a lightweight middleware specifically designed to enable seamless and efficient cross-cluster device access in edge environments. EdgeConnector consists of components deployed across superior and subordinate clusters and employs a compact mechanism leveraging eXpress Data Path (XDP) for highly efficient device communication between clusters. The middleware was evaluated in both real-world and simulated environments. A real-world case study demonstrates its practicality and usability, while experimental results from the simulated environment highlight its superior performance. Specifically, EdgeConnector achieves an 80% reduction in latency and a 90% reduction in CPU usage under high-load conditions compared to the leading existing solution for cross-cluster service access.

Abstract Researchers have recently achieved significant advances in deep learning techniques, which in turn has substantially advanced other research disciplines, such as natural language processing, image processing, speech recognition, and software engineering. Various deep learning techniques have been successfully employed to facilitate software engineering tasks, including code generation, software refactoring, and fault localization. Many studies have also been presented in top conferences and journals, demonstrating the applications of deep learning techniques in resolving various software engineering tasks. However, although several surveys have provided overall pictures of the application of deep learning techniques in software engineering, they focus more on learning techniques, that is, what kind of deep learning techniques are employed and how deep models are trained or fine-tuned for software engineering tasks. We still lack surveys explaining the advances of subareas in software engineering driven by deep learning techniques, as well as challenges and opportunities in each subarea. To this end, in this study, we present the first task-oriented survey on deep learning-based software engineering. It covers twelve major software engineering subareas significantly impacted by deep learning techniques. Such subareas spread out through the whole lifecycle of software development and maintenance, including requirements engineering, software development, testing, maintenance, and developer collaboration. As we believe that deep learning may provide an opportunity to revolutionize the whole discipline of software engineering, providing one survey covering as many subareas as possible in software engineering can help future research push forward the frontier of deep learning-based software engineering more systematically. For each of the selected subareas, we highlight the major advances achieved by applying deep learning techniques with pointers to the available datasets in such a subarea. We also discuss the challenges and opportunities concerning each of the surveyed software engineering subareas.

Recent code large language models (LLMs) have shown promising performance in generating standalone functions. However, they face limitations in repository-level code generation due to their lack of awareness of repository-level dependencies ( e.g., user-defined attributes), resulting in dependency errors such as undefined-variable and no-member errors. In this work, we introduce ToolGen , an approach that integrates autocompletion tools into the code LLM generation process to address these dependencies. ToolGen comprises two main phases: Trigger Insertion and Model Fine-tuning (Offline), and Tool-integrated Code Generation (Online). During the offline phase, ToolGen augments functions within a given code corpus with a special mark token, indicating positions to trigger autocompletion tools. These augmented functions, along with their corresponding descriptions, are then used to fine-tune a selected code LLM. In the online phase, ToolGen iteratively generates functions by predicting tokens step-by-step using the fine-tuned LLM. Whenever a mark token is encountered, ToolGen invokes the autocompletion tool to suggest code completions and selects the most appropriate one through constrained greedy search. We conduct comprehensive experiments to evaluate ToolGen ’s effectiveness in repository-level code generation across three distinct code LLMs: CodeGPT, CodeT5, and CodeLlama. To facilitate this evaluation, we create a benchmark comprising 671 real-world code repositories and introduce two new dependency-based metrics: Dependency Coverage and Static Validity Rate . The results demonstrate that ToolGen significantly improves Dependency Coverage by 31.4% to 39.1% and Static Validity Rate by 44.9% to 57.7% across the three LLMs, while maintaining competitive or improved performance in widely recognized similarity metrics such as BLEU-4, CodeBLEU, Edit Similarity, and Exact Match. On the CoderEval dataset, ToolGen achieves improvements of 40.0% and 25.0% in test pass rate (Pass@1) for CodeT5 and CodeLlama, respectively, while maintaining the same pass rate for CodeGPT. ToolGen also demonstrates high efficiency in repository-level code generation, with latency ranging from 0.63 to 2.34 seconds for generating each function. Furthermore, our generalizability evaluation confirms ToolGen ’s consistent performance when applied to diverse code LLMs, encompassing various model architectures and scales.

Open source software (OSS) supply chain enlarges the attack surface of a software system, which makes package registries attractive targets for attacks. Recently, multiple package registries have received intensified attacks with malicious packages. Of those package registries, NPM and PyPI are two of the most severe victims. Existing malicious package detectors are developed with features from a list of packages of the same ecosystem and deployed within the same ecosystem exclusively, which is infeasible to utilize the knowledge of a new malicious NPM package detected recently to detect the new malicious package in PyPI. Moreover, existing detectors lack support to model malicious behavior of OSS packages in a sequential way. To address the two limitations, we propose a single detection model using malicious behavior sequence, named Cerebro , to detect malicious packages in NPM and PyPI. We curate a feature set based on a high-level abstraction of malicious behavior to enable multi-lingual knowledge fusing. We organize extracted features into a behavior sequence to model sequential malicious behavior. We fine-tune the pre-trained language model to understand the semantics of malicious behavior. Extensive evaluation has demonstrated the effectiveness of Cerebro over the state-of-the-art as well as the practically acceptable efficiency. Cerebro has detected 683 and 799 new malicious packages in PyPI and NPM, and received 707 thank letters from the official PyPI and NPM teams.

The widespread use of open-source software (OSS) has led to extensive code reuse, making vulnerabilities in OSS significantly pervasive.The vulnerabilities due to code reuse in OSS are commonly known as vulnerable code clones (VCCs) or recurring vulnerabilities.Existing approaches primarily employ clone-based techniques to detect recurring vulnerabilities by matching vulnerable functions in software projects.These techniques do not incorporate specially designed mechanisms for vulnerabilities with multiple fixing functions (VM).Typically, they generate a signature for each fixing function and report VM using a matching-one-in-all approach.However, the variation in vulnerability context across diverse fixing functions results in varying accuracy levels in detecting VM, potentially limiting the effectiveness of existing methods.In this paper, we introduce VMud, a novel approach for detecting Vulnerabilities with Multiple Fixing Functions.VMud identifies vulnerable function clones (VCCs) through function matching similar to existing methods.However, VMud takes a different approach by only selecting the critical functions from VM for signature generation, which are a subset of the fixing functions.This step ensures that VMud focuses on fixing functions that offer sufficient knowledge about the VM.To cope with the potential decrease in recall due to excluding the remaining fixing functions, VMud employs semantic equivalent statement matching using these critical functions.It aims to uncover more VM by creating two signatures of each critical function and matching precisely by contextual semantic equivalent statement mapping on the two signatures.Our evaluation has demonstrated that VMud surpasses state-of-the-art vulnerability detection approaches by 30.30% in terms of F1-Score.Furthermore,

Recently, many large language models (LLMs) have been proposed, showing advanced proficiency in code generation. Meanwhile, many efforts have been dedicated to evaluating LLMs on code generation benchmarks such as HumanEval. Although being very helpful for comparing different LLMs, existing evaluation focuses on a simple code generation scenario (i.e., function-level or statement-level code generation), which mainly asks LLMs to generate one single code unit (e.g., a function or a statement) for the given natural language description. Such evaluation focuses on generating independent and often small-scale code units, thus leaving it unclear how LLMs perform in real-world software development scenarios.

Code review is an essential activity for ensuring the quality and maintainability of software projects. However, it is a time-consuming and often error-prone task that can significantly impact the development process. Recently, ChatGPT, a cutting-edge language model, has demonstrated impressive performance in various natural language processing tasks, suggesting its potential to automate code review processes. However, it is still unclear how well ChatGPT performs in code review tasks. To fill this gap, in this paper, we conduct the first empirical study to understand the capabilities of ChatGPT in code review tasks, specifically focusing on automated code refinement based on given code reviews. To conduct the study, we select the existing benchmark CodeReview and construct a new code review dataset with high quality. We use CodeReviewer, a state-of-the-art code review tool, as a baseline for comparison with ChatGPT. Our results show that ChatGPT outperforms CodeReviewer in code refinement tasks. Specifically, our results show that ChatGPT achieves higher EM and BLEU scores of 22.78 and 76.44 respectively, while the state-of-the-art method achieves only 15.50 and 62.88 on a high-quality code review dataset. We further identify the root causes for ChatGPT's underperformance and propose several strategies to mitigate these challenges. Our study provides insights into the potential of ChatGPT in automating the code review process, and highlights the potential research directions.

Software composition analysis (SCA) tools have been widely adopted to identify vulnerable libraries used in software applications. Such SCA tools depend on a vulnerability database to know affected libraries of each vulnerability. However, it is labor-intensive and error prone for a security team to manually maintain the vulnerability database. While several approaches adopt extreme multi-label learning to predict affected libraries for vulnerabilities, they are practically ineffective due to the limited library labels and the unawareness of ecosystems.

Modern microservice systems have become increasingly complicated due to the dynamic and complex interactions and runtime environment. It leads to the system vulnerable to performance issues caused by a variety of reasons, such as the runtime environments, communications, coordinations, or implementations of services. Traces record the detailed execution process of a request through the system and have been widely used in performance issues diagnosis in microservice systems. By identifying the execution processes and attribute value combinations that are common in anomalous traces but rare in normal traces, engineers may localize the root cause of a performance issue into a smaller scope. However, due to the complex structure of traces and the large number of attribute combinations, it is challenging to find the root cause from the huge search space. In this paper, we propose TraceContrast, a trace-based multi-dimensional root cause localization approach. TraceContrast uses a sequence representation to describe the complex structure of a trace with attributes of each span. Based on the representation, it combines contrast sequential pattern mining and spectrum analysis to localize multi-dimensional root causes efficiently. Experimental studies on a widely used microservice benchmark show that TraceContrast outperforms existing approaches in both multi-dimensional and instance-dimensional root cause localization with significant accuracy advantages. Moreover, Trace-Contrast is efficient and its efficiency can be further improved by parallel execution.

Pre-trained code models have achieved notable success in the field of Software Engineering (SE). However, existing studies have predominantly focused on improving model performance, with limited attention given to other critical aspects such as model calibration. Model calibration, which refers to the accurate estimation of predictive uncertainty, is a vital consideration in practical applications. Therefore, in order to advance the understanding of model calibration in SE, we conduct a comprehensive investigation into the calibration of pre-trained code models in this paper. Our investigation focuses on five pre-trained code models and four code understanding tasks, including analyses of calibration in both in-distribution and out-of-distribution settings. Several key insights are uncovered: (1) pre-trained code models may suffer from the issue of over-confidence; (2) temperature scaling and label smoothing are effective in calibrating code models in in-distribution data; (3) the issue of over-confidence in pre-trained code models worsens in different out-of-distribution settings, and the effectiveness of temperature scaling and label smoothing diminishes. All materials used in our experiments are available at https://github.com/queserasera22/Calibration-of-Pretrained-Code-Models.

Open source software brings benefit to the software community but also introduces legal risks caused by license violations, which result in serious consequences such as lawsuits and financial losses. To mitigate legal risks, some approaches have been proposed to identify licenses, detect license incompatibilities and inconsistencies, and recommend licenses. As far as we know, however, there is no prior work to understand modification terms in open source licenses or to detect and fix violations of modification terms. To bridge this gap, we first empirically characterize modification terms in 48 open source licenses. These licenses all require certain forms of “notice” to describe the modifications made to the original work. Inspired by our study, we then design LiVo to automatically detect and fix violations of modification terms in open source licenses during forking. Our evaluation has shown the effectiveness and efficiency of LiVo. 18 pull requests for fixing modification term violations have received positive responses. 8 have been merged.

Flaky tests hinder the development process by exhibiting uncertain behavior in regression testing. A flaky test may pass in some runs and fail in others while running on the same code version. The non-deterministic outcome frequently misleads the developers into debugging non-existent faults in the code. To effectively debug the flaky tests, developers need to reproduce them. The industry de facto to reproduce flaky tests is to rerun them multiple times. However, rerunning a flaky test numerous times is time and resource-consuming. This work presents a technique for rapidly and reliably reproducing timing-dependent GUI flaky tests, acknowledged as the most common type of flaky tests in Android apps. Our insight is that flakiness in such tests often stems from event racing on GUI data. Given stack traces of a failure, our technique employs dynamic analysis to infer event races likely leading to the failure and reproduces it by selectively delaying only relevant events involved in these races. Thus, our technique can efficiently reproduce a failure within minimal test runs. The experiments conducted on 80 timing-dependent flaky tests collected from 22 widely-used Android apps show our technique is efficient in flaky test failure reproduction. Out of the 80 flaky tests, our technique could successfully reproduce 73 within 1.71 test runs on average. Notably, it exhibited extremely high reliability by consistently reproducing the failure for 20 runs.

Simulation testing has been widely adopted by leading companies to ensure the safety of autonomous driving systems (ADSs). Anumber of scenario-based testing approaches have been developed to generate diverse driving scenarios for simulation testing, and demonstrated to be capable of finding safety violations. However, there is no automated way to diagnose whether these violations are caused by the ADS under test and which category these violations belong to. As a result, great effort is required to manually diagnose violations. To bridge this gap, we propose DiaVio to automatically diagnose safety violations in simulation testing by leveraging large language models (LLMs). It is built on top of a new domain specific language (DSL) of crash to align real-world accident reports described in natural language and violation scenarios in simulation testing. DiaVio fine-tunes a base LLM with real-world accident reports to learn diagnosis capability, and uses the fine-tuned LLM to diagnose violation scenarios in simulation testing. Our evaluation has demonstrated the effectiveness and efficiency of DiaVio in violation diagnosis.

Data-driven techniques are promising for automatically locating and fixing bugs, which can reduce enormous time and effort for developers. However, the effectiveness of these techniques heavily relies on the quality and scale of bug datasets. Despite that emerging approaches to automatic bug dataset construction partially provide a solution for scalability, data quality remains a concern. Specifically, it remains a barrier for humans to isolate the minimal set of bug-inducing or bug-fixing changes, known as critical changes. Although delta debugging (DD) techniques are capable of extracting critical changes on benchmark datasets in academia, the efficiency and accuracy are still limited when dealing with real-world bugs, where code change dependencies could be overly complicated. In this paper, we propose C2D2, a novel delta debugging approach for critical change extraction, which estimates the probabilities of dependencies between code change elements. C2D2 considers the probabilities of dependencies and introduces a matrix-based search mechanism to resolve compilation errors (CE) caused by missing dependencies. It also provides hybrid mechanisms for flexibly selecting code change elements during the DD process. Experiments on Defect4J and a real-world regression bug dataset reveal that C2D2 is significantly more efficient than the traditional DD algorithm ddmin with competitive effectiveness, and significantly more effective and more efficient than the state-of-the-art DD algorithm ProbDD. Furthermore, compared to human-isolated critical changes, C2D2 produces the same or better critical change results in 56% cases in Defects4J and 86% cases in the regression dataset, demonstrating its usefulness in automatically extracting critical changes and saving human efforts in constructing large-scale bug datasets with real-world bugs.

Open source software (OSS) supply chains have been attractive targets for attacks. One of the significant, popular attacks is realized by malicious packages on package registries. NPM, as the largest package registry, has been recently flooded with malicious packages. In response to this severe security risk, many detection tools have been proposed. However, these tools do not model malicious behavior in a holistic way; only consider a predefined set of sensitive APIs; and require huge manual confirmation effort due to high false positives and binary detection results. Thus, their practical usefulness is hindered.

Vulnerability reports play a crucial role in mitigating open-source software risks. Typically, the vulnerability report contains affected versions of a software. However, despite the validation by security expert who discovers and vendors who review, the affected versions are not always accurate. Especially, the complexity of maintaining its accuracy increases significantly when dealing with multiple versions and their differences. Several advances have been made to identify affected versions. However, they still face limitations. First, some existing approaches identify affected versions based on repository-hosting platforms (i.e., GitHub), but these versions are not always consistent with those in package registries (i.e., Maven). Second, existing approaches fail to distinguish the importance of different vulnerable methods and patched statements in face of vulnerabilities with multiple methods and change hunks.

The wide adoption of open source third-party libraries can propagate vulnerabilities that originally exist in third-party libraries through dependency chains to downstream projects. To mitigate this security risk, vulnerability exploitation analysis has been proposed to further reduce false positives of vulnerability reachability analysis. However, existing approaches work less effectively when the vulnerable function of the vulnerable library is indirectly invoked by a client project through a call chain of multiple steps.

Null pointer dereference raises Null Pointer Exceptions (NPEs). There are two groups of approaches to detect NPEs. Type-based approaches carry out strict type-based null safety checking. They heavily rely on annotations, and thus produce many false positives. Dataflow-based approaches leverage static forward and/or backward dataflow analysis. They mostly have a limited capability in tracking fields and interprocedural analysis, and introduce false positives and false negatives. To address these drawbacks, we propose Wheeljack to detect NPEs for Java. It does not rely on annotations, and hence can work effectively under a lack of annotations. It leverages our novel abstraction of nullness status to enhance field tracking, and our novel invocation analysis (capturing change to return value and side effect of an invocation) to enhance interprocedural analysis. Our evaluation on 28 Java projects has demonstrated that Wheeljack can mostly outperform the four state-of-the-art NPE detectors in recall without sacrificing precision. 5 and 2 new NPEs have been confirmed and fixed by developers after we submit 8 issues.

Changes on source code may propagate to distant code entities through various kinds of relationships, which may form up change propagation channels . It is however difficult for developers to reveal code change propagate channels due to sophisticated interrelationships among code entities. In this work, we propose a novel graph representation for the changed code entities and related code entities changed within a range of space and time so that the types of relationships along which the changes are propagated can be explicitly presented. Then a subgraph mining technique is used to find the frequent change propagation channels . We finally reveal 40 types of frequent change propagation channels that cover over 98% cases of code change propagation in five well-known open-source Java projects. We find evidence that the code changes propagated through an unchanged intermediate code entity consume more time than those through a changed one, indicating the difficulties in maintaining code entities that related through indirect relationships. We find that a small proportion of code entities frequently appear in the FCPCs, and confirm the semantic relationships between code entities covered by 50 instances of FCPCs, indicating potential usefulness for developers to explain the range of change impact from given source code changes.

Partial code usually involves non-fully-qualified type names (non-FQNs) and undeclared receiving objects. Resolving the FQNs of these non-FQN types and undeclared receiving objects (referred to as type inference) is the prerequisite to effective search and reuse of partial code. Existing dictionary-lookup based methods build a symbolic knowledge base of API names and code contexts, which involve significant compilation overhead and are sensitive to unseen API names and code context variations. In this article, we propose using a p rompt-tuned c o de m asked language mod e l (MLM) as a neural knowledge base for type inference, called POME, which is lightweight and has minimal requirements on code compilation. Unlike the existing symbol name and context matching for type inference, POME infers the FQNs syntax and usage knowledge encapsulated in prompt-tuned code MLM through a colze-style fill-in-blank strategy. POME is integrated as a plug-in into web and integrated development environments (IDE) to assist developers in inferring FQNs in the real world. We systematically evaluate POME on a large amount of source code from GitHub and Stack Overflow, and explore its generalization and hybrid capability. The results validate the effectiveness of the POME design and its applicability for partial code type inference, and they can be easily extended to different programming languages (PL). POME can also be used to generate a PL-hybrid type inference model for providing a one-for-all solution. As the first of its kind, our neural type inference method opens the door to many innovative ways of using partial code.

Developers’ API needs should be more pragmatic, such as seeking suggestive, explainable, and extensible APIs rather than the so-called best result. Existing API search research cannot meet these pragmatic needs because they are solely concerned with query-API relevance. This necessitates a focus on enhancing the entire query process, from query definition to query refinement through intent clarification to query results promoting divergent thinking about results. This paper designs a novel Knowledge-Aware Human-AI Dialog agent (KAHAID) which guides the developer to clarify the uncertain, under-specified query through multi-round question answering and recommends APIs for the clarified query with relevance explanation and extended suggestions (e.g., alternative, collaborating or opposite-function APIs). We systematically evaluate KAHAID. In terms of human-AI dialogue process, it achieves a high diversity of question options (the average diversity between any two options is 74.9%) and the ability to guide developers to find APIs using fewer dialogue rounds (no more than 3 rounds on average). For API recommendation, KAHAID achieves an MRR and MAP of 0.769 and 0.794, outperforming state-of-the-art API search approaches BIKER and CLEAR by at least 47% in MRR and 226.7% in MAP. For knowledge extension, KAHAID obtains an MRR and MAP of 0.815 and 0.864, surpassing state-of-the-art query clarification approaches by at least 42% in MRR and 45.2% in MAP. As the first of its kind, KAHAID opens the door to integrating the immediate response capability of API research and the interaction, clarification, explanation, and extensibility capability of social-technical information seeking.

Automatic static analysis tools (ASATs) detect source code violations to static analysis rules and are usually used as a guard for source code quality. The adoption of ASATs, however, is often challenged because of several problems such as a large number of false alarms, invalid rule priorities, and inappropriate rule configurations. Research has shown that tracking the history of the violations is a promising way to solve the above problems because the facts of violation fixing may reflect the developers' subjective expectations on the violation detection results. Precisely identifying the revisions that induce or fix a violation is however challenging because of the imprecise matching of violations between code revisions and ignorance of merge commits in the maintenance history. In this paper, we propose ViolationTracker, an approach to precisely matching the violation instances between adjacent revisions and building the life cycle of violations with the identification of inducing, fixing, deleting, and reopening of each violation case. The approach employs code entity anchoring heuristics for violation matching and considers merge commits that used to be ignored in existing research. We evaluate ViolationTracker with a manually-validated dataset that consists of 500 violation instances and 158 threads of 30 violation cases with detailed evolution history from open-source projects. Violation Tracker achieves over 93 % precision and 98 % recall on violation matching, outperforming the state-of-the-art approach, and 99.4 % precision on rebuilding the histories of violation cases. We also show that ViolationTracker is useful to identify actionable violations. A preliminary empirical study reveals the possibility to prioritize static analysis rules according to further analysis on the actionable rates of the rules.

Developers often seek solutions for their programming problems by retrieving existing questions on technical Q&A sites such as Stack Overflow. In many cases, they fail to find relevant questions due to the knowledge gap between the questions and the queries or feel it hard to choose the desired questions from the returned results due to the lack of explanations about the relevance. In this paper, we propose KGXQR, a knowledge graph based explainable question retrieval approach for programming tasks. It uses BERT-based sentence similarity to retrieve candidate Stack Overflow questions that are relevant to a given query. To bridge the knowledge gap and enhance the performance of question retrieval, it constructs a software development related concept knowledge graph and trains a question relevance prediction model to re-rank the candidate questions. The model is trained based on a combined sentence representation of BERT-based sentence embedding and graph-based concept embedding. To help understand the relevance of the returned Stack Overflow questions, KGXQR further generates explanations based on the association paths between the concepts involved in the query and the Stack Overflow questions. The evaluation shows that KGXQR outperforms the baselines in terms of accuracy, recall, MRR, and MAP and the generated explanations help the users to find the desired questions faster and more accurately.

Machine learning (ML) enabled systems are emerging with recent breakthroughs in ML. A model-centric view is widely taken by the literature to focus only on the analysis of ML models. However, only a small body of work takes a system view that looks at how ML components work with the system and how they affect software engineering for ML-enabled systems. In this paper, we adopt this system view, and conduct a case study on Rasa 3.0, an industrial dialogue system that has been widely adopted by various companies around the world. Our goal is to characterize the complexity of such a large-scale ML-enabled system and to understand the impact of the complexity on testing. Our study reveals practical implications for software engineering for ML-enabled systems.

Fault diagnosis in robotic systems is challenging due to their complex and heterogeneous structures and complex interactions with physical environments. Given the complexities and uncertainties, we think it may be helpful to diagnose faults of a robotic system by understanding its behaviors from the perspective of observability. In this paper, we conduct an empirical study to explore the efficacy of combining different kinds of common observability data (i.e., logs, traces, and trajectories) for fault diagnosis in robotic systems. In the study, we investigate root causes of 398 bug cases in robotic systems to understand their characteristics. Furthermore, we replicate 23 bugs out of them and perform a fault diagnosis study in which participants diagnose each of the replicated bug with only observability data and record how useful observability data is. The bug case analysis study revealed that the root causes of bugs in robotic systems originate from various levels, including physical environment interaction (11.81%), hardware usage (14.82%), software implementation (49.25%), and system configuration (24.12%). The fault diagnosis study shows the combination of trace and trajectory data improves the fault diagnosis success rate by 58.33% and 8.33%, respectively, compared to using only logs. Our study promotes the vision of observability-based fault diagnosis in robotic systems.

Modern large-scale service-based systems such as microservice systems have become increasingly complex, making it hard to localize anomalous services when various issues emerge. Traces record the workflows of requests through service instances and have been widely used in anomaly detection and root cause analysis. Existing trace-based approaches widely use statistical methods or learning-based techniques to detect trace anomalies and localize anomalous services. However, these approaches often suffer from the concept drift problem, i.e., the statistical properties of traces change over time in unforeseen ways. In this paper, we propose TraceStream, an anomalous service localization approach based on trace data stream clustering. TraceStream uses data stream clustering to discover potential anomalous trace clusters in evolving trace data and uses spectrum analysis to localize anomalous services based on the clusters. Moreover, TraceStream can effectively incorporate the online feedback of operation engineers based on the trace clusters to improve the accuracy for localizing anomalous services. Our evaluation confirms that TraceStream can effectively detect anomalies and localize anomalous services in an evolving microservice system. It can effectively incorporate human feedback to further improve the performance of anomalous service localization. Moreover, TraceStream is efficient and its efficiency can be further improved by sampling a small portion of traces by cluster.

Although deep learning (DL) software has been pervasive in various applications, the brittleness of deep neural networks (DNN) hinders their deployment in many tasks especially high-stake ones. To mitigate the risk accompanied with DL software fault, a variety of DNN testing techniques have been proposed such as test case selection. Among those test case selection or prioritization methods, the uncertainty-based ones such as DeepGini have demonstrated their effectiveness in finding DNN’s faults. Recently, TestRank, a learning based test ranking method has shown their out-performance over simple uncertainty-based test selection methods. However, this is achieved with a more complicated design which needs to train a graph convolutional network and a multi-layer Perceptron. In this paper, we propose a novel and lightweight DNN test selection method to enhance the effectiveness of existing simple ones. Besides the DNN model’s uncertainty on test case itself, we take into account model’s uncertainty on its neighbors. This could diversify the selected test cases and improve the effectiveness of existing uncertainty-based test selection methods. Extensive experiments on 5 datasets demonstrate the effectiveness of our approach.

A fault in large online service systems often triggers numerous alerts due to the complex business and component dependencies among services, which is known as “alert storm”. In a short time, an online service system may generate a huge amount of alert data. This poses a challenge for on-call engineers to identify alerts that are associated with a system failure for root cause analysis. In this paper, we propose DyAlert, a dynamic graph neural networks-based approach for linking alerts that might be triggered by a same fault to reduce the burden of on-call engineers in the fault analysis. Our insight is that alerts are often triggered by alert propagation when a system failure occurs, e.g., alert$a$would lead to the occurrence of alert$b$. Whether two alerts should be linked depends on if one alert is triggered by the propagation of the other. Leveraging this insight, we design a dynamic graph (namely Alert-Metric Dynamic Graph) that describes the propagation process of alerts. Based on the dynamic graph, we train a neural networks-based model to predict alert links. We evaluate DyAlert with real-world data collected from an online service system running 85 business units and about 30,000 different services in a large enterprise. The results show that DyAlert is effective in predicting alert links and it outperforms the state-of-the-art approaches with an average increase of 0.259 in F1-score.

Pre-trained code models (e.g. CodeBERT and CodeT5) have demonstrated their code intelligence in various software engineering tasks, such as code summarization. And full fine-tuning has become the typical approach to adapting these models to downstream tasks. However, full fine-tuning these large models can be computationally expensive and memory-intensive, particularly when training for multiple tasks. To alleviate this issue, several parameter-efficient fine-tuning methods (e.g. Adapter and LoRA) have been proposed to only train a small number of additional parameters, while keeping the original pre-trained parameters frozen. Although these methods claim superiority over the prior techniques, they seldom make a comprehensive and fair comparison on multiple software engineering tasks. Moreover, besides their potential in reducing fine-tuning costs and maintaining approximate performance, the effectiveness of these methods in low-resource, cross-language, and cross-project scenarios is inadequately studied. To this end, we first conduct experiments by fine-tuning state-of-the-art code models with these methods on both code understanding tasks and code generation tasks. The results show that, by tuning only 0.5% additional parameters, these methods may achieve comparable or higher performance than full fine-tuning in code understanding tasks, but they may exhibit slightly weaker performance in code generation tasks. We also investigate the impact of these methods with varying numbers of training samples and find that, a considerable number of samples (e.g. 1000 for clone detection) may be required for them to approximate the performance of full fine-tuning. Our experimental results in cross-language and cross-project scenarios demonstrate that by freezing most pre-trained parameters and tuning only 0.5% additional parameters, these methods achieve consistent improvements in models' transfer learning ability in comparison to full fine-tuning. Our code and data are available at https://github.com/anonymous-ase23/ CodeModelParameterEfficientFinetuning.

Automated code generation has been extensively studied in recent literature. In this work, we first survey 66 participants to motivate a more pragmatic code generation scenario, i.e., library-oriented code generation, where the generated code should implement the functionally of the natural language query with the given library. We then revisit existing learning-based code generation techniques and find they have limited effectiveness in such a library-oriented code generation scenario. To address this limitation, we propose a novel library-oriented code generation technique, CodeGen4Libs, which incorporates two stages: import generation and code generation. The import generation stage generates import statements for the natural language query with the given third-party libraries, while the code generation stage generates concrete code based on the generated imports and the query. To evaluate the effectiveness of our approach, we conduct extensive experiments on a dataset of 403,780 data items. Our results demonstrate that CodeGen4Libs outperforms baseline models in both import generation and code generation stages, achieving improvements of up to 97.4% on EM (Exact Match), 54.5% on BLEU, and 53.5% on Hit@All. Overall, our proposed CodeGen4Libs approach shows promising results in generating high-quality code with specific third-party libraries, which can improve the efficiency and effectiveness of software development.

Bugs and their fixes in the code evolution histories are important assets for many software engineering tasks such as deriving new state-of-the-art automatic bug fixing techniques. Existing bug datasets are either manually built which is difficult to grow efficiently to a scale large enough for massive data analysis, or lack of precise information of how bugs are introduced and fixed which is critical for in-depth analysis such as buggy/fixing code identification. Moreover, the types of the bugs are typically missing in the existing bug datasets, limiting the possibility of developing high-precision type-specific approaches for enterprise-level purposes. In this work, we propose BugMiner, an approach to automatically collecting bugs from code repositories by isolating the critical changes of the bugs. We also propose a learning-based approach for automating bug type classification with relatively small manual labels of bug types. We evaluate our approach regarding the precision of bug information and the efficiency of the bug-mining process with 2,082 bugs automatically mined from 100 open-source projects. We demonstrate the improved effectiveness and efficiency in bug-fixing location identification, compared to the SOTA BugBuilder, and high recall and precision in bug-inducing location identification. We also compare our learning-based bug classification approach to traditional baseline method, indicating about 17 % improvement in classification effectiveness under macro-F1.

As basic elements in program, variables convey essential information that is critical for program comprehension and maintenance. However, understanding the meanings of variables in program is not always easy for developers, since poor-quality variable names are prevalent while such variable are less informative for program comprehension. Therefore, in this paper, we target at generating concise natural language explanations for variables to facilitate program comprehension. In particular, there are two challenges in variable explanation generation, including the lack of training data and the association with complex code contexts around the variable. To address these issues, we propose a novel approach ZeroVar,which leverages code pre-trained models and zero-shot prompt learning to generate explanations for the variable based on its code context. ZeroVarcontains two stages: (i) a pre-training stage that continually pre-trains a base model (i.e., CodeT5) to recover the randomly-masked parameter descriptions in method docstrings; and (ii) a zero-shot prompt learning stage that leverages the pre-trained model to generate explanations for a given variable via the prompt constructed with the variable and its belonging method context. We then extensively evaluate the quality and usefulness of the variable explanations generated by ZeroVar.We construct an evaluation dataset of 773 variables and their reference explanations. Our results show that ZeroVarcan generate higher-quality explanations than baselines, not only on automated metrics such as BLEU and ROUGE, but also on human metrics such as correctness, completeness, and conciseness. Moreover, we further assess the usefulness of ZeroVAR-generated explanations on two downstream tasks related to variable naming quality, i.e., abbreviation expansion and spelling correction. For abbreviation expansion, the generated variable explanations can help improve the present rate (+13.1%), precision (+3.6%), and recall (+10.0%) of the state-of-the-art abbreviation explanation approach. For spelling correction, by using the generated explanations we can achieve higher hit@1 (+162.9(%) and hit@3 (+49.6%) than the recent variable representation learning approach.

Program synthesis aims to automatically generate an executable program that conforms to the given specification. Recent advancements have demonstrated that deep neural methodologies and large-scale pretrained language models are highly proficient in capturing program semantics. For robot programming, prior works have facilitated program synthesis by incorporating global environments. However, the assumption of acquiring a comprehensive understanding of the entire environment is often excessively challenging to achieve. In this work, we present a framework that learns to synthesize a program by rectifying potentially erroneous code segments, with the aid of partially observed environments. To tackle the issue of inadequate attention to partial observations, we propose to first learn an environment embedding space that can implicitly evaluate the impacts of each program token based on the precondition. Furthermore, by employing a graph structure, the model can aggregate both environmental and syntactic information flow and furnish smooth program rectification guidance. Extensive experimental evaluations and ablation studies on the partially observed VizDoom domain authenticate that our method offers superior generalization capability across various tasks and greater robustness when encountering noises.

Library migration, which replaces the current library with a different one to retain the same software behavior, is common in software evolution. An essential part of this is finding an analogous API for the desired functionality. However, due to the multitude of libraries/APIs, manually finding such an API is time-consuming and error-prone. Researchers created automated analogical API recommendation techniques, notably documentation-based methods. Despite potential, these methods have limitations, e.g., incomplete semantic understanding in documentation and scalability issues. In this study, we present KGE4AR, a novel documentation-based approach using knowledge graph (KG) embedding for recommending analogical APIs during library migration. KGE4AR introduces a unified API KG to comprehensively represent documentation knowledge, capturing high-level semantics. It further embeds this unified API KG into vectors for efficient, scalable similarity calculation. We assess KGE4AR with 35,773 Java libraries in two scenarios, with and without target libraries. KGE4AR notably outperforms state-of-the-art techniques (e.g., 47.1%-143.0% and 11.7%-80.6% MRR improvements), showcasing scalability with growing library counts.

Fixing crashes is challenging, and developers often discuss their encountered crashes and refer to similar crashes and solutions on online Q&A forums (e.g., Stack Overflow). However, a crash often involves very complex contexts, which includes different contextual elements, e.g., purposes, environments, code, and crash traces. Existing crash solution recommendation or general solution recommendation techniques only use an incomplete context or treat the entire context as pure texts to search relevant solutions for a given crash, resulting in inaccurate recommendation results. In this work, we propose a novel crash solution knowledge graph (KG) to summarize the complete crash context and its solution with a graph-structured representation. To construct the crash solution KG automatically, we propose to leverage prompt learning to construct the KG from SO threads with a small set of labeled data. Based on the constructed KG, we further propose a novel KG-based crash solution recommendation technique KG4CraSolver, which precisely finds the relevant SO thread for an encountered crash by finely analyzing and matching the complete crash context based on the crash solution KG. The evaluation results show that the constructed KG is of high quality and KG4CraSolver outperforms baselines in terms of all metrics (e.g., 13.4%-113.4% MRR improvements). Moreover, we perform a user study and find that KG4CraSolver helps participants find crash solutions 34.4% faster and 63.3% more accurately.

Deep learning (DL) applications, built upon a heterogeneous and complex DL stack (e.g., Nvidia GPU, Linux, CUDA driver, Python runtime, and TensorFlow), are subject to software and hardware dependencies across the DL stack. One challenge in dependency management across the entire engineering lifecycle is posed by the asynchronous and radical evolution and the complex version constraints among dependencies. Developers may introduce dependency bugs (DBs) in selecting, using and maintaining dependencies. However, the characteristics of DBs in DL stack is still under-investigated, hindering practical solutions to dependency management in DL stack. To bridge this gap, this paper presents the first comprehensive study to characterize symptoms, root causes and fix patterns of DBs across the whole DL stack with 446 DBs collected from StackOverflow posts and GitHub issues. For each DB, we first investigate the symptom as well as the lifecycle stage and dependency where the symptom is exposed. Then, we analyze the root cause as well as the lifecycle stage and dependency where the root cause is introduced. Finally, we explore the fix pattern and the knowledge sources that are used to fix it. Our findings from this study shed light on practical implications on dependency management.

Resource leaks, which are caused by acquired resources not being released, often result in performance degradation and system crashes. Resource leak detection relies on two essential components: identifying potential Resource Acquisition and Release (RAR) API pairs, and subsequently analyze code to uncover instances where the corresponding release API call is absent after an acquisition API call. Yet, existing techniques confine themselves to an incomplete pair pool, either pre-defined manually or mined from project-specific code corpus, thus limiting coverage across libraries/APIs and po- tentially overlooking latent resource leaks.

Small datasets are common in software engineering tasks such as linguistic smell detection and code runtime complexity prediction, as crafting these datasets often involves expert knowledge. Prior work usually applies machine learning algorithms (e.g., logistic regression and SVM) with hand-crafted features to tackle them, which could outperform neural models such as CNN. Recently, researchers have employed fine-tuning large pre-trained code models on various code-related tasks thanks to their transferability. However, it might be still instable and overfitting when fine-tuning on small datasets. In this paper, we firstly conduct an empirical study to fine-tune CodeBERT(a) on four code-related small datasets and observe the instability phenomenon. This could be induced by over-capacity and irrelevant features inherent in these large pre-trained code models with respective to those small datasets. To address this issue, we leverage variational information bottleneck to filter out irrelevant features when fine-tuning the models. The experiments demonstrate the out-performance of our method compared to standard fine-tuning and regularization method such as dropout and weight decay. We also experimentally study the stability of our method through varying dataset sizes. Our code and data are available at https://github.com/little-pikachu-hash/VIBCodeBERT.

Code changes are at the very core of software development and maintenance. Deep learning techniques have been used to build a model from a massive number of code changes to solve software engineering tasks, e.g., commit message generation and bug-fix commit identification. However, existing code change representation learning approaches represent code change as lexical tokens or syntactical AST (abstract syntax tree) paths, limiting the capability to learn semantics of code changes. Besides, they mostly do not consider noisy or tangled code change, hurting the accuracy of solved tasks. To address the above problems, we first propose a slice-based code change representation approach which considers data and control dependencies between changed code and unchanged code. Then, we propose a pre-trained sparse Transformer model, named CCS2VEC, to learn code change representations with three pre-training tasks. Our experiments by fine-tuning our pre-trained model on three downstream tasks have demonstrated the improvement of CCS2VEC over the state-of-the-art CC2VEC.

The physical world we live in is accelerating digitalization with the vigorous development of Internet of Things (IoT). Following this trend, Web of Things (WoT) further enables fast and efficient creation of various applications that perceive and act on the physical world using standard Web technologies. A popular way for creating WoT applications is Trigger-Action Programming (TAP), which allows users to orchestrate the capabilities of IoT devices in the form of “if trigger, then action”. However, existing TAP approaches don’t support scenario-centric WoT applications which involve abstract modeling of physical environments and complex spatio-temporal dependencies between events and actions. In this paper, we propose an approach called SCTAP which supports Scenario-Centric Trigger-Action Programming based on software-defined physical environments. SCTAP defines a structured and conceptual representation for physical environments, which provides the required programming abstractions for WoT applications. Based on the representation, SCTAP defines a grammar for specifying scenario-centric WoT applications with spatio-temporal dependencies. Furthermore, we design a service-based architecture for SCTAP which supports the integration of device access, event perception, environment representation, and rule execution in a loosely-coupled and extensible way. We implement SCTAP as a WoT infrastructure and evaluate it with two case studies including a smart laboratory and a smart coffee house. The results confirm the usability, feasibility and efficiency of SCTAP and its implementation.

Embracing LLMs is definitely a correct and even necessary direction for software enterprises to improve quality and efficiency. However, achieving systematic and comprehensive intelligent software development still requires careful consideration and there is much fundamental work to do. For enterprises, solidifying the digitization and knowledge accumulation of software development, as well as the fundamental capabilities of software engineering such as requirement analysis, design, and validation, remains crucial and is also a basic condition for achieving higher levels of intelligent development. For academic research, there is still much work to do in the direction of systematic and comprehensive intelligent software development. This also requires us have a deeper understanding of the complexity of software systems and software requirements and design, based on understanding the capabilities of LLMs.

When searching code, developers may express additional constraints (e.g., functional constraints and nonfunctional constraints) on the implementations of desired functionalities in the queries. Existing code search tools treat the queries as a whole and ignore the different implications of different parts of the queries. Moreover, these tools usually return a ranked list of candidate code snippets without any explanations. Therefore, the developers often find it hard to choose the desired results and build confidence on them. In this article, we conduct a developer survey to better understand and address these issues and induct some insights from the survey results. Based on the insights, we propose XCoS, an explainable code search approach based on query scoping and knowledge graph. XCoS extracts a background knowledge graph from general knowledge bases like Wikidata and Wikipedia. Given a code search query, XCoS identifies different parts (i.e., functionalities, functional constraints, nonfunctional constraints) from it and use the expressions of functionalities and functional constraints to search the codebase. It then links both the query and the candidate code snippets to the concepts in the background knowledge graph and generates explanations based on the association paths between these two parts of concepts together with relevant descriptions. XCoS uses an interactive user interface that allows the user to better understand the associations between candidate code snippets and the query from different aspects and choose the desired results. Our evaluation shows that the quality of the extracted background knowledge and the concept linkings in codebase is generally high. Furthermore, the generated explanations are considered complete, concise, and readable, and the approach can help developers find the desired code snippets more accurately and confidently.

Myriad of heterogeneous resources are widely distributed in the cyber, physical and social spaces. These resources are integrated by software to form diverse Cyber-Physical-Social Systems (CPSSs). Among them, the CPSSs in the form of resource choreography is receiving more attention. Traditional software development methods may not be suitable for constructing and executing CPSS applications (CPSS-Apps) with the characteristics of loosely-coupled resource collaboration and spatial-temporal constraints sensitive. In this paper we propose a comprehensive framework to support resource choreography from the perspectives of representation, modeling and execution. In the framework, a CPSS-App is represented by an application model conforming to a meta-model. An application model is generated by a multi-scene storyboard modeling tool. The model is further used to customize the capability units acting as the abstract unit of the resources providing the same service. An architecture following the microservice style is applied to achieve the choreography of the capability units by asynchronous message communication while a resource is determined by application-level service discovery. The framework is evaluated through a human experiment. The results show that the application construction and execution by the framework is feasible. The modeling tool is usable and the execution architecture is scalable in different environment settings.

AI applications often use ML/DL (Machine Learning/Deep Learning) models to implement specific AI tasks. As application developers usually are not AI experts, they often choose to integrate existing implementations of ML/DL models as libraries for their AI tasks. As an active research area, AI attracts many researchers and produces a lot of papers every year. Many of the papers propose ML/DL models for specific tasks and provide their implementations. However, it is not easy for developers to find ML/DL libraries that are suitable for their tasks. The challenges lie in not only the fast development of AI application domains and techniques, but also the lack of detailed information of the libraries such as environmental dependencies and supporting resources. In this paper, we conduct an empirical study on ML/DL library seeking questions on Stack Overflow to understand the developers' requirements for ML/DL libraries. Based on the findings of the study, we propose a task-oriented ML/DL library recommendation approach, called MLTaskKG. It constructs a knowledge graph that captures AI tasks, ML/DL models, model implementations, repositories, and their relationships by extracting knowledge from different sources such as ML/DL resource websites, papers, ML/DL frameworks, and repositories. Based on the knowledge graph, MLTaskKG recommends ML/DL libraries for developers by matching their requirements on tasks, model characteristics, and implementation information. Our evaluation shows that 92.8% of the tuples sampled from the resulting knowledge graph are correct, demonstrating the high quality of the knowledge graph. A further experiment shows that MLTaskKG can help developers find suitable ML/DL libraries using 47.6% shorter time and with 68.4% higher satisfaction.

When reusing or modifying code, developers need to understand the implicit knowledge behind a piece of code in addition to the literal meaning of code. Such implicit knowledge involves related concepts and their explanations. Uncovering and understanding the implicit knowledge in code are challenging due to the extensive use of abbreviations, scattered expressions of concepts, and ambiguity of concept mentions. In this paper, we propose an automatic approach (called CoLiCo) that can uncover implicit concepts in code and link the uncovered concepts to Wikipedia. Based on a trained identifier embedding model, CoLiCo identifies Wikipedia concepts mentioned in a given code snippet and excerpts a paragraph-level explanation from Wikipedia for each concept. During the process, CoLiCo resolves identifier abbreviation (i.e., concepts mentioned in the form of abbreviations) and identifier aggregation (i.e., concepts mentioned by an aggregation of multiple identifiers) based on identifier embedding and mining of identifier abbreviation/aggregation relations. Experimental study shows that CoLiCo outperforms a general entity linking approach by 38.7% in the correctness of concept linking and identifies 96.7% more correct concept linkings on a dataset with 629 code snippets. The concept linking is significant for program understanding in 54% code snippets. Our user study shows that CoLiCo can significantly shorten the time and improve the correctness in code comprehension tasks that intensively involve implicit knowledge.

Third-party libraries with rich functionalities facilitate the fast development of JavaScript software, leading to the explosive growth of the NPM ecosystem. However, it also brings new security threats that vulnerabilities could be introduced through dependencies from third-party libraries. In particular, the threats could be excessively amplified by transitive dependencies. Existing research only considers direct dependencies or reasoning transitive dependencies based on reachability analysis, which neglects the NPM-specific dependency resolution rules as adapted during real installation, resulting in wrongly resolved dependencies. Consequently, further fine-grained analysis, such as precise vulnerability propagation and their evolution over time in dependencies, cannot be carried out precisely at a large scale, as well as deriving ecosystem-wide solutions for vulnerabilities in dependencies.

Test failures are one of the most common reasons for broken builds in continuous integration. It is expensive to diagnose all test failures in a build. As test failures are usually caused by a few underlying faults, triaging test failures with respect to their underlying root causes can save test failure diagnosis cost. Existing failure triage methods are mostly developed for triaging crash or bug reports, and hence not applicable in the context of test failure triage in continuous integration. In this paper, we first present a large-scale empirical study on 163,371 broken builds caused by test failures to characterize test failures in real-world Java projects. Then, motivated by our study, we propose a new change-aware approach, BuildSheriff, to triage test failures in each continuous integration build such that test failures with the same root cause are put in the same cluster. Our evaluation on 200 broken builds has demonstrated that BuildSheriff can significantly improve the state-of-the-art methods on the triaging effectiveness.

A microservice system in industry is usually a large-scale distributed system consisting of dozens to thousands of services running in different machines. An anomaly of the system often can be reflected in traces and logs, which record inter-service interactions and intra-service behaviors respectively. Existing trace anomaly detection approaches treat a trace as a sequence of service invocations. They ignore the complex structure of a trace brought by its invocation hierarchy and parallel/asynchronous invocations. On the other hand, existing log anomaly detection approaches treat a log as a sequence of events and cannot handle microservice logs that are distributed in a large number of services with complex interactions. In this paper, we propose DeepTraLog, a deep learning based microservice anomaly detection approach. DeepTraLog uses a unified graph representation to describe the complex structure of a trace together with log events embedded in the structure. Based on the graph representation, DeepTraLog trains a GGNNs based deep SVDD model by combing traces and logs and detects anomalies in new traces and the corresponding logs. Evaluation on a microservice benchmark shows that DeepTraLog achieves a high precision (0.93) and recall (0.97), outperforming state-of-the-art trace/log anomaly detection approaches with an average increase of 0.37 in F1-score. It also validates the efficiency of DeepTraLog, the contribution of the unified graph representation, and the impact of the configurations of some key parameters.

Distributed tracing has been an important part of microservice infrastructure and learning-based trace analysis has been used to detect anomalies in microservice systems. Existing learning-based trace anomaly detection approaches ei-ther assume that trace patterns can be learned from normal execution or rely on fault injection to produce labeled traces (i.e., normal/anomalous ones). However, in practice it is often difficult to ensure that the normal execution does not involve anomalous traces or obtain a large variety of normal and anomalous traces through fault injection. In this paper, we propose PUTraceAD, a trace anomaly detection approach that can alleviate the above problems. PUTraceAD represents a trace as a span causal graph with node features such as operation name, response code, duration time. Based on the graph representation, PUTraceAD trains a GNN- and PU learning-based trace anomaly detection model. During the process, PU (Positive and Unlabeled) learning optimizes model parameters through estimating the data distribution. Therefore, PUTraceAD can train the model based on a small set of labeled anomalous traces and a large set of unlabeled traces. Our evaluation shows that PUTraceAD outperforms existing unsupervised trace anomaly detection approaches and only slightly underperforms a supervised learning-based approach that takes full advantage of labeled traces.

Android apps are event-driven, and their execution is often interrupted by external events. This interruption can cause data loss issues that annoy users. For instance, when the screen is rotated, the current app page will be destroyed and recreated. If the app state is improperly preserved, user data will be lost. In this work, we present an approach and tool iFixDataloss that automatically detects and fixes data loss issues in Android apps. To achieve this, we identify scenarios in which data loss issues may occur, develop strategies to reveal data loss issues, and design patch templates to fix them. Our experiments on 66 Android apps show iFixDataloss detected 374 data loss issues (284 of them were previously unknown) and successfully generated patches for 188 of the 374 issues. Out of 20 submitted patches, 16 have been accepted by developers. In comparison with state-of-the-art techniques, iFixDataloss performed significantly better in terms of the number of detected data loss issues and the quality of generated patches.

Android apps are event-driven, and their execution is often interrupted by external events. This interruption can cause data loss issues that annoy users. For instance, when the screen is rotated, the current app page will be destroyed and recreated. If the app state is improperly preserved, user data will be lost. In this work, we present a tool iFixDataloss that automatically detects and fixes data loss issues in Android apps. To achieve this, we identify scenarios in which data loss issues may occur by analyzing the Android life cycle, developing strategies to reveal data loss issues, and designing patch templates to fix them. Our experiments on 66 Android apps show iFixDataloss detected 374 data loss issues (284 of them were previously unknown) and successfully generated patches for 188 of the 374 issues. Out of 20 submitted patches, 16 have been accepted by developers. In comparison with state-of-the-art techniques, iFixDataloss performed significantly better in terms of the number of detected data loss issues and the quality of generated patches. Video Link: https://www.youtube.com/watch?v=MAPsCo-dRKs Github Link: https://github.com/iFixDataLoss/iFixDataloss22

Bug datasets lay significant empirical and experimental foundation for various SE/PL researches such as fault localization, software testing, and program repair. Current well-known datasets are constructed manually, which inevitably limits their scalability, representativeness, and the support for the emerging data-driven research.

Code clones widely exist in open-source and industrial software projects and are still recognized as a threat to software maintenance due to the additional effort required for the simultaneous maintenance of multiple clone instances and potential defects caused by inconsistent changes in clone instances. To alleviate the threat, it is essential to accurately and efficiently make the decisions of change propagation between clone instances. Based on an exploratory study on clone change propagation with five famous open-source projects, we find that a clone class can have both propagation-required changes and propagation-free changes and thus fine-grained change propagation decision is required. Based on the findings, we propose a graph-based deep learning approach to predict the change propagation requirements of clone instances. We develop a graph representation, named Fused Clone Program Dependency Graph (FC-PDG), to capture the textual and structural code contexts of a pair of clone instances along with the changes on one of them. Based on the representation, we design a deep learning model that uses a Relational Graph Convolutional Network (R-GCN) to predict the change propagation requirement. We evaluate the approach with a dataset constructed based on 51 open-source Java projects, which includes 24,672 pairs of matched changes and 38,041 non-matched changes. The results show that the approach achieves high precision (83.1%), recall (81.2%), and F1-score (82.1%). Our further evaluation with three other open-source projects confirms the generality of the trained clone change propagation prediction model.

Despite the benefits, continuous integration (CI) can incur high costs. One of the well-recognized costs is long build time, which greatly affects the speed of software development and increases the cost in computational resources. While there exist configuration options in the CI infrastructure to accelerate builds, the CI infrastructure is often not optimally configured, leading to CI configuration smells. Attempts have been made to detect or repair CI configuration smells. However, none of them is specifically designed to improve build performance in CI.

Microservice architecture design highly relies on expert experience and may often result in improper service decomposition. Moreover, a microservice architecture is likely to degrade with the continuous evolution of services. Architecture measurement is thus important for the long-term evolution of microservice architectures. Due to the independent and dynamic nature of services, source code analysis based approaches cannot well capture the interactions between services. In this paper, we propose a trace analysis based microservice architecture measurement approach. We define a trace data model for microservice architecture measurement, which enables fine-grained analysis of the execution processes of requests and the interactions between interfaces and services. Based on the data model, we define 14 architectural metrics to measure the service independence and invocation chain complexity of a microservice system. We implement the approach and conduct three case studies with a student course project, an open-source microservice benchmark system, and three industrial microservice systems. The results show that our approach can well characterize the independence and invocation chain complexity of microservice architectures and help developers to identify microservice architecture issues caused by improper service decomposition and architecture degradation.

Deep learning (DL) has been widely applied to many domains. Unique challenges in engineering DL systems are posed by the programming paradigm shift from traditional systems to DL systems, and performance is one of the challenges. Performance problems (PPs) in DL systems can cause severe consequences such as excessive resource consumption and financial loss. While bugs in DL systems have been extensively investigated, PPs in DL systems have hardly been explored. To bridge this gap, we present the first comprehensive study to i) characterize symptoms, root causes, and introducing and exposing stages of PPs in DL systems developed in TensorFLow and Keras, with 224 PPs collected from 210 StackOverflow posts, and to ii) assess the capability of existing performance analysis approaches in tackling PPs, with a constructed benchmark of 58 PPs in DL systems. Our findings shed light on the implications on developing high-performance DL systems, and detecting and localizing PPs in DL systems. To demonstrate the usefulness of our findings, we develop a static checker DeepPerf to detect three types of PPs. It has detected 488 new PPs in 130 GitHub projects. 105 and 27 PPs have been confirmed and fixed.

Developers often ask how-to questions using search engines, technical Q&A communities, and interactive Q&A systems to seek help for specific programming tasks. However, they often do not formulate the questions in a specific way, making it hard for the systems to return the best answers. We propose an approach (TaskKG4Q) that interactively helps developers formulate a programming related how-to question. TaskKG4Q is using a programming task knowledge graph (task KG in short) mined from Stack Overflow questions, which provides a hierarchical conceptual structure for tasks in terms of [actions], [objects], and [constraints]. An empirical evaluation of the intrinsic quality of the task KG revealed that 75.0% of the annotated questions in the task KG are correct. The comparison between TaskKG4Q and two baselines revealed that TaskKG4Q can help developers formulate more specific how-to questions. More so, an empirical study with novice programmers revealed that they write more effective questions for finding answers to their programming tasks on Stack Overflow.

In this work, we introduce a tool, RegMiner, to automate the process of collecting replicable regression bugs from a set of Git repositories. In the code commit history, RegMiner searches for regressions where a test can pass a regression-fixing commit, fail a regressioninducing commit, and pass a previous working commit again. Technically, RegMiner (1) identifies potential regression-fixing commits from the code evolution history, (2) migrates the test and its code dependencies in the commit over the history, and (3) minimizes the compilation overhead during the regression search. Our experients show that RegMiner can successfully collect 1035 regressions over 147 projects in 8 weeks, creating the largest replicable regression dataset within the shortest period, to the best of our knowledge. In addition, our experiments further show that (1) RegMiner can construct the regression dataset with very high precision and acceptable recall, and (2) the constructed regression dataset is of high authenticity and diversity. The source code of RegMiner is available at https://github.com/SongXueZhi/RegMiner, the mined regression dataset is available at https://regminer.github.io/, and the demonstration video is available at https://youtu.be/yzcM9Y4unok.

Open source software (OSS) vulnerabilities threaten the security of software systems that use OSS. Vulnerability databases provide valuable information (e.g., vulnerable version and patch) to mitigate OSS vulnerabilities. There arises a growing concern about the information quality of vulnerability databases. However, it is unclear what the quality of patches in existing vulnerability databases is; and existing manual or heuristic-based approaches for patch tracking are either too expensive or too specific to apply to all OSS vulnerabilities.

Due to the large amount and high complexity of trace data, microservice trace analysis tasks such as anomaly detection, fault diagnosis, and tail-based sampling widely adopt machine learning technology. These trace analysis approaches usually use a preprocessing step to map structured features of traces to vector representations in an ad-hoc way. Therefore, they may lose important information such as topological dependencies between service operations. In this paper, we propose TraceCRL, a trace representation learning approach based on contrastive learning and graph neural network, which can incorporate graph structured information in the downstream trace analysis tasks. Given a trace, TraceCRL constructs an operation invocation graph where nodes represent service operations and edges represent operation invocations together with predefined features for invocation status and related metrics. Based on the operation invocation graphs of traces TraceCRL uses a contrastive learning method to train a graph neural network-based model for trace representation. In particular, TraceCRL employs six trace data augmentation strategies to alleviate the problems of class collision and uniformity of representation in contrastive learning. Our experimental studies show that TraceCRL can significantly improve the performance of trace anomaly detection and offline trace sampling. It also confirms the effectiveness of the trace augmentation strategies and the efficiency of TraceCRL.

Uncaught runtime exceptions have been recognized as one of the commonest root causes of real-life exception bugs in Java applications. However, existing runtime exception detection techniques rely on symbolic execution or random testing, which may suffer the scalability or coverage problem. Rule-based bug detectors (e.g., SpotBugs) provide limited rule support for runtime exceptions. Inspired by the recent successes in applying deep learning to bug detection, we propose a deep learning-based technique, named Drex, to identify not only the types of runtime exceptions that a method might signal but also the statement scopes that might signal the detected runtime exceptions. It is realized by graph-based code representation learning with (i) a lightweight analysis to construct a joint graph of CFG, DFG and AST for each method without requiring a build environment so as to comprehensively characterize statement syntax and semantics and (ii) an attention-based graph neural network to learn statement embeddings in order to distinguish different types of potentially signaled runtime exceptions with interpretability. Our evaluation on 54,255 methods with caught runtime exceptions and 54,255 methods without caught runtime exceptions from 5,996 GitHub Java projects has indicated that Drex improves baseline approaches by up to 18.2% in exact accuracy and 41.6% in F1-score. Drex detects 20 new uncaught runtime exceptions in 13 real-life pro-jects, 7 of them have been fixed, while none of them is detected by rule-based bug detectors (i.e., SpotBugs and PMD).

Annotations have been widely used in Java programs to support additional compile-time, deployment-time, and runtime processing. Developers use annotations to delegate repetitive logics such as object initialization and request forwarding to compilers and runtime frameworks. Therefore, these annotations are important for the correct execution of programs. In practice, however, developers often find it hard to correctly use annotations and the misuse of annotations has led to real bugs in Java programs. In this paper, we conduct an empirical study on Stack Overflow questions to investigate the major development frameworks that are involved in questions about Java annotations and the main problems encountered by developers in the use of Java annotations. Based on the findings of the study, we propose DeepAnna, a deep learning based Java annotation recommendation and misuse detection approach. Based on a corpus of Java programs with intensive use of annotations, DeepAnna trains a deep learning based multi-label classification model by considering both the structural and textual contexts of source code. DeepAnna can recommend annotations at both class level and method level. Our evaluation with a large corpus of open-source Java projects shows that DeepAnna outperforms state-of-the-art text multi-label classification approaches in annotation recommendation and can effectively detect annotation misuses. Based on our analysis, we submit 85 bug-fixing pull requests for annotation misuses in open-source projects and 20 of them have been accepted and merged.

Microservice systems are complicated due to their runtime environments and service communications. Debugging a failure involves the deployment and manipulation of microservice systems on a containerized environment and faces unique challenges due to the high complexity and dynamism of microservices. To address these challenges, we propose a debugging approach for microservice systems based on the delta debugging algorithm, which is to minimalize failure-inducing deltas of circumstances (e.g., deployment, environmental configurations). Our approach includes novel techniques for defining, deploying/manipulating, and executing deltas during delta debugging. In particular, to construct a (failing) circumstance space for delta debugging to minimalize, our approach defines a set of circumstance dimensions that can affect the execution of microservice systems. To automate the testing of deltas, our approach includes the design of an infrastructure layer for automating deployment and manipulation of microservice systems. To optimize the delta debugging process, our approach includes the design of parallel execution for delta testing tasks. Our evaluation shows that our approach is scalable and efficient with the provided infrastructure resources and the designed parallel execution for optimization. Our experimental study on a medium-size microservice benchmark system shows that our approach can effectively identify failure-inducing deltas that help diagnose the root causes.

Context based API recommendation is an important way to help developers find the needed APIs effectively and efficiently. For effective API recommendation, we need not only a joint view of both structural and textual code information, but also a holistic view of correlated API usage in control and data flow graph as a whole. Unfortunately, existing API recommendation methods exploit structural or textual code information separately. In this work, we propose a novel API recommendation approach called APIRec-CST (API Recommendation by Combining Structural and Textual code information). APIRec-CST is a deep learning model that combines the API usage with the text information in the source code based on an API Context Graph Network and a Code Token Network that simultaneously learn structural and textual features for API recommendation. We apply APIRec-CST to train a model for JDK library based on 1,914 open-source Java projects and evaluate the accuracy and MRR (Mean Reciprocal Rank) of API recommendation with another 6 open-source projects. The results show that our approach achieves respectively a top-1, top-5, top-10 accuracy and MRR of 60.3, 81.5, 87.7 and 69.4 percent, and significantly outperforms an existing graph-based statistical approach and a tree-based deep learning approach for API recommendation. A further analysis shows that textual code information makes sense and improves the accuracy and MRR. The sensitivity analysis shows that the top-k accuracy and MRR of APIRec-CST are insensitive to the number of APIs to be recommended in a hole. We also conduct a user study in which two groups of students are asked to finish 6 programming tasks with or without our APIRec-CST plugin. The results show that APIRec-CST can help the students to finish the tasks faster and more accurately and the feedback on the usability is overwhelmingly positive.

Stack Overflow (SO) provides informal documentation for APIs in response to questions that express API related developer needs. Navigating the information available on SO and getting information related to a particular API and need is challenging due to the vast amount of questions and answers and the tag-driven structure of SO. In this paper we focus on identifying and classifying fine-grained developer needs expressed in sentences of API-related SO questions, as well as the specific information types used to express such needs, and the different roles APIs play in these questions and their answers. We derive a taxonomy, complementing existing ones, through an empirical study of 266 SO posts. We then develop and evaluate an approach for the automated identification of the fine-grained developer needs in SO threads, which takes a thread as input and outputs the corresponding developer needs, the types of information expressing them, and the roles of API elements relevant to the needs. To show a practical application of our taxonomy, we introduce and evaluate an approach for the automated retrieval of SO questions, based on these developer needs.

Programming tutorials demonstrate programming tasks with code examples. However, our study of Stack Overflow questions reveals the low utilization of high-quality programming tutorials, which is caused task description mismatch and code information overload. Neither document search nor recently proposed activity-centric search can address these two barriers. In this work, we enrich the programming task knowledge graph with actions extracted from comments in code examples and more forms of activity sentences. To overcome the task description mismatch problem, we use code matching based task search method to find relevant programming tasks and code examples to the code under development. We integrate our knowledge graph and task search method in the IDE, and develop an observe-push based tool to prompt developers with API usage examples in explicit task contexts. To alleviate the code information overload problem, our tool highlights programming task and API information in the prompted tutorial excerpts and code examples based on the underlying knowledge graph. Our evaluation confirms the high quality of the constructed knowledge graph, and show that our code matching based task search can recommend effective code solutions to programming issues asked on Stack Overflow. Through an user study, we demonstrate that our tool is useful for assisting developers in finding and using relevant programming tutorials in their programming tasks.

User-facing services are now evolving towards the microservice architecture where a service is built by connecting multiple microservice stages. While an entire service is heavy, the microservice architecture shows the opportunity to only offload some microservice stages to the edge devices that are close to the end users. However, emerging techniques often result in the violation of Quality-of-Service (QoS) of microservice-based services in cloud-edge continuum, as they do not consider the communication overhead or the resource contention between microservices.We propose Nautilus, a runtime system that effectively deploys microservice-based user-facing services in cloud-edge continuum. It ensures the QoS of microservice-based user-facing services while minimizing the required computational resources. Nautilus is comprised of a communication-aware microservice mapper, a contention-aware resource manager and a load-aware microservice scheduler. The mapper divides the microservice graph into multiple partitions based on the communication overhead and maps the partitions to the nodes. On each node, the resource manager determines the optimal resource allocation for its microservices based on reinforcement learning that may capture the complex contention behaviors. The microservice scheduler monitors the QoS of the entire service, and migrates microservices from busy nodes to idle ones at runtime. Our experimental results show that Nautilus reduces the computational resource usage by 23.9% and the network bandwidth usage by 53.4%, while achieving the required 99%-ile latency.

Libraries are widely adopted in developing software projects. Library APIs are often missing during library evolution as library developers may deprecate, remove or refactor APIs. As a result, client developers have to manually find replacement APIs for missing APIs when updating library versions in their projects, which is a difficult and expensive software maintenance task. One of the key limitations of the existing automated approaches is that they usually consider the library itself as the single source to find replacement APIs, which heavily limits their accuracy.In this paper, we first present an empirical study to understand characteristics about missing APIs and their replacements. Specifically, we quantify the prevalence of missing APIs, and summarize the knowledge sources where the replacements are found, and the code change and mapping cardinality between missing APIs and their replacements. Then, inspired by the insights from our study, we propose a heuristic-based approach, REPFINDER, to automatically find replacements for missing APIs in library update. We design and combine a set of heuristics to hierarchically search three sources (deprecation message, own library, and external library) for finding replacements. Our evaluation has demonstrated that REPFINDER can find replacement APIs effectively and efficiently, and significantly outperform the state-of-the-art approaches.

Assigning bugs to the right components is the prerequisite to get the bugs analyzed and fixed. Classification-based techniques have been used in practice for assisting bug component assignments, for example, the BugBug tool developed by Mozilla. However, our study on 124,477 bugs in Mozilla products reveals that erroneous bug component assignments occur frequently and widely. Most errors are repeated errors and some errors are even misled by the BugBug tool. Our study reveals that complex component designs and misleading component names and bug report keywords confuse bug component assignment not only for bug reporters but also developers and even bug triaging tools. In this work, we propose a learning to rank framework that learns to assign components to bugs from correct, erroneous and irrelevant bug-component assignments in the history. To inform the learning, we construct a bug tossing knowledge graph which incorporates not only goal-oriented component tossing relationships but also rich information about component tossing community, component descriptions, and historical closed and tossed bugs, from which three categories and seven types of features for bug, component and bug-component relation can be derived. We evaluate our approach on a dataset of 98,587 closed bugs (including 29,100 tossed bugs) of 186 components in six Mozilla products. Our results show that our approach significantly improves bug component assignments for both tossed and non-tossed bugs over the BugBug tool and the BugBug tool enhanced with component tossing relationships, with >20% Top-k accuracies and >30% NDCG@k (k=1,3,5,10).

Developers often rely on API documentation to learn API directives, i.e., constraints and guidelines related to API usage. Failing to follow API directives may cause defects or improper implementations. Since there are no industry-wide standards on how to document API directives, they take many forms and are often hard to understand by developers or challenging to parse with tools.

Code clones are identical or similar code in software projects. On one hand, developers clone code to achieve higher productivity and thus clones inherently exist; on the other hand, code clones demand extra effort to maintain the consistency between clone instances and may introduce bugs, and thus are often considered harmful for software maintenance and quality. We believe that not all code clones have the same level of harmfulness. A systematic way of assessing the harmfulness level of cloned code would facilitate informed decisions on how to deal with clones. We propose a model for clone harmfulness level assessment with four quantitative indicators that can be extracted from the evolution history of the clones. Specifically, we gather information, such as code clone changes and bug- fixes related to clone divergence and re-synchronization, to find objective evidence that a clone harms the software quality or brings potential risks even if no bugs are found. The assessment model consists of four harmfulness levels of clones determined by the four indicators. We also derive three harmfulness factors from the intrinsic properties of clones that potentially affect the harmfulness of clones. We conduct a large-scale empirical study with five open-source and three industry systems and find that 61.0-84.7% of the clones are not harmful in terms of consistent maintenance overhead. We find evidence in the evolution history that several factors, such as spread of clone instances, number of clone instances, and number of developers, have non-trivial correlation with clone harmfulness levels. We also propose six counter measures for clone harmfulness mitigation based on the observation of the harmfulness factors, and have collected useful feedback from industrial software architects and senior developers through an interview meeting.

Sentiment and emotion detection from textual communication records of developers have various application scenarios in software engineering (SE). However, commonly used off-the-shelf sentiment/emotion detection tools cannot obtain reliable results in SE tasks and misunderstanding of technical knowledge is demonstrated to be the main reason. Then researchers start to create labeled SE-related datasets manually and customize SE-specific methods. However, the scarce labeled data can cover only very limited lexicon and expressions. In this article, we employ emojis as an instrument to address this problem. Different from manual labels that are provided by annotators, emojis are self-reported labels provided by the authors themselves to intentionally convey affective states and thus are suitable indications of sentiment and emotion in texts. Since emojis have been widely adopted in online communication, a large amount of emoji-labeled texts can be easily accessed to help tackle the scarcity of the manually labeled data. Specifically, we leverage Tweets and GitHub posts containing emojis to learn representations of SE-related texts through emoji prediction. By predicting emojis containing in each text, texts that tend to surround the same emoji are represented with similar vectors, which transfers the sentiment knowledge contained in emoji usage to the representations of texts. Then we leverage the sentiment-aware representations as well as manually labeled data to learn the final sentiment/emotion classifier via transfer learning. Compared to existing approaches, our approach can achieve significant improvement on representative benchmark datasets, with an average increase of 0.036 and 0.049 in macro-F1 in sentiment and emotion detection, respectively. Further investigations reveal that the large-scale Tweets make a key contribution to the power of our approach. This finding informs future research not to unilaterally pursue the domain-specific resource but try to transform knowledge from the open domain through ubiquitous signals such as emojis. Finally, we present the open challenges of sentiment and emotion detection in SE through a qualitative analysis of texts misclassified by our approach.

The complexity and dynamism of microservice systems pose unique challenges to a variety of software engineering tasks such as fault analysis and debugging. In spite of the prevalence and importance of microservices in industry, there is limited research on the fault analysis and debugging of microservice systems. To fill this gap, we conduct an industrial survey to learn typical faults of microservice systems, current practice of debugging, and the challenges faced by developers in practice. We then develop a medium-size benchmark microservice system (being the largest and most complex open source microservice system within our knowledge) and replicate 22 industrial fault cases on it. Based on the benchmark system and the replicated fault cases, we conduct an empirical study to investigate the effectiveness of existing industrial debugging practices and whether they can be further improved by introducing the state-of-the-art tracing and visualization techniques for distributed systems. The results show that the current industrial practices of microservice debugging can be improved by employing proper tracing and visualization techniques and strategies. Our findings also suggest that there is a strong need for more intelligent trace analysis and visualization, e.g., by combining trace visualization and improved fault localization, and employing data-driven and learning-based recommendation for guided visual exploration and comparison of traces.

Exploring GUIs of Android apps plays a key role in many important scenarios such as functional testing (e.g., finding crash errors), security analysis (e.g., identifying malicious behav-iors) and competitive analysis (e.g., storyboarding app features). To automate GUI exploration, existing techniques often try to visit as many GUI pages as possible via specific strategies, e.g., random (like Monkey) or heuristic (like Stoat, A3E). However, their effectiveness is still unclear and much under-explored. To this end, we conducted the first study in this paper to understand and characterize their limitations by carefully analyzing the coverage reports from a set of real-world, open-source apps. Through this study, we identified three key limitations due to the lack of dependency knowledge during exploration, i.e., widget-page dependency, widget-widget dependency and system-event dependency. To overcome them, we introduce dependency-informed exploration, an automated approach that leverages static dependency analysis to effectively improve GUI exploration performance. Given an app, our approach first constructs a GUI page transition model that captures the dependencies between GUI widgets, and then guides GUI exploration during a depth-first traversal. We realized our approach as a tool named Gesda, and evaluated it on 70 open-source Android apps. The results show Gesda outperforms existing state-of-the-art GUI exploration techniques, i.e., Monkey and Stoat. Additionally, Gesda uncovers 4 previously unknown crashes in 4 apps as a by-product of GUI exploration due to the benefit of dependency knowledge, while Monkey and Stoat have not discovered them.

In this paper, we present OpenAPIDocGen2, a tool that generates on-demand class documentation based on source code and documentation analysis. For a given class, OpenAPIDocGen2 generates a combined documentation for it, which includes functionality descriptions, directives, domain concepts, usage examples, class/method roles, key methods, relevant classes/methods, characteristics and concepts classification, and usage scenarios.

This report introduces the approach that we have designed and implemented for the DeClutter challenge of Doc-Gen2, which detects non-informative code comments. The approach combines both comment based text classification and code context based prediction. Based on the approach, our "fduse" team achieved the best F1 score (0.847) in the competition.

Third-party libraries play a key role in software development as they can relieve developers of the heavy burden of re-implementing common functionalities. However, third-party libraries and client projects evolve asynchronously. As a result, out-dated third-party libraries might be used in client projects while developers are not aware of the potential risk (e.g., security bug). Outdated third-party libraries may be updated in client projects in a delayed way, and developers may be less aware of the potential risk (e.g., API incompatibility) in updates. Developers of third-party libraries may be unaware of how their third-party libraries are used or updated in client projects. Therefore, a quantitative and holistic study on usages, updates and risks of third-party libraries in open-source projects can provide concrete evidences on these problems, and practical insights to improve the ecosystem. In this paper, we contribute such a study in Java ecosystem. In particular, we conduct a library usage analysis (e.g., usage intensity and outdatedness) and library update analysis (e.g., update intensity and delay) on 806 open-source projects and 13,565 third- party libraries. Then, we carry out a library risk analysis (e.g., usage risk and update risk) on 806 open-source projects and 544 security bugs. These analyses aim to quantify the usage and update practices and the potential risk of using and updating outdated third-party libraries with respect to security bugs from two holistic perspectives (i.e., open-source projects and third-party libraries). Our findings suggest practical implications to developers and researchers on problems and potential solutions in maintaining third-party libraries (e.g., smart alerting and automated updating of outdated third-party libraries). To indicate the usefulness of our findings, we design a smart alerting system for assisting developers to make confident decisions when updating third-party libraries. 33 and 24 open-source projects have confirmed and updated third-party libraries after receiving our alerts.

Humans are social creatures which enjoy participating in group activities. Existing platforms such as event-based social networks and social-matching applications empower people to organize and participate in different kinds of interest-based activities. However, credibility issues are inevitable since the participants’ commitment to participate in activities on time can hardly be guaranteed. As a result, many activities are canceled due to lack of participation, which impairs people’s will for attending activities and increases the difficulty of coalescing activity groups.

Long build times in continuous integration (CI) can greatly increase the cost in human and computing resources, and thus become a common barrier faced by software organizations adopting CI. Build outcome prediction has been proposed as one of the remedies to reduce such cost. However, the state-of-the-art approaches have a poor prediction performance for failed builds, and are not designed for practical usage scenarios. To address the problems, we first conduct an empirical study on 2,590,917 builds to characterize build times in real-world projects, and a survey with 75 developers to understand their perceptions about build outcome prediction. Then, motivated by our study and survey results, we propose a new history-aware approach, named BuildFast, to predict CI build outcomes cost-efficiently and practically. We develop multiple failure-specific features from closely related historical builds via analyzing build logs and changed files, and propose an adaptive prediction model to switch between two models based on the build outcome of the previous build. We investigate a practical online usage scenario of BuildFast, where builds are predicted in chronological order, and measure the benefit from correct predictions and the cost from incorrect predictions. Our experiments on 20 projects have shown that BuildFast improved the state-of-the-art by 47.5% in F1-score for failed builds.

Developers are concerned with the comparison of similar APIs in terms of their commonalities and (often subtle) differences. Our empirical study of Stack Overflow questions and API documentation confirms that API comparison questions are common and can often be answered by knowledge contained in API reference documentation. Our study also identifies eight types of API statements that are useful for API comparison. Based on these findings, we propose a knowledge graph based approach APIComp that automatically extracts API knowledge from API reference documentation to support the comparison of a pair of API classes or methods from different aspects. Our approach includes an offline phase for constructing an API knowledge graph, and an online phase for generating an API comparison result for a given pair of API elements. Our evaluation shows that the quality of different kinds of extracted knowledge in the API knowledge graph is generally high. Furthermore, the comparison results generated by APIComp are significantly better than those generated by a baseline approach based on heuristic rules and text similarity, and our generated API comparison results are useful for helping developers in API selection tasks.

Microservice systems are highly dynamic and complex. For such systems, operation engineers and developers highly rely on trace analysis to understand architectures and diagnose various problems such as service failures and quality degradation. However, the huge number of traces produced at runtime makes it challenging to capture the required information in real-time. To address the faced challenges, in this paper, we propose a graph-based microservice trace analysis approach GMTA for understanding architecture and diagnosing various problems. Built on a graph-based representation, GMTA includes efficient processing of traces produced on the fly. It abstracts traces into different paths and further groups them into business flows. To support various analytical applications, GMTA includes an efficient storage and access mechanism by combining a graph database and a real-time analytics database and using a carefully designed storage structure. Based on GMTA, we construct analytical applications for architecture understanding and problem diagnosis, these applications support various needs such as visualizing service dependencies, making architectural decisions, analyzing the changes of services behaviors, detecting performance issues, and locating root causes. GMTA has been implemented and deployed in eBay. An experimental study based on trace data produced by eBay demonstrates GMTA's effectiveness and efficiency for architecture understanding and problem diagnosis. Case studies conducted in eBay's monitoring team and Site Reliability Engineering (SRE) team further confirm GMTA's substantial benefits in industrial-scale microservice systems.

As a mixed result of intensive dependency on third-party libraries, flexible mechanisms to declare dependencies and increased number of modules in a project, different modules of a project directly depend on multiple versions of the same third-party library. Such library version inconsistencies could increase dependency maintenance cost, or even lead to dependency conflicts when modules are inter-dependent. Although automated build tools (e.g., Maven's enforcer plugin) provide partial support to detect library version inconsistencies, they do not provide any support to harmonize inconsistent library versions.

Due to the lexical gap between functionality descriptions and user queries, documentation-based API retrieval often produces poor results.Verb phrases and their phrase patterns are essential in both describing API functionalities and interpreting user queries. Thus we hypothesize that API retrieval can be facilitated by explicitly recognizing and matching between the fine-grained structures of functionality descriptions and user queries. To verify this hypothesis, we conducted a large-scale empirical study on the functionality descriptions of 14,733 JDK and Android API methods. We identified 356 different functionality verbs from the descriptions, which were grouped into 87 functionality categories, and we extracted 523 phrase patterns from the verb phrases of the descriptions. Building on these findings, we propose an API method recommendation approach based on explicit matching of functionality verb phrases in functionality descriptions and user queries, called PreMA. Our evaluation shows that PreMA can accurately recognize the functionality categories (92.8%) and phrase patterns (90.4%) of functionality description sentences; and when used for API retrieval tasks, PreMA can help participants complete their tasks more accurately and with fewer retries compared to a baseline approach.

Clone detection on large code repository is necessary for many big code analysis tasks. The goal is to provide rich information on identical and similar code across projects. Detecting near-miss code clones on big code is challenging since it requires intensive computing and memory resources as the scale of the source code increases. In this work, we propose SAGA, an efficient suffix-array based code clone detection tool designed with sophisticated GPU optimization. SAGA not only detects Type-l and Type-2 clones but also does so for cross-project large repositories and for the most computationally expensive Type-3 clones. Meanwhile, it also works at segment granularity, which is even more challenging. It detects code clones in 100 million lines of code within 11 minutes (with recall and precision comparable to state-of-the-art approaches), which is more than 10 times faster than state-of-the-art tools. It is the only tool that efficiently detects Type-3 near-miss clones at segment granularity in large code repository (e.g., within 11 hours on 1 billion lines of code). We conduct a preliminary case study on 85,202 GitHub Java projects with 1 billion lines of code and exhibit the distribution of clones across projects. We find about 1.23 million Type-3 clone groups, containing 28 million lines of code at arbitrary segment granularity, which are only detectable with SAGA. We believe SAGA is useful in many software engineering applications such as code provenance analysis, code completion, change impact analysis, and many more.

Accomplishing a program task usually involves performing multiple activities in a logical order. Task-solving activities may have different relationships, such as subactivityof, precede-follow, and different attributes, such as location, condition, API, code. We refer to task-solving activities and their relationships and attributes as know-how knowledge. Programming task know-how knowledge is commonly documented in semi-structured textual tutorials. A formative study of the 20 top-viewed Android-tagged how-to questions on Stack Overflow suggests that developers are faced with three information barriers (incoherent modeling of task intent, tutorial information overload and unstructured task activity description) for effectively discovering and understanding task-solving knowledge in textual tutorials. Knowledge graph has been shown to be effective in representing relational knowledge and supporting knowledge search in a structured way. Unfortunately, existing knowledge graphs extract only know-what information (e.g., APIs, API caveats and API dependencies) from software documentation. In this paper, we devise open information extraction (OpenIE) techniques to extract candidates for task activities, activity attributes and activity relationships from programming task tutorials. The resulting knowledge graph, TaskKG, includes a hierarchical taxonomy of activities, three types of activities relationships and five types of activity attributes, and enables activity-centric knowledge search. As a proof-of-concept, we apply our approach to Android Developer Guide. A comprehensive evaluation of TaskKG shows high accuracy of our OpenIE techniques. A user study shows that TaskKG is promising in helping developers finding correct answers to programming how-to questions.

Frequent co-changes to multiple files, i.e., evolutionary coupling, can demonstrate active relations among files, explicit or implicit. Although evolutionary coupling has been used to analyze software quality, there is no systematic study on the categorization of frequent co-changes between files which may used for characterizing various quality problems. In this paper, we report an empirical study on 27,087 co-change commits of 6 open-source systems with the purpose of understanding the observed evolutionary coupling. We extracted fine-grained change information from version control system to investigate whether two files exhibit particular kinds of co-change relationships. We consider code changes on 5 types of program entities (i.e., field, method, control statement, non-control statement, and class) and identified 6 types of dominating co-change relationships. Our manual analysis showed that each of the 6 types can be explained by structural coupling, semantic coupling, or implicit dependencies. Temporal analysis further shows that files may exhibit different co-change relationships at different phases in the evolution history. Finally, we investigated co-changes among multiple files by combining co-change relationships between related file pairs and showed with live examples that rich information embedded in the fine-grained co-change relationships may help developers to change code at multiple locations. Moreover, we analyzed how these co-change relationship types can be used to facilitate change impact analysis and to pinpoint design problems.

Source code summaries are concise representations, in form of text and/or code, of complex code elements and are meant to help developers gain a quick understanding that in turns help them perform specific tasks. Generation of summaries that are task-specific is still a challenge in the automatic code summarization field. We propose an approach for generating on-demand, extrinsic hybrid summaries for API classes, relevant to a programming task, formulated as a natural language query. The summaries include the most relevant sentences extracted from the API reference documentation and the most relevant methods.

A domain glossary that organizes domain-specific concepts and their aliases and relations is essential for knowledge acquisition and software development. Existing approaches use linguistic heuristics or term-frequency-based statistics to identify domain specific terms from software documentation, and thus the accuracy is often low. In this paper, we propose a learning-based approach for automatic construction of domain glossary from source code and software documentation. The approach uses a set of high-quality seed terms identified from code identifiers and natural language concept definitions to train a domain-specific prediction model to recognize glossary terms based on the lexical and semantic context of the sentences mentioning domain-specific concepts. It then merges the aliases of the same concepts to their canonical names, selects a set of explanation sentences for each concept, and identifies "is a", "has a", and "related to" relations between the concepts. We apply our approach to deep learning domain and Hadoop domain and harvest 5,382 and 2,069 concepts together with 16,962 and 6,815 relations respectively. Our evaluation validates the accuracy of the extracted domain glossary and its usefulness for the fusion and acquisition of knowledge from different documents of different projects.

Continuous Integration (CI) is a widely-used software development practice to reduce risks. CI builds often break, and a large amount of efforts are put into troubleshooting broken builds. Despite that compiler errors have been recognized as one of the most frequent types of build failures, little is known about the common types, fix efforts and fix patterns of compiler errors that occur in CI builds of open-source projects. To fill such a gap, we present a large-scale empirical study on 6,854,271 CI builds from 3,799 open-source Java projects hosted on GitHub. Using the build data, we measured the frequency of broken builds caused by compiler errors, investigated the ten most common compiler error types, and reported their fix time. We manually analyzed 325 broken builds to summarize fix patterns of the ten most common compiler error types. Our findings help to characterize and understand compiler errors during CI and provide practical implications to developers, tool builders and researchers.

In the production environment, a large part of microservice failures are related to the complex and dynamic interactions and runtime environments, such as those related to multiple instances, environmental configurations, and asynchronous interactions of microservices. Due to the complexity and dynamism of these failures, it is often hard to reproduce and diagnose them in testing environments. It is desirable yet still challenging that these failures can be detected and the faults can be located at runtime of the production environment to allow developers to resolve them efficiently. To address this challenge, in this paper, we propose MEPFL, an approach of latent error prediction and fault localization for microservice applications by learning from system trace logs. Based on a set of features defined on the system trace logs, MEPFL trains prediction models at both the trace level and the microservice level using the system trace logs collected from automatic executions of the target application and its faulty versions produced by fault injection. The prediction models thus can be used in the production environment to predict latent errors, faulty microservices, and fault types for trace instances captured at runtime. We implement MEPFL based on the infrastructure systems of container orchestrator and service mesh, and conduct a series of experimental studies with two opensource microservice applications (one of them being the largest open-source microservice application to our best knowledge). The results indicate that MEPFL can achieve high accuracy in intraapplication prediction of latent errors, faulty microservices, and fault types, and outperforms a state-of-the-art approach of failure diagnosis for distributed systems. The results also show that MEPFL can effectively predict latent errors caused by real-world fault cases.

Software systems are increasingly required to autonomously adapt their architectural structures and/or behaviors to runtime environmental changes. However, existing architecture-based self-adaptation approaches mostly focus on structural adaptations within a predefined space of architectural alternatives (e.g., switching between two alternative services) while merely considering quality constraints (e.g., reliability and performance). In this paper, we propose a new architecture-based self-adaptation approach, which performs behavioral adaptations with automatically generated alternatives and supports relaxed functional constraints from the perspective of business value. Specifically, we propose a technique to automatically generate behavioral alternatives of a software system from the currently-employed architectural behavioral specification. We employ business value to comprehensively evaluate the behavioral alternatives while capturing the trade-offs among relaxed functional and quality constraints. We also introduce a genetic algorithm-based planning technique to efficiently search for the optimal (sometimes a near-optimal) behavioral alternative that can provide the best business value. The experimental study on an online order processing benchmark has shown promising results that the proposed approach can improve adaptation flexibility and business value with acceptable performance overhead.

API documentation provides important knowledge about the functionality and usage of APIs. In this paper, we focus on API caveats that developers should be aware of in order to avoid unintended use of an API. Our formative study of Stack Overflow questions suggests that API caveats are often scattered in multiple API documents, and are buried in lengthy textual descriptions. These characteristics make the API caveats less discoverable. When developers fail to notice API caveats, it is very likely to cause some unexpected programming errors. In this paper, we propose natural language processing(NLP) techniques to extract ten subcategories of API caveat sentences from API documentation and link these sentences to API entities in an API caveats knowledge graph. The API caveats knowledge graph can support information retrieval based or entity-centric search of API caveats. As a proof-of-concept, we construct an API caveats knowledge graph for Android APIs from the API documentation on the Android Developers website. We study the abundance of different subcategories of API caveats and use a sampling method to manually evaluate the quality of the API caveats knowledge graph. We also conduct a user study to validate whether and how the API caveats knowledge graph may improve the accessibility of API caveats in API documentation.

StackOverflow provides answers for a huge number of software development questions that are frequently encountered by developers. However, searching relevant questions in StackOverflow is not always easy using the keyword based search engine provided by StackOverflow. A software development question can be characterized by multiple attributes, such as, its concern (e.g., configuration problem, error handling, sample code, etc.), programming language, operating system, and involved middleware, framework, library and software technology. We propose a multi-faceted and interactive approach for searching StackOverflow questions (called MFISSO), which leverages these attributes of the questions. Our approach starts with an initial keyword-based query and extracts a multifaceted categorization from all the candidate questions using natural language processing and data mining. It then allows developers to iteratively refine the search results through an interactive process. We evaluated an implementation of MFISSO in a controlled experiments with 20 computing students, solving ten software development tasks using StackOverflow. The experiment shows that MFISSO can help developers find relevant questions faster and with higher accuracy.

Analyzing and understanding source code changes is important in a variety of software maintenance tasks. To this end, many code differencing and code change summarization methods have been proposed. For some tasks (e.g. code review and software merging), however, those differencing methods generate too fine-grained a representation of code changes, and those summarization methods generate too coarse-grained a representation of code changes. Moreover, they do not consider the relationships among code changes. Therefore, the generated differences or summaries make it not easy to analyze and understand code changes in some software maintenance tasks.

Debugging microservice systems involves the deployment and manipulation of microservice systems on a containerized environment and faces unique challenges due to the high complexity and dynamism of microservices. To address these challenges, in this paper, we propose a debugging approach for microservice systems based on the delta debugging algorithm, which is to minimize failureinducing deltas of circumstances (e.g., deployment, environmental configurations) for effective debugging. Our approach includes novel techniques for defining, deploying/manipulating, and executing deltas following the idea of delta debugging. In particular, to construct a (failing) circumstance space for delta debugging to minimize, our approach defines a set of dimensions that can affect the execution of microservice systems. Our experimental study on a medium-size microservice benchmark system shows that our approach can effectively identify failure-inducing deltas that help diagnose the root causes.

Searching and reusing online code has become a common practice in software development. Two important characteristics of online code have not been carefully considered in current tool support. First, many pieces of online code are largely similar but subtly different. Second, several pieces of code may form complex relations through their differences. These two characteristics make it difficult to properly rank online code to a search query and reduce the efficiency of examining search results. In this paper, we present an exploratory online code search approach that explicitly takes into account the above two characteristics of online code. Given a list of methods returned for a search query, our approach uses clone detection and code differencing techniques to analyze both commonalities and differences among the methods in the search results. It then produces an exploration graph that visualizes the method differences and the relationships of methods through their differences. The exploration graph allows developers to explore search results in a structured view of different method groups present in the search results, and turns implicit code differences into visual cues to help developers navigate the search results. We implement our approach in a web-based tool called CodeNuance. We conduct experiments to evaluate the effectiveness of our CodeNuance tool for search results examination, compared with ranked-list and code-clustering based search results examination. We also compare the performance and user behavior differences in using our tool and other exploratory code search tools.

Some user needs can only be met by leveraging the capabilities of others to undertake particular tasks that require intelligence and labor. Crowdsourcing such capabilities is one way to achieve this. But providing a service that leverages crowd intelligence and labor is a challenge, since various factors need to be considered to enable reliable service provisioning. For example, the selection of an optimal set of workers from those who bid to perform a task needs to be made based on their reliability, expected reward, and distance to the target locations. Moreover, for an application involving multiple services, the overall cost and time constraints must be optimally allocated to each involved service. In this article, we develop a framework, named C rowd S ervice , that supplies crowd intelligence and labor as publicly accessible crowd services via mobile crowdsourcing. The article extends our earlier work by providing an approach for constraints synthesis and worker selection. It employs a genetic algorithm to dynamically synthesize and update near-optimal cost and time constraints for each crowd service involved in a composite service and selects a near-optimal set of workers for each crowd service to be executed. We implement the proposed framework on Android platforms and evaluate its effectiveness, scalability, and usability in both experimental and user studies.

Users increasingly depend on mobile applications to get access to software services, social networks, and physical devices. When using mobile applications, users often want to achieve personal goals rather than merely perform individual tasks. To achieve a goal, a user often needs to combine software services, social cooperation, and possibly manual work. Moreover, a goal can often be achieved in different ways, each of which involves an alternative sequence of tasks. Accordingly, mobile applications should be customizable, to accommodate user preferences, and adaptive in changing their configuration automatically if the current configuration is failing. In this paper, we propose an improved runtime goal model that can manage runtime lifecycle of goals and adaptively schedule the activation of goals and execution of tasks. Based on the model, we propose an agent-based framework called MobiGoal, which combines software services, social cooperation, and manual work for achieving user goals and provides an infrastructure for developing customizable and adaptive mobile applications for personal goals. We have developed an implementation for Android platform and conducted an empirical study. The results show that MobiGoal applications can effectively support users to adaptively achieve their goals and MobiGoal can significantly save effort of application development for specific goals.

Collaborative work supported by mobile applications has become more and more popular. Mobile collaboration in some cases needs to be conducted in an interactive way to allow the sharing of the requester screen with the collaborator. Existing interactive screen sharing techniques, however, may cause heavy network traffic and high latency and lack fine-grained control of the scope of collaboration. In this paper, we propose CollaDroid, a lightweight and UI Description based technique for interactive collaboration of Android applications. CollaDroid can automatically transform an Android application to a collaboration augmented application with which a requester can interactively collaborate with a remote collaborator by synchronizing UI (User Interface) content and events. The results of our experimental study show that CollaDroid is applicable for a large part of applications in the Android Market and can provide an efficient collaboration mechanism with low network traffic and latency. And the results of our user study show that the collaboration mechanism implemented by CollaDroid is well accepted by users.

Debugging is difficult. Recent studies show that automatic bug localization techniques have limited usefulness. One of the reasons is that programmers typically have to understand why the program fails before fixing it. In this work, we aim to help programmers understand a bug by automatically generating likely invariants which are violated in the failed tests. Given a program with an initial assertion and at least one test case failing the assertion, we first generate random test cases, identify potential bug locations through bug localization, and then generate program state mutation based on active learning techniques to identify a predicate "explaining" the cause of the bug. The predicate is a classifier for the passed test cases and failed test cases. Our main contribution is the application of invariant learning for bug explanation, as well as a novel approach to overcome the problem of lack of test cases in practice. We apply our method to real-world bugs and show the generated invariants are often correlated to the actual bug fixes.

In this paper, we propose an approach to detecting project-specific recurring designs in code base and abstracting them into design templates as reuse opportunities. The mined templates allow programmers to make further customization for generating new code. The generated code involves the code skeleton of recurring design as well as the semi-implemented code bodies annotated with comments to remind programmers of necessary modification. We implemented our approach as an Eclipse plugin called MICoDe. We evaluated our approach with a reuse simulation experiment and a user study involving 16 participants. The results of our simulation experiment on 10 open source Java projects show that, to create a new similar feature with a design template, (1) on average 69% of the elements in the template can be reused and (2) on average 60% code of the new feature can be adopted from the template. Our user study further shows that, compared to the participants adopting the copy-paste-modify strategy, the ones using MICoDe are more effective to understand a big design picture and more efficient to accomplish the code reuse task.

In Online-to-Offline (O2O) commerce, customer services may need to be composed from online and offline services. Such composition is challenging, as it requires effective selection of appropriate services that, in turn, support optimal combination of both online and offline services. In this paper, we address this challenge by proposing an approach to O2O service composition which combines offline route planning and social collaboration to optimize service selection. We frame general O2O service composition problems using timed automata and propose an optimization procedure that incorporates: (1) a Markov Chain Monte Carlo (MCMC) algorithm to stochastically select a concrete composite service, and (2) a model checking approach to searching for an optimal collaboration plan with the lowest cost given certain time constraint. Our procedure has been evaluated using the simulation of a rich scenario on effectiveness and scalability.

When performing feature location tasks, developers often need to explore a large number of program elements by following a variety of clues (such as program element location, dependency, and content). As there are often complex relationships among program elements, it is likely that some relevant program elements are omitted, especially when the implementations for a feature or concern scatter across several source files. In this paper, we propose an approach for recommending potentially relevant program elements in an interactive feature location process. The two characteristics of our approach are: considering ongoing user context (i.e., confirmed or negated elements) in an interactive manner; performing an example-based reasoning to determine relevance of program elements. Based on an initial set of program elements confirmed by developers, our approach recommends additional program elements in an iterative process, in which developers can confirm relevant results, negate irrelevant results, and obtain an updated recommendation list. We have implemented our approach as an Eclipse plug-in called RecFL and conducted an experimental study. The results show that the participants using RecFL achieved a much better performance in their feature location tasks than the participants not using RecFL. The participants using RecFL also felt it easier to accomplish their feature location tasks with the support of RecFL.

Abstract The life cycle of a large‐scale software system can undergo many releases. Each release often involves hundreds or thousands of revisions committed by many developers over time. Many code changes are made in a systematic and collaborative way. However, such systematic and collaborative code changes are often undocumented and hidden in the evolution history of a software system. It is desirable to recover commonalities and associations among dispersed code changes in the evolutionary trajectory of a software system. In this paper, we present Summarizing Evolutionary Trajectory by Grouping and Aggregation (SETGA), an approach to summarizing historical commit records as trajectory patterns by grouping and aggregating relevant code changes committed over time. The SETGA extracts change operations from a series of commit records from version control systems. It then groups extracted change operations by their common properties from different dimensions such as change operation types, developers, and change locations. After that, SETGA aggregates relevant change operation groups by mining various associations among them. We implement SETGA and conduct an empirical study with 3 open‐source systems. We investigate underlying evolution rules and problems that can be revealed by the identified patterns and analyze the evolution of trajectory patterns in different periods. The results show that SETGA can identify various types of trajectory patterns that are useful for software evolution management and quality assurance.

Some user needs in real life can only be accomplished by leveraging the intelligence and labor of other people via crowdsourcing tasks. For example, one may want to confirm the validity of the description of a secondhand laptop by asking someone else to inspect the laptop on site. To integrate these crowdsourcing tasks into user applications, it is required that crowd intelligence and labor be provided as easily accessible services (e.g., Web services), which can be called crowd services. In this paper, we develop a framework named CROWDSERVICE which supplies crowd intelligence and labor as publicly accessible crowd services via mobile crowdsourcing. We implement the proposed framework on the Android platform and evaluate the usability of the framework with a user study.

Architectural refactorings can contain hundreds of steps and experienced developers could carry them out over several weeks. Moreover, developers need to explore a correct sequence of refactorings steps among many more incorrect alternatives. Thus, carrying out architectural refactorings is costly, risky, and challenging. In this paper, we present Refactoring Navigator: a tool-supported and interactive recommendation approach for aiding architectural refactoring. Our approach takes a given implementation as the starting point, a desired high-level design as the target, and iteratively recommends a series of refactoring steps. Moreover, our approach allows the user to accept, reject, or ignore a recommended refactoring step, and uses the user's feedback in further refactoring recommendations. We evaluated the effectiveness of our approach and tool using a controlled experiment and an industrial case study. The controlled experiment shows that the participants who used Refactoring Navigator accomplished their tasks in 77.4% less time and manually edited 98.3% fewer lines than the control group. The industrial case study suggests that Refactoring Navigator has the potential to help with architectural refactorings in practice.

Regression testing is essential to ensure software quality during software evolution. Two widely-used regression testing techniques, test case selection and prioritization, are used to maximize the value of the continuously enlarging test suite. However, few works consider both these two techniques together, which decreases the usefulness of the independently studied techniques in practice. In the presence of changes during program evolution, regression testing is usually conducted by selecting the test cases that cover the impact results of the changes. It seldom considers the false-positives in the information covered. Hence, the effectiveness of such regression testing techniques is decreased. In this paper, we propose an approach, ComboRT, which combines test case selection and prioritization together to directly generate a ranked list of test cases. It is based on the impact results predicted by the change impact analysis (CIA) technique, FCA–CIA, which generates a ranked list of impacted methods. Test cases which cover these impacted methods are included in the new test suite. As each method predicted by FCA–CIA is assigned with an impact factor value corresponding to the probability of this method to be impacted, test cases are then ordered according to the impact factor values of the impacted methods. Empirical studies on four Java based software systems demonstrate that ComboRT can be effectively used for regression testing in object-oriented Java-based software systems during their evolution.

Internetware is required to respond quickly to emergent user requirements or requirements changes by providing application upgrade or making context-aware recommendations. As user requirements in Internet computing environment are often changing fast and new requirements emerge more and more in a creative way, traditional requirements engineering approaches based on requirements elicitation and analysis cannot ensure the quick response of Internetware. In this paper, we propose an approach for mining context-aware user requirements from crowd contributed mobile data. The approach captures behavior records contributed by a crowd of mobile users and automatically mines context-aware user behavior patterns (i.e., when, where and under what conditions users require a specific service) from them using Apriori-M algorithm. Based on the mined user behaviors, emergent requirements or requirements changes can be inferred from the mined user behavior patterns and solutions that satisfy the requirements can be recommended to users. To evaluate the proposed approach, we conduct an experimental study and show the effectiveness of the requirements mining approach.

Developers often need to modify pasted code when programming with copy-and-paste practice. Some modifications on pasted code could involve lots of editing efforts, and any missing or wrong edit could incur bugs. In this paper, we propose a clone-based and interactive approach to recommending where and how to modify the pasted code. In our approach, we regard clones of the pasted code as the results of historical copy-and-paste operations and their differences as historical modifications on the same piece of code. Our approach first retrieves clones of the pasted code from a clone repository and detects syntactically complete differences among them. Then our approach transfers each clone difference into a modification slot on the pasted code, suggests options for each slot, and further mines modifying regulations from the clone differences. Based on the mined modifying regulations, our approach dynamically updates the suggested options and their ranking in each slot according to developer's modifications on the pasted code. We implement a proof-of-concept tool CCDemon based on our approach and evaluate its effectiveness based on code clones detected from five open source projects. The results show that our approach can identify 96.9% of the to-be-modified positions in pasted code and suggest 75.0% of the required modifications. Our human study further confirms that CCDemon can help developers to accomplish their modifications of pasted code more efficiently.

The lifecycle of a large-scale software system can undergo many releases. Each release often involves hundreds or thousands of revisions committed by many developers over time. Many code changes are made in a systematic and collaborative way. However, such systematic and collaborative code changes are often undocumented and hidden in the evolution history of a software system. It is desirable to recover commonalities and associations among dispersed code changes in the evolutionary trajectory of a software system. In this paper, we present SETGA (Summarizing Evolutionary Trajectory by Grouping and Aggregation), an approach to summarizing historical commit records as trajectory patterns by grouping and aggregating relevant code changes committed over time. SETGA extracts change operations from a series of commit records from version control systems. It then groups extracted change operations by their common properties from different dimensions such as change operation types, developers and change locations. After that, SETGA aggregates relevant change operation groups by mining various associations among them. The proposed approach has been implemented and applied to three open-source systems. The results show that SETGA can identify various types of trajectory patterns that are useful for software evolution management and quality assurance.

When performing software maintenance tasks, developers often need to understand a series of background knowledge based on information distributed in different software repositories such as source codes, version control systems and bug tracking systems. An effective way to support developers to understand such knowledge is to provide an integrated knowledge base and allow them to ask questions using natural language. Existing approaches cannot well support natural language questions that involve a series of conceptual relationships and are phrased in a flexible way. In this paper, we propose an interactive approach for understanding developers' natural language queries. The approach can understand a developer's natural language questions phrased in different ways by generating a set of ranked and human-readable candidate questions and getting feedback from the developer. Based on the candidate question confirmed by the developer, the approach can then synthesize an answer by constructing and executing a structural query to the knowledge base. We have implemented a tool following the proposed approach and conducted a user study using the tool. The results show that our approach can help developers get the desired answers more easily and accurately.

Developers work in the IDE, but search online resources in the web browser. The separation of the working and search context often cause the ignorance of the working context during online search. Several tools have been proposed to integrate the web browser into the IDE so that developers can search and use online resources directly in the IDE. These tools enable only the shallow integration of the web browser and the IDE. Some tools allow the developer to augment search queries with program entities in the current snapshot of the code. In this paper, we present an in-IDE ambient search agent to bridge the separation of the developer's working context and search context. Our approach considers the developers' working context in the IDE as a time-series stream of programming event observed from the developer's interaction with the IDE over time. It supports the deeper integration of the working context in the entire search process from query formulation, custom search, to search results refinement and representation. We have implemented our ambient search agent and integrate it into the Eclipse IDE. We conducted a user study to evaluate our approach and the tool support. Our evaluation shows that our ambient search agent can better aid developers in searching and using online programming resources while working in the IDE.

In an uncertain and changing environment, a composite service needs to continuously optimize its business process and service selection through runtime adaptation. To achieve the overall satisfaction of stakeholder requirements, quality tradeoffs are needed to adapt the composite service in response to the changing environments. Existing approaches on service selection and composition, however, are mostly based on quality preferences and business processes decisions made statically at the design time. In this paper, we propose a requirements-driven self-optimization approach for composite services. It measures the quality of services (QoS), estimates the earned business value, and tunes the preference ranks through a feedback loop. The detection of unexpected earned business value triggers the proposed self-optimization process systematically. At the process level, a preference-based reasoner configures a requirements goal model according to the tuned preference ranks of QoS requirements, reconfiguring the business process according to its mappings from the goal configurations. At the service level, selection decisions are optimized by utilizing the tuned weights of QoS criteria. We used an experimental study to evaluate the proposed approach. Results indicate that the new approach outperforms both fixed-weighted and floating-weighted service selection approaches with respect to earned business value and adaptation flexibility.

Socio-technical systems (STSs) consist of human, hardware and software agents that work in tandem to fulfill stakeholder requirements. A specification for an STS consists of a set of (social) commitments among participating agents that serve as a contract among them. However, by their very nature, STSs are open, dynamic and continuously evolving along with their environments. To ensure that such systems continue to satisfy their requirements, the agents that comprise an STS must continuously adapt their behaviors to take into account risks and opportunities that arise at runtime. This paper presents a decision-theoretic self-adaptation framework that proposes candidate adaptation strategies for participating agents. These strategies are implementable by reconfiguration of plans or even changes in contractual commitments among agents. The adaptation procedure involves negotiating commitment changes and possible compensations to/from creditor agents. To evaluate the proposal, the paper also presents the results of an experimental study with a simulated STS, which confirms that success rates for achieving stakeholder goals and overall trade off utility can be improved significantly by incorporating evolving commitments to support STS adaptation.

A self-adaptive system uses runtime models to adapt its architecture to the changing requirements and contexts. However, there is no one-to-one mapping between the requirements in the problem space and the architectural elements in the solution space. Instead, one refined requirement may crosscut multiple architectural elements, and its realization involves complex behavioral or structural interactions manifested as architectural design decisions. In this paper we propose to combine two kinds of self-adaptations: requirements-driven self-adaptation, which captures requirements as goal models to reason about the best plan within the problem space, and architecture-based self-adaptation, which captures architectural design decisions as decision trees to search for the best design for the desired requirements within the contextualized solution space. Following these adaptations, component-based architecture models are reconfigured using incremental and generative model transformations. Compared with requirements-driven or architecture-based approaches, the case study using an online shopping benchmark shows promise that our approach can further improve the effectiveness of adaptation (e.g. system throughput in this case study) and offer more adaptation flexibility.

Clone detectors find similar code fragments (i.e., instances of code clones) and report large numbers of them for industrial systems. To maintain or manage code clones, developers often have to investigate differences of multiple cloned code fragments. However,existing program differencing techniques compare only two code fragments at a time. Developers then have to manually combine several pairwise differencing results. In this paper, we present an approach to automatically detecting differences across multiple clone instances. We have implemented our approach as an Eclipse plugin and evaluated its accuracy with three Java software systems. Our evaluation shows that our algorithm has precision over 97.66% and recall over 95.63% in three open source Java projects. We also conducted a user study of 18 developers to evaluate the usefulness of our approach for eight clone-related refactoring tasks. Our study shows that our approach can significantly improve developers’performance in refactoring decisions, refactoring details, and task completion time on clone-related refactoring tasks. Automatically detecting differences across multiple clone instances also opens opportunities for building practical applications of code clones in software maintenance, such as auto-generation of application skeleton, intelligent simultaneous code editing.

Code clones have to be made explicit and be managed in software maintenance. Researchers have developed many clone detection tools to detect and analyze code clones in software systems. These tools report code clones as similar code fragments in source files. However, clone-related maintenance tasks (e.g., refactorings) often involve a group of code clones appearing in larger syntactic context (e.g., code clones in sibling classes or code clones calling similar methods). Given a list of low-level code-fragment clones, developers have to manually summarize from bottom up low-level code clones that are relevant to the syntactic context of a maintenance task. In this paper, we present a clone summarization technique to summarize code clones with respect to their common syntactic context. The clone summarization allows developers to locate and maintain code clones in a top-down manner by type hierarchy and usage dependencies. We have implemented our approach in the Clonepedia tool and conducted a user study on JHotDraw with 16 developers. Our results show that Clonepedia users can better locate and refactor code clones, compared with developers using the Clone Detective tool.

Requirements-driven approaches provide an effective mechanism for self-adaptive systems by reasoning over their runtime requirements models to make adaptation decisions. However, such approaches usually assume that the relations among alternative behaviours, environmental parameters and requirements are clearly understood, which is often simply not true. Moreover, they do not consider the influence of the current behaviour of an executing system on adaptation decisions. In this paper, we propose an improved requirements-driven self-adaptation approach that combines goal reasoning and case-based reasoning. In the approach, past experiences of successful adaptations are retained as adaptation cases, which are described by not only requirements violations and contexts, but also currently deployed behaviours. The approach does not depend on a set of original adaptation cases, but employs goal reasoning to provide adaptation solutions when no similar cases are available. And case-based reasoning is used to provide more precise adaptation decisions that better reflect the complex relations among requirements violations, contexts, and current behaviours by utilizing past experiences. Our experimental study with an online shopping benchmark shows that our approach outperforms both requirements-driven approach and case-based reasoning approach in terms of adaptation effectiveness and overall quality of the system.

Feature location is a human-oriented and information-intensive process. When performing feature location tasks with existing tools, developers often feel it difficult to formulate an accurate feature query (e.g., keywords) and determine the relevance of returned results. In this paper, we propose a feature location approach that supports multi-faceted interactive program exploration. Our approach automatically extracts and mines multiple syntactic and semantic facets from candidate program elements. Furthermore, it allows developers to interactively group, sort, and filter feature location results in a centralized, multi-faceted, and intelligent search User Interface (UI). We have implemented our approach as a web-based tool MFIE and conducted an experimental study. The results show that the developers using MFIE can accomplish their feature location tasks 32% faster and the quality of their feature location results (in terms of F-measure) is 51% higher than that of the developers using regular Eclipse IDE.

Software systems contain many implicit application-specific business and programming rules. These rules represent high-level logical structures and processes for application-specific business and programming concerns. They are crucial for program understanding, consistent evolution, and systematic reuse. However, existing pattern mining and analysis approaches cannot effectively mine such application-specific rules. In this paper, we present an approach for mining logical clones in software that reveal high-level business and programming rules. Our approach extracts a program model from source code, and enriches the program model with code clone information, functional clusters (i.e., a set of methods dealing with similar topics or concerns), and abstract entity classes (representing sibling entity classes). It then analyzes the enriched program model for mining recurring logical structures as logical clones. We have implemented our approach in a tool called MiLoCo (Mining Logical Clone) and conducted a case study with an open-source ERP and CRM software. Our results show that MiLoCo can identify meaningful and useful logical clones for program understanding, evolution and reuse.

In this paper, we address the optimization problem of SLA-constrained service composition. Focusing on the main drawbacks of traditional approaches surveyed:1) the difficulties in preference definition and weight assignment, 2) the limitation of linear utility function for identifying preferred skyline solutions, and 3) the poor efficiency and scalability of algorithms, we present a systematic approach of combining the weighted Tchebycheff distance with skyline computation to cope with this optimization problem. More specifically, we first propose a fuzzy linguistic preference model that can help service composer elicit, represent and establish consistent preference relations upon QoS dimensions. Then we present a weighting procedure to transform the preference relations into numeric weights that are used in the Tchebycheff distance as quantified measurement of preference for skyline solutions. Finally we propose a hybrid evolutionary algorithm to heuristically find preferred skyline solutions in an efficient way. The algorithm is further evaluated by a set of experimental studies.

Effective clone management is essential for developers to recognize the introduction and evolution of code clones, to judge their impact on software quality, and to take appropriate measures if required. Our previous study shows that cloning practice is not simply a technical issue. It must be interpreted and considered in a larger context from technical, personal, and organizational perspectives. In this paper, we propose a contextual and on-demand code clone management approach called CCEvents (Code Cloning Events). Our approach provides timely notification about relevant code cloning events for different stakeholders through continuous monitoring of code repositories. It supports on-demand customization of clone monitoring strategies in specific technical, personal, and organizational contexts using a domain-specific language. We implemented the proposed approach and conducted an empirical study with an industrial project. The results confirm the requirements for contextual and on-demand code clone management and show the effectiveness of CCEvents in providing timely code cloning notifications and in helping to achieve effective clone management.

Self-repairing approaches have been proposed to alleviate the runtime requirements satisfaction problem by switching to appropriate alternative solutions according to the feedback monitored. However, little has been done formally on analyzing the relations between specific environmental failures and corresponding repairing decisions, making it a challenge to derive a set of alternative solutions to withstand possible environmental failures at runtime. To address these challenges, we propose a requirements-driven self-repairing approach against environmental failures, which combines both development-time and runtime techniques. At the development phase, in a stepwise manner, we formally analyze the issue of self-repairing against environmental failures with the support of the model checking technique, and then design a sufficient and necessary set of alternative solutions to withstand possible environmental failures. The runtime part is a runtime self-repairing mechanism that monitors the operating environment for unsatisfiable situations, and makes self-repairing decisions among alternative solutions in response to the detected environmental failures.

SUMMARY Developers often have to locate the parts of source code that contribute to a specific feature during software maintenance tasks. This activity, referred to as feature location in software engineering, is a human‐intensive and knowledge‐intensive process. Researchers have investigated (semi‐)automatic analysis‐based techniques to assist developers in such feature location activities. However, little work has been carried out on better understanding how developers perform feature location tasks. In this paper, we report an exploratory study of feature location process, consisting of three experiments in which developers were given unfamiliar systems and asked to complete six feature location tasks. Our study suggests that feature location process can be understood hierarchically at three levels of granularity: phase, pattern, and action. Furthermore, our statistical analysis shows that these feature location phases, patterns, and actions can be effectively imparted to junior developers and consequently improve their performance on feature location tasks. Our qualitative observations and interviews also suggest that external factors, for example, human factors, task properties, and in‐process feedbacks, affect the choices and usage of different feature location patterns and actions. Our results open up new opportunities to feature location research, which could lead to better tool support and more rigorous feature location process. Copyright © 2013 John Wiley & Sons, Ltd.

Running with static requirements and design decisions, a software system cannot always perform optimally in a highly uncertain and rapidly changing environment. Quality-driven self-adaptation, which enables a software system to continually adapt its structure and behavior to improve the overall quality satisfaction, thus becomes a promising capability of software systems. Existing researches on self-adaptive systems, although having proposed effective methods and techniques on requirements-driven self-adaptation and reflective components, do not well address the gap between requirements and runtime architecture. In this paper, we propose a quality-driven self-adaptation approach, which incorporates both requirements- and architecture-level adaptations. At the requirements level, value-based quality tradeoff decisions are made with the aim of maximizing system-level value propositions. At the architecture level, component-based architecture adaptations are conducted. To bridge the gap between requirements and runtime architecture, design decisions capturing alternative design options and their rationales are introduced to help map requirements adaptations and context changes to adaptation operations on the runtime architecture. To validate the effectiveness, we implement the approach based on a reflective component model and conduct an experimental study on it. The results show that the approach leads to better performance compared with traditional software and the overall quality satisfaction is kept maintained. Furthermore, the development effort is affordable but the approach still has shortage in extensibility.

Self-adaptive systems are now facing the anticipation of mass customization. Therefore, the Software Product Line (SPL) engineering for developing Self-Adaptive systems (SPL4SA) can be an effective way. At the first sight, SPL4SA is the straightforward combination of the two methodologies of SPL engineering and self-adaptive systems. However, the direct and unsystematic combination will bring difficulty in the domain requirements analysis and in the customization process. In this paper, in order to give a solution to the practical problems, we propose a domain requirements meta-model in SPL4SA. It is described with different point of views and the variability binding constraints inside are emphasized. Based on it, a guidance is concluded to support the consistent customization towards the domain model. In addition, an experimental study about a web-based business product line involving self-adaptation capability is conducted to evaluate the model.

Modern information systems, such as enterprise applications and e-commerce applications, often consist of databases surrounded by a large variety of software applications depending on the databases. During the evolution and deployment of such information systems, developers have to ensure the global consistency between database schemas and surrounding software applications. However, in such situations as Enterprise Application Integration (EAI), databases are shared by a number of software applications contributed by different independent parties, and the developers of those applications often have little or no control on when and how database schema evolves over time. As a result, databases and software applications may not always remain in sync. Such inconsistency may lead to data loss, program failures, or decreased performance. The fundamental challenge in evolving and deploying such database-centric information systems is the fact that databases and their surrounding software applications are subject to independent, asynchronous, and potentially conflicting evolution processes. In this paper, we present an approach to automatically adapting software applications to the evolution of their underlying databases by graph-based schema differencing and aspect-oriented dynamic patching. Our empirical study shows that our approach can automatically adapt software applications to a number of common types of database schema evolution, which accounts for over 87.5% of all schema evolution in the subject system. Our approach allows database schema maintainer to evolve database schema more freely without being afraid of breaking surrounding software applications; it also allows application developers to catch up database schema evolution more quickly without diverting too much from their main business concerns.

Code clones are similar code segments. Researchers have proposed many techniques to detect, understand and eliminate code clones. However, due to lack of deeper understanding of reasons of cloning practices, especially from personal and organizational perspectives, little effective support can be provided to alleviate maintenance problems caused by code clones. In this paper, we report an industrial study on investigating reasons of cloning practices in large-scale software development from technical, personal, and organizational perspectives. Our study involves code analysis, questionnaire survey, and interviews with developers, and gathers solid empirical data about how developers clone and why during different phases of clones' lifecycle in industrial development. The results of our study suggest that cloning is not simply a technical issue; it must be interpreted and understood in larger context in which code clones occur and evolve. Within these contexts, there are several adjustable factors and two critical points that affect the introduction, existence, and removal of clones. These adjustable factors and critical points reveal opportunities to improve cloning practices in industrial development from technical, personal, and organizational perspectives.

Socio-technical systems consist of human, hardware and software components that work in tandem to fulfill stakeholder requirements. By their very nature, such systems operate under uncertainty as components fail, humans act in unpredictable ways, and the environment of the system changes. Self-repair refers to the ability of such systems to restore fulfillment of their requirements by relying on monitoring, reasoning, and diagnosing on the current state of individual requirements. Self-repair is complicated by the multi-agent nature of socio-technical systems, which demands that requirements monitoring and self-repair be done in a decentralized fashion. In this paper, we propose a stateful requirements monitoring approach by maintaining an instance of a state machine for each requirement, represented as a goal, with runtime monitoring and compensation capabilities. By managing the interactions between the state machines, our approach supports hierarchical goal reasoning in both upward and downward directions. We have implemented a customizable Java framework that supports experimentation by simulating a socio-technical system. Results from our experiments suggest effective and precise support for a wide range of self-repairing decisions in a socio-technical setting.

Fine-grained software configuration management has been proven to offer substantial benefits for software development in many fields, overcoming developmental problems, such as complexity management and support for communication and coordination, among others. The same problems exist in a collaborative ontology development environment that uses a simple versioning and configuration management system. The author presents a general mechanism to support hierarchy versioning and configuration item management in collaborative ontology development. This fine-grained configuration management mechanism can help solve collaborative developmental problems. An implementation system is presented to illustrate the capability and efficiency of this mechanism.

Locating the program element(s) relevant to a particular feature is an important step in efficient maintenance of a software system. The existing feature location techniques analyze each feature independently and perform a one-time analysis after being provided an initial input. As a result, these techniques are sensitive to the quality of the input, and they tend to miss the nonlocal interactions among features. In this paper, we propose to address the proceeding two issues in feature location using an iterative context-aware approach. The underlying intuition is that the features are not independent of each other, and the structure of source code resembles the structure of features. The distinguishing characteristics of the proposed approach are: 1) it takes into account the structural similarity between a feature and a program element to determine their relevance; 2) it employs an iterative process to propagate the relevance of the established mappings between a feature and a program element to the neighboring features and program elements. Our initial evaluation suggests the proposed approach is more robust and can significantly increase the recall of feature location with a slight decrease in precision.

Developers often have to locate the parts of the source code that contribute to a specific feature during software maintenance tasks. This activity, referred to as feature location in software engineering, is a human- and knowledge-intensive process. Researchers have investigated information retrieval, static/dynamic analysis based techniques to assist developers in such feature location activities. However, little work has been done on better understanding how developers perform feature location tasks. In this paper, we report an exploratory study of feature location process, consisting of two experiments in which developers were given unfamiliar systems and asked to complete six feature location tasks in two hours. Our study suggests that feature location process can be understood hierarchically at three levels of granularities: phase, pattern, and action. Furthermore, our study suggests that these feature-location phases, patterns and actions can be effectively imparted to junior developers and consequently improve their performance on feature location tasks. Our results open up new opportunities to feature location research, which could lead to better tool support and more rigorous feature location process.

It is common in practice that a Software Product Line (SPL) is constructed by reengineering a set of existing variant products. To alleviate the problems of high risks of failures and the limitations of resources and cost, incremental reengineering towards a SPL is a natural choice in many cases. However, several problems remain unaddressed properly, such as how to define increments, how to satisfy regular product delivery in parallel with reengineering, and how to achieve early successes. In this paper, we report an industrial case study on a successful SPL-targeted reengineering project conducted in Alcatel-Lucent. In this project, the project team applied the principles of agile development in the process of SPL reengineering. The key practices of the project include value-based increment definition, domain-driven responsibility alignment, iterative component refactoring and integration. We analyze the reengineering process of a major component qualitatively and quantitatively, with the focus on initial investment required, trend of investment, returns on investment and quality improvement. Our case study shows that incremental and iterative approach with stakeholder-value considerations can help to achieve steady and successful SPL reengineering in a cost-effective manner. We also find that SPL adoption can be regarded as an emergent result of the reconstruction and improvement of existing product assets.

Running in a highly uncertain and greatly complex environment, Web systems cannot always provide full set of services with optimal quality, especially when work loads are high or subsystem failures are frequent. Hence, it is significant to continuously maintain a high satisfaction level of survivability, hereafter survivability assurance, while relaxing or sacrificing certain quality or functional requirements that are not crucial to the survival of the entire system. After giving a value-based interpretation to survivability assurance to facilitate a quantitative analysis, we propose a requirements-driven self-tuning method for the survivability assurance of Web systems. Maintaining an enriched and live goal model, our method adapts to runtime tradeoff decisions made by our PID (proportional-integral-derivative) controller and goal-oriented reasoner for both quality and functional requirements. The goal-based configuration plans produced by the reasoner is carried out on the live goal model, and then mapped into system architectural configurations. Experiments on an online shopping system are conducted to validate the effectiveness of the proposed method.

In the long-term evolution of a Software Product Line (SPL), how to ensure the alignment between the reference and application architectures is a critical problem. Existing ad-hoc methods for architecture synchronization cannot ensure the completeness. In this paper, we propose a model-driven method for synchronized SPL architecture evolution using bidirectional transformation, a well-developed technique with solid mathematical foundation. Based on the model-based architecture representation, we capture the variability-intensive consistency relations between reference and application architectures and specify them with Beanbag, a declarative language supporting operation-based synchronization. Then, with the generated synchronizer and additional mechanisms, we can achieve coordinated architecture evolution through periodic synchronizations.

Quality requirements of a software system cannot be optimally met, especially when it is running in an uncertain and changing environment. In principle, a controller at runtime can monitor the change impact on quality requirements of the system, update the expectations and priorities from the environment, and take reasonable actions to improve the overall satisfaction. In practice, however, existing controllers are mostly designed for tuning low-level performance indicators rather than high-level requirements. By linking the overall satisfaction to a business value indicator as feedback, we propose a control theoretic self-tuning method that can dynamically adjust the tradeoff decisions among different quality requirements. A preference-based reasoning algorithm is involved to configure hard goals accordingly to guide the following architecture reconfiguration.

The increasingly complex environments in which software systems are running today have made runtime software quality unstable and hardly in an optimal state, especially for those systems in open and dynamic environments, e.g. Internetware. In this paper, we explore the effectiveness of software cybernetics and feedback control theory in runtime software quality optimization. We propose a method of runtime quality optimization by using feedback control theory. Specially, we consider the problem of runtime optimization for a specific quality attribute, namely throughput, for Web-based systems. We design a double-layer feedback control model for the problem and implement the runtime optimization control method. In the method, runtime feedbacks are collected and used by the control model to adjust related control parameters. The experimental study has demonstrated the effectiveness of software cybernetics and feedback control theory in runtime quality optimization.

Commercial-off-the-shelf components (COTS) are widely reused at present and black-box composition is the unique way to integrate them into the target system. However, various mismatches among components often hamper the integration of COTS. While the existing approaches to modeling software systems often neglect the issue of COTS component-mismatch detection and elimination. Aiming at solving this problem, this paper proposes an adaptive software architecture model. Based on this model, we first analyze and conclude the mismatches among heterogeneous components, and then we propose the corresponding solutions to eliminate these mismatches and provide a seamless integration for COTS components. At the same time, we employ an example of Web-based application system to illustrate the efficiency of our approach.

The coordination-policy that components interactions satisfied often determines the properties of nowadays component-based information systems, e.g. safety, liveness and fairness etc. Therefore, how to derive coordination-policy satisfying behavior all out of such system to achieve better system properties is of a significant problem that needs to be solved. Aim to this problem, we propose an optimistic policy- satisfying behavior derivation approach in this paper. The main idea of the approach is to automatically construct a Coordination Environment (CE) for such composed system that system components can work together in a deadlock-free and policy-satisfying manner, and so as to obtain desired system properties. In this approach, component-based information system is modeled by interface automaton network (IAN), and component coordination-policies are specified by LTL. To explain the correctness and validity of this approach, we give a corresponding example certification.

Lose sight of the existence of system crosscutting concerns, e.g. safety and quality etc, often causes the system hard to maintain and evolve according to the changing environment and requirements. In this paper we propose an incremental aspect-oriented (AO) approach to ease this kind of evolution problem in architecture level. In this approach we introduce a novel connector, namely aspect weaving connector (AWC), to support the seamless integration of AOSD and software architecture modeling. Concretely crosscutting concerns are encapsulated into aspects and modeled as software components. AWC acts as a connector wrapper coordinating the interaction between aspectual and regular components. In order to provide a formal basic to AWC, we propose a conceptual model of it, which formalizes the underlying mechanisms of aspect dynamic weaving in architecture level using process algebra CSP. Then we verify the model's properties with FDR2 and prove that our connector-centric AO architecture modeling approach can give system an architectural dynamism and make it easier to maintain and evolve.

Dynamic evolution has been an essential requirement for more and more business systems which attempt to provide 7(days) x 24(hours) availability and flexible adaptability on the changing business environment. Therefore, these systems are expected to be self-adaptable at run-time with little user intervention. CBSD provides an architectural way for self-adaptation, in which adaptation can be performed on the macro level of architecture and easier to control. However, the big gap between the problem space (business goal and environment) and the solution space (software architecture and components), and the runtime decision-making for adaptation are two difficulties for the implementation of business-oriented self-adaptation. In this paper, we propose an approach of decision support for dynamic adaptation of business systems based on feature binding analysis. In the method, feature model is introduced to represent the business policy and bridge the gap. So, dynamic adaptation can first be performed on feature binding analysis. The other characteristic of the method is CBR (case based reasoning) based adaptation decision on environment factors captured by all kinds of sensors.

In a component-based system components are basic computation units implementing specific business functions, and their interactions are explicitly represented by connectors. If the system is required to be adaptable with dynamic architectural evolutions, the connectors must have the capability of being adapted at runtime to different interaction context. Aiming at this requirement, we propose an automatic connector generation method in this paper. In the method, connectors are generated on analysis of behavior and data specifications of two components to be assembled. First a state machine for connector behavior is created, and then the connector can be executed on the state machine, data adaptation scheme and predefined stub templates for various component types. Mismatches on data are resolved by model transformation. An example of payment query in e-business is referred throughout the paper to illustrate our method.