Recent advancements in end-to-end autonomous driving systems (ADSs) underscore their potential for perception and planning capabilities. However, challenges remain. Complex driving scenarios contain rich semantic information, yet ambiguous or noisy semantics can compromise decision reliability, while interference between multiple driving tasks may hinder optimal planning. Furthermore, prolonged inference latency slows decision-making, increasing the risk of unsafe driving behaviors. To address these challenges, we propose ExpertAD, a novel framework that enhances the performance of ADS with Mixture of Experts (MoE) architecture. We introduce a Perception Adapter (PA) to amplify task-critical features, ensuring contextually relevant scene understanding, and a Mixture of Sparse Experts (MoSE) to minimize task interference during prediction, allowing for effective and efficient planning. Our experiments show that ExpertAD reduces average collision rates by up to 20% and inference latency by 25% compared to prior methods. We further evaluate its multi-skill planning capabilities in rare scenarios (e.g., accidents, yielding to emergency vehicles) and demonstrate strong generalization to unseen urban environments. Additionally, we present a case study that illustrates its decision-making process in complex driving scenarios.

Open source software (OSS) has become the foundation of modern applications, but its transitive dependencies make it especially vulnerable to supply chain attacks. One common tactic is to inject malicious code into third-party packages. NPM, in particular, due to its widespread use and large volume of packages, has become the popular target of malicious code injection. While various detectors have been proposed, they suffer three limitations, i.e., inadequate behavior modeling of obfuscated code, ignoring object-centric features of JavaScript, and lack of synergy between static and dynamic analysis. These limitations lead to imprecise modeling of program behavior and hinder detection effectiveness.To address these limitations, we propose ProfMal to identify malicious NPM packages, which leverages the synergy between static and dynamic analysis to construct behavior graphs for each package. Specifically, our static analysis constructs the behavior graphs through object-sensitive analysis, while identifying sensitive API calls and locating statically unresolved calls. Our dynamic analysis augments the behavior graphs by resolving those statically unresolved calls. Based on these comprehensive behavior graphs, we train a graph-based classifier to identify maliciousness. Our evaluation has indicated that ProfMal achieves the highest F1-score of 92.4%, outperforming the state-of-the-arts by 6.2% to 48.8%. During a three-month real-world detection, ProfMal has detected 496 previously unknown malicious NPM packages, and all of them have been confirmed and removed from NPM.

End-to-end autonomous driving systems (ADSs), with their strong capabilities in environmental perception and generalizable driving decisions, are attracting growing attention from both academia and industry. However, once deployed on public roads, ADSs are inevitably exposed to diverse driving hazards that may compromise safety and degrade system performance. This raises a strong demand for resilience of ADSs, particularly the capability to continuously monitor driving hazards and adaptively respond to potential safety violations, which is crucial for maintaining robust driving behaviors in complex driving scenarios.To bridge this gap, we propose a resilience-oriented runtime framework, named Argus, to mitigate the driving hazards, thus preventing potential safety violations and improving the driving performance of an ADS. Argus continuously monitors the trajectories generated by the ADS for potential hazards and, whenever the EGO vehicle is deemed unsafe, seamlessly takes control via a hazard mitigator. We integrate Argus with three state-of-the-art end-to-end ADSs, i.e., TCP, UniAD and VAD. Our evaluation has demonstrated that Argus effectively and efficiently enhances the resilience of ADSs, improving the driving score of ADSs by 150.30% on average, and preventing 64.38% of the violations, with little additional time overhead.

Open source software (OSS) supply chain enlarges the attack surface of a software system, which makes package registries attractive targets for attacks. Recently, multiple package registries have received intensified attacks with malicious packages. Of those package registries, NPM and PyPI are two of the most severe victims. Existing malicious package detectors are developed with features from a list of packages of the same ecosystem and deployed within the same ecosystem exclusively, which is infeasible to utilize the knowledge of a new malicious NPM package detected recently to detect the new malicious package in PyPI. Moreover, existing detectors lack support to model malicious behavior of OSS packages in a sequential way. To address the two limitations, we propose a single detection model using malicious behavior sequence, named Cerebro , to detect malicious packages in NPM and PyPI. We curate a feature set based on a high-level abstraction of malicious behavior to enable multi-lingual knowledge fusing. We organize extracted features into a behavior sequence to model sequential malicious behavior. We fine-tune the pre-trained language model to understand the semantics of malicious behavior. Extensive evaluation has demonstrated the effectiveness of Cerebro over the state-of-the-art as well as the practically acceptable efficiency. Cerebro has detected 683 and 799 new malicious packages in PyPI and NPM, and received 707 thank letters from the official PyPI and NPM teams.

The widespread use of open-source software (OSS) has led to extensive code reuse, making vulnerabilities in OSS significantly pervasive.The vulnerabilities due to code reuse in OSS are commonly known as vulnerable code clones (VCCs) or recurring vulnerabilities.Existing approaches primarily employ clone-based techniques to detect recurring vulnerabilities by matching vulnerable functions in software projects.These techniques do not incorporate specially designed mechanisms for vulnerabilities with multiple fixing functions (VM).Typically, they generate a signature for each fixing function and report VM using a matching-one-in-all approach.However, the variation in vulnerability context across diverse fixing functions results in varying accuracy levels in detecting VM, potentially limiting the effectiveness of existing methods.In this paper, we introduce VMud, a novel approach for detecting Vulnerabilities with Multiple Fixing Functions.VMud identifies vulnerable function clones (VCCs) through function matching similar to existing methods.However, VMud takes a different approach by only selecting the critical functions from VM for signature generation, which are a subset of the fixing functions.This step ensures that VMud focuses on fixing functions that offer sufficient knowledge about the VM.To cope with the potential decrease in recall due to excluding the remaining fixing functions, VMud employs semantic equivalent statement matching using these critical functions.It aims to uncover more VM by creating two signatures of each critical function and matching precisely by contextual semantic equivalent statement mapping on the two signatures.Our evaluation has demonstrated that VMud surpasses state-of-the-art vulnerability detection approaches by 30.30% in terms of F1-Score.Furthermore,

Code review is an essential activity for ensuring the quality and maintainability of software projects. However, it is a time-consuming and often error-prone task that can significantly impact the development process. Recently, ChatGPT, a cutting-edge language model, has demonstrated impressive performance in various natural language processing tasks, suggesting its potential to automate code review processes. However, it is still unclear how well ChatGPT performs in code review tasks. To fill this gap, in this paper, we conduct the first empirical study to understand the capabilities of ChatGPT in code review tasks, specifically focusing on automated code refinement based on given code reviews. To conduct the study, we select the existing benchmark CodeReview and construct a new code review dataset with high quality. We use CodeReviewer, a state-of-the-art code review tool, as a baseline for comparison with ChatGPT. Our results show that ChatGPT outperforms CodeReviewer in code refinement tasks. Specifically, our results show that ChatGPT achieves higher EM and BLEU scores of 22.78 and 76.44 respectively, while the state-of-the-art method achieves only 15.50 and 62.88 on a high-quality code review dataset. We further identify the root causes for ChatGPT's underperformance and propose several strategies to mitigate these challenges. Our study provides insights into the potential of ChatGPT in automating the code review process, and highlights the potential research directions.

Software composition analysis (SCA) tools have been widely adopted to identify vulnerable libraries used in software applications. Such SCA tools depend on a vulnerability database to know affected libraries of each vulnerability. However, it is labor-intensive and error prone for a security team to manually maintain the vulnerability database. While several approaches adopt extreme multi-label learning to predict affected libraries for vulnerabilities, they are practically ineffective due to the limited library labels and the unawareness of ecosystems.

Open source software brings benefit to the software community but also introduces legal risks caused by license violations, which result in serious consequences such as lawsuits and financial losses. To mitigate legal risks, some approaches have been proposed to identify licenses, detect license incompatibilities and inconsistencies, and recommend licenses. As far as we know, however, there is no prior work to understand modification terms in open source licenses or to detect and fix violations of modification terms. To bridge this gap, we first empirically characterize modification terms in 48 open source licenses. These licenses all require certain forms of “notice” to describe the modifications made to the original work. Inspired by our study, we then design LiVo to automatically detect and fix violations of modification terms in open source licenses during forking. Our evaluation has shown the effectiveness and efficiency of LiVo. 18 pull requests for fixing modification term violations have received positive responses. 8 have been merged.

Simulation testing has been widely adopted by leading companies to ensure the safety of autonomous driving systems (ADSs). Anumber of scenario-based testing approaches have been developed to generate diverse driving scenarios for simulation testing, and demonstrated to be capable of finding safety violations. However, there is no automated way to diagnose whether these violations are caused by the ADS under test and which category these violations belong to. As a result, great effort is required to manually diagnose violations. To bridge this gap, we propose DiaVio to automatically diagnose safety violations in simulation testing by leveraging large language models (LLMs). It is built on top of a new domain specific language (DSL) of crash to align real-world accident reports described in natural language and violation scenarios in simulation testing. DiaVio fine-tunes a base LLM with real-world accident reports to learn diagnosis capability, and uses the fine-tuned LLM to diagnose violation scenarios in simulation testing. Our evaluation has demonstrated the effectiveness and efficiency of DiaVio in violation diagnosis.

Data-driven techniques are promising for automatically locating and fixing bugs, which can reduce enormous time and effort for developers. However, the effectiveness of these techniques heavily relies on the quality and scale of bug datasets. Despite that emerging approaches to automatic bug dataset construction partially provide a solution for scalability, data quality remains a concern. Specifically, it remains a barrier for humans to isolate the minimal set of bug-inducing or bug-fixing changes, known as critical changes. Although delta debugging (DD) techniques are capable of extracting critical changes on benchmark datasets in academia, the efficiency and accuracy are still limited when dealing with real-world bugs, where code change dependencies could be overly complicated. In this paper, we propose C2D2, a novel delta debugging approach for critical change extraction, which estimates the probabilities of dependencies between code change elements. C2D2 considers the probabilities of dependencies and introduces a matrix-based search mechanism to resolve compilation errors (CE) caused by missing dependencies. It also provides hybrid mechanisms for flexibly selecting code change elements during the DD process. Experiments on Defect4J and a real-world regression bug dataset reveal that C2D2 is significantly more efficient than the traditional DD algorithm ddmin with competitive effectiveness, and significantly more effective and more efficient than the state-of-the-art DD algorithm ProbDD. Furthermore, compared to human-isolated critical changes, C2D2 produces the same or better critical change results in 56% cases in Defects4J and 86% cases in the regression dataset, demonstrating its usefulness in automatically extracting critical changes and saving human efforts in constructing large-scale bug datasets with real-world bugs.

Open source software (OSS) supply chains have been attractive targets for attacks. One of the significant, popular attacks is realized by malicious packages on package registries. NPM, as the largest package registry, has been recently flooded with malicious packages. In response to this severe security risk, many detection tools have been proposed. However, these tools do not model malicious behavior in a holistic way; only consider a predefined set of sensitive APIs; and require huge manual confirmation effort due to high false positives and binary detection results. Thus, their practical usefulness is hindered.

Vulnerability reports play a crucial role in mitigating open-source software risks. Typically, the vulnerability report contains affected versions of a software. However, despite the validation by security expert who discovers and vendors who review, the affected versions are not always accurate. Especially, the complexity of maintaining its accuracy increases significantly when dealing with multiple versions and their differences. Several advances have been made to identify affected versions. However, they still face limitations. First, some existing approaches identify affected versions based on repository-hosting platforms (i.e., GitHub), but these versions are not always consistent with those in package registries (i.e., Maven). Second, existing approaches fail to distinguish the importance of different vulnerable methods and patched statements in face of vulnerabilities with multiple methods and change hunks.

The wide adoption of open source third-party libraries can propagate vulnerabilities that originally exist in third-party libraries through dependency chains to downstream projects. To mitigate this security risk, vulnerability exploitation analysis has been proposed to further reduce false positives of vulnerability reachability analysis. However, existing approaches work less effectively when the vulnerable function of the vulnerable library is indirectly invoked by a client project through a call chain of multiple steps.

Null pointer dereference raises Null Pointer Exceptions (NPEs). There are two groups of approaches to detect NPEs. Type-based approaches carry out strict type-based null safety checking. They heavily rely on annotations, and thus produce many false positives. Dataflow-based approaches leverage static forward and/or backward dataflow analysis. They mostly have a limited capability in tracking fields and interprocedural analysis, and introduce false positives and false negatives. To address these drawbacks, we propose Wheeljack to detect NPEs for Java. It does not rely on annotations, and hence can work effectively under a lack of annotations. It leverages our novel abstraction of nullness status to enhance field tracking, and our novel invocation analysis (capturing change to return value and side effect of an invocation) to enhance interprocedural analysis. Our evaluation on 28 Java projects has demonstrated that Wheeljack can mostly outperform the four state-of-the-art NPE detectors in recall without sacrificing precision. 5 and 2 new NPEs have been confirmed and fixed by developers after we submit 8 issues.

Machine learning (ML) enabled systems are emerging with recent breakthroughs in ML. A model-centric view is widely taken by the literature to focus only on the analysis of ML models. However, only a small body of work takes a system view that looks at how ML components work with the system and how they affect software engineering for ML-enabled systems. In this paper, we adopt this system view, and conduct a case study on Rasa 3.0, an industrial dialogue system that has been widely adopted by various companies around the world. Our goal is to characterize the complexity of such a large-scale ML-enabled system and to understand the impact of the complexity on testing. Our study reveals practical implications for software engineering for ML-enabled systems.

Although deep learning (DL) software has been pervasive in various applications, the brittleness of deep neural networks (DNN) hinders their deployment in many tasks especially high-stake ones. To mitigate the risk accompanied with DL software fault, a variety of DNN testing techniques have been proposed such as test case selection. Among those test case selection or prioritization methods, the uncertainty-based ones such as DeepGini have demonstrated their effectiveness in finding DNN’s faults. Recently, TestRank, a learning based test ranking method has shown their out-performance over simple uncertainty-based test selection methods. However, this is achieved with a more complicated design which needs to train a graph convolutional network and a multi-layer Perceptron. In this paper, we propose a novel and lightweight DNN test selection method to enhance the effectiveness of existing simple ones. Besides the DNN model’s uncertainty on test case itself, we take into account model’s uncertainty on its neighbors. This could diversify the selected test cases and improve the effectiveness of existing uncertainty-based test selection methods. Extensive experiments on 5 datasets demonstrate the effectiveness of our approach.

Bugs and their fixes in the code evolution histories are important assets for many software engineering tasks such as deriving new state-of-the-art automatic bug fixing techniques. Existing bug datasets are either manually built which is difficult to grow efficiently to a scale large enough for massive data analysis, or lack of precise information of how bugs are introduced and fixed which is critical for in-depth analysis such as buggy/fixing code identification. Moreover, the types of the bugs are typically missing in the existing bug datasets, limiting the possibility of developing high-precision type-specific approaches for enterprise-level purposes. In this work, we propose BugMiner, an approach to automatically collecting bugs from code repositories by isolating the critical changes of the bugs. We also propose a learning-based approach for automating bug type classification with relatively small manual labels of bug types. We evaluate our approach regarding the precision of bug information and the efficiency of the bug-mining process with 2,082 bugs automatically mined from 100 open-source projects. We demonstrate the improved effectiveness and efficiency in bug-fixing location identification, compared to the SOTA BugBuilder, and high recall and precision in bug-inducing location identification. We also compare our learning-based bug classification approach to traditional baseline method, indicating about 17 % improvement in classification effectiveness under macro-F1.

Deep learning (DL) applications, built upon a heterogeneous and complex DL stack (e.g., Nvidia GPU, Linux, CUDA driver, Python runtime, and TensorFlow), are subject to software and hardware dependencies across the DL stack. One challenge in dependency management across the entire engineering lifecycle is posed by the asynchronous and radical evolution and the complex version constraints among dependencies. Developers may introduce dependency bugs (DBs) in selecting, using and maintaining dependencies. However, the characteristics of DBs in DL stack is still under-investigated, hindering practical solutions to dependency management in DL stack. To bridge this gap, this paper presents the first comprehensive study to characterize symptoms, root causes and fix patterns of DBs across the whole DL stack with 446 DBs collected from StackOverflow posts and GitHub issues. For each DB, we first investigate the symptom as well as the lifecycle stage and dependency where the symptom is exposed. Then, we analyze the root cause as well as the lifecycle stage and dependency where the root cause is introduced. Finally, we explore the fix pattern and the knowledge sources that are used to fix it. Our findings from this study shed light on practical implications on dependency management.

Code changes are at the very core of software development and maintenance. Deep learning techniques have been used to build a model from a massive number of code changes to solve software engineering tasks, e.g., commit message generation and bug-fix commit identification. However, existing code change representation learning approaches represent code change as lexical tokens or syntactical AST (abstract syntax tree) paths, limiting the capability to learn semantics of code changes. Besides, they mostly do not consider noisy or tangled code change, hurting the accuracy of solved tasks. To address the above problems, we first propose a slice-based code change representation approach which considers data and control dependencies between changed code and unchanged code. Then, we propose a pre-trained sparse Transformer model, named CCS2VEC, to learn code change representations with three pre-training tasks. Our experiments by fine-tuning our pre-trained model on three downstream tasks have demonstrated the improvement of CCS2VEC over the state-of-the-art CC2VEC.

Software performance is a critical quality attribute that determines the success of a software system. However, practitioners lack comprehensive and holistic understanding of how real-life performance issues are caused and resolved in practice from the technical, engineering, and economic perspectives. This paper presents a large-scale empirical study of 570 real-life performance issues from 13 open source projects from various problem domains, and implemented in three popular programming languages, Java (192 issues), C/C++ (162 issues), and Python (216 issues). From the technical perspective, we summarizeeightgeneral types of performance issues with corresponding root causes and resolutions that apply for all three languages. We also identify available tools for detecting and resolving different types of issues from the literature. In addition, we found that 27% of the 570 issues are resolved by design-level optimization—coordinated revision of a group of related source files and their design structure. We reveal four typical design-level optimization patterns, includingclassic design patterns,change propagation,optimization clone, andparallel optimizationthat practitioners should be aware of in resolving performance issues. From the engineering perspective, this study analyzes how test code changes in performance optimization. We found that only 15% of the 570 performance issues involve revision of test code. In most cases, the revised test cases focus on the functional logic of the performance optimization, rather than directly evaluate the performance improvement. This finding points to the potential lack of engineering standard for formally verifying performance optimization in regression testing. Finally, from the economic perspective, we analyze the“Return On Investment”of performance optimization. We found that design-level optimization usually requires more investment, but not always yields to higher performance improvement. However, developers tend to use design-level optimization when they concern about other quality attributes, such as maintainability and readability.

Third-party libraries with rich functionalities facilitate the fast development of JavaScript software, leading to the explosive growth of the NPM ecosystem. However, it also brings new security threats that vulnerabilities could be introduced through dependencies from third-party libraries. In particular, the threats could be excessively amplified by transitive dependencies. Existing research only considers direct dependencies or reasoning transitive dependencies based on reachability analysis, which neglects the NPM-specific dependency resolution rules as adapted during real installation, resulting in wrongly resolved dependencies. Consequently, further fine-grained analysis, such as precise vulnerability propagation and their evolution over time in dependencies, cannot be carried out precisely at a large scale, as well as deriving ecosystem-wide solutions for vulnerabilities in dependencies.

Test failures are one of the most common reasons for broken builds in continuous integration. It is expensive to diagnose all test failures in a build. As test failures are usually caused by a few underlying faults, triaging test failures with respect to their underlying root causes can save test failure diagnosis cost. Existing failure triage methods are mostly developed for triaging crash or bug reports, and hence not applicable in the context of test failure triage in continuous integration. In this paper, we first present a large-scale empirical study on 163,371 broken builds caused by test failures to characterize test failures in real-world Java projects. Then, motivated by our study, we propose a new change-aware approach, BuildSheriff, to triage test failures in each continuous integration build such that test failures with the same root cause are put in the same cluster. Our evaluation on 200 broken builds has demonstrated that BuildSheriff can significantly improve the state-of-the-art methods on the triaging effectiveness.

Despite the benefits, continuous integration (CI) can incur high costs. One of the well-recognized costs is long build time, which greatly affects the speed of software development and increases the cost in computational resources. While there exist configuration options in the CI infrastructure to accelerate builds, the CI infrastructure is often not optimally configured, leading to CI configuration smells. Attempts have been made to detect or repair CI configuration smells. However, none of them is specifically designed to improve build performance in CI.

To enhance the compatibility in the version control of Java Third-party Libraries (TPLs), Maven adopts Semantic Versioning (SemVer) to standardize the underlying meaning of versions, but users could still confront abnormal execution and crash after upgrades even if compilation and linkage succeed. It is caused by semantic breaking (SemB) issues, such that APIs directly used by users have identical signatures but inconsistent semantics across upgrades. To strengthen compliance with SemVer rules, developers and users should be alerted of such issues. Unfortunately, it is challenging to detect them statically, because semantic changes in the internal methods of APIs are difficult to capture. Dynamic testing can confirmingly uncover some, but it is limited by inadequate coverage.

Deep learning (DL) has been widely applied to many domains. Unique challenges in engineering DL systems are posed by the programming paradigm shift from traditional systems to DL systems, and performance is one of the challenges. Performance problems (PPs) in DL systems can cause severe consequences such as excessive resource consumption and financial loss. While bugs in DL systems have been extensively investigated, PPs in DL systems have hardly been explored. To bridge this gap, we present the first comprehensive study to i) characterize symptoms, root causes, and introducing and exposing stages of PPs in DL systems developed in TensorFLow and Keras, with 224 PPs collected from 210 StackOverflow posts, and to ii) assess the capability of existing performance analysis approaches in tackling PPs, with a constructed benchmark of 58 PPs in DL systems. Our findings shed light on the implications on developing high-performance DL systems, and detecting and localizing PPs in DL systems. To demonstrate the usefulness of our findings, we develop a static checker DeepPerf to detect three types of PPs. It has detected 488 new PPs in 130 GitHub projects. 105 and 27 PPs have been confirmed and fixed.

Open source software (OSS) vulnerabilities threaten the security of software systems that use OSS. Vulnerability databases provide valuable information (e.g., vulnerable version and patch) to mitigate OSS vulnerabilities. There arises a growing concern about the information quality of vulnerability databases. However, it is unclear what the quality of patches in existing vulnerability databases is; and existing manual or heuristic-based approaches for patch tracking are either too expensive or too specific to apply to all OSS vulnerabilities.

Uncaught runtime exceptions have been recognized as one of the commonest root causes of real-life exception bugs in Java applications. However, existing runtime exception detection techniques rely on symbolic execution or random testing, which may suffer the scalability or coverage problem. Rule-based bug detectors (e.g., SpotBugs) provide limited rule support for runtime exceptions. Inspired by the recent successes in applying deep learning to bug detection, we propose a deep learning-based technique, named Drex, to identify not only the types of runtime exceptions that a method might signal but also the statement scopes that might signal the detected runtime exceptions. It is realized by graph-based code representation learning with (i) a lightweight analysis to construct a joint graph of CFG, DFG and AST for each method without requiring a build environment so as to comprehensively characterize statement syntax and semantics and (ii) an attention-based graph neural network to learn statement embeddings in order to distinguish different types of potentially signaled runtime exceptions with interpretability. Our evaluation on 54,255 methods with caught runtime exceptions and 54,255 methods without caught runtime exceptions from 5,996 GitHub Java projects has indicated that Drex improves baseline approaches by up to 18.2% in exact accuracy and 41.6% in F1-score. Drex detects 20 new uncaught runtime exceptions in 13 real-life pro-jects, 7 of them have been fixed, while none of them is detected by rule-based bug detectors (i.e., SpotBugs and PMD).

Annotations have been widely used in Java programs to support additional compile-time, deployment-time, and runtime processing. Developers use annotations to delegate repetitive logics such as object initialization and request forwarding to compilers and runtime frameworks. Therefore, these annotations are important for the correct execution of programs. In practice, however, developers often find it hard to correctly use annotations and the misuse of annotations has led to real bugs in Java programs. In this paper, we conduct an empirical study on Stack Overflow questions to investigate the major development frameworks that are involved in questions about Java annotations and the main problems encountered by developers in the use of Java annotations. Based on the findings of the study, we propose DeepAnna, a deep learning based Java annotation recommendation and misuse detection approach. Based on a corpus of Java programs with intensive use of annotations, DeepAnna trains a deep learning based multi-label classification model by considering both the structural and textual contexts of source code. DeepAnna can recommend annotations at both class level and method level. Our evaluation with a large corpus of open-source Java projects shows that DeepAnna outperforms state-of-the-art text multi-label classification approaches in annotation recommendation and can effectively detect annotation misuses. Based on our analysis, we submit 85 bug-fixing pull requests for annotation misuses in open-source projects and 20 of them have been accepted and merged.

Libraries are widely adopted in developing software projects. Library APIs are often missing during library evolution as library developers may deprecate, remove or refactor APIs. As a result, client developers have to manually find replacement APIs for missing APIs when updating library versions in their projects, which is a difficult and expensive software maintenance task. One of the key limitations of the existing automated approaches is that they usually consider the library itself as the single source to find replacement APIs, which heavily limits their accuracy.In this paper, we first present an empirical study to understand characteristics about missing APIs and their replacements. Specifically, we quantify the prevalence of missing APIs, and summarize the knowledge sources where the replacements are found, and the code change and mapping cardinality between missing APIs and their replacements. Then, inspired by the insights from our study, we propose a heuristic-based approach, REPFINDER, to automatically find replacements for missing APIs in library update. We design and combine a set of heuristics to hierarchically search three sources (deprecation message, own library, and external library) for finding replacements. Our evaluation has demonstrated that REPFINDER can find replacement APIs effectively and efficiently, and significantly outperform the state-of-the-art approaches.

Due to the broad attack surface and the lack of runtime protection, potential safety and security threats hinder the real-life adoption of autonomous vehicles. Although efforts have been made to mitigate some specific attacks, there are few works on the protection of the autonomous driving system, i.e., the control software system performing such as perception, decision making, and motion tracking. This article presents a decentralized self-protection framework called Guardauto to protect the autonomous driving system against runtime threats. First, Guardauto proposes an isolation model to decouple the autonomous driving system and isolate its components with a set of partitions. Second, Guardauto provides self-protection mechanisms for each target component, which combines different methods to monitor the target execution and plan adaption actions accordingly. Third, Guardauto provides cooperation among local self-protection mechanisms to identify the root-cause component in the case of cascading failures affecting multiple components. A prototype has been implemented and evaluated on the open-source autonomous driving system Autoware. Results show that Guardauto could effectively mitigate runtime failures and attacks, and protect the control system with acceptable performance overhead.

The design structure of a system continuously evolves as the consequence of fast-paced code revisions. Agile techniques, such as continuous testing, ensures the function goals of a system with every code revision. However, there lacks an efficient approach that can continuously model the design difference resulting from every single code revision to facilitate comprehension and ensure the design quality. This paper contributes a novel design modeling approach, called Design Differencing (DESIGNDIFF), that models and visualizes the highlevel design differences resulting from every code revision. This paper defines a complete and general set of 17 design change operators to capture the design difference from any code revision. We evaluated the potential of DESIGNDIFF in three aspects. First, a user study of 10 developers indicated that DESIGNDIFF can help practitioners to faster and better understand high-level design differences from real-life software commits. Second, DESIGNDIFF analyzed 14,832 real-life commits in five real-life projects: finding 4,189 commits altered the software design, 855 commits introduced and 337 commits eliminated design flaws. The latency between the flaw introduction and elimination is on average 2 months to 2 years! With an affordable performance overhead, DESIGNDIFF has great potential to benefit practitioners in more applications.

Performance issues widely exist in modern software systems. Existing performance optimization approaches, such as dynamic profiling, usually fail to consider the impacts of architectural connections among methods on performance issues. This paper contributes an architectural approach, Butterfly Space modeling, to investigate performance issues. Each Butterfly Space is composed of 1) a seed method; 2) methods in the "upper wing" that call the seed directly or transitively; and 3) methods in the "lower wing" that are called by the seed, directly or transitively. The rationale is that the performance of the seed method impacts and is impacted by all the other methods in the space because of the call relationship. As such, developers can more efficiently investigate groups of connected performance improvement opportunities in Butterfly Spaces. We studied three real-world open source Java projects to evaluate such potential. Our findings are three-fold: 1) If the seed method of a Butterfly Space contains performance problems, up to 60% of the methods in the space also contain performance problems; 2) Butterfly Spaces can potentially help to non-trivially increase the precision/recall and reduce the costs in identifying performance improvement opportunities, compared to dynamic profiling; and 3) Visualizing dynamic profiling metrics with Butterfly Spaces simultaneously help to reveal two typical patterns, namely Expensive Callee and Inefficient Caller, that are responsible for performance problems and provide insights on where to improve next. We believe that Butterfly Space modeling has great potential for investigating performance issues.

Third-party libraries play a key role in software development as they can relieve developers of the heavy burden of re-implementing common functionalities. However, third-party libraries and client projects evolve asynchronously. As a result, out-dated third-party libraries might be used in client projects while developers are not aware of the potential risk (e.g., security bug). Outdated third-party libraries may be updated in client projects in a delayed way, and developers may be less aware of the potential risk (e.g., API incompatibility) in updates. Developers of third-party libraries may be unaware of how their third-party libraries are used or updated in client projects. Therefore, a quantitative and holistic study on usages, updates and risks of third-party libraries in open-source projects can provide concrete evidences on these problems, and practical insights to improve the ecosystem. In this paper, we contribute such a study in Java ecosystem. In particular, we conduct a library usage analysis (e.g., usage intensity and outdatedness) and library update analysis (e.g., update intensity and delay) on 806 open-source projects and 13,565 third- party libraries. Then, we carry out a library risk analysis (e.g., usage risk and update risk) on 806 open-source projects and 544 security bugs. These analyses aim to quantify the usage and update practices and the potential risk of using and updating outdated third-party libraries with respect to security bugs from two holistic perspectives (i.e., open-source projects and third-party libraries). Our findings suggest practical implications to developers and researchers on problems and potential solutions in maintaining third-party libraries (e.g., smart alerting and automated updating of outdated third-party libraries). To indicate the usefulness of our findings, we design a smart alerting system for assisting developers to make confident decisions when updating third-party libraries. 33 and 24 open-source projects have confirmed and updated third-party libraries after receiving our alerts.

The binary-level function matching has been widely used to detect whether there are 1-day vulnerabilities in released programs. However, the high false positive is a challenge for current function matching solutions, since the vulnerable function is highly similar to its corresponding patched version. In this paper, the Binary X-Ray (BinXray), a patch based vulnerability matching approach, is proposed to identify the specific 1-day vulnerabilities in target programs accurately and effectively. In the preparing step, a basic block mapping algorithm is designed to extract the signature of a patch, by comparing the given vulnerable and patched programs. The signature is represented as a set of basic block traces. In the detection step, the patching semantics is applied to reduce irrelevant basic block traces to speed up the signature searching. The trace similarity is also designed to identify whether a target program is patched. In experiments, 12 real software projects related to 479 CVEs are collected. BinXray achieves 93.31% accuracy and the analysis time cost is only 296.17ms per function, outperforming the state-of-the-art works.

The dynamic typing discipline of Python allows developers to program at a high level of abstraction. However, type related bugs are commonly encountered in Python systems due to the lack of type declaration and static type checking. Especially, the misuse of dynamic typing discipline produces underlying bugs and increases maintenance efforts. In this paper, we introduce six types of dynamic typing related practices in Python programs, which are the common but potentially risky usage of dynamic typing discipline by developers. We also implement a tool named PYDYPE to detect them. Based on this tool, we conduct an empirical study on nine real-world Python systems (with the size of more than 460KLOC) to understand dynamic typing related practices. We investigate how widespread the dynamic typing related practices are, why they are introduced into the systems, whether their usage correlates with increased likelihood of bug occurring, and how developers fix dynamic typing related bugs. The results show that: (1) dynamic typing related practices exist inconsistently in different systems and Inconsistent Variable Types is most prevalent; (2) they are introduced into systems mainly during early development phase to promote development efficiency; (3) they have a significant positive correlation with bug occurring; (4) developers tend to add type checks or exception handling to fix dynamic typing related bugs. These results benefit future research in coding convention, language design, bug detection and fixing.

Long build times in continuous integration (CI) can greatly increase the cost in human and computing resources, and thus become a common barrier faced by software organizations adopting CI. Build outcome prediction has been proposed as one of the remedies to reduce such cost. However, the state-of-the-art approaches have a poor prediction performance for failed builds, and are not designed for practical usage scenarios. To address the problems, we first conduct an empirical study on 2,590,917 builds to characterize build times in real-world projects, and a survey with 75 developers to understand their perceptions about build outcome prediction. Then, motivated by our study and survey results, we propose a new history-aware approach, named BuildFast, to predict CI build outcomes cost-efficiently and practically. We develop multiple failure-specific features from closely related historical builds via analyzing build logs and changed files, and propose an adaptive prediction model to switch between two models based on the build outcome of the previous build. We investigate a practical online usage scenario of BuildFast, where builds are predicted in chronological order, and measure the benefit from correct predictions and the cost from incorrect predictions. Our experiments on 20 projects have shown that BuildFast improved the state-of-the-art by 47.5% in F1-score for failed builds.

As a mixed result of intensive dependency on third-party libraries, flexible mechanisms to declare dependencies and increased number of modules in a project, different modules of a project directly depend on multiple versions of the same third-party library. Such library version inconsistencies could increase dependency maintenance cost, or even lead to dependency conflicts when modules are inter-dependent. Although automated build tools (e.g., Maven's enforcer plugin) provide partial support to detect library version inconsistencies, they do not provide any support to harmonize inconsistent library versions.

Empirical experience regarding how real-life performance issues are caused and resolved can provide valuable insights for practitioners to effectively and efficiently prevent, detect, and fix performance issues. Prior work shows that most performance issues have their roots in poor architectural decisions. This paper contributes a large scale empirical study of 192 real-life performance issues, with an emphasis on software design. First, this paper contributes a holistic view of eight common root causes and typical resolutions that recur in different projects, and surveyed existing literature, in particular, tools, that can detect and fix each type of performance issue. Second, this study is first-of-its-kind to investigate performance issues from a design perspective. In the 192 issues, 33% required design-level optimization, i.e. simultaneously revising a group of related source files for resolving the issues. We reveal four design-level optimization patterns, which have shown different prevalence in resolving different root causes. Finally, this study investigated the Return on Investment for addressing performance issues, to help practitioners choose between localized or design-level optimization resolutions, and to prioritize issues due to different root causes.

Identifying potentially vulnerable locations in a code base is critical as a pre-step for effective vulnerability assessment; i.e., it can greatly help security experts put their time and effort to where it is needed most. Metric-based and pattern-based methods have been presented for identifying vulnerable code. The former relies on machine learning and cannot work well due to the severe imbalance between non-vulnerable and vulnerable code or lack of features to characterize vulnerabilities. The latter needs the prior knowledge of known vulnerabilities and can only identify similar but not new types of vulnerabilities. In this paper, we propose and implement a generic, lightweight and extensible framework, LEOPARD, to identify potentially vulnerable functions through program metrics. LEOPARD requires no prior knowledge about known vulnerabilities. It has two steps by combining two sets of systematically derived metrics. First, it uses complexity metrics to group the functions in a target application into a set of bins. Then, it uses vulnerability metrics to rank the functions in each bin and identifies the top ones as potentially vulnerable. Our experimental results on 11 real-world projects have demonstrated that, LEOPARD can cover 74.0% of vulnerable functions by identifying 20% of functions as vulnerable and outperform machine learning-based and static analysis-based techniques. We further propose three applications of LEOPARD for manual code review and fuzzing, through which we discovered 22 new bugs in real applications like PHP, radare2 and FFmpeg, and eight of them are new vulnerabilities.

In recent years, coverage-based greybox fuzzing has proven itself to be one of the most effective techniques for finding security bugs in practice. Particularly, American Fuzzy Lop (AFL for short) is deemed to be a great success in fuzzing relatively simple test inputs. Unfortunately, when it meets structured test inputs such as XML and JavaScript, those grammar-blind trimming and mutation strategies in AFL hinder the effectiveness and efficiency. To this end, we propose a grammar-aware coverage-based greybox fuzzing approach to fuzz programs that process structured inputs. Given the grammar (which is often publicly available) of test inputs, we introduce a grammar-aware trimming strategy to trim test inputs at the tree level using the abstract syntax trees (ASTs) of parsed test inputs. Further, we introduce two grammar-aware mutation strategies (i.e., enhanced dictionary-based mutation and tree-based mutation). Specifically, tree-based mutation works via replacing subtrees using the ASTs of parsed test inputs. Equipped with grammar-awareness, our approach can carry the fuzzing exploration into width and depth. We implemented our approach as an extension to AFL, named Superion; and evaluated the effectiveness of Superion using large- scale programs (i.e., an XML engine libplist and three JavaScript engines WebKit, Jerryscript and ChakraCore). Our results have demonstrated that Superion can improve the code coverage (i.e., 16.7% and 8.8% in line and function coverage) and bug-finding capability (i.e., 34 new bugs, among which we discovered 22 new vulnerabilities with 19 CVEs assigned and 3.2K USD bug bounty rewards received) over AFL and jsfunfuzz.

Continuous Integration (CI) is a widely-used software development practice to reduce risks. CI builds often break, and a large amount of efforts are put into troubleshooting broken builds. Despite that compiler errors have been recognized as one of the most frequent types of build failures, little is known about the common types, fix efforts and fix patterns of compiler errors that occur in CI builds of open-source projects. To fill such a gap, we present a large-scale empirical study on 6,854,271 CI builds from 3,799 open-source Java projects hosted on GitHub. Using the build data, we measured the frequency of broken builds caused by compiler errors, investigated the ten most common compiler error types, and reported their fix time. We manually analyzed 325 broken builds to summarize fix patterns of the ten most common compiler error types. Our findings help to characterize and understand compiler errors during CI and provide practical implications to developers, tool builders and researchers.

Vtable reuse attack, as a novel type of code reuse attacks, is introduced to bypass most binary-level control flow integrity enforcement and vtable integrity enforcement. So far, two binary-level defenses (TypeArmor and vfGuard) are proposed to defend against vtable reuse attacks. Both techniques use semantic information as the control flow integrity enforcement policy, i.e., TypeArmor and vfGuard utilize argument register count and dispatch offset at virtual callsite as the signature to check the validity of target functions, respectively. In this paper, we propose layered object-oriented programming (LOOP), an advanced vtable reuse attack, to show that the coarse-grained control flow integrity strategies are still vulnerable to vtable reuse attacks. In LOOP, we introduce argument expansion gadgets and transfer gadgets to, respectively, bypass TypeArmor and vfGuard. We generalize the characteristics of both gadgets and develop a tool to discover them at the binary level. We demonstrated that under the protection of TypeArmor and vfGuard, Firefox, Adobe Flash Player, and Internet Explorer are all vulnerable to LOOP attacks. Furthermore, we show the availability of argument expansion gadgets and transfer gadgets in common software or libraries.

Software systems are increasingly required to autonomously adapt their architectural structures and/or behaviors to runtime environmental changes. However, existing architecture-based self-adaptation approaches mostly focus on structural adaptations within a predefined space of architectural alternatives (e.g., switching between two alternative services) while merely considering quality constraints (e.g., reliability and performance). In this paper, we propose a new architecture-based self-adaptation approach, which performs behavioral adaptations with automatically generated alternatives and supports relaxed functional constraints from the perspective of business value. Specifically, we propose a technique to automatically generate behavioral alternatives of a software system from the currently-employed architectural behavioral specification. We employ business value to comprehensively evaluate the behavioral alternatives while capturing the trade-offs among relaxed functional and quality constraints. We also introduce a genetic algorithm-based planning technique to efficiently search for the optimal (sometimes a near-optimal) behavioral alternative that can provide the best business value. The experimental study on an online order processing benchmark has shown promising results that the proposed approach can improve adaptation flexibility and business value with acceptable performance overhead.

Analyzing loops is very important for various software engineering tasks such as bug detection, test case generation and program optimization. However, loops are very challenging structures for program analysis, especially when (nested) loops contain multiple paths that have complex interleaving relationships. In this paper, we propose the path dependency automaton (PDA) to capture the dependencies among the multiple paths in a loop. Based on the PDA, we first propose a loop classification to understand the complexity of loop summarization. Then, we propose a loop analysis framework, named Proteus, which takes a loop program and a set of variables of interest as inputs and summarizes path-sensitive loop effects (i.e., disjunctive loop summary) on the variables of interest. An algorithm is proposed to traverse the PDA to summarize the effect for all possible executions in the loop. We have evaluated Proteus using loops from five open-source projects and two well-known benchmarks and applying the disjunctive loop summary to three applications: loop bound analysis, program verification and test case generation. The evaluation results have demonstrated that Proteus can compute a more precise bound than the existing loop bound analysis techniques; Proteus can significantly outperform the state-of-the-art tools for loop program verification; and Proteus can help generate test cases for deep loops within one second, while symbolic execution tools KLEE and Pex either need much more time or fail.

Grey-box fuzzing is a practically effective approach to test real-world programs. However, most existing grey-box fuzzers lack directedness, i.e. the capability of executing towards user-specified target sites in the program. To emphasize existing challenges in directed fuzzing, we propose Hawkeye to feature four desired properties of directed grey-box fuzzers. Owing to a novel static analysis on the program under test and the target sites, Hawkeye precisely collects the information such as the call graph, function and basic block level distances to the targets. During fuzzing, Hawkeye evaluates exercised seeds based on both static information and the execution traces to generate the dynamic metrics, which are then used for seed prioritization, power scheduling and adaptive mutating. These strategies help Hawkeye to achieve better directedness and gravitate towards the target sites. We implemented Hawkeye as a fuzzing framework and evaluated it on various real-world programs under different scenarios. The experimental results showed that Hawkeye can reach the target sites and reproduce the crashes much faster than state-of-the-art grey-box fuzzers such as AFL and AFLGo. Specially, Hawkeye can reduce the time to exposure for certain vulnerabilities from about 3.5 hours to 0.5 hour. By now, Hawkeye has detected more than 41 previously unknown crashes in projects such as Oniguruma, MJS with the target sites provided by vulnerability prediction tools; all these crashes are confirmed and 15 of them have been assigned CVE IDs.

Performance problems widely exist in modern software systems. Existing performance optimization techniques, including profiling-based and pattern-based techniques, usually fail to consider the architectural impacts among methods that easily slow down the overall system performance. This paper contributes a new approach, named Speedoo, to identify groups of methods that should be treated together and deserve high priorities for performance optimization. The uniqueness of Speedoo is to measure and rank the performance optimization opportunities of a method based on 1) the architectural impact and 2) the optimization potential. For each highly ranked method, we locate a respective Optimization Space based on 5 performance patterns generalized from empirical observations. The top ranked optimization spaces are suggested to developers as potential optimization opportunities. Our evaluation on three real-life projects has demonstrated that 18.52% to 42.86% of methods in the top ranked optimization spaces indeed undertook performance optimization in the projects. This outperforms one of the state-of-the-art profiling tools YourKit by 2 to 3 times. An important implication of this study is that developers should treat methods in an optimization space together as a group rather than as individuals in performance optimization. The proposed approach can provide guidelines and reduce developers' manual effort.

Dangling pointers have become an important class of software bugs that can lead to use-after-free and double-free vulnerabilities. So far, only a few approaches have been proposed to protect against dangling pointers, while most of them suffer from high overhead. In this paper, we propose a lightweight approach, named DangDone, to eliminate dangling pointers at compile time. Built upon the root cause of a dangling pointer, i.e., a pointer and its aliases are not nullified but the memory area they point to is deallocated, DangDone realizes the protection by inserting an intermediate pointer between the pointers (i.e., a pointer and its aliases) and the memory area they point to. Hence, nullifying the intermediate pointer will nullify the pointer and its aliases, which mitigates the vulnerabilities caused by dangling pointers. Experimental results have demonstrated that DangDone can protect target programs (i.e., the SPEC CPU benchmarks and the programs with known CVEs) with negligible runtime overhead (i.e., around 1% on average).

Analyzing and understanding source code changes is important in a variety of software maintenance tasks. To this end, many code differencing and code change summarization methods have been proposed. For some tasks (e.g. code review and software merging), however, those differencing methods generate too fine-grained a representation of code changes, and those summarization methods generate too coarse-grained a representation of code changes. Moreover, they do not consider the relationships among code changes. Therefore, the generated differences or summaries make it not easy to analyze and understand code changes in some software maintenance tasks.

Greybox fuzzing is one of the most effective approaches for detecting software vulnerabilities. Various new techniques have been continuously emerging to enhance the effectiveness and/or efficiency by incorporating novel ideas into different components of a greybox fuzzer. However, there lacks a modularized fuzzing framework that can easily plugin new techniques and hence facilitate the reuse, integration and comparison of different techniques. To address this problem, we propose a fuzzing framework, namely Fuzzing Orchestration Toolkit (FOT). FOT is designed to be versatile, configurable and extensible. With FOT and its extensions, we have found 111 new bugs from 11 projects. Among these bugs, 18 CVEs have been assigned. Video link: https://youtu.be/O6Qu7BJ8RP0.

Software vulnerability is one of the major threats to software security. Once discovered, vulnerabilities are often fixed by applying security patches. In that sense, security patches carry valuable information about vulnerabilities, which could be used to discover, understand and fix (similar) vulnerabilities. However, most existing patch analysis approaches work at the source code level, while binary-level patch analysis often heavily relies on a lot of human efforts and expertise. Even worse, some vulnerabilities may be secretly patched without applying CVE numbers, or only the patched binary programs are available while the patches are not publicly released. These practices greatly hinder patch analysis and vulnerability analysis. In this paper, we propose a scalable binary-level patch analysis framework, named SPAIN, which can automatically identify security patches and summarize patch patterns and their corresponding vulnerability patterns. Specifically, given the original and patched versions of a binary program, we locate the patched functions and identify the changed traces (i.e., a sequence of basic blocks) that may contain security or non-security patches. Then we identify security patches through a semantic analysis of these traces and summarize the patterns through a taint analysis on the patched functions. The summarized patterns can be used to search similar patches or vulnerabilities in binary programs. Our experimental results on several real-world projects have shown that: i) SPAIN identified security patches with high accuracy and high scalability, ii) SPAIN summarized 5 patch patterns and their corresponding vulnerability patterns for 5 vulnerability types, and iii) SPAIN discovered security patches that were not documented, and discovered 3 zero-day vulnerabilities.

Coverage-based fuzzing is one of the most effective techniques to find vulnerabilities, bugs or crashes. However, existing techniques suffer from the difficulty in exercising the paths that are protected by magic bytes comparisons (e.g., string equality comparisons). Several approaches have been proposed to use heavy-weight program analysis to break through magic bytes comparisons, and hence are less scalable. In this paper, we propose a program-state based binary fuzzing approach, named Steelix, which improves the penetration power of a fuzzer at the cost of an acceptable slow down of the execution speed. In particular, we use light-weight static analysis and binary instrumentation to provide not only coverage information but also comparison progress information to a fuzzer. Such program state information informs a fuzzer about where the magic bytes are located in the test input and how to perform mutations to match the magic bytes efficiently. We have implemented Steelix and evaluated it on three datasets: LAVA-M dataset, DARPA CGC sample binaries and five real-life programs. The results show that Steelix has better code coverage and bug detection capability than the state-of-the-art fuzzers. Moreover, we found one CVE and nine new bugs.

Loop termination is an important problem for proving the correctness of a system and ensuring that the system always reacts. Existing loop termination analysis techniques mainly depend on the synthesis of ranking functions, which is often expensive. In this paper, we present a novel approach, named Loopster, which performs an efficient static analysis to decide the termination for loops based on path termination analysis and path dependency reasoning. Loopster adopts a divide-and-conquer approach: (1) we extract individual paths from a target multi-path loop and analyze the termination of each path, (2) analyze the dependencies between each two paths, and then (3) determine the overall termination of the target loop based on the relations among paths. We evaluate Loopster by applying it on the loop termination competition benchmark and three real-world projects. The results show that Loopster is effective in a majority of loops with better accuracy and 20 ×+ performance improvement compared to the state-of-the-art tools.

Programs that take highly-structured files as inputs normally process inputs in stages: syntax parsing, semantic checking, and application execution. Deep bugs are often hidden in the application execution stage, and it is non-trivial to automatically generate test inputs to trigger them. Mutation-based fuzzing generates test inputs by modifying well-formed seed inputs randomly or heuristically. Most inputs are rejected at the early syntax parsing stage. Differently, generation-based fuzzing generates inputs from a specification (e.g., grammar). They can quickly carry the fuzzing beyond the syntax parsing stage. However, most inputs fail to pass the semantic checking (e.g., violating semantic rules), which restricts their capability of discovering deep bugs. In this paper, we propose a novel data-driven seed generation approach, named Skyfire, which leverages the knowledge in the vast amount of existing samples to generate well-distributed seed inputs for fuzzing programs that process highly-structured inputs. Skyfire takes as inputs a corpus and a grammar, and consists of two steps. The first step of Skyfire learns a probabilistic context-sensitive grammar (PCSG) to specify both syntax features and semantic rules, and then the second step leverages the learned PCSG to generate seed inputs. We fed the collected samples and the inputs generated by Skyfire as seeds of AFL to fuzz several open-source XSLT and XML engines (i.e., Sablotron, libxslt, and libxml2). The results have demonstrated that Skyfire can generate well-distributed inputs and thus significantly improve the code coverage (i.e., 20% for line coverage and 15% for function coverage on average) and the bug-finding capability of fuzzers. We also used the inputs generated by Skyfire to fuzz the closed-source JavaScript and rendering engine of Internet Explorer 11. Altogether, we discovered 19 new memory corruption bugs (among which there are 16 new vulnerabilities and received 33.5k USD bug bounty rewards) and 32 denial-of-service bugs.

Analyzing performance and understanding the potential best-case, worst-case and distribution of program execution times are very important software engineering tasks. There have been model-based and program analysis-based approaches for performance analysis. Model-based approaches rely on analytical or design models derived from mathematical theories or software architecture abstraction, which are typically coarse-grained and could be imprecise. Program analysis-based approaches collect program profiles to identify performance bottlenecks, which often fail to capture the overall program performance. In this paper, we propose a performance analysis framework PerfPlotter. It takes the program source code and usage profile as inputs and generates a performance distribution that captures the input probability distribution over execution times for the program. It heuristically explores high-probability and low-probability paths through probabilistic symbolic execution. Once a path is explored, it generates and runs a set of test inputs to model the performance of the path. Finally, it constructs the performance distribution for the program. We have implemented PerfPlotter based on the Symbolic PathFinder infrastructure, and experimentally demonstrated that PerfPlotter could accurately capture the best-case, worst-case and distribution of program execution times. We also show that performance distributions can be applied to various important tasks such as performance understanding, bug validation, and algorithm selection.

Loops are challenging structures for program analysis, especially when loops contain multiple paths with complex interleaving executions among these paths. In this paper, we first propose a classification of multi-path loops to understand the complexity of the loop execution, which is based on the variable updates on the loop conditions and the execution order of the loop paths. Secondly, we propose a loop analysis framework, named Proteus, which takes a loop program and a set of variables of interest as inputs and summarizes path-sensitive loop effects on the variables. The key contribution is to use a path dependency automaton (PDA) to capture the execution dependency between the paths. A DFS-based algorithm is proposed to traverse the PDA to summarize the effect for all feasible executions in the loop. The experimental results show that Proteus is effective in three applications: Proteus can 1) compute a more precise bound than the existing loop bound analysis techniques; 2) significantly outperform state-of-the-art tools for loop verification; and 3) generate test cases for deep loops within one second, while KLEE and Pex either need much more time or fail.

In an uncertain and changing environment, a composite service needs to continuously optimize its business process and service selection through runtime adaptation. To achieve the overall satisfaction of stakeholder requirements, quality tradeoffs are needed to adapt the composite service in response to the changing environments. Existing approaches on service selection and composition, however, are mostly based on quality preferences and business processes decisions made statically at the design time. In this paper, we propose a requirements-driven self-optimization approach for composite services. It measures the quality of services (QoS), estimates the earned business value, and tunes the preference ranks through a feedback loop. The detection of unexpected earned business value triggers the proposed self-optimization process systematically. At the process level, a preference-based reasoner configures a requirements goal model according to the tuned preference ranks of QoS requirements, reconfiguring the business process according to its mappings from the goal configurations. At the service level, selection decisions are optimized by utilizing the tuned weights of QoS criteria. We used an experimental study to evaluate the proposed approach. Results indicate that the new approach outperforms both fixed-weighted and floating-weighted service selection approaches with respect to earned business value and adaptation flexibility.

A self-adaptive system uses runtime models to adapt its architecture to the changing requirements and contexts. However, there is no one-to-one mapping between the requirements in the problem space and the architectural elements in the solution space. Instead, one refined requirement may crosscut multiple architectural elements, and its realization involves complex behavioral or structural interactions manifested as architectural design decisions. In this paper we propose to combine two kinds of self-adaptations: requirements-driven self-adaptation, which captures requirements as goal models to reason about the best plan within the problem space, and architecture-based self-adaptation, which captures architectural design decisions as decision trees to search for the best design for the desired requirements within the contextualized solution space. Following these adaptations, component-based architecture models are reconfigured using incremental and generative model transformations. Compared with requirements-driven or architecture-based approaches, the case study using an online shopping benchmark shows promise that our approach can further improve the effectiveness of adaptation (e.g. system throughput in this case study) and offer more adaptation flexibility.

Requirements-driven approaches provide an effective mechanism for self-adaptive systems by reasoning over their runtime requirements models to make adaptation decisions. However, such approaches usually assume that the relations among alternative behaviours, environmental parameters and requirements are clearly understood, which is often simply not true. Moreover, they do not consider the influence of the current behaviour of an executing system on adaptation decisions. In this paper, we propose an improved requirements-driven self-adaptation approach that combines goal reasoning and case-based reasoning. In the approach, past experiences of successful adaptations are retained as adaptation cases, which are described by not only requirements violations and contexts, but also currently deployed behaviours. The approach does not depend on a set of original adaptation cases, but employs goal reasoning to provide adaptation solutions when no similar cases are available. And case-based reasoning is used to provide more precise adaptation decisions that better reflect the complex relations among requirements violations, contexts, and current behaviours by utilizing past experiences. Our experimental study with an online shopping benchmark shows that our approach outperforms both requirements-driven approach and case-based reasoning approach in terms of adaptation effectiveness and overall quality of the system.

Running in a highly uncertain and greatly complex environment, Web systems cannot always provide full set of services with optimal quality, especially when work loads are high or subsystem failures are frequent. Hence, it is significant to continuously maintain a high satisfaction level of survivability, hereafter survivability assurance, while relaxing or sacrificing certain quality or functional requirements that are not crucial to the survival of the entire system. After giving a value-based interpretation to survivability assurance to facilitate a quantitative analysis, we propose a requirements-driven self-tuning method for the survivability assurance of Web systems. Maintaining an enriched and live goal model, our method adapts to runtime tradeoff decisions made by our PID (proportional-integral-derivative) controller and goal-oriented reasoner for both quality and functional requirements. The goal-based configuration plans produced by the reasoner is carried out on the live goal model, and then mapped into system architectural configurations. Experiments on an online shopping system are conducted to validate the effectiveness of the proposed method.

Quality requirements of a software system cannot be optimally met, especially when it is running in an uncertain and changing environment. In principle, a controller at runtime can monitor the change impact on quality requirements of the system, update the expectations and priorities from the environment, and take reasonable actions to improve the overall satisfaction. In practice, however, existing controllers are mostly designed for tuning low-level performance indicators rather than high-level requirements. By linking the overall satisfaction to a business value indicator as feedback, we propose a control theoretic self-tuning method that can dynamically adjust the tradeoff decisions among different quality requirements. A preference-based reasoning algorithm is involved to configure hard goals accordingly to guide the following architecture reconfiguration.

The increasingly complex environments in which software systems are running today have made runtime software quality unstable and hardly in an optimal state, especially for those systems in open and dynamic environments, e.g. Internetware. In this paper, we explore the effectiveness of software cybernetics and feedback control theory in runtime software quality optimization. We propose a method of runtime quality optimization by using feedback control theory. Specially, we consider the problem of runtime optimization for a specific quality attribute, namely throughput, for Web-based systems. We design a double-layer feedback control model for the problem and implement the runtime optimization control method. In the method, runtime feedbacks are collected and used by the control model to adjust related control parameters. The experimental study has demonstrated the effectiveness of software cybernetics and feedback control theory in runtime quality optimization.