Merge conflicts often arise when developers integrate changes from different software branches. The conflicts can result from overlapping edits in programs (i.e., textual conflicts), or cause build and test errors (i.e., build and test conflicts). They degrade software quality and hinder programmer productivity. While several tools detect build conflicts, few offer meaningful support for resolving them. To overcome limitations of existing tools, we introduce BuCoR (BUild COnflict Resolver), a new conflict resolver. BuCoR first detects conflicts by comparing three versions related to a merging scenario: base b, left l, and right r. To resolve conflicts, it employs two complementary strategies: example-based transformation (BuCoR-E) and rule-based transformation (BuCoR-R). BuCoR-R applies predefined rules to resolve conflicts in frequently suggested or conventional ways. BuCoR-E mines branch versions (l and r) for exemplar edits applied to fix related build errors. From these examples, it infers and generalizes program transformation patterns to resolve conflicts in project-specific or unconventional ways.

As the innovation of smart devices and internet-of-things (IoT), smart homes have become prevalent. People tend to transform residences into smart homes by customizing off-the-shelf smart home platforms, instead of creating IoT systems from scratch. Among the alternatives, Home Assistant (HA) is one of the most popular platforms. It allows programmers (i.e., home residents or smart-home creators) to smartify homes by (S1) integrating selected devices into the system and (S2) programming YAML-based software to control those devices. Unfortunately, due to the diversity of devices and complexity of automatic configurations, many programmers have difficulty correctly creating YAML files. Consequently, their smart homes may not work as expected, causing frustration and concern in people. This article presents a novel study on issues of YAML-based automation configuration in smart homes (issues related to S2). We mined the online forum Home Assistant Community for discussion threads related to programming of automation configuration. By manually inspecting 190 threads, we revealed 3 categories of concerns: implementation, optimization, and debugging. Under each category, we classified discussions based on the issue locations and technical concepts involved. Among debugging discussions, we further classified discussions based on users’ resolution strategies; we also applied existing analysis tools to buggy YAML files, to assess the tool effectiveness. Our study reveals the common challenges faced by programmers and frequently applied resolution strategies. There are 129 (68%) examined issues concerning debugging, but existing tools can detect at most 14 of the issues and fix none. It implies that existing tools provide limited assistance in automation configuration. Our research sheds light on future directions in smart home programming.

In software development, many documents (e.g., tutorials for tools and mobile application websites) contain screenshots of GUIs to illustrate functionalities. Although screenshots are critical in such documents, screenshots can become outdated, especially if document developers forget to update them. Outdated screenshots can mislead users and diminish the credibility of documentation. Identifying screenshots manually is tedious and error-prone, especially when documents are numerous. However, no existing tools are proposed to detect outdated screenshots in GUI documents. To mitigate manual efforts, we propose DOSUD , a novel approach for detecting outdated screenshots. It is challenging to identify outdated screenshots since the differences are subtle and only specific areas are useful to identify such screenshots. To address the challenges, DOSUD automatically extracts and labels screenshots and trains a classification model to identify outdated screenshots. As the first exploration, we focus on Android applications and the most popular IDE, VS Code. We evaluated DOSUD on a benchmark comprising 10 popular applications, achieving high F1-scores. When applied in the wild, DOSUD identified 20 outdated screenshots across 50 Android application websites and 17 outdated screenshots in VS Code documentation. VS Code developers have confirmed and fixed all our bug reports.

Developers often build software on top of third-party libraries (Libs) to improve programmer productivity and software quality. The libraries may contain vulnerabilities exploitable by hackers to attack the applications (Apps) built on top of them. Such attacks are known as software supply chain attacks, the documented number of which has increased 742% in 2022. Researchers and developers created tools to mitigate such attacks, by scanning the library dependencies of Apps, identifying the usage of vulnerable library versions, and suggesting secure alternatives to vulnerable dependencies. However, recent studies show that many developers do not trust the reports by these tools; they need code or evidence to demonstrate how library vulnerabilities lead to security exploits, in order to assess vulnerability severity and modification necessity. Unfortunately, manually crafting demos of application-specific attacks is challenging and timeconsuming, and there is insufficient tool support to automate that procedure.To help developers enhance software security, in this study, we systematically explored the usage of a large language model (LLM)–ChatGPT-4.0–to generate security tests, which unit tests demonstrate how vulnerable library dependencies facilitate the supply chain attacks to givenApps. In our exploration, we defined prompt templates to take in the various vulnerability-relevant information we manually collected, and generated prompts from those templates to query ChatGPT for security test generation. We found that ChatGPT-generated tests demonstrated 24 evidence or proof of vulnerability for 49 Apps. To assess the consistency of test generation, we also evaluated another five state-of-the-art LLMs. All the models generated security tests for at least 17 cases that successfully demonstrate the vulnerabilities. We filed six reports for the newly revealed vulnerabilities in Apps, and got four Common Vulnerability Entries (CVEs) assigned. Our use of ChatGPT outperformed two state-of-the-art security test generators (TRANSFER and SIEGE), by generating a lot more tests and achieving more attacks. Our research will shed light on new research in security test generation.

When Computer Science (CS) students try to use or extend open-source software (OSS) projects, they often encounter the common challenge of OSS failing to build on their local machines. Even though OSS often provides ready-to-build packages, subtle differences in local environment setups can lead to build issues, costing students tremendous time and effort in debugging. Despite the prevalence of build issues faced by CS students, there is a lack of studies exploring this topic. To investigate the build issues frequently encountered by CS students and explore methods to help them resolve these issues, we conducted a novel dual-phase study involving 330 build tasks among 55 CS students. Phase I characterized the build issues students faced, their resolution attempts, and the effectiveness of those attempts. Based on these findings, Phase II introduced an intervention method that emphasized key information (e.g., recommended programming language versions) to students. The study demonstrated the effectiveness of our intervention in improving build success rates. Our research will shed light on future directions in related areas, such as CS education on best practices for software builds and enhanced tool support to simplify the build process.

Advanced persistent threats (APTs) have caused significant financial losses for enterprises, making the development of effective detection systems a critical priority. While existing provenance graph-based APT defenses demonstrate high accuracy, the high complexity and cost of graph operations (e.g., construction, iteration) require extensive processing time and computational resources, making them impractical for real-time detection. To address this challenge, we introduce Madeline, a graph-free, lightweight APT detection system that leverages historical system statistics. Through a multi-step state score calculation for a set of behavioral attributes, MADE-LINE meticulously captures subtle, gradual system changes indicative of stealthy APT activities. Using an LSTM autoencoder, Madeline performs anomaly detection effectively without the need for prior knowledge or manual labeling of attacks. Additionally, Madeline supports continuous monitoring, enhancing the assessment of ongoing risks. This feature helps reduce the need for extensive investigative resources by prioritizing genuine high-risk periods. Our experiments show that Madeline achieves comparable detection accuracy with the state-of-the-art APT detection, with 0.996 recall and 0.011 false positive rate on average. Madeline also significantly reduces the computational overhead, with over 1000x reduction in processing time and 5x in memory utilization.

API developers evolve software libraries to fix bugs, add new features, or refactor code, but the evolution can introduce API-breaking changes (e.g., API renaming). To benefit from such evolution, the programmers of client projects have to repetitively upgrade the callsites of libraries, since API-breaking changes introduce many compilation errors. It is tedious and error-prone to resolve such errors, especially when programmers are often unfamiliar with the API usages of newer versions. To migrate client code, the prior approaches either mine API mappings or learn edit scripts, but both the research lines have inherent limitations. For example, mappings alone cannot handle complex cases, and there is no sufficient source (e.g., migration commits) for learning edit scripts.

Developers regularly have to resolve merge conflicts, i.e., two conflicting sets of changes to the same files in different branches, which can be tedious and error-prone. To resolve conflicts, developers typically: keep the local version (KL) or the remote version (KR) of the code. They also sometimes manually edit both versions into a single one (ME). However, most existing techniques only support merging the local and remote versions (the ME strategy).

For many years now, modern software is known to be developed in multiple languages (hence termed as multilingual or multi-language software). Yet, to date, we still only have very limited knowledge about how multilingual software systems are constructed. For instance, it is not yet really clear how different languages are used, selected together, and why they have been so in multilingual software development. Given the fact that using multiple languages in a single software project has become a norm, understanding language use and selection (i.e., language profile ) as a basic element of the multilingual construction in contemporary software engineering is an essential first step. In this article, we set out to fill this gap with a large-scale characterization study on language use and selection in open-source multilingual software. We start with presenting an updated overview of language use in 7,113 GitHub projects spanning the 5 past years by characterizing overall statistics of language profiles, followed by a deeper look into the functionality relevance/justification of language selection in these projects through association rule mining. We proceed with an evolutionary characterization of 1,000 GitHub projects for each of the 10 past years to provide a longitudinal view of how language use and selection have changed over the years, as well as how the association between functionality and language selection has been evolving. Among many other findings, our study revealed a growing trend of using three to five languages in one multilingual software project and the noticeable stableness of top language selections. We found a non-trivial association between language selection and certain functionality domains, which was less stable than that with individual languages over time. In a historical context, we also have observed major shifts in these characteristics of multilingual systems both in contrast to earlier peer studies and along the evolutionary timeline. Our findings offer essential knowledge on the multilingual construction in modern software development. Based on our results, we also provide insights and actionable suggestions for both researchers and developers of multilingual systems.

In this article, we conduct a measurement study to comprehensively compare the accuracy impacts of multiple embedding options in cryptographic API completion tasks. Embedding is the process of automatically learning vector representations of program elements. Our measurement focuses on design choices of three important aspects, program analysis preprocessing , token-level embedding , and sequence-level embedding . Our findings show that program analysis is necessary even under advanced embedding. The results show 36.20% accuracy improvement, on average, when program analysis preprocessing is applied to transfer bytecode sequences into API dependence paths. With program analysis and the token-level embedding training, the embedding dep2vec improves the task accuracy from 55.80% to 92.04%. Moreover, only a slight accuracy advantage (0.55%, on average) is observed by training the expensive sequence-level embedding compared with the token-level embedding. Our experiments also suggest the differences made by the data. In the cross-app learning setup and a data scarcity scenario, sequence-level embedding is more necessary and results in a more obvious accuracy improvement (5.10%).

Extensive research has been conducted to explore cryptographic API misuse in Java. However, despite the tremendous popularity of the Python language, uncovering similar issues has not been fully explored. The current static code analysis tools for Python are unable to scan the increasing complexity of the source code. This limitation decreases the analysis depth, resulting in more undetected cryptographic misuses. In this research, we propose Cryptolation, a Static Code Analysis (SCA) tool that provides security guarantees for complex Python cryptographic code. Most existing analysis tools for Python solely focus on specific Frameworks such as Django or Flask. However, using a SCA approach, Cryptolation focuses on the language and not any framework. Cryptolation performs an inter-procedural data-flow analysis to handle many Python language features through variable inference (statically predicting what the variable value is) and SCA. Cryptolation covers 59 Python cryptographic modules and can identify 18 potential cryptographic misuses that involve complex language features. In this paper, we also provide a comprehensive analysis and a state-of-the-art benchmark for understanding the Python cryptographic Application Program Interface (API) misuses and their detection. Our state-of-the-art benchmark PyCryptoBench includes 1,836 Python cryptographic test cases that covers both 18 cryptographic rules and five language features. PyCryptoBench also provides a framework for evaluating and comparing different cryptographic scanners for Python. To evaluate the performance of our proposed cryptographic Python scanner, we evaluated Cryptolation against three other state-of-the-art tools: Bandit, Semgrep, and Dlint. We evaluated these four tools using our benchmark PyCryptoBench and manual evaluation of (four Top-Ranked and 939 Un-Ranked) real-world projects. Our results reveal that, overall, Cryptolation achieved the highest precision throughout our testing; and the highest accuracy on our benchmark. Cryptolation had 100% precision on PyCryptoBench, and the highest precision on the real-world projects.

Virtual Reality (VR) is an emerging technique that has been applied to more and more areas such as gaming, remote conference, and education. Since VR user interface has very different characteristics compared with traditional graphic user interface (GUI), VR applications also require new testing techniques for quality assurance. Recently, some frameworks (e.g., VRTest) have been proposed to automate VR user interface testing by automatically controlling the player camera. However, their testing strategies are not able to address VR-specific testing challenges such as object occlusion and movement. In this paper, we propose a novel testing technique called VRGuide to explore VR scenes more efficiently. In particular, VRGuide adapts a computer geometry technique called Cut Extension to optimize the camera routes for covering all interact-able objects. We compared the testing strategy with VRTest on eight top VR software projects with scenes. The results show that VRGuide is able to achieve higher test coverage upon testing timeout in two of the projects, and achieve saturation coverage with averagely 31% less testing time than VRTest on the remaining six projects. Furthermore, VRGuide detected and reported four unknown bugs confirmed by developers, only one of which is also detected by VRTest.

Spectrum-based fault localization (SBFL) techniques can aid in debugging, but their practicality in industrial settings has been limited due to the large number of tests needed to execute before applying SBFL. Previous research has explored different trigger modes for SBFL and found that applying it immediately after the first test failure is also effective. However, this study only considered single-location bugs, while multi-location bugs are prevalent in real-world scenarios and especially at our company Cvent, which is interested in integrating SBFL to its CI/CD workflow.

Continuous integration (CI) practices encourage developers to frequently integrate code into a shared repository. Each integration is validated by automatic build and testing such that errors are revealed as early as possible. When CI failures or integration errors are reported, existing techniques are insufficient to automatically locate the root causes for two reasons. First, a CI failure may be triggered by faults in source code and/or build scripts, whereas current approaches consider only source code. Second, a tentative integration can fail because of build failures and/or test failures, whereas existing tools focus on test failures only. This article presents UniLoc, the first unified technique to localize faults in both source code and build scripts given a CI failure log, without assuming the failure’s location (source code or build scripts) and nature (a test failure or not). Adopting the information retrieval (IR) strategy, UniLoc locates buggy files by treating source code and build scripts as documents to search and by considering build logs as search queries. However, instead of naïvely applying an off-the-shelf IR technique to these software artifacts, for more accurate fault localization, UniLoc applies various domain-specific heuristics to optimize the search queries, search space, and ranking formulas. To evaluate UniLoc, we gathered 700 CI failure fixes in 72 open source projects that are built with Gradle. UniLoc could effectively locate bugs with the average mean reciprocal rank value as 0.49, mean average precision value as 0.36, and normalized discounted cumulative gain value as 0.54. UniLoc outperformed the state-of-the-art IR-based tool BLUiR and Locus. UniLoc has the potential to help developers diagnose root causes for CI failures more accurately and efficiently.

In collaborative software development, programmers create software branches to add features and fix bugs tentatively, and then merge branches to integrate edits. When edits from different branches textually overlap (i.e., textual conflicts ) or lead to compilation and runtime errors (i.e., build and test conflicts ), it is challenging for developers to remove such conflicts. Prior work proposed tools to detect and solve conflicts. They investigate how conflicts relate to code smells and the software development process. However, many questions are still not fully investigated, such as what types of conflicts exist in real-world applications and how developers or tools handle them. For this article, we used automated textual merge, compilation, and testing to reveal three types of conflicts in 208 open-source repositories: textual conflicts, build conflicts (i.e., conflicts causing build errors), and test conflicts (i.e., conflicts triggering test failures). We manually inspected 538 conflicts and their resolutions to characterize merge conflicts from different angles. Our analysis revealed three interesting phenomena. First, higher-order conflicts (i.e., build and test conflicts) are harder to detect and resolve, while existing tools mainly focus on textual conflicts. Second, developers manually resolved most higher-order conflicts by applying similar edits to multiple program locations; their conflict resolutions share common editing patterns implying great opportunities for future tool design. Third, developers resolved 64% of true textual conflicts by keeping complete edits from either a left or right branch. Unlike prior studies, our research for the first time thoroughly characterizes three types of conflicts, with a special focus on higher-order conflicts and limitations of existing tool design. Our work will shed light on future research of software merge.

The Java platform provides various cryptographic APIs to facilitate secure coding. However, correctly using these APIs is challenging for developers who lack cybersecurity training. Prior work shows that many developers misused APIs and consequently introduced vulnerabilities into their software. To eliminate such vulnerabilities, people created tools to detect and/or fix cryptographic API misuses. However, it is still unknown (1) how current tools are designed to detect cryptographic API misuses, (2) how effectively the tools work to locate API misuses, and (3) how developers perceive the usefulness of tools’ outputs. For this paper, we conducted an empirical study to investigate the research questions mentioned above. Specifically, we first conducted a literature survey on existing tools and compared their approach design from different angles. Then we applied six of the tools to three popularly used benchmarks to measure tools’ effectiveness of API-misuse detection. Next, we applied the tools to 200 Apache projects and sent 57 vulnerability reports to developers for their feedback. Our study revealed interesting phenomena. For instance, none of the six tools was found universally better than the others; however, CogniCrypt, CogniGuard, and Xanitizer outperformed SonarQube. More developers rejected tools’ reports than those who accepted reports (30 versus 9) due to their concerns on tools’ capabilities, the correctness of suggested fixes, and the exploitability of reported issues. This study reveals a significant gap between the state-of-the-art tools and developers’ expectations; it sheds light on future research in vulnerability detection.

The Java libraries JCA and JSSE offer cryptographic APIs to facilitate secure coding. When developers misuse some of the APIs, their code becomes vulnerable to cyber-attacks. To eliminate such vulnerabilities, people built tools to detect security-API misuses via pattern matching. However, most tools do not (1) fix misuses or (2) allow users to extend tools' pattern sets. To overcome both limitations, we created Seader---an example-based approach to detect and repair security-API misuses. Given an exemplar (insecure, secure) code pair, Seader compares the snippets to infer any API-misuse template and corresponding fixing edit. Based on the inferred info, given a program, Seader performs inter-procedural static analysis to search for security-API misuses and to propose customized fixes.

In software merge, the edits from different branches can textually overlap (i.e., textual conflicts) or cause build and test errors (i.e., build and test conflicts), jeopardizing programmer productivity and software quality. Existing tools primarily focus on textual conflicts; few tools detect higher-order conflicts (i.e., build and test conflicts). However, existing detectors of build conflicts are limited. Due to their heavy usage of automatic build, current detectors (e.g., Crystal) only report build errors instead of identifying the root causes; developers have to manually locate conflicting edits. These detectors only help when the branches-to-merge have no textual conflict.

A play test is the process in which testers play video games for software quality assurance. Manual testing is expensive and time-consuming, especially when there are many mobile games to test and every game version requires extensive testing. Current testing frameworks (e.g., Android Monkey) are limited as they adopt no domain knowledge to play games. Learning-based tools (e.g., Wuji) require tremendous manual effort and ML expertise of developers. This paper presents LIT-a lightweight approach to generalize play test tactics from manual testing, and to adopt the tactics for automatic testing. Lit has two phases: tactic generalization and tactic concretization. In Phase I, when a human tester plays an Android game$G$for a while (e.g., eight minutes), Lit records the tester's inputs and related scenes. Based on the collected data, Lit infers a set of context-aware, abstract play test tactics that describe under what circumstances, what actions can be taken. In Phase II, LIttests$G$based on the generalized tactics. Namely, given a randomly generated game scene, Lit tentatively matches that scene with the abstract context of any inferred tactic; if the match succeeds, Lit customizes the tactic to generate an action for playtest. Our evaluation with nine games shows Lit to outperform two state-of-the-art tools and a reinforcement learning (RL)-based tool, by covering more code and triggering more errors. Lit complements existing tools and helps developers test various casual games (e.g., match3, shooting, and puzzles).

Ever since its first release in 2009, the Go programming language (Golang) has been well received by software communities. A major reason for its success is the powerful support of library-based development, where a Golang project can be conveniently built on top of other projects by referencing them as libraries. As Golang evolves, it recommends the use of a new library-referencing mode to overcome the limitations of the original one. While these two library modes are incompatible, both are supported by the Golang ecosystem. The heterogeneous use of library-referencing modes across Golang projects has caused numerous dependency management (DM) issues, incurring reference inconsistencies and even build failures. Motivated by the problem, we conducted an empirical study to characterize the DM issues, understand their root causes, and examine their fixing solutions. Based on our findings, we developed Hero, an automated technique to detect DM issues and suggest proper fixing solutions. We applied Hero to 19,000 popular Golang projects. The results showed that Hero achieved a high detection rate of 98.5% on a DM issue benchmark and found 2,422 new DM issues in 2,356 popular Golang projects. We reported 280 issues, among which 181 (64.6%) issues have been confirmed, and 160 of them (88.4%) have been fixed or are under fixing. Almost all the fixes have adopted our fixing suggestions.

Fault localization is important for software development and maintenance. Among existing techniques, spectrum-based fault localization (SBFL) is effective to locate bugs based on the execution coverage of passed and failed tests. However, current SBFL tools require the execution of all tests before suggesting any ranked list of suspicious locations. In reality, such all-test execution can be very time-consuming; SBFL's outputs based on the whole-suite execution can significantly delay developers' debugging activities and jeopardize their productivity. For this paper, we were curious whether we can apply SBFL immediately after seeing one or several test failures, instead of waiting for all tests to finish their run. Specifically, with 28 injected bugs and 13 real bugs in a close-sourced software product, we collected the statement-level coverage for each test case, and investigated the usage of 25 alternative SBFL formulas. We triggered SBFL in five modes: (i) after the first test failure, (ii) after the first failure and some extra passed tests, (iii) after every test failure, (iv) at a specified time interval (e.g., every 2 minutes), or (v) after the complete execution of all tests. Our study shows interesting results. Compared with whole-suite execution, triggering SBFL formulas earlier based on partial execution helps locate bugs more effectively. Among the five modes, the first-failure-driven mode works best. Additionally, we conducted similar experiments on 57 real bugs from the Defects4J dataset and observed similar phenomena. Our observations imply that instead of waiting for the completion of all test runs, it is quite promising to apply SBFL formulas immediately after the initial test failure. In this way, developers are likely to get better suggestions within a shorter period of time. Our research will help developers better adopt SBFL in practice.

Developers change software programs for various purposes (e.g., bug fixes, feature additions, and code refactorings), but the intents of code changes are often not recorded or are poorly documented. To automatically infer the change intent of each program commit (i.e., a set of code changes), existing work classifies commits based on commit messages and/or the sheer counts of edited files, lines, or abstract syntax tree (AST) nodes. However, none of these tools reason about the syntactic or semantic dependencies between co-applied changes, neither do they adopt any deep learning method. To better characterize program commits, in this paper, we present CClassifier—a new approach that classifies commits by (1) using advanced static program analysis to comprehend relationship between co-applied edits, (2) representing edits and their relationship via graphs, and (3) applying convolutional neural networks (CNN) to classify those graphs. Compared with prior work, CClassifier extracts a richer set of features from program changes; it is the first to classify program commits using CNN. For evaluation, we prepared a benchmark that contains 7,414 code changes from 5 open-source Java projects. On this benchmark, we empirically compared CClassifier and the state-of-the-art approach with five-fold cross validation. On average, when predicting bug-fixing commits within the same projects, CClassifier improved the prediction accuracy from 70% to 72%. More importantly, prior work seldom identifies feature-addition commits; CClassifier can successfully identify such commits in a lot more scenarios. Our evaluation shows that CClassifier outperforms prior work due to its usage of advanced program analysis and CNN.

Programming tasks provide an opportunity for students to improve their problem-solving skills (PSS). However, when programming tasks are challenging, students could become demotivated and lose the opportunity to improve PSS in the process. To scaffold the difficulty of programming tasks and better motivate students to enhance PSS via coding, this paper introduces PDL (Problem Description Language). Given the natural-language description of a combinatorial optimization problem (COP), PDL requires students to describe (i) inputs, (ii) constraints, (iii) the optimization objective, and (iv) outputs, based on their problem comprehension. PDL then validates each problem description by (1) compiling a solution program from the description and (2) executing the generated program with predefined test cases. Based on the compiling and testing results, PDL provides feedback to students, and assists students to adjust their problem comprehension and improve problem descriptions.

Combinatorial problems (CPs) arise in many areas, and people use constraint solvers to automatically solve these problems. However, the state-of-the-art constraint solvers (e.g., Gecode and Chuffed) have overly complicated software architectures; they compute solutions inefficiently. This paper presents a novel and model-driven approach---SoGen---to synthesize efficient problem-specific solvers from constraint models. Namely, when users model a CP with our domain-specific language PDL (short for Problem Description Language), SoGen automatically analyzes various properties of the problem (e.g., search space, value boundaries, function monotonicity, and overlapping subproblems), synthesizes an efficient solver algorithm based on those properties, and generates a C program as the problem solver. PDL is unique because it can create solvers that resolve constraints via dynamic programming (DP) search.

Among various bug reports (BRs), security bug reports (SBRs) are unique because they require immediate concealment and fixes. When SBRs are not identified in time, attackers can exploit the vulnerabilities. Prior work identifies SBRs via text mining, which requires a predefined keyword list and trains a classifier with known SBRs and non-security bug reports (NSBRs). The former approach is not reliable, because (1) as the contexts of security vulnerabilities and terminology of SBRs change over time, the predefined list will become out-dated; and (2) users may have insufficient SBRs for training. We introduce a semi-supervised learning-based approach, Sais, to adaptively and reliably identify SBRs. Given a project's BRs containing some labeled SBRs, many more NSBRs, and unlabeled BRs, Sais iteratively mines keywords, trains a classifier based on the keywords from the labeled data, classifies unlabeled BRs, and augments its training data with the newly labeled BRs. Our evaluation shows that Sais is useful for identifying SBRs.

Cryptographic API misuses seriously threatens software security. Automatic screening of cryptographic misuse vulnerabilities has been a popular and important line of research over the years. However, the vision of producing a scalable detection tool that developers can routinely use to screen millions of line of code has not been achieved yet. Our main technical goal is to attain a high precision and high throughput approach based on specialized program analysis. Specifically, we design inter-procedural program slicing on top of a new on-demand flow-, context- and field- sensitive data flow analysis. Our current prototype named CryptoGuard can detect a wide range of Java cryptographic API misuses with a precision of 98.61%, when evaluated on 46 complex Apache Software Foundation projects (including, Spark, Ranger, and Ofbiz). Our evaluation on 6,181 Android apps also generated many security insights. We created a comprehensive benchmark named CryptoApi-Bench with 40-unit basic cases and 131-unit advanced cases for in-depth comparison with leading solutions (e.g., SpotBugs, CrySL, Coverity). To make CryptoGuard widely accessible, we are in the process of integrating CryptoGuard with the Software Assurance Marketplace (SWAMP). SWAMP is a popular no-cost service for continuous software assurance and static code analysis.

Developers build programs based on software libraries to reduce coding effort. If a program inappropriately sets an API parameter, the program may exhibit unexpected runtime behaviors. To help developers correctly use library APIs, researchers built tools to mine API parameter rules. However, it is still unknown (1) what types of parameter rules there are, and (2) how these rules distribute inside documents and source files. In this paper, we conducted an empirical study to investigate the above-mentioned questions. To analyze as many parameter rules as possible, we took a hybrid approach that combines automatic localization of constrained parameters with manual inspection. Our automatic approach---PaRu---locates parameters that have constraints either documented in Javadoc (i.e., document rules) or implied by source code (i.e., code rules). Our manual inspection (1) identifies and categorizes rules for the located parameters, and (2) establishes mapping between document and code rules. By applying PaRu to 9 widely used libraries, we located 5,334 parameters with either document or code rules. Interestingly, there are only 187 parameters that have both types of rules, and 79 pairs of these parameter rules are unmatched. Additionally, PaRu extracted 1,688 rule sentences from Javadoc and code. We manually classified these sentences into six categories, two of which are overlooked by prior approaches. We found that 86.2% of parameters have only code rules; 10.3% of parameters have only document rules; and only 3.5% of parameters have both document and code rules. Our research reveals the challenges for automating parameter rule extraction. Based on our findings, we discuss the potentials of prior approaches and present our insights for future tool design.

Node.js has been popularly used for web application development, partially because of its large software ecosystem known as NPM (Node Package Manager) packages. When using open-source NPM packages, most developers download prebuilt packages on npmjs.com instead of building those packages from available source, and implicitly trust the downloaded packages. However, it is unknown whether the blindly trusted prebuilt NPM packages are reproducible (i.e., whether there is always a verifiable path from source code to any published NPM package). Therefore, for this paper, we conducted an empirical study to examine the reproducibility of NPM packages, and to understand why some packages are not reproducible.Specifically, we downloaded versions/releases of 226 most popularly used NPM packages and then built each version with the available source on GitHub. Next, we applied a differencing tool to compare the versions we built against versions downloaded from NPM, and further inspected any reported difference. Among the 3,390 versions of the 226 packages, only 2,087 versions are reproducible. Based on our manual analysis, multiple factors contribute to the non-reproducibility issues, such as flexible versioning information in package.json file and the divergent behaviors between distinct versions of tools used in the build process. Our investigation reveals challenges of verifying NPM reproducibility with existing tools, and provides insights for future verifiable build procedures.

When building enterprise applications on Java frameworks (e.g., Spring), developers often specify components and configure operations with a special kind of XML files named "deployment descriptors (DD)". Maintaining such XML files is challenging and time-consuming; because (1) the correct configuration semantics is domain-specific but usually vaguely documented, and (2) existing compilers and program analysis tools rarely examine XML files. To help developers ensure the quality of DD, this paper presents a novel approach---Xeditor---that extracts configuration couplings (i.e., frequently co-occurring configurations) from DD, and adopts the coupling rules to validate new or updated files.

Cryptographic API misuses seriously threaten software security. Automatic screening of cryptographic misuse vulnerabilities has been a popular and important line of research over the years. However, the vision of producing a scalable detection tool that developers can routinely use to screen millions of line of code has not been achieved yet. Our main technical goal is to attain a high precision and high throughput approach based on specialized program analysis. Specifically, we design inter-procedural program slicing on top of a new on-demand flow-, context- and field- sensitive data flow analysis. Our current prototype named CryptoGuard can detect a wide range of Java cryptographic API misuses with a precision of 98.61%,, when evaluated on 46 complex Apache Software Foundation projects (including, Spark, Ranger, and Ofbiz). Our evaluation on 6,181 Android apps also generated many security insights. We created a comprehensive benchmark named CryptoAPI-Bench with 40-unit basic cases and 131-unit advanced cases for in-depth comparison with leading solutions (e.g., SpotBugs, CrySL, Coverity). To make CryptoGuard widely accessible, we are in the process of integrating CryptoGuard with the Software Assurance Marketplace (SWAMP). SWAMP is a popular no-cost service for continuous software assurance and static code analysis.

Stack Overflow (SO) is the most popular online Q&A site for developers to share their expertise in solving programming issues. Given multiple answers to a certain question, developers may take the accepted answer, the answer from a person with high reputation, or the one frequently suggested. However, researchers recently observed that SO contains exploitable security vulnerabilities in the suggested code of popular answers, which found their way into security-sensitive high-profile applications that millions of users install every day. This observation inspires us to explore the following questions: How much can we trust the security implementation suggestions on SO? If suggested answers are vulnerable, can developers rely on the community's dynamics to infer the vulnerability and identify a secure counterpart? To answer these highly important questions, we conducted a comprehensive study on security-related SO posts by contrasting secure and insecure advice with the community-given content evaluation. Thereby, we investigated whether SO's gamification approach on incentivizing users is effective in improving security properties of distributed code examples. Moreover, we traced the distribution of duplicated samples over given answers to test whether the community behavior facilitates or prevents propagation of secure and insecure code suggestions within SO. We compiled 953 different groups of similar security-related code examples and labeled their security, identifying 785 secure answer posts and 644 insecure answer posts. Compared with secure suggestions, insecure ones had higher view counts (36,508 vs. 18,713), received a higher score (14 vs. 5), and had significantly more duplicates (3.8 vs. 3.0) on average. 34% of the posts provided by highly reputable so-called trusted users were insecure. Our findings show that based on the distribution of secure and insecure code on SO, users being laymen in security rely on additional advice and guidance. However, the community-given feedback does not allow differentiating secure from insecure choices. The reputation mechanism fails in indicating trustworthy users with respect to security questions, ultimately leaving other users wandering around alone in a software security minefield.

Constraint optimization problems (COP) on finite domains are typically solved via search. Many problems (e.g., 0-1 knapsack) involve redundant search, making a general constraint solver revisit the same subproblems again and again. Existing approaches use caching, symmetry breaking, subproblem dominance, or search with decomposition to prune the search space of constraint problems. In this paper we present a different approach--DPSolver--which uses dynamic programming (DP) to efficiently solve certain types of constraint optimization problems (COPs). Given a COP modeled with MiniZinc, DPSolver first analyzes the model to decide whether the problem is efficiently solvable with DP. If so, DPSolver refactors the constraints and objective functions to model the problem as a DP problem. Finally, DPSolver feeds the refactored model to Gecode--a widely used constraint solver--for the optimal solution. Our evaluation shows that DPSolver significantly improves the performance of constraint solving.

Developers build programs based on software libraries. When a library evolves, programmers need to migrate their client code from the library's old release(s) to new release(s). Due to the API backwards incompatibility issues, such code migration may require developers to replace API usage and apply extra edits (e.g., statement insertions or deletions) to ensure the syntactic or semantic correctness of migrated code. Existing tools extract API replacement rules without handling the additional edits necessary to fulfill a migration task. This paper presents our novel approach, Meditor, which extracts and applies the necessary edits together with API replacement changes. Meditor has two phases: inference and application of migration edits. For edit inference, Meditor mines open source repositories for migration-related (MR) commits, and conducts program dependency analysis on changed Java files to locate and cluster MR code changes. From these changes, Meditor further generalizes API migration edits by abstracting away unimportant details (e.g., concrete variable identifiers). For edit application, Meditor matches a given program with inferred edits to decide which edit is applicable, customizes each applicable edit, and produces a migrated version for developers to review. We applied Meditor to four popular libraries: Lucene, CraftBukkit, Android SDK, and Commons IO. By searching among 602,249 open source projects on GitHub, Meditor identified 1,368 unique migration edits. Among these edits, 885 edits were extracted from single updated statements, while the other 483 more complex edits were from multiple co-changed statements. We sampled 937 inferred edits for manual inspection and found all of them to be correct. Our evaluation shows that Meditor correctly applied code migrations in 218 out of 225 cases. This research will help developers automatically adapt client code to different library versions.

Android app testing is challenging and time-consuming because fully testing all feasible execution paths is difficult. Nowadays apps are usually tested in two ways: human testing or automated testing. Prior work compared different automated tools. However, some fundamental questions are still unexplored, including (1) how automated testing behaves differently from human testing, and (2) whether automated testing can fully or partially substitute human testing.

Most existing Android malware detection and categorization techniques are static approaches, which suffer from evasion attacks, such as obfuscation. By analyzing program behaviors, dynamic approaches are potentially more resilient against these attacks. Yet existing dynamic approaches mostly rely on characterizing system calls which are subject to system-call obfuscation. This paper presents DroidCat, a novel dynamic app classification technique, to complement existing approaches. By using a diverse set of dynamic features based on method calls and inter-component communication (ICC) Intents without involving permission, app resources, or system calls while fully handling reflection, DroidCat achieves superior robustness than static approaches as well as dynamic approaches relying on system calls. The features were distilled from a behavioral characterization study of benign versus malicious apps. Through three complementary evaluation studies with 34 343 apps from various sources and spanning the past nine years, we demonstrated the stability of DroidCat in achieving high classification performance and superior accuracy compared with the two state-of-the-art peer techniques that represent both static and dynamic approaches. Overall, DroidCat achieved 97% F1-measure accuracy consistently for classifying apps evolving over the nine years, detecting or categorizing malware, 16%-27% higher than any of the two baselines compared. Furthermore, our experiments with obfuscated benchmarks confirmed higher robustness of DroidCat over these baseline techniques. We also investigated the effects of various design decisions on DroidCat's effectiveness and the most important features for our dynamic classification. We found that features capturing app execution structure such as the distribution of method calls over user code and libraries are much more important than typical security features such as sensitive flows.

The Java platform and its third-party libraries provide useful features to facilitate secure coding. However, misusing them can cost developers time and effort, as well as introduce security vulnerabilities in software. We conducted an empirical study on StackOverflow posts, aiming to understand developers' concerns on Java secure coding, their programming obstacles, and insecure coding practices.

Researchers have recently proposed various automatic program repair (APR) approaches that reuse past fixes to fix new bugs. However, some fundamental questions, such as how new fixes overlap with old fixes, have not been investigated. Intuitively, the overlap between old and new fixes decides how APR approaches can construct new fixes with old ones. Based on this intuition, we systematically designed six overlap metrics, and performed an empirical study on 5,735 bug fixes to investigate the usefulness of past fixes when composing new fixes. For each bug fix, we created delta dependency graphs (i.e., program dependency graphs for code changes), and identified how bug fixes overlapped with each other in terms of the content, code structure, and identifier names of fixes. Our results show that if an APR approach composes new fixes by fully or partially reusing the content of past fixes, only 2.1% and 3.2% new fixes can be created from single or multiple past fixes in the same project, compared with 0.9% and 1.2% fixes created from past fixes across projects. However, if an APR approach composes new fixes by fully or partially reusing the code structure of past fixes, up to 41.3% and 29.7% new fixes can be created.

A flaky test is a test that may fail or pass for the same code under testing (CUT). Flaky tests could be harmful to developers because the non-deterministic test outcome is not reliable and developers cannot easily debug the code. A prior study characterized the root causes and fixing strategies of flaky tests by analyzing commits of 51 Apache open source projects, without analyzing any Android app. Due to the popular usage of Android devices and the multitude of interactions of Android apps with third-party software libraries, hardware, network, and users, we were curious to find if the Android apps manifested unique flakiness patterns and called for any special resolution for flaky tests as compared to the existing literature. For this paper, we conducted an empirical study to characterize the flaky tests in Android apps. By classifying the root causes and fixing strategies of flakiness, we aimed to investigate how our proposed characterization for flakiness in Android apps varies from prior findings, and whether there are domain-specific flakiness patterns. After mining GitHub, we found 29 Android projects containing 77 commits that were relevant to flakiness. We identified five root causes of Android apps' flakiness. We revealed three novel causes -Dependency, Program Logic, and UI. Five types of resolution strategies were observed to address the flaky behavior. Many of the examined commits show developers' attempt to fix flakiness by changing software implementation in various ways. However, there are still 13% commits that simply skipped or removed the flaky tests. Our observations provide useful insights for future research on flaky tests of Android apps.

Prior studies showed that developers applied repeated bug fixes-similar or identical code changes-to multiple locations. According to the observation, researchers built tools to automatically generate candidate patches from the repeated bug-fixing patterns. However, all such research focuses on the recurring change patterns within single methods. We are curious whether there are also repeated bug fixes that change multiple program entities (e.g., classes, methods, and fields); and if so, how we can leverage such recurring change patterns to further help developers fix bugs. In this paper, we present a comprehensive empirical study on multi-entity bug fixes in terms of their frequency, composition, and semantic meanings. Specifically for each bug fix, we first used our approach InterPart to perform static inter-procedural analysis on partial programs (i.e., the old and new versions of changed Java files), and to extract change dependency graphs (CDGs)-graphs that connect multiple changed entities based on their syntactic dependencies. By extracting common subgraphs from the CDGs of different fixes, we identified the recurring change patterns. Our study on Aries, Cassandra, Derby, and Mahout shows that (1) 52-58% of bug fixes involved multi-entity changes; (2) 6 recurring change patterns commonly exist in all projects; and (3) 19-210 entity pairs were repetitively co-changed mainly because the pairs invoked the same methods, accessed the same fields, or contained similar content. These results helped us better understand the gap between the fixes generated by existing automatic program repair (APR) approaches and the real fixes. Our observations will shed light on the follow-up research of automatic program comprehension and modification.

When many clones are detected in software programs, not all clones are equally important to developers. To help developers refactor code and improve software quality, various tools were built to recommend clone-removal refactorings based on the past and the present information, such as the cohesion degree of individual clones or the co-evolution relations of clone peers. The existence of these tools inspired us to build an approach that considers as many factors as possible to more accurately recommend clones. This paper introduces CREC, a learning-based approach that recommends clones by extracting features from the current status and past history of software projects. Given a set of software repositories, CREC first automatically extracts the clone groups historically refactored (R-clones) and those not refactored (NR-clones) to construct the training set. CREC extracts 34 features to characterize the content and evolution behaviors of individual clones, as well as the spatial, syntactical, and co-change relations of clone peers. With these features, CREC trains a classifier that recommends clones for refactoring. We designed the largest feature set thus far for clone recommendation, and performed an evaluation on six large projects. The results show that our approach suggested refactorings with 83% and 76% F-scores in the within-project and cross-project settings. CREC significantly outperforms a state-of-the-art similar approach on our data set, with the latter one achieving 70% and 50% F-scores. We also compared the effectiveness of different factors and different learning algorithms.

Programmers produce code clones when developing software. By copying and pasting code with or without modification, developers reuse existing code to improve programming productivity. However, code clones present challenges to software maintenance: they may require consistent application of the same or similar bug fixes or program changes to multiple code locations. To simplify the maintenance process, various tools have been proposed to automatically detect clones [1], [2], [3], [4], [5], [6]. Some tools tokenize source code, and then compare the sequence or frequency of tokens to reveal clones [1], [3], [4], [5]. Some other tools detect clones using tree-matching algorithms to compare the Abstract Syntax Trees (ASTs) of source code [2], [6]. In this paper, we present CCLEARNER, the first solely token-based clone detection approach leveraging deep learning. CCLEARNER extracts tokens from known method-level code clones and nonclones to train a classifier, and then uses the classifier to detect clones in a given codebase. To evaluate CCLEARNER, we reused BigCloneBench [7], an existing large benchmark of real clones. We used part of the benchmark for training and the other part for testing, and observed that CCLEARNER effectively detected clones. With the same data set, we conducted the first systematic comparison experiment between CCLEARNER and three popular clone detection tools. Compared with the approaches not using deep learning, CCLEARNER achieved competitive clone detection effectiveness with low time cost.

Programmers always fix bugs when maintaining software. Previous studies showed that developers apply repeated bug fixes-similar or identical code changes-to multiple locations. Based on the observation, researchers built tools to identify code locations in need of similar changes, or to suggest similar bug fixes to multiple code fragments. However, some fundamental research questions, such as what are the characteristics of repeated bug fixes, are still unexplored. In this paper, we present a comprehensive empirical study with 341,856 bug fixes from 3 open source projects to investigate repeated fixes in terms of their frequency, edit locations, and semantic meanings. Specifically, we sampled bug reports and retrieved the corresponding fixing patches in version history. Then we chopped patches into smaller fixes (edit fragments). Among all the fixes related to a bug, we identified repeated fixes using clone detection, and put a fix and its repeated ones into one repeated-fix group. With these groups, we characterized the edit locations, and investigated the common bug patterns as well as common fixes.Our study on Eclipse JDT, Mozilla Firefox, and LibreOffice shows that (1) 15-20% of bugs involved repeated fixes; (2) 73-92% of repeated-fix groups were applied purely to code clones; and (3) 39% of manually examined groups focused on bugs relevant to additions or deletions of whole if-structures. These results deepened our understanding of repeated fixes. They enabled us to assess the effectiveness of existing tools, and will further provide insights for future research directions in automatic software maintenance and program repair.

Bug localization is challenging and time-consuming. Given a bug report, a developer may spend tremendous time comprehending the bug description together with code in order to locate bugs. To facilitate bug report comprehension, information retrieval (IR)-based bug localization techniques have been proposed to automatically search for and rank potential buggy code elements (i.e., classes or methods). However, these techniques do not leverage any dynamic execution information of buggy programs. In this paper, we perform the first systematic study on how dynamic execution information can help with static IR-based bug localization. More specifically, with the fixing patches and bug reports of 157 real bugs, we investigated the impact of various execution information (i.e. coverage, slicing, and spectrum) on three IR-based techniques: the baseline technique, BugLocator, and BLUiR. Our experiments demonstrate that both the coverage and slicing information of failed tests can effectively reduce the search space and improve IR-based techniques at both class and method levels. Using additional spectrum information can further improve bug localization at the method but not the class level. Some of our investigated ways of augmenting IR-based bug localization with execution information even outperform a state-of-the-art technique, which merges spectrum with an IR-based technique in a complicated way. Different from prior work, by investigating various easy-to-understand ways to combine execution information with IR-based techniques, this study shows for the first time that execution information can generally bring considerable improvement to IR-based bug localization.

When developers add features and fix bugs, they often make systematic edits-similar edits to multiple locations. Systematic edits may indicate that developers should instead refactor to eliminate redundancy. This paper explores this question by designing and implementing a fully automated refactoring tool called RASE, which performs clone removal. RASE (1) extracts common code guided by a systematic edit; (2) creates new types and methods as needed; (3) parameterizes differences in types, methods, variables, and expressions; and (4) inserts return objects and exit labels based on control and data flow. To our knowledge, this functionality makes RASE the most advanced refactoring tool for automated clone removal. We evaluate RASE with real-world systematic edits and compare to method based clone removal. RASE successfully performs clone removal in 30 of 56 method pairs (n=2) and 20 of 30 method groups (n≥3) with systematic edits. We find that scoping refactoring based on systematic edits (58%), rather than the entire method (33%), increases the applicability of automated clone removal. Automated refactoring is not feasible in the other 42% cases, which indicates that automated refactoring does not obviate the need for systematic editing.

Adding features and fixing bugs in software often require systematic edits which are similar, but not identical, changes to many code locations. Finding all edit locations and editing them correctly is tedious and error-prone. In this paper, we demonstrate an Eclipse plug-in called Lase that (1) creates context-aware edit scripts from two or more examples, and uses these scripts to (2) automatically identify edit locations and (3) transform the code. In Lase, users can view syntactic edit operations and corresponding context for each input example. They can also choose a different subset of the examples to adjust the abstraction level of inferred edits. When Lase locates target methods matching the inferred edit context and suggests customized edits, users can review and correct LASE's edit suggestion. These features can reduce developers' burden in repetitively applying similar edits to different methods. The tool's video demonstration is available at https://www.youtube.com/ watch?v=npDqMVP2e9Q.

Adding features and fixing bugs often require systematic edits that make similar, but not identical, changes to many code locations. Finding all the relevant locations and making the correct edits is a tedious and error-prone process for developers. This paper addresses both problems using edit scripts learned from multiple examples. We design and implement a tool called LASE that (1) creates a context-aware edit script from two or more examples, and uses the script to (2) automatically identify edit locations and to (3) transform the code. We evaluate LASE on an oracle test suite of systematic edits from Eclipse JDT and SWT. LASE finds edit locations with 99% precision and 89% recall, and transforms them with 91% accuracy. We also evaluate LASE on 37 example systematic edits from other open source programs and find LASE is accurate and effective. Furthermore, we confirmed with developers that LASE found edit locations which they missed. Our novel algorithm that learns from multiple examples is critical to achieving high precision and recall; edit scripts created from only one example produce too many false positives, false negatives, or both. Our results indicate that LASE should help developers in automating systematic editing. Whereas most prior work either suggests edit locations or performs simple edits, LASE is the first to do both for nontrivial program edits.

Software modifications are often systematic ---they consist of similar, but not identical, program changes to multiple contexts. Existing tools for systematic program transformation are limited because they require programmers to manually prescribe edits or only suggest a location to edit with a related example. This paper presents the design and implementation of a program transformation tool called SYDIT. Given an example edit, SYDIT generates a context-aware, abstract edit script, and then applies the edit script to new program locations. To correctly encode a relative position of the edits in a new location, the derived edit script includes unchanged statements on which the edits are control and data dependent. Furthermore, to make the edit script applicable to a new context using different identifier names, the derived edit script abstracts variable, method, and type names. The evaluation uses 56 systematic edit pairs from five large software projects as an oracle. SYDIT has high coverage and accuracy. For 82% of the edits (46/56), SYDIT matches the context and applies an edit, producing code that is 96% similar to the oracle. Overall, SYDIT mimics human programmers correctly on 70% (39/56) of the edits. Generation of edit scripts seeks to improve programmer productivity by relieving developers from tedious, error-prone, manual code updates. It also has the potential to guide automated program repair by creating program transformations applicable to similar contexts.

Bug fixes and feature additions to large code bases often require systematic edits-similar, but not identical, coordinated changes to multiple places. This process is tedious and error-prone. Our prior work introduces a systematic editing approach that creates generalized edit scripts from exemplar edits and applies them to user-selected targets. This paper describes how the Sydit plug-in integrates our technology into the Eclipse integrated development environment. A programmer provides an example edit to Sydit that consists of an old and new version of a changed method. Based on this one example, Sydit generates a context-aware, abstract edit script. To make transformations applicable to similar but not identical methods, Sydit encodes control, data, and containment dependences and abstracts position, type, method, and variable names. Then the programmer selects target methods and Sydit customizes the edit script to each target and displays the results for the programmer to review and approve. Sydit thus automates much of the systematic editing process. To fully automate systematic editing, future tool enhancements should include automated selection of targets and testing of Sydit generated edits.

Monitoring sensitive events is of great significance for adaptive software. Specifying a sufficient set of constraints prior to software deployment is necessary to detect the presence of such events during execution. Here we propose a visual constraint specifying approach, which can generate monitoring code automatically, so as to make the constraint specification easy, and the adaptation of software easy. The detailed representing approach, generating algorithm, and a typical implementation are introduced also