Large Language Model-Generated Code (LLMgCode) has become increasingly common in software development. So far LLMgCode has more quality issues than Human-Authored Code (HaCode). It is common for LLMgCode to mix with HaCode in a code change, while the change is signed by only human developers, without being carefully examined. Many automated methods have been proposed to detect LLMgCode from HaCode, in which the perplexity-based method ( Perplexity for short) is the state-of-the-art method. However, the efficacy evaluation of Perplexity has focused on detection accuracy. Yet it is unclear whether Perplexity is good enough in a wider range of realistic evaluation settings. To this end, we carry out a family of experiments to compare Perplexity against feature- and pre-training-based methods from three perspectives: detection accuracy , detection speed , and generalization capability . The experimental results show that Perplexity has the best generalization capability while having limited detection accuracy and detection speed. Based on that, we discuss the strengths and limitations of Perplexity , e.g., Perplexity is unsuitable for high-level programming languages. Finally, we provide recommendations to improve Perplexity and apply it in practice. As the first large-scale investigation on detecting LLMgCode from HaCode, this article provides a wide range of findings for future improvement.

Small and focused commits are highly valued in modern software development. However, developers sometimes submit a commit with more than one concern, represented by several lines of code changes for a specific purpose, e.g., adding new features or fixing bugs. Such composite commits confuse developers during code reviews as well as other software activities, resulting in various issues. Existing studies predominantly leverage code structure to untangle composite commits, but without considering code semantics that have been demonstrated to be important in many related studies. In this article, we propose UNTCC, a new approach that uses structural and semantic information forUNTanglingCompositeCommits. To achieve structural information, we propose the code change graph, a fine-grained, text-attributed graph representation of a commit, incorporating before-change and after-change code dependencies; and UNTCC employs the graph autoencoder to learn its structural representation. To achieve semantic information, UNTCC leverages a large language model (Llama-3.2-3B) to learn joint embeddings of the raw commit and its aligned graph representation, which guide the division of different concerns within a composite commit. The experimental evaluation using 27,853 composite commits from 9 C# and 10 Java projects shows that in terms of Accuracya/Accuracyc, UNTCC achieves 94%/74% in C# and 77%/54% in Java, outperforming state-of-the-art approaches by 2%—623%/32%—573% in C# and 22%—285%/35%—286% in Java. The results indicate that UNTCC can effectively untangle composite commits.

Commit messages are valuable resources for describing why code changes are committed to repositories in version control systems (e.g., Git).They effectively help developers understand code changes and better perform software maintenance tasks.Unfortunately, developers often neglect to write high-quality commit messages in practice.Therefore, a growing body of work is proposed to generate commit messages automatically.These works all demonstrated that how to organize and represent code changes is vital in generating good commit messages, including the use of fine-grained graphs or embeddings to better represent code changes.In this study, we choose an alternative way to condense code changes before generation, i.e., proposing brief yet concise text templates consisting of the following three parts: (1) summarized code changes, (2) elicited comments, and (3) emphasized code identifiers.Specifically, we first condense code changes by using our proposed templates with the help of a heuristic-based tool named ChangeScribe, and then fine-tune CodeLlama-7B on the pairs of our proposed templates and corresponding commit messages.Our proposed templates better utilize pre-trained language models, while being naturally brief and readable to complement generated commit messages for developers.

Modern software development reuses third-party libraries to cut costs but may introduce vulnerabilities. A critical practice is to verify the security of third-party libraries against public vulnerability reports. Many automated methods have been proposed to identify vulnerable libraries from vulnerability reports. Existing methods are designed for the generic identification of vulnerable libraries, considering the security of all software libraries. Generic identification is inherently challenging, resulting in limited accuracy. However, organizations only consider the security of libraries they trust and use, by self-managing a library whitelist. Therefore, we propose LibGuard, a framework to adapt existing methods to help organizations secure the libraries they use. LibGuard supplies a library whitelist for existing methods and filters the results according to a threshold, facilitating the discovery of risks overlooked by organizations while controlling false alarms. LibGuard is implemented in two ways. The first attaches the whitelist after existing methods. The second integrates the whitelist into existing methods. We evaluated LibGuard using 5,107 vulnerability reports and the library whitelist built from 79 Google projects and 29 Huawei projects. The results show that the two implementations of LibGuard increase the average F1 score by 10.25% and 11.77%, respectively. Moreover, LibGuard performs stably during the extension of whitelists. To our knowledge, this paper is the first study dedicated to securing self-managed third-party libraries, offering insights into adapting generic software security management to self-managed contexts.

Background:Along with the rapid development and widespread adoption of blockchain technology, many common practices have been summarized into blockchain-based design patterns for application development. However, the numerous and scattered patterns may cause confusion among practitioners. Therefore, adopting appropriate patterns to meet various requirements has become a major challenge, as it requires deep development experience and blockchain technology knowledge.Objective:To address this problem, this paper proposes a decision-support solution to assist with the selection of design patterns during the blockchain-based application development, including a layered taxonomy of design patterns, mappings of quality attributes with the patterns, and a decision model incorporating the taxonomy and mappings.Method:We collected 72 distinct and state-of-the-art design patterns via a Systematic Literature Review (SLR) to establish a layered taxonomy, and 18 unified quality attribute metrics were proposed for blockchain-based pattern assessment and mapping establishment. Based on the pattern taxonomy and quality attribute mappings, we developed a decision model that can provide intuitive guidance for pattern selection.Results:The proposed solution was evaluated through a case study in a seafood supply chain, in which we examined how well the decision model could help identify design flaws and provide reasonable solutions. Additionally, interviews and a questionnaire-based survey were conducted to measure the completeness, correctness, and usefulness of the proposed decision model. The evaluation results indicate that the proposed decision-support solution provides developers with comprehensive guidance, facilitates targeted decision making, and supports intuitive understanding.Conclusions:Our decision-support solution can improve the development efficiency of blockchain-based applications, especially in addressing potential design flaws, achieving targeted quality attributes, and reducing development costs.

Abstract “Shift To Left” is the cornerstone of the successful implementation of DevSecOps. By testing projects for vulnerabilities in the early stages of development, teams can save overall costs before security issues reach the build phase. As one of the popular practices in “Shift To Left,” the Software Composition Analysis (SCA) system aims to leverage the Software Bill of Materials (SBOM) to enhance software supply chain security. However, the SBOM lacks mature generation and distribution mechanisms, requiring incentive measures to drive industry consensus. Additionally, the data and tools associated with the SBOM lack effective record‐keeping and monitoring, making it challenging to ensure data integrity and tool security. Traditional SCA systems treat SBOM as a regular data format for external service provision, yet fail to solve problems such as lack of shared platforms, inability to guarantee data integrity and tool security, as well as issues with poor interoperation compatibility. This paper introduces blockchain technology into the SCA system, utilizing smart contracts to provide core SBOM tool services and microservices to improve the operational efficiency of smart contract deployment and maintenance. The proposed SCA system effectively provides a shared platform for SBOM with reliable data integrity, guaranteed tool security, and good interoperability.

Abstract By adopting agile and lean practices, DevOps aims to achieve rapid value delivery by speeding up development and deployment cycles, which however lead to more security concerns that cannot be fully addressed by an isolated security role only in the final stage of development. DevSecOps promotes security as a shared responsibility integrated into the DevOps process that seamlessly intertwines development, operations, and security from the start throughout to the end of cycles. While some companies have already begun to embrace this new strategy, both industry and academia are still seeking a common understanding of the DevSecOps movement. The goal of this study is to report the state‐of‐the‐practice of DevSecOps, including the impact of DevOps on security, practitioners' understanding of DevSecOps, and the practices associated with DevSecOps as well as the challenges of implementing DevSecOps. The authors used a mixed‐methods approach for this research. The authors carried out a grey literature review on DevSecOps, and surveyed the practitioners of DevSecOps in industry of China. The status quo of DevSecOps in industry is summarized. Three major software security risks are identified with DevOps, where the establishment of DevOps pipeline provides opportunities for security‐related activities. The authors classify the interpretations of DevSecOps into three core aspects of DevSecOps capabilities, cultural enablers, and technological enablers. To materialise the interpretations into daily software production activities, the recommended DevSecOps practices from three perspectives—people, process, and technology. Although a preliminary consensus is that DevSecOps is regarded as an extension of DevOps, there is a debate on whether DevSecOps is a superfluous term. While DevSecOps is attracting an increasing attention by industry, it is still in its infancy and more effort needs to be invested to promote it in both research and industry communities.

Hyperledger Fabric is another development of blockchain technology after Ethereum, which is more suitable as an operating platform for smart contracts. However, the testing technology of Hyperledger Fabric smart contracts (also known as chaincode) is not yet mature currently. Based on this, this paper studies the vulnerability detection of Golang chaincodes. Firstly, we summarize 17 kinds of Golang chaincode vulnerabilities by investigating existing research. Secondly, taking the high accuracy of dynamic detection and the high efficiency of static detection into consideration, we propose a chaincode vulnerability detection framework that combines the dynamic symbolic execution and the static abstract syntax tree analysis technology. We also implement a supporting-tool that can detect the above 15 types of vulnerabilities. Finally, we test the tool by 15 chaincodes collected from GitHub and unknown vulnerabilities were detected in 13 projects. The precision turned out to be 91% after manual inspection. In order to verify the recall rate, we manually inject 30 vulnerabilities into the collected chaincodes and all of them are detected. The evaluation results show the accuracy of the proposed vulnerability detection method for Hyperledger Fabric smart contracts.

Hyperledger Fabric has become one of the most widely used consortium blockchain frameworks with the ability to execute custom smart contracts. Performance modeling and network evaluation are necessary for performance estimation and optimization of the Fabric blockchain platform. The compatibility and effectiveness of existing performance modeling methods must be improved. For this reason, we proposed a compatible performance modeling method using queuing theory for Fabric considering the limited transaction pool. Taking the 2.0 version of Fabric as a case, we have established the model for the transaction process in the Fabric network. By analyzing the two-dimensional continuous-time Markov process of this model, we solved the system stationary equation and obtained the analytical expressions of performance indicators such as the system throughput, the system steady-state queue length, and the system’s average response time. We collected the required parameter values through the official test suite. An extensive analysis and simulation was performed to verify the accuracy and the effectiveness of the model and formula. We believe that this method can be extended to a wide range of scenarios in other blockchain systems.

Despite widespread agreement on the benefits of code review, its outcomes may not be as expected. The complications can undermine the purpose of the development process and even destroy the entire development cycle. Both academia and the industrial communities have invested a great deal of time and effort into code reviews. When a project team adheres to the best practices and creates a conducive environment, it is likely that code reviews could be conducted effectively and efficiently. By reviewing peer-reviewed scientific publications and gray literature on code review best practices, we summarized 57 practices as well as 19 code review pains that they address. Our review has shown that following best practices can ease the process of code review considerably. Multiple actionable practices are needed to support code review pains at the same time. To enable the adoption of best practices, OSS and industrial communities alike invest in integrating automatic techniques with code review tools. We hope that this review will provide researchers and practitioners with a comprehensive understanding of code review practices, aiding them in conducting code reviews more successfully.

Nowadays, software permeates almost every aspect of our lives. To produce complex and large-scale software products, a large number of software engineers are required. Accordingly, researchers and educators recognize the importance of Software Engineering Education (SEE), and many studies related to SEE have been published in recent years. To synthesize the large amount of research in SEE, some Systematic Literature Reviews (SLRs) focusing on different areas of SEE have been conducted and reported. However, due to their limited focuses, none of these SLRs is able to depict an overall state-of-the-art for SEE. To remedy this, we conducted a tertiary study on SEE, which identifies 26 relevant SLRs published between 2004 and 2019. By classifying and positioning these SLRs in two dimensions, i.e. the education methods/tools applied for SEE and the research topics related to SEE, we present a landscape of SEE, which locates the SLRs on SEE and their research dimensions. Further, we collected the issues studied in the published research and those that need to be addressed for instructors. This paper also discusses the challenges of the current SEE research landscape.

Context: Following on other scientific disciplines, such as health sciences, the use of Grey Literature (GL) has become widespread in Software Engineering (SE) research. Whilst the number of papers incorporating GL in SE is increasing, there is little empirically known about different aspects of the use of GL in SE research.

Context: Following on other scientific disciplines, such as health sciences, the use of grey literature (GL) is becoming widespread in Software Engineering (SE) research. Whilst the number of papers incorporating GL on SE is increasing, there is little empirically known about different aspects of the use of GL in SE research. In particular, there is a lack of excellent evaluation standard for the quality of GL. Aim: Our research is aimed at systematically reviewing the use of GL in SE, empirically exploring SE researchers' views on GL and providing a guide for using GL in SE and for quality assessment of the GL to be included. Method: We used a mixed-methods approach for this research. We carried out a Systematic Literature Review (SLR) of the use of GL in SE. Then we surveyed the authors of the papers included in the SLR (as GL users) and the invited experts in the SE community on the use of GL in SE research. Results: We systematically selected and reviewed 102 SE secondary studies that incorporate GL in SE research, from which we identified two groups based on their reporting: 1) 76 reviews only claim their use of GL; 2) 26 reviews report the results by including GL. We also obtained 20 replies from the GL users and 24 replies from the invited SE experts. Conclusion: There is no common understanding of the meaning of GL in SE. Researchers define the scopes and the definitions of GL in a variety of ways. We found five main reasons of using GL in SE research. The findings have enabled us to propose a conceptual model for how GL works in SE research lifecycle. In the next workThere is a need for research to develop guidelines for using GL in SE and for assessing quality of GL.

Context: Data synthesis is one of the most significant tasks in Systematic Literature Review (SLR). Software Engineering (SE) researchers have adopted a variety of methods of synthesizing data that originated in other disciplines. One of the qualitative data synthesis methods is meta-ethnography, which is being used in SE SLRs. Objective: We aim at studying the adoption of meta-ethnography in SE SLRs in order to understand how this method has been used in SE. Method: We conducted a tertiary study of the use of meta-ethnography by reviewing sixteen SLRs. We carried out an empirical inquiry by integrating SLR and confirmatory email survey. Results: There is a general lack of knowledge, or even awareness, of different aspects of meta-ethnography and/or how to apply it. Conclusion: There is a need of investment in gaining in-depth knowledge and skills of correctly applying meta-ethnography in order to increase the quality and reliability of the findings generated from SE SLRs. Our study reveals that meta-ethnography is a suitable method to SE research. We discuss challenges and propose recommendations of adopting meta-ethnography in SE. Our effort also offers a preliminary checklist of the systematic considerations for doing meta-ethnography in SE and improving the quality of meta-ethnographic research in SE.

Software Engineering (SE) community has recently been investing significant amount of effort in qualitative research to study the human and social aspects of SE processes, practices, and technologies. Ethnography is one of the major qualitative research methods, which is based on constructivist paradigm that is different from the hypothetic-deductive research model usually used in SE. Hence, the adoption of ethnographic research method in SE can present significant challenges in terms of sufficient understanding of the methodological requirements and the logistics of its applications. It is important to systematically identify and understand various aspects of adopting ethnography in SE and provide effective guidance. We carried out an empirical inquiry by integrating a systematic literature review and a confirmatory survey. By reviewing the ethnographic studies reported in 111 identified papers and 26 doctoral theses and analyzing the authors' responses of 29 of those papers, we revealed several unique insights. These identified insights were then transformed into a preliminary checklist that helps improve the state-of-the-practice of using ethnography in SE. This study also identifies the areas where methodological improvements of ethnography are needed in SE.

Synthesizing data extracted from primary studies is an integral component of the methodologies in support of Evidence Based Software Engineering (EBSE) such as System Literature Review (SLR). Since a large and increasing number of studies in Software Engineering (SE) incorporate qualitative data, it is important to systematically review and understand different aspects of the Qualitative Research Synthesis (QRS) being used in SE. We have reviewed the use of QRS methods in 328 SLRs published between 2005 and 2015. We also inquired the authors of 274 SLRs to confirm whether or not any QRS methods were used in their respective reviews. 116 of them provided the responses, which were included in our analysis. We found eight QRS methods applied in SE research, two of which, narrative synthesis and thematic synthesis, have been predominantly adopted by SE researchers for synthesizing qualitative data. Our study determines that a significant amount of missing knowledge and incomplete understanding of the defined QRS methods in the community. Our effort also identifies an initial set factors that may influence the selection and use of appropriate QRS methods in SE.

Context: The assessment of Threats to Validity (TTVs) is critical to secure the quality of empirical studies in Software Engineering (SE). In the recent decade, Systematic Literature Review (SLR) was becoming an increasingly important empirical research method in SE. One of the mechanisms of insuring the level of scientific value in the findings of an SLR is to rigorously assess its validity. Hence, it is necessary to realize the status quo and issues of TTVs of SLRs in SE. Objective: This study aims to investigate the-state-of-the-practice of TTVs of the SLRs published in SE, and further support SE researchers to improve the assessment and strategies against TTVs in order to increase the quality of SLRs in SE. Method: We conducted a tertiary study by reviewing the SLRs in SE that report the assessment of TTVs. Results: We identified 316 SLRs published from 2004 to the first half of 2015, in which TTVs are discussed. The issues associated to TTVs were also summarized and categorized. Conclusion: The common TTVs related to SLR research, such as internal validity and reliability, were thoroughly discussed in most SLRs. The threats to construct validity and external validity drew less attention. Moreover, there are few strategies and tactics being reported to cope with the various TTVs.