In vulnerability detection approaches based on deep learning, fine-tuning with Pre-trained Language Models (PLMs) is a prevalent technique. Unfortunately, a natural gap exists between model pre-training tasks and vulnerability detection tasks due to different input formats, and the performance of fine-tuning relies on downstream dataset scales. Recently, prompt tuning has been used to alleviate these issues. However, it has not received enough attention in vulnerability detection. To assess the effectiveness of prompt tuning, we consider three classical vulnerability detection tasks: within-domain vulnerability detection, cross-domain vulnerability detection, and vulnerability type detection. Our empirical study considers three popular PLMs: CodeBERT, CodeT5, and CodeGPT. Then we use Devign, BigVul, and Reveal datasets as our experimental subjects. Our empirical results indicate that (1) compared to fine-tuning, prompt tuning can increase the accuracy of three tasks by an average of 42 %, 38%, and 41 %, respectively; (2) different prompt templates can have up to an 8 % impact on accuracy; (3) in data scarcity scenarios, the superiority of prompt tuning over fine-tuning is more obvious. Our research demonstrates that using prompt tuning can help to achieve better performance in vulnerability detection tasks and is a promising research direction in the future.

The code comment generation task is designed for developers to understand programs more quickly during development and maintenance.However, the existing automatic code comment generation models can not generate valuable comments for developers.It is necessary to explore a technology that can optimize the performance of code comment generation models without changing the model.We consider adversarial training as the experimental object, which can improve the robustness and generalization of the model.We present a large-scale study to experimentally validate the performance of gradient-based adversarial training methods in the code comment generation task.The results show that adversarial training can improve the model performance by generating adversarial examples without changing the model.Our empirical study can provide a new perspective for researchers to improve the performance of code comment generation models.

Fault localization techniques have been developed for decades. Spectrum Based Fault Localization (SBFL) is a popular strategy in this research topic. However, SBFL is well known for low accuracy, mainly due to simply using a coverage matrix of program executions. In this paper, we propose a method based on graph neural network (AGFL), characterized by the adjacent matrix of the abstract syntax tree and the word vector of each program token. Referring to the Dstar, we calculate the suspiciousness of the statements and rank these statements. The experiment carried on Defects4J, a widely used benchmark, reveals that AGFL can locate 178 of the 262 studied bugs within Top-1, while state-of-the-art techniques at most locate 148 within Top-1. We also investigate the impacts of hyper-parameters (e.g., epoch and learning rate). The results show that AGFL has the best effect when the epoch is 100 and the learning rate is 0.0001. This value of epoch and learning rate increases by 66% compared to the worst on Top-1.

Abstract Security vulnerability prediction (SVP) can construct models to identify potentially vulnerable program modules via machine learning. Two kinds of features from different points of view are used to measure the extracted modules in previous studies. One kind considers traditional software metrics as features, and the other kind uses text mining to extract term vectors as features. Therefore, gathered SVP data sets often have numerous features and result in the curse of dimensionality. In this article, we mainly investigate the impact of filter‐based ranking feature selection (FRFS) methods on SVP, since other types of feature selection methods have too much computational cost. In empirical studies, we first consider three real‐world large‐scale web applications. Then we consider seven methods from three FRFS categories for FRFS and use a random forest classifier to construct SVP models. Final results show that given the similar code inspection cost, using FRFS can improve the performance of SVP when compared with state‐of‐the‐art baselines. Moreover, we use McNemar's test to perform diversity analysis on identified vulnerable modules by using different FRFS methods, and we are surprised to find that almost all the FRFS methods can identify similar vulnerable modules via diversity analysis.

Software defect prediction can identify potential defective modules in advance and then provide guidances for software testers to allocate more testing resources on these modules. During the gathering process for defect prediction datasets, if multiple metrics are used to measure the program modules, it will result in curse of dimensionality. Feature selection is one of effective methods to alleviate this problem. However, designing effective feature selection methods is a great challenge. Motivated by the idea of search based software engineering, we formalize this problem as a multi-objective optimization problem, and then propose novel method MOFES. To verify the effectiveness of our proposed method, we choose PROMISE dataset gathered from real projects, and compare MOFES with some classical baseline methods. Final results show that our method has the advantages of selecting less features and achieving better prediction performance in most projects while its computational cost is acceptable.

The recent studies indicate that predicate-based statistical fault localization suffered from the control dependence confounding effect and the failure flow confounding effect, which decrease the measurement accuracy of fault localization. However, the extent of the potentially confounding effect of data dependence is uncertain. This paper presents a novel approach that accounts for the effects of program dependences to mitigate the confounding effect during statistical predicate-based fault localization. First, we present a variable type-based predicate designation technique to improve the ability of fault-relevant predicate identification. Then, we conduct dependence analysis to examine the extent of the potentially confounding effect of data dependence in fault localization. Finally, we propose a linear regression-based method to mitigate both the data dependence confounding effect and the control dependence confounding effect. Using the open-source software systems, we find that the fault-relevant predicate can be identified effectively by the proposed predicate design technique, and the effectiveness of fault localization can be significantly improved after mitigating the dependence confounding effect.

In recent years, many approaches have been developed to determine the order of tested classes in interclass integration test. However, existing approaches are inaccurate, as they ignore the influence of abstract classes and polymorphism. In this paper, we propose a test-level-based approach to deal with class-integration-test order, in which both abstract classes and polymorphism are taken into account. First, based on interclass dependence analysis, we develop an edge-removing algorithm to eliminate cycles caused by static and dynamic dependencies, taking abstract classes and polymorphism into account. Then, after eliminating cycles, we propose a class-integration-test order algorithm based on test levels, including static and dynamic test levels. In this algorithm, we take into account the fact of some test levels infeasible caused by the characteristic of abstract classes that they cannot be instantiated and offer corresponding adjustment strategy. Finally, we design and implement a test level order generator. The experimental results show that the proposed strategy needs less test stubs than the most typically graph-based approaches. Copyright © 2014 John Wiley & Sons, Ltd.