Software vulnerability detection is crucial for ensuring the security of software systems, representing a significant and challenging task. Recently, some studies have integrated large language models and graph neural networks to extract code features from different modalities (code sequences and graphs) for vulnerability detection. Unfortunately, current solutions struggle to fully leverage the complementary knowledge between modalities, thereby undermining their effectiveness in practical applications. In this paper, we proposeClara, a novel cross-modal learning approach that integrates multi-modal information from both global and local perspectives for effective detection. Specifically, for local fusion, we design an information interaction module guided by prompts, which employs learnable prompts to enhance feature extraction through the interaction of information between modalities. For global fusion, we devise a Cross-attention Adaptive Fusion module that adaptively adjusts the fusion weights of embeddings from different modalities using attention mechanisms. Experimental results on two benchmark datasets demonstrate thatClaraachieves improvements of 21.37% and 11.86% in F1 score over state-of-the-art vulnerability detection methods, respectively.

The software development process is characterized by an iterative cycle of continuous functionality implementation and debugging, essential for the enhancement of software quality and adaptability to changing requirements. This process incorporates two isolatedly studied tasks: Code Search (CS), which retrieves reference code from a code corpus to aid in code implementation, and Fault Localization (FL), which identifies code entities responsible for bugs within the software project to boost software debugging. The basic observation of this study is that these two tasks exhibit similarities since they both address search problems. Notably, CS techniques have demonstrated greater effectiveness than FL ones, possibly because of the precise semantic details of the required code offered by natural language queries, which are not readily accessible to FL methods. Drawing inspiration from this, we hypothesize that a fault localizer could achieve greater proficiency if semantic information about the buggy methods were made available. Based on this idea, we propose \(\texttt{CosFL}\) , an FL approach that decomposes the FL task into two steps: query generation , which describes the functionality of the problematic code in natural language, and fault retrieval , which uses CS to find program elements semantically related to the query, allowing for finishing the FL task from a CS perspective. Specifically, to depict the buggy functionalities and generate high-quality queries, \(\texttt{CosFL}\) extensively harnesses the code analysis, semantic comprehension, text generation, and decision-making capabilities of LLMs. Moreover, to enhance the accuracy of CS, \(\texttt{CosFL}\) captures varying levels of context information and employs a multi-granularity CS strategy, which facilitates a more precise identification of buggy methods from a holistic view. The evaluation on 835 real bugs from 23 Java projects shows that \(\texttt{CosFL}\) successfully localizes 324 bugs within Top-1, which significantly outperforms the state-of-the-art approaches by 26.6%–57.3%. The ablation study and sensitivity analysis further validate the importance of different components and the robustness of \(\texttt{CosFL}\) across different backend models.

As software evolves, test code must be co-maintained to ensure quality, but it often becomes obsolete, leading to failures that mislead developers and increase maintenance overhead. While recent Large Language Model (LLM)-based approaches show promise for repairing obsolete tests, their effectiveness is constrained by a critical challenge: providing comprehensive, repository-level context without overwhelming the models’ input limits. Fixed retrieval strategies often fail to capture the diverse dependencies required for complex repairs. In this paper, we present Trace, a retrieve-agent-based repository-level test repair method. The Trace selectively retrieves crucial context by analyzing (1) class-level structures to understand internal changes, (2) caller methods to capture real-world usage patterns, and (3) related files along the call graph to trace transitive dependencies. This multi-faceted context provides the LLM with a precise and concise understanding of the necessary code environment. We evaluated Trace on a dataset of real-world Java test updates, where it demonstrated superior performance compared to existing state-of-the-art baselines. Our results confirm that a structured, adaptive retrieval process is key to unlocking the full potential of LLMs for automated test maintenance.

Retrieval-Augmented Code Generation (RACG) leverages external knowledge to enhance Large Language Models (LLMs) in code synthesis, improving the functional correctness of the generated code. However, existing RACG systems largely overlook security, leading to substantial risks. Especially, the poisoning of malicious code into knowledge bases can mislead LLMs, resulting in the generation of insecure outputs, which poses a critical threat in modern software development. To address this, we propose a security-hardening framework for RACG systems, CodeGuarder, that shifts the paradigm from retrieving only functional code examples to incorporating both functional code and security knowledge. Our framework constructs a security knowledge base by analyzing real-world vulnerabilities from the ReposVul dataset. For each code generation query, a retriever decomposes the query into fine-grained sub-tasks and fetches relevant security knowledge. To prioritize critical security guidance, we introduce a re-ranking and filtering mechanism by leveraging the LLMs' susceptibility to different vulnerability types. This filtered security knowledge is seamlessly integrated into the generation prompt. Our evaluation shows CodeGuarder significantly improves code security rates across various LLMs, achieving average improvements of 20.12% in standard RACG, and 31.53% and 21.91% under two distinct poisoning scenarios without compromising functional correctness. Furthermore, CodeGuarder demonstrates strong generalization, enhancing security even when the targeted language's security knowledge is lacking. This work presents CodeGuarder as a pivotal advancement towards building secure and trustworthy RACG systems.

Pre-trained code models are essential for various code intelligence tasks. Yet, their effectiveness is heavily influenced by the quality of the pre-training dataset, particularly human-written reference comments, which usually serve as a bridge between the programming language and natural language. One significant challenge is that such comments could become inconsistent with the corresponding code as the software evolves, leading to suboptimal model performance. Large language models (LLMs) have demonstrated superior capabilities in generating high-quality code comments. This work investigates whether substituting original human-written comments with LLM-generated ones can improve pre-training datasets for more effective pretrained code models. As existing reference-based metrics cannot evaluate the quality of human-written reference comments themselves, to enable direct comparison between LLM-generated and human reference comments, we introduce two auxiliary tasks as novel reference-free metrics, including code-comment inconsistency detection and semantic code search. Experimental results show that LLM-generated comments exhibit superior semantic consistency with the code compared to human-written reference comments. Our manual evaluation also corroborates this conclusion, which indicates the potential of utilizing LLMs to enhance the quality of the pre-training dataset. Based on this finding, we rebuilt the CodeSearchNet dataset with LLM-generated comments and re-pre-trained the CodeT5 model. Evaluations on multiple code intelligence tasks demonstrate that models pretrained by LLM-enhanced data outperform their counterparts (pre-trained by original human reference comments data) on code summarization, code generation, and code translation tasks. This research validates the feasibility of rebuilding the pre-training dataset by LLMs to advance code intelligence tasks. It advocates rethinking the reliance on human reference comments for coderelated tasks.

Fault localization (FL) is a critical but time-consuming part of software debugging. With the improvement of the Large Language Models (LLMs) in their code capabilities, the increasing demand for automated software development has encouraged more research on building LLM-based Fault Localization (LLMFL) systems. However, existing LLMFL techniques are typically restricted to predicting bug locations by analyzing static code, while overlooking crucial dynamic program state of the software. This lack of context makes LLMs prone to generating "hallucinations", incorrectly identifying bug-free code as suspicious. To address this, this paper introduces PingFL, the LLMFL system that incorporates program dynamic information for more accurate automatic fault localization. PingFL comprises a Fault Localization (FL) agent and a Print Debugging (PD) agent. The FL agent is tasked with understanding the root cause through a set of callable tools. When the FL agent nominates a location as suspicious, it would entrust the PD agent to verify the suspected issue through multiple rounds of print debugging. In particular, these two agents communicate efficiently by conveying the textual thought generated by the LLM. The evaluation on 812 real-world bugs from the Defects4J benchmark shows that PingFL can localize 450 bugs within Top-1, which significantly outperforms other LLM-based approaches by 41% to 122%. A deeper dive into PingFL’s performance reveals that it exhibits specific FL strategies and tool usage patterns even without explicit instructions. Finally, PingFL proves to be cost-effective, spending an average of $0.23 and 104.62 seconds per bug, with the print debugging mechanism accounting for only $0.07 and 48.14 seconds.

As software grows in complexity to accommodate diverse features and platforms, software bloating has emerged as a significant challenge, adversely affecting performance and security. However, existing approaches inadequately address the dual objectives of debloating: maintaining functionality by preserving essential features and enhancing security by reducing security issues. Specifically, current software debloating techniques often rely on input-based analysis, using user inputs as proxies for the specifications of desired features. However, these approaches frequently overfit provided inputs, leading to functionality loss and potential security vulnerabilities. To address these limitations, we proposeLEADER, a program debloating framework enhanced by Large Language Models (LLMs), which leverages their semantic understanding, generative capabilities, and decision-making strengths.LEADERmainly consists of two modules: (1) a documentation-guided test augmentation module designed to preserve functionality, which leverages LLMs to comprehend program documentation and generates sufficient tests to cover the desired features comprehensively, and (2) a multi-advisor-aided program debloating module that employs a neuro-symbolic pipeline to ensure that the security of the software can be perceived during debloating. This module combines debloating and security advisors for analysis and employs an LLM as a decision-maker to eliminate undesired code securely. Extensive evaluations on widely used benchmarks demonstrate the efficacy ofLEADER. It achieves a 95.5% test case pass rate and reduces program size by 42.5%. Notably, it reduces the introduction of vulnerabilities during debloating by 79.1% and decreases pre-existing vulnerabilities by 16.5% more than CovA. These results demonstrate thatLEADERsurpasses the state-of-the-art tool CovA in functionality and security. These results underscore the potential ofLEADERto set a new standard in program debloating by effectively balancing functionality and security.

Software vulnerability detection is crucial for high-quality software development. Recently, some studies utilizing Graph Neural Networks (GNNs) to learn the graph representation of code in vulnerability detection tasks have achieved remarkable success. However, existing graph-based approaches mainly face two limitations that prevent them from generalizing well to large code graphs: (1) the interference of noise information in the code graph; (2) the difficulty in capturing long-distance dependencies within the graph. To mitigate these problems, we propose a novel vulnerability detection method,ANGEL, whose novelty mainly embodies the hierarchical graph refinement and context-aware graph representation learning. The former hierarchically filters redundant information in the code graph, thereby reducing the size of the graph, while the latter collaboratively employs the Graph Transformer and GNN to learn code graph representations from both the global and local perspectives, thus capturing long-distance dependencies. Extensive experiments demonstrate promising results on three widely used benchmark datasets: our method significantly outperforms several other baselines in terms of the accuracy and F1 score. Particularly, in large code graphs,ANGELachieves an improvement in accuracy of 34.27%-161.93% compared to the state-of-the-art method, AMPLE. Such results demonstrate the effectiveness ofANGELin vulnerability detection tasks.

Fault Localization (FL) is an essential step during the debugging process. With the strong capabilities of code comprehension, the recent Large Language Models (LLMs) have demonstrated promising performance in diagnosing bugs in the code. Nevertheless, due to LLMs’ limited performance in handling long contexts, existing LLM-based fault localization remains on localizing bugs within asmall code scope(i.e., a method or a class), which struggles to diagnose bugs for alarge code scope(i.e., an entire software system). To address the limitation, this paper presents SoapFL, which builds an LLM-driven standard operating procedure (SOP) to automatically localize buggy methods from the entire software. By simulating the behavior of a human developer, SoapFL models the FL task as a three-step process, which involves comprehension, navigation, and confirmation. Within specific steps, SoapFL provides useful test behavior or coverage information to LLM through program analysis. Particularly, we adopt a series of auxiliary strategies such as Test Behavior Tracking, Document-Guided Search, and Multi-Round Dialogue to overcome the challenges in each step. The evaluation on the widely used Defects4J-V1.2.0 benchmark shows that SoapFL can localize 175 out of 395 bugs within Top-1, which outperforms the other LLM-based approaches and exhibits complementarity to the state-of-the-art learning-based techniques. Additionally, we confirm the indispensability of the components in SoapFL with the ablation study and demonstrate the usability of SoapFL through a user study. Finally, the cost analysis shows that SoapFL spends an average of only 0.081 dollars and 92 seconds for a single bug.

Vulnerabilities exert great burden on developers in terms of debugging and maintenance. Automated Vulnerability Repair(AVR) is considered as a promising approach to alleviate the burden of developers. Template-based automated program repair techniques have shown their effectiveness in fixing general bugs. However, due to the diverse root causes of vulnerabilities, it is challenging to construct sufficient repair templates to cover various vulnerabilities. In this paper, we introduce a Template-guided Retrieval-Augmented Patch generation approach, named T-RAP. Inspired by retrieval-augmented techniques that effectively utilize historical data, our approach leverages repair templates to extract similar vulnerability repair patches from the codebase. These patches then guide the process of generating vulnerability patches. To extract similar patches, we also propose a matching algorithm specifically designed for the retrieval-augmented vulnerability repair. This involves identifying similarities between numerous templates and vulnerabilities during the template-guided stage. Experimental results demonstrate that T-RAP outperforms all the studied AVR approaches, repairing 56.8% more vulnerabilities than VulRepair and 30.24% more than VulMaster. It can also accurately repair more types of real-world vulnerabilities than VulMaster. Additionally, we evaluated the effectiveness of our patch retriever. The results indicate that our template-guided retriever, which is based on our matching algorithm, outperforms the retrieval algorithm proposed in the recent retrieval-augmented patch generation approach RAP-Gen.

Software vulnerabilities inevitably arise during software development and may leave behind huge security risks. In order to detect and mitigate vulnerabilities before they can be exploited, various fine-grained deep learning (DL)-based vulnerablity detection (VD) approaches have been proposed to locate vulnerable statements, among which the Transformer-based methods have shown promising performances. However, existing Transformer-based statement-level approaches still suffer from a crucial limitation: they ignore the intrinsic data/control dependency relations between the statements. In this work, we propose a novel Transformer-based model MatsVD, which aims to address the above challenge from two aspects: Firstly, inspired by the hierarchical structure of code (i.e., tokens, statements, and functions), MatsVD comprises three different Transformer-based layers (i.e., statement embedding layer, statement representation layer, and function representation layer) to gradually aggregate the basic code tokens into meaningful statement/function representations; Secondly, to further exploit the data/control dependencies between statements, we replace the original attention mechanism of the Transformer with a novel dependency-based attention by masking irrelevant attention scores according to the program dependency graph. We comprehensively evaluate MatsVD on the widely used C/C++ vulnerability dataset Big-Vul. The results show that MatsVD significantly outperforms 6 other statement-level methods on both binary classification and ranking metrics. In particular, MatsVD obtains an F1 score of 86% and a Top-1 Accuracy of 93% on statement-le, which improves by respectively 22.97% and 7.76% compared to the state-of-the-art method VELVET.

Transformers are now widely utilized in code intelligence tasks. To better fit highly structured source code, various structure information is passed into Transformer, such as positional encoding and abstract syntax tree (AST) based structures. However, it is still not clear how these structural features affect code intelligence tasks, such as code summarization. Addressing this problem is of vital importance for designing Transformer-based code models. Existing works are keen to introduce various structural information into Transformers while lacking persuasive analysis to reveal their contributions and interaction effects. In this paper, we conduct an empirical study of frequently-used code structure features for code representation, including two types of position encoding features and AST-based structure features. We propose a couple of probing tasks to detect how these structure features perform in Transformer and conduct comprehensive ablation studies to investigate how these structural features affect code semantic summarization tasks. To further validate the effectiveness of code structure features in code summarization tasks, we assess Transformer models equipped with these code structure features on a structural dependent summarization dataset. Our experimental results reveal several findings that may inspire future study: (1) there is a conflict between the influence of the absolute positional embeddings and relative positional embeddings in Transformer; (2) AST-based code structure features and relative position encoding features show a strong correlation and much contribution overlap for code semantic summarization tasks indeed exists between them; (3) Transformer models still have space for further improvement in explicitly understanding code structure information.

Fault detection has long been a hot research issue for industry. Many common algorithms such as principal component analysis, recursive transformed component statistical analysis and moments-based robust principal component analysis can deal with static processes only, whereas most industrial processes are dynamic. Therefore, dynamic principal component analysis and recursive dynamic transformed component statistical analysis have been proposed to deal with dynamic processes by expanding the dimensions. The computational complexity of these algorithms are greatly increased, and these algorithms cannot divide the data space accurately. In this paper, we propose a novel algorithm called recursive innovational component statistical analysis (RICSA), which estimates the dynamic structure of the data, accurately divides the data space into dynamic components and innovational components. In unsteady state process, the statistical characteristics of data will change, and RICSA can classify these characteristics into dynamic components by dividing the data space, instead of treating them as faults, thereby reducing the false alarm rate. Through a series of comparative experiments, especially on practical coal pulverizing system in the 1000-MW ultra-supercritical thermal power plant, Zhoushan Power Plant, we found the recursive innovational component statistical analysis to realize a higher accuracy rate and a lower false alarm rate and detection delay, which verifies its superiority. We also discuss the reduced computational complexity associated with the recursive innovational component statistical analysis. Note to Practitioners—Aiming at the dynamic processes, the recursive innovational component statistical analysis algorithm proposed in this paper can divide the data space into dynamic components and innovational components by estimating the dynamic structure of the data. In addition, in the monitoring process, computational complexity is also a key point. Compared with recursive dynamic transformed component statistical analysis, recursive innovational component statistical analysis has lower computational complexity and higher accuracy. After multiple sets of experiments, it can be verified that recursive innovational component statistical analysis has a great monitoring effect in the actual industrial process, and can provide early warning of faults.

Regression testing is a widely adopted approach to expose change-induced bugs as well as to verify the correctness/robustness of code in modern software development settings. Unfortunately, the occurrence of flaky tests leads to a significant increase in the cost of regression testing and eventually reduces the productivity of developers (i.e., their ability to find and fix real problems). State-of-the-art approaches leverage dynamic test information obtained through expensive re-execution of test cases to effectively identify flaky tests. Towards accounting for scalability constraints, some recent approaches have built on static test case features, but fall short on effectiveness. In this paper, we introduce Peeler, a new fully static approach for predicting flaky tests through exploring a representation of test cases based on the data dependency relations. The predictor is then trained as a neural network based model, which achieves at the same time scalability (because it does not require any test execution), effectiveness (because it exploits relevant test dependency features), and practicality (because it can be applied in the wild to find new flaky tests). Experimental validation on 17,532 test cases from 21 Java projects shows that Peeler outperforms the state-of-the-art FlakeFlagger by around 20 percentage points: we catch 22% more flaky tests while yielding 51% less false positives. Finally, in a live study with projects in-the-wild, we reported to developers 21 flakiness cases, among which 12 have already been confirmed by developers as being indeed flaky.

Abstract Context Tangled commits are changes to software that address multiple concerns at once. For researchers interested in bugs, tangled commits mean that they actually study not only bugs, but also other concerns irrelevant for the study of bugs. Objective We want to improve our understanding of the prevalence of tangling and the types of changes that are tangled within bug fixing commits. Methods We use a crowd sourcing approach for manual labeling to validate which changes contribute to bug fixes for each line in bug fixing commits. Each line is labeled by four participants. If at least three participants agree on the same label, we have consensus. Results We estimate that between 17% and 32% of all changes in bug fixing commits modify the source code to fix the underlying problem. However, when we only consider changes to the production code files this ratio increases to 66% to 87%. We find that about 11% of lines are hard to label leading to active disagreements between participants. Due to confirmed tangling and the uncertainty in our data, we estimate that 3% to 47% of data is noisy without manual untangling, depending on the use case. Conclusion Tangled commits have a high prevalence in bug fixes and can lead to a large amount of noise in the data. Prior research indicates that this noise may alter results. As researchers, we should be skeptics and assume that unvalidated data is likely very noisy, until proven otherwise.

Smart contracts with natural economic attributes have been widely and rapidly developed in various fields. However, the bugs and vulnerabilities in smart contracts have brought huge economic losses, which has strengthened people's attention to the security issues of smart contracts. The immutability of smart contracts makes people more willing to conduct security checks before deploying smart contracts. Nonetheless, existing smart contract vulnerability detection techniques are far away from enough: static analysis approaches rely heavily on manually crafted heuristics which is difficult to reuse across different types of vulnerabilities while deep learning based approaches also have unique limitations. In this study, we propose a novel approach, Peculiar, which uses Pre-training technique for detection of smart contract vulnerabilities based on crucial data flow graph. Compared against the traditional data flow graph which is already utilized in existing approach, crucial data flow graph is less complex and does not bring an unnecessarily deep hierarchy, which makes the model easy to focus on the critical features. Moreover, we also involve pre-training technique in our model due to the dramatic improvements it has achieved on a variety of NLP tasks. Our empirical results show that Peculiar can achieve 91.80 % precision and 92.40 % recall in detecting reentrancy vulnerability, one of the most severe and common smart contract vulnerabilities, on 40,932 smart contract files, which is significantly better than the state-of-the-art methods (e.g., Smartcheck achieves 79.37% precision and 70.50% recall). Meanwhile, another experiment shows that Peculiar is more discerning to reentrancy vulnerability than existing approaches. The ablation experiment reveals that both crucial data flow graph and pre-trained model contribute significantly to the performances of Peculiar.

The literature of Automated Program Repair is largely dominated by approaches that leverage test suites not only to expose bugs but also to validate the generated patches. Unfortunately, beyond the widely-discussed concern that test suites are an imperfect oracle because they can be incomplete, they can include tests that are flaky. A flaky test is one that can be passed or failed by a program in a non-deterministic way. Such tests are generally carefully removed from the repair benchmarks. In practice, however, flaky tests are available test suite of software repositories. To the best of our knowledge, no study has discussed this threat to validity for evaluation of program repair. In this work, we highlight this threat and further investigate the impact of flaky tests by reverting their removal from the Defects4J benchmark. Our study aims to characterize the impact of flaky tests for localizing bugs and the eventual influence on the repair performance. Among other insights, we find that (1) although flaky tests are few (≈0.3%) of total tests, they affect experiments related to a large proportion (98.9%) of Defects4J real-world faults; (2) most flaky tests (98%) actually provide deterministic results under specific environment configurations (with the jdk version influencing the results); (3) flaky tests drastically hinder the effectiveness of spectrum-based fault localization (e.g., the rankings of 90 bugs drop down while none of the bugs obtains better location results compared with results achieved without flaky tests); and (4) the repairability of APR tools is greatly affected by the presence of flaky tests (e.g., 10 state of the art APR tools can now fix significantly fewer bugs than when the benchmark is manually curated to remove flaky tests). Given that the detection of flaky tests is still nascent, we call for the program repair community to relax the artificial assumption that the test suite is free from flaky tests. One direction that we propose is to consider developing strategies where patches that partially-fix bugs are considered worthwhile: a patch may make the program pass some test cases but fail some (which may actually be the flaky ones).

Automated Program Repair (APR) is becoming a hot topic in Software Engineering community with many approaches being proposed and experiments being performed over the years. The results obtained from different experiments can be used as practical guidance to advance APR techniques. However, researchers have generally ignored the biases with respect to the unexpected results generated by various APR techniques, in which case the repair process cannot be finished normally and is terminated with unexpected exceptions (referred to as the non-repairability factors). In this paper, we aim to thoroughly understand the reasons for such non-repairability factors of various APR techniques, thus to provide practical insights for diverse stakeholders to establish an unbiased evaluation of APR techniques. To achieve so, we performed a systematic study on the existing execution logs that are ended with unexpected exceptions collected from different APR studies. Specifically, we investigated different types of exceptions with their frequencies of occurrence, the behind reasons of such occurrences, as well as the impact of such exceptions on the repairability of APR techniques. Our experimental results reveal that: 1) non-repairability factors happen in 25.7% of our studied logs and are widespread among diverse combinations of APR tools with FL strategies; 2) Inherent defect of APR tools is the most common reason for the occurrence of the non-repairability factors; 3) the impact of the non-repairability factors on the performance of APR tools can be rather significant. Our empirical study indicates that it is of great importance to eliminate the biases from the non-repairability factors. We also highlight several implications for actions that we can take to eliminate such biases.

Test-based automated program repair (APR) has attracted huge attention from both industry and academia. Despite the significant progress made in recent studies, the overfitting problem (i.e., the generated patch is plausible but overfitting) is still a major and long-standing challenge. Therefore, plenty of techniques have been proposed to assess the correctness of patches either in the patch generation phase or in the evaluation of APR techniques. However, the effectiveness of existing techniques has not been systematically compared and little is known to their advantages and disadvantages. To fill this gap, we performed a large-scale empirical study in this paper. Specifically, we systematically investigated the effectiveness of existing automated patch correctness assessment techniques, including both static and dynamic ones, based on 902 patches automatically generated by 21 APR tools from 4 different categories. Our empirical study revealed the following major findings: (1) static code features with respect to patch syntax and semantics are generally effective in differentiating overfitting patches over correct ones; (2) dynamic techniques can generally achieve high precision while heuristics based on static code features are more effective towards recall; (3) existing techniques are more effective towards certain projects and types of APR techniques while less effective to the others; (4) existing techniques are highly complementary to each other. For instance, a single technique can only detect at most 53.5% of the overfitting patches while 93.3% of them can be detected by at least one technique when the oracle information is available. Based on our findings, we designed an integration strategy to first integrate static code features via learning, and then combine with others by the majority voting strategy. Our experiments show that the strategy can enhance the performance of existing patch correctness assessment techniques significantly.