The number of disclosed vulnerabilities in open-source projects has been increasing steadily over the years, and thus it is important to deploy patches to repair vulnerabilities in a timely manner. However, due to the widespread reuse and customization of open-source software, there are often multiple versions or branches of the same project that co-exist in the ecosystem. Therefore, it is often challenging and tricky to guarantee that an exposed vulnerability can be repaired thoroughly. Driven by this, plenty of 1-day vulnerability analysis tools have been proposed recently, such as function-level vulnerability detection and patch presence test tools. Despite the fact that code evolution is common for open-source projects, existing analysis tools often neglect the important fact that the patched code is also constantly evolving. In this study, we take the first look to systematically investigate the phenomenon of security patch evolution in open-source projects. In particular, we performed extensive experiments on a large-scale dataset containing 1,046 distinct CVEs with 2,633 patches collected from popular open-source projects (e.g., linux, openssl). This study reveals interesting yet important findings with respect to the aspects of patch evolution frequency, patch evolution patterns, and the evolution impact on downstream 1-day vulnerability analysis tools. We believe that this study can shed important light on future researches on patch analysis.

Transformer-based pre-trained models of code (PTMC) have been widely utilized and have achieved state-of-the-art performance in many mission-critical applications. However, they can be vulnerable to adversarial attacks through identifier substitution or coding style transformation, which can significantly degrade accuracy and may further incur security concerns. Although several approaches have been proposed to generate adversarial examples for PTMC, the effectiveness and efficiency of such approaches, especially on different code intelligence tasks, has not been well understood. To bridge this gap, this study systematically analyzes five state-of-the-art adversarial attack approaches from three perspectives: effectiveness, efficiency, and the quality of generated examples. The results show that none of the five approaches balances all these perspectives. Particularly, approaches with a high attack success rate tend to be time-consuming; the adversarial code they generate often lack naturalness, and vice versa. To address this limitation, we explore the impact of perturbing identifiers under different contexts and find that identifier substitution within for and if statements is the most effective. Based on these findings, we propose a new approach that prioritizes different types of statements for various tasks and further utilizes beam search to generate adversarial examples. Evaluation results show that it outperforms the state-of-the-art ALERT in terms of both effectiveness and efficiency while preserving the naturalness of the generated adversarial examples.