Large language models (LLMs) have emerged as a promising tool for detecting code vulnerabilities, potentially offering advantages over traditional rule-based methods. This paper proposes an enhanced framework for vulnerability detection using LLMs, incorporating various prompt engineering strategies to improve performance. We evaluate several techniques, including role-based prompting, zero-shot chain-of-thought, and structured prompting approaches, on the DiverseVul dataset of C/C++ vulnerabilities. Our experiments assess the framework’s performance across different code structures, contextual information levels, and LLM capabilities. Our results show that using our dynamic prompt engineering technique, you can improve the F1 score by up to 100% with GPT-3.5, a widely used LLM model. We also observe that GPT-4o, Gemini 2.0 Flash, and Meta Llama 3.1 generally outperform GPT-3.5, and all models are very poor when it comes to correctly identifying the type of vulnerability in the code, with the best F1 score of 0.16 observed. However, our follow-up experiments on LLM-based vulnerability correction (i.e., patching) show a 45.77% success rate using GPT-4o, demonstrating promising results in leveraging LLMs for enhancing software security and providing insights into optimizing prompt engineering for vulnerability detection tasks.

It is still challenging to effectively identify rumors due to rapid changes in people's interests and perceptions. To enhance rumor detectors, we first need to better understand which rumors are effective (in terms of bypassing detection) and their characteristics. In this paper, we introduce ARGH, a novel framework to automatically generate rumors using recent advancements in natural language processing, customized to target and generate specific topics. To show the effectiveness of ARGH, we conducted a user study with 212 participants and analyzed how well humans can detect the rumors generated by ARGH, and we also tested its performance against the state-of-the-art rumor detection model PLAN [17]. Surprisingly, the experimental results demonstrate that the generated rumors are significantly harder to identify as rumors than hand-written rumors, degrading the detection accuracy by both humans and machines by 18.87% and 17.62%, respectively. We believe that ARGH will be a useful tool to obtain high quality and evasive rumor datasets quickly, which is often a tedious and time consuming task. Further, our analysis results provide valuable insight into how to characterize evasive rumors and how they can be generated, which will help to enhance the existing rumor detection techniques.