Abstract With the evolution of modern software development paradigms, component reuse, and low-code approaches have emerged as mainstream in software development. However, developers often lack an in-depth understanding of reused code. The inability of components to operate autonomously leads to insufficient testing of software functionalities and security, further exacerbating the contradiction between the increasing complexity of software architectures and the demand for accurate and efficient software automation testing. This, in turn, increases the frequency of software supply chain security incidents. This paper proposes a test-driven generation framework, LLM4TDG, based on large language models (LLMs). By formally defining the constraint dependency graph and converting it into context constraints, LLMs’ ability to understand natural language descriptions such as test requirements and documents is enhanced. Constraint reasoning and backtracking mechanisms are then used to generate test drivers that satisfy the defined constraints automatically. Using the EvalPlus dataset, we evaluate the comprehensive capabilities of LLM4TDG in test case generation using four general-domain LLMs and five code-generation-domain LLMs. The experimental results indicate that our approach significantly enhances LLMs’ ability to comprehend constraints in testing objectives, achieving a 47.62% increase in constraint understanding across 147 testing tasks. Employing LLM4TDG significantly improves the average pass@k metric of all LLMs by 10.41%. The pass@k metric for CodeQwen-chat has improved by up to 18.66%. The metric surpasses the state-of-the-art GPT-4, with a performance of 92.16% on HUMANEVAL and 87.14% on HUMANEVAL+, which enhances the error correction and functional correctness in test-driven code generation. Meanwhile, Our experiments were conducted on a dataset of Python third-party libraries containing malicious behavior in the context of security testing tasks, validating the effectiveness of our method in real-world applications and its generalization capabilities.

APIs are increasingly prevalent in application environments, carrying the core business logic and sensitive data of enterprises, and have increasingly become the target of cyber attackers. The proportion of web attacks targeting APIs has exceeded half. The widespread use of APIs has expanded the attack surface, posing serious security challenges. Security risks, such as unauthorized access, misuse of business logic, data breaches, and complex cyber attacks, have intensified. Tr aditional security measures have proven inadequate in addressing API threats. There is an urgent demand for a more contextually aware and intelligent security mechanism capable of effectively mitigating API attacks. We proposed a novel model TAD-LLM based on Large Language Model for anomaly detection in API traffic. By using S2GS data transformation method, prompt optimization algorithm and LoRA fine-tuning technique, enables the model to acquire a profound comprehension of domain-specific knowledge in more elaborate detail, thereby enhancing the overall detection capability. Experimental results demonstrate that the proposed model TAD-LLM makes a significant advancement in securing APIs against cyber threats. The average f1-score of TAD-LLM reaches 99.27% in complex API attack scenarios. There are also notable improvements in precision, recall, and accuracy. Moreover, the overall performance of the model indicates that the model we proposed outperforms other models significantly and exhibits superior capability in handling complex API attack scenarios and advanced API attack techniques. It is worth noting that our model also shows strong performance on CSIC 2010, a widely used common http traffic dataset.

With the increasing complexity of software composition, code defects have become a long-term problem in software security. Traditional static analysis techniques cannot exhaustively enumerate all unsafe modes, and problems such as low path coverage rate brought by dynamic detection techniques make software security vulnerability detection inefficient. Methods based on Natural Language Processing have promoted the research of code defect detection tasks; however, there are problems of insufficient code semantic learning and limited data processing by pre-trained models. To solve these problems, from the perspective of enriching model input semantics and improving the model’s ability to process data, based on the Transformer model, we propose a hierarchical compression encoder model CPGBERT to detect whether the target function has defects. By using the regularity of the program context and structure, the program code is sliced for the input-output variables related to the objective function and dependencies on the codes’ propagation paths. Extract multiple code property graph information on rich semantics from the sliced program code for graph fusion, and embed the fused code property graph into the model by grouping. During the learning process, the independent hidden layer features are compressed and aggregated to make the model focus on the deep semantic learning of the objective function. The experiment uses the CodeXGLUE benchmark dataset and compares 6 kinds of code defect detection models having better performance to perform defect detection and effect evaluation on actual engineering code. The results show that the accuracy of the CPGBERT detection model is 67.97%, which is 5.89% higher than the CodeBERT model proposed by Microsoft and 1.35% higher than the state-of-the-art model CoTexT.