Large Language Models (LLMs) have been increasingly integrated into computer-use agents, which can autonomously operate tools on a user's computer to accomplish complex tasks. However, due to the inherently unstable and unpredictable nature of LLM outputs, they may issue unintended tool commands or incorrect inputs, leading to potentially harmful operations. Unlike traditional security risks stemming from insecure user prompts, tool execution results from LLM-driven decisions introduce new and unique security challenges. These vulnerabilities span across all components of a computer-use agent. To mitigate these risks, we propose AgentSentinel, an end-to-end, real-time defense framework designed to mitigate potential security threats on a user's computer. AgentSentinel intercepts all sensitive operations within agent-related services and halts execution until a comprehensive security audit is completed. Our security auditing mechanism introduces a novel inspection process that correlates the current task context with system traces generated during task execution. To thoroughly evaluate AgentSentinel, we present BadComputerUse, a benchmark consisting of 60 diverse attack scenarios across six attack categories. The benchmark demonstrates a 87% average attack success rate on four state-of-the-art LLMs. Our evaluation shows that AgentSentinel achieves an average defense success rate of 79.6%, significantly outperforming all baseline defenses.

Cyber-physical systems (CPSs) are used extensively in critical infrastructure, underscoring the need for anomaly detection systems that are able to catch even the most motivated attackers. Traditional anomaly detection techniques typically do ‘one-off’ training on datasets crafted by experts or generated by fuzzers, potentially limiting their ability to generalize to unseen and more subtle attack strategies. Stopping at this point misses a key opportunity: a defender can actively challenge the attacker to find more nuanced attacks, which in turn can lead to more effective detection capabilities. Building on this concept, we propose Evo-Defender, an evolutionary framework that iteratively strengthens CPS defenses through a dynamic attacker-defender interaction. Evo-Defender includes a smart attacker that employs guided fuzzing to explore diverse, non-redundant attack strategies, while the self-evolving defender uses incremental learning to adapt to new attack patterns. We implement Evo-Defender on two realistic CPS testbeds: the Tennessee Eastman process and a Robotic Arm Assembly Workstation, injecting over 600 attack scenarios. In end-to-end attack detection experiments, EvoDefender achieves up to $2.7 \times$ higher performance than state-of-the-art baselines on unseen scenarios, while utilizing training data more efficiently for faster and more robust detection.

In high-stakes process industries, the Basic Process Control System (BPCS) relies on conventional control to enhance productivity, while the Safety Instrumented System (SIS) uses safety functions to maintain safety. Compared to the BPCS, attackers targeting the SIS can modify safety function activation conditions to trigger them prematurely or to evade the activation of the safety function. While various attack detection methods focus on the BPCS, they often overlook the SIS. This can lead to undetected safety breaches, significantly increasing the risk of catastrophic fault. Recent methods face three key limitations that hinder their practical application to SIS. First, both attackers and engineers can exploit the hot update mechanism of SIS to add or modify control logic. However, current methods lack verification for the newly added or modified logic. Second, current methods are unable to assess the rationality of dangerous value ranges. Third, these methods struggle to distinguish between faults and attacks, making it difficult to determine the appropriate time to activate the SIS’s safety function. To overcome these limitations, we propose SecureSIS, a method for securing SIS safety functions by leveraging the safety attributes of the SIS and incorporating information from the BPCS. The core of SecureSIS includes: 1) using the safety attributes of the SIS to verify automatically extracted candidate control logic detection rules; 2) utilizing information from the BPCS to verify automatically extracted candidate value range detection rules; and 3) distinguishing between safety function attacks and industrial process faults with validated rules and integration of process data from BPCS. Our scheme was evaluated using a Tricon SIS controller deployed on a gas pipeline network platform. The results indicate that SecureSIS achieved 97.3% accuracy in detecting data injection attacks and a detection accuracy of 96.0% for control logic modification attacks. Compared with the other representative detection approaches, our scheme has better detection performance.

The rapid progress of autonomous vehicles (AVs) has brought the prospect of a driverless future closer than ever. Recent fatalities, however, have emphasized the importance of safety validation through large-scale testing. Multiple approaches achieve this fully automatically using high-fidelity simulators, i.e., by generating diverse driving scenarios and evaluating autonomous driving systems (ADSs) against different test oracles. While effective at finding violations, these approaches do not identify the decisions and actions that caused them---information that is critical for improving the safety of ADSs. To address this challenge, we propose ACAV, an automated framework designed to conduct causality analyses for AV accident recordings in two stages. First, we apply feature extraction schemas based on the messages exchanged between ADS modules, and use a weighted voting method to discard frames of the recording unrelated to the accident. Second, we use safety specifications to identify safety-critical frames and deduce causal events by applying CAT---our causal analysis tool---to a station-time graph. We evaluated ACAV on the Apollo ADS, finding that it can identify five distinct types of causal events in 93.64% of 110 accident recordings generated by an AV testing engine. We further evaluated ACAV on 1206 accident recordings collected from versions of Apollo injected with specific faults, finding that it can correctly identify causal events in 96.44% of the accidents triggered by prediction errors, and 85.73% of the accidents triggered by planning errors.

Large Language Models (LLMs) have showcased their remarkable capabilities in diverse domains, encompassing natural language understanding, translation, and even code generation. The potential for LLMs to generate harmful content is a significant concern. This risk necessitates rigorous testing and comprehensive evaluation of LLMs to ensure safe and responsible use. However, extensive testing of LLMs requires substantial computational resources, making it an expensive endeavor. Therefore, exploring cost-saving strategies during the testing phase is crucial to balance the need for thorough evaluation with the constraints of resource availability. To address this, our approach begins by transferring the moderation knowledge from an LLM to a small model. Subsequently, we deploy two distinct strategies for generating malicious queries: one based on a syntax tree approach, and the other leveraging an LLM-based method. Finally, our approach incorporates a sequential filter-test process designed to identify test cases that are prone to eliciting toxic responses. By doing so, we significantly curtail unnecessary or unproductive interactions with LLMs, thereby streamlining the testing process. Our research evaluated the efficacy of DistillSeq across four LLMs: GPT-3.5, GPT-4.0, Vicuna-13B, and Llama-13B. In the absence of DistillSeq, the observed attack success rates on these LLMs stood at 31.5% for GPT-3.5, 21.4% for GPT-4.0, 28.3% for Vicuna-13B, and 30.9% for Llama-13B. However, upon the application of DistillSeq, these success rates notably increased to 58.5%, 50.7%, 52.5%, and 54.4%, respectively. This translated to an average escalation in attack success rate by a factor of 93.0% when compared to scenarios without the use of DistillSeq. Such findings highlight the significant enhancement DistillSeq offers in terms of reducing the time and resource investment required for effectively testing LLMs.

Large language models (LLMs) like ChatGPT and Gemini have significantly advanced natural language processing, enabling various applications such as chatbots and automated content generation. However, these models can be exploited by malicious individuals who craft toxic prompts to elicit harmful or unethical responses. These individuals often employ jailbreaking techniques to bypass safety mechanisms, highlighting the need for robust toxic prompt detection methods. Existing detection techniques, both blackbox and whitebox, face challenges related to the diversity of toxic prompts, scalability, and computational efficiency. In response, we propose ToxicDetector, a lightweight greybox method designed to efficiently detect toxic prompts in LLMs. ToxicDetector leverages LLMs to create toxic concept prompts, uses embedding vectors to form feature vectors, and employs a Multi-Layer Perceptron (MLP) classifier for prompt classification. Our evaluation on various versions of the LLama models, Gemma-2, and multiple datasets demonstrates that ToxicDetector achieves a high accuracy of 96.39% and a low false positive rate of 2.00%, outperforming state-of-the-art methods. Additionally, ToxicDetector's processing time of 0.0780 seconds per prompt makes it highly suitable for real-time applications. ToxicDetector achieves high accuracy, efficiency, and scalability, making it a practical method for toxic prompt detection in LLMs.

As the Industrial Control System (ICS) increasingly merges with the Internet, the security threats have been increasing from internal users and external hackers. These challenges are further intensified by the facts: industrial control devices and protocols, leading to the inadequacy of traditional access control models in tackling the intricacies of ICS. We identify attributes that are optimally aligned with the specific needs of the ICS environment and propose the Multidimensional Industrial Control Attribute-Based Access Control Model (MICABAC) as a customized solution. MICABAC model significantly improves access control security and is finer granularity by selecting and evaluating required attributes within various ICS. We have been validated in two real-world ICS environments: the Gas Pipe Network System (GPNS) and the Computer Numerical Control (CNC) machine tool. Experiments indicate that by integrating MICABAC into the existing system, the maximum delay for access requests is 63.83 ms. In terms of accuracy in defending against malicious attacks, the GPNS achieves 96.49% and the CNC reaches 94.86%. Finally, we discuss the advantages and limitations of MICABAC and explore potential directions for future research.

Industrial control systems (ICSs) are prevalent in critical infrastructures, where programmable logic controllers (PLCs) and physical instruments are integrated. However, multiple successful attacks against PLC control logic programs have caused significant damage to ICSs, which has led to an urgent need for detection and forensics of such attacks. Although several off-the-shelf defending mechanisms have been presented in the past, few of them can detect and locate the control logic attacks at run time. In this article, we propose a practical and automatic control logic attack detection and forensics framework (CLADF) to conduct control logic attack detection and forensics in ICSs. Specifically, the core of CLADF includes: 1) a control application extraction module to extract PLC binary control applications by simulating PLC normal upload functionality; 2) a control application reverse engineering module to disassemble binary control applications; and 3) an attack detection and forensics module for verifying the integrity of PLC control applications, recovering the normal control application, and locating the modified control instructions. We extensively evaluated CLADF in five different application scenarios and two real-world Schneider PLCs. For each PLC, we generated three types of 150 mutated control logic attacks. The results demonstrate that CLADF can effectively extract the run-time binary control application in different application scenarios and disassemble these binary control applications into assembly instructions. Moreover, CLADF can accurately detect the attacks and locate the modified subroutines.

The era of the Industrial Internet of Things has led to an escalating menace of Cyber-Physical Manufacturing Systems (CPMS) to cyber-attacks. Presently, the field of intrusion detection for CPMS has significant advancements. However, current methodologies require significant costs for collecting historical data to train detection models, which are tailored to specific machining scenarios. Evolving machining scenarios in the real world challenge the adaptability of these methods. In this paper, We found that the machining code of the CPMS contains a complete machining process, which is an excellent detection basis. Therefore we propose MPI-CNC, an intrusion detection approach based on Machining Process Invariant in the machining code. Specifically, MPI-CNC automates the analysis of the machining codes to extract machining process rules and key parameter rules, which serve as essential detection rules. Then, MPI-CNC actively acquires runtime status from the CPMS and matches the detection rules to identify cyber-attacks behavior. MPI-CNC was evaluated using two FANUC CNC machine tools across ten real machining scenarios. The experiment demonstrated the exceptional adaptability capability of MPI-CNC. Furthermore, MPI-CNC showed superior accuracy in detecting cyber-attacks against CPMS compared to existing state-of-the-art detection methods while ensuring normal machining operations.

Differential cryptanalysis is a powerful algorithmic-level attack, playing a central role in evaluating the security of symmetric cryptographic primitives. In general, the resistance against differential cryptanalysis can be characterized by the maximum expected differential characteristic probability. In this paper, we present generic and extensible approaches based on mixed integer linear programming (MILP) to bound such probability. We design a high-level cryptography-specific language EasyBc tailored for block ciphers and provide various rigorous procedures as differential denotational semantics, to automate the generation of MILP from block ciphers written in EasyBc . We implement an open-sourced tool that provides support for fully automated resistance evaluation of block ciphers against differential cryptanalysis. The tool is extensively evaluated on 23 real-life cryptographic primitives including all the 10 finalists of the NIST lightweight cryptography standardization process. The experiments confirm the expressivity of EasyBc and show that the tool can effectively prove the resistance against differential cryptanalysis for all block ciphers under consideration. EasyBc makes resistance evaluation against differential cryptanalysis easily accessible to cryptographers.

Motion control systems, extensively utilized in domains like 3D printing, CNC machining, and robotic arm operations, are pivotal in modern manufacturing and automation processes. Consequently, a specific category of attacks, designed to target these systems, can manipulate the movements of controlled objects while replaying false sensor readings to evade existing tools, thereby severely disrupting these essential operations without being detected. To make things worse, the limited computing resources of embedded devices in these systems constrain the implementation of robust security protections and monitoring mechanisms locally. To solve this, we propose a novel side-channel method that leverages current signals emitted by motors to reconstruct trajectories for attack detection. In this paper, we design and implement a two-stage detection framework, dubbed PowerGuard. In the offline learning stage, PowerGuard first captures the current signals emitted by the servo motors and models the correlation between these signals and corresponding movement trajectories. In the real-time monitoring stage, PowerGuard finds outliers that deviate from the desired trajectory described in the benign G-code file. We have evaluated PowerGuard using a typical motion control system that contains CNC machine tools from different vendors (e.g., Siemens 828D, 840D-sl, Fanuc 0i-md, 0i-tf). We conducted extensive experiments to evaluate the reconstruction accuracy and attack detection performance. Experimental results show that PowerGuard can reconstruct movement trajectories with an error of 0.047mm, and detect 93.35% of various trajectory anomalies.

Industrial control systems (ICSs) are types of cyber-physical systems in which programs, written in languages such as ladder logic or structured text, control industrial processes through sensing and actuating. Given the use of ICSs in critical infrastructure, it is important to test their resilience against manipulations of sensor/actuator inputs. Unfortunately, existing methods fail to test them comprehensively, as they typically focus on finding the simplest-to-craft manipulations for a testing goal, and are also unable to determine when a test is simply a minor permutation of another, i.e. based on the same causal events. In this work, we propose a guided fuzzing approach for finding 'meaningfully different’ tests for an ICS via a general formalisation of sensor/actuator-manipulation strategies. Our algorithm identifies the causal events in a test, generalises them to an equivalence class, and then updates the fuzzing strategy so as to find new tests that are causally different from those already identified. An evaluation of our approach on a real-world water treatment system shows that it is able to find 106% more causally different tests than the most comparable fuzzer. While we focus on diversifying the test suite of an ICS, our formalisation may be useful for other fuzzers that intercept communication channels.

Cyber–physical systems (CPSs) are ubiquitous in critical infrastructures, where programmable logic controllers (PLCs) and physical components intertwine. However, multiple successful attacks targeting safety-related CPSs, in particular the PLCs, manifest their vulnerability toward malicious cyber attacks, which may cause significant damage consequently. Though several kinds of defending techniques exist in the literature, few of them can be practically and widely applied to real-world CPSs equipped with PLCs from leading vendors, primarily due to the lack of specific hardware or unrealistic defense assumptions. In this article, we propose PLC-READER, a practical memory attacks detection and response framework to secure the CPS. The core of PLC-READER includes: 1) a comprehensive semantic analysis solution specifically for PLC’s proprietary protocol based on software reverse engineering and network traffic difference analysis and 2) a fine-grained memory structure analysis solution to identify the critical memory data. Based on the results of such reverse engineering, PLC-READER further performs sanity checks for the PLC’s critical memory by periodically checking the hash values and dynamic checksum values of these memory data. We extensively evaluated PLC-READER against four types of 366 different memory attacks, with some newly developed ones which got six CVE IDs from Schneider and Rockwell, by analyzing three kinds of proprietary protocols and six kinds of memory structures in six kinds of real-world PLCs from three leading manufacturers. The results demonstrate that the PLC-READER can detect all memory attacks with an accuracy of 100% and perform corresponding emergency responses in time.

Cyber-physical systems (CPSs) automating critical public infrastructure face a pervasive threat of attack, motivating research into different types of countermeasures. Assessing the effectiveness of these countermeasures is challenging, however, as benchmarks are difficult to construct manually, existing automated testing solutions often make unrealistic assumptions, and blindly fuzzing is ineffective at finding attacks due to the enormous search spaces and resource requirements. In this work, we proposeactive sensor fuzzing, a fully automated approach for building test suites without requiring anya priorknowledge about a CPS. Our approach employs active learning techniques. Applied to a real-world water treatment system, our approach manages to find attacks that drive the system into 15 different unsafe states involving water flow, pressure, and tank levels, including nine that were not covered by an established attack benchmark. Furthermore, we successfully generate targeted multi-point attacks which have been long suspected to be possible. We reveal that active sensor fuzzing successfully extends the attack benchmarks generated by our previous work, an ML-guided fuzzing tool, with two more kinds of attacks. Finally, we investigate the impact of active learning on models and the reason that the model trained with active learning is able to discover more attacks.

Autonomous vehicle (AV) systems must be comprehensively tested and evaluated before they can be deployed. High-fidelity simulators such as CARLA or LGSVL allow this to be done safely in very realistic and highly customizable environments. Existing testing approaches, however, fail to test simulated AVs systematically, as they focus on specific scenarios and oracles (e.g., lane following scenario with the “no collision” requirement) and lack any coverage criteria measures. In this paper, we propose$\mathtt {AVUnit}$, a framework for systematically testing AV systems against customizable correctness specifications. Designed modularly to support different simulators,$\mathtt {AVUnit}$consists of two new languages for specifying dynamic properties of scenes (e.g. changing pedestrian behaviour after waypoints) and fine-grained assertions about the AV's journey.$\mathtt {AVUnit}$further supports multiple fuzzing algorithms that automatically search for test cases that violate these assertions, using robustness and coverage measures as fitness metrics. We evaluated the implementation of$\mathtt {AVUnit}$for the LGSVL+Apollo simulation environment, finding 19 kinds of issues in Apollo, which indicate that the open-source Apollo does not perform well in complex intersections and lane-changing related scenarios.

Autonomous driving systems (ADSs) must be tested thoroughly before they can be deployed in autonomous vehicles. High-fidelity simulators allow them to be tested against diverse scenarios, including those that are difficult to recreate in real-world testing grounds. While previous approaches have shown that test cases can be generated automatically, they tend to focus on weak oracles (e.g. reaching the destination without collisions) without assessing whether the journey itself was undertaken safely and satisfied the law. In this work, we propose , an automated framework for testing ADSs against real-world traffic laws, which is designed to be compatible with different scenario description languages. provides a rich driver-oriented specification language for describing traffic laws, and a fuzzing engine that searches for different ways of violating them by maximising specification coverage. To evaluate our approach, we implemented it for Apollo+LGSVL and specified the traffic laws of China. was able to find 14 violations of these laws, including 173 test cases that caused accidents.

Cyber-physical systems (CPSs) are widespread in critical domains, and significant damage can be caused if an attacker is able to modify the code of their programmable logic controllers (PLCs). Unfortunately, traditional techniques for attesting code integrity (i.e. verifying that it has not been modified) rely on firmware access or roots-of-trust, neither of which proprietary or legacy PLCs are likely to provide. In this paper, we propose a practical code integrity checking solution based on privacy-preserving black box models that instead attest the input/output behaviour of PLC programs. Using faithful offline copies of the PLC programs, we identify their most important inputs through an information flow analysis, execute them on multiple combinations to collect data, then train neural networks able to predict PLC outputs (i.e. actuator commands) from their inputs. By exploiting the black box nature of the model, our solution maintains the privacy of the original PLC code and does not assume that attackers are unaware of its presence. The trust instead comes from the fact that it is extremely hard to attack the PLC code and neural networks at the same time and with consistent outcomes. We evaluated our approach on a modern six-stage water treatment plant testbed, finding that it could predict actuator states from PLC inputs with near-100% accuracy, and thus could detect all 120 effective code mutations that we subjected the PLCs to. Finally, we found that it is not practically possible to simultaneously modify the PLC code and apply discreet adversarial noise to our attesters in a way that leads to consistent (mis-)predictions.

Cyber-physical systems (CPSs) in critical infrastructure face a pervasive threat from attackers, motivating research into a variety of countermeasures for securing them. Assessing the effectiveness of these countermeasures is challenging, however, as realistic benchmarks of attacks are difficult to manually construct, blindly testing is ineffective due to the enormous search spaces and resource requirements, and intelligent fuzzing approaches require impractical amounts of data and network access. In this work, we propose active fuzzing, an automatic approach for finding test suites of packet-level CPS network attacks, targeting scenarios in which attackers can observe sensors and manipulate packets, but have no existing knowledge about the payload encodings. Our approach learns regression models for predicting sensor values that will result from sampled network packets, and uses these predictions to guide a search for payload manipulations (i.e. bit flips) most likely to drive the CPS into an unsafe state. Key to our solution is the use of online active learning, which iteratively updates the models by sampling payloads that are estimated to maximally improve them. We evaluate the efficacy of active fuzzing by implementing it for a water purification plant testbed, finding it can automatically discover a test suite of flow, pressure, and over/underflow attacks, all with substantially less time, data, and network access than the most comparable approach. Finally, we demonstrate that our prediction models can also be utilised as countermeasures themselves, implementing them as anomaly detectors and early warning systems.

The threat of attack faced by cyber-physical systems (CPSs), especially when they play a critical role in automating public infrastructure, has motivated research into a wide variety of attack defence mechanisms. Assessing their effectiveness is challenging, however, as realistic sets of attacks to test them against are not always available. In this paper, we propose smart fuzzing, an automated, machine learning guided technique for systematically finding 'test suites' of CPS network attacks, without requiring any knowledge of the system's control programs or physical processes. Our approach uses predictive machine learning models and metaheuristic search algorithms to guide the fuzzing of actuators so as to drive the CPS into different unsafe physical states. We demonstrate the efficacy of smart fuzzing by implementing it for two real-world CPS testbeds—a water purification plant and a water distribution system—finding attacks that drive them into 27 different unsafe states involving water flow, pressure, and tank levels, including six that were not covered by an established attack benchmark. Finally, we use our approach to test the effectiveness of an invariant-based defence system for the water treatment plant, finding two attacks that were not detected by its physical invariant checks, highlighting a potential weakness that could be exploited in certain conditions.

Cyber-physical systems (CPS) consist of sensors, actuators, and controllers all communicating over a network; if any subset becomes compromised, an attacker could cause significant damage. With access to data logs and a model of the CPS, the physical effects of an attack could potentially be detected before any damage is done. Manually building a model that is accurate enough in practice, however, is extremely difficult. In this paper, we propose a novel approach for constructing models of CPS automatically, by applying supervised machine learning to data traces obtained after systematically seeding their software components with faults ("mutants"). We demonstrate the efficacy of this approach on the simulator of a real-world water purification plant, presenting a framework that automatically generates mutants, collects data traces, and learns an SVM-based model. Using cross-validation and statistical model checking, we show that the learnt model characterises an invariant physical property of the system. Furthermore, we demonstrate the usefulness of the invariant by subjecting the system to 55 network and code-modification attacks, and showing that it can detect 85% of them from the data logs generated at runtime.