Safe Reinforcement Learning (RL) has been applied to synthesize control policies that maximize task rewards while adhering to safety constraints within simulated secure cyber-physical systems. However, the vulnerability of safe RL to adversarial attacks remains largely unexplored. We argue that understanding the safety vulnerabilities of learned control policies is crucial for ensuring true safety in real-world scenarios. To address this gap, we first formally define the safe RL problem with formal language (signal temporal logic) and demonstrate that even optimal policies are susceptible to observation perturbations. We then introduce novel safety violation attacks that exploit adversarial models trained with reversed safety constraints to induce unsafe behaviors. Lastly, through both theoretical analysis and experimental results, we demonstrate that our approach is more effective at violating safety constraints than existing adversarial RL methods, which primarily focus on reducing task rewards rather than compromising safety.

We study the vulnerability of Cyber-physical systems (CPS) under stealthy sensor attacks in black-box scenarios. “Black-box” refers to scenarios where the attacker has minimal knowledge of the target system. Designing a stealthy sensor attack sequence under this scenario has two main challenges. The first one lies in ensuring the stealthiness of the sensor attack, meaning does not trigger an alert when applying the generated sensor attack sequence to the CPS. The second one is maintaining stealthiness throughout the attack generation process, indicating the limitation on the alarm frequency when generating the attack sequence. To address the above challenges, we develop a querybased black-box stealthy attack framework to violate the safety of the CPS. To maintain stealthiness during training, an active learning method has been introduced to extract the detector’s information to a time series model. The stealthy attack sequence is then generated from that model. Experiments on four numerical simulations and a high-fidelity simulator demonstrate the effectiveness of the proposed framework.

Cyber-Physical Systems(CPS) are the integration of sensing, control, computation, and networking with physical components and infrastructure connected by the internet. The autonomy and reliability are enhanced by the recent development of safe reinforcement learning (safe RL). However, the vulnerability of safe RL to adversarial conditions has received minimal exploration. In order to truly ensure safety in physical world applications, it is crucial to understand and address these potential safety weaknesses in learned control policies. In this work, we demonstrate a novel attack to violate safety that induces unsafe behaviors by adversarial models trained using reversed safety constraints. The experiment results show that the proposed method is more effective than existing works.

Safe reinforcement learning (RL) aims to derive a control policy that navigates a safety-critical system while avoiding unsafe explorations and adhering to safety constraints. While safe RL has been extensively studied, its vulnerabilities during the policy training have barely been explored in an adversarial setting. This article bridges this gap and investigates the training time vulnerability of formal language-guided safe RL. Such vulnerability allows a malicious adversary to inject backdoor behavior into the learned control policy. First, we formally define backdoor attacks for safe RL and divide them into active and passive ones depending on whether to manipulate the observation. Second, we propose two novel algorithms to synthesize the two kinds of attacks, respectively. Both algorithms generate backdoor behaviors that may go unnoticed after deployment but can be triggered when specific states are reached, leading to safety violations. Finally, we conduct both theoretical analysis and extensive experiments to show the effectiveness and stealthiness of our methods.

There are various applications of Cyber-Physical systems (CPSs) that are life-critical where failure or malfunction can result in significant harm to human life, the environment, or substantial economic loss. Therefore, it is important to ensure their reliability, security, and robustness to the attacks. However, there is no widely used toolbox to simulate CPS and target security problems, especially the simulation of sensor attacks and defense strategies against them. In this work, we introduce our toolbox CPSim, a user-friendly simulation toolbox for security problems in CPS. CPSim aims to simulate common sensor attacks and countermeasures to these sensor attacks. We have implemented bias attacks, delay attacks, and replay attacks. Additionally, we have implemented various recovery-based methods against sensor attacks. The sensor attacks and recovery methods configurations can be customized with the given APIs. CPSim has built-in numerical simulators and various implemented benchmarks. Moreover, CPSim is compatible with other external simulators and can be deployed on a real testbed for control purposes. 1

Modern microprocessors employ register files (RFs) for performance enhancement and achieving instruction level parallelism simultaneously. However, RF incurs large power consumption owing to the highly frequent access. Meanwhile, as technology scales, bias temperature instability has become a major reliability concern for RF designers. This paper presents an aging-aware trimodal register file (TM-RF) design to enhance the power efficiency. As instructions pass through the pipeline, TM-RF places the bit-cells in different modes based on the register activity, thereby achieving significant power reduction. To meet design constraints of different applications, we present four schemes to implement the proposed design, providing design flexibility. Additionally, with device selection and worst case sizing methodology, we mitigate aging-effect-induced RF reliability degradation. Simulation results on SPEC 2000 benchmarks demonstrate that TM-RF achieves up to 81.4% power savings and 17% reliability improvement on average, with minimal impact on performance.