Cyber-Physical Systems (CPS) often leverage Reinforcement Learning (RL) techniques to adapt dynamically to changing environments and optimize performance. However, it is challenging to construct safety cases for RL components. We therefore propose the SAFE-RL (Safety and Accountability Framework for Evaluating Reinforcement Learning) for supporting the development, validation, and safe deployment of RL-based CPS. We adopt a design science approach to construct the framework and demonstrate its use in three RL applications in small Uncrewed Aerial systems (sUAS).

Cyber-Physical Systems (CPS) increasingly leverage Reinforcement Learning (RL) to adapt dynamically to changing environments and optimize performance over time. While RL enhances efficiency and safety by enabling autonomous adjustments to unexpected conditions and hazard avoidance, it also introduces significant risks, as learned behaviors may lead to unpredictable or unsafe actions in real-world deployment. Therefore, integrating risk management into RL system design is essential. In this paper, we propose the QuestRL Framework, a question-driven approach that translates high-level safety guidelines into RL-specific considerations. This framework helps RL practitioners address key risks early in development, informing new or existing system requirements while ensuring traceability to risk management objectives. To evaluate its effectiveness, we conducted a study across two use cases, engaging six RL experts in developing system requirements with and without the framework. Our findings suggest that the framework promotes critical thinking and helps practitioners identify additional risk factors, ultimately supporting safer RL deployment.

CyberPhysical systems (CPS) must be closely monitored to identify and potentially mitigate emergent problems that arise during their routine operations. However, the multivariate time-series data which they typically produce can be complex to understand and analyze. While formal product documentation often provides example data plots with diagnostic suggestions, the sheer diversity of attributes, critical thresholds, and data interactions can be overwhelming to non-experts who subsequently seek help from discussion forums to interpret their data logs. Deep learning models, such as Long Short-term memory (LSTM) networks can be used to automate these tasks and to provide clear explanations of diverse anomalies detected in real-time multivariate data-streams. In this paper we present RESAM, a requirements process that integrates knowledge from domain experts, discussion forums, and formal product documentation, to discover and specify requirements and design definitions in the form of time-series attributes that contribute to the construction of effective deep learning anomaly detectors. We present a case-study based on a flight control system for small Uncrewed Aerial Systems and demonstrate that its use guides the construction of effective anomaly detection models whilst also providing underlying support for explainability. RESAM is relevant to domains in which open or closed online forums provide discussion support for log analysis.