In safe offline reinforcement learning, the objective is to develop a policy that maximizes cumulative rewards while strictly adhering to safety constraints, utilizing only offline data. Traditional methods often face difficulties in balancing these constraints, leading to either diminished performance or increased safety risks. We address these issues with a novel approach that begins by learning a conservatively safe policy through the use of Conditional Variational Autoencoders, which model the latent safety constraints. Subsequently, we frame this as a Constrained Reward-Return Maximization problem, wherein the policy aims to optimize rewards while complying with the inferred latent safety constraints. This is achieved by training an encoder with a reward-Advantage Weighted Regression objective within the latent constraint space. Our methodology is supported by theoretical analysis, including bounds on policy performance and sample complexity. Extensive empirical evaluation on benchmark datasets, including challenging autonomous driving scenarios, demonstrates that our approach not only maintains safety compliance but also excels in cumulative reward optimization, surpassing existing methods. Additional visualizations provide further insights into the effectiveness and underlying mechanisms of our approach.

Abstract Cyber-physical systems are complex systems that require the integration of diverse software, firmware, and hardware to be practical and useful. This increased complexity is impacting the management of models necessary for designing cyber-physical systems that are able to take into account a number of “-ilities”, such that they are safe and secure and ultimately resilient to disruption of service. We propose an ontological metamodel for system design that augments an already existing industry metamodel to capture the relationships between various model elements (requirements, interfaces, physical, and functional) and safety, security, and resilient considerations. Employing this metamodel leads to more cohesive and structured modeling efforts with an overall increase in scalability, usability, and unification of already existing models. In turn, this leads to a mission-oriented perspective in designing security defenses and resilience mechanisms to combat undesirable behaviors. We illustrate this metamodel in an open-source GraphQL implementation, which can interface with a number of modeling languages. We support our proposed metamodel with a detailed demonstration using an oil and gas pipeline model.

Our work focuses on modeling the security of systems from their component-level designs. Towards this goal, we develop a categorical formalism to model attacker actions. Equipping the categorical formalism with algebras produces two interesting results for security modeling. First, using the Yoneda lemma, we can model attacker reconnaissance missions. In this context, the Yoneda lemma shows us that if two system representations, one being complete and the other being the attacker’s incomplete view, agree at every possible test, they behave the same. The implication is that attackers can still successfully exploit the system even with incomplete information. Second, we model the potential changes to the system via an exploit. An exploit either manipulate the interactions between system components, such as providing the wrong values to a sensor, or changes the components themselves, such as controlling a global positioning system (GPS). One additional benefit of using category theory is that mathematical operations can be represented as formal diagrams, helpful in applying this analysis in a model-based design setting. We illustrate this modeling framework using an unmanned aerial vehicle (UAV) cyber-physical system model. We demonstrate and model two types of attacks (1) a rewiring attack, which violates data integrity, and (2) a rewriting attack, which violates availability.

Safe navigation of autonomous agents in human centric environments requires the ability to understand and predict motion of neighboring pedestrians. However, predicting pedestrian intent is a complex problem. Pedestrian motion is governed by complex social navigation norms, is dependent on neighbors' trajectories and is multimodal in nature. In this work, we propose SCAN, a Spatial Context Attentive Network that can jointly predict socially-acceptable multiple future trajectories for all pedestrians in a scene. SCAN encodes the influence of spatially close neighbors using a novel spatial attention mechanism in a manner that relies on fewer assumptions, is parameter efficient, and is more interpretable compared to state-of-the-art spatial attention approaches. Through experiments on several datasets we demonstrate that our approach can also quantitatively outperform state of the art trajectory prediction methods in terms of accuracy of predicted intent.

Cyber-physical systems require the construction and management of various models to assure their correct, safe, and secure operation. These various models are necessary because of the coupled physical and computational dynamics present in cyber-physical systems. However, to date the different model views of cyber-physical systems are largely related informally, which raises issues with the degree of formal consistency between those various models of requirements, system behavior, and system architecture. We present a category-theoretic framework to make different types of composition explicit in the modeling and analysis of cyber-physical systems, which could assist in verifying the system as a whole. This compositional framework for cyber-physical systems gives rise to unified system models, where system behavior is hierarchically decomposed and related to a system architecture using the systems-as-algebras paradigm. As part of this paradigm, we show that an algebra of (safety) contracts generalizes over the state of the art, providing more uniform mathematical tools for constraining the behavior over a richer set of composite cyber-physical system models, which has the potential of minimizing or eliminating hazardous behavior.

Static analysis tools are programs that run on source code prior to their compilation to binary executables and attempt to find flaws or defects in the code during the early stages of development. If left unresolved, these flaws could pose security risks. While numerous static analysis tools exist, there is no single tool that is optimal. Therefore, many static analysis tools are often used to analyze code. Further, some of the alerts generated by the static analysis tools are low-priority or false alarms. Machine learning algorithms have been developed to distinguish between true alerts and false alarms, however significant man hours need to be dedicated to labeling data sets for training. This study investigates the use of active learning to reduce the number of labeled alerts needed to adequately train a classifier. The numerical experiments demonstrate that a query by committee active learning algorithm can be utilized to significantly reduce the number of labeled alerts needed to achieve similar performance as a classifier trained on a data set of nearly 60,000 labeled alerts.

The performance of autonomous cars can be greatly enhanced through wireless coordination. However, mobility has traditionally been a challenge for wireless networks due to rapid fluctuation of the signal quality. Current control systems handle this challenge by slowing down the vehicle to preserve safety. However, in this research, we demonstrate that we can robustly characterize the channel quality by mapping the multipath signals to the dynamics of the physical environment, thus controlling the trajectory of the mobile agent to a safe efficient motion path. This allows mobile systems to realize the performance benefits of wireless coordination while providing safety.

Today, there is a plethora of software security tools employing visualizations that enable the creation of useful and effective interactive security analyst dashboards. Such dashboards can assist the analyst to understand the data at hand and, consequently, to conceive more targeted preemption and mitigation security strategies. Despite the recent advances, model-based security analysis is lacking tools that employ effective dashboards-to manage potential attack vectors, system components, and requirements. This problem is further exacerbated because model-based security analysis produces significantly larger result spaces than security analysis applied to realized systems-where platform specific information, software versions, and system element dependencies are known. Therefore, there is a need to manage the analysis complexity in model-based security through better visualization techniques. Towards that goal, we propose an interactive security analysis dashboard that provides different views largely centered around the system, its requirements, and its associated attack vector space. This tool makes it possible to start analysis earlier in the system lifecycle. We apply this tool in a significant area of engineering design-the design of cyber-physical systems-where security violations can lead to safety hazards.

Recent advancements in Intelligent Transportation Systems suggest that the roads will gradually be filled with autonomous vehicles that are able to drive themselves while communicating with each other and the infrastructure. Autonomous intersection management is among the more challenging traffic scenarios, which involves coordinating the movement of autonomous vehicles through a conflict zone. Intersection management is also potentially one of the more beneficial traffic scenarios in terms of mobility and environmental impact. In this paper we propose a platoon-based approach for the cooperative intersection management problem. We assert that leveraging the platooning capability of autonomous vehicles could improve the efficiency of any policy at an intersection, in terms of average delay time per vehicle and can reduce the communication overhead in the vicinity of intersections by a factor of up to the average platoon size. We also develop a new autonomous intersection management method that guarantees the safety of traffic by allowing one platoon in the conflict zone at any time. We examine the effects of platooning on a simple stop sign at a single 4-way intersection in a simulated environment and report the results in terms of average delay per vehicle and communication overhead. Moreover, we evaluate the performance of the proposed method in a simulated environment and compare the results in terms of average wait time per vehicle and variance in delay with that of a stop sign.

The so-called “ilities” of a system-such as safety, interoperability, and efficiency-should be designed into systems from their very conception, which can be achieved by integrating more powerful analysis techniques into the general systems engineering process. The primary barrier to achieving this objective is the lack of effectiveness of the existing analytical tools during early concept development. This paper introduces a new technique, which is based on general systems theory and hierarchical control theory, that can capture behaviors that are prevalent in complex human- and software-intensive systems. This paper presents a new technique that supports rigorous, systematic analysis of future concepts in order to identify potentially adverse scenarios and undocumented assumptions that may inhibit mission objectives. Furthermore, these techniques are intended to address issues associated with the interaction, integration, and coordination of agents with heterogeneous decision-making capability and qualities, varying levels of automation, and changing roles of human operators.

As transportation systems become increasingly complex and the roles of human operators and autonomous software continue to evolve, traditional safety-related analytical methods are becoming inadequate. Traditional hazard analysis tools are based on an accident causality model that does not capture many of the complex behaviors found in modern engineered systems. Additionally, these traditional approaches are most effective during the late stages of system development, when detailed design information is available. However, system safety cannot be cost-effectively assured by discovering problems at these late stages and adding expensive updates to the design. Rather, safety should be designed into complex intelligent transportation systems from their very conception, which can be achieved by integrating powerful hazard analysis techniques into the general systems engineering process. The primary barrier to achieving this objective is the lack of effectiveness of the existing analytical tools during early concept development. This paper introduces a new technique, which is based on a systems- and control-theoretic model of accident causality that can capture behaviors that are prevalent in these complex software-intensive systems. The goals are to (1) develop rigorous systematic tools for the analysis of future concepts to identify potentially hazardous scenarios and undocumented assumptions and to (2) extend these tools to assist stakeholders in the development of concepts using a safety-driven approach. Current work focuses on air transportation, but future goals of this research are to extend to and generalize all modes of transportation.