Bayesian Optimization (BO) is a widely used technique for optimizing black-box functions, whose mathematical form is unknown and can only be evaluated through costly simulations. In this work, we focus on optimizing functions that are the \(\min/\max\) of several components, where each component is typically a non-linear, non-convex black-box function. Traditional BO approaches may suffer from the masking effect , where components with lower mean values are sampled more, even if they do not contain the global optimum. In our prior work, Minimum Bayesian Optimization ( minBO ) addressed this issue by proposing a sampling approach that uses a surrogate for each component and samples based on predicted improvement across all components. While effective, minBO treats each component independently and does not capture potential dependencies between components. Additionally, estimating surrogates for each component can limit the scalability of the approach. We introduce Conjunctive Bayesian Optimization ( conBO ), a novel approach that overcomes these limitations. We propose a paired sampling algorithm ( conBO-PS ) that considers dependencies between components by analyzing all pairs of functions and estimating the distribution of the minimum of two functions. While conBO-PS accounts for dependencies, it is computationally expensive. To improve efficiency, we introduce conBO large-scale ( conBO-LS ), which adapts conBO-PS by considering a subset of components chosen based on their potential impact, allowing users to control computational effort. We evaluate the performance of these algorithms on non-linear synthetic functions and also compare them to state-of-the-art methods in the context of falsifying conjunctive safety requirements for Cyber-Physical Systems (CPS). A conjunctive safety requirement refers to a set of safety conditions (requirements) tested together, where if at-least one condition is falsified, the entire conjunctive requirement is considered falsified. In fact, in such context the function to be minimized is the minimum of several sub-components. Results show that conBO-PS and conBO-LS outperform existing approaches, offering better solution quality and computational efficiency. In the CPS application, the proposed approaches achieve faster falsification and improved falsification rates across all benchmarks.

Requirements driven search-based testing (also known as falsification) has proven to be a practical and effective method for discovering erroneous behaviors in Cyber-Physical Systems. Despite the constant improvements on the performance and applicability of falsification methods, they all share a common characteristic. Namely, they are best-effort methods which do not provide any guarantees on the absence of erroneous behaviors (falsifiers) when the testing budget is exhausted. The absence of finite time guarantees is a major limitation which prevents falsification methods from being utilized in certification procedures. In this paper, we address the finite-time guarantees problem by developing a new stochastic algorithm. Our proposed algorithm not only estimates (bounds) the probability that falsifying behaviors exist, but also identifies the regions where these falsifying behaviors may occur. We demonstrate the applicability of our approach on standard benchmark functions from the optimization literature and on the F16 benchmark problem.Note to Practitioners—The safety assurance problem for Cyber-Physical Systems (CPS) remains an open challenge. To demonstrate functional safety, practitioners must collect evidence that establishes that a system performs as expected under certain assumptions. The expected system behavior is typically captured through functional correctness requirements. In the case of CPS, evidence typically takes the form of test cases that are executed both on a model of the system and/or on the actual system. One of the challenges in producing such evidence is how to automatically generate test cases which are representative of the infinite execution space of CPS. Search-based test generation (SBTG) is a class of methods that can automatically generate test cases for CPS while being guided by the functional requirements. As SBTG methods try to discover test cases that invalidate, i.e., falsify, the requirements, they also collect validating, i.e., satisfying, test cases that can be used as evidence. This work introduces a method that can assess whether enough test cases have been executed given a finite testing budget. The sufficiency of the test suite is assessed by computing the probability that invalidating system behaviors may exist but have not yet been discovered. The practitioner can then adjust the number of test cases generated until a desired degree of confidence on the probability is achieved. Hence, our method not only works as an automated test case generation algorithm, but also as a method that provides formal functional performance guarantees on the system. Future directions will investigate extensions of our method to stochastic CPS.

We propose a framework for security vulnerability analysis for Cyber-Physical Systems (CPS). Our framework imposes only minimal assumptions on the structure of the CPS. Namely, we consider CPS with feedback control loops, state observers, and anomaly detection algorithms. Moreover, our framework does not require any knowledge about the dynamics or the algorithms used in the CPS. Under this common CPS architecture, we develop tools that can identify vulnerabilities in the system and their impact on the functionality of the CPS. We pose the CPS security problem as a falsification (or Search Based Test Generation (SBTG)) problem guided by security requirements expressed in Signal Temporal Logic (STL). We propose two different categories of security requirements encoded in STL: (1) detectability (stealthiness) and (2) effectiveness (impact on the CPS function). Finally, we demonstrate in simulation on an inverted pendulum and on an Unmanned Aerial Vehicle (UAV) that both specifications are falsifiable using our SBTG techniques.

In the paper titled " Stealthy attacks formalized as STL formulas for Falsification of CPS Security", we investigate a broad class of attacks on the sensor and actuation blocks in the form of additive perturbation that impacts the measurement and control, respectively. In this demo, we demonstrate the usage of our framework and the underlying technologies along with a case study on aviation systems using Microsoft Flight Simulator (MSFS).