A metastable failure is a self-sustaining congestive collapse in which a system degrades in response to a transient stressor (e.g., a load surge) but fails to recover after the stressor is removed. These rare but potentially catastrophic events are notoriously hard to diagnose and mitigate, sometimes causing prolonged outages affecting millions of users.

Learning to control an unknown dynamical system with respect to high-level temporal specifications is an important problem in control theory. We present the first regret-free online algorithm for learning a controller for linear temporal logic (LTL) specifications for systems with unknown dynamics. We assume that the underlying (unknown) dynamics is modeled by a finite-state and action Markov decision process (MDPs). Our core technical result is a regret-free learning algorithm for infinite-horizon reach-avoid problems on MDPs. For general LTL specifications, we show that the synthesis problem can be reduced to a reach-avoid problem once the graph structure is known. Additionally, we provide an algorithm for learning the graph structure, assuming knowledge of a minimum transition probability, which operates independently of the main regret-free algorithm. Our LTL controller synthesis algorithm provides sharp bounds on how close we are to achieving optimal behavior after a finite number of learning episodes. In contrast, previous algorithms for LTL synthesis only provide asymptotic guarantees, which give no insight into the transient performance during the learning phase.

This paper presents a step towards a formal controller design method for autonomous agents based on knowledge awareness to improve decision-making. Our approach is to first create an organized repository of information (a knowledge base) for autonomous agents which can be accessed and then translated into temporal specifications. Secondly, to develop a controller with formal guarantees that meets a combination of mission-specific objective and the specification from the knowledge base, we utilize an abstraction-based controller design (ABCD) approach, capable of managing both nonlinear dynamics and temporal requirements. Unlike the conventional offline ABCD approach, our method dynamically updates the controller whenever the knowledge base prompts changes in the specifications. A three-dimensional nonlinear car model navigating an urban road scenario with traffic signs and obstacles is considered for validation. Results show the effectiveness of the method in guiding the autonomous agents to the target while complying with the knowledge base and the mission-specific objective.

Abstraction-based techniques are an attractive approach for synthesizing correct-by-construction controllers to satisfy high-level temporal requirements. A main bottleneck for successful application of these techniques is the memory requirement, both during controller synthesis (to store the abstract transition relation) and in controller deployment (to store the control map). We propose memory-efficient methods for mitigating the high memory demands of the abstraction-based techniques using neural network representations . To perform synthesis for reach-avoid specifications, we propose an on-the-fly algorithm that relies on compressed neural network representations of the forward and backward dynamics of the system. In contrast to usual applications of neural representations, our technique maintains soundness of the end-to-end process. To ensure this, we correct the output of the trained neural network such that the corrected output representations are sound with respect to the finite abstraction. For deployment, we provide a novel training algorithm to find a neural network representation of the synthesized controller and experimentally show that the controller can be correctly represented as a combination of a neural network and a look-up table that requires a substantially smaller memory. We demonstrate experimentally that our approach significantly reduces the memory requirements of abstraction-based methods. We compare the performance of our approach with the standard abstraction-based synthesis on several models. For the selected benchmarks, our approach reduces the memory requirements respectively for the synthesis and deployment by a factor of 1.31× 10 5 and 7.13× 10 3 on average, and up to 7.54× 10 5 and 3.18× 10 4 . Although this reduction is at the cost of increased off-line computations to train the neural networks, all the steps of our approach are parallelizable and can be implemented on machines with higher number of processing units to reduce the required computational time.

We study fundamental reachability problems on pseudo-orbits of linear dynamical systems. Pseudo-orbits can be viewed as a model of computation with limited precision and pseudo-reachability can be thought of as a robust version of classical reachability. Using an approach based on $o$-minimality of $\reals_{\exp}$ we prove decidability of the discrete-time pseudo-reachability problem with arbitrary semialgebraic targets for diagonalisable linear dynamical systems. We also show that our method can be used to reduce the continuous-time pseudo-reachability problem to the (classical) time-bounded reachability problem, which is known to be conditionally decidable.

We study fundamental decision problems on linear dynamical systems in discrete time. We focus on pseudo-orbits, the collection of trajectories of the dynamical system for which there is an arbitrarily small perturbation at each step. Pseudo-orbits are generalizations of orbits in the topological theory of dynamical systems. We study the pseudo-orbit problem, whether a state belongs to the pseudo-orbit of another state, and the pseudo-Skolem problem, whether a hyperplane is reachable by an ε-pseudo-orbit for every ε. These problems are analogous to the well-studied orbit problem and Skolem problem on unperturbed dynamical systems. Our main results show that the pseudo-orbit problem is decidable in polynomial time and the Skolem problem on pseudo-orbits is decidable. The former extends the seminal result of Kannan and Lipton from orbits to pseudo-orbits. The latter is in contrast to the Skolem problem for linear dynamical systems, which remains open for proper orbits.

We consider the time-bounded reachability problem for continuous-time Markov decision processes. We show that the problem is decidable subject to Schanuel’s conjecture. Our decision procedure relies on the structure of optimal policies and the conditional decidability (under Schanuel’s conjecture) of the theory of reals extended with exponential and trigonometric functions over bounded domains. We further show that any unconditional decidability result would imply unconditional decidability of the bounded continuous Skolem problem, or equivalently, the problem of checking if an exponential polynomial has a non-tangential zero in a bounded interval. We note that the latter problems are also decidable subject to Schanuel’s conjecture but finding unconditional decision procedures remain longstanding open problems.

We propose an optimization for space-efficient implementations of explicit model-predictive controllers (MPC) for robust control of linear time-invariant (LTI) systems on embedded platforms. We obtain an explicit-form robust model-predictive controller as a solution to a multi-parametric linear programming problem. The structure of the controller is a polyhedral decomposition of the control domain, with an affine map for each domain. While explicit MPC is suited for embedded devices with low computational power, the memory requirements for such controllers can be high. We provide an optimization algorithm for a mixed-precision implementation of the controller, where the deviation of the implemented controller from the original one is within the robustness margin of the robust control problem. The core of the mixed-precision optimization is an iterative static analysis that co-designs a robust controller and a low-bitwidth approximation that is statically guaranteed to always be within the robustness margin of the original controller. We have implemented our algorithm and show on a set of benchmarks that our optimization can reduce space requirements by up to 20.9% and on average by 12.6% compared to a minimal uniform precision implementation of the original controller.