Abstract Advances in deep learning have revolutionized cyber‐physical applications, including the development of autonomous vehicles. However, real‐world collisions involving autonomous control of vehicles have raised significant safety concerns regarding the use of deep neural networks (DNNs) in safety‐critical tasks, particularly perception. The inherent unverifiability of DNNs poses a key challenge in ensuring their safe and reliable operation. In this work, we propose perception simplex ( ), a fault‐tolerant application architecture designed for obstacle detection and collision avoidance. We analyse an existing LiDAR‐based classical obstacle detection algorithm to establish strict bounds on its capabilities and limitations. Such analysis and verification have not been possible for deep learning‐based perception systems yet. By employing verifiable obstacle detection algorithms, identifies obstacle existence detection faults in the output of unverifiable DNN‐based object detectors. When faults with potential collision risks are detected, appropriate corrective actions are initiated. Through extensive analysis and software‐in‐the‐loop simulations, we demonstrate that provides deterministic fault tolerance against obstacle existence detection faults, establishing a robust safety guarantee.

Entities in multiagent systems may seek conflicting subobjectives, and this leads to competition between them. To address performance degradation due to competition, we consider a bilevel lottery where a social planner at the high level selects a reward first and, sequentially, a set of players at the low level jointly determine a Nash equilibrium given the reward. The social planner is faced with efficiency losses where a Nash equilibrium of the lottery game may not coincide with the social optimum. We propose an optimal bilevel lottery design problem as finding the least reward and perturbations such that the induced Nash equilibrium produces the socially optimal payoff. We formally characterize the price of anarchy and the behavior of public goods and Nash equilibrium with respect to the reward and perturbations. We relax the optimal bilevel lottery design problem via a convex approximation and identify mild sufficient conditions under which the approximation is exact.

Perception of obstacles remains a critical safety concern for autonomous vehicles. Real-world collisions have shown that the autonomy faults leading to fatal collisions originate from obstacle existence detection. Open source autonomous driving implementations show a perception pipeline with complex interdependent Deep Neural Networks. These networks are not fully verifiable, making them unsuitable for safety-critical tasks. In this work, we present a safety verification of an existing LiDAR based classical obstacle detection algorithm. We establish strict bounds on the capabilities of this obstacle detection algorithm. Given safety standards, such bounds allow for determining LiDAR sensor properties that would reliably satisfy the standards. Such analysis has as yet been unattainable for neural network based perception systems. We provide a rigorous analysis of the obstacle detection system with empirical results based on real-world sensor data.

Mobile robots such as unmanned vehicles integrate heterogeneous capabilities in sensing, computation, and control. They are representative cyber-physical systems where the cyberspace and the physical world are strongly coupled. However, the safety of mobile robots is significantly threatened by cyber/physical attacks and software/hardware failures. These threats can thwart normal robot operations and cause robot misbehaviors. In this paper, we propose a novel anomaly detection system, which leverages physical dynamics of mobile robots to detect misbehaviors in sensors and actuators. We explore issues raised in real-world implementations, e.g., distinctive robot dynamic models, sensor quantity and quality, decision parameters, etc., for practicality purposes. We implement the detection system on two types of mobile robots and evaluate the detection performance against various misbehavior scenarios, including signal interference, sensor spoofing, logic bomb and physical jamming. The experiments show detection effectiveness and small detection delays.