In safety-critical reinforcement learning tasks such as robot navigation, autonomous driving, and industrial control, real-time monitoring of agent behavior is paramount to prevent unsafe actions from compromising system reliability. We introduce Safe Fuzzy Control Barrier Functions (Safe Fuzzy-CBF), a monitoring framework that continuously evaluates agent risk using a smooth control barrier function learned from data via Takagi-Sugeno fuzzy rules, rather than relying on a predefined model-based barrier. In the offline phase, we extract fuzzy rules from trajectory data and construct a continuously differentiable barrier that unifies state, action, and cost into a single risk metric. During online execution, we estimate the barrier’s time derivative using finite differences, apply Lipschitz-based corrections, and when the barrier condition is violated, we solve a small quadratic program in the normalized control space to compute a safe corrective action. Evaluated on three Safety-Gymnasium navigation tasks and the MetaDrive autonomous driving environment, our framework detects emerging safety violations in real time and enforces runtime safety without degrading the underlying RL policy’s performance, achieving up to an 86.8% reduction in accumulated cost.

In this paper we investigate a supervisory control problem of discrete event systems under attacks. Specially, we consider a type of sensor deception attacks, called replacement attacks, under which the intruder confuses the observation of events by substituting the occurrence of an observation with another. We use labeled Petri nets as the reference formalism to model a discrete event system and represent control specifications in terms of generalized mutual exclusion constraints (GMECs). The concept of a monitor function is proposed to describe the satisfiability of GMECs given an observation by counting the number of occurrences of each label. Due to the existence of attacks, some labels generated by a plant are prone to be altered by an attacker, interfering with a supervisor such that it cannot make correct control decisions. For assisting the monitor function to estimate the number of occurrences of those altered-prone labels, generated by the plant, the notion of label dependency is introduced. Accordingly, a monitor-based supervisor is designed with low online computational effort, avoiding the marking estimation or the reachability analysis of the system. It is verified that the proposed supervisor not only enforces all GMECs no matter whether or not replacement attacks occur, but also keeps the system's behavior as permissive as possible. Note to Practitioners—Cyber physical systems (CPSs) have exhibited multifaceted applications in various fields such as process control systems, smart grids, distributed robotics, autonomous vehicles. Due to the over-reliance on communication networks, CPSs are vulnerable to attacks that can tamper the data collection processes and interfere safety critical decision making processes, resulting in catastrophic damages to the systems. In the frame of discrete event systems, most of existing supervisory control strategies of CPSs under attacks rely on an exhaustive reachability analysis, which is computationally expensive, making such approaches hardly applicable to large systems. In order to address this issue, this work considers a type of sensor deception attacks, called replacement attacks, and proposes a monitor-based supervisor policy, enforcing the control specifications of the systems in the presence of replacement attacks. Without requiring tedious analysis, the designed online supervisor has low computational effort and control decisions only depend on a direct analysis of the observation sequence.

For automated manufacturing systems, the reachability graph analysis based on Petri nets faces with the state explosion challenge. This paper proposes an effective partial order technique for generalized system of simple sequential processes with unreliable resources by constructing improved weak-persistent graph. First, we review the partition technique of reachability graph which divides the marking type into legal and forbidden markings. Then, the definition of weak-persistent sets is introduced. Comparing with the original persistent set, the weak-persistent set achieves a better reduction in the state-space generation. However, it cannot be used directly in the system with unreliable resource due to uncontrollable transitions. Therefore, the concept of improved weak-persistent sets is proposed to reduce the reachability graph of generalized system of simple sequential processes with unreliable resources. Furthermore, it is showed that the improved weak-persistent graph can provide significant reduction compared with reachability graph while preserving all forbidden markings. Finally, the examples are given to demonstrate our method.

This paper proposes an adaptive supervisory control policy for an automated manufacturing system (AMS) with multiple types of unreliable resources. The considered AMS is a generalized system of simple sequential process with resources (GS$^{3}$PR). In order to model resource failures and recoveries, modified recovery subnets are developed, which model more complex resource failure scenarios compared with traditional recovery subnets. For the purpose of guaranteeing the execution of a system’s production successfully, from the perspective of the structural analysis of the system, a siphon control-based technique is introduced and a type of controller is developed. It is verified that the proposed adaptive control policy assures the deadlock-freeness of the controlled system no matter if there exist resource failures, and retains all behavior of the original system. Finally, examples are presented to demonstrate the proposed method.Note to Practitioners—Resources failures are ubiquitous in an automated manufacturing system (AMS). Their occurrences can lead partial or global system stagnation, which causes unnecessary downtime and even catastrophic results. Most of the existing robust deadlock control strategies for AMSs with unreliable resources usually restrict system evolution in an unnecessarily conservative way such that part of permissive behavior is excluded if no resource failure occurs, which affects the utilization of resources and reduces the productivity of the system. In order to address the issue, this work considers a type of AMSs where each processing stage may require an arbitrary number of units, but all from a single resource type. For such an AMS, we propose an adaptive deadlock control policy that makes the controlled system deadlock-free, keeping all the system behavior as in the case that unreliable resources do not break down. The proposed method is realized by dynamically adjusting the working modes of a system to adapt to resource failures and recoveries, which is readily accessible to practitioners from a technical viewpoint.

In an automated manufacturing system (AMS), resources are, in general, subject to unpredictable failures, which invalidate many existing deadlock control strategies. In this article, we propose an adaptive deadlock control policy for an AMS with multiple types of unreliable resources. The considered AMS is modeled with a system of simple sequential processes with resources. First, based on an elementary siphon control method, monitors are added for elementary siphons and some particular dependent siphons to ensure the liveness of a system if there are no resource failures. By considering the fact that an unreliable resource may fail in a system, recovery subnets are added to describe the resource failures and recoveries. Since a monitor added for a siphon may not be able to guarantee that the corresponding siphon is always marked if the failure of a resource in the siphon occurs, the concept of switch controllers is presented so as to make the siphon always remarked if it is emptied by resource failures. It is verified that the adaptive controller proposed in this article can guarantee the liveness of the controlled system no matter whether unreliable resources break down or not. More importantly, if there is no resource failure, the system can maintain predefined production without degrading planned system performance. Finally, examples are presented to illustrate the validity of the proposed method.

Due to extensive use and wear of equipment, the occurrence of failures in automated manufacturing systems (AMSs) remains inevitable. However, with maintenance policies, the reliability and availability of such systems can be increased. This paper presents different basic models based on stochastic Petri nets allowing the modeling of the integration of corrective and preventive maintenance in unreliable manufacturing systems and analysis of their effects on system's productivity. In all these models, we make sure that after having undergone a preventive or corrective operation, a resource is functioning normally. This property avoids the execution in the model of an infinite failure loop. Finally, we show how the proposed modeling of these maintenance strategies can be integrated in unreliable controlled manufacturing systems without causing failure-induced deadlocks.

The paper focuses on robust scheduling for flexible manufacturing systems (FMSs) with unreliable resources modelled by timed Petri nets (TPNs). For this purpose, the paper shows that schedulers based on beam search method can be combined with liveness-enforcing supervisors in a straightforward way. This combination is motivated by some properties in the set of reachable markings of a TPN. The use of the liveness-enforcing supervisor improves the robustness of the scheduling without degrading the performance for a particular class of job-shop systems. An example is given to illustrate the proposed strategy.

Resource failures may happen in automated manufacturing systems (AMSs) because of different reasons in the real world, making most existing deadlock control policies unapplicable. This paper develops methods for the robust deadlock control of AMSs with unreliable resources based on Petri nets. The considered AMSs are modeled with generalized systems of simple sequential processes with resources (GS3PR). First, a method based on reachability graph partition technique is provided to analyze the robust legal markings and the forbidden ones in an unreliable GS3PR (U-GS3PR), in which resource failures and recovery procedures are modeled with recovery subnets. Then, the control problem for such a system is converted into a problem for controlling the forbidden states in a U-GS3PR and control places can be designed by solving the maximal number of forbidden markings problems. Since the robust legal reachability spaces computed may be nonconvex and such a system cannot be optimally controlled by the conjunctions of linear constraints, we propose an interval-inhibitor-arc-based robust deadlock control policy for a system with nonconvex legal reachability spaces by solving the maximal number of tq-critical marking/transition separation instances problems (MNTMPs(tq)). Finally, examples are presented to demonstrate the proposed methods.

Siphons play an important role in the development of deadlock control methods by using Petri nets. The number of siphons increases exponentially with respect to the size of a Petri net. This article presents a symbolic approach to the computation of minimal siphons in Petri nets by using binary decision diagrams (BDD). The siphons of a Petri net can be found via a set of logic conditions. The logic conditions are symbolically modeled by using Boolean algebras. The operations of Boolean algebras are implemented by BDD that are capable of representing large sets of siphons with small shared data structures. The proposed method first uses BDD to compute all siphons of a Petri net and then a binary relation is designed to extract all minimal siphons. Finally, by using a number of examples, the efficiency of the proposed method is verified through different-sized problems.

In each of Fig. 1(a), (c), (e), and (f) in the above-named article [ibid., vol. 42, no. 1, pp. 178-191, Jan. 2012], there is an arc from transition t3 to place p6, which is incorrect. The arc between the transition and the place should be from p6 to t3. The Petri net depicted in Fig. 1(a) is a well-known example that has been widely utilized in the literature.

Deadlocks are an undesirable situation in automated flexible manufacturing systems (FMS). Their occurrences often deteriorate the utilization of resources and may lead to catastrophic results. Finding an optimal supervisor is NP-hard. A computationally efficient method often ends up with a suboptimal one. This paper develops a deadlock prevention method that makes a good tradeoff between optimality and computational tractability for a class of Petri nets, which can model many FMS. The theory of regions guides our efforts toward the development of near-optimal solutions for deadlock prevention. Given a plant net, a minimal initial marking is first decided by structural analysis, and an optimal live controlled system is computed. Then, a set of inequality constraints is derived with respect to the markings of monitors and the places in the model such that no siphon can be insufficiently marked. A method is proposed to identify the redundancy condition for constraints. For a new initial marking of the plant net, a deadlock-free controlled system can be obtained by regulating the markings of the monitors such that the inequality constraints are satisfied, without changing the structure of the controlled system. The near-optimal performance of a controlled net system via the proposed method is shown through several examples.

In the above paper, it proposes a deadlock prevention policy for a flexible manufacturing system (FMS), which needs the complete state enumeration of the FMS that is modeled with Petri nets. The reachability graph of a Petri-net model is divided into two parts: the live zone (LZ) and the deadlock zone (DZ). The states in the LZ of the reachability graph of a Petri net constitute the legal behavior of the net from the viewpoint of deadlock prevention. The concept of first-met bad markings is proposed. A first-met bad marking is a node in DZ, whose father nodes are in LZ. The deadlock prevention policy is used in an iterative way. At each iteration, a first-met bad marking is identified from the reachability graph of a Petri net to be controlled. The reachability of a first-met bad marking is prohibited by adding a monitor, establishing a marking invariance relationship between the additional monitor and the activity places that are marked under the first-met bad marking. To achieve this, without a formal proof, [Lemma 1] is developed as shown in this article.