Anomaly detection and diagnosis algorithms for Cyber-Physical Systems, especially Cyber-Physical Production Systems, are increasingly data-driven and rely on sufficient and representative data to be fitted. However, recording such data comes with cost, especially for rare or unsafe operational states. While simulation offers a scalable solution, by generating synthetic data, it often comes with a gap between simulated and real-world environments. In this paper, we investigate the impact of a pretraining with simulation-generated data on anomaly detection algorithms in a case study of a model Cyber-Physical Production Systems and its simulation. In a first step, we therefore train different anomaly detection algorithms on simulated data, and subsequently continue training with real data. We examine, (i) whether pretraining with additional synthetic data enhances the performance of anomaly detection algorithms, and (ii) how the proportion of real versus synthetic data affects a model’s effectiveness when operating on a fixed data budget. Our findings show that pretraining on simulation generated data can increase the performance of anomaly detection algorithms, however, solely training on simulated data is leading to a decrease in performance.

Reliable sensor data are essential for the effective operation and safety of cyber-physical systems (CPS) in industrial environments. However, sensors frequently experience faults or degradation, leading to compromised system performance. In order to increase the resilience of CPS, this paper proposes a novel approach to creating virtual sensors capable of reconstructing missing or faulty sensor data through gradient-based input reconstruction by leveraging neural networks. Specifically, we employ an LSTM-based autoencoder architecture trained both conventionally and with a masking strategy to handle potential sensor data loss scenarios effectively. Our method involves using automatic differentiation and gradient descent to iteratively optimize missing sensor inputs, guided by the pretrained network. We evaluate this approach comprehensively on both simulated and real-world plant data from cyber-phyiscal process plants, demonstrating robust reconstruction performance across various sensor failure scenarios. Additionally, we explore the efficacy of modular clustering methods versus single comprehensive models, highlighting the advantages and limitations inherent to each approach. Our findings reveal significant potential for improving system resilience and maintaining operational continuity in CPS through advanced virtual sensor implementations.

A long-term objective for many research fields, such as anomaly detection, discretization and root-cause diagnosis in Cyber-Phyiscal Production Systems is the realization of resilient and highly autonomous systems. Instances of those systems range from the control and operation of single production systems to controlling entire plants. While notable progress has been made, a key challenge remains unaddressed: the availability of comprehensive and standardized datasets necessary for advancing machine learning based solutions. Typical evaluation datasets comprise systems of (too) little complexity or are either suitable for only data-driven or only for symbolic methods. Yet, a comprehensive dataset and model for developing, training, and testing machine learning methods from anomaly detection to fault diagnosis in a structured and comparable manner does not exist. To bridge this gap, we present a benchmark specifically designed to support both data-driven methods and symbolic reasoning approaches, by extending the BerFiPl benchmark introduced by Ehrhardt et al. [1]. By providing hybrid data streams, labeled system states, and a modular, interpretable system structure, the dataset offers a unique opportunity to develop, train, test, and compare hybrid AI methods to bridge the gap between data-driven and symbolic paradigms.

Empirical research plays a fundamental role in the machine learning domain. At the heart of impactful empirical research lies the development of clear research hypotheses, which then shape the design of experiments. The execution of experiments must be carried out with precision to ensure reliable results, followed by statistical analysis to interpret these outcomes. This process is key to either supporting or refuting initial hypotheses. Despite its importance, there is a high variability in research practices across the machine learning community and no uniform understanding of quality criteria for empirical research. To address this gap, we propose a model for the empirical research process, accompanied by guidelines to uphold the validity of empirical research. By embracing these recommendations, greater consistency, enhanced reliability and increased impact can be achieved.

In this paper, we tackle the problem of planning in complex numerical domains, where actions are indexed by control parameters, and their effects may be described by neural networks. We propose a lazy, hierarchical approach based on two ingredients. First, a Satisfiability Modulo Theory solver looks for an abstract plan where the neural networks in the model are abstracted into uninterpreted functions. Then, we attempt to concretize the abstract plan by querying the neural network to determine the control parameters. If the concretization fails and no valid control parameters could be found, suitable information to refine the abstraction is lifted to the Satisfiability Modulo Theory model. We contrast our work against the state of the art in NN-enriched numerical planning, where the neural network is eagerly and exactly represented as terms in Satisfiability Modulo Theories over nonlinear real arithmetic. Our systematic evaluation on four different planning domains shows that avoiding symbolic reasoning about the neural network not only leads to substantial efficiency improvements, but also enables their integration as black-box models.

Autonomously detecting anomalous behavior based on system observations is a fundamental task for Cyber-Physical Systems (CPS). Due to the high system complexity and large number of subsystems in modern CPS, rule- or knowledge-based approaches for anomaly detection are more and more replaced by Machine Learning (ML) approaches which leverage historical CPS data. Typically, ML approaches learn a system model based on the CPS data and identify anomalous behavior based on the distance of the real CPS behavior to the predicted model behavior. However, most classical ML approaches for anomaly detection are monolithic, meaning a single ML model is fitted on a global CPS observation, making them frail to spurious correlations and confounders that originate on CPS subsystem level. We hence propose a modular approach toward anomaly detection in CPS, specifically a novel Modular Neural Network (MNN) architecture. Our architecture not only models the behavior of individual CPS sub-systems in individual MNN modules, but additionally models the dependencies of the CPS subsystems into the MNN architecture. Thereby, we omit confounding effects and spurious correlations, enabling us to identify and allocate anomalies within the CPS at subsystem level. We benchmark our MNN architecture against monolithic Neural Networks and MNN architectures that do not explicitly model CPS subsystem dependencies using a real-world dataset of an industrial robot with different anomalies. We show that by modeling real-world dependencies into a MNN architecture, we can improve the performance of autonomous anomaly detection in CPS.

Cyber-Physical Production Systems (CPPS) are becoming increasingly important in modern manufacturing, which leads to a growing need for automated anomaly detection, maintenance decision-making, and fault diagnosis. Artificial Intelligence (AI) and Machine Learning (ML) algorithms are often used to perform these tasks. However, there is a shortage of real data sets with fault modes, making it difficult to train ML algorithms. To overcome this problem, we propose the use of the Flexible Production Simulation (FliPSi) to generate simulated data for the development, training and evaluation of ML algorithms. FliPSi is a simulation tool developed in Unity, a game engine, which is used to simulate modular CPPS. It allows the construction of different CPPS, data collection, and generating data which could realistically originate from CPPS. The focus of FliPSi is to generate data for the development, training and testing of anomaly detection and diagnosis algorithms. In this paper, we show that data generated with FliPSi can be used to train ML algorithms, specifically recurrent neural networks (RNNs) and variations such as Long Short-Term Memory (LSTM) and Gated Recurrent Units (GRU), which are specifically designed to process sequential data and that the algorithms trained with these data sets can be employed to detect simulated anomalies. With this, we show that FliPSi can be employed to overcome a blocking issue in using ML algorithms for anomaly detection.

To improve the autonomy of Cyber-Physical Production Systems (CPPS), a growing number of approaches in Artificial Intelligence (AI) is developed. However, implementations of such approaches are often validated on individual use-cases, offering little to no comparability. Though CPPS automation includes a variety of problem domains, existing benchmarks usually focus on single or partial problems. Additionally, they often neglect to test for AI-specific performance indicators, like asymptotic complexity scenarios or runtimes. Within this paper we identify minimum common set requirements for AI benchmarks in the domain of CPPS and introduce a comprehensive benchmark, offering applicability on diagnosis, reconfiguration, and planning approaches from AI. The benchmark consists of a grid of datasets derived from 16 simulations of modular CPPS from process engineering, featuring multiple functionalities, complexities, and individual and superposed faults. We evaluate the benchmark on state-of-the-art AI approaches in diagnosis, reconfiguration, and planning. The benchmark is made publicly available on GitHub.