The transition to connected vehicles (CVs) is shaping the future of autonomous and highly connected vehicles. As the vehicles’ reliance on increasingly complex software increases, more central points of failure are introduced. This necessitates more thorough and scalable anomaly and intrusion detection systems. Conventional approaches can struggle with the complex and dynamic structures of CVs while not providing any automatic response actions. Therefore, machine learning-based anomaly detection methods are gaining popularity. Reinforcement Learning especially has shown the potential to increase configurability and provide end-to-end solutions. Thus, this paper presents a Systematic Literature Review (SLR) on Reinforcement Learning (RL) for anomaly and intrusion detection, applicable for CVs. In it, different RL algorithms and the analysis of how they can be used for training and detecting anomalies are presented. The problem formulations for different approaches are compared, their advantages and challenges are discussed. Additionally, the potential of using RL to react to anomalies is presented, discussed and advocated for. The results of the survey indicate that many of the identified RL formulations are not inherently superior from supervised learning approaches. However, RL techniques can result in better configurability of anomaly detection system, enabling site reliability engineers to efficiently adjust the model via reward functions. Furthermore, it becomes evident that the strengths of RL-based anomaly detection are best leveraged when used for reacting to anomalies. Therefore, the authors propose to focus on the contextual or sequential decision making formulation including corrective actions, compared to the classification-based detection formulations using RL.

In safety-critical applications such as automotive or railway technology, prediction of the condition and remaining life of electronic components is a major challenge. To determine the condition, a comprehensive digital fingerprint of the components is needed, which already starts during the production phase. Currently, inspection data such as SPI, AOI and AXI data are used to group the components into two groups ("OK" and "not OK"). In the future, however, this inspection data will also be used to predict the service life and current condition of the assembly. This article deals with the investigation and processing of the imaging inspection data, as well as the geometry parameters determined from it. First, a dimensional reduction of the input data is carried out with the help of mathematical methods (PCA, t-SNE and Autoencoder). This reduction extracts the relevant information from the input data from the point of view of the AI. So far, clustering methods such as K-Means, DBSCAN and Hierarchical Clustering have been used to generate groups. In the future, these groupings can be used to predict the service life together with the loads on the components and thus make a statement about the current condition of the components

The rise of software-defined automation systems is accompanied by an increase in interconnected services and networks. Among other implications, this leads to an expansion of the attack surface for cyber threats, which in turn can lead to significant security issues and financial losses. Against this background, this paper investigates the effectiveness of anomaly-based intrusion detection systems (IDS) for detecting and preventing new and evolving cyber threats. The paper especially considers the challenge of concept drift, which describes the phenomenon that the statistical properties of the network traffic or behavior patterns that an IDS is trying to detect change over time. To accomplish this, a comprehensive literature review, prioritizing recent studies since 2019, highlights the challenges of concept drift and the application of reinforcement learning in IDS. The results suggest that the integration of adaptive strategies and reinforcement learning in IDS enables significant improvements in anomaly detection.

Industrial automation is driven by trends such as autonomy, intelligence and networking. Increasing energy demands, scarce resources and shorter product lifecycles pose challenges to production systems such as interoperability, flexibility and extensibility. The concept of the Digital Twin, acting as a virtual representation of a production system, can address these challenges. This paper presents a concept that realizes a dynamic production scheduling as partial function of the Digital Twin using the Asset Administration Shell and a service-oriented architecture. The authors specifically address automated asset production scheduling and propose an MQTT broker with a Functionality Dictionary server. Through the Asset Administration Shell interface, the proposed production scheduler communicates with the proposed product calendar to generate a specialized production schedule for each product. The evaluation scenario in a cyber-physical laboratory shows the advantages of this concept.

The cyber-physical systems (CPS) are characterized by a high degree of complexity due to the presence of networked heterogeneous components. This complexity makes it crucial to prevent error propagation in the system. Therefore, error detection and mitigation are necessary requirements in CPS. Recently, DL-based techniques have emerged as popular solutions for error detection in CPS. However, the main concern of DL-based error detection models in power constrained CPS is the trade-off between accuracy and speed. This leads to the necessity of designing optimized, accurate, and lightweight models.This paper proposes an optimized lightweight error detection model based on prediction approach. The paper addresses the limitations of conventional DL-based approaches in error detection for hardware-constrained CPS, particularly an exoskeleton system. The model adopted state-of-the-art efficient architecture that comprises in parallel CNN and LSTM layers, which is then transformed into a lightweight network through data quantization and network pruning techniques. The effectiveness of the proposed method is demonstrated through its application in error detection of the exoskeleton system’s data.

In partially automated manufacturing, humans work together with mobile robots. Trajectory prediction, i.e. predicting future positions of human workers, improves collaboration and coexistence between humans and robots on the shop floor. In this paper, we discuss the interrelated research questions of how human motion trajectories can be predicted and how mobile robots such as Autonomous Mobile Robots and Automated Guided Vehicles can take such predictions into account in their pathfinding and navigation. On the robot side, advanced D* pathfinding algorithms allow robots to take dynamic obstacles into account. For trajectory prediction, the position of human workers is determined by an Ultra-Wideband-based Real-Time Locating System. A trajectory prediction framework is introduced to support the implementation and use of pattern- and planning-based trajectory prediction algorithms. The evaluation is based on scenarios from the addressed problem area of manufacturing.