Abstract The domain of cyber‐physical systems (CPS) has recently seen strong growth, for example, due to the rise of the Internet of Things (IoT) in industrial domains, commonly referred to as “Industry 4.0.” However, CPS challenges like the strong hardware focus can impact modern software development practices, especially in the context of modernizing legacy systems. While microservices and DevOps have been widely studied for enterprise applications, there is insufficient coverage for the CPS domain. Our goal is therefore to analyze the peculiarities of such systems regarding challenges and practices for using and migrating towards microservices and DevOps. We conducted a rapid review based on 146 scientific papers, and subsequently validated our findings in an interview‐based case study with nine CPS professionals in different business units at Siemens AG. The combined results picture the specifics of microservices and DevOps in the CPS domain. While several differences were revealed that may require adapted methods, many challenges and practices are shared with typical enterprise applications. Our study supports CPS researchers and practitioners with a summary of challenges, practices to address them, and research opportunities.

A characteristic of reinforcement learning is the ability to develop unforeseen strategies when solving problems. While such strategies sometimes yield superior performance, they may also result in undesired or even dangerous behavior. In industrial scenarios, a system's behavior also needs to be predictable and lie within defined ranges. To enable the agents to learn (how) to align with a given specification, this paper proposes to explicitly transfer functional and non-functional requirements into shaped rewards. Experiments are carried out on the smart factory, a multi-agent environment modeling an industrial lot-size-one production facility, with up to eight agents and different multi-agent reinforcement learning algorithms. Results indicate that compliance with functional and non-functional constraints can be achieved by the proposed approach.

Abstract Systems are becoming increasingly more adaptive, using techniques like machine learning to enhance their behavior on their own rather than only through human developers programming them. We analyze the impact the advent of these new techniques has on the discipline of rigorous software engineering, especially on the issue of quality assurance. To this end, we provide a general description of the processes related to machine learning and embed them into a formal framework for the analysis of adaptivity, recognizing that to test an adaptive system a new approach to adaptive testing is necessary. We introduce scenario coevolution as a design pattern describing how system and test can work as antagonists in the process of software evolution. While the general pattern applies to large-scale processes (including human developers further augmenting the system), we show all techniques on a smaller-scale example of an agent navigating a simple smart factory. We point out new aspects in software engineering for adaptive systems that may be tackled naturally using scenario coevolution. This work is a substantially extended take on Gabor et al. (International symposium on leveraging applications of formal methods, Springer, pp 137–154, 2018).

Adversarial learning has been established as a successful paradigm in reinforcement learning. We propose a hybrid adversarial learner where a reinforcement learning agent tries to solve a problem while an evolutionary algorithm tries to find problem instances that are hard to solve for the current expertise of the agent, causing the intelligent agent to co-evolve with a set of test instances or scenarios. We apply this setup, called scenario co-evolution, to a simulated smart factory problem that combines task scheduling with navigation of a grid world. We show that the so trained agent outperforms conventional reinforcement learning. We also show that the scenarios evolved this way can provide useful test cases for the evaluation of any (however trained) agent.

In this paper, we extend an existing approach for monitoring work-pieces in Self-Organizing Industrial Systems (SOIS) to be applicable to more complex SOIS settings, where autonomous agents can move and regulate their velocity completely free. The problem of having infeasible many possible valid routes is handled by extracting significant sub-paths (pathlets), by which valid routes can be reconstructed. We show that meaningful pathlets can also be extracted for complex SOIS settings. By adding a temporal component to the dictionary, we are able to detect anomalous trajectories regarding spatial movement and temporal flow. Furthermore, environmental changes, like added/removed machines, can be located using information gained from the reconstruction process. Additionally, we show that the global movement behavior of the agents can be analyzed, too, by monitoring specific movement parameters for the dictionary pathlets. The data for the experiments are generated using a SOIS simulation which implements a state-of-the-art architecture for multi-agent systems.

In this work, we present a monitoring system for Self-Organizing Industrial Systems (SOIS). It is based on an anomaly detection approach which evaluates the movement of objects within a factory by putting them together from sub-trajectories. By introducing two metrics - relative user frequency and pathlet occurence per user - the existing method is extended so that not only anomalous trajectories and omitted production stations can be detected, but also loops, shifts in the load distribution and novel valid paths. For this purpose, suitable visualization techniques are presented: For loop detection the pathlet occurence per user is monitored and evaluated using box plots. Shifts in the load distribution and novel valid paths are detected using heat maps. The work-flow of the monitoring system is illustrated based on data which is generated by a simplified simulation model.

The main challenge for anomaly detection in Self-Organizing Industrial Systems (SOIS) is the high degree of freedom of the system, which causes a state-space explosion. Since the system is free to choose at runtime any solution out of the vast amount of possible ones, to ensure that the production process is optimal at all times, classic anomaly detection techniques can not be used one-to-one in SOISs. For this reason, we already presented in previous work, a novel anomaly detection method, which exploits the idea that many products will share a larger fraction of the production process. Accordingly, it learns at first such recurrent “building blocks” of object movements and represents then incoming movements in relation to these known building blocks. With it, anomalous trajectories and global anomalous events like the omitting of a system component, can be detected. In this paper, we present a new algorithm which extracts such “building blocks” more efficiently. In particular, the new approach scales linear with the number of samples per trajectory, while the existing approach scales quadratic.