While essential for learning, errors are a costly constraint in the industrial sector. Reinforcement Learning (RL), an inherently error-driven technique, shows strong potential in this context. Digital Twins (DTs) help mitigate this limitation by offering risk-free simulation environments where RL agents can learn. This article presents a systematic review of studies that integrate RL and DT to address industrial challenges. Focusing on their convergence, the review organizes the discussion by disciplinary domains, types of RL algorithms, and the collaborative architectures emerging from their integration. Key findings include strategies to manage DT fidelity gaps, wide collaboration in connected automation and manufacturing, and growing applications in specialized domains. The review also identifies patterns in the use of various RL methods and a rising adoption of hybrid architectures across industrial sectors. The complexity of the addressed problems and the maturity of the reported solutions indicate a promising and expanding role for RL-DT integration in industrial digitalization.

Automating Assembly Sequence Planning (ASP) is a core task in digitalizing industrial manufacturing. The objective of ASP is to find a feasible and efficient sequence, in which the components of a given assembly can be assembled. Often, the problem is tackled using an “assembly-by-disassembly” approach, i.e., a disassembly sequence is generated an then reversed to obtain an assembly sequence. Recently, approaches to ASP based on Reinforcement Learning (RL) and physics-based simulation environments have gained attention, wherein an RL- agent acts against the simulation environment to learn feasible disassembly sequences. We propose a combination of this approach with the path finding algorithm RRT*(Rapidly Exploring Random Tree *). The path finding algorithm generates lists of components that are in principle accessible at different stages of assembly, taking into account the space needed for the hand of the fitter or for a robotic arm. These lists are passed on to the RL agent, which then only attempts to manipulate and disassemble the currently accessible components. Using a validation assembly, we demonstrate that this approach can meaningfully improve the performance, compared to an unconstrained RL agent.

In the era of digital transformation, automating industrial workflows is a key driver of productivity enhancement. One complex challenge in this field is “entity matching” in heterogeneous system landscapes, a task both time-consuming and error-prone when performed manually. The aim of this study is to explore the automation of a mapping process for a fleet of Air Separation Units (ASUs), which involves reconciling local measurement tags from various ASUs to a given standardized global naming scheme. We implemented four different machine learning models, combined with the evaluation of several pre- and post-processing algorithms. To achieve more stable and accurate predictions, we pursued an ensemble approach, combining the outputs of the individual models. In our real-world data set, 68% of local measurement tags could be correctly mapped by this algorithm, which is too low for a fully automated process. Therefore, we combined the ensemble learning approach with a threshold on the confidence scores of the ensemble learner, such as to automatically map only those tags, for which the mapping is most likely correct. While this only partially automates the workflow, it meaningfully reduces the manual effort without compromising the quality of results.

One of the concepts of Industry 4.0 is a decentralized, highly flexible and self-organizing factory. A key enabler for this vision is semantic interoperability, i.e., a shared and unambiguous understanding across all components of a production system about the meaning of data exchanged and services offered by components. We review current approaches towards semantic interoperability and propose a transformation framework that allows factory operators to assess their current maturity level and to determine measures for gradually transforming their production systems towards full semantic interoperability. For each level in this transformation framework, we propose suitable technology options.

This paper deals with controlling a process that requires a constant air temperature at its inlet. Temperature control is achieved by partially recirculating the hot discharge flow from the process and mixing it with cooler ambient air. The system is characterized by a deadtime in the recirculation path, lagging temperature measurements at the process inlet and discharge, and by slow dynamics of the actuator influencing the recirculation fraction. As a control structure we propose a three-point controller with a deadband and hysteresis, combined with a delayed feedback. Due to the nonlinear nature of the system, the parameters of the delayed feedback are subject to scheduling, depending on the current recirculation fraction. In order to swiftly react to changes in the process load, an additional feedforward path from the process exhaust is proposed. Performance and robustness of the controller are confirmed in simulations and field experiments.

The decorrelating and the linear, minimum mean-squared error (MMSE) detectors for asynchronous code-division multiple-access communications ideally are infinite memory-length detectors. Finite memory approximations of these detectors require the inversion of a correlation matrix whose dimension is given by the product of the number of active users and the length of the processing window. With increasing number of active users or increasing length of the processing window, the calculation of the inverse may soon become numerically very expensive. In this paper, we prove that the decorrelating and the linear MMSE detector can both be realized by linear multi-stage interference cancellation algorithms with ideally an infinite number of stages. It is shown that for serial multi-stage interference cancellation, depending on the signal-to-noise ratio and the number of active users, only a few stages are necessary to obtain the same BER performance as the ideal detectors. Thus, the complexity can be reduced considerably.

In coded aperture imaging the attainable quality of the reconstructed images strongly depends on the choice of the aperture pattern. Optimum mask patterns can be designed from binary arrays with constant sidelobes of their periodic autocorrelation function, the so-called URAs. However, URAs exist for a restricted number of aperture sizes and open fractions only. Using a mismatched filter decoding scheme, artifact-free reconstructions can be obtained even if the aperture array violates the URA condition. A general expression and an upper bound for the signal-to-noise ratio as a function of the aperture array and the relative detector noise level are derived. Combinatorial optimization algorithms, such as the great deluge algorithm, are employed for the design of near-optimum aperture arrays. The signal-to-noise ratio of the reconstructions is predicted to be only slightly inferior to the URA case while no restrictions with respect to the aperture size or open fraction are imposed.

In the past, different multiuser detectors for asynchronous code-division multiple-access communications have been proposed, many of them may be characterized as zero-forcing detectors, e.g., the decorrelation detector. We show that linear interference cancellation schemes are asymptotically zero-forcing which means that they are equivalent to the decorrelating detector if the number of stages approaches infinity. These detectors have been found to be superior to the conventional matched filter detector. However, the design of spreading sequences optimized especially for these receivers has not been considered up to now. Usually, spreading sequences are designed to have a low peak correlation parameter. Pursley (1977) has shown that the average interference parameter (AIP) is an important design parameter since it is related to the average signal-to-interference ratio of the conventional receiver. In this paper, we consider the construction of spreading sequences for zero-forcing multiuser detectors that are optimal in the sense of performance and near-far resistance. It is shown that sequences with a low AIP are near-optimal. This, again, stresses the importance of the AIP for the design of spreading sequences for CDMA systems employing any kind of receiver. Numerical examples indicate that by using optimized sequences the average signal-to-noise ratio (SNR) can be improved by about 1-2 dB for lengths of interest in applications.