This article introduces a novel integration of deep model-based learning with modular state-based Stackelberg games (Mod-SbSG) for distributed self-optimization in manufacturing systems, using a sample-efficient approach. Model-free Mod-SbSG requires frequent interactions with real systems to find optimal solutions, which can be costly, time-consuming, and risky in industrial settings. Prior studies handled this by using digital representations to train Mod-SbSG players, but accurate representations are often difficult to develop. Hence, our framework replaces digital representations with deep learning methods that learn system dynamics, optimize policies within Mod-SbSG, and reduce real-world interactions. The method includes two main steps: 1) designing deep learning models to predict system dynamics and 2) training Mod-SbSG players in virtual environments. We evaluate single- and multistep predictors and demonstrate network reuse for transfer learning in adaptable systems, which reduces real system interactions by 77.78% in a laboratory testbed industrial control scenario.

This article presents a novel game-theoretical (GT) framework, distributed Stackelberg strategies in state-based potential games (DS2-SbPGs), for autonomous multiobjective optimization in decentralized manufacturing systems. Existing approaches, including multiagent reinforcement learning (MARL) and native SbPG, struggle with scalability, coordination inefficiencies, and the complexity of tuning combined objective functions in real-world settings. DS2-SbPG integrates potential games and Stackelberg games, which improves the cooperative tradeoff capabilities of potential games and the multiobjective optimization handling by Stackelberg games. Notably, all training procedures are conducted in a fully distributed manner. DS2-SbPG offers a promising solution to finding optimal tradeoffs between objectives by eliminating the complexities of setting up combined objective optimization functions for individual players in self-learning domains, particularly in real-world industrial settings with diverse and numerous objectives between the subsystems. We formally prove that DS2-SbPG constitutes a dynamic potential game with guaranteed convergence. Experimental validation on a laboratory-scale testbed demonstrates the effectiveness of DS2-SbPG and its two variants: one with a single-leader–follower structure and another (Stack DS2-SbPG) for multileader–follower scenarios. Both variants significantly outperform native SbPG, achieving up to 10.61% reduction in power consumption while enhancing overall system performance, which signals the potential of DS2-SbPG in real-world applications.

In this paper, we introduce novel gradient-based optimization methods for state-based potential games (SbPGs) within self-learning distributed production systems. SbPGs are recognised for their efficacy in enabling self-optimizing distributed multi-agent systems and offer a proven convergence guarantee, which facilitates collaborative player efforts towards global objectives. Our study strives to replace conventional ad-hoc random exploration-based learning in SbPGs with contemporary gradient-based approaches, which aim for faster convergence and smoother exploration dynamics, thereby shortening training duration while upholding the efficacy of SbPGs. Moreover, we propose three distinct variants for estimating the objective function of gradient-based learning, each developed to suit the unique characteristics of the systems under consideration. To validate our methodology, we apply it to a laboratory testbed, namely Bulk Good Laboratory Plant, which represents a smart and flexible distributed multi-agent production system. The incorporation of gradient-based learning in SbPGs reduces training times and achieves more optimal policies than its baseline.

In this research, we discuss the impact of combining model-based deep learning and game theory in dynamic games to develop a sample-efficient self-learning methodology for smart production systems. We propose a novel approach, namely Model-Based Game Theory (MBGT), by incorporating model-based deep learning into a successful self-learning strategy of State-based Potential Games. Most of the advanced self-learning approaches are established based on machine learning and artificial intelligence, including reinforcement learning and game-theoretical method. However, because of the iterative behaviour during the learning processes, the learning processes are almost impossible to be conducted directly in a real-world manufacturing environment. One potential approach is to conduct the learning processes using the digital representation of the systems, e.g. the digital twin or simulation. Nonetheless, such representations in industrial settings are not always available and developing a digital representation of a complex system is immensely challenging. Hence, this problem can be solved through model-based learning, in which deep learning models are trained to predict the dynamics of the systems with a small margin of error. Then, the trained models are deployed as virtual environments for the learning processes of the GT-based control policies, instead of learning in the real world. In this study, we implement MBGT to a bulk good system and thoroughly analyse the performances between training in the real system and virtual environment, which eventually turn out to be positive effects.

The integration of advanced robotic technologies with modern software platforms has enabled significant advancements in industrial automation. This paper aims to introduce a novel workflow to integrate ABB industrial robot manipulators and the Robot Operating System (ROS), where communication is built through OPC Unified Architecture or a virtual controller with EGM+RWS. Moreover, we also provide a method to deploy the ROS environment using container technology in our workflow. The developed workflow was tested and evaluated in a real-world welding application, demonstrating its capability to automate the welding process of steel looped hooks for supermarket shelves with high accuracy and efficiency, in which the path planner for the robot is obtained from MoveIt by ROS. The introduced workflow can also be integrated with the MLPro framework, which enriches the workflow with machine learning-related algorithms, e.g. for path planning. The results of this study demonstrate the potential of integrating ABB industrial robot manipulators with ROS for industrial process automation and highlight the benefits of using ROS as a software platform for industrial robotics applications.

This article describes a novel concept to optimize manufacturing systems distributively through data-based learning. We propose a game-theoretic (GT) learning set-up that is incorporated with accessible control code of the programmable logic controller (PLC) to accelerate the optimal policies learning procedures, instead of learning everything from scratch. Therefore, we offer to process the accessible and available control code into a GT-based learning framework which is subsequently optimized in a fully distributed manner. To this end, we employ the recently developed framework of state-based potential games (PGs) and prove that under mild conditions PLC-informed (PLCi) learning forms a state-based PG framework. We conduct the experiment on a laboratory scale testbed in numerous production scenarios. The experiment's results highlight the major potential of using the PLCi GT-learning, which is the reduction of energy consumption of the production timescales and improvement of production efficiency while nearly halven the learning times.

In this paper, we discuss the impact of communication and memory-based learners on distributed self-optimization of smart and flexible manufacturing units. Specifically, we employ the recently proposed framework of state-based potential games, which has proven to be successful in allowing distributed optimization in multi-agent systems. We first augment the framework with additional communication capabilities for the individual players and analyze the efficacy of state and action communications within the different players. Second, we incorporate memory states within the learning dynamics of the players and analyze their impact on the learning performance. The proposed method is inspired by the promising results of memory-based reinforcement learning. However, previous studies have rarely dealt with distributed manufacturing control. We believe that it will be important to explore the potential use of the communication and memory-based approaches in manufacturing control with multi-agent settings. Hence, the proposed method is applied to a bulk good laboratory plant providing a thorough experimental analysis of the effect of the various improvements with very encouraging results.