Nowadays, industrial intelligence has become a strong driving force for the progress of smart manufacturing. In this paper, a multi-Agent reinforcement learning framework based on Multi-Head Attention as Mixture-of-Head Attention Network (MOHN) is proposed to effectively solve the classical problem in flexible manufacturing, the flexible job shop scheduling problem (FJSP). To begin with, an innovative approach is introduced for formulating FJSP as a Markov decision process (MDP). This methodology integrates both process selection and machine allocation into a unified action space. Additionally, a dynamic decision timing mechanism is ingeniously incorporated, significantly enhancing the effectiveness of the MDP's action space. Second, in order to efficiently extract more comprehensive and potential state feature information from the action space, this paper proposes a MOHA-based state extraction method to capture the complex linkages between processes and machines in the manufacturing process, so as to improve the decision-making efficiency of the agent. Experiments on a large number of instances at different scales show that the method outperforms the traditional PDRs algorithm with strong generalization ability.

The no-wait flow-shop scheduling problem (NFSSP) with sequence-dependent setup times (SDSTs) and release times (RTs) is applicable in many areas, such as steel production, food processing, and chemical processing. Estimation of the distribution algorithm (EDA) has recently been recognized as a prominent metaheuristic methodology in the field of evolutionary computation due to its excellent performance of global exploration. In this article, an innovative matrix-cube-based (i.e., 3-D) EDA (MCEDA) is first proposed to minimize the total earliness and tardiness (TET) of the NFSSP with SDSTs and RTs. This problem is NP-hard in the strong sense. First, a 3-D matrix cube is devised to learn the valuable information from promising solutions or excellent individuals. Second, an EDA model or probabilistic model based on the matrix cube and a special sampling method is presented to perform effective exploration in solution space and find promising regions. Third, based on a series of newly defined subneighborhoods, a new local search with both a speed-up scanning method and one search strategy is developed to execute exploitation from promising regions. Fourth, a speed-up evaluation method based on the problem’s property is designed to reduce the computational complexity for calculating criterion and accelerate the search process. Owing to the reasonable hybridization of exploration and exploitation, MCEDA can perform very efficient search in solution space. Extensive test results on instances of such a just-in-time problem first show that MCEDA can achieve better solution than state-of-the-art algorithms in obviously less computation time. Additional experiments on instances of various NFSSPs further confirm the efficiency and robustness of MCEDA.