Robotic tasks often require multiple manipulators to enhance task efficiency and speed, but this increases complexity in terms of collaboration, collision avoidance, and the expanded state-action space. To address these challenges, we propose a multi-level approach combining Reinforcement Learning (RL) and Dynamic Movement Primitives (DMP) to generate adaptive, real-time trajectories for new tasks in dynamic environments using a demonstration library. This method ensures collision-free trajectory generation and efficient collaborative motion planning. We validate the approach through experiments in the PyBullet simulation environment with UR5e robotic manipulators. Project Website: https://sites.google.com/virginia.edu/oncoldmp/home

Manufacturers are pursuing energy-efficient production in response to the fluctuating energy price, growing global competition, rigorous international laws, and severe environmental crisis. This article proposes to boost the energy efficiency of production systems by controlling the production. It extends the existing energy-saving control research by presenting integrated modeling, analyzing, and controlling approaches. The work starts from the modeling of the production systems and establishes an analytical model to systematically quantify the production loss resulted from energy-saving control and the various disruptions. A dynamic control algorithm is proposed to reduce energy consumption and maintain desirable productivity. Simulation studies are utilized to demonstrate the application of the proposed method and verify its effectiveness. Note to Practitioners—The previous research indicates that it is possible to strategically turn off stations during production for energy saving. However, production systems are complex dynamic systems consisting of interconnected stations and supporting subsystems. Similar to station random failures, turning off stations for energy saving can severely jeopardize the production, and deviate the production from the desired target. Therefore, a quantitative method is established to calculate the production loss resulted from the energy-saving control and disruptions. The method is important to understand the real control cost. Based on the analysis, a dynamic control algorithm is formulated to balance the achieved control benefit and the production loss. It provides plant managers a useful tool to make energy-saving control decisions with a thorough understanding of production system dynamics. The presented research is established for serial batch production systems. The examples of batch stations include the refrigerator foaming equipment in refrigerator assembly lines and the vacuum oven in battery assembly lines. The model cannot be directly applied in serial-parallel production systems.

This article formulates gantry real-time scheduling in a gantry work cell, where the material transfer is driven by gantries, as a Markov decision process (MDP). Classical learning methods and planning methods for solving the optimization problems in MDP are discussed. An innovative method, called “Q-ADP,” is proposed to integrate reinforcement learning (RL) with approximate dynamic programming (ADP). Q-ADP uses model-free Q-learning algorithm to learn state values through interactions with the environment, meanwhile, planning steps during the learning process opt for ADP to keep updating state values through several sample paths. A model of one-step transition probabilities is built based on the machines' reliability model, and serves the ADP algorithm. To demonstrate the effectiveness of this method, a numerical study is performed to show the production performance, compared to a standard Q-learning algorithm. The simulation results show that Q-ADP outperforms standard Q-learning under the same length of training process. It is also shown that with the benefit of repeated updating state values through sample paths, Q-ADP requires less data for gantry policy to converge, which makes the method promising when real data are limited.

Energy places an important role in a new scale of urbanization, digitization, and industrialization. Going “energy-efficient” then becomes a major component of the missions for manufacturers and service providers to stay globally competitive. In recent years, the newly emerging intelligent technologies are enhancing the production process and control management in an energy-effective and -efficient manner. In order to apply and implement these innovations, many new challenges and opportunities have emerged and significantly expanded the scopes of typical production and service automation.

It becomes more and more critical for manufacturing enterprises to improve energy efficiency because of the escalating energy prices, increasing global competitions, and more rigorous government regulations. In this paper, a systematic method is developed to improve the energy efficiency of a multistage manufacturing system through production control. The method aims at reducing energy consumption with minimal negative impact on production. We start from the analysis of system dynamics and develop quantitative methods to estimate energy saving opportunities. A supervisory control algorithm is developed to improve system energy efficiency by periodically taking the saving opportunities. Simulation case studies are performed to validate the effectiveness of the control algorithm.

In this paper, the performance of a typical composite work cell structure that includes multiple machines and a material handling gantry is studied. A mathematic model is established to analyze the performance of such gantry systems. Due to the unique features of the gantry system, cycle time and waiting time of the gantry and machines are discussed and formulated based on two basic scenarios. To measure the impact of disruption events, permanent production loss is evaluated by using the sensor data collected from plant floors. Data-driven diagnostic methods are developed to identify the bottleneck machine and to evaluate permanent production loss attribution of each machine. The results provide a solid base to efficiently and effectively improve the performance of a composite work system. Performance diagnostics of gantry systems provides a theoretical and practical basis for production improvement. A gantry system for composite lay-up processes is used for numerical simulation case studies, by which the performance diagnostic methodologies and improvements are demonstrated.

Mathematical models are often required in process modeling, control, and evaluation. To meet the increasing demand of production system real-time performance evaluation and improvement, in this paper: 1) an innovative event-based, data-driven mathematical model is established for network structured manufacturing systems; 2) important properties of network structured manufacturing systems are obtained through the concept development of a virtual ideal clean system; and 3) a system performance diagnostic method is developed based on the mathematical model and system properties, as well as available sensor data. The mathematical model and system performance identification methodology are studied analytically and validated by simulation studies. This data-driven mathematical framework and system real-time performance diagnostic methodology are invaluable for real-time production control to improve system responsiveness and efficiency.

Advanced manufacturing systems are characterized by their complex dynamics, which are subject to constant changes caused by technology insertion, engineering modification, as well as disruption events. To support daily operation, distributed sensors are utilized to monitor the status of each process. The advantages of the sensor data are not fully realized in current production system analysis due to a lack of data-driven system level modeling. Motivated by this need, we propose a data-driven manufacturing system model to describe production dynamics and develop a systematic method to identify the causes of permanent production loss. This research enables real-time production system performance diagnosis, which is invaluable in increasing system responsiveness and improving real-time production control to effectively enhance overall system efficiency.

Current manufacturing facilities lack proper performance indicators that can accurately pinpoint areas of energy inefficiency on the manufacturing line. This paper establishes new indices that utilize readily available, real sensor information from the production line, such as buffer levels, machine throughput, etc., to find the machine that is the least energy efficient. This is the machine that causes the line to waste the most energy without producing parts. The energy structure of the production line is analyzed to better understand the complex system dynamics and to find the root cause of the energy inefficiencies. A baseline energy consumption is established, which is the least amount of energy that is needed to produce a certain number of parts on the line. Using this knowledge, the new sustainable manufacturing performance indicators are defined to properly monitor the performance of the line. These indices utilize the energy structure to illustrate the static (energy used when there are no downtime events) and dynamic (energy lost due to downtime) portions of the energy consumption. The concepts of the downtime energy bottleneck and the rated power bottleneck are introduced and proven analytically. These methods are verified using simulation studies in Simulink/MATLAB.

In modern manufacturing facilities it is imperative to maximize profits by reducing energy costs while maintaining or enhancing production operations. To achieve this goal, one must understand the complex energy dynamics of the manufacturing system to properly identify inefficiencies and energy savings opportunities on the production line. The energy system dynamics focus on how energy is consumed by the production line at any given time. The energy profit bottleneck (EP-BN) is defined, which provides the machine that, when the downtime is reduced, leads to the highest overall profit increase. This EP-BN is a more general definition over the traditional throughput bottleneck. The EP-BN explicitly considers the cost of energy, which was previously ignored or treated as insignificant in traditional throughput analysis. This bottleneck is used in conjunction with an energy opportunity window control methodology that reduces overall energy consumption while maintaining throughput. The energy opportunity window duration and frequency is numerically analyzed for a real-time control scheme.

Accurate evaluation of the impacts of machine downtime incidents, especially those that cause the most production loss, is of great importance for deciding where to allocate limited resources in multistage manufacturing systems. In this paper, we focus on quantitative analysis of the impacts of downtime events in terms of production loss based on available production information. A complex station concept is introduced to represent a single station or a parallel structure in a production line. The dynamics of complex stations are studied. The real-time analysis of the impact of a downtime event in a parallel production line and in a serial production line is demonstrated to be consistent. Apart from the analytic study, a computational procedure using real-time production information is developed. A case study is conducted to demonstrate its potential on real-time production control and resource/budget allocation in a multistage manufacturing system.

In this paper, a market demand driven modeling framework is developed. The market demand is modeled as an end-of-line virtual machine based on which the market demand dissatisfaction (MDD) can be measured as production loss using event-based analysis. A general Markovian continuous-flow model is developed for market demand-driven systems with multistage production networks combining manual and automatic processes. Machine failure bottlenecks (MF-BNs) and machine capacity bottlenecks (MC-BNs) are defined and identified based on event-based indicators. A supervisory control algorithm is integrated in the framework to reduce MDD and improve system productivity through identification and mitigation of MF-BNs and MC-BNs. Simulation-based analysis will also be utilized, and case studies are performed to validate the effectiveness of the modeling framework and the supervisory control policies.

Modern manufacturing facilities waste many energy savings opportunities (ESO) due to the lack of integration between the facility and the production system. To explore the energy savings opportunities, this paper combines the two largest energy consumers in a manufacturing plant: the production line and the heating, ventilation, and air conditioning (HVAC) system. The concept of the energy opportunity window (OW) is utilized, which allows each machine to be turned off at set periods of time without any throughput loss. The recovery time of each machine is the minimum amount of time a machine must be operational between opportunity windows to guarantee zero production loss and it is explored both analytically and numerically. The opportunity window for the production line is synced with the peak periods of energy demand for the HVAC system to create a heuristic rule to optimize the energy cost savings. This integrated system is modeled and tested using simulation studies.

Conventionally, improving production efficiency, flexibility and responsiveness has been the primary research focus of production management, while energy consumption has received relatively little attention. Energy consumption plays a more and more important role in the manufacturing environment. This is mainly driven by energy cost and environmental concerns. When the energy system becomes complicated and coupled with ongoing production, it is very difficult to hunt the “hidden treasure” which affects the overall benefit of a manufacturing system. This paper provides a systematic method to search for energy saving opportunities and strategies. We start from dynamic production transient analysis and provide quantitative analysis for identifying energy saving opportunity in a system. Furthermore, energy saving strategy is justified through cost analysis for tradeoffs between energy savings and throughput loss. A case study is conducted to demonstrate its potential on energy savings in a multistage manufacturing system.