In a mobile edge computing environment where mobile devices are powered by green energy, minimizing power consumption and latency under long-term MEC energy constraints faces several challenges: unpredictable tasks, keeping both the privacy of the mobile device’s battery and efficient task scheduling, stabilization of power consumption and latency accumulation, and non-convex optimization for a continuous-discrete decision space. To tackle these challenges, we investigate the latency-aware resource-constrained scheduling (LARCS) problem in a hybrid-energy edge-cloud MEC system to minimize energy consumption and latency under long-term MEC energy constraints. For such a scenario, for the first time, we address protecting battery privacy and mitigating energy rebound peaks, which indicate sudden power surges from concurrent high-load tasks. We model the LARCS using mixed integer nonlinear programming (MINLP) and prove its NP-completeness. Then, it is reformulated as a Markov game. We adopt the multi-actor-attention-critic learning mechanism to preserve user privacy and employ a reward shape in reward functions for mitigating energy rebound peaks. On the above basis, we propose a latency-aware resource-constrained scheduling algorithm on the basis of multi-agent deep reinforcement learning (LARC-MADRL) to coordinate optimizations for multiple mobile users. This proposed algorithm requires no prior knowledge of uncertain parameters, operates independently of hybrid-energy dynamic models, and maintains user privacy. Numerical simulations validate that our proposed algorithm outperforms benchmark algorithms in minimizing power consumption, latency, and dropout rate while achieving long-term benefits, better convergence, improved stability, and enhanced scalability.

With the advancement of semiconductor manufacturing technology, multifunctional process modules (MPMs) have been widely adopted in cluster tools to enhance production flexibility and efficiency by handling multiple operations concurrently. However, the MPMs lead to challenges in scheduling the robot due to a variety of configurations, complex robot action sequences, and regulating wafer postprocessing residency time. For scheduling such a dual-arm cluster tool (DACT) with MPMs, this article proposes a specific reinforcement learning-based scheduling method. We first develop an adaptive algorithm to generate all feasible MPM configurations. We then employ the masking technique and prioritize a replay experience buffer to improve the dueling double deep$Q$-network (D3QN), enabling it to train and identify scheduling strategies that minimize makespan and reduce wafer residency time under each configuration. We conduct experiments to demonstrate that the proposed method ensures high productivity, providing a robust and flexible scheduling solution for cluster tools in semiconductor manufacturing, and significantly enhances overall production performance.

In recent years, deep reinforcement learning (DRL) has demonstrated significant potential to address complex and dynamic supply chain optimization (SCO) problems. However, existing DRL algorithms often encounter challenges when dealing with large-scale and high-dimensional supply chain decisions, making it difficult to effectively coordinate production and transportation. To address these issues, this paper proposes an innovative deep reinforcement learning algorithm—Action Decomposition Actor-Critic (ADAC). This algorithm significantly reduces learning complexity by decomposing complex decision tasks into multiple subtasks. Based on real-world supply chain scenarios, we construct a complex multi-stage supply chain environment. Additionally, we employ action decomposition-based Proximal Policy Optimization (PPO) and Soft Actor-Critic (SAC) algorithms to learn optimal policies in continuous action spaces, enabling fine-grained control of production and transportation. To verify the effectiveness of the algorithm, we conduct extensive experiments on both simulation and real-world datasets. Experimental results demonstrate that the ADAC algorithm outperforms traditional heuristic algorithms and general DRL algorithms in multiple complex supply chain scenarios. This shows the strong robustness and wide applicability of the ADAC algorithm in SCO problems.

3D hand mesh reconstruction from 2D images is crucial for various computer visual tasks such as virtual reality and human-computer interaction, while it remains a challenging problem due to changed hand poses and diverse self-/cross-hand occlusions. In this paper, we propose a graph-based reasoning network for 3D hand mesh reconstruction from a single 2D RGB image by recovering the fine-grained 3D hand mesh keypoints in a coarse-to-fine manner. First, we extract the hand-joint-specific features to capture the overall hand structure via a CNN backbone with a linear projection sampling operation. Based on the hand-joint locations, we further progressively learn more hand mesh keypoints to construct the coarse hand shape via hierarchical attention-embedded graph learning layers. Finally, we leverage the 2D shallow semantic features to refine the coarse hand mesh keypoints into fine-grained 3D hand mesh vertices coordinates via cascaded graph learning layer with linear mapping. Experimental results on the widely used hand databases show that our method achieves outstanding performance in both single-hand and two-interactive-hand 3D mesh reconstruction.

Cluster tools are vital in semiconductor manufacturing, where multifunctional process modules (MPMs) enhance flexibility and efficiency. However, variable MPMs and processing time in dual-arm cluster tools (DACTs) complicate scheduling, as variable MPM allocation patterns yield distinct productivity. This paper proposes a reinforcement learning-based method for DACTs with MPMs. Firstly, an algorithm enumerates all valid MPM allocation patterns. Then, an adaptive deep Q-Network (DQN) with masking techniques efficiently selects the most efficient pattern and generates robot schedules, minimizing makespan and wafer post-processing residency time across diverse DACT configurations. Experiments validate the proposed approach that offers robust, flexible scheduling solutions to boost semiconductor manufacturing productivity.

How to effectively optimize the operation for a complex supply chain environment has been high on the agenda. Although the existing deep reinforcement learning methods have achieved success in certain applications, they still face limitations in complex supply chain environments, including difficulties in data sharing and a lack of digital collaboration, especially in multi-agent systems. In order to meet this challenge, we propose a novel multi-agent reinforcement learning algorithm based on dynamic optimistic weights (DO-QMIX), aiming at solving the shortcomings of the traditional weighted QMIX algorithm (WQMIX) in the simplicity of the weighting function. WQMIX employs two weighting schemes to handle multi-agent issues. However, its fixed weighting function restricts algorithm performance and hinders adaptability to dynamic, complex supply chain challenges. Therefore, we propose a dynamic weighting mechanism, which can adjust the weighting function in real time based on the changes in the environment, thus improving the overall efficiency. We construct a complex multi-stage supply chain environment in the real-world supply chain scenario and conduct many experiments using both real-world and simulated datasets. The experimental results demonstrate that DO-QMIX is significantly superior to the traditional multi-agent reinforcement learning algorithm in complex supply chain scenarios, especially in dealing with dynamic changes and complex decisions.

Multi-cluster tools are widely adopted in semiconductor manufacturing. When a process module (PM) at a step fails, a multi-cluster tool cannot complete the process recipe of work-in-process wafers and must be forced to enter a closedown process to be empty. To increase the throughput of a wafer fab, it is economically significant to shorten the failure-closedown process. However, due to the wafer residency time constraints, it is highly challenging to respond to a PM failure and find a corresponding optimal schedule. By assuming no parallel PM at a step, this work is the first to study this important issue for scheduling multi-cluster tools. We analyze the task sequences and synchronization conditions for robot activities to avoid deadlock in a shared buffer module. Upon these analyses, for process-dominant multi-cluster tools whose optimal steady-state schedule is known, algorithms are proposed to synthesize the proper sequences for robots in case of PM failures, then, a nonlinear program model is proposed to find an optimal schedule for the corresponding closedown process or decide no feasible solutions. The proposed model can uniformly deal with different scenarios of PM failures. Examples are given to illustrate the application of the proposed method.Note to Practitioners—In a wafer fab, there are hundreds, even thousands, of cluster tools. It is common that a failure of a processing module happens in a cluster tool. How to intelligently respond to such a random failure in a multi-cluster tool is an important issue in real-world production. This paper studies the scheduling problem of a multi-cluster tool in case of a failure at a PM. For a multi-cluster tool in case of a failure module, the proposed method can significantly reduce the loss of work-in-process wafers and shorten the closedown process.

Cluster tools are widely adopted in semiconductor manufacturing. When a process module fails, a cluster tool must undergo a close-down process, transiting to an idle state. Minimizing the makespan of a close-down process is economically significant in increasing the throughput of a wafer fab. However, the optimization of a close-down process is challenging due to wafer residency time constraints. Existing linear programming models must be constructed when specific processing time and robot activity time parameters are given. To address the issue, we propose a scheduling method based on deep reinforcement learning. For this reinforcement learning algorithm, the specific states, actions, and reward functions are designed for the close-down process to follow the Markov property, and a deep Q-network is adopted to find an optimal scheduling policy. Experimental results demonstrate that, in contrast to traditional scheduling methods, the proposed method achieves an optimal policy and also excels in generalization, enabling adaptive real-time production scheduling.

As data volume increases and data parallelism strengthens, cloud service providers must reduce energy consumption by using effective cloud scheduling methods. This paper investigates scheduling multiple servers in cloud data centers to achieve more balanced energy consumption over the long term while ensuring task completion. We propose a method based on long short-term memory (LSTM) and the African vulture optimization algorithm (AVOA), combining prediction and scheduling. The proposed method is divided into two main modules: the prediction phase and the scheduling phase. The purpose of the prediction is to reserve partial resources on servers for large tasks, thereby reducing energy consumption by minimizing the number of active servers. Therefore, we use a prediction method based on LSTM to predict upcoming resource demands and anticipate server requirements. In the scheduling phase, we adjust the priority of tasks to ensure task completion while maximizing the processing of tasks with larger demands. Experiments are performed to validate the application and effectiveness of the proposed method, which outperforms the benchmark algorithms.

Mobile edge computing boosts the application performance on mobile devices by collaborating with cloud platforms. This paper studies the task offloading and computing resource allocation problem in a multibase, multiserver, and multiuser scenario subject to resource constraints. The goal is to maximize the users' task offloading utility, including improvements in task completion time, energy consumption, and communication cost. The addressed problem is formulated as a mixed integer nonlinear programming (MINLP) model. In this paper, we decompose the MINLP and the optimal computing resource allocation policy under a deterministic offloading strategy obtained by the Karush-Kuhn-Tucker conditions. Then, a hybrid adaptive large neighborhood search (HALNS) algorithm is proposed to conduct task offloading. The adaptive large neighborhood search and the variable neighborhood descent stages are jointly employed in HALNS. The proposed algorithm, an improved simulated annealing algorithm, and a modified variable neighborhood search algorithm are executed to evaluate their performances. Digital experimental results show that our proposed algorithm achieves higher system utility, lower delays, and less energy consumption.

In semiconductor manufacturing, multicluster tools are widely employed for many wafer fabrication processes. With the demand for high-mix integrated circuit chips and shrinkage of circuit width, a scheme in which multiple wafer types are fabricated inside multicluster tools is adopted by wafer foundries to make more profits. Multiple wafer types, multiple robots, and wafer residency time constraints make the resulting scheduling problems challenging. This work focuses on scheduling a single-arm multicluster tool to process two wafer types concurrently subject to wafer residency time constraints in which a conventional one-wafer cycle and backward strategy are not efficient. With such properties, several necessary and sufficient conditions are presented to check the feasibility of a periodic schedule. Polynomial-time-complex algorithms are proposed to examine a tool's schedulability and coordinate multiple robots to handle wafers for schedulable scenarios. The cycle time of an obtained schedule can reach the lower bound. A practical example is used to show the effectiveness of the proposed algorithm.

In today’s semiconductor manufacturing industry, wafer foundries often face the challenge of producing a variety of integrated circuit chip products using a single manufacturing line. To address this, multicluster tools have become a popular choice for processing multiple wafer types simultaneously. Operating such tools involves coordinating the robots in adjacent individual tools to transport multitype wafers through a shared buffer. This study aims to develop a scheduling method for the concurrent fabrication processes of two wafer types, performed by a multicluster tool with wafer residency time constraints. The proposed approach presents a two-backward sequence, based on a backward strategy of a single wafer type, to convert a one-wafer cyclic schedule into a one-wafer-per-type cyclic schedule while revealing its temporal properties. To ensure a smooth operation of a single-arm multicluster tool system and synchronize multiple robots, several necessary and sufficient conditions are derived for the first time. Two efficient algorithms are then proposed to determine the feasibility of a periodic schedule and obtain a schedule that achieves the lower-bound cycle time under a two-backward strategy, maximizing the productivity of such a multicluster tool. Finally, numerical simulations and two practical examples are presented to demonstrate the applications and performance of the proposed approach.

Mobile edge computing (MEC) is a promising way to improve the application performance on mobile devices in recent years, especially in terms of energy saving. However, users have to pay additional service costs such as communication costs when using services provided by MEC. In this paper, we manage to minimize both the average energy consumption and the average communication cost of mobile devices in a mobile edge computing environment with multiple users. We consider a MEC system that consists of multiple cloudlets and a cloud. Users submit their applications consisting of tasks with dependencies to this MEC system. We combine computation offloading and dynamic voltage frequency scaling (DVFS) technologies to reduce the power consumption of mobile devices. We formulate this problem into a mixed-integer nonlinear programming model and propose an adaptive multiobjective evolutionary algorithm based on Non-dominated Sorting Genetic Algorithm III (NSGA-III) to solve it. Simulation experiments are conducted to compare the proposed algorithm with three baseline algorithms and two multi-objective optimization algorithms, which validates the superiority of the proposed algorithms.

As wafer circuit widths shrink down, stringent quality control is required during wafer fabrication processes. Therefore, a cleaning operation that removes chemical residuals inside a processing chamber is recently demanded in wafer fabs. To make a trade-off between quality and productivity, a condition-based chamber cleaning in practice is introduced to execute a cleaning operation with a known state of a chamber. Aiming to schedule a time-constrained single-arm cluster tool with a condition-based chamber cleaning operation, we present the necessary and sufficient conditions to check the schedulability of such a cluster tool. Efficient scheduling algorithms for a feasible schedule are derived. An example is given to show the application of the proposed approach.

A kind of facilities for wafer fabrication, cluster tools (CTs) need to close down to an idle state from time to time because of periodical maintenance and switches from one type of lots to another, which is called a normal close-down process (NCDP). It is crucial to optimize such a transient process since it tends to occur more and more frequently due to customization. Also, process modules (PMs) in CTs are known to be failure-prone. Once a PM failure occurs, a tool needs to close down to an idle state as well, which is different from NCDP and is called a failure close-down process (FCDP). With wafer residency time constraints (WRTCs) being imposed, close-down process optimization for such a tool is challenging, since one needs to not only finish this process as soon as possible but also meet WRTCs during this transient process. In order to tackle this problem, this article first introduces steady state scheduling problems. Then, with a presented backward robot task sequence, a linear programming model is first proposed to optimize NCDP. To deal with the PM failures, efficient PM failure response policies are formulated for the cases in which a PM fails. Then, four linear programs are proposed to optimize an FCDP. Finally, industrial case studies are given to show the usefulness of the proposed approaches.

The semiconductor manufacturing industry adopts multi-cluster tools as wafer fabrication equipment. If one of process modules fails, a multi-cluster tool cannot follow the normal schedule to complete the process recipe of work-in-process wafers and must empty these wafers by a closing down phase. It is economically paramount to shorten the close-down process with a process module failure (FCDP) and make no loss of wafers. However, due to the wafer residency time constraints, it is rather difficult to respond to a module failure and find a corresponding optimal schedule. By assuming no parallel modules at a step, this work studies this important issue for multi-cluster tools. For process-dominant multi-cluster tools whose optimal steady state schedule is known, we analyze the task sequences to avoid deadlock at the shared buffer modules. Algorithms are proposed to synthesize the proper sequences for robots in case of a process module failure, then, a nonlinear program model is derived to find an optimal schedule for the corresponding close-down process or decide no feasible solutions. An example is present to show the application of our proposed method.

As a kind of robotic cells, cluster tools are widely used for semiconductor wafer fabrication processes since they provide a reconfigurable and efficient environment. With recent advances in new semiconductor materials, the circuit line width has continuously being shrunk down, which brings new challenges for manufacturers. They require that a wafer should be moved away from a processing chamber as soon as possible after its processing is finished. To ensure high-quality integrated circuits in a wafer, its post-processing residency time must be minimized. It is also highly desirable to maximize the throughput of robotic cluster tools. This article aims at scheduling such tools with multiple objectives subject to wafer residency time constraints. To do so, new algorithms are proposed to calculate robot waiting time delicately upon the analysis of particular events of robot waiting for dual-blade robotic tools and optimally schedule such tools. The numerical results of industrial examples show that the proposed algorithms can provide an effective method to find schedules for dual-blade cluster tools such that multiple objectives are optimized.

Wafer revisiting and residency time constraints complicate the scheduling problem of cluster tools in semiconductor manufacturing. Random disturbance to the activity time in operating a tool further complicates such a scheduling problem. To solve this challenging problem, this paper proposes a robust real-time schedule which consists of a real-time controller (RTC) and an off-line schedule. The former is developed to offset the activity time disturbance such that the wafer sojourn time fluctuation in a process module is minimized. With the RTC, to find the off-line schedule, necessary and sufficient schedulability conditions under which a feasible schedule exists are derived and these conditions can be easily checked. Then, the off-line schedule can be efficiently found by the proposed algorithms based on nondisturbed activity time if a feasible schedule exists. With the obtained real-time schedule, it is shown that the productivity of the system is maximized. Finally, examples are used to illustrate the proposed approach.

A robotic cluster tool involves many activities whose time is subject to some disturbance, thus leading to the activity time variation. It results in wafer sojourn time fluctuation in a process module, which may in turn violate wafer residency time constraints. Some wafer fabrication requires a revisiting process. With wafer revisiting, the effect of activity time variation on wafer sojourn time fluctuation is so complicated that no analysis was reported to the best knowledge of the authors. It is vitally important to accurately analyze it. To do so, this paper adopts a Petri net model to describe the dynamical behavior of cluster tools. With this model, a real-time control policy is proposed to offset the effect of the activity time variation on wafer sojourn time fluctuation as much as possible. Then, the wafer sojourn time delay is analyzed and algorithms are developed to calculate its exact upper bound. With the proposed method, one can check if a given schedule is feasible under bounded activity time variation. Some practical examples are given to show the application of the proposed approach.

In wafer fabrication, a robotic cluster tool is required to be closed down in order for engineers to perform its on-demand and preventive maintenance and switch between different wafer lots. They often deal with a close-down process subject to wafer residency time constraints, i.e., a wafer must exit from a processing chamber before its quality degradation within a certain time limit. To obtain higher yield, it is very important to optimize a close-down process for a cluster tool. Yet the existing literature pays no or little attention to this issue. By focusing on a time-constrained single-arm cluster tool, this paper intends: 1) to build its Petri net model to analyze its schedulability and 2) to develop computationally efficient algorithms to find an optimal and feasible schedule for its closing-down process under different workloads at its steps. Industrial examples are used to illustrate the application of the proposed method.

Semiconductor manufacturing industry has adopted multi-cluster tools as wafer fabrication equipment that is extraordinarily pricey but highly attractive owing to their higher productivity than single cluster tools can achieve. A challenging issue is how to schedule these tools. It is especially difficult to schedule their frequently occurring close-down processes subject to wafer residency constraints. Such processes appear frequently as caused by wafer lot switches and preventive and emergency maintenances. They are dynamical and non-cyclic. We analyze the synchronization conditions for multiple robots to perform concurrent activities. Upon these conditions, for the situations that an optimal schedule can be found in the steady state, a linear program model is proposed to find a feasible and optimal schedule for close-down processes. An example shows the application and efficiency of our proposed method.

High-mix and low-volume wafer fabrication leads to more and more lot switches in cluster tools. Practitioners must thus deal with more transient processes during such switches, including start-up and close-down. To obtain higher throughput, it is critical to shorten these processes. Much effort has been put into the steady state modeling and scheduling of cluster tools and some for start up processes. However, no attention is paid to a close-down process for single-arm cluster tools with wafer residency constraints. This work aims to do so by 1) developing a Petri net model to analyze their properties and 2) proposing Petri net-based methods to solve their close-down optimal scheduling problems under different workloads among their process steps. An industrial example is given to illustrate their application.

It is very challenging to schedule a multi-cluster tool to maximize its throughput. This work studies its one-wafer optimal periodic schedule. It is found that the key to schedule it is to determine its robots waiting times. A resource-oriented Petri net model is developed for it such that the robot waiting times are well modeled. Based on the model, optimality conditions are derived and the scheduling problem is reduced to the determination of robot waiting times. By the derived conditions, an optimal one-wafer optimal periodic schedule for a multi-cluster tool can be obtained by scheduling its individual cluster tools one by one. Then, a highly efficient algorithm is proposed to compute it for an entire multi-cluster tool.