In recent years, a number of dynamic multiobjective evolutionary algorithms (DMOEAs) have been proposed for tackling dynamic multiobjective optimization problems (DMOPs). Most of DMOEAs adopt learning methods to extract search experiences from past environments, trying to predict a promising initial population in new environments. However, they often ignore the use of search experiences to guide the evolutionary trajectory in new environments, which is also important to accelerate their convergence. Thus, this paper proposes a historical search-guided evolutionary (HSGE) framework for tackling DMOPs, which designs a neural network-based pattern learning (NNPL) strategy and a historical direction-guided evolutionary (HDGE) strategy. First, the NNPL strategy trains a neural network to effectively extract search experiences from historical environments. Then, based on these search experiences, the HDGE strategy is designed to steer the evolutionary direction of population, aiming to speed up its convergence in new environments. After embedding four dynamic response mechanisms into the HSGE framework, the corresponding DMOEAs have shown superior performance over the original DMOEAs on most test DMOPs. Moreover, the experimental results also validate the advantages of HSGE over two state-of-the-art optimization frameworks for tackling DMOPs.

This paper proposes a multiobjective many-tasking evolutionary algorithm with enhanced search direction-based knowledge transfer (MMaTEA-ESD). Specifically, the search directions for all tasks are first dynamically computed based on the populations obtained during the evolutionary search process. Subsequently, the search direction of the target task is divided into multiple subsearch directions. After that, the most similar subsearch directions from candidate source tasks are identified by measuring their similarity to those of the target task. Finally, the selected subsearch directions are combined to form the enhanced search direction for knowledge transfer, effectively mitigating negative transfer. Numerical results on a widely used multiobjective many-tasking benchmark test suite demonstrate the competitive performance of MMaTEA-ESD over some state-of-the-art algorithms.

To efficiently tackle problems with multiple conflicting objectives, several Multi-Objective Reinforcement Learning (MORL) algorithms utilize a universal policy network that takes preference weights as input to represent optimal policies for all different preferences. However, it is quite challenging to train such a universal policy as it is easy to forget or fail to learn skills for some preferences. To alleviate this issue, we propose an efficient Population-Based MORL (PB-MORL) method that trains multiple agents with universal policy networks using a shared replay buffer. Each agent is biased towards optimizing specific objectives by applying differentiated weights to the rewards sampled from the buffer. Therefore, the policy of each agent only needs to handle the specific part of the preference space rather than the entire space, simplifying the training task. Meanwhile, the experiences in the common buffer facilitate the information sharing among individuals, which can significantly reduce the number of interaction steps for training multiple agents. Experiments on both continuous and discrete tasks demonstrate the superiority of PB-MORL over several state-of-the-art MORL methods.

Many combinatorial multiobjective optimization problems involve very costly-to-evaluate objectives and constraints. It is very difficult, if not impossible, for traditional heuristics to solve these problems with an acceptable amount of computational time. In this paper, we show that offline machine learning can be very useful to assist multiobjective evolutionary algorithms to tackle this kind of problem. We take a complicated real-life multiobjective routing-packing problem as the test bed. We propose to use offline machine learning methods to replace time-consuming packing heuristics for packing feasibility prediction. Experiments show that the machine learning models can be 1,000 times faster than some commonly used packing heuristics and their accuracy can be as high as 98%. We adopt MOEA/D to decompose the problem into a number of single objective subproblems and solve them in a collaborative manner. We propose an encoding strategy to represent each routing scheme and use genetic operators to generate new routes. Experimental studies have been conducted on 100 instances from HUAWEI’s real-world logistics application and two test suites from the literature. Our proposed method can solve each HUAWEI instance in around one minute. Our solutions on the two test suites are comparable to other existing algorithms, and the overall computational cost of our method is significantly lower than others.

The integration of Evolutionary Algorithm (EA) and Reinforcement Learning (RL) has emerged as a promising approach for tackling some challenges in RL, such as sparse rewards, lack of exploration, and brittle convergence properties. However, existing methods often employ actor networks as individuals of EA, which may constrain their exploratory capabilities, as the entire actor population will stop evolution when the critic network in RL falls into local optimal. To alleviate this issue, this paper introduces a Two-stage Evolutionary Reinforcement Learning (TERL) framework that maintains a population containing both actor and critic networks. TERL divides the learning process into two stages. In the initial stage, individuals independently learn actor-critic networks, which are optimized alternatively by RL and Particle Swarm Optimization (PSO). This dual optimization fosters greater exploration, curbing susceptibility to local optima. Shared information from a common replay buffer and PSO algorithm substantially mitigates the computational load of training multiple agents. In the subsequent stage, TERL shifts to a refined exploitation phase. Here, only the best individual undergoes further refinement, while the rest individuals continue PSO-based optimization. This allocates more computational resources to the best individual for yielding superior performance. Empirical assessments, conducted across a range of continuous control problems, validate the efficacy of the proposed TERL paradigm.

In recent years, the global repercussions of SARS-CoV-2 and its variants have posed significant challenges to various areas, including the economic order, transportation, healthcare, and education, and the mitigation and prevention of SARS-CoV-2 and other infectious diseases have received much attention. This paper tries to develop a rational and effective intervention strategy in response to the dynamic evolution of the SARS-CoV-2 epidemic. To address this issue, we first use the SEIR model to analyze the population distribution under the SARS-CoV-2 epidemic, and then integrate a deep Q-neural network (DQN) to reflect an economic quantification model contextualized within the SARS-CoV-2 scenario. Next, we propose an evolutionary deep reinforcement learning to generate an optimal and adaptive intervention strategy, by optimizing the weight sequence of the DQN. The experimental results show the effectiveness of the proposed method in maximizing economic benefits, while ensuring the uninterrupted functionality of medical institutions. Importantly, the learned intervention strategy exhibits the applicability to other infectious diseases.

Recently, the application of transfer learning within dynamic multiobjective evolutionary algorithms (DMOEAs) has shown significant potential to solve dynamic multiobjective optimization problems (DMOPs). This approach utilizes the transfer learning technique which reuses information from previous environments to accelerate the search process in the new environment. However, the risk of negative transfer can lead to an erroneous search direction and inefficient use of computational resources. To address this issue, this paper proposes a novel multistage transfer learning DMOEA, named MSTL. The algorithm is designed to enhance the convergence and diversity of the population through the combination of the pre-transfer learning stage and the transfer learning and validation stage when environmental changes occur. In the pre-transfer learning stage, a suitable target domain, source domain, and validation classifier sample sets are generated guided by historical information. In the transfer learning and validation stage, the TrAdaboost technique to transfer knowledge from the target to the source domain, with the results verified by two validation classifiers to mitigate negative transfer. Experimental results demonstrate that our proposed algorithm outperforms four competing algorithms in terms of diversity and convergence on a widely recognized benchmark suite.

A number of Neural Architecture Search (NAS) methods have been proposed to automate the design of Convolutional Neural Networks (CNNs). However, they typically require significant computational resources and exhibit search instability. To save computational costs, many NAS methods design only a single group of cell architectures (i.e., a small part of the total network architecture), which are simply stacked to form the final network. This leads to a lack of diversity in network architecture design, thereby reducing network performance. To alleviate this problem, we propose a Particle Swarm Optimization (PSO)-based Stage-Wise NAS algorithm (named NAS-SW). Initially, we introduce a stage-wise architecture search strategy, which can find different cell architectures at various search stages. Then, we propose a grouping architecture update strategy, which can realize effective information transfer between cell architectures rapidly. Importantly, these strategies do not add extra computational costs. Experiments demonstrate that NAS-SW has superior performance and better efficiency. NAS-SW only needs 0.37 GPU days to design a CNN on CIFAR-10. Additionally, on the CIFAR-10 and CIFAR-100 datasets, the network architectures searched by NAS-SW achieve top-1 validation accuracies of 97.55% and 83.78%, respectively.

The Vehicle Routing Problem with Simultaneous Pickup-Delivery and Time Windows (VRPSPDTW) is a highly challenging issue in complex logistics distribution scenarios, requiring an optimal balance between cost and efficiency. Traditional methods often rely on single heuristic or metaheuristic algorithms, which perform not so well when dealing with VRPSPDTW. To overcome this challenge, we propose a deep reinforcement learning-based multi-agent algorithm (DRL-MA) to tackle the VRPSPDTW. Our algorithm includes explorative, exploitative, and perturbative agents, which are responsible for balancing exploration and exploitation. The action space of each agent comprises a combination of neighborhood operators, and then the Deep Q-network (DQN) is used to learn effective neighborhood transition sequences from a long-term perspective, which can effectively explore large and complex solution spaces. The cooperation and competition among agents during the search process offer a more flexible and effective strategy. Experimental studies conducted on a real test suite of large-scale VRPSPDTW instances validate the superiority of our proposed DRL-MA over some state-of-the-art algorithms.

In autonomous transportation, the vehicle routing problem with simultaneous pickup-delivery and time windows (VRPSPDTW) plays a crucial role in enhancing transportation efficiency, thereby garnering significant research attention over the past decade. However, due to the increased complexity of VRPSPDTW, traditional heuristic algorithms have been inadequate in solving this problem. To address this challenge, this paper proposes a multitask-based evolutionary algorithm (MBEA) with knowledge transfer designed specifically for solving VRPSPDTW in autonomous transportation. MBEA tackles large-scale VRPSPDTW instances by utilizing multiple auxiliary tasks to aid the optimization process. Initially, MBEA generates$k$different auxiliary tasks by randomly selecting a subset of customers from the original VRPSPDTW. Subsequently, an evolutionary multitasking approach is employed to generate offspring solutions. This enables the transfer of valuable routing information among tasks, facilitating the evolutionary search for the original VRPSPDTW. Experimental studies conducted on a practical test suite of large-scale VRPSPDTW instances validate the superiority of our proposed algorithm over several recently proposed approaches.Note to Practitioners—The vehicle routing problem with simultaneous pickup-delivery and time windows (VRPSPDTW) is prevalent in autonomous transportation and finds application in various industrial sectors. It arises when manufacturing companies need to collect scrap products from customers for recycling purposes or deliver purchased commodities within specified time windows. However, traditional optimization methods face challenges in effectively addressing these problems due to their large-scale requirements and the necessity to provide satisfactory service. Thus, this paper proposes a multitask-based evolutionary algorithm with knowledge transfer that aims to obtain optimal solutions for VRPSPDTW. Our algorithm utilizes multiple auxiliary tasks to facilitate the optimization process for the original large-scale VRPSPDTW. Experimental results demonstrate the effectiveness of our approach in dealing with practical large-scale VRPSPDTW within a reasonable time frame.

Dynamic pickup and delivery problems (DPDPs) with various constraints, such as docks, time windows, capacity, and last-in-first-out loading, have posed significant challenges for existing vehicle routing algorithms, as most of them only optimize a single weighted objective function, which makes it difficult to maintain the solutions’ diversity and may easily become stuck in local optima. To alleviate this issue, this paper introduces a decomposition-based multiobjective evolutionary algorithm with tabu search for solving the above DPDPs. First, our algorithm leverages multiobjectivization and reformulates the DPDP as a multiobjective optimization problem (MOP), which is further decomposed into multiple subproblems. Then, these subproblems are approached simultaneously and collaboratively by using a crossover process to enhance the diversity of the solutions, followed by using an efficient tabu search to speed up the convergence. In this way, our algorithm can better balance the trade-off between exploration and exploitation for solving this MOP, and then one promising solution can be selected from the population to complete some pickup and delivery tasks in an interval of the DPDP. Simulation results on 64 test problems from a practical scenario of Huawei demonstrate that the proposed algorithm outperforms other competitive algorithms for tackling DPDPs. Additionally, more experiments are conducted on 20 large-scale distribution problems within JD Logistics to validate the generalization capability of our algorithm.

Traditional reproduction operators in many-objective evolutionary algorithms (MaOEAs) seem to not be so effective to tackle many-objective optimization problems (MaOPs). This is mainly because the population size cannot be set to an arbitrarily large value if the computational efficiency is of concern. In such a case, the distance between the parents becomes remarkably large and, consequently, it is not easy to reproduce a superior offspring in high-dimensional objective space. To alleviate this problem, an elite gene-guided (EGG) reproduction operator is proposed to tackle MaOPs in this article. In this operator, an elite gene pool is built by collecting the knee points from the current population. Then, the offspring is produced by exchanging the genes with this elite gene pool under an exchange rate, aiming to reserve more promising genes into the next generation. In order to provide new genes for the population, other genes will be disturbed under a disturbance rate. The settings and functional analysis of the exchange rate and disturbance rate are studied using several experiments. The proposed EGG operator is easy to implement and can be embedded to any MaOEA. As examples, we show the embedding of the proposed EGG operator into four competitive MaOEAs, that is, MOEA/D, NSGA-III, θ -DEA, and SPEA2-SDE provide some advantages over simulated binary crossover, differential evolution, and an evolutionary path-based reproduction operator on solving a number of benchmark problems with 3 to 15 objectives.

Overall constraint violation functions are commonly used in multiobjective evolutionary algorithms (MOEAs) for handling constraints. Constraints could cause these algorithms stuck in two stagnation states: 1) since the feasible region of a multiobjective optimization problem can consist of several disconnected feasible subregions, the search can be easily trapped in a feasible subregion which does not contain all the global Pareto optimal solutions and 2) an overall constraint violation function may have many nonzero minimal points, it can make the search stuck in an unfeasible area. To address these two issues, this article proposes a strategy to detect whether or not the search is stuck in these two stagnation states and then escape from them. Our proposed detect-and-escape strategy uses the feasible ratio and the change rate of overall constraint violation to detect stagnation, and adjusts the weight of the constraint violation for guiding the search to escape from stagnation states. We develop and implement a decomposition-based constrained MOEA with this strategy. Extensive experiments on a number of benchmark problems demonstrate the competitiveness of our proposed algorithm when compared to five other state-of-the-art constrained evolutionary algorithms.

Decomposition-based constrained multiobjective evolutionary algorithms decompose a constrained multiobjective problem into a set of constrained single-objective subproblems. For each subproblem, the aggregation function and the overall constraint violation need to be minimized simultaneously, which however may conflict with each other during the evolutionary process. To solve this issue, this paper proposes a novel decomposition-based constrained multiobjective evolutionary algorithm with two types of weight vectors, respectively emphasizing convergence and diversity. The solutions associated to the convergence weight vectors are updated only considering the aggregation function in order to search the whole search space freely, while the ones associated to the diversity weight vectors are renewed by considering both the aggregation function and the overall constraint violation, which encourages to search around the feasible region found so far. Once the replacement of solutions does not happen for the diversity weight vectors in a period, the corresponding diversity weight vectors will be transferred to convergence one. Thereafter, all solutions will finally search around the feasible region, which helps to find more feasible or superior solutions. The proposed constraint handling technique can have a good balance to search the feasible and infeasible regions and show the promising performance, which is validated when tackling several constrained multi-objective problems.

Recently, it was found that most multiobjective particle swarm optimizers (MOPSOs) perform poorly when tackling many-objective optimization problems (MaOPs). This is mainly because the loss of selection pressure that occurs when updating the swarm. The number of nondominated individuals is substantially increased and the diversity maintenance mechanisms in MOPSOs always guide the particles to explore sparse regions of the search space. This behavior results in the final solutions being distributed loosely in objective space, but far away from the true Pareto-optimal front. To avoid the above scenario, this paper presents a balanceable fitness estimation method and a novel velocity update equation, to compose a novel MOPSO (NMPSO), which is shown to be more effective to tackle MaOPs. Moreover, an evolutionary search is further run on the external archive in order to provide another search pattern for evolution. The DTLZ and WFG test suites with 4-10 objectives are used to assess the performance of NMPSO. Our experiments indicate that NMPSO has superior performance over four current MOPSOs, and over four competitive multiobjective evolutionary algorithms (SPEA2-SDE, NSGA-III, MOEA/DD, and SRA), when solving most of the test problems adopted.

The selection of swarm leaders (i.e., the personal best and global best), is important in the design of a multiobjective particle swarm optimization (MOPSO) algorithm. Such leaders are expected to effectively guide the swarm to approach the true Pareto optimal front. In this paper, we present a novel external archive-guided MOPSO algorithm (AgMOPSO), where the leaders for velocity update are all selected from the external archive. In our algorithm, multiobjective optimization problems (MOPs) are transformed into a set of subproblems using a decomposition approach, and then each particle is assigned accordingly to optimize each subproblem. A novel archive-guided velocity update method is designed to guide the swarm for exploration, and the external archive is also evolved using an immune-based evolutionary strategy. These proposed approaches speed up the convergence of AgMOPSO. The experimental results fully demonstrate the superiority of our proposed AgMOPSO in solving most of the test problems adopted, in terms of two commonly used performance measures. Moreover, the effectiveness of our proposed archive-guided velocity update method and immune-based evolutionary strategy is also experimentally validated on more than 30 test MOPs.