This paper introduces the Augmented Decision Space Optimization (ADSO) method for sparsity-driven optimization of mixed-strategies in Stackelberg Security Games (SSGs). The proposed method enhances traditional strategy optimization by combining binary variables to represent the presence of pure strategies with real-valued variables to refine their selection probabilities. Specifically, instead of waiting for an evolutionary process to gradually discover sparse solutions, the binary variables in ADS allow the real-valued variables to be switched on or off, thereby directly enforcing sparsity. This dual codification scheme achieves targets such as sparsification and computational efficiency in large-scale games. We demonstrate that ADS outperforms existing heuristic methods, offering superior solution quality, scalability, and stability. Empirical results across three different benchmark games show that ADS generates compact strategies with minimal computational overhead, achieving performance close to the exact methods. Furthermore, state-of-the-art results are obtained for problems where exact methods fail to scale effectively. Our framework promises broad applicability beyond SSGs, encompassing a wide range of game-theoretic and combinatorial optimization problems.

This paper presents an evolvable conditional diffusion method such that black-box, non-differentiable multi-physics models, as are common in domains like computational fluid dynamics and electromagnetics, can be effectively used for guiding the generative process to facilitate autonomous scientific discovery. We formulate the guidance as an optimization problem where one optimizes for a desired fitness function through updates to the descriptive statistic for the denoising distribution, and derive an evolution-guided approach from first principles through the lens of probabilistic evolution. Interestingly, the final derived update algorithm is analogous to the update as per common gradient-based guided diffusion models, but without ever having to compute any derivatives. We validate our proposed evolvable diffusion algorithm in two AI for Science scenarios: the automated design of fluidic topology and meta-surface. Results demonstrate that this method effectively generates designs that better satisfy specific optimization objectives without reliance on differentiable proxies, providing an effective means of guidance-based diffusion that can capitalize on the wealth of black-box, non-differentiable multi-physics numerical models common across Science.

Transfer multiobjective optimization promises sample-efficient discovery of near Pareto-optimal solutions to a target task by utilizing experiential priors from related source tasks. In this paper, we show that in domains where evaluation data is at a premium, e.g., in scientific and engineering disciplines involving time-consuming computer simulations or complex real-world experimentation, knowledge transfer through surrogate models can be pivotal in saving sample evaluation costs. While state-of-the-art algorithms (without transfer) typically assume budgets in the order of only a few hundred evaluations, we seek to explore how far we can get on even tighter budgets. The uniqueness of our proposed Expensive Transfer Evolutionary Multiobjective Optimizer (ExTrEMO) is that it can maximally utilize external information from hundreds of source datasets, including those that may be negatively correlated with the target task. This is achieved by melding evolutionary search with factorized transfer Gaussian process surrogates, capturing varied source-target correlations in potentially decentralized computation environments. We provide a regret bound analysis for ExTrEMO that translates to a theoretical proof of increasingly faster convergence as a result of multi-source transfers. The theory is experimentally verified on benchmark functions and toward accelerated design of biomedical microdevices. We release our code at https://github.com/LiuJ-2023/ExTrEMO.

In an era of pervasive digitalization, the growing volume and variety of data streams poses a new challenge to the efficient running of data-driven optimization algorithms. Targeting scalable multiobjective evolution under large-instance data, this article proposes the general idea of using subsampled small-data tasks as helpful minions (i.e., auxiliary source tasks) to quickly optimize for large datasets-via an evolutionary multitasking framework. Within this framework, a novel computational resource allocation strategy is designed to enable the effective utilization of the minions while guarding against harmful negative transfers. To this end, an intertask empirical correlation measure is defined and approximated via Bayes' rule, which is then used to allocate resources online in proportion to the inferred degree of source-target correlation. In the experiments, the performance of the proposed algorithm is verified on: 1) sample average approximations of benchmark multiobjective optimization problems under uncertainty and 2) practical multiobjective hyperparameter tuning of deep neural network models. The results show that the proposed algorithm can obtain up to about 73% speedup relative to existing approaches, demonstrating its ability to efficiently tackle real-world multiobjective optimization involving evaluations on large datasets.

Transfer optimization enables data-efficient optimization of a target task by leveraging experiential priors from related source tasks. This is especially useful in multiobjective optimization settings where a set of tradeoff solutions is sought under tight evaluation budgets. In this article, we introduce a novel concept of inverse transfer in multiobjective optimization. Inverse transfer stands out by employing Bayesian inverse Gaussian process models to map performance vectors in the objective space to population search distributions in task-specific decision space, facilitating knowledge transfer through objective space unification . Building upon this idea, we introduce the first Inverse Transfer Evolutionary Multiobjective Optimizer (invTrEMO). A key highlight of invTrEMO is its ability to harness the common objective functions prevalent in many application areas, even when decision spaces do not precisely align between tasks. This allows invTrEMO to uniquely and effectively utilize information from heterogeneous source tasks as well. Furthermore, invTrEMO yields high-precision inverse models as a significant byproduct, enabling the generation of tailored solutions on-demand based on user preferences. Empirical studies on multi- and many-objective benchmark problems, as well as a practical case study, showcase the faster convergence rate and modeling accuracy of the invTrEMO relative to state-of-the-art evolutionary and Bayesian optimization algorithms. The source code of the invTrEMO is made available at https://github.com/LiuJ-2023/invTrEMO .

In today's digital world, we are faced with an explosion of data and models produced and manipulated by numerous large-scale cloud-based applications. Under such settings, existing transfer evolutionary optimization (TrEO) frameworks grapple with simultaneously satisfying two important quality attributes, namely: 1) scalability against a growing number of source tasks and 2) online learning agility against sparsity of relevant sources to the target task of interest. Satisfying these attributes shall facilitate practical deployment of transfer optimization to scenarios with big task instances, while curbing the threat of negative transfer. While applications of existing algorithms are limited to tens of source tasks, in this article, we take a quantum leap forward in enabling more than two orders of magnitude scale-up in the number of tasks; that is, we efficiently handle scenarios beyond 1000 source task instances. We devise a novel TrEO framework comprising two co-evolving species for joint evolutions in the space of source knowledge and in the search space of solutions to the target problem. In particular, co-evolution enables the learned knowledge to be orchestrated on the fly, expediting convergence in the target optimization task. We have conducted an extensive series of experiments across a set of practically motivated discrete and continuous optimization examples comprising a large number of source task instances, of which only a small fraction indicate source-target relatedness. The experimental results show that not only does our proposed framework scale efficiently with a growing number of source tasks but is also effective in capturing relevant knowledge against sparsity of related sources, fulfilling the two salient features of scalability and online learning agility.

Deep reinforcement learning (DRL) policies have been shown to be deceived by perturbations (e.g., random noise or intensional adversarial attacks) on state observations that appear at test time but are unknown during training. To increase the robustness of DRL policies, previous approaches assume that explicit adversarial information can be added into the training process, to achieve generalization ability on these perturbed observations as well. However, such approaches not only make robustness improvement more expensive but may also leave a model prone to other kinds of attacks in the wild. In contrast, we propose an adversary agnostic robust DRL paradigm that does not require learning from predefined adversaries. To this end, we first theoretically show that robustness could indeed be achieved independently of the adversaries based on a policy distillation (PD) setting. Motivated by this finding, we propose a new PD loss with two terms: 1) a prescription gap maximization (PGM) loss aiming to simultaneously maximize the likelihood of the action selected by the teacher policy and the entropy over the remaining actions and 2) a corresponding Jacobian regularization (JR) loss that minimizes the magnitude of gradients with respect to the input state. The theoretical analysis substantiates that our distillation loss guarantees to increase the prescription gap and hence improves the adversarial robustness. Furthermore, experiments on five Atari games firmly verify the superiority of our approach compared to the state-of-the-art baselines.

Transfer evolutionary optimization (TrEO) has emerged as a computational paradigm to leverage related problem-solving information from various source tasks to boost convergence rates in a target task. State-of-the-art Tr EO algorithms have utilized a source-target similarity capture method with probabilistic priors that grants the ability to reduce negative transfers. A recent work makes use of an additional solution representation learning module to induce high ordinal correlation between source and target objective functions through source-to-target search space mappings, with the aim of promoting positive transfers between them. However, current implementations of this approach are found to be data-intensive - calling for all generated source data to be cached - leading to high storage costs in practice. As an alternative, this paper investigates the feasibility of a data-lean variant of the aforesaid approach, labeled as (1, G)-TrEO, in which only the first and final (Gth) generations of source data are used for solution representation learning and transfer. We conduct experimental analyses of (1, G)-TrEO using multi-objective benchmark functions as well as a practical example in vehicle crashworthiness design. Our results show that a simple data-lean transfer optimizer is able to achieve competitive performance. While this paper presents a first investigation of (1, G)-TrEO, we hope that the findings would inspire future forms of data-lean TrEO algorithms.

Policy distillation (PD) has been widely studied in deep reinforcement learning (RL), while existing PD approaches assume that the demonstration data (i.e., state-action pairs in frames) in a decision making sequence is uniformly distributed. This may bring in unwanted bias since RL is a reward maximizing process instead of simple label matching. Given such an issue, we denote the frame importance as its contribution to the expected reward on a particular frame, and hypothesize that adapting such frame importance could benefit the performance of the distilled student policy. To verify our hypothesis, we analyze why and how frame importance matters in RL settings. Based on the analysis, we propose an importance prioritized PD framework that highlights the training on important frames, so as to learn efficiently. Particularly, the frame importance is measured by the reciprocal of weighted Shannon entropy from a teacher policy's action prescriptions. Experiments on Atari games and policy compression tasks show that capturing the frame importance significantly boosts the performance of the distilled policies.

Evolutionary multitasking, which solves multiple optimization tasks simultaneously, has gained increasing research attention in recent years. By utilizing the useful information from related tasks while solving the tasks concurrently, improved performance has been shown in various problems. Despite the success enjoyed by the existing evolutionary multitasking algorithms, still there is a lack of theoretical studies guaranteeing faster convergence compared to the conventional single task case. To analyze the effects of transferred information from related tasks, in this article, we first put forward a novel multitask gradient descent (MTGD) algorithm, which enhances the standard gradient descent updates with a multitask interaction term. The convergence of the resulting MTGD is derived. Furthermore, we present the first proof of faster convergence of MTGD relative to its single task counterpart. Utilizing MTGD, we formulate a gradient-free evolutionary multitasking algorithm called multitask evolution strategies (MTESs). Importantly, the single task evolution strategies (ESs) we utilize are shown to asymptotically approximate gradient descent and, hence, the faster convergence results derived for MTGD extend to the case of MTES as well. Numerical experiments comparing MTES with single task ES on synthetic benchmarks and practical optimization examples serve to substantiate our theoretical claim.

Adversarial attack can be deemed as a necessary prerequisite evaluation procedure before the deployment of any reinforcement learning (RL) policy. Most existing approaches for generating adversarial attacks are gradient based and are extensive, viz., perturbing every pixel of every frame. In contrast, recent advances show that gradient-free selective perturbations (i.e., attacking only selected pixels and frames) could be a more realistic adversary. However, these attacks treat every frame in isolation, ignoring the relationship between neighboring states of a Markov decision process; thus resulting in high computational complexity that tends to limit their real-world plausibility due to the tight time constraint in RL. Given the above, this article showcases the first study of how transferability across frames could be exploited for boosting the creation of minimal yet powerful attacks in image-based RL. To this end, we introduce three types of frame-correlation transfers (FCTs) (i.e., anterior case transfer, random projection-based transfer, and principal components-based transfer) with varying degrees of computational complexity in generating adversaries via a genetic algorithm. We empirically demonstrate the tradeoff between the complexity and potency of the transfer mechanism by exploring four fully trained state-of-the-art policies on six Atari games. Our FCTs dramatically speed up the attack generation compared to existing methods, often reducing the computation time required to nearly zero; thus, shedding light on the real threat of real-time attacks in RL.

It is our pleasure to introduce this special issue on multitask evolutionary computation (MTEC), focusing on novel methodologies and applications of evolutionary algorithms (EAs) crafted to perform multiple search and optimization tasks jointly. EAs are population-based methods inspired by principles of natural evolution that have provided a gradient-free path to solving complex learning and optimization problems. However, unlike the natural world where evolution has engendered diverse species and produced differently skilled subpopulations,in silicoEAs are typically designed to evolve a set of solutions specialized for just a single target task. This convention of problem solving in isolation tends to curtail the power ofimplicit parallelismof a population. Skills evolved for a given problem instance do not naturally transfer to populations tasked to solve another. Hence, convergence rates remain restrained, even in settings where related tasks with overlapping search spaces, similar optimal solutions, or with other forms of reusable information, are routinely recurring.

Many decisions in a machine learning (ML) pipeline involve nondifferentiable and discontinuous objectives and search spaces. Examples include feature selection, model selection, and hyperparameter tuning, where candidate solutions in an outer optimization loop must be evaluated via a learning subsystem. Evolutionary algorithms (EAs) are prominent gradient-free methods to handle such tasks. However, EAs are known to pose steep computational challenges, especially when dealing with large-instance datasets. As opposed to prior works that often fall back on parallel computing hardware to resolve this big data problem of EAs, in this article, we propose a novel algorithm-centric solution based onevolutionary multitasking. Our approach involves the creation of a band ofminions, i.e., small data proxies to the main target task, that are constructed by subsampling a fraction of the large dataset. We then combine the minions with the main task in a single multitask optimization framework, boosting evolutionary search by using small data to quickly optimize for the large dataset. Our key algorithmic contribution in this setting is to allocate computational resources to each of the tasks in a principled manner. The article considers wrapper-based feature selection as an illustrative case study of the broader idea of using multitasking to speedup outer loop evolutionary configurations of any ML subsystem. The experiments reveal that multitasking can indeed speedup baseline EAs, by more than 40% on some datasets.

The Vehicle Routing Problem (VRP) is a well-known NP-hard combinatorial optimization problem, which has wide spread applications in real world, such as logistics, bus route planning, and urban path planning. To solve VRP, traditional optimization methods usually start the search from scratch and ignore the VRPs solved in the past, which could lead to repeated explorations of the search space of related problems, and thus results in slow optimization process involving unnecessary computational cost. Keeping this in mind, to speed up the optimization for vehicle routing, this article presents a new study towards faster vehicle routing by transferring knowledge from customer representations which are learned from past solved VRPs. In particular, we propose to capture the useful traits buried in previous optimized routing solutions by learning a new customer representation, which can be transferred across VRPs, serving as the prior knowledge, to bias the optimization in the target VRP. In contrast to existing approaches, the proposed knowledge transfer is consist of a learning of new customer representation based on the optimized routing solution, which is general to VRPs possessing different structural properties, and a weighted$l_{1}$norm-regularized formulation for building sparse mapping across VRPs, that is easy to solve. Further, the proposed knowledge transfer across VRPs occurs along the whole optimization search process, and is thus able to guide the routing optimization process consistently. To verify the efficacy of the proposed method, by using population-based optimization method as the VRP solver, comprehensive empirical studies on both commonly used VRP benchmarks and real world vehicle routing application are presented.

Black-box optimization algorithms typically start a search from scratch, assuming little prior knowledge about the task at hand. In practice, this approach can be prohibitive for computationally expensive problems, as a large number of costly function evaluations are often needed before a suitable (near-optimal) solution is found. Under this observation, recent efforts have incorporatedtransfer learningcapabilities into sequential model-based Bayesian optimization (BO) solvers, resulting in substantial performance speed-ups by leveraging information from related past problems. However, a common simplifying assumption in existing approaches is that the search spaces of a previously encounteredsourceand the ongoingtargettask bear the same features and dimensionality, with the difference lying in their respective objective functions. In this article, we present a generalizedtransfer BOalgorithm that relaxes the aforementioned assumption. Our method jointly transforms source features while training probabilistic transfer regression models for the target, thus applying to practical use-cases where (in addition to the difference in objective functions) the number of features could change across the source and target tasks; for example, features can be added and/or removed. The theoretical basis of our proposal is analyzed, and its empirical performance is demonstrated on synthetic benchmark functions as well as in realistic examples spanning engineering design and the automated configuration of a machine learning model. Note to Practitioners—Problems of industrial interest have a tendency of being repetitive in nature. For this reason, domain experts are always in high demand, as they are able to harness their experience of similar problems to come up with fast solutions in difficult situations. However, domain experts are not easy to find. Given this fact, the present paper puts forth a method for automating the process of knowledge extraction (through experiential learning) and transfer across problems in the domain of computationally expensive black-box optimization. The key novelty and motivation of this work lies in enabling the adaptive transfer of knowledge even when the number of features changes across the source and target problems. Our proposed approach is verified experimentally on a range of benchmarks as well as real-world problems of a computationally expensive nature, highlighting the utility of an optimization engine that is able to learn from experience without the need for constant human intervention.

Humans have the ability to identify recurring patterns in diverse situations encountered over a lifetime, constantly understanding relationships between tasks and efficiently solving them through knowledge reuse. The capacity of artificial intelligence systems to mimic such cognitive behaviors for effective problem solving is deemed invaluable, particularly when tackling real-world problems where speed and accuracy are critical. Recently, the notion of evolutionary multitasking has been explored as a means of solving multiple optimization tasks simultaneously using a single population of evolving individuals. In the presence of similarities (or even partial overlaps) between high-quality solutions of related optimization problems, the resulting scope for intertask genetic transfer often leads to significant performance speedup-as the cost of re-exploring overlapping regions of the search space is reduced. While multitasking solvers have led to recent success stories, a known shortcoming of existing methods is their inability to adapt the extent of transfer in a principled manner. Thus, in the absence of any prior knowledge about the relationships between optimization functions, a threat of predominantly negative (harmful) transfer prevails. With this in mind, this article presents a realization of a cognizant evolutionary multitasking engine within the domain of multiobjective optimization. Our proposed algorithm learns intertask relationships based on overlaps in the probabilistic search distributions derived from data generated during the course of multitasking-and accordingly adapts the extent of genetic transfers online. The efficacy of the method is substantiated on multiobjective benchmark problems as well as a practical case study of knowledge transfers from low-fidelity optimization tasks to substantially reduce the cost of high-fidelity optimization.

Recently, evolutionary multitasking (EMT) has been proposed in the field of evolutionary computation as a new search paradigm, for solving multiple optimization tasks simultaneously. By sharing useful traits found along the evolutionary search process across different optimization tasks, the optimization performance on each task could be enhanced. The autoencoding-based EMT is a recently proposed EMT algorithm. In contrast to most existing EMT algorithms, which conduct knowledge transfer across tasks implicitly via crossover, it intends to perform knowledge transfer explicitly among tasks in the form of task solutions, which enables the employment of task-specific search mechanisms for different optimization tasks in EMT. However, the autoencoding-based explicit EMT can only work on continuous optimization problems. It will fail on combinatorial optimization problems, which widely exist in real-world applications, such as scheduling problem, routing problem, and assignment problem. To the best of our knowledge, there is no existing effort working on explicit EMT for combinatorial optimization problems. Taking this cue, in this article, we thus embark on a study toward explicit EMT for combinatorial optimization. In particular, by using vehicle routing as an illustrative combinatorial optimization problem, the proposed explicit EMT algorithm (EEMTA) mainly contains a weighted l1-norm-regularized learning process for capturing the transfer mapping, and a solution-based knowledge transfer process across vehicle routing problems (VRPs). To evaluate the efficacy of the proposed EEMTA, comprehensive empirical studies have been conducted with the commonly used vehicle routing benchmarks in multitasking environment, against both the state-of-the-art EMT algorithm and the traditional single-task evolutionary solvers. Finally, a real-world combinatorial optimization application, that is, the package delivery problem (PDP), is also presented to further confirm the efficacy of the proposed algorithm.

With the emergence of crowdshipping and sharing economy, vehicle routing problem with occasional drivers (VRPOD) has been recently proposed to involve occasional drivers with private vehicles for the delivery of goods. In this article, we present a generalized variant of VRPOD, namely, the vehicle routing problem with heterogeneous capacity, time window, and occasional driver (VRPHTO), by taking the capacity heterogeneity and time window of vehicles into consideration. Furthermore, to meet the requirement in today's cloud computing service, wherein multiple optimization tasks may need to be solved at the same time, we propose a novel evolutionary multitasking algorithm (EMA) to optimize multiple VRPHTOs simultaneously with a single population. Finally, 56 new VRPHTO instances are generated based on the existing common vehicle routing benchmarks. Comprehensive empirical studies are conducted to illustrate the benefits of the new VRPHTOs and to verify the efficacy of the proposed EMA for multitasking against a state-of-art single task evolutionary solver. The obtained results showed that the employment of occasional drivers could significantly reduce the routing cost, and the proposed EMA is not only able to solve multiple VRPHTOs simultaneously but also can achieve enhanced optimization performance via the knowledge transfer between tasks along the evolutionary search process.

The design of the evolutionary algorithm with learning capability from past search experiences has attracted growing research interests in recent years. It has been demonstrated that the knowledge embedded in the past search experience can greatly speed up the evolutionary process if properly harnessed. Autoencoding evolutionary search (AEES) is a recently proposed search paradigm, which employs a single-layer denoising autoencoder to build the mapping between two problems by configuring the solutions of each problem as the input and output for the autoencoder, respectively. The learned mapping makes it possible to perform knowledge transfer across heterogeneous problem domains with diverse properties. It has shown a promising performance of learning and transferring the knowledge from past search experiences to facilitate the evolutionary search on a variety of optimization problems. However, despite the success enjoyed by AEES, the linear autoencoding model cannot capture the nonlinear relationship between the solution sets used in the mapping construction. Taking this cue, in this article, we devise a kernelized autoencoder to construct the mapping in a reproducing kernel Hilbert space (RKHS), where the nonlinearity among problem solutions can be captured easily. Importantly, the proposed kernelized autoencoding method also holds a closed-form solution which will not bring much computational burden in the evolutionary search. Furthermore, a kernelized autoencoding evolutionary-search (KAES) paradigm is proposed that adaptively selects the linear and kernelized autoencoding along the search process in pursuit of effective knowledge transfer across problem domains. To validate the efficacy of the proposed KAES, comprehensive empirical studies on both benchmark multiobjective optimization problems as well as real-world vehicle crashworthiness design problem are presented.

Memetic multi-agent system (MeMAS) is recently proposed as an enhanced version that integrates meme concept into multi-agent system (MAS) wherein all meme-inspired agents have an improvement in learning performance via meme evolution independently or social interaction. In the process of solving the black box optimization problem, the potential advantages of MeMAS have not been utilized well, which makes it a fertile area for further exploration. This paper presents a memetic multi-agent optimization paradigm through coordinate rotation (MeMAO-R) to combine MeMAS with evolutionary algorithms (EAs) to improve optimization efficiency. Based on MeMAS, the particular interest of MeMAO-R is placed on assisting original complex optimization task with new tasks generated by coordinate rotation. Further, MeMAO-R constructs the social interaction mechanism which facilitates to improve their convergence speed for solving the target optimization problem by utilizing meaningful information transferred across multiple agents with differing views of the target problem. Besides, MeMAO-R employs one or more classical EAs as the fundamental population based evolutionary solvers for multiple agents to optimize multiple tasks in a multi-agent scenario. Lastly, to testify the efficacy of the proposed MeMAO-R, comprehensive empirical studies on basic optimization problems are provided.

Humans rarely tackle every problem from scratch. Given this observation, the motivation for this paper is to improve optimization performance through adaptive knowledge transfer across related problems. The scope for spontaneous transfers under the simultaneous occurrence of multiple problems unveils the benefits of multitasking. Multitask optimization has recently demonstrated competence in solving multiple (related) optimization tasks concurrently. Notably, in the presence of underlying relationships between problems, the transfer of high-quality solutions across them has shown to facilitate superior performance characteristics. However, in the absence of any prior knowledge about the intertask synergies (as is often the case with general black-box optimization), the threat of predominantly negative transfer prevails. Susceptibility to negative intertask interactions can impede the overall convergence behavior. To allay such fears, in this paper, we propose a novel evolutionary computation framework that enables online learning and exploitation of the similarities (and discrepancies) between distinct tasks in multitask settings, for an enhanced optimization process. Our proposal is based on the principled theoretical arguments that seek to minimize the tendency of harmful interactions between tasks, based on a purely data-driven learning of relationships among them. The efficacy of our proposed method is validated experimentally on a series of synthetic benchmarks, as well as a practical study that provides insights into the behavior of the method in the face of several tasks occurring at once.

Sample-based evolutionary algorithms (EAs) are widely used for optimizing problems with multi (greater than one but less than four) or even many (greater than or equal to four) objectives of interest. In general, the difficulty of a problem exponentially increases with the number of objectives, serving as a clear example of the curse of dimensionality. The exploratory approach an EA takes in these cases has led to it being thought of as a big data generator, progressively sampling and evaluating solutions in high performing regions of a decision space to guide the search towards optimal solutions. Notably, in both multi- and many-objective EAs, the sampled data can be further utilized for building inverse generative models, mapping points in objective space back to solutions in the decision space. Such models offer immense flexibility to a decision maker in generating new target solutions on the fly, thereby facilitating real-time a posteriori preference incorporation into the search. In this paper, we show that the data distribution resulting from a many-objective formulation is in fact more conducive to building accurate inverse models than its multiobjective counterpart. Given the potential utility of these models, we in turn shed light on a rare blessing of dimensionality that is yet to be explored in the context of optimization. We first present simple theoretical arguments supporting our claim. Thereafter, experimental studies of Gaussian process-based inverse modeling for a synthetic and a real-world example are carried out to further confirm the theory.

Transfer optimization is an emerging concept that promises to enhance productivity of planning and decision-making processes by allowing for the adaptive reuse of knowledge (data) drawn from various “source” problems in a related ongoing “target” task of interest. Despite the recent advances in transfer optimization, however, a continuing challenge is the scalability of associated algorithms given big data of source problem instances. This paper tackles the scaling problem of an online adaptive knowledge transfer framework under big source data. We propose an efficient source selection algorithm based on the theory of multi-armed bandits such that the most related source task to the target is chosen for knowledge transfer, as opposed to extracting knowledge from all sources simultaneously. For this purpose, we introduce a novel and principled reward measure to reflect the source-target similarities. The efficacy of our proposed approach is assessed on the well-known knapsack problem that has practical implications in optimization of supply chain and manufacturing processes. Extensive experiments are conducted under big data of source problem instances. The numerical results clearly reveal that the incorporation of the proposed source selection mechanism in the existing adaptive knowledge transfer framework makes it successfully feasible for fast/real-time decision-making in the big data source setting.

In this paper, we propose a memetic multi-agent optimization (MeMAO) paradigm to enhance the search efficacy of classical EAs (i.e., Differential Evolution (DE)) in solving the complex optimization problems. The essential backbone of MeMAO is a recently proposed memetic multi-agent learning system wherein agents acquire increasing learning capabilities by interacting with the environment mainly in a reinforcement learning manner. Differing from MeMAS, the particular interest of MeMAO is placed on addressing the specific challenges when applying classical EAs to optimize the high dimensional optimization problems with a "low effective dimensionality". To achieve this, the target optimization problem is firstly re-formulated into multiple low dimensional tasks via random embedding methods. Further, MeMAO employs DE as the fundamental population based evolutionary solver for multiple agents to optimize multiple low dimensional tasks in a multi-agent scenario. Importantly, MeMAO constructs the social interaction mechanisms among multiple agents, hence improves their convergence speed for solving the target optimization problem by sharing the beneficial information across multiple agents. Lastly, to testify the efficacy of the proposed MeMAO, comprehensive empirical studies on 8 synthetic optimization problems with a dimensionality of 2,000 are provided.

Recently, evolutionary multi-tasking (EMT) has been proposed as a new evolutionary search paradigm that op-timizes multiple problems simultaneously. Due to the knowledge transfer across optimization tasks occurs along the evolutionary search process, EMT has been demonstrated to outperform the traditional single-task evolutionary search algorithms on many complex optimization problems, such as multimodal continuous optimization problems, NP-hard combinatorial optimization problems, and constrained optimization problems. Today, EMT has attracted lots of attentions, and many EMT algorithms have been proposed in the literature. The explicit EMT algorithm (EEMTA) is a recent proposed new EMT algorithm. In contrast to most of existing EMT algorithms, which employ a single population using unified space and common search operators for solving multiple problems, the EEMTA uses multiple populations which possess problem-specific solution representations and search mechanisms for different problems in evolutionary multi-tasking, which thus could lead to enhanced optimization performance. However, the original EEMTA was proposed for solving only two tasks. As knowledge transfer from inappropriate tasks may lead to negative effect on the evolutionary optimization process, additional designs of identifying task pairs for knowledge transfer is necessary in EEMTA for evolutionary multi-tasking with tasks more than two. To the best of our knowledge, there is no research effort has been conducted on this issue. Keeping this in mind, in this paper, we present a preliminary study on the task selection in EEMTA for many-task optimization. As task similarity may lose to capture the usefulness between tasks in evolutionary search, instead of using similarity measures for task selection, here we propose a credit assignment approach for selecting proper task to conduct knowledge transfer in explicit evolutionary many-tasking. The proposed approach is based on the feedbacks from the transferred solutions across tasks, which is adaptively updated along the evolutionary search. To confirm the efficacy of the proposed method, empirical studies on the many-task optimization problem, which consists of 7 commonly used optimization benchmarks, have been presented and discussed.

In transfer learning , we aim to improve the predictive modeling of a target output by using the knowledge from some related source outputs. In real-world applications, the data from the target domain is often precious and hard to obtain, while the data from source domains is plentiful. Thus, since the complexity of Gaussian process based multi-task/transfer learning approaches grows cubically with the total number of source＋ target observations, the method becomes increasingly impractical for large ( > 1 0 4 ) source data inputs even with a small amount of target data. In order to scale known transfer Gaussian processes to large-scale source datasets , we propose an efficient aggregation model in this paper, which combines the predictions from distributed (small-scale) local experts in a principled manner. The proposed model inherits the advantages of single-task aggregation schemes, including efficient computation, analytically tractable inference, and straightforward parallelization during training and prediction. Further, a salient feature of the proposed method is the enhanced expressiveness in transfer learning — as a byproduct of flexible inter-task relationship modelings across different experts. When deploying such models in real-world applications, each local expert corresponds to a lightweight predictor that can be embedded in edge devices, thus catering to cases of online on-mote processing in fog computing settings.

This paper draws motivation from the remarkable ability of humans to extract useful building-blocks of knowledge from past experiences and spontaneously reuse them for new and more challenging tasks. It is contended that successfully replicating such capabilities in computational solvers, particularly global black-box optimizers, can lead to significant performance enhancements over the current state-of-the-art. The main challenge to overcome is that in general black-box settings, no problem-specific data may be available prior to the onset of the search, thereby limiting the possibility of offline measurement of the synergy between problems. In light of the above, this paper introduces a novel evolutionary computation framework that enables online learning and exploitation of similarities across optimization problems, with the goal of achieving an algorithmic realization of the transfer optimization paradigm. One of the salient features of our proposal is that it accounts for latent similarities which while being less apparent on the surface, may be gradually revealed during the course of the evolutionary search. A theoretical analysis of our proposed framework is carried out, substantiating its positive influences on optimization performance. Furthermore, the practical efficacy of an instantiation of an adaptive transfer evolutionary algorithm is demonstrated on a series of numerical examples, spanning discrete, continuous, as well as single- and multi-objective optimization.

Evolutionary multitasking (EMT) is an emerging research topic in the field of evolutionary computation. In contrast to the traditional single-task evolutionary search, EMT conducts evolutionary search on multiple tasks simultaneously. It aims to improve convergence characteristics across multiple optimization problems at once by seamlessly transferring knowledge among them. Due to the efficacy of EMT, it has attracted lots of research attentions and several EMT algorithms have been proposed in the literature. However, existing EMT algorithms are usually based on a common mode of knowledge transfer in the form of implicit genetic transfer through chromosomal crossover. This mode cannot make use of multiple biases embedded in different evolutionary search operators, which could give better search performance when properly harnessed. Keeping this in mind, this paper proposes an EMT algorithm with explicit genetic transfer across tasks, namely EMT via autoencoding, which allows the incorporation of multiple search mechanisms with different biases in the EMT paradigm. To confirm the efficacy of the proposed EMT algorithm with explicit autoencoding, comprehensive empirical studies have been conducted on both the single- and multi-objective multitask optimization problems.

This paper proposes a Gaussian process (GP) based co-sub-Pareto front surrogate augmentation strategy for evolutionary optimization of computationally expensive multiobjective problems. In the proposed algorithm, a multiobjective problem is decomposed into a number of subproblems, the solution of each of which is used to approximate a portion or sector of the Pareto front (i.e., a subPF). Thereafter, a multitask GP model is incorporated to exploit the correlations across the subproblems via joint surrogate model learning. A novel criterion for the utility function is defined on the surrogate landscape to determine the next candidate solution for evaluation using the actual expensive objectives. In addition, a new management strategy for the evaluated solutions is presented for model building. The novel feature of our approach is that it infers multiple subproblems jointly by exploiting the possible dependencies between them, such that knowledge can be transferred across subPFs approximated by the subproblems. Experimental studies under several scenarios indicate that the proposed algorithm outperforms state-of-the-art multiobjective evolutionary algorithms for expensive problems. The parameter sensitivity and effectiveness of the proposed algorithm are analyzed in detail.

In most real-world settings, designs are often gradually adapted and improved over time. Consequently, there exists knowledge from distinct (but possibly related) design exercises, which have either been previously completed or are currently in-progress, that may be leveraged to enhance the optimization performance of a particular target optimization task of interest. Further, it is observed that modern day design cycles are typically distributed in nature, and consist of multiple teams working on associated ideas in tandem. In such environments, vast amounts of related information can become available at various stages of the search process corresponding to some ongoing target optimization exercise. Successfully exploiting this knowledge is expected to be of significant value in many practical settings, where solving an optimization problem from scratch may be exorbitantly costly or time consuming. Accordingly, in this paper, we propose an adaptive knowledge reuse framework for surrogate-assisted multiobjective optimization of computationally expensive problems, based on the novel idea of multiproblem surrogates. This idea provides the capability to acquire and spontaneously transfer learned models across problems, facilitating efficient global optimization. The efficacy of our proposition is demonstrated on a series of synthetic benchmark functions, as well as two practical case studies.

In some real-world applications, it has been found that the performance of multiobjective optimization evolutionary algorithms (MOEAs) may deteriorate when boundary solutions in the Pareto front (PF) are more difficult to approximate than others. Such a problem feature, referred to as difficult-to-approximate (DtA) PF boundaries, is seldom considered in existing multiobjective optimization test problems. To fill this gap and facilitate possible systematic studies, we introduce a new test problem generator. The proposed generator enables the design of test problems with controllable difficulties regarding the feature of DtA PF boundaries. Three representative MOEAs, NSGA-II, SMS-EMOA, and MOEA/D-DRA, are performed on a series of test problems created using the proposed generator. Experimental results indicate that all the three algorithms perform poorly on the new test problems. Meanwhile, a modified variant of MOEA/D-DRA, denoted as MOEA/D-DRA-UT, is validated to be more effective in dealing with these problems. Subsequently, it is concluded that the rational allocation of computational resources between different PF parts is crucial for MOEAs to handle the problems with DtA PF boundaries.

Many-objective optimization problems bring great difficulties to the existing multiobjective evolutionary algorithms, in terms of selection operators, computational cost, visualization of the high-dimensional tradeoff front, and so on. Objective reduction can alleviate such difficulties by removing the redundant objectives in the original objective set, which has become one of the most important techniques in many-objective optimization. In this paper, we suggest to view objective reduction as a multiobjective search problem and introduce three multiobjective formulations of the problem, where the first two formulations are both based on preservation of the dominance structure and the third one utilizes the correlation between objectives. For each multiobjective formulation, a multiobjective objective reduction algorithm is proposed by employing the nondominated sorting genetic algorithm II to generate a Pareto front of nondominated objective subsets that can offer decision support to the user. Moreover, we conduct a comprehensive analysis of two major categories of objective reduction approaches based on several theorems, with the aim of revealing their strengths and limitations. Lastly, the performance of the proposed multiobjective algorithms is studied extensively on various benchmark problems and two real-world problems. Numerical results and comparisons are then shown to highlight the effectiveness and superiority of the proposed multiobjective algorithms over existing state-of-the-art approaches in the related field.

Multiobjective evolutionary algorithms (MOEAs) have proven their effectiveness and efficiency in solving problems with two or three objectives. However, recent studies show that MOEAs face many difficulties when tackling problems involving a larger number of objectives as their behavior becomes similar to a random walk in the search space since most individuals are nondominated with respect to each other. Motivated by the interesting results of decomposition-based approaches and preference-based ones, we propose in this paper a new decomposition-based dominance relation to deal with many-objective optimization problems and a new diversity factor based on the penalty-based boundary intersection method. Our reference point-based dominance (RP-dominance), has the ability to create a strict partial order on the set of nondominated solutions using a set of well-distributed reference points. The RP-dominance is subsequently used to substitute the Pareto dominance in nondominated sorting genetic algorithm-II (NSGA-II). The augmented MOEA, labeled as RP-dominance-based NSGA-II, has been statistically demonstrated to provide competitive and oftentimes better results when compared against four recently proposed decomposition-based MOEAs on commonly-used benchmark problems involving up to 20 objectives. In addition, the efficacy of the algorithm on a realistic water management problem is showcased.

Recent analytical studies have revealed that in spite of promising success in problem solving, the performance of evolutionary multitasking deteriorates with decreasing similarity between constitutive tasks. The present day multifactorial evolutionary algorithm (MFEA) is susceptible to negative knowledge transfer between uncorrelated tasks. To alleviate this issue, we propose a linearized domain adaptation (LDA) strategy that transforms the search space of a simple task to the search space similar to its constitutive complex task. This high order representative space resembles high correlation with its constitutive task and provides a platform for efficient knowledge transfer via crossover. The proposed framework, LDA-MFEA is tested on several benchmark problems constituting of tasks with different degrees of similarities and intersecting global optima. Experimental results demonstrate competitive performances against MFEA and shows that our proposition dramatically improves the performance relative to optimizing each task independently.

To date, dynamic vehicle routing problem (DVRP) has attracted great research attentions due to its wide range of real world applications. In contrast to traditional static vehicle routing problem, the whole routing information in DVRP is usually unknown and obtained dynamically during the routing execution process. To solve DVRP, many heuristic and metaheuristic methods have been proposed in the literature. In this paper, we present a novel evolutionary search paradigm with learning capability for solving DVRP. In particular, we propose to capture the structured knowledge from optimized routing solution in early time slot, which can be further reused to bias the customer-vehicle assignment when dynamic occurs. By extending our previous research work, the learning of useful knowledge, and the scheduling of dynamic customer requests are detailed here. Further, to evaluate the efficacy of the proposed search paradigm, comprehensive empirical studies on 21 commonly used DVRP instances with diverse properties are also reported.

In recent decades, the field of multiobjective optimization has attracted considerable interest among evolutionary computation researchers. One of the main features that makes evolutionary methods particularly appealing for multiobjective problems is the implicit parallelism offered by a population, which enables simultaneous convergence toward the entire Pareto front. While a plethora of related algorithms have been proposed till date, a common attribute among them is that they focus on efficiently solving only a single optimization problem at a time. Despite the known power of implicit parallelism, seldom has an attempt been made to multitask, i.e., to solve multiple optimization problems simultaneously. It is contended that the notion of evolutionary multitasking leads to the possibility of automated transfer of information across different optimization exercises that may share underlying similarities, thereby facilitating improved convergence characteristics. In particular, the potential for automated transfer is deemed invaluable from the standpoint of engineering design exercises where manual knowledge adaptation and reuse are routine. Accordingly, in this paper, we present a realization of the evolutionary multitasking paradigm within the domain of multiobjective optimization. The efficacy of the associated evolutionary algorithm is demonstrated on some benchmark test functions as well as on a real-world manufacturing process design problem from the composites industry.

To enhance the search performance of evolutionary algorithms, reusing knowledge captured from past optimization experiences along the search process has been proposed in the literature, and demonstrated much promise. In the literature, there are generally three types of approaches for reusing knowledge from past search experiences, namely exact storage and reuse of past solutions, the reuse of model-based information, and the reuse of structured knowledge captured from past optimized solutions. In this paper, we focus on the third type of knowledge reuse for enhancing evolutionary search. In contrast to existing works, here we focus on knowledge transfer across heterogeneous continuous optimization problems with diverse properties, such as problem dimension, number of objectives, etc., that cannot be handled by existing approaches. In particular, we propose a novel autoencoding evolutionary search paradigm with learning capability across heterogeneous problems. The essential ingredient for learning structured knowledge from search experience in our proposed paradigm is a single layer denoising autoencoder (DA), which is able to build the connections between problem domains by treating past optimized solutions as the corrupted version of the solutions for the newly encountered problem. Further, as the derived DA holds a closed-form solution, the corresponding reusing of knowledge from past search experiences will not bring much additional computational burden on the evolutionary search. To evaluate the proposed search paradigm, comprehensive empirical studies on the complex multiobjective optimization problems are presented, along with a real-world case study from the fiber-reinforced polymer composites manufacturing industry.

Traditionally, single-objective and multi-objective optimization have only considered a single problem in one run. However, the notion of evolutionary multitasking, which aims at solving multiple optimization problems simultaneously, has recently emerged in Evolutionary Computation (EC). It is inspired by the implicit parallelism of population-based search, which attempts to take advantage of implicit genetic transfer in a multitasking environment. According to optimization literature, transforming a single-objective optimization (SOO) problem into a multi-objective optimization (MOO) problem has often been found to remove local optima. Motivated by the aforementioned idea and the concept of multitasking, in this paper, we introduce a new strategy for tackling complex multi-modal problems. In particular, we solve the original (or target) SOO task together with an artificially formulated MOO task in a multitask setting. Therein, the MOO task is expected to provide a useful inductive bias to the search progress of the target SOO task by leveraging on the transferable knowledge shared between them, thereby helping overcome local optima and effectively guiding the population towards more promising regions of the search space.

Over the years, the algorithms of evolutionary computation have emerged as popular tools for tackling complex real-world optimization problems. A common feature among these algorithms is that they focus on efficiently solving a single problem at a time. Despite the availability of a population of individuals navigating the search space, and the implicit parallelism of their collective behavior, seldom has an effort been made to multitask. Considering the power of implicit parallelism, we are drawn to the idea that population-based search strategies provide an idyllic setting for leveraging the underlying synergies between objective function landscapes of seemingly distinct optimization tasks, particularly when they are solved together with a single population of evolving individuals. As has been recently demonstrated, allowing the principles of evolution to autonomously exploit the available synergies can often lead to accelerated convergence for otherwise complex optimization tasks. With the aim of providing deeper insight into the processes of evolutionary multitasking, we present in this paper a conceptualization of what, in our opinion, is one possible interpretation of the complementarity between optimization tasks. In particular, we propose a synergy metric that captures the correlation between objective function landscapes of distinct tasks placed in synthetic multitasking environments. In the long run, it is contended that the metric will serve as an important guide toward better understanding of evolutionary multitasking, thereby facilitating the design of improved multitasking engines.

Compression Resin Transfer Moulding is a popular method for high volume production of superior quality fibre-reinforced polymer composite parts. However, the process involves a large number of design variables that must be carefully chosen in order to reduce cycle time, capital layout and running costs, while maximizing final part quality. These objectives are principally governed by two separate phases of the manufacturing cycle, namely the resin filling and curing phases. It turns out that independently optimizing either phase (which is the general practice) may often lead to conditions that significantly restrict or even adversely affect the progress of the other. In light of this fact, a novel approach of modelling the entire composites manufacturing problem as bi-level program, one that assimilates both phases, has been adopted in this paper. In particular, an efficient multi-objective bi-level evolutionary algorithm is designed to effectively deal with the computationally expensive simulation-based optimization problem. The unique feature of the algorithm is that it incorporates a Pareto Rank Learning scheme, together with surrogate assistance for the upper level problem, in order to eliminate several expensive but redundant objective function evaluations. The optimization process is therefore considerably accelerated, assisting manufacturers in making improved decisions for this complex engineering design problem.

The Robust Vehicle Routing Problem with Time Windows has been gaining popularity over the past few years due to its focus on tackling uncertainty inherent to real world problems. Most of the current approaches in generating robust solutions require prior knowledge on the uncertainties, such as uncertainties in travel time. Hence, they are less than favorable to use in the absence of data, i.e., in the case of data starvation. In this paper, we present an evolutionary algorithm that in the absence of data on travel time uncertainty, provides a decision maker with a collection of solutions, each with a corresponding level of trade-off between total travel distance and solution robustness. In particular, we present a novel realization of route flexibility and its relation to solution robustness. Furthermore, we propose a bi-objective evolutionary algorithm for the vehicle routing problem with time windows where the objectives are (a) total travel distance and (b) solution flexibility. The proposed algorithm is tested on the well-known Solomon benchmarks and a trade-off analysis between total distance and solution flexibility is provided based on the obtained test results. Based on observations from the trade-off analysis, a number of suggestions to improve the current logistics system are provided.

The design of evolutionary algorithms has typically been focused on efficiently solving a single optimization problem at a time. Despite the implicit parallelism of population-based search, no attempt has yet been made to multitask, i.e., to solve multiple optimization problems simultaneously using a single population of evolving individuals. Accordingly, this paper introduces evolutionary multitasking as a new paradigm in the field of optimization and evolutionary computation. We first formalize the concept of evolutionary multitasking and then propose an algorithm to handle such problems. The methodology is inspired by biocultural models of multifactorial inheritance, which explain the transmission of complex developmental traits to offspring through the interactions of genetic and cultural factors. Furthermore, we develop a cross-domain optimization platform that allows one to solve diverse problems concurrently. The numerical experiments reveal several potential advantages of implicit genetic transfer in a multitasking environment. Most notably, we discover that the creation and transfer of refined genetic material can often lead to accelerated convergence for a variety of complex optimization functions.

Bilevel optimization is a type of mathematical program in which one optimization problem (called the lower level problem) is nested within another (called the upper level problem). In recent years, there has been considerable interest in the development of algorithms that can handle multiple objective functions at both levels. The challenge lies in the implication that every upper level decision leads to a set of Pareto optimal solutions at the lower level. As a result, a single point in the upper level decision space maps to a set of points in the upper level objective space. Since standard multiobjective evolutionary algorithms (MOEAs) are not designed for such point-to-set mappings, the state-of-the-art solution methods often dictate several enhancements to existing MOEAs. In this paper, we propose an adaptive scalarization based approach to solving bilevel programs with multiple objectives at both levels, such that any off-the-shelf MOEA can be used with minimum modification. Subsequently, we put forward a surrogate-assistance technique that can significantly lower the computational cost commonly associated with such problems. Finally, proof-of-concept numerical experiments are carried out in order to demonstrate the potential of our proposed methods.

Profitable tour problem (PTP) belongs to the class of vehicle routing problem (VRP) with profits seeking to maximize the difference between the total collected profit and the total cost incurred. Traditionally, PTP involves single vehicle. In this paper, we consider PTP with multiple vehicles. Unlike the classical VRP that seeks to serve all customers, PTP involves the strategic-level customer selection so as to maximize the total collected profit and the operational-level route optimization to minimize the total cost incurred. Therefore, PTP is essentially the knapsack problem at the strategic level with VRP at the operational level. That means the evolutionary bi-level programming would be a suitable choice of methodology for solving the NP-hard PTP. Employing some evolutionary method to solve the bi-level program naively would undoubtedly be prohibitively expensive. We thus present in this paper the notion of knowledge adoption to approximate the initial solution to the lower-level optimization problem for a given trial solution of the upper-level decision variables. One may consider the knowledge adoption as a special case of knowledge transfer in which the transfer takes place within the same problem domain. Refining the approximate initial solution with local search forces it to quickly converge to some locally optimal solution. The better the estimation of the initial solution, the closer the local optimum will be to the global one. PTP finds its important application in the fields of transportation and logistics. In addressing last-mile problem using auction at the urban consolidation center (UCC), PTP plays a significant role in the winner determination problem (WDP) that follows. Empirical study demonstrates the efficacy of the proposed approach in solving the PTP-based WDP, yielding significantly higher profit, utilization, and service level compared to when the UCC adopts the conventional WDP based on multiple knapsack problem (i.e. the MKP-based WDP).