Dynamic multi-objective optimization problems (DMOPs) with a changing number of objectives may have Pareto-optimal set (PS) manifold expanding or contracting over time. Knowledge transfer has been used for solving DMOPs, since it can transfer useful information from solving one problem instance to solve another related problem instance. However, we show that the state-of-the-art transfer approach based on heuristic lacks diversity on problem with extremely strong bias and loses convergence on problems with multi-modality and variable correlation, after the number of objectives increases and decreases, respectively. Therefore, we propose a novel transfer strategy based on learning, called learning to expand and contract PS (denoted as LEC) for enhancing diversity and convergence after number of objective increases and decreases, respectively. It firstly learns potentially good directions for expansion and contraction separately via principal component analysis. Then, the most promising expansion and contraction directions are selected from their candidates according to whether they help diversity and convergence, respectively. Lastly, PS is learnt to be expanded and contracted based on these most promising directions. Comprehensive studies using 13 DMOP benchmarks with a changing number of objectives demonstrate that our proposed LEC is effective on improving solution quality, not only right after changes but also after optimization of different generations, compared to state-of-the-art algorithms.

Rapid advancements in text-to-3D shape synthesis using generative AI models have achieved remarkable general-ization performance, generating novel designs based on com-binations of different concepts. However, designs synthesized through automated optimization often result in ill-defined shapes, which render such designs impractical for engineering design applications. We present the first end-to-end prompt evolution design optimization (PREDO) framework contextualized in a vehicle design scenario that leverages a vision-language model to penalize impractical car designs synthesized by a generative model. The backbone of our framework is an evolutionary strategy coupled with an optimization objective function that comprises a physics-based solver and a vision-language model for practical or functional guidance in the generated car designs. In the prompt evolution search, the optimizer iteratively generates a population of text prompts, which embed user specifications on the aerodynamic performance and visual preferences of the 3D car designs. Then, in addition to the computational fluid dynamics simulations, the pre-trained vision-language model is used to penalize impractical designs and, thus, foster the evolutionary algorithm to seek more viable designs. Our investigations on a car design optimization problem show a wide spread of potential car designs generated at the early phase of the search, which indicates a good diversity of designs in the initial populations, and an increase of over 20 % in the probability of generating practical designs compared to a baseline framework without using a vision-language model. Visual inspection of the designs against the performance results demonstrates prompt evolution as a very promising paradigm for finding novel designs with good optimization performance while providing ease of use in specifying design specifications and preferences via a natural language interface.

Abstract In the early stages of engineering design, multitudes of feasible designs can be generated using structural optimization methods by varying the design requirements or user preferences for different performance objectives. Data mining such potentially large datasets is a challenging task. An unsupervised data-centric approach for exploring designs is to find clusters of similar designs and recommend only the cluster representatives for review. Design similarity can be defined not only on a purely functional level but also based on geometric properties, such as size, shape, and topology. While metrics such as chamfer distance measure the geometrical differences intuitively, it is more useful for design exploration to use metrics based on geometric features, which are extracted from high-dimensional 3D geometric data using dimensionality reduction techniques. If the Euclidean distance in the geometric features is meaningful, the features can be combined with performance attributes resulting in an aggregate feature vector that can potentially be useful in design exploration based on both geometry and performance. We propose a novel approach to evaluate such derived metrics by measuring their similarity with the metrics commonly used in 3D object classification. Furthermore, we measure clustering accuracy, which is a state-of-the-art unsupervised approach to evaluate metrics. For this purpose, we use a labeled, synthetic dataset with topologically complex designs. From our results, we conclude that Pointcloud Autoencoder is promising in encoding geometric features and developing a comprehensive design exploration method.

Hanabiis a cooperative game that brings the problem of modeling other players to the forefront. In this game, coordinated groups of players can leverage preestablished conventions to great effect. In this article, we focus onad hocsettings with no previous coordination between partners. We introduce a “Bayesian Meta-Agent” that maintains a belief distribution over hypotheses of partner policies. The policies that serve as initial hypotheses are generated using MAP-Elites, to ensure behavioral diversity. We evaluate an “Adaptive” version of the agent, which selects a response policy based on the updated belief distribution and a “Generalist” version, which selects a response based on the uniform prior. In short episodes of ten games with a consistent partner, the “Adaptive” version outperforms the “Generalist” when the training and evaluation populations are the same. This presents a first step toward an agent that can model its partner and adapt within a time frame that is compatible with human interaction.

Topology optimization optimizes material layout in a design space for a given objective, such as crash energy absorption, and a set of boundary conditions. In industrial applications, multi-objective topology optimization requires expensive simulations to evaluate the objectives and generate multiple Pareto-optimal solutions. So, it is more economical to identify preferred regions on the Pareto front and generate only the desired solutions. Clustering methods, a widely used subclass of machine learning methods, provide an unsupervised approach to summarize the dataset, which eases the identification of the preferred set of designs. However, generating solutions similar to the preferred designs based on different metrics is a challenging task. In this paper, we present an interactive method to generate designs similar to a preferred set using one of the state-of-the-art weighted-sum approaches called scaled energy weighting - hybrid cellular automata (SEW-HCA). To avoid unnecessary computations, metamodels are used to predict the desired weight vectors needed by SEW-HCA. We evaluate an application of our method for cooperative topology optimization using a cantilever multi-load-case problem and a crashworthiness optimization problem. Using the proposed method, we could successfully generate designs that are similar to preferred solutions based on geometry and performance. We believe that this is a crucial component that will improve the usefulness of multi-objective topology optimization in real-world applications.

The dynamic capacitated arc routing problem (DCARP) aims at re-scheduling the service plans of agents, such as vehicles in a city scenario, when dynamic events deteriorate the quality of the current schedule. Various algorithms have been proposed to solve DCARP instances in different dynamic scenarios. However, most existing work evaluated their algorithms' performance based on artificially constructed dynamic environments instead of using more realistic traffic simulations which are built on actual traffic data. In this paper, we constructed a novel DCARP benchmarking framework based on the Simulation of Urban MObility (SUMO) transportation simulation software, which allows to include real-world traffic environments for generating a set of DCARP instances from dynamic events, such as road congestion or task changes. The flexibility of the framework allows to develop DCARP optimization algorithms and evaluate their effectiveness more comprehensively. We use the benchmarking framework to generate 12 different dynamic instances using real-world traffic data of Dublin City. We then demonstrate the value of our framework by using these instances to compare our previously proposed hybrid local search algorithm (HyLS) with a state-of-the-art meta-heuristic optimization algorithm. The generated benchmark scenarios indicate that HyLS is a very effective optimizer on DCARP scenarios with real traffic data for reducing the total service cost. They also demonstrate the importance of our DCARP benchmarking framework for the development and benchmarking of optimization algorithms in more realistic scenarios.

The Capacitated Arc Routing Problem (CARP) aims at assigning vehicles to serve tasks which are located at different arcs in a graph. However, the originally planned routes are easily affected by different dynamic events like newly added tasks. This gives rise to Dynamic CARP (DCARP) instances, which need to be efficiently optimized for new high-quality service plans in a short time. However, it is unknown which dynamic events make DCARP instances especially hard to solve. Therefore, in this paper, we provide an investigation of the influence of different dynamic events on DCARP instances from the perspective of fitness landscape analysis based on a recently proposed hybrid local search (HyLS) algorithm. We generate a large set of DCARP instances based on a variety of dynamic events and analyze the fitness landscape of these instances using several different measures such as fitness correlation length. From the empirical results we conclude that cost-related events have no significant impact on the difficulty of DCARP instances, but instances which require more new vehicles to serve the remaining tasks are harder to solve. These insights improve our understanding of the DCARP instances and pave the way for future work on improving the performance of DCARP algorithms.

Traditional methods for solving problems within computer science rely mostly upon the application of handcrafted algorithms. As however manual engineering of them can be considered to be a tedious process, it is interesting to consider how far internal mechanisms can be directly learned in an end-to-end manner instead. This is especially tempting to consider for metaheuristic and evolutionary optimization routines which inherently rely upon creating abundant amounts of data during run-time. To implement such an approach for these types of algorithms, it effectively requires a pipeline to first acquire deran-domized algorithm components in a domain-dependent manner and secondly a mapping to select them based upon characteristic features which unveil the black box character of an optimization problem. While in principle, within our prior work we proposed methods for extracting spatial features from metadata, these unfortunately fail to acknowledge the time-dependent nature of it. Thus, fail in scenarios when the inputs generated from initial iterations are not expressive enough. For this reason we specifically develop within this work architectures for spatio-temporal data processing. Particularly, we find that our proposed GCN-GRU and LSTM architectures, which take inspiration from CNN-LSTMs originally proposed for activity recognition in multimedia data-streams, demonstrate high efficiency and most consistent performance on time series of variable length. Further, we can also demonstrate that the class activation map (CAM) for interpretable learning with time series data helps to understand and reflects problem-dependent properties of the search behavior of an optimization algorithm.

Recent advancements in machine learning comprise generative models such as autoencoders (AE) for learning and compressing 3D data to generate low-dimensional latent representations of 3D shapes. Learning latent representations that disentangle the underlying factors of variations in 3D shapes is an intuitive way to achieve generalization in generative models. However, it remains an open problem to learn a generative model of 3D shapes such that the latent variables are disentangled and represent different interpretable aspects of 3D shapes. In this paper, we propose Split-AE, which is an autoencoder-based architecture for partitioning the latent space into two sets, named as content and style codes. The content code represents global features of 3D shapes to differentiate between semantic categories of shapes, while style code represents distinct visual features to differentiate between shape categories having similar semantic meaning. We present qualitative and quantitative experiments to verify feature disentanglement using our Split-AE. Further, we demonstrate that, given a source shape as an initial shape and a target shape as a style reference, the trained Split-AE combines the content of a source and style of a target shape to generate a novel augmented shape, that possesses the distinct features of the target shape category yet maintains the similarity of the global features with the source shape. We conduct a qualitative study showing that the augmented shapes exhibit a realistic interpretable mixture of content and style features across different shape classes with similar semantic meaning.

Dynamic optimization, for which the objective functions change over time, has attracted intensive investigations due to the inherent uncertainty associated with many real-world problems. For its robustness with respect to noise, evolutionary algorithms (EAs) have been expected to have great potential for dynamic optimization. Many dynamic optimization methods, such as diversity-driven methods, memory methods, and prediction methods have been proposed based on EAs to deal with environmental changes. However, they face difficulties in adapting to fast changes in dynamic optimization as EAs normally need quite a few fitness evaluations to find a near-optimum solution. To address this issue, this article proposes a new framework of applying EAs in the context of dynamic optimization to deal with fast changing environments. We suggest that instead of online evolving (searching) solutions for the ever-changing objective function, EAs are more suitable for acquiring an archive of solutions in an offline way, which could be adopted to construct a system to provide high-quality solutions efficiently in a dynamic environment. To be specific, we formulate the offline search as a static set-oriented optimization problem. Then, a set of solutions is obtained by an EA for this set-oriented optimization problem. After this, the obtained solution set is adopted to do fast adaptation to the corresponding dynamic optimization problem. The general framework is instantiated for continuous dynamic-constrained optimization problems, and the empirical results show the potential of the proposed framework. The superiority of the framework is also verified on a dynamic vehicle routing problem with changing demands.

The choice of design representations, as of search operators, is central to the performance of evolutionary optimization algorithms, in particular, for multitask problems. The multitask approach pushes further the parallelization aspect of these algorithms by solving simultaneously multiple optimization tasks using a single population. During the search, the operators implicitly transfer knowledge between solutions to the offspring, taking advantage of potential synergies between problems to drive the solutions to optimality. Nevertheless, in order to operate on the individuals, the design space of each task has to be mapped to a common search space, which is challenging in engineering cases without clear semantic overlap between parameters. Here, we apply a 3-D point cloud autoencoder to map the representations from the Cartesian to a unified design representation: the latent space of the autoencoder. The transfer of latent space features between design representations allows the reconstruction of shapes with interpolated characteristics and maintenance of common parts, which potentially improves the performance of the designs in one or more tasks during the optimization. Compared to traditional representations for shape optimization, such as free-form deformation, the latent representation enables more representative design modifications, while keeping the baseline characteristics of the learned classes of objects. We demonstrate the efficiency of our approach in an optimization scenario where we minimize the aerodynamic drag of two different car shapes with common underbodies for cost-efficient vehicle platform design.

The capacitated arc routing problem (CARP) is a challenging combinatorial optimization problem abstracted from many real-world applications, such as waste collection, road gritting, and mail delivery. However, few studies considered dynamic changes during the vehicles’ service, which can cause the original schedule infeasible or obsolete. The few existing studies are limited by the dynamic scenarios considered, and by overly complicated algorithms that are unable to benefit from the wealth of contributions provided by the existing CARP literature. In this article, we first provide a mathematical formulation of dynamic CARP (DCARP) and design a simulation system that is able to consider dynamic events while a routing solution is already partially executed. We then propose a novel framework which can benefit from the existing static CARP optimization algorithms so that they could be used to handle DCARP instances. The framework is very flexible. In response to a dynamic event, it can use either a simple restart strategy or a sequence transfer strategy that benefits from the past optimization experience. Empirical studies have been conducted on a wide range of DCARP instances to evaluate our proposed framework. The results show that the proposed framework significantly improves over state-of-the-art dynamic optimization algorithms.

Large-scale, high-quality data sets are central to the development of advanced machine learning techniques that increase the effectiveness of existing optimization methods or even inspire novel ones. Especially in the engineering domain, such high-quality data sets are rare due to confidentiality concerns and generation costs, be it computational or manual efforts. We, therefore, introduce the OSU-Honda Automobile Hood Dataset (CarHoods10k), an industry-grade 3-D vehicle hood data set of over 10 000 shapes along with mechanical performance data that were validated against real-world hood designs by industry experts. CarHoods10k offers researchers and practitioners the unique opportunity to develop novel methods on realistic data with relevance to real-world vehicle design. To illustrate central use cases, we first apply methods from geometric deep learning to learn a compact latent representation for design space exploration. Second, we use machine learning models to predict mechanical hood performance from the learned latent representation. We thus demonstrate the effectiveness of machine learning for building metamodels, which are used in design optimization whenever possible to replace costly engineering simulations. Third, we integrate CarHoods10k in a topology optimization approach based on evolutionary algorithms to demonstrate its capability to search for high-performing structures, while maintaining manufacturability constraints.

Methods for learning on 3D point clouds became ubiquitous due to the popularization of 3D scanning technology and advances of machine learning techniques. Among these methods, point-based deep neural networks have been utilized to explore 3D designs in optimization tasks. However, engineering computer simulations require high-quality meshed models, which are challenging to automatically generate from unordered point clouds. In this work, we propose Point2FFD: A novel deep neural network for learning compact geometric representations and generating simulation-ready meshed models. Built upon an autoencoder architecture, Point2FFD learns to compress 3D point clouds into a latent design space, from which the network generates 3D polygonal meshes by selecting and deforming simulation-ready mesh templates. Through benchmark experiments, we show that our proposed network achieves comparable shape-generative performance than existing state-of-the-art point-based generative models. In real world-inspired vehicle aerodynamic optimizations, we demonstrate that Point2FFD generates simulation-ready meshes of realistic car shapes and leads to better optimized designs than the benchmarked networks.

Methods for learning and compressing high-dimensional data allow designers to generate novel and low-dimensional design representations for shape optimization problems. By using compact design spaces, global optimization algorithms require less function evaluations to characterize the problem landscape. Furthermore, data-driven representations are often domain-agnostic and independent of the user expertise, and thus potentially capture more relevant design features than a human designer would suggest. However, more factors than the dimensionality play a role in the efficiency of design representations. In this paper, we perform a comparative analysis of design representations for 3D shape optimization problems obtained with principal component analysis, kernel-principal component analysis and a 3D point cloud autoencoder, which we apply on a benchmark data set of computer aided engineering car models. We evaluate the shape-generative capabilities of these methods and show that we can modify the geometries more locally with the autoencoder than with the remaining methods. In a vehicle aerodynamic optimization framework, we verify that this property of the autoencoder representation improves the optimization performance by enabling potentially complementary degrees of freedom for the optimizer. With our study, we provide insights on the qualitative properties and quantifiable measures on the efficiency of deep neural networks as shape generative models for engineering optimization problems, as well as analyses of geometric representations for engineering optimization with evolutionary algorithms.

In the early design phase of automotive digital development, one of the key challenges for the designer is to consider multiple-criteria like aerodynamics and structural efficiency besides aesthetic aspects for designing a car shape. In our research, we imagine a cooperative design system in the automotive domain which provides guidance to the designer for finding sets of design options or well-performing designs for preferred search areas. In the present paper, we focus on two perspectives for this multi-criteria decision-making problem: First, a scenario without prior information about design preferences, where the designer aims to explore the search space for a diverse set of design alternatives. Second, a scenario where the designer has a prior intuition on preferred solutions of interest. For both scenarios, we assume that historic 3D car shape data exists, which we can utilize to learn a compact low-dimensional design representation based on a variational autoencoder (VAE). In contrast to evolutionary multi-objective optimization approaches where starting populations are randomly initialized, we propose to seed the population more efficiently by exploiting the advantage of linear interpolation in the latent space of the VAE. In our experiments, we demonstrate that the multi-objective optimization converges faster and achieves a diverse set of solutions. For the second scenario, when specifying design preferences by weights, we improve on the weighted-sum method, which simplifies the multi-objective problem and propose a strategy for efficiently adapting the weights towards the preferred design solution.

The imbalanced classification problem is very relevant in both academic and industrial applications. The task of finding the best machine learning model to use for a specific imbalanced dataset is complicated due to a large number of existing algorithms, each with its own hyperparameters. The Combined Algorithm Selection and Hyperparameter optimization (CASH) has been introduced to tackle both aspects at the same time. However, CASH has not been studied in detail in the class imbalance domain, where the best combination of resampling technique and classification algorithm is searched for, together with their optimized hyperparameters. Thus, we target the CASH problem for imbalanced classification. We experiment with a search space of 5 classification algorithms, 21 resampling approaches and 64 relevant hyperparameters in total. Moreover, we investigate performance of 2 well-known optimization approaches: Random search and Tree Parzen Estimators approach which is a kind of Bayesian optimization. For comparison, we also perform grid search on all combinations of resampling techniques and classification algorithms with their default hyperparameters. Our experimental results show that a Bayesian optimization approach outperforms the other approaches for CASH in this application domain.

Recent years have seen the advancement of data-driven paradigms in population-based and evolutionary optimization. This reflects on one hand the mere abundance of available data, but on the other hand also progresses in the refinement of previously available machine learning methods. Surprisingly, deep pattern recognition methods emerging from the studies of neural networks have only been sparingly applied. This comes unexpected, as the complex data generated by evolutionary search algorithms can be considered tedious and intractable for manual analysis with mere practical intuitions. In this work, we therefore explore opportunities to employ deep networks to directly learn problem characteristics of continuous optimization problems. Particularly, with data obtained during initial runs of an optimization algorithm. We find that a graph neural network, trained upon a graph representation of continuous search spaces, shows in comparison to more traditional approaches higher validation accuracy and retrieves characteristics within the latent space which are better at distinguishing different continuous optimization problems. We hope that our study can pave the way towards new approaches which allow us to learn problem-dependent algorithm components and recall these from predictions of inputs generated during the run-time of an optimization algorithm.

Evolutionary algorithms are a class of population-based meta-heuristic methods partially inspired by natural evolution. Specifically, they rely on stochastic variation and selection processes to sequentially find optimal solutions of a function of interest. We attempt in this work to extract preferences in these stochastic evolutionary operators in form of empirical and improved distributions as basis for model-based mutation operators. The latter can be considered as representing problem-tailored search operators which exist independently from the optimisation run and thus can be transferred to similar problem instances. This offline approach is different to existing model-based optimisation techniques, e.g. EDA's, CMA-ES and Bayesian approaches, where adaption happens rather in an online manner without the influence of prior experience. Our approach can be rather considered to follow the recent line of research on knowledge transfer in optimisation, which until now heavily relies upon the transfer of candidate solutions across different optimisation tasks. We investigate in this paper the interplay between algorithm and optimisation task, its influence on the retrieved distributions and explore whether or not these can lead to performance improvements on a selected range of problems, as well as when transferring them across problems. At last, we make a comparison of built distributions in the hope of relating similarity in statistical distances between distributions to possible performance gains.

Transfer learning has been used for solving multiple optimization and dynamic multi-objective optimization problems, since transfer learning is believed to be able to transfer useful information from one problem instance to help solving another related problem instance. This paper aims to study how effective transfer learning is in dynamic multi-objective optimization (DMO). Through computation time analysis of transfer learning, we show that the `inner' optimization problem introduced by transfer learning is very time-consuming. In order to enhance the efficiency, two alternatives are computationally investigated on a number of dynamic bi- and tri-objective test problems. Experimental results have shown that the greatly enhanced efficiency does not result in much degeneration on the performance of transfer learning. Considering the high computational cost of transfer learning, it is likely that the original purpose of using transfer learning in DMO might be negated. In other words, the computation time saved in optimization is eaten up by computationally expensive transfer learning. As a result, there is less gain than expected in the overall computational efficiency. To verify this, experiments have been conducted, regarding using computational cost of transfer learning to optimize randomly generated solutions. The results have demonstrated that the convergence and diversity of final solutions generated from the random solutions are significantly better than those generated from transferred solutions under the same total computational budget.

In order to reduce the dimensionality of 3D point cloud representations, autoencoder architectures generate increasingly abstract, compressed features of the input data. Visualizing these features is central to understanding the learning process, however, while successful visualization techniques exist for neural networks applied to computer vision tasks, similar methods for geometric, especially non-Euclidean, input data are currently lacking. Hence, we propose a first-of-kind method to project the features learned by point cloud autoencoders into a 3D-space augmented with color maps. Our proposal explores the properties of 1D-convolutions, used in state-of-the art point cloud autoencoder architectures to handle the input data, which leads to an intuitive interpretation of the visualized features. Furthermore, we tackle the search for relevant co-activations in the feature space by clustering the input data in the latent space, where we explore the correspondence between network features and geometric characteristics of typical shapes of the clusters. We tested our approach with experiments on a benchmark data set, and with three different configurations of a point cloud autoencoder, where we show that the features learned by the autoencoder correlate with the occupancy of the input space by the training data.

Clinical time series are known for irregular, highly-sporadic and strongly-complex structures and are consequently difficult to model by traditional state-space models. In this paper, we investigate the potential of applying variational recurrent neural networks (VRNNs) for forecasting clinical time series extracted from electronic health records (EHRs) of patients. Variational recurrent neural networks (VRNNs) combine recurrent neural networks (RNNs) and variational inference (VI) and are state-of-the-art methods to model highly-variable sequential data such as text, speech, time series and multimedia signals in a generative fashion. We propose to incorporate multiple correlated time series to improve the forecasting of VRNNs. The selection of these correlated time series is based on the similarity of the supplementary medical information e.g., disease diagnostics, ethnicity and age etc. between the patients. We evaluate the effectiveness of utilizing such supplementary information with root mean square error (RMSE), on clinical benchmark data-set "Medical Information Mart for Intensive Care (MIMIC III)" for multi-step-ahead prediction. We further perform subjective analysis to highlight the effects of the similarity of the supplementary medical information on individual temporal features e.g., Systolic Blood Pressure (SBP), Heart Rate (HR) etc. of the patients from the same data-set. Our results clearly show that incorporating the correlated time series based on the supplementary medical information can help improving the accuracy of the VRNNs for clinical time series forecasting.

The notion of learning from different problem instances, although an old and known one, has in recent years regained popularity within the optimization community. Notable endeavors have been drawing inspiration from machine learning methods as a means for algorithm selection and solution transfer. However, surprisingly approaches which are centered around internal sampling models have not been revisited. Even though notable algorithms have been established in the last decades. In this work, we progress along this direction by investigating a method that allows us to learn an evolutionary search strategy reflecting rough characteristics of a fitness landscape. This latter model of a search strategy is represented through a flexible mixture-based distribution, which can subsequently be transferred and adapted for similar problems of interest. We validate this approach in two series of experiments in which we first demonstrate the efficacy of the recovered distributions and subsequently investigate the transfer with a systematic from the literature to generate benchmarking scenarios.

Although the anomaly detection problem can be considered as an extreme case of class imbalance problem, very few studies consider improving class imbalance classification with anomaly detection ideas. Most data-level approaches in the imbalanced learning domain aim to introduce more information to the original dataset by generating synthetic samples. However, in this paper, we gain additional information in another way, by introducing additional attributes. We propose to introduce the outlier score and four types of samples (safe, borderline, rare, outlier) as additional attributes in order to gain more information on the data characteristics and improve the classification performance. According to our experimental results, introducing additional attributes can improve the imbalanced classification performance in most cases (6 out of 7 datasets). Further study shows that this performance improvement is mainly contributed by a more accurate classification in the overlapping region of the two classes (majority and minority classes). The proposed idea of introducing additional attributes is simple to implement and can be combined with resampling techniques and other algorithmic-level approaches in the imbalanced learning domain.

In complex scenarios where a model of other actors is necessary to predict and interpret their actions, it is often desirable that the model works well with a wide variety of previously unknown actors. Hanabi is a card game that brings the problem of modeling other players to the forefront, but there is no agreement on how to best generate a pool of agents to use as partners in ad-hoc cooperation evaluation. This paper proposes Quality Diversity algorithms as a promising class of algorithms to generate populations for this purpose and shows an initial implementation of an agent generator based on this idea. We also discuss what metrics can be used to compare such generators, and how the proposed generator could be leveraged to help build adaptive agents for the game.

Engineering designs can involve multiple stages, where at each stage, the design models are incrementally modified and optimized. In contrast to traditional dynamic optimization problems, where the changes are caused by some objective factors, the changes in such incremental optimization problems (IOPs) are usually caused by the modifications made by the decision makers during the design process. While existing work in the literature is mainly focused on traditional dynamic optimization, little research has been dedicated to solving such IOPs. In this paper, we study how to adopt cooperative coevolution to efficiently solve a specific type of IOPs, namely, those with increasing decision variables. First, we present a benchmark function generator on the basis of some basic formulations of IOPs with increasing decision variables and exploitable modular structure. Then, we propose a contribution-based cooperative coevolutionary framework coupled with an incremental grouping method for dealing with them. On one hand, the benchmark function generator is capable of generating various benchmark functions with various characteristics. On the other hand, the proposed framework is promising in solving such problems in terms of both optimization accuracy and computational efficiency. In addition, the proposed method is further assessed using a real-world application, i.e., the design optimization of a stepped cantilever beam.

Real-world evolutionary design optimizations of complex shapes can efficiently be solved using linear deformation representations, but the optimization performance crucially depends on the initial deformation setup. For instance, when modeling the deformation by radial basis functions (RBF) the convergence speed depends on the condition number of the involved kernel matrix, which previous work therefore tried to optimize through careful placement of RBF kernels. We show that such representation-specific techniques are inherently limited and propose a generic, representation-agnostic optimization based on orthogonalization of the deformation matrix. This straightforward black-box optimization projects any given linear deformation setup to optimal condition number without changing its design space, which, as we show through extensive numerical experiments, can boost the convergence speed of evolutionary optimizations by up to an order of magnitude.

As a well-known engineering practice, concurrent engineering (CE) considers all elements involved in a product's life cycle from the early stages of product development, and emphasizes executing all design tasks simultaneously. As a result, there exist various complex design problems in CE, which usually have many design parameters or require different disciplinary knowledge to solve them. To address these problems and enable concurrent design, different methods have been developed. The original problem is usually divided into small subproblems so that each subproblem can be solved individually and simultaneously. However, good decomposition, optimization, and communication strategies among subproblems are still needed in the field of CE. This paper attempts to study and analyze cooperative co-evolution (CC) based design optimization in CE by employing a parallel CC framework. Furthermore, it aims to develop new concurrent design methods based on parallel CC to solve different kinds of CE problems. To achieve this goal, a new novelty-driven CC is developed for design problems with complex structures and a novel concurrent design method is presented for quasi-separable multidisciplinary design optimization (MDO) problems. The efficacy of the new methods is studied on universal electric motor design problems and a general MDO problem, and compared to that of some existing methods. Additionally, this paper studies how the communication frequency among subpopulations affects the performance of the proposed methods. The optimal communication frequencies under different communication costs are reported as experimental results for both proposed methods on the test problems. Based on this paper, an effective self-adaptive method is proposed to be used in both optimization schemes, which is able to adapt the communication frequency during the optimization process.

A dynamic industrial design optimization requires high-quality optimization algorithms as well as adaptive representations to find the global solution for a given problem. For adapting the representation to changing environments or to new input we utilize the concept of evolvability, which in our interpretation consists of three criteria: variability, regularity, and improvement potential, where regularity and improvement potential characterize conflicting goals between exploration and exploitation. Our goal is the efficient adaptation of the representation according to a given preference weight between regularity and improvement potential. We propose a combination of two heuristics, Lloyd sampling and orthogonal least squares sampling, to initialize the adaptation process for a given preference weight. We show that this initialization improves the convergence speed of the adaptation process as well as the resulting fitness. We then realize a stepwise design optimization procedure by alternating the adaptation of the representation with optimization of the design. During the design optimization process we extract information which we exploit in the next adaptation phase. We show that an intermediate preference weight, balancing between regularity and improvement potential, allows to exploit this information and is robust to erroneous initial information. Thereby, we increase the performance of the whole design optimization process.

Industrial product design is characterized by increasing complexity due to the high number of involved parameters, objectives, and boundary conditions, all typically changing over time. Population-based evolutionary design optimization targets to solve these kinds of application problems, offering efficient algorithms striving for high-quality solutions. An important factor in the optimization setup is the representation, which defines the encoding of the design and the mapping from parameter space to design space. Being able to numerically quantify the quality of different representation settings would strengthen the optimal choice of encoding. Motivated by the biological concept of evolvability, we propose three criteria, namely variability, regularity, and improvement potential, to evaluate linear deformation representations for their use in shape optimization problems. The first aspect characterizes the exploration potential of the design space, the second measures the expected convergence speed, and the third determines the expected improvement of the quality of a design. We propose and experimentally analyze mathematical definitions for each of the three criteria. We demonstrate the successful application of our model to two evolutionary optimization scenarios: fitting of 1D height fields and fitting of 3D face scans, both based on RBF deformations. Due to the general character of our definition we expect the transferability of our concepts to alternative deformation methods.

A successful design optimization crucially depends on the underlying representation, which has to adapt to a variety of demands and changing boundary conditions. Complex system engineering addresses these challenges through key features like self-organization, modularity, locality, or evolution. The representation covers the parameter setup (location and quantity) and the mapping between parameter space (genotype) and design space (phenotype), and should allow for both adaptation and specialization of a design. To quantify the potential of a representation, suitable quality criteria are needed. Evolvability is such a criterion, which has been derived from biological analysis. However, many biological and technical studies propose different definitions of evolvability. We analyze, interpret, and extend them in order to derive an evolvability criterion suitable for complex system engineering. This can be used as a basis for future design optimization problems.

For a successful business model the efficient development and design of a comprehensive product family plays a crucial part in many real world applications. A product family as it occurs, e.g., in the automotive domain consists of a product platform which covers the commonalities of product variants and the derived product variants. While product variants need to be fast and flexibly adjusted to market needs, from manufacturing and development point of view an underlying product platform with a large number of common parts is required to increase cost efficiency. For the design and evaluation of optimization methods for product family development, in the present paper the universal electric motor (UEM) family problem is considered, as it provides a fair trade-off between complexity and computational costs compared to real world application scenarios in the automotive domain. Since especially solving this problem without usage of pre-knowledge comes with high computational costs, a cascaded evolutionary multi-objective optimization based on NSGA-II with concatenation of product Pareto fronts is proposed in the present paper to efficiently reduce computational time. Besides providing sets of Pareto solutions to the unbiased UEM family problem the effects of considering solutions of prior platform optimizations as starting point for follow-up optimizations under changing requirements are evaluated.

To deal with complex optimization problems plagued with computationally expensive fitness functions, the use of surrogates to replace the original functions within the evolutionary framework is becoming a common practice. However, the appropriate datacentric approximation methodology to use for the construction of surrogate model would depend largely on the nature of the problem of interest, which varies from fitness landscape and state of the evolutionary search, to the characteristics of search algorithm used. This has given rise to the plethora of surrogate-assisted evolutionary frameworks proposed in the literature with ad hoc approximation/surrogate modeling methodologies considered. Since prior knowledge on the suitability of the data centric approximation methodology to use in surrogate-assisted evolutionary optimization is typically unavailable beforehand, this paper presents a novel evolutionary framework with the evolvability learning of surrogates (EvoLS) operating on multiple diverse approximation methodologies in the search. Further, in contrast to the common use of fitness prediction error as a criterion for the selection of surrogates, the concept of evolvability to indicate the productivity or suitability of an approximation methodology that brings about fitness improvement in the evolutionary search is introduced as the basis for adaptation. The backbone of the proposed EvoLS is a statistical learning scheme to determine the evolvability of each approximation methodology while the search progresses online. For each individual solution, the most productive approximation methodology is inferred, that is, the method with highest evolvability measure. Fitness improving surrogates are subsequently constructed for use within a trust-region enabled local search strategy, leading to the self-configuration of a surrogate-assisted memetic algorithm for solving computationally expensive problems. A numerical study of EvoLS on commonly used benchmark problems and a real-world computationally expensive aerodynamic car rear design problem highlights the efficacy of the proposed EvoLS in attaining reliable, high quality, and efficient performance under a limited computational budget.

In this paper, we present a novel cross-surrogate assisted memetic algorithm (CSAMA) as a manifestation of multi co-objective evolutionary computation to enhance the search on computationally expensive problems by means of transferring, sharing and reusing information across objectives. In particular, the construction of surrogate for one objective is augmented with information from other related objectives to improve the prediction quality. The process is termed as a cross-surrogate modelling methodology, which will be used in lieu with the original expensive functions during the evolutionary search. Analyses on the prediction quality of the cross-surrogate modelling and the search performance of the proposed algorithm are conducted on the benchmark problems with assessments made against several state-of-the-art multiobjective evolutionary algorithms. The results obtained highlight the efficacy of the proposed CSAMA in attaining high quality Pareto optimal solutions under limited computational budget.

The performance of design optimizations which target the improvement of certain physical aspects of real world objects like e.g. in the automotive or aeronautical domain depends on the efficient interplay of optimization algorithm, evaluation method and shape representation. For the development of complex aerodynamic components, evolutionary algorithms as global stochastic optimization algorithms have been successfully coupled to shape morphing methods. Instead of a direct representation of the shapes' boundary, shape morphing methods like free-form deformation (FFD) apply scalable changes to a baseline prototype using a moderate number of parameters mapped to control point movements. The initial spatial arrangement of the control points influences strongly the design flexibility and the optimization performance in combination with the normal distributed mutation operator in evolutionary optimization algorithms. In the present paper, a method is proposed to support the generation process of initial FFD control volumes which is usually carried out manually in practice. The method is based on the concept of evolvability which is considered as the property of initial control volumes to generate favorable design variations within a moderate number of iterations while avoiding unfeasible mutations. We introduce mathematically translational design variability, mutational design variability and the central robust control volume as key features to compute an evolvable distribution of control points. In target shape matching experiments using an evolutionary strategy, the performance for different configurations of evolvability-tuned initial control volumes is shown empirically.

Evolutionary design optimization for improving the performance of real world objects, like e.g. car shapes in the context of aerodynamic efficiency, usually depends on a well balanced combination of representation, optimizer and design evaluation method. Shape representation requires a fair trade-off between minimum number of design parameters and design flexibility which likewise guarantees a good optimization convergence while allowing manifold design variations. Recently, shape morphing methods have gained increased attention because of their capability to represent complex shapes with a reasonable number of parameters, especially powerful if coupled with numerical simulations for measuring design performance. Free-form deformation, as prominent shape morphing representative, relies on an initial grid of control points, the control volume, which allows the modification of the embedded shape. The set-up of the control volume is a crucial process which in practice is done manually based on the experience of the human user. Here, a method for the automated construction of control volumes is suggested based on a proposed measure ECVwhich relies on the concept of evolvability as a potential capacity of representations to produce successful designs in a reasonable time. It is shown for target shape matching experiments that optimizations based on ECV-tuned control volumes provide a significantly better performance in design optimization.

During CAD development and any kind of design optimisation over years a huge amount of geometries accumulate in a design department. To organize and structure these designs with respect to reusability, a hierarchical set of components on different scalings is extracted by the designers. This hierarchy allows to compose designs from several parts and to adapt the composition to the current task. Nevertheless, this hierarchy is imposed by humans and relies on their experiences. In the present paper a computational method is proposed for an unsupervised extraction of design components from a large repository of geometries. Methods known from the field of object and pattern recognition in images are transferred to the 3D design space to detect relevant features of geometries. The non-negative matrix factorization algorithm (NMF) is extended and tuned to the given task for an autonomous detection of design components. The results of the NMF additionally provide an overview on the distribution of these components in the design repository. The extracted components sum up in a parts-based representation which serves as a base for manual or computational design development or optimisation respectively.