Code completion is a context-aware code completion function available in certain programming environments. It can accelerate the coding process of applications, enhance development efficiency, and reduce costs. Compared with the code generation, code completion has advantages such as flexible adaptation to changes, high precision and pertinence, and lower performance overhead. Intelligent code completion uses artificial intelligence technology to achieve code completion more effectively. However, it faces issues such as poor interpretability, unstable performance, and high demand for computing resources. To make full use of the powerful semantic understanding, information retrieval, and text generation capabilities of Large Language Models (LLMs), as well as the strong global optimization and adaptive search capabilities of Evolutionary Computation (EC), an integrated network framework, the Evolutionary Dual-Channel Network (EDN), is proposed. EDN leverages the precise context comprehension and feature extraction capabilities of large language models to predict the nodes of the abstract syntax tree, thereby achieving code completion. To enhance the performance and interpretability of the entire framework, we use the genetic algorithm (GA), with its powerful search ability, as the main optimization method for EDN. This proposed framework combining LLMs and EC in a novel way. By using EC to optimize certain components in the network framework of LLMs, it enables a more flexible integration of LLMs and EC. Experiment results on three public corpora consisting of multiple programming languages demonstrate the effectiveness of the proposed method.

In practical applications, dynamic multi-objective optimization problems inevitably face large-scale scenarios. Massive search space and high-dimensional historical data pose some serious challenges to existing prediction-based change response mechanisms, which affects their ability in maintaining population diversity and accurately predicting convergent solutions. In this paper, we propose a generative model-driven approach that trains a generative model for a new environment only through historical Pareto optimal sets. It can generate both convergent and diverse population for the new environment. We show that by introducing a new training scheme and loss function for adversarial autoencoder, the training of the generative model can achieve stable convergence under high-dimensional coupled data conditions. In addition, the trained generative model can maintain the diversity of generative candidate solutions by smooth sampling in the latent space. Extensive experiments were conducted on a typical dynamic multi-objective testing suite with problem settings ranging from 30 to 600 dimensions. These experimental results demonstrate that the optimization performance of the proposed approach outperforms those of the existing state-of-the-art designs.

Mixture of experts (MoE) has shown great potential in enhancing large language models, such as DeepSeek. In MoE, expert networks collaboratively process tokens routed by the gating network, enabling advanced capabilities such as semantic understanding, computational reasoning, and code generation. To address security threats such as DDoS in these dynamic environments, it is essential to implement tailored backup measures for these experts. However, traditional backup methods often lead to inefficiencies and excessive resource consumption. To overcome these challenges, we propose a novel approach for backing up experts in MoE. We formally define and mathematically model a new problem, termed the security-aware off-site pairwise protection (SOPP) problem, and prove its NP-hardness. To solve this problem, we develop three novel techniques of latency-aware MoE construction (LMC) to reduce backup latency, partitioned backup selection (PBS) to trade off security levels and resource consumption, as well as pairwise selective identifier (PSI) to determine the appropriate backup pairwise nodes. On the basis of these techniques, we propose an efficient heuristic algorithm called the off-site distributed pairwise nodes protection (OD-PNP), providing theoretical performance guarantees. Through extensive simulations and analyses, we demonstrate that our proposed algorithm outperforms state-of-the-art methods in terms of both protection efficiency and resource consumption.

Dynamic multiobjective optimization problems (DMOPs) are often scaled to large-scale scenarios in real-world applications, which inevitably must face the triple challenges of massive search space, dynamic environmental changes and multiobjective conflicts simultaneously. This article proposes an adversarial autoencoder-based large-scale dynamic multiobjective evolutionary framework. It integrates deep generative modeling techniques and large-scale multiobjective evolutionary algorithms (LMOEAs) to solve large-scale DMOPs effectively and efficiently. Specifically, an adversarial autoencoder-based deep generative network training architecture is proposed for high-dimensional decision variables in large-scale DMOPs. It can transfer a generative model trained on Pareto-optimal solutions in the current environment to a new environment using only the auxiliary information exhibited through the movement trajectories of historical Pareto-optimal solutions, resulting in the generation of quality initial populations for the new environment. Meanwhile, any proven LMOEA can be integrated into the proposed framework without extensive modifications. Experimental results on a typical dynamic multiobjective test suite with problem settings from 30 to 1000 dimensions demonstrate that the optimization performance of the proposed framework outperforms existing state-of-the-art designs. Especially in large-scale scenarios, the proposed framework is considered superior in terms of solution quality and computational efficiency.

Generalized zero-shot learning (GZSL) shows great potential for improving generalization to unseen classes in real-world scenarios. However, most GZSL methods depend on benchmark datasets with per-class attribute annotations, which creates a large semantic gap and worsens the domain shift problem in the visual-semantic space. To address these challenges, instance-level attributes offer an intuitive solution, but they require expensive manual annotation. In this paper, we propose a simple yet effective approach called per-instance attribute synthesis (PIAS) to generate diverse semantic representations for each instance. Our method first uses the Vision Transformer (ViT) model to extract visual features and then generates per-instance attributes. The patch splitting, positional embedding, and multi-head self-attention mechanisms in ViT improve the discriminability of both visual and semantic representations. Next, we define the generated attributes of class-average images as class anchor points. These anchor points are calibrated in the semantic space by minimizing the cosine similarity between the anchor points and per-class attribute annotations. Finally, we improve the diversity of generated per-instance attributes by aligning the topological structure between per-class attribute annotations and synthesized per-instance attributes with that between class-average visual features and per-instance visual features. We conduct comprehensive experiments on three challenging ZSL datasets: AWA2, CUB, and SUN. The results show that PIAS significantly outperforms state-of-the-art methods under both ZSL and GZSL settings. We further demonstrate the generalization ability of PIAS by applying it to attribute-based zero-shot image retrieval tasks.

Many problems in science and engineering can be mathematically modeled using partial differential equations (PDEs), which are essential for fields like computational fluid dynamics (CFD), molecular dynamics, and dynamical systems. Although traditional numerical methods like the finite difference/element method are widely used, their computational inefficiency, due to the large number of iterations required, has long been a challenge. Recently, deep learning (DL) has emerged as a promising alternative for solving PDEs, offering new paradigms beyond conventional methods. Despite the growing interest in techniques like physics-informed neural networks (PINNs), a systematic review of the diverse neural network (NN) approaches for PDEs is still missing. This survey fills that gap by categorizing and reviewing the current progress of deep NNs (DNNs) for PDEs. Unlike previous reviews focused on specific methods like PINNs, we offer a broader taxonomy and analyze applications across scientific, engineering, and medical fields. We also provide a historical overview, key challenges, and future trends, aiming to serve both researchers and practitioners with insights into how DNNs can be effectively applied to solve PDEs.

Recently, large language models (LLMs) have showcased remarkable capabilities across a diverse range of applications, including general natural language processing (NLP) and domain-specific tasks. Empirical evidence indicates that LLMs have matched or even surpassed human performance in various areas, such as language translation, reading comprehension, and logical reasoning. However, preliminary research reveals that LLMs struggle with basic character and word editing, which is crucial for practical tasks such as creating 1000-word articles or modifying specific text information. To comprehensively assess the capabilities of LLMs in character and word understanding and manipulation (CWUM), we introduce the CWUM benchmark in Chinese and English. CWUM comprises 23 tasks focusing on text editions, including counting, identification, insertion, and reversal. A comprehensive evaluation of nine advanced LLMs on CWUM is conducted, which highlights significant failures of existing LLMs on CWUM tasks that humans can solve perfectly with 100% accuracy. Meanwhile, specific deficiencies of LLMs in basic language understanding and manipulation are revealed by performing quality and quantity analysis. Furthermore, in the experiment part, various methods are investigated to improve model performance, demonstrating the effectiveness of supervised fine-tuning (SFT) in enhancing model performance on CWUM while maintaining generalization abilities on unseen tasks.

Aiming to identify entity pairs with biomedical semantic relations and assign specific relation types, biomedical relation extraction (BioRE) plays a critical role in biomedical text mining and information extraction (IE). Recent studies indicate that general large language models (LLMs) have made some breakthroughs in general relation extraction (RE) tasks. However, even the advanced open-source LLMs struggle with BioRE tasks. For example, WizardLM-70B and LLaMA-2-70B achieve F-scores of 14.05 and 12.21 on the BioRED dataset, respectively, significantly lagging behind the state-of-the-art (SOTA) method which scores 65.17. To address this gap, a multitask instruction-tuning framework is proposed, which can transform general LLMs into BioRE-specialized models with our meticulously curated instruction dataset, REInstruct, comprising 150000 diverse and quality instruction-response pairs. Consequently, we introduce REaMA, a series of open-source LLMs with sizes of 7B and 13B specifically tailored for BioRE tasks. Experimental results on seven representative BioRE datasets show that both REaMA-2-7B and REaMA-2-13B acquire promising performance on all datasets. Remarkably, the larger REaMA-2-13B outperforms the current SOTA method on five out of seven datasets. The result exhibits the effectiveness of instruction-tuning on REInstruct in eliciting strong RE capabilities in LLMs. Furthermore, we show that incorporating chain of thought (CoT) into REInstruct can further enhance the generalization ability of REaMA. The project is available at https://github.com/stzpp/REaMA.

Learning from a limited amount of data, namely Few-Shot Learning, stands out as a challenging computer vision task. Several works exploit semantics and design complicated semantic fusion mechanisms to compensate for rare representative features within restricted data. However, relying on naive semantics such as class names introduces biases due to their brevity, while acquiring extensive semantics from external knowledge takes a huge time and effort. This limitation severely constrains the potential of semantics in Few-Shot Learning. In this paper, we design an automatic way called Semantic Evolution to generate high-quality semantics. The incorporation of high-quality semantics alleviates the need for complex network structures and learning algorithms used in previous works. Hence, we employ a simple two-layer network termed Semantic Alignment Network to transform semantics and visual features into robust class prototypes with rich discriminative features for few-shot classification. The experimental results show our framework outperforms all previous methods on six benchmarks, demonstrating a simple network with high-quality semantics can beat intricate multimodal modules on few-shot classification tasks. Code is available at https://github.com/zhangdoudou123/SemFew.

The rapid growth of electric vehicles (EVs) has led to significant challenges in providing efficient and sustainable charging solutions. This paper addresses the battery swapping station (BSS) recommendation problem by proposing a novel conflict-free genetic algorithm (CFGA) integrated with a Nash equilibrium seeking (NES) approach to identify optimal Nash equilibrium (ONE) solutions to such a non-cooperative optimization problem. The CFGA employs specialized crossover and mutation operators to generate offspring that satisfy the constraints of the problem, ensuring that each EV decides a unique battery swap strategy without conflict. Firstly, an order crossover operator is proposed to preserve the order of genes in the chromosomes. Secondly, a replacement and exchange mutation operator is proposed to enhance mutation diversity. The resulting optimal solution is then used as the initial strategy for the NES, which iteratively converges to the ONE. The proposed CFGA with NES algorithm is evaluated under both small-scale and large-scale cases, demonstrating its effectiveness in achieving a balance between costs for EVs and utilization for BSSs. The study's findings have practical implications for the smart grid and EV integration, offering a robust method for optimizing EV infrastructure and operations.

Multiscenario optimization problems involve multiple scenarios to be optimized simultaneously, where each scenario corresponds to a multiobjective optimization problem with specific operating conditions. The goal is to find a group of public compromised optimal solutions (PCOSs) achieving compromised optimal in every scenario. This type of optimization problems widely exists in real-world applications, but the research on it is few. In this article, a Multiscenario Optimization Evolutionary Algorithm based on a transfer framework is proposed. In one iteration, for each scenario, its knee solutions are recognized as its transfer candidates and are used to construct a constraint hyperplane, which determines whether transfer candidates from other scenarios can be the transferable solutions in this underlying scenario. Then, among all transfer candidates, those accepted by all scenarios are identified as the PCOSs and stored in archive. Afterwards, the archive updating process is applied to guarantee the optimality of solutions in archive and control the size of archive. Finally, each scenario’s accepted transferable solutions are utilized in its offspring generation, thus achieving information transfer between different scenarios. Experimental results on a group of benchmark functions verify the superiority of the proposed design in terms of both optimality and computational efficiency over existing approaches.

In recent years, the signal diagnosis on devices operating under industrial environment has attracted increasing attention. Most data-driven signal diagnosis methods are based on a closed-set assumption that class sets of training and test data are the same. However, in industrial scenarios, during the running process of the device, the operating environment and condition may change over time, continuing generating data belonging to unknown classes with new characteristics and distribution. The unknown classes usually reflect new modes or faults of the device needed to be captured. They are unavailable in training phase, contradicting the closed-set assumption. Existing methods are inappropriate to this type of open-set classification, requiring to classify known classes and recognize unknown classes. To address this challenging problem, this article proposes a generic open-set signal classification method. First, we apply Fourier transform to convert the sensor signals from time domain to frequency domain, then data in the time and frequency domains are fused. Next, a variational encoder-classifier network is proposed to classify known classes and learn the distribution of feature space to extract robust latent features. Finally, based on extreme value theory and entropy, a pair of discriminators determine whether samples belong to unknown or not. The experimental results on two vibration-signal datasets from bearings and nuclear reactor demonstrate the effectiveness and superiority of our proposed open-set signal classification method, especially in practical applications.

Abstract Recently, many studies have been carried out on model compression to handle the high computational cost and high memory footprint brought by the implementation of deep neural networks. In this paper, model compression of convolutional neural networks is constructed as a multiobjective optimization problem with two conflicting objectives, reducing the model size and improving the performance. A novel structured pruning method called Conventional-based and Evolutionary Approaches Guided Multiobjective Pruning (CEA-MOP) is proposed to address this problem, where the power of conventional pruning methods is effectively exploited for the evolutionary process. A delicate balance in pruning rate and model accuracy has been automated achieved by a multiobjective optimization evolutionary model. First, an ensemble framework integrates pruning metrics to establish a codebook for further evolutionary operations. Then, an efficient coding method is developed to shorten the length of chromosome, thus ensuring its superior scalability. Finally, sensitivity analysis is automatically carried out to determine the upper bound of pruning rate for each layer. Notably, on CIFAR-10, CEA-MOP reduces more than 50% FLOPs on ResNet-110 and improves the relative accuracy. Moreover, on ImageNet, CEA-MOP reduces more than 50% FLOPs on ResNet-101 with negligible top-1 accuracy drop.

In this article, we focus on the vehicle routing problem (VRP) with time windows under uncertainty. To capture the uncertainty characteristics in a real-life scenario, we design a new form of disturbance on travel time and construct robust multiobjective VRP with the time window, where the perturbation range of travel time is determined by the maximum disturbance degree. Two conflicting objectives include: 1)the minimization of both the total distance and: 2)the number of vehicles. A robust multiobjective particle swarms optimization approach is developed by incorporating an advanced encoding and decoding scheme, a robustness measurement metric, as well as the local search strategy. First, through particle flying in the decision space, the problem space characteristic under deterministic environment is fully exploited to provide guidance for robust optimization. Then, a designed metric is adopted to measure the robustness of solutions and help to search for the robust optimal solutions during the particle flying process. In addition to the updating process of particle, two local search strategies, problem-based local search and route-based local search, are developed for further improving the performance of solutions. For comparison, we develop several robust optimization problems by adding disturbances on selected benchmark problems. The experimental results validate our proposed algorithm has a distinguished ability to generate enough robust solutions and ensure the optimality of these solutions.

In this article, we propose a structure-aligned generative adversarial network framework to improve zero-shot learning (ZSL) by mitigating the semantic gap, domain shift, and hubness problem. The proposed framework contains two parts, i.e., a generative adversarial network with a softmax classifier part, and a structure-aligned part. In the first part, the generative adversarial network aims at generating pseudovisual features through the guiding generator and discriminator play the minimax two-player game together. At the same time, the softmax classifier is committed to increasing the interclass distance and reducing intraclass distance. Then, the harmful effect of domain shift and hubness problems can be mitigated. In another part, we introduce a structure-aligned module where the structural consistency between visual space and semantic space is learned. By aligning the structure between visual space and semantic space, the semantic gap between them can be bridged. The performance of classification is improved when the structure-aligned visual-semantic embedding space is transferred to the unseen classes. Our framework reformulates the ZSL as a standard fully supervised classification task using the pseudovisual features of unseen classes. Extensive experiments conducted on five benchmark data sets demonstrate that the proposed framework significantly outperforms state-of-the-art methods in both conventional and generalized settings.

Catastrophic forgetting is a chronic problem during the online training process of deep neural networks. That is, once a new data set is used to train an existing neural network, the network will lose the ability to recognize the original data set. In literature, online contrastive divergence (CD) with generative replay (GR) exploits the generative capacity of the neural network to facilitate online training. It greatly alleviates catastrophic forgetting but cannot totally eliminate it. To overcome this shortcoming and further solve the challenging issue, in this article, we propose a novel approach named asynchronous dual-channel online restricted Boltzmann machine, where online CD with dual-channel GR plays an important role in further eliminating catastrophic forgetting. The asynchronous gradient estimation, by which the Markov chain sampling and the network calculation are conducted asynchronously on separate computing nodes, is designed to speed up training. The experimental results show that the proposed method outperforms several algorithms in increasing training speed and minimizing catastrophic forgetting. Besides, online learning with dual-channel can be effectively extended to other online learning neural networks with GR and has achieved excellent results in our verification experiments.

Filter pruning has been widely applied on compression of deep convolutional neural networks. Especially, the filter pruning approach using evolutionary algorithms has exhibited powerful capabilities of automatic model compression and global optimal solution search. However, the exponential increasing of search space brought by the extremely large scale of deep networks makes it difficult to achieve a well-compressed model with limited time and energy. In this paper, we design an effective method aiming at improving the efficiency in evolutionary pruning, called IEEPruner. Multiple pruning criteria are first utilized to initialize an efficient search space, and then a novel encoding scheme is proposed to further reduce the search space. After that, we formulate the search of the optimal pruned structure as a multiobjective optimization problem, and integrate a multiobjective evolutionary algorithm to achieve a good balance between model size and accuracy in an automatic manner. Thus, we automatically search for the optimal pruned model in this search space. Extensive experiments on widely used convolutional neural networks including VGGNet, ResNet, and LeNet have demonstrated the efficiency of the proposed evolutionary pruning method.

Although numerous effective and efficient multiobjective evolutionary algorithms have been developed in recent years to search for a well-converged and well-diversified Pareto optimal front, most of these designs are computationally expensive and have to maintain a large population of individuals throughout the evolutionary process. Once the Pareto optimal front is found satisfactorily, the cognitive burden is then imposed upon decision makers to handpick one solution for implementation among a massive number of candidates even with powerful multicriteria decision-making tools. With the increase in the number of decision variables and objective functions in the face of real-world applications, these problems have become a daunting challenge. In this article, we propose a recursive evolutionary algorithm, called EvoKneer, to directly search for global knee solutions, but also multiple local knee solutions using the minimum Manhattan distance approach as opposed to an enormous number of Pareto optimal solutions. Compared with the traditional evolutionary approaches, the proposed design herein only preserves nondominated solutions in rank one in each generation. Boundary Individuals Selection is tailored to select only M2 boundary individuals where M is the number of objectives. Relieving the burden of maintaining a large population size and its diversity throughout a lengthy evolutionary process, this design with a very low computational cost allows the evolutionary algorithm to converge to knee solutions quickly. To facilitate the experimental validations, a simulator with a graphical user interface is developed under the Delphi XE7 platform and made available for public use. In addition, the proposed algorithm is evaluated with the DO2DK, DEB2DK, DEB2DK2, and DEB3DK benchmark functions. The comparison results validate that the proposed EvoKneer algorithm is computationally and efficiently finding all global and local knee solutions.

As an essential component in multi- and many-objective optimization, decision-making process either selects a subset of solutions from the whole Pareto front or guides the search toward a small part of the Pareto front during the evolutionary process. In recent years, for many-objective optimization problems (MaOPs), a number of evolutionary algorithms have been developed to search for Pareto optimal solutions. However, there is a lack of research works focusing on designing decision-making approaches. In order to overcome this deficiency, we propose a novel knee-based decision-making method to search for several solutions of interest (SOIs) from a large number of solutions on the Pareto front, each of which contains the best convergence performance at least within its neighborhood and can be identified as a global or local knee solution. The optimization performance achieved by all SOIs approximates the performance of the whole Pareto front as much as possible. Furthermore, in order to relieve the difficulties in the decision-making process on MaOPs, a new visualization approach is developed based on this proposed decision-making approach. It provides information about the shape and location of the Pareto front, the possible bulge, as well as the convergence degree and distribution of solutions. The experimental results on several benchmark functions demonstrate the superiority of the proposed design in the selection of SOIs and visualization of high-dimensional objective space.

Uncertainty is an important feature abstracted from real-world applications. Multiobjective optimization problems (MOPs) with uncertainty can always be characterized as robust MOPs (RMOPs). Over recent years, multiobjective optimization evolutionary algorithms (EAs) have demonstrated the success in solving MOPs. However, most of them do not consider disturbance in the design. In order to handling the uncertainty in the optimization problem, we first give a thorough analysis of three important issues on robust optimization. Then, a novel EA called multiobjective optimization EA with robustness enhancement is developed, where the seamless integration of robustness and optimality is achieved by a proposed novel archive updating mechanism applied on the evolutionary process as well as the new robust optimal front building strategy designed to construct the final robust optimal front. Furthermore, the new designed archive updating mechanism makes the robust optimization process free of the enormous computational workload induced from sampling. The experimental results on a set of benchmark functions show the superiority of the proposed design in terms of both solutions' quality under the disturbance and computational efficiency in solving RMOPs.

Small-sample learning involves training a neural network on a small-sample data set. An expansion of the training set is a common way to improve the performance of neural networks in small-sample learning tasks. However, improper constraints in expanding training data will reduce the performance of the neural networks. In this article, we present certain conditions for incorporation of additional training data. According to these conditions, we propose a neural network framework for self-training using self-generated data called small-sample learning network (SSLN). The SSLN consists of two parts: the expression learning network and the sample recall generative network, both of which are constructed based on restricted Boltzmann machine (RBM). We show that this SSLN can converge as well as the RBM. Moreover, the experiment results on MNIST Digit, SVHN, CIFAR10, and STL-10 data sets reveal the superiority of the SSLN over other models.

Uncertainty inadvertently exists in most real-world applications. In the optimization process, uncertainty poses a very important issue and it directly affects the optimization performance. Nowadays, evolutionary algorithms (EAs) have been successfully applied to various multiobjective optimization problems (MOPs). However, current researches on EAs rarely consider uncertainty in the optimization process and existing algorithms often fail to handle the uncertainty, which have limited EAs' applications in real-world problems. When MOPs come with uncertainty, they are referred to as robust MOPs (RMOPs). In this paper, we aim at solving RMOPs using EA-based optimization search. We propose a novel robust multiobjective optimization EA (RMOEA) with two distinct, yet complement, parts: 1) multiobjective optimization finding global Pareto optimal front ignoring disturbance at first and 2) robust optimization searching for the robust optimal front afterward. Furthermore, a comprehensive performance evaluation method is proposed to quantify the performance of RMOEA in solving RMOPs. Experimental results on a group of benchmark functions demonstrate the superiority of the proposed design in terms of both solutions' quality under the disturbance and computational efficiency in solving RMOPs.

A many-objective problems (MaOP) refer to the optimization problem involving more than three objectives. Particle swarm optimization (PSO) is one of the potential heuristic methods suited for solving MaOPs. The personal best selection strategy, the global best selection strategy, and the archive maintenance strategy are the three key components in the design of a Many-Objective Particle Swarm Optimization (MaOPSO). The personal best and global best selection strategies determine the direction where particles will fly. The archive maintenance strategy has an important impact on convergence and diversity of its algorithm. In MaOPs, the high dimensionality in the objective space decreases the probability of a solution to be dominated by the other solutions in the population. Thus, it becomes more difficult for PSO to select the good leaders from so many non-dominated solutions. In this paper, a virtual Inverted Generational Distance indicator is proposed to evaluate the comprehensive quality of a solution in the external archive according to a constructed virtual Pareto front (vPF). Accordingly, a new indicator-based MaOPSO using vPF (MaOPSO/vPF) is developed to improve the convergence and diversity of the approximate Pareto front. Experimental results on the MaF test suites demonstrate that the proposed MaOPSO/vPF performs better than some selected competing Multi-objective Optimization Evolutionary Algorithms.

In many-objective optimization, visualization of population in the high-dimensional objective space provides a critical understanding of the Pareto front. First, visualization throughout the evolutionary process can be exploited in developing effective many-objective evolutionary algorithms. Furthermore, visualization is a crucial component of multi-criteria decision making. By directly observing the performance of each solution, the trade-off between objectives, and distribution of the approximate front, the decision maker can easily decide which solution should be chosen from. In this paper, we make a detailed summary for existing visualization approaches and group them into five different categories. Then, three evaluation criteria for visualization approaches are designed, according to which, five state-of-the-arts are compared under the created data sets. Experimental results show that all approaches can satisfy each criterion to some degree but no one can fully achieve all of these criteria. There is a need to develop the new approach emphasis on fully satisfy all criteria simultaneously. Then, based on the comparison results, two future research directions for visualization approach are proposed.

Selection strategy, including mating selection and environmental selection, is a key ingredient in the design of evolutionary multiobjective optimization algorithms. Existing approaches, which have shown competitive performance in low-dimensional multiobjective optimization problems with two or three objectives, often encounter considerable challenges in many-objective optimization, where the number of objectives exceeds 3. This paper first provides a comprehensive analysis on the selection strategies in the current evolutionary many-objective optimization algorithms. Afterward, we propose a coordinated selection strategy to improve the performance of evolutionary algorithms in many-objective optimization. This selection strategy considers three crucial factors: 1) the new mating selection criterion considers both the quality of each selected parent and the effectiveness of the combination of selected parents; 2) the new environmental selection criterion directly focuses on the performance of the whole population rather than single individual alone; and 3) both selection steps are complement to each other and the coordination between them in the evolutionary process can achieve a better performance than each of them used individually. Furthermore, in order to handle the curse of dimensionality in many-objective optimization problems, a new convergence measure by distance and a new diversity measure by angle are developed in both selection steps. Experimental results on both DTLZ and WFG benchmark functions demonstrate the superiority of the proposed algorithm in comparison with six state-of-the-art designs in terms of both solution quality and computational efficiency.

In a high-dimensional objective space, visualization of population in an approximate Pareto front is crucial to decision making process. By directly observing the performance of each solution, the trade-off between objectives, and distribution of approximate front, the decision maker can effectively decide which solution should be chosen from. Furthermore, visualization throughout the evolution process can also be exploited in designing effective many-objective evolutionary algorithms. Recently, a new visualization approach was developed by constructing a mapping from a high dimensional objective space into a two dimensional polar coordinate system, where a group of predefined direction vectors divide the whole space into a number of sub regions and each individual is associated with one weight vector. This method can be scalable to any dimensions, and simultaneously deal with a large number of individuals and multiple Pareto fronts for the purpose of visual comparison. It faithfully preserves shape, location, range, and distribution of Pareto front. However, distributions of Pareto front within each subregion and relations between different sub regions cannot be observed by this method. In this paper, in order to overcome this deficiency, we incorporate a modified multi-dimensional scaling (MDS) approach into this method. Experimental results show that the modified MDS is a suitable complementary to the existing method. Furthermore, the new design combined with existing visualization approach provides a comprehensive mean to visualize all important information in a many-objective optimization problem.

Evolutionary algorithms have been successfully applied for exploring both converged and diversified approximate Pareto-optimal fronts in multiobjective optimization problems, two- or three-objective in general. However, when solving problems with many objectives, nearly all algorithms perform poorly due to the loss of selection pressure in fitness evaluation. An extremely large objective space could inadvertently deteriorate the effect of an evolutionary operator. In this paper, we propose a new approach to directly handle the challenges to solve many-objective optimization problems (MaOPs). This novel design includes two stages: first, the whole population quickly approaches a small number of “target” points near the true Pareto front; then, the proposed diversity improvement strategy is applied to facilitate these individuals to spread and well distribute. As a case study, the proposed algorithm based on this design is compared with five state-of-the-art algorithms. Experimental results show that the proposed method exhibits improved performance in both convergence and diversity for solving MaOPs.

Visualization of population in a high-dimensional objective space throughout the evolution process presents an attractive feature that could be well exploited in designing many-objective evolutionary algorithms (MaOEAs). In this paper, a new visualization method is proposed. It maps individuals from a high-dimensional objective space into a 2-D polar coordinate plot while preserving Pareto dominance relationship, retaining shape and location of the Pareto front, and maintaining distribution of individuals. From it, a decision-maker can observe the evolution process, estimate location, range, and distribution of Pareto front, assess quality of the approximated front and tradeoff between objectives, and easily select preferred solutions. Furthermore, its applications can be scalable to any dimensions, handle a large number of individuals on front, and simultaneously visualize multiple fronts for comparison. Based on this visualization tool, a performance metric, named polar-metric, is designed. The convergence of the approximate front is measured by radial values of all population members on that front. Meanwhile, the diversity performance is mainly determined by niche count of each subregion in a high-dimensional objective space. Experimental results show that it can provide a comprehensive and reliable comparison among MaOEAs.

Decomposition-Based Multi-Objective Evolutionary Algorithms (DBMOEA), such as Multiple Single Objective Pareto Sampling (MSOPS) and Multiobjective Evolutionary Algorithm based on Decomposition (MOEA/D), have been successfully applied in finding Pareto-optimal fronts in Multiobjective Optimization Problems (MOPs), two or three-objective in general. DBMOEA decomposes one MOP into multiple Single-objective Optimization Problems (SOPs) where the convergence of approximated front is facilitated by finding the optimal solution of each SOP and its diversity is preserved by a group of well distributed SOPs. However, when solving problems with many objectives, one single solution can be the optimal solution of multiple SOPs which inadvertently leads to a severe loss of population diversity. In this paper, we propose a new diversity improvement method incorporated into a modified DBMOEA to directly handle this challenge. The design includes two steps. First, a few number of weight vectors guide the whole population towards a small number of solutions nearby the true Pareto front. Afterwards, initialize a subpopulation around each solution and diversify them toward well distribution. As a case study, a new algorithm based on this design is compared with three state-of-the-art DBMOEAs, MOEA/D, MSOPS, and MO-NSGA-II. Experimental results show that the proposed methods exhibit better performance in both convergence and diversity than the chosen competitors for solving many-objective optimization problems.

Evolutionary algorithms have been effectively used to solve multiobjective optimization problems with a small number of objectives, two or three in general. However, when problems with many objectives are encountered, nearly all algorithms perform poorly due to loss of selection pressure in fitness evaluation solely based upon the Pareto optimality principle. In this paper, we introduce a new fitness evaluation mechanism to continuously differentiate individuals into different degrees of optimality beyond the classification of the original Pareto dominance. The concept of fuzzy logic is adopted to define a fuzzy Pareto domination relation. As a case study, the fuzzy concept is incorporated into the designs of NSGA-II and SPEA2. Experimental results show that the proposed methods exhibit better performance in both convergence and diversity than the original ones for solving many-objective optimization problems.

Evolutionary algorithms have been successfully exploited to solve multiobjective optimization problems. In the literature, a heuristic approach is often taken. For a chosen benchmark problem with specific problem characteristics, the performance of multiobjective evolutionary algorithms (MOEAs) is evaluated via some heuristic chosen performance metrics. The conclusion is then drawn based on statistical findings given the preferable choices of performance metrics. The conclusion, if any, is often indecisive and reveals no insight pertaining to which specific problem characteristics the underlying MOEA could perform the best. In this paper, we introduce an ensemble method to compare MOEAs by combining a number of performance metrics using double elimination tournament selection. The double elimination design allows characteristically poor performance of a quality algorithm to still be able to win it all. Experimental results show that the proposed metric ensemble can provide a more comprehensive comparison among various MOEAs than what could be obtained from a single performance metric alone. The end result is a ranking order among all chosen MOEAs, but not quantifiable measures pertaining to the underlying MOEAs.

In this study, we have compared six state-of-the-art Multiobjective Evolutionary Algorithms (MOEAs) designed specifically for many-objective optimization problems under a number of carefully crafted benchmark problems. Using the performance metrics ensemble, we aim at providing a comprehensive measure and more importantly revealing insight pertaining to specific problem characteristics that the underlying MOEA could perform the best. The experimental results confirm the finding from the No Free Lunch theorem: any algorithm's elevated performance over one class of problems is exactly paid for in loss over another class. In addition, the experimental results show that the performance of MOEA to solve many-objective optimization problems depends on two distinct aspects: the ability of MOEA to tackle the specific characteristics of the problem and the ability of MOEA to handle high-dimensional objective space.

Evolutionary algorithms have been effectively used to solve multiobjective optimization problems with a small number of objectives, two or three in general. However, when encounter problems with many objectives (more than five), nearly all algorithms performs poorly because of loss of selection pressure in fitness evaluation solely based upon Pareto domination. In this paper, we introduce a new fitness evaluation mechanism to continuously differentiate solutions into different degrees of optimality beyond the classification of the original Pareto dominance. Here, the concept of fuzzy logic is adopted to define fuzzy-dominated relation. As a case study, the fuzzy concept is incorporated into the NSGA-II, instead of the original Pareto dominance principle. Experimental results show that the proposed method exhibits a better performance in both convergence and diversity than the original NSGA-II for solving many-objective optimization problems. More importantly, it enables a fast convergence process.

Evolutionary algorithms have been effectively exploited to solve multiobjective optimization problems. In literature, a heuristic approach is often taken. For a chosen benchmark problem, the performance of multiobjective evolutionary algorithms (MOEAs) is evaluated via some heuristic chosen performance metrics. The conclusion is then drawn based on statistical findings given the preferable choices of performance metrics. The conclusion, if any, is often indecisive and reveals no insight pertaining to specific problem characteristics that the underlying MOEA could perform the best. In this paper, we introduce an ensemble method to compare MOEAs by combining a number of performance metrics using double elimination tournament selection. Double elimination design allows characteristically poor performance of a quality algorithm under the special environment to still be able to win it all. Experimental results show that the proposed metrics ensemble can provide a more comprehensive comparison among various MOEAs than what could be obtained from single performance metric alone.