In the last decade, a variety of multiobjective evolutionary algorithms (MOEAs) with specific enhancements have been developed for solving multiobjective optimization problems (MOPs). In this article, unlike MOEAs, we provide a new artificial intelligence approach to solve MOPs, which adopts an imitation learning-based end-to-end method, namely AlphaMOEA. AlphaMOEA is entirely a model composed of neural networks that mainly follow the architecture of multitask learning (MTL). It has two training stages: the supervised learning (SL) stage and the reinforcement learning (RL) stage. In the SL stage, AlphaMOEA fits the solutions in the decision space provided by a number of selected MOEAs. Since neural networks in AlphaMOEA are composed of parameters with high dimensions, the fitting process can be viewed as a transformation of the solutions from a low-dimensional space into a high-dimensional space. This allows AlphaMOEA to obtain different valuable knowledge from a perspective of high dimensionality. Then, AlphaMOEA is trained in the RL stage to obtain good performance for MOPs with various problem characteristics in a self-driven manner. The RL stage relies on several designed components, including a similarity-based state design to measure the distance between solutions, an evolution operator-based action set to provide exploration behavior, and an indicator-guided reward to produce an incremental evaluation. Experimental results demonstrate that AlphaMOEA can learn valuable information about the decision space in high-dimensional representations, thereby achieving a desirable balance between exploration and exploitation. AlphaMOEA can further improve the performance for solving MOPs with various problem characteristics in a reasonable time.

Time-series forecasting is considered a critical task with extensive applications across diverse domains. However, effectively capturing both cross-dimension and cross-time dependencies in nonstationary time series remains a significant challenge, particularly due to the confounding effects of environmental factors. These factors often introduce spurious correlations that obscure the learning of meaningful temporal features. In this article, the novel framework Caformer (Causal Transformer) is proposed for time-series forecasting grounded in causal reasoning, the science of identifying causality. The framework consists of four key modules: the dynamic learner, environment learner, temporal learner, and decompose learner. The dynamic learner uncovers dynamic interactions among features to model cross-dimension dependencies, while the temporal learner infers cross-time dependencies under causal constraints. The environment learner, together with the decompose learner, extracts environmental factors and applies a backdoor adjustment to mitigate the confounding effects on the time series. Extensive experiments demonstrate that Caformer achieves the state-of-the-art performance in both long-term and short-term forecasting. It achieves up to a 26.2% mean square error (mse) reduction on the traffic dataset and 21.8% on the electricity dataset compared to PatchTST, with consistent gains across eight long-term benchmarks. On the M4 dataset, Caformer ranks first across all 15 short-term forecasting categories. In addition to strong predictive accuracy, Caformer provides interpretable insights into the learned dependencies.

Code completion is a context-aware code completion function available in certain programming environments. It can accelerate the coding process of applications, enhance development efficiency, and reduce costs. Compared with the code generation, code completion has advantages such as flexible adaptation to changes, high precision and pertinence, and lower performance overhead. Intelligent code completion uses artificial intelligence technology to achieve code completion more effectively. However, it faces issues such as poor interpretability, unstable performance, and high demand for computing resources. To make full use of the powerful semantic understanding, information retrieval, and text generation capabilities of Large Language Models (LLMs), as well as the strong global optimization and adaptive search capabilities of Evolutionary Computation (EC), an integrated network framework, the Evolutionary Dual-Channel Network (EDN), is proposed. EDN leverages the precise context comprehension and feature extraction capabilities of large language models to predict the nodes of the abstract syntax tree, thereby achieving code completion. To enhance the performance and interpretability of the entire framework, we use the genetic algorithm (GA), with its powerful search ability, as the main optimization method for EDN. This proposed framework combining LLMs and EC in a novel way. By using EC to optimize certain components in the network framework of LLMs, it enables a more flexible integration of LLMs and EC. Experiment results on three public corpora consisting of multiple programming languages demonstrate the effectiveness of the proposed method.

Existing constrained multiobjective evolutionary algorithms (CMOEAs) frequently employ the information provided by the unconstrained Pareto front (UPF) to facilitate the identification of the constrained Pareto front (CPF) for constrained multiobjective optimization problems (CMOPs). However, obtaining a UPF with favorable convergence and diversity is not straightforward, and the obtained UPF is sometimes difficult to effectively assist in the identification of CPF for certain complex CMOPs. To this end, a novel algorithm called DREMCO is proposed, which endeavors to obtain a good UPF and is capable of utilizing the obtained UPF to consistently assist in the identification of CPF. DREMCO consists of a main population for the original problem and a two-phase (propulsion phase and recovery phase) auxiliary population with a dynamics regulation mechanism. In the propulsion phase, the auxiliary population ignores constraints and employs an improved aggregation function to obtain a good UPF, thereby pulling the main population across infeasible regions. In the recovery phase, the auxiliary population uses a penalty function method to converge to CPF and continuously refines the CPF of the main population. Concurrently, a novel phase judgment method is proposed for seamless transition between phases. Furthermore, an information-sharing strategy is proposed, which is capable of sharing information of the parents and offspring in offspring generation and environment selection, respectively. The experimental results with 11 state-of-the-art CMOEAs on five benchmark suites and eight real-world CMOPs demonstrate the efficacy of the proposed algorithm.

In practical applications, dynamic multi-objective optimization problems inevitably face large-scale scenarios. Massive search space and high-dimensional historical data pose some serious challenges to existing prediction-based change response mechanisms, which affects their ability in maintaining population diversity and accurately predicting convergent solutions. In this paper, we propose a generative model-driven approach that trains a generative model for a new environment only through historical Pareto optimal sets. It can generate both convergent and diverse population for the new environment. We show that by introducing a new training scheme and loss function for adversarial autoencoder, the training of the generative model can achieve stable convergence under high-dimensional coupled data conditions. In addition, the trained generative model can maintain the diversity of generative candidate solutions by smooth sampling in the latent space. Extensive experiments were conducted on a typical dynamic multi-objective testing suite with problem settings ranging from 30 to 600 dimensions. These experimental results demonstrate that the optimization performance of the proposed approach outperforms those of the existing state-of-the-art designs.

Effectively managing convergence, diversity, and feasibility constitutes a fundamental trinity of tasks in optimizing constrained multiobjective optimization problems (CMOPs). Nevertheless, contemporary constrained multiobjective evolutionary algorithms (CMOEAs) frequently encounter challenges in reconciling these imperatives simultaneously. Drawing inspiration from overwhelming success in artificial intelligence, we propose a deep reinforcement learning-guided coevolutionary algorithm (DRLCEA) to tackle this predicament. DRLCEA employs two populations to optimize the original and unconstrained versions of the CMOP, respectively and then fosters cooperation between them according to the guidance of DRL. The established DRL employs two evaluation metrics to appraise population convergence, diversity, and feasibility, thus remarkably proficient in reflecting and steering the coevolution . Therefore, the proposed DRLCEA could effectively locate the feasible regions and approximate the constrained Pareto front. We assess the proposed algorithm on 32 benchmark CMOPs and one real-world UAV emergency track planning (UETP) application. Experimental results undoubtedly demonstrate the superiority and robustness of the proposed DRLCEA.

Enhancing energy efficiency is pivotal for advancing robotic lower limb prostheses. However, the conventional control strategy in powered prostheses displays notable variance in energy economy across terrains during walking. This challenge could be managed by developing control optimization methods to effectively reduce the subject’s metabolic cost and the device’s energy expenditure, thereby improving the walking economy in diverse environments. In this study, we investigate a multi-objective optimization problem (MOP) for controlling above-knee prostheses, intending to reduce prosthetic power consumption and the subject’s metabolic costs during walking activities. To bolster real-time performance in the optimization loop, we introduce a knee-guided evolutionary algorithm (KGEA) for MOP, which efficiently reduces exploration space and time complexity, thus enabling the practical implementation of a limited number of solutions in each iteration. The effectiveness of the proposed optimization-based strategy was evaluated through experiments involving two lower limb amputees engaging in slope, stair, and flat walking. Our results demonstrated that the optimization-based strategy significantly reduced the subjects’ metabolic consumption, with an average reduction of 16.53% for subject 1 and 10.23% for subject 2 across different terrains, compared to using the prosthesis without the optimization strategy. This promising outcome marks a substantial advancement in the development of energy-efficient, multi-degree-of-freedom power prostheses.Note to Practitioners—Lower Limb amputees using powered prostheses experience energy efficiency challenges due to limited battery life, particularly with multi-degree-of-freedom prostheses over varied terrains. Additionally, compared to passive alternatives, the increased weight of powered prosthetics poses a challenge in managing users’ metabolic costs, requiring suitable optimization strategies to offset the additional burden and optimize body economy during locomotion. The proposed KGEA-based multi-objective optimization method, designed explicitly for above-knee prostheses, targets the reduction of prosthetic power consumption and subject’s metabolic costs during walking activities. This innovation will likely benefit individuals with lower limb amputations, offering them improved mobility and comfort across various terrains. By addressing variance in varied terrains and inefficiencies in energy consumption caused by traditional control strategies, this work can assist practitioners in creating more effective and user-friendly prosthetic devices. Furthermore, we explore continuous refinement of the multi-objective optimization model and KGEA to accommodate a broader range of user-profiles and walking scenarios, ensuring its applicability in real-world situations.

This paper proposes a novel navigation and optimization approach for mobile robots operating in intelligent manufacturing (IM) scenarios with unknown obstacles. First, we introduce a set of evaluation functions that simultaneously consider multiple metrics, including speed, security, sampling step, and obstacle avoidance ability, for scenarios with unknown dynamic obstacles. We then adopt a knee-guided multi-objective evolutionary algorithm capable of balancing parameter size and performance to optimize these conflicting objectives. Finally, we present a local navigation framework that integrates navigation and multi-objective optimization to enable obstacle avoidance in unstructured environments. The proposed algorithm is validated through simulations and practical scenarios, demonstrating its effectiveness in both static and dynamic scenarios.Note to Practitioners—The Human-cyber-physical System (HCPS) is the basic principle of the new generation of intelligent manufacturing (IM). It presents new requirements for the production method, and robot operate completely autonomously is expected to be the future solution for some complex tasks. However, traditional navigation methods often fail in these scenarios due to environmental changes and restricted workspaces. To address this challenge, this paper proposes a multi-objective optimization-based local navigation approach (MONA) that transforms the local navigation problem into a multi-objective optimization problem. The approach uses the knee-guided multi-objective evolutionary algorithm to optimize the control input for the mobile robot, ensuring safe and fast navigation in complex environments. The proposed approach has been validated through simulation and real scenario experiments, demonstrating its effectiveness.

Bilevel optimization problems (BLOPs) are characterized by an interactive hierarchical structure, where the upper level seeks to optimize its strategy while simultaneously considering the response of the lower level. Evolutionary algorithms are commonly used to solve complex bilevel problems in practical scenarios, but they face significant resource consumption challenges due to the nested structure imposed by the implicit lower-level optimality condition. This challenge becomes even more pronounced as problem dimensions increase. Although recent methods have enhanced bilevel convergence through task-level knowledge sharing, further efficiency improvements are still hindered by redundant lower-level iterations that consume excessive resources while generating unpromising solutions. To overcome this challenge, this article proposes an efficient dynamic resource allocation framework for evolutionary bilevel optimization, named DRC-BLEA. Compared to existing approaches, DRC-BLEA introduces a novel competitive quasi-parallel paradigm, in which multiple lower-level optimization tasks, derived from different upper-level individuals, compete for resources. A continuously updated selection probability is used to prioritize execution opportunities to promising tasks. Additionally, a cooperation mechanism is integrated within the competitive framework to further enhance efficiency and prevent premature convergence. Experimental results compared with chosen state-of-the-art algorithms demonstrate the effectiveness of the proposed method. Specifically, DRC-BLEA achieves competitive accuracy across diverse problem sets and real-world scenarios, while significantly reducing the number of function evaluations and overall running time.

Dynamic multiobjective optimization problems (DMOPs) are often scaled to large-scale scenarios in real-world applications, which inevitably must face the triple challenges of massive search space, dynamic environmental changes and multiobjective conflicts simultaneously. This article proposes an adversarial autoencoder-based large-scale dynamic multiobjective evolutionary framework. It integrates deep generative modeling techniques and large-scale multiobjective evolutionary algorithms (LMOEAs) to solve large-scale DMOPs effectively and efficiently. Specifically, an adversarial autoencoder-based deep generative network training architecture is proposed for high-dimensional decision variables in large-scale DMOPs. It can transfer a generative model trained on Pareto-optimal solutions in the current environment to a new environment using only the auxiliary information exhibited through the movement trajectories of historical Pareto-optimal solutions, resulting in the generation of quality initial populations for the new environment. Meanwhile, any proven LMOEA can be integrated into the proposed framework without extensive modifications. Experimental results on a typical dynamic multiobjective test suite with problem settings from 30 to 1000 dimensions demonstrate that the optimization performance of the proposed framework outperforms existing state-of-the-art designs. Especially in large-scale scenarios, the proposed framework is considered superior in terms of solution quality and computational efficiency.

Neural architecture search (NAS) is widely used to automate the design of high-accuracy deep architectures, which are often vulnerable to adversarial attacks in practice due to the lack of adversarial robustness. Existing methods focus on the direct utilization of regularized optimization process to address this critical issue, which causes the lack of interpretability for the end users to learn how the robust architecture is constructed. In this paper, we introduce a robust enhanced plugin (REP) method for differentiable NAS to search for robust neural architectures. Different from existing peer methods, REP focuses on the robust search primitives in the search space of NAS methods, and naturally has the merit of contributing to understanding how the robust architectures are progressively constructed. Specifically, we first propose an effective sampling strategy to sample robust search primitives in the search space. In addition, we also propose a probabilistic enhancement method to guarantee natural accuracy and adversarial robustness simultaneously during the search process. We conduct experiments on both convolutional neural networks and graph neural networks with widely used benchmarks against state of the arts. The results reveal that REP can achieve superiority in terms of both the adversarial robustness to popular adversarial attacks and the natural accuracy of original data. REP is flexible and can be easily used by any existing differentiable NAS methods to enhance their robustness without much additional effort.

Existing domain adaptation semantic segmentation (DASS) methods under adverse conditions often depend on pseudo-labels for network training. However, these pseudo-labels are frequently plagued by noise and bias toward high-confidence predictions, thereby impeding the enhancement of segmentation performance. This article tackles the above challenge by proposing a novel boundary-based active domain adaptation (ADA) framework, which efficiently selects both informative low-confidence samples and high-confident but misclassified samples to be labeled while maximizing the segmentation performance under a limited annotation budget. For the evaluation of sample confidence and informativeness, we first propose ranking weighted feature space impurity (RWFSI) metric to quantify category distribution among a sample's nearest neighbors within the feature space and consider the samples with higher RWFSI values as low-confidence samples around the decision boundary, which can also alleviate the category imbalance of active labels. Subsequently, we apply Gaussian mixture models (GMMs) to model the distribution across source and target domains. Using the spatial arrangement of each GMM component, we define the intraclass domain shift score (ICDSS), which identifies samples with high ICDSS values as those more likely to be high-confidence but misclassified, aiding in refining sample selection. Extensive experiments demonstrate that our method is superior to the existing state-of-the-art domain adaptation and active learning (AL) methods and comparable with those of full supervision. The code will be released at https://github.com/1061018609/BADA.

Recently, large language models (LLMs) have showcased remarkable capabilities across a diverse range of applications, including general natural language processing (NLP) and domain-specific tasks. Empirical evidence indicates that LLMs have matched or even surpassed human performance in various areas, such as language translation, reading comprehension, and logical reasoning. However, preliminary research reveals that LLMs struggle with basic character and word editing, which is crucial for practical tasks such as creating 1000-word articles or modifying specific text information. To comprehensively assess the capabilities of LLMs in character and word understanding and manipulation (CWUM), we introduce the CWUM benchmark in Chinese and English. CWUM comprises 23 tasks focusing on text editions, including counting, identification, insertion, and reversal. A comprehensive evaluation of nine advanced LLMs on CWUM is conducted, which highlights significant failures of existing LLMs on CWUM tasks that humans can solve perfectly with 100% accuracy. Meanwhile, specific deficiencies of LLMs in basic language understanding and manipulation are revealed by performing quality and quantity analysis. Furthermore, in the experiment part, various methods are investigated to improve model performance, demonstrating the effectiveness of supervised fine-tuning (SFT) in enhancing model performance on CWUM while maintaining generalization abilities on unseen tasks.

Aiming to identify entity pairs with biomedical semantic relations and assign specific relation types, biomedical relation extraction (BioRE) plays a critical role in biomedical text mining and information extraction (IE). Recent studies indicate that general large language models (LLMs) have made some breakthroughs in general relation extraction (RE) tasks. However, even the advanced open-source LLMs struggle with BioRE tasks. For example, WizardLM-70B and LLaMA-2-70B achieve F-scores of 14.05 and 12.21 on the BioRED dataset, respectively, significantly lagging behind the state-of-the-art (SOTA) method which scores 65.17. To address this gap, a multitask instruction-tuning framework is proposed, which can transform general LLMs into BioRE-specialized models with our meticulously curated instruction dataset, REInstruct, comprising 150000 diverse and quality instruction-response pairs. Consequently, we introduce REaMA, a series of open-source LLMs with sizes of 7B and 13B specifically tailored for BioRE tasks. Experimental results on seven representative BioRE datasets show that both REaMA-2-7B and REaMA-2-13B acquire promising performance on all datasets. Remarkably, the larger REaMA-2-13B outperforms the current SOTA method on five out of seven datasets. The result exhibits the effectiveness of instruction-tuning on REInstruct in eliciting strong RE capabilities in LLMs. Furthermore, we show that incorporating chain of thought (CoT) into REInstruct can further enhance the generalization ability of REaMA. The project is available at https://github.com/stzpp/REaMA.

Symbolic regression is commonly considered in wide-ranging applications due to its inherent capability for learning both structure and weighting parameters of an interpretable model. However, for those scenarios that require fitting multiple expressions (MEs) synchronously, existing symbolic regression algorithms need to run multiple times step by step asynchronously for fitting such a group of expressions. Due to lacking mechanisms to explicitly capture and leverage the relationships between these expressions, the coupling information among MEs will be lost in this approach. A multiexpression symbolic regression algorithm (ME-SR) is proposed in this article to address the issue in learning MEs. Additionally, a methodology for extracting the approximate maximum common subexpression (aMCSE) among these MEs is suggested to disclose the relationships. A new metric is formulated to measure the quality of an aMCSE in ME-SR by imitating the concept of intersection over union. Furthermore, an adaptive cross matrix is incorporated into the algorithm to balance the search efforts between intertask and intratask domains. The proposed ME-SR demonstrates superior performance when compared to its counterparts of single expression symbolic regression on the designed test set. Finally, the efficacy of the method is well verified by a real-world circuit design case.

The large-scale multiobjective optimization problem (LSMOP) is characterized by simultaneously optimizing multiple conflicting objectives and involving hundreds of decision variables. Many real-world applications in engineering can be modeled as LSMOPs; simultaneously, engineering applications require insensitivity in performance. This requirement typically means that the algorithm should not only produce good results in terms of performance for every run but also the performance of multiple runs should not fluctuate too much. However, existing large-scale multiobjective optimization algorithms often focus on improving algorithm performance, but pay little attention to improving the insensitivity characteristic of algorithms. This directly leads to substantial limitations when solving practical problems. In this work, we propose an evolutionary algorithm called large-scale multiobjective optimization algorithm via Monte Carlo tree search, which is based on the Monte Carlo tree search and aims to improve the performance and insensitivity of solving LSMOPs. The proposed method samples decision variables to construct new nodes on the Monte Carlo tree for optimization and evaluation, and it selects nodes with good evaluations for further searches in order to reduce the performance sensitivity caused by large-scale decision variables. We propose two metrics to measure the sensitivity of the algorithm and compare the proposed algorithm with several state-of-the-art designs on different benchmark functions and metrics. The experimental results confirm the effectiveness and performance insensitivity of the proposed design for solving LSMOPs.

Synthetic aperture radar (SAR) is capable of obtaining the high-resolution 2-D image of the interested target scene, which enables advanced remote sensing and military applications, such as missile terminal guidance. In this article, the terminal trajectory planning for SAR imaging guidance is first investigated. It is found that the guidance performance of an attack platform is determined by the adopted terminal trajectory. Therefore, the aim of the terminal trajectory planning is to generate a set of feasible flight paths to guide the attack platform toward the target and meanwhile obtain the optimized SAR imaging performance for enhanced guidance precision. The trajectory planning is then modeled as a constrained multiobjective optimization problem given a high-dimensional search space, where the trajectory control and SAR imaging performance are comprehensively considered. By utilizing the temporal-order-dependent property of the trajectory planning problem, a chronological iterative search framework (CISF) is proposed. The problem is decomposed into a series of subproblems, where the search space, objective functions, and constraints are reformulated in chronological order. The difficulty of solving the trajectory planning problem is thus significantly alleviated. Then, the search strategy of CISF is devised to solve the subproblems successively. The optimization results of the preceding subproblem can be utilized as the initial input of the subsequent subproblems to enhance the convergence and search performance. Finally, a trajectory planning method is put forward based on CISF. Experimental studies demonstrate the effectiveness and superiority of the proposed CISF compared with the state-of-the-art multiobjective evolutionary methods. The proposed trajectory planning method can generate a set of feasible terminal trajectories with optimized mission performance.

Dynamic constrained multiobjective optimization involves irregular changes in the distribution of the true Pareto-optimal fronts, drastic changes in the feasible region caused by constraints, and the movement directions and magnitudes of the optimal distance variables due to diverse changing environments. To solve these problems, we propose a multi-population evolution based dynamic constrained multiobjective optimization algorithm. In this algorithm, we design a tribe classification operator to divide the population into different tribes according to a feasibility check and the objective values, which is beneficial for driving the population toward the feasible region and Pareto-optimal fronts. Meanwhile, a population selection strategy is proposed to identify promising solutions from tribes and exploit them to update the population. The optimal values of the distance variables vary differently with dynamic environments, thus, we design a dynamic response strategy for solutions in different tribes that estimates their distances to approach the Pareto-optimal fronts and regenerates a promising population when detecting environmental changes. In addition, a scalable generator is designed to simulate diverse movement directions and magnitudes of the optimal distance variables in real-world problems under dynamic environments, obtaining a set of improved test problems. Experimental results show the effectiveness of test problems, and the proposed algorithm is impressively competitive with several chosen state-of-the-art competitors.

Multimodal multiobjective optimization problems (MMOPs) are commonly seen in real-world applications. Many evolutionary algorithms have been proposed to solve continuous MMOPs. However, little effort has been made to solve combinatorial (or discrete) MMOPs. Searching for equivalent Pareto-optimal solutions in the discrete decision space is challenging. Moreover, the true Pareto-optimal solutions of a combinatorial MMOP are usually difficult to know, which has limited the development of its optimizer. In this article, we first propose a test problem generator for multimodal multiobjective traveling salesman problems (MMTSPs). It can readily generate MMTSPs with known Pareto-optimal solutions. Then, we propose a novel evolutionary algorithm to solve MMTSPs. In our proposed algorithm, we develop two new edge assembly crossover operators, which are specialized in searching for superior solutions to MMTSPs. Moreover, the proposed algorithm uses a new environmental selection operator to maintain a good balance between the objective space diversity and decision space diversity. We compare our algorithm with five state-of-the-art designs. Experimental results convincingly show that our algorithm is powerful in solving MMTSPs.

Deep neural networks (DNNs) have shown remarkable performance in solving a wide variety of real-world problems, ranging from image recognition to natural language processing and self-driving vehicles. In principle, the achievements of DNNs are mainly contributed by their deep architectures, which can learn meaningful representations at different levels. This can greatly enhance the performance of the subsequent machine-learning algorithms. However, manually designing an optimal deep architecture for a particular problem requires a rich knowledge of both the investigated problem domain and the DNNs, which is not necessarily held by every end user interested in this area.

Multimodal multiobjective optimization problems (MMOPs) possess multiple Pareto optimal sets (PSs) corresponding to the identical Pareto optimal front (PF). To handle MMOPs, we propose a bi-objective evolutionary algorithm (BOEA), which transforms an MMOP into a bi-objective optimization problem. This problem is constructed by the penalty boundary intersection technique and a diversity indicator to obtain multiple PSs. The first objective reflects the population convergence and factors in the population diversity in the objective space, while the other objective concentrates more on the population diversity in the decision space. Furthermore, an environmental selection strategy is designed to choose the offspring solutions, which consists of nondominated sorting based on the transformed optimization problem and hierarchical clustering for selecting promising solutions. Experiments on 34 MMOPs demonstrate that BOEA performs better than selected state-of-the-art representatives, including 22 MMOPs from CEC2019 and 12 imbalanced MMOPs. In addition, the effectiveness of BOEA is further validated by six feature selection problems in real-world applications.

Usually, data-driven multiobjective optimization problems (DD-MOPs) are indirectly solved by evolutionary algorithms through the built surrogate model which is well-trained from sample data. However, in most DD-MOPs, only a few available data can be practicably collected from real engineering experiments due to the unaffordable cost and time. The key challenge in such a DD-MOP is to prevent the serious deterioration on the accuracy of the obtained approximate Pareto front. In this article, two novel strategies, critical fitness for evolutionary algorithms and data augmentation for a surrogate model, are complementarily imposed by a generative adversarial network (GAN) to tackle with the challenges in DD-MOPs. In the critical fitness strategy, a new critical fitness, composed of the critical score from the discriminator of GAN and the prediction value of the surrogate model, is proposed to improve the accuracy of the approximate Pareto front of a DD-MOP. In the data augmentation strategy, some new samples are synthetized by the generator of GAN to build a better-trained surrogate model. As a result, the GAN concurrently serves the critical fitness strategy and the data augmentation strategy as the roles of “killing two birds with one stone.” The performance of the proposed algorithm for DD-MOPs was well-verified over 26 benchmark problems and successfully applied to discover new NdFeB materials.

For the past many years, several constrained multiobjective evolutionary algorithms (CMOEAs) have been designed for solving constrained multi-objective optimization problems (CMOPs). In these CMOEAs, some constraint-handling techniques (CHTs) were proposed to balance the convergence and constrained satisfaction, however, they still face some serious challenges, such as premature convergence to the local optimal region and labor-intensive tuning of parameters for a specific CMOP. Furthermore, most of the existing CHTs are derived by solving constrained single-objective optimization. The information hidden from the feasible non-dominated set (FNDS) has not been fully utilized. This study proposed a novel parameter-less constraint handling technique, which divides the entire population into three mutually exclusive subsets dynamically: FNDS, the subset dominated by FNDS, and the subset not dominated by FNDS. According to the proposed division of labor, it is not necessary to balance the convergence and constrained satisfaction in each subset. To avoid being entrapped in local optima, the proposed algorithm adopts a two-stage strategy to solve CMOPs. In the first stage, the proposed algorithm focuses solely on converging toward the unconstrained Pareto front without considering the constrained satisfaction. In the second stage, the FNDS constraint handling technique is adopted to guide the population converging toward constrained Pareto front effectively. The performance of the proposed algorithm was compared to that of nine state-of-the-art CMOEAs, and the comparison results show that the proposed algorithm performs significantly better on the CF, MW, and LIRCMOP test suites.

Generative Adversarial Networks (GANs) have achieved remarkable success in image translation tasks. However, its prohibitive computational overhead has been a major hurdle for deployment on resource constrained platforms. Due to the training instability and the complicated network architecture, existing model acceleration techniques cannot appropriately handle the GAN compression problem. To cope with these difficulties, we propose a multi-objective evolutionary algorithm to compress GAN models in the context of image translation tasks, which is termed as MEGC. Particularly, the conflict between the computational cost of the GAN model and the quality of generated image is explicitly modeled as a two-objective optimization problem, and the evolved Pareto set is utilized to guide the sampling process during supernet training, which can in turn divert the focus of the supernet training to well-performing compact subnets. Besides, an evaluation-free strategy is introduced to facilitate exploration in the search space while incurring no extra computational cost. Based on the above design, the proposed MEGC eliminates the requirement of subnet searching in the post-processing procedure. Experiments on image translation tasks under paired and unparied settings demonstrate the effectiveness of the proposed MEGC on reducing the computational cost of GANs while improving the quality of generated images compared to those of the full models.

Neural architecture search (NAS) has recently gained extensive interest in the deep learning community because of its great potential in automating the construction process of deep models. Among a variety of NAS approaches, evolutionary computation (EC) plays a pivotal role with its merit of gradient-free search ability. However, a massive number of the current EC-based NAS approaches evolve neural architectures in an absolutely discrete manner, which makes it tough to flexibly handle the number of filters for each layer, since they often reduce it to a limit set rather than searching for all possible values. Moreover, EC-based NAS methods are often criticized for their inefficiency in performance evaluation, which usually requires laborious full training for hundreds of candidate architectures generated. To address the inflexible search issue on the number of filters, this work proposes a split-level particle swarm optimization (PSO) approach. Each dimension of the particle is subdivided into an integer part and a fractional part, encoding the configurations of the corresponding layer, and the number of filters within a large range, respectively. In addition, the evaluation time is greatly saved by a novel elite weight inheritance method based on an online updating weight pool, and a customized fitness function considering multiple objectives is developed to well control the complexity of the searched candidate architectures. The proposed method, termed split-level evolutionary NAS (SLE-NAS), is computationally efficient, and outperforms many state-of-the-art peer competitors at much lower complexity across three popular image classification benchmark datasets.

Early time series classification predicts the class label of a given time series before it is completely observed. In time-critical applications, such as arrhythmia monitoring in ICU, early treatment contributes to the patient's fast recovery, and early warning could even save lives. Hence, in these cases, it is worthy of trading, to some extent, classification accuracy in favor of earlier decisions when the time series data are collected over time. In this article, we propose a novel deep reinforcement learning-based framework, snippet policy network V2 (SPN-V2), for long and varied-length multi-lead electrocardiogram (ECG) early classification. The proposed SNP-V2 contains two main components: snippet representation learning (SRL) and early classification timing learning (ECTL). The SRL is proposed to encode inner-snippet spatial correlations and inter-snippet temporal correlations into the hidden representations of the subsegment (snippet) of the input ECG. ECTL aims to learn a decision agent to classify the time series early and accurately. To optimize the proposed framework, we design a novel knee-guided neuroevolution algorithm (KGNA) to solve cardiovascular diseases' early classification problem, automatically optimizing the proposed SPN-V2 regarding the tradeoff between accuracy and earliness. In addition, we conduct a series of experiments on two real-world ECG datasets. The experimental results show the superiority of the proposed algorithm over the state-of-the-art competing methods.

Monocular depth estimation is one of the fundamental tasks in environmental perception and has achieved tremendous progress by virtue of deep learning. However, the performance of trained models tends to degrade or deteriorate when employed on other new datasets due to the gap between different datasets. Though some methods utilize domain adaptation technologies to jointly train different domains and narrow the gap between them, the trained models cannot generalize to new domains that are not involved in training. To boost the transferability of self-supervised monocular depth estimation models and mitigate the issue of meta-overfitting, we train the model in the pipeline of meta-learning and propose an adversarial depth estimation task. We adopt model-agnostic meta-learning (MAML) to obtain universal initial parameters for further adaptation and train the network in an adversarial manner to extract domain-invariant representations for easing meta-overfitting. In addition, we propose a constraint to impose upon cross-task depth consistency to compel the depth estimation to be identical in different adversarial tasks, which improves the performance of our method and smoothens the training process. Experiments on four new datasets demonstrate that our method adapts quite fast to new domains. Our method trained after 0.5 epoch achieves comparable results with the state-of-the-art methods trained at least 20 epochs.

An expensive multimodal optimization problem (EMMOP) is that the computation of the objective function is time consuming and it has multiple global optima. This article proposes a decomposition differential evolution (DE) based on the radial basis function (RBF) for EMMOPs, called D/REM. It mainly consists of two phases: the promising subregions detection (PSD) and the local search phase (LSP). In PSD, a population update strategy is designed and the mean-shift clustering is employed to predict the promising subregions of EMMOP. In LSP, a local RBF surrogate model is constructed for each promising subregion and each local RBF surrogate model tracks a global optimum of EMMOP. In this way, an EMMOP is decomposed into many expensive global optimization subproblems. To handle these subproblems, a popular DE variant, JADE, acts as the search engine to deal with these subproblems. A large number of numerical experiments unambiguously validate that D/REM can solve EMMOPs effectively and efficiently.

Highly constrained multiobjective optimization problems (HCMOPs) refer to constrained multiobjective optimization problems (CMOPs) with complex constraints and small feasible regions, which are commonly encountered in many real-world applications. Current constraint-handling techniques will face two difficulties when dealing with HCMOPs: 1) feasible solution is hard to be found and too much search effort is spent in locating the feasible region and 2) since the total feasible region of an HCMOP can consist of several disconnected subregions, the search process might be stuck in the comparatively larger feasible subregion, which does not contain the whole Pareto front (PF). To address these two issues, an evolutionary algorithm with constraint relaxation strategy based on differential evolution algorithm, that is, CRS-DE, is proposed in this article. In each generation, the CRS-DE relaxes the constraints by dividing the infeasible solutions into two subpopulations based on total constraint violation, that is, the "semifeasible" subpopulation (SF) and "infeasible" subpopulation (IF), respectively. The SF provides information on the promising regions of finding the feasible solution and is the driving force for convergence toward the PF, while the IF focuses on global exploration for new promising regions. Corresponding reproduction and selection strategies are devised for the SF, IF, and feasible subpopulations, which create a clear division of labor with cooperation to facilitate the search for feasible solutions. To leverage the influence of CRS and prevent the population from premature convergence, a mobility restriction mechanism is developed to restrict the individuals in the SF and IF from entering the feasible subpopulation and enhance the diversity of the whole population. Comprehensive experiments on a series of benchmark test problems and a real-world CMOP demonstrate the competitiveness of our method compared with other representative algorithms in terms of effectiveness and reliability in finding a set of well-distributed optimal solutions for HCMOPs.

In our earlier study, an energy-efficient passive UAV radar imaging system was formulated, which comprehensively analyzed the system performance. In this article, based on the evaluator set, a mission planning framework for the underlying energy-efficient passive UAV radar imaging system is proposed to achieve optimized mission performance for a given remote sensing task. First, the mission planning problem is defined in the context of the proposed synthetic aperture radar (SAR) system and a general framework is outlined, including mission specification, illuminator selection, and path planning. It is found that the performance of the system is highly dependent upon the flight path adopted by the UAV platform in a 3-D terrain environment, which offers the potential of optimizing the mission performance by adjusting the UAV path. Then, the path planning problem is modeled as a single-objective optimization problem with multiple constraints. Path planning can be divided into two substages based on different mission orientations and low mutual correlation. Based on this property, a path planning method, called substage division collaborative search (Sub-DiCoS), is proposed. The problem is divided into two subproblems with the corresponding decision space and subpopulation, which significantly relax the constraints for each subproblem and facilitates the search for feasible solutions. Then, differential evolution and the whole-stage best guidance technique are devised to cooperatively lead the subpopulations to search for the best solution. Finally, simulations are presented to demonstrate the effectiveness of the proposed Sub-DiCoS method. The result of the mission planning method can be used to guide the UAV platform to safely travel through a 3-D rough terrain in an energy-efficient manner and achieve optimized SAR imaging and communication performance during the flight.

In recent years, numerous efficient many-objective optimization evolutionary algorithms have been proposed to find well-converged and well-distributed nondominated optimal solutions. However, their scalability performance may deteriorate drastically to solve large-scale many-objective optimization problems (LSMaOPs). Encountering high-dimensional solution space with more than 100 decision variables, some of them may lose diversity and trap into local optima, while others may achieve poor convergence performance. This article proposes a multipopulation-based differential evolution algorithm, called LSMaODE, which can solve LSMaOPs efficiently and effectively. In order to exploit and explore the exponential decision space, the proposed algorithm divides the population into two groups of subpopulations, which are optimized with different strategies. First, the randomized coordinate descent technique is applied to 10% of individuals to exploit the decision variables independently. This subpopulation maintains diversity in the decision space to avoid premature convergence into local optimum. Second, the remaining 90% of individuals are optimized with the nondominated guided random interpolation strategy, which interpolates individual among three nondominated solutions randomly. The strategy can guide the population convergent toward the nondominated solutions quickly, meanwhile, maintain good distribution in objective space. Finally, the proposed LSMaODE is evaluated on the LSMOP test suites from the scalability in both decision and objective dimensions. The performance is compared against five state-of-the-art large-scale many-objective evolutionary algorithms. The experimental results show that LSMaODE provides highly competitive performance.

Problem decomposition is crucial for coping with large-scale global optimization problems, which relies heavily on highly precise variable grouping methods. The state-of-the-art decomposition methods identify separability based on the finite differences principle, which is valid only for additively separable functions but not applicable to non-additively separable functions. Therefore, we need to investigate separability in more depth in order to propose a more general principle and design more universal decomposition methods. In this paper, we conduct a comprehensive theoretical investigation on separability, the core of which is proposing an innovative separability identification principle: the minimum points shift principle. By utilizing the new principle, we develop a general separability grouping (GSG) method that can handle both additively and non-additively separable functions with high accuracy. In addition, we design a new set of benchmark functions based on non-additive separability, which compensates for the lack of non-additively separable functions in the previous test suites. Extensive experiments demonstrate that the proposed GSG achieves high grouping accuracy on both new and CEC series benchmark problems, especially on non-additively separable problems Finally, we verify that the proposed GSG can effectively improve the optimization performance of non-additively separable problems through optimization experiments.

Convolutional neural networks (CNNs) are a superb computing paradigm in deep learning, and their architectures are considered to be the key to performance breakthroughs in various tasks. Recently, neural architecture search (NAS) methods have been proposed to automate the process of network architecture design, many of which have discovered novel CNN architectures that are superior to human-designed ones. However, most of the current NAS methods suffer from either prohibitively high computational complexity of resulting architectures which have inadvertently affected the deep model deployment, or limitations which impede the flexibility of architecture design. To address these deficiencies, this work proposes an evolutionary computation (EC)-based method for compact and flexible NAS. A valuable search space with parameter-efficient mobile-inverted bottleneck convolution blocks as the primitive components is proposed to ensure the initial quality of the compact architectures. In addition, a two-level variable-length particle swarm optimization (PSO) approach is devised to evolve both the microarchitecture and macroarchitecture of CNNs. Furthermore, this study proposes an effective scheme by integrating multiple computationally reduced methods to greatly speed up the evaluation process. Experimental results on the CIFAR-10, CIFAR-100, and ImageNet datasets show the superiority of the proposed method against the state-of-the-art algorithms in terms of the classification performance, search cost, and resulting architecture complexity.

Multiscenario optimization problems involve multiple scenarios to be optimized simultaneously, where each scenario corresponds to a multiobjective optimization problem with specific operating conditions. The goal is to find a group of public compromised optimal solutions (PCOSs) achieving compromised optimal in every scenario. This type of optimization problems widely exists in real-world applications, but the research on it is few. In this article, a Multiscenario Optimization Evolutionary Algorithm based on a transfer framework is proposed. In one iteration, for each scenario, its knee solutions are recognized as its transfer candidates and are used to construct a constraint hyperplane, which determines whether transfer candidates from other scenarios can be the transferable solutions in this underlying scenario. Then, among all transfer candidates, those accepted by all scenarios are identified as the PCOSs and stored in archive. Afterwards, the archive updating process is applied to guarantee the optimality of solutions in archive and control the size of archive. Finally, each scenario’s accepted transferable solutions are utilized in its offspring generation, thus achieving information transfer between different scenarios. Experimental results on a group of benchmark functions verify the superiority of the proposed design in terms of both optimality and computational efficiency over existing approaches.

Arrhythmia detection from ECG is an important research subject in the prevention and diagnosis of cardiovascular diseases. The prevailing studies formulate arrhythmia detection from ECG as a time series classification problem. Meanwhile, early detection of arrhythmia presents a real-world demand for early prevention and diagnosis. In this paper, we address a problem of cardiovascular diseases early classification, which is a varied-length and long-length time series early classification problem as well. For solving this problem, we propose a deep reinforcement learning-based framework, namely Snippet Policy Network (SPN), consisting of four modules, snippet generator, backbone network, controlling agent, and discriminator. Comparing to the existing approaches, the proposed framework features flexible input length, solves the dual-optimization solution of the earliness and accuracy goals. Experimental results demonstrate that SPN achieves an excellent performance of over 80% in terms of accuracy. Compared to the state-of-the-art methods, at least 7% improvement on different metrics, including the precision, recall, F1-score, and harmonic mean, is delivered by the proposed SPN. To the best of our knowledge, this is the first work focusing on solving the cardiovascular early classification problem based on varied-length ECG data. Based on these excellent features from SPN, it offers a good exemplification for addressing all kinds of varied-length time series early classification problems.

With the rise of various smart electronics and mobile/edge devices, many existing high-accuracy convolutional neural network (CNN) models are difficult to be applied in practice due to the limited resources, such as memory capacity, power consumption, and spectral efficiency. In order to meet these constraints, researchers have carefully designed some lightweight networks. Meanwhile, to reduce the reliance on manual design on expert experience, some researchers also work to improve neural architecture search (NAS) algorithms to automatically design small networks, exploiting the multiobjective approaches that consider both accuracy and other important goals during optimization. However, simply searching for smaller network models is not consistent with the current research belief of "the deeper the better" and may affect the effectiveness of the model and thus waste the limited resources available. In this article, we propose an automatic method for designing CNNs architectures under constraint handling, which can search for optimal network models meeting the preset constraint. Specifically, an adaptive penalty algorithm is used for fitness evaluation, and a selective repair operation is developed for infeasible individuals to search for feasible CNN architectures. As a case study, we set the complexity (the number of parameters) as a resource constraint and perform multiple experiments on CIFAR-10 and CIFAR-100, to demonstrate the effectiveness of the proposed method. In addition, the proposed algorithm is compared with a state-of-the-art algorithm, NSGA-Net, and several manual-designed models. The experimental results show that the proposed algorithm can successfully solve the problem of the uncertain size of the optimal CNN model under the random search strategy, and the automatically designed CNN model can satisfy the predefined resource constraint while achieving better accuracy.

Deep neural networks (DNNs) have achieved great success in many applications. The architectures of DNNs play a crucial role in their performance, which is usually manually designed with rich expertise. However, such a design process is labor-intensive because of the trial-and-error process and also not easy to realize due to the rare expertise in practice. Neural architecture search (NAS) is a type of technology that can design the architectures automatically. Among different methods to realize NAS, the evolutionary computation (EC) methods have recently gained much attention and success. Unfortunately, there has not yet been a comprehensive summary of the EC-based NAS algorithms. This article reviews over 200 articles of most recent EC-based NAS methods in light of the core components, to systematically discuss their design principles and justifications on the design. Furthermore, current challenges and issues are also discussed to identify future research in this emerging field.

Robots for high-speed manipulation require to produce motions beyond the performance limitations set by the traditional approaches. Recent results integrate the properties associated with dynamic coupling driving and structural mechanics to compute optimal smooth arm motions; however, when accelerating the convergence speed of potential solutions, those approaches cannot avoid premature convergence. In this article, we propose an autonomous motion planning method at the torque level for a class of robots considering multiple conflicting performance metrics. Specifically, we focus on the hyper dynamic manipulation of a golf swing robot using a knee-guided multiobjective optimization algorithm. Compared with traditional planning methods in position or velocity level, it can study motor performance limitations with strong nonlinear dynamic coupling beyond the motion limits designed by the manufacturers. First, the robot’s joint torque is approximated by the B-spline method using the solution at each iteration. Then, we transform the motion planning problem into a multiobjective optimization problem with soft constraints of torque limits and hard constraints of joint stops, and develop a knee-guided evolutionary algorithm to find the optimization solution with the quality tradeoffs between the scale of parameters and metrics. Finally, we conduct the simulation to demonstrate the dynamics performance of the golf swing robot. The results indicate that our approach can generate superior dynamics performance beyond limits with low energy consumption and high precision.

In this article, we focus on the vehicle routing problem (VRP) with time windows under uncertainty. To capture the uncertainty characteristics in a real-life scenario, we design a new form of disturbance on travel time and construct robust multiobjective VRP with the time window, where the perturbation range of travel time is determined by the maximum disturbance degree. Two conflicting objectives include: 1)the minimization of both the total distance and: 2)the number of vehicles. A robust multiobjective particle swarms optimization approach is developed by incorporating an advanced encoding and decoding scheme, a robustness measurement metric, as well as the local search strategy. First, through particle flying in the decision space, the problem space characteristic under deterministic environment is fully exploited to provide guidance for robust optimization. Then, a designed metric is adopted to measure the robustness of solutions and help to search for the robust optimal solutions during the particle flying process. In addition to the updating process of particle, two local search strategies, problem-based local search and route-based local search, are developed for further improving the performance of solutions. For comparison, we develop several robust optimization problems by adding disturbances on selected benchmark problems. The experimental results validate our proposed algorithm has a distinguished ability to generate enough robust solutions and ensure the optimality of these solutions.

Neural architecture search (NAS), which automatically designs the architectures of deep neural networks, has achieved breakthrough success over many applications in the past few years. Among different classes of NAS methods, evolutionary computation-based NAS (ENAS) methods have recently gained much attention. Unfortunately, the development of ENAS is hindered by unfair comparison between different ENAS algorithms due to different training conditions and high computational cost caused by expensive performance evaluation. This article develops a platform named BenchENAS, in short for benchmarking evolutionary NAS, to address these issues. BenchENAS makes it easy to achieve fair comparisons between different algorithms by keeping them under the same settings. To accelerate the performance evaluation in a common lab environment, BenchENAS designs a novel and generic efficient evaluation method for the population characteristics of evolutionary computation. This method has greatly improved the efficiency of the evaluation. Furthermore, BenchENAS is easy to install and highly configurable and modular, which brings benefits in good usability and easy extensibility. This article conducts efficient comparison experiments on eight ENAS algorithms with high GPU utilization on this platform. The experiments validate that the fair comparison issue does exist in the current ENAS algorithms, and BenchENAS can alleviate this issue. A Website has been built to promote BenchENAS athttps://benchenas.com, where interested researchers can obtain the source code and document of BenchENAS for free.

Real-world multiobjective optimization problems usually involve conflicting objectives that change over time, which requires the optimization algorithms to quickly track the Pareto-optimal front (POF) when the environment changes. In recent years, evolutionary algorithms based on prediction models have been considered promising. However, most existing approaches only make predictions based on the linear correlation between a finite number of optimal solutions in two or three previous environments. These incomplete information extraction strategies may lead to low prediction accuracy in some instances. In this article, an incremental support vector machine (ISVM)-based dynamic multiobjective evolutionary algorithm, in short called ISVM-DMOEA, is proposed. We treat the solving of dynamic multiobjective optimization problems (DMOPs) as an online learning process, using the continuously obtained optimal solution to update an ISVM without discarding the solution information at earlier time. ISVM is then used to filter random solutions and generate an initial population for the next moment. To overcome the obstacle of insufficient training samples, a synthetic minority oversampling strategy is implemented before the training of ISVM. The advantage of this approach is that the nonlinear correlation between solutions can be explored online by ISVM, and the information contained in all historical optimal solutions can be exploited to a greater extent. The experimental results and comparison with the chosen state-of-the-art algorithms demonstrate that the proposed algorithm can effectively tackle DMOPs.

Cardiovascular diseases (CVDs) are considered the greatest threat to human life according to World Health Organization. Early classification of CVDs and the appropriate follow-up treatment are crucial for preventing sudden deaths. Electrocardiogram (ECG) is one of the most common non-invasive tools used to evaluate the state of the heart, which can be exploited to automatically diagnose as well. However, the importance of diagnosing CVDs is varying in different context-specific scenarios. For example, ST-segment elevation (STE) is an acute myocardial infarction indicator for patients associated with chest pain and cardiac biomarker. In in-hospital healthcare, STE should be diagnosed with a higher priority than the other phenotypes of ECG. Hence, the context-specific requirements should be considered in ECG early classification problems. We formalize the ECG early classification problem as the context-specific time series classification problem. We propose a novel Constraint-based Knee-guided Neuroevolutionary Algorithm (CKNA) based on the Snippet Policy Networks V2 to solve this problem. To validate the proposed method, we perform a series of experiments on two public ECG datasets under various context-specific simulated scenarios after consulting with physicians specializing in the area. Experimental results show that CKNA significantly improves the average recall of disease classification by 5.5% compared to the competing baseline under user-specified requirements. Moreover, experimental results prove that CKNA presents a feasible solution for the early classifying of cardiac arrhythmias under different user-specified scenarios.

Neural networks have been demonstrated to be trainable even with hundreds of layers, which exhibit remarkable improvement on expressive power and provide significant performance gains in a variety of tasks. However, the prohibitive computational cost has become a severe challenge for deploying them on resource-constrained platforms. Meanwhile, widely adopted deep neural network architectures, for example, ResNets or DenseNets, are manually crafted on benchmark datasets, which hamper their generalization ability to other domains. To cope with these issues, we propose an evolutionary algorithm-based method for shallowing deep neural networks (DNNs) at block levels, which is termed as ESNB. Different from existing studies, ESNB utilizes the ensemble view of block-wise DNNs and employs the multiobjective optimization paradigm to reduce the number of blocks while avoiding performance degradation. It automatically discovers shallower network architectures by pruning less informative blocks, and employs knowledge distillation to recover the performance. Moreover, a novel prior knowledge incorporation strategy is proposed to improve the exploration ability of the evolutionary search process, and a correctness-aware knowledge distillation strategy is designed for better knowledge transferring. Experimental results show that the proposed method can effectively accelerate the inference of DNNs while achieving superior performance when compared with the state-of-the-art competing methods.

Filter pruning has been widely applied on compression of deep convolutional neural networks. Especially, the filter pruning approach using evolutionary algorithms has exhibited powerful capabilities of automatic model compression and global optimal solution search. However, the exponential increasing of search space brought by the extremely large scale of deep networks makes it difficult to achieve a well-compressed model with limited time and energy. In this paper, we design an effective method aiming at improving the efficiency in evolutionary pruning, called IEEPruner. Multiple pruning criteria are first utilized to initialize an efficient search space, and then a novel encoding scheme is proposed to further reduce the search space. After that, we formulate the search of the optimal pruned structure as a multiobjective optimization problem, and integrate a multiobjective evolutionary algorithm to achieve a good balance between model size and accuracy in an automatic manner. Thus, we automatically search for the optimal pruned model in this search space. Extensive experiments on widely used convolutional neural networks including VGGNet, ResNet, and LeNet have demonstrated the efficiency of the proposed evolutionary pruning method.

A multi-modal multi-objective optimization problem (MMOP) may have equivalent Pareto optimal solutions. These solutions are different in the decision space but correspond to the same objective vector. Searching for equivalent Pareto optimal solutions with evolutionary algorithms is a hot topic in recent years. However, most existing researches are about continuous MMOPs, whereas there are few studies on discrete MMOPs. In this paper, we discuss the property of the multi-modal multi-objective traveling salesman problem and present a set of test problems. Then, we propose an evolutionary optimizer to solve the problem. Experimental results show that our evolutionary optimizer can find more equivalent Pareto optimal solutions than traditional multi-objective evolutionary optimizers on the test problems.

Currently, Deep Convolutional Neural Networks (DCNNs) are used to solve all kinds of problems in the field of machine learning and artificial intelligence due to their learning and adaptation capabilities. However, most successful DCNN models have a high computational complexity making them difficult to deploy on mobile or embedded platforms. This problem has prompted many researchers to develop algorithms and approaches to help reduce the computational complexity of such models. One of them is called filter pruning, where convolution filters are eliminated to reduce the number of parameters and, consequently, the computational complexity of the given model. In the present work, we propose a novel algorithm to perform filter pruning by using Multi-Objective Evolution Strategy (ES) algorithm, called DeepPruningES. Our approach avoids the need for using any knowledge during the pruning procedure and helps decision-makers by returning three pruned CNN models with different trade-offs between performance and computational complexity. We show that DeepPruningES can significantly reduce a model's computational complexity by testing it on three DCNN architectures: Convolutional Neural Networks (CNNs), Residual Neural Networks (ResNets), and Densely Connected Neural Networks (DenseNets).

Deep Convolutional Neural Networks (DCNNs) have the potential to revolutionize the field of Medical Imaging Diagnostics due to their capabilities of learning by using only raw data. However, DCNNs can only learn when trained using thousands of data points, which is not always available when dealing with medical data. Moreover, due to patient privacy concerns and the small prevalence of certain diseases in the population, medical data often presents unbalanced classes and fewer data points than other data types. Researchers often rely on Generative Adversarial Networks (GANs) to synthesize more data from a given distribution to solve this problem. Nevertheless, GANs are computationally intensive models requiring the use of powerful hardware to run. In the present work, an algorithm for pruning GANs based on Evolution Strategy (ES) and Multi-Criteria Decision Making (MCDM) is proposed in which a model with the best trade-off between computational complexity and synthesis performance can be found without the use of any trade-off parameter. In the proposed algorithm, the model with the best trade-off is defined geometrically as the candidate solution with the minimum Manhattan distance (MMD) in a two-dimensional objective space established by the number of Floating-Point Operations (FLOPs) and the Wasserstein distance of all candidate solutions, also known as the knee solution. The results show that the pruned GAN model achieves similar performance compared with the original model with up to 70% fewer Floating-Point Operations.

A rational leader selection strategy can enhance a swarm to manage the convergence and diversity during the entire search process. In this article, a novel adaptive multiobjective particle swarm optimization (MOPSO) is proposed on the basis of an evolutionary state estimation mechanism, which is used to detect the evolutionary environment whether in exploitation or exploration state. During the search process, different types of leaders, such as a convergence global best solution (c-gBest) and several diversity global best solutions (d-gBests), are to be selected from the external archive for particles under different evolutionary environments. The c-gBest is selected for improving the convergence when the swarm is in an exploitation state, while the d-gBests are chosen for enhancing the diversity in an exploration state. Furthermore, a modified archive maintenance strategy based on some predefined reference points is adopted to maximize the diversity of the Pareto solutions in the external archive. The experimental results demonstrate that the proposed algorithm performs significantly better than the several state-of-the-art multiobjective PSO algorithms and multiobjective evolutionary algorithms on 31 benchmark functions in terms of convergence and diversity of those obtained approximate Pareto fronts.

Over the past two decades, numerous multi- and many-objective evolutionary algorithms (MOEAs and MaOEAs) have been proposed to solve the multi- and many-objective optimization problems (MOPs and MaOPs), respectively. It is known that the difficulty of maintaining the convergence and diversity performances rapidly grows as the number of objectives increases. This phenomenon is especially evident for the Pareto-dominance-based EAs, because the nondominated sorting often fails to provide enough convergent pressure toward the Pareto front (PF). Therefore, many researchers came up with some non-Pareto-dominance-based EAs, which are based on indicator, decomposition, and so on. In this article, we propose a polar-metric ( p -metric)-based EA (PMEA) for tackling both MOPs and MaOPs. p -metric is a recently proposed performance indicator which adopts a set of uniformly distributed direction vectors. In PMEA, we use a two-phase selection which combines both nondominated sorting and p -metric. Moreover, a modification is proposed to adjust the direction vectors of p -metric dynamically. In the experiments, PMEA is compared with six state-of-the-art EAs in total and is measured by three performance metrics, including p -metric. According to the empirical results, PMEA shows promising performances on most of the test problems, involving both MOPs and MaOPs.

In recent years, numerous many-objective evolutionary algorithms (MaOEAs) have been developed to search for well-diversified and well-converged Pareto optimal solutions for high-dimensional many-objective optimization problems (MaOPs). However, existing MaOEAs have to tackle some daunting challenges, including the emergence of dominance resistance solutions, effective diversity preservation scheme, management of a large population size, extremely high computational complexity, sensitivity to the shape of Pareto front (PF), and overly relying on high-quality reference points. In this article, we present an evolution strategy (ES) for solving MaOPs, called MaOES, which can solve these challenges efficiently and effectively. Inspired by the Vector Equilibrium phenomenon in magnetic fields, isotropic magnetic particles would automatically repel from each other, keep the uniform distance from the nearest neighbors, and extend the entire magnetic fields as far as possible, all at the same time. In the proposed algorithm, an efficient self-adaptive Precision-Controllable Mutation operator is designed for individuals to explore and exploit the decision space. In addition, the Maximum Extension Distance strategy, which emulates the isotropic magnetic particle behavior in a magnetic field, is developed to guide individuals to keep uniform distance and extension to approximate the entire PF. As a result, the MaOES can obtain a well-converged and well-diversified PF with much less population size and far lower computational complexity. The larger the number of individuals, the sharper the contour the resulting approximate PF will be. Finally, the proposed algorithm is evaluated by the scalable MaOPs test suites on DTLZ and WFG. The experimental results have been demonstrated to provide a competitive and oftentimes better performance when compared against some chosen state-of-the-art MaOEAs.

Although numerous effective and efficient multiobjective evolutionary algorithms have been developed in recent years to search for a well-converged and well-diversified Pareto optimal front, most of these designs are computationally expensive and have to maintain a large population of individuals throughout the evolutionary process. Once the Pareto optimal front is found satisfactorily, the cognitive burden is then imposed upon decision makers to handpick one solution for implementation among a massive number of candidates even with powerful multicriteria decision-making tools. With the increase in the number of decision variables and objective functions in the face of real-world applications, these problems have become a daunting challenge. In this article, we propose a recursive evolutionary algorithm, called EvoKneer, to directly search for global knee solutions, but also multiple local knee solutions using the minimum Manhattan distance approach as opposed to an enormous number of Pareto optimal solutions. Compared with the traditional evolutionary approaches, the proposed design herein only preserves nondominated solutions in rank one in each generation. Boundary Individuals Selection is tailored to select only M2 boundary individuals where M is the number of objectives. Relieving the burden of maintaining a large population size and its diversity throughout a lengthy evolutionary process, this design with a very low computational cost allows the evolutionary algorithm to converge to knee solutions quickly. To facilitate the experimental validations, a simulator with a graphical user interface is developed under the Delphi XE7 platform and made available for public use. In addition, the proposed algorithm is evaluated with the DO2DK, DEB2DK, DEB2DK2, and DEB3DK benchmark functions. The comparison results validate that the proposed EvoKneer algorithm is computationally and efficiently finding all global and local knee solutions.

Deep neural networks (DNNs) have been regarded as fundamental tools for many disciplines. Meanwhile, they are known for their large-scale parameters, high redundancy in weights, and extensive computing resource consumptions, which pose a tremendous challenge to the deployment in real-time applications or on resource-constrained devices. To cope with this issue, compressing DNNs for accelerating its inference has drawn extensive interest recently. The basic idea is to prune parameters with little performance degradation. However, the overparameterized nature and the conflict between parameters reduction and performance maintenance make it prohibitive to manually search the pruning parameter space. In this paper, we formally establish filter pruning as a multiobjective optimization problem, and propose a knee-guided evolutionary algorithm (KGEA) that can automatically search for the solution with quality tradeoff between the scale of parameters and performance, in which both conflicting objectives can be optimized simultaneously. In particular, by incorporating a minimum Manhattan distance approach, the search effort in the proposed KGEA is explicitly guided toward the knee area, which greatly facilitates the manual search for a good tradeoff solution. Moreover, the parameter importance is directly estimated on the criterion of performance loss, which can robustly identify the redundancy. In addition to the knee solution, a performance-improved model can also be found in a fine-tuning-free fashion. The experiments on compressing fully convolutional LeNet and VGG-19 networks validate the superiority of the proposed algorithm over the state-of-the-art competing methods.

Deep learning is a timely research direction in machine learning, where breakthrough progress has been made in both academe and industries, bringing promising results in speech recognition, computer vision, industrial control and automation, etc. The motivation of deep learning is primarily to establish a model to simulate the neural connection structure of the human brain. While dealing with complex tasks, deep learning adopts a number of transformation stages to deliver the in-depth description and interpretation of the data. Deep learning achieves exceptional power and flexibility by learning to represent the task through a nested hierarchy of layers, with more abstract representations formed successively in terms of less abstract ones. One of the key issues of existing deep learning approaches is that the meaningful representations can be learned only when their hyperparameter settings are properly specified beforehand, and general parameters are learned during the training process. Until now, not much research has been dedicated to automatically set the hyperparameters, and accurately find the globally optimal general parameters. However, this problem can be formulated as optimization problems, including discrete optimization, constrained optimization, large-scale global optimization, and multiobjective optimization, by engaging mechanisms of evolutionary computation.

As an essential component in multi- and many-objective optimization, decision-making process either selects a subset of solutions from the whole Pareto front or guides the search toward a small part of the Pareto front during the evolutionary process. In recent years, for many-objective optimization problems (MaOPs), a number of evolutionary algorithms have been developed to search for Pareto optimal solutions. However, there is a lack of research works focusing on designing decision-making approaches. In order to overcome this deficiency, we propose a novel knee-based decision-making method to search for several solutions of interest (SOIs) from a large number of solutions on the Pareto front, each of which contains the best convergence performance at least within its neighborhood and can be identified as a global or local knee solution. The optimization performance achieved by all SOIs approximates the performance of the whole Pareto front as much as possible. Furthermore, in order to relieve the difficulties in the decision-making process on MaOPs, a new visualization approach is developed based on this proposed decision-making approach. It provides information about the shape and location of the Pareto front, the possible bulge, as well as the convergence degree and distribution of solutions. The experimental results on several benchmark functions demonstrate the superiority of the proposed design in the selection of SOIs and visualization of high-dimensional objective space.

Dynamic multiobjective optimization problems (DMOPs) are optimization problems with multiple conflicting optimization objectives, and these objectives change over time. Transfer learning-based approaches have been proven to be promising; however, a slow solving speed is one of the main obstacles preventing such methods from solving real-world problems. One of the reasons for the slow running speed is that low-quality individuals occupy a large amount of computing resources, and these individuals may lead to negative transfer. Combining high-quality individuals, such as knee points, with transfer learning is a feasible solution to this problem. However, the problem with this idea is that the number of high-quality individuals is often very small, so it is difficult to acquire substantial improvements using conventional transfer learning methods. In this article, we propose a knee point-based transfer learning method, called KT-DMOEA, for solving DMOPs. In the proposed method, a trend prediction model (TPM) is developed for producing the estimated knee points. Then, an imbalance transfer learning method is proposed to generate a high-quality initial population by using these estimated knee points. The advantage of this approach is that the seamless integration of a small number of high-quality individuals and the imbalance transfer learning technique can greatly improve the computational efficiency while maintaining the quality of the solution. The experimental results and performance comparisons with some chosen state-of-the-art algorithms demonstrate that the proposed design is capable of significantly improving the performance of dynamic optimization.

Evolutionary neural architecture search (ENAS) can automatically design the architectures of deep neural networks (DNNs) using evolutionary computation algorithms. However, most ENAS algorithms require an intensive computational resource, which is not necessarily available to the users interested. Performance predictors are a type of regression models which can assist to accomplish the search, while without exerting much computational resource. Despite various performance predictors have been designed, they employ the same training protocol to build the regression models: 1) sampling a set of DNNs with performance as the training dataset; 2) training the model with the mean square error criterion; and 3) predicting the performance of DNNs newly generated during the ENAS. In this article, we point out that the three steps constituting the training protocol are not well thought-out through intuitive and illustrative examples. Furthermore, we propose a new training protocol to address these issues, consisting of designing a pairwise ranking indicator to construct the training target, proposing to use the logistic regression to fit the training samples, and developing a differential method to build the training instances. To verify the effectiveness of the proposed training protocol, four widely used regression models in the field of machine learning have been chosen to perform the comparisons on two benchmark datasets. The experimental results of all the comparisons demonstrate that the proposed training protocol can significantly improve the performance prediction accuracy against the traditional training protocols.

In recent years, numerous efficient and effective multimodal multiobjective evolutionary algorithms (MMOEAs) have been developed to search for multiple equivalent sets of Pareto optimal solutions simultaneously. However, some of the MMOEAs prefer convergent individuals over diversified individuals to construct the mating pool, and the individuals with slightly better decision space distribution may be replaced by significantly better objective space distribution. Therefore, the diversity in the decision space may become deteriorated, in spite of the decision and objective diversities have been taken into account simultaneously in most MMOEAs. Because the Pareto optimal subsets may have various shapes and locations in the decision space, it is very difficult to drive the individuals converged to every Pareto subregion with a uniform density. Some of the Pareto subregions may be overly crowded, while others are rather sparsely distributed. Consequently, many existing MMOEAs obtain Pareto subregions with imbalanced density. In this article, we present a two-stage double niched evolution strategy, namely DN-MMOES, to search for the equivalent global Pareto optimal solutions which can address the above challenges effectively and efficiently. The proposed DN-MMOES solves the multimodal multiobjective optimization problem (MMOP) in two stages. The first stage adopts the niching strategy in the decision space, while the second stage adapts double niching strategy in both spaces. Moreover, an effective decision density self-adaptive strategy is designed for improving the imbalanced decision space density. The proposed algorithm is compared against eight state-of-the-art MMOEAs. The inverted generational distance union (IGDunion) performance indicator is proposed to fairly compare two competing MMOEAs as a whole. The experimental results show that DN-MMOES provides a better performance to search for the complete Pareto Subsets and Pareto Front on IDMP and CEC 2019 MMOPs test suite.

The field of medical imaging diagnostic makes use of a modality of imaging tests, e.g., X-rays, ultrasounds, computed tomographies, and magnetic resonance imaging, to assist physicians with the diagnostic of patients' illnesses. Due to their state-of-the-art results in many challenging image classification tasks, deep neural networks (DNNs) are suitable tools for use by physicians to provide diagnostic support when dealing with medical images. To further advance the field, the present work proposes a two-phase algorithm capable of automatically generating compact DNN architectures given a database, called here DNNDeepeningPruning. In the first phase, also called the deepening phase, the algorithm grows a DNN by adding blocks of residual layers one after another until the model overfits the given data. In the second phase, called the pruning phase, the algorithm prunes the created DNN model from the first phase to produce a DNN with a small amount of floating-point operations guided by some preference given by the user. The proposed algorithm unifies the two separate fields of DNN architecture searching and pruning under a single framework, and it is tested in two medical imaging data sets with satisfactory results.

Previous work has shown that adversarial learning can be used for unsupervised monocular depth and visual odometry (VO) estimation, in which the adversarial loss and the geometric image reconstruction loss are utilized as the mainly supervisory signals to train the whole unsupervised framework. However, the performance of the adversarial framework and image reconstruction is usually limited by occlusions and the visual field changes between the frames. This article proposes a masked generative adversarial network (GAN) for unsupervised monocular depth and ego-motion estimations. The MaskNet and Boolean mask scheme are designed in this framework to eliminate the effects of occlusions and impacts of visual field changes on the reconstruction loss and adversarial loss, respectively. Furthermore, we also consider the scale consistency of our pose network by utilizing a new scale-consistency loss, and therefore, our pose network is capable of providing the full camera trajectory over a long monocular sequence. Extensive experiments on the KITTI data set show that each component proposed in this article contributes to the performance, and both our depth and trajectory predictions achieve competitive performance on the KITTI and Make3D data sets.

Multi-modal multi-objective optimization problems may have different Pareto optimal solutions with the same objective vector. A number of evolutionary multi-modal multiobjective algorithms have been developed to solve these problems. They aim to search for a Pareto optimal solution set with good diversity in both the objective and decision spaces. Although the normalization in both the objective and decision spaces is very important for these algorithms, there are few studies on this topic. In this paper, we investigate the effect of four normalization methods on two evolutionary multi-modal multiobjective algorithms. Six distance minimization problems are chosen as test problems. The experimental results show that the effect of normalization in evolutionary multi-modal multiobjective optimization is algorithm- and problem-dependent.

When dynamic multi-objective optimization evolutionary algorithms (DMOEA) are used to solve real world problems, these are not only required to be able to find the Pareto-Optimal Set (POS) quickly, but also the results obtained can be easily used by decision makers. The classic DMOEAs have much room for improvement in both aspects. Recently, the transfer learning based DMOEAs have been proved that these methods can significantly improve the quality of the solution, but there are still too many individuals in the POS obtained by these algorithms. The resulting problems are twofold: this not only consumes a lot of computing resources to those solutions that will not be used, but also makes it more difficult for decision makers to choose. In this paper, we proposed a dynamic multiobjective optimization evolutionary algorithm which combines knee solutions with transfer learning method, and the feature of the proposed method is that it only outputs a very small number of solutions, which can greatly improve the efficiency of decisionmaking. The proposed algorithm divides the whole decision space into different subspaces, and find a local knee solutions in each subspace, then a transfer learning framework, Tr-DMOEA, is used to predict the knee solutions of the optimization problem at the next moment by using the local knee solutions and a global knee solution. The experimental results show the effectiveness of our design.

Pareto-based multi-objective evolutionary algorithms experience grand challenges in solving many-objective optimization problems due to their inability to maintain both convergence and diversity in a high-dimensional objective space. Exiting approaches usually modify the selection criteria to overcome this issue. Different from them, we propose a novel meta-objective (MeO) approach that transforms the many-objective optimization problems in which the new optimization problems become easier to solve by the Pareto-based algorithms. MeO converts a given many-objective optimization problem into a new one, which has the same Pareto optimal solutions and the number of objectives with the original one. Each meta-objective in the new problem consists of two components which measure the convergence and diversity performances of a solution, respectively. Since MeO only converts the problem formulation, it can be readily incorporated within any multi-objective evolutionary algorithms, including those non-Pareto-based ones. Particularly, it can boost the Pareto-based algorithms' ability to solve many-objective optimization problems. Due to separately evaluating the convergence and diversity performances of a solution, the traditional density-based selection criteria, for example, crowding distance, will no longer mistake a solution with poor convergence performance for a solution with low density value. By penalizing a solution in term of its convergence performance in the meta-objective space, the Pareto dominance becomes much more effective for a many-objective optimization problem. Comparative study validates the competitive performance of the proposed meta-objective approach in solving many-objective optimization problems.

Convolutional neural networks (CNNs) have gained remarkable success on many image classification tasks in recent years. However, the performance of CNNs highly relies upon their architectures. For the most state-of-the-art CNNs, their architectures are often manually designed with expertise in both CNNs and the investigated problems. Therefore, it is difficult for users, who have no extended expertise in CNNs, to design optimal CNN architectures for their own image classification problems of interest. In this article, we propose an automatic CNN architecture design method by using genetic algorithms, to effectively address the image classification tasks. The most merit of the proposed algorithm remains in its "automatic" characteristic that users do not need domain knowledge of CNNs when using the proposed algorithm, while they can still obtain a promising CNN architecture for the given images. The proposed algorithm is validated on widely used benchmark image classification datasets, compared to the state-of-the-art peer competitors covering eight manually designed CNNs, seven automatic + manually tuning, and five automatic CNN architecture design algorithms. The experimental results indicate the proposed algorithm outperforms the existing automatic CNN architecture design algorithms in terms of classification accuracy, parameter numbers, and consumed computational resources. The proposed algorithm also shows the very comparable classification accuracy to the best one from manually designed and automatic + manually tuning CNNs, while consuming fewer computational resources.

The community detection in dynamic networks is essential for important applications such as social network analysis. Such detection requires simultaneous maximization of the clustering accuracy at the current time step while minimization of the clustering drift between two successive time steps. In most situations, such objectives are often in conflict with each other. This article proposes the concept of consensus community. Knowledge from the previous step is obtained by extracting the intrapopulation consensus communities from the optimal population of the previous step. Subsequently, the intrapopulation consensus communities of the previous step obtained is voted by the population of the current time step during the evolutionary process. A subset of the consensus communities, which receives a high support rate, will be recognized as the interpopulation consensus communities of the previous and current steps. Interpopulation consensus communities are the knowledge that can be transferred from the previous to the current step. The population of the current time step can evolve toward the direction similar to the population in the previous time step by retaining such interpopulation consensus community during the evolutionary process. Community structure is subjected to evaluation, update, and mutation events, which are directed by using interpopulation consensus community information during the evolutionary process. The experimental results over many artificial and real-world dynamic networks illustrate that the proposed method produces more accurate and robust results than those of the state-of-the-art approaches.

Uncertainty is an important feature abstracted from real-world applications. Multiobjective optimization problems (MOPs) with uncertainty can always be characterized as robust MOPs (RMOPs). Over recent years, multiobjective optimization evolutionary algorithms (EAs) have demonstrated the success in solving MOPs. However, most of them do not consider disturbance in the design. In order to handling the uncertainty in the optimization problem, we first give a thorough analysis of three important issues on robust optimization. Then, a novel EA called multiobjective optimization EA with robustness enhancement is developed, where the seamless integration of robustness and optimality is achieved by a proposed novel archive updating mechanism applied on the evolutionary process as well as the new robust optimal front building strategy designed to construct the final robust optimal front. Furthermore, the new designed archive updating mechanism makes the robust optimization process free of the enormous computational workload induced from sampling. The experimental results on a set of benchmark functions show the superiority of the proposed design in terms of both solutions' quality under the disturbance and computational efficiency in solving RMOPs.

There may exist more than one Pareto optimal solution with the same objective vector to a multimodal multiobjective optimization problem (MMOP). The difficulties in finding such solutions can be different. Although a number of evolutionary multimodal multiobjective algorithms (EMMAs) have been proposed, they are unable to solve such an MMOP due to their convergence-first selection criteria. They quickly converge to the Pareto optimal solutions which are easy to find and therefore lose diversity in the decision space. That is, such an MMOP features an imbalance between achieving convergence and preserving diversity in the decision space. In this article, we first present a set of imbalanced distance minimization benchmark problems. Then we propose an evolutionary algorithm using a convergence-penalized density method (CPDEA). In CPDEA, the distances among solutions in the decision space are transformed based on their local convergence quality. Their density values are estimated based on the transformed distances and used as the selection criterion. We compare CPDEA with five state-of-the-art EMMAs on the proposed benchmarks. Our experimental results show that CPDEA is clearly superior in solving these problems.

Convolutional neural networks (CNNs) have shown remarkable performance in various real-world applications. Unfortunately, the promising performance of CNNs can be achieved only when their architectures are optimally constructed. The architectures of state-of-the-art CNNs are typically handcrafted with extensive expertise in both CNNs and the investigated data, which consequently hampers the widespread adoption of CNNs for less experienced users. Evolutionary deep learning (EDL) is able to automatically design the best CNN architectures without much expertise. However, the existing EDL algorithms generally evaluate the fitness of a new architecture by training from scratch, resulting in the prohibitive computational cost even operated on high-performance computers. In this paper, an end-to-end offline performance predictor based on the random forest is proposed to accelerate the fitness evaluation in EDL. The proposed performance predictor shows the promising performance in term of the classification accuracy and the consumed computational resources when compared with 18 state-of-the-art peer competitors by integrating into an existing EDL algorithm as a case study. The proposed performance predictor is also compared with the other two representatives of existing performance predictors. The experimental results show the proposed performance predictor not only significantly speeds up the fitness evaluations but also achieves the best prediction among the peer performance predictors.

Evolutionary paradigms have been successfully applied to neural network designs for two decades. Unfortunately, these methods cannot scale well to the modern deep neural networks due to the complicated architectures and large quantities of connection weights. In this paper, we propose a new method using genetic algorithms for evolving the architectures and connection weight initialization values of a deep convolutional neural network to address image classification problems. In the proposed algorithm, an efficient variable-length gene encoding strategy is designed to represent the different building blocks and the potentially optimal depth in convolutional neural networks. In addition, a new representation scheme is developed for effectively initializing connection weights of deep convolutional neural networks, which is expected to avoid networks getting stuck into local minimum that is typically a major issue in the backward gradient-based optimization. Furthermore, a novel fitness evaluation method is proposed to speed up the heuristic search with substantially less computational resource. The proposed algorithm is examined and compared with 22 existing algorithms on nine widely used image classification tasks, including the state-of-the-art methods. The experimental results demonstrate the remarkable superiority of the proposed algorithm over the state-of-the-art designs in terms of classification error rate and the number of parameters (weights).

This article presents a novel evolution strategy-based evolutionary algorithm, named DMOES, which can efficiently and effectively solve multiobjective optimization problems in dynamic environments. First, an efficient self-adaptive precision controllable mutation operator is designed for individuals to explore and exploit the decision space. Second, the simulated isotropic magnetic particles niching can guide the individuals to keep uniform distance and extent to approximate the entire Pareto front automatically. Third, the nondominated solutions (NDS) guided immigration can facilitate the population convergence with two different strategies for the NDSs and the dominated solutions, respectively. As a result, our algorithm can track the new approximate Pareto set and approximate Pareto front as quickly as possible when the environment changes. In addition, DMOES can obtain a well-converged and well-diversified Pareto front with much less population size and far lower computational cost. The larger the number of individuals, the sharper the contour of the resulted approximate Pareto front will be. Finally, the proposed algorithm is evaluated by the FDA, dMOP, UDF, and ZJZ test suites. The experimental results have been demonstrated to provide a competitive and oftentimes better performance when compared against some chosen state-of-the-art dynamic multiobjective evolutionary algorithms.

The performance of convolutional neural networks (CNNs) highly relies on their architectures. In order to design a CNN with promising performance, extensive expertise in both CNNs and the investigated problem domain is required, which is not necessarily available to every interested user. To address this problem, we propose to automatically evolve CNN architectures by using a genetic algorithm (GA) based on ResNet and DenseNet blocks. The proposed algorithm is completely automatic in designing CNN architectures. In particular, neither preprocessing before it starts nor postprocessing in terms of CNNs is needed. Furthermore, the proposed algorithm does not require users with domain knowledge on CNNs, the investigated problem, or even GAs. The proposed algorithm is evaluated on the CIFAR10 and CIFAR100 benchmark data sets against 18 state-of-the-art peer competitors. Experimental results show that the proposed algorithm outperforms the state-of-the-art CNNs hand-crafted and the CNNs designed by automatic peer competitors in terms of the classification performance and achieves a competitive classification accuracy against semiautomatic peer competitors. In addition, the proposed algorithm consumes much less computational resource than most peer competitors in finding the best CNN architectures.

Biomedical image segmentation is lately dominated by deep neural networks (DNNs) due to their surpassing expert-level performance. However, the existing DNN models for biomedical image segmentation are generally highly parameterized, which severely impede their deployment on real-time platforms and portable devices. To tackle this difficulty, we propose an evolutionary compression method (ECDNN) to automatically discover efficient DNN architectures for biomedical image segmentation. Different from the existing studies, ECDNN can optimize network loss and number of parameters simultaneously during the evolution, and search for a set of Pareto-optimal solutions in a single run, which is useful for quantifying the tradeoff in satisfying different objectives, and flexible for compressing DNN when preference information is uncertain. In particular, a set of novel genetic operators is proposed for automatically identifying less important filters over the whole network. Moreover, a pruning operator is designed for eliminating convolutional filters from layers involved in feature map concatenation, which is commonly adopted in DNN architectures for capturing multi-level features from biomedical images. Experiments carried out on compressing DNN for retinal vessel and neuronal membrane segmentation tasks show that ECDNN can not only improve the performance without any retraining but also discover efficient network architectures that well maintain the performance. The superiority of the proposed method is further validated by comparison with the state-of-the-art methods.

The existing multiobjective evolutionary algorithms (EAs) based on nondominated sorting may encounter serious difficulties in tackling many-objective optimization problems (MaOPs), because the number of nondominated solutions increases exponentially with the number of objectives, leading to a severe loss of selection pressure. To address this problem, some existing many-objective EAs (MaOEAs) adopt Euclidean or Manhattan distance to estimate the convergence of each solution during the environmental selection process. Nevertheless, either Euclidean or Manhattan distance is a special case of Minkowski distance with the order P=2 or P=1 , respectively. Thus, it is natural to adopt Minkowski distance for convergence estimation, in order to cover various types of Pareto fronts (PFs) with different concavity-convexity degrees. In this paper, a Minkowski distance-based EA is proposed to solve MaOPs. In the proposed algorithm, first, the concavity-convexity degree of the approximate PF, denoted by the value of P , is dynamically estimated. Subsequently, the Minkowski distance of order P is used to estimate the convergence of each solution. Finally, the optimal solutions are selected by a comprehensive method, based on both convergence and diversity. In the experiments, the proposed algorithm is compared with five state-of-the-art MaOEAs on some widely used benchmark problems. Moreover, the modified versions for two compared algorithms, integrated with the proposed P -estimation method and the Minkowski distance, are also designed and analyzed. Empirical results show that the proposed algorithm is very competitive against other MaOEAs for solving MaOPs, and two modified compared algorithms are generally more effective than their predecessors.

Uncertainty inadvertently exists in most real-world applications. In the optimization process, uncertainty poses a very important issue and it directly affects the optimization performance. Nowadays, evolutionary algorithms (EAs) have been successfully applied to various multiobjective optimization problems (MOPs). However, current researches on EAs rarely consider uncertainty in the optimization process and existing algorithms often fail to handle the uncertainty, which have limited EAs' applications in real-world problems. When MOPs come with uncertainty, they are referred to as robust MOPs (RMOPs). In this paper, we aim at solving RMOPs using EA-based optimization search. We propose a novel robust multiobjective optimization EA (RMOEA) with two distinct, yet complement, parts: 1) multiobjective optimization finding global Pareto optimal front ignoring disturbance at first and 2) robust optimization searching for the robust optimal front afterward. Furthermore, a comprehensive performance evaluation method is proposed to quantify the performance of RMOEA in solving RMOPs. Experimental results on a group of benchmark functions demonstrate the superiority of the proposed design in terms of both solutions' quality under the disturbance and computational efficiency in solving RMOPs.

There have been few researches on solving multimodal multiobjective optimization problems, whereas they are commonly seen in real-world applications but difficult for the existing evolutionary optimizers. In this paper, we propose a novel multimodal multiobjective evolutionary algorithm using two-archive and recombination strategies. In the proposed algorithm, the properties of decision variables and the relationships among them are analyzed at first to guide the evolutionary search. Then, a general framework using two archives, i.e., the convergence and the diversity archives, is adopted to cooperatively solve these problems. Moreover, the diversity archive simultaneously employs a clustering strategy to guarantee diversity in the objective space and a niche-based clearing strategy to promote the same in the decision space. At the end of evolution process, solutions in the convergence and the diversity archives are recombined to obtain a large number of multiple Pareto optimal solutions. In addition, a set of benchmark test functions and a performance metric are designed for multimodal multiobjective optimization. The proposed algorithm is empirically compared with two state-of-the-art evolutionary algorithms on these test functions. The comparative results demonstrate that the overall performance of the proposed algorithm is significantly superior to the competing algorithms.

Convergence and diversity are interdependently handled during the evolutionary process by most existing many-objective evolutionary algorithms (MaOEAs). In such a design, the degraded performance of one would deteriorate the other, and only solutions with both are able to improve the performance of MaOEAs. Unfortunately, it is not easy to constantly maintain a population of solutions with both convergence and diversity. In this paper, an MaOEA based on two independent stages is proposed for effectively solving many-objective optimization problems (MaOPs), where the convergence and diversity are addressed in two independent and sequential stages. To achieve this, we first propose a nondominated dynamic weight aggregation method by using a genetic algorithm, which is capable of finding the Pareto-optimal solutions for MaOPs with concave, convex, linear and even mixed Pareto front shapes, and then these solutions are employed to learn the Pareto-optimal subspace for the convergence. Afterward, the diversity is addressed by solving a set of single-objective optimization problems with reference lines within the learned Pareto-optimal subspace. To evaluate the performance of the proposed algorithm, a series of experiments are conducted against six state-of-the-art MaOEAs on benchmark test problems. The results show the significantly improved performance of the proposed algorithm over the peer competitors. In addition, the proposed algorithm can focus directly on a chosen part of the objective space if the preference area is known beforehand. Furthermore, the proposed algorithm can also be used to effectively find the nadir points.

Deep learning (DL) aims at learning the meaningful representations. A meaningful representation gives rise to significant performance improvement of associated machine learning (ML) tasks by replacing the raw data as the input. However, optimal architecture design and model parameter estimation in DL algorithms are widely considered to be intractable. Evolutionary algorithms are much preferable for complex and nonconvex problems due to its inherent characteristics of gradient-free and insensitivity to the local optimal. In this paper, we propose a computationally economical algorithm for evolving unsupervised deep neural networks to efficiently learn meaningful representations, which is very suitable in the current big data era where sufficient labeled data for training is often expensive to acquire. In the proposed algorithm, finding an appropriate architecture and the initialized parameter values for an ML task at hand is modeled by one computational efficient gene encoding approach, which is employed to effectively model the task with a large number of parameters. In addition, a local search strategy is incorporated to facilitate the exploitation search for further improving the performance. Furthermore, a small proportion labeled data is utilized during evolution search to guarantee the learned representations to be meaningful. The performance of the proposed algorithm has been thoroughly investigated over classification tasks. Specifically, error classification rate on MNIST with 1.15% is reached by the proposed algorithm consistently, which is considered a very promising result against state-of-the-art unsupervised DL algorithms.

Inverted generational distance (IGD) has been widely considered as a reliable performance indicator to concurrently quantify the convergence and diversity of multiobjective and many-objective evolutionary algorithms. In this paper, an IGD indicator-based evolutionary algorithm for solving many-objective optimization problems (MaOPs) has been proposed. Specifically, the IGD indicator is employed in each generation to select the solutions with favorable convergence and diversity. In addition, a computationally efficient dominance comparison method is designed to assign the rank values of solutions along with three newly proposed proximity distance assignments. Based on these two designs, the solutions are selected from a global view by linear assignment mechanism to concern the convergence and diversity simultaneously. In order to facilitate the accuracy of the sampled reference points for the calculation of IGD indicator, we also propose an efficient decomposition-based nadir point estimation method for constructing the Utopian Pareto front (PF) which is regarded as the best approximate PF for real-world MaOPs at the early stage of the evolution. To evaluate the performance, a series of experiments is performed on the proposed algorithm against a group of selected state-of-the-art many-objective optimization algorithms over optimization problems with 8-, 15-, and 20-objective. Experimental results measured by the chosen performance metrics indicate that the proposed algorithm is very competitive in addressing MaOPs.

Convolutional autoencoders (CAEs) have shown their remarkable performance in stacking to deep convolutional neural networks (CNNs) for classifying image data during the past several years. However, they are unable to construct the state-of-the-art CNNs due to their intrinsic architectures. In this regard, we propose a flexible CAE (FCAE) by eliminating the constraints on the numbers of convolutional layers and pooling layers from the traditional CAE. We also design an architecture discovery method by exploiting particle swarm optimization, which is capable of automatically searching for the optimal architectures of the proposed FCAE with much less computational resource and without any manual intervention. We test the proposed approach on four extensively used image classification data sets. Experimental results show that our proposed approach in this paper significantly outperforms the peer competitors including the state-of-the-art algorithms.

Deep learning algorithms have shown their superiority especially in addressing challenging machine learning tasks. The best performance of deep learning algorithms can be reached only when their hyper-parameters have been successfully optimized. However, the hyper-parameter optimization problem is non-convex and non-differentiable, and traditional optimization algorithms are incapable of addressing them well. Evolutionary algorithms are a class of meta-heuristic search algorithms, preferred for optimizing real-world problems due largely to their no mathematical requirements on the problems to be optimized. Although most researchers from the community of deep learning are aware of the effectiveness of evolutionary algorithms in optimizing the hyper-parameters of deep learning algorithms, they still believe that the grid search method is more effective when the number of hyper-parameters is small. To clarify this, we design a hyper-parameter optimization method by using particle swarm optimization that is a widely used evolutionary algorithm, to perform 192 experimental comparisons for stacked auto-encoders that are a class of deep learning algorithms with a relative small number of hyper-parameters, investigate and compare the classification accuracy and computational complexity with those of the grid search method on eight widely used image classification benchmark datasets. The experimental results show that the proposed algorithm can achieve the comparative classification accuracy but saving 10×-100× computational complexity compared with the grid search method.

A many-objective problems (MaOP) refer to the optimization problem involving more than three objectives. Particle swarm optimization (PSO) is one of the potential heuristic methods suited for solving MaOPs. The personal best selection strategy, the global best selection strategy, and the archive maintenance strategy are the three key components in the design of a Many-Objective Particle Swarm Optimization (MaOPSO). The personal best and global best selection strategies determine the direction where particles will fly. The archive maintenance strategy has an important impact on convergence and diversity of its algorithm. In MaOPs, the high dimensionality in the objective space decreases the probability of a solution to be dominated by the other solutions in the population. Thus, it becomes more difficult for PSO to select the good leaders from so many non-dominated solutions. In this paper, a virtual Inverted Generational Distance indicator is proposed to evaluate the comprehensive quality of a solution in the external archive according to a constructed virtual Pareto front (vPF). Accordingly, a new indicator-based MaOPSO using vPF (MaOPSO/vPF) is developed to improve the convergence and diversity of the approximate Pareto front. Experimental results on the MaF test suites demonstrate that the proposed MaOPSO/vPF performs better than some selected competing Multi-objective Optimization Evolutionary Algorithms.

Reliable prediction of movie box-office revenues can greatly reduce the financial risk in the film industry, but accurate prediction is not easy to obtain. Recently, deep neural networks has been applied on movie box-office revenues prediction problems as a promising solution. However, the architecture has a significant impact on its performance, and generally involves a heavy burden of manually designing which is unable to traverse the space of possible architectures efficiently. As a result, the applicability and performance of deep neural networks are severely limited. This paper proposes a new evolutionary algorithm for evolving deep neural networks for movie box-office revenues prediction. In particular, a deep neural network that fuses features extracted from movie posters by a convolutional neural network is introduced first, then a set of novel genetic operators are designed correspondingly. The proposed method can automate the deep neural network architecture designing and aim to search the optimal architecture for movie box-office revenues prediction. Experiments carried out on the Internet Movie Database (IMDB) dataset show that the proposed algorithm achieves superior performance compare to other competitive approaches.

One of the major distinguishing features of the dynamic multiobjective optimization problems (DMOPs) is that optimization objectives will change over time, thus tracking the varying Pareto-optimal front becomes a challenge. One of the promising solutions is reusing “experiences” to construct a prediction model via statistical machine learning approaches. However, most existing methods neglect the nonindependent and identically distributed nature of data to construct the prediction model. In this paper, we propose an algorithmic framework, called transfer learning-based dynamic multiobjective evolutionary algorithm (EA), which integrates transfer learning and population-based EAs to solve the DMOPs. This approach exploits the transfer learning technique as a tool to generate an effective initial population pool via reusing past experience to speed up the evolutionary process, and at the same time any population-based multiobjective algorithms can benefit from this integration without any extensive modifications. To verify this idea, we incorporate the proposed approach into the development of three well-known EAs, nondominated sorting genetic algorithm II, multiobjective particle swarm optimization, and the regularity model-based multiobjective estimation of distribution algorithm. We employ 12 benchmark functions to test these algorithms as well as compare them with some chosen state-of-the-art designs. The experimental results confirm the effectiveness of the proposed design for DMOPs.

The performance of multiobjective evolutionary algorithms deteriorates appreciably in solving many-objective optimization problems (MaOPs) which encompass more than three objectives. One of the known rationales is the loss of selection pressure which leads to the selected parents not generating promising offspring toward Pareto-optimal front (PF) with diversity. Estimation of distribution algorithms sample new solutions with a probabilistic model built from the statistics extracting over the existing solutions so as to mitigate the adverse impact of genetic operators. In this paper, an improved regularity-based estimation of distribution algorithm is proposed to effectively tackle unconstrained MaOPs. In the proposed algorithm, diversity repairing mechanism is utilized to mend the areas, where need nondominated solutions with a closer proximity to the PF. Then favorable solutions are generated by the model built from the regularity of the solutions surrounding a group of representatives. These two steps collectively enhance the selection pressure which gives rise to the superior convergence of the proposed algorithm. In addition, dimension reduction technique is employed in the decision space to speed up the estimation search of the proposed algorithm. Finally, by assigning the Pareto-optimal solutions to the uniformly distributed reference vectors, a set of solutions with excellent diversity and convergence is obtained. To measure the performance, NSGA-III, GrEA, MOEA/D, HypE, MBN-EDA, and RM-MEDA are selected to perform comparison experiments over DTLZ and DTLZ-test suites with 3-, 5-, 8-, 10-, and 15-objective. Experimental results quantified by the selected performance metrics reveal that the proposed algorithm shows considerable competitiveness in addressing unconstrained MaOPs.

The class imbalance problem in machine learning occurs when certain classes are underrepresented relative to the others, leading to a learning bias toward the majority classes. To cope with the skewed class distribution, many learning methods featuring minority oversampling have been proposed, which are proved to be effective. To reduce information loss during feature space projection, this study proposes a novel oversampling algorithm, named minority oversampling in kernel adaptive subspaces (MOKAS), which exploits the invariant feature extraction capability of a kernel version of the adaptive subspace self-organizing maps. The synthetic instances are generated from well-trained subspaces and then their pre-images are reconstructed in the input space. Additionally, these instances characterize nonlinear structures present in the minority class data distribution and help the learning algorithms to counterbalance the skewed class distribution in a desirable manner. Experimental results on both real and synthetic data show that the proposed MOKAS is capable of modeling complex data distribution and outperforms a set of state-of-the-art oversampling algorithms.

In many-objective optimization, visualization of population in the high-dimensional objective space provides a critical understanding of the Pareto front. First, visualization throughout the evolutionary process can be exploited in developing effective many-objective evolutionary algorithms. Furthermore, visualization is a crucial component of multi-criteria decision making. By directly observing the performance of each solution, the trade-off between objectives, and distribution of the approximate front, the decision maker can easily decide which solution should be chosen from. In this paper, we make a detailed summary for existing visualization approaches and group them into five different categories. Then, three evaluation criteria for visualization approaches are designed, according to which, five state-of-the-arts are compared under the created data sets. Experimental results show that all approaches can satisfy each criterion to some degree but no one can fully achieve all of these criteria. There is a need to develop the new approach emphasis on fully satisfy all criteria simultaneously. Then, based on the comparison results, two future research directions for visualization approach are proposed.

In traditional many-objective evolutionary algorithms (MaOEAs), solutions survived to the next generation are individually selected which leads to the favorable quality upon the population composed of these selected solutions not necessarily to be gained. However, MaOEAs which are widely used in solving many-objective optimization problems (MaOPs) are considerably preferred due to their population-based nature. In this paper, a global view-based selection mechanism has been proposed to concern the quality of the selected solutions from a global prospective, which is capable of simultaneously facilitating the performance of the entire selected solutions. Indeed, the proposed selection mechanism is equivalent to solve a linear assignment problem whose cost matrix is constructed by the entries concurrently measuring the convergence and diversity of each solution. In addition, this design principle is also utilized for the mating selection to guarantee the convergence and diversity of the selected parents to generate promising offspring. As a case study, the proposed global view-based selection mechanism is integrated into NSGA-III (i.e., GS-NSGA-III). To validate the proposed selection mechanism, extensive experiments are performed by GS-NSGA-III against four state-of-the-art MaOEAs over 8-, 10-, and 15-objective DTLZ1-DTLZ7 test problems. The results measured by the selected performance metric reveal that GS-NSGA-III shows considerable competitiveness in addressing MaOPs.

It is a daunting challenge to balance the convergence and diversity of an approximate Pareto front in a many-objective optimization evolutionary algorithm. A novel algorithm, named many-objective particle swarm optimization with the two-stage strategy and parallel cell coordinate system (PCCS), is proposed in this paper to improve the comprehensive performance in terms of the convergence and diversity. In the proposed two-stage strategy, the convergence and diversity are separately emphasized at different stages by a single-objective optimizer and a many-objective optimizer, respectively. A PCCS is exploited to manage the diversity, such as maintaining a diverse archive, identifying the dominance resistant solutions, and selecting the diversified solutions. In addition, a leader group is used for selecting the global best solutions to balance the exploitation and exploration of a population. The experimental results illustrate that the proposed algorithm outperforms six chosen state-of-the-art designs in terms of the inverted generational distance and hypervolume over the DTLZ test suite.

Domain adaptation learning (DAL) investigates how to perform a task across different domains. In this paper, we present a kernelized local-global approach to solve domain adaptation problems. The basic idea of the proposed method is to consider the global and local information regarding the domains (e.g., maximum mean discrepancy and intraclass distance) and to convert the domain adaptation problem into a bi-object optimization problem via the kernel method. A solution for the optimization problem will help us identify a latent space in which the distributions of the different domains will be close to each other in the global sense, and the local properties of the labeled source samples will be preserved. Therefore, classic classification algorithms can be used to recognize unlabeled target domain data, which has a significant difference on the source samples. Based on the analysis, we validate the proposed algorithm using four different sources of data: synthetic, textual, object, and facial image. The experimental results indicate that the proposed method provides a reasonable means to improve DAL algorithms.

Selection strategy, including mating selection and environmental selection, is a key ingredient in the design of evolutionary multiobjective optimization algorithms. Existing approaches, which have shown competitive performance in low-dimensional multiobjective optimization problems with two or three objectives, often encounter considerable challenges in many-objective optimization, where the number of objectives exceeds 3. This paper first provides a comprehensive analysis on the selection strategies in the current evolutionary many-objective optimization algorithms. Afterward, we propose a coordinated selection strategy to improve the performance of evolutionary algorithms in many-objective optimization. This selection strategy considers three crucial factors: 1) the new mating selection criterion considers both the quality of each selected parent and the effectiveness of the combination of selected parents; 2) the new environmental selection criterion directly focuses on the performance of the whole population rather than single individual alone; and 3) both selection steps are complement to each other and the coordination between them in the evolutionary process can achieve a better performance than each of them used individually. Furthermore, in order to handle the curse of dimensionality in many-objective optimization problems, a new convergence measure by distance and a new diversity measure by angle are developed in both selection steps. Experimental results on both DTLZ and WFG benchmark functions demonstrate the superiority of the proposed algorithm in comparison with six state-of-the-art designs in terms of both solution quality and computational efficiency.

In a high-dimensional objective space, visualization of population in an approximate Pareto front is crucial to decision making process. By directly observing the performance of each solution, the trade-off between objectives, and distribution of approximate front, the decision maker can effectively decide which solution should be chosen from. Furthermore, visualization throughout the evolution process can also be exploited in designing effective many-objective evolutionary algorithms. Recently, a new visualization approach was developed by constructing a mapping from a high dimensional objective space into a two dimensional polar coordinate system, where a group of predefined direction vectors divide the whole space into a number of sub regions and each individual is associated with one weight vector. This method can be scalable to any dimensions, and simultaneously deal with a large number of individuals and multiple Pareto fronts for the purpose of visual comparison. It faithfully preserves shape, location, range, and distribution of Pareto front. However, distributions of Pareto front within each subregion and relations between different sub regions cannot be observed by this method. In this paper, in order to overcome this deficiency, we incorporate a modified multi-dimensional scaling (MDS) approach into this method. Experimental results show that the modified MDS is a suitable complementary to the existing method. Furthermore, the new design combined with existing visualization approach provides a comprehensive mean to visualize all important information in a many-objective optimization problem.

Real world optimization problems always possess multiple objectives which are conflict in nature. Multi-objective evolutionary algorithms (MOEAs), which provide a group of solutions in region of Pareto front, increasingly draw researchers attention for their excellent performance. In this regard, solutions with a wide diversity would be more favored as they give decision makers more choices to evaluate upon their problems. Based on the insight of investigating the evolution, the Pareto front often lies in a manifold space, not Euclidian space. However, most MOEAs utilize Euclidian distance as a sole mechanism to keep a wide range of diversity for solutions, which is not suitable somewhat from this aspect. To this end, manifold dimension reduction algorithm which has the ability to map solutions in the same front of objective space into Euclidian space is adapted in further. And then, general clustering algorithm are utilized. At the end, we use this technology to replace the crowding distance technology in NSGA-II to choose individuals when there is not enough slots in mating selection process. Based on a range of experiments over benchmark problems against state-of-the-art, it is fully expected benefit of performance improvement will be more significant when applied in many objectives optimization problems. This will be pursuit in our future study.

Most real-world optimization problems are dynamic in nature. To deal with the optimization problems in a dynamic environment, a general framework inspired from immune system is proposed. Three types of subpopulations mimicking the states of B-cells are introduced. Among them, innate population globally explores the promising area, while adaptive population locally searches the optima, and the memory population reuses the history information. Moreover, activation rule is brought to transfer the global search to local search and suppression works for eliminating the redundancy. Experimental simulation shows that the proposed algorithm is competitive comparing with the state-of-the-art designs in the tested dynamic environments modeled by the most commonly used test function-moving peaks benchmark.

When solving constrained optimization problems by evolutionary algorithms, an important issue is how to balance constraints and objective function. This paper presents a new method to address the above issue. In our method, after generating an offspring for each parent in the population by making use of differential evolution (DE), the well-known feasibility rule is used to compare the offspring and its parent. Since the feasibility rule prefers constraints to objective function, the objective function information has been exploited as follows: if the offspring cannot survive into the next generation and if the objective function value of the offspring is better than that of the parent, then the offspring is stored into a predefined archive. Subsequently, the individuals in the archive are used to replace some individuals in the population according to a replacement mechanism. Moreover, a mutation strategy is proposed to help the population jump out of a local optimum in the infeasible region. Note that, in the replacement mechanism and the mutation strategy, the comparison of individuals is based on objective function. In addition, the information of objective function has also been utilized to generate offspring in DE. By the above processes, this paper achieves an effective balance between constraints and objective function in constrained evolutionary optimization. The performance of our method has been tested on two sets of benchmark test functions, namely, 24 test functions at IEEE CEC2006 and 18 test functions with 10-D and 30-D at IEEE CEC2010. The experimental results have demonstrated that our method shows better or at least competitive performance against other state-of-the-art methods. Furthermore, the advantage of our method increases with the increase of the number of decision variables.

A minimum Manhattan distance (MMD) approach to multiple criteria decision making in multiobjective optimization problems (MOPs) is proposed. The approach selects the final solution corresponding with a vector that has the MMD from a normalized ideal vector. This procedure is equivalent to the knee selection described by a divide and conquer approach that involves iterations of pairwise comparisons. Being able to systematically assign weighting coefficients to multiple criteria, the MMD approach is equivalent to a weighted-sum (WS) approach. Because of the equivalence, the MMD approach possesses rich geometric interpretations that are considered essential in the field of evolutionary computation. The MMD approach is elegant because all evaluations can be performed by efficient matrix calculations without iterations of comparisons. While the WS approach may encounter an indeterminate situation in which a few solutions yield almost the same WS, the MMD approach is able to determine the final solution discriminately. Since existing multiobjective evolutionary algorithms aim for a posteriori decision making, i.e., determining the final solution after a set of Pareto optimal solutions is available, the proposed MMD approach can be combined with them to form a powerful solution method of solving MOPs. Furthermore, the approach enables scalable definitions of the knee and knee solutions.

Evolutionary algorithms have been successfully applied for exploring both converged and diversified approximate Pareto-optimal fronts in multiobjective optimization problems, two- or three-objective in general. However, when solving problems with many objectives, nearly all algorithms perform poorly due to the loss of selection pressure in fitness evaluation. An extremely large objective space could inadvertently deteriorate the effect of an evolutionary operator. In this paper, we propose a new approach to directly handle the challenges to solve many-objective optimization problems (MaOPs). This novel design includes two stages: first, the whole population quickly approaches a small number of “target” points near the true Pareto front; then, the proposed diversity improvement strategy is applied to facilitate these individuals to spread and well distribute. As a case study, the proposed algorithm based on this design is compared with five state-of-the-art algorithms. Experimental results show that the proposed method exhibits improved performance in both convergence and diversity for solving MaOPs.

Visualization of population in a high-dimensional objective space throughout the evolution process presents an attractive feature that could be well exploited in designing many-objective evolutionary algorithms (MaOEAs). In this paper, a new visualization method is proposed. It maps individuals from a high-dimensional objective space into a 2-D polar coordinate plot while preserving Pareto dominance relationship, retaining shape and location of the Pareto front, and maintaining distribution of individuals. From it, a decision-maker can observe the evolution process, estimate location, range, and distribution of Pareto front, assess quality of the approximated front and tradeoff between objectives, and easily select preferred solutions. Furthermore, its applications can be scalable to any dimensions, handle a large number of individuals on front, and simultaneously visualize multiple fronts for comparison. Based on this visualization tool, a performance metric, named polar-metric, is designed. The convergence of the approximate front is measured by radial values of all population members on that front. Meanwhile, the diversity performance is mainly determined by niche count of each subregion in a high-dimensional objective space. Experimental results show that it can provide a comprehensive and reliable comparison among MaOEAs.

Inspired by the fact that in modern society, team cooperation and the division of labor play important roles in accomplishing a task, this paper proposes a dual-population differential evolution (DPDE) with coevolution for constrained optimization problems (COPs). The COP is treated as a bi-objective optimization problem where the first objective is the actual cost or reward function to be optimized, while the second objective accounts for the degree of constraint violations. At each generation during the evolution process, the whole population is divided into two based on the solution's feasibility to treat the both objectives separately. Each subpopulation focuses on only optimizing the corresponding objective which leads to a clear division of work. Furthermore, DPDE makes use of an information-sharing strategy to exchange search information between the different subpopulations similar to the team cooperation. The comparison of the proposed method on a number of benchmark functions with selected state-of-the-art constraint-handling algorithms indicates that the proposed technique performs competitively and effectively.

In the field of evolutionary computation, there has been a growing interest in applying evolutionary algorithms to solve multimodal optimization problems (MMOPs). Due to the fact that an MMOP involves multiple optimal solutions, many niching methods have been suggested and incorporated into evolutionary algorithms for locating such optimal solutions in a single run. In this paper, we propose a novel transformation technique based on multiobjective optimization for MMOPs, called MOMMOP. MOMMOP transforms an MMOP into a multiobjective optimization problem with two conflicting objectives. After the above transformation, all the optimal solutions of an MMOP become the Pareto optimal solutions of the transformed problem. Thus, multiobjective evolutionary algorithms can be readily applied to find a set of representative Pareto optimal solutions of the transformed problem, and as a result, multiple optimal solutions of the original MMOP could also be simultaneously located in a single run. In principle, MOMMOP is an implicit niching method. In this paper, we also discuss two issues in MOMMOP and introduce two new comparison criteria. MOMMOP has been used to solve 20 multimodal benchmark test functions, after combining with nondominated sorting and differential evolution. Systematic experiments have indicated that MOMMOP outperforms a number of methods for multimodal optimization, including four recent methods at the 2013 IEEE Congress on Evolutionary Computation, four state-of-the-art single-objective optimization based methods, and two well-known multiobjective optimization based approaches.

Multiobjective evolutionary algorithms have become prevalent and efficient approaches for solving multiobjective optimization problems. However, their performances deteriorate severely when handling many-objective optimization problems (MaOPs) due to the loss of selection pressure to drive the search toward the Pareto front and the ineffective design in diversity maintenance mechanism. This paper proposes a many-objective evolutionary algorithm (MaOEA) based on directional diversity (DD) and favorable convergence (FC). The main features are the enhancement of two selection schemes to facilitate both convergence and diversity. In the algorithm, a mating selection based on FC is applied to strengthen selection pressure while an environmental selection based on DD and FC is designed to balance diversity and convergence. The proposed algorithm is tested on 64 instances of 16 MaOPs with diverse characteristics and compared with seven state-of-the-art algorithms. Experimental results show that the proposed MaOEA performs competitively with respect to chosen state-of-the-art designs.

Managing convergence and diversity is essential in the design of multiobjective particle swarm optimization (MOPSO) in search of an accurate and well distributed approximation of the true Pareto-optimal front. Largely due to its fast convergence, particle swarm optimization incurs a rapid loss of diversity during the evolutionary process. Many mechanisms have been proposed in existing MOPSOs in terms of leader selection, archive maintenance, and perturbation to tackle this deficiency. However, few MOPSOs are designed to dynamically adjust the balance in exploration and exploitation according to the feedback information detected from the evolutionary environment. In this paper, a novel method, named parallel cell coordinate system (PCCS), is proposed to assess the evolutionary environment including density, rank, and diversity indicators based on the measurements of parallel cell distance, potential, and distribution entropy, respectively. Based on PCCS, strategies proposed for selecting global best and personal best, maintaining archive, adjusting flight parameters, and perturbing stagnation are integrated into a self-adaptive MOPSO (pccsAMOPSO). The comparative experimental results show that the proposed pccsAMOPSO outperforms the other eight state-of-the-art competitors on ZDT and DTLZ test suites in terms of the chosen performance metrics. An additional experiment for density estimation in MOPSO illustrates that the performance of PCCS is superior to that of adaptive grid and crowding distance in terms of convergence and diversity.

Parallel Cell Coordinate System (PCCS) was proposed to evaluate the individual fitness in an archive and access the population progress in the evolutionary environment. In a Many-objective Optimization Problem (MaOP), it is much harder to tradeoff the convergence and diversity than in a Multiobjective Optimization Problem. To more effectively tackle the MaOPs, the PCCS and the aggregation-based approach are integrated into a Many-objective Optimization Particle Swarm Optimization (MaOPSO). In this paper, the sensitivity of PCCS is examined with respect to the number of objectives and the maximum size of an archive. The experimental results indicate that the MaOPSO performs better than MOEA/D in terms of IGD and HV metrics on the WFG test suit, and PCCS is not sensitive to the number of objectives and the maximum size of an archive.

Differential evolution is regarded as one of the most efficient evolutionary algorithms to tackle multiobjective optimization problems. The key to success of any multiobjective evolutionary algorithms (MOEAs) is maintaining a delicate balance between exploration and exploitation throughout the evolution process. In this paper, we propose a Fuzzy-based Multiobjective Differential Evolution (FMDE) that uses performance metrics, specifically, hypervolume, spacing, and maximum spread, to measure the state of the evolution process. We apply the inference rules to these metrics in order to dynamically adjust the associated control parameters of a chosen mutation strategy used in this algorithm. One parameter controls the degree of greedy or exploitation, while another regulates the degree of diversity or exploration of the reproduction phase. Therefore, we can appropriately adjust the degree of exploration and exploitation through performance feedback. The performance of FMDE is evaluated on well-known ZDT and DTLZ test suites in addition two representative functions in WFG. The results show that the proposed algorithm is competitive with respect to chosen state-of-the-art MOEAs.

The performances of Pareto-based multi-objective evolutionary algorithms deteriorate severely when solving many-objective optimization problems (MaOPs) mainly due to the loss of selection pressure and inappropriate design in diversity maintenance mechanism. To handling MaOPs, this paper proposes a many-objective evolutionary algorithm (MaOEA) based on directional diversity and favorable convergence (MaOEA-DDFC). In the algorithm, the mating selection based on favorable convergence and Pareto-dominance is applied to strengthen the selection pressure while an environmental selection considering directional diversity and favorable convergence is designed in order to make a good trade-off between diversity and convergence. To validate algorithm performance, seven DTLZ problems with 3, 5, 7 and 10 objectives are tested. Experimental results show that the proposed MaOEA-DDFC performs better than five state-of-the-art MaOEAs in terms of inverted generational distance and hypervolume indicators.

A novel differential evolution with coevolution (CDE) for constrained optimization problems (COPs) is proposed in this paper. The proposed CDE treats a COP as a bi-objective optimization problem where the first objective remains the same objective function itself while the second objective accounts for the degree of constraint violations. It adopts a simple and straightforward idea to solve COPs by dividing the whole population into dual subpopulations based on their feasibility. Each subpopulation corresponds with only one objective alone. An information-sharing strategy is incorporated to exchange search information between the different subpopulations. The comparison of the proposed approach with selected state-of-the-art constraint handling techniques demonstrates on a number of benchmark problems that the proposed framework performs competitively and effectively.

Decomposition-Based Multi-Objective Evolutionary Algorithms (DBMOEA), such as Multiple Single Objective Pareto Sampling (MSOPS) and Multiobjective Evolutionary Algorithm based on Decomposition (MOEA/D), have been successfully applied in finding Pareto-optimal fronts in Multiobjective Optimization Problems (MOPs), two or three-objective in general. DBMOEA decomposes one MOP into multiple Single-objective Optimization Problems (SOPs) where the convergence of approximated front is facilitated by finding the optimal solution of each SOP and its diversity is preserved by a group of well distributed SOPs. However, when solving problems with many objectives, one single solution can be the optimal solution of multiple SOPs which inadvertently leads to a severe loss of population diversity. In this paper, we propose a new diversity improvement method incorporated into a modified DBMOEA to directly handle this challenge. The design includes two steps. First, a few number of weight vectors guide the whole population towards a small number of solutions nearby the true Pareto front. Afterwards, initialize a subpopulation around each solution and diversify them toward well distribution. As a case study, a new algorithm based on this design is compared with three state-of-the-art DBMOEAs, MOEA/D, MSOPS, and MO-NSGA-II. Experimental results show that the proposed methods exhibit better performance in both convergence and diversity than the chosen competitors for solving many-objective optimization problems.

Multimodal optimization is one of the most challenging tasks for optimization. It requires an algorithm to effectively locate multiple global and local optima, not just single optimum as in a single objective global optimization problem. To address this objective, this paper first investigates a cluster-based differential evolution (DE) for multimodal optimization problems. The clustering partition is used to divide the whole population into subpopulations so that different subpopulations can locate different optima. Furthermore, the self-adaptive parameter control is employed to enhance the search ability of DE. In this paper, the proposed multipopulation strategy and the self-adaptive parameter control technique are applied to two versions of DE, crowding DE (CDE) and species-based DE (SDE), which yield self-CCDE and self-CSDE, respectively. The new algorithms are tested on two different sets of benchmark functions and are compared with several state-of-the-art designs. The experiment results demonstrate the effectiveness and efficiency of the proposed multipopulation strategy and the self-adaptive parameter control technique. The proposed algorithms consistently rank top among all the competing state-of-the-art algorithms.

An artificial immune system inspired by the fundamental principle of the vertebrate immune system, for solving constrained optimization problems, is proposed. The analogy between the mechanism of biological immune response and constrained optimization formulation is drawn. Individuals in population are classified into feasible and infeasible groups according to their constraint violations that closely match with the two states, inactivated and activated, of B-cells in the immune response. Feasible group focuses on exploitation in the feasible areas through clonal selection, recombination, and hypermutation, while infeasible group facilitates exploration along the feasibility boundary via location update. Direction information is extracted to promote the interactions between these two groups. This approach is validated by the benchmark functions proposed most recently and compared with those of the state of the art from various branches of evolutionary computation paradigms. The performance achieved is considered fairly competitive and promising.

Evolutionary algorithms have been effectively used to solve multiobjective optimization problems with a small number of objectives, two or three in general. However, when problems with many objectives are encountered, nearly all algorithms perform poorly due to loss of selection pressure in fitness evaluation solely based upon the Pareto optimality principle. In this paper, we introduce a new fitness evaluation mechanism to continuously differentiate individuals into different degrees of optimality beyond the classification of the original Pareto dominance. The concept of fuzzy logic is adopted to define a fuzzy Pareto domination relation. As a case study, the fuzzy concept is incorporated into the designs of NSGA-II and SPEA2. Experimental results show that the proposed methods exhibit better performance in both convergence and diversity than the original ones for solving many-objective optimization problems.

Evolutionary algorithms have been successfully exploited to solve multiobjective optimization problems. In the literature, a heuristic approach is often taken. For a chosen benchmark problem with specific problem characteristics, the performance of multiobjective evolutionary algorithms (MOEAs) is evaluated via some heuristic chosen performance metrics. The conclusion is then drawn based on statistical findings given the preferable choices of performance metrics. The conclusion, if any, is often indecisive and reveals no insight pertaining to which specific problem characteristics the underlying MOEA could perform the best. In this paper, we introduce an ensemble method to compare MOEAs by combining a number of performance metrics using double elimination tournament selection. The double elimination design allows characteristically poor performance of a quality algorithm to still be able to win it all. Experimental results show that the proposed metric ensemble can provide a more comprehensive comparison among various MOEAs than what could be obtained from a single performance metric alone. The end result is a ranking order among all chosen MOEAs, but not quantifiable measures pertaining to the underlying MOEAs.

In this study, we have compared six state-of-the-art Multiobjective Evolutionary Algorithms (MOEAs) designed specifically for many-objective optimization problems under a number of carefully crafted benchmark problems. Using the performance metrics ensemble, we aim at providing a comprehensive measure and more importantly revealing insight pertaining to specific problem characteristics that the underlying MOEA could perform the best. The experimental results confirm the finding from the No Free Lunch theorem: any algorithm's elevated performance over one class of problems is exactly paid for in loss over another class. In addition, the experimental results show that the performance of MOEA to solve many-objective optimization problems depends on two distinct aspects: the ability of MOEA to tackle the specific characteristics of the problem and the ability of MOEA to handle high-dimensional objective space.

Leader selection and archive maintenance are the two key issues, which have an important impact on the performance of the obtained approximate Pareto front, to be tackled when extending Single-Objective Particle Swarm Optimization to Multi-Objective Particle Swarm Optimization (MOPSO). In this paper, a new method of density estimation is proposed for selecting leaders and maintaining archive in MOPSO. The density of a nondominated solution in archive is calculated according to the Parallel Cell Distance after the archive is mapped from Cartesian Coordinate System into Parallel Cell Coordinate System. A new MOPSO is proposed based on this method of density estimation for selecting leaders and maintaining archive to improve the performance of convergence and diversity. The experimental results show that the proposed algorithm is significantly superior to the five chosen state-of-the-art MOPSOs on 12 test problems in term of hypervolume performance indicator.

This paper presents an artificial immune system for solving dynamic optimization problems. For effectively solving optimization problems in dynamic environments, search population should be able to fast converge in each environment and redistribute when change occurs and subsequently track the change. This paper presents three modifications made to the basic artificial immune system to meet these requirements. The first one is the modified gradient-based mechanism called quasi-gradient which is beneficial to speed up the convergence of population. The second one is clustering strategy which is adopted to effectively distribute the population. Memory mechanism is the last one which takes advantage of the historical information from the last scenario preparing for the upcoming new environment. Applications on classic simple test-case generator and moving peak problem validate the performance of the proposed algorithm.

Evolutionary algorithms have been effectively used to solve multiobjective optimization problems with a small number of objectives, two or three in general. However, when encounter problems with many objectives (more than five), nearly all algorithms performs poorly because of loss of selection pressure in fitness evaluation solely based upon Pareto domination. In this paper, we introduce a new fitness evaluation mechanism to continuously differentiate solutions into different degrees of optimality beyond the classification of the original Pareto dominance. Here, the concept of fuzzy logic is adopted to define fuzzy-dominated relation. As a case study, the fuzzy concept is incorporated into the NSGA-II, instead of the original Pareto dominance principle. Experimental results show that the proposed method exhibits a better performance in both convergence and diversity than the original NSGA-II for solving many-objective optimization problems. More importantly, it enables a fast convergence process.

This paper proposes an artificial immune system based algorithm for solving constrained optimization problems, inspired by the principle of the vertebrate immune system. The analogy between the mechanism of vertebrate immune system and constrained optimization formulation is first given. The population is divided into two groups- feasible individuals and infeasible individuals. The infeasible individuals are viewed as the inactivated immune cells approaching the feasible regions by decreasing the constraint violations whereas the feasible individuals are treated as activated immune cells searching for the optima. The interaction between them through the extracted directional information is facilitated mimicking the functionality of T cells. This mechanism not only encourages infeasible individuals approaching feasibility regions, but facilitates exploring the boundary between the feasible and infeasible regions in which optima are often located. This approach is validated and performance is quantified by the benchmark functions used in related researches through statistical means with those of the state-of-the-art from various branches of evolutionary computation paradigms. The performance obtained is fairly competitive and in some cases even better.

Particle swarm optimization (PSO) has been recently adopted to solve constrained optimization problems. In this paper, a cultural-based constrained PSO is proposed to incorporate the information of the objective function and constraint violation into four sections of the belief space, specifically normative knowledge, spatial knowledge, situational knowledge, and temporal knowledge. The archived information facilitates communication among swarms in the population space and assists in selecting the leading particles in three different levels: personal, swarm, and global levels. Comprehensive comparison of the proposed heuristics over a number of benchmark problems with selected state-of-the-art constraint-handling techniques demonstrates that the proposed cultural framework helps the multiple-swarm PSO to perform competitively with respect to selected designs.

Many practical optimization problems are with the existence of uncertainties, among which a significant number belong to the dynamic optimization problem (DOP) category in which the fitness function changes through time. In this study, we propose the cultural based particle swarm optimization (PSO) to solve DOP problems. A cultural framework is introduced that incorporates the required information from the PSO into five sections of the belief space, namely situational knowledge, temporal knowledge, domain knowledge, normative knowledge and spatial knowledge. The stored information will be adopted to detect the changes in the environment and assists response to the change through a diversity based repulsion among particles and migration among swarms in the population space, also helps in selecting the leading particles in three different levels, personal, swarm and global level. Comparison of the proposed cultural based dynamic PSO demonstrates the better or equal performance with respect to other selected state-of-the-art dynamic PSO heuristics.

Evolutionary algorithms have been effectively exploited to solve multiobjective optimization problems. In literature, a heuristic approach is often taken. For a chosen benchmark problem, the performance of multiobjective evolutionary algorithms (MOEAs) is evaluated via some heuristic chosen performance metrics. The conclusion is then drawn based on statistical findings given the preferable choices of performance metrics. The conclusion, if any, is often indecisive and reveals no insight pertaining to specific problem characteristics that the underlying MOEA could perform the best. In this paper, we introduce an ensemble method to compare MOEAs by combining a number of performance metrics using double elimination tournament selection. Double elimination design allows characteristically poor performance of a quality algorithm under the special environment to still be able to win it all. Experimental results show that the proposed metrics ensemble can provide a more comprehensive comparison among various MOEAs than what could be obtained from single performance metric alone.

Multiobjective particle swarm optimization algorithms (MOPSO) have been broadly adopted to solve multiobjective optimization problems. In our previous work, a cultural MOPSO with dynamic flight parameters had been introduced in which some parameters had to be tuned. In this paper, the sensitivity analysis of the cultural MOPSO with respect to nine tuning parameters is investigated. The extensive study is performed on different sets of tuning parameters over six benchmark test problems. The additive binary epsilon indicator is adopted as a mean to compare every two algorithms. The graphical box plot comparisons demonstrate the robustness of cultural MOPSO.

Multiobjective particle swarm optimization (MOPSO) algorithms have been widely used to solve multiobjective optimization problems. Most MOPSOs use fixed momentum and acceleration for all particles throughout the evolutionary process. In this paper, we introduce a cultural framework to adapt the personalized flight parameters of the mutated particles in a MOPSO, namely momentum and personal and global accelerations, for each individual particle based upon various types of knowledge in "belief space," specifically situational, normative, and topographical knowledge. A comprehensive comparison of the proposed algorithm with chosen state-of-the-art MOPSOs on benchmark test functions shows that the movement of the individual particle using the adapted parameters assists the MOPSO to perform efficiently and effectively in exploring solutions close to the true Pareto front while exploiting a local search to attain diverse solutions.

In this study, a novel cultural-based constrained particle swarm optimization (CPSO) is proposed that incorporates the information of the objective function and constraints violation into four sections of the belief space, that is normative knowledge, spatial knowledge, situational knowledge, and temporal knowledge. The stored information facilitates the communication among swarms in the population space. It also assists in selecting the leading particles using a three-level PSO: personal, swarm and global levels. Simulation results of the proposed heuristics over a number of benchmark problems demonstrate that this novel cultural framework helps the multiple swarm CPSO to perform very efficiently.

This research exploits the fact embedded in immune system to communicate between innate immune response and adaptive immune response levels. We first sort infeasible antibodies based on their constraint violation values and feasible antibodies based on their objective function values. The next processes assesses in what directions individuals with high affinities should move to improve their objective function values and reduce constraint violations. This information is used to estimate the mutation direction to infeasible antibodies. This approach is validated and tested using benchmark functions used in related researches and the results obtained are compared with studies made in similar area. The performance acquired is competitive and in some cases even better than those of state-of-the-art. The algorithm is also designed to obtain feasible solutions in every run executed.

In this paper, the proposed constrained multiobejctive particle swarm optimization (MOPSO) adopts the multiobjective constraint handling framework and includes the following design features: An infeasible global best archive to guide the infeasible particles towards feasible region(s); procedures to update the personal best archive are designed to encourage finding feasible regions and convergence towards the Pareto front; acceleration constants in the particle swarm optimization equation are adjusted during the search process to encourage finding more feasible particles or to search for better solutions; and mutation operators are adopted to encourage global and local searches. The simulation results indicate that the proposed algorithm is highly competitive in solving the benchmark problems.

This paper emulates a biological notion in vaccines to promote exploration in the search space for solving multimodal function optimization problems using artificial immune systems (AISs). In this method, we first divide the decision space into equal subspaces. The vaccine is then randomly extracted from each subspace. A few of these vaccines, in the form of weakened antigens, are then injected into the algorithm to enhance the exploration of global and local optima. The goal of this process is to lead the antibodies to unexplored areas. Using this biologically motivated notion, we design the vaccine-enhanced AIS for multimodal function optimization, achieving promising performance.

Many real-world optimization problems have to be solved under the presence of uncertainties. A significant number of these uncertainty problems falls into the dynamic optimization category in which the fitness function varies through time. For this class of problems, an evolutionary algorithm is expected to perform satisfactorily in spite of different degrees and frequencies of change in the fitness landscape. In addition, the dynamic evolutionary algorithm should warrant an acceptable performance improvement to justify the additional computational cost. Effective reuse of previous evolutionary information is a must as it facilitates a faster convergence after a change has occurred. This paper proposes a new dynamic evolutionary algorithm that uses variable relocation to adapt already converged or currently evolving individuals to the new environmental condition. The proposed algorithm relocates those individuals based on their change in function value due to the change in the environment and the average sensitivities of their decision variables to the corresponding change in the objective space. The relocation occurs during the transient stage of the evolutionary process, and the algorithm reuses as much information as possible from the previous evolutionary history. As a result, the algorithm shows improved adaptation and convergence. The newly adapted population is shown to be fitter to the new environment than the original or most randomly generated population. The algorithm has been tested by several dynamic benchmark problems and has shown competitive results compared to some chosen state-of-the-art dynamic evolutionary approaches.

This paper proposes a constraint handling technique for multiobjective evolutionary algorithms based on an adaptive penalty function and a distance measure. These two functions vary dependent upon the objective function value and the sum of constraint violations of an individual. Through this design, the objective space is modified to account for the performance and constraint violation of each individual. The modified objective functions are used in the nondominance sorting to facilitate the search of optimal solutions not only in the feasible space but also in the infeasible regions. The search in the infeasible space is designed to exploit those individuals with better objective values and lower constraint violations. The number of feasible individuals in the population is used to guide the search process either toward finding more feasible solutions or favor in search for optimal solutions. The proposed method is simple to implement and does not need any parameter tuning. The constraint handling technique is tested on several constrained multiobjective optimization problems and has shown superior results compared to some chosen state-of-the-art designs.

We study 3-D retinal curvature estimation from multiple images that provides the fundamental geometry of the human retina and could be used for 3-D retina visualization and disease diagnosis purposes. An affine camera model is used for 3-D reconstruction due to its simplicity, linearity, and robustness. A major challenge is that a series of optics is involved in the retinal imaging process, including an actual fundus camera, a digital camera, and the optics of the human eye, all of which cause significant nonlinear distortions in retinal images. In this paper, we develop a new constrained optimization method that considers both the geometric shape of the human retina and nonlinear lens distortions. Moreover, we examine a variety of lens distortion models to approximate the optics of the human eye in order to create a smooth spherical surface for curvature estimation. The experimental results on both synthetic data and real retinal images validate the proposed algorithm.

This paper proposes an adaptive penalty function for solving constrained optimization problems using genetic algorithms. The proposed method aims to exploit infeasible individuals with low objective value and low constraint violation. The number of feasible individuals in the population is used to guide the search process either toward finding more feasible individuals or searching for the optimum solution. The proposed method is simple to implement and does not need any parameter tuning. The performance of the algorithm is tested on 22 benchmark functions in the literature. The results show that the proposed approach is able to find very good solutions comparable to the chosen state-of-the-art designs. Furthermore, it is able to find feasible solutions in every run for all of the benchmark functions tested.

A multiple-swarm multiobjective particle swarm optimization (PSO) algorithm, named dynamic multiple swarms in multiobjective PSO, is proposed in which the number of swarms is adaptively adjusted throughout the search process via the proposed dynamic swarm strategy. The strategy allocates an appropriate number of swarms as required to support convergence and diversity criteria among the swarms. Additional novel designs include a PSO updating mechanism to better manage the communication within a swarm and among swarms and an objective space compression and expansion strategy to progressively exploit the objective space during the search process. Comparative study shows that the performance of the proposed algorithm is competitive in comparison to the selected algorithms on standard benchmark problems. In particular, when dealing with test problems with multiple local Pareto fronts, the proposed algorithm is much less computationally demanding. Sensitivity analysis indicates that the proposed algorithm is insensitive to most of the user-specified design parameters.

Multiobjective particle swarm optimization algorithms (MOPSO) have been widely used to solve multiobjective optimization problems. Most of MOPSOs use fixed momentum and acceleration for all particles throughout the evolutionary process. In this paper, we introduce a cultural framework to adapt the flight parameters of the MOPSO namely momentum, personal, and global acceleration for each individual particle based upon the various types of knowledge in belief space, specifically situational knowledge, normative knowledge, and topographical knowledge. Movement of the particles using the adapted parameters helps the MOPSO to perform efficiently and effectively in multiobjective optimization.

Facial feature tracking for model-based coding has evolved over the past decades. Of particular interest is its application in very low bit rate coding in which optimization is used to analyze head and shoulder sequences. We present the results of a computational experiment in which we apply a combination of non-dominated sorting genetic algorithm (NSGA-II) and a deterministic search to find optimal facial animation parameters at many bandwidths, simultaneously. As objective functions are concerned, peak signal-to-noise ratio is chosen to be maximized while the total number of facial animation parameters is chosen to be minimized. Particularly, the algorithm is tested for efficiency and reliability. The results show that the overall methodology works effectively, but that a better error assessment function is needed.

In this paper, the improvement of two design components (swarm growing strategy and objective space compression and expansion strategy) from the existing multiple swarms MOPSO, namely DSMOPSO, is presented. In addition, sensitivity analysis is conducted to study the impact of the five tuning parameters on its performance through two performance metrics. Simulation results show the improved design is robust with respect to the tuning parameters.

This paper proposes a two-tiered, evolutionary architecture for computer based synthesis of textures. In this architecture, a traditional tree based texture generation system is controlled by a set of evolutionary agents. The main contribution of this work is that the user is able to choose the degree of interaction and control they exert over the system. Evolutionary agents are designed to contain information about desirable image features, and they evolve based on user feedback. The agents in turn control the main evolutionary engine for generating textures. This system allows the computer to continue working when the designer leaves without limiting the designerpsilas ability to control the texture generation process when they are available to interact with the system. An experimental implementation is developed to verify the utility of the proposed architecture for texture synthesis. Results show significant improvements in the average user ranking of the agents as the genetic algorithm progresses.

This paper proposes the use of vaccine to promote exploration in the search space for solving multimodal function optimization problems using artificial immune system. In this method, first we divide the decision space into equal subspaces. Vaccine is then extracted randomly from each subspace. A few of these antigens are then injected into the algorithm to enhance the exploration of global and local optima. The vaccine is introduced in the form of suppressed antibodies. The goal of this process is to allocate the available antibodies at unexplored areas. Using this biologically motivated notion we design the vaccine enhanced artificial immune system for multimodal function optimization.

This paper proposes a method to exchange information among multiple swarms in particle swarm optimization (PSO) to facilitate evolutionary search. The algorithm is developed to solve problems having landscapes with a large number of local optima. Each swarm maintains two sets of particles; one set includes the particles to be shared with other swarms, while the other involves the particles to be replaced by individuals from other swarms. The proposed algorithm also provides a new design to search for neighboring swarms in order to share common interests among the swarm's neighborhood. The particle's movement is according to one variation of PSO with three basic terms, each one to lead the particles toward the best particle in the swarm, in the neighborhood, and in the whole population. Demonstrated through a suite of benchmark test functions, the proposed algorithm shows competitive performance with improved convergence speed.

Diabetic retinopathy is a leading cause of blindness in developed countries. Diabetic patients can prevent severe visual loss by attending regular eye examinations and receiving timely treatments. In the United States, standard protocols have been developed and refined for years to provide better screening and evaluation procedures of the fundus images. Due to the emerging number of diabetic retinopathy cases, accurate and efficient evaluations of the fundus images have become a serious burden for the ophthalmologists or care providers. While diabetic retinopathy remains too complicated to call for an automatic diagnosis system, an efficient tool to facilitate the grading process with a limited number of personnel is in great demand. The current study is to develop a sorting system with a user-friendly interface, based upon the standardized early treatment diabetic retinopathy study (ETDRS) protocol, to assist the professional graders. The raw fundus images will be screened and assigned to different graders according to their skill levels and experiences. The developed hierarchical sorting process will greatly support the graders and enhance their efficiency and throughput. The proposed hybrid intelligent system with multilevel knowledge representation is used to construct this sorting system. A preliminary case study is conducted using only the features of the spot lesion group coupled with the ETDRS standard to demonstrate its feasibility and performance. The results obtained from the case study show a promising future.

The self-organizing map (SOM) is an efficient tool for visualizing high-dimensional data. In this paper, the clustering and visualization capabilities of the SOM, especially in the analysis of textual data, i.e., document collections, are reviewed and further developed. A novel clustering and visualization approach based on the SOM is proposed for the task of text mining. The proposed approach first transforms the document space into a multidimensional vector space by means of document encoding. Afterwards, a growing hierarchical SOM (GHSOM) is trained and used as a baseline structure to automatically produce maps with various levels of detail. Following the GHSOM training, the new projection method, namely the ranked centroid projection (RCP), is applied to project the input vectors to a hierarchy of 2-D output maps. The RCP is used as a data analysis tool as well as a direct interface to the data. In a set of simulations, the proposed approach is applied to an illustrative data set and two real-world scientific document collections to demonstrate its applicability.

Recently, various multiobjective particle swarm optimization (MOPSO) algorithms have been developed to efficiently and effectively solve multiobjective optimization problems. However, the existing MOPSO designs generally adopt a notion to "estimate" a fixed population size sufficiently to explore the search space without incurring excessive computational complexity. To address the issue, this paper proposes the integration of a dynamic population strategy within the multiple-swarm MOPSO. The proposed algorithm is named dynamic population multiple-swarm MOPSO. An additional feature, adaptive local archives, is designed to improve the diversity within each swarm. Performance metrics and benchmark test functions are used to examine the performance of the proposed algorithm compared with that of five selected MOPSOs and two selected multiobjective evolutionary algorithms. In addition, the computational cost of the proposed algorithm is quantified and compared with that of the selected MOPSOs. The proposed algorithm shows competitive results with improved diversity and convergence and demands less computational cost.

This study examines the optimization of the Job Shop Scheduling Problem (JSP) by a search space division scheme and use of the meta-heuristic method of Particle Swarm Optimization (PSO) to solve it. The Job Shop Scheduling Problem (JSP) is a well known huge combinatorial problem from the field of Deterministic Scheduling. It is considered the one of the hardest in the class of NP-Hard problems. The objective is to optimally schedule a finite number of operations to a finite number of resources while complying with ordering constraints. The Particle Swarm Optimization Algorithm (PSO) is a new meta heuristic optimization method modeled after the behavior of a flock of birds in flight. "Particles" are initialized in the search space of a particular problem by assigning them a position, which represents a solution to the objective function, and a velocity. They "fly" through the search space with out direct control, but are given both a cognitive personal component and a global or social component of the best positions (thereby solutions) in space. The PSO algorithm is considered a very fast algorithm and is emerging as a widely studied widely used algorithm for optimization problems. The proposed method uses this meta heuristic to solve the JSP by assigning each machine in a JSP an independent swarm of particles.

In this paper, a multiple swarms MOPSO (called DSMOPSO) in which the number of swarms is dynamically adjusted is proposed to solve for multiobjective optimization. Three novel ideas are introduced to DSMOPSO: the dynamic swarm strategy to allocate an appropriate number of swarms as needed and justified, the modified PSO update mechanism to better manage the convergence and communication among and within swarms, and objective space compression and expansion strategy to progressively exploit the objective space during different stages of search process. Compared with some state of-the-art designs, the proposed algorithm shows competitive results in producing well extended and near optimum Pareto fronts.

This paper introduces a new constraint handling technique for multi-objective evolutionary algorithms based on adaptive penalty functions and distance measures of an individual. These two values are used to modify the objective space. The modified objective functions are used in the non- dominance sorting so that the algorithm evolves feasible optimal solutions not only from the feasible space but also from the infeasible space. The search in the infeasible space is designed to encourage those individuals with better objective value and low constraint violation. The number of feasible individuals in the population is used to guide the search process either toward finding more feasible solutions or toward finding optimum solutions. The proposed method is simple to implement and does not need any parameter tuning. The constraint handling technique was tested on several constrained multi-objective problems and has shown superior results.

Many real-world optimization problems have to be performed under the presence of various uncertainties. A significant number of these uncertainty problems fall into the dynamic optimization category. For this class of problems, an evolutionary algorithm is expected to perform well under different levels and frequencies of change in the landscape. In addition, the dynamic evolutionary algorithm should warrant an acceptable performance improvement to justify the additional computational cost. Effective reuse of previous evolutionary information is a must as it facilitates a faster convergence after a change has occurred. This paper introduces a new dynamic evolutionary algorithm that uses variable relocation vectors to adapt already converged or currently evolving individuals to the new environmental condition. The proposed algorithm relocates those individuals based on their change in functional value due to the change in the environment and the average sensitivities of their decision variables to the corresponding change in the objective space. The relocated population is shown to be better fit to the new environment than the original or any other randomly generated population. The algorithm has been tested for several dynamic benchmark problems and has shown better results.

In order to effectively restrain inter-area oscillations in power systems, a local measurement based neural excitation controller is proposed to generate global stable signal. This is to replace the global measurement based power system stabilizer (GPSS). The proposed neural controller is constructed by two recurrent neural networks: a recurrent neural identifier (RNID) and a recurrent neural controller (RNCT). Non-measurable global dynamics in large-scale multi-machine power systems is estimated by the RNID and is provided to RNCT in order to generate global stable signals for a higher hierarchy of supplementary excitation control. Both RNID and RNCT are trained offline first to approximate the function of GPSS before online application. Simulation results based on Kundur's 2-area 4-machine power system model proved the effectiveness of the proposed local signal based neural identifier and controller in damping inter-area oscillations.

Most existing multiobjective particle swarm optimization (MOPSO) designs generally “ estimate” a fixed population size sufficiently to explore the search space without incurring excessive computational complexity. In this paper, we propose a MOPSO design to solve multiobjective optimization problems, known as the dynamic population multiple-swarm MOPSO (DMOPSO). The proposed algorithm incorporates strategies to adjust the population size to enhance exploration capability. An additional feature, adaptive local archive, is designed to improve the diversity within each swarm. Compared with some state-of-the-art MOPSO algorithms, the proposed algorithm shows competitive results with improved diversity and convergence.

A new threshold time series model is proposed, whose submodels are extended from AR to SARIMA and whose domains having thresholds are extended to two. By these two extensions, the newly proposed models would offer more flexibility to piecewisely approximate nonstationary time series by a finite number of local stationary models. Genetic algorithm is applied to simultaneously search for appropriate model structures, estimate the optimal model coefficients, as well as to partition space by finding appropriate thresholds. The resulting model is applied to a synthetic multi-frequency sine wave and two financial time series with improved modeling quality. The proposed model is also applied for seismogram analysis to recognize earthquake wave pattern in order to locate arrival time of different waves.

This paper proposes a self adaptive penalty function for solving constrained optimization problems using genetic algorithms. In the proposed method, a new fitness value, called distance value, in the normalized fitness-constraint violation space, and two penalty values are applied to infeasible individuals so that the algorithm would be able to identify the best infeasible individuals in the current population. The method aims to encourage infeasible individuals with low objective function value and low constraint violation. The number of feasible individuals in the population is used to guide the search process either toward finding more feasible solutions or toward finding the optimum solution. The proposed method is simple to implement and does not need parameter tuning. The performance of the algorithm is tested on 13 benchmark functions in the literature. The results show that the approach is able to find very good solutions comparable to other state-of-the-art designs. Furthermore, it is able to find feasible solutions in every run for all of the benchmark functions.

This paper proposes a method to exchange information among multiple swarms in particle swarm optimization. The provided algorithm is developed to solve problems that have landscapes with a high number of local optima. Each swarm provides two sets of particles; one set is the particles to be sent to another swarm, while the other set is the particles to be replaced by individuals from other swarms. Proposed algorithm also provides a new paradigm to search for neighboring swarms in order to share common interests in the swarm's neighborhood. The particle's movement is according to one variation of PSO with three basic terms, each one to lead the particles toward the best particle in the swarm, in the neighborhood, and in the whole population. Demonstrated through a suite of benchmark test functions, the proposed algorithm is shown competitive performance with improved convergence speed.

In this paper, the clustering and visualization capabilities of the SOM, specifically tailored for the analysis of textual data, are reviewed and further developed. A novel clustering and visualization approach is proposed for the task of textual data mining. The proposed approach first transforms the document space into a multi-dimensional vector space by means of citation patterns. An intuitive and effective projection method, namely the ranked centroid projection (RCP), is then applied in conjunction with a dynamic SOM model, the growing hierarchical self-organizing map, which automatically produces document maps with various levels of details. The RCP is used both as a data analysis tool as well as a direct interface to the data. We also extend the RCP to address the problem of the incremental clustering of dynamic document collections. In a set of simulations, the proposed approach is applied to a synthetic data set and two real-world scientific document collections, to demonstrate its applicability.

An integrated fault tolerant control solution calls for a nonlinear adaptive controller with universal approximation capability and guaranteed stability. To fulfill this requirement, we propose the use of neural networks trained online under a globalized dual heuristic programming architecture supervised by a decision logic capable of identifying controller malfunctions in early stages and providing new avenues with greater probability of convergence using information from a dynamic model bank. The classification and distinction of controller malfunctions and of the faults in the system is achieved through three independent quality indexes. Proof-of- the-concept simulations of nonlinear plants demonstrate the approach legitimacy.

The self-organizing map (SOM) is an efficient tool for visualizing high-dimensional data as it performs a topology-presenting projection of the input space on a low-dimensional grid. To utilize the information provided by the SOM and obtain an approximation of the data structure, a separate data projection method is usually needed. However, most of the SOM projection methods are computationally expensive when the size of the data set becomes large. In this paper, we present an intuitive and effective SOM projection method with comparatively low computational complexity for the purpose of cluster visualization. This method maps data vectors on the output space based on their responses to different prototype vectors. High-resolution maps can be obtained with a relatively small network size. The proposed method is demonstrated using both an artificial and a real world data set.

An advanced reconfigurable controller enhanced by a multiple model architecture is proposed as a tool to achieve fault tolerance in complex nonlinear systems. The most complete adaptive critic design, globalized dual heuristic programming (GDHP), constitutes a highly flexible nonlinear adaptive controller responsible for the generation of new control solutions for novel plant dynamics introduced by unknown faults. The main contribution of the presented work focuses on a novel fault tolerant control supervisor. Working on a higher hierarchical level, the proposed supervisor makes use of two quality indices to perform fault detection, identification and isolation based on the knowledge stored in a dynamic model bank (DMB). In the event of abrupt known faults, such knowledge is then used to greatly reduce the reconfiguration time of the GDHP controller. The synergy of the proposed supervisor and GDHP goes beyond, as solutions designed by the controller to previously unknown faults are autonomously added to the model bank. The fine interrelations between the algorithm's subsystems and its advanced capabilities are illustrated through extensive numerical simulations of a single-input single-output (SISO) linear system and of a multiple-input multiple-output (MIMO) nonlinear system, both subject to a series of fault scenarios involving expected and unexpected, abrupt and incipient faults. Note to Practitioners-The raising complexity of physical plants and control missions inevitably leads to increasing occurrence, diversity and severity of faults. Take automated production process as an example, the extent of time a plant is capable of maintaining acceptable performance levels is now considered to be the single factor with the highest impact on profitability. For a growing number of plants, it has become impractical to list all possible fault scenarios in order to take the necessary steps to assure continuous healthy operation. Therefore, it is now essential to have a control algorithm dedicated to the provision of tailored control solutions capable of maintaining stability and as much performance as possible during the occurrence of faults. This is the goal of fault tolerant control. In the presented paper, the most complete adaptive critic design, globalized dual heuristic programming (GDHP), is responsible for the generation of new control solutions for novel plant dynamics introduced by faults unknown at design time. A highly flexible nonlinear adaptive controller, GDHP is capable of dealing with both abrupt and incipient (gradually changing dynamics) faults. Working on a higher hierarchical level, a novel fault supervisor is introduced which makes use of two quality indices to perform fault detection, identification and isolation based on the knowledge stored in a dynamic model bank (DMB). In the event of abrupt known faults, such knowledge is then used to greatly reduce the convergence time of the GDHP controller. The synergy of the proposed supervisor and GDHP goes further, as solutions designed by the controller to previously unknown faults are autonomously added to the model bank in fact allowing the supervisor to learn new fault scenarios and their solutions as they occur. The fine interrelations between the algorithm's subsystems and its advanced capabilities are illustrated through extensive numerical simulations.

In this paper, we propose a generic, two-phase framework for solving constrained optimization problems using genetic algorithms. In the first phase of the algorithm, the objective function is completely disregarded and the constrained optimization problem is treated as a constraint satisfaction problem. The genetic search is directed toward minimizing the constraint violation of the solutions and eventually finding a feasible solution. A linear rank-based approach is used to assign fitness values to the individuals. The solution with the least constraint violation is archived as the elite solution in the population. In the second phase, the simultaneous optimization of the objective function and the satisfaction of the constraints are treated as a biobjective optimization problem. We elaborate on how the constrained optimization problem requires a balance of exploration and exploitation under different problem scenarios and come to the conclusion that a nondominated ranking between the individuals will help the algorithm explore further, while the elitist scheme will facilitate in exploitation. We analyze the proposed algorithm under different problem scenarios using Test Case Generator-2 and demonstrate the proposed algorithm's capability to perform well independent of various problem characteristics. In addition, the proposed algorithm performs competitively with the state-of-the-art constraint optimization algorithms on 11 test cases which were widely studied benchmark functions in literature.

N-partite networks are natural representations of complex multientity databases. However, processing these networks can be a highly memory and computation-intensive task, especially when positive correlation exists between the degrees of vertices from different partitions. In order to improve the scalability of this process, this paper proposes two algorithms that make use of sampling for obtaining less expensive approximate results. The first algorithm is optimal for obtaining homogeneous discovery rates with a low memory requirement, but can be very slow in cases where the combined branching factor of these networks is too large. A second algorithm that incorporates concepts from evolutionary computation aims toward dealing with this slow convergence in the case when it is more interesting to increase approximation convergence speed of elements with high feature values. This algorithm makes use of the positive correlation between "local" branching factors and the feature values. Two applications examples are demonstrated in searching for most influential authors in collections of journal articles and in analyzing most active earthquake regions from a collection of earthquake events.

This paper presents an optimization strategy based on the social algorithm and collective behaviors. The new algorithm proposed incorporates the information of the individuals within the society introduced as their talent and the collective behavior of the society in the civilization called the liberty rate. The algorithm has been demonstrated on two benchmark problems and shown promising results for further investigation.

The growth of use of graph-structured databases modeled on n-partite graphs has increased the ability to generate more flexible databases. However, the calculation of certain features in these databases may be highly resource-consuming. This work proposes a method for approximating these features by sampling. A discussion of the difficulty of sampling in n-partite graphs is made and an evolutionary algorithm-based method is presented that uses the information from a smaller subset of the graph to infer the amount of sampling needed for the rest of the graph. Experimental results are shown on a publications database on Anthrax for finding the most important authors.

In this paper we propose a novel approach for solving constrained optimization problems using genetic algorithms. The main emphasis of this algorithm is to be problem independent and to produce consistent results in terms of the quality of feasible solutions. The basic characteristic of this algorithm is the complete ignorance of the objective function till at least one feasible solution is found. The elitist scheme is used to assure consistent results and to help guide the stochastic search to the more fruitful regions of the parameter space. We have used rank based fitness assignment and have experimented with two ranking schemes. We have developed an empirical analysis and supporting experimental comparisons to favor one ranking scheme over the other. Irrespective of the ranking scheme used, our algorithm has performed well providing at least one feasible solution for every run of the algorithm and producing results that are comparable to the best published before.

This work details the development of a hybrid evolutionary algorithm for solving the traveling salesman problem (TSP). The strategy of the algorithm is to complement and extend the successful results of a genetic algorithm (GA) using a distance preserving crossover (DPX) by incorporating memory in the form of ant pheromone during the city selection process. The synergistic combination of the DPX-GA with city selection based on probability determined by both distance and previous success incorporates additional information into the search mechanism. This combination into a hybrid GA facilitates finding quality solutions for TSP problems with lower computation complexity. This study represents a preliminary investigation with direct comparison to show the feasibility and promise of this hybrid approach.

Autonomous temporal linguistic rule extraction is an application of growing interest for its relevance to both decision support systems and fuzzy controllers. In the presented work, rules are evaluated using three qualitative metrics based on their representation on the truth space diagram. Each metric is then treated as a conflicting optimization goal and multiple objective evolutionary algorithm is used to obtain a set of optimal non-dominant rules. Novel techniques for data pre-processing and rule set post-processing are designed. Data collected from a simulated hot and cold water mixer is used to validate the proposed procedure.

A complete fault tolerant control solution calls for a nonlinear adaptive controller with universal approximation capability and guaranteed stability. To fulfill this requirement, we propose the use of neural networks trained online under a globalized dual heuristic programming architecture supervised by a decision logic capable of identifying controller malfunctions in the early stages and providing new avenues with greater probability of convergence using information from a dynamic model bank. The classification and distinction of controller malfunctions and of the faults in the system are achieved through three independent quality indexes. Proof-of-the-concept simulations of nonlinear plants demonstrate the approach legitimacy.

The Self-Organizing Map (SOM) is an efficient tool for visualizing high-dimensional data. In this paper, an intuitive and effective SOM projection method is proposed for mapping high-dimensional data onto the two-dimensional SOM structure with a growing self-organizing map. In the learning phase, a growing SOM is trained and the growing cell structure is used as the baseline framework. After the learning phase, the new projection method is used to map the input vector so that the input data is mapped to the structure of the SOM without having to plot the weight values, resulting in easy visualization of the data. The projection method is demonstrated on two data sets with promising results and a significantly reduced network size.

In this paper, we propose a genetic algorithm based design procedure for a radial-basis function neural network. A Hierarchical Rank Density Genetic Algorithm (HRDGA) is used to evolve the neural network's topology and parameters simultaneously. Compared with traditional genetic algorithm based designs for neural networks, the hierarchical approach addresses several deficiencies highlighted in literature. In addition, the rank-density based fitness assignment technique is used to optimize the performance and topology of the evolved neural network to deal with the confliction between the training performance and network complexity. Instead of producing a single optimal solution, HRDGA provides a set of near-optimal neural networks to the designers so that they can have more flexibility for the final decision-making based on certain preferences. In terms of searching for a near-complete set of candidate networks with high performances, the networks designed by the proposed algorithm prove to be competitive, or even superior, to three other traditional radial-basis function networks for predicting Mackey–Glass chaotic time series.

A heuristic, automatic facial feature extraction approach is presented in this paper. The method is based on the edge density distribution of the image, containing non-occluded, nearly frontal view of one or more faces. In the preprocessing stage, a face is approximated to an ellipse, and genetic algorithm is applied to search for the best ellipse region match. In the feature extraction stage, genetic algorithm is applied to extract the facial features, such as the eyes (with eyebrows), nose and mouth, in the predefined subregions. The simulation results validates that the proposed method is capable of automatically extracting features from various video images effectively under natural lighting environments and in the presence of complex backgrounds, certain amount of artificial noise and of multi-face oriented with angles. This preliminary study advanced from a rich literature provides robust facial feature detection under certain variations in lighting conditions and backgrounds.

The Self-Organizing Map (SOM) is an efficient tool for visualizing high-dimensional data. In this paper, an intuitive and effective SOM projection method is proposed for mapping high-dimensional data onto the two-dimensional grid structure with a growing self-organizing mechanism. In the learning phase, a growing SOM is trained and the growing cell structure is used as the baseline framework. In the ordination phase, the new projection method is used to map the input vector so that the input data is mapped to the structure of the SOM without having to plot the weight values, resulting in easy visualization of the data. The projection method is demonstrated on four different data sets, including a 118 patent data set and a 399 checical abstract data set related to polymer cements, with promising results and a significantly reduced network size.

Using optimization tools such as genetic algorithms to construct a fuzzy expert system (FES) focusing only on its accuracy without considering the comprehensibility may not result in a system that produces understandable expressions. To exploit the transparency characteristics of FES for reasoning in a higher-level knowledge representation, a FES should provide high comprehensibility while preserving its accuracy. The completeness of fuzzy sets and rule structures should also be considered to guarantee that every data point has a response output. This paper proposes some quantitative measures for a FES to determine the degree of the accuracy, the comprehensibility of the fuzzy sets, and the completeness of fuzzy rule structure. These quantitative measures are then used as a fitness function for a genetic algorithm in optimally refining a FES.

Abstract Research fronts, defined as clusters of documents that tend to cite a fixed, time invariant set of base documents, are plotted as time lines for visualization and exploration. Using a set of documents related to the subject of anthrax research, this article illustrates the construction, exploration, and interpretation of time lines for the purpose of identifying and visualizing temporal changes in research activity through journal articles. Such information is useful for presentation to members of expert panels used for technology forecasting.

Concerns the use of evolutionary algorithms (EA) in solving multiobjective optimization problems (MOP). We propose the use of a rank-density-based genetic algorithm (RDGA) that synergistically integrates selected features from existing algorithms in a unique way. A new ranking method, automatic accumulated ranking strategy, and a "forbidden region" concept are introduced, completed by a revised adaptive cell density evaluation scheme and a rank-density-based fitness assignment technique. In addition, four types of MOP features, such as discontinuous and concave Pareto front, local optimality, high-dimensional decision space and high-dimensional objective space are exploited and the corresponding MOP test functions are designed. By examining the selected performance indicators, RDGA is found to be statistically competitive with four state-of-the-art algorithms in terms of keeping the diversity of the individuals along the tradeoff surface, tending to extend the Pareto front to new areas and finding a well-approximated Pareto optimal front.

This paper proposes a new evolutionary approach to multiobjective optimization problems - the dynamic multiobjective evolutionary algorithm (DMOEA). In DMOEA, a novel cell-based rank and density estimation strategy is proposed to efficiently compute dominance and diversity information when the population size varies dynamically. In addition, a population growing and declining strategies are designed to determine if an individual will survive or be eliminated based on some qualitative indicators. Meanwhile, an objective space compression strategy is devised to continuously refine the quality of the resulting Pareto front. By examining the selected performance metrics on three recently designed benchmark functions, DMOEA is found to be competitive with or even superior to five state-of-the-art MOEAs in terms of maintaining the diversity of the individuals along the tradeoff surface, tending to extend the Pareto front to new areas, and finding a well-approximated Pareto optimal front. Moreover, DMOEA is evaluated by using different parameter settings on the chosen test functions to verify its robustness of converging to an optimal population size, if it exists. Simulations show that DMOEA has the potential of autonomously determining the optimal population size, which is found insensitive to the initial population size chosen.

In this paper, we propose a novel hybrid intelligent system (HIS) which provides a unified integration of numerical and linguistic knowledge representations. The proposed HIS is a hierarchical integration of an incremental learning fuzzy neural network (ILFN) and a linguistic model, i.e., fuzzy expert system (FES), optimized via the genetic algorithm (GA). The ILFN is a self-organizing network. The linguistic model is constructed based on knowledge embedded in the trained ILFN or provided by the domain expert. The knowledge captured from the low-level ILFN can be mapped to the higher level linguistic model and vice versa. The GA is applied to optimize the linguistic model to maintain high accuracy, comprehensibility, completeness, compactness, and consistency. The resulted HIS is capable of dealing with low-level numerical computation and higher level linguistic computation. After the system is completely constructed, it can incrementally learn new information in both numerical and linguistic forms. To evaluate the system's performance, the well-known benchmark Wisconsin breast cancer data set was studied for an application to medical diagnosis. The simulation results have shown that the proposed HIS performs better than the individual standalone systems. The comparison results show that the linguistic rules extracted are competitive with or even superior to some well-known methods. Our interest is not only on improving the accuracy of the system, but also enhancing the comprehensibility of the resulted knowledge representation.

In this paper, a new evolutionary approach, the rank-density based genetic algorithm (RDGA), to multiobjective optimization problems is proposed. In RDGA, a new ranking method, called an automatic accumulated ranking strategy and a "forbidden region" concept are introduced, completed by the revised adaptive cell density evaluation scheme and rank-density based fitness assignment technique. By examining the selected performance indicators on two benchmark problems, RDGA is found to be statistically competitive with two state-of-the-art multiobjective evolutionary algorithms, in terms of keeping the diversity of the individuals along the trade-off surface, extending the Pareto front to new areas, and finding a well-approximated Pareto optimal front.

The authors propose a new evolutionary approach to multiobjective optimization problems; the Dynamic Multiobjective Evolutionary Algorithm (DMOEA). In DMOEA, a population growing and population decline strategies are designed, and several important indicators are defined in order to determine the adaptive individual "killing" scheme. By examining the selected performance indicators of a test function, DMOEA is found to be effective in directing the population into an optimal population size, keeping the diversity of the individuals along the trade-off surface, tending to extend the Pareto front to new areas, and finding a well-approximated Pareto optimal front.

In this paper, we propose a genetic algorithm based design procedure for a radial-basis function neural network. A hierarchical rank density genetic algorithm (HRDGA) is used to evolve both the neural network's topology and parameters. In addition, the rank-density based fitness assignment technique is used to optimize the performance and topology of the evolved neural network to deal with the confliction between the training performance and network complexity. Instead of producing a single optimal network, HRDGA provides a set of near-optimal neural networks to the designers or the decision makers so that they can have more flexibility for the final decision-making based on their preferences. In terms of searching for a near-complete set of candidate networks with high performances, the networks designed by the proposed algorithm prove to be competitive, or even superior, to three selected traditional radial-basis function networks for predicting Mackey-Glass chaotic time series.

An automatic facial feature extraction method is presented in this paper. The method is based on the edge density distribution of the image. In the preprocessing stage a face is approximated to an ellipse, and a genetic algorithm is applied to search for the best ellipse region match. In the feature extraction stage, the genetic algorithm is applied to extract the facial features, such as the eyes, nose and mouth, in the predefined sub regions. The simulation results validates that the proposed method is capable of automatically extracting features from various video images effectively under natural lighting environments and in the presence of certain amount of artificial noise and of multi-face oriented with angles.

Takagi-Sugeno (TS) fuzzy models have been the popular choice for multi-model based nonlinear system identification for controller design. However, the interpretation of the model from a dynamical systems point of view is still unclear. We investigate the interpretation of the TS fuzzy models in terms of local dynamics of nonlinear systems in state space. The investigation is based on the comparison between the constant-affine recursive i/o equation and constant-affine state space representations. We show that the true local dynamics is not identifiable by TS fuzzy models by analysis and example. For the identification of constant-affine state space models, a subspace identification algorithm is proposed to identify the true local dynamics with zero initial conditions.

Using optimization tools such as genetic algorithms to construct a fuzzy expert system (FES), focusing only on its accuracy without considering comprehensibility may result in a system that is not easy to understand or the so called a black box model. To exploit the transparency features of FESs for explanation in higher-level knowledge representation, a FES should provide high comprehensibility while preserving its accuracy. The completeness of fuzzy sets and rule structures should also be considered to guarantee that every data point has a response output. This paper proposes some quantitative measures to determine the degree of the accuracy, comprehensibility, and completeness of FESs. These quantitative measures are then used as a fitness function for a genetic algorithm in an optimally built FES.

The growing demand in system reliability and survivability under failures has urged ever-increasing research effort on the development of fault diagnosis and accommodation. In this paper, the on-line fault tolerant control problem for dynamic systems under unanticipated failures is investigated from a realistic point of view without any specific assumption on the type of system dynamical structure or failure scenarios. The sufficient conditions for system on-line stability under catastrophic failures have been derived using the discrete-time Lyapunov stability theory. Based upon the existing control theory and the modern computational intelligence techniques, an on-line fault accommodation control strategy is proposed to deal with the desired trajectory-tracking problems for systems suffering from various unknown and unanticipated catastrophic component failures. Theoretical analysis indicates that the control problem of interest can be solved on-line without a complete realization of the unknown failure dynamics provided an on-line estimator satisfies certain conditions. Through the on-line estimator, effective control signals to accommodate the dynamic failures can be computed using only the partially available information of the faults. Several on-line simulation studies have been presented to demonstrate the effectiveness of the proposed strategy. To investigate the feasibility of using the developed technique for unanticipated fault accommodation in hardware under the real-time environment, an on-line fault tolerant control test bed has been constructed to validate the proposed technology. Both on-line simulations and the real-time experiment show encouraging results and promising futures of on-line real-time fault tolerant control based solely upon insufficient information of the system dynamics and the failure dynamics.

In this paper, we propose a genetic algorithm based design procedure for a multi layer feed forward neural network. A hierarchical genetic algorithm is used to evolve both the neural networks topology and weighting parameters. Compared with traditional genetic algorithm based designs for neural networks, the hierarchical approach addresses several deficiencies, including a feasibility check highlighted in literature. A multi objective cost function is used herein to optimize the performance and topology of the evolved neural network simultaneously. In the prediction of Mackey Glass chaotic time series, the networks designed by the proposed approach prove to be competitive, or even superior, to traditional learning algorithms for the multi layer Perceptron networks and radial basis function networks. Based upon the chosen cost function, a linear weight combination decision making approach has been applied to derive an approximated Pareto optimal solution set. Therefore, designing a set of neural networks can be considered as solving a two objective optimization problem.

Automatic recognition of frog vocalization is considered a valuable tool for a variety of biological research and environmental monitoring applications. In this research an automatic monitoring system, which can recognize the vocalizations of four species of frogs and can identify different individuals within the species of interest, is proposed. For the desired monitoring system, species identification is performed first with the proposed filtering and grouping algorithm. Individual identification, which can estimate frog population within the specific species, is performed in the second stage. Digital signal pre-processing, feature extraction, dimensionality reduction, and neural network pattern classification are performed step by step in this stage. Wavelet Packet feature extraction together with two different dimension reduction algorithms are synergistically integrated to produce final feature vectors, which are to be fed into a neural network classifier. The simulation results show the promising future of deploying an array of continuous, on-line environmental monitoring systems based upon nonintrusive analysis of animal calls.

In this paper, a method for automatic construction of a fuzzy rule-based system from numerical data using the Incremental Learning Fuzzy Neural (ILFN) network and the Genetic Algorithm is presented. The ILFN network was developed for pattern classification applications. The ILFN network, which employed fuzzy sets and neural network theory, equips with a fast, one-pass, on-line, and incremental learning algorithm. After trained, the ILFN network stored numerical knowledge in hidden units, which can then be directly interpreted into if then rule bases. However, the rules extracted from the ILFN network are not in an optimized fuzzy linguistic form. In this paper, a knowledge base for fuzzy expert system is extracted from the hidden units of the ILFN classifier. A genetic algorithm is then invoked, in an iterative manner, to reduce number of rules and select only discriminate features from input patterns needed to provide a fuzzy rule-based system. Three computer simulations using a simulated 2-D 3-class data, the well-known Fisher's Iris data set, and the Wisconsin breast cancer data set were performed. The fuzzy rule-based system derived from the proposed method achieved 100% and 97.33% correct classification on the 75 patterns for training set and 75 patterns for test set, respectively. For the Wisconsin breast cancer data set, using 400 patterns for training and 299 patterns for testing, the derived fuzzy rule-based system achieved 99.5% and 98.33% correct classification on the training set and the test set, respectively.

An innovative neuro-fuzzy network appropriate for fault detection and classification in a machinery condition health monitoring environment is proposed. The network, called an incremental learning fuzzy neural (ILFN) network, uses localized neurons to represent the distributions of the input space and is trained using a one-pass, on-line, and incremental learning algorithm that is fast and can operate in real time. The ILFN network employs a hybrid supervised and unsupervised learning scheme to generate its prototypes. The network is a self-organized structure with the ability to adaptively learn new classes of failure modes and update its parameters continuously while monitoring a system. To demonstrate the feasibility and effectiveness of the proposed network, numerical simulations have been performed using some well-known benchmark data sets, such as the Fisher's Iris data and the Deterding vowel data set. Comparison studies with other well-known classifiers were performed and the ILFN network was found competitive with or even superior to many existing classifiers. The ILFN network was applied on the vibration data known as Westland data set collected from a U.S. Navy CH-46E helicopter test stand, in order to assess its efficiency in machinery condition health monitoring. Using a simple fast Fourier transform (FFT) technique for feature extraction, the ILFN network has shown promising results. With various torque levels for training the network, 100% correct classification was achieved for the same torque Levels of the test data.

As dynamic systems become more complex, experience more rapidly changing environments, and encounter a greater variety of unexpected component failures, solving the control problems of such systems is a grand challenge for control engineers. We investigate the fault tolerant control problem and propose an intelligent sliding mode control strategy using artificial neural networks to handle the desired trajectories tracking problem for systems suffering from catastrophic faults or incipient failures. The approach is to continuously monitor the system performance and identify what the system's current state is by using a fault detection method based upon our best knowledge of the nominal system and nominal controller. Once a fault is detected, the proposed intelligent controller will adjust its control signal by adding a robust term to confine the system performance within a boundary layer. At the same time, an artificial neural network is initialized and compensates for the unknown fault dynamics online. Once the online learning process converges, the control input is tuned again by using the output of the identification model and a new least upper bound for the remaining uncertainty is estimated to further reduce the tracking error. The simulation results show a significant improvement in trajectory following performance based upon the proposed intelligent control strategy.

A method for automatic construction of a fuzzy expert system from numerical data using the ILFN network and a genetic algorithm is presented. The incremental learning fuzzy neural (ILFN) network was developed for pattern classification problems. The ILFN network is a fast, one-pass, on-line, and incremental learning algorithm. A knowledge base for fuzzy expert systems is extracted from the hidden units of the ILFN classifier. The genetic algorithm is then used, in an iterative manner, to reduce the number of rules and select important input pattern features needed to generate a comprehensible fuzzy rule-based system.

Condition monitoring of dynamic systems based on vibration signatures has generally relied upon Fourier based analysis as a means of translating vibration signals in time domain into the frequency domain. However, Fourier analysis provided a poor representation of signals well localized in time. The wavelet packet transform is introduced as an alternative means of extracting time-frequency information from vibration signature. Moreover, with the aid of statistical based feature selection criteria, a lot of feature components containing little discriminant information could be discarded resulting in a feature subset with reduced number of parameters. This significantly reduces the long training time that is often associated with neural network classifier and increases the generalization ability of the neural network classifier.

A new learning algorithm suitable for pattern classification in machine condition health monitoring based on fuzzy neural networks called an "incremental learning fuzzy neuron network" (ILFN) has been developed. The ILFN, using Gaussian neurons to represent the distributions of the input space, is an online one-pass incremental learning algorithm. The network is a self-organized classifier with the ability to adaptively learn new information without forgetting old knowledge. To prove the concept the simulations have been performed with the vibration data known as Westland vibration data set. Furthermore, the classification performance of the network has been tested on other benchmark data sets, such as Fisher's iris data (1936) and a vowel data set. For the generalization capability, comparison studies among other well-known classifiers were performed and the ILFN was found competitive with or even superior to many existing classifiers. Additionally the ILFN uses far less training time than conventional classifiers.

We propose a computationally efficient synthesis procedure for a class of bidirectional associative memories. Networks are described by a system of first-order ordinary difference equations which are defined on a closed hypercube of the state-space with solutions extended to the corner of the hypercube. The proposed algorithm possesses several advantages since it is possible: 1) to exert control over the number of spurious states; 2) to estimate the basins of attraction of the stable memories; and 3) under certain constraints, to effectively store a number of desired stable memories which by far exceed the order of the network. The applicability of the present results is demonstrated by means of a specific application in reconfigurable control of flexible structures.