To address the ethical and social risks of artificial intelligence, there is a growing interest in transparency and fairness, and various approaches that consider fairness in highly transparent artificial intelligence have attracted significant attention. In particular, inherently interpretable models can be powerful tools in scenarios where transparency and fairness are important, as they enable fairness to be addressed on the basis of an understanding of their internal mechanisms. A fuzzy system is a representative inherently interpretable model that can make flexible decisions considering real-world uncertainties. Multi-objective fuzzy genetics-based machine learning generates a number of fuzzy classifiers considering trade-offs among multiple objectives by using an evolutionary multi-objective optimization algorithm. In this study, we analyze in detail a set of fuzzy classifiers obtained by multi-objective fuzzy genetics-based machine learning, which simultaneously optimizes accuracy and fairness. We compare this set with other sets of inherently interpretable models and investigate the internal mechanisms of the fuzzy classifiers. Furthermore, we investigate the effects of combining fairness-aware optimization with fairness through unawareness on the fairness of fuzzy classifiers. The experimental results show that fuzzy systems are useful as inherently interpretable and fair models, and provide several insights that offer valuable guidance for fair artificial intelligence design.

This paper introduces a clustering-based sample selection method for a replay buffer to improve the performance of the continual Self-Supervised Learning (SSL) with a replay-based approach. Specifically, we first extract latent representations of a task using the encoder network after the task has been learned. Next, clustering is applied to these latent representations for obtaining cluster centroids (i.e., nodes). Then, the training sample with the most similar latent representation to each node is selected. Finally, reservoir sampling is applied to these selected training samples for updating a replay buffer. In this paper, we apply the proposed method to continual SSL and verify its effectiveness through numerical experiments on real-world datasets. The source code is available at https://github.com/Masuyama-lab/LUMP_CAplus.

While a Gaussian centralized TSK fuzzy system seeks full interpretability, its existing training method may become infeasible. In this study, we investigate the system's promising modeling performance and full interpretability by revisiting it from a probabilistic perspective. This approach allows us to precisely identify its output as the mathematical expectation and to derive its output variance as a novel Output Stability (OS) metric, which can be used to measure output stability and generate a calibrated probability output for model calibration. Subsequently, a novel OS-based training method for a fully interpretable Gaussian centralized TSK fuzzy classifier is developed to enhance its modeling performance. In addition, another potential value of OS as its new application is also exploited in linear aggregation learning of such fully interpretable fuzzy subsystems. Experimental results on 14 benchmark binary datasets demonstrate the effectiveness of both the OS-based training method and the OS-based linear aggregation learning in terms of average testing classification performance, interpretability, and training time.

As solution evaluation costs increase, data-driven multiobjective optimization is becoming more critical. Using multiple computers in parallel helps manage computation times. Federated learning offers a privacy-friendly, cost-effective way to handle distributed data. Integration of these two approaches is known as federated data-driven multiobjective optimization. However, to the best of our knowledge, a few studies tackle this emerging topic. This paper introduces multiobjective evolutionary algorithms (MOEAs) to federated clustering via adaptive resonance theory-based clustering (FCAC) and proposes FCAC-MOEA as a solver system of federated data-driven optimization problems. The computational experiments showed that the proposed method achieves both high search efficiency and privacy preservation on various multiobjective optimization problems.

Generally, clustering algorithms based on Adaptive Resonance Theory (ART) require the setting of data-dependent parameters such as similarity thresholds. Previous studies have shown that Correntropy-Induced Metric (CIM)-based ART+ (CA+), which automatically determines similarity thresholds based on the diversity of training data, exhibits superior clustering performance. CA+ is a non-hierarchical clustering algorithm that autonomously and adaptively generates nodes by the training data. This paper introduces a hierarchical structure to CA+ to enhance clustering performance. We discuss the clustering performance and clustering characteristics of the proposed algorithm through comparative experiments with other algorithms using real-world datasets.

While the recently developed first-order Takagi–Sugeno–Kang (TSK) fuzzy regressor FIMG-TSK shares its full interpretability, this study leverages the concisely expressed output variance of FIMG-TSK to explore its high feasibility in being a wide-ensemble component. In this way, the regression performance can be enhanced and simultaneously FIMG-TSKs overdependence on the rule weights can be alleviated to a certain extent. To this end, a wide ensemble of all base regressors (i.e., FIMG-TSKs) called EFIMG-TSKs is proposed. In the ensemble-strategic aspect, EFIMG-TSK has its internally and generatively decorrelated ensemble strategy with a quintuply diversity guarantee for its strong generalization capability. In the learning aspect, the learning objective of EFIMG-TSKs reflects the internally and generatively decorrelated ensemble learning of all base FIMG-TSKs and accordingly is optimized globally with an analytical solution to the weights of all fuzzy rules in each base FIMG-TSK. The experimental results on 16 benchmarking datasets demonstrate the effectiveness of EFIMG-TSKs in terms of regression performance, training time, and interpretability.

This article proposes a multi-label classification algorithm capable of continual learning by applying an Adaptive Resonance Theory (ART)-based clustering algorithm and the Bayesian approach for label probability computation. The ART-based clustering algorithm adaptively and continually generates prototype nodes corresponding to given data, and the generated nodes are used as classifiers. The label probability computation independently counts the number of label appearances for each class and calculates the Bayesian probabilities. Thus, the label probability computation can cope with an increase in the number of labels. Experimental results with synthetic and real-world multi-label datasets show that the proposed algorithm has competitive classification performance to other well-known algorithms while realizing continual learning.

Most existing multiobjective evolutionary algorithms treat all decision variables as a whole to perform genetic operations and optimize all objectives with one population at the same time. Considering different control attributes, different decision variables have different optimization effects on each objective, so decision variables can be divided into convergence- or diversity-related variables. In this article, we propose a new metric called the optimization degree of the convergence-related decision variable to each objective to calculate the contribution objective of each decision variable. All decision variables are grouped according to their contribution objectives. Then, a multiobjective evolutionary algorithm, namely, decision variable contributing to objectives evolutionary algorithm (DVCOEA), has been proposed. In order to balance the convergence and diversity of the population, the DVCOEA algorithm combines the multipopulation multiobjective framework, where two different optimization strategies are designed to optimize the subpopulation and individuals in the external archive, respectively. Finally, DVCOEA is compared with several state-of-the-art algorithms on a number of benchmark functions. Experimental results show that DVCOEA is a competitive approach for solving large-scale multi/many-objective problems.

While interpretable antecedent parts of first-order Takagi–Sugeno–Kang (TSK) fuzzy rules can be properly acquired by adopting some clustering methods, this study aims at avoiding the commonly used yet fully incomprehensive consequent parts and their intractable training, and simultaneously seeking for enhanced generalization performance by determining the weight of each rule. The central idea is to build a mathematically equivalent bridge between a Gaussian mixture model (GMM) and a fully interpretable first-order TSK fuzzy system called FIMG-TSK, with the help of Gaussian-mixture's mean. The resultant FIMG-TSK has a simple expected output expression without summation-to-one defuzzification, which will be helpful in inducing both smaller output variance and a negative entropic and rule-stability-based regularizer for enhancing the generalization performance. After revealing three factors affecting the output stability of FIMG-TSK, the negative entropic and rule-stability-based regularizer is designed through both these factors and the squared entropy to make the output variance of FIMG-TSK as small as possible. Accordingly, a novel training method, whose objective function takes the proposed regularizer as an additional term and hence compromises both accuracy and output stability of FIMG-TSK, is developed to quickly provide an analytical solution to the weight of each rule. The effectiveness of the proposed training method is manifested by the experimental results on ten regression datasets.

Explainable artificial intelligence (XAI) is an important research topic in the field of machine learning. A fuzzy rule-based classifier is a promising XAI technique thanks to its high interpretability. We can linguistically explain its classification result because a set of linguistically explainable fuzzy if-then rules are used for classification. In real-world data mining applications, multiple class labels are assigned to a single instance. Such a dataset is called a multi-label dataset (MLD). For MLDs, multiobjective fuzzy genetics-based machine learning for multi-label classification (MoFGBMLML) has been proposed. MoFGBMLMLaims to search for explainable fuzzy classifiers by explicitly considering the accuracy-complexity tradeoff that exists in explainable classifier design. In the field of multi-label classification, different accuracy metrics have been proposed to evaluate classifier performance. As a result, different multiobjective optimization problems (MOPs) can be defined using each accuracy metric together with a complexity metric. Usually, MoFGBMLMLsolves each MOP independently. In this paper, we incorporate the idea of multi-tasking optimization into MoFGBMLMLso that multiple MOPs are solved simultaneously. We also propose a new information sharing method to improve the effectiveness of multi-tasking optimization in MoFGBMLML. Our experimental results show that multiple accuracy metrics can be simultaneously optimized through the multi-tasking optimization framework and the proposed information sharing method improves the classification accuracy of fuzzy classifiers obtained by MoFGBMLML.

This paper proposes an Adaptive Resonance Theory (ART)-based clustering algorithm for a dataset which contains numerical and categorical attributes simultaneously. In the proposed algorithm, similarity between numerical attributes is calculated by the correntropy-based nonlinear similarity measurement, while similarity between categorical attributes is defined by a hamming distance-based approach. One advantage of the proposed algorithm is that the algorithm continually and adaptively generates a sufficient number of nodes for clustering from given data points. Empirical studies on various datasets show that the proposed algorithm has comparable clustering performance to the representative mixed data clustering algorithms.

For many application scenarios where raw and even multidomain training data can be easily collected, and at the same time, validation data (as ground-truth data) are available, it becomes naturally desirable for us to perform an enhanced classification/prediction on only validation data with the appropriate leverage of training data. In this article, a novel ensemble framework EP-TSK-FK of Takagi–Sugeno–Kang (TSK) fuzzy subclassifiers, is proposed to achieve the following distinctive characteristics: 1) each interpretable TSK fuzzy subclassifier on each training subset can be quickly built in parallel such that its outputs provide the values of the corresponding augmented features of the original validation data space; 2) as a novel ensemble method of fuzzy subclassifiers, EP-TSK-FK trains all the interpretable TSK fuzzy subclassifiers only once and does not explicitly reuse them while predicting a testing sample, which thereby reduces the computational complexity of the ensemble process for prediction; 3) after running the proposed iterative fuzzy c-means clustering algorithm iterative fuzzy C-means clustering (IFCM) on the augmented validation data to obtain the representative centroids, the fast classification/prediction of EP-TSK-FK on the testing samples is realized by using the$k$-nearest neighbor (KNN) method on the representative centroids with the original features; and 4) enhanced classification performance by the IFCM & KNN method is theoretically revealed, and the experimental results on the benchmarking datasets indicate the effectiveness of EP-TSK-FK and its parallel learning method in the sense of enhanced classification performance, running time, and interpretability.

A multi-modal multi-objective optimization problem (MMOP) may have equivalent Pareto optimal solutions. These solutions are different in the decision space but correspond to the same objective vector. Searching for equivalent Pareto optimal solutions with evolutionary algorithms is a hot topic in recent years. However, most existing researches are about continuous MMOPs, whereas there are few studies on discrete MMOPs. In this paper, we discuss the property of the multi-modal multi-objective traveling salesman problem and present a set of test problems. Then, we propose an evolutionary optimizer to solve the problem. Experimental results show that our evolutionary optimizer can find more equivalent Pareto optimal solutions than traditional multi-objective evolutionary optimizers on the test problems.

As the first attempt, this article considers how to provide a design methodology for style clustering on stylistic data, where each cluster depends on both the similarities between data samples and its latently or apparently distinguishable style. By taking our previous fuzzy k plane clustering algorithm as the basic framework, a fuzzy style k-plane clustering (S-KPC) algorithm is proposed to have its distinctive merits: First, the nuances between styles of clusters can be well identified by using the proposed twofold data representation. That is to say, style matrices are used to express the structure, hence style information of each cluster, whereas the augmentation of the original features of data with enhanced nodes is taken as an abstract representation so as to move the manifold structure of data apart. Such a twofold data representation can make us realize S-KPC readily in an incremental way. Second, by means of alternating optimization strategy, the objective function of S-KPC can be optimized such that each discriminant function of each cluster shares the advantages of both simple regression models and functional-link neural networks. Extensive experiments on synthetic and real-world datasets demonstrate that S-KPC has comparable clustering performance with several compared methods on the adopted ordinary datasets, and yet it obviously outperforms them on stylistic datasets.

Although high-order Takagi–Sugeno–Kang (TSK) fuzzy systems have demonstrated their computational advantages and simultaneously circumvent the weakness that the number of rules with the number of input variables and membership functions grows exponentially in both zero-order and first-order TSK fuzzy systems for complex modeling tasks, they still face two serious issues: incapability for a changing environment and no interpretability of the coefficients in high-order polynomial used in the consequent part of each fuzzy rule. In order to circumvent these two challenges, a novel stacked architecture of an interpretable deep higher order TSK fuzzy classifier called DHO-TSK and its deep learning method are proposed by proving the equivalence between a high-order TSK fuzzy classifier and a deep ensemble of interpretable zero-order TSK fuzzy classifiers in this article. DHO-TSK can be built by assembling interpretable zero-order TSK fuzzy classifiers in a special stacked way. Each zero-order TSK fuzzy classifier can be learnt by randomly selecting input features, randomly assigning an antecedent fuzzy subset from a fixed fuzzy partition to each of the selected input features, and then multiplying the output of each TSK fuzzy classifier by a randomly selected feature. Except for the abovementioned solid theoretical equivalence, DHO-TSK is featured in the following aspects: first, the consequent part of each fuzzy rule in DHO-TSK becomes interpretable and the output expression of each layer in DHO-TSK becomes comprehensible due to the adopted stacked ensemble; second, its enhanced classification performance can be achieved in a stacked deep learning way; third, DHO-TSK has its adoptability for changing environments owing to random selection of both features and fuzzy membership functions. Our experimental results on the benchmarking UCI and KEEL datasets and a real dataset indicate the effectiveness of DHO-TSK and its learning method in the sense of both classification performance and interpretability.

Recently, Evolutionary Multiobjective Multitask optimization (EMMO) was proposed as a new research topic in the field of Evolutionary Multiobjective optimization (EMO). In contrast to conventional EMO algorithms, EMMO algorithms solve multiple multiobjective optimization problems (multiple tasks) in their single run. Most EMMO algorithms have the same number of populations as the number of tasks to be solved simultaneously, and each population corresponds to a different task. The main feature of EMMO algorithms is that offspring solutions are generated by not only intra-task crossover but also inter-task crossover. Local mating in intra-task crossover improves the search performance of EMO algorithms that use uniformly distributed weight vectors during a search, such as MOEA/D. Therefore, local mating in inter-task crossover is a promising idea for EMMO algorithms. In this paper, we propose a simple extension of MOEA/D for EMMO algorithms and a local mating method in inter-task crossover based on uniformly distributed weight vectors. Through computational experiments, we examine the effects of local mating in inter-task crossover on the search performance of the proposed algorithm. Experimental results show that the local mating improves the search performance of the proposed algorithm.

Multi-modal multi-objective optimization problems may have different Pareto optimal solutions with the same objective vector. A number of evolutionary multi-modal multiobjective algorithms have been developed to solve these problems. They aim to search for a Pareto optimal solution set with good diversity in both the objective and decision spaces. Although the normalization in both the objective and decision spaces is very important for these algorithms, there are few studies on this topic. In this paper, we investigate the effect of four normalization methods on two evolutionary multi-modal multiobjective algorithms. Six distance minimization problems are chosen as test problems. The experimental results show that the effect of normalization in evolutionary multi-modal multiobjective optimization is algorithm- and problem-dependent.

In multi-label classification problems, multiple class labels are assigned to each instance. Two approaches have been studied in the literature. One is a data transformation approach, which transforms a multi-label dataset into a number of singlelabel datasets. However, this approach often loses the correlation information among classes in the multi-class assignment. The other is a method adaptation approach where a conventional classification method is extended to multi-label classification. Recently, some explainable classification models for multi-label classification have been proposed. Their high interpretability has also been discussed with respect to the transparency of the classification process. Although the explainability is a well-known advantage of fuzzy systems, their applications to multi-label classification have not been well studied. Since multi-label classification problems often have vague class boundaries, fuzzy systems seem to be a promising approach to multi-label classification. In this paper, we propose a new multiobjective evolutionary fuzzy system, which can be categorized as a method adaptation approach. The proposed algorithm produces nondominated classifiers with different tradeoffs between accuracy and complexity. We examine the behavior of the proposed algorithm using synthetic multi-label datasets. We also compare the proposed algorithm with five representative algorithms. Our experimental results on real-world datasets show that the obtained fuzzy classifiers with a small number of fuzzy rules have high transparency and comparable generalization ability to the other examined multi-label classification algorithms.

Dominance resistant solutions (DRSs) in multi-objective problems have very good values for some objectives and very bad values for other objectives. Whereas DRSs are far away from the Pareto front, they are hardly dominated by other solutions due to some very good objective values. It is well known that the existence of DRSs severely degrades the search ability of Pareto dominance-based algorithms such as NSGA-II and SPEA2. In this paper, we examine the effect of DRSs on the search ability of NSGA-II on the DTLZ test problems with many objectives. We slightly change their problem formulation to increase the size of the DRS region. Through computational experiments, we show that DRSs have a strong negative effect on the search ability of NSGA-II whereas they have almost no effect on MOEA/D with the PBI function. We also show that a slightly modified NSGA-II for decreasing the negative effect of DRSs works well on many-objective DTLZ test problems (its performance is similar to NSGA-III and MOEA/D). These results suggest that DTLZ is not an appropriate test suite for evaluating many-objective evolutionary algorithms. This issue is further addressed through computational experiments on newly formulated test problems with no distance function.

Clustering based on Adaptive Resonance Theory (ART) has been actively studied. In previous studies, ART-based clustering algorithms with a topological structure have been proposed and showed their superior self-organizing ability. However, this method deteriorates the clustering performance at high noise ratios. In this paper, we propose a multilayer clustering algorithm based on a topological ART-based clustering for improving a noise reduction ability. Simulation experiments show that the proposed algorithm achieves excellent clustering performance on a 2D synthetic dataset in high noise environments.

The performance of decomposition-based multiobjective evolutionary algorithms (MOEAs) often deteriorates clearly when solving multiobjective optimization problems with irregular Pareto fronts (PFs). The main reason is the improper settings of reference vectors and scalarizing functions. In this paper, we propose a decomposition-based MOEA guided by a growing neural gas network, which learns the topological structure of the PF. Both reference vectors and scalarizing functions are adapted based on the topological structure to enhance the evolutionary algorithm's search ability. The proposed algorithm is compared with eight state-of-the-art optimizers on 34 test problems. The experimental results demonstrate that the proposed method is competitive in handling irregular PFs.

There may exist more than one Pareto optimal solution with the same objective vector to a multimodal multiobjective optimization problem (MMOP). The difficulties in finding such solutions can be different. Although a number of evolutionary multimodal multiobjective algorithms (EMMAs) have been proposed, they are unable to solve such an MMOP due to their convergence-first selection criteria. They quickly converge to the Pareto optimal solutions which are easy to find and therefore lose diversity in the decision space. That is, such an MMOP features an imbalance between achieving convergence and preserving diversity in the decision space. In this article, we first present a set of imbalanced distance minimization benchmark problems. Then we propose an evolutionary algorithm using a convergence-penalized density method (CPDEA). In CPDEA, the distances among solutions in the decision space are transformed based on their local convergence quality. Their density values are estimated based on the transformed distances and used as the selection criterion. We compare CPDEA with five state-of-the-art EMMAs on the proposed benchmarks. Our experimental results show that CPDEA is clearly superior in solving these problems.

Many practical scenarios have demanded that we should classify unlabeled data more accurately based on both physical features (e.g., color, distance, or similarity) and implicit style features of data. As most extant classification algorithms classify unlabeled data based only on their physical features, they become weak in achieving expected classification results for many scenarios. To work around this drawback in this paper, a novel classification method (FuCM) from the perspective of fuzzy social network based on both physical and implicit style features of data is proposed. Based on the proposed fuzzy social network and its dynamics about fuzzy influences of nodes, FuCM comprises two stages. In its training stage, after the fuzzy social network has been built, it learns the topological structure, reflecting physical features and implicit style features of data by carrying out fuzzy influence dynamics in the built network. In its prediction stage, both physical and implicit style features of data are effectively integrated to yield the double structure efficiency characterized by fuzzy influences of nodes. FuCM classifies unlabeled data according to the strongest connection measure based on the proposed double structure efficiency. FuCM does not assume that both data distribution and the classification by physical features or by both physical and implicit style features of data must be known in advance. Thus, it is a novel unified classification framework in this sense. In contrast to all the nine comparative methods, FuCM experimentally demonstrates its comparable classification performance on most synthetic, UCI and KEEL datasets, which can be well classified based only on physical features of data. Furthermore, it displays distinctive superiority on five case studies where satisfactory classification certainly depends on both physical and implicit style features.

Recently high performance of decomposition-based algorithms such as MOEA/D, NSGA-III and MOEA/DD for many-objective optimization has been repeatedly reported. When they are applied to many-objective problems, weight vectors are usually generated by a two-layered approach with boundary and inside layers. However, the specification of the two layers has not been discussed in detail in the literature. They are usually intuitively specified: More weight vectors are included in the boundary layer than the inside layer, and the size of the simplex in the inside layer is a half of that in the boundary layer. In this paper, we discuss the following two issues about the specification of the two layers: (i) how to specify the number of weight vectors in each layer, and (ii) how to specify the size of the simplex in the inside layer. We address these two issues through computational experiments on many-objective problems with six types of Pareto fronts: linear triangular, convex triangular, concave triangular, linear inverted triangular, convex inverted triangular, and concave inverted triangular. Our experimental results clearly demonstrate that the appropriate specification of the two layers strongly depends on the problem (i.e., the shape of the Pareto front) and the performance indicator. We also address the two issues from a viewpoint of the relation between the weight vector distribution for each shape of the Pareto front and the optimal distribution of solutions for each performance indicator.

This paper applies a genetic algorithm and fuzzy markup language to construct a human and smart machine cooperative learning system on game of Go. The genetic fuzzy markup language (GFML)-based Robot Agent can work on various kinds of robots, including Palro, Pepper, and TMU's robots. We use the parameters of FAIR open source Darkforest and OpenGo AI bots to construct the knowledge base of Open Go Darkforest (OGD) cloud platform for student learning on the Internet. In addition, we adopt the data from AlphaGo Master's sixty online games as the training data to construct the knowledge base and rule base of the co-learning system. First, the Darkforest predicts the win rate based on various simulation numbers and matching rates for each game on the OGD platform, then the win rate of OpenGo is as the final desired output. The experimental results show that the proposed approach can improve knowledge base and rule base of the prediction ability based on Darkforest and OpenGo AI bot with various simulation numbers.

Local Pareto optimal solutions may exist in multi-modal multi-objective optimization problems. Traditional multi-objective evolutionary algorithms usually try to escape from local Pareto optima. However, these solutions may be good enough for the decision makers and are additional options if Pareto optimal solutions are infeasible. In this paper, we modify our previous double-niched evolutionary algorithm (DNEA) to search for local Pareto optimal solutions. The new version is termed as DNEA-L. We apply DNEA-L to 3- and 4-objective polygon-based problems with local Pareto optima. The experimental results show that DNEA-L is efficient to find a large number of local Pareto optimal solutions with good diversity.

The performance of a Multiobjective Evolutionary Algorithm (MOEA) for many-objective optimization is often evaluated by multiobjective scalable test problems like DTLZ and WFG problems. This is because the scalable test problems are quite useful for an MOEA analysis. However, the scalable test problems do not have enough diversity of the shapes of the Pareto front and the feasible region to evaluate the capability of MOEAs. Previous studies showed that these shapes have a great impact on the performance of MOEAs. Thus, MOEAs should be evaluated on more test problems with different shapes of the Pareto front and the feasible region. In this study, the shapes of the Pareto front in the existing scalable test problems are examined from some viewpoints such as the distribution of optimal or worst solutions for each objective and the degree of the correspondence with the distribution of the weight vectors. The shape of the feasible region is also examined from the viewpoint of the spread of an initial population and the existence of dominance resistant solutions. According to the observations, we propose new shapes of the Pareto front and the feasible region to design a new scalable test suite. Experimental results show that the proposed test suite has totally different properties from the existing test problems.

Various distance minimization problems (DMPs) have been proposed to visualize the search behaviors of evolutionary multiobjective optimization (EMO) algorithms in solving many-objective problems, multiobjective multimodal problems, and dynamic multiobjective problems. Among those DMPs, only the box constraints are considered. In this paper, we propose several constraint DMPs to visualize the behaviors of EMO algorithms with constraint handling techniques. In the proposed constraint DMPs, constraints are simply specified in the two-dimensional decision space. In the same manner, high-dimensional problems and multimodal problems can also be generated. Computational experiments show different behaviors by different algorithms in various constraint DMPs.

The hypervolume has been frequently used as an indicator to compare the performance of evolutionary multi-objective optimization (EMO) algorithms. It has also been used in indicator-based algorithms (e.g., SMS-EMOA and HypE). In such an EMO algorithm, a multi-objective problem is handled as a single-objective problem to maximize the hypervolume of a pre-specified number of solutions. Whereas a reference point is needed for hypervolume calculation, its specification has not been discussed in detail in many studies. This may be because the reference point specification has almost no effect on experimental results when hypervolume-based EMO algorithms are applied to frequently-used scalable test problems such as DTLZ and WFG with triangular Pareto fronts. However, when the Pareto front of a test problem is not triangular (e.g., minus-DTLZ and minus-WFG), the reference point specification has a dominant effect on solution sets obtained by hypervolume-based EMO algorithms. In this paper, first we explain the importance of an appropriate reference point specification in SMS-EMOA. Then we examine the use of a dynamically changing specification of the reference point in SMS-EMOA.

A promising idea for evolutionary constrained optimization is to efficiently utilize not only feasible solutions (feasible individuals) but also infeasible ones. In this paper, we propose a simple implementation of this idea in MOEA/D. In the proposed method, MOEA/D has two grids of weight vectors. One is used for maintaining the main population as in the standard MOEA/D. In the main population, feasible solutions always have higher fitness than infeasible ones. Among infeasible solutions, solutions with smaller constraint violations have higher fitness. The other grid is for maintaining a secondary population where non-dominated solutions with respect to scalarizing function values and constraint violations are stored. More specifically, a single non-dominated solution with respect to the scalarizing function and the total constraint violation is stored for each weight vector. A new solution is generated from a pair of neighboring solutions in the two grids. That is, there exist three possible combinations of two parents: both from the main population, both from the secondary population, and each from each population. The proposed MOEA/D variant is compared with the standard MOEA/D and other evolutionary algorithms for constrained multiobjective optimization through computational experiments.

In recent years, evolutionary multiobjective optimization (EMO) algorithms have frequently been used for engineering problems with some conflicting objective functions to be simultaneously optimized. EMO algorithms can provide a number of Pareto optimal solutions to users. Two scenarios are considered in the practical use of EMO algorithms. One is that a decision maker selects a single solution from the obtained ones after the EMO process. The other is that a decision maker utilizes the solutions to analyze the relationship between design variables and objective functions of the corresponding problem. In this paper, we apply fuzzy genetics-based machine learning to the second scenario in order to generate if-then rule-based classifiers which represent the relationship between design variables and objective functions. We also utilize this method during the EMO process to pre-screen candidate offspring solutions. The classifier detects non-promising offspring solutions. Then, they are discarded before their fitness evaluation, so that the computation resource is used only for promising solutions. We apply this method to one engineering problem and examine its effect on the search performance of an EMO algorithm.

A Multiobjective Evolutionary Algorithm (MOEA) is one of the effective approaches for solving Multiobjective Optimization Problems (MOPs). The performance of MOEAs is evaluated mainly by scalable MOP test suites where the number of objectives can be arbitrarily specified. However, the number of scalable MOP test suites is quite limited and their properties are similar. Thus, there is a risk that the current research on MOEAs is specialized for some properties (i.e., a shape of feasible regions, a shape of the Pareto front, and a distance function) of existing scalable MOP test suites. In this paper, we focus on the above properties of two popular MOP test suites (i.e., DTLZ and WFG). Based on DTLZ and WFG, we create 12 MOPs which have partially different properties from those of DTLZ and WFG. Computational experiments show that the search performance of the state-of-the-art MOEAs strongly depends on three properties.

The hypervolume indicator has frequently been used for comparing evolutionary multi-objective optimization (EMO) algorithms. A reference point is needed for hypervolume calculation. However, its specification has not been discussed in detail from a viewpoint of fair performance comparison. A slightly worse point than the nadir point is usually used for hypervolume calculation in the EMO community. In this paper, we propose a reference point specification method for fair performance comparison of EMO algorithms. First, we discuss the relation between the reference point specification and the optimal distribution of solutions for hypervolume maximization. It is demonstrated that the optimal distribution of solutions strongly depends on the location of the reference point when a multi-objective problem has an inverted triangular Pareto front. Next, we propose a reference point specification method based on theoretical discussions on the optimal distribution of solutions. The basic idea is to specify the reference point so that a set of well-distributed solutions over the entire linear Pareto front has a large hypervolume and all solutions in such a solution set have similar hypervolume contributions. Then, we examine whether the proposed method can appropriately specify the reference point through computational experiments on various test problems. Finally, we examine the usefulness of the proposed method in a hypervolume-based EMO algorithm. Our discussions and experimental results clearly show that a slightly worse point than the nadir point is not always appropriate for performance comparison of EMO algorithms.

The hypervolume and the inverted generational distance (IGD) have been frequently used for the comparison of evolutionary multiobjective optimization algorithms. For the hypervolume, the relation between the location of a reference point and the optimal distribution of solutions has been studied in the literature. However, such a relation has not been studied for the IGD whereas IGD-based comparison results depend on the specification of reference points. Our intention is to clearly demonstrate the dependency of IGD-based comparison results on reference point specification. First, we explain difficulties of fair comparison in the following two cases: one is the use of all nondominated solutions among obtained solutions by compared algorithms as reference points, and the other is the use of a small number of uniformly sampled reference points. Discussions on these two cases show the necessity of a large number of uniformly sampled reference points on the entire Pareto front. Then, we show a bias of the IGD with such a reference point set through computational experiments. It is shown that the IGD tends to favor a solution set with much smaller diversity than a fully expanded solution set over the entire Pareto front. Finally, we propose a new specification method where reference points are uniformly sampled not only from the Pareto front but also from outside the Pareto front.

In this paper, we propose a new method for solving multiobjective optimization problems with a large number of decision variables. The proposed method called weighted optimization framework is intended to serve as a generic method that can be used with any population-based metaheuristic algorithm. After explaining some general issues of large-scale optimization, we introduce a problem transformation scheme that is used to reduce the dimensionality of the search space and search for improved solutions in the reduced subspace. This involves so-called weights that are applied to alter the decision variables and are also subject to optimization. Our method relies on grouping mechanisms and employs a population-based algorithm as an optimizer for both original variables and weight variables. Different grouping mechanisms and transformation functions within the framework are explained and their advantages and disadvantages are examined. Our experiments use test problems with 2-3 objectives 40-5000 variables. Using our approach on three well-known algorithms and comparing its performance with other large-scale optimizers, we show that our method can significantly outperform most existing methods in terms of solution quality as well as convergence rate on almost all tested problems for many-variable instances.

Hypervolume subset selection is to find a pre-specified number of solutions for hypervolume maximization. The optimal distribution of solutions on the Pareto front has been theoretically studied for two-objective problems in the literature. In this paper, we discuss hypervolume subset selection for three-objective problems with triangular and inverted triangular Pareto fronts. Our contribution is to show that the effect of the location of a reference point for hypervolume calculation on the optimal distribution of solutions is totally different between triangular and inverted triangular Pareto fronts. When the reference point is far from the Pareto front, most solutions are on the sides of the inverted triangular Pareto front while they are evenly distributed over the entire triangular Pareto front. These properties seem to hold in multiobjective problems with four or more objectives. We also show that the effect of the location of a reference point on the optimal distribution is totally different between maximization and minimization problems with the same triangular Pareto fronts. This property is supported by the fact that maximization problems with triangular Pareto fronts are equivalent to minimization problems with inverted triangular Pareto fronts. The optimal distribution of solutions is also discussed when the reference point is close to the Pareto front (i.e., when its location is between the nadir point and the Pareto front).

Various evolutionary multiobjective optimization (EMO) algorithms have been used in the field of evolutionary fuzzy systems (EFS), because EMO algorithms can easily handle multiple objective functions such as the accuracy maximization and complexity minimization for fuzzy system design. Most EMO algorithms used in EFS are Pareto dominance-based algorithms such as NSGA-II, SPEA2, and PAES. There are a few studies where other types of EMO algorithms are used in EFS. In this paper, we apply a multiobjective evolutionary algorithm based on decomposition called MOEA/D to EFS for fuzzy classifier design. MOEA/D is one of the most well-known decomposition-based EMO algorithms. The key idea is to divide a multiobjective optimization problem into a number of single-objective problems using a set of uniformly distributed weight vectors in a scalarizing function. We propose a new scalarizing function called an accuracy-oriented function (AOF) which is specialized for classifier design. We examine the effects of using AOF in MOEA/D on the search ability of our multiobjective fuzzy genetics-based machine learning (GBML). We also examine the synergy effect of MOEA/D with AOF and parallel distributed implementation of fuzzy GBML on the generalization ability.

Hypervolume has been frequently used as a performance indicator for comparing evolutionary multiobjective optimization (EMO) algorithms. Hypervolume has been also used in indicator-based algorithms. Whereas a reference point is needed for hypervolume calculation, its specification has not been discussed in detail from a viewpoint of fair comparison. This may be because a slightly worse reference point than the nadir point seems to work well. In this paper, we tackle this issue: How to specify a reference point for fair comparison. First we discuss an appropriate specification of a reference point for multiobjective problems. Our discussions are based on the well-known theoretical results about the optimal solution distribution for hypervolume maximization. Next we examine various specifications by computational experiments. Experimental results show that a slightly worse reference point than the nadir point works well only for test problems with triangular Pareto fronts. Then we explain why this specification is not always appropriate for test problems with inverted triangular Pareto fronts. We also report a number of solution sets obtained by SMS-EMOA with various specifications of a reference point.

One research direction in the field of evolutionary multiobjective optimization (EMO) is a post-analytical process of non-dominated solutions in order to analyze the relationship between design variables and objective functions for optimization problems. For this purpose, data mining techniques have been used in some studies. From a practical point of view, this process itself should be considered as a multiobjective optimization problem. In this paper, multiobjective genetic fuzzy rule selection is applied to the post-analytical process of solutions obtained by EMO algorithms. First, multiple regions of interest are specified in the objective space. Each region with a number of solutions is handled as a different class. A set of patterns is generated by the labeled solutions. Second, a number of fuzzy if-then rules are generated by classification rule mining. Finally, an EMO algorithm is applied to combinatorial optimization of fuzzy if-then rules in order to obtain a number of non-dominated fuzzy classifiers with respect to accuracy and complexity. Through computational experiments using two engineering problems, we show that we can obtain various classifiers with a variety of complexity-accuracy tradeoff.

Recently a number of evolutionary multiobjective optimization algorithms have been proposed in the framework of MOEA/D (Multi-Objective Evolutionary Algorithm based on Decomposition). A multiobjective problem is decomposed into multiple single-objective problems using a set of weight vectors in MOEA/D. The number of single-objective problems is the same as the number of weight vectors, which is also the same as the population size. It is well known that the performance of MOEA/D depends on the shape of the Pareto front. Weight vectors in MOEA/D are specified by using a triangular simplex lattice structure. Thus MOEA/D works well on multiobjective problems with triangular Pareto fronts, and has difficulties in handling inverted triangular Pareto fronts. One may wonder what happens if an inverted triangular simplex lattice structure is used for generating weight vectors in MOEA/D. This is our research question. In this paper, first we explain how to use an inverted triangular simplex lattice structure for generating weight vectors. Next we analytically explain inherent difficulties in the use of inverted triangular weight vectors in MOEA/D. Then, through computational experiments, we examine the performance of two types of MOEA/D: One is with triangular weight vectors, and the other is with inverted triangular weight vectors. Our discussions show that the use of inverted triangular weight vectors is not a good choice except for some special cases.

Portfolio optimization is always an attractive topic for research. In our previous approach, we proposed a method for mining a group stock portfolio that used grouping genetic algorithms. The derived group stock portfolio represents stocks in the same group that may have similar properties; consequ ently, a variety of stock portfolios could be offered to investors. However, the evaluation process used by this previous approach is time consuming when the number of stocks or groups increases. To address this problem, the map-reduce technique is considered. Map-reduce is a well-known approach for speeding up the mining process. This paper proposes a map-reduce-based approach to mine groups of stock portfolios and speed up the evolution process while still achieving results as similar as possible to the previous approach. Here, a chromosome represents a mapper number, a group number, a stock part and a portfolio part. Utilizing the mapper number, the chromosomes in a population are divided into subsets and sent to respective mappers, while the reducers execute fitness evaluation and genetic operations. The evolution process is repeated until the terminal conditions are reached. Finally, experiments on a real dataset are conducted to demonstrate the efficiency of the proposed approach.

Recently, a number of high performance many-objective evolutionary algorithms with systematically generated weight vectors have been proposed in the literature. Those algorithms often show surprisingly good performance on widely used DTLZ and WFG test problems. The performance of those algorithms has continued to be improved. The aim of this paper is to show our concern that such a performance improvement race may lead to the overspecialization of developed algorithms for the frequently used many-objective test problems. In this paper, we first explain the DTLZ and WFG test problems. Next, we explain many-objective evolutionary algorithms characterized by the use of systematically generated weight vectors. Then we discuss the relation between the features of the test problems and the search mechanisms of weight vector-based algorithms such as multiobjective evolutionary algorithm based on decomposition (MOEA/D), nondominated sorting genetic algorithm III (NSGA-III), MOEA/dominance and decomposition (MOEA/DD), and θ-dominance based evolutionary algorithm (θ-DEA). Through computational experiments, we demonstrate that a slight change in the problem formulations of DTLZ and WFG deteriorates the performance of those algorithms. After explaining the reason for the performance deterioration, we discuss the necessity of more general test problems and more flexible algorithms.

In data science applications, it is very often to require predictive models satisfying monotonicity with respect to the explanatory variables involved in the dataset. In ordinal classification or regression, this occurs when the output variable or class label do not decrease when input variables increase, or vice versa. This problem is commonly known as monotonic classification, and most existing classification techniques are not able to manage this kind of constraints or they require first to monotonize the data. In the literature, the monotonicity has been considered in linguistic fuzzy models, fuzzy-inference methods, and fuzzy rule-based control systems. However, to the best of our knowledge, there is no fuzzy rule-based system designed to produce monotonic fuzzy rule-based models for classification problems. In this paper, we propose to incorporate some mechanisms based on monotonicity indexes for addressing such problems in two popular and competitive evolutionary fuzzy systems algorithms for classification and regression tasks: FARC-HD and FSmogfse+Tune. In addition, the proposals are able to handle any kind of classification dataset without the necessity of preprocessing. The quality of our approaches is analyzed using statistical analysis and comparing with well-known monotonic classifiers.

Recently a number of evolutionary many-objective algorithms have been proposed using uniformly generated weight vectors. Those algorithms can be viewed as improved versions of MOEA/D with the PBI (penalty-based boundary intersection) function. Reference lines are uniformly specified using the weight vectors in the normalized objective space. The basic idea of those algorithms is to find a single solution along each reference line. Whereas a different search mechanism has been devised in each algorithm, solution assignment to reference lines is commonly based on the distance to the nearest reference line. This solution assignment can be interpreted as using a larger penalty value in MOEA/D with PBI. Actually, MOEA/D with PBI works well on frequently-used many-objective test problems DTLZ1-4 when a large penalty value is used. However, the shape of the contour lines of the PBI function suggests the use of a small penalty value for many-objective problems. Moreover, good results have been reported in the literature for many-objective knapsack problems when a small penalty value was used. In this paper, we discuss why good results are obtained from a large penalty value from a viewpoint of characteristics of DTLZ1-4 as test problems. Our discussions on the use of a large penalty value also explain why good results are obtained by recently-proposed weight vector-based evolutionary many-objective algorithms.

Recently NSGA-III has been frequently used for performance comparison of newly proposed evolutionary many-objective optimization algorithms. That is, NSGA-III has been used as a benchmark algorithm for evolutionary many-objective optimization. However, unfortunately, its source code is not available from the authors of the NSGA-III paper. This leads to an undesirable situation where a different implementation is used in a different study. Moreover, comparison is usually performed on DTLZ and WFG test problems. As a result, the performance of NSGA-III on a wide variety of many-objective test problems is still unclear whereas it has been frequently used for performance comparison in the literature. In this paper, we evaluate the performance of NSGA-III in comparison with NSGA-II on four totally different types of many-objective test problems with 3-10 objectives: DTLZ1-4 problems, their maximization variants, distance minimization problems, and knapsack problems. We use two different implementations of NSGA-II and NSGA-III. We show through computational experiments that NSGA-III does not always outperform NSGA-II even for ten-objective problems. That is, their comparison results depend not only on the number of objectives but also on the type of test problems. The choice of test problems has a larger effect on their comparison results than the number of objectives in our computational experiments. We also demonstrate that totally different results are obtained from different implementations of NSGA-III for some test problems.

The inverted generational distance (IGD) indicator has been frequently used for performance evaluation of many-objective algorithms. In this paper, we discuss the sensitivity of performance evaluation results by the IGD to the specification of a reference point set. Through computational experiments, we demonstrate that misleading evaluation results can be obtained by the use of the IGD. The reason for the misleading evaluation results is that the IGD tends to favor a solution set with a similar distribution to the reference point set. We demonstrate that such an undesirable bias of the IGD can be decreased by increasing the size of the reference point set and the size of solution sets to be compared. It is also shown that the bias can be decreased by using a modified IGD indicator called the inverted generational distance plus (IGD+). However, the bias becomes more severe by increasing the number of objectives. Our experimental results clearly demonstrate the necessity of very careful examination of performance comparison results by the IGD and IGD+indicators.

In the evolutionary multi-objective optimization community, algorithm comparison is usually performed under the same population size. However, this is not always fair because its best specification is usually different in each algorithm. In many-objective optimization, the number of solutions to be found may depend on the situation. If the decision maker wants to analyze the entire Pareto front, thousands of solutions may be needed. If the decision maker wants to choose a single final solution from some candidates after their quick checks, only a small number of representative solutions may be needed. In this paper, we discuss how to evaluate the ability of evolutionary many-objective optimization algorithms to find an arbitrarily specified number of non-dominated solutions. Our idea is the use of solution selection after the termination of each algorithm. We examine two scenarios: One is solution selection from the final population, and the other is from all of the examined solutions. Through computational experiments, first we demonstrate that performance comparison heavily depends on the population size. Then we examine the effects of solution selection from the final population and the examined solutions on comparison results.

A multiobjective test problem is called “scalable” when the number of its objectives can be arbitrarily specified. Evolutionary many-objective optimization algorithms are usually evaluated using scalable test problems. However, their design is not easy due to the difficulty in formulating a Pareto front and a feasible region in a high-dimensional objective space. As a result, a wide variety of scalable test problems have not been proposed. First, in this paper, existing scalable test problems are examined from some new viewpoints such as the uniqueness of the optimal solution for an objective and the presence of an optimal solution for multiple objectives. It is shown that most existing scalable test problems have some common properties. Next, a new framework for test problem design is proposed. The proposed framework enables us to design the Pareto front in a highly flexible manner. More specifically, we can specify not only its curvature property (e.g., convex, concave and linear) but also its shape (e.g., triangle, rotated triangle, pentagon and hexagon). The feasible region of the objective space can be also designed flexibly. Finally, some scalable test problems are generated by the proposed framework. It is clearly shown that the generated test problems have totally different properties from existing scalable test problems.

Pittsburgh-style genetics-based machine learning (GBML) algorithms have strong search ability for obtaining rule-based classifiers. However, when we apply them to data mining from large data, we need huge computation time for fitness evaluation. In our previous studies, we have proposed parallel distributed implementation of fuzzy GBML for fuzzy classifier design from large data. The basic idea of our parallel distributed implementation is to divide not only a population but also a training data set into N sub-populations and N training data subsets, respectively. A pair of a sub-population and a training data subset is assigned to each of N CPU cores in a workstation or a cluster. This dual division strategy achieved a quadratic speedup (i.e., N2times faster than the use of a single CPU core) while maintaining the generalization ability on test data. In this paper, we apply our parallel distributed implementation to GAssist which is a non-fuzzy Pittsburgh-style GBML algorithm. We examine the effects of the number of divisions on the search ability comparing with the parallel distributed fuzzy GBML.

Evolution of cooperation has been actively studied in the evolutionary computation (EC) community mainly for the iterated prisoner's dilemma (IPD) game. One of the frequently examined settings is a noisy environment where a player chooses a different action from the suggested one by its strategy with a pre-specified error probability. The use of the error probability in the IPD game usually makes the evolution of cooperation very difficult. This is because occasional defection by error disturbs mutual cooperation. In our former study, we examined the effect of the error probability on the evolution of cooperation among players with binary string strategies under various settings of memory length. Then we found that a higher average payoff was obtained by increasing the error probability from zero to a small value only when we used 7-bit binary string strategies with a memory about the opponent's previous two actions. That is, a small error probability helped the evolution of cooperation only under this particular setting. This behavior was not observed in the other settings of memory usage. In this paper, we further examine this behavior through computational experiments for a wide variety of settings of various factors. Experimental results show that this behavior is observed only under special settings of the following factors: the number of players (i.e., population size), memory length, memory usage, and a crossover operator.

Selecting a proper set of test problems is essential for fair performance comparison of evolutionary multi-objective optimization (EMO) algorithms. This is because the comparison results strongly depend on the choice of test problems. Test problems are also very important for examining the behavior of each algorithm. In general, it is advisable to prepare a set of various test problems including both easy and difficult ones for each algorithm. Our idea is to use a meta-optimization technique for generating such a set of test problems. More specifically, we use a two-level meta-optimization model. In the upper level, test problems are optimized. That is, test problems are handled as solutions. In the lower level, each test problem is evaluated using multiple EMO algorithms. The point of our idea is high flexibility in the definition of an objective function in the upper level. For example, when we want to design a difficult test problem only for a particular EMO algorithm, the minimization of its relative performance can be used as an objective function. By maximizing its relative performance, we can also design an easy test problem only for that algorithm. By generating both easy and difficult problems for each algorithm in this manner, we can prepare an appropriate test problem set for fair performance comparison. Through computational experiments, we demonstrate that we can generate a wide variety of test problems, each of which is difficult for a different type of EMO algorithms.

Classifier design for a classification problem with M classes can be viewed as finding an optimal partition of its pattern space into M disjoint subspaces. However, this is not always a good strategy especially when training patterns from different classes are heavily overlapping in the pattern space. A simple but practically useful idea is the use of a reject option. In this case, the pattern space is partitioned into (M+1) disjoint subspace where the classification of new patterns is rejected in the (M+1)th subspace. In this paper, we discuss the design of fuzzy rule-based classifiers with a reject option. The rejection subspace is specified by a threshold value for the difference of a kind of matching degrees between the best matching class and the second best matching class. The important research question is how to specify the threshold value. We examine the following two approaches: One is manual specification after designing a fuzzy rule-based classifier, and the other is simultaneous multiobjective optimization of a threshold value and a fuzzy rule-based classifier. In the latter approach, we use three objectives: maximization of the correct classification, and minimization of the rejection and the complexity of the classifier.

Recently a number of evolutionary multi-objective optimization (EMO) algorithms have been proposed using the framework of MOEA/D (multi-objective evolutionary algorithm based on decomposition). Those algorithms are characterized by the use of uniformly distributed normalized weight vectors from which a set of uniformly distributed reference lines is generated. Their basic idea is to search for a Pareto optimal solution along each reference line. While they are different in various aspects such as fitness evaluation, solution assignment to each reference line, and solution replacement, they share the same basic idea (i.e., to search for a Pareto optimal solution along each reference line). The importance of weight vector specification has been emphasized in the literature. However, the specification of a reference point has not been examined in detail whereas it plays an important role as a starting point of all reference lines. The reference point usually consists of the best value of each objective over the examined solutions, which is an approximation of the ideal point. However, this approximation is not good in early generations where the true ideal point may be much better than the best value of each objective over the examined solutions (even if it is very good in later generations). Based on these discussions, we propose a reference point specification method.

In general, an M-objective continuous optimization problem has an (M - 1)-dimensional Pareto front in the objective space. If its dimension is smaller than (M - 1), it is called a degenerate Pareto front. Deb-Thiele-Laumanns-Zitzler (DTLZ)5 and Walking Fish Group (WFG)3 have often been used as many-objective continuous test problems with degenerate Pareto fronts. However, it was noted that DTLZ5 has a nondegenerate part of the Pareto front. Constraints have been proposed to remove the nondegenerate part. In this letter, first we show that WFG3 also has a nondegenerate part. Then, we derive constraints to remove the nondegenerate part. Finally, we show that the existence of the nondegenerate part makes WFG3 an interesting test problem through computational experiments.

When the performance of different evolutionary multiobjective optimization (EMO) algorithms is compared, the same population size is usually used for all EMO algorithms in computer simulations. This setting is to obtain a solution set of the same size from a different algorithm. However, in general, each algorithm may have its own best parameter specifications for each test problem. Thus, it may be difficult to appropriately specify the same population size for all algorithms for their fair comparison. A different algorithm may be evaluated as being the best for a different specification of the population size. An alternative setting is to allow each algorithm to use its own best population size. In this setting, a solution set of different size is obtained from each algorithm. It may be difficult to perform fair comparison using solution sets of different size. In this paper, we discuss the difficulty in comparing EMO algorithms under these two settings of the population size: the same specification for all algorithms and a different specification for each algorithm. First, we discuss the effect of the number of non-dominated solutions on some performance indicators. Next we show the difficulty in the first setting: Performance of each algorithm depends on the population size. Then we discuss the difficulty in the second setting: The size of a solution set obtained by each algorithm is not the same. In this setting, we examine the use of solution selection as a post-processing procedure to choose the same number of solutions from each solution set of different size. The selected solutions are used for performance comparison.

Fuzzy genetics-based machine learning (FGBML) has frequently been used for fuzzy classifier design. It is one of the promising evolutionary machine learning (EML) techniques from the viewpoint of data mining. This is because FGBML can generate accurate classifiers with linguistically interpretable fuzzy if-then rules. Of course, a classifier with tens of thousands of if-then rules is not linguistically understandable. Thus, the complexity minimization of fuzzy classifiers should be considered together with the accuracy maximization. In previous studies, we proposed hybrid FGBML and its multiobjective formulation (MoFGBML) to handle both the accuracy maximization and the complexity minimization simultaneously. MoFGBML can obtain a number of non-dominated classifiers with different tradeoffs between accuracy and complexity. In this paper, we focus on heuristic rule generation in MoFGBML to improve the search performance. In the original heuristic rule generation, each if-then rule is generated from a randomly-selected training pattern in a heuristic manner. This operation is performed at population initialization and during evolution. To generate more generalized rules according to the training data, we propose new heuristic rule generation where each rule is generated from multiple training patterns. Through computational experiments using some benchmark data sets, we discuss the effects of the proposed operation on the search performance of our MoFGBML.

In our previous study, we proposed an ensemble action selection model where each player has multiple strategies with different memory length for the iterated prisoner's dilemma (IPD) game. An action was suggested by each strategy based on its memory about opponent's single, two or three actions. Majority vote was used for action selection. Under these settings, the evolution of cooperation was examined for various ensembles (i.e., various combinations of strategies). In this paper, we extend our ensemble model to a more general case where strategies have different memory usage. Each strategy of a player has a memory of opponent's and/or player's previous actions. For example, a memory of a strategy can be opponent's single and player's two actions. Another strategy's memory can be player's three actions. Various combinations of strategies for ensemble action selection are examined in this paper. It is shown through computational experiments that the use of ensemble action selection enhances the evolution of cooperation. It is also shown that no cooperation is evolved among strategies with no memory about opponent's actions. An interesting observation is that cooperation is evolved by players with the combination of the following three strategies: two strategies with no memory about opponent's actions, and a single strategy with a memory of both player's and opponent's actions.

The prisoner's dilemma (PD) game is a well-known non-zero sum game. Its iterated version (IPD game) has been widely used to study the evolution of cooperative strategies. In this paper, we assume a noisy environment where a player chooses a different action from the suggested one by its own strategy with a pre-specified error probability. Generally, the noise in action selection makes the evolution of cooperation difficult because the player cannot distinguish between an intentional defection by the opponent's strategy and an unintentional defection by error. However, when a 7-bit binary string with a memory about opponent's two actions was used as a strategy of each player, we observed strange evolution behavior where the use of a small error probability increased the average payoff to the level close to the complete mutual cooperation. That is, the use of a small error probability seems to help the evolution of cooperation. Such a strange behavior was not clearly observed by other types of strategies (e.g., 3-bit binary string with a memory about opponent's single action, 15-bit binary strings with a memory about opponent's three actions). In this paper, we report our simulation results where our focus is placed on the strange evolution behavior of 7-bit binary string strategies. We also try to analyze their strange evolution behavior.

An important implementation issue in the design of hybrid evolutionary multiobjective optimization algorithms such as multiobjective genetic local search (MOGLS) is how to combine local search with evolutionary algorithms. It has been demonstrated that the performance of MOGLS strongly depends on the order of global search and local search. A balance between local search and global search also affects its search ability. We can use three ideas for designing high-performance MOGLS algorithms. One idea is to choose one of two options: local search after global search or global search after local search. In general, their appropriate order depends on the problem. Another idea is to use tuned parameter values to appropriately specify their balance. The other idea is to change both their order and the parameter values during the execution of MOGLS. This idea can be implemented by dividing the whole search period into some sub-periods (i.e., dividing all generations into some intervals of generations). The appropriate order and parameter values are assigned to each sub-period. In this paper, we propose off-line algorithm structure optimization for MOGLS. The effectiveness of the proposed idea is examined by computational experiments on a two-objective knapsack problem and a two-objective flowshop scheduling problem. Based on experimental results, we discuss the importance of structure optimization of MOGLS.

In this paper, we assume that we have two types of datasets for classifier design. One is an in-house dataset which is fully available for classifier design as training data. The other is an external dataset which is kept under a very severe privacy preserving policy. We assume that the available information on the external dataset is only the error rate of a presented classifier. No other information is available such as the number of patterns, attribute values of each pattern, and its class label. Thus, the external dataset can be viewed as a black-box model where the error rate is calculated as an output for an input classifier. In this paper, we discuss how such a black-box type dataset can be utilized in fuzzy genetics-based machine leaning (GBML). We use a hybrid fuzzy GBML algorithm where its Michigan-style part is applied to each individual of a Pittsburgh-style part. Since a fuzzy rule-based classifier is an individual in the Pittsburgh-style part, a black-box type dataset can be utilized for fitness evaluation. Through computational experiments, we examine the effect of using a black-box type dataset in comparison with fuzzy rule-based classifiers design only from a fully available dataset.

Fuzzy genetics-based machine learning (FGBML) is one of the representative approaches to obtain a set of fuzzy if-then rules by evolutionary computation. A number of FGBML methods have been proposed so far. Among them, Michigan-style approaches are popular thanks to thier lower computational cost than Pittsburgh approaches. In this study, we introduce two simple modifications for our Michigan-style FGBML. One is related to heuristic rule generation. In the original FGBML, each rule in an initial population is generated from a randomly-selected training pattern in a heuristic manner. The heuristic rule generation also performs during evolution where each rule is generated from a misclassified pattern. As its modification, we propose the use of multiple patterns to generate each fuzzy if-then rule. The other is related to the fitness calculation. In the original FGBML, the fitness of each rule is calculated as the number of correctly classified training patterns, while the number of misclassified patterns is ignored. As its modification, we incorporate a penalty term into the fitness function. Through computational experiments using 20 benchmark data sets, we examine the effects of these two modifications on the search ability of our Michigan-style FGBML.

The inverted generational distance (IGD) has been frequently used as a performance indicator for many-objective problems where the use of the hypervolume is difficult. However, since IGD is not Pareto compliant, it is possible that misleading Pareto incompliant results are obtained. Recently, a simple modification of IGD was proposed by taking into account the Pareto dominance relation between a solution and a reference point when their distance is calculated. It was also shown that the modified indicator called IGD+ is weakly Pareto compliant. However, actual effects of the modification on performance comparison have not been examined. Moreover, IGD+ has not been compared with other distance-based weakly Pareto compliant indicators such as the additive epsilon indicator and the D1 indicator (i.e., IGD with the weighted achievement scalarizing function). In this paper, we examine the effect of the modification by comparing IGD+ with IGD for multiobjective and many-objective problems. In computational experiments, we generate a large number of ordered pairs of non-dominated solution sets where one is better than the other. Two solution sets in each pair are compared by the above-mentioned performance indicators. We examine whether each indicator can correctly say which solution set is better between them.

We examine the behavior of three classes of evolutionary multiobjective optimization (EMO) algorithms on many-objective knapsack problems. They are Pareto dominance-based, scalarizing function-based, and hypervolume-based algorithms. NSGA-II, MOEA/D, SMS-EMOA, and HypE are examined using knapsack problems with 2-10 objectives. Our test problems are generated by randomly specifying coefficients (i.e., profits) in objectives. We also generate other test problems by combining two objectives to create a dependent or correlated objective. Experimental results on randomly generated many-objective knapsack problems are consistent with well-known performance deterioration of Pareto dominance-based algorithms. That is, NSGA-II is outperformed by the other algorithms. However, it is also shown that NSGA-II outperforms the other algorithms when objectives are highly correlated. MOEA/D shows totally different search behavior depending on the choice of a scalarizing function and its parameter value. Some MOEA/D variants work very well only on two-objective problems while others work well on many-objective problems with 4-10 objectives. We also obtain other interesting observations such as the performance improvement by similar parent recombination and the necessity of diversity improvement for many-objective knapsack problems.

Various evolutionary multiobjective optimization (EMO) algorithms have been proposed in the literature. They have different search mechanisms for increasing the diversity of solutions and improving the convergence to the Pareto front. As a result, each algorithm has different characteristics in its search behavior. Multiobjective search behavior can be visually shown in an objective space for a test problem with two or three objectives. However, such a visual examination is difficult in a high-dimensional objective space for many-objective problems. The use of distance minimization problems has been proposed to examine many-objective search behavior in a two-dimensional decision space. This idea has an inherent limitation: the number of decision variables should be two. In our former study, we formulated a four-objective distance minimization problem with 10, 100, and 1000 decision variables. In this paper, we generalize our former study to many-objective problems with an arbitrary number of objectives and decision variables by proposing an idea of specifying reference points on a plane in a high-dimensional decision space. As test problems for computational experiments, we generate six-objective and eight-objective problems with 10, 100, and 1000 decision variables. Our experimental results on those test problems show that the number of decision variables has large effects on multiobjective search in comparison with the choice of an EMO algorithm and the number of objectives.

Iterated prisoner's dilemma (IPD) games have been frequently used for examining the evolution of cooperative game strategies. It has been pointed out in some studies that the choice of a representation scheme (i.e., coding mechanism) has a large effect on the evolution. A choice of a different representation scheme often leads to totally different results. In those studies on IPD games, a single representation scheme is assigned to all players. That is, all players have the same representation scheme. In our former studies, we reported experimental results in an inhomogeneous setting where a different representation scheme was assigned to each player. The evolution of cooperation among different types of game strategies was examined. In this paper, we report experimental results in another interesting setting where each player is assumed to have multiple strategies with different representation schemes. The next action of each player is determined by a majority vote by its strategies. That is, each player is assumed to have an ensemble decision making system. Experimental results in such an ensemble IPD model are compared with those in the standard IPD model where each player has a single strategy.

Discretization of continuous attributes is a key issue in classifier design from numerical data. In the machine learning community, continuous attributes are discretized into intervals. An entropy measure is often used to determine the cutting points for interval discretization. In the fuzzy system community, continuous attributes are usually discretized into overlapping fuzzy sets. Learning and optimization techniques are used to adjust the membership function of each fuzzy set. One interesting research issue is a comparison between interval partitions and fuzzy partitions. We address this issue by using an entropy-based interval discretization method in hybrid fuzzy genetics-based machine learning (GBML). Our hybrid fuzzy GBML algorithm is applied to a number of data sets where interval discretization is fuzzified with different fuzzification grades from zero (i.e., interval partitions) to one (i.e., completely fuzzified partitions). Experimental results from various fuzzification grades are compared with each other.

A large number of non-dominated solutions are usually obtained as a result of a single run of an EMO (evolutionary multi-objective optimization) algorithm. When the number of objectives is two, all the obtained non-dominated solutions can be easily shown in the objective space. A single final solution is to be chosen by the decision maker from the presented solutions. The increase in the number of objectives makes it very difficult to present the obtained non-dominated solutions in a visually understandable manner. It is also very difficult for the decision maker to examine the presented solutions for choosing a single final solution when the number of objectives is large. These discussions suggest the use of a small population in an EMO algorithm. However, a large population is needed to search for the entire Pareto front of a many-objective problem. In this paper, we discuss the selection of a small number of solutions to be presented to the decision maker from a large number of the obtained non-dominated solutions. This is to satisfy the following two requests: (i) A large population is needed to search for the entire Pareto front, and (ii) the decision maker does not want to manually examine a large number of solutions. We propose a use of a two-step solution set selection approach. The first step is offline multi-objective optimization where a large number of nondominated solutions are obtained. The second step is solution set selection where only a small number of solutions are chosen from a large number of obtained solutions. The selected solutions are presented to the decision maker. We explain some strategies for solution set selection in the second step. Our focus is not how to choose a single final solution but how to select a small number of promising solutions to be presented to the decision maker.

Expert judgment is widely used for activity duration estimation in software project management. While there are both advantages and disadvantages of expert judgment-based estimation, we propose the use of fuzzy inference rules for semi-automatic estimation to reduce the potential negative aspects of the expert judgment-based estimation. Fourteen fuzzy inference rules are introduced to elicit and adjust expert tacit knowledge, and expert judgment-based estimation results are complemented by fuzzy inference rules. The results from expert judgment and fuzzy inference rules are compared with the expert judgment-based approach using surveys and one-on-one interviews with project managers from different disciplines through analyses with data from past software projects. The use of fuzzy inference rules improves the estimation accuracy of the expert judgment-based approach by 39.35%. The proposed approach facilitates the experts to derive a more realistic and reliable activity duration estimation in software project management.

This paper discusses parallel distributed genetics-based machine learning (GBML) of fuzzy rule-based classifiers from multiple data sets. We assume that each data set has a similar but different set of attributes. In other words, each data set has different missing attributes. Our task is the design of a fuzzy rule-based classifier from those data sets. In this paper, we first show that fuzzy rules can handle missing attributes easily. Next we explain how parallel distributed fuzzy GBML can handle multiple data sets with different missing attributes. Then we examine the accuracy of obtained fuzzy rule-based classifiers from various settings of available training data such as a single data set with no missing attribute and multiple data sets with many missing attributes. Experimental results show that the use of multiple data sets often increases the accuracy of obtained fuzzy rule-based classifiers even when they have missing attributes. We also discuss the learning from a data set under a severe privacy preserving policy where only the error rate of each candidate classifier is available. It is assumed that no information about each individual pattern is available. This means that we cannot use any information on the class label or the attribute values of each pattern. We explain how such a black-box data set can be utilized for classifier design.

An important implementation issue in the design of hybrid evolutionary multiobjective optimization algorithms with local search (i.e., multiobjective memetic algorithms) is how to strike a balance between local search and global search. If local search is applied to all individuals at every generation, almost all computation time is spent by local search. As a result, global search ability of memetic algorithms is not well utilized. We can use three ideas for decreasing the computation load of local search. One idea is to apply local search to only a small number of individuals. This idea can be implemented by introducing a local search probability, which is used to choose only a small number of initial solutions for local search from the current population. Another idea is a periodical (i.e., intermittent) use of local search. This idea can be implemented by introducing a local search interval (e.g., every 10 generations), which is used to specify when local search is applied. The other idea is an early termination of local search. Local search for each initial solution is terminated after a small number of neighbors are examined. This idea can be implemented by introducing a local search length, which is the number of examined neighbors in a series of iterated local search from a single initial solution. In this paper, we discuss the use of these three ideas to strike a local-global search balance. Through computational experiments on a two-objective 500-item knapsack problem, we compare various settings of local search such as short local search from all individuals at every generation, long local search from only a few individuals at every generation, and periodical long local search from all individuals. Global search in this paper means genetic search by crossover and mutation in multiobjective memetic algorithms.

In the development of evolutionary multiobjective optimization (EMO) algorithms, it is important to implement a good balancing mechanism between the convergence of solutions towards the Pareto front and their diversity over the Pareto front. When an EMO algorithm is applied to a two-objective problem, the balance can be easily visualized by showing all solutions at each generation in the two-dimensional objective space. However, such a visual examination of the multiobjective search is difficult for many-objective problems with four or more objectives. The use of many-objective test problems with two decision variables has been proposed in some studies to visually examine the search behavior of EMO algorithms. Such test problems are defined by a number of points in a two-dimensional decision space where the distance minimization from each point is an objective. Thus the number of objectives is the same as the number of points. The search behavior of EMO algorithms can be visually examined in the two-dimensional decision space. In this paper, we propose the use of many-objective test problems for visual examination of the search behavior in a high-dimensional decision space. More specifically, our m-objective test problem with n variables is generated by specifying m points on a plane in an n-dimensional decision space. We examine the behavior of EMO algorithms through computational experiments on such an m-objective n-variable test problem. Our experimental results show that the number of variables has a large effect on the search behavior of EMO algorithms with respect to the diversity of solutions.

In our former study, we have already proposed a parallel distributed model for the speedup of fuzzy genetics-based machine learning (GBML). Our model is an island model for parallel implementation of fuzzy GBML algorithms where a population is divided into multiple subpopulations. A single subpopulation is assigned to each island. Training data are also divided and distributed over the islands. When we have N islands (i.e., N CPUs for parallel computation), the speedup is the order of the square of N. This is because both the population and the training data are divided into N subsets. One characteristic feature of our parallel distributed model is training data rotation over the islands. Each of the N training data subsets is assigned to one of the N islands. The assigned training data subsets are rotated over the islands periodically (e.g., every 100 generations). This means that the environment of each island is changed periodically. The focus of this paper is how to update existing fuzzy rules at each island after the training data rotation. One extreme setting is to totally update fuzzy rules using the newly assigned training data subset. Another extreme setting is to use existing fuzzy rules with no changes. In this paper, we examine incremental learning, which can be viewed as an intermediate mechanism between the two extreme settings.

Various settings of iterated prisoner's dilemma (IPD) games have been studied to examine the evolution of cooperation among players in the literature. In recent studies, players are often spatially placed in a network. Each player plays the IPD game against its neighbors, which are defined by connections between players in the network. As in the case of cellular IPD games in a two-dimensional lattice, the number of opponents is usually very small in network-based IPD games. In this paper, we examine the effect of the number of opponents on the evolution of cooperation through computational experiments using various networks. The main feature of our computational experiments is to randomly choose a pre-specified number of opponents for each player from its neighbors in one setting and from all players in another setting. In this manner, we examine a wide range of specifications of the number of opponents.

Over the past few decades, fuzzy systems have been widely used in several application fields, thanks to their ability to model complex systems. The design of fuzzy systems has been successfully performed by applying evolutionary and, in particular, genetic algorithms, and recently, this approach has been extended by using multiobjective evolutionary algorithms, which can consider multiple conflicting objectives, instead of a single one. The hybridization between multiobjective evolutionary algorithms and fuzzy systems is currently known as multiobjective evolutionary fuzzy systems. This paper presents an overview of multiobjective evolutionary fuzzy systems, describing the main contributions on this field and providing a two-level taxonomy of the existing proposals, in order to outline a well-established framework that could help researchers who work on significant further developments. Finally, some considerations of recent trends and potential research directions are presented.

We propose a parallel distributed model of a hybrid fuzzy genetics-based machine learning (GBML) algorithm to drastically decrease its computation time. Our hybrid algorithm has a Pittsburgh-style GBML framework where a rule set is coded as an individual. A Michigan-style rule-generation mechanism is used as a kind of local search. Our parallel distributed model is an island model where a population of individuals is divided into multiple islands. Training data are also divided into multiple subsets. The main feature of our model is that a different training data subset is assigned to each island. The assigned training data subsets are periodically rotated over the islands. The best rule set in each island also migrates periodically. We demonstrate through computational experiments that our model decreases the computation time of the hybrid fuzzy GBML algorithm by an order or two of magnitude using seven parallel processors without severely degrading the generalization ability of obtained fuzzy rule-based classifiers. We also examine the effects of the training data rotation and the rule set migration on the search ability of our model.

Evolution of cooperative behavior has been examined in many studies on the IPD (Iterated Prisoner's Dilemma) game under various conditions. In some studies, each agent is allowed to play against itself. However, this setting is somewhat strange because we do not play any real-world games against ourselves. In this paper, we examine the effect of this somewhat strange setting on the evolution of cooperative behavior in a spatial IPD game. Two cases are compared with each other: Each agent is allowed to play against itself in one case and not allowed to do so in the other case. It is shown through computational experiments that similar results are obtained from these two cases when opponents of each agent are selected from a large number of its neighbors. However, the difference between the two cases is large when the number of neighbors is small. Actually the evolution of cooperative behavior is strongly facilitated by allowing each agent to play against itself when the number of neighbors is small. Our computational experiments are performed on a spatial IPD game with various specifications of the neighborhood size where binary and real number strings are used as game strategies.

We examine the evolution of strategies for a spatial IPD (Iterated Prisoner's Dilemma) game, which are encoded using different representation schemes. Each agent at a cell in a two-dimensional grid-world has its own representation scheme for encoding its strategy. In general, strategies with different representation schemes cannot be recombined. Thus a population of agents can be viewed as a mixture of different species (i.e., an ecology with different species). When the size of a neighborhood structure is small and/or the number of representation schemes is large, it is likely that some agents have no neighbors with the same representation scheme. We discuss the handling of those agents because they cannot generate their new strategies through recombination. In computational experiments, we use four types of strings (i.e., four representation schemes). Agents in our spatial IPD game are randomly divided into four groups of the same size. One string type is assigned to each group. Recombination is performed between strings of neighboring agents with the same string type. With respect to the IPD game, we compare two settings with each other. In one setting, the IPD game is played between any pair of neighboring agents regardless of their string types. In the other setting, it is played only between neighboring agents with the same string type. Using these two settings, we examine the effect of the IPD game between agents with different representation schemes on strategy evolution. We also examine the effect of the number of different representation schemes in a population (i.e., the number of species) on strategy evolution.

Real world data sets are often imbalanced with respect to the class distribution. Classifier design from those data sets is relatively new challenge. The main problem is the lack of positive class patterns in the data sets. To deal with this problem, there are two main approaches. One is to additionally sample minority class patterns (i.e., over-sampling). The other is to sample a part of majority class patterns (i.e., under-sampling). In our previous research, we have proposed a parallel distributed genetics-based machine learning for large data sets. In our method, not only a population but also a training data set is divided into subgroups, respectively. A pair of a sub-population and a training data subset is assigned to an individual CPU core in order to reduce the computation time. In this paper, our parallel distributed approach is applied to imbalanced data sets. The training data subsets are constructed by a composition of subsets divided majority class patterns with the entire set of non-divided minority class patterns. Through computational experiments, we show the effectiveness of our parallel distributed approach with the proposed data subdivision schemes for imbalanced data sets.

Objective functions in combinatorial optimization are discrete. The number of possible values of a discrete objective function is totally different from problem to problem. Optimization of a discrete objective function is often very difficult. In the case of multiobjective optimization, a different objective function has a different number of possible values. This means that each axis of the objective space has a different granularity. Some axes may have fine granularities while others are coarse. In this paper, we examine the effect of discrete objective functions with different granularities on the search behavior of EMO (evolutionary multiobjective optimization) algorithms through computational experiments. Experimental results show that a discrete objective function with a coarse granularity slows down the search of EMO algorithms along that objective. An interesting observation is that such a slow-down along one objective often leads to the speed-up of the search along other objectives. We also examine the effect of adding a small random noise to each discrete objective function in order to increase the number of possible objective values.

Recently it has been pointed out in many studies that evolutionary multi-objective optimization (EMO) algorithms with Pareto dominance-based fitness evaluation do not work well on many-objective problems with four or more objectives. In this paper, we examine the behavior of well-known and frequently used EMO algorithms such as NSGA-II, SPEA2 and MOEA/D on many-objective problems with correlated or dependent objectives. First we show that good results on many-objective 0/1 knapsack problems with randomly generated objectives are not obtained by Pareto dominance-based EMO algorithms (i.e., NSGA-II and SPEA2). Next we show that the search ability of NSGA-II and SPEA2 is not degraded by the increase in the number of objectives when they are highly correlated or dependent. In this case, the performance of MOEA/D is deteriorated. As a result, NSGA-II and SPEA2 outperform MOEA/D with respect to the convergence of solutions toward the Pareto front for some many objective problems. Finally we show that the addition of highly correlated or dependent objectives can improve the performance of EMO algorithms on two-objective problems in some cases.

Explanation ability of a fuzzy rule-based classifier is its ability to explain why an input pattern is classified as a particular class in a convincing way. This ability is important especially when fuzzy rule-based classifiers are used as support systems for human users. This is because human users often want to know why the current input pattern is classified as a particular class. The explanation ability looks similar to the interpretability. They are, however, clearly different concepts. Whereas the explanation ability is directly related to the classification of each pattern, the interpretability is usually independent of classification results. The interpretability has been taken into account in multiobjective design of fuzzy rule-based classifiers. However, the explanation ability has not been used for fuzzy rule-based classifier design. This is because its quantitative definition is very difficult. In this paper, we discuss various factors that are related to quantitative definition of the explanation ability of fuzzy rule-based classifiers. Using simple numerical examples, we explain that the complexity minimization of fuzzy rule-based classifiers does not always lead to the explanation ability maximization. We also explain that the accuracy of fuzzy rules is related to the explanation ability.

Tens of thousands of classifiers have been proposed so far. There is no best classifier among them for all the existing data sets. The performance of each classifier often depends on the data sets used for comparison. Even for a single classifier, suitable parameters of the classifier also depend on the data sets. That is, there is a possibility that a suited classifier and its parameter specification can be chosen beforehand if the target data sets or their characteristics were known. In recent years, a number of data complexity measures have been proposed to characterize data sets. The aim of this study is to develop a meta classifier for selecting an appropriate classifier and/or its appropriate parameter specification by means of data complexity measures. In this paper, we focus on the parameter specification of fuzzy classifiers using data complexity measures as a preliminary study. To construct a meta-classifier, we generate a large number of artificial data sets from Keel benchmark data sets. Then we generate meta-patterns which are composed of the values of data complexity measures as inputs and an appropriate fuzzy partition as an output. Using meta-patterns, a meta classifier is designed by multiobjective genetic fuzzy rule selection. We evaluate the proposed method through leave one-group out cross-validation.

Recently distance minimization problems in a two-dimensional decision space have been utilized as many-objective test problems to visually examine the behavior of evolutionary multi-objective optimization (EMO) algorithms. Such a test problem is usually defined by a single polygon where the distance from a solution to each vertex is minimized in the decision space. We can easily generate different test problems from different polygons. We can also easily generate test problems with multiple equivalent Pareto optimal regions using multiple polygons of the same shape and the same size. Whereas these test problems have a number of advantages, they have no clear relevance to real-world situations since they are artificially generated unrealistic test problems. In this paper, we generate a distance minimization problem from a real-world map. Our test problem has four objectives, which are to minimize the distances to the nearest elementary school, junior high school, railway station, and convenience store. Using our test problem, we examine the behavior of well-known and frequently-used EMO algorithms in terms of their diversity maintenance ability in the two-dimensional decision space.

The iterated prisoner's dilemma (IPD) game has been frequently used to examine the evolution of cooperative behavior among agents in the field of evolutionary computation. It has been demonstrated that various factors are related to the evolution of cooperative behavior. One well-known factor is spatial relations among agents. The IPD game is often played in a 2-D grid world. Such a spatial IPD game has a neighborhood structure, which is used to choose opponents for the IPD game and parents for genetic operations. Another important factor is the choice of a representation scheme to encode the strategy of each agent. Different representation schemes often lead to different results. Whereas the choice of a representation scheme is known to be important, a mixture of different representation schemes has not been examined for the spatial IPD game in the literature. That is, a population of homogeneous agents with the same representation scheme has been usually assumed in the literature. In this paper, we introduce the use of different representation schemes in a single population to the spatial IPD game in order to examine the evolution of cooperative behavior under more general assumptions. With the use of different representation schemes, we can examine the evolution of cooperative behavior in various settings such as partial interaction through the IPD game, partial interaction through crossover, full interaction through the IPD game and crossover, and no interaction between different subpopulations of agents.

Evolutionary algorithms have been actively applied to knowledge discovery, data mining and machine learning under the name of genetics-based machine learning (GBML). The main advantage of using evolutionary algorithms in those application areas is their flexibility: Various knowledge extraction criteria such as accuracy and complexity can be easily utilized as fitness functions. On the other hand, the main disadvantage is their large computation load. It is not easy to apply evolutionary algorithms to large data sets. The scalability improvement to large data sets is one of the main research issues in GBML. In our former studies, we proposed an idea of parallel distributed implementation of GBML and examined its effectiveness for genetic fuzzy rule selection. The point of our idea was to realize a quadratic speed-up by dividing not only a population but also training data. Training data subsets were periodically rotated over sub-populations in order to prevent each sub-population from over-fitting to a specific training data subset. In this paper, we propose the use of parallel distributed implementation for the design of ensemble classifiers. An ensemble classifier is designed by combining base classifiers, each of which is obtained from each sub-population. Through computational experiments on parallel distributed genetic fuzzy rule selection, we examine the generalization ability of designed ensemble classifiers under various settings with respect to the size of training data subsets and their rotation frequency.

Evolutionary multiobjective optimization (EMO) algorithms have often been used to search for a number of non-dominated fuzzy rule-based classifiers with respect to their accuracy and complexity. It is, however, pointed out in some studies that the entire accuracy-complexity tradeoff surface is not always found by well-known and frequently-used EMO algorithms such as NSGA-II. Especially it is very difficult for EMO algorithms to find fuzzy rule-based classifiers with high accuracy around the edge of the tradeoff surface. One simple idea for the design of accurate fuzzy rule-based classifiers is the use of fine fuzzy partitions with a number of small antecedent fuzzy sets. The use of fine fuzzy partitions usually improves the accuracy of fuzzy rule-based classifiers on training data. It may, however, have some side-effects such as the deterioration of classification accuracy on test data and the increase in the search space for fuzzy system design. In this paper, we examine the use of fine fuzzy partitions in the evolutionary multiobjective design of fuzzy rule-based classifiers. Experimental results show that the use of fine fuzzy partitions almost always increases the number of obtained non-dominated fuzzy rule-based classifiers, almost always improve their training data accuracy, and often improve their test data accuracy for some data sets. We also examine the relation between the granularity of fuzzy partitions and the number of antecedent conditions (i.e., rule length).

Data mining is a very active and rapidly growing research area in the field of computer science. Its goal is to obtain useful knowledge for users from a database. Association rule mining from a database is one of the most well-known data mining techniques. In general, a large number of if-then rules are extracted by specifying minimum support and confidence levels. They are, however, too complicated as knowledge for users to understand many rules at one time. Multiobjective genetic fuzzy rule selection from Pareto-optimal and near Pareto-optimal rules is a promising approach which can obtain an accurate and simple rule set by considering the accuracy maximization and the complexity minimization. In this paper, we propose two extensions of multiobjective genetic fuzzy rule selection for designing more accurate fuzzy rule-based classifiers. One extension is to add compatible rules with misclassified patterns into candidate rules for genetic fuzzy rule selection. The other is to tune membership functions after genetic fuzzy rule selection. We examine the effects of these extensions through computational experiments on imbalanced data sets.

The use of Pareto dominance for fitness evaluation has been the mainstream in evolutionary multiobjective optimization for the last two decades. Recently, it has been pointed out in some studies that Pareto dominance-based algorithms do not always work well on multiobjective problems with many objectives. Scalarizing function-based fitness evaluation is a promising alternative to Pareto dominance especially for the case of many objectives. A representative scalarizing function-based algorithm is MOEA/D (multiobjective evolutionary algorithm based on decomposition) of Zhang & Li (2007). Its high search ability has already been shown for various problems. One important implementation issue of MOEA/D is a choice of a scalarizing function because its search ability strongly depends on this choice. It is, however, not easy to choose an appropriate scalarizing function for each multiobjective problem. In this paper, we propose an idea of using different types of scalarizing functions simultaneously. For example, both the weighted Tchebycheff (Chebyshev) and the weighted sum are used for fitness evaluation. We examine two methods for implementing our idea. One is to use multiple grids of weight vectors and the other is to assign a different scalarizing function alternately to each weight vector in a single grid.

Pareto dominance-based algorithms have been the main stream in the field of evolutionary multiobjective optimization (EMO) for the last two decades. It is, however, well-known that Pareto-dominance-based algorithms do not always work well on many-objective problems with more than three objectives. Currently alternative frameworks are studied in the EMO community very actively. One promising framework is the use of an indicator function to find a good solution set of a multiobjective problem. EMO algorithms with this framework are called indicator-based evolutionary algorithms (IBEAs) where the hypervolume measure is frequently used as an indicator. IBEAs with the hypervolume measure have strong theoretical support and high search ability. One practical difficult of such an IBEA is that the hypervolume calculation needs long computation time especially when we have many objectives. In this paper, we propose an idea of using a scalarizing function-based hypervolume approximation method in IBEAs. We explain how the proposed idea can be implemented in IBEAs. We also demonstrate through computational experiments that the proposed idea can drastically decrease the computation time of IBEAs without severe performance deterioration.

In the design of evolutionary multiobjective optimization (EMO) algorithms, it is important to strike a balance between diversity and convergence. Traditional mask-based crossover operators for binary strings (e.g., one-point, two-point, and uniform) tend to decrease the spread of solutions along the Pareto front in EMO algorithms while they improve the convergence to part of the Pareto front. This is because such a crossover operator, which is called geometric crossover, always generates an offspring in the segment between its two parents under the Hamming distance in the genotype space. That is, the sum of the distances from the generated offspring to its two parents is always equal to the distance between the two parents. In this paper, we first propose a non-geometric binary crossover operator to generate an offspring outside the segment between its two parents. Next, we show some properties of our crossover operator. Then we examine its effects on the behavior of EMO algorithms through computational experiments on knapsack problems with two, four, and six objectives. Experimental results show that our crossover operator can increase the spread of solutions along the Pareto front in EMO algorithms without severely degrading their convergence property. As a result, our crossover operator improves some overall performance measures such as the hypervolume.

Cellular evolutionary algorithms usually use a single neighborhood structure for local selection. When a new solution is to be generated by crossover and/or mutation for a cell, a pair of parent solutions is selected from its neighbors. The current solution at the cell is replaced with the newly generated offspring if the offspring has the higher fitness value than the current one. That is, the ldquoreplace-if-betterrdquo policy is used for the replacement of the current solution. Local selection, crossover, mutation and replacement are iterated at every cell in cellular algorithms. A recently proposed multiobjective evolutionary algorithm called MOEA/D by Zhang and Li (2007) can be viewed as a cellular algorithm where each cell has its own scalarizing fitness function with a different weight vector. We can introduce a spatial structure to MOEA/D by the Euclidean distance between weight vectors. Its main difference from standard cellular algorithms is that a newly generated offspring for a cell is compared with not only the current solution of the cell but also its neighbors for local replacement in MOEA/D. In this paper, we examine the effect of local replacement on the search ability of a cellular version of MOEA/D. Whereas the same neighborhood structure was used for local selection and local replacement in the original MOEA/D, we examine the use of different neighborhood structures for local selection and local replacement. It is shown through computational experiments on multiobjective 0/1 knapsack problems with two, four and six objectives that local replacement plays an important role in MOEA/D especially for many-objective optimization problems.

This paper proposes an idea of approximating the hypervolume of a non-dominated solution set using a number of achievement scalarizing functions with uniformly distributed weight vectors. Each achievement scalarizing function with a different weight vector is used to measure the distance from the reference point of the hypervolume to the attainment surface of the non-dominated solution set along its own search direction specified by its weight vector. Our idea is to approximate the hypervolume by the average distance from the reference point to the attainment surface over a large number of uniformly distributed weight vectors (i.e., over various search directions). We examine the effect of the number of weight vectors (i.e., the number of search directions) on the approximation accuracy and the computation time of the proposed approach. As expected, experimental results show that the approximation accuracy is improved by increasing the number of weight vectors. It is also shown that the proposed approach needs much less computation time than the exact hypervolume calculation for a six-objective knapsack problem even when we use about 100,000 weight vectors.

Recently, multiobjective evolutionary algorithms have been applied to improve the difficult tradeoff between interpretability and accuracy of fuzzy rule-based systems. It is known that both requirements are usually contradictory, however, these kinds of algorithms can obtain a set of solutions with different trade-offs. The application of multiobjective evolutionary algorithms to fuzzy rule-based systems is often referred to as multiobjective genetic fuzzy systems. The first study on multiobjective genetic fuzzy systems was multiobjective genetic fuzzy rule selection in order to simultaneously achieve accuracy maximization and complexity minimization. This approach is based on the generation of a set of candidate fuzzy classification rules by considering a previously fixed granularity or multiple fuzzy partitions with different granularities for each attribute. Then, a multiobjective evolutionary optimization algorithm is applied to perform fuzzy rule selection. Although the multiple granularity approach is one of the most promising approaches, its interpretability loss has often been pointed out. In this work, we propose a mechanism to generate single granularity-based fuzzy classification rules for multiobjective genetic fuzzy rule selection. This mechanism is able to specify appropriate single granularities for fuzzy rule extraction before performing multiobjective genetic fuzzy rule selection. The results show that the performance of the obtained classifiers can be even improved by avoiding multiple granularities, which increases the linguistic interpretability of the obtained models.

Recently fuzzy system design has been frequently formulated as multiobjective optimization problems with two conflicting goals: maximization of accuracy and interpretability. Whereas the formulation of accuracy maximization is usually straightforward in each application task, it is not easy to define the interpretability of fuzzy rule-based systems. As a result, interpretability maximization is often handled as complexity minimization. In this paper, we discuss whether the complexity minimization leads to the interpretability maximization in the design of fuzzy rule-based systems for pattern classification problems. Using very simple artificial test problems, we show that the complexity minimization does not always lead to the interpretability maximization. We also discuss the explanation capability of fuzzy rule-based systems to explain their reasoning results to human users in an understandable manner. We show that the interpretability maximization is closely related to but different from the explanation capability maximization.

Recently evolutionary multiobjective optimization (EMO) algorithms have been actively used for the design of accurate and interpretable fuzzy rule-based systems. This research area is often referred to as multiobjective genetic fuzzy systems where EMO algorithms are used to search for a number of non-dominated fuzzy rule-based systems with respect to their accuracy and interpretability. The main advantage of the use of EMO algorithms for fuzzy system design over single-objective optimizers is that multiple alternative fuzzy rule-based systems with different accuracy-interpretability tradeoffs are obtained by their single run. The decision maker can choose a single fuzzy rule-based system according to their preference. There still exist several important issues to be discussed in this research area such as the definition of interpretability, the formulation of interpretability measures, the visualization of tradeoff relations, and the interpretability of the explanation of fuzzy reasoning results. In this paper, we discuss the ability of EMO algorithms as multiobjective optimizers to search for Pareto optimal or near Pareto optimal fuzzy rule-based systems. More specifically, we examine whether EMO algorithms can find non-dominated fuzzy rule-based systems that approximate the entire Pareto fronts of multiobjective fuzzy system design problems.

The iterated prisoner's dilemma (IPD) game has been frequently used to examine the evolution of cooperative behavior among agents in the field of evolutionary computation. A number of factors are known to be related to the evolution of cooperative behavior. One well-known factor is spatial relations among agents. The IPD game is often played in a grid-world. Such a spatial IPD game has a neighborhood structure which is used for local opponent selection in the IPD game and local parent selection in genetic operations. Another important factor is the choice of a representation scheme to encode each strategy. Different representation schemes often lead to totally different results. Whereas the choice of a representation scheme is known to be important, a mixture of different representation schemes has not been examined for the spatial IPD game in the literature. This means that a population of homogeneous agents with the same representation scheme has been assumed. In this paper, we introduce a different situation to the spatial IPD game in order to examine the evolution of cooperative behavior under more general assumptions. The main novelty of our spatial IPD game is the use of a mixture of different representation schemes. This means that we use a population of inhomogeneous agents with different representation schemes. Another novelty is the use of two neighborhood structures, each of which is used for local opponent selection and local parent selection. Under these specifications, we show a number of interesting observations on the evolution of cooperative behavior.

A large number of non-dominated solutions are often obtained by a single run of an evolutionary multiobjective optimization (EMO) algorithm. In the EMO research area, it is usually assumed that a single solution is to be chosen from the obtained non-dominated solutions by the decision maker. It is, however, time-consuming and not easy for the decision maker to examine a large number of obtained non-dominated solutions. Motivated by these discussions, we proposed single-objective and multiobjective formulations of solution selection problems to present only a small number of representative non-dominated solutions to the decision maker in our former study. The basic idea is to minimize the number of solutions to be presented while maximizing their hypervolume. A number of single-objective formulations can be derived from such a two-objective solution selection problem. In this paper, single-objective rule selection is performed as a post-processing procedure of EMO algorithms to select a prespecified number of non-dominated solutions (e.g., 10 or 20 solutions). Through computational experiments on multiobjective 0/1 knapsack problems, we examine the characteristic features of selected non-dominated solutions. We also examine the effect of the choice of a reference point for hypervolume calculation on the distribution of selected non-dominated solutions.

A new trend in evolutionary multi-objective optimization (EMO) is the handling of a multi-objective problem as an optimization problem of an indicator function. A number of approaches have been proposed under the name of indicator-based evolutionary algorithms (IBEAs). In IBEAs, the entire population usually corresponds to a solution of the indicator optimization problem. In this paper, we show how hypervolume maximization can be handled as single-objective and multi-objective problems by coding a set of solutions of the original multi-objective problem as an individual. Our single-objective formulation maximizes the hypervolume under constraint conditions on the number of nondominated solutions. On the other hand, our multi-objective formulation minimizes the number of non-dominated solutions while maximizing their Hypervolume.

Genetic fuzzy rule selection has been successfully used to design accurate and interpretable fuzzy classifiers from numerical data. In our former study, we proposed its parallel distributed implementation which can drastically decrease the computational time by dividing both a population and a training data set into sub-groups. In this paper, we examine the effect of data reduction on the generalization ability of fuzzy rule-based classifiers designed by our parallel distributed approach. Through computational experiments, we show that data reduction can be realized without severe deterioration in the generalization ability of the designed fuzzy classifiers.

Evolutionary multiobjective optimization (EMO) is an active research area in the field of evolutionary computation. EMO algorithms are designed to find a non-dominated solution set that approximates the entire Pareto front of a multiobjective optimization problem. Whereas EMO algorithms usually work well on two-objective and three-objective problems, their search ability is degraded by the increase in the number of objectives. One difficulty in the handling of many-objective problems is the exponential increase in the number of non-dominated solutions necessary for approximating the entire Pareto front. A simple countermeasure to this difficulty is to use large populations in EMO algorithms. In this paper, we examine the behavior of EMO algorithms with large populations (e.g., with 10,000 individuals) through computational experiments on multiobjective and many-objective knapsack problems with two, four, six, eight and ten objectives. We examine two totally different algorithms: NSGA-II and MOEA/D. NSGA-II is a Pareto dominance-based algorithm while MOEA/D uses scalarizing functions. Their search ability is examined for various specifications of the population size under the fixed computation load. That is, we use the total number of examined solutions as the stopping condition of each algorithm. Thus the use of a very large population leads to the termination at an early generation (e.g., 20th generation). It is demonstrated through computational experiments that the use of too large populations makes NSGA-II very slow and inefficient. On the other hand, MOEA/D works well even when it is executed with a very large population. We also discuss why MOEA/D works well even when the population size is unusually large.

It is well-known that Pareto dominance-based evolutionary multiobjective optimization (EMO) algorithms do not work well on many-objective problems. This is because almost all solutions in each population become non-dominated with each other when the number of objectives is large. That is, the convergence property of EMO algorithms toward the Pareto front is severely deteriorated by the increase in the number of objectives. Currently the design of scalable EMO algorithms is a hot issue in the EMO community. In this paper, we examine the scalability of multiobjective genetic local search (MOGLS) to many-objective problems using a hybrid algorithm of NSGA-lI and local search. Multiobjective knapsack problems with 2, 4, 6, 8, and 10 objectives are used in computational experiments. It is shown by experimental results that the performance of NSGA-lI is improved by the hybridization with local search independent of the number of objectives in the range of 2 to 10 objectives.

Whereas evolutionary multiobjective optimization (EMO) algorithms have successfully been used in a wide range of real-world application tasks, difficulties in their scalability to many-objective problems have also been reported. In this paper, first we demonstrate those difficulties through computational experiments. Then we review some approaches proposed in the literature for the scalability improvement of EMO algorithms. Finally we suggest future research directions in evolutionary many-objective optimization.

In the field of data mining, two rule evaluation criteria called confidence and support are often used to evaluate a rule. Pareto-optimality of rules can be defined using these two criteria. The rules that are Pareto-optimal in the maximization of confidence and support are called Pareto-optimal rules. In this paper, we examine the effectiveness of designing fuzzy rule-based classifiers from Pareto-optimal rules and near Pareto-optimal rules. To show the effectiveness, we compare the Pareto-optimal (and near Pareto-optimal) rules with rules extracted by various rule evaluation criteria. In the design of classifiers, we use evolutionary multiobjective rule selection to obtain simple and accurate classifiers. Through computational experiments, we show that the best fuzzy rule with respect to each rule evaluation criterion is one of Pareto-optimal rules. We also show that fuzzy rule-based classifiers designed from Pareto-optimal rules have higher accuracy.

Recently a number of approaches have been proposed to improve the scalability of evolutionary multiobjective optimization (EMO) algorithms to many-objective problems. In this paper, we examine the effectiveness of those approaches through computational experiments on multiobjective knapsack problems with two, four, six, and eight objectives. First we briefly review related studies on evolutionary many-objective optimization. Next we explain why Pareto dominance-based EMO algorithms do not work well on many-objective optimization problems. Then we explain various scalability improvement approaches. We examine their effects on the performance of NSGA-II through computational experiments. Experimental results clearly show that the diversity of solutions is decreased by most scalability improvement approaches while the convergence of solutions to the Pareto front is improved. Finally we conclude this paper by pointing out future research directions.

The worst one-max solver competition task in GECCO 2007 was to develop a one-max solver that can find the optimal solution of the 15-bit one-max problem as late as possible within 1000 generations. There are two conflicting issues in developing such a one-max solver. One is to slow down the evolution of solutions toward the optimal solution (i.e., not to find the optimal solution in early generations). The other is to find the optimal solution in a very late generation. In this paper, we examine the effect of using a non-geometric binary crossover operator through computational experiments on the worst one-max solver competition task.

This paper empirically examines the effect of the specification of the local search application probability on the performance of multiobjective memetic algorithms. In each generation of multiobjective memetic algorithms, local search is probabilistically applied to each solution. We handle the local search application probability as a controllable parameter. In computational experiments in this paper, we examine the effect of dynamically changing the probability using the five control strategies: constant, step-wise increase, step-wise decrease, linear increase, and linear decrease. Better results are obtained for almost all test problems by changing the local search application probability than specifying it as a constant value. An interesting observation is that the choice of an appropriate control strategy is problem-dependent. Gradually decreasing its value leads to good results for many problems. Such a control strategy, however, does not work on some test problems.

An emerging trend in the design of evolutionary multiobjective optimization algorithms is to directly optimize a quality indicator of non-dominated solution sets such as the hypervolume measure. Some algorithms have been proposed to search for a set of a pre-specified number of non-dominated solutions that maximizes the given quality indicator. In this paper, we propose an iterative approach to indicator-based evolutionary multiobjective optimization. The main feature of our approach is that only a single solution is obtained by its single run. Thus multiple runs are needed to find a solution set. In each run, our approach searches for a solution with the maximum contribution to the hypervolume of the solution set obtained by its previous runs. We discuss several issues related to the implementation of such an iterative approach.

We have examined the effect of spatial structures on the evolution of iterated prisoner’s dilemma (IPD) game strategies. In our former study, we used two neighborhood structures, which follow the concept of structured demes. One is for the interaction among players through the IPD game. A player in each cell in a grid-world plays against its neighbors defined by this neighborhood structure. The other is for the mating of strategies by genetic operations. A new strategy for a player is generated by genetic operations from a pair of parent strings, which are selected from its neighbors defined by the second neighborhood structure. In this paper, we extend our IPD game simulation to a more realistic problem while keeping the simplicity of the original IPD game. We employ a stochastic strategy represented by a string of real numbers between 0 and 1. Each real number in the string denotes the probability of cooperation. We examine the effects of spatial structures on the evolution of IPD game strategies with probabilistic decision making in various payoff matrices. From simulation results, it is shown that cooperative behavior is evolved only when the interaction neighborhood is small and the mating neighborhood is also small for some payoff matrices.

Genetic fuzzy rule selection has been successfully used to design accurate and interpretable fuzzy classifiers. However there exists a computational complexity problem for large data sets. This paper proposes a simple but effective idea to improve the applicability of genetic fuzzy rule selection to large data sets. Our idea is based on the parallel distributed implementation of genetic fuzzy rule selection. We examine the advantage of the proposed approach through computational experiments on some benchmark data sets.

In the design of evolutionary multiobjective optimization (EMO) algorithms, it is important to strike a balance between diversity and convergence. Traditional mask-based crossover operators for binary strings (e.g., one-point and uniform) tend to decrease the diversity of solutions in EMO algorithms while they improve the convergence to the Pareto front. This is because such a crossover operator, which is called geometric crossover, always generates an offspring in the segment between its two parents under the Hamming distance in the genotype space. That is, the sum of the distances from the generated offspring to its two parents is always equal to the distance between the parents. In this paper, first we propose a non-geometric binary crossover operator to generate an offspring outside the segment between its parents. Next we examine the effect of the use of non-geometric binary crossover on single-objective genetic algorithms. Experimental results show that non-geometric binary crossover improves their search ability. Then we examine its effect on EMO algorithms. Experimental results show that non-geometric binary crossover drastically increases the diversity of solutions while it slightly degrades their convergence to the Pareto front. As a result, some performance measures such as hypervolume are clearly improved.

It has been demonstrated in the literature that a similarity-based mating scheme can increase the diversity of solutions in evolutionary multiobjective optimization (EMO) algorithms. In the similarity-based mating scheme, an extreme solution is chosen from the current population as one parent. A similar solution to the selected parent is chosen from the current population as the other parent (i.e., as a mate of the first parent). In this paper, we first demonstrate that the similarity-based mating scheme works well when it is incorporated into NSGA-II. Next we point out a problematic side effect of choosing extreme solutions as parents using SPEA. That is, the similarity-based mating scheme does not always widen the population along the Pareto front but also lengthen it toward the Pareto front. This is because poor solutions far from the Pareto front have relatively high selection probabilities to be chosen as parents. Then we propose a simple trick to prevent such a poor solution from being selected as the first parent. Finally we demonstrate that the modified similarity-based mating scheme works well in SPEA as well as NSGA-II through computational experiments on multiobjective 0/1 knapsack problems.

We compare single-objective genetic algorithms (SOGAs) with multi-objective genetic algorithms (MOGAs) in their applications to multi-objective knapsack problems. First we discuss difficulties in comparing a single solution by SOGAs with a solution set by MOGAs. We also discuss difficulties in comparing several solutions from multiple runs of SOGAs with a large number of solutions from a single run of MOGAs. It is shown that existing performance measures are not necessarily suitable for such comparison. Then we compare SOGAs with MOGAs through computational experiments on multi-objective knapsack problems. Experimental results on two-objective problems show that MOGAs outperform SOGAs even when they are evaluated with respect to a scalar fitness function used in SOGAs. This is because MOGAs are more likely to escape from local optima. On the other hand, experimental results on four-objective problems show that the search ability of MOGAs is degraded by the increase in the number of objectives. Finally we suggest a framework of hybrid algorithms where a scalar fitness function in SOGAs is probabilistically used in MOGAs to improve the convergence of solutions to the Pareto front.

In this paper, we demonstrate that multiobjective genetic rule selection can significantly improve the accuracy-complexity tradeoff curve of fuzzy rule-based classification systems generated by a heuristic rule extraction procedure for classification problems with many continuous attributes. First a prespecifled number of fuzzy rules are extracted in a heuristic manner based on a rule evaluation criterion. This step can be viewed as fuzzy data mining. Then multiobjective genetic rule selection is applied to the extracted rules to find a number of non-dominated rule sets with respect to accuracy maximization and complexity minimization. This step can be viewed as a postprocessing procedure in fuzzy data mining. Experimental results show that multiobjective genetic rule selection finds a number of smaller rule sets with higher classification accuracy than heuristically extracted rule sets. That is, the accuracy-complexity tradeoff curve of heuristically extracted rule sets in fuzzy data mining is improved by multiobjective genetic rule selection. This observation suggests that multiobjective genetic rule selection plays an important role in fuzzy data mining as a postprocessing procedure.

In this paper, we show the usefulness of multiobjective genetic rule selection as a postprocessing procedure in data mining for pattern classification problems. First we extract a prespecified number of rules using a data mining technique. Then we apply multiobjective genetic rule selection to the extracted rules. Experimental results show that multiobjective genetic rule selection significantly decreases the number of extracted rules while improving their classification accuracy.

The main characteristic feature of evolutionary multiobjective optimization (EMO) is that no a priori information about the decision maker's preference is utilized in the search phase. EMO algorithms try to find a set of well-distributed Pareto-optimal solutions with a wide range of objective values. It is, however, very difficult for EMO algorithms to find a good solution set of a multiobjective combinatorial optimization problem with many decision variables and/or many objectives. In this paper, we propose an idea of incorporating the decision maker's preference into EMO algorithms to efficiently search for Pareto-optimal solutions of such a hard multiobjective optimization problem.

This paper compares fuzzy rules with interval rules through computational experiments on benchmark data sets from the UCI database using an evolutionary multiobjective rule selection method. In the design of fuzzy and interval rule-based systems for classification problems, we use three types of partitions: homogeneous fuzzy partitions, inhomogeneous entropy-based interval partitions, and inhomogeneous fuzzy partitions derived from the interval partitions. A large number of rule-based systems are designed from each type of partitions using our evolutionary multiobjective rule selection method with three objectives: to maximize the number of correctly classified training patterns, to minimize the number of rules, and to minimize the total number of antecedent conditions. Experimental results show that the fuzzification of interval rules improves their generalization ability for many data sets

We examine three methods for improving the ability of evolutionary multiobjective optimization (EMO) algorithms to find a variety of fuzzy rule-based classification systems with different tradeoffs with respect to their accuracy and complexity. The accuracy of each fuzzy rule-based classification system is measured by the number of correctly classified training patterns while its complexity is measured by the number of fuzzy rules and the total number of antecedent conditions. One method for improving the search ability of EMO algorithms is to remove overlapping rule sets in the three-dimensional objective space. Another method is to choose similar rule sets as parents for crossover operations. The other method is to bias the selection probability of parents toward rule sets with high accuracy. The effectiveness of each method is examined through computational experiments on benchmark data sets

We focus on the handling of overlapping solutions in evolutionary multiobjective optimization (EMO) algorithms. First we show that there exist a large number of overlapping solutions in each population when EMO algorithms are applied to multiobjective combinatorial optimization problems with only a few objectives. Next we implement three strategies to handle overlapping solutions. One strategy is the removal of overlapping solutions in the objective space. In this strategy, overlapping solutions in the objective space are removed during the generation update phase except for only a single solution among them. As a result, each solution in the current population has a different location in the objective space. Another strategy is to remove overlapping solutions so that each solution in the current population has a different location in the decision space. The other strategy is the modification of Pareto ranking where overlapping solutions in the objective space are allocated to different fronts. As a result, each solution in each front has a different location in the objective space. Effects of each strategy on the performance of the NSGA-II algorithm are examined through computational experiments on multiobjective 0/1 knapsack problems, multiobjective flowshop scheduling problems, and multiobjective fuzzy rule selection problems.

Evolutionary multiobjective fuzzy rule selection can find a large number of non-dominated fuzzy rule-based classifiers with different tradeoffs between complexity and accuracy. Very simple fuzzy rule-based classifiers with high interpretability are usually not accurate while complicated classifiers with high accuracy are not interpretable. In this paper, fuzzy rule-based classifiers with different tradeoffs are used as an ensemble classifier. Three multiobjective formulations of fuzzy rule selection are compared with each other in terms of the generalization ability of constructed ensemble classifiers. Those ensemble classifiers are also compared with individual fuzzy rule-based classifiers obtained from the corresponding three single-objective formulations based on weighted sums of accuracy and complexity measures.

The aim of This work is to develop partner robots that can obtain and accumulate human-friendly behaviors. To realize it, we use a concept of structured learning which emphasizes the importance of an interactive learning of several modules through interaction with its environment. In a proposed method, a robot obtains hand-to-hand behavior by using an interactive evolutionary computation based on human evaluations estimated by fuzzy state-value functions. Moreover, a self-organizing map is used for clustering human hand positions. A state-value function and a knowledge database are assigned to each clustered positions. Furthermore, the best trajectory is stored in the knowledge database to reuse it in the same situation. Some experimental results show the effectiveness of the proposed method.

This paper proposes a method for generating behaviors based on imitation of a partner robot interacting with a human. First of all, we discuss the role of imitation, and explain the method for imitative behavior generation of the robot based on computational intelligence. The robot searches for a human by using a CCD camera. A human hand motion pattern is extracted from a series of images taken from the CCD camera. Next, the position sequence of the extracted human hand is used as inputs to a spiking neural network in order to recognize it as a gesture. Furthermore, the trajectory for a behavior is generated and updated by a steady-state genetic algorithm based on human motions. Furthermore, a self-organizing map is used for clustering human hand motion patterns as gestures. Finally, we show several experimental results of imitative behavior generation through interaction with a human.

This paper is concerned with a local learning method of a multi-objective behavior coordination for a mobile robot. The multiobjective behavior coordination plays a role in integrating outputs of basic behavioral modules. A behavioral weight is assigned to each behavioral module represented by fuzzy rules, production rules, and so on. By updating these behavioral weights, the mobile robot can take a multi-objective situated action. However, the coordination rule is designed suitably static environments and the mobile robot must learn or update coordination rule in dynamic environments with moving obstacles. Therefore, we propose a local episode-based learning which is a learning method using self-reference of the relationship between previous perception and action in short-term memory.

Deals with a pet robot with an emotional model. The robot requires several capabilities, such as perceiving, acting, communicating and surviving. Furthermore, it should learn various behaviors through interaction with its owner. This paper focuses on teaching a pet robot tricks or to dance. Basically, the owner can teach these tricks by simple communication based on trial and error. The robot performs the tricks by using a fuzzy controller, and further acquires tricks by a delta rule for online learning and a genetic algorithm for off-line learning. We use "Rag Warrior" as our pet robot. Experimental results show that this robot performs tricks through interaction with its owner.

This paper deals with a multi-objective behavior coordinate for a mobile robot using fuzzy control and neural network. A task given to a mobile robot includes various objectives such as collision avoiding, target tracing, and wall following. We apply fuzzy control for describing each behavior of the robot. However, a behavior might share some fuzzy rules with other behaviors. Therefore, this paper proposes a reconfiguring method for a set of fuzzy rules. The combination of fuzzy rules is updated dynamically by a neural network according to the perceptual information. Furthermore, this paper describes a learning method of the neural network and fuzzy rules based on error functions. Simulation results show that the robot can take multi-objective behavior by the proposed method.