Recently, several approaches have gained attention by integrating large language models (LLMs) into evolutionary algorithms. Building on this trend, we introduce Language Model Particle Swarm Optimization (LMPSO), a novel method that incorporates an LLM into the swarm intelligence framework of Particle Swarm Optimization (PSO). In LMPSO, the velocity of each particle is defined as a part of the prompt that generates the next candidate solution, leveraging an LLM to produce solutions while respecting the PSO paradigm. This integration enables an LLM-driven search process that adheres to the foundational principles of PSO. We evaluate LMPSO on the Traveling Salesman Problem (TSP) and on a heuristic improvement task for TSP, where solutions are represented as program code. Heuristic improvement aims to enhance existing heuristics by searching for new ones; it is challenging for standard PSO due to the solution representation in the form of program code. Experimental results demonstrate that LMPSO can generate high-quality solutions for small TSP instances and improve existing TSP heuristics while following the PSO search framework. By incorporating LLMs into PSO, LMPSO expands the applicability of swarm intelligence and highlights the potential of LLMs for addressing complex optimization challenges.

Feature engineering is a necessary step in the machine learning pipeline. Together with other preprocessing methods, it allows the conversion of raw data into a dataset containing only the necessary features to solve the task at hand, reducing the computational complexity of inducing models and creating models that are potentially simpler, more robust, and more interpretable. We use M3GP, a wrapper-based feature engineering algorithm, to induce a set of features that are adapted in number and in shape to several classifiers with different levels of predictive power, from decision trees with depth 3 to random forests with 100 estimators and no depth limit. Intuition tells us that classifiers that are restricted in the number of features should compensate for this restriction by using features with a high degree of correlation with the target objective. By opposition, the principle behind the boosting algorithm tells us that we can create a strong classifier using a large set of weak features. This indicates that classifiers with no restrictions should prefer many but weaker features. Our results confirm this hypothesis while also revealing that M3GP induces unnecessarily complex features. We measure complexity using several structural complexity metrics found in the literature and show that, although our pipeline consistently obtains good results, the structural complexity of the induced models varies drastically across runs. Additionally, while the test performance peaks in the early stages of the evolution, the complexity of the feature engineering models continues to grow, with little to no return in test performance. This work promotes using several complexity metrics to measure model interpretability and identifies issues related to model complexity in M3GP, proposing solutions to improve the computational cost of inducing models and the complexity of the final models.

Genetic algorithm (GA) is a common approach for multi-objective path planning. However, conventional GA performs poorly on large-scale complex maps due to the lack of an efficient initialization method and the infeasible solutions generated during the GA search. In this paper, first, we propose an innovative initialization method. The proposed method extracts the division points of the map and constructs a topological map, allowing the initialization of feasible paths based on the fitness function. Second, we calculate the estimated fitness value of each path in the topological map. Paths with low estimated fitness values will not be initialized, reducing the search space. In addition, we improve the crossover of GA, preventing the generation of infeasible paths by utilizing the topological map. The proposed method is compared against Theta*, A *(adjusted to consider smoothness), and conventional genetic algorithms on small and large-scale maps. The proposed method has outperformed previous methods regarding fitness value, reducing the runtime by more than 37% on large-scale maps.

Recent improvements in pretrained Large Language Models (LLMs) have demonstrated increasing capabilities in various natural language processing tasks, such as instruction following. However, effectively utilizing LLMs in interactive environments that require advanced reasoning, planning, and decision-making skills remains a challenge. In this study, we integrate Learning Classifier Systems (LCS) with LLMs, thereby extending their capabilities to interact in natural language domains. To accommodate the integration, especially the rule represented in natural language, we propose a novel Rule Optimization method using LLMs for Imitation Learning (ROIL) in text-based interactive environments. ROIL addresses rule learning by lever-aging LLMs to optimize rules based on human demonstrations, thus eliminating the need for trial-and-error learning processes and ensuring both interpretability and safety. It also tackles the challenge of learning transferable skills in these environments. We evaluate the efficacy of ROIL in the WebShop environment, a text-based e-commerce website navigation problem. The method achieved a significant performance and efficiency improvement over a strong baseline, while demonstrating performance close to a gradient-based imitation learning approach. Additionally, this study explores the enhancement of ROIL using meta-heuristic optimization algorithms, providing foundational research for further investigation in this area.

In the realm of robot design, co-design aims to optimize both the structure and the controller of a robot concurrently. One approach integrates genetic algorithms to optimize the soft robot's structure with deep reinforcement learning for the controller. A significant challenge in this approach is the inheritance of the controller due to the mismatch of the sensors and actuators of the robots across generations. In this study, we propose a Lamarckian co-design method to inherit the controller optimized by deep reinforcement learning through transfer learning. In experimental evaluations through the Evogym benchmark, we demonstrate that our proposed method achieves an average reduction of 41.7% in the optimization time for robots compared to existing methods and concurrently leads to an average performance improvement of 118.5%. Furthermore, we show that combining the inheritance of controllers with the crossover of structure genomes from two robots allows for additional reductions in optimization time and improvements in performance in several tasks.

The premise of continual learning is to continuously learn new tasks using the same model in an environment where multiple tasks are given sequentially while retaining the knowledge learned in previous tasks. Typical deep learning models suffer from catastrophic forgetting, in which knowledge of past tasks is drastically lost when learning new tasks, and LSTM (Long Short-Term Memory) is no exception. One of the promising methods for continual learning are replay-based methods. However, they are prone to overfitting, harming generalization. Meanwhile, Ant Colony Optimization (ACO) is an algorithm widely used for combinatorial optimization problems and has been applied to the structural optimization of LSTM. In this study, we propose a continual learning method for LSTM using ACO to reduce catastrophic forgetting. The method iteratively optimizes the internal structure of the LSTM in parallel with the training of the current task, using the model performance as fitness. It also extends the replay-based method and utilizes two kinds of memory buffers to reduce overfitting to the memory. The proposed method was tested on four benchmark problems, and the results indicate its effectiveness, especially in the small memory size scenarios.

Neural architecture search (NAS) aims to automatically design suitable architectures of artificial neural networks (ANNs) under various situations. Recently, NAS based on zero-cost proxies can predict the performance of ANNs with the cost of a single forward/backward propagation pass at most. While zero-cost proxies can speed up NAS by orders of magnitude, the gap between the predicted and actual performance of ANNs prevents zero-cost proxies from identifying ANNs with top performance.

Text-to-image generation using deep learning based models has become a popular research topic, allowing users to generate custom artworks from specified text input. However, generating suitable prompts that produce creative and desirable images remains a significant challenge. To address this challenge, we propose a novel method that incorporates interactive evolutionary computation (IEC) to evolve the latent array. By integrating human perception into the system, our approach enables users to search for and generate images that align with their desired specifications through interaction with the system. We demonstrate the effectiveness of our proposed method in generating images that align with the users' mental images from an initial image using genetic algorithms through a series of experiments. Furthermore, the results from our user studies show that our proposed method enables users to generate images that match their desired mental images with less effort and in less time compared to conventional generation methods. Overall, this study contributes to the field of text-to-image generation by introducing a human-in-the-loop approach that enhances user control and specificity in the image generation process.

This paper introduces a new Genetic Programming (GP) based classification framework for multiclass classification problems. The proposed framework uses a binary decomposition-based GP method to extract new features to enhance the performance of classifiers in the multiclass classification task. We firstly introduce a random binary decomposition method that uses a part-vs-part strategy to decompose the multiclass problems which increase the number of binary problems that can be decomposed from a multiclass problem. Then the details of combining GP with this binary decomposition method for feature extraction are explained. Finally, we compare our method to several popular ML methods and traditional GP methods in a broad set of benchmark problems. The outcome shows the performance of classifiers is enhanced for multi-class classification tasks when combined with this technique. The effect of applying this framework to different classifiers and large real-world data set is also explored. The results suggest the effectiveness and universality of our method.

This paper proposes a hybrid basis function construction method (GP-RVM) for Symbolic Regression problem, which combines an extended version of Genetic Programming called Kaizen Programming and Relevance Vector Machine to evolve an optimal set of basis functions. Different from traditional evolutionary algorithms where a single individual is a complete solution, our method proposes a solution based on linear combination of basis functions built from individuals during the evolving process. RVM which is a sparse Bayesian kernel method selects suitable functions to constitute the basis. RVM determines the posterior weight of a function by evaluating its quality and sparsity. The solution produced by GP-RVM is a sparse Bayesian linear model of the coefficients of many non-linear functions. Our hybrid approach is focused on nonlinear white-box models selecting the right combination of functions to build robust predictions without prior knowledge about data. Experimental results show that GP-RVM outperforms conventional methods, which suggest that it is an efficient and accurate technique for solving SR. The computational complexity of GP-RVM scales in O(M3), where M is the number of functions in the basis set and is typically much smaller than the number N of training patterns.

In recent years, the possibility of using recurrent neural networks (RNNs) to model long-term dependencies in music and to generate novel pieces of music has been actively investigated. However, previous work on music generation by RNNs has simply generated music autonomously without user control. Thereby, previous studies have not been effective in assisting human composers. To overcome this, we incorporated the method of interactive evolutionary computation (IEC) in order to search through the latent space of music modeled by recurrent variational auto-encoders. By using IEC, we were able to integrate human perception into the system and enable the user to search for their desired type of music by interacting with the system. The proposed method effectively combines the impressive sequential data modeling capabilities of RNNs with the interactivity and user-friendliness of IEC-based systems. We conducted a user study to evaluate the usefulness of our method in comparison to conventional composition methods. The results revealed that our proposed method was effective in aiding human composition.

A linear SVM scales linearly with the size of a dataset, and hence is very desirable as a classifier for large datasets. However, it is not able to classify a dataset having a nonlinear decision boundary between the classes unless the dataset has been transformed by some mapping function so that the decision boundary becomes linear or it is a good approximation to a linear boundary. Often these mapping functions may result in a dataset with very large dimension or even infinite dimension. To avoid the curse of dimensionality, kernel functions are used as mapping functions. However, a kernel SVM has quadratic time complexity, and hence does not scale very well with large datasets. Moreover, the choice of a kernel function and its parameter optimization are arduous tasks. Therefore, a replacement of kernel function with an explicit mapping function is desirable in the case of large datasets. In this paper, we propose a novel co-evolutionary approach to find an explicit mapping function. We use GA to evolve an n-tuple of GP trees as a mapping function, and GP to evolve each individual GP tree. The dataset is then transformed using the found mapping function so that a linear SVM can be used. Besides the fact that the proposed algorithm allows us to use a fast linear SVM, the results also show that the proposed algorithm outperforms the kernel trick and even performs as good as the kernel trick combined with feature selection.

How can we boost Escherichia coli (E. coli) growth (i.e., production) without modifying genes themselves? This remains a challenging and fruitful goal that would facilitate the mass production of biofuels, biomedicine, and engineered genomes in synthetic biology. In this paper, we focus on rear-ranging gene order within an operon to optimize gene expression. Optimizing more than five genes remains laborious without predictive modeling as the number of gene orders increases factorially - a five-gene operon possesses 120 gene orders, but a ten-gene operon possesses 3,628,800 gene orders. To handle a ten-gene operon, we propose consultation algorithms utilizing LCS to analyze the relationship between gene order and growth rate, and then verify predicted gene orders with high growth rates using wet-lab experiments. “Consultation” refers to optimizing gene orders in different machine learning algorithms and choosing gene orders with high growth rates in each algorithm to avoid over-fitting. We address the following research questions: (RQ1) How can we predict E. coli growth according to gene orders? (RQ2) Can definite rules easily understood by biologists be extracted? (RQ3) Can new E. coli strains surpass the highest growth rate of the dataset? Our first computational approach shows that consultation algorithms utilizing LCS can identify gene orders that significantly control E. coli growth and create novel E. coli strains with high growth rates using these operon construction rules.

In symbolic regression, which aims to find a function that satisfies the target values for all data points, one of the major challenges is that the solutions cannot be uniquely determined. Genetic programming (GP) provides a powerful approach to symbolic regression in that it does not require models of functions to be fixed. However, it is known that GP suffers from a phenomenon known as bloat, meaning that candidate functions attain an excessively complicated form during the search, which is undesirable in many applications. While the majority of approaches for regulating bloat introduce anti-bloat genetic operators or anti-bloat selection schemes, most of these are derived from heuristics and/or require well-tuned hyper-parameters. In the present study, we propose a novel approach in which genetic trees of GP are reduced during the search using a basis of a set of polynomials (vanishing ideal) that are equivalent to zero for the data points of symbolic regression. The vanishing ideal is computed using an algebraic approach, and because it only requires data points as input, our approach does not involve the tuning of any hyper-parameters. The proposed approach regulates bloat and efficiently determines simple solutions. We compare our approach with standard GP with a penalty term for the height of trees in the fitness, and demonstrate the effectiveness of our approach to two tasks (real-valued symbolic regression and the 6-parity problem).

Many feature selection methods are based on the assumption that important features are highly correlated with their corresponding classes, but mainly uncorrelated with each other. Often, this assumption can help eliminate redundancies and produce good predictors using only a small subset of features. However, when the predictability depends on interactions between features, such methods will fail to produce satisfactory results. In this paper a method that can find important features, both independently and dependently discriminative, is introduced. This method works by performing two different types of permutation tests that classify each of the features as either irrelevant, independently predictive or dependently predictive. It was evaluated using a classifier based on an ensemble of genetic programs. The attributes chosen by the permutation tests were shown to yield classifiers at least as good as the ones obtained when all attributes were used during training - and often better. The proposed method also fared well when compared to other attribute selection methods such as RELIEFF and CFS. Furthermore, the ability to determine whether an attribute was independently or dependently predictive was confirmed using artificial datasets with known dependencies.

The design of genetic networks has been studied for implementing desired biological systems, and in particular, some researchers have proposed automatic design methods using optimization techniques. However, it is difficult to implement genetic networks designed by previous methods due to overly simplified model descriptions whose parameters are infeasible in the real world. Additionally, the methods do not ensure robustness against parameter perturbation. In this paper, we propose a two-stage design method and a fitness function evaluating robustness to create genetic networks which can be implemented experimentally. Further, we suggest the knowledge about robust network structures from results of optimization.

S-systems are mathematical models based on the power-law formalism, which are widely employed for the investigation of Gene Regulatory Networks (GRNs). Because of their complex dynamics - characterized by multi-modality and nonlinearity-the parameterization of S-systems is far from straightforward, demanding global optimization techniques. The problem of parameter estimation of S-systems is further complicated when the desired dynamics is characterized by oscillations. In this work, we describe a novel methodology based on Particle Swarm Optimization for the automatic parameterization of oscillating Ssystems. In this methodology, two swarms perform independent optimizations, and cooperate by periodically exchanging the best particles. The two swarms exploit two different fitness functions: a traditional point-to-point distance, and a spectra-based fitness function. We show that this cooperative approach allows the double swarm to outperform the common methodology, based on a single swarm exploiting a single fitness function. We demonstrate the effectiveness of our method using a GRN of five genes, performing tests of increasing complexity, up to the simultaneous inference of 17 parameters.

In this paper, we introduce our approach for evolving reaction networks. It is an efficient derivative of the neuroevolution of augmenting topologies algorithm directed at the evolution of biochemical systems or molecular programs. Our method addresses the problem of meaningful crossovers between two chemical reaction networks of different topologies. It also builds on features such as speciation to speed up the search, to the point where it can deal with complete, realistic mathematical models of the biochemical processes. We demonstrate this framework by evolving credible biochemical answers to challenging autonomous molecular problems: in vitro batch oscillatory networks that match specific oscillation shapes. Our experimental results suggest that the search space is efficiently covered and that, by using crossover and preserving topological innovations, significant improvements in performance can be obtained for the automatic design of molecular programs.

Group Counseling Optimization (GCO) has recently been proposed in an attempt to emulate the human social behavior in solving life problems through counseling within a group. After its promising results in solving unconstrained singleobjective and multi-objective optimization problems, in this paper, GCO is extended to solve the constrained optimization problems for the first time. Also, a hybrid parameter-less constraint handling technique is proposed, which uses two wellknown constraint handling techniques: feasible rules and penalty function. The Constrained Group Counseling Optimization (CGCO) uses gradient-based mutation only in the case of all-equality-constraints COPs to reach the extremely small feasible region easily. Moreover, CGCO performance is tested by solving the constrained benchmarks function of the CEC 2010 competition. The results demonstrate that CGCO is competitive to other state-of-the-art algorithms and consistently reaches feasible solutions. Introduction Many real-world applications necessitate the solution of Constrained Optimization Problems (COPs). The solution of such problems means optimizing a given objective function while satisfying a set of imposed constraints. A COP can be defined as follows (Mezura-Montes and Coello C., 2011): minimize f (x) sub ject to g j(x)≤ 0, j = 1, ...,q (1) h j(x) = 0, j = q+1, ...,m Ld ≤ xd ≤Ud , d = 1, ...,D where x = (x1,x2, ...,xD) ∈ R D is a real-valued Ddimensional vector, f is the real valued objective function, g j and h j are q inequality constraints and (m− q) equality constraints, respectively, Ld and Ud are the lower and upper bounds of xd , respectively. To solve COPs, researchers used well-known natureinspired algorithms such as Particle Swarm optimization (PSO) (Kennedy et al., 1995), and Differential Evolution (DE) (Storn and Price, 1997), etc. Originally, all of these algorithms are mainly proposed to deal with unconstrained optimization problems so that it should add a Constraint Handling Technique (CHT) to enable them to deal with COPs. Currently, seven categories of CHTs are known in the literature (Mezura-Montes and Coello C., 2011): Feasibility Rules (FR) (Deb, 2000), Stochastic Ranking (SR) (Runarsson and Yao, 2000), e-constrained method (Takahama et al., 2005), Novel Penalty Functions, Novel special operators, Multi-objective concepts, Ensemble of constraint-handling techniques. In FR, a solution with less constraint violation is preferred. If two solutions have the same value of constraint violation, the fitter solution is preferred. In SR, a user-defined parameter called p f determines which criterion to use when comparing infeasible solutions: (1) based on their sum of constraints violation or (2) based only on their objective function values. In e-constrained method, the value of e> 0 relaxes the limit of considering a solution as feasible. In Novel Penalty Functions, researchers recently proposed two penalty-based approaches namely adaptive penalty function and dynamic penalty function. Adaptive penalty function (Tessema and Yen, 2009) calculates the penalty factor based on the status of the candidate solutions in the search space. Dynamic penalty functions (Tasgetiren and Suganthan, 2006) adopts the current generation number to decrease the penalty factor. In Novel special operators, operator such as boundary operator (Leguizamon and Coello C., 2009) is suggested. In Multi-objective concepts, a COP is turned into a bi-objective optimization problem (objective function and sum of constraints violation) (Wang et al., 2007). In ensemble of constraint-handling techniques (Mallipeddi and Suganthan, 2010a), more than one of the aforementioned categories are hybridized to get the advantages of each category. For simplicity, the Constrained Group Counseling Optimization (CGCO) proposes the use of two parameter-less CHTs: FR and penalty function without any penalty factor. Gradient-based mutation operator (Takahama and Sakai, 2006) is also used to deal with COPs that have only equality constraints. CGCO is applied to a set of standard benchmark ALIFE 14: Proceedings of the Fourteenth International Conference on the Synthesis and Simulation of Living Systems COPs of the IEEE CEC 2010 competition (Mallipeddi and Suganthan, 2010b) and the results are compared with eDEag (Takahama and Sakai, 2010), the winner of this competition, and another recently proposed algorithm Co-CLPSO (Liang et al., 2010). This paper is organized as follows: section 2 outlines the related work. Section 3 provides an overview of GCO. Section 4 presents an overview of the CHTs and gradient-based mutation used by CGCO. Section 5 introduces the proposed CGCO. In section 6, CGCO is tested on COPs to evaluate its performance. Finally, the conclusions and future work are put forward in section 7. Related Works Here, some well-known nature-inspired algorithms, namely DE (Storn and Price, 1997), PSO (Kennedy et al., 1995) are briefly addressed. In (Takahama and Sakai, 2006), an approach, called eDE, has been suggested to solve COPs using e-constrained method as CHT and gradient-based mutation as a repair operator. Gradient-based mutation helps eDE handle COPs whose constraints are equality. This kind of COPs is difficult because the feasible region is very small. Afterwards, Takahama and Sakai added the concept of archive in their new approach eDEag (Takahama and Sakai, 2010). The archive increases the diversity of candidate solutions so that it gives better stability. In eDEag, a new controlling method of e level is adopted. eDEag yielded promising results and was the winner of the CEC2010 competition (Mallipeddi and Suganthan, 2010b). Liang et al. proposed an approach using PSO to solve COPs which is called cooperation comprehensive learning PSO (Co-CLPSO) (Liang et al., 2010). In Co-CLPSO, a novel CHT in which the population is divided into two subswarms is used. These two sub-swarms cooperate with each other in an attempt to solve the COPs. Particles of each swarm are responsible to deal with different constraints. Two swarms exchange their experiences to benefit each other. Sequential quadratic programming (SQP) is used as a local optimizer to improve the obtained solutions. CoCLPSO ranked the fifth position in the CEC2010 competition. Group Counseling Optimization “Instead of mimicking the behavior of biological organisms such as birds, fish, ants, and bees, GCO is inspired by the human social behavior in solving life problems through counseling within a group” (Eita and Fahmy, 2010, 2014). Counseling (Burnard, 2002) is a well-established branch in sociology and psychology. This was the first time that a connection is found between population-based optimization and group counseling (Berg et al., 2006). Based on the group counseling concept, GCO is developed (Eita and Fahmy, 2010, 2014). Four parameters affect the behavior of GCO: • Number of group members acting as counselors, c, (c≤ m–1). • Counseling probability, cp. • Search range reduction coefficient, red, set into the range [0,1]. • Transition rate from the stage of exploration to that of exploitation, tr. The GCO algorithm is illustrated in the following steps: Step 1 At the very beginning, the algorithm initializes randomly a population with m D-dimensional candidate solutions X i in the search space according to a beta distribution (Gentle, 2003); (Owen, 2008), β (x) = xa−1(1− x)b−1 B(a,b) 0 < x < 1

In this paper, we propose a method using a genetic algorithm (GA) for motion imitation between two different types of humanoid robots. Although motion imitation between humans and robots has been a popular research topic for a long time, the imitation between different types of robots still remains an unsolved task. The selection of the correct joint angles is critical for robot motion. However, different robots have different anatomies, with each joint's position and movable range uniquely defined for each type of robot. This discrepancy is an obstacle when converting a motion to another type of robot. The proposed method uses a genetic algorithm in order to find the conversion matrix needed to map one robot's joint angles to joint angles of another robot. This is done with two objectives in mind; one is to reduce the difference between the sample imitation and the converted imitation. The other one is to keep the stability. Two experiments were conducted; one stable and one unstable experiment. The experiments were made with two different types of robots in a simulation environment. The stable experiment showed a concordance rate of 93.7% with the test motion. The imitation also tested with the real robot and succeeded to keep standing. In the unstable experiment, the student robot keeps its balance for most of the simulation time. It showed a concordance rate of 95.5%, which is slightly higher than that in the stable experiment. These results show great promise for the proposed method as a way to realize motion imitation between different types of robots.

Evolutionary algorithms (EAs) have been successfully used in many studies for evolving both the structure and parameters of biological networks including gene regulatory networks that demonstrate different functionalities. However, most of these studies have used only mutation as the genetic operator in the evolutionary framework, perhaps due to the difficulty of implementing the crossover operation that generates the feasible network models. Nevertheless, crossover is considered to be the most powerful operator of EA which preserves the building blocks and promote quick convergence to a global optima. In this work we propose to use a Messy Genetic Algorithm (MGA) for evolving biological reaction networks that can calculate mathematical functions. The tactful encoding of MGA for reaction networks using a variable length chromosome, allows the use of crossover as well as mutation for the problem in hand that results in a fully functional EA. Earlier MGA has been used for solving many complex problems for which solution encoding is difficult. We used the proposed MGA for evolving different types of mathematical function calculating networks and the success was very encouraging. The evolved networks were able to calculate the target functions for mutually exclusive test data sets satisfactorily. Comparing with some other existing method based on Asexual Evolution (AE), the proposed method was superior in terms of different functions it could successfully evolve and the accuracy at which it could calculate those functions.

The unimodal Gaussian has been the distribution of choice for many extensions in Estimation of Distribution Algorithms (EDA). Some groups have used clustering algorithms, like k-means, to use multimodal distributions in different modifications of EDA. Most proposals use a fixed number of groups or clusters, and other works use heuristic approaches to find the right number of clusters in the search space without any previous information. The heuristic methods, however, lack the mathematical rigor required in the inference of a probability distribution's parameters. In this work, we propose the use of the Nonparametric Bayesian Model known as Dirichlet Process to fit the number of clusters given the data in a modified Population Based Incremental Learning (PBIL) model. We compare our approach with similar techniques that also use multimodal probability distributions to enhance the quality of the search in other EDA approaches. Our approach shows improvements by reducing the number of generations needed to find good results that are comparable to the state of the art in clustered EDA.

Ranking the set of search results according to their relevance to a user query is an important task in an Information Retrieval (IR) systems such as a Web Search Engine. Learning the optimal ranking function for this task is a challenging problem because one must consider complex non-linear interactions between numerous factors such as the novelty, authority, contextual similarity, etc. of thousands of documents that contain the user query. We model this task as a non-linear ranking problem, for which we propose Rank-PMBGP, an efficient algorithm to learn an optimal non-linear ranking function using Probabilistic Model Building Genetic Programming. We evaluate the proposed method using the LETOR dataset, a standard benchmark dataset for training and evaluating ranking functions for IR. In our experiments, the proposed method obtains a Mean Average Precision (MAP) score of 0.291, thereby significantly outperforming a non-linear baseline approach that uses Genetic Programming.

Proteins are composed by amino acids, which are created by genes. To understand how different genes interact to create different proteins, we need to model the gene regulatory networks (GRNs) of different organisms. There are different models that attempt to model GRNs. In this paper, we use the popular S-System to model small networks. This model has been solved with different evolutionary computation techniques, which have obtained good results; yet, there are no models that achieve a perfect reconstruction of the network. We implement a variation of particle swarm optimization (PSO), called dissipative PSO (DPSO), to optimize the model; we also research the use of an L1 regularizer and compare it with other evolutionary computing approaches. To the best of our knowledge, neither the DPSO nor L1 optimizer has been jointly used to solve the S-System. We find that the combination of S-System and DPSO offers advantages over previously used methods, and presents promising results for inferencing larger and more complex networks.

Gene Regulatory Networks (GRNs) describe the interactions between different genes. One of the most important tasks in biology is to find the right regulations in a GRN given observed data. The problem, is that the data is often noisy and scarce, and we have to use models robust to noise and scalable to hundreds of genes. Recently, Recursive Neural Networks (RNNs) have been presented as a viable model for GRNs, which is robust to noise and can be scaled to larger networks. In this paper, to optimize the parameters of the RNN, we implement a classic Population Based Incremental Learning (PBIL), which in certain scenarios has outperformed classic GA and other evolutionary techniques like Particle Swarm Optimization (PSO). We test this implementation on a small and a large artificial networks. We further study the optimal tunning parameters and discuss the advantages of the method.

Estimation of distribution algorithms (EDAs) which deal with tree structures as GP are called as probabilistic model building GPs (PMBGPs), and they show better search performance than GP in many problems. A problem of prototype tree-based method, a type of PMBGPs, is that samplings do not always generate the most probable solution, which is the individual with the highest probability and reflects a learned distribution most. This problem wastes a part of learning and increases the number of evaluations to get an optimum solution. In order to overcome this difficulty, this paper proposes a hybrid approach using Belief propagation (BP) in sampling process. BP is an inference algorithm on graphical models and can generate the most probable solution. By applying our approach to benchmark tests, we show that the proposed method is more effective than PLS alone.

The Portfolio Optimization problem is an example of a resource allocation problem with money as the resource to be allocated to assets. We first have to select the assets from a pool of them available in the market and then assign proper weights to them to maximize the return and minimize the risk associated with the Portfolio. In our work, we have introduced a new “greedy coordinate ascent mutation operator” and we have also included the trading volumes concept. We performed simulations with the past data of NASDAQ100 and DowJones30, concentrating mainly on the 2008 recession period. We also compared our results with the indices and the simple Genetic Algorithms approach.

Guiding visitors through an exhibit space such as a museum is an important, early application for mobile robots, and commercial robots designed for this purpose have become available. We consider the problem of using a single mobile robot to simultaneously direct multiple groups of visitors through a museum or exhibition, and formulate an objective function for this task. We show that an evolutionary robotics approach using a simple, low-fidelity simulator and genetic programming can automatically generate robot controllers which can perform this task better than hand-coded controllers as well as humans in both simulation and on a real robot.

The performance of Differential Evolution (DE) algorithm is significantly affected by its parameter setting. But the choice of parameters is heavily dependent on the problem characteristics. Therefore, recently a couple of adaptation schemes that automatically adjust DE parameters have been proposed. The current work presents another adaptation scheme for DE parameters namely amplification factor and crossover rate. We systematically analyze the effectiveness of the proposed adaptation scheme for DE parameters using a standard benchmark suite consisting of ten functions. The undertaken empirical study shows that the proposed adaptive DE (aDE) algorithm exhibits an overall better performance compared to other prominent adaptive DE algorithms as well as canonical DE.

This paper proposes cellular differential evolution (cDE) algorithm for solving dynamic economic dispatch (DED) problems with valve-point effects. DEDs are high dimensional optimization problems with many equality and inequality constraints. The problem of premature convergence in solving high dimensional optimization problems using evolutionary algorithms (EAs) could be fought using population structuring. This work investigates the suitability a structured DE algorithm, called cDE, in solving these large dimensional optimization tasks. The suitability and effectiveness of the proposed algorithm is validated using two test systems consisting of 10 and 13 thermal units respectively. Numerical results clearly show that the proposed method outperforms existing methods in terms of solution quality and robustness.

Learning a ranking function is important for numerous tasks such as information retrieval (IR), question answering, and product recommendation. For example, in information retrieval, a Web search engine is required to rank and return a set of documents relevant to a query issued by a user. We propose RankDE, a ranking method that uses differential evolution (DE) to learn a ranking function to rank a list of documents retrieved by a Web search engine. To the best of our knowledge, the proposed method is the first DE-based approach to learn a ranking function for IR. We evaluate the proposed method using LETOR dataset, a standard benchmark dataset for training and evaluating ranking functions for IR. In our experiments, the proposed method significantly outperforms previously proposed rank learning methods that use evolutionary computation algorithms such as Particle Swam Optimization (PSO) and Genetic Programming (GP), achieving a statistically significant mean average precision (MAP) of 0.339 on TD2003 dataset and 0.430 on the TD2004 dataset. Moreover, the proposed method shows comparable results to the state-of-the-art non-evolutionary computational approaches on this benchmark dataset. We analyze the feature weights learnt by the proposed method to better understand the salient features for the task of learning to rank for information retrieval.

Baldwinian evolution is a type of hybridization of population-based global search and individual local search. The individuals take local refining processes, then in selection benefit from the improved fitness, but do not pass on the refined traits the data in to the offspring. The lost information of the refined phenotype implies that the inheritance encoded in genotypes is not directly benefit traits, but the traits having potential to achieve high fitness through the lifetime interaction with the environment. As the result, it is necessary to study how learning works comparing to the previous generation, in addition to how much it improves on the current population. The children may imitate what their parents performed and catch up with them, or alternatively, explore elsewhere and have no idea of where the parents arrived. In this paper, the trade-off is investigated, and it is revealed that in Baldwinian learning, the capability to follow the parents' footprints benefits. With higher imitation tendency, the evolving population can maintain a greater scale of learning potential, and the search results in better speed and convergence.

The performance of a memetic algorithm (MA) largely depends on the synergy between its global and local search counterparts. The amount of global exploration and local exploitation to be carried out, for optimal performance, varies with problem type. Therefore, an algorithm should intelligently allocate its computational efforts between genetic search and local search. In this work we propose an adaptive local search method that adjusts the effort for local tuning of individuals, taking feedback from the search. We implemented an MA hybridizing this adaptive local search method with differential evolution algorithm. Experimenting with a standard benchmark suite it was found that the proposed MA can utilize its global and local search components adaptively. The proposed algorithm also exhibited very competitive performance with other existing algorithms.

Recent study has introduced the powerful selection scheme in cellular genetic algorithm that can produce all ranges of selective pressure. The parameters used in that study, however, are empirically estimated by numbers of experiments. In this study, we propose the idea of performing a parameter estimation from a theoretical perspective. In the concept of maximizing the probability to find the new best solution together with hill-climbing optimization, enabling search for an optimal parameter in each generation. The selection scheme with the optimal parameter yields the numbers of mating that maximizes the probability of finding better solutions. This optimal parameter changes during run and it is adaptive to the behavior of a particular evolution. In order to confirm the capability of this parameter estimation method, we have conducted experiments to compare the manually tuned static parameter and the estimated dynamic parameter obtained from this method. Result from the experiment shows that the algorithm with estimated parameter performed better than the former method, even with the best tuned parameter. Therefore, by applying this parameter estimation to the selection scheme stated at the beginning, we would be able to create a new universal adaptive paradigm for the cellular evolutionary algorithm.

This paper presents an IEC (Interactive Evolutionary Computation) system named ConBreO to render expressive music performance using Genetic Programming. The central problem of IEC is the limitation of number of fitness evaluations because of user fatigue. In the system, we introduce two support techniques for IEC. The first one is a hybrid approach of IEC and automated evolution which allows the system to evolve both of IEC and automated evolution. The second one is the selective presentation which selects a new individual to be evaluated by the user based on its expected improvement of fitness. Using the system, obtained expression rule won an award at a performance rendering contest which evaluates computer systems generating expressive musical performances. Our experiment shows that the selective presentation reduces the number of fitness evaluations required to construct the fitness prediction model and prevents the system evaluating unfruitful individuals.

BACKGROUND: Gene regulatory network is an abstract mapping of gene regulations in living cells that can help to predict the system behavior of living organisms. Such prediction capability can potentially lead to the development of improved diagnostic tests and therapeutics. DNA microarrays, which measure the expression level of thousands of genes in parallel, constitute the numeric seed for the inference of gene regulatory networks. In this paper, we have proposed a new approach for inferring gene regulatory networks from time-series gene expression data using linear time-variant model. Here, Self-Adaptive Differential Evolution, a versatile and robust Evolutionary Algorithm, is used as the learning paradigm. RESULTS: To assess the potency of the proposed work, a well known nonlinear synthetic network has been used. The reconstruction method has inferred this synthetic network topology and the associated regulatory parameters with high accuracy from both the noise-free and noisy time-series data. For validation purposes, the proposed approach is also applied to the simulated expression dataset of cAMP oscillations in Dictyostelium discoideum and has proved it's strength in finding the correct regulations. The strength of this work has also been verified by analyzing the real expression dataset of SOS DNA repair system in Escherichia coli and it has succeeded in finding more correct and reasonable regulations as compared to various existing works. CONCLUSION: By the proposed approach, the gene interaction networks have been inferred in an efficient manner from both the synthetic, simulated cAMP oscillation expression data and real expression data. The computational time of this approach is also considerably smaller, which makes it to be more suitable for larger network reconstruction. Thus the proposed approach can serve as an initiate for the future researches regarding the associated area.

The Portfolio Optimization problem consists of the selection of a group of assets to a long-term fund in order to minimize the risk and maximize the return of the investment. This is a multi-objective (risk, return) resource allocation problem, where the aim is to correctly assign weights to the set of available assets, which determines the amount of capital to be invested in each asset.

The generation of profitable trading rules for Foreign Exchange (FX) investments is a difficult but popular problem. The use of Machine Learning in this problem allows us to obtain objective results by using information of the past market behavior. In this paper, we propose a Genetic Algorithm (GA) system to automatically generate trading rules based on Technical Indexes. Unlike related researches in the area, our work focuses on calculating the most appropriate trade timing, instead of predicting the trading prices.

This paper describes a new program evolution method named PORTS (Program Optimization by Random Tree Sampling) which is motivated by the idea of preservation and control of tree fragments. We hypothesize that to reconstruct building blocks efficiently, tree fragments of any size should be preserved into the next generation, according to their differential fitnesses. PORTS creates a new individual by sampling from the promising trees by traversing and transition between trees instead of subtree crossover and mutation. Because the size of a fragment preserved during a generation update follows a geometric distribution, merits of the method are that it is relatively easy to predict the behavior of tree fragments over time and to control sampling size, by changing a single parameter. Our experimental results on three benchmark problems show that the performance of PORTS is competitive with SGP (Simple Genetic Programming). And we observed that there is a significant difference of fragment distribution between PORTS and simple GP.

In recent years, program evolution algorithms based on the estimation of distribution algorithm (EDA) have been proposed to improve search ability of genetic programming (GP) and to overcome GP-hard problems. One such method is the probabilistic prototype tree (PPT) based algorithm. The PPT based method explores the optimal tree structure by using the full tree whose number of child nodes is maximum among possible trees. This algorithm, however, suffers from problems arising from function nodes having different number of child nodes. These function nodes cause intron nodes, which do not affect the fitness function. Moreover, the function nodes having many child nodes increase the search space and the number of samples necessary for properly constructing the probabilistic model. In order to solve this problem, we propose binary encoding for PPT. Here, we convert each function node to a subtree of binary nodes where the converted tree is correct in grammar. Our method reduces ineffectual search space, and the binary encoded tree is able to express the same tree structures as the original method. The effectiveness of the proposed method is demonstrated through the use of two computational experiments.

In order to get a better understanding of different types of cancers and to find the possible biomarkers for diseases, recently, many researchers are analyzing the gene expression data using various machine learning techniques. However, due to a very small number of training samples compared to the huge number of genes and class imbalance, most of these methods suffer from overfitting. In this paper, we present a majority voting genetic programming classifier (MVGPC) for the classification of microarray data. Instead of a single rule or a single set of rules, we evolve multiple rules with genetic programming (GP) and then apply those rules to test samples to determine their labels with majority voting technique. By performing experiments on four different public cancer data sets, including multiclass data sets, we have found that the test accuracies of MVGPC are better than those of other methods, including AdaBoost with GP. Moreover, some of the more frequently occurring genes in the classification rules are known to be associated with the types of cancers being studied in this paper.

Estimation of distribution algorithms are evolutionary algorithms using probabilistic techniques instead of traditional genetic operators. Recently, the application of probabilistic techniques to program and function evolution has received increasing attention, and this approach promises to provide a strong alternative to the traditional genetic programming techniques. Although a probabilistic context-free grammar (PCFG) is a widely used model for probabilistic program evolution, a conventional PCFG is not suitable for estimating interactions among nodes because of the context freedom assumption. In this paper, we have proposed a new evolutionary algorithm named programming with annotated grammar estimation based on a PCFG with latent annotations, which allows this context freedom assumption to be weakened. By applying the proposed algorithm to several computational problems, it is demonstrated that our approach is markedly more effective at estimating building blocks than prior approaches.

Recently, a number of works have been done on how to use Genetic Algorithms to solve the Portfolio Optimization problem, which is an instance of the Resource Allocation problem class. Almost all these works use a similar genomic representation of the portfolio: An array, either real, where each element represents the weight of an asset in the portfolio, or binary, where each element represents the presence or absence of an asset in the portfolio.

Genetic programming (GP) is a powerful optimization algorithm that has been applied to a variety of problems. This algorithm can, however, suffer from problems arising from the fact that a crossover, which is a main genetic operator in GP, randomly selects crossover points, and so building blocks may be destroyed by the action of this operator. In recent years, evolutionary algorithms based on probabilistic techniques have been proposed in order to overcome this problem. In the present study, we propose a new program evolution algorithm employing a Bayesian network for generating new individuals. It employs a special chromosome called theexpandedparsetree,which significantly reduces the size of the conditional probability table (CPT). Prior prototype tree-based approaches have been faced with the problem of huge CPTs, which not only require significant memory resources, but also many samples in order to construct the Bayesian network. By applying the present approach to three distinct computational experiments, the effectiveness of this new approach for dealing with deceptive problems is demonstrated.

We propose a crossover-based adaptive local search (LS) operation for enhancing the performance of standard differential evolution (DE) algorithm. Incorporating LS heuristics is often very useful in designing an effective evolutionary algorithm for global optimization. However, determining a single LS length that can serve for a wide range of problems is a critical issue. We present a LS technique to solve this problem by adaptively adjusting the length of the search, using a hill-climbing heuristic. The emphasis of this paper is to demonstrate how this LS scheme can improve the performance of DE. Experimenting with a wide range of benchmark functions, we show that the proposed new version of DE, with the adaptive LS, performs better, or at least comparably, to classic DE algorithm. Performance comparisons with other LS heuristics and with some other well-known evolutionary algorithms from literature are also presented.

Research on the application of Interactive Evolutionary Computation(IEC) to the field of musical computation has been improved in recent years, marking an interesting parallel to the current trend of applying human characteristics or sensitivities to computer systems. However, past techniques developed for IEC-based composition have not necessarily proven very effective for professional use. This is due to the large difference between data representation used by IEC and authored classical music composition. To solve this difficulties, the authors purpose a new IEC approach to music composition based on classical music theory. In this paper, the authors describe an established system according to the above idea, and detail of making success of composition a piece.

Portfolio optimization by GA is a problem that has recently received a lot of attention. However, most works in this area have so far ignored the effects of cost on Portfolio Optimization, and haven't directly addressed the problem of portfolio management (continuous optimization of a portfolio over time). In this work, we use the Euclidean Distance between the portfolio selection in two consecutive time periods as measure of cost, and the objective sharing method to balance the goals of maximizing returns and minimizing distance over time. We also improve the GA method by adding genetic material from previous runs into the new population (seeding). We experiment our method on historical monthly data from the NASDAQ and NIKKEI indexes, and obtain a better result than pure GA, defeating the index under non-bubble market conditions.

Genetic Programming (GP) which mimics the natural evolution to optimize functions and programs, has been applied to many problems. In recent years, evolutionary algorithms are seen from the viewpoint of the estimation of distribution. Many algorithms called EDAs (Estimation of Distribution Algorithms) based on probabilistic techniques have been proposed. Although probabilistic context free grammar (PCFG) is often used for the function and program evolution, it assumes the independence among the production rules. With this simple PCFG, it is not able to induce the building-blocks from promising solutions. We have proposed a new function evolution algorithm based on PCFG using latent annotations which weaken the independence assumption. Computational experiments on two subjects (the royal tree problem and the DMAX problem) demonstrate that our new approach is highly effective compared to prior approaches.

We present a memetic algorithm for evolving the structure of biomolecular interactions and inferring the effective kinetic parameters from the time series data of gene expression using the decoupled Ssystem formalism. We propose an Information Criteria based fitness evaluation for gene network model selection instead of the conventional Mean Squared Error (MSE) based fitness evaluation. A hill-climbing local-search method has been incorporated in our evolutionary algorithm for efficiently attaining the skeletal architecture which is most frequently observed in biological networks. The suitability of the method is tested in gene circuit reconstruction experiments, varying the network dimension and/or characteristics, the amount of gene expression data used for inference and the noise level present in expression profiles. The reconstruction method inferred the network topology and the regulatory parameters with high accuracy. Nevertheless, the performance is limited to the amount of expression data used and the noise level present in the data. The proposed fitness function has been found more suitable for identifying correct network topology and for estimating the accurate parameter values compared to the existing ones. Finally, we applied the methodology for analyzing the cell-cycle gene expression data of budding yeast and reconstructed the network of some key regulators.

Genetic programming (GP) is a powerful optimization algorithm and has been applied to many problems. GP is an extension of genetic algorithm (GA) which can handle programs, functions, etc. GP evolves with genetic operators such as crossover and mutation. The crossover operator in GP however selects sub-trees randomly and this selection is done regardless of the problem. This gives rise to the destruction of good building blocks. Recently, probabilistic model building techniques have been applied to GP to estimate the building blocks properly. This type of algorithm is called probabilistic model building GP (PMBGP). Because GP uses many types of nodes, prior PMBGPs have been faced with the problem of huge CPT (Conditional Probability Table) size. The large CPT not only consumes a lot of memory but also requires many samples to construct networks. We propose a new PMBGP that uses Bayesian network for generating new individuals. In our approach, a special chromosome called expanded parse tree is used to improve the problem of huge CPT size.

Noise is inevitable in microarray data. The real challenge lies in identifying the biomolecular interactions in spite of the significant noise level present in the expression profiles that current technology offers. In this paper, we study the usefulness of an evolutionary approach in reverse engineering the biomolecular connections in a gene circuit from observed system dynamics that is contaminated with noise. The method uses an Information Criteria based fitness evaluation for selecting models, represented in decoupled S-system formalism, instead of the conventional mean squared error (MSE) based fitness evaluation. The suitability of the method is tested in experiments of reconstructing an artificial network from gene expression profiles with varying noise levels. The proposed fitness function has been found more suitable for identifying correct network topology and for estimating the accurate parameter values compared to the existing one.

Recently, genetic programming (GP) has been applied to the classification of gene expression data. In its typical implementation, using training data, a single rule or a single set of rules is evolved with GP, and then it is applied to test data to get generalized test accuracy. However, in most cases, the generalized test accuracy is not higher. In this paper, we propose a majority voting technique for prediction of the labels of test samples. Instead of a single rule or a single set of rules, we evolve multiple rules with GP and then apply those rules to test samples to determine their labels by using the majority voting technique. We demonstrate the effectiveness of our proposed method by performing different types of experiments on two microarray data sets.

Recently modular robots have become more capable of practical applications with the recent improvement of sensors and actuators. Among them, a robot has been developed which can change its pattern according to the landscape or for its own purpose. However, deciding the appropriate pattern and controller manually is not easy with the increased number of modules. In this paper, we propose a new approach to automatically building patterns of block-type robots by means of artificial-life morphogenesis. We empirically show the emergence of the effective patterns in both virtual and real worlds, some of which seem to be surprisingly counter-intuitive.

Scleroderma is an autoimmune disease of the connective tissues, which thickens and hardens the affected areas. Recently, researchers have found evidence that genes are important factors for this disease, and there exist consistent differences in the patterns of gene expressions of skin biopsies from affected and non-affected individuals. In this paper, we apply genetic programming (GP) on the gene expression data of scleroderma and normal biopsies to evolve the classification rules that can differentiate between them. In these evolved rules, we have found six genes that have differential gene expression levels in scleroderma and normal biopsies and thus individually can classify all the samples correctly. In addition to these genes, we have also found some simple rules containing two or more genes that can classify all the samples perfectly

In this paper we present an evolutionary approach for inferring the structure and dynamics in gene circuits from observed expression kinetics. For representing the regulatory interactions in a genetic network the decoupled S-system formalism has been used. We proposed an Information Criteria based fitness evaluation for model selection instead of the traditional Mean Squared Error (MSE) based fitness evaluation. A hill climbing local search method has been incorporated in our evolutionary algorithm for attaining the skeletal architecture which is most frequently observed in biological networks. Using small and medium-scale artificial networks we verified the implementation. The reconstruction method identified the correct network topology and predicted the kinetic parameters with high accuracy.

We present a modified version of Differential Evolution (DE) for locating the global minimum at a higher convergence velocity. The proposed model differs from conventional DE by applying selection both for reproduction and survival, whereas the original model applies exclusively "knock-out" selection mechanism for survival. Because of its one-to-one reproduction strategy DE often consumes too many fitness evaluations to locate the global optimum. In this work we show that selecting parents for breeding and offspring for survival, DE's search capability can be further accelerated, which will be particularly useful for expensive function optimizations. Computational results using many benchmark functions are reported which show significant improvements in the convergence characteristics of the proposed algorithm over the original one.

Recognition of motifs in multiple unaligned sequences provides an insight into protein structure and function. The task of discovering these motifs is very challenging because most of these motifs exist in different sequences in different mutated forms of the original consensus motif and thus have weakly conserved regions. Different score metrics and algorithms have been proposed for motif recognition. In this paper, we propose a new genetic algorithm based method for identification of multiple motifs instances in multiple biological sequences. The experimental results on simulated and real data show that our algorithm can identify multiple occurrences of a weak motif in single sequences as well as in multiple sequences. Moreover, it can identify weakly conserved regions more accurately than other genetic algorithm based motif discovery methods.

We propose a new approach to generating the motion of humanoid robots intuitively by means of Interactive Evolutionary Computation (IEC). In our system, novice users are able to design effective motions through the subjective evaluation of displayed individuals, even if they do not have any technical knowledge. The motions evolved by the IEC system are not necessarily stable nor feasible in real environments. Thus, appropriate adjustments are required to revise the motions. For this purpose, we use a real-valued GA in a dynamic simulator. We empirically show the effectiveness of our approach by designing a kick motion for a humanoid robot. Categories and Subject Descriptors

The multi-agent cooperation has been proved useful in executing many complex tasks. In our previous paper, we proposed a path planning algorithm based on a random sampling for the sake of the multi-agent cooperation. However, the action path of the robots is liable to be deviated by the noise in the real world. Thus, some correction mechanism is required to reduce the differences between the planned path and the real one. In this paper, we propose a re-planning method to solve the above-mentioned difficulty. The applicability of this method is confirmed with the experiment using two humanoid robots, in which they have re-generated path plans according to their locations detected by their own cameras

To execute a task consisting of multiple subtasks using a robot, we need to determine the subtask execution order. We investigated the problem of cooperative object transport by multiple robots in a previous study. This is a task in which multiple robots carry an object to a specified goal by passing the object between each other. This paper proposes two algorithms that automatically determine the passing order between robots. This is a difficult task to find an optimum solution. Our algorithms reduce the amount of information that the operator needs to give to the robots and makes robot operations easy. The effectiveness of the algorithms was verified through simulation

In this work we present an improved evolutionary method for inferring S-system model of genetic networks from the time series data of gene expression. We employed Differential Evolution (DE) for optimizing the network parameters to capture the dynamics in gene expression data. In a preliminary investigation we ascertain the suitability of DE for a multimodal and strongly non-linear problem like gene network estimation. An extension of the fitness function for attaining the sparse structure of biological networks has been proposed. For estimating the parameter values more accurately an enhancement of the optimization procedure has been also suggested. The effectiveness of the proposed method was justified performing experiments on a genetic network using different numbers of artificially created time series data.

In this paper, we proposed Fittest Individual Refinement (FIR), a crossover based local search method for Differential Evolution (DE). The FIR scheme accelerates DE by enhancing its search capability through exploration of the neighborhood of the best solution in successive generations. The proposed memetic version of DE (augmented by FIR) is expected to obtain an acceptable solution with a lower number of evaluations particularly for higher dimensional functions. Using two different implementations DEfirDE and DEfirSPX we showed that proposed FIR increases the convergence velocity of DE for well known benchmark functions as well as improves the robustness of DE against variation of population. Experiments using multimodal landscape generator showed our proposed algorithms consistently outperformed their parent algorithms. A performance comparison with reported results of well known real coded memetic algorithms is also presented.

Identification of those genes that might anticipate the clinical behavior of different types of cancers is challenging due to availability of a smaller number of patient samples compared to huge number of genes, and the noisy nature of microarray data. After selection of some good genes based on signal-to-noise ratio, unsupervised learning like clustering and supervised learning like k-nearest neighbor (kNN) classifier are widely used in cancer researches to correlate the pathological behavior of cancers with the gene expression levels' differences in cancerous and normal tissues. By applying adaptive searches like Probabilistic Model Building Genetic Algorithm (PMBGA), it may be possible to get a smaller size gene subset that would classify patient samples more accurately than the above methods. In this paper, we propose a new PMBGA based method to extract informative genes from microarray data using Support Vector Machine (SVM) as a classifier. We apply our method to three microarray data sets and present the experimental results. Our method with SVM obtains encouraging results on those data sets as compared with the rank based method using kNN as a classifier.

This paper proposes a novel technique for a program evolution based on probabilistic models. In the proposed method, two probabilistic distribution models with probabilistic dependencies between variables are used together. We empirically comfirm that our proposed method has higher search performance. Thereafter, we discuss the effectiveness of its distribution models.

The cooperation of several robots is needed for complex tasks. The cooperation methods for multiple robots generally require exact goal or sub-goal positions. However, it is difficult to direct the goal or sub-goal positions to multiple robots for the sake of cooperation with each other. Planning algorithms reduce the burden for this purpose. In this paper, we propose a multi-agent planning algorithm based on a random sampling method. This method doesn't require the exact sub-goal positions nor the times at which cooperation occurs. The effectiveness of this approach is empirically shown by simulation results.

We propose an integrated technique of genetic programming (GP) and reinforcement learning (RL) to enable a real robot to adapt its actions to a real environment. Our technique does not require a precise simulator because learning is achieved through the real robot. In addition, our technique makes it possible for real robots to learn effective actions. Based on this proposed technique, we acquire common programs, using GP, which are applicable to various types of robots. Through this acquired program, we execute RL in a real robot. With our method, the robot can adapt to its own operational characteristics and learn effective actions. In this paper, we show experimental results from two different robots: a four-legged robot "AIBO" and a humanoid robot "HOAP-1." We present results showing that both effectively solved the box-moving task; the end result demonstrates that our proposed technique performs better than the traditional Q-learning method.

This paper proposes a method to capture the dynamics of gene expression data using S-system formalism and construct genetic network models. The proposed method exploits the probabilistic heuristic search and divide-and-conquer approach to generate candidate network structures. In evaluating the network structure, we attempt a primitive integration of other knowledge to the statistical criterion. The robustness analysis uses Z-score to identify significant parameters from results of stochastic search. We evaluated the proposed method on artificial generated data and E.coli mRNA expression data.

In this paper, we propose an approach to the behavior acquisition required for humanoid robots to learn a cooperative transportation task. In case of object transportation with two humanoid robots, mutual position shifts may occur due to the body swinging of robots. Therefore, it is necessary to correct the position in a real-time manner. Many efforts are needed to develop the position shift correction system. We propose to solve the problem by learning required behaviors with two learning algorithms. Successful cooperation of two HOAP-1 humanoid robots in the transportation task obtained by classifier system and Q-learning has been confirmed experimentally in our work.

In order to acquire useful motions of a real-world robot, it is necessary to carry out learning in a real environment. However, learning is difficult within a real environment. In addition, the acceleration of learning is required for a practical execution. We propose an approach to the learning acceleration using data retrieved from the real environment. This consists of the method of automatically constructing the simulator from real data and of learning a robot controller with the simulator. The experimental results suggest that our GP-based technique enables the effective controller learning.

We propose a hierarchical approach called "CBR augmented GP" to evolve robust control programs for humanoid robots. Humanoid robots are high-dimensional systems; thus it is very difficult for GP to generate control programs for humanoid robots. The key idea in our approach is to extract control rules with GP in simplified simulation and get a prototype of the control program then interpret and interpolate it with case-based reasoning (CBR) in the real world environments. Accordingly, our proposed approach consists of two stages: the evolution stage and the adaptation stage. In the first stage, the prototype of the control program is evolved based on abstract primitive behaviors in a highly simplified simulation. In the second stage, the best control program is applied to a physical robot thereby adapting it to the real world environments by using CBR. Experimental results show that this approach can generate robust control programs that can easily overcome gaps between simplified simulation and real world. Furthermore, the robot can adapt to new environments which it never encountered in simulation.

A very effective means to study the gene networks is visualization. With rapid increase of the size of gene networks, it has become more realistic to identify the collaborating genes in the network, which will facilitate the behavioral study of the groups and the network as a whole. In our previous paper, we presented a layered approach for visualizing gene regulatory networks. In this paper, we present a 3D layout model for visualizing gene networks, which clusters the correlated genes depending on their causal relationships. To demonstrate the effectiveness of the approach, we visualize real gene networks of different sizes. The experimental results show the superiority and usefulness of the new model when compared with previous results.

Recently, there has been a growing interest in classification of patient samples based on gene expressions. Here the classification task is made more difficult by the noisy nature of the data, and by the overwhelming number of genes relative to the number of available training samples in the data set. Moreover, many of these genes are irrelevant for classification and have negative effect on the accuracy and on the required learning time for the classifier. We propose a new evolutionary computation method to select the most useful subset of genes for molecular classification. We apply this method to three benchmark data sets and present our unbiased experimental results.

Constructing genetic network model from microarray data is an important approach to understanding the functions of the genes. Proposed in this paper is the use of pair-wise correlation analysis to capture regulation and co-regulation among genes. It considers pair-wise p-metrics correlation between the expression of the genes and also the change in expression of the genes. The method is used alongside meta heuristic approach to construct gene regulatory networks of difference and differential equation model from microarray data. The evolutionary algorithms are used to find the structure of the model with highest criteria. Using the result of the analysis improves the performance of the meta heuristics and allows us to extract relations from gene expression of E. coli and S. cerevisiae.

In recent years, base sequences have been increasingly unscrambled through attempts represented by the human genome project. Accordingly, the estimation of the genetic network has been accelerated. However, no definitive method has become available for drawing a large effective graph. This paper proposes a method which allows for coping with an increase in the number of nodes by laying out genes on planes of several layers and then overlapping these planes. This layout involves an optimization problem which requires maximizing the fitness function. To demonstrate the effectiveness of our approach, we show some graphs using actual data on 82 genes, 552 genes, and artificial data modeled from a scale-free network of 1,000 genes. We also describe how to lay out nodes by means of stochastic searches, e.g., stochastic hill-climbing and simulating annealing methods. The experimental results show the superiority and usefulness of stochastic searches in comparison with the simple random search.

We introduce a technique that allows a real robot to execute a real-time learning, in which GP and RL are integrated. In our former research, we showed the result of an experiment with a real robot "AIBO" and proved the technique performed better than the traditional Q-learning method. Based on the proposed technique, we can acquire the common programs using a GP, applicable to various types of robots. We execute reinforcement learning with the acquired program in a real robot. In this way, the robot can adapt to its own operational characteristics and learn effective actions. In this paper, we show the experimental results in which a humanoid robot "HOAP-1" has been evolved to perform effectively to solve the box-moving task.

We present the emergence of cooperative behaviors of heterogeneous robots by means of evolutionary subsumption in both simulation and real world environments. The key idea of evolutionary subsumption is to apply GP to the design of subsumption architecture, thus hierarchically constructs the control architecture of robots, and enables us to build domain knowledge into the genetic programming system. We claim that this method can facilitate the transformation from simulation to real world. Our approach is evaluated with an "eye"-"hand" cooperation problem. The domain knowledge of this problem is that the "eye" is an observer and the "hand" is an actor, namely we let the "eye" to observe each action of the "hand" and give appraisement as reinforcement signal to the "hand", thus endow the "hand" with learning ability. Experimental results show that by applying this approach, multirobot system exhibits identical behaviors in simulation and real world.

We propose estimation of distribution programming (EDP) based on a probability distribution expression using a Bayesian network. EDP is a population-based program search method, in which the population probability distribution is estimated, and individuals are generated based on the results. We focus our attention on the fact that the dependency relationship of nodes of the program (expressed as a tree structure) is explicit, and estimate the probability distribution of the program population using a Bayesian network. We compare EDP with GP (genetic programming) on several benchmark tests, i.e., a max problem and a Boolean function problem. We also discuss the trends of problems that are the forte of EDP.

In generating a program automatically, if we do not know whether the problem is solvable or not in advance, then the representation of the program must be turing-complete, i.e. the representation must be able to express any algorithms. However, a tree structure used by the standard genetic programming is not turing-complete. We propose a representation scheme, which is a recurrent network consisting of trees. It makes genetic programming turing-complete without introducing any new non-terminals. In addition, we empirically show how it succeeds in evolving language classifiers.

In this paper we present particle swarm optimization with Gaussian mutation combining the idea of the particle swarm with concepts from evolutionary algorithms. This method combines the traditional velocity and position update rules with the ideas of Gaussian mutation. This model is tested and compared with the standard PSO and standard GA. The comparative experiments have been conducted on unimodal functions and multimodal functions. PSO with Gaussian mutation is able to obtain a result superior to GA. We also apply the PSO with Gaussian mutation to a gene network. Consequently, it has succeeded in acquiring better results than those by GA and PSO alone.

This paper presents an approach to learning polynomial feedforward neural networks (PFNNs). The approach suggests, first, finding the polynomial network structure by means of a population-based search technique relying on the genetic programming paradigm, and second, further adjustment of the best discovered network weights by an especially derived backpropagation algorithm for higher order networks with polynomial activation functions. These two stages of the PFNN learning process enable us to identify networks with good training as well as generalization performance. Empirical results show that this approach finds PFNN which outperform considerably some previous constructive polynomial network algorithms on processing benchmark time series.

We present an evolutionary modeling method for modeling genetic regulatory networks. The method features a hybrid algorithm of genetic programming with statistical analysis to derive systems of differential equations. Genetic programming and the least mean squares method were combined to identify a concise form of regulation between the variables from a given set of time series. Results of multiple runs were statistically analyzed to indicate the term with robust and significant influence. Our approach was evaluated in artificial data and real world data.

This paper proposes a new method for island model GP. The proposed method applies a traditional genetic operator to an aborigine and a depth-dependent crossover to the immigrants according to their ages, which show how long they survive in the island. This method can provide both local and global search strategies. The experimental results have shown that our approach works effectively.

This paper proposes several techniques for avoiding overfitting in the genetic programming (GP) of polynomials. The model specification flexibility is increased by: (1) a polynomial block reformulation, which reduces the statistical bias, and, (2) complexity tuning using local ridge regression and regularized weight subset selection, which reduce the statistical variance. Another contribution is the designed fitness function for search navigation towards highly predictive models. Experimental results on time-series forecasting show that these techniques help GP to find accurate, less complex and better forecasting polynomials than traditional Koza-style GP (J.R. Koza, 1992) and the previous Stroganoff system (H. Iba et al., 1994, 2001).

This paper presents a genetic programming system that evolves polynomial harmonic networks. These are multilayer feed-forward neural networks with polynomial activation functions. The novel hybrids assume that harmonics with non-multiple frequencies may enter as inputs the activation polynomials. The harmonics with non-multiple, irregular frequencies are derived analytically using the discrete Fourier transform. The polynomial harmonic networks have tree-structured topology which makes them especially suitable for evolutionary structural search. Empirical results show that this hybrid genetic programming system outperforms an evolutionary system manipulating polynomials, the traditional Koza-style genetic programming, and the harmonic GMDH network algorithm on processing time series.

Presents an application of genetic algorithms (GAs) to the gene network inference problem; this is one of the active topics in recent bioinformatics. The objective is to predict a regulating network structure of the interacting genes from the observed outcome, i.e. expression pattern. The task consists of modeling the rules of regulation and inferring the network structure from the observed data. The GA is applied to training the model with observed data in order to predict the regulatory pathways, represented as an influence matrix. We have implemented a reverse engineering method based on GAs in a quantitative and linear biological framework. The merit of this approach is that it can be applied with a small amount of data, it can optimize large numbers of parameters simultaneously and it can be applied to nonlinear models. The GA implementation includes multi-stage evolution and matrix chromosomes. This method has been applied to both simulated and experimentally observed gene expression patterns. In this research, we used the knowledge of designing an electric circuit by a GA.

This paper presents a Genetic Programming (GP) system that evolves polynomial harmonic networks. The hybrid tree-structured network representation suggests that terminal harmonics with non-multiple frequencies may enter polynomial function nodes as variables. The harmonics with non-multiple, irregular frequencies are derived analytically using the discrete Fourier transform. The development of polynomial harmonic GP includes also design of a regularized statistical fitness function for improved search control and overfitting avoidance. Empirical results show that this hybrid version outperforms the previous GP system manipulating polynomials STROGANOFF, the traditional Koza-style GP, and the harmonic GMDH network algorithm on processing time series.

The paper presents a genetic programming (GP) system that evolves polynomial harmonic networks. The hybrid tree-structured network representation suggests that terminal harmonics with non-multiple frequencies may enter polynomial function nodes as variables. The harmonics with non-multiple, irregular frequencies are derived analytically using the discrete Fourier transform. The development of polynomial harmonic GP includes also design of a regularized statistical fitness function for improved search control and overfitting avoidance. Empirical results show that this hybrid version outperforms the previous GP system manipulating polynomials STROGANOFF, the traditional Koza-style GP, and the harmonic GMDH network algorithm on processing time series.

Describes an evolutionary method for identifying a gene regulatory network from the observed time series data of the gene's expression. We use a system of ordinary differential equations as a model of the network and infer their right-hand sides by using genetic programming (GP). To explore the search space more effectively in the course of evolution, the least mean squares (LMS) method is used along with ordinary GP. We apply our method to three target networks and empirically show how successfully GP infers the systems of differential equations.

This paper presents an approach to regularization of inductive genetic programming tuned for learning polynomials. The objective is to achieve optimal evolutionary performance when searching high-order multivariate polynomials represented as tree structures. We show how to improve the genetic programming of polynomials by balancing its statistical bias with its variance. Bias reduction is achieved by employing a set of basis polynomials in the tree nodes for better agreement with the examples. Since this often leads to over-fitting, such tendencies are counteracted by decreasing the variance through regularization of the fitness function. We demonstrate that this balance facilitates the search as well as enables discovery of parsimonious, accurate, and predictive polynomials. The experimental results given show that this regularization approach outperforms traditional genetic programming on benchmark data mining and practical time-series prediction tasks.

This paper proposes a system of evolving analog circuits based on a variable length chromosome. Methods featured are the chromosome of a component list, the multi-stage evolution, and the pressure on the circuit size. A set of experiments are described to confirm the system's robustness, the scalability of a circuit, and the efficiency of time and the memory consumption. The first experiment shows the robustness supplied by the evolutionary method. The second one compares several types of chromosome implementation schemes. We also provide experiments to evaluate the multi-stage and scaling methods.

The problem of identifying the trend in financial data series in order to forecast them for profit increase is addressed using genetic programming (GP). We enhance a GP system that searches for polynomial models of financial data series and relate it to a traditional GP manipulating functional models. Two of the key issues in the development are: 1) preprocessing of the series which includes data transformations and embedding; and 2) design of a proper fitness function that navigates the search by favouring parsimonious and predictive models. The two GP systems are applied for stock market analysis, and examined with real Tokyo Stock Exchange data. Using statistical and economical measures to estimate the results, we show that the GP could evolve profitable polynomials.

This paper presents the application of genetic programming (GP) to the prediction of price data in the Japanese stock market. The goal of this task is to choose the best stocks when making an investment and to decide when and how many stocks to sell or buy. There have been several applications of genetic algorithms (GAs) to financial problems, such as portfolio optimization, bankruptcy prediction, financial forecasting, fraud detection and scheduling. GP has also been applied to many problems in time-series prediction. However, relatively few studies have been made for the purpose of predicting stock market data by means of GP. This paper describes how successfully GP is applied to predicting stock data so as to gain a high profit. Comparative experiments are conducted with neural networks to show the effectiveness of the GP-based approach.

info:eu-repo/semantics/published

This paper introduces a new approach to genetic programming (GP), based on a numerical technique, which integrates a GP-based adaptive search of tree structures, and a local parameter tuning mechanism employing statistical search (a system identification technique). In traditional GP, recombination can cause frequent disruption of building blocks or mutation can cause abrupt changes in the semantics. To overcome these difficulties, we supplement traditional GP with a local hill-climbing search, using a parameter tuning procedure. More precisely, we integrate the structural search of traditional GP with a multiple regression analysis method and establish our adaptive program, called STROGANOFF (STructured Representation On Genetic Algorithms for NOn-linear Function Fitting). The fitness evaluation is based on a minimum description length (MDL) criterion, which effectively controls the tree growth in GP. We demonstrate its effectiveness by solving several system identification (numerical) problems and compare the performance of STROGANOFF with traditional GP and another standard technique (radial basis functions). We then extend STROGANOFF to symbolic (nonnumerical) reasoning by introducing multiple types of nodes, using a modified MDL-based selection criterion and a pruning of the resultant trees. The effectiveness of this numerical approach to GP is demonstrated by successful application to symbolic regression problems.