Genetic programming-based feature construction has achieved significant success in recent years as an automated machine learning technique to enhance learning performance. However, overfitting remains a challenge that limits its broader applicability. To improve generalization, we prove that vicinal risk, estimated through noise perturbation or mixup-based data augmentation, is bounded by the sum of empirical risk and a regularization termb–either finite difference or the vicinal Jensen gap. Leveraging this decomposition, we propose an evolutionary feature construction framework that jointly optimizes empirical risk and the vicinal Jensen gap to control overfitting. Since datasets may vary in noise levels, we develop a noise estimation strategy to dynamically adjust regularization strength. Furthermore, to mitigate manifold intrusionb–where data augmentation may generate unrealistic samples that fall outside the data manifoldb–we propose a manifold intrusion detection mechanism. Experimental results on 58 datasets demonstrate the effectiveness of Jensen gap minimization compared to other complexity measures. Comparisons with 15 machine learning algorithms further indicate that genetic programming with the proposed overfitting control strategy achieves superior performance.

Genetic programming-based symbolic regression is a widely used machine learning technique, but its effectiveness can be limited as the number of input features increases. In genetic programming, two-stage feature selection has been extensively applied to enhance performance when dealing with a large number of input features. Existing two-stage feature selection methods typically require reinitializing new GP trees based on the selected features after feature selection, which disrupts the building blocks accumulated during evolution. In this paper, we propose a crossover operator that is aware of the selected features to leverage the feature selection results, thereby bypassing the need for reinitialization. This operator guides the crossover process to prioritize selected features, gradually eliminating unimportant features while preserving evolved building blocks. Experimental results validate the proposed method across three different feature-selection mechanisms on 98 datasets, demonstrating its effectiveness and broad applicability across various feature-selection strategies.

Neutrality, characterized by pathways in the genotype space that do not alter the phenotype or fitness, enables a broad exploration of evolutionary search. Simplicity bias describes the tendency of evolutionary systems to favor low-complexity solutions. This study investigates how neutrality contributes to simplicity bias in evolutionary systems using a Boolean Linear Genetic Programming framework. We introduce two fitness functions that utilize symmetry in solutions to promote neutrality, to analyze their effects on neutral network connectivity and search dynamics. Our results demonstrate that simpler phenotypes, characterized by lower Kolmogorov complexity, exhibit greater redundancy and connectivity, making them more accessible during neutral exploration. In addition, the proposed fitness functions significantly improve search success rates, especially for complex target phenotypes, by expanding neutral pathways. These findings shed light on the role of neutrality in shaping simplicity bias and provide practical insights to improve the effectiveness of evolutionary algorithms.

Lexicase parent selection considers training cases separately, postulating that aggregated fitness reduces the information about the behavior of individuals. Originally lexicase was proposed in the context of program synthesis, characterized by uncompromising problems that require qualitatively different actions for different inputs, but it has since been extended to regression problems. To facilitate valley-crossing a relaxation parameter ϵ was added broadening the pass condition at a given training case. Although ϵ-lexicase has demonstrated superior effectiveness, it was compared against selection methods that aggregated squared (or absolute) errors. Recent contributions, however, demonstrate that correlation fitness functions can lead to significant performance gains over the root mean square error (RMSE) in tournament-guided evolution for symbolic regression. Here we compare ϵ-lexicase (with and without down-sampling) against tournament selection using both error- and correlation-based fitness to guide Genetic Programming (GP). We also assess batch ϵ-lexicase selection as an intermediate condition. Finally, we explore different selection pressures to assess the exploration-exploitation trade-off. We analyze the experimental results using different metrics, including code redundancy, sharpness-awareness and selection impact. Our results demonstrate that tournament selection with correlation fitness function significantly outperforms ϵ-lexicase on regression problems and that its batch variant also benefits from correlation-based aggregation.

Symbolic regression is a key task in machine learning, aiming to discover mathematical expressions that best describe a dataset. While deep learning has increased interest in using neural networks for symbolic regression, many existing approaches rely on pre-trained models. These models require significant computational resources and struggle with regression tasks involving unseen functions and variables. A pre-training-free paradigm is needed to better integrate with search-based symbolic regression algorithms. To address these limitations, we propose a novel framework for symbolic regression that integrates evolutionary feature construction with a neural network, without the need for pre-training. Our approach adaptively generates symbolic trees that align with the desired semantics in real-time using a language model trained via online supervised learning, providing effective building blocks for feature construction. To mitigate hallucinations from the language model, we design a retrieval-augmented generation mechanism that explicitly leverages searched symbolic expressions. Additionally, we introduce a scale-invariant data augmentation technique that further improves the robustness and generalization of the model. Experimental results demonstrate that our framework achieves state-of-the-art accuracy across 25 regression algorithms and 120 regression tasks.

The deployment of satellite constellations for internet access has garnered significant interest in recent years. Current research trends focus on optimizing parameters of uniformly distributed symmetric constellations such as Starlink and Kuiper to better suit evaluation metrics like land coverage or ground station revisit time. These studies largely do not take into account the real distribution of people around the world, nor do they explore the field of Asymmetric Constellations. This work presents a novel optimization algorithm that uniquely designs asymmetric satellite constellations to focus on maximizing population coverage. Through our system evaluation with a fitness function and extensive simulations, our optimized constellations reduce redundancy and enhance efficiency. Results demonstrate that our asymmetrically designed constellations outperform standard uniform constellations in several use cases representing the United States, Europe, and a global scale, highlighting the potential for high-impact satellite deployment.

This article examines various methods of computing uncertainty and diversity for active learning in genetic programming. We found that the model population in genetic programming can be exploited to select informative training data points by using a model ensemble combined with an uncertainty metric. We explored several uncertainty metrics and found that differential entropy performed the best. We also compared two data diversity metrics and found that correlation as a diversity metric performs better than minimum Euclidean distance, although there are some drawbacks that prevent correlation from being used on all problems. Finally, we combined uncertainty and diversity using a Pareto optimization approach to allow both to be considered in a balanced way to guide the selection of informative and unique data points for training.

Searching for symbolic models plays an important role in a wide range of domains such as neural architecture search and automatic program synthesis. Genetic programming is a promising stochastic method for searching effective symbolic models within an acceptable time. The genetic programming performance is closely related to the hardness of the fitness landscape. A better fitness landscape with less local optima normally implies that it is easier to search for better solutions. In recent years, there have been many studies enhancing genetic programming performance by forming better fitness landscapes. However, the better design of the fitness landscape highly relies on specific domain knowledge and consumes a lot of expert effort. This paper proposes a fitness landscape optimization method to automatically design better fitness landscapes for genetic programming search than the manually designed ones. We optimize the landscapes by optimizing the neighborhood structures of symbolic solutions. We verify the effectiveness of the proposed method in both supervised learning and combinatorial optimization problems. The results show that the proposed method significantly reduces the hardness of fitness landscapes. By simply searching against the automatically optimized fitness landscapes, a genetic programming method can have a very competitive performance with state-of-the-art methods.

In this contribution, we explore Cartesian Genetic Programming for image analysis of biomedical data. Producing large quantities of human-labeled biomedical data is an expensive task. Here, we introduce a way for CGP to use a small amount of training data, without loss in performance. To define the size of the training data, we utilize an Active Learning method to direct the algorithm towards informative samples. We examine how sampling a small set of data from the CELLPOSE dataset affects the performance of CGP. We also study the effects of restarting CGP with Active Learning. We found that using several restarts can lead to a more diverse set of the highest-performing solutions with fewer active nodes while maintaining similar performance to standard CGP.

In this contribution, we discuss the basic concepts of genotypes and phenotypes in tree-based GP (TGP), and then analyze their behavior using five real-world datasets. We show that TGP exhibits the same behavior that we can observe in other GP representations: At the genotypic level trees show frequently unchecked growth with seemingly ineffective code, but on the phenotypic level, much smaller trees can be observed. To generate phenotypes, we provide a unique technique for removing semantically ineffective code from GP trees. The approach extracts considerably simpler phenotypes while not being limited to local operations in the genotype. We generalize this transformation based on a problem-independent parameter that enables a further simplification of the exact phenotype by coarse-graining to produce approximate phenotypes. The concept of these phenotypes (exact and approximate) allows us to clarify what evolved solutions truly predict, making GP models considered at the phenotypic level much better interpretable.

Evolutionary feature construction is a technique that has been widely studied in the domain of automated machine learning. A key challenge that needs to be addressed in feature construction is its tendency to overfit the training data. Instead of the traditional approach to control overfitting by reducing model complexity, this paper proposes to control overfitting based on bias-variance decomposition. Specifically, this paper proposes reducing the variance of a model, i.e., reducing the variance of predictions when exposed to data with injected noise, to improve its generalization performance within a multi-objective optimization framework. Experiments conducted on 42 datasets demonstrate that the proposed method effectively controls overfitting and outperforms six model complexity measures for overfitting control. Moreover, further analysis reveals that controlling overfitting adhering to bias-variance decomposition outperforms several plausible variants, highlighting the importance of controlling overfitting based on solid machine learning theory.

Industrialized supply chains significantly impact the environment by accelerated greenhouse gas emissions. As supply chains get complex, they suffer from fragmentation in terms of sharing knowledge among participants. Fragmented chains such as the meat business, encompasses sub-stages like feed production, processing, distribution and retail but incorporate bare minimum vertical integration. This hinders measurement of carbon footprint against products being shipped. Lack of in-frastructure to estimate emissions at different independent stages results in lost opportunity to minimize emissions from end-to-end. To address issues arising from isolated supply chain participants, we propose a decentralized framework leveraging blockchain functions, internet of things, and distributed databases to allow to capture fine-grained greenhouse gas emissions across supply chain for joint optimization of underlying resource consumption. The proposed framework facilitates formation of a mix of local and global collaboration groups for precise carbon emission tracing while ensuring privacy and transparency. Key frame-work features include system's extensibility and scalability for integration of diverse information sources, secure data capture mechanism, and propagation of data and policies via blockchain and internet of things infrastructure. Our proposed solution aims to offer a flexible, comprehensive, and collaborative approach to recording, monitoring and optimizing carbon footprint across complex disjoint supply chains, thereby promoting improved environmental oupnut management.

Gene regulatory networks are networks of interactions in organisms responsible for determining the production levels of proteins and peptides. Mathematical and computational models of gene regulatory networks have been proposed, some of them rather abstract and called artificial regulatory networks. In this contribution, a spatial model for gene regulatory networks is proposed that is biologically more realistic and incorporates an artificial chemistry to realize the interaction between regulatory proteins called the transcription factors and the regulatory sites of simulated genes. The result is a system that is quite robust while able to produce complex dynamics similar to what can be observed in nature. Here an analysis of the impact of the initial states of the system on the produced dynamics is performed, showing that such models are evolvable and can be directed toward producing desired protein dynamics.

Evolutionary feature construction is a key technique in evolutionary machine learning, with the aim of constructing high-level features that enhance performance of a learning algorithm. In real-world applications, engineers typically construct complex features based on a combination of basic features, re-using those features as modules. However, modularity in evolutionary feature construction is still an open research topic. This paper tries to fill that gap by proposing a modular and hierarchical multitree genetic programming (GP) algorithm that allows trees to use the output values of other trees, thereby representing expressive features in a compact form. Based on this new representation, we propose a macro parent-repair strategy to reduce redundant and irrelevant features, a macro crossover operator to preserve interactive features, and an adaptive control strategy for crossover and mutation rates to dynamically balance the trade-off between exploration and exploitation. A comparison with seven bloat control methods on 98 regression datasets shows that the proposed modular representation achieves significantly better results in terms of test performance and smaller model size. Experimental results on the state-of-the-art symbolic regression benchmark demonstrate that the proposed symbolic regression method outperforms 22 existing symbolic regression and machine learning algorithms, providing empirical evidence for the superiority of the modularized evolutionary feature construction method.

In recent years, genetic programming has achieved impressive results on evolutionary feature construction tasks. To increase search effectiveness, researchers have developed many semantic-based crossover and mutation operators to guide genetic programming searches toward the target semantics. However, semantics has not yet been explored for the hoist mutation operator, which is an operator designed for controlling the bloat effect. Although the hoist mutation operator can significantly reduce model sizes, the most informative subtree may be disrupted by the randomness in mutation. To address this issue, we develop a semantic-based hoist mutation operator in this paper to preserve the most informative subtree that has the largest cosine similarity between its semantics and the target semantics. Experimental results on 98 regression datasets from the Penn Machine Learning Benchmark show that using this operator not only significantly reduces model size, but also improves the test accuracy of features constructed by genetic programming. A comparison with seven bloat control methods shows that the proposed operator achieves the best trade-off between accuracy and model size. Moreover, an experiment on the state-of-the-art symbolic regression benchmark shows that genetic programming with the semantic-based hoist mutation operator achieves the best test accuracy and competitive model sizes compared with 22 symbolic regression and machine learning algorithms.

The difficulty of learning optimal coefficients in regression models using only genetic operators has long been a challenge in genetic programming for symbolic regression. As a simple but effective remedy it has been proposed to perform linear scaling of model outputs prior to a fitness evaluation. Recently, the use of a correlation coefficient-based fitness function with a post-processing linear scaling step for model alignment has been shown to outperform error-based fitness functions in generating symbolic regression models. In this study, we compare the impact of four evaluation strategies on relieving genetic programming (GP) from learning coefficients in symbolic regression and focusing on learning the more crucial model structure. The results from 12 datasets, including ten real-world tasks and two synthetic datasets, confirm that all these strategies assist GP to varying degrees in learning coefficients. Among the them, correlation fitness with one-time linear scaling as post-processing, due to be the most efficient while bringing notable benefits to the performance, is the recommended strategy to relieve GP from learning coefficients.

Automated machine learning (AutoML) greatly eases human efforts in architecture engineering. However, mainstream AutoML methods like neural architecture search (NAS) are customized for well-designed search spaces wherein promising architectures are densely distributed. In contrast, AutoML-Zero builds machine-learning algorithms using basic primitives and can explore novel architectures beyond human knowledge. AutoML-Zero shows the potential to deploy machine learning systems by not taking advantage of either feature engineering or architectural engineering. In its current form, it only optimizes a single objective like accuracy and has no mechanism to ensure that the constraints of real-world applications are satisfied. We propose a multi-objective variant of AutoML-Zero called MOAZ, that distributes solutions on a Pareto front by trading off accuracy against the computational complexity of the machine learning algorithm. In addition to generating different Pareto-optimal solutions, MOAZ can effectively explore the sparse search space to improve search efficiency. Experimental results on linear regression tasks show MOAZ reduces the median complexity by 87.4% compared to AutoML-Zero while accelerating the median target performance achievement speed by 82%. In addition, our preliminary results on non-linear regression tasks show the potential for further improvements in search accuracy and for reducing the need for human intervention in AutoML.

Evolutionary feature construction is an important technique in the machine learning domain for enhancing learning performance. However, traditional genetic programming-based feature construction methods often suffer from bloat, which means the sizes of constructed features increase excessively without improved performance. To address this issue, this paper proposes a double-stage lexicase selection operator to control bloat while not damaging search effectiveness. This new operator contains a two-stage selection process, where the first stage selects individuals based on fitness values and the second stage selects individuals based on tree sizes. Therefore, the proposed operator can control bloat meanwhile leveraging the advantage of the lexicase selection operator. Experimental results on 98 regression datasets show that compared to the traditional bloat control method of having a depth limit, the proposed selection operator not only significantly reduces the sizes of constructed features on all datasets but also keeps a similar level of predictive performance. A comparative experiment with seven bloat control methods shows that the double lexicase selection operator achieves the best trade-off between the model performance and the model size.

Autonomous robots deployed in the real world will need control policies that rapidly adapt to environmental changes. To this end, we propose AutoRobotics-Zero (ARZ), a method based on AutoML-Zero that discovers zero-shot adaptable policies from scratch. In contrast to neural network adaption policies, where only model parameters are optimized, ARZ can build control algorithms with the full expressive power of a linear register machine. We evolve modular policies that tune their model parameters and alter their inference algorithm on-the-fly to adapt to sudden environmental changes. We demonstrate our method on a realistic simulated quadruped robot, for which we evolve safe control policies that avoid falling when individual limbs suddenly break. This is a challenging task in which two popular neural network baselines fail. Finally, we conduct a detailed analysis of our method on a novel and challenging non-stationary control task dubbed Cataclysmic Cartpole. Results confirm our findings that ARZ is significantly more robust to sudden environmental changes and can build simple, interpretable control policies.

We evolve floating point Sextic polynomial populations of genetic programming binary trees for up to a million generations. We observe continued innovation but this is limited by tree depth. We suggest that deep expressions are resilient to learning as they disperse information, impeding evolvability, and the adaptation of highly nested organisms, and we argue instead for open complexity. Programs with more than 2,000,000,000 instructions (depth 20,000) are created by crossover. To support unbounded long-term evolution experiments in genetic programming (GP), we use incremental fitness evaluation and both SIMD parallel AVX 512-bit instructions and 16 threads to yield performance equivalent to 1.1 trillion GP operations per second, 1.1 tera GPops, on an Intel Xeon Gold 6136 CPU 3.00GHz server.

The modern economy is both a complex self-organizing system and an innovative, evolving one. Contemporary theory, however, treats it essentially as a static equilibrium system. Here we propose a formal framework to capture its complex, evolving nature. We develop an agent-based model of an economic system in which firms interact with each other and with consumers through market transactions. Production functions are represented by a pair of von Neumann technology matrices, and firms implement production plans taking into account current price levels for their inputs and output. Prices are determined by the relation between aggregate demand and supply. In the absence of exogenous perturbations the system fluctuates around its equilibrium state. New firms are introduced when profits are above normal, and are ultimately eliminated when losses persist. The varying number of firms represents a recurrent perturbation. The system thus exhibits dynamics at two levels: the dynamics of prices and output, and the dynamics of system size. The model aims to be realistic in its fundamental structure, but is kept simple in order to be computationally efficient. The ultimate aim is to use it as a platform for modeling the structural evolution of an economic system. Currently the model includes one form of structural evolution, the ability to generate new technologies and new products.

We propose a new surrogate-assisted evolutionary algorithm for expensive multiobjective optimization. Two classification-based surrogate models are used, which can predict the Pareto dominance relation and$\theta $-dominance relation between two solutions, respectively. To make such surrogates as accurate as possible, we formulate dominance prediction as an imbalanced classification problem and address this problem using deep learning techniques. Furthermore, to integrate the surrogates based on dominance prediction with multiobjective evolutionary optimization, we develop a two-stage preselection strategy. This strategy aims to select a promising solution to be evaluated among those produced by genetic operations, taking proper account of the balance between convergence and diversity. We conduct an empirical study on a number of well-known multiobjective and many-objective benchmark problems, over a relatively small number of function evaluations. Our experimental results demonstrate the superiority of the proposed algorithm compared with several representative surrogate-assisted algorithms.

Neural architecture search (NAS) has emerged as a promising avenue for automatically designing task-specific neural networks. Existing NAS approaches require one complete search for each deployment specification of hardware or objective. This is a computationally impractical endeavor given the potentially large number of application scenarios. In this paper, we propose Neural Architecture Transfer (NAT) to overcome this limitation. NAT is designed to efficiently generate task-specific custom models that are competitive under multiple conflicting objectives. To realize this goal we learn task-specific supernets from which specialized subnets can be sampled without any additional training. The key to our approach is an integrated online transfer learning and many-objective evolutionary search procedure. A pre-trained supernet is iteratively adapted while simultaneously searching for task-specific subnets. We demonstrate the efficacy of NAT on 11 benchmark image classification tasks ranging from large-scale multi-class to small-scale fine-grained datasets. In all cases, including ImageNet, NATNets improve upon the state-of-the-art under mobile settings ( ≤ 600M Multiply-Adds). Surprisingly, small-scale fine-grained datasets benefit the most from NAT. At the same time, the architecture search and transfer is orders of magnitude more efficient than existing NAS methods. Overall, experimental evaluation indicates that, across diverse image classification tasks and computational objectives, NAT is an appreciably more effective alternative to conventional transfer learning of fine-tuning weights of an existing network architecture learned on standard datasets. Code is available at https://github.com/human-analysis/neural-architecture-transfer.

Convolutional neural networks (CNNs) are the backbones of deep learning paradigms for numerous vision tasks. Early advancements in CNN architectures are primarily driven by human expertise and by elaborate design processes. Recently, neural architecture search was proposed with the aim of automating the network design process and generating task-dependent architectures. While existing approaches have achieved competitive performance in image classification, they are not well suited to problems where the computational budget is limited for two reasons: 1) the obtained architectures are either solely optimized for classification performance, or only for one deployment scenario and 2) the search process requires vast computational resources in most approaches. To overcome these limitations, we propose an evolutionary algorithm for searching neural architectures under multiple objectives, such as classification performance and floating point operations (FLOPs). The proposed method addresses the first shortcoming by populating a set of architectures to approximate the entire Pareto frontier through genetic operations that recombine and modify architectural components progressively. Our approach improves computational efficiency by carefully down-scaling the architectures during the search as well as reinforcing the patterns commonly shared among past successful architectures through Bayesian model learning. The integration of these two main contributions allows an efficient design of architectures that are competitive and in most cases outperform both manually and automatically designed architectures on benchmark image classification datasets: CIFAR, ImageNet, and human chest X-ray. The flexibility provided from simultaneously obtaining multiple architecture choices for different compute requirements further differentiates our approach from other methods in the literature.

Modularity represents a recurring theme in the attempt to scale evolution to the design of complex systems. However, modularity rarely forms the central theme of an artificial approach to evolution. In this work, we report on progress with the recently proposed Tangled Program Graph (TPG) framework in which programs are modules. The combination of the TPG representation and its variation operators enable both teams of programs and graphs of teams of programs to appear in an emergent process. The original development of TPG was limited to tasks with, for the most part, complete information. This work details two recent approaches for scaling TPG to tasks that are dominated by partially observable sources of information using different formulations of indexed memory. One formulation emphasizes the incremental construction of memory, again as an emergent process, resulting in a distributed view of state. The second formulation assumes a single global instance of memory and develops it as a communication medium, thus a single global view of state. The resulting empirical evaluation demonstrates that TPG equipped with memory is able to solve multi-task recursive time-series forecasting problems and visual navigation tasks expressed in two levels of a commercial first-person shooter environment.

Tangled Program Graphs (TPG) is a framework for genetic programming which has shown promise in challenging reinforcement learning problems with discrete action spaces. The approach has recently been extended to incorporate temporal memory mechanisms that enable operation in environments with partial-observability at multiple timescales. Here we propose a highly-modular memory structure that manages temporal properties of a task and enables operation in problems with continuous action spaces. This significantly broadens the scope of real-world applications for TPGs, from continuous-action reinforcement learning to time series forecasting. We begin by testing the new algorithm on a suite of symbolic regression benchmarks. Next, we evaluate the method in 3 challenging time series forecasting problems. Results generally match the quality of state-of-the-art solutions in both domains. In the case of time series prediction, we show that temporal memory eliminates the need to pre-specify a fixed-size sliding window of previous values, or autoregressive state, which is used by all compared methods. This is significant because it implies that no prior model for a time series is necessary, and the forecaster may adapt more easily if the properties of a series change significantly over time.

Convolutional neural networks (CNNs) are the backbones of deep learning paradigms for numerous vision tasks. Early advancements in CNN architectures are primarily driven by human expertise and elaborate design. Recently, neural architecture search (NAS) was proposed with the aim of automating the network design process and generating task-dependent architectures. This paper introduces NSGA-Net -- an evolutionary search algorithm that explores a space of potential neural network architectures in three steps, namely, a population initialization step that is based on prior-knowledge from hand-crafted architectures, an exploration step comprising crossover and mutation of architectures, and finally an exploitation step that utilizes the hidden useful knowledge stored in the entire history of evaluated neural architectures in the form of a Bayesian Network. The integration of these components allows an efficient design of architectures that are competitive and in many cases outperform both manually and automatically designed architectures on CIFAR-10 classification task. The flexibility provided from simultaneously obtaining multiple architecture choices for different compute requirements further differentiates our approach from other methods in the literature.

Bug repair is a major component of software maintenance, which requires a huge amount of manpower. Evolutionary computation, particularly genetic programming (GP), is a class of promising techniques for automating this time-consuming and expensive process. Although recent research in evolutionary program repair has made significant progress, major challenges still remain. In this article, we propose ARJA-e, a new evolutionary repair system for Java code that aims to address challenges for the search space, search algorithm, and patch overfitting. To determine a search space that is more likely to contain correct patches, ARJA-e combines two sources of fix ingredients (i.e., the statement-level redundancy assumption and repair templates) with contextual analysis-based search space reduction, thereby leveraging their complementary strengths. To encode patches in GP more properly, ARJA-e unifies the edits at different granularities into statement-level edits and then uses a lower-granularity patch representation that is characterized by the decoupling of statements for replacement and statements for insertion. ARJA-e also uses a finer-grained fitness function that can make full use of semantic information contained in the test suite, which is expected to better guide the search of GP. To alleviate patch overfitting, ARJA-e further includes a postprocessing tool that can serve the purposes of overfit detection and patch ranking. We evaluate ARJA-e on 224 real Java bugs from Defects4J and compare it with the state-of-the-art repair techniques. The evaluation results show that ARJA-e can correctly fix 39 bugs in terms of the patches ranked first, achieving substantial performance improvements over the state of the art. In addition, we analyze the effect of the components of ARJA-e qualitatively and quantitatively to demonstrate their effectiveness and advantages.

Automated program repair is the problem of automatically fixing bugs in programs in order to significantly reduce the debugging costs and improve the software quality. To address this problem, test-suite based repair techniques regard a given test suite as an oracle and modify the input buggy program to make the entire test suite pass. GenProg is well recognized as a prominent repair approach of this kind, which uses genetic programming (GP) to rearrange the statements already extant in the buggy program. However, recent empirical studies show that the performance of GenProg is not fully satisfactory, particularly for Java. In this paper, we propose ARJA, a new GP based repair approach for automated repair of Java programs. To be specific, we present a novel lower-granularity patch representation that properly decouples the search subspaces of likely-buggy locations, operation types and potential fix ingredients, enabling GP to explore the search space more effectively. Based on this new representation, we formulate automated program repair as a multi-objective search problem and use NSGA-II to look for simpler repairs. To reduce the computational effort and search space, we introduce a test filtering procedure that can speed up the fitness evaluation of GP and three types of rules that can be applied to avoid unnecessary manipulations of the code. Moreover, we also propose a type matching strategy that can create new potential fix ingredients by exploiting the syntactic patterns of existing statements. We conduct a large-scale empirical evaluation of ARJA along with its variants on both seeded bugs and real-world bugs in comparison with several state-of-the-art repair approaches. Our results verify the effectiveness and efficiency of the search mechanisms employed in ARJA and also show its superiority over the other approaches. In particular, compared to jGenProg (an implementation of GenProg for Java), an ARJA version fully following the redundancy assumption can generate a test-suite adequate patch for more than twice the number of bugs (from 27 to 59), and a correct patch for nearly four times of the number (from 5 to 18), on 224 real-world bugs considered in Defects4J. Furthermore, ARJA is able to correctly fix several real multi-location bugs that are hard to be repaired by most of the existing repair approaches.

This paper introduces NSGA-Net --- an evolutionary approach for neural architecture search (NAS). NSGA-Net is designed with three goals in mind: (1) a procedure considering multiple and conflicting objectives, (2) an efficient procedure balancing exploration and exploitation of the space of potential neural network architectures, and (3) a procedure finding a diverse set of trade-off network architectures achieved in a single run. NSGA-Net is a population-based search algorithm that explores a space of potential neural network architectures in three steps, namely, a population initialization step that is based on prior-knowledge from hand-crafted architectures, an exploration step comprising crossover and mutation of architectures, and finally an exploitation step that utilizes the hidden useful knowledge stored in the entire history of evaluated neural architectures in the form of a Bayesian Network. Experimental results suggest that combining the dual objectives of minimizing an error metric and computational complexity, as measured by FLOPs, allows NSGA-Net to find competitive neural architectures. Moreover, NSGA-Net achieves error rate on the CIFAR-10 dataset on par with other state-of-the-art NAS methods while using orders of magnitude less computational resources. These results are encouraging and shows the promise to further use of EC methods in various deep-learning paradigms.

Lexicase selection achieves very good solution quality by introducing ordered test cases. However, the computational complexity of lexicase selection can prohibit its use in many applications. In this paper, we introduce Batch Tournament Selection (BTS), a hybrid of tournament and lexicase selection which is approximately one order of magnitude faster than lexicase selection while achieving a competitive quality of solutions. Tests on a number of regression datasets show that BTS compares well with lexicase selection in terms of mean absolute error while having a speed-up of up to 25 times. Surprisingly, BTS and lexicase selection have almost no difference in both diversity and performance. This reveals that batches and ordered test cases are completely different mechanisms which share the same general principle fostering the specialization of individuals. This work introduces an efficient algorithm that sheds light onto the main principles behind the success of lexicase, potentially opening up a new range of possibilities for algorithms to come.

This paper presents an automatic software repair system that combines the characteristic components of several typical evolutionary computation based repair approaches into a unified repair framework so as to take advantage of their respective component strengths. We exploit both the redundancy assumption and repair templates to create a search space of candidate repairs. Then we employ a multi-objective evolutionary algorithm with a low-granularity patch representation to explore this search space, in order to find simple patches. In order to further reduce the search space and alleviate patch overfitting we introduce replacement similarity and insertion relevance to select more related statements as promising fix ingredients, and we adopt anti-patterns to customize the available operation types for each likely-buggy statement. We evaluate our system on 224 real bugs from the Defects4J dataset in comparison with the state-of-the-art repair approaches. The evaluation results show that the proposed system can fix 111 out of those 224 bugs in terms of passing all test cases, achieving substantial performance improvements over the state-of-the-art. Additionally, we demonstrate the ability of ARJA-e to fix multi-location bugs that are unlikely to be addressed by most of existing repair approaches.

We present AutoMap, a pair of methods for automatic generation of evolvable genotype-phenotype mappings. Both use an artificial neural network autoencoder trained on phenotypes harvested from fitness peaks as the basis for a genotype-phenotype mapping. In the first, the decoder segment of a bottlenecked autoencoder serves as the genotype-phenotype mapping. In the second, a denoising autoencoder serves as the genotype-phenotype mapping. Automatic generation of evolvable genotype-phenotype mappings are demonstrated on the n-legged table problem, a toy problem that defines a simple rugged fitness landscape, and the Scrabble string problem, a more complicated problem that serves as a rough model for linear genetic programming. For both problems, the automatically generated genotype-phenotype mappings are found to enhance evolvability.

In nature, gene regulatory networks are a key mediator between the information stored in the DNA of living organisms (their genotype) and the structural and behavioral expression this finds in their bodies, surviving in the world (their phenotype). They integrate environmental signals, steer development, buffer stochasticity, and allow evolution to proceed. In engineering, modeling and implementations of artificial gene regulatory networks have been an expanding field of research and development over the past few decades. This review discusses the concept of gene regulation, describes the current state of the art in gene regulatory networks, including modeling and simulation, and reviews their use in artificial evolutionary settings. We provide evidence for the benefits of this concept in natural and the engineering domains.

In this contribution we propose a hybrid genetic programming approach for evolving a decision making system in the domain of RoboCup Soccer (Simulation League). Genetic programming has been rarely used in this domain in the past, due to the difficulties and restrictions of the soccer simulation. The real-time requirements of robot soccer and the lengthy evaluation time even for simulated games provide a formidable obstacle to the application of evolutionary approaches. Our new method uses two evolutionary phases, each of which compensating for restrictions and limitations of the other. The first phase produces some evolved GP individuals applying an off-game evaluation system which can be trained on snapshots of game situations as they actually happened in earlier games, and corresponding decisions tagged as correct or wrong. The second phase uses the best individuals of the first phase as input to run another GP system to evolve players in a real game environment where the quality of decisions is evaluated through winning or losing during real-time runs of the simulator. We benchmark the new system against a baseline system used by most simulation league teams, as well as against winning systems of the 2016 tournament.

An important challenge for traffic signal control is adapting to irregular changes in traffic. In recent years, different heuristics have been developed to address this issue. However, most of them are tested in artificial scenarios under controlled circumstances. In this paper, we present the first implementation of Genetic Programming in the evolution of traffic signal controllers for a real-world scenario. The evolved controllers are compared with a static control and an actuated control. The results indicate a significant improvement over traditional methods. Moreover, additional experiments indicate that the evolved controllers have the ability to adapt to unplanned changes in traffic conditions.

In an evolutionary system, robustness describes the resilience to mutational and environmental changes, whereas evolvability captures the capability of generating novel and adaptive phenotypes. The research literature has not seen an effective quantification of phenotypic evolvability able to predict the evolutionary potential of the search for novel phenotypes. In this study, we propose to characterize the mutational potential among different phenotypes using the phenotype network, where vertices are phenotypes and edges represent mutational connections between them. In the framework of such a network, we quantitatively analyze the evolvability of phenotypes by exploring a number of vertex centrality measures commonly used in complex networks. In our simulation studies we use a Linear Genetic Programming system and a population of random walkers. Our results suggest that the weighted eigenvector centrality serves as the best estimator of phenotypic evolvability.

Searching the k-nearest matching patches for each patch in an input image, i.e., computing the k-nearest-neighbor fields ($k$-NNF), is a core part of various computer vision/graphics algorithms. In this paper, we show that $k$-NNF can be efficiently computed using a novel artificial multi-bee-colony (AMBC) algorithm, where each patch uses a dedicated bee colony to search for its k-nearest matches. As a population-based algorithm, AMBC is capable of escaping local optima. The added communication among different colonies further allows good matches to be quickly propagated across the image. In addition, AMBC makes no assumption about the neighborhood structure or communication direction, making it directly applicable to image sets and suitable for parallel processing. Quantitative evaluations show that AMBC can find solutions that are much closer to the ground truth than the generalized PatchMatch algorithm does. It also outperforms the PatchMatch Graph over image sets.

We describe the content and outcomes of the First Workshop on Open-Ended Evolution: Recent Progress and Future Milestones (OEE1), held during the ECAL 2015 conference at the University of York, UK, in July 2015. We briefly summarize the content of the workshop's talks, and identify the main themes that emerged from the open discussions. Two important conclusions from the discussions are: (1) the idea of pluralism about OEE-it seems clear that there is more than one interesting and important kind of OEE; and (2) the importance of distinguishing observable behavioral hallmarks of systems undergoing OEE from hypothesized underlying mechanisms that explain why a system exhibits those hallmarks. We summarize the different hallmarks and mechanisms discussed during the workshop, and list the specific systems that were highlighted with respect to particular hallmarks and mechanisms. We conclude by identifying some of the most important open research questions about OEE that are apparent in light of the discussions. The York workshop provides a foundation for a follow-up OEE2 workshop taking place at the ALIFE XV conference in Cancún, Mexico, in July 2016. Additional materials from the York workshop, including talk abstracts, presentation slides, and videos of each talk, are available at http://alife.org/ws/oee1 .

Recombination is a commonly used genetic operator in artificial and computational evolutionary systems. It has been empirically shown to be essential for evolutionary processes. However, little has been done to analyze the effects of recombination on quantitative genotypic and phenotypic properties. The majority of studies only consider mutation, mainly due to the more serious consequences of recombination in reorganizing entire genomes. Here we adopt methods from evolutionary biology to analyze a simple, yet representative, genetic programming method, linear genetic programming. We demonstrate that recombination has less disruptive effects on phenotype than mutation, that it accelerates novel phenotypic exploration, and that it particularly promotes robust phenotypes and evolves genotypic robustness and synergistic epistasis. Our results corroborate an explanation for the prevalence of recombination in complex living organisms, and helps elucidate a better understanding of the evolutionary mechanisms involved in the design of complex artificial evolutionary systems and intelligent algorithms.

We propose an evolutionary feature creator (EFC) to explore a non-linear and offline method for generating features in image recognition tasks. Our model aims at extracting low-level features automatically when provided with an arbitrary image database. In this work, we are concerned with the addition of algorithmic depth to a genetic programming (GP) system, hypothesizing that it will improve the capacity for solving problems that require high-level, hierarchical reasoning. For this we introduce a network superstructure that co-evolves with our low-level GP representations. Two approaches are described: the first uses our previously used "shallow" GP system, the second presents a new "deep" GP system that involves this network superstructure. We evaluate these models against a benchmark object recognition database. Results show that the deep structure outperforms the shallow one in generating features that support classification, and does so without requiring significant additional computational time. Further, high accuracy is achieved on the standard ETH-80 classification task, also outperforming many existing specialized techniques. We conclude that our EFC is capable of data-driven extraction of useful features from an object recognition database.

This work examines a novel method that provides a parallel search of a very large network space consisting of fisheries management data. The parallel search solution is capable of determining global maxima of the search space using exhaustive search, compared to local optima located by machine learning solutions such as evolutionary computation. The actual solutions from the best machine learning technique, called Probabilistic Adaptive Mapping Developmental Genetic Algorithm, are compared by a fisheries expert to the global maxima solutions returned by parallel search. The time required for parallel search, for both CPU and GPU-based solutions, are compared to those required for machine learning solutions. The GPU parallel computing solution was found to have a speedup of 12x over a multi-threaded CPU solution. An expert found that overall the machine learning solutions produced more interesting results by locating local optima than global optima determined by parallel processing.

A number of researchers who apply genetic programming (GP) to the analysis of financial data have had success in using predictability pretests to determine whether the time series under analysis by a GP contains patterns that are actually inherently predictable. However, most studies to date apply no such pretests, or pretests of any kind. Most previous work in this area has attempted to use filters to ensure inherent predictability of the data within a window of a time series, whereas other works have used multiple time frame windows under analysis by the GP to provide one overall GP recommendation. This work, for the first time, analyzes the use of external information about the price trend of a stock's market sector. This information is used in a filter to bolster confidence of a GP-based alert regarding formation of a trend for the chosen stock. Our results indicate a significant improvement in trend identification for the majority of stocks analyzed using intraday data.

An extension to a non-linear offline method for generating features for image recognition is introduced. It aims at generating low-level features automatically when provided with some arbitrary image database. First, a general representation of prioritized pixel-neighbourhoods is described. Next, genetic programming is used to specify functions on those representations. The result is a set of transformations on the space of grayscale images. These transforms are utilized as a step in a classification process, and evolved in an evolutionary algorithm. The technique is shown to match the efficiency of the state-of-the-art on a medical image classification task. Further, the approach is shown to self-select an appropriate solution structure and complexity. Finally, we show that competitive co-evolution is a viable means of combating over-fitting. It is concluded that the technique generally shows good promise for the creation of novel image features in situations where pixel-level features are complex or unknown, such as medical images.

Self Modifying Cartesian Genetic Programming is a general purpose, graph-based, developmental form of Cartesian Genetic Programming. Using a combination of computational functions and special functions that can modify the phenotype at runtime, it has been employed to find general solutions to certain Boolean circuits and mathematical problems. In the present work, a new version, of SMCGP is proposed and demonstrated. Compared to the original SMCGP both the representation and the function set have been simplified. However, the new representation is also two-dimensional and it allows evolution and development to have more ways to solve a given problem. Under most situations we show that the new method makes the evolution of solutions to even parity and binary addition faster than with previous version of SMCGP.

Traditionally, a genetic algorithm is used to analyze networks by maximizing the modularity (Q) measure to create a favorable community. A coevolutionary algorithm is used here to not only find the appropriate community division for a network, but to find interesting networks containing substantial changes in data within a very large network space. The network is one of the largest, if not the largest, analyzed by evolutionary computation techniques to date and is created using a real world data set consisting of fisheries catch data in the north Atlantic Ocean off the coast of Canada. This work examines the quantitative performance of two types of coevolutionary algorithms against both a standard GA that uses a natural (but not necessarily optimal) division of the data set into communities, and simulated annealing. The goal for all search algorithms was to automatically find anomalies (differences in catch) within the data. To measure practical usefulness of the system, a fisheries expert analyzed the best networks located by the search algorithms using an existing visualization software prototype. The expert indicated that a refined version of coevolutionary GA known as PAMDGA was found to most reliably locate subnetworks containing catch differences of biological relevance.

A number of researchers have attempted to take successful GP trading systems and make them even better through the use of filters. We investigate the use of a linear genetic programming (LGP) system that combines GP signals provided over multiple intraday time frames to produce one trading action. Four combinations of time frames stretching further into the past are examined. Two different decision mechanisms for evaluating the overall signal given the GP signals over all time frames are also examined, one based on majority vote and another based on temporal proximity to the buying decision. Results indicated that majority vote outperformed emphasis on proximity of time frames to the current trading decision. Analyses also indicated that longer time frame combinations were more conservative and outperformed shorter combinations for both overall upward and downward price trends.

This paper aims to improve the capability of genetic programming to tackle the evolution of cooperation: evolving multiple partial solutions that collaboratively solve structurally and functionally complex problems. A multilevel genetic programming approach is presented based on a new computational multilevel selection framework [19]. This approach considers biological group selection theory to encourage cooperation, and a new cooperation operator to build solutions hierarchically. It extends evolution from individuals to multiple group levels, leading to good performance on both individuals and groups. The applicability of this approach is evaluated on 7 multi-class classification problems with different features, such as non-linearity, skewed data distribution and large feature space. The results, when compared to other cooperative evolutionary algorithms in the literature, demonstrate that this approach improves solution accuracy and consistency, and simplifies solution complexity. In addition, the problem is decomposed as a result of evolution without human interference.

A handful of researchers who apply genetic programming (GP) to the analysis of financial markets have devised predictability pretests to determine whether the time series that is being supplied to GP contains patterns that can be predicted, but most studies apply no such pretests. By applying predictability pretests, researchers can have greater confidence that the GP system is solving a problem which is actually there and that it will be less likely to make questionable investment decisions based on non-existent patterns. Previous work in this area has applied regression to randomized versions of time series training data to create a functional model that is applied over a future window of time. This work presents two types of predictability filters with low computational overhead, namely frequency-based and information theoretic, that complement the previous function-based continuous output predictability models. Results indicate that either filter can be beneficial for particular trend types, but the information-based filter involves a greater chance of missing opportunities for profit. In contrast, the frequency-based filter always outperforms, or is competitive with, the filterless implementation.

Self Modifying Cartesian Genetic Programming (SMCGP) aims to be a general purpose form of developmental genetic programming. The evolved programs are iterated thus allowing an infinite sequence of phenotypes (programs) to be obtained from a single evolved genotype. In previous work this approach has already shown that it is possible to obtain mathematically provable general solutions to certain problems. We extend this class in this paper by showing how SMCGP can be used to find algorithms that converge to mathematical constants (pi and e). Mathematical proofs are given that show that some evolved formulae converge to pi and e in the limit as the number of iterations increase.

Foreign exchange (forex) market trading using evolutionary algorithms is an active and controversial area of research. We investigate the use of a linear genetic programming (LGP) system for automated forex trading of four major currency pairs. Fitness functions with varying degrees of conservatism through the incorporation of maximum drawdown are considered. The use of the fitness types in the LGP system for different currency value trends are examined in terms of performance over time, underlying trading strategies, and overall profitability. An analysis of trade profitability shows that the LGP system is very accurate at both buying to achieve profit and selling to prevent loss, with moderate levels of trading activity.

To successfully search multiple coadaptive subcomponents in a solution, we developed a novel cooperative evolutionary algorithm based on a new computational multilevel selection framework. This algorithm constructs cooperative solutions hierarchically by implementing the idea of group selection. We show that this simple and straightforward algorithm is able to accelerate evolutionary speed and improve solution accuracy on string covering problems as compared to other EAs used in literature. In addition, the structure of the solution and the roles played by each subcomponent in the solution emerge as a result of evolution without human interference.

A genetic algorithm is combined with two variants of the modularity (Q) network analysis metric to examine a substantial amount fisheries catch data. The data set produces one of the largest networks evaluated to date by genetic algorithms applied to network community analysis. Rather than using GA to decide community structure that simply maximizes modularity of a network, as is typical, we use two fuzzy community membership functions applied to natural temporal divisions in the network so the GA is used to find interesting areas of the search space through maximization of modularity. The work examines the performance of the genetic algorithm against simulated annealing using both types of fuzzy community membership functions. The algorithms are used in an existing visualization software prototype, where the solutions are evaluated by a fisheries expert.

Self modifying CGP (SMCGP) is a developmental form of Cartesian genetic programming(CGP). It differs from CGP by including primitive functions which modify the program. Beginning with the evolved genotype the self-modifying functions produce a new program (phenotype) at each iteration. In this paper we have applied it to a well known digital circuit building problem: even-parity. We show that it is easier to solve difficult parity problems with SMCGP than either with CGP or modular CGP, and that the increase in efficiency grows with problem size. More importantly, we prove that SMCGP can evolve general solutions to arbitrary-sized even parity problems.

In biology, the importance of environmental feedback to the process of embryogenesis is well understood. In this paper we explore the introduction of a local fitness to an artificial developmental system, providing an artificial analogue to the natural phenomenon. First, we define a highly simplified model of vasculogenesis, an environment-based toy problem in which we can evaluate our strategies. Since the use of a global fitness function for local feedback is likely too computationally expensive, we introduce the notion of a neighbourhood-based ldquolocal fitnessrdquo function. This local fitness serves as an environmental-feedback guide for the developmental system. The result is a developmental analogue of guided hill-climbing, one which significantly improves the performance of an artificial embryogeny in the evolution of a simplified vascular system. We further evaluate our model in a collection of randomly generated two-dimensional geometries, and show that inclusion of local fitness helps allay some of the problem difficulty in irregular environments. In the process, we also introduce a novel and systematic means of generating bounded, connected two-dimensional geometries.

In this paper, we use Genetic Programming (GP) to define a set of transforms on the space of greyscale images. The motivation is to allow an evolutionary algorithm means of transforming a set of image patterns into a more classifiable form. To this end, we introduce the notion of a transform-based evolvable feature (TEF), a moment value extracted from a GP-transformed image, used in a classification task. Unlike many previous approaches, the TEF allows the whole image space to be searched and augmented. TEFs are instantiated through Cartesian Genetic Programming, and applied to a medical image classification task, that of detecting muscular dystrophy-indicating inclusions in cell images. It is shown that the inclusion of a single TEF allows for significantly superior classification relative to predefined features alone.

During the legal investigation of Enron Corporation, the U.S. Federal Regulatory Commission (FERC) made public a substantial data set of the company's internal corporate emails. This work presents a genetic algorithm (GA) approach to social network analysis (SNA) using the Enron corpus. Three SNA metrics, degree, density, and proximity prestige, were applied to the detection of networks with high email activity and presence of important actors with respect to email transactions. Quantitative analysis revealed that density and proximity prestige captured networks of high activity more so than degree. Subsequent qualitative analysis indicated that there were trade-offs in the selection of SNA metrics. Examination of the discovered social networks showed that density and proximity prestige isolated networks involving key actors to a greater extent than degree. In particular, density picked out interesting patterns in terms of email volume, while proximity prestige better isolated key actors at Enron. The roles of the particular actors picked out by the networks as reasons for their prominence are also discussed.

Self-Modifying Cartesian Genetic Programming (SMCGP) is a form of genetic programming that integrates developmental (self-modifying) features as a genotype-phenotype mapping. This paper asks: Is it possible to evolve a learning algorithm using SMCGP?

The notion of evolvability has been put forward to describe the "core mechanism" of natural and artificial evolution. Recently, studies have revealed the influence of the environment upon a system's evolvability. In this contribution, we study the evolvability of a system in various environmental situations. We consider neutrality and variability as two sides of evolvability. The former makes a system tolerant to mutations and provides a hidden staging ground for future phenotypic changes. The latter produces explorative variations yielding phenotypic improvements. Which of the two dominates is influenced by the environment. We adopt two tools for this study of evolvability: 1) the rate of adaptive evolution, which captures the observable adaptive variations driven by evolvability; and 2) the variability of individuals, which measures the potential of an individual to vary functionally. We apply these tools to a Linear Genetic Programming system and observe that evolvability is able to exploit its two sides in different environmental situations.

Efficient and effective deployment of IEEE 802.16 networks to service an area of users with certain traffic demands is an important network planning problem. We resort to an evolutionary approach in order to yield good approximation solutions. In our method, novel genetic variation operations are proposed to incorporate the feature of this real-world application of evolutionary algorithm.

Recently, a form of memory usage was introduced for genetic programming (GP) called "soft memory." Rather than have a new value completely overwrite the old value in a register, soft memory combines the new and old register values. This work examines the performance of a soft memory linear GP and developmental GP implementation for stock trading. Soft memory is known to more slowly adapt solutions compared to traditional GP. Thus, it was expected to perform well on stock data which typically exhibit local turbulence in combination with an overall longer term trend. While soft memory and standard memory were both found to provide similar impressive accuracy in buys that produced profit and sells that prevented losses, the softer memory settings traded more actively. The trading of the softer memory systems produced less substantial cumulative gains than traditional memory settings for the stocks tested with climbing share price trends. However, the trading activity of the softer memory settings had moderate benefits in terms of cumulative profit compared to buy-and-hold strategy for share price trends involving a drop in prices followed later by gains.

We describe how to harness the graphics processing abilities of a consumer video game console (Xbox 360) for general programming on graphics processing unit (GPGPU) purposes. In particular, we implement a linear GP (LGP) system to solve classification and regression problems. We conduct inter- and intra-platform benchmarking of the Xbox 360 and PC, using GPU and CPU implementations on both architectures. Platform benchmarking confirms highly integrated CPU and GPU programming flexibility of the Xbox 360, having the potential to alleviate typical GPGPU decisions of allocating particular functionalities to CPU or GPU.

We define the rate of evolution Re in a GP system based on the rate of efficient genetic variations being accepted. This definition is motivated by the measurement of amino acid to synonymous substitution ratio ka/ks in biology. Experimental applications of this rate of evolution measurement show that Re well reflects how evolution proceeds underneath fitness development and quantifies the rate of innovation through efficient genetic variations.

A major difficulty for anomaly detection lies in discovering boundaries between normal and anomalous behavior, due to the deficiency of abnormal samples in the training phase. In this paper, a novel coevolutionary algorithm which attempts to simulate territory establishment in ecology is conceived to tackle anomaly detection problems. Two species in normal and abnormal behavior pattern space coevolve competitively and cooperatively. Competition prevents individuals in one species from invading the other's territory; cooperation aims to achieve complete pattern coverage by adjusting the evolutionary environment according to the pressure coming from neighbors. In a sense, we extend the definition of cooperative coevolution from "coupled fitness" to "interaction of the evolutionary environment". This coevolutionary algorithm, enhanced with features like niching inside of species, global and local fitness, and fuzzy sets, tries to balance overfitting and overgeneralization. This provides an accurate boundary definition. Experimental results on transactional data from a real financial institution show that this coevolutionary algorithm is more effective than the evolutionary algorithm in evolving normal or abnormal behavior patterns only.

In nature, systems with enormous numbers of components (i.e. cells) are evolved from a relatively small genotype. It has not yet been demonstrated that artificial evolution is sufficient to make such a system evolvable. Consequently researchers have been investigating forms of computational development that may allow more evolvable systems. The approaches taken have largely used re-writing, multi- cellularity, or genetic regulation. In many cases it has been difficult to produce general purpose computation from such systems.In this paper we introduce computational development using a form of Cartesian Genetic Programming that includes self-modification operations. One advantage of this approach is that ab initio the system can be used to solve computational problems. We present results on a number of problems and demonstrate the characteristics and advantages that self-modification brings.

In this paper we describe the genetic programming system GGP operating on graphs and introduce the notion of graph isomorphisms to explain how they influence the dynamics of GP. It is shown empirically how fitness databases can improve the performance of GP and how mapping graphs to a canonical form can increase these improvements by saving considerable evaluation time.

Algorithmic Chemistries are Artificial Chemistries that aim at algorithms. In this contribution we present a new algorithm to execute Algorithmic Chemistries during evolution. This algorithm ensures synthesizes of the whole program and cuts off execution of unneeded instructions without restricting the stochastic way of execution. We demonstrate benefits of the new algorithm for evolution of Algorithmic Chemistries and discuss the relation of Algorithmic Chemistries with Estimation of Distribution Algorithms.

A large training set of fitness cases can critically slow down genetic programming, if no appropriate subset selection method is applied. Such a method allows an individual to be evaluated on a smaller subset of fitness cases. In this paper we suggest a new subset selection method that takes the problem structure into account, while being problem independent at the same time. In order to achieve this, information about the problem structure is acquired during evolutionary search by creating a topology (relationship) on the set of fitness cases. The topology is induced by individuals of the evolving population. This is done by increasing the strength of the relation between two fitness cases, if an individual of the population is able to solve both of them. Our new topology-based subset selection method chooses a subset, such that fitness cases in this subset are as distantly related as is possible with respect to the induced topology. We compare topology-based selection of fitness cases with dynamic subset selection and stochastic subset sampling on four different problems. On average, runs with topology-based selection show faster progress than the others.

This work is a first study of search spaces and fitness landscapes in the context of quantum program evolution. Considering small instances of the Deutsch-Josza problem as a staring point for explorations of quantum program search spaces, we analyze the structure of mutation landscapes using autocorrelation characteristics and information measures. Our motivation is to obtain insights into the relationship between landscape characteristic and quantum circuit evolution with the aim to improve the efficiency of evolutionary search.

This article reviews the growing body of scientific work in artificial chemistry. First, common motivations and fundamental concepts are introduced. Second, current research activities are discussed along three application dimensions: modeling, information processing, and optimization. Finally, common phenomena among the different systems are summarized. It is argued here that artificial chemistries are "the right stuff" for the study of prebiotic and biochemical evolution, and they provide a productive framework for questions regarding the origin and evolution of organizations in general. Furthermore, artificial chemistries have a broad application range of practical problems, as shown in this review.

This article demonstrates a new method of programming artificial chemistries. It uses the emerging capabilities of the system's dynamics for information-processing purposes. By evolution of metabolisms that act as control programs for a small robot one achieves the adaptation of the internal metabolic pathways as well as the selection of the most relevant available exteroceptors. The underlying artificial chemistry evolves efficient information-processing pathways with most benefit for the desired task, robot navigation. The results show certain relations to such biological systems as motile bacteria.

We introduce a new form of linear genetic programming (GP). Two methods of acceleration of our GP approach are discussed: 1) an efficient algorithm that eliminates intron code and 2) a demetic approach to virtually parallelize the system on a single processor. Acceleration of runtime is especially important when operating with complex data sets, because they are occurring in real-world applications. We compare GP performance on medical classification problems from a benchmark database with results obtained by neural networks. Our results show that GP performs comparably in classification and generalization.

We analyze different levels of meta-evolution using a graph based GP system. The system allows one to represent individuals of the search space and genetic variation operators in a coherent way as graph programs differing only in the operator set. Seven variants of meta-evolution are tested on three real world classification problems. The most complex variant consists of three meta-levels where graph programs on meta-level 1 recombine individuals of the search space (base level), graph programs on meta-level 2 recombine programs on meta-level 1, and programs on meta-level 3 recombine programs on meta-level 2 and themselves. The empirical results shows that the use of meta levels is advantageous.

Many machine learning tasks are just too hard to be solved with a single processor machine, no matter how efficient the algorithms are and how fast our hardware is. Luckily genetic programming is well suited for parallelization compared to standard serial algorithms. The paper describes the first parallel implementation of an AIM-GP system, creating the potential for an extremely fast system. The system is tested on three problems and several variants of demes and migration are evaluated. Most of the results are applicable to both linear and tree based systems.

info:eu-repo/semantics/published

We examine the qualitative dynamics of a catalytic self-organizing system of binary strings that is inspired by the chemical information processing metaphor. A string is interpreted in two different ways: either (a) as raw data or (b) as a machine that is able to process another string as data in order to produce a third one. This article focuses on the phenomena of evolution whose appearance is notable because no explicit mutation, recombination, or artificial selection operators are introduced. We call the system self-evolving because every variation is performed by the objects themselves in their machine form.

We discuss a family of competitive dynamics useful for pattern recognition purposes. Derived from a physical model of mode competition, they generalize former concepts to include populations of cells working as grandmother cell assemblies. Also the notion of unfair competition is introduced.