Exposing an evolutionary algorithm that is used to evolve robot controllers to variable conditions is necessary to obtain solutions which are robust and can cross the reality gap. However, we do not yet have methods for analyzing and understanding the impact of the varying morphological conditions which impact the evolutionary process, and therefore for choosing suitable variation ranges. By morphological conditions, we refer to the starting state of the robot, and to variations in its sensor readings during operation due to noise. In this paper, we introduce a method that permits us to measure the impact of these morphological variations and we analyze the relation between the amplitude of variations, the modality with which they are introduced, and the performance and robustness of evolving agents. Our results demonstrate that (i) the evolutionary algorithm can tolerate morphological variations which have a very high impact, (ii) variations affecting the actions of the agent are tolerated much better than variations affecting the initial state of the agent or of the environment, and (iii) improving the accuracy of the fitness measure through multiple evaluations is not always useful. Moreover, our results show that morphological variations permit generating solutions which perform better both in varying and non-varying conditions.

The possibility of using competitive evolutionary algorithms to generate long-term progress is normally prevented by the convergence on limit cycle dynamics in which the evolving agents keep progressing against their current competitors by periodically rediscovering solutions adopted previously. This leads to local but not to global progress (i.e., progress against all possible competitors). We propose a new competitive algorithm that produces long-term global progress by identifying and filtering out opportunistic variations, that is, variations leading to progress against current competitors and retrogression against other competitors. The efficacy of the method is validated on the coevolution of predator and prey robots, a classic problem that has been used in related researches. The accumulation of global progress over many generations leads to effective solutions that involve the production of articulated behaviors. The complexity of the behavior displayed by the evolving robots increases across generations, although progress in performance is not always accompanied by behavior complexification.

We demonstrate how evaluating candidate solutions in a limited number of stochastically varying conditions that vary over generations at a moderate rate is an effective method for developing high quality robust solutions. Indeed, agents evolved with this method for the ability to solve an extended version of the double-pole balancing problem, in which the initial state of the agents and the characteristics of the environment in which the agents are situated vary, show the ability to solve the problem in a wide variety of environmental circumstances and for prolonged periods of time without the need to readapt. The combinatorial explosion of possible environmental conditions does not prevent the evolution of robust solutions. Indeed, exposing evolving agents to a limited number of different environmental conditions that vary over generations is sufficient and leads to better results with respect to control experiments in which the number of experienced environmental conditions is greater. Interestingly the exposure to environmental variations promotes the evolution of convergent strategies in which the agents act so to exhibit the required functionality and so to reduce the complexity of the control problem.

Previous evolutionary studies demonstrated how robust solutions can be obtained by evaluating agents multiple times in variable environmental conditions. Here we demonstrate how agents evolved in environments that vary across generations outperform agents evolved in environments that remain fixed. Moreover, we demonstrate that best performance is obtained when the environment varies at a moderate rate across generations, that is, when the environment does not vary every generation but every N generations. The advantage of exposing evolving agents to environments that vary across generations at a moderate rate is due, at least in part, to the fact that this condition maximizes the retention of changes that alter the behavior of the agents, which in turn facilitates the discovery of better solutions. Finally, we demonstrate that moderate environmental variations are advantageous also from an evolutionary computation perspective, that is, from the perspective of maximizing the performance that can be achieved within a limited computational budget.

In this paper we show how evolving robots can develop behaviors displaying a modular organization characterized by semi-discrete and semi-dissociable sub-behavioral units playing different functions. In our experiments, the development of differentiated behaviors is not realized through the sub-division of the control system into modules and/or through the utilization of differentiated training processes. Instead, it simply originates as a consequence of the adaptive advantage provided by the possibility to display and use functionally specialized behaviors. These are selected by evolution not only with respect to their capability to perform a given sub-function but also with respect to the capability to support smooth and effective transition with other behaviors. This is achieved by having different co- adapted behaviors and by evaluating the variation affecting the behaviors on the basis of the impact they have on the overall performance of the robots. Moreover this process enables the development of t...

Coevolving systems are notoriously difficult to understand. This is largely due to the Red Queen effect that dictates heterospecific fitness interdependence. In simulation studies of coevolving systems, master tournaments are often used to obtain more informed fitness measures by testing evolved individuals against past and future opponents. However, such tournaments still contain certain ambiguities. We introduce the use of a phenotypic cluster analysis to examine the distribution of opponent categories throughout an evolutionary sequence. This analysis, adopted from widespread usage in the bioinformatics community, can be applied to master tournament data. This allows us to construct behavior-based category trees, obtaining a hierarchical classification of phenotypes that are suspected to interleave during cyclic evolution. We use the cluster data to establish the existence of switching-genes that control opponent specialization, suggesting the retention of dormant genetic adaptations, that is, genetic memory. Our overarching goal is to reiterate how computer simulations may have importance to the broader understanding of evolutionary dynamics in general. We emphasize a further shift from a component-driven to an interaction-driven perspective in understanding coevolving systems. As yet, it is unclear how the sudden development of switching-genes relates to the gradual emergence of genetic adaptability. Likely, context genes gradually provide the appropriate genetic environment wherein the switching-gene effect can be exploited.

Artificial Life (hereafter ALife) is an interdisciplinary research field that has the purpose of gaining a better understanding of biological life by artificially synthesizing simple and novel forms of life, as well as reproducing lifelike properties of living systems. Synthesis of artificial cells, simulation of large-scale biological networks, intelligent use of exponentially growing amounts of biochemical data, exploitation of biological substrates for computation and control, and deployment of bio-inspired engineering are just some of the main issues of ALife and nowadays are cutting-edge topics.ALife is at the intersection between a theoretical perspective, namely, the scientific explanations of different levels of life organization (e.g., molecules, compartments, cells, tissues, organs, organisms, societies, and collective and social phenomena) and advanced technological applications (bio-inspired algorithms and techniques for building up effective solutions such as in the fields of robotics, big data analysis, and computational medicine).This special issue presents the top six articles—revised and extended—that were presented at ECAL 2013, the twelfth European Conference on Artificial Life, which was held in Taormina, Sicily, Italy, September 2–6, 2013 (http://www.dmi.unict.it/ecal2013/). ECAL 2013 was a grand scientific celebration with hundreds of paper and poster presentations (for a total of 267 submissions, 10 workshops, 16 tracks, and more than 350 attendants) by leading scientists in the field, which have described an impressive array of results, ideas, technologies, and applications, thus showing the current state of the art of ALife. The aim of this special issue is to give an incentive for boosting interest and new ideas in designing life and lifelike processes at different levels of complexity.After a careful peer-review process, only the top six research manuscripts, extended and revised, of ECAL 2013 have been selected for inclusion in this special issue.In “Multi-crease Self-folding by Global Heating,” by Shuhei Miyashita, Cagdas Onal, and Daniela Rus, the authors present a scheme for the autonomous folding of the body of a 3D robot from a 2D sheet using heat. They developed an easy, fast, and reliable fabrication technique for constructing a self-folding sheet. In particular, they exploit the thermal deformation of a constructive sheet sandwich made of rigid structural layers, and they utilize the V-fold method to attain the targeted folding angles. The authors also developed a mobile robot that uses such a self-folding body and that performs locomotion by using two vibration motors. The locomotion of the robot proves the functionality of the self-folding origami structure.In “The Search for Candidate Relevant Subsets of Variables in Complex Systems,” by M. Villani, A. Roli, A. Filisetti, M. Fiorucci, I. Poli, and R. Serra, the authors present a method to identify relevant subsets of variables for the comprehension of the organization of a dynamical system. Thus, the dynamical cluster index (DCI)—an information-theoretic measure—is introduced; the DCI relies on observations of current values of the relationships among the system variables, and does not require any previous knowledge. The usefulness of this method was tested in different application domains, where a known dynamical model generates the data, and the aim of the DCI is to uncover significant aspects of the organization of the model. One of the main novelties of this work is in the use of truly dynamical systems.In “Indirectly Encoding Running and Jumping Sodarace Creatures for Artificial Life,” by Paul Szerlip and Kenneth O. Stanley, the authors present a platform for evolving two-dimensional artificial creatures, whose aim is to serve for future artificial life experiments in evolving creatures. They introduce a new, indirectly encoded Sodarace (IESoR) system, which extends the original Sodarace by enabling the evolution of significantly more complex and regular creature morphologies. The capability of the system has been proved in both walking and jumping domains, in which IESoR discovered a wide breadth of strategies through the novel approach of search with local competition.In “Experiments on and Numerical Modeling of the Capture and Concentration of Transcription-Translation Machinery inside Vesicles,” by Fabio Mavelli and Pasquale Stano, the authors present a mathematical model of the encapsulation of transcription-translation (TX-TL) coupled reactions based on a minimal protein synthesis model and on different solute partition functions. Such a model can be used to predict the time span and the amount of protein produced starting from any pure system composition. The proposed approach highlights the role of stochastic events in synthetic-cell research, and emphasizes the importance of the integration of stochastic simulations with experimental approaches. The results presented show clearly that experimental data are compatible with an entrapment model, which follows a power law rather than a Gaussian distribution.In “Lessons from Speciation Dynamics: How to Generate Selective Pressure Towards Diversity,” by Heiko Hamann, the author detects how methods for generating selective pressure towards diversity (SPTD) can be transferred from the domain of artificial ecology (AE) to evolutionary robotics (ER). Furthermore, he also investigates how SPTD is generated without task-specific behavioral features or other forms of a priori knowledge. A promising finding of this work is that selective pressure towards unpopulated regions of the search space is generated for both systems. In particular, a beneficial finding is the analogy between the behavioral distances in ER and self-organizing ecology, in terms of the selective pressure generated.In “Cell-Division Behavior in a Heterogeneous Swarm Environment,” by Adam Erskine and J. Michael Herrmann, the authors present a system of virtual particles that interact using simple kinetic rules. In particular, they present a two-species three-dimensional swarm in which the behavior that emerges resembles cell division. In addition, they have proved that behavior resembling repeated cell division emerges from the low-level kinetic interactions of a heterogeneous swarm. Such division behavior exists in a small but finite volume of the swarm chemistry parameter space. Furthermore, from the outcomes it emerges how the behaviors differ depending on whether the swarm is moving in two- or three-dimensional space.The guest editors would like to thank all the authors of this special issue for their excellent work, constituting a valuable contribution to the state of the art and the scientific growth of ALife. We would also like to thank everyone who helped to ensure high scientific quality both of ECAL 2013 and of this special issue: the plenary speakers, Prof. Paolo Arena, Prof. Roberto Cingolani, Prof. Roberto Cipolla, Prof. Martin Hanczyc, Prof. Henrik Hautop Lund, Prof. Didier Keymeulen, Prof. Steve Oliver, Prof. Bernhard Ø. Palsson, and Prof. Rolf Pfeifer; all technical program committee members; the workshop chair; the tutorial chair; the publicity chair; and the local organizers. Special thanks go to the editor-in-chief of Artificial Life, Prof. Mark Bedau, for encouraging and accepting this special issue, and to the editorial assistant, Linda Reedijk, for her useful and important support in the realization of this volume.

In this paper we illustrate how the capacity to select the most appropriate actions when handling contexts affording multiple conflicting actions can be solved either through a selective attention strategy (in which the stimuli affording alternative actions are filtered out at the perceptual level through top-down regulation) or at later processing stages through an action selection strategy (through the suppression of the premotor information eliciting alternative actions). By carrying out a series of experiments in which a neuro-robot develops an ability to choose between conflicting actions, we were able to identify the conditions that lead to the development of solutions based on one strategy or another. Overall, the results indicate that the selective attention strategy constitutes the most simple and straightforward mechanism enabling the acquisition of such capacities. Moreover, the characteristics of the adaptive/learning process influence whether the adaptive robot converges towards a selective attention and/or action selection strategy.

A highly efficient corner finding algorithm, named ACT-CORNER, is introduced. The algorithm uses active sub-sampling: it searches for corners by exploiting information from local image samples. ACT-CORNER is compared to both the Harris and FAST corner detectors. It is computationally much more efficient than the Harris corner detector. The comparison with FAST depends on the image size, with an advantage for ACT-CORNER for image dimensions of 320 × 240 pixels or larger. ACT-CORNER's performance is evaluated in the context of optic flow based Time-To-Contact (TTC) estimation. Image zoom experiments show that the accuracy of TTC estimations is similar when ACT-CORNER or FAST are used for corner detection. Finally, experiments with a Parrot AR drone show that the TTC estimates based on ACT-CORNER correspond well to sonar-based estimates.

Evolutionary robotics (ER) is a powerful approach for the automatic synthesis of robot controllers, as it requires little a priori knowledge about the problem to be solved in order to obtain good solutions. This is particularly true for collective and swarm robotics, in which the desired behavior of the group is an indirect result of the control and communication rules followed by each individual. However, the experimenter must make several arbitrary choices in setting up the evolutionary process, in order to define the correct selective pressures that can lead to the desired results. In some cases, only a deep understanding of the obtained results can point to the critical aspects that constrain the system, which can be later modified in order to re-engineer the evolutionary process towards better solutions. In this article, we discuss the problem of engineering the evolutionary machinery that can lead to the desired result in the swarm robotics context. We also present a case study about self-organizing synchronization in a swarm of robots, in which some arbitrarily chosen properties of the communication system hinder the scalability of the behavior to large groups. We show that by modifying the communication system, artificial evolution can synthesize behaviors that scale properly with the group size.

In this paper, we present an artificial vision system that is trained with a genetic algorithm for categorising five different kinds of images (letters) of different sizes. The system, which has a limited field of view, can move its eye so as to explore the images visually. The analysis of the system at the end of the training process indicates that correct categorisation is achieved by (1) exploiting sensory-motor coordination so as to experience stimuli that facilitate discrimination, and (2) integrating perceptual and/or motor information over time through a process of accumulation of partially conflicting evidence. We discuss our results with respect to the possible different strategies for categorisation and to the possible roles that action can play in perception.

Active perception refers to a theoretical approach to the study of perception grounded on the idea that perceiving is a way of acting, rather than a process whereby the brain constructs an internal representation of the world. The operational principles of active perception can be effectively tested by building robot-based models in which the relationship between perceptual categories and the body-environment interactions can be experimentally manipulated. In this paper, we study the mechanisms of tactile perception in a task in which a neuro-controlled anthropomorphic robotic arm, equipped with coarse-grained tactile sensors, is required to perceptually categorize spherical and ellipsoid objects. We show that best individuals, synthesized by artificial evolution techniques, develop a close to optimal ability to discriminate the shape of the objects as well as an ability to generalize their skill in new circumstances. The results show that the agents solve the categorization task in an effective and robust way by self-selecting the required information through action and by integrating experienced sensory-motor states over time.

In this paper we present a model which allows to co-evolve the morphology and the control system of realistically simulated robots (creatures). The method proposed is based on an artificial ontogenetic process in which the genotype does not specify directly the characteristics of the creatures but rather the growing rules that determine how an initial artificial embryo will develop on a fully formed individual. More specifically, the creatures are generated through a developmental process which occurs in time and space and which is realized through the progressive addition of both structural parts and regulatory substances which affect the successive course of the morphogenetic process. The creatures are provided with a distributed control system made up of several independent neural controllers embedded in the different body parts which only have access to local sensory information and which coordinate through the effects of physical actions mediated by the external environment through the emission/detection of signals which diffuse locally in space. The analysis of evolved creatures shows how they display effective morphology and control mechanisms which allow them to walk effectively and robustly both on regular and irregular terrains in all the replications of the experiment. Moreover, the obtained results show how the possibility to develop such skills can be improved by also selecting individuals on the basis of a task-independent component which reward them for the ability to coordinate the movements of their parts.

Active perception refers to a theoretical approach to the study of perception grounded on the idea that perceiving is a way of acting, rather than a cognitive process whereby the brain constructs an internal representation of the world. The operational principles of active perception can be effectively tested by building robot-based models in which the relationship between perceptual categories and the body-environment interactions can be experimentally manipulated. In this paper, we study the mechanisms of tactile perception in a task in which a neuro-controlled anthropomorphic robotic arm, equipped with coarse-grained tactile sensors, is required to perceptually discriminate between spherical and ellipsoid objects. The results of this work demonstrate that evolved continuous time nonlinear neural controllers can bring forth strategies to allow the arm to effectively solve the discrimination task.

This paper analyses the strengths and weaknesses of self-organising approaches, such as evolutionary ro-botics, and direct design approaches, such as behav-iour-based controllers, for the production of autono-mous robots? controllers, and shows how the two approaches can be usefully combined. In particular, the paper proposes a method for encoding evolved neural-network based behaviours into motor schema-based controllers and then shows how these control-lers can be modified and combined to produce robots capable of solving new tasks. The method has been validated in the context of a collective robotics sce-nario in which a group of physically assembled simulated autonomous robots are requested to pro-duce different forms of coordinated behaviours (e.g., coordinated motion, walled-arena exiting, and light pursuing).

Self-organized synchronization is a common phenomenon observed in many natural and artificial systems: simple coupling rules at the level of the individual components of the system result in an overall coherent behavior. Owing to these properties, synchronization appears particularly interesting for swarm robotics systems, as it allows robust temporal coordination of the group while minimizing the complexity of the individual controllers. The goal of the experiments presented in this paper is the study of self-organizing synchronization for robots that present an individual periodic behavior. In order to design the robot controllers, we make use of artificial evolution, which proves to be capable of synthesizing minimal synchronization strategies based on the dynamical coupling between robots and environment. The obtained results are analyzed under a dynamical system perspective, which allows us to uncover the evolved mechanisms and to predict the scalability properties of the self-organizing synchronization with respect to varying group size.

In this paper we investigate whether social learning or the combination of individual and social learning can provide an adaptive advantage for artificial embodied agents that have to develop behavioural abilities that are too difficult or too costly to be acquired through individual learning. Obtained results demonstrate that social learning provides an adaptive advantage when individuals are allowed to learn socially from experienced individuals and individually. Moreover, the obtained results indicate how the iteration of a social and individual learning through out generations lead to a cumulative cultural evolutionary process in which novelties are integrated with previously developed skills and are successfully transmitted in further generations

In this paper we describe how a population of evolving robots can autonomously develop forms of spatial representation that allow them to discriminate different locations of their environment. Developed representation forms consist of patterns of activations of internal neurons that are generated by integrating sequences of sensory-motor states while the robots interact autonomously with their environment. Moreover, these representation forms are allocentric, i.e. they allow evolved robots to identify a given spatial location independently from the robots' initial position and orientation in the environment. The analysis on the robots' representation system indicate that it can be characterized as a limit cycle resulting from the transient dynamic between fixed attractor points that alternate while the robot move in the environment. We also demonstrate how the evolved representation systems display remarkable generalization properties. We conclude the paper by discussing the characteristics of the representation system developed by the robots and its relation with other models described in the literature

The ability to synchronise the individual actions within large groups is an adaptive response observed in many biological systems. Indeed, synchrony can increase the efficiency of a group by maximising the global outcome or by minimising the interference among individuals. In any case, synchronisation appears desirable for a robotic system as it allows to coordinate through time the activities of the group. The main goal of the experiments presented in this paper is the study of self-organising synchronisation behaviours for a group of robots. To do so, we do not postulate the need of internal dynamics. Instead, we stress the importance of the dynamical coupling between robots and environment, which can be exploited for synchronisation, allowing to keep a minimal complexity of both the behavioural and the communication level. We use artificial evolution to synthesise the robot controllers, and we show how very simple communication strategies can produce self-organising synchronisation behaviours that scale to very large groups and that can be transfered to physical robots.

In this paper, we present the results of an experiment in which a collection of simulated robots that have been evolved for the ability to solve a collective navigation problem develop a communication system that allows them to co-operate better. The analysis of the results obtained indicates how evolving robots develop a non-trivial communication system and exploit different communication modalities. The results also indicate how the possibility of co-adapting the robots’ individual and social/communicative behaviour plays a key role in the development of progressively more complex and effective individuals.

An important goal of collective robotics is the design of control systems that allow groups of robots to accomplish common tasks by coordinating without a centralized control. In this paper, we study how a group of physically assembled robots can display coherent behavior on the basis of a simple neural controller that has access only to local sensory information. This controller is synthesized through artificial evolution in a simulated environment in order to let the robots display coordinated-motion behaviors. The evolved controller proves to be robust enough to allow a smooth transfer from simulated to real robots. Additionally, it generalizes to new experimental conditions, such as different sizes/shapes of the group and/or different connection mechanisms. In all these conditions the performance of the neural controller in real robots is comparable to the one obtained in simulation.

Distributed coordination of groups of individuals accomplishing a common task without leaders, with little communication, and on the basis of self-organizing principles, is an important research issue within the study of collective behavior of animals, humans, and robots. The article shows how distributed coordination allows a group of evolved, physically linked simulated robots (inspired by a robot under construction) to display a variety of highly coordinated basic behaviors such as collective motion, collective obstacle avoidance, and collective approach to light, and to integrate them in a coherent fashion. In this way the group is capable of searching and approaching a lighted target in an environment scattered with obstacles, furrows, and holes, where robots acting individually fail. The article shows how the emerged coordination of the group relies upon robust self-organizing principles (e.g., positive feedback) based on a novel sensor that allows the single robots to perceive the group's "average" motion direction. The article also presents a robust solution to a difficult coordination problem, which might also be encountered by some organisms, caused by the fact that the robots have to be capable of moving in any direction while being physically connected. Finally, the article shows how the evolved distributed coordination mechanisms scale very well with respect to the number of robots, the way in which robots are assembled, the structure of the environment, and several other aspects.

In this paper we investigate the emergence of communication in competitive multi-agent systems. A competitive environment is created with two teams of agents competing in an exploration task; the quickest team to explore the largest area wins. One team uses indirect communication and is controlled by an artificial neural network evolved using a Pareto multi-objective approach. The second team uses direct communication and a fixed strategy for exploration. A comparison is made between agents with and without communication. Results show that as the fitness function vary differing exploration strategies emerge. Experiments with communication produced cooperative strategies; while the experiments without communication produced effective strategies but with individuals acting independently.

In this paper, I discuss in which conditions a population of embodied and situated agents that have to solve problems that require co-operation might develop forms of ritualized interaction and communication. After reviewing the most relevant literature, I shall try to identify the main open research problems and the most promising research directions. More specifically, I shall discuss: (a) the type of problems, the agents’ characteristics and the environmental/social conditions that might facilitate the emergence of an ability to interact and communicate; and (b) the behavioural and cognitive capabilities that are crucial for the development of forms of communication of different complexity.

In this paper, we discuss the limitations of current evolutionary robotics models and we propose a new framework that might solve some of these problems and lead to an open-ended evolutionary process in hardware. More specifically, the paper describes a novel approach where the usual concepts of population, generations and fitness are made implicit in the system. Individuals co-evolve embedded in their environment. Exploiting the self-assembling capabilities of the (simulated) robots, the genotype of a successful individual can spread in the population. In this way, interesting behaviours emerge spontaneously, resulting in chasing and evading other individuals, collective obstacle avoidance and co-ordinated motion of self-assembled structures.

Evolutionary robotics (Cliff et al. 1993, Nolfi and Floreano 2000) is the study and application of autonomous robots developed through a self-organizational method based on artificial evolution. Th...

We present a set of experiments in which simulated robots are evolved for the ability to aggregate and move together toward a light target. By developing and using quantitative indexes that capture the structural properties of the emerged formations, we show that evolved individuals display interesting behavioral patterns in which groups of robots act as a single unit. Moreover, evolved groups of robots with identical controllers display primitive forms of situated specialization and play different behavioral functions within the group according to the circumstances. Overall, the results presented in the article demonstrate that evolutionary techniques, by exploiting the self-organizing behavioral properties that emerge from the interactions between the robots and between the robots and the environment, are a powerful method for synthesizing collective behavior.

The evolution of simulated robots with three different architectures is studied in this article. We compare a nonmodular feed-forward network, a hardwired modular, and a duplication-based modular motor control network. We conclude that both modular architectures outperform the non-modular architecture, both in terms of rate of adaptation as well as the level of adaptation achieved. The main difference between the hardwired and duplication-based modular architectures is that in the latter the modules reached a much higher degree of functional specialization of their motor control units with regard to high-level behavioral functions. The hardwired architectures reach the same level of performance, but have a more distributed assignment of functional tasks to the motor control units. We conclude that the mechanism through which functional specialization is achieved is similar to the mechanism proposed for the evolution of duplicated genes. It is found that the duplication of multifunctional modules first leads to a change in the regulation of the module, leading to a differentiation of the functional context in which the module is used. Then the module adapts to the new functional context. After this second step the system is locked into a functionally specialized state. We suggest that functional specialization may be an evolutionary absorption state.

We propose that the ability to extract regularities from time series through prediction learning can be enhanced if we use a hierarchical architecture in which higher layers are trained to predict the internal state of lower layers when such states change significantly. This hierarchical organization has two functions: (a) it forces the system to recode sensory information progressively so as to enhance useful regularities and filter out useless information; and (b) it progressively reduces the length of the sequences which should be predicted going from lower to higher layers. This, in turn, allows higher levels to extract higher-level regularities which are hidden at the sensory level. By training an architecture of this type to predict the next sensory state of a robot navigating in an environment divided into two rooms, we show how the first-level prediction layer extracts low-level regularities such as 'walls', 'corners' and 'corridors', while the second-level prediction layer extracts higher-level regularities such as 'the left side wall of the large room'. The extraction of these regularities allows the robot to localize its position in the environment and to detect changes in the environment (e.g. the presence of a new object or the fact that a door has been closed).

Coevolution (i.e., the evolution of two or more competing populations with coupled fitness) has several features that may potentially enhance the power of adaptation of artificial evolution. In particular, as discussed by Dawkins and Krebs [3], competing populations may reciprocally drive one another to increasing levels of complexity by producing an evolutionary "arms race." In this article we will investigate the role of coevolution in the context of evolutionary robotics. In particular, we will try to understand in what conditions coevolution can lead to "arms races." Moreover, we will show that in some cases artificial coevolution has a higher adaptive power than simple evolution. Finally, by analyzing the dynamics of coevolved populations, we will show that in some circumstances well-adapted individuals would be better advised to adopt simple but easily modifiable strategies suited for the current competitor strategies rather than incorporate complex and general strategies that may be effective against a wide range of opposing counter-strategies.

In this paper I claim that one of the main characteristics that makes the evolutionary robotics approach suitable for the study of adaptive behavior in natural and artificial agents is the possibility of relying largely on a self-organization process. Indeed, by using artificial evolution the role of the designer may be limited to the specification of a fitness function which measures the ability of a given robot to perform a desired task. From an engineering point of view, the main advantage of relying on self-organization is the fact that the designer does not need to divide the desired behavior into simple basic behaviors to be implemented in separate layers (or modules) of the robot control system. By selecting individuals for their ability to perform the desired behavior as a whole, simple basic behaviors can emerge from the interaction between several processes in the control system and from the interaction between the robot and the environment. From the point of view of the study of natural systems, the possibility of evolving robots that are free to select their way to solve a task by interacting with their environment may help us to understand how natural organisms produce adaptive behavior. Finally, the attempt to scale up to more complex tasks may help us to identify what the critical features of natural evolution are which allowed the emergence of the extraordinary variety of highly adapted life forms present on the planet.

The problem of the validity of simulation is particularly relevant for methodologies that use machine learning techniques to develop control systems for autonomous robots, as, for instance, the artificial life approach known as evolutionary robotics. In fact, although it has been demonstrated that training or evolving robots in real environments is possible, the number of trials needed to test the system discourages the use of physical robots during the training period. By evolving neural controllers for a Khepera robot in computer simulations and then transferring the agents obtained to the real environment we show that (a) an accurate model of a particular robot-environment dynamics can be built by sampling the real world through the sensors and the actuators of the robot; (b) the performance gap between the obtained behaviors in simulated and real environments may be significantly reduced by introducing a “conservative” form of noise; (c) if a decrease in performance is observed when the system is transferred to a real environment, successful and robust results can be obtained by continuing the evolutionary process in the real environment for a few generations.