StarCraft: Brood War remains a challenging benchmark for artificial intelligence research, particularly in the domain of macromanagement, where long-term strategic planning is required. Traditional approaches to StarCraft AI rely on rule-based systems or supervised deep learning, both of which face limitations in adaptability and computational efficiency. In this work, we introduce NeuroPAL, a neuroevolutionary framework that integrates Neuroevolution of Augmenting Topologies (NEAT) with Punctuated Anytime Learning (PAL) to improve the efficiency of evolutionary training. By alternating between frequent, low-fidelity training and periodic, high-fidelity evaluations, PAL enhances the sample efficiency of NEAT, enabling agents to discover effective strategies in fewer training iterations. We evaluate NeuroPAL in a fixed-map, single-race scenario in StarCraft: Brood War and compare its performance to standard NEAT-based training. Our results show that PAL significantly accelerates the learning process, allowing the agent to reach competitive levels of play in approximately half the training time required by NEAT alone. Additionally, the evolved agents exhibit emergent behaviors such as proxy barracks placement and defensive building optimization, strategies commonly used by expert human players. These findings suggest that structured evaluation mechanisms like PAL can enhance the scalability and effectiveness of neuroevolution in complex real-time strategy environments.

This paper investigates the application of Neuroevolution of Augmenting Topologies (NEAT) to automate gameplay in Dark Souls, a notoriously challenging action role-playing game characterized by complex combat mechanics, dynamic environments, and high-dimensional visual inputs. Unlike traditional reinforcement learning or game playing approaches, our method evolves neural networks directly from raw pixel data, circumventing the need for explicit game-state information. To facilitate this approach, we introduce the Dark Souls API (DSAPI), a novel Python framework leveraging real-time computer vision techniques for extracting critical game metrics, including player and enemy health states. Using NEAT, agents evolve effective combat strategies for defeating the Asylum Demon, the game's initial boss, without predefined behaviors or domain-specific heuristics. Experimental results demonstrate that evolved agents achieve up to a 35% success rate, indicating the viability of neuroevolution in addressing complex, visually intricate gameplay scenarios. This work represents an interesting application of vision-based neuroevolution, highlighting its potential use in a wide range of challenging game environments lacking direct API support or well-defined state representations.

In this research, we evolve gaits for an arachnid-inspired robot. The method used is an expansion upon previous research on the incremental evolution of gaits for hexapod robots with two degrees of freedom per leg, which we now apply to a more complex, eight-legged robot with three degrees of freedom per leg. Incremental evolution handles gait generation for legged robots in two discrete increments. The first increment uses a cyclic genetic algorithm to learn the activations (pulse instructions to the servos) required for each leg to perform a single-leg cycle. This learning program takes into account the way each leg is mounted on the body and the range of movement provided by the three servos on each leg to produce a smooth, straight, and efficient leg cycle. The second increment uses a genetic algorithm to select the best combination of leg cycles for each leg and to learn the timing to execute each leg cycle to coordinate them all together into a single gait. In this work, we learn the gait incrementally in a simulation and transfer the final gaits to the real robot to confirm the method's viability.

In this paper we describe a method using genetic algorithms to replicate natural allopatric speciation. There are many versions of evolutionary computation (EC) that have some characteristics of speciation, but none that match natural processes. In an effort to develop such a form of EC, we created a simple model that we used to experiment in developing an EC that mimics natural speciation. Our long term goal for speciation is to have a single population eventually become two populations that are reproductively isolated even though they reside in the same environment. In previous work we developed an environment where we replicated adaptation, survival of the fittest, and migration of a population. In this paper, we report on research where we used this environment to explore the possibility of speciation. We use a genetic algorithm that alters the agents in the environment as we allow a population of intermixing individuals to develop and become established. We then add a physical barrier that separates the individuals in the population and then remove the barrier after several generations to see if the initially single population becomes two reproductively isolated populations despite no longer being physically isolated. In this way we were able to replicate the initial stages of allopatric speciation.

A robust obstacle avoidance control program was developed for a mobile robot in the context of tight, dynamic indoor environments. Deep Learning was applied in order to produce a refined classifier for decision making. The network was trained on low quality raw RGB images. A fine-tuning approach was taken in order to leverage pre-learned parameters from another network and to speed up learning time. The robot successfully learned to avoid obstacles as it drove autonomously in a tight classroom/laboratory setting.

In this paper we present an implementation of and a proposed algorithm for an easily expandable hardware Artificial Neural Network (ANN) capable of learning using inexpensive, off-the-shelf microprocessors. While significant work has been done in hardware ANN implementations, this research offers a unique, general use, unspecialized, and inexpensive model with a flexible architecture representation. Using Arduino Pro Mini microprocessors and a flexible data communication framework that makes use of the built-in circuit bus called the Inter-Integrated Circuit, this implementation involves the programming of one neuron per microchip. This one to one ratio allows for the computational parallelism inherent in neural networks and provides for the flexibility of building various ANN architectures. The prototype that was developed consists of an input layer element microchip, two hidden layer neuron microchips, and an output layer neuron microchip. Learning happens completely on hardware via backpropagation without a data connection to a computer. Tests showed that the prototype can learn the logical operations OR, AND, XOR, and XNOR, and that the system can accommodate dynamic changes in learning between logical operations.

Previously we developed an implementation for an easily expandable hardware Artificial Neural Network (ANN) capable of learning using inexpensive, off-the-shelf Arduino Pro Mini microprocessors. This ANN system is unique, general use, unspecialized, and inexpensive. The implementation involves a one neuron per microchip representation, a ratio which allows for computational parallelism and ANN architecture flexibility inherent to biological neural networks. Learning happens completely on hardware via backpropagation without the need for communication with a computer. Tests showed successful, dynamic learning of the logical operations OR, AND, XOR, and XNOR. In this paper, we demonstrate the usage and strength of this implementation by applying the same framework to learn live obstacle avoidance and autonomous navigation for a 1:24 scale model car equipped with ultrasonic distance sensors. This test of the application involved a user who supervised the learning and a method to easily transition between testing and training the ANN on the car via Bluetooth. Results show that the hardware ANN consistently learns to navigate the car through an obstacle course from entrance to exit and vice versa with no collisions.

This work involves learning the use schedule of an academic building in order to intelligently control various aspects of the environment. Motion sensors are used to monitor and record the activity of each of the rooms in the building. After a basic preprocessing of the data, a Cyclic Genetic Algorithm (CGA) is used to pick out the patterns of use of the rooms. The CGA is seen as ideal for such a problem because of its ability to find repetitive cyclic patterns in the data. Our results show that a CGA has the ability to pick out such patterns and construct a schedule of use for a room.

A new deterministic greedy genetic algorithm selection operator with very high selection pressure, dubbed the "Jugate Adaptive Method" is examined. Its performance and behavior are compared to those of a canonical genetic algorithm with tournament selection, and a random-restarting next-ascent stochastic hill-climber. All three algorithms are tuned using parameter sweeps to optimize their success rates on five combinatorial optimization problems, tuning each algorithm for each problem independently. Results were negative in that the new method was outperformed in nearly all experiments. Experimental data show the hill climber to be the clear winner in four of five test problems.

Various selection schemes have been described for use in genetic algorithms. This paper investigates the effects of adding greediness to the standard roulette-wheel selection. The results of this study are tested on a Cyclic Genetic Algorithm (CGA) used for learning gaits for a hexapod servo-robot. The effectiveness of CGA in learning optimal gaits with selection based on roulette-wheel selection with and without greediness is compared. The results were analyzed based on fitness of the individual gaits, convergence time of the evolution process, and the fitness of the entire population evolved. Results demonstrate that selection with too much greediness tends to prematurely converge with a sub-optimal solution, which results in poorer performance compared to the standard roulette-wheel selection. On the other hand, roulette-wheel selection with very low greediness evolves more diverse and fitter populations with individuals that result in the desired optimal gaits.

Generating walking gaits for legged robots is a challenging task. Gait generation with proper leg coordination involves a series of actions that are continually repeated to create sustained movement. In this paper we present the use of a Cyclic Genetic Algorithm (CGA) to learn gaits for a quadruped servo robot with three degrees of movement per leg. An actual robot was used to generate a simulation model of the movement and states of the robot. The CGA used the robot's unique features and capabilities to develop gaits specific for that particular robot. Tests done in simulation show the success of the CGA in evolving a reasonable control program and preliminary tests on the robot show that the resultant control program produces a suitable gait.

Anytime Learning with Fitness Biasing has been shown in previous works to be an effective tool for evolving hexapod gaits. In this paper, we present the use of Anytime Learning with Fitness Biasing to evolve the controller for a robot learning the box pushing task. The robot that was built for this task, was measured to create an accurate model. The model was used in simulation to test the effectiveness of Anytime Learning with Fitness Biasing for the box pushing task. This work is the first step in new research where an automated system to test the viability of Fitness Biasing will be created, as well as the first application of Fitness Biasing to a high level task such as box pushing.

It is a difficult task to generate optimal walking gaits for mobile legged robots. Generating and coordinating an optimal gait involves continually repeating a series of actions in order to create a sustained movement. In this work, we present the use of a Cyclic Genetic Algorithm (CGA) to learn near optimal gaits for an actual quadruped servo-robot with three degrees of movement per leg. This robot was used to create a simulation model of the movement and states of the robot which included the robot's unique features and capabilities. The CGA used this model to learn gaits that were optimized for this particular robot. Tests done in simulation show the success of the CGA in evolving gait control programs and tests on robot show that these control programs produce reasonable gaits.

Evolving a robot's sensor morphology along with its control program has the potential to significantly improve its effectiveness in completing the assigned task, plus accommodates the possibility of allowing it to adapt to significant changes in the environment. In previous work, we presented a learning system where the angle, range, and type of sensors on a hexapod robot, along with the control program, were evolved. The evolution was done in simulation and the tests, which were also done in simulation, showed that effective sensor morphologies and control programs could be co-learned by the system. In this paper, we describe the learning system and show that the simulated results are confirmed by tests on the actual hexapod robot.

Real-time learning is the process of an artificial intelligence agent learning behavior(s) at the same pace as it operates in the real world. Video games tend to be an excellent locale for testing real-time learning agents, as the action happens at real speeds with a good visual feedback mechanism, coupled with the possibility of comparing human performance to that of the agent's. In addition, players want to be competing against a consistently challenging opponent. This paper is a discussion of a controller for an agent in the space combat game Xpilot and the evolution of said controller using two different methods. The controller is a multilayer neural network, which controls all facets of the agent's behavior that are not created in the initial set-up. The neural network is evolved using 1-to-1 evolution strategies in one method and genetic algorithms in the other method. Using three independent trials per methodology, it was shown that evolution strategies learned faster, while genetic algorithms learned more consistently, leading to the idea that genetic algorithms may be superior when there is ample time before use, but evolution strategies are better when pressed for learning time as in real-time learning.

It is advantageous for colony robots to be autonomous and self-sufficient. This requires them to perform their duties while maintaining enough energy to operate. Previously, we reported the equipping of power storage for legged robots with high capacitance capacitors, the configuration of one of these robots to effectively use its power storage in a colony recharging system, and the learning of a control program that enabled the robot to navigate to a charging station in simulation. In this work, we report the learning of a control program that allowed the simulated robot to perform area coverage in a self-sufficient framework that made available the best pre-learned navigation behavior module.

The Core is a unique learning environment where agents compete to evolve controllers without the need of a fitness function. In this paper we use it with a cyclic genetic algorithm to evolve agents in the network game Xpilot, where the agents are engaged in space combat. The agents interact locally through tournament selection, crossover, and mutation to produce offspring in the evolution of controllers. The system is highly parallel, can be easily distributed among a network of computers, and has an element of simple co-evolution as the environment (population of agents) evolves to continually challenge individual agents evolving in the environment

It is desirable that colony robots be autonomous and self-sufficient, which requires that they can perform their duties while maintaining enough energy to operate. In previous work, we reported the equipping of legged robots with high capacitance capacitors for power storage and the configuration of one of these robots to make practical use of its power storage in a colony recharging system. Research reported in this paper involves the learning of a control program that allows this robot to navigate to a charging station. The viability of the configuration and the learned control program were verified by observing the actual robot as it operated using the best of the solutions produced in simulation.

An unconstrained cyclic genetic algorithm (CGA) is presented as a means to evolve multi-loop behavior for an autonomous agent. The evolved programs control autonomous agents in the network space combat game Xpilot. Ultimately, the agent's goal is survival; it must learn to move within a restricted area, avoiding obstacles while engaged in combat with an opposing agent. The CGA learned multi-loop control programs that significantly improved the agent's survivability in the hostile Xpilot environment.

Xpilot, a network game where agents engage in space combat, has been shown to be a good test bed for controller learning systems. In this paper, we introduce the Core, an Xpilot learning environment where a population of learning agents interact locally through tournament selection, crossover, and mutation to produce offspring in the evolution of controllers. The system does not require the researcher to develop a fitness function or suitable agents to engage with the evolving agent. Instead, it employs a form of co-evolution where the environment, made up of the population of agents, evolves to continually challenge individual agents evolving within it. Tests show its successful use in evolving controllers for combat agents in Xpilot.

In the research presented in this paper, we use incremental evolution to learn multifaceted neural network (NN) controllers for agents operating in the space game Xpilot. Behavioral components specific to the accomplishment of specific tasks, such as bullet-dodging, shooting, and closing on an enemy, are learned in the first increment. These behavioral components are used in the second increment to evolve a NN that prioritizes the output of a two-layer NN depending on that agent's current situation.

In this paper, we describe a distributed learning system used to evolve a control program for an agent operating in the network game Xpilot. This system, which we refer to as a queue genetic algorithm, is a steady state genetic algorithm that uses stochastic selection and first-in-first-out replacement. We employ it to distribute fitness evaluations over a local network of dissimilar computers. The system made full use of our available computers while evolving successful controller solutions that were comparable to those evolved using a regular generational genetic algorithm.

Simulated evolution by the use of genetic algorithms (GA) is presented as the solution to a two-faceted problem: the challenge for an autonomous agent to learn the reactive component of multiple survival strategies, while simultaneously determining the relative importance of these strategies as the agent encounters changing multivariate obstacles. The agent's ultimate purpose is to prolong its survival; it must learn to navigate its space avoiding obstacles while engaged in combat with an opposing agent. The GA learned rule-based controller significantly improved the agent's survivability in the hostile Xpilot environment.

Interactive combat games are useful as test-beds for learning systems employing evolutionary computation. Of particular value are games that can be modified to accommodate differing levels of complexity. In this paper, the authors presented the use of Xpilot as a learning environment that can be used to evolve primitive reactive behaviors, yet can be complex enough to require combat strategies and team cooperation. In addition, this environment was used with a genetic algorithm to learn the weights for an artificial neural network controller that provides both offensive and defensive reactive control for an autonomous agent.

Cyclic genetic algorithms can be used to generate single loop control programs for robots. While successful in generating controllers for individual leg movement, gait generation, and area search path finding, cyclic genetic algorithms have had limited use when dealing with control problems that require different behaviors in response to sensor inputs. For such behaviors, there is a need for modifications that will allow the generation of multi-loop control programs, which can properly react to sensor input. In this work, we present modifications to the standard cyclic genetic algorithm that enables it to learn multi-loop control programs with branching that allows the control to jump from one loop to another. Preliminary tests show the success of our modification.

The controlled use of a continuous power supply for robots of a colony is presented. This work builds on previous work where capacitors were used as an onboard power supply, and where robots with metallic probes were charged at a power station. An onboard controller was implemented to direct the hexapod colony robot behavior according to its power supply status. A PIC chip, implemented as an analog-to-digital voltage converter, was used as the robot's voltage sensor. A BASIC Stamp II was used in conjunction with photocells as the robot's light sensors while a light source was placed at the power station. Tests were performed and revealed the plausibility of implementing a voltage sensor and light sensor/source for colony robots powered with capacitors.

The evolution of a team of heterogeneous agents is challenging. To allow the greatest level of specialization team members must be evolved in separate populations, but finding acceptable partners for evaluation at trial time is difficult. Testing too few partners blinds the GA from recognizing fit solutions while testing too many partners makes the computation time unmanageable. We developed a system based on punctuated anytime learning that periodically tests a number of partner combinations to select a single individual from each population to be used at trial time. We previously tested our method with a two agent box-pushing task. In this work, we show the efficiency of our method by applying it to the predator-prey scenario.

Partial recombination is a type of crossover for genetic algorithms that focuses on a subset of the chromosome. It provides a means of doing crossover were only the genes involved in producing the fitness are affected. In this paper, we use it to evolve the parameters for a model that represents the capabilities of a robot. The values of these parameters are evolved as the robot periodically performs an action that is also being performed by the model. Each action performed by the robot does not include every possible turn command so using partial recombination allows the system to only change the parameters involved in the action. Tests show that partial recombination makes a significant difference in the co-evolution of model parameters.

The evolution of a heterogeneous team is a complex problem. Evolving teams if a single population can retard the GA's ability to specialize emergent behavior, but co-evolution requires a system for evaluation at trial time. If two few combinations of partners are tested, the GA is unable to recognize fit agents; if too many agents are tested, the resultant computation time becomes excessive. We created a system based on punctuated anytime learning that only periodically tests samples of partner combinations to reduce computation time and tested a variety of sample sizes. In this paper, we present a successful method of varying the sample sizes, dependent on the level of fitness, using a box pushing task for comparison.

Anytime learning with fitness biasing was shown in an earlier work to be an effective tool for learning leg cycles for a hexapod robot. This learning system was capable of adapting to changes in the environment. Although the leg cycles were appropriate for rougher terrain, the gaits produced with them by a standard genetic algorithm were not capable of bearing the robot's load. In this paper, we present the use of anytime learning with fitness biasing to improve the gaits produced by allowing the learning system to adapt to unforeseen changes in the environment and the robot's capabilities. Training and tests were done in simulation, with the resultant gaits tested on the actual robot.

Evolving heterogeneous behavior for cooperative agents is a complex challenge. The co-evolution of separate populations requires a system for evaluation at trial time. If too few combinations of partners are tested, the GA is unable to recognize fit agents, but if too many agents are tested the required computation time becomes unreasonable. To resolve this issue, we created a system based on punctuated anytime learning that periodically tests partner combinations to reduce computation time. In continued research, we discovered that by testing fewer combinations the GA maintains accuracy while further reducing computation time. In this paper we propose a method that concurrently tests varying numbers of partner combinations and the spacing between these combinations at trial time to determine which is optimal for any stage of the co-evolution. We chose a box pushing task to compare these methods.

We describe a system that uses evolutionary computation to evolve tower-like structures. The construction takes place in a computer simulated gravitational environment. The evolution targets the morphology; each chromosome carries structural description of the entity. Fitness functions evaluate the structural integrity and "goodness" of each individual based on indicators such as joint tension, center of gravity, position in space, height, etc. Twelve evolution-tests were performed and all successfully reached tower solutions.

The evolution of heterogeneous team behaviour can be a very demanding task. In order to promote the greatest level of specialization team members should be evolved in separate populations. The greatest complication in the evolution of separate populations is finding suitable partners for evolution at trial time. If too few combinations are tested, the genetic algorithm loses its ability to recognize possible solutions and if too many combinations are tested the algorithm becomes too computationally expensive. In previous work a method of punctuated anytime learning was employed to test all combinations of possible partners at periodic generations to reduce the number of evaluations. In further works, it was found that by varying the number of combinations tested, the sample size, the GA could produce an accurate and even less computationally expensive solution. In this paper, we compare different sampling sizes to determine the most effective approach to finding the solution. We use a box pushing task to compare these different sampling sizes.

This paper discusses the use of fitness biasing to alter the control of a seven-microprocessor robot as it shifts from one environment to another. The robot was initially using a gait evolved to work on a smooth surface (tile). When tested on a rough surface (carpet) the learned gait was found to be inappropriate because the legs were causing drag as they repositioned. An efficient move to reposition on the smooth surface did not work on the rough surface. Anytime learning with fitness biasing was applied to the continued evolution of the individual leg cycles as the simulated robot moved from an area of smooth to rough terrain. An actual robot was used to test the results. Following training using fitness biasing, the robot's gait was more appropriate for a rough surface as it learned to raise its leg more before initiating the return movement.

The incremental evolution of neural networks to control hexapod robot locomotion can be separated into two main parts: the evolution of leg controllers the cycle action of single legs (leg cycles) and the evolution of the coordination of these individual leg controllers to produce a gait. In this paper, we use a genetic algorithm to do the first of these steps, to evolve the structure of an artificial neural network that produces leg cycles for a hexapod robot. The robot has 12 servo effectors; two per leg to produce horizontal and vertical movement. The servos are controlled by pulses that are provided by the leg's controller. A cycle of these pulses produces a leg cycle. With minimal restrictions on the structure of the neural network, a genetic algorithm was used to evolve in simulation the parameters of neurons and their connections. Neural networks were implemented on a BASIC Stamp II SX microcomputer and found to generate smooth leg cycles on the hexapod robot.

Punctuated anytime learning is presented as the solution for two problems: the use of anytime learning with an off-line learning module and the linking of the actual robot to its simulation during evolutionary robotics. Two methods of punctuated anytime learning, fitness biasing and the co-evolution of model parameters, are described and compared using the common task of gait generation for a hexapod robot with changing capabilities.