A wide variety of tasks may be performed by humans using only visual data as input. Creating artificial intelligence that adequately uses visual data allows controllers to use single cameras for input and to interact with computer games by merely reading the screen render. In this research, we use the Quake II game environment to compare various techniques that train neural network (NN) controllers to perform a variety of behaviors using only raw visual input. First, it is found that a humanlike retina, which has greater acuity in the center and less in the periphery, is more useful than a uniform acuity retina, both having the same number of inputs and interfaced to the same NN structure, when learning to attack a moving opponent in a visually simple room. Next, we use the same humanlike retina and NN in a more visually complex room, but, finding it is unable to learn successfully, we use a Lamarckian learning algorithm with a nonvisual hand-coded controller as a supervisor to help train the visual controller via backpropagation. Last, we replace the hand-coded supervising nonvisual controller with an evolved nonvisual NN controller, eliminating the human aspect from the supervision, and it solves a problem for which a solution was not previously known.

Watermarking based DRM implementations insert imperceptible information or watermark in digital media to trace owner of the content and deter the illegal distribution of media. In geometry based 3D watermarking algorithms, a watermark is inserted by modifying the coordinates of vertices in the mesh. It is a requirement of watermarking algorithms that this change in vertex coordinates shouldn't cause perceptible distortion. It has always been a challenge to select vertices in the 3D model which would not cause perceptible distortion on addition of watermark. This paper proposes a novel approach to overcome this challenge using Artificial Neural Networks (ANN). Feature vectors representing the geometry of the vertex and its surrounding vertices are extracted and used to train and simulate ANN. ANN is used as a classifier to determine which vertices should be selected for watermarking. Experimental results simulate various attacks to test the robustness of the algorithm.

A combination of backpropagation and neuroevolution is used to train a neural network visual controller for agents in the Quake II environment. The agents must learn to shoot an enemy opponent in a semi-visually complex environment using only raw visual inputs. A comparison is made between using normal neuroevolution and using neuroevolution combined with backpropagation for Lamarckian adaptation. The supervised backpropagation imitates a hand-coded controller that uses non-visual inputs. Results show that using backpropagation in combination with neuroevolution trains the visual neural network controller much faster and more successfully.

We evolve a neural network controller for a boat that learns to maintain a given bearing and range with respect to a moving target in the Lagoon 3D game environment. Simulating realistic physics makes maneuvering boats difficult and thus makes an evolutionary approach an attractive alternative to hand coded methods. We evolve the weights of simple recurrent neural networks trained with a fitness function designed to combine multiple fitness objectives based on speed, heading, and position to create a robust maintain station behavior. Results with an enforced subpopulation neural-evolution genetic algorithm indicate that we can consistently evolve robust maintain controllers for realistically simulated boats in Lagoon.

The first-person-shooter Quake II is used as a platform to test neuro-visual control and retina input layouts. Agents are trained to shoot a moving enemy as quickly as possible in a visually simple environment, using a neural network controller with evolved weights. Two retina layouts are tested, each with the same number of inputs: first, a graduated density retina which focuses near the center of the screen and blurs outward; second, a uniform retina which focuses evenly across the screen. Results show that the graduated density retina learns more successfully than the uniform retina.

It is sometimes useful to provide intelligent agents with some degree of stochastic behavior, particularly when used in games and simulators. The less-predictable behavior that results from the randomness can make the agents seem more believable, and would encourage the players or users to address the genuine problems presented by a game or simulator rather than simply learning to exploit the embedded agents' predictability. However, such randomized behavior should not harm performance in the agents' designated tasks. This paper introduces a method, called stochastic sharpening, for training artificial neural networks as stochastic controllers for agents in discrete-state environments. Stochastic sharpening reinforces the representation of confidence values in the outputs of networks with localist encodings, and thus produces networks that recommend alternative actions on the basis of their expected utility. Such networks can be used to introduce stochastic behavior with minimal disruption of task performance, resulting in agents that are more believable and less subject to exploitation based on predictability.

In most modern video games, character behavior is scripted; no matter how many times the player exploits a weakness, that weakness is never repaired. Yet, if game characters could learn through interacting with the player, behavior could improve as the game is played, keeping it interesting. This paper introduces the real-time Neuroevolution of Augmenting Topologies (rtNEAT) method for evolving increasingly complex artificial neural networks in real time, as a game is being played. The rtNEAT method allows agents to change and improve during the game. In fact, rtNEAT makes possible an entirely new genre of video games in which the player trains a team of agents through a series of customized exercises. To demonstrate this concept, the Neuroevolving Robotic Operatives (NERO) game was built based on rtNEAT. In NERO, the player trains a team of virtual robots for combat against other players' teams. This paper describes results from this novel application of machine learning, and demonstrates that rtNEAT makes possible video games like NERO where agents evolve and adapt in real time. In the future, rtNEAT may allow new kinds of educational and training applications through interactive and adapting games.

We introduce the adaptive team of agents (ATA), a system of homogeneous agents with identical control policies which nevertheless adopt heterogeneous roles appropriate to their environment. ATAs have applications in domains such as games, and can be evolved through neuroevolution. In this paper we show how ATAs can be evolved to solve the problem posed by a simple strategy game and discuss their application to richer environments.

A potentially powerful application of evolutionary computation (EC) is to evolve neural networks for automated control tasks. However, in such tasks environments can be unpredictable and fixed control policies may fail when conditions suddenly change. Thus, there is a need to evolve neural networks that can adapt, i.e. change their control policy dynamically as conditions change. In this paper, we examine two methods for evolving neural networks with dynamic policies. The first method evolves recurrent neural networks with fixed connection weights, relying on internal state changes to lead to changes in behavior. The second method evolves local rules that govern connection weight changes. The surprising experimental result is that the former method can be more effective than evolving networks with dynamic weights, calling into question the intuitive notion that networks with dynamic synapses are necessary for evolving solutions to adaptive tasks.