Cross-entropy optimization (CE) has proven to be a powerful tool for search in control environments. In the basic scheme, a distribution over proposed solutions is repeatedly adapted by evaluating a sample of solutions and refocusing the distribution on a percentage of those with the highest scores. We show that, in the kind of noisy evaluation environments that are common in decision-making domains, this percentage-based refocusing does not optimize the expected utility of solutions, but instead a quantile metric. We provide a variant of CE (Proportional CE) that effectively optimizes the expected value. We show using variants of established noisy environments that Proportional CE can be used in place of CE and can improve solution quality.

This paper examines how the choice of the selection mechanism in an evolutionary algorithm impacts the objective function it optimizes, specifically when the fitness function is noisy. We provide formal results showing that, in an abstract infinite-population model, proportional selection optimizes expected fitness, truncation selection optimizes order statistics, and tournament selection can oscillate. The "winner" in a population depends on the choice of selection rule, especially when fitness distributions differ between individuals resulting in variable risk. These findings are further developed through empirical results on a novel stochastic optimization problem called "Die4", which, while simple, extends existing benchmark problems by admitting a variety of interpretations of optimality.

Recent advancements in model-based reinforcement learning have shown that the dynamics of many structured domains (e.g. DBNs) can be learned with tractable sample complexity, despite their exponentially large state spaces. Unfortunately, these algorithms all require access to a planner that computes a near optimal policy, and while many traditional MDP algorithms make this guarantee, their computation time grows with the number of states. We show how to replace these over-matched planners with a class of sample-based planners — whose computation time is independent of the number of states — without sacrificing the sample-efficiency guarantees of the overall learning algorithms. To do so, we define sufficient criteria for a sample-based planner to be used in such a learning system and analyze two popular sample-based approaches from the literature. We also introduce our own sample-based planner, which combines the strategies from these algorithms and still meets the criteria for integration into our learning system. In doing so, we define the first complete RL solution for compactly represented (exponentially sized) state spaces with efficiently learnable dynamics that is both sample efficient and whose computation time does not grow rapidly with the number of states.

The purpose of this paper is to present a comparison between two methods of building adaptive controllers for robots. In spite of the wide range of techniques which are used for defining autonomous robot architectures, few attempts have been made in comparing their performance under similar circumstances. This comparison is particularly important in establishing benchmarks and in determining the best approach methods. The robotic tasks in our research concern mainly the convergence of behaviors like obstacle avoidance, hitting targets and shortest path finding using various methods of synthesizing control architectures' parameters. The first approach that has been used combines Neural Networks and Genetic Algorithms in a simple yet robust controller using an Evolutionary Robotics technique. The second one introduces a manner of using Reinforcement Learning with a Neural Network based architecture. The experiments take place in a simulated 3D environment, which was designed to allow the development, testing and comparison of various controllers in terms of advantages and disadvantages in order to establish a benchmark for autonomous robots.