Typical real world machine control tasks have some characteristics which makes them difficult to solve: Their state spaces are high-dimensional and continuous, and it may be impossible to reach a satisfying target state by exploration or human control. To overcome these problems, in this paper, we propose (1) to use radial basis functions for value function approximation in continuous space reinforcement learning and (2) the use of learned inverse projection functions for state space exploration. We apply our approach to path planning in dynamic environments and to an aircraft autolanding simulation, and evaluate its performance.

Many robot learning tasks are very difficult to solve: their state spaces are high dimensional, variables and command parameters are continuously valued, and system states are only partly observable. In this paper, we propose to learn a continuous space value function for reinforcement learning using neural networks trained from data of exploration runs. The learned function is guaranteed to be a lower bound for, and reproduces the characteristic shape of, the accurate value function. We apply our approach to two robot navigation tasks, discuss how to deal with possible problems occurring in practice, and assess its performance.

In many robot applications, autonomous robots must be capable of localizing the objects they are to manipulate. In this paper we address the object localization problem by fitting a parametric curve model to the object contour in the image. The initial prior of the object pose is iteratively refined to the posterior distribution by optimizing the separation of the object and background. The local separation criteria are based on local statistics which are iteratively computed from the object and background region. No prior knowledge on color distributions is needed. Experiments show that the method is capable of localizing objects in a cluttered and textured scene even under strong variations of illumination. The method is able to localize a soccer ball within frame rate.

With the services that autonomous robots are to provide becoming more demanding, the states that the robots have to estimate become more complex. In this paper, we develop and analyze a probabilistic, vision-based state estimation method for individual autonomous robots. This method enables a team of mobile robots to estimate their joint positions in a known environment and track the positions of autonomously moving objects. The state estimators of different robots cooperate to increase the accuracy and reliability of the estimation process. This cooperation between the robots enables them to track temporarily occluded objects and to faster recover their position after they have lost track of it. The method is empirically validated based on experiments with a team of physical robots.

Most multi-robot navigation planning methods make assumptions about the kind of navigation problems they are to solve and the capabilities of the robots they are to control. In this paper, we propose to select problem-adequate navigation planning methods based on empirical investigations, that is, the robots should learn by experimentation to use the best planning methods. To support this development strategy we provide software tools that enable the robots to automatically learn predictive models for the performance of different navigation planning methods in a given application domain. We show, in the context of robot soccer, that the hybrid planning method which selects planning methods based on a learned predictive model outperforms the individual planning methods. The results are validated in extensive experiments using a realistic and accurate robot simulator that has learned the dynamic model of the real robots.

With the services that autonomous robots are to provide becoming more demanding, the states that the robots have to estimate become more complex. We develop and analyze a probabilistic, vision-based state estimation method for individual, autonomous robots. This method enables a team of mobile robots to estimate their joint positions in a known environment and track the positions of autonomously moving objects. The state estimators of different robots cooperate to increase the accuracy and reliability of the estimation process. This cooperation between the robots enables them to track temporarily occluded objects and to faster recover their position after they have lost track of it. The method is empirically validated based on experiments with a team of physical robots.

Most artificial neural networks used in practical applications are based on simple neuron types in a multi-layer architecture. Here, we propose to solve optimization problems using a fully recurrent network of spiking neurons mimicking the response behavior of biological neurons. Such networks can compute a series of different solutions for a given problem and converge into a periodical sequence of such solutions. The goal of this paper is to prove that neural networks like the SRM (Spike Response Model) are able to solve nonlinear optimization problems. We demonstrate this for the traveling salesman problem. Our network model is able to compute multiple solutions and can use its dynamics to leave local minima in which classical models would be stuck. For adapting the model, we introduce a suitable network architecture and show how to encode the problem directly into the network weights.