This paper sketches and discusses design options for complex probabilistic state estimators and investigates their interactions and their impact on performance. We consider, as an example, the estimation of game states in autonomous robot soccer. We show that many factors other than the choice of algorithms determine the performance of the estimation systems. We propose empirical investigations and learning as necessary tools for the development of successful state estimation systems.

Typical real world machine control tasks have some characteristics which makes them difficult to solve: Their state spaces are high-dimensional and continuous, and it may be impossible to reach a satisfying target state by exploration or human control. To overcome these problems, in this paper, we propose (1) to use radial basis functions for value function approximation in continuous space reinforcement learning and (2) the use of learned inverse projection functions for state space exploration. We apply our approach to path planning in dynamic environments and to an aircraft autolanding simulation, and evaluate its performance.

Many robot learning tasks are very difficult to solve: their state spaces are high dimensional, variables and command parameters are continuously valued, and system states are only partly observable. In this paper, we propose to learn a continuous space value function for reinforcement learning using neural networks trained from data of exploration runs. The learned function is guaranteed to be a lower bound for, and reproduces the characteristic shape of, the accurate value function. We apply our approach to two robot navigation tasks, discuss how to deal with possible problems occurring in practice, and assess its performance.

In many robot applications, autonomous robots must be capable of localizing the objects they are to manipulate. In this paper we address the object localization problem by fitting a parametric curve model to the object contour in the image. The initial prior of the object pose is iteratively refined to the posterior distribution by optimizing the separation of the object and background. The local separation criteria are based on local statistics which are iteratively computed from the object and background region. No prior knowledge on color distributions is needed. Experiments show that the method is capable of localizing objects in a cluttered and textured scene even under strong variations of illumination. The method is able to localize a soccer ball within frame rate.

With the services that autonomous robots are to provide becoming more demanding, the states that the robots have to estimate become more complex. In this paper, we develop and analyze a probabilistic, vision-based state estimation method for individual autonomous robots. This method enables a team of mobile robots to estimate their joint positions in a known environment and track the positions of autonomously moving objects. The state estimators of different robots cooperate to increase the accuracy and reliability of the estimation process. This cooperation between the robots enables them to track temporarily occluded objects and to faster recover their position after they have lost track of it. The method is empirically validated based on experiments with a team of physical robots.

Most multi-robot navigation planning methods make assumptions about the kind of navigation problems they are to solve and the capabilities of the robots they are to control. In this paper, we propose to select problem-adequate navigation planning methods based on empirical investigations, that is, the robots should learn by experimentation to use the best planning methods. To support this development strategy we provide software tools that enable the robots to automatically learn predictive models for the performance of different navigation planning methods in a given application domain. We show, in the context of robot soccer, that the hybrid planning method which selects planning methods based on a learned predictive model outperforms the individual planning methods. The results are validated in extensive experiments using a realistic and accurate robot simulator that has learned the dynamic model of the real robots.

With the services that autonomous robots are to provide becoming more demanding, the states that the robots have to estimate become more complex. We develop and analyze a probabilistic, vision-based state estimation method for individual, autonomous robots. This method enables a team of mobile robots to estimate their joint positions in a known environment and track the positions of autonomously moving objects. The state estimators of different robots cooperate to increase the accuracy and reliability of the estimation process. This cooperation between the robots enables them to track temporarily occluded objects and to faster recover their position after they have lost track of it. The method is empirically validated based on experiments with a team of physical robots.

The approach presented in this paper allows a team of mobile robots to estimate cooperatively their poses, i.e. positions and orientations, and the poses of other observed objects from images. The images are obtained by calibrated color cameras mounted on the robots. Model knowledge of the robot environment, the geometry of observed objects, and the characteristics of the cameras are represented in curve functions which describe the relation between model curves in the image and the sought pose parameters. The pose parameters are estimated by minimizing the distance between model curves and actual image curves. Observations from possibly different view points obtained at different times are fused by a method similar to the extended Kalman filter. In contrast to the extended Kalman filter, which is based on a linear approximation of the measurement equations, we use an iterative optimization technique which takes nonlinearities into account. The approach has been successfully used in robot soccer, where it reliably maintained a joint pose estimate for the players and the ball.