This paper describes quasi-online reinforcement learning: while a robot is exploring its environment, in the background a probabilistic model of the environment is built on the fly as new experiences arrive; the policy is trained concurrently based on this model using an anytime algorithm. Prioritized sweeping, directed exploration, and transformed reward functions provide additional speed-ups. The robot quickly learns goal-directed policies from scratch, requiring few interactions with the environment and making efficient use of available computation time. From an outside perspective it learns the behavior online and in real time. We describe comparisons with standard methods and show the individual utility of each of the proposed techniques

This work presents a method for probabilistic plane fitting and an application to robotic 3D mapping. The plane is fitted in an orthogonal least-square sense and the output complies with the conventions of the symmetries and perturbation model (SPmodel). In the second part of the paper, the presented plane fitting method is used within a 3D mapping application. It is shown that by using probabilistic information, high precision 3D maps can be generated.

This paper relates first experiences using a state-of-the-art, time-of-flight sensor that is able to deliver 3D images. The properties and capabilities of the sensor make it a potential powerful tool for applications within mobile robotics especially for real-time tasks, as the sensor features a frame rate of up to 30 frames per second. Its capabilities in terms of basic obstacle avoidance and local path-planning are evaluated and compared to the performance of a standard laser scanner.

It is difficult to apply traditional reinforcement learning algorithms to robots, due to problems with large and continuous domains, partial observability, and limited numbers of learning experiences. This paper deals with these problems by combining: (1) reinforcement learning with memory, implemented using an LSTM recurrent neural network whose inputs are discrete events extracted from raw inputs; (2) online exploration and offline policy learning. An experiment with a real robot demonstrates the methodology's feasibility.