This paper presents a purely visual semantic mapping framework using omnidirectional images. The approach rests upon the robust segmentation of the robot's local free space, replacing conventional range sensors for the generation of occupancy grid maps. The perceptions are mapped into a bird's eye view allowing an inverse sensor model directly by removing the non-linear distortions of the omnidirectional camera mirror. The system relies on a place category classifier to label the navigation relevant categories: room, corridor, doorway, and open room. Each place class maintains a separated grid map that are fused with the range-based occupancy grid for building a dense semantic map.

This paper presents a spatial layout recovery approach from single omnidirectional images. Vertical structures in the scene are extracted via classification from heterogeneous features computed at the superpixel level. Vertical surfaces are further classified according to their main orientation by fusing oriented line features, floor-wall boundary features and histogram of oriented gradients (HOG) with a Random Forest classifier. Oriented line features are used to build an orientation map that considers the main vanishing points. The floor-wall boundary feature attempts to reconstruct the scene shape as if it were observed from a bird's-eye view. Finally, the HOG descriptors are aggregated per superpixel and summarize the gradient distribution at homogeneous appearance regions. Compared to existing methods in the literature which rely only on corners or lines, our method gains statistical support from multiple cues aggregated per superpixel which provide more robustness against noise, occlusion, and clutter.

The design of visual robotic behaviors constitutes a substantial challenge. It requires to draw meaningful relation ships and constraints between the acquired visual perception and the geometry of the environment both empirically and programmatically. This contribution proposes a novel robot learning framework to classify and acquire scenario specific autonomous behaviors through demonstration. During demonstration, robocentric 3D range and omnidirectional images are recorded as training instances of typical robot navigation situations pertaining to different contexts in multiple indoor scenarios. A programming by demonstration approach generalizes the demonstrated trajectories to a general mapping between visual features extracted from the omnidirectional image onto a corresponding robot motion. The approach is able to distinguish among different traversing scenarios and further identifies the best matching context within the scenario to predict an appropriate robot motion. As a comparison to context matching, the behaviors are trained by means of an artificial neural network and its generalization ability is evaluated against the former. The experimental validation on the mobile robot indicates that the acquired visual behavior is robust and generalizes meaningful actions beyond the specific environments and scenarios presented during training.

This paper presents a novel approach for floor obstacle segmentation in omnidirectional images which rests upon the fusion of multiple classification generated from heterogeneous segmentation schemes. The individual naive Bayes classifiers rely on different features and cues to determine a pixel's class label. Ground truth data for training and testing the classifiers is obtained from the superposition of 3D scans captured by a photonic mixer device camera. The classification is supported by edge detection which indicate the presence of obstacles and sonar range data. The complementary expert decisions are aggregated by stacked generalization, behavior knowledge space or voting combination. The combined floor classifier achieves a classification accuracy of up to 0.96 true positive rate with only 0.03 false positive rate. A robust robot navigation is accomplished by arbitration among a reactive obstacle avoidance and a corridor following behavior using the robots local free space as perception.

This paper describes a novel approach for purely vision based mobile robot navigation. The visual obstacle avoidance and corridor following behavior rely on the segmentation of the traversable floor region in the omnidirectional robocentric view. The image processing employs a supervised approach in which the segmentation optimal with respect to the appearance of the local environment is determined by cross validation over 3D scans captured by a photonic mixer device (PMD) camera. The range data in the front view provides the seeds and validation data to supervise the appearance based segmentation in the omniview. Segmentation relies on histogram backprojection which maintains separate appearance models for floor, obstacles and background. A naive Bayes classifier predicts the occupancy of the robots local environment by fusing the evidence provided by different segmentations and models. The classification error is analyzed on ground truth data generated by a PMD camera and manually segmented scenes. The scheme is highly robust with respect to ambiguous and misleading visual appearances of obstacles and floor, thus enabling the robot to navigate safely in unstructured environments of diverse appearance, texture and illumination. The proposed vision algorithm and the navigation behavior demonstrate a robust performance in extensive robotic experiments across several hours of autonomous operation.