This paper presents an automated robotic system for generating semantic maps of inventory in retail environments. In retail settings, semantic maps are labeled maps of stores where each discrete section of shelving is assigned a department label describing the types of products on that shelf. Starting from a metric map of the store, the robot autonomously extracts the shelf boundaries, generates a distance-optimal tour of the store to view every shelf, and follows the tour while avoiding unmapped clutter and moving people. The robot creates a point cloud of the store using the data collected from this tour. We introduce a novel soft-object assignment algorithm to create a virtual map and a dynamic programming algorithm to segment this map. These algorithms use a priori information about the products to boost data from laser and camera sensors in order to recognize and semantically label objects. The primary contribution of this paper is the integration of multiple systems for automated path planning, navigation, object recognition, and semantic mapping. This paper represents an important contribution toward deploying mobile robots in dynamic human environments.

The class of objects that can be represented by surfaces of revolution (SoRs) is highly prevalent in human work and living spaces. Due to their prevalence and convenient geometric properties, SoRs have been employed over the past thirty years for single-view camera calibration and pose estimation, and have been studied in terms of SoR object reconstruction and recognition. Such treatment has provided techniques for the automatic identification and classification of important SoR structures, such as apparent contours, cross sections, bitangent points, creases, and inflections. The presence of these structures are crucial to most SoR-based image metrology algorithms. This paper develops single-view and two-view pose recovery and reconstruction formulations that only require apparent contours, and no other SoR features.The primary objective of this paper is to present and experimentallyvalidate the minimal problems pertaining toSoR metrology from apparent contours. For a single view with a known reference model, this includes absolute pose recovery. For many views and no reference model this is extended to structure from motion (SfM). Assuming apparent contours as input that have been identified and segmented with reasonable accuracy, the minimal problems aredemonstrated to produce accurate SoR pose and shape results when used as part of a RANSAC-based hypothesis generation and evaluation pipeline.

In many scenarios, robots encounter rotationally symmetric objects for which no known 3D model exists. To be able to grasp such objects using existing grasp point computation schemes, an estimate of their 3D-pose and shape is necessary. In this paper, we address the problem of recovering 3D-pose and shape of an unknown surface of revolution from two perspective views of known relative orientation. We propose a new algorithm for simultaneous estimation of pose and shape without making use of any cross-sections or bi-tangent points needed by other approaches. Our algorithm builds upon existing single-view SOR reconstruction approaches and couples the pose estimation and reconstruction process. Pose is optimized to minimize discrepancies between reconstructed shapes from two views. Our method works even in the presence of only one of the two apparent contours of a surface of revolution. We test our algorithm by comparing to ground-truth poses and shapes as well as performing grasping experiments. We introduce a new dataset of rotationally symmetric objects in a variety of poses and backgrounds and with a measured ground-truth pose and shape.

We present a novel approach for detecting objects and estimating their 3D pose in single images of cluttered scenes. Objects are given in terms of 3D models without accompanying texture cues. A deformable parts-based model is trained on clusters of silhouettes of similar poses and produces hypotheses about possible object locations at test time. Objects are simultaneously segmented and verified inside each hypothesis bounding region by selecting the set of superpixels whose collective shape matches the model silhouette. A final iteration on the 6-DOF object pose minimizes the distance between the selected image contours and the actual projection of the 3D model. We demonstrate successful grasps using our detection and pose estimate with a PR2 robot. Extensive evaluation with a novel ground truth dataset shows the considerable benefit of using shape-driven cues for detecting objects in heavily cluttered scenes.