In this paper, we present a novel approach to 2D street map-based localization for mobile robots that navigate mainly in urban sidewalk environments. Recently, localization based on the map built by Simultaneous Localization and Mapping (SLAM) has been widely used with great success. However, such methods limit robot navigation to environments whose maps are prebuilt. In other words, robots cannot navigate in environments that they have not previously visited. We aim to relax the restriction by employing existing 2D street maps for localization. Finding an exact match between sensor data and a street map is challenging because, unlike maps built by robots, street maps lack detailed information about the environment (such as height and color). Our approach to coping with this difficulty is to maximize statistical dependence between sensor data and the map, and localization is achieved through maximization of a Mutual Information-based criterion. Our method employs a computationally efficient estimator of Squared-loss Mutual Information through which we achieved near real-time performance. The effectiveness of our method is evaluated through localization experiments using real-world data sets.

This paper proposes a method of map merging using cycle consistency check and spanning tree selection by RANSAC. Key issues in map merging are the transformation of the coordinate frame of each submap to a global coordinate frame and the rejection of outliers (false arcs) in a pose graph. The proposed method reduces outliers using cycle consistency constraints. To cope with the exponential increase of cycles, cycle consistency check is applied iteratively with a limited number of chordless cycles. Then, spanning trees having only inlier arcs are selected by RANSAC to find the transformation of each submap to a global coordinate frame. Experiments using public datasets show that the proposed method successfully obtained good spanning trees at high outlier ratio.

This paper proposes a method of building a 3D map from straight line segments using a parallel stereo camera. In the proposed method, 3D lines are reconstructed using 3D edge points obtained by edge-based SLAM. This is effective for longitudinal lines, e.g., in a corridor, which are hard to reconstruct due to degeneracy and small disparities. A parallel stereo camera cannot reconstruct horizontal lines in the images, and we reconstruct them using the points of intersection with nearby lines. Furthermore, we propose 1D occupancy cell representation for a 3D line segment to determine its range from multiple view images. Experiments show our approach successfully built line-based maps of indoor environments.

In this study, we present a novel road recognition method using a single image for mobile robot navigation. Vision-based road recognition in outdoor environments remains a significant challenge. Our approach exploits digital street maps, the robot position, and prior knowledge of the environment. We segment an input image into superpixels, which are grouped into various object classes such as roadway, sidewalk, curb, and wall. We formulate the classification problem as an energy minimization problem and employ graph cuts to estimate the optimal object classes in the image. Although prior information assists recognition, erroneous information can lead to false recognition. Therefore, we incorporate localization into our recognition method to correct errors in robot position. The effectiveness of our method was verified through experiments using real-world urban datasets.

This paper presents a map-merging method which builds a 3D visual map by connecting part maps. The purpose is to increase flexibility and robustness in 3D mapping. Map merging is performed by combining image retrieval using bag-of-words, geometric verification, and pose adjustment. A key issue is robust data association. To cope with false matches, we perform group matching with pose propagation, which checks the geometric consistency of point correspondences over multiple frames. Also, we perform path consistency check, which examines the accumulated errors along a loop to eliminate inconsistent part map connections. Experiments show our method successfully built detailed 3D maps of indoor environments.

In this study, we aim to achieve autonomous navigation for robots in environments that they have not previously visited. Many of the existing methods for autonomous navigation require a map to be built beforehand, typically by manually navigating the robot. Navigation without maps, i.e., without any prior information about the environment, is very difficult. We propose to use existing digital street maps for autonomous navigation. Nowadays digital street maps (e.g., those provided by Google Maps) are widely available and used routinely. Reuse of existing maps for robots eliminates extra cost of building maps. One of the difficulties in using existing street maps is data association between a robot's observation and the map, because the physical entities that correspond to the boundary lines in the map are unknown. We address this issue by using region annotations such as roads and buildings and prior knowledge. We introduce a probabilistic framework that simultaneously estimates a robot's position and the road's boundaries.We evaluated our method in complex urban environments. Our method successfully localized in environments that includes both roadways and pedestrian walkways.

This paper proposes a method of planar reconstruction for robotic mapping using image edge points. A major problem in image-based planar reconstruction is how to assign planes to non-textured regions. Our method samples seed points from the pixels and generates plane hypotheses using the image edge points surrounding the seed points. After pruning the plane hypotheses using visibility constraints and clustering, the optimal plane is assigned to each seed point using a graph cut algorithm. A seed point graph represents continuity constraints between seed points for the smoothness terms in the graph cut in order to correct false planes generated by occlusions. Our method can be applied to both non-textured and textured environments, and also it can reconstruct planes even from curved edges. Experiments show that our method successfully reconstructed planes in various environments.

We propose a novel localization method for outdoor mobile robots using High Dynamic Range (HDR) vision technology. To obtain an HDR image, multiple images at different exposures is typically captured and combined. However, since mobile robots can be moving during a capture sequence, images cannot be fused easily. Instead, we generate a set of keypoints that incorporates those detected in each image. The position of the robot is estimated using the keypoint sets to match measured positions with a map. We conducted experimental comparisons of HDR and auto-exposure images, and our HDR method showed higher robustness and localization accuracy.

This paper presents a 3D laser scanner that can gaze at an arbitrary region. By modulating the secondary rotation speed of roundly swinging 3D laser scanner, it gazes at a specific region and measures with high density measurement points. The proposed method uses a secondary rotation motor to control the measurement point density and no extra motor is required. The 3D scanner can keep the field of view of and scan cycle time while gazing at any region. The system is evaluated on experimental targets for different scenarios using a mobile robot.

This paper proposes a new localization method for outdoor navigation using a stereo camera only. Vision-based navigation in outdoor environments is still challenging because of large illumination changes. To cope with various illumination conditions, we use 2D occupancy grid maps generated from 3D point clouds obtained by a stereo camera. Furthermore, we incorporate salient line segments extracted from the ground into the grid maps. This grid map building is not much affected by illumination conditions. On the grid maps, the robot poses are estimated using a particle filter that combines visual odometry and map-matching. Experimental results showed the effectiveness and robustness of the proposed method under various weather and illumination conditions.

This paper presents a method of 3D localization using image edge-points detected from binocular stereo image sequences. The proposed method calculates camera poses using visual odometry, and updates the poses by reducing the accumulated errors using landmark recognition. Landmark recognition is done based on robust and scalable image-retrieval using image edge-points with SIFT descriptors and a vocabulary tree. A randomized-ICP algorithm is employed to accurately estimate the 6-DOF camera pose from a landmark image and an edge-point based 3D map. Experiments show our edge-point based approach outperforms approaches using corners and Laplacian points.

This paper proposes a light-weight sensor platform that consists of gyro-assisted odometry and a 3D laser scanner for localization of human-scale robots. The gyro-assisted odometry provides highly accurate positioning only by dead-reckoning. The 3D laser scanner has a wide field of view and uniform measuring-point distribution. Robust and computationally inexpensive localization is implemented on the sensor platform using a particle filter on a 2D grid map generated by projecting 3D points on to the ground. The system uses small and low-cost sensors, and can be applied to a variety of mobile robots in human-scale environments. Outdoor navigation experiments were performed at the Tsukuba Challenge 2009, which is an open proving ground for human-scale robots. Our robot successfully navigated the assigned 1-km course in a fully autonomous mode multiple times.

Most vision-based SLAM systems utilize corner-like features, and may be unstable in non-textured environments where only a few corner features can be extracted. To cope with this problem, we employ edge points to perform SLAM with a stereo camera. The edge-point based SLAM is applicable to non-textured environments since plenty of edge points can be obtained even from a small number of lines. The proposed method estimates camera poses and builds detailed 3D maps robustly by aligning edge points between frames using the ICP algorithm. In indoor experiments, the method successfully built detailed 3D maps even under noisy condition.

This paper presents a method of 3D mapping using a binocular stereo camera with automatic recovery from registration failure. We employ edge points as map element, and apply a variant of ICP algorithm to the inter-frame registration. The method detects failure in the registration due to erratic camera motion or moving objects and recovers from the failure by searching a good image to resume the registration using edge-point SIFT descriptors. In experiments, our method successfully coped with registration failure in 3D mapping in indoor and outdoor environments under various conditions.

Our goal is that the robot learns specific objects (not object category) from images. The major problem here is how to separate the target object from the background. We create a scene model from an image sequence. The scene model contains both the target object and background. We separate the target object from the background by matching the scene model and training images having different backgrounds. A scene model consists of a 3D model and 2D models. We utilize edge points to represent detailed object shape. The 3D model is composed of the 3D points reconstructed from image edge points using structure-from-motion technique. A 2D model consists of an image in the input image sequence, edge points in the image, and the camera pose from which the image was taken. Each edge point has a SIFT descriptor for edge-point matching. The scale space analysis is done to obtain scale-invariant edge points.

This paper presents a method of 3D SLAM using a single camera. We utilize a Rao-Blackwellised particle filter (RBPF) to deal with a large number of landmarks. A difficulty in monocular SLAM is robustness to outliers and noise, which may cause false estimates especially under short baseline conditions. We propose an exhaustive pose-space search that finds all the plausible hypotheses efficiently using epipolar geometry. The obtained pose hypotheses are refined by the RBPF. Simulations and experiments show that the proposed method successfully performed 3D SLAM with a small number of particles.

This paper introduces a planning method for handling furniture which exists in real world. We propose a method which is easily expandable its handle able furniture such as closet, shelf and so on. If the robot can handle such furniture autonomously, it is expected that multiple daily tasks, for example, storing a small object in a drawer, can be achieved by the robot. Because perplexing processes is needed to give the knowledge of furniture handling to the robot manually, we propose direct teaching based approach which can easily give not only how to handle the furniture but also an appearance and 3D shape of it. Combining general knowledge given in advance and manipulation procedure instructed by human directly, the robot acquires how to manipulate the storing places. The performance of the proposed method is illustrated by experiments.

This paper presents a method of dense 3-D object modeling based on surface interpolation with line segments. The proposed method first reconstructs edges from an image sequence, and then creates line segments by connecting edge points to cover the object surface. False segments are eliminated using visibility constraints. The method can reconstruct dense models since line segments can be created with plenty of edge points. We conducted experiments with objects in indoor environments and confirmed the effectiveness of the method

This paper describes a grasp planning for a mobile manipulator which works in real environment. Mobile robot studies up to now that manipulate an object in real world practically use ID tag on an object or an object model which is given to the robot in advance. The authors aim to develop a mobile manipulator that can acquire an object model through video images and can manipulate the object. In this approach, the robot can manipulate an unknown object autonomously. A grasp planning proposed in this paper can find a stable grasp pose from the automatically generated model which contains redundant data and the shape error of the object. Experiments show the effectiveness of the proposed method

This paper presents a method of object map building using object models created from image sequences captured by a single camera. Object map is a highly structured map, which is built by placing 3-D object models on the floor plane according to object recognition results. To increase the efficiency of object map building, we propose a framework to integrate dense shape and recognition features into an object model. Experimental results show that an object map of a room was built successfully using the proposed object models

This paper describes motion planning for a mobile manipulator which works in real environment. Mobile manipulators that can manipulate an object was developed up to now. We assume that object grasping is not restricted to just one grasp way and robot can move freely as far as it is in its operational area of the object. This paper proposes the motion planning for object grasping under the previous explained conditions. There are two major issues to detect the robot pose. The first problem is pose error of the robot hand. The second problem is kinematical redundancy of the mobile manipulator. The pose is evaluated from the amount of joint motions for recovery of hand pose error. Experiments show the effectiveness of the proposed method.

This paper presents a system of 3-D simultaneous localization and mapping (SLAM) using monocular vision-based on the structure-from-motion scheme. An crucial issue in applying structure-from-motion to SLAM is that accuracy depends heavily on the baseline distance. We address this problem by selecting an appropriate baseline based on criteria for the tradeoff between the baseline distance and the number of feature points visible in the images. Experimental results show that full 3-D sparse maps with camera trajectory were built from images captured with a handy camera.

This paper presents a method of improving the performance of global scan matching with a laser range finder using saliency-based scan point resampling. The proposed method calculates the saliency of each scan point according to the amount of information the scan point has, or the length of the line segment on which the scan point lies. Based on the saliency, the method preserves important scan points and discards redundant scan points. Experimental results show that the proposed method improves the efficiency and accuracy of global scan matching without degrading matching rates.

This work presents a new scan matching method for mobile robot localization and mapping. The proposed method is based on the geometric hashing scheme, which utilizes Euclidean invariant features in order to match an input scan with reference scans without an initial alignment The method is applicable to global localization in an environment having curved objects. Experimental results show that a map of a large cyclic environment was built with high accuracy using the proposed method.

This paper presents a method of building a 3-D map that consists of 3-D object models. First, we build a 2-D map from laser range finder (LRF) scans, and detect objects by matching a scan template with the 2-D map. Then, we verify the detected object candidates using vision-based object recognition based on the constraint that the estimated pose of a true positive must be consistent between scan matching and object recognition. We have conducted an experiment in a room having desks and cabinets to show the effectiveness of the method.

This paper describes 3-D shape modeling performed by a mobile robot. We propose a method in which the robot in the real environment acquires the accurate shape and constructs a 3-D model of an object using Computer Vision techniques under the condition that position and shape of the object is unknown. 3-D modeling is performed from image streams which are captured by a camera placed on the robot. The results show the effectiveness of our method.

This paper presents a global localization method in which the robot can relocalize itself when getting lost due to large slippage and kidnapping. In the proposed method, the robot generates pose hypotheses using a global scan matching method. The robot selects one hypothesis as the correct pose by filtering out false hypotheses using a multiple hypothesis tracking scheme. While the robot is tracking a single pose, the map is updated based on the SLAM framework. Experimental results show that the robot successfully localized itself robustly to disturbances including noises and kidnapping.

It is important and necessary to develop a teleoperated mobile robot to have reconnaissance operation inside of the extreme earthquake stricken building or underground town. The present authors aim at developing such a teleoperated mobile robot. In this paper, a prototype of the teleoperation and man-robot interface system is proposed. The tele-operated robot and the remote operation station is linked with wireless network and the operator can command the robot. One feature point of this system is to provide a live omni-directional panoramic view image and live map to the operator on the display of the remote operation station. The live map represents the floor plan of the moving environment and it is provided by SLAM technology with LRF scan data. It proves very effective to have combined use of the panoramic image and the live map. Experimental set up and integrated experiments are also reported in this paper.

This paper describes how to acquire 3-D shape of an object and to estimate robot pose by observing an unknown object. A single camera is placed on the robot, image streams of the target object are captured by the camera on a mobile robot moving around the object. Computer vision techniques are utilized under the condition that shape, size and accurate position information of the target object is not given in advance, but the robot finds the target object and reconstructs the 3-D object shape by itself. In addition, it is necessary that the robot grasps relative pose between itself and the target object. Experimental result show the effectiveness of our method in point of robustness and accuracy for the mobile robot.

This paper presents a method of building a structured map, which consists of objects such as furniture. We represent a map as a graph, in which a node represents an object and an arc represents a relative pose between objects. The robot localizes itself and builds a map using odometry readings and sensor data obtained by object recognition. To correct the map distortion caused by errors in the data, the robot utilizes loops as geometric constraints, and imports geometric knowledge provided by a hand-made map. Experiments show that the root successfully built a map of corridors, and a map of a room having many objects.

This paper presents a method of path planning for a mobile robot with a pan-tilt camera to find target objects under object-pose uncertainties caused by environmental changes or map inaccuracy. The uncertainties of object poses are represented by probability distributions on both a geometric map and a grid map. Based on these maps, the proposed method generates a weighted Voronoi graph, and plans a path with a short travel time, high passability, and high visibility of the target objects. The total path is the concatenation of paths to reach a goal and paths to observe a target object. The paper proposes a two-layered planning scheme, in which the candidate paths between subgoals are planned on the lower layer, and they are integrated on the upper layer.

This paper presents a navigation system based on an inaccurate map. The map we consider here is a hybrid of a topological map and a geometrical map so that it can be built easily and permits changes in object locations. A localization scheme and an object recognition technique are introduced in order to keep the robot pose consistent on such an inaccurate map. To cope with the problem that a path may also be inaccurate on an inaccurate map, the robot corrects the path based on its pose estimated by the localization method while tracking the path. An experiment shows that the robot successfully navigated in an indoor environment, recognizing several kinds of objects.

Proposes a method of global localization on a map with large uncertainty. Since the pose of a mobile robot represented in a global frame of reference might be inconsistent on an inaccurate map, the method represents the robot pose in a local frame attached to a landmark. When the robot finds a new landmark, the robot transforms its pose from the old landmark frame to the new one based on the relative pose between the landmarks. Map errors are represented by probability density functions defined on these relative poses. The paper presents an extension of Markov localization which is augmented so as to incorporate the map errors into data fusion process, and shows experimental results.

Navigation in unknown environments requires the robot to obtain the destination positions without a map. The utilization of model-based object recognition would be a solution, where the robot can estimate the destination positions from geometric relationships between the recognized objects and the robot. This paper presents a robot system for this kind of navigation, in which the robot navigates itself to the room designated by room number. The robot has an environment model including a corridor and a door with a room number plate, and utilizes the model for the efficient recognition of the objects and the estimation of their positions.

A geometric map which quantitatively represents the locations of entities in the environment are widely employed for indoor navigation. Navigation based on a geometric map provides high accuracy in positioning, but making an accurate map for a large area is difficult. To cope with this problem, this paper proposes a framework of indoor navigation based on a partially quantitative map. The map comprises the models of geometric entities in the environment, the locations of which are partially defined. Using the partial information, the robot searches and recognizes entities as a landmark while navigating in the environment.

Event notification is essential in network management. Many components in a distributed NMS need an event notification mechanism, and so an event notification framework is expected to facilitate the development of the components. CORBA is a promising platform to build such a framework, but its performance is not sufficient for monitoring large-scale networks. This paper proposes an event notification framework that has high performance and customizability. The framework is based on Java and CORBA, and has customization points at which event handling capabilities can be added. Java gives the framework flexibility in the sense that event handling capabilities call be added and replaced dynamically by using downloadable Java programs. High throughput of more than 1000 events per second has been achieved by event batching.

An animation system has been developed which generates animations from stories written in natural language. The system consists of three modules: story understanding module, stage directing module and action generating module.