In this study, we present a method for all-around depth estimation from multiple omnidirectional images for indoor environments. In particular, we focus on plane-sweeping stereo as the method for depth estimation from the images. We propose a new icosahedron-based representation and ConvNets for omnidirectional images, which we name "CrownConv" because the representation resembles a crown made of origami. CrownConv can be applied to both fisheye images and equirect- angular images to extract features. Furthermore, we propose icosahedron-based spherical sweeping for generating the cost volume on an icosahedron from the extracted features. The cost volume is regularized using the three-dimensional CrownConv, and the final depth is obtained by depth regression from the cost volume. Our proposed method is robust to camera alignments by using the extrinsic camera parameters; therefore, it can achieve precise depth estimation even when the camera alignment differs from that in the training dataset. We evaluate the proposed model on synthetic datasets and demonstrate its effectiveness. As our proposed method is computationally efficient, the depth is estimated from four fisheye images in less than a second using a laptop with a GPU. Therefore, it is suitable for real-world robotics applications. Our source code is available at https://github.com/matsuren/crownconv360depth.

This paper presents a dense method of real-time registration of RGB-D image pair. So far, we have proposed the “see-through system”, in which multiple images acquired from RGB-D sensors are integrated to present images that is useful for confirming the shape or the positions of objects behind obstacles. However, errors of positional relation of sensors result in see-through images in which some objects are doubled or appear at incorrect positions. It is difficult to align the images because they are captured from distant viewpoints and there are few shared field of view. In the proposed method, positional relation of two RGB-D sensors is corrected using a new IRLS framework, fast and robust minimization strategy.

Conventional techniques for frame-to-frame camera motion estimation rely on tracking a set of sparse feature points. However, images taken from spherical cameras have high distortion which can induce mistakes in feature point tracking, offsetting the advantage of their large fields-of-view. Hence, in this research, we attempt a novel approach of using dense optical flow for distortion-robust spherical camera motion estimation. Dense optical flow incorporates smoothing terms and is free of local outliers. It encodes the camera motion as well as dense 3D information. Our approach decomposes dense optical flow into epipolar geometry and the dense disparity map, and reprojects this disparity map to estimate 6 DoF camera motion. The approach handles spherical image distortion in a natural way. We experimentally demonstrate its accuracy and robustness.

Localization is an important task for mobile service robots in indoor spaces. In this research, we propose a novel technique for indoor localization using a spherical camera. Spherical cameras can obtain a complete view of the surroundings allowing the use of global environmental information. We take advantage of this in order to estimate camera position and the orientation with respect to a known 3D line map of an indoor environment, using a single image. We robustly extract 2D line information from the spherical image via spherical-gradient filtering and match it to 3D line information in the line map. Our method requires no information about the 3D-2D line correspondences. In order to avoid a complicated six degrees of freedom (6 DoF) search for position and orientation, we use a Manhattan world assumption to decompose the line information in the image. The 6 DoF localization process is divided into two phases. First, we estimate the orientation by extracting the three principle directions from the image. Then, the position is estimated by robustly matching the distribution of lines between the image and the 3D model via a spherical Hough representation. This decoupled search can robustly localize a spherical camera using a single image, as we demonstrate experimentally.

This paper introduces a new methodology of robotic hammering inspection for the maintenance of social infrastructures. In particular, the estimation of material defect conditions, such as delamination depth of concrete, is focused upon. Development of an automated diagnosis methodology is necessary for the maintenance of superannuated social infrastructures. The hammering test, which is an efficient inspection method, has attracted considerable attention in the context of automated inspection using robots. In this study, to apply the hammering test to robotic inspection, in which material conditions of infrastructures must be diagnosed in detail, an estimation method of the defect conditions is proposed, and an integration technique of plural classifiers for improving the inspection accuracy is introduced. Furthermore, an inspection system that can decrease the influence of the mechanical running-noise is implemented. Our experimental results using concrete test pieces demonstrate the effectiveness of the proposed method; the accuracy of the defect detection and defect condition estimation was validated.

Three-dimensional (3D) measurement is an important means for robots to acquire information about their environment. Structure from Motion is one of these 3D measurement methods. The 3D reconstruction of objects in the environment can be obtained from pictures captured with single camera in Structure from Motion. Furthermore, the camera motion can be obtained simultaneously. Because of its simplicity, Structure from Motion has been implemented in various ways. However there is an essential problem in that the scale of the measured objects cannot be computed by Structure from Motion. In order to compute the absolute scale, other information is required. However this is difficult for robots in an unknown situation. In this paper, we propose a method that can reconstruct the absolute scale of objects using refraction. Refraction changes the light ray path between the objects and the camera. This method is implemented using only a refractive plate and single camera. The results of simulations show the effectiveness of the proposed method in both air and other media (e.g., water).

An automated diagnosis methodology is necessary for the maintenance of superannuated social infrastructures. In this context, the hammering test is an efficient inspection method, and it has been widely used because of the resulting accuracy and efficiency of operation. While robotic automation of the hammering inspection method is highly desirable, the development of an automatic diagnostic algorithm that can operate at actual inspection sites is essential. Furthermore, portability of the diagnostic algorithm is also highly desirable. In this study, in order to construct reliable detectors and to improve their portability for the performance of the hammering test, we propose a boosting-based defect detector that is robust against variations in environmental conditions. In particular, we present the construction of a noise-robust classifier with a refinement of the feature values extracted from hammering sounds and an updating rule of template vectors of its evaluation function. Our experimental results in a concrete tunnel demonstrate the effectiveness of the proposed method; the accuracy of the classifier at an actual site and adaptivity to environmental noise are confirmed.

We propose a method for generating robot motions by integrating multiple policies effective for task achievement. The method obtains a new policy efficiently based on the applied policies. However, there might be some states in which the applied policies fail to achieve the task. The failing states are found by means of a decrease in the state values. The policies for the states are then modified. In this paper, we applied this method to clutch assembly in order to demonstrate its validity for assembly tasks. We integrated insertion motion and search motion for the task and finally obtained the effective motion that accords to the task states.

Parameter tuning of force control is important for successful robotic assembly and ensuring high efficiency operations. In this paper, we present a method of designing damping control parameters for general assembly operations by considering the cycle time. In this method, optimal parameters are obtained through iterative simulations of assembly operations because it is difficult to estimate the cycle time analytically. We have applied the method to clutch assembly, which is a complicated insertion of a splined axis into movable toothed plates, and demonstrate how the operation can be sped up using the obtained parameters.