A new convenient calibration algorithm is proposed for unsynchronized multi-camera networks having large capture volume. The proposed method gives a simple and accurate calibration mean using a small 3D reference object. Extrinsic and intrinsic parameters are recovered simultaneously by capturing the object placed arbitrarily in different locations in the capture volume. The proposed method first solves the problem linearly with factorizing projection matrices into the camera and the object pose parameters. Due to the multi-view constraints imposed on the factorization, the consistency of rigid transformations among cameras and objects can be acquired. These consistent estimation results can be further refined by a non-linear optimization process. The proposed algorithm is evaluated via simulated and real experiments to verify that it is more efficient than previous methods.

This paper proposes a novel framework using geometric information on parallelograms for camera calibration and scene modeling. The proposed method solves the problem linearly with factorizing a measurement matrix into the camera and plane parameters. The measurement matrix contains canonic planar homographies between camera views and planes on which arbitrary parallelograms lie. Since the factorization based approaches recover all camera poses simultaneously, the consistency of rigid transformations among cameras can be ensured. The previous parallelogram-based approaches solving the problem linearly cannot guarantee this consistency because they are based on the infinite homographies extracted from the individual pairs of camera images. The proposed method is more useful than the previous factorization based approach using parallelepipeds because parallelograms are more basic scene elements. The results of the experiments with real outdoor images are presented to demonstrate the feasibility of the proposed method.

To insert a virtual object into a real image, the position of the object must appear seamlessly as the camera moves. This requires camera tracking with estimations of all internal and external parameters in each frame with an adequate degree of stability to ensure negligible visible drift between the real and virtual elements. In the post production of film, matchmoving software based on SfM is typically used in the camera tracking process. However, most of this type of software fails when attempting to track the camera in a dynamic scene in which a moving foreground object such as a real actor occupies a large part of the background. Therefore, this study proposes a camera tracking system that uses an auxiliary camera to estimate the motion of the main shooting camera and 3D position of background features in a dynamic scene. A novel reconstruction and connection method was developed for feature tracks that are occluded by a foreground object. Experimentation with a 2K sequence demonstrated the feasibility of the proposed method.

There have been many approaches to obtain 3D modeling and camera calibration simultaneously from uncalibrated images using parallelism, orthogonality and self-calibration constraints. These approaches can give more stable results with fewer images and allow us to gain the results with only linear operations in most cases. It has been proved that the estimation results are accurate enough to be used as the initial values for nonlinear optimization to refine the results. In this paper, it is shown that all the linear constraints used in the previous works performed independently up to now can be implemented easily in the proposed linear method. The proposed method uses a stratified approach, in which affine reconstruction is performed first and then metric reconstruction. In this procedure, the additional constraints newly extracted in this paper have an important role for affine reconstruction in practical situations. The study on the situations that can not be dealt with by the previous approaches is presented and it is shown that the proposed method being able to handle the cases is more flexible in use.

A novel method is introduced to estimate the camera parameters when there is no useful information in the scene but in which two or more parallelograms exist. First, the proposed method acquires the infinite homography by rectification of the imaged parallelograms instead of the general method that uses vanishing point correspondences directly. Due to this peculiar approach, the additional constraints that arise from the parallelograms can be acquired. Second, the infinite homography estimated together with the additional constraints is applied to the calibration of a camera. Owing to the effect of the additional constraints, more accurate estimation results can be acquired. Moreover, even if the number of vanishing points induced from parallelograms is three, the infinite homography and, consequently, camera parameters can be computed without the need for an estimation of the fundamental matrix that requires many point correspondences between the views for accuracy.

This paper deals with structure and motion estimation by using image sequences captured by a multi-camera system moving in a scene. Single camera systems with a relatively small field of view have limited accuracy because of the inherent confusion between translation and rotation. This is also the case for the stereo camera systems if the systems need the intersection of the fields of view for stereo correspondence. The cameras constituting the multi-camera system considered ultimately in this paper are arranged so that there is no, or a small, intersection of the fields of view. This configuration improves the accuracy of the structure and motion estimation because the ambiguity mentioned above decreases due to a large field of view. We propose a recursive algorithm for real time applications of the multi-camera system using long term image sequences, which have appearing and disappearing point features. The results of the experiments with long term real image sequences are presented to demonstrate the feasibility of the algorithm. 1

This paper presents a vision based SLAM method by using stereo SFM technique. The proposed method is based on the stereo SFM presented in our former paper. The method do not need stereo correspondence but, nevertheless, can determine absolute scale factor, which is an important factor for long term navigation and SLAM. The method use only motion correspondence basically, which is easier to solve than stereo correspondence because the SFM algorithm can use a sequence of images taken at short time intervals. However, we infer the stereo correspondence inversely from the absolute estimates of structure and motion parameters and utilize this information to improve the performance of our method. Consequently, the method maintain the robustness to the stereo correspondence ambiguity and can avoid the degenerate configuration reported in the former paper. We also propose a simple initialization technique for the proposed method based on extended Kalman filter, which is critical issue for the methods using bearing-only measurements. The experimental results demonstrate the effectiveness of the algorithm.

The stereo SFM combining stereo and motion is a useful vision cue to estimate both the motion of the cameras and the structure of a scene with an absolute scale factor. Most of the works related to the stereo SFM have used both motion correspondence and stereo correspondence. Because it is information redundancy to use these two correspondence simultaneously, they can give accurate results. However, the methods to solve the stereo correspondence problem are not yet reliable enough to have been used in practical applications, while the motion correspondence problem is easy to solve. Therefore, if an absolute scale factor can be acquired by only using motion correspondence without stereo correspondence, it is not necessary to use stereo correspondence for accuracy at the cost of reliability. This paper shows that the scale factor can be determined by using only motion correspondence. Moreover, it is shown that the degenerate cases reported in this paper are all the cases and can be easily avoided. This fact is verified from the two theorems and the simulation results. This paper also describes an implementation of the proposed algorithm. This implementation uses multiple images acquired continuously when a stereo camera moves in a scene and is formulated to be a recursive form for real-time applications. Some peculiar implementation techniques contribute to fast convergence rate and good stability in spite of large initial uncertainty. The simulation and experimental results demonstrate the effectiveness of the algorithm.

Vision sensors are attractive equipments for an autonomous mobile robot because they are information-rich and rarely have restrictions in range and various applications. However, there exists a difficult problem in using vision sensors to reconstruct geometrical information, which is a correspondence or matching problem. We present a simultaneous localization and map building (SLAM) algorithm for an autonomous mobile robot based on a vision sensor robust to the correspondence or matching problem with cooperation of structure from motion (SFM) and stereo. We use an omni-directional stereo vision sensor, which is suitable for SLAM. Due to its large field of view, we can acquire robust estimates and reduce the effect of motion drift from the fiducial coordinate frame. Results on real images illustrate the performance of the proposed method.

Describes a method to construct a stationary environmental map and estimate the ego-motion of a mobile robot from unknown planar motion by using an omni-directional vision sensor. Most environments where a mobile robot works are limited to two-dimensional space and the environmental map which is necessary for mobile robot navigation has also two dimensions. However conventional "structure from motion (SFM)" algorithms cannot be applied to two-dimensional space in perspective projection. We propose a SFM algorithm that can be applied to two-dimensional space. The proposed SFM algorithm exploits the azimuths of features which are obtained from an omni-directional vision sensor and gives robust results against the noise of image information by taking advantage of the large field of view. A relation between observed azimuths and motion parameters of a robot are constrained by a nonlinear equation and our method obtains all the motion parameters and an environmental map through a two-step procedure of solving the equation.