We propose a hybrid camera pose estimation method using an inclination sensor value and correspondence-free line segments. In this method, possible azimuths of the camera pose are hypothesized by a voting method under an inclination constraint. Then some camera positions for each possible azimuth are calculated based on the detected line segments that affirmatively voted for the azimuth. Finally, the most consistent one is selected as the camera pose out of the multiple sets of the camera positions and azimuths. Unlike many other tracking methods, our method does not use past information but rather estimates the camera pose using only present information. This feature is useful for an initialization measure of registration in augmented reality (AR) systems. This paper describes the details of the method and shows its effectiveness with experiments in which the method is actually used in an AR application.

This paper describes an evaluation system using an industrial robot, constructed for the purpose of evaluating registration technology for Mixed Reality. In this evaluation system, the tip of the robot arm plays the role of the user's head, where a head- mounted display is mounted. By using an industrial robot, we can obtain the ground truth of the camera pose with a high level of accuracy and robustness. Additionally, we have the ability to play back the same specified operations repeatedly under identical conditions. In addition to the system implementation, we propose evaluation methods for motion robustness, relative orientation robustness, relative distance robustness, jitter, and an overall evaluation. We verify the validity of this system through some experiments.

This work describes a new hybrid vision-based registration method with an inclination sensor. In the method, a camera is tracked by solving linear equations under inclination constraint. Linear operations are faster than the nonlinear optimization process, but the output does not usually satisfy the orthonormality constraint. On the contrary, the proposed method calculates the camera position and azimuth directly, thus the result satisfies the orthonormality constraint. Many hybrid approaches using inertia sensors have been proposed for AR/MR. However, such methods still depend on vision-based methods in initialization processes. On the other hand, the proposed method is totally hybrid in that the inclination measured by the sensor is always incorporated in pose calculation process as well as vision information. The method can be used in initialization process of conventional hybrid methods as well as it can be used as an independent registration method. The proposed method can be applied to not only inside-out-style camera tracking but also outside-in-style object tracking.

This paper describes a new head tracking method utilizing a gyroscope mounted on a head-mounted display (HMD) and a bird's-eye view camera that observes the HMD from a fixed third-person viewpoint. Furthermore, we propose an extension of this method to hybrid registration, combining it with user's view cameras. The HMD is equipped with a gyroscope and a marker. The gyroscope measures the orientation of the user s view camera so that the number of pose parameters to be solved can be reduced. The other parameters are to be estimated by the bird's-eye view camera that observes the marker. This method is an improvement over the conventional outside-in-style vision-based tracker method, which only uses visual information. Hence, it can be thought of as an alternative to a physical head tracker such as a magnetic sensor and to an inside-out-style vision-based tracker. In addition to theoretical discussions, this paper demonstrates the effectiveness of our methods by experiments. We also propose methods for calibrating the gyroscope and the marker on HMD, which are essential in implementing the tracking method.

This paper describes new vision-based registration methods utilizing not only cameras on a user's head-mounted display but also a bird's-eye view camera that observes the user from an objective viewpoint. Two new methods, the line constraint method (LCM) and global error minimization method (GEM), are proposed. The former method reduces the number of unknown parameters concerning the user's viewpoint by restricting it to be on the line of sight from the bird's-eye view. The other method minimizes the sum of errors, which is the sum of the distance between the fiducials on the view and the calculated positions of them based on the current viewing parameters, for both the user's view and the bird's-eye view. The methods proposed here reduce the number of points that should be observed from the user's viewpoint for registration, thus improving the stability. In addition to theoretical discussions, this paper demonstrates the effectiveness of our methods by experiments in comparison with methods that use only a user's view camera or a bird's-eye view camera.

This paper describes a platform package, called "MR Platform," which we have been implementing for research and development of augmented reality technology and applications. This package includes a parallax-less stereo video see-through HMD and a software development kit (SDK) for a Linux PC environment. The SDK is composed of a C++ class library for making runtime MR applications and related utilities such as a camera calibration tool. By using the SDK, the following functions are available: capturing video, handling a six degree-of-freedom (DOF) sensor, image processing such as color detection, estimating head position and orientation, displaying the real world image, and calibrating sensor placement and camera parameters of two cameras mounted on the HMD.

In stereo algorithms with more than two cameras, the improvement of accuracy is often reported since they are robust against noise. However, another important aspect of the polynocular stereo, that is the ability of occlusion detection, has been paid less attention. We intensively analyzed the occlusion in the camera matrix stereo (SEA) and developed a simple but effective method to detect the presence of occlusion and to eliminate its effect in the correspondence search. By considering several statistics on the occlusion and the accuracy in the SEA, we derived a few base masks which represent occlusion patterns and are effective for the detection of occlusion. Several experiments using typical indoor scenes showed quite good performance to obtain dense and accurate depth maps even at the occluding boundaries of objects.

In stereo algorithms with more than two cameras, the improvement of accuracy in correspondence search is often reported. On the other hand, another important aspect of the polynocular stereo is the ability of occlusion detection. The camera matrix stereo SEA, which we have developed, offers a simple but effective framework to detect the presence of occlusion and to obtain reliable correspondence. SEA can produce a dense and accurate depth map with sharp object profiles. In this paper, we made some systematic comparison of several algorithms for occlusion detection in SEA. The results are quite interesting and reasonable. They are useful to design an actual polynocular stereo system.