In conventional imaging, the geometry of image formation is described by the pinhole camera model, and it allows multiview reconstruction based on the perspective projection. If a light ray passes through different media, however, the pin-hole camera model becomes invalid due to refraction, causing changes in direction. In this paper, we present a method for 3D trajectory reconstruction under refraction at a cylindrical surface. In the proposed method, the refractive geometry is represented by a geometric mapping from image points to refracted rays, instead of physically modeling it. 3D points inside the cylindrical surface are estimated by computing intersection points between segments of corresponding refracted rays in multiview. In our experiments, we demonstrate that 3D trajectories of multiple moving objects inside a transparent cylinder, which is filled with water, are successfully recovered using multiple cameras.

This paper presents a method for interactive deformation of a real object. Our method uses a predefined 3D model of a target object for tracking and deformation. A camera pose relative to the target object is estimated using 3D model-based tracking. Object region in camera image is obtained by projecting the 3D model onto image plane with the estimated camera pose, and a texture map is extracted from the object region and mapped to the 3D model. Then a texture-mapped model is rendered based on a mesh deformed by user via Laplacian operation. Experimental results demonstrate that our method provides user interactions with 3D real objects on real scenes, not augmented virtual contents.

Edge-based tracking is a fast and plausible approach for textureless 3D object tracking, but its robustness is still very challenging in highly cluttered backgrounds due to numerous local minima. To overcome this problem, we propose a novel method for fast and robust textureless 3D object tracking in highly cluttered backgrounds. The proposed method is based on optimal local searching of 3D-2D correspondences between a known 3D object model and 2D scene edges in an image with heavy background clutter. In our searching scheme, searching regions are partitioned into three levels (interior, contour, and exterior) with respect to the previous object region, and confident searching directions are determined by evaluating candidates of correspondences on their region levels; thus, the correspondences are searched among likely candidates in only the confident directions instead of searching through all candidates. To ensure the confident searching direction, we also adopt the region appearance, which is efficiently modeled on a newly defined local space (called a searching bundle). Experimental results and performance evaluations demonstrate that our method fully supports fast and robust textureless 3D object tracking even in highly cluttered backgrounds.

This paper presents 3-D visual tracking on a mobile phone, which is one of the next promising challenges of visual tracking for mobile augmented reality applications. Our approach is based on model-based tracking using a 3-D model of an object on a target scene. A camera pose is estimated by iteratively updating infinitesimal motions of a camera, explicitly represented by 3-D rigid-body transformations. The 3-D visual tracking is implemented on recent Android smartphones. Its feasibility is shown by experimental results and discussed along with its performance. To our knowledge, this is the first work that demonstrates the 3-D model-based tracking on a mobile phone.

Abstract Model‐based camera tracking is a technology that estimates a precise camera pose based on visual cues (e.g., feature points, edges) extracted from camera images given a 3D scene model and a rough camera pose. This paper proposes an automatic method for flexibly adjusting the confidence of visual cues in model‐based camera tracking. The adjustment is based on the conditions of the target object/scene and the reliability of the initial or previous camera pose. Under uncontrolled or less‐controlled working environments, the proposed object‐adaptive tracking method works flexibly at 20 frames per second on an ultra mobile personal computer (UMPC) with an average tracking error within 3 pixels when the camera image resolution is 320 by 240 pixels. This capability enabled the proposed method to be successfully applied to a mobile augmented reality (AR) guidance system for a museum. Copyright © 2009 John Wiley & Sons, Ltd.

In projection-based augmented reality (AR) alleviating visual distraction of patterns has been a great challenge. As a representative one, a method of embedding patterns and their complements (hereafter, we call the pairs complementary patterns) into AR images alternately has been proposed recently. This paper presents subjective evaluation results and their statistical analysis on the visual perceptibility of embedding complementary patterns in different ways in a standard hardware environment. Then, we explore the constraints for embedded complementary patterns to be less perceptible. As expected, high projector refresh rate and low pattern strength were the general conditions for a decrease in the perception of embedded complementary patterns. However, reducing pattern size and projecting complementary patterns with an interval were also among the factors affecting the results. Detailed constraints are given in the experimental results. Also, we present which constraint is more dominant for pattern perceptibility.

We propose a nonintrusive method for generating all-focused projection. Unlike traditional restoration algorithms that operate on a blurred image to recover the original, the nature of our problem requires that the correction be applied to the original image before blurring. To accomplish this, a pattern image is generally projected and a series of spatially varying point-spread-functions (PSFs) across the projection input image were estimated from its camera image. In this paper, the pattern image was imperceptibly projected and thus not visible to the observers. Experimental results demonstrate that the proposed method effectively removes the out-of-focus projection blur in dynamic environments where the projector or screen object moves.

Existing geometric and radiometric compensation methods for direct-projected augmented reality focus on static projection surfaces rather than dynamic surfaces (with varying geometry in time). We aim at providing an effective framework for projecting a sequence of augmented reality images onto dynamic surfaces without geometric and radiometric distortion. We present our design of a special pattern image for simultaneous geometric and radiometric compensation and evaluate two different techniques for embedding the pattern image into augmented reality images. The validity of the proposed method is examined through a variety of experiments with a mobile projector-camera system.