Keystone correction is an essential operation for projector-based applications, especially in mobile scenarios. In this paper, we propose a handheld movable projection method that can freely project keystone-free content on a general flat surface without adding any markings or boundary on it. Such a projection system can give the user greater freedom of display control (such as viewing angle, distance, etc.), without suffering from keystone distortion. To achieve this, we attach a camera to the projector to form a camera-projector pair. A green frame with the same resolution as the projector screen is projected onto the screen. Particle filter is employed to track the green frame and the correction of the display content is then achieved by rectifying the projection region of interest into a rectangular area. We built a prototype system to validate the effectiveness of the method. Experimental results show that our method can continuously project distortion free content in real time with good performance.

In this paper, we propose a hand-held mobile projection system that can freely project keystone free content on a general flat surface without any markings. Such a projection system can give the user great freedom of control of the display such as viewing angle and distance without suffering from distortion. We attach a camera to the projector to form a stereo pair. The correction of the display content is then achieved by rectifying the projection region of interest into a rectangular region and pre-warping the display content. Experimental results show that our system can continuously project distortion free content in real time with reasonable precision.

In this paper, we proposed a movable hand-held display system which uses a projector to project display content onto an ordinary cardboard which can move freely within the projection area. Such a system can give users greater freedom of control of the display such as the viewing angle and distance. At the same time, the size of the cardboard can be made to a size that fits one's application. A projector-camera pair is calibrated and used as the tracking and projection system. We present a vision based algorithm to detect an ordinary cardboard and track its subsequent motion. Display content is then pre-warped and projected onto the cardboard at the correct position. Experimental results show that our system can project onto the cardboard in reasonable precision.

The extended Kalman filter (EKF) is suitable for real-time pose estimation due its low computational demand and ability to handle the nonlinear perspective camera model. There are many EKF based approaches in the literature; some are very recent while others exist for about two decades. These methods differ in two main aspects: the number and arrangement of cameras, and the number and usage of filters. In this work, we will compare these approaches using simulations and real experiments. As far as we know, it is the first attempt to do this with such details. We will show which is suitable under different motion patterns, and explain the effect of the bas-relief ambiguity upon the accuracy of the different approaches. Additionally, we will discuss how to solve the scale factor ambiguity, and suggest the best strategy to deal with the features fed to the filter.

Processing of stereo images has become more and more important in recent years because of the availability of various stereo displaying devices. In particular, stabilizing of stereo images is important and useful especially when the images are obtained from cameras held by inexperienced hands or placed on unstable platforms. In this paper, we propose a new frame warping method for such a problem. Most video stabilization methods to date use 2-D geometric transform to approximate the changes between frames. However, these methods fail when there is a large depth variation between the foreground and the background in the scene. We try to solve this by estimating the 3-D motion parameters of the cameras by tri-focal tensor and the extended Kalman filter. And use the motion parameters to stabilize the images. We test the method using synthetic and real images and the results show that the performance of our proposed method is accurate even if the background contains large relative depth variations.

In this work, we solve the pose estimation problem for robot motion by placing multiple cameras on the robot. In particular, we use four cameras arranged as two back-to-back stereo pairs combined with the extended Kalman filter (EKF). The reason for using multiple cameras is that the pose estimation problem is more constrained for multiple cameras than for a single camera. Back-to-back cameras are used since they provide more information. Stereo information is used in self initialization and outlier rejection. Different approaches to solve the long-sequence-drift have been suggested. Both the simulations and the real experiments show that our approach is fast, robust, and accurate.

Traditional vision-based 3-D motion estimation algorithms for robots require given or calculated 3-D models while the motion is being tracked. We propose a high-speed extended-Kalman-filter-based approach that recovers position and orientation from stereo image sequences without prior knowledge as well as the procedure for the reconstruction of 3-D structures. Empowered by the use of the trifocal tensor, the computation step of 3-D models can be eliminated. The algorithm is thus more flexible and can be applied to a wide range of domains. The twist motion model is also adopted to parameterize the 3-D motion such that the motion representation in the proposed algorithm is robust and minimal. As the number of parameters to be estimated is reduced, our algorithm is more efficient, stable and accurate compared to traditional approaches. The proposed method has been verified using a real image sequence with ground truth.

Resolution improvement from several images is typically restricted to simple planar rotations and translations. In this paper, a super-resolution algorithm that allows 3D ego-motion of the camera system is proposed. By exploiting the trifocal tensor constraint of a stereo camera system, the intermediate step of scene structure recovery is effectively skipped. 3D motion is estimated recursively through an extended Kalman filter and used by a novel image warping procedure to perform resolution enhancement. Real image experiment with comparison to well known Irani-Peleg approach confirms the validity of the algorithm.

An innovative extended Kalman filter (EKF) algorithm for pose tracking has been proposed in this paper. It has the advantages of both structure and motion-based (SAM-based) and traditional model-based pose estimation algorithms. With no prior information about the scene, the pose sequence can be computed directly from images while the updating of the 3-D structure is not necessary. To achieve the goal, a constant velocity motion model is used as the dynamic system and the trifocal tensor point transfer function is applied to the measurement model of the filter. The resulting algorithm is stable, accurate and efficient. An empirical comparison with existing EKFs which deal with the same problem has been made and shows that our approach outperformed the others. The proposed method has been tested with various video sequences to demonstrate its performance in real situations.

Inserting synthetic objects into video sequences has gained much interest in recent years. Fast and robust vision-based algorithms are necessary to make such an application possible. Traditional pose tracking schemes using recursive structure from motion techniques adopt one Kalman filter and thus only favor a certain type of camera motion. We propose a robust simultaneous pose tracking and structure recovery algorithm using the interacting multiple model (IMM) to improve performance. In particular, a set of three extended Kalman filters (EKFs), each describing a frequently occurring camera motion in real situations (general, pure translation, pure rotation), is applied within the IMM framework to track the pose of a scene. Another set of EKFs,one filter for each model point, is used to refine the positions of the model features in the 3-D space. The filters for pose tracking and structure refinement are executed in an interleaved manner. The results are used for inserting virtual objects into the original video footage. The performance of the algorithm is demonstrated with both synthetic and real data. Comparisons with different approaches have been performed and show that our method is more efficient and accurate.

In this paper, an innovative extended Kalman filter (EKF) algorithm for pose tracking using the trifocal tensor is proposed. In the EKF, a constant-velocity motion model is used as the dynamic system, and the trifocal-tensor constraint is incorporated into the measurement model. The proposed method has the advantages of those structure- and-motion-based approaches in that the pose sequence can be computed with no prior information on the scene structure. It also has the strengths of those model-based algorithms in which no updating of the three-dimensional (3-D) structure is necessary in the computation. This results in a stable, accurate, and efficient algorithm. Experimental results show that the proposed approach outperformed other existing EKFs that tackle the same problem. An extension to the pose-tracking algorithm has been made to demonstrate the application of the trifocal constraint to fast recursive 3-D structure recovery.

Finding the pose and structure of an unknown object from an image sequence has many applications in graphics, virtual reality, and multimedia processing. In this paper, we address this problem by using a two-stage iterative method. Starting from an initial guess of the structure, the first stage estimates the pose of the object. The second stage uses the estimated pose information to refine the structure. This process is repeated until the difference between the observed data and data re-projected from the estimated model is minimized. This method is a variation of the classical bundle adjustment method, but is faster in execution and is simpler to implement. We used the Kanade-Lucas-Tomasi feature tracker for obtaining the image features. Synthetic and real data have been tested with good results.

This paper describes a genetic algorithm that tackles the pose-estimation problem in computer vision. Our genetic algorithm can find the rotation and translation of an object accurately when the three-dimensional structure of the object is given. In our implementation, each chromosome encodes both the pose and the indexes to the selected point features of the object. Instead of only searching for the pose as in the existing work, our algorithm, at the same time, searches for a set containing the most reliable feature points in the process. This mismatch filtering strategy successfully makes the algorithm more robust under the presence of point mismatches and outliers in the images. Our algorithm has been tested with both synthetic and real data with good results. The accuracy of the recovered pose is compared to the existing algorithms. Our approach outperformed the Lowe's method and the other two genetic algorithms under the presence of point mismatches and outliers. In addition, it has been used to estimate the pose of a real object. It is shown that the proposed method is applicable to augmented reality applications.

A recursive two-step method to recover structure and motion from image sequences based on Kalman filtering is described in this paper. The algorithm consists of two major steps. The first step is an extended Kalman filter (EKF) for the estimation of the object's pose. The second step is a set of EKFs, one for each model point, for the refinement of the positions of the model features in the three-dimensional (3-D) space. These two steps alternate from frame to frame. The initial model converges to the final structure as the image sequence is scanned sequentially. The performance of the algorithm is demonstrated with both synthetic data and real-world objects. Analytical and empirical comparisons are made among our approach, the interleaved bundle adjustment method, and the Kalman filtering-based recursive algorithm by Azarbayejani and Pentland. Our approach outperformed the other two algorithms in terms of computation speed without loss in the quality of model reconstruction.

A robust simultaneous pose tracking and structure recovery algorithm based on the Interacting Multiple Model (IMM) for augmented reality applications is proposed in this paper. A set of three extended Kalman filters (EKFs), each describes a frequently occurring camera motion in real situations (general, pure translation, pure rotation), is applied within the IMM framework to track the pose of an object. Another set of EKFs, one filter for each model point, is used to refine the positions of the model features in the 3D space. The filters for pose tracking and structure refinement are executed in an interleaved manner. The results are used for inserting virtual objects into the original video footage. The performance of the algorithm is demonstrated with both synthetic and real data. Comparisons with different approaches have been performed and show that our method is more efficient and accurate.

A stroke order and number free method for on-line recognition of Chinese characters is proposed. Both input characters and the model characters are represented with complete attributed relational graphs (ARGs). The ARGs of the model base are built according to the human knowledge of the segment relation structure of Chinese characters. For the recognition purpose, an optimal matching measure between two ARGs is defined, and the graph matching is formulated as a search problem of finding the minimum cost path in a state space tree, using the A * algorithm. Moreover, to reduce the search time of the A *, besides a heuristic estimate, a novel strategy is employed which again uses the human knowledge of the segment position structure of Chinese characters to prune the tree. Our experimental results demonstrate the efficiency of the proposed method.

A structural method for on-line recognition of Chinese characters is proposed, which is stroke order and stroke number free. Both input characters and the model characters are represented with complete attributed relational graphs (ARGs). A new optimal matching measure between two ARGs is defined. Classification of an input character can be implemented by matching its ARG against every ARG of the model base. The matching procedure is formulated as a search problem of finding the minimum cost path in a state space tree, using the A* algorithm. In order to speed up the search of the A*, besides a heuristic estimate, a novel strategy that utilizes the geometric position information of stroke segments of Chinese characters to prune the tree is employed. The efficiency of our method is demonstrated by the promising experimental results.