In this paper we propose a very efficient and low-cost method for real-time 3D object pose estimation, suitable for operation under severe changes in the illumination conditions. The method consists of a geometry-based and learning-based stage: first a geometry-based pose estimation algorithm is used to obtain pose parameter values, which are utilized as supervised information during a following learning-based stage. The efficacy of the proposed approach is illustrated on a real-world application in a head pose based facial-gesture interface for automatic wheelchair navigation.

We propose a method of identifying persons who are using wheelchairs in real environments on the basis of HOG features extracted from disparity images. First, we use USV, which is a stereo vision system for detecting humans, to determine the 3-D location of the humans moving within the monitored area, and cut out the corresponding regions in the disparity image. Then, we compute a HOG feature vector from those cut-out images, divide the results into the two classes of persons in wheelchairs and pedestrians and perform training and distinguishing by SVM. The data dealt with here involved people moving around without restriction in real environments. Also, there was no guarantee that the photographed environment was the same at the time of training data acquisition and at the time of recognition processing. We conducted training and recognition experiments on this method using video data acquired in the laboratory that confirmed a per-frame recognition rate of 99% or higher. Furthermore, after training with the same laboratory data and application to video data of visitors to an assistive products exhibition, a per-frame correct recognition rate of 80% was confirmed.

Dozens of people are killed every year at railroad crossings, a situation that requires urgent action by railroad companies, especially in major cities. We are therefore developing a ubiquitous stereo vision based system for ensuring safety at railroad crossings. In this system, stereo cameras are installed at four corners and are pointed toward the center of the railroad crossing to monitor the passage of people. The system determines automatically and in real-time whether anyone or anything is inside the railroad crossing, and whether anyone remains in the crossing. The system can be configured to automatically switch over to a surveillance monitor or automatically connect to an emergency brake system in the event of trouble.

We propose a novel fiducial-less 3-D object tracking method. Our method consists of three components: 1) bottom-up approach (BUA), 2) top-down approach (TDA), and 3) automatic database addition (ADA). An experimental result shows an accuracy and robustness of our method.

We have developed a method for recognizing and tracking known three-dimensional objects in scene images captured from a user's viewpoint. The method uses multiple color histogram matching and condensation tracking. We implemented a prototype 3-D online manual system based on our method.

This paper describes a fast and robust approach to recovering structure and motion from video frames. It first describes a robust recursive factorization method for affine projection. Using the least median of squares (LMedS) criterion, the method estimates the dominant 3D affine motion and discards feature points regarded as outliers. The computational cost of the overall procedure is reduced by combining this robust statistics-based method with a recursive factorization method that can at each frame provide the updated 3D structure of an object at a fixed computational cost by using the principal component analysis. Experiments with synthetic data and with real image sequences demonstrate that the method can be used to estimate the dominant structure and the motion robustly and in real-time, on an off-the-shelf PC. Finally this paper describes preliminary online experiments on selecting feature points, which have the dominant 3D motion, from live video frames by using a PC-cluster system.

This paper describes a method of estimating the position and orientation of a camera for constructing a mixed reality (MR) system. In an MR system 3D virtual objects should be merged into a 3D real environment at a right position in real time. To acquire the user's viewing position and orientation is the main technical problem of constructing an MR system. The user's viewpoint can be determined by estimating the position and orientation of a camera using images taken at the viewpoint. Our method estimates the camera pose using screen coordinates of captured color fiducial markers whose 3D positions are known. The method consists of three algorithms for perspective n-points problems and rises each algorithm selectively. The method also estimates the screen coordinates of untracked markers that are occluded or are out of the view. It has been found that an experimental MR system that is based on the proposed method can seamlessly merge 3D virtual objects into a 3D real environment at right position in real-time and allows users to look around an area in which numbers are placed.

To attain smooth human interaction we propose a system which simultaneously utilizes the stereo disparity and optical flow information of real-time stereo gray and multi resolution images to recognize objects and gestures. For real-time calculation of the disparity and optical flow information of a stereo image the system first creates pyramid images by utilizing a Gaussian filter. The system then determines the disparity and optical flow of a low density image and extracts regions in front of a certain depth. The three foremost regions are recognized by higher order local autocorrelation features and a linear discriminant analysis. With this process the system recognizes the face and hand signs of users which are displayed foremost, and roughly recognizes movements within the region in real-time. With this framework, the system can discriminate, the face of a user, can monitor the basic movements of the user, can smoothly learn a presented object by users and can communicate with users from hand signs learned in advance.

We present the intuitive interpretation of affine epipolar geometry for the orthographic, scaled orthographic, and paraperspective projection models in terms of the factorization method for the generalized affine projection (GAP) model proposed by Fujiki and Kurata (1997). Using the GAP model introduced by Mundy and Zisserman (1992), each affine projection model can be resolved into the orthographic projection model by the introduction of virtual image planes, then the affine epipolar geometry can be simply obtained from the estimates of the factorization, method.

Registration and integration of measured data of real objects are becoming important in 3D modeling for computer graphics and computer-aided design. We propose a new algorithm of registration and integration of multiple range images for producing a geometric object surface model. The registration algorithm determines a set of rigid motion parameters that register a range image to a given mesh-based geometric model. The algorithm is an integration of the iterative closest point algorithm with the least median of squares estimator. After registration, points in the input range are classified into inliers and outliers according to registration errors between each data point and the model. The outliers are appended to the surface model to be used by registration with the following range images. The parts classified as inlier by at least one registration result is segmented out to be integrated. This process consisting of registration and integration is iterated until all views are integrated. We successfully experimented with the proposed method on real range image sequences taken by a rangefinder. The method does not need any preliminary processes.

In order to develop a shape and decorative pattern database of old pottery and ceramics objects, it is necessary to obtain the surface pattern on the 3-dimensional shape of the object. This paper describes the recovery of the revolution surface from a monocular image of unknown camera parameters and the retrieval of the 2-dimensional pattern. The camera parameters are obtained by using a genetic algorithm (GA). After the revolution surfaces are reconstructed, these surfaces are developed into a 2-dimensional plane. We show that a scanner-digitized image of an old ceramic object can be analyzed by GA to reconstruct the revolution surface and to develop the 2-dimensional image on the 3-dimensional object.

MIRACLE-IV is discussed, which is capable of obtaining an internal structure of an object from a series of silhouette images with no initial explicit models about the object. The images are derived from only one object, but the form is varied. The system is composed of two subsystems: a model acquisition part (the modeler) and an image processing strategy part (the strategist). On the assumption that the object consists of hinges, slides, and solids, the modeler learns their number in the object and the relationship among them. The strategist binds the functional features as hinges or slides with visual features in the actual image data. The image processing sequence for the extraction of the visual feature is not given but is learned automatically through trial and error.>