This paper proposes real-time face tracking and recognition by combining two eigen-based methods. The first method is a novel extension of eigenface called augmented eigenface and the second method is a sparse 3D eigentemplate tracker controlled by a particle filter. The augmented eigenface is an eigenface augmented by an associative mapping to 3D shape that is specified by a set of volumetric face models. This paper discusses how to make up the augmented eigenface and how it can be used for inference of 3D shape from partial images. The associative mapping is also generalized to subspace-to-one mappings to cover photometric image changes for a fixed shape. A novel technique, called photometric adjustment, is introduced for simple implementation of associative mapping when an image subspace should be combined to a shape. The sparse 3D eigentemplate tracker is an extension of the 3D template tracker proposed by Oka et al. In combination with the augmented eigenface, the sparse 3D eigentemplate tracker facilitates real-time 3D tracking and recognition when a monocular image sequence is provided. In the tracking, sparse 3D eigentemplate is updated by the augmented eigenface while face pose is estimated by the sparse eigentracker. Since the augmented eigenface is constructed on the conventional eigenfaces, face identification and expression recognition are also accomplished efficiently during the tracking. Therefore, the task of face tracking and recognition is accomplished in the particle tracker framework. In the experiment, an augmented eigenface was constructed from 25 faces where 24 images were taken in different lighting conditions for each face. Experimental results show that the augmented eigenface works with the 3D eigentemplate tracker for real-time tracking and recognition. Real-time expression recognition was also reported where the augmented eigenface was constructed from several expressions of a particular person. The real-time tracker works at 30fps with quad-core implementation on Intel Core i7 CPU 940 (2.93GHz).

Tracking 3d pose of a known object is one of the most important problems in computer vision. This paper proposes an appearance-based approach to this problem by combining the sparse template matching and the particle filter. Although the combination of them has already been discussed for 2d tracker, it has not been applied for efficient 3d tracking. This paper discusses an appearance-based tracker when a surface model of the target 3d object and the initial pose are given. The fundamental framework of the particle filter is provided at first for implementing a pose tracker based on sparse 3d template matching. Then, the coarse-to-fine approach is introduced for efficient implementation. Although the fundamental particle filter often requires a lot of particles for sufficient tracking, the number of particles can be effectively reduced by the coarse-to-fine strategy. Experimental results for both the simulation data and real images show how the proposed method works in frame rate on 3GHz Core 2 Quad (single thread) without using GPU or other special hardware.

Eigenspaces are often utilized in computer vision for various applications. However, careful manipulations are required for sensitive applications in photometric analyses and human interface applications. While several formulations have already been utilized in each application, a unified framework of eigenspace manipulations should be created for further progress of computer vision. This paper provides a concept of homogeneous eigenspaces for this purpose. After brief discussion of eigenspace construction, this paper focuses on how an optimal projection of an input image should be done. Three approaches are presented for the projection and normalization in order to minimize the effects of noise. Both the theoretical and experimental comparisons show that the normalization after projection can suppress the effects of noise more effectively than the other methods. The normalization-after-projection formulation can be reformulated by the homogeneous eigenspace. In the framework of homogeneous eigenspace, a projection onto the normalized eigenspace is accomplished without using an explicit normalization. Furthermore, an optimal partial projection to the normalized eigenspace is also reduced to linear mapping in this framework. Two applications are also discussed for illumination analysis and for photometric classification. These problems are efficiently solved by partial projections onto homogeneous eigenspaces.

The best method for estimating the fundamental matrix and/or the epipole over a given set of point correspondences between two images is a nonlinear minimization, which searches a rank-2 fundamental matrix that minimizes the geometric error cost function. When convenience is preferred to accuracy, we often use a linear approximation method, which searches a rank-3 matrix that minimizes the algebraic error. Although it has been reported that the algebraic error causes very poor results, it is currently thought that the relatively inaccurate results of a linear estimation method are a consequence of neglecting the rank-2 constraint, and not a result of exploiting the algebraic error. However, the reason has not been analyzed fully. In the present paper, we analyze the effects of the cost function selection and the rank-2 constraint based on covariance matrix analyses and show theoretically and experimentally that it is more important to enforce the rank-2 constraint than to minimize the geometric cost function.

Face recognition under various lighting condition's is discussed to cover cases when too few images are available for registration. This paper proposes decomposition of an eigenface into two orthogonal eigenspaces for realizing robust face recognition under such conditions. The decomposed eigenfaces consisting of two eigenspaces are constructed for each person even if only one image is available. A universal eigenspace called the canonical space (CS) plays an important role in creating the eigenspaces by way of decomposition, where CS is constructed a priori by principal component analysis (PCA) over face images of many people under many lighting conditions. In the registration stage, an input face image is decomposed to a projection image in CS and the residual of the projection. Then two eigenspaces are created independently in CS and in the orthogonal complement CS/sup /spl perp//. Some refinements of the two eigenspaces are also discussed. By combining the two eigenspaces, we can easily realize face identification that is robust to illumination change, even when too few images are registered. Through experiments, we show the effectiveness of the decomposed eigenfaces as compared with conventional methods.

A Photometric Image-Based Rendering (PIER) concept is proposed that facilitates the generation of an image with an arbitrary illumination. Based on this concept, we aim to generate both diffuse and specular reflections. It is not necessary to explicitly recover 3D shape and reflection properties of the scene. In order to control appearance changes caused by modifications in the lighting conditions, we utilize a set of real images taken, in a variety of lighting conditions. Since the diffuse and specular reflection components have different characteristics, we separate these components and apply different methods to each. A photometric linearization is introduced to control diffuse reflections as well as for separating the other components. This also facilitates the treatment of attached shadows as a part of the diffuse reflection. A morphing technique is utilized to generate specular reflections. This is an effective technique for dealing with glossy objects, even when the light shape is clearly observed in the specular reflection. Experimental results show that realistic images can be successfully generated using this technique.

Recognizing a target in synthetic-aperture radar (SAR) images is an important, yet challenging, application of the model-based vision technique. This paper describes a model-based SAR recognition system based on invariant histograms and deformable template matching techniques. An invariant histogram is a histogram of invariant values defined by geometric features such as points and lines in SAR images. Although a few invariants are sufficient to recognize a target, we use a histogram of all invariant values given by all possible target feature pairs. This redundant histogram enables robust recognition under severe occlusions typical in SAR recognition scenarios. Multi-step deformable template matching examines the existence of an object by superimposing templates over potential energy field generated from images or primitive features. It determines the template configuration which has the minimum deformation and the best alignment of the template with features. The deformability of the template absorbs the instability of SAR features. We have implemented the system and evaluated the system performance using hybrid SAR images, generated from synthesized model signatures and real SAR background signatures.

In this paper, we propose the concept of robotic task design for task directed robotics in order to develop a framework for achieving versatile robotic task execution system. This system consists of the following subsystems: 1) a robotic task design system; 2) a teaching (programming) system; and 3) a task execution system. These have equal importance in accomplishing the task intended by the user with the robot system instead of over emphasizing the task execution system which has been the main research area of robotics in terms of motion control. Based on this proposed concept, a robotic task execution controller called NOAC is designed. The main concept in the structure of NOAC is to separate function (1) which is commonly needed by every task in the task class (task class function) and function (2) which is a task-oriented function. They are to be parallel processed on NOAC system. This separation simplifies software development. A deburring task is designed based on the nature of the burr and implemented onto the NOAC system with the above concept. This system is shown to be simple and expandable.>

The authors propose robust algorithms for line-based pose enumeration from a single view and report on their evaluations by simulations. The proposed algorithms incorporate two major refinements into the algorithms originally proposed by T. Shakunaga. The first refinement, introduction of zone-crossing detection to the 1-D search, decreases the rate of overlooking a correct pose. The second refinement, adaptive selection of a perspective angle transform pair, considerably reduces the average estimation error. Simulation results show that pose estimation precision depends primarily on the precision of line detection. Although the refinements are effective, they are better for more precise line detection. For 99% of rigid body samples, the algorithm can estimate rotation with an error of less than 2/spl deg/, and for 99.9% of the samples, the error is less than 10/spl deg/. Simulation experiments for articulated objects show similar results by using the second algorithm.>

This paper formalizes and analyzes problems of estimating a pose from a single view when an object model and a camera model are provided. For discussing recovery of the rotation of flexible objects, an object model called the vector body model is introduced. This model is represented as a graph consisting of vector bodies and joint vectors, both of which are abstractions of a rigid body and a rotational joint. Any object can be represented as a graph, and a particular rotation estimation problem from object-image correspondence can be derived from its subgraph. Using this formalization, this paper identifies primitive problems that are solvable but would be unsolvable if any vector in the graph were not available. Problems are analyzed from both topological and metrical viewpoints.

This paper presents the framework for a CAD-based monocular vi- sion system that estimates an object pose. Estimation of pose from a single view is discussed according to an a priori 3-d ob- ject model that includes key primitive ele- ments and the relations among them. Es- timation strategy is discussed briefly, and several algorithms used in a pose estima- tion are presented. The proposed sys- tem - implemented in four subsystems for ob ject-image correspondence, feasible pose enumeration, pose selection, and estima- tion refinement - integrates sensor infor- mation and a priori knowledge about the treated objects. Experimental results for synthesized and real images show the effec- tiveness of this system for pose estimation and 3-d pose tracking.

The author discusses the modeling of a 3-D indoor scene from a single view using a generic object model. It is assumed that an image is made by a well-calibrated camera, and that the camera height above the floor is known. The image is assumed to be a projection of a natural corridor scene from which a generic model is known, but the specific model is unknown. This system can model any corridor that satisfies the following conditions: (1) the corridor is a rectangular parallelepiped, (2) there are several lines along each axis of the rectangle parallelepiped, (3) there are not many parallel lines in directions other than along the principal axes, and (4) corridor height is within a known range. The system for 3-D modeling from a single view of such a corridor consists of a robust bottom-up image processing part and a top-down, model-based interpretation part. Experimental results show that specific corridor models can be recovered from a single view with an approximately 1% estimation error.>

A framework is proposed for model-based monocular vision covering not only conventional 3-D rigid models, but also flexible structures made up of 3-D rigid bodies connected by rotational joints. Pose-estimation problems from a single view are defined and discussed according to this object model composed of rigid bodies and rotational axes, respectively represented by sets of unit vectors and by single unit vectors. The authors define primitive problems as those which are solvable, but which would be unsolvable if any vector in the problem were invisible. A theorem is derived to extract a primitive problem family, members of which correspond to models containing rigid bodies and invisible rotational axes. Two generic rotation-estimation algorithms applicable to this problem family are also constructed. Experimental results from several primitive problems show the effectiveness of the proposed framework.>

This paper provides a mathematical framework for shape-from-angle problems, and discusses four primitive problems based on it. For three of them, algebraic equations are provided from a geometric analysis. A generic lD search algorithm is proposed for designing a 1-D search algorithm, usable for any shape-from-angle problems. By designing the problem-dependent parts of the generic algorithm, specific l-D search algorithms are constructed for each problem. The generic algorithm finds all the feasible configurations by detecting the zero-crossing points of a problem-dependent test function on a constraint curve. Experimental results show the effectiveness of each specific algorithm and the applicability of the generic algorithm to several types of shape-from-angle problems.