Glint features have important roles in gaze-tracking systems. However, when the operation range of a gaze-tracking system is enlarged, the performance of glint-feature-based (GFB) approaches will be degraded mainly due to the curvature variation problem at around the edge of the cornea. Although the pupil contour feature may provide complementary information to help estimating the eye gaze, existing methods do not properly handle the cornea refraction problem, leading to inaccurate results. This paper describes a contour-feature-based (CFB) 3-D gaze-tracking method that is compatible to cornea refraction. We also show that both the GFB and CFB approaches can be formulated in a unified framework and, thus, they can be easily integrated. Furthermore, it is shown that the proposed CFB method and the GFB method should be integrated because the two methods provide complementary information that helps to leverage the strength of both features, providing robustness and flexibility to the system. Computer simulations and real experiments show the effectiveness of the proposed approach for gaze tracking.

In this work, we develop an appearance-based gaze tracking system allowing user to move their head freely. The main difficulty of the appearance-based gaze tracking method is that the eye appearance is sensitive to head orientation. To overcome the difficulty, we propose a 3-D gaze tracking method combining head pose tracking and appearance-based gaze estimation. We use a random forest approach to model the neighbor structure of the joint head pose and eye appearance space, and efficiently select neighbors from the collected high dimensional data set. L1-optimization is then used to seek for the best solution for regression from the selected neighboring samples. Experiment results shows that it can provide robust binocular gaze tracking results with less constraints but still provides moderate estimation accuracy of gaze estimation.

Glint features have important roles in gaze tracking systems. But when the operation range of a gaze tracking system is enlarged, the performance of glint-feature-based (GFB) approaches will be degraded mainly due to the curvature variation problem at around the edge of the cornea. Although the pupil contour feature may provide complementary information to help estimating the eye gaze, existing methods do not properly handle the cornea refraction problem, leading to inaccurate results. This paper describes a contour-feature-based (CFB) 3-D gaze tracking method that is compatible to cornea refraction. Experiments show the effectiveness of the proposed approach for gaze tracking.

This paper shows that, by simply adding a triangle aperture (TA) in front of a camera lens, iris autofocus can be easily achieved. Through the TA, the corneal reflection of a light source forms a triangle glint on the image plane. The size and orientation of the glint can be used to infer the amount and the direction of the focus adjustment. A gradient-descent autofocus control law is proposed for uncalibrated lenses. Results from theoretical analysis and real experiments show that the proposed method is more efficient and accurate than the conventional circular aperture approach.

This work aims to develop a system for predicting age progression in children faces. Age progression prediction in children faces is critical to assist missing children searching. An integral module including feature extraction, distance measurement, and face synthesis is devised in this paper to predict faces at different ages. In the proposed method, a curvature-weighted plus bending-energy distance is employed for selecting similar facial components from an aging database. The growth curve of each facial component is used to predict the shape, size, and location of each component at a different age. Thin plate spline method is employed to synthesize a 3-D face model from the predicted components by minimizing the bending energy. Experiments are conducted to test the proposed method with various subjects and the results show that the proposed method is very promising.

Reconfiguring the PTZ parameters of a camera network is an combinatorial optimization problem and computing the optimal solution is very time consuming. Therefore, existing methods can only provide sub-optimal solutions. In this paper, a nonlinear objective function for better utilizing the cameras to track multiple targets is proposed. Furthermore, it is shown that by expanding the unknown parameters and imposing new constraints, the nonlinear objective function can be converted into a linear production game (LPG) problem. Since an LPG possesses an optimal solution which can be evaluated with polynomial time, the proposed method is efficient and accurate. Computer simulations have been conducted and the results show that the proposed method is very promising.

This paper proposes a non-orthogonal view iris recognition system comprising a new iris imaging module, an iris segmentation module, an iris feature extraction module and a classification module. A dual-charge-coupled device camera was developed to capture four-spectral (red, green, blue, and near-infrared) iris images which contain useful information for simplifying the iris segmentation task. An intelligent random sample consensus iris segmentation method is proposed to robustly detect iris boundaries in a four-spectral iris image. In order to match iris images acquired at different off-axis angles, we propose a circle rectification method to reduce the off-axis iris distortion. The rectification parameters are estimated using the detected elliptical pupillary boundary. Furthermore, we propose a novel iris descriptor which characterizes an iris pattern with multiscale step/ridge edge-type maps. The edge-type maps are extracted with the derivative of Gaussian and the Laplacian of Gaussian filters. The iris pattern classification is accomplished by edge-type matching which can be understood intuitively with the concept of classifier ensembles. Experimental results show that the equal error rate of our approach is only 0.04% when recognizing iris images acquired at different off-axis angles within ±30°.

This paper presents a biometric iris recognition using 2D-LDA with embedding 2D-PCA. A new approach that the 2D-PCA is embedded into the 2D-LDA to improve its performance is proposed. The approach first finds the most concentrated training samples in each class, and uses the sample mean to represent the class. Then the 2D-PCA is adopted to find the projection matrix which can scatter the variance between classes. The results show that the new approach has an encouraging performance. The recognition rate up to 99.20% can be achieved.

This paper presents a novel biometric recognition system fusing the palmprint and hand geometry of a human hand based on morphology. We utilize the image morphology and concept of Voronoi diagram to cut the image of the front of the whole palm apart into several irregular blocks in accordance with the hand geometry. Furthermore, statistic characteristics of the gray level in the blocks are employed as characteristic values. The experimental results show that the proposed system has an encouraging performance. The false acceptance rate (FAR) and false rejection rate (FRR) are reduced down to 0.0035% and 5.7692%, respectively.

In this paper, we propose a framework for unsupervised analysis of human behavior based on manifold learning. First, a pairwise human posture distance matrix is calculated from a training action sequence. Then, the isometric feature mapping (Isomap) algorithm is applied to construct a low-dimensional structure from the distance matrix. The data points in the Isomap space are consequently represented as a time-series of low-dimensional points. A temporal segmentation technique is then applied to segment the time series into subseries corresponding to atomic actions. Next, a dynamic time warping (DTW) approach is applied for clustering atomic action sequences. Finally, we use the clustering results to learn and classify atomic actions using the nearest neighbor rule. Experiments conducted on real data demonstrate the efficacy of the proposed method.

Visual analysis of human behavior has generated considerable interest in the field of computer vision because of its wide spectrum of potential applications. Human behavior can be segmented into atomic actions, each of which indicates a basic and complete movement. Learning and recognizing atomic human actions are essential to human behavior analysis. In this paper, we propose a framework for handling this task using variable-length Markov models (VLMMs). The framework is comprised of the following two modules: a posture labeling module and a VLMM atomic action learning and recognition module. First, a posture template selection algorithm, based on a modified shape context matching technique, is developed. The selected posture templates form a codebook that is used to convert input posture sequences into discrete symbol sequences for subsequent processing. Then, the VLMM technique is applied to learn the training symbol sequences of atomic actions. Finally, the constructed VLMMs are transformed into hidden Markov models (HMMs) for recognizing input atomic actions. This approach combines the advantages of the excellent learning function of a VLMM and the fault-tolerant recognition ability of an HMM. Experiments on realistic data demonstrate the efficacy of the proposed system.

Producing an appropriate extent of visually salient regions in video sequences is a challenging task. In this work, we propose a novel approach for modeling dynamic visual attention based on spatiotemporal analysis. Our model first detects salient points in three-dimensional video volumes, and then uses them as seeds to search the extent of salient regions in a motion attention map. To determine the extent of attended regions, the maximum entropy in the spatial domain is used to analyze the dynamics obtained from spatiotemporal analysis. The experiment results show that the proposed dynamic visual attention model can effectively detect visual saliency through successive video volumes.

This paper presents a video surveillance system that is capable of detecting and classifying moving targets in real-time. The system extracts moving targets from a video stream and classifies them into predefined categories according to their spatiotemporal properties. Classification of the moving targets is completed via a combination of a temporal boosted classifier and spatiotemporal “motion energy” analysis. We illustrate that a temporal boosted classifier can be designed that successfully recognizes five object categories: person(s), bicycle, motorcycle, vehicle, and person with umbrella. The proposed temporal boosted classifier has the unique ability to improve weak classifiers by allowing them to make use of previous information when evaluating the current frame. In addition, we demonstrate a method to further process targets in the “person(s)” category to determine if they are single moving individuals or crowds. It is shown that this challenging task of moving crowd recognition can be effectively performed using spatiotemporal motion energies.

This paper presents a new method for the relaxation of multiview registration error. The multiview registration problem is represented using a graph. Each node and each edge in the graph represents a 3-D data set and a pairwise registration, respectively. Assuming that all the pairwise registration processes have converged to fine results, this paper shows that the multiview registration problem can be converted into a quadratic programming problem of Lie algebra parameters. The constraints are obtained from every cycle of the graph to eliminate the accumulation errors of global registration. A linear solution is proposed to distribute the accumulation error to proper positions in the graph, as specified by the quadratic model. Since the proposed method does not involve the original 3-D data, it has low time and space complexity. Additionally, the proposed method can be embedded into a trust-region algorithm and, thus, can correctly handle the nonlinear effects of large accumulation errors, while preserving the global convergence property to the first-order critical point. Experimental results confirm both the efficiency and the accuracy of the proposed method.

This paper presents a video surveillance system in the environment of a stationary camera that can extract moving targets from a video stream in real time and classify them into predefined categories according to their spatiotemporal properties. Targets are detected by computing the pixel-wise difference between consecutive frames, and then classified with a temporally boosted classifier and ldquospatiotemporal-oriented energyrdquo analysis. We demonstrate that the proposed classifier can successfully recognize five types of objects: a person, a bicycle, a motorcycle, a vehicle, and a person with an umbrella. In addition, we process targets that do not match any of the AdaBoost-based classifier's categories by using a secondary classification module that categorizes such targets as crowds of individuals or non-crowds. We show that the above classification task can be performed effectively by analyzing a target's spatiotemporal-oriented energies, which provide a rich description of the target's spatial and dynamic features. Our experiment results demonstrate that the proposed system is extremely effective in recognizing all predefined object classes.

A framework for human action modeling and recognition in continuous action sequences is proposed. A star figure enclosed by a bounding convex polygon is used to effectively represent the extremities of the silhouette of a human body. Thus, human actions are recorded as a sequence of the star figure's parameters, which is then used for action modeling. To model human actions in a compact manner while characterizing their spatio-temporal patterns, star figure parameters are represented by a 2-D feature map, which is used and regarded as a spatio-temporal template. Experiments to evaluate the performance of the proposed framework show that it can recognize human actions in an efficient and effective manner.

In this paper we propose a novel fusion strategy which fuses information from multiple physical traits via a cascading verification process. In the proposed system users are verified by each individual modules sequentially in turns of face, voice and iris, and would be accepted once he/she is verified by one of the modules without performing the rest of the verifications. Through adjusting thresholds for each module, the proposed approach exhibits different behavior with respect to security and user convenience. We provide a criterion to select thresholds for different requirements and we also design an user interface which helps users find the personalized thresholds intuitively. The proposed approach is verified with experiments on our in-house face-voice-iris database. The experimental results indicate that besides the flexibility between security and convenience, the proposed system also achieves better accuracy than its most accurate module.

Visual analysis of human behavior has generated considerable interest in the field of computer vision because it has a wide spectrum of potential applications. Atomic human action recognition is an important part of a human behavior analysis system. In this paper, we propose a language modeling framework for this task. The framework is comprised of two modules: a posture labeling module, and an atomic action learning and recognition module. A posture template selection algorithm is developed based on a modified shape context matching technique. The posture templates form a codebook that is used to convert input posture sequences into training symbol sequences or recognition symbol sequences. Finally, a variable-length Markov model technique is applied to learn and recognize the input symbol sequences of atomic actions. Experiments on real data demonstrate the efficacy of the proposed system.

In this paper, a framework of automatic human action segmentation and recognition in continuous action sequences is proposed. A star-like figure is proposed to effectively represent the extremities in the silhouette of human body. The human action, thus, is recorded as a sequence of the star-like figure parameters, which is used for action modeling. To model human actions in a compact manner while characterizing their spatio-temporal distributions, star-like figure parameters are represented by Gaussian mixture models (GMM). In addition, to address the intrinsic nature of temporal variations in a continuous action sequence, we transform the time sequence of star-like figure parameters into frequency domain by discrete cosine transform (DCT) and use only the first few coefficients to represent different temporal patterns with significant discriminating power. The performance shows that the proposed framework can recognize continuous human actions in an efficient way

This paper presents a biometric recognition system, based on the iris of a human eye, using a wavelet transform. The proposed system includes three modules: image preprocessing, feature extraction, and recognition. The feature extraction module adopts the gradient direction (i.e., angle) of the wavelet transform as the discriminating texture features. The system encodes the features to generate its iris feature codes using two efficient coding techniques: binary Gray encoding and delta modulation. Experimental results show that recognition rates up to 95.27%, 95.62%, 96.21%, and 99.05%, respectively, using different coding methods, can be achieved.

A novel approach to three-dimensional (3-D) gaze tracking using 3-D computer vision techniques is proposed in this paper. This method employs multiple cameras and multiple point light sources to estimate the optical axis of user's eye without using any user-dependent parameters. Thus, it renders the inconvenient system calibration process which may produce possible calibration errors unnecessary. A real-time 3-D gaze tracking system has been developed which can provide 30 gaze measurements per second. Moreover, a simple and accurate calibration method is proposed to calibrate the gaze tracking system. Before using the system, each user only has to stare at a target point for a few (2-3) seconds so that the constant angle between the 3-D line of sight and the optical axis can be estimated. The test results of six subjects showed that the gaze tracking system is very promising achieving an average estimation error of under 1 degrees.

In this paper, the Cramér-Rao lower bound (CRLB) of image registration error using an isotropic fiducial mark is derived. The derived CRLB is a function of the intensity profile of the fiducial mark. Following the development of the CRLB, a new method for designing an isotropic fiducial mark, suitable for digital image registration, is presented. A parameterization method of the fiducial intensity profile is introduced which guarantees no aliasing effect when the fiducial mark is digitized with the proper sampling rate. A method for computing the fiducial intensity profile, based on minimization of the CRLB registration error, and subject to certain practical constraints, is developed. For imaging systems with a significant low-pass effect, it is proposed to pre-emphasize the high frequency components of the fiducial mark by converting the designed gray-scale fiducial marks into binary fiducial marks. Experimental results show that the designed fiducial mark can provide very accurate registration results and that the registration accuracy is independent of its location.

This paper presents a simple and efficient method of calibrating the intrinsic camera parameters for all the lens settings of a motorized zoom lens. We fix the aperture setting and perform the camera calibration, adaptively, over the ranges of the room and focus settings. Bilinear interpolation is used to provide the values of the intrinsic camera parameters for those lens settings where no observations are taken. Our experiments show that the proposed method can provide accurate intrinsic camera parameters for all the lens settings, even though camera calibration is performed only for a small number of sampled lens settings. A calibration object suitable for zoom lens calibration is also presented.

We propose a novel method to estimate and track the 3D line of sight of a person based on 3D computer vision techniques. Most of the existing nonintrusive gaze tracking methods share a common drawback that users have to perform certain experiments in calibrating the user-dependent parameters before using the gaze tracking systems. These parameters are functions of the radius of the cornea, the position of the pupil, the position of user's head, etc. Our approach, in contrast, employs multiple cameras and multiple point light sources to estimate the light of sight without using any of the user-dependent parameters. As a consequence, the users can avoid the inconvenient calibration process which may produce possible calibration errors. Computer simulations were performed to confirm the proposed method.

This paper presents a new automatic approach to reconstructing a model for the 3D environment by use of an active binocular head (the IIS head). To efficiently store and access the depth estimates, we propose the use of the inverse polar octree. The depth estimates are computed by using the asymptotic Bayesian estimation method. The path of the local motion required by the asymptotic Bayesian method is determined online automatically to reduce the ambiguity of stereo matching. Some rules for checking the consistency between the new observation and the previous observations have been developed. Our experimental results have shown that the proposed approach is very promising for automatic generation of 3D models which can be used for rendering a 3D scene in a virtual reality system.

A systematic approach for integrating virtual objects into real images is developed in this paper.We propose the P3P-IC.method to solve the camera pose estimation problem.A robust tracking method is developed via the combination of the LMedS technique and the P3P-ICP method.With the 3D models and the robust tracking methods, we can determine the camera poses associated with each frame in the image sequence.Knowing the camera poses for each image frame, we can then integrate virtual objects into a video segment. 1.1

In this paper, we show how an active binocular head, the IIS head, can be easily calibrated with very high accuracy. Our calibration method can also be applied to many other binocular heads. In addition to the proposal and demonstration of a four-stage calibration process, there are three major contributions in this paper. First, we propose a motorized-focus lens (MFL) camera model which assumes constant nominal extrinsic parameters. The advantage of having constant extrinsic parameters is to having a simple head/eye relation. Second, a calibration method for the MFL camera model is proposed in this paper, which separates estimation of the image center and effective focal length from estimation of the camera orientation and position. This separation has been proved to be crucial; otherwise, estimates of camera parameters would be very noise-sensitive. Thirdly, we show that, once the parameters of the MFL camera model is calibrated, a nonlinear recursive least-square estimator can be used to refine all the 35 kinematic parameters. Real experiments have shown that the proposed method can achieve accuracy of one pixel prediction error and 0.2 pixel epipolar error, even when all the joints, including the left and right focus motors, are moved simultaneously. This accuracy is good enough for many 3D vision applications, such as navigation, object tracking and reconstruction.

This paper proposes a new closed-form solution for identifying the kinematic parameters of an active binocular head having four revolute joints and two prismatic joints by using three-dimensional (3-D) point (position) measurements of a calibration point. Since this binocular head is composed of off-the-shelf components, its kinematic parameters are unknown. Therefore, we can not directly apply those existing nonlinear optimization methods. Even if we want to use the nonlinear optimization methods, a closed-form solution can be first applied to obtain accurate enough initial values. Hence, this paper considers only methods that provide closed-form solutions, i.e., those requiring no initial estimates. Notice that most existing closed-form solutions require pose (i.e., both position and orientation) measurements. However, as far as we know, there is no inexpensive technique which can provide accurate pose measurements. Therefore, existing closed-form solutions based on pose measurements can not give us the required accuracy. As a result, we have developed a new method that does not require orientation measurements and can use only the position measurements of a calibration point to obtain highly accurate estimates of kinematic parameters using closed-form solutions. The proposed method is based on the complete and parametrically continuous (CPC) kinematic model, and can be applied to any kind of kinematic parameter identification problems with or without multiple end-effecters, providing that the links are rigid, the joints are either revolute or prismatic and no closed-loop kinematic chain is included.

In camera calibration, due to the correlations between certain camera parameters, an estimate of a set of camera parameters which minimizes a given criterion does not guarantee that the estimates of the physical camera parameter are themselves accurate. This problem has not drawn much attention from our computer vision society because conventional computer vision applications require only accurate 3D measurements and do not care much about the values of the physical parameters as long as their composite effect is satisfactory. However, when dealing with an active vision system, accuracy of the physical parameters is very critical because we need that accuracy to establish the relation between the motor positions and the camera parameters (both intrinsic and extrinsic). The contribution of this work is mainly in error analysis of camera calibration, especially in the accuracy of the physical camera parameters themselves, for four different types of calibration problems. With our error analysis, the most suitable camera calibration technique and calibration configuration for providing accurate camera parameters can be determined. Based on this error analysis, we have developed a method for calibrating our eight-degree-of-freedom binocular system. Real experiments have shown that this method can achieve accuracy of one pixel prediction error and 0.2 pixel epipolar error, even when all the eight joints, including the focus motors, are moved simultaneously. This high accuracy greatly alleviates the difficulties encountered in applying the active vision approach to automatic reconstruction of 3D objects and environments, which is our current endeavor.

This paper proposes a new closed-form solution for identifying the kinematic parameters of an active binocular head by using a single calibration point. This method is based on the complete and parametrically continuous (CPC) kinematic model, and can be applied to any kind of kinematic parameter identification problems with or without multiple end-effectors, providing that the links are rigid, the joints are either revolute or prismatic and no closed-loop kinematic chain is included. As a practical example, this paper focuses on the calibration of a binocular head having four revolute joints and two prismatic joints. Simulation and real experiments have shown that the proposed method of using point measurements can achieve much higher accuracy than that of using pose measurements.

The paper by Zhuang and Roth (ibid. vol.9, p.174-85 (1993)) presents a linear solution to kinematic parameter identification of robot manipulators. With their method, the orientation parameters for all revolute joints have to be solved first before solving for the translation parameters for all revolute joints and the orientation parameters for all prismatic joints simultaneously. Our major modification here is to decompose the kinematic parameters estimation problem into many subproblems of a single joint axis such that the complexity is reduced and easier implementation is derived. In addition, this gives a unified solution for any serial manipulator with an arbitrary combination of prismatic and revolute joints.>

The authors investigate the effect of neglecting lens distortion, and present a theoretical analysis of the calibration accuracy. The error bound derived is a function of a few factors, including the number of calibration points, the observation error of 2-D image points, the radial lens distortion coefficient, and the image size and resolution. This error bound provides a guideline for selecting both a proper camera calibration configuration and an appropriate camera model while providing the desired accuracy. Experimental results from both computer simulations and real experiments are given.>