We present a complete three-dimensional (3-D) ear recognition system combining local and holistic features in a computationally efficient manner. The system is comprised of four primary components, namely: 1) ear image segmentation; 2) local feature extraction and matching; 3) holistic feature extraction and matching; and 4) a fusion framework combining local and holistic features at the match score level. For the segmentation component, we introduce a novel shape-based feature set, termed the Histograms of Indexed Shapes (HIS), to localize a rectangular region containing the ear. For the local feature extraction and representation component, we extend the HIS feature descriptor to an object-centered 3-D shape descriptor, the Surface Patch Histogram of Indexed Shapes (SPHIS), for local ear surface representation and matching. For the holistic matching component, we introduce a voxelization scheme for holistic ear representation from which an efficient, voxel-wise comparison of gallery-probe model pairs can be made. The match scores obtained from both the local and holistic matching components are fused to generate the final match scores. Experimental results conducted on the University of Notre Dame (UND) collection G dataset, containing range images of 415 subjects yielded a rank-one recognition rate of 98.3% and an equal error rate of 1.7%. These results demonstrate that the proposed approach outperforms state-of-the-art 3-D ear biometric systems. Additionally, the method is considerably more efficient compared to the state-of-the-art because it employs a sparse set of features rather than using the dense model.

We present a novel voxelization framework for holistic Three-Dimensional (3D) object representation that accounts for distinct surface features. A voxelization of an object is performed by encoding an attribute or set of attributes of the surface region contained within each voxel occupying the space that the object resides in. To our knowledge, the voxel structures employed in previous methods consist of uniformly-sized voxels. The proposed framework, in contrast, generates structures consisting of variable-sized voxels that are adoptively distributed in higher concentration near distinct surface features. The primary advantage of the proposed method over its fixed resolution counterparts is that it yields a significantly more concise feature representation that is demonstrated to achieve a superior recognition performance. An evaluation of the method is conducted on a 3D ear recognition task. The ear provides a challenging case study be- cause of its high degree of inter-subject similarity.

In this paper, we present a robust method for 2D ear recognition using color SIFT features. Firstly, we extend the Scale Invariant Feature Transform (SIFT) algorithm originally performed on the intensity channel [1] to the RGB color channels to maximize the robustness of the SIFT feature descriptor. Secondly, a feature matching algorithm for ear recognition is proposed by fusion of the features extracted from the different color channels. Experiments conducted on the University of Notre Dame (UND) and the West Virginia University (WVU) ear biometric datasets indicate that our method can achieve better recognition rates than the state-of-the-art methods applied on the same datasets.

We present a method that automatically detects chewing events in surveillance video of a subject. Firstly, an Active Appearance Model (AAM) is used to track a subject's face across the video sequence. It is observed that the variations in the AAM parameters across chewing events demonstrate a distinct periodicity. We utilize this property to discriminate between chewing and non-chewing facial actions such as talking. A feature representation is constructed by applying spectral analysis to a temporal window of model parameter values. The estimated power spectra subsequently undergo non-linear dimensionality reduction via spectral regression. The low-dimensional representations of the power spectra are employed to train a Support Vector Machine (SVM) binary classifier to detect chewing events. Experimental results yielded a cross validated percentage agreement of 93.4%, indicating that the proposed system provides an efficient approach to automated chewing detection.

We describe a novel approach for determining the audio-visual synchrony of a monologue video sequence utilizing vocal pitch and facial landmark trajectories as descriptors of the audio and visual modalities, respectively. The visual component is represented by the horizontal and vertical displacement of corresponding facial landmarks between subsequent frames. These facial landmarks are acquired using the statistical modeling technique, known as the Active Shape Model (ASM). The audio component is represented by the fundamental frequency, or pitch, obtained using the subharmonic-to-harmonic ratio (SHR). The synchrony between the audio and visual feature vectors is computed using Gaussian mutual information. The raw synchrony estimates obtained using this method may contain spurious synchrony values due to over-sensitivity. A filtering method is employed for discarding synchrony values that occur during non-associated audio/visual events. The human visual system is capable of distinguishing rigid and non-rigid motion of an articulator during speech. In an attempt to emulate this process, we separate rigid and non-rigid motion and compute the synchrony attributed to each. Experiments are conducted on a dataset of monologue video clip pairs. Each pair is composed of an asynchronous and synchronous version of the video clip. For the asynchronous video clips, the audio signal is displaced with respect to the visual signal. Experimental results indicate that the proposed approach is successful in detecting facial regions that demonstrate synchrony, and in distinguishing between synchronous and asynchronous sequences. © 2009. The copyright of this document resides with its authors.

In this paper, we present a novel method for 3D face recognition using adaboosted geodesic distance features. Firstly, a generic model is finely conformed to each face model contained within a 3D face dataset. Secondly, the geodesic distance between anatomical point pairs are computed across each conformed generic model. Adaboost then generates a strong-classifier based on a collection of geodesic distances that are most discriminative for face recognition. Experiments conducted on the face recognition grand challenge (FRGC) database D collection indicate that the system can achieve over a 95% rank-one recognition rate.

In this paper we describe a multi-modal ear and face biometric system. The system is comprised of two components: a 3D ear recognition component and a 2D face recognition component. For the 3D ear recognition, a series of frames is extracted from a video clip and the region of interest (i.e., ear) in each frame is independently reconstructed in 3D using Shape From Shading. The resulting 3D models are then registered using the iterative closest point algorithm. We iteratively consider each model in the series as a reference model and calculate the similarity between the reference model and every model in the series using a similarity cost function. Cross validation is performed to assess the relative fidelity of each 3D model. The model that demonstrates the greatest overall similarity is determined to be the most stable 3D model and is subsequently enrolled in the database. For the 2D face recognition, a set of facial landmarks is extracted from frontal facial images using the Active Shape Model. Then, the response of facial images to a series of Gabor filters at the locations of facial landmarks are calculated. The Gabor features (attributes) are stored in the database as the face model for recognition. The similarity between the Gabor features of a probe facial image and the reference models are utilized to determine the best match. The match scores of the ear recognition and face recognition modalities are fused to boost the overall recognition rate of the system. Experiments are conducted using a gallery set of 402 video clips and a probe of 60 video clips (images). As a result, a rank-one identification rate of 100% was achieved using the weighted sum technique for fusion.

We describe a novel approach for 3D ear biometrics using video. A series of frames are extracted from a video clip and the region-of-interest (ROI) in each frame is independently reconstructed in 3D using Shape from Shading (SFS). The resulting 3D models are then registered using the Iterative Closest Point (ICP) algorithm. We iteratively consider each model in the series as a reference and calculate the similarity between the reference model and every model in the series using a similarity cost function. Cross validation is performed to assess the relative fidelity of each 3D model. The model that demonstrates the greatest overall similarity is determined to be the most stable 3D model and is subsequently enrolled in the database. Experiments are conducted using a gallery set of 402 video clips and a probe of 60 video clips. The results (95.0% rank-1 recognition rate) indicate that the proposed approach can produce recognition rates comparable to systems that use 3D range data. To the best of our knowledge, we are the first to develop a 3D ear biometric system that obtains 3D ear structure from a video sequence.

We describe a novel approach for 3-D ear biometrics using video. A series of frames is extracted from a video clip and the region of interest in each frame is independently reconstructed in 3-D using shape from shading. The resulting 3-D models are then registered using the iterative closest point algorithm. We iteratively consider each model in the series as a reference model and calculate the similarity between the reference model and every model in the series using a similarity cost function. Cross validation is performed to assess the relative fidelity of each 3-D model. The model that demonstrates the greatest overall similarity is determined to be the most stable 3-D model and is subsequently enrolled in the database. Experiments are conducted using a gallery set of 402 video clips and a probe of 60 video clips. The results (95.0% rank-1 recognition rate and 3.3% equal error rate) indicate that the proposed approach can produce recognition rates comparable to systems that use 3-D range data. To the best of our knowledge, we are the first to develop a 3-D ear biometric system that obtains a 3-D ear structure from a video sequence.