In this paper, we propose a two-level integrated model for accurate 3D tracking of rigid head motion and non-rigid facial animation. At the lowest level, the 2D shape of facial features is robustly extracted using a regularized shape model and a cascade multi-stage algorithm. At the highest level, we estimate both the facial animation and 3D pose parameters via minimizing an energy function comprising three terms. The first quantifies the error in matching a 3D wireframe face model (Candide) to the image sequence, while the remaining terms impose temporal and spatial motion-smoothness constraints over the 3D model points. Through extensive experiments, we demonstrate the feasibility and effectiveness of the proposed method and also show it outperforms the state-of-the-art algorithms such as the Bayes Tangent Shape Model.

In this paper, two multi-stream asynchrony Dynamic Bayesian Network models (MS-ADBN model and MM-ADBN model) are proposed for audio-visual speech recognition (AVSR). The proposed models, with different topology structures, loose the asynchrony of audio and visual streams to word level. For MS-ADBN model, both in audio stream and in visual stream, each word is composed of its corresponding phones, and each phone is associated with observation vector. MM- ADBN model is an augmentation of MS-ADBN model, a level of hidden nodes--state level, is added between the phone level and the observation node level, to describe the dynamic process of phones. Essentially, MS-ADBN model is a word model, while MM-ADBN model is a phone model. Speech recognition experiments are done on a digit audio-visual (A-V) database, as well as on a continuous A-V database. The results demonstrate that the asynchrony description between audio and visual stream is important for AVSR system, and MM-ADBN model has the best performance for the task of continuous A-V speech recognition.

A new approach of tracking objects in image sequences is proposed, in which the constant changes of the size and orientation of the target can be precisely described. For each incoming frame, a probability distribution image of the target is created, where the target's area turns into a blob. The scale of this blob can be determined based on the local maxima of differential scale-space filters. We employ the QP_TR trust region algorithm to search the local maxima of orientational multi-scale normalized Laplacian filter of the probability distribution image to locate the target as well as to determine its scale and orientation. Based on the tracking results of sequence examples, the new method is proven to be capable of describing the target more accurately and thus achieves much better tracking precision.

Multifractal analysis is becoming more and more popular in image segmentation community, in which the box-counting based multifractal dimension estimations are most commonly used. However, in spite of its computational efficiency, the regular partition scheme used by various box-counting methods intrinsically produces less accurate results. In this paper, a novel multifractal estimation algorithm based on mathematical morphology is proposed and a set of new multifractal descriptors, namely the local morphological multifractal exponents is defined to characterize the local scaling properties of textures. A series of cubic structure elements and an iterative dilation scheme are utilized so that the computational complexity of the morphological operations can be tremendously reduced. Both the proposed algorithm and the box-counting based methods have been applied to the segmentation of texture mosaics and real images. The comparison results demonstrate that the morphological multifractal estimation can differentiate texture images more effectively and provide more robust segmentations.

Although simple and efficient, traditional feature-based texture segmentation methods usually suffer from the intrinsical less inaccuracy, which is mainly caused by the oversimplified assumption that each textured subimage used to estimate a feature is homogeneous. To solve this problem, an adaptive segmentation algorithm based on the coupled Markov random field (CMRF) model is proposed in this paper. The CMRF model has two mutually dependent components: one models the observed image to estimate features, and the other models the labeling to achieve segmentation. When calculating the feature of each pixel, the homogeneity of the subimage is ensured by using only the pixels currently labeled as the same pattern. With the acquired features, the labeling is obtained through solving a maximum a posteriori problem. In our adaptive approach, the feature set and the labeling are mutually dependent on each other, and therefore are alternately optimized by using a simulated annealing scheme. With the gradual improvement of features' accuracy, the labeling is able to locate the exact boundary of each texture pattern adaptively. The proposed algorithm is compared with a simple MRF model based method in segmentation of Brodatz texture mosaics and real scene images. The satisfying experimental results demonstrate that the proposed approach can differentiate textured images more accurately.

Trajectory matching is an important way to describe and classify behaviors of moving objects in a computer visual system. In this paper, we present two trajectory description methods, time-sampling sequence and space-sampling sequence, which can be used in different matching applications. We then propose two general trajectory matching schemes based on Levenshtein distance and relaxation matching respectively. Trajectory Levenshtein distance scheme is a good way to compare the topological shapes and directions of trajectories, and can be performed quickly. Trajectory relaxation matching scheme can gain the statistical optimal matching. Finally, we propose a top-to-bottom hierarchical clustering algorithm to classify trajectories, and several experiments demonstrate that our schemes are efficient in matching and classifying different shape and direction trajectories

Compared with ID grayscale histogram analysis, 2D entropic thresholding makes use of local average as well as pixel gray level. However, it is time consuming to search the threshold vector in the 2D histogram. In the paper, a fast algorithm using wavelet decomposition is proposed, with which a set of candidates of the vector was first obtained in the decomposed histogram. The optimal threshold vector is then obtained without exhaustive searching. Experimental results have shown that our algorithm not only finds the threshold vector as well as Brink's method (1992) but also saves computation costs, using up only 0.53% of the processing time taken by exhaustive searching.

Because of the large number of Chinese characters and many different writing styles involved, the recognition of handwritten Chinese characters remains a very challenging task. It is well recognized that a good feature set plays a key role in a successful recognition system. Shape normalization is as well an essential step toward achieving translation, scale, and rotation invariance in recognition. Many shape normalization methods and different feature sets have been proposed in the literature. We first review five commonly used shape normalization schemes and then discuss various feature extraction techniques usually used in handwritten Chinese character recognition. Based on numerous experiments conducted on 3,755 handwritten Chinese characters (GB2312-80), we discuss the matches made between the normalization schemes and the feature sets and suggest the best match between them in terms of classification performance. The nearest neighbor classifier was adopted in our experiments with templates obtained by using the K-means clustering algorithm.

This paper presents a hidden Markov mesh random field (HMMRF) based approach for off-line handwritten Chinese characters recognition using statistical observation sequences embedded in the strokes of a character. Due to a large set of Chinese characters and many different writing styles, the recognition of handwritten Chinese characters is very challenging. In our approach, the binary image is first normalized by a nonlinear shape normalization scheme to adjust the width, length, and the correlation of strokes. Two types of stroke-based features are then extracted to represent the observation sequence. The estimation of model parameters and state sequence decoding algorithms are also discussed in the paper. Experimental results on 470 isolated handwritten Chinese characters demonstrate the effectiveness of our approach.

An image segmentation approach based on modified fuzzy c-mean clustering algorithm is developed. This method deals with the global and local image information at a gross scene level, which incorporates the local information including the edge map and the spatial relationship of the pixels into the parameters of its objective function. But the current clustering based segmentation methods usually incorporate the local information into the feature space, or integrate the global and local information at a local level. In addition, we also propose a fuzzy Gaussian basis function neural network to complete fuzzy clustering on the grey-histogram of image as the initial solution, which can automatically determine the number of clusters, and is strong and robust.