This paper presents a spatio-temporal approach in recognizing six universal facial expressions from visual data and using them to compute levels of interest. The classification approach relies on a two-step strategy on the top of projected facial motion vectors obtained from video sequences of facial expressions. First a linear classification bank was applied on projected optical flow vectors and decisions made by the linear classifiers were coalesced to produce a characteristic signature for each universal facial expression. The signatures thus computed from the training data set were used to train discrete hidden Markov models (HMMs) to learn the underlying model for each facial expression. The performances of the proposed facial expressions recognition were computed using five fold cross-validation on Cohn-Kanade facial expressions database consisting of 488 video sequences that includes 97 subjects. The proposed approach achieved an average recognition rate of 90.9% on Cohn-Kanade facial expressions database. Recognized facial expressions were mapped to levels of interest using the affect space and the intensity of motion around apex frame. Computed level of interest was subjectively analyzed and was found to be consistent with "ground truth" information in most of the cases. To further illustrate the efficacy of the proposed approach, and also to better understand the effects of a number of factors that are detrimental to the facial expression recognition, a number of experiments were conducted. The first empirical analysis was conducted on a database consisting of 108 facial expressions collected from TV broadcasts and labeled by human coders for subsequent analysis. The second experiment (emotion elicitation) was conducted on facial expressions obtained from 21 subjects by showing the subjects six different movies clips chosen in a manner to arouse spontaneous emotional reactions that would produce natural facial expressions.

This paper compares the performances of linear and non-linear data projection techniques in classifying six universal facial expressions from only visual data. Three different data projection techniques, namely, principal component analysis (PCA), non-negative matrix factorization (NMF) and local linear embedding (LLE) were experimented to project the facial motion onto lower dimensional subspace to estimate the intrinsic dimension of the facial expression data. The classification approach relies on a two-step strategy on the top of projected facial motion information obtained from sequence of facial expression images. First, a bank of linear classifier was applied on projected data and decision made by the linear classifiers was coalesced to produce a characteristic signature for each universal facial expressions. The signatures thus computed from the training data set were used to train discrete hidden Markov models (HMMs) to learn the underlying model for each facial expression. The performances of each data representations in classifying the facial expressions were compared using five fold cross validation on a database of 488 video sequences that include 97 subjects. To further illustrate the efficacy of the proposed approach and to better understand the effects of a number of factors that are detrimental to the facial expression recognition using only visual data were conducted. The first empirical analysis was conducted on a database consisting of 108 facial expressions collected from TV broad cast and labeled by human coder for subsequent analysis. The second experiment was conducted on facial expressions obtained from 21 subjects by showing the subjects six different clips of movies chosen in a manner to arouse spontaneous emotional reactions that would produce natural facial expression.