We present a fully automated three-dimensional model-based, real-time capable approach to distinguish six universal facial expressions from visual images of human faces. The face model is fitted to the images of a publicly available data base. From the model parameters two sets of features are computed, person-specific and non-person-specific to estimate the facial expression visible in the current image sequence. We integrate the complex, state-of-the-art Candide-3 face model, which is specifically appropriate for facial expression recognition and is also publicly available, which further increases the comparability of our approach. Our experimental evaluation presents recognition rates of the six basic facial expressions defined by Ekman et al.

Computers have been widely deployed to our daily lives, but human-computer interaction still lacks intuition. Researchers intend to resolve these shortcomings by augmenting traditional systems with human-like interaction capabilities. Knowledge about human emotion, behavior, and intention is necessary to construct convenient interaction mechanisms. Today, dedicated hardware often infers the emotional state from human body measures. Similar to humans interpreting facial expressions, our approach acquires video information using standard hardware that does not interfere with people to accomplish this task. It exploits model-based techniques that accurately localize facial features, seamlessly track them through image sequences, and finally interpret the visible information. We make use of state-of-the-art techniques and specifically adapt most of the components involved to this scenario, which provides high accuracy and real-time capability. We base our experimental evaluation on publicly available databases and compare its results to related approaches. Our proof-of-concept demonstrates the feasibility of our approach and shows promising for integration into various applications.

Our system runs at 10 fps on a 2.0 GHz processor and an image resolution of 640times480 pixels. High quality objective functions that are learned from annotated example images ensure both an accurate and fast computation of the model parameters. Our demonstrator for facial expression estimation has been presented at several events with political audience and on TV. However, the approach of robust face models fitting, forms the basis of various more applications such as gaze detection or gender estimation. The drawback of our approach is that the data base from which the objective function is learned needs to cover all aspects of face properties. If, for instance, the database did not contain images of bearded men the objective function will fail when confronted with such an image. Furthermore, the data base has to be manually annotated. Although no expert knowledge is required, this task requires a considerable amount of time. An online fitting demonstration is available.

Due to their use of information contained in texture, active appearance models (AAM) generally outperform active shape models (ASM) in terms of fitting accuracy. Although many extensions and improvements over the original AAM have been proposed, on of the main drawbacks of AAMs remains its dependence on good initial model parameters to achieve accurate fitting results. In this paper, we determine the initial model parameters for AAM fitting with ASM fitting, and use machine learning techniques to improve the scope and accuracy of ASM fitting. Combining the precision of AAM fitting with the large radius of convergence of learned ASM fitting improves the results by an order of magnitude, as our empirical evaluation on a database of publicly available benchmark images demonstrates.

Efficient and accurate localization of the components of human faces, such as skin, lips, eyes, and brows, provides benefit to various real-world applications. However, high intra-class and small inter-class variations in color prevent simple but quick pixel classifiers from yielding robust results. In contrast, more elaborate classifiers consider shape or region features but they do not achieve real-time performance. In this paper, we show that it definitely is possible to robustly determine the facial components and achieve far more than real-time performance. We also use quick pixel-level classifiers and provide them with a set of pixel features that are adapted to the image characteristics beforehand. We do not manually select the pixel features and specify the calculation rules. In contrast, our idea is to provide a multitude of features and let the machine learning algorithm decide which of them are important. The evaluation draws a comparison to fixed approaches that do not adapt the computation of the features to the image content in any way. The obtained accuracy is precise enough to be used for real-world applications such as for model-based interpretation of human faces.

While the " 'quasi-state-of-the-art'" towards acoustic emotion recognition relies on multivariate time-series analysis of e.g. pitch, energy, or MFCC by statistical functionals as moments or extrema, only few respect statistical noise by outliers due to too long segments as turns. Such noise can be overcome by hierarchical functionals as means of extrema over smaller units as words or chunks. Segmentation of such units however usually relies on transcription. We therefore discuss hierarchical functionals based on automatic segmentation and their systematic generation as opposed to common expert-driven selection. To cope with rapidly growing feature spaces iquest5k, we discuss data-driven two-stage compression based on SVM- SFFS. Extensive test-runs are carried out on two known emotion and behavior corpora, and show superiority of the suggested approach.

Surveillance of drivers, pilots or passengers possesses significant potential for increased security within passenger transport. In an automotive setting the interaction can e.g. be improved by social awareness of an MMI. As further example security marshals can be efficiently positioned guided by according systems. Within this scope the detection of security relevant behavior patterns as aggressiveness or stress is discussed. The focus lies on real-life usage respecting online processing, subject independency, and noise robustness. The approach introduced employs multivariate time-series analysis for the synchronization and data reduction of audio and video by brute-force feature generation. By combined optimization of the large audiovisual space accuracy is boosted. Extensive results are reported on aviation behavior, as well as in particular for the audio channel on numerous standard corpora. The influence of noise will be discussed by representative carnoise overlay.

Model-based techniques have proven to be successful in interpreting the large amount of information contained in images. Associated fitting algorithms search for the global optimum of an objective function, which should correspond to the best model fit in a given image. Although fitting algorithms have been the subject of intensive research and evaluation, the objective function is usually designed ad hoc, based on implicit and domain-dependent knowledge. In this article, we address the root of the problem by learning more robust objective functions. First, we formulate a set of desirable properties for objective functions and give a concrete example function that has these properties. Then, we propose a novel approach that learns an objective function from training data generated by manual image annotations and this ideal objective function. In this approach, critical decisions such as feature selection are automated, and the remaining manual steps hardly require domain-dependent knowledge. Furthermore, an extensive empirical evaluation demonstrates that the obtained objective functions yield more robustness. Learned objective functions enable fitting algorithms to determine the best model fit more accurately than with designed objective functions.

Great interest is recently shown in behavior modeling, especially in public surveillance tasks. In general it is agreed upon the benefits of use of several input cues as audio and video. Yet, synchronization and fusion of these information sources remains the main challenge. We therefore show results for a feature space combination, which allows for overall feature space optimization. Audio and video features are thereby firstly derived as low-level-descriptors. Synchronization and feature combination is achieved by multivariate time-series analysis. Test-runs on a database of aggressive, cheerful, intoxicated, nervous, neutral, and tired behavior in an airplane situation show a significant improvement over each single modality.

Model-based image interpretation extracts high-level information from images using a priori knowledge about the object of interest. The computational challenge in model fitting is to determine the model parameters that best match a given image, which corresponds to finding the global optimum of the objective function. When it comes to the robustness and accuracy of fitting models to specific images, humans still outperform state- of-the-art model fitting systems. Therefore, we propose a method in which non-experts can guide the process of designing model fitting algorithms. In particular, this paper demonstrates how to obtain robust objective functions for face model fitting applications, by learning their calculation rules from example images annotated by humans. We evaluate the obtained function using a publicly available image database and compare it to a recent state-of-the-art approach in terms of accuracy.

Model-based methods in computer vision have proven to be a good approach for compressing the large amount of information in images. Fitting algorithms search for those parameters of the model that optimise the objective function given a certain image. Although fitting algorithms have been the subject of intensive research and evaluation, the objective function is usually designed ad hoc and heuristically with much implicit domain-dependent knowledge. This paper formulates a set of requirements that robust objective functions should satisfy. Furthermore, we propose a novel approach that learns the objective function from training images that have been annotated with the preferred model parameters. The requirements are automatically enforced during the learning phase, which yields generally applicable objective functions. We compare the performance of our approach to other approaches. For this purpose, we propose a set of indicators that evaluate how well an objective function meets the stated requirements. 1