Facial dynamics contain useful information for facial expression recognition (FER). However, exploiting dynamics in FER is challenging. This is mainly due to a variety of expression transitions. For example, video sequences belonging to a same emotion class may have different characteristics in transition duration and/or transition type (e.g., onset versus offset). The temporal mismatches between query and training video sequences could degrade the FER. This paper proposes a new partial matching framework that aims to overcome the temporal mismatch of expression transition. During the training stage, we construct an over-complete transition dictionary where many possible partial expression transitions are contained. During the test stage, we extract a number of partial expression transitions from a query video sequence. Each partial expression transition is analyzed individually. This increases the possibility of matching a partial expression transition in the query video sequence against the partial expression transitions in the over-complete transition dictionary. To make a partial matching subject-independent and robust to the temporal mismatch, each partial expression transition is defined as facial shape displacement between a pair of face clusters. Experimental results show that the proposed method is robust to variations of transition duration and transition type in subject-independent recognition.

Considerable research efforts have been made for face recognition in various real-world applications. However, degraded face images, acquired in the real-world, make face recognition difficult. In this paper, we propose a new face image quality assessment that aims to realize a robust and reliable face recognition system. The proposed method considers two factors for face image quality, i.e., visual quality and mismatch between training and test face images. A face image quality assessor is learned based on the two factors to discriminate useful faces from unuseful ones. The proposed face image quality assessment model is robust and adaptive to face recognition systems by employing a learned assessment. Our experimental results on a challenging database show significant improvement in face recognition accuracy by the proposed method.

Recently, high-performance face recognition has attracted research attention in real-world scenarios. Thanks to the advances in sensor technology, face recognition system equipped with multiple sensors has been widely researched. Among them, face recognition system with near-infrared imagery has been one important research topic. In this paper, complementary effect resided in face images captured by nearinfrared and visible rays is exploited by combining two distinct spectral images (i.e., face images captured by near-infrared and visible rays). We propose a new texture feature (i.e., multispectral texture feature) extraction method with synthesized face images to achieve high-performance face recognition with illumination-invariant property. The experimental results show that the proposed method enhances the discriminative power of features thanks the complementary effect.

Recently, recognizing spontaneous facial expression has gained increasing attention in various emerging applications related to human affect. Spontaneous facial expression may generally have different temporal characteristics across subjects, emotion types, and so on. In this paper, we proposed a facial expression recognition (FER) method which adaptively magnifies a subtle facial motion based on its temporal characteristics. In training stage, we learn the relations between the temporal characteristics of facial motions and their discriminative temporal filtering. The learned model is used to automatically predict the most discriminative temporal filtering that magnifies the subtle facial motion in a test sequence. Experimental result shows that the proposed FER using the adaptive motion magnification performed clearly better than FER using non-adaptive motion magnification as well as FER without motion magnification.

In real world facial expression recognition, blurred face images could hamper achieving high performance due to the lack of distinct edges and textures. In this paper, we propose a new feature extraction method that is robust to blurred face images for facial expression recognition. In the proposed method, the facial feature is extracted adaptively depending on the image sharpness, aiming to achieve robustness against blurred face images. Experimental results on blurred face images demonstrate that the proposed method outperforms the exiting feature extraction method.

Age classification in the real world is a very challenging task due to the large variation of face appearances (e.g., a variety of human races, genders, facial expressions, poses etc.). In this paper, we propose a new age classification method using local modeling of age group to deal with large variation problem. The local modeling is built by clustering training faces within an age group. Nearest face clusters in the local modeling to a test face contribute in determining the age group of the test face. This enables us to reduce the effect of the variation unrelated to age. For comparing the test face with the face clusters, we combine two complementary similarities that consider the cluster centroid and the intra-cluster variation. Experimental result on a real-world dataset shows that our local modeling based approach is superior to global modeling based approach, achieving state-of-the-art performance.

Recently, the development of practical face recognition (FR) system has received much attention. Despite of its extensive study, the FR performance could be severely degraded in real-life scenario (e.g., CCTV surveillance), due to uncontrolled face image conditions of pose/alignment, blur, and brightness. This paper proposes new automated face quality assessment (FQA) framework built-in to a practical FR system. In the proposed framework, three quality factors in face images are rapidly evaluated owing to a cascaded classification. Only face images that have been verified by the FQA are used in recognition phase. Our experiment shows that the cascaded FQA can successfully discard face images that could negatively affect FR.

Automatic facial expression recognition (FER) is becoming increasingly important in the area of affective computing systems because of its various emerging applications such as human-machine interface and human emotion analysis. Recently, sparse representation based FER has become popular and has shown an impressive performance. However, sparse representation could often produce less meaningful sparse solution for FER due to intra-class variation such as variation in identity or illumination. This paper proposes a new sparse representation based FER method, aiming to reduce the intra-class variation while emphasizing the facial expression in a query face image. To that end, we present a new method for generating an intra-class variation image of each expression by using training expression images. The appearance of each intra-class variation image could be close to the appearance of the query face image in identity and illumination. Therefore, the differences between the query face image and its intra-class variation images are used as the expression features for sparse representation. Experimental results show that the proposed FER method has high discriminating capability in terms of improving FER performance. Further, the intra-class variation images of non-neutral expressions are complementary with that of neutral expression, for improving FER performance.

This paper proposes a novel face descriptor based on color information, i.e., so-called local color vector binary patterns (LCVBPs), for face recognition (FR). The proposed LCVBP consists of two discriminative patterns: color norm patterns and color angular patterns. In particular, we have designed a method for extracting color angular patterns, which enables to encode the discriminating texture patterns derived from spatial interactions among different spectral-band images. In order to perform FR tasks, the proposed LCVBP feature is generated by combining multiple features extracted from both color norm patterns and color angular patterns. Extensive and comparative experiments have been conducted to evaluate the proposed LCVBP feature on five public databases. Experimental results show that the proposed LCVBP feature is able to yield excellent FR performance for challenging face images. In addition, the effectiveness of the proposed LCVBP feature has successfully been tested by comparing other state-of-the-art face descriptors.

This paper proposes local color vector binary pattern (LCVBP) as a new color based face descriptor for face recognition (FR). In the proposed LCVBP, color pixel pattern of different spectral images is extracted as discriminative information. Further, to maximize a complementary effect taken from different color channels, color angle pattern between a pair of different spectral images is incorporated into the creation of LCVBP. Experimental results on three public databases (CMU-PIE, XM2VTSDB, and Color FERET) show that the proposed color texture feature is able to significantly improve FR performance, compared to the conventional grayscale texture features including Gabor wavelet and Local Binary Pattern (LBP). In particular, the proposed LCVBP is able to provide excellent FR performance for face images taken under severe illumination variation.

In this paper, we propose a new color face recognition (FR) method which effectively employs feature selection algorithm in order to find the set of optimal color components (from various color models) for FR purpose. The proposed FR method is also designed to improve FR accuracy by combining the selected color components at the feature level. The effectiveness of the proposed color FR method has been successfully demonstrated using two public CMU-PIE and Color FERET face databases (DB). In our comparative experiments, traditional grayscale-based FR, previous color-based FR, and popular local binary pattern (LBP) based FR methods were compared with the proposed method. Experimental results show that our color FR method performs better than the aforementioned three different FR approaches. In particular, the proposed method can achieve 7.81% and 18.57% improvement in FR performance on the CMU-PIE and Color FERET DB, respectively, compared to representative color-based FR solutions previously developed.

A new approach for on-line cursive script recognition that combines a letter spotting technique with an island-driven lattice search algorithm is presented. Initially, all plausible letter components within an input pattern are detected, using a letter spotting technique based on hidden Markov models. A word hypothesis lattice is generated as a result of the letter spotting. Then an island-driven search algorithm is performed to find the optimal path on the word hypothesis lattice, which corresponds to the most probable word among the dictionary words. The results of this experiment suggest that the proposed method works effectively in recognizing English cursive words. In a word recognition test, the average 85.4% word accuracy was obtained.