The performance of face recognition (FR) systems for video surveillance has been shown to improve when the design data is augmented through synthetic face generation. This is true, for instance, with pair-wise matchers (e.g., deep Siamese networks) that rely on a reference gallery, typically with one still image per individual. However, generating synthetic images based on stills (from the source domain) may not improve performance during operations due to the domain shift w.r.t. the target domain. Moreover, despite the emergence of Generative Adversarial Networks (GANs) for realistic synthetic generation, it is often difficult to control the conditions under which synthetic faces are generated. In this paper, a cross-domain face synthesis approach is proposed that integrates a new Controllable GAN (C-GAN). It employs an off-the-shelf 3D face model as a simulator to generate facial images under various poses. The simulated images and noise are input to the C-GAN for realism refinement. It relies on an additional adversarial game as a third player to preserve the identity and specific facial attributes of the refined images. This allows generating realistic synthetic face images that reflect capture conditions in the target domain, while controlling the GAN output such that faces may be generated under desired pose conditions. Experiments were performed using videos from the Chokepoint and COX-S2V datasets, and a deep Siamese network for FR with a single reference still per person. Results indicate that the proposed approach can provide a higher level of accuracy compared to state- of-the-art approaches for synthetic data augmentation1.

Sparse representation-based classification (SRC) techniques have been shown to achieve a high level of performance in video-based face recognition (FR). However, matching faces captured in uncontrolled video conditions against a gallery with a single reference facial still per individual typically yields low accuracy. To improve robustness to intra-class variations, SRC techniques for FR have recently been extended to incorporate variational information from an external generic set into an auxiliary variational dictionary. Despite their success in handling linear variations, probe facial images with non-linear variations due to e.g., changes in pose and expressions, cannot be accurately reconstructed with a linear combination of images from gallery and auxiliary dictionaries because they do not share the same type of variations. In this paper, a new synthetic plus variational model is proposed to account for the non-linearities, particularly with pose variations. It reconstructs a probe image using (1) an auxiliary variational dictionary and (2) an augmented gallery dictionary enriched with a set of synthetic images generated from the reference faces with a wide diversity of pose angles. By solving a newly formulated simultaneous sparsity-based optimization problem, the augmented gallery dictionary is encouraged to share the same sparsity pattern with the variational dictionary for the same pose angles. In this way, each synthetic face in the augmented dictionary is combined with similar facial viewpoint in the variational dictionary. Experimental results obtained on Chokepoint and COX-S2V datasets, using different face representations, indicate that the proposed approach can outperform state-of-the-art SRC-based methods for still-to-video FR with a SSPP.

In video surveillance, face recognition (FR) systems are employed to detect individuals of interest appearing over a distributed network of cameras. The performance of still-to-video FR systems can decline significantly because faces captured in unconstrained operational domain (OD) over multiple video cameras have a different underlying data distribution compared to faces captured under controlled conditions in the enrollment domain with a still camera. This is particularly true when individuals are enrolled to the system using a single reference still. To improve the robustness of these systems, it is possible to augment the reference set by generating synthetic faces based on the original still. However, without the knowledge of the OD, many synthetic images must be generated to account for all possible capture conditions. FR systems may, therefore, require complex implementations and yield lower accuracy when training on many less relevant images. This paper introduces an algorithm for domain-specific face synthesis (DSFS) that exploits the representative intra-class variation information available from the OD. Prior to operation (during camera calibration), a compact set of faces from unknown persons appearing in the OD is selected through affinity propagation clustering in the captured condition space (defined by pose and illumination estimation). The domain-specific variations of these face images are then projected onto the reference still of each individual by integrating an image-based face relighting technique inside the 3-D reconstruction framework. A compact set of synthetic faces is generated that resemble individuals of interest under the capture conditions relevant to the OD. In a particular implementation based on sparse representation classification, the synthetic faces generated with the DSFS are employed to form a cross-domain dictionary that accounts for structured sparsity, where the dictionary blocks combine the original and synthetic faces of each individual. Experimental results obtained with videos from the Chokepoint and COX-S2V data sets reveal that augmenting the reference gallery set of still-to-video FR systems using the proposed DSFS approach can provide a significantly higher level of accuracy compared with the state-of-the-art approaches, with only a moderate increase in its computational complexity.

In still-to-video face recognition (FR), the faces captured with surveillance cameras are matched against reference stills of target individuals enrolled to the system. FR is a challenging problem in video surveillance due to uncontrolled capture conditions (variations in pose, expression, illumination, blur, scale, etc.), and the limited number of reference stills to model target individuals. This paper introduces a new approach to generate multiple synthetic face images per reference still based on camera-specific capture conditions to deal with illumination variations. For each reference still, a diverse set of faces from non-target individuals appearing in the camera viewpoint are selected based on luminance and contrast distortion. These face images are then decomposed into detail layer and large scale layer using an edge-preserving image decomposition to obtain their illumination dependent component. Finally, the large scale layers of these images are morphed with each reference still image to generate multiple synthetic reference stills that incorporate illumination and contrast conditions. Experimental results obtained with the ChokePoint dataset reveal that these synthetic faces produce an enhanced face model. As the number of synthetic faces grows, the proposed approach provides a higher level of accuracy and robustness across a range of capture conditions.

In this study, a novel approach is proposed for mental stress recognition through automatic analysis of eye video sequences. The proposed system consists of five stages: video capturing, fuzzy image processing, signal processing, feature extraction and, classification. The pupil parameters including Pupil Diameter (PD) and Pupil Dilation Acceleration (PDA) are measured using soft computing techniques wherein the eye region is detected using the genetic algorithm (GA), and a fuzzy filter is designed for noise reduction. Edge detection is performed based on fuzzy reasoning and linking is done using Hough transform. Then, signal processing technique is applied to the pupil parameters to extract their most relevant features. Extracted features are imported into the learning system to classify the affective states between "stress" and "relaxed". The Fuzzy SVM (FSVM) is applied to this classification process. In order to induce the stress in subjects, a Stroop color-word test is designed. Also, the results obtained from the pupil parameters are compared with two other physiological signals including Electrocardiogram (ECG) and Photoplethysmogram (PPG). The experimental results indicate the pupil parameters have great potential for stress recognition compared to the other two physiological signals and, proposed stress recognition system is promising.

This paper presents a facial expression recognition approach to recognize the affective states. Feature extraction is a vital step in the recognition of facial expressions. In this work, a novel facial feature extraction method based on Intersecting Cortical Model (ICM) is proposed. The ICM network which is a simplified model of Pulse-Coupled Neural Network (PCNN) model has great potential to perform pixel grouping. In the proposed method the normalized face image is segmented into two regions including mouth, eyes using fuzzy c-means clustering (FCM). Segmented face images are imported into an ICM network with 300 iteration number and pulse image produced by the ICM network is chosen as the face code, then the support vector machine (SVM) is trained for discrimination of different expressions to distinguish the different affective states. In order to evaluate the performance of the proposed algorithm, the face image dataset is constructed and the proposed algorithm is used to classify seven basic expressions including happiness, sadness, fear, anger, surprise and hate The experimental results confirm that ICM network has great potential for facial feature extraction and the proposed method for human affective recognition is promising. Fast feature extraction is the most advantage of this method which can be useful for real world application.