Facial micro-expressions (MEs) refer to brief spontaneous facial movements that can reveal a person's genuine emotion. They are valuable in lie detection, criminal analysis, and other areas. While deep learning-based ME recognition (MER) methods achieved impressive success, these methods typically require pre-processing using conventional optical flow-based methods to extract facial motions as inputs. To overcome this limitation, we proposed a novel MER framework using self-supervised learning to extract facial motion for ME (SelfME). To the best of our knowledge, this is the first work using an automatically self-learned motion technique for MER. However, the self-supervised motion learning method might suffer from ignoring symmetrical facial actions on the left and right sides of faces when extracting fine features. To address this issue, we developed a symmetric contrastive vision transformer (SCViT) to constrain the learning of similar facial action features for the left and right parts of faces. Experiments were conducted on two benchmark datasets showing that our method achieved state-of-the-art performance, and ablation studies demonstrated the effectiveness of our method.

Facial micro-expression (ME) refers to a brief spontaneous facial movement that can reveal the genuine emotion of a person. The absence of data is a major problem for ME. Thankfully, generative deep neural network models can aid in producing desired samples. In this work, we proposed a deep learning based adaptive dual motion model (ADMM) for generating facial ME samples. A dual motion extraction (DME) module extracts robust motions from two modalities: original color images and edge-based grayscale images, with dual streams. Using edge-based grayscale images can help the method focus on learning subtle movements by eliminating the influences of noises and illumination variants. The motions extracted by the dual streams are fed into an adaptive motion fusion (AMF) module for combing the motions adaptively to generate the dense motion. Our method was trained on the CASME II, SMIC, and SAMM datasets. The evaluation and analysis of the results demonstrated the effectiveness of our method.

Facial micro-expression (FME) refers to a brief spontaneous facial movement that can reveal a person's genius emotion. One challenge in facial micro-expression is the lack of data. Fortunately, generative deep neural network models can assist in the creation of desired images. However, the issues for micro-expressions are the facial variations are too subtle to capture, and the limited training data may make feature extraction difficult. To address these issues, we developed a deep motion retargeting and transfer learning based facial micro-expression generation model (DMT-FMEG). First, to capture subtle variations, we employed a deep motion retargeting (DMR) network that can learn keypoints in an unsupervised manner, estimate motions, and generate desired images. Second, to enhance the feature extraction ability, we applied deep transfer learning (DTL) by borrowing knowledge from macro-expression images. We evaluated our method on three datasets, CASME II, SMIC, and SAMM, and found that it showed satisfactory results on all of them. With the effectiveness of the method, we won the second place in the generation task of the FME 2021 challenge.

Dynamic facial expression recognition has many useful applications in social networks, multimedia content analysis, security systems and others. This challenging process must be done under recurrent problems of image illumination and low resolution which changes at partial occlusions. This paper aims to produce a new facial expression recognition method based on the changes in the facial muscles. The geometric features are used to specify the facial regions i.e., mouth, eyes, and nose. The generic Fourier shape descriptor in conjunction with elliptic Fourier shape descriptor is used as an attribute to represent different emotions under frequency spectrum features. Afterwards a multi-class support vector machine is applied for classification of seven human expression. The statistical analysis showed our approach obtained overall competent recognition using 5-fold cross validation with high accuracy on well-known facial expression dataset.