Distal facial Electromyography (EMG) can be used to detect smiles and frowns with reasonable accuracy. It capitalises on volume conduction to detect relevant muscle activity, even when the electrodes are not placed directly on the source muscle. The main advantage of this method is to prevent occlusion and obstruction of the facial expression production, whilst allowing EMG measurements. However, measuring EMG distally entails that the exact source of the facial movement is unknown. Therefore, we investigated whether we could identify specific Facial Action Units (AUs) from distal facial EMG after an initial calibration phase with Computer Vision (CV). We compared Support Vector Machines (SVM) and Random Forest (RF) with several types of feature engineering and early fusion of the two modalities. The detection performance for AU6 (Orbicularis Oculi) and AU12 (Zygomaticus Major) was estimated by calculating the agreement with Facial Action Coding System (FACS) certified coders. The best results were achieved using Random Forest. Using a fusion of CV and EMG features resulted in F1 scores of 0.83 for AU6; and the fusion of engineered EMG plus CV returned an F1 score of 0.81 for AU12. Both these results are well above the CV baseline that shows F1 scores of 0.56 and 0.62 for AU6 and AU12 respectively. This demonstrates the potential of distal EMG to detect individual facial movements. It also enables researchers to compare the results measured with this wearable device to psychological research on facial expressions using FACS. Using a wearable enables measurements with higher ecological validity. Finally, we observed that EMG activity starts before the onset of visually perceived movement. Because of this, the agreement between EMG-based methods and FACS coders might be underestimating the ground truth.

Electro-Dermal Activity (EDA)and head movement have been shown to correlate with felt affect. Given the easiness to measure them, they are suitable as a wearable affective assessment tool. Arguably, autonomic affective responses such as EDA are less affected by volition than the production of other embodied cues of affect such as facial expressions. Moreover, head movement has been shown as a reliable source of information about the intention behind a facial expression. Therefore, we explored the feasibility of using EDA measured from the neck and head movement to make inferences about the nature of facial expressions, in particular, smiles. EDA was measured simultaneously from the hand and the neck of participants displaying spontaneous and posed smiles. Our results show that both measurement locations are highly correlated. Furthermore, EDA signals carry information about the spontaneity of the measured smiles, as shown by a classification accuracy of about 90%. Finally, head movement turned out to be rather revealing, with classification accuracy reaching about 99 %.

Positive experiences are a success metric in product and service design. Quantifying smiles is a method of assessing them continuously. Smiles are usually a cue of positive affect, but they can also be fabricated voluntarily. Automatic detection is a promising complement to human perception in terms of identifying the differences between smile types. Computer vision (CV) and facial distal electromyography (EMG) have been proven successful in this task. This is the first study to use a wearable EMG that does not obstruct the face to compare the performance of CV and EMG measurements in the task of distinguishing between posed and spontaneous smiles. The results showed that EMG has the advantage of being able to identify covert behavior not available through vision. Moreover, CV appears to be able to identify visible dynamic features that human judges cannot account for. This sheds light on the role of non-observable behavior in distinguishing affect-related smiles from polite positive affect displays.