Most micro- and macro-expression spotting methods in untrimmed videos suffer from the burden of video-wise collection and frame-wise annotation. Weakly supervised expression spotting (WES) based on video-level labels can potentially mitigate the complexity of frame-level annotation while achieving fine-grained frame-level spotting. However, we argue that existing weakly supervised methods are based on multiple instance learning (MIL) involving inter-modality, inter-sample, and inter-task gaps. The inter-sample gap is primarily from the sample distribution and duration. Therefore, we propose a novel and simple WES framework, MC-WES, using multi-consistency collaborative mechanisms that include modal-level saliency, video-level distribution, label-level duration and segment-level feature consistency strategies to implement fine frame-level spotting with only video-level labels to alleviate the above gaps and merge prior knowledge. The modal-level saliency consistency strategy focuses on capturing key correlations between raw images and optical flow. The video-level distribution consistency strategy utilizes the difference of sparsity in temporal distribution. The label-level duration consistency strategy exploits the difference in the duration of facial muscles. The segment-level feature consistency strategy emphasizes that features under the same labels maintain similarity. Experimental results on three challenging datasets-CAS(ME)$^{2}$2, CAS(ME)$^{3}$3, and SAMM-LV-demonstrate that MC-WES is comparable to state-of-the-art fully supervised methods.

Micro- and macro-expression spotting in an untrimmed video is a challenging task, due to the mass generation of false positive samples. Most existing methods localize higher response areas by extracting hand-crafted features or cropping specific regions from all or some key raw images. However, these methods either neglect the continuous temporal information or model the inherent human motion paradigms (background) as foreground. Consequently, we propose a novel two-stream network, named Local suppression and Global enhancement Spotting Network (LGSNet), which takes segment-level features from optical flow and videos as input. LGSNet adopts anchors to encode expression intervals and selects the encoded deviations as the object of optimization. Furthermore, we introduce a Temporal Multi-Receptive Field Feature Fusion Module (TMRF$^{3}$M) and a Local Suppression and Global Enhancement Module (LSGEM), which help spot short intervals more precisely and suppress background information. To further highlight the differences between positive and negative samples, we set up a large number of random pseudo ground truth intervals (background clips) on some discarded sliding windows to accomplish background clips modeling to counteract the effect of non-expressive face and head movements. Experimental results show that our proposed network achieves state-of-the-art performance on the CAS(ME)$^{2}$, CAS(ME)$^{3}$and SAMM-LV datasets.

Macro- and micro-expression spotting is a very challenging task to locate their occurrence intervals in long face videos. In this paper, we propose an efficient two-stream network named location suppression based spotting network (LSSNet), which includes three parts. First, the optical flow is extracted using the traditional TV-L1 algorithm which captures subtle facial movements while adding temporal information to alleviate the problem of insufficient samples. Then, fixed length features are extracted from the sampled optical flow and raw images by an I3D model, which is used to set sliding windows. Finally, location suppression modules (LSMs) are added to the pyramidal convolutional neural network (CNN) to reduce the proposals with too long and too short intervals. In addition, we use two different methods, named top_k and top_threshold, for validation. We adopt leave-one-subject-out (LOSO) to train our model on CAS(ME)2 and SAMM-LV. Experimental results show that our LSSNet achieves the state-of-the-art result with top_threshold, especially on the CAS(ME)2 dataset. The code is available at https://github.com/williamlee91/mer_spot.