In forgery detection, temporal forgery localization offers a more nuanced perspective than binary detection by providing more precise temporal boundaries of manipulations. However, its need for frame-wise annotations limits real-world practicality. Therefore, we present the task of Weakly Supervised Temporal Forgery Localization (WS-TFL), which aims to localize forgery segments in videos given only video-level labels for training. To tackle the absence of frame-wise annotations in weakly supervised settings, pseudo-label learning presents a viable solution for WS-TFL. However, pseudo labels often suffer from inaccuracy (i.e. contain noise) due to lack of supervision. Inspired by the ability of curriculum learning in handling noisy data, this work proposes Curriculum Pseudo Labeling (CPL), a simple yet effective strategy to address noise in pseudo labels. Specifically, pseudo-label learning leverages proposals generated from a base model as coarse pseudo labels to retrain the model for better performance. To refine pseudo labels and reduce noise, we design a learning curriculum that ranks them by quality, where we analyze the multiple-instance learning essence of WS-TFL and further introduce forgery prior knowledge. Experiments show that CPL consistently improves performance across different baselines, with a 35.24% and 27.63% increase in average precision to the default baseline on Lav-DF and TVIL datasets, respectively.

The impressive development of facial manipulation techniques has raised severe public concerns. Identity-aware methods, especially suitable for protecting celebrities, are seen as one of promising face forgery detection approaches with additional reference video. However, without in-depth observation of fake video’s characteristics, most existing identity-aware algorithms are just naive imitation of face verification model and fail to exploit discriminative information. In this article, we argue that it is necessary to take both spatial and temporal perspectives into consideration for adequate inconsistency clues and propose a novel forgery detector named SpatioTemporal IDentity network (STIDNet). To effectively capture heterogeneous spatiotemporal information in a unified formulation, our STIDNet is following a knowledge distillation architecture that the student identity extractor receives supervision from a spatial information encoder (SIE) and a temporal information encoder (TIE) through multiteacher training. Specifically, a regional sensitive identity modelling paradigm is proposed in SIE by introducing facial blending augmentation but with uniform identity label, thus encourage model to focus on spatial discriminative region like outer face. Meanwhile, considering the strong temporal correlation between audio and talking face video, our TIE is devised in a cross-modal pattern that the audio information is introduced to supervise model exploiting temporal personalized movements. Benefit from knowledge transfer from SIE and TIE, STIDNet is able to capture individual’s essential spatiotemporal identity attributes and sensitive to even subtle identity deviation caused by manipulation. Extensive experiments indicate the superiority of our STIDNet compared with previous works. Moreover, we also demonstrate STIDNet is more suitable for real-world implementation in terms of model complexity and reference set size.

Face forgery detection has attracted much attention due to the ever-increasing social concerns caused by facial manipulation techniques. Recently, identity-based detection methods have made considerable progress, which is especially suitable in the celebrity protection scenario. However, they still suffer from two main limitations: (a) generic identity extractor is not specifically designed for forgery detection, leading to nonnegligibleIdentity Representation Biasto forged images. (b) existing methods only analyze the identity representation of each image individually, but ignores the query-reference interaction for inconsistency exploiting. To address these issues, a novelInconsistency Exploiting based Identity Rectification Network(IEIRNet) is proposed in this paper. Firstly, for the identity bias rectification, the IEIRNet follows an effective two-branches structure. Besides theGeneric Identity Extractor(GIE) branch, an essentialBias Diminishing Module(BDM) branch is proposed to eliminate the identity bias through a novelAttention-based Bias Rectification(ABR) component, accordingly acquiring the ultimate discriminative identity representation. Secondly, for query-reference inconsistency exploiting, anInconsistency Exploiting Module(IEM) is applied in IEIRNet to comprehensively exploit the inconsistency clues from both spatial and channel perspectives. In the spatial aspect, an innovative region-aware kernel is derived to activate the local region inconsistency with deep spatial interaction. Afterward in the channel aspect, a coattention mechanism is utilized to model the channel interaction meticulously, and accordingly highlight the channel-wise inconsistency with adaptive weight assignment and channel-wise dropout. Our IEIRNet has shown effectiveness and superiority in various generalization and robustness experiments.

The malicious usage of facial manipulation techniques boosts the desire of face forgery detection research. Recently, identity-based approaches have attracted much attention due to the effective observation of identity inconsistency. However, there are still several nonnegligible problems: (1) generic identity extractor is totally trained on real images, leading to enormous identity representation bias during processing forged content; (2) the identity information of forged image is hybrid and presents regional distribution, while the single global identity feature is hard to reflect this local identity inconsistency. To solve the above problems, in this paper a novel Region-Aware Identity Rectification Network (RAIRNet) is proposed to effectively rectify the identity bias and adaptively exploit the inconsistency local region. Firstly, for the identity bias problem, our RAIRNet is devised in a two-branch architecture, which consists of a Generic Identity Extractor (GIE) branch and a Bias Diminishing Module (BDM) branch. The BDM branch is designed to rectify the bias introduced by GIE branch through a prototype-based training schema. This two-branch architecture effectively promotes model to adapt to forged content while maintaining the focus on identity space. Secondly, for local identity inconsistency exploiting, a novel Meta Identity Filter Generator (MIFG) is devised in a meta-learning way to generate the region-aware filter based on identity prior. This region-aware filter can adaptively exploit the local inconsistency clues and activate the discriminative local region. Moreover, to balance the local-global information and highlight the forensic clues, an Adaptive Weight Assignment Mechanism (AWAM) is proposed to assign adaptive importance weight to two branches. Extensive experiments on various datasets show the superiority of our RAIRNet. In particular, on the challenging DFDCp dataset, our approach outperforms previous binary-based and identity-based methods by 10.3% and 5.5% respectively.