The advancement of generative artificial intelligence has led to the creation of more diverse and realistic fake facial images. This poses serious threats to personal privacy and can contribute to the spread of misinformation. Existing deepfake detection methods usually utilize prior knowledge about forged clues to design complex modules, achieving excellent performance in the intra-domain settings. However, their performance usually suffers from a significant decline in unseen forgery patterns. It is thus desirable to develop a generalized deepfake detection method using a neat network structure. In this paper, we propose a simple yet efficient framework to transfer a powerful large-scale vision model like ViT to the downstream deepfake detection task, namely the generalized deepfake detection framework (GenDF). Concretely, we first propose a deepfake-specific representation learning (DSRL) scheme to learn different discontinuity patterns across patches inside a fake facial image and continuity between patches within a real counterpart in a low-dimensional space. To further alleviate the distribution mismatch between generic real images and human facial images consisting of both real and fake, we introduce a feature space redistribution (FSR) scheme to separately optimize the distributions of real and fake feature space, enabling the model to learn more distinctive representations. Furthermore, to enhance the generalization performance on unseen forgery patterns produced by constantly evolving facial manipulation techniques and diverse variations on real faces, we propose a classification-invariant feature augmentation (CIFAug) function without trainable parameters. CIFAug expands the scopes of real and fake feature space along directions orthogonal to the classification direction, enabling the model to learn more generalizable features while preserving discrimination. Extensive experiments demonstrate that our method achieves state-of-the-art generalization performance in cross-domain and cross-manipulation settings with only 0.28M trainable parameters.

Self-supervised monocular visual odometry has a crucial advantage of not depending on labels and has shown significant performance in autonomous driving and robotics. However, recent methods suffer from limited feature representations as they depend on coarse semantic masks to handle dynamic objects, resulting in diminished accuracy in dynamic environments. In contrast to these coarse-grained methods, we present Fine-MVO, a novel self-supervised monocular visual odometry that aims to address dynamic objects using implicit fine-grained feature representations, thus achieving excellent accuracy and robustness in dynamic environments. First, Fine-MVO provides an efficient cross-feature augmentation module and a novel loss weight balance strategy to effectively leverage fine-grained features with implicit semantic information, leading to a great improvement in the depth estimation accuracy, especially on object boundaries in the scenes. Secondly, we design a novel pose-feature enhancement module and an effective two-stage training policy to empower the pose network to focus on robust static regions and temporal information, thereby enhancing the pose estimation performance in dynamic and long-term environments. Extensive experimental results demonstrate the excellent accuracy and generalization of Fine-MVO. Specifically, Fine-MVO achieves a remarkable 36.80% improvement in pose accuracy over the state-of-the-art method on the KITTI dataset, which even breaks through the performance of loop closure within geometry-based visual odometry methods. Furthermore, Fine-MVO exhibits satisfactory generalization on the outdoor dataset AirDOS-Shibuya, attaining a notable improvement of 28.22% over current advanced method. Excitingly, Fine-MVO also reveals outstanding generalization on the indoor dataset TUM-RGBD.

Yang Ping, JunYu Lu, Ruyi Gan, Junjie Wang, Yuxiang Zhang, Pingjian Zhang, Jiaxing Zhang. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2023.