Audio-driven talking face synthesis (TFS) enables the creation of realistic speaking videos by combining a single facial image with a speech audio clip. Unlike other facial features that naturally deform during speech, teeth represent unique rigid structures whose shape and size should remain constant throughout the video sequence. However, current methods often produce temporal inconsistencies and artifacts in the teeth region, resulting in a less realistic appearance of the generated videos. To address this, we propose OrthoNet, a plug-and-play framework designed to eliminate unrealistic teeth effects in audio-driven TFS. Our method introduces a Detail-oriented Teeth Aligner module, designed to preserve teeth details and adapt to their shape. It works with a Memory-guided Teeth Stabilizer that integrates a long-term memory bank for global teeth structure and a short-term memory module for local temporal dynamics. Through this framework, OrthoNet acts like an orthodontist for existing Audio2Video methods, ensuring that teeth maintain natural rigidity and temporal consistency even under varying degrees of teeth occlusion. Extensive experiments demonstrate that our method makes the teeth in generated videos appear more natural during speech, significantly enhancing the temporal consistency and structural stability of audio-driven video generation.

Few-shot generative model adaption is a challenging task that aims to adapt a generative model pre-trained on a large-scale source domain dataset to the target domain with limited training samples. Current methods do their best to transfer source domain knowledge to the target generator in different ways. However, the over-fitting problem has always been a thorny problem in model adaption and is hard to solve due to the limited training data. To overcome such issues, we revisit the training process of model adaption and devise hypothetical experiments. Our research has found that current adaption methods fail to fully use the learned knowledge in the source generator. Meanwhile, in the all-parameter training mode, parameters independent of the source domain are also fine-tuned when fitting the few-shot target samples. In order to circumvent such issues, we propose the optimal kernel modulation method for effective few-shot generative model adaption. The idea of optimal transport theory is leveraged to measure the importance of model parameters for knowledge preservation and transfer. Meanwhile, to realize the control of parameter optimization, we adopt the parameter-efficient kernel modulation method according to its importance. Extensive quantitative and qualitative experiments prove the effectiveness and superiority of our method.

Owing to advancements in image synthesis techniques, stylization methodologies for large models have garnered remarkable outcomes. However, when it comes to processing facial images, the outcomes frequently fall short of expectations. Facial stylization is predominantly challenged by two significant hurdles. Firstly, obtaining a large dataset of high-quality stylized images is difficult. The scarcity and diversity of artistic styles make it impractical to compile comprehensive datasets for each style. Secondly, while many methods can transfer colors and strokes from style images, these elements alone cannot fully capture a specific style, which encompasses both concrete and abstract visual elements. Additionally, facial stylization often alters the visual features of the face, making it challenging to balance these changes with the need to retain facial information. To address these issues, we propose a novel method called ACFun, which uses only one style image and one facial image for facial stylization. ACFun comprises an Abstract Fusion Module (AFun) and a Concrete Fusion Module (CFun), which separately learn the abstract and concrete features of the style and face. We also design a Face and Style Imagery Alignment Loss to align the style image with the face image in the latent space. Finally, we generate styled facial images from noise directly to complete the facial stylization task. Experiments show that our method outperforms others in facial stylization, producing highly artistic and visually pleasing results.

Few-shot generative model adaptation aims to obtain an excellent model that generates high-quality and high-diversity images with a few training data. However, a small number of training samples often leads to overfitting of the model, which leads the generated images to lose generative diversity. Existing methods either fail to preserve structural information, leading to overfitting phenomena, or maintain too much structure in the source domain, failing to transfer styles well. To solve these problems, we propose an effective generative model adaptation method with style-guided prompt to balance generative diversity and style transformation. Firstly, by freezing the structure-related parameters of the pre-trained model, we preserve the robustness and diversity of the source domain's generative model, which helps to mitigate overfitting and maintain diversity in the generated images. Secondly, the proposed style-guided prompt method allows us to capture the target domain's style more naturally, facilitating more accurate and efficient style transfer in the generated images. Thirdly, the multi-layer deep contrastive loss is designed to further enhance the generated images' diversity and quality by preserving the generative diversity in the source domain without using extra target domain data. Extensive quantitative and qualitative experiments prove the effectiveness and superiority of our method. Project Page:https://github.com/Patricia886/SGP/.