The widespread emergence of Deepfake content has made it increasingly important to distinguish real and fake faces. Although many methods focus on detecting Deepfake content, only a few address the protection of real faces against forgery. Therefore, this paper proposes a proactive Deepfake detection scheme named FALU, which combines the uniqueness of facial identity features with the robustness of average hashing. The method first divides the input image into facial and non-facial regions, extracts identity-related features from the facial region, encodes them using average hashing, and embeds the result as a watermark into the non-facial region. During detection, the watermark is extracted from the non-facial region and compared with a newly generated hash code from the facial region. High correlation indicates authenticity, while low correlation suggests Deepfake forgery. To facilitate efficient and reliable watermark embedding, FALU integrates the symmetric sampling structure of U-Net with Mamba-like linear attention mechanism, proposing a lightweight encoder network. This scheme ensures the persistent presence of secret information before and after manipulation, thereby enhancing face source detection and tampering identification. Experimental results demonstrate that the proposed scheme effectively counters traditional Deepfake techniques and shows significant potential for preserving personal privacy.

Recent advances have emphasized the importance of meme stickers in open-domain dialogue systems.However, previous studies overlook the one-to-many issue that a single sticker could represent various emotions in different dialogue contexts.Additionally, they require retraining the model for new stickers which did not appear in previous training.To address the above issues, we propose in this paper an Emotion-Aware and Efficient Meme Sticker Dialogue generation framework.In the framework, we design an Emotion Adaptive Prompt to capture the emotional cues from the dialogue history, which is sent to an Emotion-Aware Fusion Decoder to guide the generation of text responses and to a meme sticker selector to choose the corresponding sticker.Furthermore, to improve the stickers' selection efficiency, we further incorporate the few-shot learning strategy into the proposed framework to avoid extensive model retraining for unseen meme stickers.Through extensive experiments, we demonstrate the superior performance of the proposed E 2 MSD compared to existing methods regarding the quality of response generation and the efficiency of meme sticker retrieval.

Multimodal Dialogue agents are often required to respond to conversation history using both textual and visual content. Even though current dialogue studies predominantly strive to generate natural texts or images, they fall short in considering the relevance of multimodal responses within a dialogue context, consequently confining agents from making prudent choices based on multiple alternatives and their associated relevance scores for decision-making. In this paper, we present a bidirectional multimodal dialogue framework that skillfully combines the forward generation of multiple text and image response candidates with reverse selection guided by relevance scores evaluated on dialogue context, facilitating agents in selecting the most suitable multimodal responses. Specifically, the forward generation aspect of our framework leverages a stage-wise approach, first producing textual replies and composite visual descriptions from the dialogue context, followed by the generation of visual responses aligned with the descriptions. In the reverse selection process, visual responses are translated into tangible descriptive texts that, in conjunction with textual responses, are inversely tied back to the dialogue context for relevance assessment, assigning a reference score to each multimodal response candidate to assist the intelligent agent in making informed decisions. Experimental outcomes demonstrate that our proposed bidirectional dialogue response framework markedly elevates performance in both automatic and human evaluations, yielding a range of contextually fitting multimodal responses for selection.

The easy sharing of multimedia content on social media has caused a rapid dissemination of fake news, which threatens society’s stability and security. Therefore, fake news detection has garnered extensive research interest in the field of social forensics. Current methods primarily concentrate on the integration of textual and visual features but fail to effectively exploit multi-modal information at both fine-grained and coarse-grained levels. Furthermore, they suffer from an ambiguity problem due to a lack of correlation between modalities or a contradiction between the decisions made by each modality. To overcome these challenges, we present a Multi-grained Multi-modal Fusion Network (MMFN) for fake news detection. Inspired by the multi-grained process of human assessment of news authenticity, we respectively employ two Transformer-based pre-trained models to encode token-level features from text and images. The multi-modal module fuses fine-grained features, taking into account coarse-grained features encoded by the CLIP encoder. To address the ambiguity problem, we design uni-modal branches with similarity-based weighting to adaptively adjust the use of multi-modal features. Experimental results demonstrate that the proposed framework outperforms state-of-the-art methods on three prevalent datasets.