With the continuous emergence of various social media platforms frequently used in daily life, the multimodal meme understanding (MMU) task has been garnering increasing attention. MMU aims to explore and comprehend the meanings of memes from various perspectives by performing tasks such as metaphor recognition, sentiment analysis, intention detection, and offensiveness detection. Despite making progress, limitations persist due to the loss of fine-grained metaphorical visual clue and the neglect of multimodal text-image weak correlation. To overcome these limitations, we propose a multi-granular multimodal clue fusion model (MGMCF) to advance MMU. Firstly, we design an object-level semantic mining module to extract object-level image feature clues, achieving fine-grained feature clue extraction and enhancing the model's ability to capture metaphorical details and semantics. Secondly, we propose a brand-new global-local cross-modal interaction model to address the weak correlation between text and images. This model facilitates effective interaction between global multimodal contextual clues and local unimodal feature clues, strengthening their representations through a bidirectional cross-modal attention mechanism. Finally, we devise a dual-semantic guided training strategy to enhance the model's understanding and alignment of multimodal representations in the semantic space. Experiments conducted on the widely-used MET-MEME bilingual dataset demonstrate significant improvements over state-of-the-art baselines. Specifically, there is an 8.14% increase in precision for offensiveness detection task, and respective accuracy enhancements of 3.53%, 3.89%, and 3.52% for metaphor recognition, sentiment analysis, and intention detection tasks. These results, underpinned by in-depth analyses, underscore the effectiveness and potential of our approach for advancing MMU.

Li Zheng, Sihang Wang, Hao Fei, Zuquan Peng, Fei Li, Jianming Fu, Chong Teng, Donghong Ji. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025.

Detecting malicious URLs is of great significance to reduce cyber crimes and maintain Internet security. Currently, Deep Learning (DL) techniques have been widely used to improve the classical malicious URL detection models, as DL-based detection models can perform an in-depth analysis of the text information of the URL, and detect the fishing URLs of unknown cyber attack types with high accuracy. Any missed blocking of malicious URLs can potentially result in a huge loss of information and property. In this paper, we focus on the vulnerability of the existing DL-based malicious URL detection models and show that they are sensitive to adversarial samples. First, we construct URL adversarial samples based on the component-level and character-level perturbations and use them to attack mainstream DL-based detection models, resulting in obvious decreases in the detection accuracies. Meanwhile, the perturbations are under the constraints that each adversarial sample URL is hardly distinguished from the original URL with naked eyes. Furthermore, under most circumstances, the adversarial samples constructed by replacing 14 types of characters and perturbing other all components except the scheme component lead to the largest increased number of missed blocking of malicious URLs, i.e., a bigger drop in the accuracy than other constructed methods. Finally, extensive experiments demonstrate the effectiveness of our adversarial examples. Even if the adversarial training is used against our adversarial samples, the adversarial samples still work and bring oblivious decreases in their accuracy.