Dasol Choi, DongGeon Lee, Brigitta Jesica Kartono, Helena Berndt, Taeyoun Kwon, Joonwon Jang, Haon Park, Hwanjo Yu, Minsuk Kahng. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.

Rapid deployment of vision-language models (VLMs) magnifies safety risks, yet most evaluations rely on artificial images.This study asks: How safe are current VLMs when confronted with meme images that ordinary users share?To investigate this question, we introduce MEMESAFETYBENCH, a 50,430-instance benchmark pairing real meme images with both harmful and benign instructions.Using a comprehensive safety taxonomy and LLMbased instruction generation, we assess multiple VLMs across single and multi-turn interactions.We investigate how real-world memes influence harmful outputs, the mitigating effects of conversational context, and the relationship between model scale and safety metrics.Our findings demonstrate that VLMs are more vulnerable to meme-based harmful prompts than to synthetic or typographic images.Memes significantly increase harmful responses and decrease refusals compared to textonly inputs.Though multi-turn interactions provide partial mitigation, elevated vulnerability persists.These results highlight the need for ecologically valid evaluations and stronger safety mechanisms.MEMESAFETYBENCH is publicly available at https://github.com/ oneonlee/Meme-Safety-Bench.Warning: This paper includes examples of harmful language and images that may be sensitive or uncomfortable.Reader discretion is recommended.

Consistency within a document is a crucial feature indicative of its quality. Recently, within the vast amount of information produced across various media, there exists a significant number of low-quality documents that either lack internal consistency or contain content utterly unrelated to their headlines. Such low-quality documents induce fatigue in readers and undermine the credibility of the media source that provided them. Consequently, research to automatically detect these low-quality documents based on natural language processing is imperative. In this study, we introduce a hierarchical graph convolutional network (HGCN) that can detect internal inconsistencies within a document and incongruences between the title and body. Moreover, we constructed the Inconsistency Dataset, leveraging published news data and its meta-data, to train our model to detect document inconsistencies. Experimental results demonstrated that the HGCN achieved superior performance with an accuracy of 91.20% on our constructed Inconsistency Dataset, outperforming other comparative models. Additionally, on the publicly available incongruent-related dataset, the proposed methodology demonstrated a performance of 92.00%, validating its general applicability. Finally, an ablation study further confirmed the significant impact of meta-data utilization on performance enhancement. We anticipate that our model can be universally applied to detect and filter low-quality documents in the real world.

Joonwon Jang, Sanghwan Jang, Wonbin Kweon, Minjin Jeon, Hwanjo Yu. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2024.