Yu Ying Chiu, Liwei Jiang, Bill Yuchen Lin, Chan Young Park, Shuyue Stella Li, Sahithya Ravi, Mehar Bhatia, Maria Antoniak, Yulia Tsvetkov, Vered Shwartz, Yejin Choi. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025.

Despite recent advancements in visionlanguage models, their performance remains suboptimal on images from non-western cultures, due to underrepresentation in training datasets.Various benchmarks have been proposed to test models' cultural inclusivity, but they have limited coverage of cultures and do not adequately assess cultural diversity across universal as well as culture-specific local concepts.To address these limitations, we introduce the GLOBALRG benchmark, comprising two challenging tasks: retrieval across universals and cultural visual grounding.The former task entails retrieving culturally-diverse images for universal concepts from 50 countries, while the latter aims at grounding culture-specific concepts within images from 15 countries.Our evaluation across a wide range of models reveals that the performance varies significantly across cultures -underscoring the necessity for enhancing multicultural understanding in vision-language models.Our

With the increasing integration of AI into everyday life, it's becoming crucial to design AI systems that serve users from diverse backgrounds by making them culturally aware.In this paper, we present GD-COMET, a geo-diverse version of the COMET commonsense inference model.GD-COMET goes beyond Western commonsense knowledge and is capable of generating inferences pertaining to a broad range of cultures.We demonstrate the effectiveness of GD-COMET through a comprehensive human evaluation across 5 diverse cultures, as well as extrinsic evaluation on a geo-diverse task.The evaluation shows that GD-COMET captures and generates culturally nuanced commonsense knowledge, demonstrating its potential to benefit NLP applications across the board and contribute to making NLP more inclusive.PersonX eats a dutch baby PersonX wants PersonX feels Others feel sad, disgusting, happy, angry COMET PersonX is seen as hungry, greedy, disgusting, mean, starving As a result, a full stomach, a full belly, to satisfy hunger, to be full, to have a snack PersonX wants satiated, full, happy PersonX feels Others feel empty, craving, happy, keen PersonX is seen as hungry, ravenous, starving, greedy 🌎 GD- COMET satisfied, full, hungry, happy, guilty to throw up, drink water, wash the baby