Guido Ivetta, Marcos J Gomez, Sofía Martinelli, Pietro Palombini, M Emilia Echeveste, Nair Carolina Mazzeo, Beatriz Busaniche, Luciana Benotti. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025.

Warning: This paper contains explicit statements of offensive stereotypes which may be upsetting The study of bias, fairness and social impact in Natural Language Processing (NLP) lacks resources in languages other than English. Our objective is to support the evaluation of bias in language models in a multilingual setting. We use stereotypes across nine types of biases to build a corpus containing contrasting sentence pairs, one sentence that presents a stereotype concerning an underadvantaged group and another minimally changed sentence, concerning a matching advantaged group. We build on the French CrowS-Pairs corpus and guidelines to provide translations of the existing material into seven additional languages. In total, we produce 11,139 new sentence pairs that cover stereotypes dealing with nine types of biases in seven cultural contexts. We use the final resource for the evaluation of relevant monolingual and multilingual masked language models. We find that language models in all languages favor sentences that express stereotypes in most bias categories. The process of creating a resource that covers a wide range of language types and cultural settings highlights the difficulty of bias evaluation, in particular comparability across languages and contexts.

Visual Question Answering~(VQA) is an important task in multimodal AI, which requires models to understand and reason on knowledge present in visual and textual data. However, most of the current VQA datasets and models are primarily focused on English and a few major world languages, with images that are Western-centric. While recent efforts have tried to increase the number of languages covered on VQA datasets, they still lack diversity in low-resource languages. More importantly, some datasets extend the text to other languages, either via translation or some other approaches, but usually keep the same images, resulting in narrow cultural representation. To address these limitations, we create CVQA, a new Culturally-diverse Multilingual Visual Question Answering benchmark dataset, designed to cover a rich set of languages and regions, where we engage native speakers and cultural experts in the data collection process. CVQA includes culturally-driven images and questions from across 28 countries in four continents, covering 26 languages with 11 scripts, providing a total of 9k questions. We benchmark several Multimodal Large Language Models (MLLMs) on CVQA, and we show that the dataset is challenging for the current state-of-the-art models. This benchmark will serve as a probing evaluation suite for assessing the cultural bias of multimodal models and hopefully encourage more research efforts towards increasing cultural awareness and linguistic diversity in this field.

In this paper we present the results of a manual classification of all ethical consideration sections for ACL 2021.We also compare how many papers had an ethics consideration section per track and per world region in ACL 2021.We classified papers according to the ethical issues covered (research benefits, potential harms, and vulnerable groups affected) and whether the paper was marked as requiring ethics review by at least one reviewer.Moreover, we discuss recurring obstacles we have observed (highlighting some interesting texts we found along the way) and conclude with three suggestions.We think that this paper may be useful for anyone who needs to write -or review -an ethics section and would like to get an overview of what others have done.

Collaborative grounding is a fundamental aspect of human-human dialog which allows people to negotiate meaning.In this paper we argue that it is missing from current deep learning approaches to dialog and interactive systems.Our central point is that making mistakes and being able to recover from them collaboratively is a key ingredient in grounding meaning.We illustrate the pitfalls of being unable to ground collaboratively, discuss what can be learned from the language acquisition and dialog systems literature, and reflect on how to move forward.

Visual Dialog is assumed to require the dialog history to generate correct responses during a dialog.However, it is not clear from previous work how dialog history is needed for visual dialog.In this paper we define what it means for visual questions to require dialog history and we propose a methodology for identifying them.We release a subset of the Guesswhat?! questions for which their dialog history completely changes their responses.We propose a novel interpretable representation that visually grounds dialog history: the Region under Discussion.It constrains the image's spatial features according to a semantic representation of the history inspired by the information structure notion of Question under Discussion.We evaluate the architecture on task-specific multimodal models and the visual transformer model LXMERT and show that there is still room for improvement.

In order to interpret the communicative intents of an utterance, it needs to be grounded in something that is outside of language; that is, grounded in world modalities.In this paper we argue that dialogue clarification mechanisms make explicit the process of interpreting the communicative intents of the speaker's utterances by grounding them in the various modalities in which the dialogue is situated.This paper frames dialogue clarification mechanisms as an understudied research problem and a key missing piece in the giant jigsaw puzzle of natural language understanding.We discuss both the theoretical background and practical challenges posed by this problem, and propose a recipe for obtaining grounding annotations.We conclude by highlighting ethical issues that need to be addressed in future work.

This paper describes an elementary school intervention to teach a text-based programming language to 10-11 year old students. We compare students with no previous programming experience with students with 3 semesters of experience with a block-based programming language. We analyze students' performance and learning based on detailed logs in an online programming platform and on multiple choice tests. Although both groups have a similar percentage of syntactical errors, the experienced group showed a better performance on exam scores and a lower number of test case errors. These findings suggest that, 10-11 year old students benefit from block-based experience when learning a new text-based programming language.

In this paper we do three things. First, we describe a web-based coding tool that is open-source, publicly available and provides formative feedback and assessment. Second, we compare several metrics on student performance in courses that use the tool versus courses that do not use it when learning to program in Haskell. We find that the dropout rates are significantly lower in those courses that use the tool at two different universities. Finally we apply the technology acceptance model to analyse students perceptions.

This paper describes an introductory Computer Science (CS) Professional Development (PD) course for K-12 teachers in Argentina that integrates pedagogical content knowledge and teacher classroom practice. We analyzed teachers' learning of what CS entails and the implementation of inquirybased programming lessons in their schools. Based on pre and post teachers surveys and classroom observations, we found that most teachers learned about the CS object of study and about fundamental programming concepts such as conditionals, loops, variables, etc. Teachers were more likely to replicate the same activities they experienced during PD workshops in their classrooms than to produce their own. Teachers who had a previous background on CS provided in-depth explanations of CS concepts to their students while other teachers superficially introduced the content knowledge. We describe PD activities and characteristics that could explain teachers' learning and incorporation of programming lessons. Findings imply that a PD program that integrates pedagogical content knowledge and teachers classroom practice can effectively improve inquiry-based CS teaching, but may be insufficient preparation for teachers with no previous background on CS.

This paper describes a school intervention to teach fundamental Computer Science (CS) concepts to 3-11 year old students with a multilanguage robot programming platform (using drag and drop, Python and C++ languages) in Argentina. We analyze students' performance and learning process based on multiple choice test and classroom observations. Data show that all students can intuitively learn sequence, conditional, loops and parameters and that girls performed slightly better than boys. Older students can easily combine these concepts to write a program. The multilanguage platform promotes student spontaneous exploration of more sophisticated CS concepts and languages. These findings imply that introducing CS in mandatory schooling from an inquiry based approach is both achievable and beneficial.

Chatbots have been used in different scenarios for getting people interested in CS for decades. However, their potential for teaching basic concepts and their engaging effect has not been measured. In this paper we present a software platform called Chatbot designed to foster engagement while teaching basic CS concepts such as variables, conditionals and finite state automata, among others. We carried out two experiences using Chatbot and the well known platform Alice: 1) an online nation-wide competition, and 2) an in-class 15-lesson pilot course in 2 high schools. Data shows that retention and girl interest are higher with Chatbot than with Alice, indicating student engagement.

We define the problem of automatic instruction interpretation as follows. Given a natural language instruction, can we automatically predict what an instruction follower, such as a robot, should do in the environment to follow that instruction? Previous approaches to automatic instruction interpretation have required either extensive domain-dependent rule writing or extensive manually annotated corpora. This article presents a novel approach that leverages a large amount of unannotated, easy-to-collect data from humans interacting in a game-like environment. Our approach uses an automatic annotation phase based on artificial intelligence planning, for which two different annotation strategies are compared: one based on behavioral information and the other based on visibility information. The resulting annotations are used as training data for different automatic classifiers. This algorithm is based on the intuition that the problem of interpreting a situated instruction can be cast as a classification problem of choosing among the actions that are possible in the situation. Classification is done by combining language, vision, and behavior information. Our empirical analysis shows that machine learning classifiers achieve 77% accuracy on this task on available English corpora and 74% on similar German corpora. Finally, the inclusion of human feedback in the interpretation process is shown to boost performance to 92% for the English corpus and 90% for the German corpus.