Explainability methods are used to benchmark the extent to which model predictions align with human rationales i.e., are 'right for the right reasons'.Previous work has failed to acknowledge, however, that what counts as a rationale is sometimes subjective.This paper presents what we think is a first of its kind, a collection of human rationale annotations augmented with the annotators demographic information.We cover three datasets spanning sentiment analysis and common-sense reasoning, and six demographic groups (balanced across age and ethnicity).Such data enables us to ask both what demographics our predictions align with and whose reasoning patterns our models' rationales align with.We find systematic inter-group annotator disagreement and show how 16 Transformer-based models align better with rationales provided by certain demographic groups: We find that models are biased towards aligning best with older and/or white annotators.We zoom in on the effects of model size and model distillation, finding -contrary to our expectations -negative correlations between model size and rationale agreement as well as no evidence that either model size or model distillation improves fairness.

Pretrained machine learning models are known to perpetuate and even amplify existing biases in data, which can result in unfair outcomes that ultimately impact user experience.Therefore, it is crucial to understand the mechanisms behind those prejudicial biases to ensure that model performance does not result in discriminatory behaviour toward certain groups or populations.In this work, we define gender bias as our case study.We quantify bias amplification in pretraining and after fine-tuning on three families of vision-and-language models.We investigate the connection, if any, between the two learning stages, and evaluate how bias amplification reflects on model performance.Overall, we find that bias amplification in pretraining and after fine-tuning are independent.We then examine the effect of continued pretraining on gender-neutral data, finding that this reduces group disparities, i.e., promotes fairness, on VQAv2 and retrieval tasks without significantly compromising task performance.

Contrastive explanations, where one decision is explained in contrast to another, are supposed to be closer to how humans explain a decision than non-contrastive explanations, where the decision is not necessarily referenced to an alternative.This claim has never been empirically validated.We analyze four English text-classification datasets (SST2, DynaSent, BIOS and DBpedia-Animals).We fine-tune and extract explanations from three different models (RoBERTa, GTP-2, and T5), each in three different sizes and apply three post-hoc explainability methods (LRP, GradientxInput, GradNorm).We furthermore collect and release human rationale annotations for a subset of 100 samples from the BIOS dataset for contrastive and non-contrastive settings.A crosscomparison between model-based rationales and human annotations, both in contrastive and non-contrastive settings, yields a high agreement between the two settings for models as well as for humans.Moreover, model-based explanations computed in both settings align equally well with human rationales.Thus, we empirically find that humans do not necessarily explain in a contrastive manner.

Daniel Hershcovich, Stella Frank, Heather Lent, Miryam de Lhoneux, Mostafa Abdou, Stephanie Brandl, Emanuele Bugliarello, Laura Cabello Piqueras, Ilias Chalkidis, Ruixiang Cui, Constanza Fierro, Katerina Margatina, Phillip Rust, Anders Søgaard. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2022.

Pretrained multilingual language models can help bridge the digital language divide, enabling high-quality NLP models for lower-resourced languages. Studies of multilingual models have so far focused on performance, consistency, and cross-lingual generalisation. However, with their wide-spread application in the wild and downstream societal impact, it is important to put multilingual models under the same scrutiny as monolingual models. This work investigates the group fairness of multilingual models, asking whether these models are equally fair across languages. To this end, we create a new four-way multilingual dataset of parallel cloze test examples (MozArt), equipped with demographic information (balanced with regard to gender and native tongue) about the test participants. We evaluate three multilingual models on MozArt –mBERT, XLM-R, and mT5– and show that across the four target languages, the three models exhibit different levels of group disparity, e.g., exhibiting near-equal risk for Spanish, but high levels of disparity for German.