Data augmentation methods, especially SoTA interpolation-based methods such as Fair Mixup, have been widely shown to increase model fairness. However, this fairness is evaluated on metrics that do not capture model uncertainty and on datasets with only one, relatively large, minority group. As a remedy, multicalibration has been introduced to measure fairness while accommodating uncertainty and accounting for multiple minority groups. However, existing methods of improving multicalibration involve reducing initial training data to create a holdout set for post-processing, which is not ideal when minority training data is already sparse. This paper uses multicalibration to more rigorously examine data augmentation for classification fairness. We stress-test four versions of Fair Mixup on two structured data classification problems with up to 81 marginalized groups, evaluating multicalibration violations and balanced accuracy. We find that on nearly every experiment, Fair Mixup worsens baseline performance and fairness, but the simple vanilla Mixup outperforms both Fair Mixup and the baseline, especially when calibrating on small groups. Combining vanilla Mixup with multicalibration post-processing, which enforces multicalibration through post-processing on a holdout set, further increases fairness.

Weight-based model editing methods update the parametric knowledge of language models post-training.However, these methods can unintentionally alter unrelated parametric knowledge representations, potentially increasing the risk of harm.In this work, we investigate how weight editing methods unexpectedly amplify model biases after edits.We introduce a novel benchmark dataset, SEESAW-CF, for measuring bias amplification of model editing methods for demographic traits such as race, geographic origin, and gender.We use SEESAW-CF to examine the impact of model editing on bias in five large language models.Our results demonstrate that edited models exhibit, to various degrees, more biased behavior for certain demographic groups than before they were edited, specifically becoming less confident in properties for Asian and African subjects.Additionally, editing facts about place of birth, country of citizenship, or gender has particularly negative effects on the model's knowledge about unrelated properties, such as field of work, a pattern observed across multiple models.