Powered by the increasing predictive capabilities of machine learning algorithms, artificial intelligence (AI) systems have acquired the ability to improve outcomes by overruling human mistakes. However, to assess the full impact of adding AI oversight, we must understand whether its presence alters human decision-making. While a large amount of research has focused on how humans respond when being assisted by AI, very little is known about how humans respond when their decisions might be overruled by AI. Because being overruled can carry psychological costs (e.g., shame and embarrassment of being overruled) and psychological benefits (e.g., relief at having their mistakes fixed), individuals might alter their decision-making under AI oversight. We provide the first field evidence that AI oversight carries psychological costs that can impact human decision-making.