Public sector AI has the potential to harm citizens, with risks increasing as its use expands. Recent work positions public procurement as a way to shape public sector AI in line with public interests, using the state’s purchasing power to influence which AI systems are procured and under what conditions. This paper examines how this potential can be realised in practice by drawing on semi-structured interviews with UK and EU buyers, providers, and procurement experts. Our findings result in six promising procurement practices that enable the public sector to shape AI in line with public interests, alongside concrete mechanisms to support their uptake. Further, we find that AI-specific procurement approaches remain immature and systems often enter through informal channels with less scrutiny. We provide directions for both research and practice on how public procurement can be used as a governance mechanism for better aligning AI with public interests.

Calls for participatory AI development often assume that stakeholders can and should be empowered to substantially shape the system. However, competing demands of e.g. safety-critical contexts might limit this. Exploring this tension, we conducted a case study of Air Traffic Control (ATC) system development. Interviews with ATC operators (n=11) and a focus group including R&D employees (n=11) uncovered that controllers were dissatisfied: their input was confined to small changes; major decisions were made through opaque processes. Further, safety-related considerations often limited the extent to which their input could be incorporated. Importantly, controllers acknowledged that safety should take priority over e.g. their usability needs, instead calling for more transparency over decision-making processes and considered factors. Our findings highlight how general calls for empowerment might not align with safety-critical (and other) requirements. Therefore it is important to engage a wide range of stakeholders to explore conflicting demands before aligning/prioritising these in light of the application context. We outline ways forward for participatory practices with implications for the CHI/responsible AI communities.

Individual human decision-makers may benefit from different forms of support to improve decision outcomes, but when will each form of support yield better outcomes? In this work, we posit that personalizing access to decision support tools can be an effective mechanism for instantiating the appropriate use of AI assistance. Specifically, we propose the general problem of learning a decision support policy that, for a given input, chooses which form of support to provide to decision-makers for whom we initially have no prior information. We develop Modiste, an interactive tool to learn personalized decision support policies. Modiste leverages stochastic contextual bandit techniques to personalize a decision support policy for each decision-maker. In our computational experiments, we characterize the expertise profiles of decision-makers for whom personalized policies will outperform offline policies, including population-wide baselines. Our experiments include realistic forms of support (e.g., expert consensus and predictions from a large language model) on vision and language tasks. Our human subject experiments add nuance to and bolster our computational experiments, demonstrating the practical utility of personalized policies when real users benefit from accessing support across tasks.