I offer a preliminary conceptual framework for evaluating AI alignment projects. It is based on the concept of action guidance. In §1 and §2, I explain action guidance and its importance to AI alignment. I introduce the ‘Guidance Framework’ in §3. In §4, I show how it can be applied to two different sorts of questions: the practical question of how to design a specific AI agent (my example is a fictional ocean-cleaning robot), and the theoretical question of how to evaluate a specific AI alignment proposal (my example is Stuart Russell's ‘binary approach’). In §5 I discuss limitations of the framework and opportunities for further research.

Stochastic policies often outperform deterministic ones. This is especially true for Constrained Stochastic Shortest Path (C-SSP) problems, a popular approach to planning under uncertainty with multiple objectives. Nevertheless, there are moral concerns about stochastic policies that should deter us from selecting them. In this paper, we identify some of these moral concerns and offer 'acceptability constraints' that allow only certain stochastic policies to be selected. We propose a novel C-SSP solver able to integrate our moral acceptability constraints, we evaluate its performance in a relevant test problem, and we show that our approach can successfully produce acceptable policies in morally significant domains.

In this paper we provide a proof of principle of a new method for addressing the ethics of autonomous vehicles (AVs), theData-Theories Method, in which vehicle crash data is combined with philosophical ethical theory to provide a guide to action for AV algorithm design. We use this method to model three scenarios in which an AV is exposed to risk on the road, and determine possible actions for the AV. We then examine how different philosophical perspectives on agent partiality, or the degree to which one can act in one’s own self-interest, might address each scenario. This method shows why modelling the ethics of AVs using data is essential. First, AVs may sometimes have options that human drivers do not, and designing AVs to mimic the most ethical human driver would not ensure that they do the right thing. Second, while ethical theories can often disagree about what should be done, disagreement can be reduced and compromises found with a more complete understanding of the AV’s choices and their consequences. Finally, framing problems around thought experiments may elicit preferences that are divergent with what individuals might prefer once they are provided with information about the real risks for a scenario. Our method provides a principled and empirical approach to productively address these problems and offers guidance on AV algorithm design.

Artificially intelligent systems will be used to make increasingly important decisions about us. Many of these decisions will have to be made without consensus about the relevant moral facts. I argue that what makes moral disagreement especially challenging is that there are two different ways of handling it: political solutions, which aim to find a fair compromise, and epistemic solutions, which aim at moral truth.