We consider the effectiveness of multiobjective counterfactual explanations (MOCEs) in helping individuals learntactics, or rules of thumb, to apply when required to select a course of action in a specific context. In this setting, a counterfactual explanation compares one course of action against another. A MOCE presents this comparison by highlighting how the two options differ across a range of objectives or metrics. We conduct a study in which participants are presented with various scenarios alongside courses of action that could be implemented in those scenarios. Counterfactual explanations, including those involving multiple objectives, are used to identify the positive and negative aspects of the provided options. Participants were then required to identify the best course of action in various contexts. Participants trained with MOCE outperformed those given no explanations in seven of eight scenarios and those given single-objective counterfactual explanations (SOCEs) in four. SOCEs gave participants an aggregated outcome (expected rewards) without breaking these into specific objectives. MOCE improved tactic learning, but participants provided with SOCE or no explanation performed better in multitactic scenarios. These findings suggest that MOCE enhances tactical decision-making, but further research is needed for multitactic integration.

In this article, we introduce and evaluate a tool for researchers and practitioners to assess the actionability of information provided to users to support algorithmic recourse. While there are clear benefits of recourse from the user’s perspective, the notion of actionability in explainable AI research remains vague, and claims of ‘actionable’ explainability techniques are based on researchers’ intuitions. Inspired by definitions and instruments for assessing actionability in other domains, we construct a seven-item tool and investigate its effectiveness through two user studies. We show that the tool discriminates actionability across explanation types and that the distinctions align with human judgments. We illustrate the impact of context on actionability assessments, suggesting that domain-specific tool adaptations may foster more human-centred algorithmic systems. This is a valuable step forward for research and practices into actionable explainability and algorithmic recourse, providing the first clear human-centred tool for assessing actionability in explainable AI.

Prior research on AI-assisted human decision-making has explored several different explainable AI (XAI) approaches. A recent paper has proposed a paradigm shift calling for hypothesis-driven XAI through a conceptual framework called evaluative AI that gives people evidence that supports or refutes hypotheses without necessarily giving a decision-aid recommendation. In this paper, we describe and evaluate an approach for hypothesis-driven XAI based on the Weight of Evidence (WoE) framework, which generates both positive and negative evidence for a given hypothesis. Through human behavioural experiments, we show that our hypothesis-driven approach increases decision accuracy and reduces reliance compared to a recommendation-driven approach and an AI-explanation-only baseline, but with a small increase in under-reliance compared to the recommendation-driven approach. Further, we show that participants used our hypothesis-driven approach in a materially different way to the two baselines.

Displaying confidence scores in human-AI interaction has been shown to help build trust between humans and AI systems. However, most existing research uses only the confidence score as a form of communication. As confidence scores are just another model output, users may want to understand why the algorithm is confident to determine whether to accept the confidence score. In this paper, we show that counterfactual explanations of confidence scores help study participants to better understand and better trust a machine learning model's prediction. We present two methods for understanding model confidence using counterfactual explanation: (1) based on counterfactual examples; and (2) based on visualisation of the counterfactual space. Both increase understanding and trust for study participants over a baseline of no explanation, but qualitative results show that they are used quite differently, leading to recommendations of when to use each one and directions of designing better explanations.

In this article, we show that explanations of decisions made by machine learning systems can be improved by not only explaining why a decision was made but also explaining how an individual could obtain their desired outcome. We formally define the concept of directive explanations (those that offer specific actions an individual could take to achieve their desired outcome), introduce two forms of directive explanations (directive-specific and directive-generic), and describe how these can be generated computationally. We investigate people’s preference for and perception toward directive explanations through two online studies, one quantitative and the other qualitative, each covering two domains (the credit scoring domain and the employee satisfaction domain). We find a significant preference for both forms of directive explanations compared to non-directive counterfactual explanations. However, we also find that preferences are affected by many aspects, including individual preferences and social factors. We conclude that deciding what type of explanation to provide requires information about the recipients and other contextual information. This reinforces the need for a human-centered and context-specific approach to explainable AI.

In most multiagent applications, communication is essential among agents to coordinate their actions and achieve their goals. However, communication often has a related cost that affects overall system performance. In this paper, we draw inspiration from epistemic planning studies to develop a communication model for agents that allows them to cooperate and make communication decisions effectively within a planning task. The proposed model treats a communication process as an action that modifies the epistemic state of the team. We evaluate whether agents can cooperate effectively and achieve higher performance using communication protocol modeled in our epistemic planning framework in two simulated tasks. Based on an empirical study conducted using search and rescue tasks with different scenarios, our results show that the proposed model improved team performance across all scenarios than baseline models.

Prominent theories in cognitive science propose that humans understand and represent the knowledge of the world through causal relationships. In making sense of the world, we build causal models in our mind to encode cause-effect relations of events and use these to explain why new events happen by referring to counterfactuals — things that did not happen. In this paper, we use causal models to derive causal explanations of the behaviour of model-free reinforcement learning agents. We present an approach that learns a structural causal model during reinforcement learning and encodes causal relationships between variables of interest. This model is then used to generate explanations of behaviour based on counterfactual analysis of the causal model. We computationally evaluate the model in 6 domains and measure performance and task prediction accuracy. We report on a study with 120 participants who observe agents playing a real-time strategy game (Starcraft II) and then receive explanations of the agents' behaviour. We investigate: 1) participants' understanding gained by explanations through task prediction; 2) explanation satisfaction and 3) trust. Our results show that causal model explanations perform better on these measures compared to two other baseline explanation models.

In autonomous multiagent or multirobotic systems, the ability to quickly and accurately respond to threats and uncertainties is important for both mission outcomes and survivability. Such systems are never truly autonomous, often operating as part of a human-agent team. Artificial intelligent agents (IAs) have been proposed as tools to help manage such teams; e.g., proposing potential courses of action to human operators. However, they are often underutilized due to a lack of trust. Designing transparent agents, who can convey at least some information regarding their internal reasoning processes, is considered an effective method of increasing trust. How people interact with such transparency information to gain situation awareness while avoiding information overload is currently an unexplored topic. In this article, we go part way to answering this question, by investigating two forms of transparency: sequential transparency, which requires people to step through the IA's explanation in a fixed order; and demand-driven transparency, which allows people to request information as needed. In an experiment using a multivehicle simulation, our results show that demand-driven interaction improves the operators' trust in the system while maintaining, and at times improving, performance and usability.

According to Clark's seminal work on common ground and grounding, participants collaborating in a joint activity rely on their shared information, known as common ground, to perform that activity successfully, and continually align and augment this information during their collaboration. Similarly, teams of human and artificial agents require common ground to successfully participate in joint activities. Indeed, without appropriate information being shared, using agent autonomy to reduce the workload on humans may actually increase workload as the humans seek to understand why the agents are behaving as they are. While many researchers have identified the importance of common ground in artificial intelligence, there is no precise definition of common ground on which to build the foundational aspects of multi-agent collaboration. In this paper, building on previously-defined modal logics of belief, we present logic definitions for four different types of common ground. We define modal logics for three existing notions of common ground and introduce a new notion of common ground, called salient common ground. Salient common ground captures the common ground of a group participating in an activity and is based on the common ground that arises from that activity as well as on the common ground they shared prior to the activity. We show that the four definitions share some properties, and our analysis suggests possible refinements of the existing informal and semi-formal definitions.

Proper epistemic knowledge bases (PEKBs) are syntactic knowledge bases that use multi-agent epistemic logic to represent nested multi-agent knowledge and belief. PEKBs have certain syntactic restrictions that lead to desirable computational properties; primarily, a PEKB is a conjunction of modal literals, and therefore contains no disjunction. Sound entailment can be checked in polynomial time, and is complete for a large set of arbitrary formulae in logics Kn and KDn. In this paper, we extend PEKBs to deal with a restricted form of disjunction: 'knowing whether.' An agent i knows whether Q iff agent i knows Q or knows not Q; that is, []Q or []not(Q). In our experience, the ability to represent that an agent knows whether something holds is useful in many multi-agent domains. We represent knowing whether with a modal operator, and present sound polynomial-time entailment algorithms on PEKBs with the knowing whether operator in Kn and KDn, but which are complete for a smaller class of queries than standard PEKBs.

Many AI applications involve the interaction of multiple autonomous agents, requiring those agents to reason about their own beliefs, as well as those of other agents. However, planning involving nested beliefs is known to be computationally challenging. In this work, we address the task of synthesizing plans that necessitate reasoning about the beliefs of other agents. We plan from the perspective of a single agent with the potential for goals and actions that involve nested beliefs, non-homogeneous agents, co-present observations, and the ability for one agent to reason as if it were another. We formally characterize our notion of planning with nested belief, and subsequently demonstrate how to automatically convert such problems into problems that appeal to classical planning technology. Our approach represents an important first step towards applying the well-established field of automated planning to the challenging task of planning involving nested beliefs of multiple agents.

In task environments with large state and action spaces, the use of temporal and state abstraction can potentially improve the decision making performance of agents. However, existing approaches within a reinforcement learning framework typically ide

We present a novel approach for assisting pattern interpretation by data mining end-users: finding explanations for association rules based on probabilistic dependencies. In the approach, relevant variables are selected from rules and from other data sources to facilitate human-understandable interpretations. An explanation of a rule involves consideration of observable variables in the data and alignment with the conditional probability of the rule. To build explanations involving multiple, interacting variables, we use Bayesian networks to structure relationships. We illustrate the benefits of our technique for assisting pattern interpretation using Internet use survey data. The novel technique has potential in various data mining scenarios such as computer aided pattern interpretation and interactive data mining.

While argumentation-based negotiation has been accepted as a promising alternative to game-theoretic or heuristic based negotiation, no evidence has been provided to confirm this theoretical advantage. We propose a model of bilateral negotiation extending a simple monotonic concession protocol by allowing the agents to exchange information about their underlying interests and possible alternatives to achieve them during the negotiation. We present an empirical study that demonstrates (through simulation) the advantages of this interest-based negotiation approach over the more classic monotonic concession approach to negotiation.

A Bayesian Network (BN) consists of a qualitative part representing the structural assumptions of the domain and a quantitative part, the parameters. To date, knowledge engineering support has focused on parameter elicitation, with little support for designing the graphical structure. Poor design choices in BN construction can impact the network's performance, network maintenance, and the explanatory power of the output. We present a tool to help domain experts examine BN structure independently of the parameters. Our qualitative evaluation of the tool shows that it can help in identifying possible structural modeling errors and, hence, improve the quality of BN models. © 2006 Wiley Periodicals, Inc. Int J Int Syst 21: 1127–1150, 2006.