In this paper, we highlight the field of explainable sequential decision making. We discuss how the problem of explaining sequential decisions gives rise to problems and challenges that are absent from scenarios that focus on explaining single-shot decision making. We provide a short survey of some of the more prominent subareas within explainable sequential decision-making and their unique focuses and blind spots. Here, we argue that we need to go beyond simply focusing on individual subareas like explainable planning, reinforcement learning, or robotics, and move towards studying and tackling the more general problem of explainable sequential decision-making. Such a holistic approach will not only allow us to identify previously ignored problems, but also provide us with the ability to transfer ideas and intuitions from one subarea of explainable sequential decision-making to another. We end the paper with a discussion on future directions and some of the most pressing open questions.

Counterfactual explanations have emerged as a popular solution for the eXplainable AI (XAI) problem of elucidating the predictions of black-box deep-learning systems because people easily understand them, they apply across different problem domains and seem to be legally compliant. Although over 100 counterfactual methods exist in the XAI literature, each claiming to generate plausible explanations akin to those preferred by people, few of these methods have actually been tested on users (∼7%). Even fewer studies adopt a user-centered perspective; for instance, asking people for their counterfactual explanations to determine their perspective on a “good explanation”. This gap in the literature is addressed here using a novel methodology that (i) gathers human-generated counterfactual explanations for misclassified images, in two user studies and, then, (ii) compares these human-generated explanations to computationally-generated explanations for the same misclassifications. Results indicate that humans do not “minimally edit” images when generating counterfactual explanations. Instead, they make larger, “meaningful” edits that better approximate prototypes in the counterfactual class. An analysis based on “explanation goals” is proposed to account for this divergence between human and machine explanations. The implications of these proposals for future work are discussed.

Recently, counterfactuals using “if-only” explanations have become very popular in eXplainable AI (XAI), as they describe which changes to feature-inputs of a black-box AI system result in changes to a (usually negative) decision-outcome. Even more recently, semi-factuals using “even-if” explanations have gained more attention. They elucidate the feature-input changes that do not change the decision-outcome of the AI system, with a potential to suggest more beneficial recourses. Some semi-factual methods use counterfactuals to the query-instance to guide semi-factual production (so-called counterfactual-guided methods ), whereas others do not (so-called counterfactual-free methods ). In this work, we perform comprehensive tests of 8 semi-factual methods on 7 datasets using 5 key metrics, to determine whether counterfactual guidance is necessary to find the best semi-factuals. The results of these tests suggests not, but rather that computing other aspects of the decision space lead to better semi-factual XAI.

Counterfactual explanations have become a major focus for post-hoc explainability research in recent years, as they seem to provide good algorithmic recourse solutions, people can readily understand them, and they may meet legal regulations (such as GDPR in the EU). However, this large literature has only addressed the use of counterfactual explanations to explain single predictive-instances. Here, we explore a novel use case in which groups of similar instances are explained in a collective fashion using “group counterfactuals” (e.g., to highlight a repeating pattern of illness in a group of patients). Group counterfactuals potentially provide broad explanations covering multiple events/instances. A novel case-based, group-counterfactual algorithm is proposed to generate such explanations and a user study is also reported to test the psychological validity of the algorithm.

Very recently, semi-factual explanations have emerged in Explainable AI (XAI) as a new and potentially important explanation strategy. Semi-factuals employ “Even if...” reasoning, as opposed to the “If only...” reasoning of counterfactuals. Counterfactuals inform users about what feature-differences lead to changes in an outcome (e.g., “ if only you asked for a lower loan, you would have been successful.”), whereas semi-factuals inform them about what feature-differences lead to the outcome remaining the same (e.g., “ Even if you asked for a lower loan, you would still have been unsuccessful”). Semi-factuals have the potential to be as important as their popular counterfactual siblings. However, the AI/ML and XAI communities have by and large struggled to imagine useful application-scenarios for semi-factuals. In this paper, we summarize recent work on semi-factual explanation and trace a roadmap for application-focused research in the area. We begin by outlining the main constraints identified for semi-factual optimization proposed in the literature, before summarizing the applications of semi-factuals proposed to-date. Then, we sketch several directions for future applications and research using semi-factuals. Finally, though semi-factuals are highly promising (especially with regard to algorithmic recourse), they have a potential for ethical misuse that we discuss in our conclusions.

Recently, eXplainable AI (XAI) research has focused on the use of counterfactual explanations to address interpretability, algorithmic recourse, and bias in AI system decision-making. The developers of these algorithms claim they meet user requirements in generating counterfactual explanations with “plausible,” “actionable” or “causally important” features. However, few of these claims have been tested in controlled psychological studies. Hence, we know very little about which aspects of counterfactual explanations really help users understand the decisions of AI systems. Nor do we know whether counterfactual explanations are an advance on more traditional causal explanations that have a longer history in AI (e.g., in expert systems). Accordingly, we carried out three user studies to (1) test a fundamental distinction in feature-types, between categorical and continuous features, and (2) compare the relative effectiveness of counterfactual and causal explanations. The studies used a simulated, automated decision-making app that determined safe driving limits after drinking alcohol, based on predicted blood alcohol content, where users’ responses were measured objectively (using predictive accuracy) and subjectively (using satisfaction and trust judgments). Study 1 ( N \({=}\) 127) showed that users understand explanations referring to categorical features more readily than those referring to continuous features. It also discovered a dissociation between objective and subjective measures: counterfactual explanations elicited higher accuracy than no-explanation controls but elicited no more accuracy than causal explanations, yet counterfactual explanations elicited greater satisfaction and trust than causal explanations. In Study 2 ( N \({=}\) 136) we transformed the continuous features of presented items to be categorical (i.e., binary) and found that these converted features led to highly accurate responding. Study 3 ( N \({=}\) 211) explicitly compared matched items involving either mixed features (i.e., a mix of categorical and continuous features) or categorical features (i.e., categorical and categorically-transformed continuous features), and found that users were more accurate when categorically-transformed features were used instead of continuous ones. It also replicated the dissociation between objective and subjective effects of explanations. The findings delineate important boundary conditions for current and future counterfactual explanation methods in XAI.

Recently, eXplainable AI (XAI) research has focused on counterfactual explanations as post-hoc justifications for AI-system decisions (e.g., a customer refused a loan might be told “if you asked for a loan with a shorter term, it would have been approved”). Counterfactuals explain what changes to the input-features of an AI system change the output-decision. However, there is a sub-type of counterfactual, semi-factuals, that have received less attention in AI (though the Cognitive Sciences have studied them more). This paper surveys semi-factual explanation, summarising historical and recent work. It defines key desiderata for semi-factual XAI, reporting benchmark tests of historical algorithms (as well as a novel, naïve method) to provide a solid basis for future developments.

Explainable AI (XAI) has been proposed as a valuable tool to assist in downstream tasks involving human-AI collaboration. Perhaps the most psychologically valid XAI techniques are case-based approaches which display "whole" exemplars to explain the predictions of black-box AI systems. However, for such post-hoc XAI methods dealing with images, there has been no attempt to improve their scope by using multiple clear feature "parts" of the images to explain the predictions while linking back to relevant cases in the training data, thus allowing for more comprehensive explanations that are faithful to the underlying model. Here, we address this gap by proposing two general algorithms (latent and superpixel-based) which can isolate multiple clear feature parts in a test image, and then connect them to the explanatory cases found in the training data, before testing their effectiveness in a carefully designed user study. Results demonstrate that the proposed approach appropriately calibrates a user's feelings of "correctness" for ambiguous classifications in real world data on the ImageNet dataset, an effect which does not happen when just showing the explanation without feature highlighting.

Recently, eXplainable AI (XAI) research has focused on the use of counterfactual explanations to address interpretability, algorithmic recourse, and bias in AI system decision-making. The proponents of these algorithms claim they meet users’ requirements for counterfactual explanations. For instance, many claim that the output of their algorithms work as explanations because they prioritise "plausible", "actionable" or "causally important" features in their generated counterfactuals. However, very few of these claims have been tested in controlled psychological studies, and we know very little about which aspects of counterfactual explanations help users to understand AI system decisions. Furthermore, we do not know whether counterfactual explanations are an advance on more traditional causal explanations that have a much longer history in AI (in explaining expert systems and decision trees). Accordingly, we carried out two user studies to (i) test a fundamental distinction in feature-types, between categorical and continuous features, and (ii) compare the relative effectiveness of counterfactual and causal explanations. The studies used a simulated, automated decision-making app that determined safe driving limits after drinking alcohol, based on predicted blood alcohol content, and user responses were measured objectively (users’ predictive accuracy) and subjectively (users’ satisfaction and trust judgments). Study 1 (N=127) showed that users understand explanations referring to categorical features more readily than those referring to continuous features. It also discovered a dissociation between objective and subjective measures: counterfactual explanations elicited higher accuracy of predictions than no-explanation control descriptions but no higher accuracy than causal explanations, yet counterfactual explanations elicited greater satisfaction and trust judgments than causal explanations. Study 2 (N=211) found that users were more accurate for categorically-transformed features compared to continuous ones, and also replicated the results of Study 1. The findings delineate important boundary conditions for current and future counterfactual explanation methods in XAI.

Counterfactual explanations have emerged as a popular solution for the eXplainable AI (XAI) problem of elucidating the predictions of black-box deep-learning systems because people easily understand them, they apply across different problem domains and seem to be legally compliant. Although over 100 counterfactual methods exist in the XAI literature, each claiming to generate plausible explanations akin to those preferred by people, few of these methods have actually been tested on users (∼7%). Even fewer studies adopt a user-centered perspective; for instance, asking people for their counterfactual explanations to determine their perspective on a “good explanation”. This gap in the literature is addressed here using a novel methodology that (i) gathers human-generated counterfactual explanations for misclassified images, in two user studies and, then, (ii) compares these human-generated explanations to computationally-generated explanations for the same misclassifications. Results indicate that humans do not “minimally edit” images when generating counterfactual explanations. Instead, they make larger, “meaningful” edits that better approximate prototypes in the counterfactual class. An analysis based on “explanation goals” is proposed to account for this divergence between human and machine explanations. The implications of these proposals for future work are discussed.

We compared two sorts of explanations for decisions made by an AI system: counterfactual explanations about how an outcome could have been different in the past, and prefactual explanations about how it could be different in the future. We examined the effects of these alternative explanation strategies on the accuracy of users' judgments about the AI app's predictions about an outcome (inferred from information about the causes), compared to the accuracy of their judgments about the app's diagnoses of a cause (inferred from information about the outcome). The tasks were based on a simulated SmartAgriculture decision support system for grass growth outcomes on dairy farms in Experiment 1, and for an analogous alien planet domain in Experiment 2. The two experiments, with 243 participants, also tested users' confidence in their decisions, and their satisfaction with the explanations. Users made more accurate diagnoses of the presence of causes based on information about their outcome, compared to predictions of an outcome given information about the presence of causes. Their predictions and diagnoses were helped equally by counterfactual explanations and prefactual ones.

There is a growing concern that the recent progress made in AI, especially regarding the predictive competence of deep learning models, will be undermined by a failure to properly explain their operation and outputs. In response to this disquiet, counterfactual explanations have become very popular in eXplainable AI (XAI) due to their asserted computational, psychological, and legal benefits. In contrast however, semi-factuals (which appear to be equally useful) have surprisingly received no attention. Most counterfactual methods address tabular rather than image data, partly because the non-discrete nature of images makes good counterfactuals difficult to define; indeed, generating plausible counterfactual images which lie on the data manifold is also problematic. This paper advances a novel method for generating plausible counterfactuals and semi-factuals for black-box CNN classifiers doing computer vision. The present method, called PlausIble Exceptionality-based Contrastive Explanations (PIECE), modifies all “exceptional” features in a test image to be “normal” from the perspective of the counterfactual class, to generate plausible counterfactual images. Two controlled experiments compare this method to others in the literature, showing that PIECE generates highly plausible counterfactuals (and the best semi-factuals) on several benchmark measures.

In recent years, there has been an explosion of AI research on counterfactual explanations as a solution to the problem of eXplainable AI (XAI). These explanations seem to offer technical, psychological and legal benefits over other explanation techniques. We survey 100 distinct counterfactual explanation methods reported in the literature. This survey addresses the extent to which these methods have been adequately evaluated, both psychologically and computationally, and quantifies the shortfalls occurring. For instance, only 21% of these methods have been user tested. Five key deficits in the evaluation of these methods are detailed and a roadmap, with standardised benchmark evaluations, is proposed to resolve the issues arising; issues, that currently effectively block scientific progress in this field.

In this paper, we describe a post-hoc explanation-by-example approach to eXplainable AI (XAI), where a black-box, deep learning system is explained by reference to a more transparent, proxy model (in this situation a case-based reasoner), based on a feature-weighting analysis of the former that is used to find explanatory cases from the latter (as one instance of the so-called Twin Systems approach). A novel method (COLE-HP) for extracting the feature-weights from black-box models is demonstrated for a convolutional neural network (CNN) applied to the MNIST dataset; in which extracted feature-weights are used to find explanatory, nearest-neighbours for test instances. Three user studies are reported examining people's judgements of right and wrong classifications made by this XAI twin-system, in the presence/absence of explanations-by-example and different error-rates (from 3-60%). The judgements gathered include item-level evaluations of both correctness and reasonableness, and system-level evaluations of trust, satisfaction, correctness, and reasonableness. Several proposals are made about the user's mental model in these tasks and how it is impacted by explanations at an item- and system-level. The wider lessons from this work for XAI and its user studies are reviewed.

In this paper, the twin-systems approach is reviewed, implemented, and competitively tested as a post-hoc explanation-by-example solution to the eXplainable Artificial Intelligence (XAI) problem. In twin-systems, an opaque artificial neural network (ANN) is explained by “twinning” it with a more interpretable case-based reasoning (CBR) system, by mapping the feature weights from the former to the latter. Extensive comparative tests are performed, over four experiments, to determine the optimal feature-weighting method for such twin-systems. Twin-systems for traditional multilayer perceptron (MLP) networks (MLP–CBR twins), convolutional neural networks (CNNs; CNN–CBR twins), and transformers for NLP (BERT–CBR twins) are examined. In addition, Feature Activation Maps (FAMs) are explored to enhance explainability by providing an additional layer of explanatory insight. The wider implications of this research on XAI is discussed, and a code library is provided to ease replicability.

Smart agriculture (SmartAg) has emerged as a rich domain for AI-driven decision support systems (DSS); however, it is often challenged by user-adoption issues. This paper reports a case-based reasoning (CBR) system, PBI-CBR, that predicts grass growth for dairy farmers, that combines predictive accuracy and explanations to improve user adoption. PBI-CBR’s key novelty is its use of Bayesian methods for case-base maintenance in a regression domain. Experiments report the tradeoff between predictive accuracy and explanatory capability for different variants of PBI-CBR, and how updating Bayesian priors each year improves performance.

In this paper, twin-systems are described to address the eXplainable artificial intelligence (XAI) problem, where a black box model is mapped to a white box “twin” that is more interpretable, with both systems using the same dataset. The framework is instantiated by twinning an artificial neural network (ANN; black box) with a case-based reasoning system (CBR; white box), and mapping the feature weights from the former to the latter to find cases that explain the ANN’s outputs. Using a novel evaluation method, the effectiveness of this twin-system approach is demonstrated by showing that nearest neighbor cases can be found to match the ANN predictions for benchmark datasets. Several feature-weighting methods are competitively tested in two experiments, including our novel, contributions-based method (called COLE) that is found to perform best. The tests consider the ”twinning” of traditional multilayer perceptron (MLP) networks and convolutional neural networks (CNN) with CBR systems. For the CNNs trained on image data, qualitative evidence shows that cases provide plausible explanations for the CNN’s classifications.

While much is known about how people tweet and interact on Twitter, surprisingly little is known about how the news items tweeted by journalists -- news tweets -- act as a distribution channel for the news that is spread by social media reading and sharing. This paper aims to fill this gap by analyzing the dynamics of news on Twitter, by revealing what drives users to consume news, and by developing a news consumption prediction model. We present the Twitter News Model (TNM), a computational data-driven approach to elucidate the dynamics of news consumption on Twitter. We apply the TNM to a dataset of interactions between users and journalists/newspapers to reveal what drives users' consumption of news on Twitter, and predictively relate users' news beliefs, motivations, and attitudes to their consumption of news. Our findings reveal that news motivations, followed by news attitudes and news beliefs, impact users' behavior of news consumption on Twitter.

LOOM is advanced as a new visualisation for changes in ranks and trends in power-law data that is changing dynamically over time. A comparison between LOOM and existing methods for visualising such data (e.g., time-series graphs, typical analytics dashboards). Several exemplar data sets are shown, using LOOM, drawn from the tracking of news stories on Twitter. The basis for the LOOM visualisation is elaborated and it is shown how it avoids the pitfalls arising in other line-graph representations.

Conversation partners have a tendency to adapt their vocal in- tensity to each other and to other social and environmental fac- tors. A socially adequate vocal intensity level by a speech syn- thesiser that goes beyond mere volume adjustment is highly de- sirable for a rewarding and successful human-machine or ma- chine mediated human-human interaction. This paper examines the interaction of the Lombard effect and speaker entrainment in a controlled experiment conducted with a confederate inter- locutor. The interlocutor was asked to maintain either a soft, a modal or a loud voice level during the dialogues. Through half of the trials, subjects were exposed to a cocktail party noise through headphones. The analytical results suggest that both the background noise and the interlocutor’s voice level affect the dynamics of speaker entrainment. Speakers appear to still en- train to the voice level of their interlocutor in noisy conditions, though to a lesser extent, as strategies of ensuring intelligibility affect voice levels as well. These findings could be leveraged in spoken dialogue systems and speech generating devices to help choose a vocal effort level for the synthetic voice that is both intelligible and socially suited to a specific interaction.

A model of user-search-engine interaction is developed using the ACT-R cognitive architecture. We test, using an empirical evaluation, the model across different result orderings and relevance distributions, demonstrating that across a number of trials, the model approximates the characteristics of large numbers of users interacting with search-engines. These results are discussed in terms of their practical implications for search interfaces and ranking algorithms.

It is now feasible to view media at home as easily as text-based pages were viewed when the World Wide Web (WWW) first emerged. This development has supported media sharing and search services providing hosting, indexing and access to large, online media repositories. Many of these sharing services also have a social aspect to them. This paper provides an initial analysis of the social interactions on a video sharing and search service (www.youtube.com). Results show that many users do not form social networks in the online community and a very small number do not appear to contribute to the wider community. However, it does seem those people who do use the available tools have much a greater tendency to form social connections.

With the growth of social bookmarking a new approach for metadata creation called tagging has emerged. In this paper we evaluate the use of tag presentation techniques. The main goal of our evaluation is to investigate the effect of some of the different properties that can be utilized in presenting tags e.g. alphabetization, using larger fonts etc. We show that a number of these factors can affect the ease with which users can find tags and use the tools for presenting tags to users.

In this paper we investigate environmental factors that can result in users having different preferences and behaviors at different times of the day. An analysis is carried out of a large sample of user data for Wireless Application Protocol (WAP) browsing to determine whether user surfing patterns vary depending on time. We examine traffic on an hourly and daily basis, and show that accesses to particular categories of pages vary relative to time. We also build Markov models, which are temporal; to predict user navigation, and illustrate those predictive models are more accurate and beneficial to mobile Internet users than traditional methods. This analysis provides insight into improving the effectiveness and efficiency of navigation prediction.

Many systems use past behavior, preferences and environmental factors to attempt to predict user navigation on the Internet. However we believe that many of these models have shortcomings, in that they do not take into account that users may have many different sets of preferences. Here we investigate an environmental factor, namely time, in making predictions about user navigation. We present methods for creating temporal rules that describe user navigation patterns. We also show the benefit of using these rules to predict user navigation and also show the benefits of these models over traditional methods. An analysis is carried out on a sample of usage logs for Wireless Application Protocol (WAP) browsing, and the results of this analysis verify our hypothesis.

Search engine results are usually presented in some form of text summary (e.g., document title, some snippets of the page's content, a URL, etc). Based on the information contained within these summaries users make relevance judgments about what links best suit their information needs. Current research suggests that these relevance judgments are in the service of some search strategy. In this paper, we model two different search strategies (the comparison and threshold strategies) and determine how well they fit data gathered from an experiment on user search within a simulated Google environment.

There are many systems that attempt to predict user navigation on the Internet through the use of past behavior, preferences and environmental factors. We believe that many of these models have shortcomings, in that they do not take into account that users may have many different sets of preferences, specifically, we investigate time as an environmental factor in making predictions about user navigation. We present a method for segmenting log files in order to learn time dependent models to predict user navigation patterns and show the benefits of these models over traditional methods. An analysis is carried out on a sample of usage logs for wireless application protocol (WAP) browsing, and the results of this analysis verify our hypothesis.

Case based reasoning (CBR) is an artificial intelligence technique that emphasises the role of past experience during future problem solving. New problems are solved by retrieving and adapting the solutions to similar problems, solutions that have been stored and indexed for future reuse as cases in a case-base. The power of CBR is severely curtailed if problem solving is limited to the retrieval and adaptation of a single case, so most CBR systems dealing with complex problem solving tasks have to use multiple cases. The paper describes and evaluates the technique of hierarchical case based reasoning, which allows complex problems to be solved by reusing multiple cases at various levels of abstraction. The technique is described in the context of Deja Vu, a CBR system aimed at automating plant-control software design.

Abstract Once viewed as a rhetorical and superficial language phenomenon, metaphor is now recognized to serve a fundamental role in our conceptual structuring and language comprehension processes. In particular, it is argued that certain experiential metaphors based upon intuitions of spatial relations are inherent in the conceptual organization of our most abstract thoughts. In this paper we present a two‐stage computational model of metaphor interpretation which employs a spatially founded semantics to broadly characterize the meaning carried by a metaphor in terms of a conceptual scaffolding, an interim meaning structure around which a fuller interpretation is fleshed out over time. We then present a semantics for the construction of conceptual scaffolding which is based upon core metaphors of collocation, containment and orientation. The goal of this scaffolding is to maintain the intended association of ideas even in contexts in which system knowledge is insufficient for a complete interpretation. This two‐stage system of scaffolding and elaboration also models the common time lapse between initial metaphor comprehension and full metaphor appreciation. Several mechanisms for deriving elaborative inference from scaffolding structures, particularly in cases of novel or creative metaphor, are also presented. While the system developed in this paper has significant practical application, it also demonstrates that core spatial metaphors clearly play a central role in metaphor comprehension.