Explainable Artificial Intelligence (XAI) can enhance trust in AI by offering cues that support human reasoning of AI behavior. Yet its effects on trust evolution remain unclear, especially when AI makes errors. This study examines how explanations of AI predictions influence human trust in AI-assisted decision-making under varying error severities. We tested two XAI visualizations, two AI error types, and three explanation strategies in simulated baggage screening tasks through an online study. Responses from 280 participants show that XAI representation significantly affects human compliance with AI during errors, while AI error type further shapes compliance after AI errors. AI Error type also impacts verification behaviors during AI errors, such as requesting explanations or ground truth. Moreover, strategies for conveying XAI influence perceived trust in AI, highlighting important implications for generalizing XAI effects beyond lab-based trust research.

The Multisource AI Scorecard Table (MAST) is a checklist tool to inform the design and evaluation of trustworthy AI systems based on the U.S. Intelligence Community’s analytic tradecraft standards. In this study, we investigate whether MAST can be used to differentiate between high and low trustworthy AI-enabled decision support systems (AI-DSSs). Evaluating trust in AI-DSSs poses challenges to researchers and practitioners. These challenges include identifying the components, capabilities, and potential of these systems, many of which are based on the complex deep learning algorithms that drive DSS performance and preclude complete manual inspection. Using MAST, we developed two interactive AI-DSS testbeds. One emulated an identity-verification task in security screening, and another emulated a text-summarization system to aid in an investigative task. Each testbed had one version designed to reach low MAST ratings, and another designed to reach high MAST ratings. We hypothesized that MAST ratings would be positively related to the trust ratings of these systems. A total of 177 subject-matter experts were recruited to interact with and evaluate these systems. Results generally show higher MAST ratings for the high-MAST compared to the low-MAST groups, and that measures of trust perception are highly correlated with the MAST ratings. We conclude that MAST can be a useful tool for designing and evaluating systems that will engender trust perceptions, including for AI-DSS that may be used to support visual screening or text summarization tasks. However, higher MAST ratings may not translate to higher joint performance, and the connection between MAST and appropriate trust or trustworthiness remains an open question.

Machine learning models have gained traction as decision support tools for tasks that require processing copious amounts of data. However, to achieve the primary benefits of automating this part of decision-making, people must be able to trust the machine learning model's outputs. In order to enhance people's trust and promote appropriate reliance on the model, visualization techniques such as interactive model steering, performance analysis, model comparison, and uncertainty visualization have been proposed. In this study, we tested the effects of two uncertainty visualization techniques in a college admissions forecasting task, under two task difficulty levels, using Amazon's Mechanical Turk platform. Results show that (1) people's reliance on the model depends on the task difficulty and level of machine uncertainty and (2) ordinal forms of expressing model uncertainty are more likely to calibrate model usage behavior. These outcomes emphasize that reliance on decision support tools can depend on the cognitive accessibility of the visualization technique and perceptions of model performance and task difficulty.

Teams composed of human and machine members operating in complex task environments must effectively interact in response to information flow while adapting to environmental changes. This study investigates how interpersonal coordination dynamics between team members are associated with team performance and shared situation awareness in a simulated urban search and rescue (USAR) task. More specifically, this study investigates (1) how communication recurrence affected and reflected coordination dynamics between a USAR robot and human operator when they used different communication strategies, and (2) how these dynamic characteristics of the human–robot interpersonal coordination were associated with the team performance and shared situation awareness. The USAR interpersonal coordination dynamics were systematically characterized using discrete recurrence quantification analysis. Results from this study indicate that (1) teams demonstrating more flexibility in their coordination dynamics were more adaptive to changes in the task environment, and (2) while robot explanations help to improve shared situation awareness, revisiting the same communication pattern (i.e., routine coordination) was associated with better team performance, but did not improve shared situation awareness.

Accountability pressures have been found to increase worker engagement and reduce adverse biases in people interacting with automated technology, but it is unclear if these effects can be observed in a more laterally controlled human-AI task. To address this question, 40 participants were asked to coordinate with an AI agent on a resource-management task, with half of the participants expecting to justify their decision strategy, which comprised our accountability condition. We then considered the effects of accountability on performance, as measured by participants’ resource sharing behaviors, their individual, and joint task scores (throughput), and their perceived workload. Participants in the accountability group shared more resources with their AI partner, took more time to make decisions, and performed worse in the task individually, but had AI partners who performed better. We found no difference between groups on how prepared they felt they were to justify their decisions, and participants reported similar levels of workload. Results suggest accountability pressures can influence exchange strategies in human-AI tasks with lateral control.

Reciprocal cooperation is prevalent in human society. Understanding human reciprocal cooperation in human-agent interaction can help design human-agent systems that promote cooperation and joint performance. Studies have found that people reciprocate cooperative behavior when interacting with computer agents in social dilemma games. However, few studies have investigated human reciprocal cooperation with agents in complex dynamic environments. This article examines the behavioral and neurological patterns of reciprocal cooperation in a hospital management microworld experiment. The participants (n = 30) work with both high- and low-cooperation computer agents to share resources to cope with dynamic demands. Participants' resource sharing behaviors were recorded and their prefrontal cortex (PFC) activation was measured using functional near-infrared spectroscopy (fNIRS) technology. Similar to previous studies conducted with participants in the United States, results demonstrate that participants in China showed reciprocal cooperation behaviors with the agents. Specifically, participants share more resources and achieve higher performance when working with a high-cooperation agent than with a low-cooperation agent. A high activation level is detected in the right dorsolateral PFC when working with a high-cooperation agent. Other PFC activation patterns imply that cooperation could be unnecessarily mentally taxing in certain situations. These findings suggest that human cooperativeness in human-agent systems can be calibrated by an agent's cooperation behavior. System designers should design for appropriate cooperativeness and avoid the inefficient use of system resources. Neurological measures could be a useful tool to investigate the mental process in human-agent cooperation.

The aim of this study is to examine the utility of physiological compliance (PC) to understand shared experience in a multiuser technological environment involving active and passive users. Common ground is critical for effective collaboration and important for multiuser technological systems that include passive users since this kind of user typically does not have control over the technology being used. An experiment was conducted with 48 participants who worked in two-person groups in a multitask environment under varied task and technology conditions. Indicators of PC were measured from participants' cardiovascular and electrodermal activities. The relationship between these PC indicators and collaboration outcomes, such as performance and subjective perception of the system, was explored. Results indicate that PC is related to group performance after controlling for task/technology conditions. PC is also correlated with shared perceptions of trust in technology among group members. PC is a useful tool for monitoring group processes and, thus, can be valuable for the design of collaborative systems. This study has implications for understanding effective collaboration.