Explainable Artificial Intelligence (XAI) can enhance trust in AI by offering cues that support human reasoning of AI behavior. Yet its effects on trust evolution remain unclear, especially when AI makes errors. This study examines how explanations of AI predictions influence human trust in AI-assisted decision-making under varying error severities. We tested two XAI visualizations, two AI error types, and three explanation strategies in simulated baggage screening tasks through an online study. Responses from 280 participants show that XAI representation significantly affects human compliance with AI during errors, while AI error type further shapes compliance after AI errors. AI Error type also impacts verification behaviors during AI errors, such as requesting explanations or ground truth. Moreover, strategies for conveying XAI influence perceived trust in AI, highlighting important implications for generalizing XAI effects beyond lab-based trust research.

Maps are crucial in conveying geospatial data in diverse contexts such as news and scientific reports. This research, utilizing thematic maps, probes deeper into the underexplored intersection of text framing and map types in influencing map interpretation. In this work, we conducted experiments to evaluate how textual detail and semantic content variations affect the quality of insights derived from map examination. We also explored the influence of explanatory annotations across different map types (e.g., choropleth, hexbin, isarithmic), base map details, and changing levels of spatial autocorrelation in the data. From two online experiments with N = 103 participants, we found that annotations, their specific attributes, and map type used to present the data significantly shape the quality of takeaways. Notably, we found that the effectiveness of annotations hinges on their contextual integration. These findings offer valuable guidance to the visualization community for crafting impactful thematic geospatial representations.

As machine learning models continue to swiftly advance, calibrating their performance has become a major concern prior to practical and widespread implementation.Most existing calibration methods often negatively impact model accuracy due to the lack of diversity of validation data, resulting in reduced generalizability.To address this, we propose a calibration method that incorporates synthetic data without compromising accuracy.We derive the expected calibration error (ECE) bound using the Probably Approximately Correct (PAC) learning framework.Large language models (LLMs), known for their ability to mimic real data and generate text with mixed class labels, are utilized as a synthetic data generation strategy to lower the ECE bound and improve model accuracy on real test data.Additionally, we propose data generation mechanisms for efficient calibration.Testing our method on four different natural language processing tasks, we observed an average up to 34% increase in accuracy and 33% decrease in ECE.

The Multisource AI Scorecard Table (MAST) is a checklist tool to inform the design and evaluation of trustworthy AI systems based on the U.S. Intelligence Community’s analytic tradecraft standards. In this study, we investigate whether MAST can be used to differentiate between high and low trustworthy AI-enabled decision support systems (AI-DSSs). Evaluating trust in AI-DSSs poses challenges to researchers and practitioners. These challenges include identifying the components, capabilities, and potential of these systems, many of which are based on the complex deep learning algorithms that drive DSS performance and preclude complete manual inspection. Using MAST, we developed two interactive AI-DSS testbeds. One emulated an identity-verification task in security screening, and another emulated a text-summarization system to aid in an investigative task. Each testbed had one version designed to reach low MAST ratings, and another designed to reach high MAST ratings. We hypothesized that MAST ratings would be positively related to the trust ratings of these systems. A total of 177 subject-matter experts were recruited to interact with and evaluate these systems. Results generally show higher MAST ratings for the high-MAST compared to the low-MAST groups, and that measures of trust perception are highly correlated with the MAST ratings. We conclude that MAST can be a useful tool for designing and evaluating systems that will engender trust perceptions, including for AI-DSS that may be used to support visual screening or text summarization tasks. However, higher MAST ratings may not translate to higher joint performance, and the connection between MAST and appropriate trust or trustworthiness remains an open question.

Machine learning models have gained traction as decision support tools for tasks that require processing copious amounts of data. However, to achieve the primary benefits of automating this part of decision-making, people must be able to trust the machine learning model's outputs. In order to enhance people's trust and promote appropriate reliance on the model, visualization techniques such as interactive model steering, performance analysis, model comparison, and uncertainty visualization have been proposed. In this study, we tested the effects of two uncertainty visualization techniques in a college admissions forecasting task, under two task difficulty levels, using Amazon's Mechanical Turk platform. Results show that (1) people's reliance on the model depends on the task difficulty and level of machine uncertainty and (2) ordinal forms of expressing model uncertainty are more likely to calibrate model usage behavior. These outcomes emphasize that reliance on decision support tools can depend on the cognitive accessibility of the visualization technique and perceptions of model performance and task difficulty.

Deceptive visualizations are visualizations that, whether intentionally or not, lead the reader to an understanding of the data which varies from the actual data. Examples of deceptive visualizations can be found in every digital platform, and, despite their widespread use in the wild, there have been limited efforts to alert laypersons to common deceptive visualization practices. In this paper, we present a tool for annotating line charts in the wild that reads line chart images and outputs text and visual annotations to assess the line charts for distortions and help guide the reader towards an honest understanding of the chart data. We demonstrate the usefulness of our tool through a series of case studies on real-world charts. Finally, we perform a crowdsourced experiment to evaluate the ability of the proposed tool to educate readers about potentially deceptive visualization practices.

With social media being a major force in information consumption, accelerated propagation of fake news has presented new challenges for platforms to distinguish between legitimate and fake news. Effective fake news detection is a non-trivial task due to the diverse nature of news domains and expensive annotation costs. In this work, we address the limitations of existing automated fake news detection models by incorporating auxiliary information (e.g., user comments and user-news interactions) into a novel reinforcement learning-based model called REinforced Adaptive Learning Fake News Detection (REAL-FND). REAL-FND exploits cross-domain and within-domain knowledge that makes it robust in a target domain, despite being trained in a different source domain. Extensive experiments on real-world datasets illustrate the effectiveness of the proposed model, especially when limited labeled data is available in the target domain.