In AI decision support, explainability (e.g., disclosing system weaknesses) should help users spot erroneous recommendations, but its efficacy may depend on task difficulty. We ran three experiments in a simulated medical visual detection task. In Experiment 1, we manipulated error difficulty (easy vs. difficult) and explainability (non-XAI vs. XAI). Experiment 2 added a virtually impossible error difficulty. Across both, explainability consistently reduced reliance on incorrect recommendations for difficult errors, showed no benefit for easy errors, and showed a small benefit for impossible errors. Experiment 3 varied error and task difficulty within-subjects and extended these patterns; as task difficulty rose, participants behaved less rationally, exhibiting both under- and overreliance. Notably, these behavioral benefits were generally not accompanied by reduced trust in the AI system. Our findings suggest that disclosing system weaknesses enhances detection of AI errors but is most effective for tasks of moderate difficulty where AI recommendations are still verifiable.

The integration of large language models (LLMs) in voice assistants has introduced a new level of naturalness and functionality into human-assistant interactions, also in cars.These enhancements lead to more engaging interactions that carry the potential cost of increasing cognitive driver distraction.This assumption was tested in a simulated driving study, in which 30 participants interacted with an assistant adopting an informal and a formal conversational style.As a result, the informal style tends to be more distracting, especially when discussing personal topics, which provides implications for the design of LLM-based voice assistants in automotive environments.

AI-based decision support is increasingly implemented to support operators in dynamic control tasks. While these systems continuously improve, to truly achieve human–system synergy, one must also study humans’ system understanding and behavior. Accordingly, we investigated the impact of explainability instructions regarding a specific system weakness on performance and trust in two experiments (with higher task demands in Experiment 2). Participants performed a dynamic control task with support from either an explainable AI (XAI, information on a system weakness), a non-explainable AI (nonXAI, no information on system weakness), or without support (manual, only in Experiment 2). Results show that participants with XAI support outperformed those in the nonXAI group, particularly in situations where the AI actually erred. Notably, informing users of system weaknesses did not affect trust once they had interacted with the system. In addition, Experiment 2 showed the general benefit of decision support over working manually under higher task demands. These findings suggest that AI support can enhance performance in complex tasks and that providing information on potential system weaknesses aids in managing system errors and resource allocation without compromising trust. • Two experiments examined explainable AI in a dynamic supervisory control task. • One group of subjects was informed about AI weaknesses; i.e. where errors can occur. • Explainability improved performance, especially in cases where the AI erred. • Weakness information did not negatively affect trust in the AI after the interaction. • Explaining AI weaknesses enhanced human–AI collaboration without compromising trust.

In a growing number of application domains, human decision-making is being supported by automated systems. While previous research has focused extensively on the negative consequences of automation support in terms of an overuse of such systems, we argue that this focus has largely overlooked another crucial issue: Humans often deteriorate the performance of automation. Specifically, human–automation dyads commonly perform worse than the system alone because humans, in an attempt to improve decisions, unfortunately interfere with correct system recommendations. This problem will only grow as systems based on artificial intelligence (AI) become more reliable and the gap between human-only and system-only performance continues to widen. We therefore outline the need for research that addresses this persisting and increasingly relevant issue. One approach to counteract this problem is to make systems more transparent and give humans more information on the system. However, while numerous explainability approaches have been brought forward, only very few show convincing effects. To be truly useful, we argue that systems need to be explainable in terms of effective behavioral guidance. Furthermore, beyond just thinking about how to enable humans to better adapt to the system (as is the case with explainability approaches), systems should be more human-centric, taking into account human strengths and weaknesses, and ultimately adapting to humans to enable synergy between humans and AI.

Federated learning allows training machine learning (ML) models without central consolidation of the raw data. Variants of such federated learning systems enable privacy-preserving ML, and address data ownership and/or sharing constraints. However, existing work mostly adopt data-parallel parameter-server architectures for mini-batch training, require manual construction of federated runtime plans, and largely ignore the broad variety of data preparation, ML algorithms, and model debugging. Over the last years, we extended Apache SystemDS by an additional federated runtime backend for federated linear-algebra programs, federated parameter servers, and federated data preparation. In this paper, we share the system-level compiler and runtime integration, new features such as multi-tenant federated learning, selected federated primitives, multi-key homomorphic encryption, and our monitoring infrastructure. Our demonstrator showcases how composite ML pipelines can be compiled into federated runtime plans with low overhead.

Data science workflows are largely exploratory, dealing with under-specified objectives, open-ended problems, and unknown business value. Therefore, little investment is made in systematic acquisition, integration, and pre-processing of data. This lack of infrastructure results in redundant manual effort and computation. Furthermore, central data consolidation is not always technically or economically desirable or even feasible (e.g., due to privacy, and/or data ownership). The ExDRa system aims to provide system infrastructure for this exploratory data science process on federated and heterogeneous, raw data sources. Technical focus areas include (1) ad-hoc and federated data integration on raw data, (2) data organization and reuse of intermediates, and (3) optimization of the data science lifecycle, under awareness of partially accessible data. In this paper, we describe use cases, the overall system architecture, selected features of SystemDS' new federated backend (for federated linear algebra programs, federated parameter servers, and federated data preparation), as well as promising initial results. Beyond existing work on federated learning, ExDRa focuses on enterprise federated ML and related data pre-processing challenges. In this context, federated ML has the potential to create a more fine-grained spectrum of data ownership and thus, even new markets.