Shapley values are widely recognized as a principled method for attributing importance to input features in machine learning. However, the exact computation of Shapley values scales exponentially with the number of features, severely limiting the practical application of this powerful approach. The challenge is further compounded when the predictive model is probabilistic---as in Gaussian processes (GPs)---where the outputs are random variables rather than point estimates, necessitating additional computational effort in modeling higher-order moments. In this work, we demonstrate that for an important class of GPs known as FANOVA GP, which explicitly models all main effects and interactions, exact Shapley attributions for both local and global explanations can be computed in *quadratic* time. For *local, instance-wise explanations*, we define a stochastic cooperative game over function components and compute the *exact stochastic Shapley value* in quadratic time only, capturing both the expected contribution and uncertainty. For *global explanations*, we introduce a deterministic, variance-based value function and compute exact Shapley values that quantify each feature’s contribution to the model’s overall sensitivity. Our methods leverage a closed-form (stochastic) Möbiusrepresentation of the FANOVA decomposition and introduce recursive algorithms, inspired by Newton's identities, to efficiently compute the mean and variance of Shapley values. Our work enhances the utility of explainable AI, as demonstrated by empirical studies, by providing more scalable, axiomatically sound, and uncertainty-aware explanations for predictions generated by structured probabilistic models.

The scalability of instructable agents in robotics or gaming is often hindered by limited data that pairs instructions with agent trajectories. However, large datasets of unannotated trajectories containing sequences of various agent behaviour (play trajectories) are often available. In a semi-supervised setup, we explore methods to extract labelled segments from play trajectories. The goal is to augment a small annotated dataset of instruction-trajectory pairs to improve the performance of an instruction-following policy trained downstream via imitation learning. Assuming little variation in segment length, recent video segmentation methods can effectively extract labelled segments. To address the constraint of segment length, we propose Play Segmentation (PS), a probabilistic model that finds maximum likely segmentations of extended subsegments, while only being trained on individual instruction segments. Our results in a game environment and a simulated robotic gripper setting underscore the importance of segmentation; randomly sampled segments diminish performance, while incorporating labelled segments from PS improves policy performance to the level of a policy trained on twice the amount of labelled data.

Shapley value-based explanations are widely utilized to demystify predictions made by opaque models. Approaches to estimating Shapley values often approximate explanation games as inessential and estimate the Shapley value directly as feature attribution with a limited capacity to quantify feature interactions. This paper introduces a new approach for calculating Shapley values that relaxes the assumption of inessential games and is proven to provide additive feature attribution. The initial formulation of the proposed approach includes the estimation of game values in their Möbius representation with exponentially many parameters, but we put forward a polynomial-time algorithm designed to manage the game's numerous values and achieve an efficient linear-time computation of the Shapley value. Moreover, this formulation uniquely enables identifying only the significant high-order feature interactions amidst a potentially exponential set. Through experiments, we demonstrate the robust performance of our methodology in game estimation and in providing explanations for multiple black-box models.

In recent years, employing Shapley values to compute feature importance has gained considerable attention. Calculating these values inherently necessitates managing an exponential number of parameters—a challenge commonly mitigated through an additivity assumption coupled with linear regression. This paper proposes a novel approach by modeling supervised learning as a multilinear game, incorporating both direct and interaction effects to establish the requisite values for Shapley value computation. To efficiently handle the exponentially increasing parameters intrinsic to multilinear games, we introduce a support vector machine (SVM)-based method for parameter estimation, its complexity is predominantly contingent on the number of samples due to the implementation of a dual SVM formulation. Additionally, we unveil an optimized dynamic programming algorithm capable of directly computing the Shapley value and interaction index from the dual SVM. Our proposed methodology is versatile and we demonstrate that it can be applied to local explanation and feature selection. Experiments underscore the competitive efficacy of our proposed methods in terms of feature selection and explanation.

The acquisition of commonsense knowledge remains a core challenge in both robotics and artificial intelligence. While Large Language Models (LLMs) encode rich latent knowledge, their unstructured and probabilistic nature limits their direct use in safety-critical domains like robotics. In this paper, we present a pipeline for extracting structured commonsense knowledge from LLMs and integrating it into a symbolic knowledge graph based on the Ontology for Robotic Knowledge Acquisition (ORKA). Grounded in the theory of conceptual spaces, our method targets physical object properties—both categorical (e.g., shape, material, location) and quantifiable (e.g., size, weight, temperature)—relevant to embodied reasoning. We evaluate the pipeline across a diverse set of LLMs, model sizes, and quantization levels, using human-annotated ground truth to assess accuracy. Results show that even compact and quantized models can produce reliable, interpretable knowledge. We also analyze the influence of measurement units and contextual relevance, highlighting trade-offs between model types and use cases. These results indicate that even smaller LLMs can serve as a useful intermediate source for deriving structured commonsense representations in robotics contexts.

Personal service robots are increasingly deployed to support daily living in domestic environments, particularly for elderly and individuals requiring assistance. These robots must perceive complex and dynamic surroundings, understand tasks, and execute context-appropriate actions. However, current systems typically rely on proprietary, hard-coded solutions tied to specific hardware and software, resulting in siloed implementations that are difficult to adapt and scale across platforms. Ontologies and Knowledge Graphs (KGs) offer a solution to enable interoperability across systems, through structured and standardized representations of knowledge and reasoning. However, symbolic systems such as KGs and ontologies struggle with raw and noisy sensory input. In contrast, multimodal language models are well suited for interpreting input such as images and natural language, but often lack transparency, consistency, and knowledge grounding. In this work, we propose a neurosymbolic framework that combines the perceptual strengths of multimodal language models with the structured representations provided by KGs and ontologies, with the aim of supporting interoperability in robotic applications. Our approach generates ontology-compliant KGs that can inform robot behavior in a platform-independent manner. We evaluated this framework by integrating robot perception data, ontologies, and five multimodal models (three LLaMA-based and two GPT-based models), each using different modes of neural-symbolic interaction. We assess the consistency and effectiveness of the generated KGs across multiple runs and configurations, and perform statistical analyzes to evaluate performance. Results show that GPT-o1 and LLaMA 4 Maverick consistently outperform other models. However, our findings also indicate that newer models do not guarantee better results, highlighting the critical role of the integration strategy in generating ontology-compliant KGs.

Understanding the rationale behind the predictions made by machine learning models holds paramount importance across numerous applications. Various explainable models have been developed to shed light on these predictions by assessing the individual contributions of features to the outcome of black-box models. However, existing methods often overlook the crucial aspect of interactions among features, restricting the explanation to isolated feature attributions. In this paper, we introduce a novel Choquet integral-based explainable method, termed ChoquEx, which not only considers the interactions among features but also enables the computation of contributions for any subset of features. To achieve this, we propose an innovative algorithm based on support vector regression that efficiently estimates the contributions of all feature subsets. Intriguingly, we leverage game-theoretic concepts, including Shapley values and interaction index, to calculate both the feature importance and interaction strength. This approach adds further interpretability and insight into the model’s decision-making process. To evaluate the effectiveness of ChoquEx, we conduct extensive experiments on diverse real-world scenarios. Our results convincingly demonstrate the superiority of the proposed model over existing explainable techniques. With its ability to unravel feature interactions and furnish comprehensive explanations, ChoquEx significantly enhances our understanding of predictive models, opening new avenues for applying machine learning in critical domains that require transparent decision-making.

Hybrid Intelligence (HI) is a rapidly growing field aiming at creating collaborative systems where humans and intelligent machines cooperate in mixed teams towards shared goals. A clear characterization of the tasks and knowledge exchanged by the agents in HI applications is still missing, hampering both standardization and reuse when designing new HI systems. Knowledge Engineering (KE) methods have been used to solve such issue through the formalization of tasks and roles in knowledge-intensive processes. We investigate whether KE methods can be applied to HI scenarios, and specifically whether common, reusable elements such as knowledge roles, tasks and subtasks can be identified in contexts where symbolic, subsymbolic and human-in-the-loop components are involved. We first adapt the well-known CommonKADS methodology to HI, and then use it to analyze several HI projects and identify common tasks. The results are (i) a high-level ontology of HI knowledge roles, (ii) a set of novel, HI-specific tasks and (iii) an open repository to store scenarios1 – allowing reuse, validation and design of existing and new HI applications.

BACKGROUND: Understanding the impact of gene interactions on disease phenotypes is increasingly recognised as a crucial aspect of genetic disease research. This trend is reflected by the growing amount of clinical research on oligogenic diseases, where disease manifestations are influenced by combinations of variants on a few specific genes. Although statistical machine-learning methods have been developed to identify relevant genetic variant or gene combinations associated with oligogenic diseases, they rely on abstract features and black-box models, posing challenges to interpretability for medical experts and impeding their ability to comprehend and validate predictions. In this work, we present a novel, interpretable predictive approach based on a knowledge graph that not only provides accurate predictions of disease-causing gene interactions but also offers explanations for these results. RESULTS: We introduce BOCK, a knowledge graph constructed to explore disease-causing genetic interactions, integrating curated information on oligogenic diseases from clinical cases with relevant biomedical networks and ontologies. Using this graph, we developed a novel predictive framework based on heterogenous paths connecting gene pairs. This method trains an interpretable decision set model that not only accurately predicts pathogenic gene interactions, but also unveils the patterns associated with these diseases. A unique aspect of our approach is its ability to offer, along with each positive prediction, explanations in the form of subgraphs, revealing the specific entities and relationships that led to each pathogenic prediction. CONCLUSION: Our method, built with interpretability in mind, leverages heterogenous path information in knowledge graphs to predict pathogenic gene interactions and generate meaningful explanations. This not only broadens our understanding of the molecular mechanisms underlying oligogenic diseases, but also presents a novel application of knowledge graphs in creating more transparent and insightful predictors for genetic research.

Enabling reinforcement learning (RL) agents to leverage a knowledge base while learning from experience promises to advance RL in knowledge intensive domains. However, it has proven difficult to leverage knowledge that is not manually tailored to the environment. We propose to use the subclass relationships present in open-source knowledge graphs to abstract away from specific objects. We develop a residual policy gradient method that is able to integrate knowledge across different abstraction levels in the class hierarchy. Our method results in improved sample efficiency and generalisation to unseen objects in commonsense games, but we also investigate failure modes, such as excessive noise in the extracted class knowledge or environments with little class structure.

This paper provides an extensive overview of the use of knowledge graphs in the context of Explainable Machine Learning. As of late, explainable AI has become a very active field of research by addressing the limitations of the latest machine learning solutions that often provide highly accurate, but hardly scrutable and interpretable decisions. An increasing interest has also been shown in the integration of Knowledge Representation techniques in Machine Learning applications, mostly motivated by the complementary strengths and weaknesses that could lead to a new generation of hybrid intelligent systems. Following this idea, we hypothesise that knowledge graphs, which naturally provide domain background knowledge in a machine-readable format, could be integrated in Explainable Machine Learning approaches to help them provide more meaningful, insightful and trustworthy explanations. Using a systematic literature review methodology we designed an analytical framework to explore the current landscape of Explainable Machine Learning. We focus particularly on the integration with structured knowledge at large scale, and use our framework to analyse a variety of Machine Learning domains, identifying the main characteristics of such knowledge-based, explainable systems from different perspectives. We then summarise the strengths of such hybrid systems, such as improved understandability, reactivity, and accuracy, as well as their limitations, e.g. in handling noise or extracting knowledge efficiently. We conclude by discussing a list of open challenges left for future research.

We tackle the problem of automatically detecting conflicting claims in research outputs. This has become even more urgent in recent years, with the increasing volume of scientific publications available. Researchers are struggling to keep pace with the literature, and to efficiently make comparisons between the results of different published studies. We hypothesise that the difficult and time-consuming process of searching and comparing results across research publications can be facilitated using machine-readable, standardised knowledge representation methods. To this end, we propose to exploit Nanopublications as the standard framework to represent the claims in research studies, and use provenance data expressed by the model as an indicator of the source of the contradiction between different claims. We evaluate this idea over the Cooperation Databank (CoDa); a repository of social science studies. Our results show that the use of provenance information can be a good factor to identify the cause of conflicting claims, and that our method can support scientists in comparing literature in a more automated way.

Deep neural networks have exploded in popularity and different types of networks are used to solve a multitude of complex tasks. One such task is anomaly detection, that a type of deep neural network called auto-encoder has become extremely proficient at solving. The low level neural activity, produced by such a network, generates extremely rich representations of the data, which can be used to extract task specific knowledge. In this paper, we built upon previous work and used co-activation graph analysis to extract knowledge from auto-encoders, that were trained for the specific task of anomaly detection. First, we outlined a method for extracting co-activation graphs from auto-encoders. Then, we performed graph analysis to discover that task specific knowledge from the auto-encoder was being encoded into the co-activation graph, and that the extracted knowledge could be used to reveal the role of individual neurons in the network.

The popularity of deep learning has increased tremendously in recent years due to its ability to efficiently solve complex tasks in challenging areas such as computer vision and language processing. Despite this success, low-level neural activity reproduced by Deep Neural Networks (DNNs) generates extremely rich representations of the data. These representations are difficult to characterise and cannot be directly used to understand the decision process. In this paper we build upon our exploratory work where we introduced the concept of a co-activation graph and investigated the potential of graph analysis for explaining deep representations. The co-activation graph encodes statistical correlations between neurons’ activation values and therefore helps to characterise the relationship between pairs of neurons in the hidden layers and output classes. To confirm the validity of our findings, our experimental evaluation is extended to consider datasets and models with different levels of complexity. For each of the considered datasets we explore the co-activation graph and use graph analysis to detect similar classes, find central nodes and use graph visualisation to better interpret the outcomes of the analysis. Our results show that graph analysis can reveal important insights into how DNNs work and enable partial explainability of deep learning models.

The focus of this work is to exploit ontologies to make robotic systems more accessible to non-expert users, therefore supporting the deployment of robot-integrated applications. Due to the increasing number of robotic platforms available for commercial use, robotic systems are nowadays being approached by users with different backgrounds, who are often more interested in the robots' high-level capabilities than their technical architecture. Without the right expertise however, using robots is restricted to the capabilities exposed by the platform provider, i.e. they can only be used as end products rather than as development platforms. Our hypothesis is that an ontological representation of the capabilities of robots could make these capabilities more accessible, reducing the complexity of robot programming and enabling non-experts to exploit these systems to a much larger extent. To demonstrate this, an ontology abstracting the capabilities exposed by the most common robotic middleware (ROS) is integrated in a system to allow non-experts to program robots of different types and capabilities without previous knowledge either of the specific robotic platform being considered, or of the intricate systems used in its implementation. Our experiments, in which non-experts users had to configure the system in order to make robots achieve different tasks, show how the efforts required for realizing basic tasks using available robotic platforms can be sensibly reduced through our approach.

In this paper, we propose an ontology design pattern for the concept of "explanation". The motivation behind this work comes from our research, which focuses on automatically identifying explanations for data patterns. If we want to produce explanations from data agnostically from the application domain, we first need a formal definition of what an explanation is, i.e. which are its components, their roles or their interactions. We analysed and surveyed works from the disciplines grouped under the name of Cognitive Sciences, with the aim of identifying differences and commonalities in the way their researchers intend the concept of explanation. We then produced not only an ontology design pattern to model it, but also the instantiations of this in each of the analysed disciplines. Besides those contributions, the paper presents how the proposed ontology design pattern can be used to analyse the validity of the explanations produced by our, and other, frameworks.