Recent knowledge graph embedding (KGE) models increasingly exploit algebraic structures to encode relational semantics.Cliffordbased models, in particular, offer strong expressiveness and geometric interpretability.In this work, we analyze the representations and decision boundaries of such models using an embedding-based reasoner as a classification function.To interpret Clifford-based geometric effects, we adapt DeepView, a visualization framework that approximates decision functions of deep classification models.This study provides one of the first systematic visual analyses of Clifford-based KGE models, helping bridge algebraic representation learning and interpretability.

Automatic label correction methods have been developed to address issues that come with a growing volume of annotated data in Machine Learning pipelines. The majority of these methods have been primarily evaluated on artificial noise in Euclidean data. Yet, particularly the subject of Geometric Deep Learning could benefit from well-performing label correction algorithms since non-Euclidean data, such as curved surfaces in 3D, is especially cumbersome and error prone to annotate. To assess the current applicability of existing label correction methods to 3D data, we evaluate automatic noise detection methods using realistic and artificial label noise in the 3D domain. We not only evaluate their effectiveness in 3D, but also examine general correction behavior regarding properties like carefulness and aggressiveness and compare those between both noise types. Our investigation shows that finding realistic noise in a 3D segmentation problem is of different nature compared to finding artificial noise.

Concept drift refers to the phenomenon that the underlying data distribution changes over time.While detection methods or model adjustment methods exist, a proper explanation of drift in high-dimensional settings is still widely unsolved.This problem is crucial since it enables an understanding of the most prominent drift characteristics.In this work, we propose to explain concept drift of high-dimensional data objects by means of concept activation vectors which give rise to local, phase, and a novel, global explanation called the Concept 2 Drift Distribution.

This paper examines how to improve interpretability of language models in the context of fairness.While traditional concept learning focuses on identifying the most important concepts for a task, this study explores how to locate the representation of sensitive attributes in pretrained language models.We address challenges such as the potential low importance and sparsity of sensitive attributes in training data, and the limited amount of labeled data for this purpose.Our experiments evaluate potential methods to obtain such identity concepts, considering factors like label sparsity, generalizability, and the influence of different language models on the representation of sensitive attributes.

In the domain of network intrusion detection, robustness against contaminated and noisy data inputs remains a critical challenge.This study introduces a probabilistic version of the Temporal Graph Network Support Vector Data Description (TGN-SVDD) model, designed to enhance detection accuracy in the presence of input noise.By predicting parameters of a Gaussian distribution for each network event, our model is able to naturally address noisy adversarials and improve robustness compared to a baseline model.Our experiments on a modified CIC-IDS2017 data set with synthetic noise demonstrate significant improvements in detection performance compared to the baseline TGN-SVDD model, especially as noise levels increase.

While 3D data is rich in information, it often comes with the drawback of being tedious to handle.Recent work in the Geometric Deep Learning community focused on developing high quality 3D datasets for tasks like mesh segmentation.However, the label quality can never be assured to be perfect.To improve label quality in 3D datasets, we propose an interactive algorithm combining DeepView, a method to visualize the classification function of neural networks, with Intrinsic Mesh CNNs, which generalize the convolution to Riemannian manifolds, to smartly select adequate sets of vertices from triangle mesh data for label correction.

Plenty of works have brought social biases in language models to attention and proposed methods to detect such biases. As a result, the literature contains a great deal of different bias tests and scores, each introduced with the premise to uncover yet more biases that other scores fail to detect. What severely lacks in the literature, however, are comparative studies that analyse such bias scores and help researchers to understand the benefits or limitations of the existing methods. In this work, we aim to close this gap for cosine based bias scores. By building on a geometric definition of bias, we propose requirements for bias scores to be considered meaningful for quantifying biases. Furthermore, we formally analyze cosine based scores from the literature with regard to these requirements. We underline these findings with experiments to show that the bias scores' limitations have an impact in the application case.

With the enourmous popularity of large language models, many researchers have raised ethical concerns regarding social biases incorporated in such models. Several methods to measure social bias have been introduced, but apparently these methods do not necessarily agree regarding the presence or severity of bias. Furthermore, some works have shown theoretical issues or severe limitations with certain bias measures.For that reason, we introduce SAME, a novel bias score for semantic bias in embeddings. We conduct a thorough theoretical analysis as well as experiments to show its benefits compared to similar bias scores from the literature. We further highlight a substantial relation of semantic bias measured by SAME with downstream bias, a connection that has recently been argued to be negligible. Instead, we show that SAME is capable of measuring semantic bias and identify potential causes for social bias in downstream tasks.

Kenneweg P, Schroeder S, Schulz A, Hammer B. Debiasing Sentence Embedders Through Contrastive Word Pairs. In: Proceedings of the 12th International Conference on Pattern Recognition Applications and Methods. Setúbal, Portugal: SCITEPRESS - Science and Technology Publications; 2023: 205-212.

Differentiable neural computers (DNCs) extend artificial neural networks with an explicit memory without interference, thus enabling the model to perform classic computation tasks, such as graph traversal. However, such models are difficult to train, requiring long training times and large datasets. In this work, we achieve some of the computational capabilities of DNCs with a model that can be trained very efficiently, namely, an echo state network with an explicit memory without interference. This extension enables echo state networks to recognize all regular languages, including those that contractive echo state networks provably cannot recognize. Furthermore, we demonstrate experimentally that our model performs comparably to its fully trained deep version on several typical benchmark tasks for DNCs.

Machine learning algorithms using deep architectures have been able to implement increasingly powerful and successful models. However, they also become increasingly more complex, more difficult to comprehend and easier to fool. So far, most methods in the literature investigate the decision of the model for a single given input datum. In this paper, we propose to visualize a part of the decision function of a deep neural network together with a part of the data set in two dimensions with discriminative dimensionality reduction. This enables us to inspect how different properties of the data are treated by the model, such as outliers, adversaries or poisoned data. Further, the presented approach is complementary to the mentioned interpretation methods from the literature and hence might be even more useful in combination with those. Code is available at https://github.com/LucaHermes/DeepView

Discriminative dimensionality reduction refers to the goal of visualizing given high-dimensional data in the plane such that the structure relevant for a specified aspect is displayed. While this framework has been successfully applied to visualize data with auxiliary label information, its extension to real-valued information is lacking. In this contribution, we propose a general way to shape data distances based on auxiliary real-valued information with the Fisher metric which is derived from a Gaussian process model of the data. This can directly be integrated into high quality non-linear dimensionality reduction methods such as t-SNE, as we will demonstrate in artificial as well as real life benchmarks.

Linear data transformations constitute essential operations in various machine learning algorithms, ranging from linear regression up to adaptive metric transformation. Often, linear scalings are not only used to improve the model accuracy, rather feature coefficients as provided by the mapping are interpreted as an indicator for the relevance of the feature for the task at hand. This principle, however, can be misleading in particular for high-dimensional or correlated features, since it easily marks irrelevant features as relevant or vice versa. In this contribution, we propose a mathematical formalisation of the minimum and maximum feature relevance for a given linear transformation which can efficiently be solved by means of linear programming. We evaluate the method in several benchmarks, where it becomes apparent that the minimum and maximum relevance closely resembles what is often referred to as weak and strong relevance of the features; hence unlike the mere scaling provided by the linear mapping, it ensures valid interpretability.