Abstract To address the task of diagram understanding we propose to distinguish between the perception of the geometric configuration of a diagram from the assignment of meaning to the geometric entities and their topological relationships. As a consequence, diagram parsing does not need to assume any particular a priori interpretations of diagrams and their constituents. Focussing on Euler diagrams, we tackle the first of these subtasks—that of identifying the geometric entities that constitute a diagram (i.e., circles, rectangles, lines, arrows, etc.) and their topological relations—as an image captioning task, using a Vision Transformer for image recognition combined with language model GPT-2 to generate qualitative spatial descriptions of Euler diagrams with an encoder-decoder model. Due to the lack of sufficient high-quality data to train the pre-trained language model for this task, we describe how we generated a synthetic dataset of Euler diagrams annotated with qualitative spatial representations based on the Region Connection Calculus (RCC8). Results showed over 95% accuracy of the transformer-based language model in the generation of meaning-carrying RCC8 specifications for given Euler diagrams.

Using norms to guide and coordinate interactions has gained tremendous attention in the multi-agent community. However, new challenges arise as the interest moves towards dynamic socio-technical systems, where human and software agents interact, and interactions are required to adapt to human's changing needs. For instance, different agents (human or software) might not have the same understanding of what it means to violate a norm (e.g., what characterizes hate speech), or their understanding of a norm might change over time (e.g., what constitutes an acceptable response time). The challenge is to address these issues by learning the meaning of a norm violation from limited interaction data. For this, we use batch and incremental learning to train an ensemble of classifiers. Ensemble learning and data-sampling handle the imbalanced class distribution of the interaction stream. At the same time, the training approaches use different strategies to ensure that the ensemble models reflect the latest community view on the meaning of norm violation. Batch learning uses weight assignment, while incremental learning continuously updates the ensemble models as community members interact. Here, we extend our previous work by creating a different balance strategy for online learning and integrating interpretability to understand norm violations. Additionally, we evaluate the proposed approaches in the context of Wikipedia article edits, where interactions revolve around editing articles, and the norm in question is prohibiting vandalism. Lastly, we conduct ablation studies to compare the ensemble's performance against a single model approach and to examine the behavior of two data sampling techniques. Results indicate that the different machine learning frameworks can learn the meaning of a norm violation in a setting with data imbalance and concept drift, although with significant differences.

Cross-community learning incorporates data from different sources to leverage task-specific solutions in a target community. This approach is particularly interesting for low-resource or newly created online communities, where data formalizing interactions between agents (community members) are limited. In such scenarios, a normative system that intends to regulate online interactions faces the challenge of continuously learning the meaning of norm violation as communities' views evolve, either with changes in the understanding of what it means to violate a norm or with the emergence of new violation classes. To address this issue, we propose the Cross-community Adapter Learning (CAL) framework, which combines adapters and transformer-based models to learn the meaning of norm violations expressed as textual sentences. Additionally, we analyze the differences in the meaning of norm violations between communities, using Integrated Gradients (IG) to understand the inner workings of our model and calculate a global relevance score that indicates the relevance of words for violation detection. Results show that cross-community learning enhances CAL's performance while explaining the differences in the meaning of norm-violating behavior based on community members' feedback. We evaluate our proposal in a small set of interaction data from Wikipedia, in which the norm prohibits hate speech.

Abstract Using norms to guide and coordinate interactions has gained tremendous attention in the multiagent community. However, new challenges arise as the interest moves towards dynamic socio-technical systems, where human and software agents interact, and interactions are required to adapt to changing human needs. For instance, different agents (human or software) might not have the same understanding of what it means to violate a norm (e.g., what characterizes hate speech), or their understanding of a norm might change over time (e.g., what constitutes an acceptable response time). The challenge is to address these issues by learning to detect norm violations from the limited interaction data and to explain the reasons for such violations. To do that, we propose a framework that combines Machine Learning (ML) models and incremental learning techniques. Our proposal is equipped to solve tasks in both tabular and text classification scenarios. Incremental learning is used to continuously update the base ML models as interactions unfold, ensemble learning is used to handle the imbalance class distribution of the interaction stream, Pre-trained Language Model (PLM) is used to learn from text sentences, and Integrated Gradients (IG) is the interpretability algorithm. We evaluate the proposed approach in the use case of Wikipedia article edits, where interactions revolve around editing articles, and the norm in question is prohibiting vandalism. Results show that the proposed framework can learn to detect norm violation in a setting with data imbalance and concept drift.

Abstract The literature on diagrammatic reasoning includes theoretical and experimental work on the effectiveness of diagrams for conveying information. One influential theoretical contribution to this field proposes that a notation that is more effective than another would have an observational advantage over it; that is, it would make certain pieces of information observable—by means of some visual, meaning-carrying relationships—that were not observable in the other. Although the notion of observational advantage captures a relevant aspect of the benefit of using one notation over another, we present here an example where this notion is not sufficient to distinguish between a more and a less effective diagram. We suggest to take the theory of observational advantage one step further by linking it to cognitive theories of human conceptualisation and reasoning. Following our previous work, we propose that the act of observing facts about set theory from the geometry of a diagram can be modeled as a conceptual blend of image schemas with parts of the geometric configuration of a diagram. Image schemas are elementary mental structures that crystallize early embodied experiences, allowing agents to make sense of what they perceive by conceptualising it in terms of these structures (e.g., container, link, scale etc.). With our approach, we can extend the theory of observational advantage to take into account the cognitive complexity of the act of observation. Concretely, we present an example of an Euler and a Hasse diagram, and we posit that, while their observational advantage is equivalent, the Hasse diagram requires a much more complex network of conceptual blends to model certain observations made from it. Thus, to reason about certain set-theoretic claims, a Hasse diagram is less cognitively effective than an Euler diagram with equivalent observational advantage. We believe our approach contributes to the theoretical discussion on what factors affect the effectiveness of a diagram, and provides new avenues for the exploration of how the embodied experiences of the users contribute to the way they reason with diagrams.

Abstract In this work, we propose a formal, computational model of the sense-making of diagrams by using the theories of image schemas and conceptual blending, stemming from cognitive linguistics. We illustrate our model here for the case of a Hasse diagram, using typed first-order logic to formalise the image schemas and to represent the geometry of a diagram. The latter additionally requires the use of some qualitative spatial reasoning formalisms. We show that, by blending image schemas with the geometrical configuration of a diagram, we can formally describe the way our cognition structures the understanding of, and the reasoning with, diagrams. In addition to a theoretical interest for diagrammatic reasoning, we also briefly discuss the cognitive underpinnings of good practice in diagram design, which are important for fields such as human-computer interaction and data visualization.

We present a computational framework for conceptual blending , a concept invention method that is advocated in cognitive science as a fundamental and uniquely human engine for creative thinking. Our framework treats a crucial part of the blending process, namely the generalisation of input concepts, as a search problem that is solved by means of modern answer set programming methods to find commonalities among input concepts. We also address the problem of pruning the space of possible blends by introducing metrics that capture most of the so-called optimality principles , described in the cognitive science literature as guidelines to produce meaningful and serendipitous blends. As a proof of concept, we demonstrate how our system invents novel concepts and theories in domains where creativity is crucial, namely mathematics and music.

In this paper we tackle the problem of semantic heterogeneity in multi-agent communication, i.e., when agents in a multi-agent system use different vocabularies for message passing, or might interpret shared vocabulary in varying ways. The problem of achieving meaningful communication in such semantically heterogeneous multi-agent interactions has been mainly tackled either by using ontology alignments to translate vocabularies, or by using methods that learn an alignment by observing how the utterance of particular terms affects the unfolding of an interaction. We propose solutions that combine these approaches and study how agents can use external alignments with possibly incomplete or erroneous mappings when communicating with each other in the context of a multi-agent interaction. We further show experimentally that with the experience gained through repeated interactions and by using simple learning techniques agents can find and repair those mappings of an ontology alignment that lead to unsuccessful interactions, thus improving the success rate of their future interactions.

Trust models as thus far described in the literature can be seen as a monolithic structure: a trust model is provided with a variety of inputs and the model performs calculations, resulting in a trust evaluation as output. The agent has no direct method of adapting its trust model to its needs in a given context. In this article, we propose a first step in allowing an agent to reason about its trust model, by providing a method for incorporating a computational trust model into the cognitive architecture of the agent. By reasoning about the factors that influence the trust calculation the agent can effect changes in the computational process, thus proactively adapting its trust model. We give a declarative formalization of this system using a multi-context system and we show that three contemporary trust models, BRS, ReGReT and ForTrust can be incorporated into a BDI reasoning system using our framework. © The Author, 2012. Published by Oxford University Press. All rights reserved.

We address the problem of how to reason about properties of knowledge transformations as they occur in distributed and decentralized interactions between large and complex artifacts, such as databases, web services, and ontologies. Based on the conceptual distinction between specifications of interactions and properties of knowledge transformations that follow from these interactions, we explore a novel mixture of process calculus and property inference by connecting interaction models with knowledge transformation rules. We aim at being generic in our exploration, hence our emphasis on abstract knowledge transformations, although we exemplify it using a lightweight specification language for interaction modeling (for which an executable peer-to-peer environment already exists) and provide a formal semantics for knowledge transformation rules using the theory of institutions. Consequently, our exploration is also an example of the gain obtained by linking current state-of-the-art distributed knowledge engineering based on web services and peer-based architectures with formal methods drawn from a long tradition in algebraic specification.

Ontology alignment foundations are hard to find in the literature. The abstract nature of the topic and the diverse means of practice make it difficult to capture it in a universal formal foundation. We argue that such a lack of formality hinders further development and convergence of practices, and in particular, prevents us from achieving greater levels of automation. In this article we present a formal foundation for ontology alignment that is based on interaction models between heterogeneous agents on the Semantic Web. We use the mathematical notion of information flow in a distributed system to ground our three hypotheses of enabling semantic interoperability and we use a motivating example throughout the article: how to progressively align two ontologies of research quality assessment through meaning coordination. We conclude the article with the presentation—in an executable specification language—of such an ontology-alignment interac-tion model.