A large number of embeddings trained on medical data have emerged, but it remains unclear how well they represent medical terminology, in particular whether the close relationship of semantically similar medical terms is encoded in these embeddings. To date, only small datasets for testing medical term similarity are available, not allowing to draw conclusions about the generalisability of embeddings to the enormous amount of medical terms used by doctors. We present multiple automatically created large-scale medical term similarity datasets and confirm their high quality in an annotation study with doctors. We evaluate state-of-the-art word and contextual embeddings on our new datasets, comparing multiple vector similarity metrics and word vector aggregation techniques. Our results show that current embeddings are limited in their ability to adequately encode medical terms. The novel datasets thus form a challenging new benchmark for the development of medical embeddings able to accurately represent the whole medical terminology.

A promising application of AI to healthcare is the retrieval of information from electronic health records (EHRs), e.g. to aid clinicians in finding relevant information for a consultation or to recruit suitable patients for a study.This requires search capabilities far beyond simple string matching, including the retrieval of concepts (diagnoses, symptoms, medications, etc.) related to the one in question.The suitability of AI methods for such applications is tested by predicting the relatedness of concepts with known relatedness scores.However, all existing biomedical concept relatedness datasets are notoriously small and consist of hand-picked concept pairs.We opensource a novel concept relatedness benchmark overcoming these issues: it is six times larger than existing datasets and concept pairs are chosen based on co-occurrence in EHRs, ensuring their relevance for the application of interest.We present an in-depth analysis of our new dataset and compare it to existing ones, highlighting that it is not only larger but also complements existing datasets in terms of the types of concepts included.Initial experiments with state-of-the-art embedding methods show that our dataset is a challenging new benchmark for testing concept relatedness models.

Diagnostic reasoning is a key component of many professions. To improve students’ diagnostic reasoning skills, educational psychologists analyse and give feedback on epistemic activities used by these students while diagnosing, in particular, hypothesis generation, evidence generation, evidence evaluation, and drawing conclusions. However, this manual analysis is highly time-consuming. We aim to enable the large-scale adoption of diagnostic reasoning analysis and feedback by automating the epistemic activity identification. We create the first corpus for this task, comprising diagnostic reasoning selfexplanations of students from two domains annotated with epistemic activities. Based on insights from the corpus creation and the task’s characteristics, we discuss three challenges for the automatic identification of epistemic activities using AI methods: the correct identification of epistemic activity spans, the reliable distinction of similar epistemic activities, and the detection of overlapping epistemic activities. We propose a separate performance metric for each challenge and thus provide an evaluation framework for future research. Indeed, our evaluation of various state-of-the-art recurrent neural network architectures reveals that current techniques fail to address some of these challenges.

Claudia Schulz, Christian M. Meyer, Jan Kiesewetter, Michael Sailer, Elisabeth Bauer, Martin R. Fischer, Frank Fischer, Iryna Gurevych. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2019.

Automated fact-checking based on machine learning is a promising approach to identify false information distributed on the web.In order to achieve satisfactory performance, machine learning methods require a large corpus with reliable annotations for the different tasks in the fact-checking process.Having analyzed existing fact-checking corpora, we found that none of them meets these criteria in full.They are either too small in size, do not provide detailed annotations, or are limited to a single domain.Motivated by this gap, we present a new substantially sized mixed-domain corpus with annotations of good quality for the core fact-checking tasks: document retrieval, evidence extraction, stance detection, and claim validation.To aid future corpus construction, we describe our methodology for corpus creation and annotation, and demonstrate that it results in substantial inter-annotator agreement.As baselines for future research, we perform experiments on our corpus with a number of model architectures that reach high performance in similar problem settings.Finally, to support the development of future models, we provide a detailed error analysis for each of the tasks.Our results show that the realistic, multi-domain setting defined by our data poses new challenges for the existing models, providing opportunities for considerable improvement by future systems.

Different semantics of abstract Argumentation Frameworks (AFs) provide different levels of decisiveness for reasoning about the acceptability of conflicting arguments.The stable semantics is useful for applications requiring a high level of decisiveness, as it assigns to each argument the label "accepted" or the label "rejected". Unfortunately, stable labellings are not guaranteed to exist, thus raising the question as to which parts of AFs are responsible for the non-existence. In this paper, we address this question by investigating a more general question concerning preferred labellings (which may be less decisive than stable labellings but are always guaranteed to exist), namely why a given preferred labelling may not be stable and thus undecided on some arguments. In particular, (1) we give various characterisations of parts of an AF, based on the given preferred labelling, and (2) we show that these parts are indeed responsible for the undecisiveness if the preferred labelling is not stable. We then use these characterisations to explain the non-existence of stable labellings.

With the rise of machine learning, and more recently the overwhelming interest in deep learning, knowledge representation and reasoning (KRR) approaches struggle to maintain their position within the wider Artificial Intelligence (AI) community. Often considered as part of thegood old-fashioned AI(Haugeland 1985) – like a memory of glorious old days that have come to an end – many consider KRR as no longer applicable (on its own) to the problems faced by AI today (Blackwell 2015; Garneloet al.2016). What they see are logical languages with symbols incomprehensible by most, inference mechanisms that even experts have difficulties tracing and debugging, and the incapability to process unstructured data like text.

Abstract Artificial intelligence (AI) approaches to problem-solving and decision-making are becoming more and more complex, leading to a decrease in the understandability of solutions. The European Union’s new General Data Protection Regulation tries to tackle this problem by stipulating a “right to explanation” for decisions made by AI systems. One of the AI paradigms that may be affected by this new regulation is answer set programming (ASP). Thanks to the emergence of efficient solvers, ASP has recently been used for problem-solving in a variety of domains, including medicine, cryptography, and biology. To ensure the successful application of ASP as a problem-solving paradigm in the future, explanations of ASP solutions are crucial. In this survey, we give an overview of approaches that provide an answer to the question ofwhyan answer set is a solution to a given problem, notably off-line justifications, causal graphs, argumentative explanations, and why-not provenance, and highlight their similarities and differences. Moreover, we review methods explaining why a set of literals isnotan answer set or why no solution exists at all.

Research in social sciences and humanities of ten involves analysing data to draw scientific conclusions. This however requires the manual coding of the data, which is highly time-consuming. A use case is the coding of students' essays in education to draw conclusions about students' reasoning and argumentation. The NeuralWeb API tackles this problem by providing automatic codings to other software components. These codings can for example be used in annotation platforms in terms of recommendations for expert coders from social sciences and humanities. After some initial manual annotations, the expert coders then merely need to verify the correctness of the automatic codings instead of manually annotating all data.

In this work, we explain how Assumption-Based Argumentation (ABA) is subsumed by Logic Programming (LP). The translation from ABA to LP (with a few restrictions on the ABA framework) results in a normal logic program whose semantics coincide with the semantics of the underlying ABA framework. Although the precise technicalities are beyond the current extended abstract (these can be found in the associated full paper) we provide a number of examples to illustrate the general idea.

Different semantics of abstract Argumentation Frameworks (AFs) provide different levels of decisiveness for reasoning about the acceptability of conflicting arguments. The stable semantics is useful for applications requiring a high level of decisiveness, as it assigns to each argument the label “accepted” or the label “rejected”. Unfortunately, stable labellings are not guaranteed to exist, thus raising the question as to which parts of AFs are responsible for the non-existence. In this paper, we address this question by investigating a more general question concerning preferred labellings (which may be less decisive than stable labellings but are always guaranteed to exist), namely why a given preferred labelling may not be stable and thus undecided on some arguments. In particular, (1) we give various characterisations of parts of an AF, based on the given preferred labelling, and (2) we show that these parts are indeed responsible for the undecisiveness if the preferred labelling is not stable. We then use these characterisations to explain the non-existence of stable labellings. We present two types of characterisations, based on labellings that are more (or equally) committed than the given preferred labelling on the one hand, and based on the structure of the given AF on the other, and compare the respective AF parts deemed responsible. To prove that our characterisations indeed yield responsible parts, we use a notion of enforcement of labels through structural revision, by means of which the preferred labelling of the given AF can be turned into a stable labelling of the structurally revised AF. Rather than prescribing how this structural revision is carried out, we focus on the enforcement of labels and leave the engineering of the revision open to fulfil differing requirements of applications and information available to users.

Assumption-Based Argumentation (ABA) has been shown to subsume various other non-monotonic reasoning formalisms, among them normal logic programming (LP). We re-examine the relationship between ABA and LP and show that normal LP also subsumes (flat) ABA. More precisely, we specify a procedure that given a (flat) ABA framework yields an associated logic program with almost the same syntax whose semantics coincide with those of the ABA framework. That is, the 3-valued stable (respectively well-founded, regular, 2-valued stable, and ideal) models of the associated logic program coincide with the complete (respectively grounded, preferred, stable, and ideal) assumption labellings and extensions of the ABA framework. Moreover, we show how our results on the translation from ABA to LP can be reapplied for a reverse translation from LP to ABA, and observe that some of the existing results in the literature are in fact special cases of our work. Overall, we show that (flat) ABA frameworks can be seen as normal logic programs with a slightly different syntax. This implies that methods developed for one of these formalisms can be equivalently applied to the other by simply modifying the syntax.

ArgTeach is a web-platform for teaching and learning the labelling semantics of abstract argumentation frameworks. The user's task is to find the complete labellings of an abstract argumentation framework, supported by ArgTeach in the form of hints about sensible next labelling steps as well as error checking of partial and total labellings. The ArgTeach web-platform considerably improves and extends an earlier java application, both with respect to functionality and design.

Abstract An answer set is a plain set of literals which has no further structure that would explain why certain literals are part of it and why others are not. We show how argumentation theory can help to explain why a literal is or is not contained in a given answer set by defining two justification methods, both of which make use of the correspondence between answer sets of a logic program and stable extensions of the assumption-based argumentation (ABA) framework constructed from the same logic program.Attack Treesjustify a literal in argumentation-theoretic terms, i.e. using arguments and attacks between them, whereasABA-Based Answer Set Justificationsexpress the same justification structure in logic programming terms, that is using literals and their relationships. Interestingly, an ABA-Based Answer Set Justification corresponds to an admissible fragment of the answer set in question, and an Attack Tree corresponds to an admissible fragment of the stable extension corresponding to this answer set.

Logic Programming and Argumentation Theory have been existing side by side as two separate, yet related, techniques in the field of Knowledge Representation and Reasoningfor many years.When Assumption-Based Argumentation (ABA) was first introduced in the nineties,the authors showed how a logic program can be encoded in an ABA framework andproved that the stable semantics of a logic program corresponds to the stable extension semantics of the ABA framework encoding this logic program.We revisit this initial work by provingthat the 3-valued stable semantics of a logic program coincides with the complete semantics of the encoding ABA framework,and that the L-stable semantics of this logic program coincides with the semi-stable semantics of the encoding ABA framework.Furthermore, we show how to graphically represent the structure of a logic program encoded in an ABA frameworkand that not only logic programming and ABA semanticsbut also Abstract Argumentation semantics can be easily applied to a logic program using these graphical representations.

Since Assumption-Based Argumentation (ABA) was introduced in the nineties,the structure and semantics of an ABA framework have been studied exclusively in logical termswithout any graphical representation.Here, we show how an ABA framework and its complete semantics can be displayed in a graph,clarifying the structure of the ABA framework as well as the resulting complete assumption labellings.Furthermore, we show that such an ABA graph can be used to represent the structureand semantics of a logic program (LP), based on the correspondence between the semantics of a LP and an ABA framework encoding this LP.

Argumentation Theory and Answer Set Programming (ASP) are two prominent theories in the field of knowledge representation and non-monotonic reasoning,where Argumentation Theory stands for a variety of approaches following similar ideas.The main difference between Argumentation Theory and ASP is that the former focusses on representing knowledge and reasoning about it in a way that resembles human reasoning, neglecting the efficiency of the reasoning procedure,whereas the latter is concerned with the efficient computation of solutions to a reasoning problem, resulting in a less human-understandable process. In recent years, ASP has been frequently applied for the computation of reasoning problems represented in argumentation-theoretical-terms and has been found an efficient method for determining solutions to problems in Argumentation Theory. My research is concerned with the opposite direction, i.e. with applying Argumentation Theory to ASP in order to explain the solutions to an ASP reasoning problem in a more human-understandable way.Developing such an explanation method also involves to investigate both the exact relationship between different approaches in Argumentation Theory in order to find the most suitable one for explanations and their connection with ASP, in particular with respect to their semantics.

Recently, argument labellings have been proposed as a new (equivalent) way to express the extension semantics of Abstract Argumentation (AA) frame-works. Here, we introduce a labelling approach for the complete semantics in Assumption-Based Argumentation (ABA), where labels are assigned to assumptions rather than whole arguments. We prove that the complete assumption labelling corresponds to the complete extension semantics in ABA, as well as to the complete extension semantics and the complete argument labelling in AA.

Following the increasing influence of the formal study of argumentation in AI research, Abstract Argumentation (AA) is now taught as part of Computer Science degrees at various universities. To support the teaching of AA we present ARGTEACH, an interactive intelligent tutor that facilitates the learning of different labelling semantics in AA. The user assigns the labels in, out, and undec to arguments in an AA framework displayed as a graph, with the aim to find all complete labellings. The user then determines which of the complete labellings are grounded, preferred, semi-stable, or stable. During the labelling process, ARGTEACH supports the user by providing hints about possible next labelling steps, using a novel method for computing complete labellings, and by checking whether the labelling done so far is correct.