This article examines LLMs' ability to correctly label simple inferences with partisan conclusions.For this, we develop a dataset with both formal and material inferences, containing logically equivalent pairs of inferences with conclusions that favor either the political left or the political right.This allows us to focus on political bias as a source of decrease in performance.Our samples are synthetically generated and thus highly controlled, covering both English and German.We assess the performance of 16 configurations of both open and proprietary state-of-the-art LLMs on that dataset, finding generally unreliable performance as well as widespread political bias which, in the case of the English samples, persists throughout our experimental settings.

In the field of computational argument quality assessment, logic and dialectic are essential dimensions used to measure the quality of argumentative texts. Both of them have found their way into the field due to their importance to argumentation theory. We trace the development of core logical concepts of validity and soundness from their first use in argumentation theory to their understanding in state-of-the-art research. We show how, in the course of this development, dialectical considerations have taken center stage, at the cost of the logical perspective. Then, we take a closer look at the quality dimensions used in the field of computational argument quality assessment. Based on an analysis of prior empirical work in this field, we show how methodological considerations from argument theory can benefit state-of-the-art methods in computational argument quality assessment. We propose an even clearer separation between the two quality dimensions not only in regards to their definitions, but also in regards to the granularity at which the argumentative text is being annotated and assessed.

Controllable and transparent text generation has been a long-standing goal in NLP.Almost as long-standing is a general idea for addressing this challenge: Parsing text to a symbolic representation, and generating from it.However, earlier approaches were hindered by parsing and generation insufficiencies.Using modern parsers and a safety supervision mechanism, we show how close current methods come to this goal.Concretely, we propose the SENTENCE-SMITH framework for English, which has three steps: 1. Parsing a sentence into a semantic graph.2. Applying human-designed semantic manipulation rules.3. Generating text from the manipulated graph.A final entailment check (4.) verifies the validity of the applied transformation.To demonstrate our framework's utility, we use it to induce hard negative text pairs that challenge text embedding models.Since the controllable generation makes it possible to clearly isolate different types of semantic shifts, we can evaluate text embedding models in a fine-grained way, also addressing an issue in current benchmarking where linguistic phenomena remain opaque.Human validation confirms that our transparent generation process produces texts of good quality.Notably, our way of generation is very resource-efficient, since it relies only on smaller neural networks.

Do LLMs fall prey to Harnad's symbol grounding problem (SGP), as it has recently been claimed?We argue that this is not the case.Starting out with countering the arguments of Bender and Koller (2020), we trace the origins of the SGP to the computational theory of mind (CTM), and we show that it only arises with natural language when questionable theories of meaning are presupposed.We conclude by showing that it would apply to LLMs only if they were interpreted in the manner of how the CTM conceives the mind, i.e., by postulating that LLMs rely on a version of a language of thought, or by adopting said questionable theories of meaning; since neither option is rational, we conclude that the SGP does not apply to LLMs.

In academic writing, the competency to argue is important. However, first-year students often have difficulties to construct good arguments. Advances in natural language processing (NLP) have made it possible to better analyze the writing quality of texts. New tools have emerged which can give students individual feedback on their texts and the structure of their arguments. While the use of these argumentation learning support tools can help create better texts, using them in an academic context also carries risks. Learning scenarios are needed that promote argumentation competency using argumentation tools while also making students aware of their limitations. To address this issue, this paper investigates how a learning design with an argumentation learning support tool can be developed to increase the argumentation competency of first-year students. The conjecture-mapping technique was used, to visualize our assumptions and illustrate the developed learning design. As part of a fi rst design cycle, the learning design was tested with 80 students in seven academic writing classes at the University of St.Gallen in Switzerland. Preliminary findings suggest that the learning design might be helpful to improve the argumentation competency as well as the data-literacy of students (in relation to argumentation tools). However, further research is necessary to confirm or reject our hypotheses.

In linguistics, there are two main perspectives on negation: a semantic and a pragmatic view.So far, research in NLP on negation has almost exclusively adhered to the semantic view.In this article, we adopt the pragmatic paradigm to conduct a study of negation understanding focusing on transformer-based PLMs.Our results differ from previous, semantics-based studies and therefore help to contribute a more comprehensive -and, given the results, much more optimistic -picture of the PLMs' negation understanding.