Moritz Altemeyer, Steffen Eger, Johannes Daxenberger, Yanran Chen, Tim Altendorf, Philipp Cimiano, Benjamin Schiller. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025.

Topic-Dependent Argument Mining (TDAM), that is extracting and classifying argument components for a specific topic from large document sources, is an inherently difficult task for machine learning models and humans alike, as large TDAM datasets are rare and recognition of argument components requires expert knowledge.The task becomes even more difficult if it also involves stance detection of retrieved arguments.In this work, we investigate the effect of TDAM dataset composition in few-and zeroshot settings.Our findings show that, while fine-tuning is mandatory to achieve acceptable model performance, using carefully composed training samples and reducing the training sample size by up to almost 90% can still yield 95% of the maximum performance.This gain is consistent across three TDAM tasks on three different datasets.We also publish a new dataset 1 and code 2 for future benchmarking.

Benjamin Schiller, Johannes Daxenberger, Iryna Gurevych. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2021.

Nandan Thakur, Nils Reimers, Johannes Daxenberger, Iryna Gurevych. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2021.

Niklas Rach, Carolin Schindler, Isabel Feustel, Johannes Daxenberger, Wolfgang Minker, Stefan Ultes. Proceedings of the 22nd Annual Meeting of the Special Interest Group on Discourse and Dialogue. 2021.

Prior work has commonly defined argument retrieval from heterogeneous document collections as a sentence-level classification task. Consequently, argument retrieval suffers both from low recall and from sentence segmentation errors making it difficult for humans and machines to consume the arguments. In this work, we argue that the task should be performed on a more fine-grained level of sequence labeling. For this, we define the task as Argument Unit Recognition and Classification (AURC). We present a dataset of arguments from heterogeneous sources annotated as spans of tokens within a sentence, as well as with a corresponding stance. We show that and how such difficult argument annotations can be effectively collected through crowdsourcing with high inter-annotator agreement. The new benchmark, AURC-8, contains up to 15% more arguments per topic as compared to annotations on the sentence level. We identify a number of methods targeted at AURC sequence labeling, achieving close to human performance on known domains. Further analysis also reveals that, contrary to previous approaches, our methods are more robust against sentence segmentation errors. We publicly release our code and the AURC-8 dataset.1

Sentence encoders map sentences to real valued vectors for use in downstream applications.To peek into these representations-e.g., to increase interpretability of their resultsprobing tasks have been designed which query them for linguistic knowledge.However, designing probing tasks for lesser-resourced languages is tricky, because these often lack largescale annotated data or (high-quality) dependency parsers as a prerequisite of probing task design in English.To investigate how to probe sentence embeddings in such cases, we investigate sensitivity of probing task results to structural design choices, conducting the first such large scale study.We show that design choices like size of the annotated probing dataset and type of classifier used for evaluation do (sometimes substantially) influence probing outcomes.We then probe embeddings in a multilingual setup with design choices that lie in a 'stable region', as we identify for English, and find that results on English do not transfer to other languages.Fairer and more comprehensive sentence-level probing evaluation should thus be carried out on multiple languages in the future.

We present an approach to evaluate argument search techniques in view of their use in argumentative dialogue systems by assessing quality aspects of the retrieved arguments. To this end, we introduce a dialogue system that presents arguments by means of a virtual avatar and synthetic speech to users and allows them to rate the presented content in four different categories (Interesting, Convincing, Comprehensible, Relation). The approach is applied in a user study in order to compare two state of the art argument search engines to each other and with a system based on traditional web search. The results show a significant advantage of the two search engines over the baseline. Moreover, the two search engines show significant advantages over each other in different categories, thereby reflecting strengths and weaknesses of the different underlying techniques.

We experiment with two recent contextualized word embedding methods (ELMo and BERT) in the context of open-domain argument search.For the first time, we show how to leverage the power of contextualized word embeddings to classify and cluster topic-dependent arguments, achieving impressive results on both tasks and across multiple datasets.For argument classification, we improve the state-of-the-art for the UKP Sentential Argument Mining Corpus by 20.8 percentage points and for the IBM Debater -Evidence Sentences dataset by 7.4 percentage points.For the understudied task of argument clustering, we propose a pre-training step which improves by 7.8 percentage points over strong baselines on a novel dataset, and by 12.3 percentage points for the Argument Facet Similarity (AFS) Corpus. 1

Argumentation mining (AM) requires the identification of complex discourse structures and has lately been applied with success monolingually. In this work, we show that the existing resources are, however, not adequate for assessing cross-lingual AM, due to their heterogeneity or lack of complexity. We therefore create suitable parallel corpora by (human and machine) translating a popular AM dataset consisting of persuasive student essays into German, French, Spanish, and Chinese. We then compare (i) annotation projection and (ii) bilingual word embeddings based direct transfer strategies for cross-lingual AM, finding that the former performs considerably better and almost eliminates the loss from cross-lingual transfer. Moreover, we find that annotation projection works equally well when using either costly human or cheap machine translations. Our code and data are available at http://github.com/UKPLab/coling2018-xling_argument_mining.

Detecting biased language in written discourse is a highly relevant area of research in political communication and other social sciences, given the large quantity of information exchanged in public online platforms. In this abstract, we discuss an approach based on the concept of "incivility"-assessing biased text on the Facebook pages of established news media. News outlets are forced to put increasing efforts into preventing heated debates from turning into disrespectful discussions on their social media platforms. By scaling the analysis from a few thousand manually coded samples to more than a million comments, we take a step towards supporting media outlets in (semi-)automatizing the detection of uncivil comments and enable a much broader analysis of the latter.

We investigate neural techniques for endto-end computational argumentation mining (AM).We frame AM both as a tokenbased dependency parsing and as a tokenbased sequence tagging problem, including a multi-task learning setup.Contrary to models that operate on the argument component level, we find that framing AM as dependency parsing leads to subpar performance results.In contrast, less complex (local) tagging models based on BiL-STMs perform robustly across classification scenarios, being able to catch longrange dependencies inherent to the AM problem.Moreover, we find that jointly learning 'natural' subtasks, in a multi-task learning setup, improves performance.

Research on peer-production suggests that as participants choose what actions to perform, prototypical activity patterns emerge. Recent work characterized these patterns and demonstrated that informal emergent roles are highly stable. Nonetheless, we know little about the ways in which contributors take on and shed emergent roles. The objectives of this study are to: (a) delineate the temporal dynamics of participants' emergent role taking behaviors, and (b) identify the motivations driving role-transition behaviors. Our study links motivation to role-transition behaviors within Wikipedia. Our first sample covered eleven years and 222,119 contributors, and was used to identify four categories of temporal role-taking behaviors, that differ in their mobility between emergent roles and across Wikipedia articles. Our second examination linked the motivations of 175 new participants to their subsequent role-taking activity over 14 months. Together, the two analyses reveal that role-taking categories can be distinguished based on participants' motivational orientation (intrinsic/extrinsic and self/others-oriented).

Argument mining has become a popular research area in NLP.It typically includes the identification of argumentative components, e.g.claims, as the central component of an argument.We perform a qualitative analysis across six different datasets and show that these appear to conceptualize claims quite differently.To learn about the consequences of such different conceptualizations of claim for practical applications, we carried out extensive experiments using state-of-the-art featurerich and deep learning systems, to identify claims in a cross-domain fashion.While the divergent conceptualization of claims in different datasets is indeed harmful to cross-domain classification, we show that there are shared properties on the lexical level as well as system configurations that can help to overcome these gaps.

In this paper, we analyze a novel set of features for the task of automatic edit category classification.Edit category classification assigns categories such as spelling error correction, paraphrase or vandalism to edits in a document.Our features are based on differences between two versions of a document including meta data, textual and language properties and markup.In a supervised machine learning experiment, we achieve a micro-averaged F1 score of .62 on a corpus of edits from the English Wikipedia.In this corpus, each edit has been multi-labeled according to a 21-category taxonomy.A model trained on the same data achieves state-of-the-art performance on the related task of fluency edit classification.We apply pattern mining to automatically labeled edits in the revision histories of different Wikipedia articles.Our results suggest that high-quality articles show a higher degree of homogeneity with respect to their collaboration patterns as compared to random articles.