Scientific Natural Language Inference (NLI) is the task of predicting the semantic relation between a pair of sentences extracted from research articles.The automatic annotation method based on distant supervision for the training set of SCINLI (Sadat and Caragea, 2022b), the first and most popular dataset for this task, results in label noise which inevitably degenerates the performance of classifiers.In this paper, we propose a novel co-training method that assigns weights based on the training dynamics of the classifiers to the distantly supervised labels, reflective of the manner they are used in the subsequent training epochs.That is, unlike the existing semi-supervised learning (SSL) approaches, we consider the historical behavior of the classifiers to evaluate the quality of the automatically annotated labels.Furthermore, by assigning importance weights instead of filtering out examples based on an arbitrary threshold on the predicted confidence, we maximize the usage of automatically labeled data, while ensuring that the noisy labels have a minimal impact on model training.The proposed method obtains an improvement of 1.5% in Macro F1 over the distant supervision baseline, and substantial improvements over several other strong SSL baselines.We make our code and data available on Github. 1

Mobashir Sadat, Cornelia Caragea. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2024.

Existing Natural Language Inference (NLI) datasets, while being instrumental in the advancement of Natural Language Understanding (NLU) research, are not related to scientific text.In this paper, we introduce SCINLI, a large dataset for NLI that captures the formality in scientific text and contains 107, 412 sentence pairs extracted from scholarly papers on NLP and computational linguistics.Given that the text used in scientific literature differs vastly from the text used in everyday language both in terms of vocabulary and sentence structure, our dataset is well suited to serve as a benchmark for the evaluation of scientific NLU models.Our experiments show that SCINLI is harder to classify than the existing NLI datasets.Our best performing model with XLNet achieves a Macro F1 score of only 78.18% and an accuracy of 78.23% showing that there is substantial room for improvement.

Automatic topic classification has been studied extensively to assist managing and indexing scientific documents in a digital collection.With the large number of topics being available in recent years, it has become necessary to arrange them in a hierarchy.Therefore, the automatic classification systems need to be able to classify the documents hierarchically.In addition, each paper is often assigned to more than one relevant topic.For example, a paper can be assigned to several topics in a hierarchy tree.In this paper, we introduce a new dataset for hierarchical multi-label text classification (HMLTC) of scientific papers called SciHTC, which contains 186, 160 papers and 1, 233 categories from the ACM CCS tree.We establish strong baselines for HMLTC and propose a multi-task learning approach for topic classification with keyword labeling as an auxiliary task.Our best model achieves a Macro-F1 score of 34.57% which shows that this dataset provides significant research opportunities on hierarchical scientific topic classification.We make our dataset and code available on Github. 1