Web claims, seen as assertions shared on the web and eligible for fact-checking, are at the heart of online discourse. They have been studied extensively on a variety of downstream tasks such as fact-checking, claim retrieval, bias detection, argument mining, or viewpoint discovery. On the other hand, claims originating from scientific publications have also been the subject of several downstream NLP tasks. However, research carried out so far has yet to focus on scientific web claims, which are scientific claims made on the web (e.g., on social media and news articles). The process of detecting and fact-checking a claim from the web can be very different depending on whether the claim is scientific or not, thus making it crucial for the developed datasets, methods, and models to make a distinction between the two. With this work, we aim at understanding what makes this distinction necessary, by understanding the linguistic differences between scientific and non-scientific claims on the web, and the impact those differences have on existing downstream tasks. To do so, we manually annotate 1,524 web claims from established benchmarks for fact-checking-related tasks, and we run statistical tests to analyze and compare the linguistic features of each group. We find that scientific claims on the web use more analytical speech, but also use more sentiment-related speech, more expressions of physical motion, and have distinct parts of speech (PoS) and punctuation styles. We also conduct experiments showing that BERT-based language models perform worse on scientific web claims by up to 17 F1 points for several downstream tasks. To understand why, we develop a novel methodology to map predictive tokens of language models to explainable linguistic features and find that language models fail to detect a specific subset of predictive features of scientific web claims. We conclude by stating that language models aimed at studying scientific web claims ought to be trained on scientific web discourse, as opposed to being trained only on generic web discourse or only on scientific text from scientific publications.

The massive amount of misinformation spreading on the internet on a daily basis has enormous negative impacts on societies. Therefore, we need systems to help fact-checkers to combat misinformation and to raise public awareness of this important problem. In this article, we propose a hybrid model which combines bidirectional encoder representations from transformer (BERT) model with various features to prioritize claims based on their check-worthiness. Features we use include domain-specific controversial topics (CT), word embeddings (WE), part-of-speech (POS) tags, and others. In addition, we explore various ways of increasing labeled data size to effectively train the models, such as increasing positive (IncPos) samples, active learning (AL), and utilizing labeled data in other languages. In our extensive experiments, we show that our model outperforms all state-of-the-art models in test collections of Conference and Labs of Evaluation Forum (CLEF) CheckThat! Lab (CTL) 2018 and 2019. In addition, when positive samples are increased in the training set, our model achieves the best mean average precision (MAP) score reported so far for the test collection of CTL 2020. Furthermore, we show that cross-lingual training is effective for prioritizing Arabic and Turkish claims, but not for English.

Massive misinformation spread over Internet has many negative impacts on our lives.While spreading a claim is easy, investigating its veracity is hard and time consuming, Therefore, we urgently need systems to help human factcheckers.However, available data resources to develop effective systems are limited and the vast majority of them is for English.In this work, we introduce TrClaim-19, which is the very first labeled dataset for Turkish checkworthy claims.TrClaim-19 consists of labeled 2287 Turkish tweets with annotator rationales, enabling us to better understand the characteristics of check-worthy claims.The rationales we collected suggest that claims' topics and their possible negative impacts are the main factors affecting their check-worthiness.