Social media users are more likely to be exposed to similar views and tend to avoid contrasting views, especially when they are part of a community of social media users. In this study, we investigate the presence of user communities and leverage them as a tool to detect misinformation on social media, specifically on X (formerly known as Twitter). We propose a misinformation detection framework, namely Similarity-based Misinformation Detection (SiMiD) that employs microblogs and utilizes user-follower interactions within a social network. Our approach extracts important textual features of social media posts using a transformer-based language model. We use contrastive learning and pseudo-labeling to fine-tune the language model. Then, we measure the similarity for each social media post based on its relevance to each user in the communities. Finally, we train a machine learning model to identify the truthfulness of social media posts using these similarity scores. We evaluate our approach on three social media datasets, compare our method with twelve state-of-the-art approaches, and answer five research questions. The experimental results, supported by statistical tests, show that contrastive learning and user communities can enhance the detection of misinformation on social media. Our model can identify misinformation content by achieving a consistently high weighted F1 score of over 90% across all datasets, even employing only a small number of users in communities. We make our implementations publicly available and provide all details that are necessary for the reproducibility of experiments. 1

The detection of hate speech is a subject extensively explored by researchers, and machine learning algorithms play a crucial role in this domain. The existing resources mostly focus on text sequence classification for the task of hate speech detection. However, the target of hateful content is another dimension that has not been studied in details due to the lack of data resources. In this study, we address this gap by introducing a novel tweet dataset for the task of joint learning of hate speech detection and target detection, called JL-Hate, for the tasks of sequential text classification and token classification, respectively. The JL-Hate dataset consists of 1,530 tweets divided equally in English and Turkish languages. Leveraging this dataset, we conduct a series of benchmark experiments. We utilize a joint learning model to concurrently perform sequence and token classification tasks on our data. Our experimental results demonstrate consistent performance with the prevalent studies, both in sequence and token classification tasks.

Misogyny is often expressed through figurative language. Some neutral words can assume a negative connotation when functioning as pejorative epithets. Disambiguating the meaning of such terms might help the detection of misogyny. In order to address such task, we present PejorativITy, a novel corpus of 1,200 manually annotated Italian tweets for pejorative language at the word level and misogyny at the sentence level. We evaluate the impact of injecting information about disambiguated words into a model targeting misogyny detection. In particular, we explore two different approaches for injection: concatenation of pejorative information and substitution of ambiguous words with univocal terms. Our experimental results, both on our corpus and on two popular benchmarks on Italian tweets, show that both approaches lead to a major classification improvement, indicating that word sense disambiguation is a promising preliminary step for misogyny detection. Furthermore, we investigate LLMs’ understanding of pejorative epithets by means of contextual word embeddings analysis and prompting.

The rapid dissemination of misinformation through online social networks poses a pressing issue with harmful consequences jeopardizing human health, public safety, democracy, and the economy; therefore, urgent action is required to address this problem. In this study, we construct a new human-annotated dataset, called MiDe22, having 5,284 English and 5,064 Turkish tweets with their misinformation labels for several recent events between 2020 and 2022, including the Russia-Ukraine war, COVID-19 pandemic, and Refugees. The dataset includes user engagements with the tweets in terms of likes, replies, retweets, and quotes. We also provide a detailed data analysis with descriptive statistics and the experimental results of a benchmark evaluation for misinformation detection.

Tokenization is an important text preprocessing step to prepare input tokens for deep language models. WordPiece and BPE are de facto methods employed by important models, such as BERT and GPT. However, the impact of tokenization can be different for morphologically rich languages, such as Turkic languages, in which many words can be generated by adding prefixes and suffixes. We compare five tokenizers at different granularity levels, that is, their outputs vary from the smallest pieces of characters to the surface form of words, including a Morphological-level tokenizer. We train these tokenizers and pretrain medium-sized language models using the RoBERTa pretraining procedure on the Turkish split of the OSCAR corpus. We then fine-tune our models on six downstream tasks. Our experiments, supported by statistical tests, reveal that the morphological-level tokenizer delivers a challenging performance with de facto tokenizers. Furthermore, we find that increasing the vocabulary size improves the performance of Morphological- and Word-level tokenizers more than that of de facto tokenizers. The ratio of the number of vocabulary parameters to the total number of model parameters can be empirically chosen as 20% for de facto tokenizers and 40% for other tokenizers to obtain a reasonable trade-off between model size and performance.

The performance of hate speech detection models relies on the datasets on which the models are trained. Existing datasets are mostly prepared with a limited number of instances or hate domains that define hate topics. This hinders large-scale analysis and transfer learning with respect to hate domains. In this study, we construct large-scale tweet datasets for hate speech detection in English and a low-resource language, Turkish, consisting of human-labeled 100k tweets per each. Our datasets are designed to have equal number of tweets distributed over five domains. The experimental results supported by statistical tests show that Transformer-based language models outperform conventional bag-of-words and neural models by at least 5% in English and 10% in Turkish for large-scale hate speech detection. The performance is also scalable to different training sizes, such that 98% of performance in English, and 97% in Turkish, are recovered when 20% of training instances are used. We further examine the generalization ability of cross-domain transfer among hate domains. We show that 96% of the performance of a target domain in average is recovered by other domains for English, and 92% for Turkish. Gender and religion are more successful to generalize to other domains, while sports fail most.

Supervised training with cross-entropy loss implicitly forces models to produce probability distributions that follow a discrete delta distribution.Model predictions in test time are expected to be similar to delta distributions if the classifier determines the class of an input correctly.However, the shape of the predicted probability distribution can become similar to the uniform distribution when the model cannot infer properly.We exploit this observation for detecting out-of-scope (OOS) utterances in conversational systems.Specifically, we propose a zero-shot post-processing step, called Distance-to-Uniform (D2U), exploiting not only the classification confidence score, but the shape of the entire output distribution.We later combine it with a learning procedure that uses D2U for loss calculation in the supervised setup.We conduct experiments using six publicly available datasets.Experimental results show that the performance of OOS detection is improved with our post-processing when there is no OOS training data, as well as with D2U learning procedure when OOS training data is available.

User intent is not restricted in human-to-machine conversations, and sometimes overshoots the scope of a designed system. Many tasks for understanding conversations require the elimination of such out-of-scope queries. We propose an out-of-scope intent detection method, called KLOOS, based on a novel feature extraction mechanism that incorporates the information accumulation of sequential word processing. Information is accumulated by KL divergence between the intent distributions of consecutive words. The performance of our approach is compared with the conventional classifiers and state-of-the-art language models fine-tuned for out-of-scope detection on three spoken query collections. The results show that KLOOS statistically significantly improves out-of-scope sensitivity in all cases, while the overall performance is not deteriorated in most cases.

As terrorists are losing against counter-terrorism efforts, they turn to manipulating cryptocurrency prices through online social communities to gain illicit profit to fund their operations. Modeling temporal online social networks (OSNs) of these communities can possibly help to provide useful intelligence about these malicious activities. However, existing techniques do not learn sufficiently from diverse features to enable prediction and simulation of online social behavior. Research on simulating temporal OSN behavior is not widely available. This research developed and validated a deep learning approach, named Temporal Network Model (TNM), to modeling the complex features and dynamic behavior exhibited in the temporal OSNs of online communities. Using extensive features extracted from fine-grained data, TNM consists of weighted time series models, user and link prediction models, and temporal dependency model that predict respectively the macroscopic behavior, microscopic user participation and events, and time stamps of the events. Evaluation was done in comparison with a benchmark approach to examine TNM's performance on predicting and simulating behavior of 42,627 users in 440,906 events on the Reddit cryptocurrency community during July-August of 2017. Results show that TNM outperformed the benchmark in 5 out of 8 simulation metrics. TNM achieved consistently better performance in user activity prediction, and performed generally better in structural (network-level) prediction. The research provides new findings on simulating temporal OSNs and new predictive analytics for understanding online social behavior.

The increasing popularity and diversity of online social networks (OSNs) have attracted more and more people to participate in multiple OSNs. Learning users' behavior and information diffusion across platforms is critical for cyber threat detection, but it is still a challenge due to the surge of users participating in multiple social platforms. Existing research on profile matching requires user identity information to be available, which may not be realistic. Little prior research payed attention to mapping behavioral patterns across platforms. We designed and implemented an efficient two-level probabilistic approach called CrossSimON to mapping user-group behavior across platforms. CrossSimON considers the activity level and network position at both individual user level and group level to correlate activities across social platforms. To evaluate the effectiveness of CrossSimON in modeling social activity across platforms, we conducted experiments on three online social platforms: GitHub, Reddit and Twitter. Our experimental results show that CrossSimON outperformed the Benchmark in 3 out of 5 simulation metrics. CrossSimON achieved better performance in user activity prediction. The research provides new strategy for cross-platform online social network simulation, and new findings on simulating OSNs and predictive analytics for understanding online social network behavior.

Simulation of human behaviour being an intrinsically difficult problem, no single algorithm or model can accurately simulate online social networks. One can obtain an optimal and reliable simulation only after combining several models focusing on diverse social aspects. Since all independent models focus on different social aspects, it is inherently difficult to combine and optimize their performance. Moreover blackbox nature of these predictive algorithm makes it difficult to integrate human-guided intelligence. Here we are presenting SimON-Feedback, an iterative ensemble algorithm to combine the prediction of several independent models into a significantly improved simulation of an online social network. To this end, we explore user posting and commenting behavior on Reddit, a large social networking platform comprised of many communities called as subreddits.

A story chain is a set of related news articles that reveal how different events are connected. This study presents a framework for discovering story chains, given an input document, in a text collection. The framework has 3 complementary parts that i) scan the collection, ii) measure the similarity between chain‐member candidates and the chain, and iii) measure similarity among news articles. For scanning, we apply a novel text‐mining method that uses a zigzagged search that reinvestigates past documents based on the updated chain. We also utilize social networks of news actors to reveal connections among news articles. We conduct 2 user studies in terms of 4 effectiveness measures—relevance, coverage, coherence, and ability to disclose relations. The first user study compares several versions of the framework, by varying parameters, to set a guideline for use. The second compares the framework with 3 baselines. The results show that our method provides statistically significant improvement in effectiveness in 61% of pairwise comparisons, with medium or large effect size; in the remainder, none of the baselines significantly outperforms our method.