Memes possess a humorous intent, yet they can also be used for malicious purposes. Analysing meme data has the potential to enhance content monitoring, identify emerging topics, and support content moderation in online platforms. Memes also represent an interesting use case for multimodal machine learning, as they combine text and image data. In this study, we explored the linguistic characteristics and analysed the convergent themes of five meme classes through common word extraction. Moreover, we compared the effectiveness of various machine learning models, i.e., unimodal (text or image) and multimodal (early fusion, late fusion) in binary and multiclass meme classification tasks. Our results on a large meme dataset showed that memes heavily adhered to current affairs, demonstrated by the high frequency of topical words across meme classes. Regarding model accuracy, early fusion achieved superior accuracy over late fusion in meme classification. Binary models outperformed multi-class classification methods. However, fusion models did not consistently surpass the accuracy of independent text or image-based models.

Social media content is increasingly subject to hate speech towards specific demographic groups. In this context, machine learning approaches appear relevant to detect malicious contents and support moderators in mitigation initiatives. However, many of the existing approaches are exclusively focused on the analysis of textual contents. On the other hand, studies that address multiple data modalities often rely on a single feature representation and a fully supervised learning setting. In this paper, we tackle multimodal hate speech detection resorting to different learning settings (one-class learning and binary classification). We also investigate the effectiveness of multiple deep learning model backbones and language models to extract embedding feature representations for text and image modalities. Our experiments with a real-world hate speech dataset show that there is a significant performance gap between one-class learning and binary classification, and that the choice of embedding representations for image and text modalities can impact the detection performance for different predictive models.

Social media platforms are pivotal in information dissemination but also contribute to the rapid spread of misinformation, especially during high-impact events like natural disasters, terrorist attacks, and political unrest. While recent advances in multi-modal learning have enhanced misinformation detection by integrating features from various modalities (e.g., text, images), certain areas remain under-explored, particularly the use of event-based multi-modal data. This paper introduces a novel approach to misinformation detection on social media using an event-based multi-modal learning framework. Our method extends beyond traditional techniques by employing latent variable modeling to capture non-linear associations in event-based multi-modal data and to generate joint features between events for classification. This approach enhances misinformation detection and enables the contextual understanding of terms across different events. We provide a detailed analysis of our dataset preparation, methodology, and results, demonstrating the effectiveness of our framework on a widely-used dataset of tweets from high-impact events. The paper concludes with insights into potential enhancements and future directions in multi-modal misinformation detection.

Accurately identifying patients with Right Ventricular Dysfunction (RVD) is critical for timely diagnosis and treatment, yet it remains a significant challenge in clinical practice due to the complexity and variability of echocardiography text reports. In this study, we address this crucial problem by evaluating the effectiveness of recurrent neural networks (RNNs) in classifying patients with and without RVD from echocardiography text reports. We demonstrate that our model exhibits strong performance, with high recall and precision in identifying RVD, underscoring its potential utility in clinical settings. Furthermore, we enhance the interpretability of the framework by leveraging the Local Interpretable Model-Agnostic Explanations (LIME) framework. This approach allows us to extract and validate meaningful semantic information related to RVD, providing deeper insights into the model’s local predictive capabilities and improving transparency in decision-making.

Online hate speech proliferation has created a difficult problem for social media platforms. A particular challenge relates to the use of coded language by groups interested in both creating a sense of belonging for its users and evading detection. Coded language evolves quickly and its use varies over time. This paper proposes a methodology for detecting emerging coded hate-laden terminology. The methodology is tested in the context of online antisemitic discourse. The approach considers posts scraped from social media platforms, often used by extremist users. The posts are scraped using seed expressions related to previously known discourse of hatred towards Jews. The method begins by identifying the expressions most representative of each post and calculating their frequency in the whole corpus. It filters out grammatically incoherent expressions as well as previously encountered ones so as to focus on emergent well-formed terminology. This is followed by an assessment of semantic similarity to known antisemitic terminology using a fine-tuned large language model, and subsequent filtering out of the expressions that are too distant from known expressions of hatred. Emergent antisemitic expressions containing terms clearly relating to Jewish topics are then removed to return only coded expressions of hatred.

Social media content can present a number of threats, including misinformation and hate speech towards specific demographic groups. One challenge is to effectively discriminate between benign and malicious posts, given the massive amount of available content. In this context, predictive models for malicious content detection can be extremely valuable, leading to the automatic removal of posts and user accounts or content being flagged for subsequent moderation. However, some of the existing detection models are limited to the analysis of a single data modality. At the same time, most multi-modal approaches operate in a fully supervised learning setting that assumes the availability of labeled data for both benign and malicious content. In this paper, we fill this gap by proposing a multimodal one-class learning approach for malicious online content detection. Our approach leverages feature extraction, dimensionality reduction, and one-class learning models to analyze text and image data in online posts simultaneously. Models learn their decision function in the challenging scenario where only benign online content is used as training data, overcoming the limitations of a fully supervised setting. Our experiments with two real-world datasets containing misinformation and hate speech posts reveal the effectiveness of different combinations of one-class learning models and dimensionality reduction techniques.