Multi-modal entity alignment aims to identify equivalent entities across diverse knowledge graphs by leveraging multiple modalities of entity information. This process is crucial for the fusion of multi-modal knowledge graphs. While current research primarily investigates how to utilize side information from entity visuals, relations, and attributes, it often overlooks the significant role of entity-type information. Furthermore, multi-modal data embedding encounters noise that negatively impacts the performance of the entity alignment task. To address these gaps, this paper introduces MTCEA, a multi-modal entity alignment method guided by entity-type information. The proposed method captures the constraints associated with entities based on the entity-type information obtained from knowledge graph ontology; then, it utilizes two embedding strategies for type constraints to enhance the model’s performance in knowledge representation. This allows effective modal fusion that integrates more fine-grained semantic constraints related to types, which improves the alignment accuracy across various cross-lingual knowledge graphs. MTCEA is validated on three subsets of DBP15K. Experimental results demonstrate that our model achieves good results overall on the Hits@1, Hits@10, and MRR metrics. In an experimental setting without using entity name, MTCEA outperforms state-of-the-art baselines.

Emotion recognition in conversation (ERC) is essential for dialogue systems to identify the emotions expressed by speakers. Although previous studies have made significant progress, accurate recognition and interpretation of similar fine-grained emotion properly accounting for individual variability remains a challenge. One particular under-explored area is the role of individual beliefs and desires in modelling emotion. Inspired by the Belief-Desire Theory of Emotion, we propose a novel method for conversational emotion recognition that incorporates both belief and desire to accurately identify emotions. We extract emotion-eliciting events from utterances and construct graphs that represent beliefs and desires in conversations. By applying message passing between nodes, our graph effectively models the utterance context, speaker’s global state, and the interaction between emotional beliefs, desires, and utterances. We evaluate our model’s performance by conducting extensive experiments on four popular ERC datasets and comparing it with multiple state-of-the-art models. The experimental results demonstrate the superiority of our proposed model and validate the effectiveness of each module in the model.

COVID-19 has spread all over the world, accounting for countless death and enormous economic loss. Since the World Health Organization (WHO) declared COVID-19 as a pandemic, governments from different countries have made various policies to prevent the pandemic from becoming worse. However, civilian reactions to the pandemic vary when they face similar situations. This behavioral variation creates a challenge when it comes to policy-making. Such differences are generally implicit, hidden in ones’ social lives. As a result, it is challenging to analyze such differences when the governments make policies. In this work, we investigate social media posts on Twitter and Weibo in order to effectively explore the difference in reactions across various countries, with the aim to understand national differences. To this end, we employ natural language processing (NLP) methods and Linguistic Inquiry and Word Count (LIWC) tools to process six languages in different countries, including the USA, Germany, France, Italy, the U.K., and China. We provide a comprehensive analysis of public reaction differences from the emotional perspective. Our findings verify that the reactions vary noticeably among various countries for some policies. Therefore, sentiment analysis can significantly influence policy-making. Our work sheds light on the mechanism of detecting the reaction differences in various countries, which can be utilized to conduct effective communication and make appropriate policy decisions.

Memes have become the popular means of communication for Internet users worldwide. Understanding the Internet meme is one of the most tricky challenges in natural language processing (NLP) tasks due to its convenient non-standard writing and network vocabulary. Recently, many linguists suggested that memes contain rich metaphorical information. However, the existing researches ignore this key feature. Therefore, to incorporate informative metaphors into the meme analysis, we introduce a novel multimodal meme dataset called MET-Meme, which is rich in metaphorical features. It contains 10045 text-image pairs, with manual annotations of the metaphor occurrence, sentiment categories, intentions, and offensiveness degree. Moreover, we propose a range of strong baselines to demonstrate the importance of combining metaphorical features for meme sentiment analysis and semantic understanding tasks, respectively. MET-Meme, and its code are released publicly for research in \urlhttps://github.com/liaolianfoka/MET-Meme-A-Multi-modal-Meme-Dataset-Rich-in-Metaphors.

Mental health is an integral part of human health and well-being. Unhealthy mentality leads to serious consequences such as self-mutilation and suicide, especially for college students. While the literature focused on analysing the relationship between mental health and a single factor such as personality or behavior, accurate prediction is yet to be achieved due to the lack of cross-dimensional analysis and multi-dimensional joint prediction. To this end, this work proposes leveraging multiple factors from three crucial dimensions of mental health: behaviors, personality, and social networks. We recruited 490 college students, and collected their behavioral records from smart cards. In addition, we extracted their psychological traits from questionnaires, and social networks by conducting the survey on the nominating community members. We created a neural network-based model to integrate behavioral, psychological, and social network factors to predict mental health problems. The experimental results verify the efficacy of the proposed model, and demonstrate that the classification model of various factors effectively predicts the students’ mental issues.