Multimodal sentiment analysis (MSA) is attracting increasing attention from researchers. Existing studies on MSA typically rely on surface-level feature extraction and fusion that can be directly obtained from multimodal data, which may often ignore the underlying semantic connection between images and texts. Recent progress in large multimodal models (LMMs) has demonstrated their impressive reasoning abilities, which can be leveraged to improve traditional MSA approaches by providing a deeper understanding of the sematic connection of the modalities. Toward this issue, in this paper, we propose a novel framework called MPT that combines traditional MSA approaches with Multi-Perspective Thinking from LMMs to improve prediction outcomes. Specifically, MPT instructs the traditional multimodal deep learning models to understand multiple-perspective rationales for different sentiment polarities, augmenting its knowledge base and enhancing its ability to make more accurate predictions. Extensive experiments on four refined datasets show that MPT can not only deliver better performance compared with existing methods, but also demonstrate good cross-modal understanding ability for recognizing user sentiment. The codes and datasets can be accessed here: https://github.com/RMJHQwQ/MPT.

Multimodal social network user sentiment analysis aims to determine users' emotional polarity (positive or negative) by mining the associations between multiple data types such as images and texts. Existing public datasets are mainly constructed from English social media platforms, while Chinese social media datasets for multimodal user sentiment analysis are extremely scarce. In terms of the posts published by Chinese social media users, it is not rare that the emotional polarity delivered by the image and the textual content is inconsistent. Given such emotional inconsistency between images and texts, how to effectively identify users' true emotion polarity is still challenging. Toward the above issues, in this paper, we firstly construct a Chinese social media dataset CH-Mits for multimodal user sentiment analysis. In order to evaluate the usability of the dataset, we conceive and implement a novel model called PEMNet, and compare it with state-of-the-art models based on the CH-Mits dataset. In the end, we analyze the performance of PEMNet on selected samples with emotional inconsistency between images and texts. The constructed dataset and codes for PEMNet are available at https://github.com/Marblrdumdore/CH-Mits.

Online social networks (OSNs) have become a commodity in our daily life. As an important concept in sociology and viral marketing, the study of social influence has received a lot of attentions in academia. Most of the existing proposals work well on dominant OSNs, such as Twitter, since these sites are mature and many users have generated a large amount of data for the calculation of social influence. Unfortunately, cold-start users on emerging OSNs generate much less activity data, which makes it challenging to identify potential influential users among them. In this work, we propose a practical solution to predict whether a cold-start user will become an influential user on an emerging OSN, by opportunistically leveraging the user’s information on dominant OSNs. A supervised machine learning-based approach is adopted, transferring the knowledge of both the descriptive information and dynamic activities on dominant OSNs. Descriptive features are extracted from the public data on a user’s homepage. In particular, to extract useful information from the fine-grained dynamic activities that cannot be represented by the statistical indices, we use deep learning technologies to deal with the sequential activity data. Using the real data of millions of users collected from Twitter (a dominant OSN) and Medium (an emerging OSN), we evaluate the performance of our proposed framework to predict prospective influential users. Our system achieves a high prediction performance based on different social influence definitions.

The proliferation of mobile devices especially smart phones brings remarkable opportunities for both industry and academia. In particular, the massive data generated from users» usage logs provide the possibilities for stakeholders to know better about consumer behaviors with the aid of data mining. In this paper, we examine the consumer behaviors across multiple platforms based on a large-scale mobile Internet dataset from a major telecom operator, which covers 9.8 million users from two regions among which 1.4 million users have visited e-commerce platforms within one week of our study. We make several interesting observations and examine users» cultural differences from different regions. Our analysis shows among the multiple e-commerce platforms available, most mobile users are loyal to their favorable sites; people (60%) tend to make quick decisions to buy something online, which usually takes less than half an hour. Furthermore, we find that people in residential areas are much easier to perform purchases than in business districts and purchases take place during non-work time. Meanwhile, people with medium socioeconomic status like browsing and purchasing on e-commerce platforms, while people with high and low socioeconomic status are much easier to conduct purchases online. We also show the predictability of cross-platform shopping behaviors with extensive experiments on the basis of our observed data. Our discoveries could be a good guide for e-commerce future strategy making.

The social triad—a group of three people—is one of the simplest and most fundamental social groups. Extensive network and social theories have been developed to understand its structure, such as triadic closure and social balance. Over the course of a triadic closure—the transition from two ties to three among three users, the strength dynamics of its social ties, however, are much less well understood. Using two dynamic networks from social media and mobile communication, we examine how the formation of the third tie in a triad affects the strength of the existing two ties. Surprisingly, we find that in about 80% social triads, the strength of the first two ties is weakened although averagely the tie strength in the two networks maintains an increasing or stable trend. We discover that (1) the decrease in tie strength among three males is more sharply than that among females, and (2) the tie strength between celebrities is more likely to be weakened as the closure of a triad than those between ordinary people. Furthermore, we formalize a triadic tie strength dynamics prediction problem to infer whether social ties of a triad will become weakened after its closure. We propose a TRIST method—a kernel density estimation (KDE)-based graphical model—to solve the problem by incorporating user demographics, temporal effects, and structural information. Extensive experiments demonstrate that TRIST offers a greater than 82% potential predictability for inferring triadic tie strength dynamics in both networks. The leveraging of the KDE and structural correlations enables TRIST to outperform baselines by up to 30% in terms of F1-score.

Human mobility data has been ubiquitously collected through cellular networks and mobile applications, and publicly released for academic research and commercial purposes for the last decade. Since releasing individual's mobility records usually gives rise to privacy issues, datasets owners tend to only publish aggregated mobility data, such as the number of users covered by a cellular tower at a specific timestamp, which is believed to be sufficient for preserving users' privacy. However, in this paper, we argue and prove that even publishing aggregated mobility data could lead to privacy breach in individuals' trajectories. We develop an attack system that is able to exploit the uniqueness and regularity of human mobility to recover individual's trajectories from the aggregated mobility data without any prior knowledge. By conducting experiments on two real-world datasets collected from both mobile application and cellular network, we reveal that the attack system is able to recover users' trajectories with accuracy about 73%~91% at the scale of tens of thousands to hundreds of thousands users, which indicates severe privacy leakage in such datasets. Through the investigation on aggregated mobility data, our work recognizes a novel privacy problem in publishing statistic data, which appeals for immediate attentions from both academy and industry.

We study the problem of group formation in online social networks. In particular, we focus on one of the most important human groups-the triad-and try to understand how closed triads are formed in dynamic networks, by employing data from a large microblogging network as the basis of our study. We formally define the problem of triadic closure prediction and conduct a systematic investigation. The study reveals how user demographics, network characteristics, and social properties influence the formation of triadic closure. We also present a probabilistic graphical model to predict whether three persons will form a closed triad in a dynamic network. Different kernel functions are incorporated into the proposed graphical model to quantify the similarity between triads. Our experimental results with the large microblogging dataset demonstrate the effectiveness (+10 percent over alternative methods in terms of F1-Score) of the proposed model for the prediction of triadic closure formation.