Abstract The primary challenge of the multimodal sentiment analysis (MSA) task is the modal fusion, and the lack of modalities may exist in the fusion process, which leads to poor prediction results. Most of the previous research on multimodal fusion is single‐stage fusion, disregarding how various modality subsets interact, as well as rarely considering relative position relationship of modality sequences, causing the fragmentation of context info. Considering the aforementioned issues, this study introduces an MSA method based on the multi‐stage graph fusion network (MSGFN) under random missing modality conditions to improve the robustness of the model to MSA under the random missing modality conditions. Firstly, for each modality, its inter‐modal and intra‐modal multi‐head attention are used to learn robust representation of the modality sequence. Meanwhile, the relative position encoding (RPE) is introduced into mechanism of attention that enables model to perceive and learn the relative position before and after the modality sequence when calculating attention, thereby better understanding the contextual info of the sequence. After that, the transformer encoder receives the learned modality features and uses the pre‐trained model to supervise the reconstruction of the missing modality information. Finally, the feature representations of different modalities have effectively fused using multi‐stage graph fusion network, and the output is used for the ultimate sentiment classification. Wide experiments are conducted on two publicly available datasets, CMU‐MOSI and IEMOCAP, and the findings indicate that the proposed method can better handle the challenges caused by modality fusion and modality missing compared with several baseline methods.

In recent years, public security incidents caused by deepfake technology have occurred frequently around the world, which makes an efficient and accurate deepfake detection model crucial. The existing advanced methods use the manipulation features in the image to realize the binary classification of real and fake images by training complex neural network models. However, these models rely on a single manipulation feature, and the detection accuracy of these methods will be greatly reduced when the forgery technology or image quality of the training dataset and the validation dataset are different. Inspired by the existing work, we propose a two-stream collaborative learning framework that combines spatial texture differences and frequency information. The average difference convolution (ADC) is designed to extract the spatial texture difference information of the image, and the gray image frequency-aware decomposition (GFAD) is used to extract the artifact information of the image in the frequency domain. At the same time, the ViT idea is combined with cross-attention mechanism for feature fusion to comprehensively mine forged features in forged images. Experimental results show that the proposed model has good detection effects on three benchmark datasets. In terms of cross-dataset evaluation, the AUC on Celeb-DF dataset reaches 82.86%, which is better than the existing advanced methods.

Abstract Image inpainting aims to repair the damaged region according to the known content in the damaged image. Recently, image inpainting methods have poor effects on high‐resolution damaged images, and the research on the inpainting of large‐area damaged images is limited. Therefore, this paper proposes an image inpainting method based on Generative Adversarial Networks (GAN) inversion and autoencoder. This work consists of two phases: first, the authors design an autoencoder‐based GAN, which learns the mapping from noise to low‐dimensional feature maps by training a generator, and then converts the generated feature maps into high‐resolution images. Thus, the difficulty of learning the mapping relationship is reduced. Second, the authors adopt the learning‐based GAN inversion to infer the closest latent code. The trained GAN is then used to reconstruct the complete image. Finally, the authors compare their method with other classical methods on the CelebAMask‐HQ, Flickr‐Faces‐HQ, and ImageNet datasets. According to the quantitative comparison, when the mask range is large, in other words, when the image has a large area of damage, the authors’ method is superior to the comparison methods. According to the qualitative comparison, the structure of the high‐resolution image inpainted by the authors’ method is more reasonable and the texture details are more realistic.

Digital disinformation, such as those occurring on online social networks (OSNs), can influence public opinion, create mistrust and division, and impact decision- and policy-making. In this study, we propose a disinformation diffusion trend analysis and identification method, which uses social situation analytics and a multilevel attention network. First, we present a division and feature representation approach of social user circle based on the content sequence (internal driving factor) and social contextual information (external driving factor) of users associated with disinformation. Second, disinformation content feature, crowd response feature, and time-series feature are represented using embedding layer and bidirectional long short-term memory neural networks (Bi-LSTMs). We also present an attention mechanism model based on multifeature fusion, which can dynamically adjust the weight of each feature. On this foundation, the fused features are fed into the multilayer perceptron to identify the propagation quantity trend. According to the experimental results of real-world OSNs and social situation metadata, we conclude that while disinformation occurs across OSN platforms, the disinformation is more likely to spread widely in the original OSN platform. We also identify four typical disinformation propagation trends based on propagation patterns and propagation peak times. Findings from our experiments demonstrate that our proposed approach accurately identifies and predicts the diffusion trend of disinformation, which can then be used to inform mitigation strategy.

The dissemination of digital disinformation in online social networks (OSNs) has been the subject of extensive research, although many challenges remain, including the analysis and control of disinformation dissemination across different platforms (i.e., cross-platform). In this article, we investigate and analyze the spreading patterns and regularities of disinformation both within a single platform and across platforms. To explore the complex relationship between user propagation desire and behaviour within the same group, a user propagation desire inference model based on propagation characteristics (behaviour characteristics and time characteristics) and a bidirectional backpropagation (B-BP) deep neural network are constructed. Then, to avoid overfitting due to the interaction of users’ propagation behaviour and the correlation among propagation characteristics, a novel adaptive weighted particle swarm optimization evolutionary algorithm is utilized to further optimize the B-BP deep neural network. We design and conduct a series of evaluation experiments on the current global hot topics including but not limited to novel coronavirus-19 pandemic (COVID-19), food safety, medical and health, and environmental protection. By using a real-world social platform and its social situation metadata analysis, the experimental results show that the proposed method not only accurately predicts the level of user propagation desire under multiple behaviour interactions but also facilitates social platform managers in handling disinformation disseminators. Our findings reveal that the intensity of social users’ desires to spread disinformation is related to the topics and groups that users are interested in, while the propagation motivation of social users is not strong under topics that users are not interested in. Our studies also demonstrate that social users with propagation desires tend to utilize their familiar social platforms and local circles for communication, and the behaviour and desire to spread disinformation to the cross-platform are not strong. We posit that these findings can help inform online and, fine-grained governance and mitigation strategies other than “one size fits all” approaches (e.g., “account prohibition and deletion”), and hopefully minimize disinformation dissemination.

A feature selection method based on mutual information and support vector machine (SVM) is proposed in order to eliminate redundant feature and improve classification accuracy. First, local correlation between features and overall correlation is calculated by mutual information. The correlation reflects the information inclusion relationship between features, so the features are evaluated and redundant features are eliminated with analyzing the correlation. Subsequently, the concept of mean impact value (MIV) is defined and the influence degree of input variables on output variables for SVM network based on MIV is calculated. The importance weights of the features described with MIV are sorted by descending order. Finally, the SVM classifier is used to implement feature selection according to the classification accuracy of feature combination which takes MIV order of feature as a reference. The simulation experiments are carried out with three standard data sets of UCI, and the results show that this method can not only effectively reduce the feature dimension and high classification accuracy, but also ensure good robustness.