Multimodal sentiment analysis refers to predicting the emotions conveyed by the joint input of multiple modalities. The primary challenge of this task is to effectively integrate emotional features across different modalities to improve the precision of emotion recognition. For visual–textual sentiment analysis, existing methods typically employ two encoders to extract text and image features separately and enhance analysis performance through cross-modal fusion. However, previous methods often overlook the fact that due to the significant noise in social media data, the correlation between text and images may be weak, and forced alignment could lead to semantic distortion. To address the problem, we propose a context-based alignment method that employs a global modality-aware attention mechanism. This mechanism allows each image or text to reference relevant information from other samples in the entire training set during the alignment process. In addition, to further enhance global alignment, we integrate self-supervised contrastive learning to refine feature representations. By leveraging global semantic information, our approach enhances the robustness of local alignment, effectively reducing alignment bias caused by data noise and improving multimodal fusion performance. Experimental results demonstrate that the proposed approach better understands and integrates information from different modalities in multimodal sentiment analysis, leading to improved accuracy and robustness in multimodal sentiment analysis.

Image fusion aims to integrate complementary information across modalities to generate high-quality fused images, thereby enhancing the performance of high-level vision tasks. While global spatial modeling mechanisms show promising results, constructing long-range feature dependencies in the spatial domain incurs substantial computational costs. Additionally, the absence of ground-truth exacerbates the difficulty of capturing complementary features effectively. To tackle these challenges, we propose a Residual Prior-driven Frequency-aware Network, termed as RPFNet. Specifically, RPFNet employs a dual-branch feature extraction framework: the Residual Prior Module (RPM) extracts modality-specific difference information from residual maps, thereby providing complementary priors for fusion; the Frequency Domain Fusion Module (FDFM) achieves efficient global feature modeling and integration through frequency-domain convolution. Additionally, the Cross Promotion Module (CPM) enhances the synergistic perception of local details and global structures through bidirectional feature interaction. During training, we incorporate an auxiliary decoder and saliency structure loss to strengthen the model's sensitivity to modality-specific differences. Furthermore, a combination of adaptive weight-based frequency contrastive loss and SSIM loss effectively constrains the solution space, facilitating the joint capture of local details and global features while ensuring the retention of complementary information. Extensive experiments validate the fusion performance of RPFNet, which effectively integrates discriminative features, enhances texture details and salient objects, and can effectively facilitate the deployment of the high-level vision task. The source code can be available at https://github.com/wang-x-1997/RPFNet.

Dimensional sentiment analysis (DSA) aims to recognize continuous real-valued annotations in multidimensional spaces such as valence-arousal space. It can serve as a sentiment analysis that is more fine-grained than traditional polarity classification. Existing methods primarily focused on supervised learning, which requires a large number of training samples. To address this issue, recent studies have suggested the use of a generative model for semi-supervised DSA. However, these methods lack the exploration of relationships between samples, which may limit their effectiveness. In this study, we proposed a sample relationships (SRs) model for semi-supervised DSA. Compared with the conventional methods, the SR module can integrate the interactive information from different samples, effectively capturing their interdependencies. The results of experiments conducted on three datasets indicated that the performance of the proposed method was better than those of the existing models and that the performance was effective and competitive even with insufficient data.