Multimodal sarcasm detection aims to identify whether utterances express sarcastic intentions contrary to their literal meaning based on multimodal information. However, existing methods fail to explore the model’s "ability to understand" the semantics expressed by sentences in the image context from semantic diversity perspectives. In this paper, we propose a multi-view semantic awareness method, which concretizes semantics from multiple perspectives to improve the model’s ability to capture different semantic features. Specifically, two learnable prefixes are attached to the text representation respectively to construct semantic representations from both the literal meaning and sarcastic intention perspectives. Then, image-text information is further fused through cross-attention to guide the semantic representation of different perspectives in the image context. Finally, the semantics expressed by prefixes are strengthened through KL divergence, thereby encouraging the model to capture two distinctive semantic features. Experiments on benchmark datasets demonstrate the effectiveness of our method.

Recent advances in few-shot novel-view synthesis based on 3D Gaussian Splatting (3DGS) have shown remarkable progress. Existing methods usually rely on carefully designed geometric regularizers to reinforce geometric supervision; however, applying multiple regularizers consistently across scenes is hard to tune and often degrades robustness. Consequently, generating reliable geometry from extremely sparse viewpoints remains a key challenge. To overcome this limitation, we introduce SREGS, a framework tailored for few-shot reconstruction whose contributions focus on two aspects: explicitly consistent geometry and multi-scale depth-guided optimization. Specifically, to explicitly optimize reconstruction consistency, we initialize the point cloud with 2D Gaussians, thereby enhancing depth consistency for the same Gaussian observed from different views. Secondly, we employ region-adaptive rapid densificationn to fill under-covered regions with additional representations, while an opacity-aware noise term injects stochasticity into each Gaussian to boost exploration in under-observed areas. In addition, to strengthen geometric refinement of the radiance field, we impose multi-scale depth constraints based on a monocular depth prior, performing geometric refinement from global to local scales and ensuring highly accurate reconstruction. Extensive experiments on LLFF, MipNeRF360, and Blender show that SREGS achieves higher synthesis quality with lower computational cost and demonstrates robust performance. The code is available at:https://github.com/LeeXiaoTong1/SREGS.

Multimodal Sentiment Analysis (MSA) is the technology of intelligently recognizing and assessing human sentiments using various data forms such as text, image, and audio. Despite current mainstream methods have made significant progress, MSA still faces the following issues: 1) most current methods train models based on pre-extracted features, lacking a sufficient understanding of sentiment diversity in multimodal data and may even lead to the loss of critical information in the raw data; and 2) textual modality, which possesses high-level semantic features, should typically dominate the fusion process, yet current methods fail to fully leverage this characteristic to balance modality information. To address the aforementioned issues, we propose a novel Multimodal Sentiment Analysis framework using Multimodal-Prefixed and Cross-Modal Attention (DB-MPCA). For the first issue, DB-MPCA employs multimodal raw data for pre-training, which not only allows for in-depth exploration of multimodal information but also significantly enhances the model’s learning capabilities and generalization, while reducing the substantial costs associated with manual annotation. Regarding the second issue, DB-MPCA introduces two prefix encoders designed to convert acoustic and visual features into prefix tokens. These tokens are then embedded into a pre-trained language model, where they are encoded together with textual tokens. Through this approach, DB-MPCA effectively learns cross-modal attention while maintaining the dominance of the textual modality, thereby optimizing the fusion of modalities. Comprehensive experiments conducted on the widely utilized dataset (CMU-MOSI) demonstrate the effectiveness of our model, highlighting its superiority over baseline models.

Knowledge-based Visual Question Answering (KBVQA) is a challenging task, which aims to answer an image related question based on external knowledge. Most of the works describe the semantic distance using the actual Euclidean distance between two nodes, which leads to distortion in modeling knowledge graphs with hierarchical and scale-free structure in KBVQA, and limits the multi-hop reasoning capability of the model. In contrast, the hyperbolic space shows exciting prospects for low-distortion embedding of graphs with hierarchical and free-scale structure. In addition, we map the different stages of reasoning into multiple adjustable hyperbolic spaces, achieving low-distortion, fine-grained reasoning. Extensive experiments on the KVQA, PQ and PQL datasets demonstrate the effectiveness of HyperMR for strong-hierarchy knowledge graphs.

Multi-hop Knowledge Graph Question Answering aims at finding an entity to answer natural language questions from knowledge graphs. When humans perform multi-hop reasoning, people tend to focus on specific relations across different hops and confirm the next entity. Therefore, most algorithms choose the wrong specific relation, which makes the system deviate from the correct reasoning path. The specific relation at each hop plays an important role in multi-hop question answering. Existing work mainly relies on the question representation as relation information, which cannot accurately calculate the specific relation distribution. In this article, we propose an interpretable assistance framework that fully utilizes the relation embeddings to assist in calculating relation distributions at each hop. Moreover, we employ the fusion attention mechanism to ensure the integrity of relation information and hence to enrich the relation embeddings. The experimental results on three English datasets and one Chinese dataset demonstrate that our method significantly outperforms all baselines. The source code of REAN will be available at https://github.com/2399240664/REAN

Emotion-cause pair extraction is an emergent natural language processing task; the target is to extract all pairs of emotion clauses and corresponding cause clauses from unannotated emotion text. Previous studies have employed two-step approaches. However, this research may lead to error propagation across stages. In addition, previous studies did not correctly handle the situation where emotion clauses and cause clauses are the same clauses. To overcome these issues, the authors first use a multitask learning model that is based on graph from the perspective of sorting, which can simultaneously extract emotion clauses, cause clauses and emotion-cause pairs via an end-to-end strategy. Then the authors propose to convert text into graph structured data, and process this scenario through a unique graph convolutional neural network. Finally, the authors design a semantic decision mechanism to address the scenario in which there are multiple emotion-cause pairs in a text.

Aspect-based sentiment analysis (ABSA) aims to predict the sentiment polarities of specific aspects in a comment sentence. Nowadays, models based on graph neural networks enhance semantic perception by using dependency relations on dependency graphs to analyze context and aspect words. However, these models ignore the importance of the dependency type information contained in word relations and do not utilize the dependency types to pay attention to semantic information and the noise problem caused by dependency tree parsing error. To solve the above problems, we propose a novel deep dependency-aware graph convolutional networks (DA-GCN) model in this paper. Among them, the DA-GCN establishes interactive relations with multi-head attention and makes use of the grammar information of dependency perception jointly to effectively learn related information from the generated graphs. We introduce multiple conditional random fields fusing structured attention to better capture specific aspect opinion words. Experimental results on five datasets prove the effectiveness and advancement of our proposed model.

A significant goal in an open-domain dialogue system is to make chatbots generate more persona coherent responses given a context. To achieve this goal, some researchers attempt to introduce persona information into neural dialogue models. However, these neural dialogue models describe excessively persona traits during the conversation, which still suffer from the problem of generating boring and meaningful responses. In this paper, we divide the open-domain personalized dialogue generation task into two processes, persona recognition and persona fusion. In the persona recognition process, we use the pre-training model to encode the personas and conversation history independently, which is beneficial to the persona information fusion. Then, we design a dynamic persona fusion mechanism to effectively mine the relevance of dialogue context and persona information, and dynamically predict whether to incorporate persona features in the process of the dialogues. Our model outperforms with 1.23% in Acc., 0.36% in BLEU, 0.92% in F1, and 0.036% in Distinct than baseline models. The experimental results on the ConvAI2 dataset illustrate that the proposed model is superior to baseline approaches for generating more coherent and persona consistent responses.

Multimodal sentiment analysis (MSA) aims to obtain the emotional polarity of language by analyzing multiple forms of human language, facial expressions, and vocal intonation. The traditional MSA model focuses on the fusion between modalities, ignoring the different contributions of language, visual, and acoustic. Thus different information of modality possesses different importance. To further explore the contributions of different modalities, we propose a highly generalized identifying modal contributions network(IMCN), which contains modality interaction module, modality fusion, and modality joint learning units in the framework. Specifically, we first designed a language modality gain detection module to make reasonable use of visual and acoustic information and reduce the noise of modal information. Secondly, crossmodal attention is used to enrich modal information. Finally, we perform joint learning of unimodal and multimodal modalities to explore the optimal solution for multimodal output. We compared with other popular multimodal sentiment analysis models and obtained better sentiment classification results on CMU-MOSI and CMU-MOSEI benchmark datasets. We also further validated the effectiveness of different modules of IMCN through ablation experiments and discussed the ideas of IMCN design.

Aspect-based sentiment analysis aims to judge the sentiment polarity of specific aspects in comments. Recent meth-ods use graph neural networks based on dependency trees to obtain the relationship between aspects and opinion words by using syntactic information. However, these models ignore the situation that the results of dependency tree parsing are incorrect and the sentences without significant syntactic structure. To solve these problems, in this paper, we propose an interactive double graph convolution networks (Inter-DGCN) model. We reconstruct the dependency tree according to the syntactic in-formation and construct the syntactic graph convolution module. Moreover, we construct the semantic graph convolution module, which uses multi-head self-attention to represent the semantic correlation between words. In addition, we add interactive feature fusion to learn the dual graph convolutional network interactively. Experimental results on three datasets demonstrate the effectiveness and advancement of our proposed model.

Code summarization aims to generate readable summaries that describe the functionality of source code pieces. The main purpose of the code summarization is to help software developers understand the code and save their precious time. However, since programming languages are highly structured, it is challenging to generate high-quality code summaries. For this reason, this paper proposes a new approach named CCHG to automatically generate code comments. Compared to recent models that use additional information such as Abstract Syntax Trees as input, our proposed method only uses the most original code as input. We believe that programming languages are the same as natural languages. Each line of code is equivalent to a sentence, representing an independent meaning. Therefore, we split the entire code snippet into several sentence-level code. Coupled with token-level code, there are two types of code that need to be processed. So we propose heterogeneous graph networks to process the sentence-level and token-level code. Even though we do not introduce additional structural knowledge, the experimental results show that our model has a considerable performance, which indicates that our model can fully learn structural information and sequence information from code snippets.

Abstract Existing word embeddings learning algorithms only employ the contexts of words, but different text documents use words and their relevant parts of speech very differently. Based on the preceding assumption, in order to obtain appropriate word embeddings and further improve the effect of text classification, this paper studies in depth a representation of words combined with their parts of speech. First, using the parts of speech and context of words, a more expressive word embeddings can be obtained. Further, to improve the efficiency of look‐up tables, we construct a two‐dimensional table that is in the format to represent words in text documents. Finally, the two‐dimensional table and a Bayesian theorem are used for text classification. Experimental results show that our model has achieved more desirable results on standard data sets. And it has more preferable versatility and portability than alternative models.