Modern task-oriented dialogue (TOD) systems increasingly rely on large language model (LLM) agents, leveraging Retrieval-Augmented Generation (RAG) and long-context capabilities for long-term memory utilization. However, these methods prioritise semantic similarity over task intent, degrading multi-session coherence. We propose MemGuide, a two-stage intent-driven memory selection framework: (1) Intent‑Aligned Retrieval retrieves goal-consistent QA‑formatted memory units; (2) Missing‑Slot Guided Filtering reranks units by slot-completion gain via a chain‑of‑thought reasoner and fine‑tuned LLaMA‑8B filter. We also introduce the MS-TOD, the first multi-session TOD benchmark with 132 diverse personas, 956 task goals, and annotated intent-aligned memory targets. Evaluations on MS-TOD show that MemGuide boosts task success rate by 11% (88%→99%) and reduces dialogue length by 2.84 turns, and matches single‑session performance.

Multimodal Stance Detection (MSD) is a crucial task for understanding public opinion on social media.Existing methods predominantly operate by learning to fuse modalities.They lack an explicit reasoning process to discern how inter-modal dynamics, such as irony or conflict, collectively shape the user's final stance, leading to frequent misjudgments.To address this, we advocate for a paradigm shift from learning to fuse to learning to reason.We introduce MIND, a Meta-cognitive Intuitivereflective Network for Dual-reasoning.Inspired by the dual-process theory of human cognition, MIND operationalizes a self-improving loop.It first generates a rapid, intuitive hypothesis by querying evolving Modality and Semantic Experience Pools.Subsequently, a meta-cognitive reflective stage uses Modality-CoT and Semantic-CoT to scrutinize this initial judgment, distill superior adaptive strategies, and evolve the experience pools themselves.These dual experience structures are continuously refined during training and recalled at inference to guide robust and context-aware stance decisions.Extensive experiments on the MMSD benchmark demonstrate that our MIND significantly outperforms most baseline models and exhibits strong generalization.

Zhiwei Zhang, Fei Zhao, Rui Wang, Zezhong Wang, Bin Liang, Jiakang Wang, Yao Hu, Shaosheng Cao, Kam-Fai Wong. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.

Zhengyi Zhao, Shubo Zhang, Yiming Du, Bin Liang, Baojun Wang, Zhongyang Li, Binyang Li, Kam-Fai Wong. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.

While large language models (LLMs) have made significant progress in expanding their context windows, they still face great challenges in effectively organizing and utilizing long-term memory to maintain conversation consistency and coherence. Summarizing historical conversations has achieved remarkable performance, which, however, loses conversational trajectory and association, making it difficult to precisely combine memories from different sessions in response to current queries. To address this, we propose the Dynamic Memory Forest (DMF), a novel Consolidation-then-Growth framework for long-term open-domain conversation, which simulates the consolidation and growth processes of human memory by dynamically organizing long-term conversation histories into a memory forest of memory trees. To be specific, inspired by the principles of synaptic consolidation and plasticity from Cognitive Science, we first consolidate each session into memory units that preserve thematic coherence ("Consolidation"). Then, we first structure these units into memory trees and then grow the forest by dynamically connecting them through an evolutionary grafting mechanism, called Group Relative Voting Optimization, which mimics synaptic connection to decide whether a new memory tree should be grafted onto the existing forest or grow independently ("Growth"). For retrieval, we design an Entropy-Driven Memory Walk, constructing a logically coherent memory path via a navigation policy that prioritizes exploring high-entropy nodes. Experiments on three long-term conversation datasets show that our DMF significantly outperforms baselines in enhancing response generation for LLMs.

Event Causality Extraction (ECE) aims to extract causal event pairs from text. Existing methods overlook the interplay between causal event pairs and their corresponding textual evidence (e.g., causal event mention pairs), and fail to effectively leverage global causal dependency information. To address these issues, we propose a Mention-Level Causal Evidence and Global Causal Graph Reasoning (MLCE-GCGR) framework to enhance ECE. First, we introduce an auxiliary Event Mention Causality Extraction (EMCE) task, which extracts causal event mention pairs, to provide evidence for the main ECE task, and design a Dual-Level Interaction Enhancement (DLIE) strategy to enhance the bidirectional interplay between event-level and mention-level causality. Second, we develop a Global Causal Graph Reasoning (GCGR) module that simulates human-like multi-turn reasoning, aiming to progressively refine the causal graph by capturing global dependencies among event mentions, types, and arguments. Experiments on four benchmark datasets show that our method outperforms state-of-the-art approaches. Moreover, by extracting causal event mention pairs as supporting evidence, our approach improves the interpretability of structured causality extraction.

Event reasoning is a fundamental ability that underlies many applications. It requires event schema knowledge to perform global reasoning and needs to deal with the diversity of the inter-event relations and the reasoning paradigms. The extent to which LLMs excel in event reasoning across various relations and reasoning paradigms has not been thoroughly investigated. Additionally, it is still unclear whether LLMs utilize event knowledge in the same way humans do. To mitigate this disparity, we comprehensively evaluate the abilities of event reasoning of LLMs on different relations, paradigms, and levels of abstraction. We introduce a novel benchmark EV2 for EValuation of EVent reasoning. EV2 consists of two levels of evaluation on schema and instance and is comprehensive in relations and reasoning paradigms. We conduct extensive experiments on EV2. We find that 1) LLMs have abilities to accomplish event reasoning but their performances are far from satisfactory. 2) There are imbalances of event reasoning abilities on different relations and paradigms. 3) LLMs have event schema knowledge, however, they're not aligned with humans on how to utilize the knowledge. Based on these findings, we guide the LLMs in utilizing the event schema knowledge as memory leading to improvements in event reasoning.

Stickers are widely used in online chatting, which can vividly express someone's intention, emotion, or attitude. Existing conversation research typically retrieves stickers based on a single session or the previous textual information, which can not adapt to the multi-modal and multi-session nature of the real-world conversation. To this end, we introduce MultiChat, a new dataset for sticker retrieval facing the multi-modal and multi-session conversation, comprising 1,542 sessions, featuring 50,192 utterances and 2,182 stickers. Based on the created dataset, we propose a novel Intent-Guided Sticker Retrieval (IGSR) framework that retrieves stickers for multi-modal and multi-session conversation history drawing support from intent learning. Specifically, we introduce sticker attributes to better leverage the sticker information in multi-modal conversation, which are incorporated with utterances to construct a memory bank. Further, we extract relevant memories for the current conversation from the memory bank to identify the intent of the current conversation, and then retrieve a sticker to respond guided by the intent. Extensive experiments on our MultiChat dataset reveal the robustness and effectiveness of our IGSR approach in multi-session, multi-modal scenarios.

Recent advancements in AI alignment techniques have significantly improved the alignment of large language models (LLMs) with static human preferences. However, the dynamic nature of human preferences can render some prior training data outdated or even erroneous, ultimately causing LLMs to deviate from contemporary human preferences and societal norms. Existing methodologies, either curation of new data for continual alignment or manual correction of outdated data for re-alignment, demand costly human resources. To address this, we propose a novel approach, LLM BehAvior Correction with INfluence FunCtion REcall and Post-Training (LANCET), which needs no human involvement. LANCET consists of two phases: (1) using a new method LinFAC to efficiently identify the training data that significantly impact undesirable model outputs, and (2) applying an novel Influence-driven Bregman Optimization (IBO) technique to adjust the model’s outputs based on these influence distributions. Our experiments show that LANCET effectively and efficiently corrects inappropriate behaviors of LLMs while preserving model utility. Further more, LANCET exhibits stronger generalization ability than all baselines under out-of-distribution harmful prompts, offering better interpretability and compatibility with real-world applications of LLMs.

Video inpainting is a crucial task with diverse applications, including fine-grained video editing, video recovery, and video dewatermarking. However, most existing video inpainting methods primarily focus on visual content completion while neglecting text information. There are only a limited number of text-guided video inpainting techniques, and these techniques struggle with maintaining visual quality and exhibit poor semantic representation capabilities. In this paper, we introduce CoCoCo, a text-guided video inpainting diffusion framework. To address the aforementioned challenges, we enhance both the training data and model structure. Specifically, we devise an instance-aware region selection strategy for masked area sampling and develop a novel motion block that incorporates efficient 3D full attention and textual cross attention. Additionally, our CoCoCo framework can be seamlessly integrated with various personalized text-to-image diffusion models through a delicate training-free transfer mechanism. Comprehensive experiments demonstrate that CoCoCo can create high-quality visual content with enhanced temporal consistency, improved text controllability, and better compatibility with personalized image models.

Bojun Jin, Jianzhu Bao, Yufang Hou, Yang Sun, Yice Zhang, Huajie Wang, Bin Liang, Ruifeng Xu. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025.

Yang Sun, Guanrong Chen, Hamid Alinejad-Rokny, Jianzhu Bao, Yuqi Huang, Bin Liang, Kam-Fai Wong, Min Yang, Ruifeng Xu. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025.

Jingqian Zhao, Bingbing Wang, Geng Tu, Yice Zhang, Qianlong Wang, Bin Liang, Jing Li, Ruifeng Xu. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025.

Fine-tuning multi-turn dialogue systems requires high-quality supervision but often suffers from degraded performance when exposed to low-quality data.Supervision errors in early turns can propagate across subsequent turns, undermining coherence and response quality.Existing methods typically address data quality via static prefiltering, which decouples quality control from training and fails to mitigate turn-level error propagation.In this context, we propose ReSURE (Regularizing Supervision UnREliability), an adaptive learning method that dynamically downweights unreliable supervision without explicit filtering.ReSURE estimates per-turn loss distributions using Welford's online statistics and reweights sample losses on the fly accordingly.Experiments on both singlesource and mixed-quality datasets show improved stability and response quality.Notably, ReSURE enjoys positive Spearman correlations (0.21 ∼ 1.0 across multiple benchmarks) between response scores and number of samples regardless of data quality, which potentially paves the way for utilizing large-scale data effectively.

Cai Ke, Yiming Du, Bin Liang, Yifan Xiang, Lin Gui, Zhongyang Li, Baojun Wang, Yue Yu, Hui Wang, Kam-Fai Wong, Ruifeng Xu. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025.

Zhengyi Zhao, Shubo Zhang, Zezhong Wang, Huimin Wang, Yutian Zhao, Bin Liang, Yefeng Zheng, Binyang Li, Kam-Fai Wong, Xian Wu. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025.

Zhengyi Zhao, Shubo Zhang, Yuxi Zhang, Yanxi Zhao, Yifan Zhang, Zezhong Wang, Huimin Wang, Yutian Zhao, Bin Liang, Yefeng Zheng, Binyang Li, Kam-Fai Wong, Xian Wu. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025.

Fact-checking, the task of reasoning about a claim’s truthfulness based on evidence, has become increasingly crucial with the rapid spread of misinformation. In real-world scenarios, fact-checking often involves checking complex claims necessitating multi-step reasoning, thus imposing a high requirement for a model’s autonomous ability. LLM-based fact-checking performs multi-step reasoning through generating natural language rationales. However, it is susceptible to error propagation problems, which means any error occurring inside rationales will result in an incorrect label. To this end, we propose an iterative rationalization finetuning approach to address this issue, enhancing the LLM’s ability by finetuning it with high-quality rationales. Specifically, we guide the generation of high-quality rationales using golden labels and, inversely, utilize these to finetune the LLM itself, thus constructing a self-improvement cycle. We demonstrate the effectiveness of the proposed method on HoVer and FEVEROUS-S benchmarks, where it achieves state-of-the-art performance, particularly in multi-step reasoning scenarios.

Image multi-emotion prediction aims to identify the emotions evoked by images in humans. In the real world, individual cognitive differences can lead to different viewers experiencing varied emotions. Most existing researchers primarily focus on analyzing image features, which are limited to the perceptual level, leading to a superficial understanding of emotions. To address this gap, we propose an Emotional Reasoning Chain (EReC) based on a multimodal large language model, which learns both perception and reasoning abilities for multi-emotion prediction. Specifically, we design a parameter-efficient fine-tuning paradigm encompassing three task instructions: perception, reasoning, and prediction instruction. This paradigm involves a progressive process to perform targeted instructions within a domain for fine-tuning, thereby optimizing the model’s capabilities in image perception, cognitive reasoning, and multi-emotion prediction. Furthermore, to alleviate hallucinations of large models, a Reason-level Alignment Score (RAS) is introduced to guide the model toward closer alignment with human cognition. Experiments on four datasets demonstrate that our EReC method, which emulates the human cognitive process of viewing and interpreting images, employs progressive tuning to integrate the perceptual, reasoning, and predictive capabilities of large models, achieving superior performance.

The in-context learning (ICL) capability of large language models (LLMs) enables them to perform challenging tasks using provided demonstrations. However, ICL is highly sensitive to the ordering of demonstrations, leading to instability in predictions. This paper shows that this vulnerability can be exploited to design a natural attack—difficult for model providers to detect—that achieves nearly 80% success rate on LLaMA-3 by simply permuting the demonstrations. Existing mitigation methods primarily rely on post-processing and fail to enhance the model's inherent robustness to input permutations, raising concerns about safety and reliability of LLMs. To address this issue, we propose Permutation-resilient learning (PEARL), a novel framework based on distributionally robust optimization (DRO), which optimizes model performance against the worst-case input permutation. Specifically, PEARL consists of a permutation-proposal network (P-Net) and the LLM. The P-Net generates the most challenging permutations by treating it as an optimal transport problem, which is solved using an entropy-constrained Sinkhorn algorithm. Through minimax optimization, the P-Net and the LLM iteratively optimize against each other, progressively improving the LLM's robustness. Experiments on synthetic pre-training and real-world instruction tuning tasks demonstrate that PEARL effectively mitigates permutation attacks and enhances performance. Notably, despite being trained on fewer shots and shorter contexts, PEARL achieves performance gains of up to 40% when scaled to many-shot and long-context scenarios, highlighting its efficiency and generalization capabilities.

Self-supervised learning (SSL) has shown great potential in learning generalizable representations for graph-structured data. However, existing SSL-based graph pre-training methods largely focus on improving graph representations by learning the structure information based on disturbing or reconstructing graphs, which ignores an important issue: the importance of different nodes in the graph structure may vary. To fill this gap, we propose a Centrality-guided Graph Pre-training (CenPre) framework to integrate the distinct importance of nodes in graph structure into the corresponding representations of nodes based on the centrality in graph theory. In this way, the different roles played by different nodes can be effectively leveraged when learning graph structure. The proposed CenPre contains three modules for node representation pre-training and alignment. The node-level importance learning module fuses the fine-grained node importance into node representation based on degree centrality, allowing the aggregation of node representations with equal/similar importance. The graph-level importance learning module characterizes the importance between all nodes in the graph based on eigenvector centrality, enabling the exploitation of graph-level structure similarities/differences when learning node representation. Finally, a representation alignment module aligns the pre-trained node representation using the original one, essentially allowing graph representations to learn structural information without losing their original semantic information, thereby leading to better graph representations. Extensive experiments on a series of real-world datasets demonstrate that the proposed CenPre outperforms the state-of-the-art baselines in the tasks of node classification, link prediction, and graph classification.

Multimodal Emotion Recognition in Conversations (MERC) aims to detect emotions expressed in each utterance within conversational videos. Graph-based methods are widely employed in MERC due to their superiority in modeling intricate speaker-sensitive and context-sensitive dependencies in conversations. Despite promising advancements made, existing graph-based methods primarily suffer from two inherent issues due to their reliance on manually predefined graph structures: structural redundancy, which burdens models with irrelevant noise aggregation, and insufficient connections, which results in a lack of cross-modal contextual cues. To address the above issues, we propose a novel graph structure learning framework for MERC, which comprises two key components: Context-aware Graph Sparsification (CGS) and Implicit Graph Relation Mining (IGR). CGS employs an edge selection network to refine the manually predefined graph, filtering out noisy information caused by structural redundancy. IGR explores potential connections that are beneficial for emotional reasoning. Experimental results on two datasets show that our proposed framework significantly improves the performance of graph-based methods in MERC.

Ang Li, Jingqian Zhao, Bin Liang, Lin Gui, Hui Wang, Xi Zeng, Xingwei Liang, Kam-Fai Wong, Ruifeng Xu. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2025.

Video content editing has a wide range of applications. With the advancement of diffusion-based generative models, video editing techniques have made remarkable progress, yet they still remain far from practical usability. Existing inversion-based video editing methods are time-consuming and struggle to maintain consistency in unedited regions. Although instruction-based methods have high theoretical potential, they face significant challenges in constructing high-quality training datasets - current datasets suffer from issues such as editing correctness, frame consistency, and sample diversity. To bridge these gaps, we introduce the Señorita-2M dataset, a large-scale, diverse, and high-quality video editing dataset. We systematically categorize editing tasks into 2 classes consisting of 18 subcategories. To build this dataset, we design four new task specialists and employ or modify 14 existing task experts to generate data samples for each subclass. In addition, we design a filtering pipeline at both the visual content and instruction levels to further enhance data quality. This approach ensures the reliability of constructed data. Finally, the Señorita-2M dataset comprises 2 million high-fidelity samples with diverse resolutions and frame counts. We trained multiple models using different base video models, i.e., Wan2.1 and CogVideoX-5B, on Señorita-2M, and the results demonstrate that the models exhibit superior visual quality, robust frame-to-frame consistency, and strong instruction following capability. More videos are available at: https://senorita-2m-dataset.github.io.

In the era of the Internet and big data, online user reviews have become a crucial source for extracting product requirements. However, these reviews are often characterized by short text length, diversity, personalization, and redundancy, making traditional requirement modeling approaches ineffective in accurately identifying and mapping user requirements to product functionalities. To address this challenge, we propose LLM-SBERT Requirement Mapping (LSRM), an automated method that enhances requirement extraction accuracy. LSRM leverages a Large Language Model (LLM) with Chain-of-Thought (CoT) reasoning to generate representative keywords for user requirements. These keywords, along with the original text, are embedded into a unified semantic space using Sentence-Bidirectional Encoder Representations from Transformers (SBERT). By applying vector concatenation and semantic similarity calculations, LSRM precisely maps diverse user requirements to standardized product functionalities. We evaluate LSRM using real-world user review data from the electric vehicle industry. Experimental results demonstrate that LSRM significantly outperforms traditional rule-based and machine learning-based approaches in terms of accuracy, precision, recall, and F1-score for requirement identification and mapping. This method presents a novel approach to automated requirement engineering, enhancing product development responsiveness and fostering user-centric innovation.

The understanding and cognition of time are the basis for large language models to understand the world. Although large language models (LLMs) have demonstrated strong capabilities in multiple reasoning tasks, they still have significant deficiencies in temporal reasoning, mainly due to the diversity of temporal expressions and the lack of temporal logic reasoning capabilities. In this study, we propose a novel Temporal Chain of Thought framework(TempCoT) to improve the performance of LLM in temporal reasoning tasks through a three-stage reasoning strategy. First, TempCoT explicitly extracts time constraints to ensure the accuracy of time references during reasoning. Second, a semantic retrieval mechanism is introduced to dynamically obtain key temporal facts to enhance the integrity and reliability of information. Finally, an explicit temporal logic reasoning module is constructed based on point algebra to improve the consistency and interpretability of reasoning. Experimental results show that TempCoT significantly improves the temporal reasoning performance of five different LLMs and shows stronger robustness on complex temporal tasks.

Multimodal Emotion Recognition in Conversations (MERC) aims to identify the emotion expressed in each utterance within conversational videos. Current efforts are directed toward modeling speaker-sensitive context dependencies and multimodal fusion. However, they still struggle to handle utterances from unseen speakers, hampering the model's generalizability. To tackle this challenge, we propose a Speaker Generalization Framework for MERC. Specifically, we build a prototype graph to learn Speaker-based Utterance Representations (SUR), leveraging prototypes as the bridge between seen and unseen speakers. Speaker-aware Contrastive Learning (CL) is then applied to refine SUR, pulling utterances (or prototypes) from the same speaker together while pushing those from different speakers apart. Further, we introduce a prototypical graph CL to generalize SUR to unseen speakers, ensuring that the same speakers exhibit similar graph structures, while dissimilar ones differ. To further enhance model generalization, we introduce Uncertainty-based Generalization for Speakers, randomly sampling SUR statistics from the estimated Gaussian distribution and probabilistically replacing the original SUR. Experimental findings on two datasets highlight that our framework substantially improves the generalization of various MERC models, surpassing state-of-the-art methods.

Cross-domain sentiment analysis (CDSA) aims to predict the sentiment polarities of reviews in the target domain using a sentiment classifier learned from the source labeled domain. Most existing studies are dominant with adversarial learning methods and focus on learning domain-invariant sentiment representations in both the source and target domains. However, since sentiment-specific features are not explicitly decoupled, the model may confuse domain features with sentiment features, thus affecting its generalization ability on target domains. Unlike previous studies, in this paper, we tackle the CDSA task from the view of disentangled representation learning, which explicitly learns the disentangled representations of review, focusing in particular on sentiment and domain semantics. Specifically, we disentangle sentiment-specific and domain-specific features from the text representation of the review by two different linear transformations. Then, we introduce a straightforward disentangled loss to disallow the sentiment-specific feature to capture domain information. Moreover, we leverage target unlabeled data to improve the quality of the learned sentiment-specific features via prototypical learning. It indirectly encourages the sentiment-specific features of target samples having potentially different classes more discriminative. Extensive experiments on widely used CDSA datasets show that our method surpasses competitive baselines and achieves new state-of-the-art results, demonstrating its effectiveness and superiority.

Sexism has become a pressing issue, driven by the rapid-spreading influence of societal norms, media portrayals, and online platforms that perpetuate and amplify gender biases. Curbing sexism has emerged as a critical challenge globally. Being capable of recognizing sexist statements and behaviors is of particular importance since it is the first step in mind change. This survey provides an extensive overview of recent advancements in sexism detection. We present details of the various resources used in this field and methodologies applied to the task, covering different languages, modalities, models, and approaches. Moreover, we examine the specific challenges these models encounter in accurately identifying and classifying sexism. Additionally, we highlight areas that require further research and propose potential new directions for future exploration in the domain of sexism detection. Through this comprehensive exploration, we strive to contribute to the advancement of interdisciplinary research, fostering a collective effort to combat sexism in its multifaceted manifestations.

Multimodal Emotion Recognition in Conversations (ERC) aims to identify the emotions conveyed by each utterance in a conversational video. Current efforts encounter challenges in balancing intra- and inter-speaker context dependencies when tackling intra-modal interactions. This balance is vital as it encompasses modeling self-dependency (emotional inertia) where speakers' own emotions affect them and modeling interpersonal dependencies (empathy) where counterparts' emotions influence a speaker. Furthermore, challenges arise in addressing cross-modal interactions that involve content with conflicting emotions across different modalities. To address this issue, we introduce an adaptive interactive graph network (IGN) called AdaIGN that employs the Gumbel Softmax trick to adaptively select nodes and edges, enhancing intra- and cross-modal interactions. Unlike undirected graphs, we use a directed IGN to prevent future utterances from impacting the current one. Next, we propose Node- and Edge-level Selection Policies (NESP) to guide node and edge selection, along with a Graph-Level Selection Policy (GSP) to integrate the utterance representation from original IGN and NESP-enhanced IGN. Moreover, we design a task-specific loss function that prioritizes text modality and intra-speaker context selection. To reduce computational complexity, we use pre-defined pseudo labels through self-supervised methods to mask unnecessary utterance nodes for selection. Experimental results show that AdaIGN outperforms state-of-the-art methods on two popular datasets. Our code will be available at https://github.com/TuGengs/AdaIGN.

Yang Sun, Muyi Wang, Jianzhu Bao, Bin Liang, Xiaoyan Zhao, Caihua Yang, Min Yang, Ruifeng Xu. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024.

Dialogue policy learning (DPL) aims to determine an abstract representation (also known as action) to guide what the response should be. Typically, DPL is cast as a sequential decision problem across a series of predefined action candidates. However, such static and narrow actions can limit response diversity and impede the dialogue agent’s adaptability to new scenarios and edge cases. To overcome these challenges, we introduce a novel Joint Transformer Reinforcement Learning framework, coined as JoTR, where a text-to-text Transformer-based model is employed to directly generate dialogue actions. More concretely, JoTR formulates a token-grained policy, facilitating more dynamic and adaptable dialogue action generation without the need for predefined action candidates. This method not only enhances the diversity of responses but also significantly improves the system’s capability to manage unfamiliar scenarios. Furthermore, JoTR utilizes Reinforcement Learning with a reward-shaping mechanism to efficiently fine-tune the token-grained policy. This allows the model to evolve through interactions, thereby enhancing its performance over time. Our extensive evaluation demonstrates that JoTR surpasses previous state-of-the-art models, showing improvements of 9% and 13% in success rate, and 34% and 37% in the diversity of dialogue actions across two benchmark dialogue modeling tasks respectively. These results have been validated by both user simulators and human evaluators. Code and data are available at ://github.com/KwanWaiChung/JoTR.

In this paper, we focus on few-shot aspect-based sentiment analysis (ABSA) and try to solve it with in-context learning (ICL) paradigm. However, the effectiveness of ICL is highly affected by retrieved in-context examples. Previous works generally leverage the semantic similarity between the candidate examples and test input to retrieve examples. However, they may yield sub-optimal results for this task. This is because considering only the overall semantic perspective may leave some useful examples, which have syntactic structural relevance to the test input or share identical sentiments and similar aspects to one unretrievable. To address this shortcoming, we advocate retrieving in-context examples for few-shot ABSA by simultaneously considering three perspectives, overall semantics, syntactic structure relevance, and aspect-sentiment semantics. To achieve this, we construct positive and negative pairs from these three perspectives and train the demonstration retriever using contrastive learning. Experimental results on four ABSA datasets show that our retrieval framework can significantly outperform baselines across the board. Moreover, to understand factors influencing ICL performance on few-shot ABSA, we conduct extensive analysis in various scenarios, which can inspire and advance future research.

Argumentative relation classification (ARC) aims to identify the relation between arguments. Previous methods that employ structured knowledge graphs to tackle the ARC task have achieved promising results. However, the prerequisite for structured knowledge to function is that the knowledge includes the topics of arguments. In practice, the topics of arguments are constantly emerging, making it impractical to construct a structured knowledge graph that contains all potential topics in advance. To address this issue, we investigate ARC from a novel perspective by utilizing unstructured knowledge to enhance the learning of ARC, where useful information for topics and arguments could be flexibly obtained from unstructured knowledge. Specifically, to retrieve diverse and comprehensive knowledge for topics and arguments, we first propose a multi-granularity retrieval method tailored for ARC, which acquires unstructured knowledge by dense retrieval at three levels of granularity: the concept level, the concept relation level, and the argument level. Further, we introduce a Knowledge-aware Heterogeneous Graph Reasoner (KHGR), which enables better utilization of retrieved knowledge to facilitate ARC. Extensive experiments on three publicly available datasets verify the superiority of our model compared with several state-of-the-art baselines. Further analysis shows that our method yields more significant benefits in low-resource scenarios.

Given the critical role of graphs in real-world applications and their high-security requirements, improving the ability of graph neural networks (GNNs) to detect out-of-distribution (OOD) data is an urgent research problem. The recent work GNNSAFE proposes a framework based on the aggregation of negative energy scores that significantly improves the performance of GNNs to detect node-level OOD data. However, our study finds that score aggregation among nodes is susceptible to extreme values due to the unboundedness of the negative energy scores and logit shifts, which severely limits the accuracy of GNNs in detecting node-level OOD data. In this paper, we propose NODESAFE: reducing the generation of extreme scores of nodes by adding two optimization terms that make the negative energy scores bounded and mitigate the logit shift. Experimental results show that our approach dramatically improves the ability of GNNs to detect OOD data at the node level, e.g., in detecting OOD data induced by Structure Manipulation, the metric of FPR95 (lower is better) in scenarios without (with) OOD data exposure are reduced from the current SOTA by 28.4% ( 22.7% ). The code is available via https://github.com/ShenzhiYang2000/NODESAFE-Bounded-and-Uniform-Energy-based-Out-of-distribution-Detection-for-Graphs.

Multimodal Emotion Recognition in Conversations (MERC) aims to identify the emotions conveyed by each utterance in a conversational video. Current efforts focus on modeling speaker-sensitive context dependencies and multimodal fusion. Despite the progress, the reliability of MERC methods remains largely unexplored. Extensive empirical studies reveal that current methods suffer from unreliable predictive confidence. Specifically, in some cases, the confidence estimated by these models increases when a modality or specific contextual cues are corrupted, defining these as uncertain samples. This contradicts the foundational principle in informatics, namely, the elimination of uncertainty. Based on this, we propose a novel calibration framework CMERC to calibrate MERC models without altering the model structure. It integrates curriculum learning to guide the model in progressively learning more uncertain samples; hybrid supervised contrastive learning to refine utterance representations, by pulling uncertain samples and others apart; and confidence constraint to penalize the model on uncertain samples. Experimental results on two datasets demonstrate the effectiveness and generalization capabilities of our CMERC across various MERC models, surpassing state-of-the-art methods.

Recent research in Multimodal Emotion Recognition in Conversations (MERC) focuses on multimodal fusion and modeling speaker-sensitive context. In addition to contextual information, personality traits also affect emotional perception. However, current MERC methods solely consider the personality influence of speakers, neglecting speaker-addressee interaction patterns. Additionally, the bottleneck problem of Emotion Shift (ES), where consecutive utterances by the same speaker exhibit different emotions has been long neglected in MERC. Early ES research fails to distinguish diverse shift patterns and simply introduces whether shifts occur as knowledge into the MERC model without considering the complementary nature of the two tasks. Based on this, we propose a Persona-infused Cross-task Graph Network (PCGNet). It first models the speaker-addressee interactive relationships by the persona-infused refinement network. Then, it learns the auxiliary task of ES Detection and the main task of MERC using cross-task connections to capture correlations across two tasks. Finally, we introduce shift-aware contrastive learning to discern diverse shift patterns. Experimental results demonstrate that PCGNet outperforms state-of-the-art methods on two widely used datasets.

Fine-grained meme understanding aims to explore and comprehend the meanings of memes from multiple perspectives by performing various tasks, such as sentiment analysis, intention detection, and offensiveness detection. Existing approaches primarily focus on simple multi-modality fusion and individual task analysis. However, there remain several limitations that need to be addressed: (1) the neglect of incongruous features within and across modalities, and (2) the lack of consideration for correlations among different tasks. To this end, we leverage metaphorical information as text modality and propose a Metaphor-aware Multi-modal Multi-task Framework (M3F) for fine-grained meme understanding. Specifically, we create inter-modality attention enlightened by the Transformer to capture inter-modality interaction between text and image. Moreover, intra-modality attention is applied to model the contradiction between the text and metaphorical information. To learn the implicit interaction among different tasks, we introduce a multi-interactive decoder that exploits gating networks to establish the relationship between various subtasks. Experimental results on the MET-Meme dataset show that the proposed framework outperforms the state-of-the-art baselines in fine-grained meme understanding.

Identifying sarcastic clues from both textual and visual information has become an important research issue, called Multimodal Sarcasm Detection. In this paper, we investigate multimodal sarcasm detection from a novel perspective, where a multimodal graph contrastive learning strategy is proposed to fuse and distinguish the sarcastic clues for textual modality and visual modality. Specifically, we first utilize object detection to derive the crucial visual regions accompanied by their captions of the images, which allows better learning of the key visual regions of visual modality. In addition, to make full use of the semantic information of the visual modality, we employ optical character recognition to extract the textual content in the images. Then, based on image regions, the textual content of visual modality, and the context of the textual modality, we build a multimodal graph for each sample to model the intricate sarcastic relations between modalities. Furthermore, we devise a graph-oriented contrastive learning strategy to leverage the correlations in the same label and differences between different labels, so as to capture better multimodal representations for multimodal sarcasm detection. Extensive experiments show that our method outperforms the previous best baseline models (with a 2.47% improvement in Accuracy, a 1.99% improvement in F-score, and a 2.20% improvement in Macro F-score). The ablation study shows that both multimodal graph structure and graph-oriented contrastive learning are important to our framework. Further, the experiments of using different pre-trained methods show that the proposed multimodal graph contrastive learning framework can directly work with various pre-trained models and achieve outstanding performance in multimodal sarcasm detection.

Multimodal aspect-based sentiment analysis (MABSA) aims to determine the sentiment polarity of each aspect mentioned in the text based on multimodal content. Various approaches have been proposed to model multimodal sentiment features for each aspect via modal interactions. However, most existing approaches have two shortcomings: (1) The representation gap between textual and visual modalities may increase the risk of misalignment in modal interactions; (2) In some examples where the image is not related to the text, the visual information may not enrich the textual modality when learning aspect-based sentiment features. In such cases, blindly leveraging information from visual modal may introduce noises in reasoning the aspect-based sentiment expressions. To tackle the shortcomings mentioned above, we propose an end-to-end MABSA framework with image conversion and noise filtration. Specifically, to bridge the representation gap in different modalities, we attempt to translate images into the input space of a pre-trained language model (PLM). To this end, we develop an image-to-text conversion module that can convert an image to an implicit sequence of token embedding. Moreover, an aspect-oriented filtration module is devised to alleviate the noise in the implicit token embeddings, which consists of two attention operations. The former aims to create an enhanced aspect embedding as a query, and the latter seeks to use this query to retrieve relevant auxiliary information from the implicit token embeddings to supplement the textual content. After filtering the noise, we leverage a PLM to encode the text, aspect, and image prompt derived from filtered implicit token embeddings as sentiment features to perform aspect-based sentiment prediction. Experimental results on two MABSA datasets show that our framework achieves state-of-the-art performance. Furthermore, extensive experimental analysis demonstrates the proposed framework has superior robustness and efficiency.

Emotion recognition in conversations (ERC) typically requires modeling both intra- and inter-speaker context dependencies. However, when modeling inter-speaker dependencies, it may not capture differences among other participants in the conversation. Recent ERC research has attempted to improve utterance representations by utilizing speakers’ commonsense knowledge. Nonetheless, these studies ignore the causal consistency in knowledge between the two participants, which contradicts the above modeling of speaker-sensitive context dependencies. Additionally, it is observed that historical utterances from various topics are blindly leveraged in context modeling, which fails the inter- and intra-topic coherence. To address these issues, we propose the topic- and causal-aware interactive graph network (TCA-IGN). Specifically, we suggest a graph encoder to model topic-level context dependencies, achieving inter- and intra-topic coherence. The topics of utterances are derived from a context-sensitive neural topic model. Then, we present a causal-aware graph attention to keep the speaker’s causal consistency in commonsense knowledge, improving speaker-level context modeling. Finally, considering the defect of modeling inter-speaker or inter-topic context dependencies, we employ supervised contrastive learning to sweeten it. Experimental results show that TCA-IGN outperforms state-of-the-art methods on three public conversational datasets.

Identifying users' stances regarding specific targets/topics is a significant route to learning public opinion from social media platforms.Most existing studies of stance detection strive to learn stance information about specific targets from the context, in order to determine the user's stance on the target.However, in real-world scenarios, we usually have a certain understanding of a target when we express our stance on it.In this paper, we investigate stance detection from a novel perspective, where the background knowledge of the targets is taken into account for better stance detection.To be specific, we categorize background knowledge into two categories: episodic knowledge and discourse knowledge, and propose a novel Knowledge-Augmented Stance Detection (KASD) framework.For episodic knowledge, we devise a heuristic retrieval algorithm based on the topic to retrieve the Wikipedia documents relevant to the sample.Further, we construct a prompt for ChatGPT to filter the Wikipedia documents to derive episodic knowledge.For discourse knowledge, we construct a prompt for ChatGPT to paraphrase the hashtags, references, etc., in the sample, thereby injecting discourse knowledge into the sample.Experimental results on four benchmark datasets demonstrate that our KASD achieves state-of-the-art performance in in-target and zero-shot stance detection.

Recent efforts have endeavored to employ the sequence-to-sequence (Seq2Seq) model in Information Extraction (IE) due to its potential to tackle multiple IE tasks in a unified manner.Under this formalization, multiple structured objects are concatenated as the target sequence in a predefined order.However, structured objects, by their nature, constitute an unordered set.Consequently, this formalization introduces a potential order bias, which can impair model learning.Targeting this issue, this paper proposes a set learning approach that considers multiple permutations of structured objects to optimize set probability approximately.Notably, our approach does not require any modifications to model structures, making it easily integrated into existing generative IE frameworks.Experiments show that our method consistently improves existing frameworks on vast tasks and datasets.

Unintended dataset biases typically exist in existing Emotion Recognition in Conversations (ERC) datasets, including label bias, where models favor the majority class due to imbalanced training data, as well as the speaker and neutral word bias, where models make unfair predictions because of excessive correlations between specific neutral words or speakers and classes.However, previous studies in ERC generally focus on capturing context-sensitive and speaker-sensitive dependencies, ignoring the unintended dataset biases of data, which hampers the generalization and fairness in ERC.To address this issue, we propose a Training-Free Debiasing framework (TFD) that operates during prediction without additional training.To ensure compatibility with various ERC models, it does not balance data or modify the model structure.Instead, TFD extracts biases from the model by generating counterfactual utterances and contexts and mitigates them using simple yet empirically robust element-wise subtraction operations.Extensive experiments on three public datasets demonstrate that TFD effectively improves generalization ability and fairness across different ERC models 1 .

Aspect Sentiment Triplet Extraction (ASTE) is an important task in sentiment analysis, aiming to extract aspect-level opinions and sentiments from user-generated reviews.The fine-grained nature of ASTE incurs a high annotation cost, while the scarcity of annotated data limits the performance of existing methods.This paper exploits data augmentation to address this issue.Traditional augmentation methods typically modify the input sentences of existing samples via heuristic rules or language models, which have shown success in text classification tasks.However, applying these methods to fine-grained tasks like ASTE poses challenges in generating diverse augmented samples while maintaining alignment between modified sentences and origin labels.Therefore, this paper proposes a target-to-source augmentation approach for ASTE.Our approach focuses on learning a generator that can directly generate new sentences based on labels and syntactic templates.With this generator, we can generate a substantial number of diverse augmented samples by mixing labels and syntactic templates from different samples.Besides, to ensure the quality of the generated sentence, we introduce fluency and alignment discriminators to provide feedback on the generated sentence and then use this feedback to optimize the generator via a reinforcement learning framework.Experiments demonstrate that our approach significantly enhances the performance of existing ASTE models. 1

Text classifiers built on Pre-trained Language Models (PLMs) have achieved remarkable progress in various tasks including sentiment analysis, natural language inference, and question-answering. However, the occurrence of uncertain predictions by these classifiers poses a challenge to their reliability when deployed in practical applications. Much effort has been devoted to designing various probes in order to understand what PLMs capture. But few studies have delved into factors influencing PLM-based classifiers’ predictive uncertainty. In this paper, we propose a novel framework, called CUE, which aims to interpret uncertainties inherent in the predictions of PLM-based models. In particular, we first map PLM-encoded representations to a latent space via a variational auto-encoder. We then generate text representations by perturbing the latent space which causes fluctuation in predictive uncertainty. By comparing the difference in predictive uncertainty between the perturbed and the original text representations, we are able to identify the latent dimensions responsible for uncertainty and subsequently trace back to the input features that contribute to such uncertainty. Our extensive experiments on four benchmark datasets encompassing linguistic acceptability classification, emotion classification, and natural language inference show the feasibility of our proposed framework. Our source code is available at https://github.com/lijiazheng99/CUE.

The state-of-the-art approaches to targeted aspect-based sentiment analysis (TABSA) are mostly built on deep neural networks with attention mechanisms. One problem is that embeddings of targets and aspects are either pre-trained from large external corpora or randomly initialized. We argue that affective commonsense knowledge and words indicative of sentiment could be used to learn better target and aspect embeddings. We therefore propose an embedding refinement framework calledRAEC(RefiningAffectiveEmbedding fromContext), in which sentiment concepts extracted from affective commonsense knowledge and word relative location information are incorporated to derive context-affective embeddings. Furthermore, a sparse coefficient vector is exploited in refining the embeddings of targets and aspects separately. In this way, embeddings of targets and aspects can capture the highly relevant affective words. Experimental results on two benchmark datasets show that our framework can be easily integrated with existing embedding-based TABSA models and achieves state-of-the-art results compared to models relying on pre-trained word embeddings or built on other embedding refinement methods.

Emotion recognition in conversations (ERC) needs to detect the emotion of each utterance in conversations. However, it is difficult for machines to recognize the emotion of utterances like humans, partly because of the lack of commonsense knowledge. Despite existing efforts gradually incorporate knowledge in ERC, they can not adaptively adjust knowledge according to different utterances and their context. In this article, we propose a knowledge selection framework SKSEC (SelectKnowledge in light ofSentimentEmotion andContext). In the SKSEC framework, first, external knowledge is eliminated by three Knowledge Elimination (KE) modules. More concretely, In word-level KE, the concept knowledge different from the sentiment corresponding to the word in utterances is randomly eliminated. In utterance- or context-level KE, If the similarity between the knowledge representation and the emotion label representation of the current utterance or its context is less than the preset threshold, the knowledge will be eliminated. Then we refine the weight of knowledge using two Graph ATtention (GAT) mechanisms. Specifically, In Sentics GAT, we employ a dimensional emotion model to measure words in utterances and their corresponding knowledge and adjust the weight of knowledge according to their emotional similarity. In Semantics GAT, the weight of knowledge is adjusted according to the semantic similarity between context and incorporated knowledge. Finally, we feed the selected knowledge to the most advanced models to evaluate the quality of knowledge. The experimental results show that the SKSEC framework can effectively improve the performance of the model by eliminating and refining external knowledge in different size and domain datasets.

Existing aspect-based/category sentiment analysis methods have shown great success in detecting sentiment polarity toward a given aspect in a sentence with supervised learning, where the training and inference stages share the same pre-defined set of aspects. However, in practice, the aspect categories are changing rather than keeping fixed over time. Dealing with unseen aspect categories is under-explored in existing methods. In this article, we formulate a new few-shot aspect category sentiment analysis (FSACSA) task, which aims to effectively predict the sentiment polarity of previously unseen aspect categories. To this end, we propose a novel Aspect-Focused Meta-Learning (AFML) framework that constructs aspect-aware and aspect-contrastive representations from external knowledge to match the target aspect with aspects in the training set. Concretely, we first construct two auxiliary contrastive sentences for a given sentence with the incorporation of external knowledge, enabling the learning of sentence representations with a better generalization. Then, we devise an aspect-focused induction network to leverage the contextual sentiment toward a given aspect to refine the label vectors. Furthermore, we employ the episode-based meta-learning algorithm to train the whole network, so as to learn to generalize to novel aspects. Extensive experiments on multiple real-life datasets show that our proposed AFML framework achieves the state-of-the-art results for the FSACSA task.

Bin Liang, Chenwei Lou, Xiang Li, Min Yang, Lin Gui, Yulan He, Wenjie Pei, Ruifeng Xu. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2022.

Zero-shot stance detection (ZSSD) aims to detect the stance for an unseen target during the inference stage.In this paper, we propose a joint contrastive learning (JointCL) framework, which consists of stance contrastive learning and target-aware prototypical graph contrastive learning.Specifically, a stance contrastive learning strategy is employed to better generalize stance features for unseen targets.Further, we build a prototypical graph for each instance to learn the target-based representation, in which the prototypes are deployed as a bridge to share the graph structures between the known targets and the unseen ones.Then a novel target-aware prototypical graph contrastive learning strategy is devised to generalize the reasoning ability of target-based stance representations to the unseen targets.Extensive experiments on three benchmark datasets show that the proposed approach achieves state-ofthe-art performance in the ZSSD task 1 .

The existing research efforts in Multimodal Sentiment Analysis (MSA) have focused on developing the expressive ability of neural networks to fuse information from different modalities. However, these approaches lack a mechanism to understand the complex relations within and across different modalities, since some sentiments may be scattered in different modalities. To this end, in this paper, we propose a novel hierarchical graph contrastive learning (HGraph-CL) framework for MSA, aiming to explore the intricate relations of intra- and inter-modal representations for sentiment extraction. Specifically, regarding the intra-modal level, we build a unimodal graph for each modality representation to account for the modality-specific sentiment implications. Based on it, a graph contrastive learning strategy is adopted to explore the potential relations based on unimodal graph augmentations. Furthermore, we construct a multimodal graph of each instance based on the unimodal graphs to grasp the sentiment relations between different modalities. Then, in light of the multimodal augmentation graphs, a graph contrastive learning strategy over the inter-modal level is proposed to ulteriorly seek the possible graph structures for precisely learning sentiment relations. This essentially allows the framework to understand the appropriate graph structures for learning intricate relations among different modalities. Experimental results on two benchmark datasets show that the proposed framework outperforms the state-of-the-art baselines in MSA.

Argument mining (AM) is a challenging task as it requires recognizing the complex argumentation structures involving multiple subtasks.To handle all subtasks of AM in an end-to-end fashion, previous works generally transform AM into a dependency parsing task.However, such methods largely require complex pre-and post-processing to realize the task transformation.In this paper, we investigate the endto-end AM task from a novel perspective by proposing a generative framework, in which the expected outputs of AM are framed as a simple target sequence.Then, we employ a pretrained sequence-to-sequence language model with a constrained pointer mechanism (CPM) to model the clues for all the subtasks of AM in the light of the target sequence.Furthermore, we devise a reconstructed positional encoding (RPE) to alleviate the order biases induced by the autoregressive generation paradigm.Experimental results show that our proposed framework achieves new state-of-the-art performance on two AM benchmarks. 1

This paper investigates the sentiment analysis task from a novel perspective by incorporating sentiment knowledge and eye movement into a graph architecture, aiming to draw the eye movement-based sentiment relationships for learning the sentiment expression of the context.To be specific, we first explore a linguistic probing eye movement paradigm to extract eye movement features based on the close relationship between linguistic features and the early and late processes of human reading behavior.Furthermore, to derive eye movement features with sentiment concepts, we devise a novel weighting strategy to integrate sentiment scores extracted from affective commonsense knowledge into eye movement features, called sentiment-eye movement weights.Then, the sentiment-eye movement weights are exploited to build the sentiment-eye movement guided graph (SEMGraph) model, so as to model the intricate sentiment relationships in the context.Experimental results on two sentiment analysis datasets with eye movement signals and three sentiment analysis datasets without eye movement signals show that the proposed SEM-Graph achieves state-of-the-art performance, and can also be directly generalized to those sentiment analysis datasets without eye movement signals.

Aspect Sentiment Triplet Extraction (ASTE) aims to extract the aspect terms along with the corresponding opinion terms and the expressed sentiments in the review, which is an important task in sentiment analysis.Previous research efforts generally address the ASTE task in an endto-end fashion through the table-filling formalization, in which the triplets are represented by a two-dimensional (2D) table of word-pair relations.Under this formalization, a term-level relation is decomposed into multiple independent word-level relations, which leads to relation inconsistency and boundary insensitivity in the face of multi-word aspect terms and opinion terms.To overcome these issues, we propose Boundary-Driven Table-Filling (BDTF), which represents each triplet as a relation region in the 2D table and transforms the ASTE task into detection and classification of relation regions.We also notice that the quality of the table representation greatly affects the performance of BDTF.Therefore, we develop an effective relation representation learning approach to learn the table representation, which can fully exploit both word-to-word interactions and relation-torelation interactions.Experiments on several public benchmarks show that the proposed approach achieves state-of-the-art performances 1 .

Existing approaches for sarcasm detection are mainly based on supervised learning, in which the promising performance largely depends on a considerable amount of labeled data or extra information. In the real world scenario, however, the abundant labeled data or extra information requires high labor cost, not to mention that sufficient annotated data is unavailable in many low-resource conditions. To alleviate this dilemma, we investigate sarcasm detection from an unsupervised perspective, in which we explore a masking and generation paradigm in the context to extract the context incongruities for learning sarcastic expression. Further, to improve the feature representations of the sentences, we use unsupervised contrastive learning to improve the sentence representation based on the standard dropout. Experimental results on six perceived sarcasm detection benchmark datasets show that our approach outperforms baselines. Simultaneously, our unsupervised method obtains comparative performance with supervised methods for the intended sarcasm dataset.

Stance detection aims to identify the stance of the text towards a target. Different from conventional stance detection, Zero-Shot Stance Detection (ZSSD) needs to predict the stances of the unseen targets during the inference stage. For human beings, we generally tend to reason the stance of a new target by linking it with the related knowledge learned from the known ones. Therefore, in this paper, to better generalize the target-related stance features learned from the known targets to the unseen ones, we incorporate the targeted background knowledge from Wikipedia into the model. The background knowledge can be considered as a bridge for connecting the meanings between known targets and the unseen ones, which enables the generalization and reasoning ability of the model to be improved in dealing with ZSSD. Extensive experimental results demonstrate that our model outperforms the state-of-the-art methods on the ZSSD task.

Zero-shot stance detection (ZSSD) is challenging as it requires detecting the stance of previously unseen targets during the inference stage. Being able to detect the target-related transferable stance features from the training data is arguably an important step in ZSSD. Generally speaking, stance features can be grouped into target-invariant and target-specific categories. Target-invariant stance features carry the same stance regardless of the targets they are associated with. On the contrary, target-specific stance features only co-occur with certain targets. As such, it is important to distinguish these two types of stance features when learning stance features of unseen targets. To this end, in this paper, we revisit ZSSD from a novel perspective by developing an effective approach to distinguish the types (target-invariant/-specific) of stance features, so as to better learn transferable stance features. To be specific, inspired by self-supervised learning, we frame the stance-feature-type identification as a pretext task in ZSSD. Furthermore, we devise a novel hierarchical contrastive learning strategy to capture the correlation and difference between target-invariant and -specific features and further among different stance labels. This essentially allows the model to exploit transferable stance features more effectively for representing the stance of previously unseen targets. Extensive experiments on three benchmark datasets show that the proposed framework achieves the state-of-the-art performance in ZSSD.

Continual relation extraction (CRE) is an important task of continual learning, which aims to learn incessantly emerging new relations between entities from texts. To avoid catastrophically forgetting old relations, some existing research efforts have focused on exploring memory replayed methods by storing typical historical learned instances or embedding all observed relations as prototypes in the episodic memory and replaying them in the subsequent training process. However, they generally fail to exploit the relation knowledge contained in the pre-trained language model (PLM), which could provide enlightening information to the representations of new relations from the known ones. To this end, we investigate the CRE from a novel perspective by generating knowledge-infused relation prototypes to leverage the relational knowledge from PLM with prompt tuning. Specifically, based on the typical samples collected from the historical learned instances with K-means algorithm, we devise novel relational knowledge-infused prompts to elicit relational knowledge from PLM for generating knowledge-infused relation prototypes. Then the prototypes are used to refine the typical examples embedding and calculate the stability-plasticity balance score for adjusting the memory replayed progress. The experimental results show that our method outperforms the state-of-the-art baseline models in CRE. The further extensive analysis presents that the proposed method is robust to memory size, task order, length of the task sequence, and the number of training instances.

Most existing aspect-based sentiment analysis (ABSA) research efforts are devoted to extracting the aspect-dependent sentiment features from the sentence towards the given aspect. However, it is observed that about 60% of the testing aspects in commonly used public datasets are unknown to the training set. That is, some sentiment features carry the same polarity regardless of the aspects they are associated with (aspect-invariant sentiment), which props up the high accuracy of existing ABSA models when inevitably inferring sentiment polarities for those unknown testing aspects. Therefore, in this paper, we revisit ABSA from a novel perspective by deploying a novel supervised contrastive learning framework to leverage the correlation and difference among different sentiment polarities and between different sentiment patterns (aspect-invariant/-dependent). This allows improving the sentiment prediction for (unknown) testing aspects in the light of distinguishing the roles of valuable sentiment features. Experimental results on 5 benchmark datasets show that our proposed approach substantially outperforms state-of-the-art baselines in ABSA. We further extend existing neural network-based ABSA models with our proposed framework and achieve improved performance.

Argument pair extraction (APE) aims to extract interactive argument pairs from two passages of a discussion.Previous work studied this task in the context of peer review and rebuttal, and decomposed it into a sequence labeling task and a sentence relation classification task.However, despite the promising performance, such an approach obtains the argument pairs implicitly by the two decomposed tasks, lacking explicitly modeling of the argument-level interactions between argument pairs.In this paper, we tackle the APE task by a mutual guidance framework, which could utilize the information of an argument in one passage to guide the identification of arguments that can form pairs with it in another passage.In this manner, two passages can mutually guide each other in the process of APE.Furthermore, we propose an inter-sentence relation graph to effectively model the interrelations between two sentences and thus facilitates the extraction of argument pairs.Our proposed method can better represent the holistic argument-level semantics and thus explicitly capture the complex correlations between argument pairs.Experimental results show that our approach significantly outperforms the current state-of-the-art model.

In this paper, we investigate the Aspect Category Sentiment Analysis (ACSA) task from a novel perspective by exploring a Beta Distribution guided aspect-aware graph construction based on external knowledge.That is, we are no longer entangled about how to laboriously search the sentiment clues for coarsegrained aspects from the context, but how to preferably find the words highly sentimentrelated to the aspects in the context and determine their importance based on the public knowledge base, so as to naturally learn the aspect-related contextual sentiment dependencies with these words in ACSA.To be specific, we first regard each aspect as a pivot to derive aspect-aware words that are highly related to the aspect from external affective commonsense knowledge.Then, we employ Beta Distribution to educe the aspect-aware weight, which reflects the importance to the aspect, for each aspect-aware word.Afterward, the aspect-aware words are served as the substitutes of the coarse-grained aspect to construct graphs for leveraging the aspectrelated contextual sentiment dependencies in ACSA.Experiments on 6 benchmark datasets show that our approach significantly outperforms the state-of-the-art baseline methods.

Sarcasm is a peculiar form and sophisticated linguistic act to express the incongruity of someone's implied sentiment expression, which is a pervasive phenomenon in social media platforms. Compared with sarcasm detection purely on texts, multi-modal sarcasm detection is more adapted to the rapidly growing social media platforms, where people are interested in creating multi-modal messages. When focusing on the multi-modal sarcasm detection for tweets consisting of texts and images on Twitter, the significant clue of improving the performance of multi-modal sarcasm detection evolves into how to determine the incongruity relations between texts and images. In this paper, we investigate multi-modal sarcasm detection from a novel perspective, so as to determine the sentiment inconsistencies within a certain modality and across different modalities by constructing heterogeneous in-modal and cross-modal graphs (InCrossMGs) for each multi-modal example. Based on it, we explore an interactive graph convolution network (GCN) structure to jointly and interactively learn the incongruity relations of in-modal and cross-modal graphs for determining the significant clues in sarcasm detection. Experimental results demonstrate that our proposed model achieves state-of-the-art performance in multi-modal sarcasm detection.

Detecting sarcastic expressions could promote the understanding of natural language in social media. In this paper, we revisit sarcasm detection from a novel perspective, so as to account for the long-range literal sentiment inconsistencies. More concretely, we explore a novel scenario of constructing an affective graph and a dependency graph for each sentence based on the affective information retrieved from external affective commonsense knowledge and the syntactical information of the sentence. Based on it, an Affective Dependency Graph Convolutional Network (ADGCN) framework is proposed to draw long-range incongruity patterns and inconsistent expressions over the context for sarcasm detection by means with interactively modeling the affective and dependency information. Experimental results on multiple benchmark datasets show that our proposed approach outperforms the current state-of-the-art methods in sarcasm detection.

Target plays an essential role in stance detection of an opinionated review/claim, since the stance expressed in the text often depends on the target. In practice, we need to deal with targets unseen in the annotated training data. As such, detecting stance for an unknown or unseen target is an important research problem. This paper presents a novel approach that automatically identifies and adapts the target-dependent and target-independent roles that a word plays with respect to a specific target in stance expressions, so as to achieve cross-target stance detection. More concretely, we explore a novel solution of constructing heterogeneous target-adaptive pragmatics dependency graphs (TPDG) for each sentence towards a given target. An in-target graph is constructed to produce inherent pragmatics dependencies of words for a distinct target. In addition, another cross-target graph is constructed to develop the versatility of words across all targets for boosting the learning of dominant word-level stance expressions available to an unknown target. A novel graph-aware model with interactive Graphical Convolutional Network (GCN) blocks is developed to derive the target-adaptive graph representation of the context for stance detection. The experimental results on a number of benchmark datasets show that our proposed model outperforms state-of-the-art methods in cross-target stance detection.

In most previous studies, the aspect-related text is considered an important clue for the Aspect-based Sentiment Analysis (ABSA) task, and thus various attention mechanisms have been proposed to leverage the interactions between aspects and context. However, it is observed that some sentiment expressions carry the same polarity regardless of the aspects they are associated with. In such cases, it is not necessary to incorporate aspect information for ABSA. More observations on the experimental results show that blindly leveraging interactions between aspects and context as features may introduce noises when analyzing those aspect-invariant sentiment expressions, especially when the aspect-related annotated data is insufficient. Hence, in this paper, we propose an Adversarial Multi-task Learning framework to identify the aspect-invariant/dependent sentiment expressions without extra annotations. In addition, we adopt a gating mechanism to control the contribution of representations derived from aspect-invariant and aspect-dependent hidden states when generating the final contextual sentiment representations for the given aspect. This essentially allows the exploitation of aspect-invariant sentiment features for better ABSA results. Experimental results on two benchmark datasets show that extending existing neural models using our proposed framework achieves superior performance. In addition, the aspect-invariant data extracted by the proposed framework can be considered as pivot features for better transfer learning of the ABSA models on unseen aspects.

In this paper, we explore a novel solution of constructing a heterogeneous graph for each instance by leveraging aspect-focused and inter-aspect contextual dependencies for the specific aspect and propose an Interactive Graph Convolutional Networks (InterGCN) model for aspect sentiment analysis. Specifically, an ordinary dependency graph is first constructed for each sentence over the dependency tree. Then we refine the graph by considering the syntactical dependencies between contextual words and aspect-specific words to derive the aspect-focused graph. Subsequently, the aspect-focused graph and the corresponding embedding matrix are fed into the aspect-focused GCN to capture the key aspect and contextual words. Besides, to interactively extract the inter-aspect relations for the specific aspect, an inter-aspect GCN is adopted to model the representations learned by aspect-focused GCN based on the inter-aspect graph which is constructed by the relative dependencies between the aspect words and other aspects. Hence, the model can be aware of the significant contextual and aspect words when interactively learning the sentiment features for a specific aspect. Experimental results on four benchmark datasets illustrate that our proposed model outperforms state-of-the-art methods and substantially boosts the performance in comparison with BERT.

Attention-based neural models were employed to detect the different aspects and sentiment polarities of the same target in targeted aspectbased sentiment analysis (TABSA).However, existing methods do not specifically pre-train reasonable embeddings for targets and aspects in TABSA.This may result in targets or aspects having the same vector representations in different contexts and losing the contextdependent information.To address this problem, we propose a novel method to refine the embeddings of targets and aspects.Such pivotal embedding refinement utilizes a sparse coefficient vector to adjust the embeddings of target and aspect from the context.Hence the embeddings of targets and aspects can be refined from the highly correlative words instead of using context-independent or randomly initialized vectors.Experiment results on two benchmark datasets show that our approach yields the state-of-the-art performance in TABSA task.