Yumeng Fu, Weitao Huang, Junjie Wu, Hao Teng, Shouduo Shang, Meishan Zhang, Bingquan Liu. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.

Emotion Recognition in Conversation (ERC), the task of identifying the emotion of each utterance in a conversation, is crucial for humanmachine interaction.Existing LLM-based ERC methods focus on standard prompting and slow thinking for emotion analysis.However, they suffer from the lack of human-like emotion reasoning and discrimination between similar emotions, thus limiting accurate emotion predictions.To this end, we present JoPR, jointing perception-curriculum learning and emotional reasoning for conversational emotion recognition.Specifically, we devise a multi-dimension curriculum with long CoT fine-tuning to clone human-like emotion reasoning.We further design an emotion-specific reward function in a novel reinforcement learning framework, thereby enhancing the discernment between similar emotions.We conduct extensive experiments on three widely used benchmark datasets, and the results demonstrate that our JoPR achieves consistent and significant improvements over baselines.

Emotion recognition in conversation (ERC), the task of discerning human emotions for each utterance within a conversation, has garnered significant attention in human-computer interaction systems. Previous ERC studies focus on speaker-specific information that predominantly stems from relationships among utterances, which lacks sufficient information around conversations. Recent research in ERC has sought to exploit pre-trained large language models (LLMs) with speaker modelling to comprehend emotional states. Although these methods have achieved the encouraging results, the extracted speaker-specific information struggles to indicate emotional dynamics. In this paper, motivated by the fact that speaker characteristics play a crucial role and LLMs have rich world knowledge, we present LaERC-S, a novel framework that stimulates LLMs to explore speaker characteristics involving the mental state and behavior of interlocutors, for accurate emotion predictions. To endow LLMs with these knowledge information, we adopt the two-stage learning to make the models reason speaker characteristics and track the emotion of the speaker in complex conversation scenarios. Extensive experiments on three benchmark datasets demonstrate the superiority of LaERC-S, reaching the new state-of-the-art.

Detecting and grounding multi-modal media manipulation (DGM4) aims to identify the authenticity of media content in the form of image-text pairs, and locate forgery contents including text tokens or image regions. Existing methods adopt joint optimization frameworks to train multiple dedicated heads for different sub-tasks. This paradigm is prone to yielding models with insufficient learning regarding manipulation traces. In this paper, we propose a two-stage learning framework, named 2S-DGM4, exploiting correlations between manipulation detection and grounding for debunking manipulation content. It introduces modality-specific manipulation types into the subsequent manipulation grounding, and naturally conducts the mutual agreement among various sub-tasks. Apart from the global context in cross-modal interaction, we develop a fine-grained refinement module to capture subtle manipulation traces. Experiments demonstrate that 2S-DGM4outperforms state-of-the-art methods on the public benchmark dataset DGM4. Further analyses verify the efficacy of 2S-DGM4on manipulation reasoning.

Large-scale pre-trained vision-language models (VLMs), like CLIP, have presented striking generalizability for adapting to image classification in a few shot setting. Most existing methods explore a set of learnable tokens, such as prompt learning, on data-efficient utilization for task adaptation. However, they focus on either the coupled-modality property by prompt projection or decoupled-modality characteristic by prompt consistency, which ignores effective interaction between prompts. To model the deep yet sufficient cross-modal interaction and enhance the generalization between both seen and unseen tasks, in this paper, we propose a novel coupled and decoupled prompt learning framework, dubbed PromptCD, for vision-language models. Specifically, we introduce a bi-directional coupled-modality mechanism to intensify the interaction between both vision and language branches. Additionally, we propose mixture consistency to further improve the generalization and discrimination of the models on unseen tasks. The integration of such a mechanism and consistency facilitates the proposed framework adaptation for various downstream tasks. We conduct extensive experiments on 11 image classification datasets under a range of evaluation protocols, including base-to-novel and domain generalization, and cross-dataset recognition. Experimental results demonstrate that our proposed PromptCD overall outperforms state-of-the-art methods.

Multimodal named entity recognition (MNER) is an essential task of vision and language, which aims to locate named entities and classify them to the predefined categories using visual scenarios. However, existing MNER studies often suffer from bias issues with fine-grained visual cue fusion, which may produce noisy coarse-grained visual cues for MNER. To accurately capture text-image relations and better refine multimodal representations, we propose a novel instruction-based Multi-granularity Cross-modality Generative framework for MNER, namely MCG-MNER. Concretely, we introduce a multi-granularity relation propagation to infer visual clues relevant to text. Then, we propose a method to jnject multi-granularity visual information into cross-modality interaction and fusion to learn a unified representation. Finally, we integrate task-specific instructions and answers for MCG-MNER. Comprehensive experimental results on three benchmark datasets, such as Twitter2015, Twitter2017 and WikiDiverse, demonstrate the superiority of our proposed method over several state-of-the-art MNER methods. We will publicly release our codes for future studies.