Emotional Support Conversation (ESC) aims to alleviate individuals’ negative emotions through multi-turn dialogues, where effective strategy planning and response generation are essential. However, existing methods often suffer from limitations in both planning reasonable support strategies and effectively expressing them in responses. To the end, we propose a novel LLM-based Emotional Support Conversation Agent (ESCA) with a plug-in strategy planner and a strategy-aligned prompt generator. The strategy planner cooperates with four aspects of the seeker’s state, including emotion intensity, trust degree, dialogue behavior, and stage of change, to enhance the rationality and effectiveness of the strategy prediction. To ensure that predicted strategies are better conveyed, the prompt generator integrates strategy-aligned instructions, knowledge, and context to generate the soft prompt for guiding the LLM to generate supportive responses. In addition to supervised fine-tuning, the prompt generator is further optimized by reinforcement learning. Experimental results demonstrate that ESCA significantly improves both response quality and the success rate of achieving the ESC task goal.

Jiayi Han, Liang Du, Hongwei Du, Xiangguo Zhou, Yiwen Wu, Yuanfang Zhang, Weibo Zheng, Donghong Han. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2025.

Open-domain multi-modal dialogue system heavily relies on visual information to generate contextually relevant responses. The existing open-domain multi-modal dialog generation methods ignore the complementary relationship between multiple modalities, and are difficult to integrate with LLMs. To tackle these challenges, we introduce AutoGraph, an innovative method for constructing visual context graphs automatically. We aim to structure complex information and seamlessly integrate it with large language models (LLMs), aligning information from multiple modalities at both semantic and structural levels. Specifically, we fully connect the text graphs and scene graphs, and then trim unnecessary edges via LLMs to automatically construct a visual context graph. Next, we design several graph sampling grammar for the first time to convert graph structures into sequence which is suitable for LLMs. Finally, we propose a two-stage fine-tuning strategy to allow LLMs to understand graph sampling grammar and generate responses. We validate our proposed method on text-based LLMs, and visual-based LLMs, respectively. Experimental results show that our proposed method achieves state-of-the-art performance on multiple public datasets.

Facial expression recognition (FER) remains a challenging task despite years of effort because of the variations in view angles and human poses and the exclusion of expression-relevant facial parts. In this work, we propose to learn effective Global receptive field and Class-sensitive metrics for FER, namely GCNet which contains a Class-sensitive metric learning module (CSMLM) and mobile dilation modules (MDMs). CSMLM fully takes advantage of the variation in human faces to extract class-sensitive and spatially consistent features to improve the effectiveness of FER. MDM utilizes cascaded dilation convolution layers to achieve a global receptive field. However, directly adding a dilation convolution layer to a given sequence of convolution layers may face the gridding problem, which leads to sparse feature maps. In this work, we find the upper bound of the dilation rate of the additional convolution layer that avoids the gridding problem. Experiments show that the proposed approach reaches state-of-the-art (SOTA) performance on the RAF-DB, FER-Plus, and SFEW2.0 datasets.