The mismatch between the growing demand for psychological counseling and the limited availability of services has motivated research into the application of Large Language Models (LLMs) in this domain. Consequently, there is a need for a robust and unified benchmark to assess the counseling competence of various LLMs. Existing works, however, are limited by unprofessional client simulation, static question-and-answer evaluation formats, and unidimensional metrics. These limitations hinder their effectiveness in assessing a model's comprehensive ability to handle diverse and complex clients. To address this gap, we introduce CARE-Bench, a dynamic and interactive automated benchmark. It is built upon diverse client profiles derived from real-world counseling cases and simulated according to expert guidelines. CARE-Bench provides a multidimensional performance evaluation grounded in established psychological scales. Using CARE-Bench, we evaluate several general-purpose LLMs and specialized counseling models, revealing their current limitations. In collaboration with psychologists, we conduct a detailed analysis of the reasons for LLMs' failures when interacting with clients of different types, which provides directions for developing more comprehensive, universal, and effective counseling models.

Detecting depression through users’ social media posting history is crucial for enabling timely intervention; however, irrelevant content within these posts negatively impacts detection performance. Thus, it is crucial to extract pertinent content from users’ complex posting history. Current methods utilize frozen screening models, which can miss critical information and limit overall performance due to isolated screening and detection processes. To address these limitations, we propose E2-LPS End-to-End Learnable Psychiatric Scale Guided Risky Post Screening Model) for jointly training our screening model, guided by psychiatric scales, alongside the detection model. We employ a straight-through estimator to enable a learnable end-to-end screening process and avoid the non-differentiability of the screening process. Experimental results show that E2-LPS outperforms several strong baseline methods, and qualitative analysis confirms that it better captures users’ mental states than others.

As an increasing number of users share their lives and mental states on social media, many studies attempt to detect depression risk through social media videos using non-verbal cues like facial expressions, posture, gaze, and intonation from individual social media platforms, a proven effective field. However, these studies have focused on single-video level analysis to detect depression. These researches fail to capture the dynamic nature of social media streams and the complex, often gradual manifestation of depression. This limitation overlooks the comprehensive mental state of users, which can only be understood through their extended video histories. To address this, we introduce the Multimodal User-level Depression Detection Dataset (MUD3). MUD3 includes the long-term video histories of depressed users on social media platforms, containing user mental states across multiple videos and treating the video histories as a continuous social media stream. This allows us to model multiple videos at the user-level and analyze users' long-term mental states. MUD3 and supplementary materials are available at https://github.com/Syx1030/MUD3.

Bichen Wang, Yuzhe Zi, Yixin Sun, Yanyan Zhao, Bing Qin. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2025.

Mental health issues pose significant challenges worldwide, leading researchers to utilize computing techniques to analyze social media posts for early detection. Currently, Large Language Models (LLMs) represent advanced AI systems with exceptional language processing capabilities. However, their effectiveness depends on the use of appropriate prompts. The design of suitable prompts specifically for analyzing mental health issues remains underexplored. Drawing inspiration from psychiatrists who use psychiatric scales to assess mental health, we have developed scale-based chains of thought (Scale-CoT) prompts. These not only enhance the LLMs’ ability to detect mental health issues but also increase clarity and explainability by integrating professional scales. Furthermore, we demonstrate that the knowledge generated by LLMs can be effectively transferred to smaller models, resulting in a better-performing and more lightweight system. Our approach, which leverages psychiatric scales, outperforms existing methods. Our experiments indicate that combining traditional psychiatry with computational techniques is worthwhile and deserving of further research and exploration.