Most recommendation benchmarks evaluate how well a model imitates user behavior. In financial advisory, however, observed actions can be noisy or short-sighted under market volatility and may conflict with a user's long-term goals. Treating what users chose as the sole ground truth, therefore, conflates behavioral imitation with decision quality. We introduce Conv-FinRe, a conversational and longitudinal benchmark for stock recommendation that evaluates LLMs beyond behavior matching. Given an onboarding interview, step-wise market context, and advisory dialogues, models must generate rankings over a fixed investment horizon. Crucially, Conv-FinRe provides multi-view references that distinguish descriptive behavior from normative utility grounded in investor-specific risk preferences, enabling diagnosis of whether an LLM follows rational analysis, mimics user noise, or is driven by market momentum. We build the benchmark from real market data and human decision trajectories, instantiate controlled advisory conversations, and evaluate a suite of state-of-the-art LLMs. Results reveal a persistent tension between rational decision quality and behavioral alignment: models that perform well on utility-based ranking often fail to match user choices, whereas behaviorally aligned models can overfit short-term noise. The dataset is publicly released on Hugging Face. https://huggingface.co/collections/TheFinAI/conv-finre, and the codebase is available on GitHub. https://github.com/The-FinAI/Conv-FinRe.

Eftekhar Hossain, Sanjeev Kumar Sinha, Naman Bansal, R. Alexander Knipper, Souvika Sarkar, John Salvador, Yash Mahajan, Sri Ram Pavan Kumar Guttikonda, Mousumi Akter, Md. Mahadi Hassan, Matthew Freestone, Matthew C. Williams Jr., Dongji Feng, Santu Karmaker. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2025.

In this paper, we conducted a comprehensive study with the latest Sentence Encoders and Large Language Models (LLMs) on the challenging task of "definition-wild zero-shot topic inference", where users define or provide the topics of interest in real-time.Through extensive experimentation on seven diverse data sets, we observed that LLMs, such as ChatGPT-3.5 and PaLM, demonstrated superior generality compared to other LLMs, e.g., BLOOM and GPT-NeoX.Furthermore, Sentence-BERT, a BERT-based classical sentence encoder, outperformed PaLM and achieved performance comparable to ChatGPT-3.5.

The COVID-19 pandemic has affected millions of people worldwide with severe health, economic, social, and political implications. Healthcare Policy Makers (HPMs) and medical experts are at the core of responding to this continuously evolving pandemic situation and are working hard to contain the spread and severity of this relatively unknown virus. Biomedical researchers are continually discovering new information about this virus and communicating the findings through scientific articles. As such, it is crucial for HPMs and funding agencies to monitor the COVID-19 research trend globally on a regular basis. However, given the influx of biomedical research articles, monitoring COVID-19 research trends has become more challenging than ever, especially when HPMs want on-demand guided search techniques with a set of topics of interest in mind. Unfortunately, existing topic trend modeling techniques are unable to serve this purpose as (1) traditional topic models are unsupervised, and (2) HPMs in different regions may have different topics of interest that they want to track. To address this problem, we introduce a novel computational task in this article calledAd-Hoc Topic Tracking, which is essentially a combination ofzero-shottopic categorization and the spatio-temporal analysis task. We then propose multiplezero-shotclassification methods to solve this task by building on state-of-the-art language understanding techniques. Next, we picked the best-performing method based on its accuracy on a separate validation dataset and then applied it to a corpus of recent biomedical research articles to track COVID-19 research endeavors across the globe using a spatio-temporal analysis. A demo website has also been developed for HPMs to create custom spatio-temporal visualizations of COVID-19 research trends. The research outcomes demonstrate that the proposedzero-shotclassification methods can potentially facilitate further research on this important subject matter. At the same time, the spatio-temporal visualization tool will greatly assist HPMs and funding agencies in making well-informed policy decisions for advancing scientific research efforts.