Accurately evaluating the word sense disambiguation (WSD) capabilities of large language models (LLMs) remains challenging, as existing studies primarily rely on single-task evaluations and classification-based metrics that overlook the fundamental differences between generative LLMs and traditional classification models.To bridge this gap, we propose RoDEval, the first comprehensive evaluation framework specifically tailored for assessing LLMbased WSD methods.RoDEval introduces four novel metrics: Disambiguation Scope, Disambiguation Robustness, Disambiguation Reliability, and Definition Generation Quality Score, enabling a multifaceted evaluation of LLMs' WSD capabilities.Experimental results using RoDEval across five mainstream LLMs uncover significant limitations in their WSD performance.Specifically, incorrect definition selections in multiple-choice WSD tasks stem not from simple neglect or forget of correct options, but rather from incomplete acquisition of the all senses for polysemous words.Instead, disambiguation reliability is often compromised by the models' persistent overconfidence.In addition, inherent biases continue to affect performance, and scaling up model parameters alone fails to meaningfully enhance their ability to generate accurate sense definitions.These findings provide actionable insights for enhancing LLMs' WSD capabilities.The source code and evaluation scripts are open-sourced at https: //github.com/DayDream405/RoDEval.

Medication recommendation aims to suggest personalized drug combinations to patients based on their longitudinal medical histories stored in electronic health record (EHR) datasets. Patients' Dynamic Treatment Regimes (DTRs) determine how patients' drug combinations change along with the evolution of disease treatment. DTRs are effective for comprehending disease-treatment dynamics and for recommending a timely and personalized combination of medications for patients. However, existing medication recommender systems (MRSs) overlook the multiple treatment pathways generated by the intervention of DTRs and can only recommend a single treatment paradigm, ignoring the fact that patients may be at different treatment stages and thus require different treatment regime. Such disregard leads to a significant limitation in recommending personalized medication combinations tailored to different treatment stages, yielding greatly compromised accuracy and applicability of MRSs. Moreover, existing methods often overlook the time interval information over patients' successive visits, which is critical to indicate patients' treatment evolution. To address these significant gaps, we propose a Time-aware Medication Recommendation Framework via Intervention of Dynamic Treatment Regimes, called MR-DTR. To explicitly illustrate the intervention processes of DTRs on similar patients, we employ a co-guided graph to connect various patient sequences. In addition, to fully utilize the time interval information, we design a time-aware guidance mechanism dedicated to the co-guided graph to efficiently learn medication representation using the patient's guidance information. We also introduce relative time intervals in the encoder to act as positional information. Extensive experiments on two real-world datasets demonstrate that MR-DTR surpasses state-of-the-art models in terms of recommendation performance. Our code is available at: https://github.com/liyifo/MR-DTR.

The goal of medication recommendation system is to recommend appropriate pharmaceutical interventions based on a patient’s diagnosis. Most of existing approaches often formulate these recommendations use data on diagnoses, procedures, and prescriptions accumulated in the electronic health records (EHR), and despite the great successes, they seem to have limitations on modelling the significance of medication to a patient’s current visit and mining fine-grained medication representation information. To address these issues, we propose a novel Significanceaware Medication Recommendation (SMRec) framework built on significance of medication to patients and fine-grained medication representation learning. Specifically, we first design a encoding mechanism to compute significance information of medications for each patient’s visit. Then, we utilize the set-level medication co-occurrence graph based on patients’ medical history which integrates temporal dependency to learn fine-grained medication representations. Experimental results on the publicly available MIMIC-III dataset demonstrate the superior effectiveness of our model compared to other approaches1.