We sincerely thank Cheng for the correspondence1 regarding our MMRAG2 work and for the insightful thoughts. We welcome the opportunity to clarify our methodological choices and to address the concerns raised. For completeness, we begin with a brief summary of the original work, followed by a detailed response to the correspondence. In our previous work, we introduced MMRAG,2 a multi-mode retrieval-augmented framework for biomedical in-context learning that supports four example-selection modes (Random, Top, Diversity, and Class). We evaluated these modes across representative biomedical NLP tasks, including named entity recognition, relation extraction, and text classification, and examined the impact of different retrievers under comparable prompting settings. Overall, our results showed that retrieval-guided example selection, particularly Top and Diversity modes, can substantially improve few-shot prompting performance compared with Random selection under limited context budgets. Below, we respond to the key points raised in Cheng’s correspondence. First, as suggested by Cheng, we now report precision and recall in Figure 1A. The results clearly show that both Top and Diversity achieve consistently higher precision and recall. This supports the conclusion that Top and Diversity outperform the other modes on the DDI dataset, consistent with our original findings in Figure 6A and B of the MMRAG paper.2

Huixue Zhou, Hengrui Gu, Zaifu Zhan, Xi Liu, Kaixiong Zhou, Yongkang Xiao, Mingfu Liang, Srinivas Prasad Govindan, Piyush Chawla, Jiyan Yang, Xiangfei Meng, Huayu Li, Buyun Zhang, Liang Luo, Wen-Yen Chen, Yiping Han, Bo Long, Rui Zhang, Tianlong Chen. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025.

OBJECTIVES: To optimize in-context learning in biomedical natural language processing by improving example selection. MATERIALS AND METHODS: We introduce a novel multi-mode retrieval-augmented generation (MMRAG) framework, which integrates 4 retrieval strategies: (1) Random Mode, selecting examples arbitrarily; (2) Top Mode, retrieving the most relevant examples based on similarity; (3) Diversity Mode, ensuring variation in selected examples; and (4) Class Mode, selecting category-representative examples. This study evaluates MMRAG on 3 core biomedical NLP tasks: Named Entity Recognition (NER), Relation Extraction (RE), and Text Classification (TC). The datasets used include BC2GM for gene and protein mention recognition (NER), DDI for drug-drug interaction extraction (RE), GIT for general biomedical information extraction (RE), and HealthAdvice for health-related text classification (TC). The framework is tested with 2 large language models (Llama-2-7B and Llama-3-8B) and 3 retrievers (Contriever, MedCPT, and BGE-Large) to assess performance across different retrieval strategies. RESULTS: The results from the Random Mode indicate that providing more examples in the prompt improves the model's generation performance. Meanwhile, Top Mode and Diversity Mode significantly outperform Random Mode on the RE (DDI) task, achieving an F1 score of 0.9669-a 26.4% improvement. Among the 3 retrievers tested, Contriever outperformed the other 2 in a greater number of experiments. Additionally, Llama 2 and Llama 3 demonstrated varying capabilities across different tasks, with Llama 3 showing a clear advantage in handling NER tasks. CONCLUSION: MMRAG effectively enhances biomedical in-context learning by refining example selection, mitigating data scarcity issues, and demonstrating superior adaptability for NLP-driven healthcare applications.

OBJECTIVE: To develop an advanced multi-task large language model (LLM) framework for extracting diverse types of information about dietary supplements (DSs) from clinical records. METHODS: We focused on 4 core DS information extraction tasks: named entity recognition (2 949 clinical sentences), relation extraction (4 892 sentences), triple extraction (2 949 sentences), and usage classification (2 460 sentences). To address these tasks, we introduced the retrieval-augmented multi-task information extraction (RAMIE) framework, which incorporates: (1) instruction fine-tuning with task-specific prompts; (2) multi-task training of LLMs to enhance storage efficiency and reduce training costs; and (3) retrieval-augmented generation, which retrieves similar examples from the training set to improve task performance. We compared the performance of RAMIE to LLMs with instruction fine-tuning alone and conducted an ablation study to evaluate the individual contributions of multi-task learning and retrieval-augmented generation to overall performance improvements. RESULTS: Using the RAMIE framework, Llama2-13B achieved an F1 score of 87.39 on the named entity recognition task, reflecting a 3.51% improvement. It also excelled in the relation extraction task with an F1 score of 93.74, a 1.15% improvement. For the triple extraction task, Llama2-7B achieved an F1 score of 79.45, representing a significant 14.26% improvement. MedAlpaca-7B delivered the highest F1 score of 93.45 on the usage classification task, with a 0.94% improvement. The ablation study highlighted that while multi-task learning improved efficiency with a minor trade-off in performance, the inclusion of retrieval-augmented generation significantly enhanced overall accuracy across tasks. CONCLUSION: The RAMIE framework demonstrates substantial improvements in multi-task information extraction for DS-related data from clinical records.