The rapid evolution of Natural Language Processing (NLP) has favoured major languages such as English, leaving a significant gap for many others due to limited resources. This is especially evident in the context of data annotation, a task whose importance cannot be underestimated, but which is time-consuming and costly. Thus, any dataset for resource-poor languages is precious, in particular when it is task-specific. Here, we explore the feasibility of repurposing an existing multilingual dataset for a new NLP task: we repurpose a subset of the BELEBELE dataset (Bandarkar et al., 2023), which was designed for multiple-choice question answering (MCQA), to enable the more practical task of extractive QA (EQA) in the style of machine reading comprehension. We present annotation guidelines and a parallel EQA dataset for English and Modern Standard Arabic (MSA). We also present QA evaluation results for several monolingual and cross-lingual QA pairs including English, MSA, and five Arabic dialects. We aim to help others adapt our approach for the remaining 120 BELEBELE language variants, many of which are deemed under-resourced. We also provide a thorough analysis and share insights to deepen understanding of the challenges and opportunities in NLP task reformulation.

Large Language Models (LLMs) have demonstrated remarkable planning abilities across various domains, including robotic manipulation and navigation. While recent work in robotics deploys LLMs for high-level and low-level planning, existing methods often face challenges with failure recovery and suffer from hallucinations in long-horizon tasks. To address these limitations, we propose a novel multi-agent LLM framework, Multi-Agent Large Language Model for Manipulation (MALMM). Notably, MALMM distributes planning across three specialized LLM agents, namely high-level planning agent, low-level control agent, and a supervisor agent. Moreover, by incorporating environment observations after each step, our framework effectively handles intermediate failures and enables adaptive re-planning. Unlike existing methods, MALMM does not rely on pre-trained skill policies or in-context learning examples and generalizes to unseen tasks. In our experiments, MALMM demonstrates excellent performance in solving previously unseen long-horizon manipulation tasks, and outperforms existing zero-shot LLM-based methods in RLBench by a large margin. Experiments with the Franka robot arm further validate our approach in real-world settings.

Rocktim Das, Simeon Hristov, Haonan Li, Dimitar Dimitrov, Ivan Koychev, Preslav Nakov. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024.

End-to-end Task-Oriented Dialog (TOD) systems typically require extensive training datasets to perform well.In contrast, large language model (LLM) based TOD systems can excel even with limited data due to their ability to learn tasks through in-context exemplars.However, these models lack alignment with the style of responses in training data and often generate comprehensive responses, making it difficult for users to grasp the information quickly.In response, we propose SyncTOD that synergizes LLMs with task-specific hints to improve alignment in low-data settings.Sync-TOD employs small auxiliary models to provide hints and select exemplars for in-context prompts.With ChatGPT, SyncTOD achieves superior performance compared to LLM-based baselines and SoTA models in low-data settings, while retaining competitive performance in full-data settings.

Medical task-oriented dialogue systems can assist doctors by collecting patient medical history, aiding in diagnosis, or guiding treatment selection, thereby reducing doctor burnout and expanding access to medical services.However, doctor-patient dialogue datasets are not readily available, primarily due to privacy regulations.Moreover, existing datasets lack comprehensive annotations involving medical slots and their different attributes, such as symptoms and their onset, progression, and severity.These comprehensive annotations are crucial for accurate diagnosis.Finally, most existing datasets are non-English, limiting their utility for the larger research community.In response, we introduce MediTOD, a new dataset of doctor-patient dialogues in English for the medical history-taking task.Collaborating with doctors, we devise a questionnairebased labeling scheme tailored to the medical domain.Then, medical professionals create the dataset with high-quality comprehensive annotations, capturing medical slots and their attributes.We establish benchmarks in supervised and few-shot settings on MediTOD for natural language understanding, policy learning, and natural language generation subtasks, evaluating models from both TOD and biomedical domains.We release MediTOD resources for future research.* Work done when authors were at IIT Delhi.[{"intent": "salutations"}] Doctor: How may I help you?[{"intent": "inform", "slots": { "positive_symptom": [ {"value": "pharyngitis", "onset": "past four days"}, {"value": "fever", "onset": "last two days"} ]}}] Patient: Yes, I just came in here today.I I've just been.Really getting like the soreness in my throat for the past, I would say four days and I also had a fever for the last two days as well. CMAS Key-Value Legends:[{"intent": "inform", "slots": { "positive_symptom": "pharyngitis", "positive_symptom": "fever", "onset": "past four days", "onset": "last two days" }] Datasets Language Annotations #utterances/ #utterancesAll TOD Tasks Comprehensive Canonicalized dialogue

Yuxia Wang, Minghan Wang, Muhammad Arslan Manzoor, Fei Liu, Georgi Nenkov Georgiev, Rocktim Jyoti Das, Preslav Nakov. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. 2024.

We systematically study how three large language models with code capabilities -CodeT5, Codex, and ChatGPT -generalize to out-ofdomain data.We consider two fundamental applications -code summarization, and code generation.We split data into domains following its natural boundaries -by an organization, by a project, and by a module within the software project.We establish that samples from each new domain present all the models with a significant challenge of distribution shift.We study how established methods adapt models to better generalize to new domains.Our experiments show that while multitask learning alone is a reasonable baseline, combining it with few-shot finetuning on examples retrieved from training data can achieve very strong performance.Moreover, this solution can outperform direct finetuning for very low-data scenarios.Finally, we consider variations of this approach to create a more broadly applicable method to adapt to multiple domains at once.We find that for code generation, a model adapted to multiple domains simultaneously performs on par with those adapted to a single domain 1 .