Shaona Ghosh, Prasoon Varshney, Makesh Narsimhan Sreedhar, Aishwarya Padmakumar, Traian Rebedea, Jibin Rajan Varghese, Christopher Parisien. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2025.

Dialogue policies play a crucial role in developing task-oriented dialogue systems, yet their development and maintenance are challenging and typically require substantial effort from experts in dialogue modeling.While in many situations, large amounts of conversational data are available for the task at hand, people lack an effective solution able to extract dialogue policies from this data.In this paper, we address this gap by first illustrating how Large Language Models (LLMs) can be instrumental in extracting dialogue policies from datasets, through the conversion of conversations into a unified intermediate representation consisting of canonical forms.We then propose a novel method for generating dialogue policies utilizing a controllable and interpretable graphbased methodology.By combining canonical forms across conversations into a flow network, we find that running graph traversal algorithms helps in extracting dialogue flows.These flows are a better representation of the underlying interactions than flows extracted by prompting LLMs.Our technique focuses on giving conversation designers greater control, offering a productivity tool to improve the process of developing dialogue policies.1 bot "Hello, how can I help you today?" intent: express greeting and offer to help user "I received this pair of boots but they are scuffed" intent: inform product received is scuffed user "I want to return and exchange them for a new pair" intent: request to return and exchange product bot "No worries, can I have your full name?"intent: ask for full name user "Alessandro Phoenix" intent: provide full name bot "And your username, email, and order ID?" intent: ask for username, email, and order id ... bot "Well then have a nice day!" intent: say goodbye bot "Hello, how can I help you today?" intent: express greeting and offer to help user "I received this pair of

Zhilin Wang, Yi Dong, Jiaqi Zeng, Virginia Adams, Makesh Narsimhan Sreedhar, Daniel Egert, Olivier Delalleau, Jane Scowcroft, Neel Kant, Aidan Swope, Oleksii Kuchaiev. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2024.

High-quality preference datasets are essential for training reward models that can effectively guide large language models (LLMs) in generating high-quality responses aligned with human preferences.As LLMs become stronger and better aligned, permissively licensed preference datasets, such as Open Assistant, HH-RLHF, and HelpSteer need to be updated to remain effective for reward modeling.Methods that distil preference data from proprietary LLMs such as GPT-4 have restrictions on commercial usage imposed by model providers.To improve upon both generated responses and attribute labeling quality, we release HelpSteer2, a permissively licensed preference dataset (CC-BY-4.0). Using a powerful Nemotron-4-340B base model trained on HelpSteer2, we are able to achieve the SOTA score (92.0%) on Reward-Bench's primary dataset, outperforming currently listed open and proprietary models, as of June 12th, 2024.Notably, HelpSteer2 consists of only ten thousand response pairs, an order of magnitude fewer than existing preference datasets (e.g., HH-RLHF), which makes it highly efficient for training reward models. Our extensive experiments demonstrate that reward models trained with HelpSteer2 are effective in aligning LLMs. Additionally, we propose SteerLM 2.0, a model alignment approach that can effectively make use of the rich multi-attribute score predicted by our reward models. HelpSteer2 is available at https://huggingface.co/datasets/nvidia/HelpSteer2 and code is available at https://github.com/NVIDIA/NeMo-Aligner

Recent generative approaches for multi-hop question answering (QA) utilize the fusion-indecoder method (Izacard and Grave, 2021) to generate a single sequence output which includes both a final answer and a reasoning path taken to arrive at that answer, such as passage titles and key facts from those passages.While such models can lead to better interpretability and high quantitative scores, they often have difficulty accurately identifying the passages corresponding to key entities in the context, resulting in incorrect passage hops and a lack of faithfulness in the reasoning path.To address this, we propose a single-sequence prediction method over a local reasoning graph (SEQGRAPH) 1 that integrates a graph structure connecting key entities in each context passage to relevant subsequent passages for each question.We use a graph neural network to encode this graph structure and fuse the resulting representations into the entity representations of the model.Our experiments show significant improvements in answer exact-match/F1 scores and faithfulness of grounding in the reasoning path on the HotpotQA dataset and achieve stateof-the-art numbers on the Musique dataset with only up to a 4% increase in model parameters.

Subword tokenization schemes are the dominant technique used in current NLP models.However, such schemes can be rigid and tokenizers built on one corpus may not adapt well to other parallel corpora.It has also been observed that in multilingual corpora, subword tokenization schemes oversegment low-resource languages, leading to a drop in translation performance.An alternative to subword tokenizers is byte-based tokenization, i.e., tokenization into byte sequences using the UTF-8 encoding scheme.Byte tokens often represent inputs at a sub-character granularity, i.e., one character can be represented by a span of byte tokens.This results in much longer byte sequences that are hard to interpret without aggregating local information from multiple byte tokens.In this paper, we propose a Local Byte Fusion (LOBEF) method for byte-based machine translation-utilizing byte n-gram and word boundaries-to aggregate local semantic information.Extensive experiments on multilingual translation, zero-shot cross-lingual transfer, and domain adaptation reveal a consistent improvement over vanilla byte-based models.Further analysis also indicates that our byte-based models are parameter-efficient and perform competitive to subword models.