High-quality datasets are essential for building effective task-oriented dialogue (TOD) systems.The existing TOD datasets often present overly simplified interactions, where users incrementally express straightforward requests that can be managed with basic slot-value style dialogue states, such as "hotel-area = east."However, this approach does not reflect reallife scenarios in which users may express complex constraints and preferences.To address this gap, in this paper, we propose SQLWOZ, a novel TOD dataset designed to capture complex, real-world user requirements.The user requirements in SQLWOZ include the four categories: 1) multiple values for a slot, 2) excluded values within a slot, 3) preferred or prioritized values, and 4) conditional values based on other conditions.We utilize SQL statements as a formalized and expressive representation of dialogue states within SQLWOZ.To evaluate the dataset, we adapt large language models as dialogue agents and conduct extensive experiments on the SQL-based dialogue state tracking, dialogue response generation, and end-to-end TOD tasks.The experimental results demonstrate the complexity and quality of SQLWOZ, establishing it as a new benchmark for advancing TOD research.

Task-oriented dialogue (TOD) systems play a vital role in numerous assistance and service scenarios, significantly improving people’s daily lives. Conventionally, a TOD system adheres to a fixed paradigm, where it must first extract user goals and query external databases before it can generate the final response. However, this fixed extract-and-query paradigm is not always optimal for all dialogue turns, which is redundant for the simple turns that do not need external information, and is inadequate for the complex turns that need to interact with the external world multiple times. To address the limitations, in this article, we propose AgentTOD, a novel TOD framework that uses a large language model (LLM) as the intelligent agent to achieve a flexible dialogue paradigm. AgentTOD deprecates the traditional modular architecture (including dialogue state tracking and dialogue policy) by utilizing an LLM as the controller brain to determine when and how to call the provided APIs to obtain external information. It can choose to call APIs any number of times with various parameters until it’s enough to reply to the user. Besides, to train AgentTOD, we construct a large and comprehensive TOD dataset, called TrajsTOD (Trajectories of TODs), which consists of 66k+ user-agent dialogue trajectories converted from eight popular TOD datasets covering 60 domains. TrajsTOD is constructed with minimal dialogue annotations where only the API calling logs are needed and can empower AgentTOD with the general ability to call APIs and generate responses according to the task definition. Extensive experimental results on the MultiWOZ-series and SGD datasets demonstrate AgentTOD has superior performance on TODs as well as a superior adaptability to new task scenarios.

Task-oriented dialogue (TOD) systems are predominantly designed to be composed of several functional modules (e.g.dialogue state tracker, dialogue policy, natural language generation) whether they are pipeline or end-to-end architectures.However, this modular design not only heavily relies on massive fully-annotated data, but also suffers from many intrinsic drawbacks, such as serious error accumulation, poor generalization ability, high customization cost, and low fault tolerance rate.In this paper, we rethink the architecture of the task-oriented dialogue systems and propose a novel fully zeroshot autonomous TOD agent, named AutoTOD, where all the delicate modules in traditional TOD systems are deprecated and all it needs is a general-purpose instruction-following language model (e.g.GPT-4).AutoTOD only leverages a simple instruction schema consisting of the description of tasks and external APIs, and can autonomously decide what to do at each dialogue turn, including asking for information, calling APIs, summarizing API results, and correcting previous mistakes.Moreover, we propose a simulation-based evaluation framework to better validate the abilities of TOD models in real-life scenarios.Extensive experiments conducted on the MultiWOZ and SGD datasets show the superior task completion ability and flexible language skills of AutoTOD. 1