Text-to-Vis is an emerging task in the data engineering and mining area that aims to automatically generate data visualizations from natural language questions (NLQs). Despite their progress, existing text-to-vis models often heavily rely on lexical matching between words in the questions and tokens in data schemas. This overreliance on lexical matching may lead to a diminished level of model robustness against input variations. In this study, we thoroughly examine the robustness of current text-to-vis models, an area that has not previously been explored. In particular, we construct the first robustness dataset nvBench-Rob, which contains diverse lexical and phrasal variations based on the original text-to-vis benchmark nvBench. Then, we found that the performance of existing text-to-vis models on this new dataset dramatically drops, implying that these methods exhibit inadequate robustness overall. Finally, we propose a novel framework based on Retrieval-Augmented Generation (RAG) technique, named GRED, specifically designed to address input perturbations in these two variants. The framework consists of three parts: NLQ-Retrieval Generator, Visualization Query-Retrieval Retuner and Annotation-based Debugger, which are used to tackle the challenges posed by natural language variants, programming style differences and data schema variants, respectively. Extensive experimental evaluations show that, compared to the state-of-the-art model Prompt4Vis in the Text-to-Vis field, GRED performs better in terms of model robustness, with a 40% increase in accuracy on the proposed nvBench-Rob dataset.

Given a massive dataset, data visualization (DV) could efficiently express the insights and summaries behind the massive raw data by employing vivid visual representations. To create suitable DVs, users are required to get a comprehensive understanding of the raw data and then transfer their ideas into DVs by composing a suitable and accurate specification in some declarative visualization languages (DVLs, e.g., Vega-Lite). A specification is a JSON object defining the properties of the DVs, like the selected data, the transformations, the visual details, and so on. Due to its complicated grammar and details, DV has quite a steep learning curve, even for data analysts. In this paper, we propose a new task named FeVisQA, referring to Free-form Question Answering over data Visualizations. More specifically,-given a raw dataset, a related DV (in the form of a specification), and a question, FeVisQA aims to predict a textual answer automatically. As a particular case of the general CodeQA (i.e., QA over general programming code like Python and Java) task, FeVisQA enables people to better comprehend data and its DVs by conducting logical reasoning when answering these questions. Since FeVisQA has not been studied in the literature, we first construct a benchmark dataset containing 152 datasets, 14,406 DVs, and 83,890 QA pairs. To tackle this new task, we design a novel neural network named FeVisQANet with advanced multi-modal encoder and adaptive decoder structures, and we also design a novel multi-step framework called VisQA for Multi-modal Large Language Models (MLLMs) based on Retrieval-augmented Generation (RAG) technology. Extensive experiments on our constructed datasets validate the rationale and effectiveness of this proposed FeVisQA task and the proposed model. While research on QA over text and table, machine reading comprehension, and CodeQA develops rapidly, prior works have yet to draw attention to question-answering over DVs. This study connects two important subareas, QA from the natural language process area and DV from the data engineering area. We hope this new dataset and model can serve as a helpful benchmark that would benefit the development of both fields.

Question Answering (QA) systems playa vital role in knowledge acquisition. CodeQA refers to question answering (QA) over source code for code comprehension purpose. However, existing CodeQA studies mainly focus on questions related to general-purpose programming languages (GPLs) (e.g., Java and Python), and no study has been conducted on QA over declarative visualization languages (DVLs) (e.g., Vega-Lite), a kind of programming languages used for creating data visualization (DV). DVLs enjoys specific grammars that are instinct different from GPLs. This demonstration presents the first neural-based QA system for DVL, FeVisQASystem. FeVisQASystem is based on a new task named Fevisqa, short for Free-form QA over data Visualization, which takes natural language questions and DV specification as inputs to predict the answers to the questions. As a particular case of the CodeQA task, Fe VisQA enables people to better comprehend data and its DV s by conducting logical reasoning when answering these questions. Although research on question-answering and machine reading comprehension is progressing quickly, little attention has previously been paid to FeVisQA. This new system and the task can serve as a helpful pioneering study for DV comprehension. The video can be accessed via https://ldrv.ms/f/s!Ah2vhboIPBFMhk6jTYOtaIRnLC2K?e=OkJqOq