Large Language Models (LLMs) have demonstrated strong capabilities in transforming text descriptions or tables to data visualizations via instruction-tuning methods.However, it is not straightforward to apply these methods directly for a more real-world use case of visualizing data from long documents based on user-given intents, as opposed to the user pre-selecting the relevant content manually.We introduce the task of intent-based chart generation from documents: given a user-specified intent and document(s), the goal is to generate a chart adhering to the intent and grounded on the document(s) in a zero-shot setting.We propose an unsupervised, two-staged framework in which an LLM first extracts relevant information from the document(s) by decomposing the intent and iteratively validates and refines this data.Next, a heuristic-guided module selects an appropriate chart type before final code generation.To assess the data accuracy of the generated charts, we propose an attribution-based metric that uses a structured textual representation of charts, instead of relying on visual decoding metrics that often fail to capture the chart data effectively.To validate our approach, we curate a dataset comprising of 1,242 tuples from two domains, finance and scientific, in contrast to the existing datasets that are largely limited to parallel text descriptions/ tables and their corresponding charts.We compare our approach with baselines using single-shot chart generation using LLMs and query-based retrieval methods; our method outperforms by upto 9 points and 17 points in terms of chart data accuracy and chart type respectively over the best baselines.

Anirudh Phukan, Divyansh Divyansh, Harshit Kumar Morj, Vaishnavi Vaishnavi, Apoorv Saxena, Koustava Goswami. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2025.

Accurately attributing answer text to its source document is crucial for developing a reliable question-answering system.However, attribution for long documents remains largely unexplored.Post-hoc attribution systems are designed to map answer text back to the source document, yet the granularity of this mapping has not been addressed.Furthermore, a critical question arises: What exactly should be attributed?This involves identifying the specific information units within an answer that require grounding.In this paper, we propose and investigate a novel approach to the factual decomposition of generated answers for attribution, employing template-based in-context learning.To accomplish this, we utilize the question and integrate negative sampling during few-shot in-context learning for decomposition.This approach enhances the semantic understanding of both abstractive and extractive answers.We examine the impact of answer decomposition by providing a thorough examination of various attribution approaches, ranging from retrieval-based techniques to LLM-based attributors.

Generating presentation slides from a long document with multimodal elements such as text and images is an important task.This is time consuming and needs domain expertise if done manually.Existing approaches for generating a rich presentation from a document are often semi-automatic or only put a flat summary into the slides ignoring the importance of a good narrative.In this paper, we address this research gap by proposing a multi-staged end-to-end model which uses a combination of LLM and VLM.We have experimentally shown that compared to applying LLMs directly with state-ofthe-art prompting, our proposed multi-staged solution is better in terms of automated metrics and human evaluation.

Aditya Sharma, Apoorv Saxena, Chitrank Gupta, Mehran Kazemi, Partha Talukdar, Soumen Chakrabarti. Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics. 2023.

While recent developments in text-to-image generative models have led to a suite of high-performing methods capable of producing creative imagery from free-form text, there are several limitations. By analyzing the cross-attention representations of these models, we notice two key issues. First, for text prompts that contain multiple concepts, there is a significant amount of pixel-space overlap (i.e., same spatial regions) among pairs of different concepts This eventually leads to the model being unable to distinguish between the two concepts and one of them being ignored in the final generation. Next, while these models attempt to capture all such concepts during the beginning of denoising (e.g., first few steps) as evidenced by cross-attention maps, this knowledge is not retained by the end of denoising (e.g., last few steps). Such loss of knowledge eventually leads to inaccurate generation outputs.To address these issues, our key innovations include two test-time attention-based loss functions that substantially improve the performance of pretrained baseline text-to-image diffusion models. First, our attention segregation loss reduces the cross-attention overlap between attention maps of different concepts in the text prompt, thereby reducing the confusion/conflict among various concepts and the eventual capture of all concepts in the generated output. Next, our attention retention loss explicitly forces text-to-image diffusion models to retain cross-attention information for all concepts across all denoising time steps, thereby leading to reduced information loss and the preservation of all concepts in the generated output. We conduct extensive experiments with the proposed loss functions on a variety of text prompts and demonstrate they lead to generated images that are significantly semantically closer to the input text when compared to baseline text-to-image diffusion models.

Knowledge graph embedding (KGE) models represent each entity and relation of a knowledge graph (KG) with low-dimensional embedding vectors.These methods have recently been applied to KG link prediction and question answering over incomplete KGs (KGQA).KGEs typically create an embedding for each entity in the graph, which results in large model sizes on real-world graphs with millions of entities.For downstream tasks these atomic entity representations often need to be integrated into a multi stage pipeline, limiting their utility.We show that an off-the-shelf encoder-decoder Transformer model can serve as a scalable and versatile KGE model obtaining state-of-the-art results for KG link prediction and incomplete KG question answering.We achieve this by posing KG link prediction as a sequence-tosequence task and exchange the triple scoring approach taken by prior KGE methods with autoregressive decoding.Such a simple but powerful method reduces the model size up to 98% compared to conventional KGE models while keeping inference time tractable.After finetuning this model on the task of KGQA over incomplete KGs, our approach outperforms baselines on multiple large-scale datasets without extensive hyperparameter tuning. 1

Apoorv Saxena, Soumen Chakrabarti, Partha Talukdar. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.

Knowledge Graphs (KG) are multi-relational graphs consisting of entities as nodes and relations among them as typed edges.Goal of the Question Answering over KG (KGQA) task is to answer natural language queries posed over the KG.Multi-hop KGQA requires reasoning over multiple edges of the KG to arrive at the right answer.KGs are often incomplete with many missing links, posing additional challenges for KGQA, especially for multi-hop KGQA.Recent research on multihop KGQA has attempted to handle KG sparsity using relevant external text, which isn't always readily available.In a separate line of research, KG embedding methods have been proposed to reduce KG sparsity by performing missing link prediction.Such KG embedding methods, even though highly relevant, have not been explored for multi-hop KGQA so far.We fill this gap in this paper and propose EmbedKGQA.EmbedKGQA is particularly effective in performing multi-hop KGQA over sparse KGs.EmbedKGQA also relaxes the requirement of answer selection from a prespecified neighborhood, a sub-optimal constraint enforced by previous multi-hop KGQA methods.Through extensive experiments on multiple benchmark datasets, we demonstrate EmbedKGQA's effectiveness over other stateof-the-art baselines.