EVOLVE provides a thorough exploration of the methodology and strategic approach employed in appraising text-to-sql for enterprise-scale applications. It delves into our unique tactics, evaluative measures and iterative enhancement processes aimed at steadily augmenting text-to-sql’s proficiency for robust SQL generation. With the advent of big data, the complexity and volume of database queries have exponentially increased. Navigating this intricate landscape, text-to-sql revolutionizes the querying process by enabling Non-SQL experts to use natural language queries. This ease of accessibility, however, underscores the criticality of accurately evaluating these queries to ensure their correctness and efficacy. Thus, our study emphasizes the development of appropriate validation metrics, which are essential in handling the complexities of big data queries. By focusing on these novel methodologies and metrics, we present a holistic view of our rigorous evaluation approach of text-to-sql.

This study introduces T2I-RISE, an innovative approach aimed at enabling natural language querying of databases, which could transform data-driven decision making. Currently, database insights are mostly generated by a limited group of experts. Business Intelligence tools provide some insights, but these are often static. We propose a comprehensive solution tailored for enterprise applications to bridge the gap between user needs and existing capabilities. Despite the industry’s increasing use of Text2SQL powered by Large Language Models (LLM), implementation challenges remain, particularly in managing large-scale database schema representations. Our study explores the critical elements needed for user intent comprehension and the generation of corresponding SQL and insights. T2I-RISE offers a comprehensive framework and thorough analysis of these components, including our unique optimization strategies.

This paper presents our contribution to the Financial Document Causality Detection (FinCausal) task, a component of the FNP-2023 workshop. The FinCausal challenge centers on the extraction of cause-and-effect relationships from financial texts written in both English and Spanish. Recent advancements in Generative AI and Large Language Models (LLMs) have instigated investigations into their reasoning abilities, propelling our exploration of LLMs’ potential for causal reasoning within the financial domain. This study also ventures into the domain of non-English languages, aiming to uncover the capacity of LLMs on this front as well. Our investigation revealed that LLMs exhibit a remarkable ability to identify causal relationships, particularly when provided with few task-specific relevant examples. Additionally, our research demonstrates the effectiveness of LLMs in processing non-English languages when given the same English prompts along with language comprehension instructions. We conducted a comparative analysis between OpenAI GPT3.5 and 4, concluding that GPT-4 model is better-suited for this purpose. Our study unveils that LLMs yield semantically similar cause and effects. This discovery highlights LLMs don’t rely solely on content for the predictions and so the necessity of adopting an evaluation approach for this task, one that emphasizes also on semantic similarity metrics.

This paper presents our submission to the Financial Narrative Summarization (FNS) task at the FNP-2023 workshop. The FNS task involves the generation of concise summaries, not exceeding 1000 words, for annual financial reports composed in English, Spanish, and Greek. In our prior work, presented in FNP-2022, we introduced DiMSum [1], a novel framework designed to automatically identify crucial narrative sections within financial reports and quantify their weighted contributions. The field of Generative AI and Large Language Models (LLMs) has recently witnessed significant advancements, prompting us to explore their utility in summarizing financial reports. Our investigation revealed that LLMs, when left to their own devices often struggle to effectively summarize complex financial documents, necessitating external guidance. In this study, we demonstrate how LLMs, when guided by the DiMSum framework, exhibit substantial improvements in the quality of financial report summarization. To the best of our knowledge, this research marks the first instance of applying LLMs to the FNS task, offering a novel approach to enhancing the summarization of financial reports.