Jianqi Gao, Hang Yu, Jian Cao, Ranran Bu, Jinghua Tang, Nengjun Zhu, Yonggang Zhang. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.

Knowledge base question answering (KBQA) aims to answer natural language questions using large-scale knowledge bases (KBs). Among various KBQA approaches, semantic parsing-based (SP-based) methods have demonstrated strong effectiveness by generating concise logical forms (LFs) that capture complex subgraph structures and semantic information. Recent research suggests that integrating large language models (LLMs) with SP can achieve significant improvements in the performance and efficiency of KBQA by facilitating the direct generation of LFs with minimal retrieval. However, generating complete LFs with LLMs continues to pose a challenge due to the complexity of the required graph structures and constraints, leading to the significant issue of non-executability. To address these challenges, we propose GCA-KBQA, a step-wise fine-tuned LLM-based framework that employs hop-wise generation, knowledge-assisted calibration, and path-level assembly to construct complete LFs for KBQA. Specifically, we decompose the complex SP process into manageable steps: first, we iteratively generate LFs for each topic entity one hop at a time using a fine-tuned LLM, leveraging KB knowledge to calibrate intermediate outputs and mitigate error propagation. Subsequently, we guide the LLM in assembling path-level LFs from different topic entities, resulting in optimized final LF. We evaluate the proposed method on four KBQA benchmarks spanning two distinct KBs, demonstrating its superior performance compared to state-of-the-art baselines. The code is available at https://github.com/pvfeldt/GCA-KBQA.

Knowledge base question answering (KBQA) refers to the task of answering natural language questions using factual information from large-scale knowledge bases (KBs). To obtain accurate answers, recent research optimizes semantic parsing methods, a major KBQA approach, with large language models (LLMs), where concise logical forms (LFs) are generated by LLMs and executed in KBs. Although these methods demonstrate superior performance, they still encounter the problem that some generated LFs fail to yield answers when executed, significantly limiting their effectiveness. To mitigate this issue, we propose KARV, a Knowledge-Assisted reasoning path Reconstruction and hierarchical Voting approach for non-executable LFs. This method extracts semantic knowledge from KBs as guidance to correct and reconstruct reasoning paths, deriving answers through a voting-based strategy. The insight is that non-executable LFs generated by LLMs still contain rich semantic information, and the knowledge retrieved from KBs can effectively correct them. Specifically, we fine-tune LLMs to generate high-quality LFs, and the nonexecutable LFs are decomposed into multiple path branches based on mentioned entities. Semantic knowledge from KBs is then leveraged to correct the entities and relations within these branches, effectively reconstructing the reasoning paths. To obtain precise final answers, we apply a hierarchical voting strategy both within and across the non-executable LFs. Our proposed method achieves state-of-the-art performance on benchmarks including WebQuestionSP (WebQSP), ComplexWebQuestions (CWQ), and FreebaseQA.

Knowledge base question answering (KBQA) refers to the system that produces answers to user queries by reasoning with a large-scale structured knowledge base. Advanced works have achieved great success either by generating logical forms (LF) or directly generating answers. Although the former typically yields better performance, these generated LF could be inaccurate, e.g., non-executable. In this regard, large language models (LLMs) have shown exciting potential for accurate generation. However, it is challenging to fine-tune LLMs to generate LF. This is because the context retrieved for prediction typically leads to an excessive number of reasoning paths. In this context, LLMs can generate numerous LF corresponding to these reasoning paths, but a few LF can result in correct answers. Thus, fine-tuning LLMs to generate answer-relevant LF would conflict with the prior knowledge of the LLMs. In this work, we propose a novel learning framework, FM-KBQA, to fine-tune LLMs using multi-task learning for KBQA. Specifically, we propose to fine-tune LLMs using an additional objective: generating the index of reasoning paths that lead to correct answers. This will direct LLMs to pay attention to answer-relevant paths among numerous reasoning paths by completing a simple task where the selected reasoning paths can be supplementary for non-executable LF. Directly generating answers can make LLMs pay attention to the answer-relevant reasoning paths, but it is much more challenging than generating the index of reasoning paths. To verify FM-KBQA's effectiveness, we conduct experiments on mainstream benchmarks, such as WebQuestionsSP (WQSP) and ComplexWebQuestions (CWQ). Extensive evaluations across two public benchmark datasets underscore the superiority of FM-KBQA over current state-of-the-art methods.