Entity and relation linking critically impact the accuracy of knowledge graph question answering (KGQA), often limiting the performance of downstream tasks like query generation. While recent advances in large language models (LLMs) offer promising solutions, their role in linking remains underexplored. This work studies how different linking strategies – including both traditional and LLM-based approaches– affect the quality of generated KG queries. We design multiple linking pipelines and use their output to guide structured query construction. Our study not only evaluates linking accuracy, but also the end-to-end impact on query generation. Our experiments show that LLM-based linkers significantly outperform non-LLM methods, particularly in recall. Moreover, we find that high recall—even at the cost of precision—can lead to better overall performance, as LLMs are resilient to input noise. These findings highlight the importance of recall-oriented linking in modern KGQA pipelines.

The manual creation of gold standards for named entity recognition and entity linking is time-and resource-intensive.Moreover, recent works show that such gold standards contain a large proportion of mistakes in addition to being difficult to maintain.We hence present BENGAL, a novel automatic generation of such gold standards as a complement to manually created benchmarks.The main advantage of our benchmarks is that they can be readily generated at any time.They are also cost-effective while being guaranteed to be free of annotation errors.We compare the performance of 11 tools on benchmarks in English generated by BENGAL and on 16 benchmarks created manually.We show that our approach can be ported easily across languages by presenting results achieved by 4 tools on both Brazilian Portuguese and Spanish.Overall, our results suggest that our automatic benchmark generation approach can create varied benchmarks that have characteristics similar to those of existing benchmarks.

Implementing the multilingual Semantic Web vision requires transforming unstructured data in multiple languages from the Document Web into structured data for the multilingual Web of Data. We present the multilingual version of FOX, a knowledge extraction suite which supports this migration by providing named entity recognition based on ensemble learning for five languages. Our evaluation results show that our approach goes beyond the performance of existing named entity recognition systems on all five languages. In our best run, we outperform the state of the art by a gain of 32.38% F1-Score points on a Dutch dataset. More information and a demo can be found at http://fox.aksw.org as well as an extended version of the paper descriping the evaluation in detail.

We present GERBIL, an evaluation framework for semantic entity annotation. The rationale behind our framework is to provide developers, end users and researchers with easy-to-use interfaces that allow for the agile, fine-grained and uniform evaluation of annotation tools on multiple datasets. By these means, we aim to ensure that both tool developers and end users can derive meaningful insights pertaining to the extension, integration and use of annotation applications. In particular, GERBIL provides comparable results to tool developers so as to allow them to easily discover the strengths and weaknesses of their implementations with respect to the state of the art. With the permanent experiment URIs provided by our framework, we ensure the reproducibility and archiving of evaluation results. Moreover, the framework generates data in machine-processable format, allowing for the efficient querying and post-processing of evaluation results. Finally, the tool diagnostics provided by GERBIL allows deriving insights pertaining to the areas in which tools should be further refined, thus allowing developers to create an informed agenda for extensions and end users to detect the right tools for their purposes. GERBIL aims to become a focal point for the state of the art, driving the research agenda of the community by presenting comparable objective evaluation results.