Extracting structured knowledge from text in the form of (subject, predicate, object) triples is a key task for many artificial intelligence applications, in particular for knowledge graphs (KGs) construction. In closed information extraction (cIE) where the extracted triples are constrained by a predefined KG schema, most existing approaches rely on Wikidata for entities and relation extraction. As a result, they often lack the flexibility to adapt to other KGs without prior retraining and costly data annotation. In this paper, to address these limitations, we propose FlexCIE, an approach for cIE which leverages Large Language Models (LLMs) and a KG completeness analysis tool. Given an input text, it identifies a list of entity mentions, links them to entities in the target KG using embedding techniques combined with LLMs, and constructs triples from these extracted entities using relevant properties retrieved from the KG. This enables the use of LLMs for cIE, while ensuring that the generated triples are accurate and compliant with the KG schema. Therefore, our work contributes to making cIE more practical and flexible, particularly for domain specific or enterprise KGs.

Fact checking is the task of determining if a given claim holds. Several algorithms have been developed to check claims with reference information in the form of facts in a knowledge base. While individual algorithms have been experimentally evaluated in the past, we provide the first comprehensive and publicly available benchmark infrastructure for evaluating methods across a wide range of assumptions about the claims and the reference information. We show how, by changing the popularity, transparency, homogeneity, and functionality properties of the facts in an experiment, it is possible to influence significantly the performance of the fact checking algorithms. We introduce a benchmark framework to systematically enforce such properties in training and testing datasets with fine tune control over their properties. We then use our benchmark to compare fact checking algorithms with one another, as well as with methods that can solve the link prediction task in knowledge bases. Our evaluation shows the impact of the four data properties on the qualitative performance of the fact checking solutions and reveals a number of new insights concerning their applicability and performance.

Fact checking is the task of determining if a given claim holds. Several algorithms have been developed to check facts with reference information in the form of knowledge bases. We demonstrate BUCKLE, an open-source benchmark for comparing and evaluating fact checking algorithms in a level playing field across a range of scenarios. The demo is centered around three main lessons. To start, we show how, by changing the properties of the training and test facts, it is possible to influence significantly the performance of the algorithms. We then show the role of the reference data. Finally, we discuss the performance for algorithms designed on different principles and assumptions, as well as approaches that address the link prediction task in knowledge bases.