We propose a method to enable Large Language Models to access Knowledge Graph (KG) data and justify their text generation by showing the specific graph data the model accessed during inference. For this, we combine Language Models with methods from Neurosymbolic Artificial Intelligence designed to answer queries on Knowledge Graphs. This is done by modifying the model so that at different stages of inference it outputs an Existential Positive First-Order (EPFO) query, which is then processed by an additional query appendix. In turn, the query appendix uses neural link predictors along with description aware embeddings to resolve these queries. After that, the queries are logged and used as an explanation of the inference process of the complete model. Lastly, we train the model using a Linear Temporal Logic (LTL) constraint-based loss function to measure the consistency of the queries among each other and with the final model output.