Question generation is the task of natural language processing where the goal is to generate fluent, grammatically correct, error-free questions based on a given input context and optionally an answer. Multi-hop question generation is a more complex task compared to traditional single-hop question generation, as it requires reasoning over multiple information from multiple input contexts in generating multi-hop questions. In this paper, we have addressed the challenge of building a multi-hop question generation system by combining the knowledge graphs with large language models. We have designed a framework KG4QG (Knowledge Graph for Question Generation), where knowledge graphs are generated from the input contexts. For the knowledge graph embedding, we have used Graph Attention Network, and for input text embedding, we have leveraged Sentence Transformer. Finally, we apply BART and T5 models as Large Language Models to generate multi-hop questions from our proposed model. Using HotpotQA dataset to evaluate the performance of our KG4QG framework, our proposed methodology has shown an enhancement of performance over the previous methodologies.