Knowledge Graph (KG) embedding has emerged as a very active area of research over the last few years, resulting in the development of several embedding methods.These KG embedding methods represent KG entities and relations as vectors in a high-dimensional space.Despite this popularity and effectiveness of KG embeddings in various tasks (e.g., link prediction), geometric understanding of such embeddings (i.e., arrangement of entity and relation vectors in vector space) is unexplored -we fill this gap in the paper.We initiate a study to analyze the geometry of KG embeddings and correlate it with task performance and other hyperparameters.To the best of our knowledge, this is the first study of its kind.Through extensive experiments on real-world datasets, we discover several insights.For example, we find that there are sharp differences between the geometry of embeddings learnt by different classes of KG embeddings methods.We hope that this initial study will inspire other follow-up research on this important but unexplored problem.