Recognition of historical documents is still an active research field due to the relatively low recognition accuracy achieved when processing old fonts or low-quality images. In this work, we investigate the use of Large Language Models (LLMs) for the correction of the OCR for old Greek documents. We examine two different old Greek datasets, one machine printed and one typewritten, using a Deep Network based OCR together with several known and easy-to-use LLMs for the correction of the result. Additionally, we synthetically produce erroneous texts and change the LLM prompts in order to further study the behavior of LLMs for correcting old Greek noisy text. Experimental results highlight the potential of LLMs for OCR correction of old Greek documents especially for the cases that the recognition results are relatively poor.