Academic writing skills are crucial for students' success but can feel overwhelming without proper guidance and practice, particularly when writing in a second language. Traditionally, students ask instructors or search dictionaries, which are not universally accessible. Early writing assistants emerged as rule-based systems that focused on detecting misspellings, subject-verb disagreements, and basic punctuation errors but are inaccurate and lack contextual understanding. Machine learning-based assistants demonstrate a strong ability for language understanding but are expensive to train. Large language models (LLMs) have shown remarkable capabilities in generating responses in natural languages based on given prompts, but they have a fundamental limitation in education: they generate essays without teaching, which can have detrimental effects on learning when misused. To address this limitation, we develop CoachGPT, which leverages LLMs to assist academic writing for those with limited educational resources and those who prefer self-paced learning. CoachGPT is an AI agent-based web application that (1) takes instructions from experienced educators (2) converts instructions into sub-tasks, and (3) provides real-time feedback and suggestions using large language models. This unique scaffolding structure makes CoachGPT unique among existing writing assistants. Compared with existing writing assistants, CoachGPT provides a more immersed writing experience with personalized messages. Our user studies prove the usefulness of CoachGPT and the potential of large language models for academic writing.