Transformer-based architectures have revolutionized natural language processing and represent a significant promise for advancing policy learning in generalized planning tasks. In particular, PlanGPT demonstrated remarkable results in generating solution plans in different planning domains. For each domain, it is trained on a domain-specific dataset composed of randomly generated planning problems and corresponding solution plans. This work proposes a novel extension to the PlanGPT framework by incorporating macro-actions, high-level actions that encapsulate sequences of primitive actions, which are a well-established technique in classical planning known to improve planning efficiency by guiding exploration through shortcuts. Leveraging this concept, we investigate the impact of macro-actions on the learning process of PlanGPT and whether they mitigate the occurrence of violated preconditions caused by complex object relationships, particularly in domains where these interactions frequently lead to invalid plans due to precondition violations. Experimental results indicate that integrating macroactions improves coverage in several challenging domains and reduces generation time, highlighting the enhanced learning capabilities of PlanGPT when supported by macro-actions.

Transformer-based architectures, such as T5, BERT and GPT, have demonstrated revolutionary capabilities in Natural Language Processing. Several studies showed that deep learning models using these architectures not only possess remarkable linguistic knowledge, but they also exhibit forms of factual knowledge, common sense, and even programming skills. However, the scientific community still debates about their reasoning capabilities, which have been recently tested in the context of automated AI planning; the literature presents mixed results, and the prevailing view is that current transformer-based models may not be adequate for planning. In this paper, we address this challenge differently. We introduce a GPT-based model customised for planning (PLANGPT) to learn a general policy for classical planning by training the model from scratch with a dataset of solved planning instances. Once PLANGPT has been trained for a domain, it can be used to generate a solution plan for an input problem instance in that domain. Our training procedure exploits automated planning knowledge to enhance the performance of the trained model. We build and evaluate our GPT model with several planning domains, and we compare its performance w.r.t. other recent deep learning techniques for generalised planning, demonstrating the effectiveness of the proposed approach.