Catastrophic forgetting poses a significant challenge for graph neural networks in continuously updating their knowledge base with data streams. To address this issue, much of the research has focused on node-level continual learning using parameter regularization or rehearsal-based strategies, while little attention given to graph-level tasks. Furthermore, current paradigms for continual graph learning may inadvertently capture spurious correlations for specific tasks through shortcuts, thereby exacerbating the forgetting of previous knowledge when new tasks are introduced. To tackle these challenges, we propose a novel paradigm, Rationale Learning GNN (RL-GNN), for graph-level continual graph learning. Specifically, we harness the invariant learning principle to incorporate environmental interventions into both the current and historical distributions, aiming to uncover rationales by minimizing empirical risk across all environments. The rationale serves as the sole factor guiding the learning process. Therefore, continual graph learning is redefined as capturing these invariant rationales within task sequences, alleviating catastrophic forgetting caused by spurious features. Extensive experiments on real-world datasets with varying task lengths demonstrate the effectiveness of our RL-GNN in continuous knowledge assimilation and reduction of catastrophic forgetting.

Graph distillation endeavors to transfer knowledge from large, complex teacher models, such as Graph Neural Networks (GNNs), to smaller, more efficient student models such as Multi-Layer Perceptrons (MLPs). Our work is motivated by a critical observation: as the complexity and depth of the teacher GNN increase, the performance of the distilled student MLP models tends to decline significantly. This issue is not limited to a specific distillation method but is a prevalent challenge across various GNN to MLP approaches. To address this gap, we introduce a novel prompt-driven graph distillation framework that enhances the student model’s input space by appending or integrating learned prompts with the original features. These prompts, derived from the teacher’s input, hidden and output knowledge, provide additional context that assists the MLP student in distilling information, thereby circumventing the need to directly capture any teacher’s knowledge into the simplest single-layer MLP. Our empirical experiment on benchmark graph datasets reveals a counter-intuitive phenomenon: the more capable the teacher, the greater the student’s ability to utilize prompts to outperform the teacher. This finding highlights the strength of our approach: a well-prompted student can indeed surpass its teacher, at achieving the best performance with an accuracy of 87.2% on the Cora standard split with a single-layer MLP, while also maintaining efficiency and robustness. In addition to testing on benchmark graph datasets, we applied our framework to the TUH EEG Epilepsy Corpus (TUEP), condensing complex GNN models to simple MLPs while achieving good performance and efficiency in epilepsy classification.