Existing graph neural network (GNN) methods are typically built upon the i.i.d. assumption, emphasizing the enhancement of the test performance for in-distribution (ID) data. However, there has been limited exploration of their adaptability to scenarios involving unknown distribution data. On the one hand, in real-world application scenarios, graph data often expands continuously with the acquisition of external knowledge, which means that new nodes with unknown categories may be added to the graph data. The gap between the new node distribution and the original node distribution can make existing GNN methods less effective. On the other hand, existing out-of-distribution (OOD) detection methods often rely on the softmax confidence score, which makes the OOD data suffer from overconfident posterior distributions. To address the above issues, we propose an Energy Propagation-based Graph Neural Network (EPGNN), which improves the OOD generalization ability by endowing GNN with the capacity to detect the OOD nodes in the graph. Specifically, we first construct GNN encoder to obtain node embedding that incorporates neighborhood structural information. Then, we design a plug-and-play energy-based OOD evaluator by assigning corresponding energy values to different nodes. Finally, we construct a plug-and-play structure-aware energy propagation module and joint alignment regularization, which make the node energy more flexible during the training process. Extensive experiments on benchmark datasets demonstrate the superiority of our method.

Graph prompt learning represents a novel paradigm aimed at enhancing the performance of graph learning models on a variety of downstream tasks by providing specific graph prompts. Despite its promise, current graph prompt learning methods are limited by the following limitations. On the one hand, existing methods often rely on manually selected graph information or simple learnable vectors, which can introduce human biases and lack expressiveness. These methods also fall short in guiding models to induce historical prior knowledge and improve generalization. Furthermore, the direct end-to-end tuning strategy of prompts lacks a necessary gentle transition, which impacts model stability and generalization. To overcome these limitations, we introduce the generalizable graph prompt learning framework (GGPL), which incorporates model-level prompt injection and a two-stage prompt tuning strategy. GGPL focuses on encoding subgraph structures and attributes during pre-training and uses SimGRACE to predict subgraph similarities, enhancing the base model's generalization. The model-level prompt injection module, with its prompt embedding backbone and self-prompt generation, seamlessly integrates invariant knowledge. Our two-stage tuning strategy, including transition and task-specific tuning, ensures better guidance and stability. By designing learnable prompt tokens and fine-tuning them with task-specific information, GGPL enables the model to generalize more robustly to downstream tasks. We conduct extensive experiments on six benchmark datasets to verify the model's effectiveness.

Recent researches have demonstrated pre-training Graph Neural Networks (GNNs) via meta learning can enhance their performance on learning representations from unlabeled data. The main idea behind them is to learn the transferable priors that work across the distribution of tasks. However, existing methods often leverage a uniform task sampling strategy and ignore the relations between their original graphs and them. This may lead the model learning the redundant information in the pre-training process. In this work, we propose Meta Graph Neural Network (MGNN), a graph pre-training method via weighted meta learning. MGNN trys to learn the transferable experiences from diverse distributions without the side-effect of redundant information. First, we break up a task in traditional meta learning into several sub-tasks to construct the minimum evaluation units. Then, to fully utilize the graph information, we use a two-stage optimization with contrastive loss functions to learn the experience priors on node- and graph-levels in meta-training process. Third, to qualify redundant information, we design an evaluation module which calculates the mutual information in a graph view. Finally, we propose a new optimization objective in meta-testing process, which reduce the model’s attention to redundant information to alleviate the negative impact of them. To validate the effectiveness of the proposed method, extensive experiments are conducted on Cora, Citeseer and Pubmed datasets with several GNNs architectures. Experimental results show that our proposed method outperforms existing GNN pre-training algorithms.

Deep learning models are often trained on datasets that are limited in size and distribution, which may not fully represent the entire range of data encountered in practice. Thus, making deep learning models generalize to out-of-distribution data has received a significant amount of attention in recent studies due to the critical importance of this ability in real-world applications. Meta learning as an effective knowledge transfer paradigm, which learns a base model with high generalization ability to adapt to new data distributions by minimizing domain shifts across tasks during meta-training. However, most existing meta learning methods assume that the base model can access the labels of different domains, and this assumption is demanding in many real application scenarios. In addition, these methods focus on narrowing data-level domain shifts, while ignoring task-level domain shifts, which may lead to inadequate or even negative transfer. Inspired by human learners who use induction to learn and master new tasks, we propose a novel domain-aware meta learning framework for out-of-distribution generalization, termed SMLG. This framework enables the base model to generalize effectively to unseen domains without relying on domain-specific labels. Specifically, we develop a domain-aware transformation module to obtain meta representation and pseudo domain labels. As a result, the base model can be trained robustly without the need for direct domain label input. Furthermore, to investigate the impact of domain shifts at different levels, we introduce a joint loss function that combines cross-entropy with a domain alignment constraint. Extensive experiments on benchmark datasets demonstrate the efficacy of our framework.

Despite the promising representation learning of graph neural networks (GNNs), the supervised training of GNNs notoriously requires large amounts of labeled data from each application. An effective solution is to apply the transfer learning in graph: using easily accessible information to pre-train GNNs, and fine-tuning them to optimize the downstream task with only a few labels. Recently, many efforts have been paid to design the self-supervised pretext tasks, and encode the universal graph knowledge among the various applications. However, they rarely notice the inherent training objective gap between the pretext and downstream tasks. This significant gap often requires costly fine-tuning for adapting the pre-trained model to downstream problem, which prevents the efficient elicitation of pre-trained knowledge and then results in poor results. Even worse, the naive pre-training strategy usually deteriorates the downstream task, and damages the reliability of transfer learning in graph data. To bridge the task gap, we propose a novel transfer learning paradigm to generalize GNNs, namely graph pre-training and prompt tuning (GPPT). Specifically, we first adopt the masked edge prediction, the most simplest and popular pretext task, to pre-train GNNs. Based on the pre-trained model, we propose the graph prompting function to modify the standalone node into a token pair, and reformulate the downstream node classification looking the same as edge prediction. The token pair is consisted of candidate label class and node entity. Therefore, the pre-trained GNNs could be applied without tedious fine-tuning to evaluate the linking probability of token pair, and produce the node classification decision. The extensive experiments on eight benchmark datasets demonstrate the superiority of GPPT, delivering an average improvement of 4.29% in few-shot graph analysis and accelerating the model convergence up to 4.32X. The code is available in: https://github.com/MingChen-Sun/GPPT.