In recent years, the field of heterogeneous graph learning has garnered significant interest. Various efforts have been made towards learning heterogeneous graph representations, such as designing meta-paths to mine implicit graph knowledge or directly applying Graph Neural Networks (GNNs) for graph representation. However, these methods fail to fully capture available graph knowledge while ensuring scalability across diverse graph settings. In this paper, we address these challenges by introducing IEGraph, a heterogeneous Graph learning approach that capitalizes on both implicit and explicit graph knowledge. This encompasses two training stages: the implicit label-free stage and the explicit label-based stage, fostering comprehensive utilization of graph information. The label-free stage extracts implicit graph knowledge by constructing local and global training samples for contrastive pre-training, while the label-based stage further employs explicit labeled data to fine-tune the model. We carry out experiments on diverse heterogeneous graphs, and the results show that IEGraph achieves commendable performance compared to other state-of-the-art baselines.

Generative graph self-supervised learning (SSL), represented by masked graph autoencoders (GAEs), has shown great potential in graph representation learning. Existing masked GAEs typically rely on reconstruction criteria, such as mean squared error, to measure the discrepancy between the input graph and the reconstructed output. However, this learning paradigm struggles with perturbed graph characteristics, hindering the learning of robust graph representations. To address this, we introduce AMGAE -- an Adversarial Masked Graph AutoEncoder, which enhances the robustness of masked GAEs by integrating an adversarial learning strategy. Specifically, we design AMGAE to comprise a generator and a discriminator, optimized alternately and interconnected by a binary discrimination task (BDT). We treat the entire masked GAE as the generator, which produces a reconstructed output using the visible graph features. Then, we synthesize the reconstructed output by substituting the visible node features with the corresponding raw input features. Finally, we employ an additional GNN layer as the discriminator to determine the authenticity of the node-level features synthesized by BDT. By introducing the adversarial strategy, AMGAE reformulates masked GAE learning into a min-max game, which facilitates the learning of robust graph representations. We conduct extensive experiments on three graph tasks, demonstrating that AMGAE performs favorably against diverse baselines.

In the existing spectral GNNs, polynomial-based methods occupy the mainstream in designing a filter through the Laplacian matrix. However, polynomial combinations factored by the Laplacian matrix naturally have limitations in message passing (e.g., over-smoothing). Furthermore, most existing spectral GNNs are based on polynomial bases, which struggle to capture the high-frequency parts of the graph spectral signal. Additionally, we also find that even increasing the polynomial order does not change this situation, which means polynomial-based models have a natural deficiency when facing high-frequency signals. To tackle these problems, we propose WaveNet, which aims to effectively capture the high-frequency part of the graph spectral signal from the perspective of wavelet bases through reconstructing the message propagation matrix. We utilize Multi-Resolution Analysis (MRA) to model this question, and our proposed method can reconstruct arbitrary filters theoretically. We also conduct node classification experiments on real-world graph benchmarks and achieve superior performance on most datasets. Our code is available at https://github.com/Bufordyang/WaveNet

Generative self-supervised learning, represented by graph autoencoders (GAEs), has begun to exhibit significant potential in addressing graph tasks. However, GAEs often rely on Graph Neural Networks (GNNs) for encoding and decoding, this can pose a computation challenge due to the inherent complexities of the aggregation mechanism in GNNs. Furthermore, the bipartite structure of GAEs introduces additional computational burdens. In contrast, Multi-Layer Perceptrons (MLPs) have no graph dependency and can train much faster than GNNs. Motivated by this, in this work, we introduce a simple yet effective alternative: the GNN-Free Masked AutoEncoder (GFMAE), which employs MLPs rather than GNNs to serve as the backbone model to speed up training. Additionally, we devise comprehensive decoding strategies to compensate for the inability of MLPs in characterizing the graph. Our comprehensive experiments conducted on eight datasets demonstrate that GFMAE achieves performance comparable to GNNs while also enhancing the training efficiency of generative models with GNNs as the backbone.