Graph Masked Autoencoders (GMAEs) have emerged as a notable self-supervised learning approach for graph-structured data. Existing GMAE models primarily focus on reconstructing node-level information, categorizing them as single-scale GMAEs. This methodology, while effective in certain contexts, tends to overlook the complex hierarchical structures inherent in many real-world graphs. For instance, molecular graphs exhibit a clear hierarchical organization in the form of the atoms-functional groups-molecules structure. Therefore, the inability of single-scale GMAE models to incorporate these hierarchical relationships often results in an inadequate capture of crucial high-level graph information, leading to a noticeable decline in performance. To address this limitation, we propose Hierarchical Graph Masked AutoEncoders (Hi-GMAE), a novel multi-scale GMAE framework designed to handle the hierarchical structures within graphs. First, Hi-GMAE constructs a multi-scale graph hierarchy through graph pooling, enabling the exploration of graph structures across different granularity levels. To ensure masking uniformity of subgraphs across these scales, we propose a novel coarse-to-fine strategy that initiates masking at the coarsest scale and progressively back-projects the mask to finer scales. Furthermore, we integrate a gradual recovery strategy with the masking process to mitigate the learning challenges posed by completely masked subgraphs. Diverging from the standard graph neural network (GNN) used in GMAE models, Hi-GMAE modifies its encoder and decoder into hierarchical structures. This entails using GNN at the finer scales for detailed local graph analysis and employing a graph transformer at coarser scales to capture global information. Such a design enables Hi-GMAE to effectively capture the multi-level information inherent in complex graph structures. Our experiments on 17 graph datasets, covering two graph learning tasks, consistently demonstrate that Hi-GMAE outperforms 29 state-of-the-art self-supervised competitors in capturing comprehensive graph information.

Molecular representation learning is pivotal in predicting molecular properties and advancing drug design. Traditional methodologies, which predominantly rely on homogeneous graph encoding, are limited by their inability to integrate external knowledge and represent molecular structures across different levels of granularity. To address these limitations, we propose a paradigm shift by encoding molecular graphs into heterogeneous structures, introducing a novel framework: Knowledge-aware Contrastive Heterogeneous Molecular Graph Learning. This approach leverages contrastive learning to enrich molecular representations with embedded external knowledge. KCHML conceptualizes molecules through three distinct graph views-molecular, elemental, and pharmacological-enhanced by heterogeneous molecular graphs and a dual message-passing mechanism. This design offers a comprehensive representation for property prediction, as well as for downstream tasks such as drug-drug interaction prediction. Extensive benchmarking demonstrates KCHML's superiority over state-of-the-art molecular property prediction models, underscoring its ability to capture intricate molecular features.

Node and graph-level clustering hold considerable significance for a wide range of applications, including drug target identification and protein function prediction. Recently, contrastive learning has surpassed numerous unsupervised learning methods and become increasingly useful for various deep clustering procedures, achieving commendable results. However, two primary obstacles impede further deployment of graph contrastive clustering: (1) its inherent tendency to separate node representations, which contradicts the clustering objective of forming meaningful groups and impedes effective cluster creation, and (2) the occurrence of false negative samples, which similarly obstructs cluster formation. Hence, this paper proposes a novel graph clustering algorithm, which employs a dual contrastive learning approach, encompassing element and cluster contrasts, and a strategy for debiasing false negative samples. The proposed algorithm utilizes element-level contrastive learning on embeddings derived from the encoder, integrating detailed node or graph characteristics. Then, clustering and cluster-level contrastive learning are executed in the embedding space to refine the results. Furthermore, the algorithm effectively addresses the potential false negatives and imbalanced prediction challenges during the dual-contrast process by implementing an optimization mechanism based on reliable results, thereby enhancing the clustering performance. Rigorous experiments across three node and graph-level benchmarks validate our proposed algorithm's efficacy.

The introduction of knowledge graphs (KG) into recommendation systems (RS) has been proven to be effective because KG introduces a variety of relations between items. In fact, users have different relation preferences depending on the relationship in KG. Existing GNN-based models largely adopt random neighbor sampling strategies to process propagation; however, these models cannot aggregate biased relation preference local information for a specific user, and thus cannot effectively reveal the internal relationship between users' preferences. This will reduce the accuracy of recommendations, while also limiting the interpretability of the results.