Self-supervised heterogeneous graph representation learning (SSHGRL) is a key technique for embedding heterogeneous graphs, enabling effective analysis and modeling of social networks and other graph-structured data, which are central to knowledge discovery and the study of social systems. However, existing SSHGRL methods are hardly applied to large-scale heterogeneous graph environments due to the normally used metapath decomposing mechanism being graph-size-sensitive. Moreover, the existing self-supervised signals are normally created from Shared Mutual Information (SMI) of different graph views that ignore the Non-SMI (NMI) contained in the same view. This results in the model tending to learn insufficient graph representation. To this end, this article proposes a designated masking propagation (DMP) mechanism to process heterogeneous graphs without using metapath. Moreover, based on the DMP graph view, a novel sufficient representation is proposed to learn the effective graph representation by combining both NMI and SMI. Extensive experiments on eight large- and medium-scale heterogeneous graph datasets demonstrate the superiority of our method, setting new state-of-the-art performance in various big data contexts.

Real-world graph data environments intrinsically exist noise (e.g., link and structure errors) that inevitably disturb the effectiveness of graph representation and downstream learning tasks. For homogeneous graphs, the latest works use original node features to synthesize a similarity graph that can correct the structure of the noised graph. This idea is based on the homogeneity assumption, which states that similar nodes in the homogeneous graph tend to have direct links in the original graph. However, similar nodes in heterogeneous graphs usually do not have direct links, which can not be used to correct the original noise graph. This causes a significant challenge in noised heterogeneous graph learning. To this end, this paper proposes a novel synthesized similarity-based graph neural network compatible with noised heterogeneous graph learning. First, we calculate the original feature similarities of all nodes to synthesize a similarity-based high-order graph. Second, we propose a similarity-aware encoder to embed original and synthesized graphs with shared parameters. Then, instead of graph-to-graph supervising, we synchronously supervise the original and synthesized graph embeddings to predict the same labels. Meanwhile, a target-based graph extracted from the synthesized graph contrasts the structure of the metapath-based graph extracted from the original graph to learn the mutual information. Extensive experiments in numerous real-world datasets show the proposed method achieves state-of-the-art records in the noised heterogeneous graph learning tasks. In highlights, +5~6\% improvements are observed in several noised datasets compared with previous SOTA methods.

As the application of knowledge graphs becomes increasingly widespread, the issue of knowledge graph incompleteness has garnered significant attention. As a classical type of non-euclidean spatial data, knowledge graphs possess various complex structural types. However, most current knowledge graph completion models are developed within a single space, which makes it challenging to capture the inherent knowledge information embedded in the entire knowledge graph. This limitation hinders the representation learning capability of the models. To address this issue, this paper focuses on how to better extend the representation learning from a single space to Riemannian manifolds, which are capable of representing more complex structures. We propose a new knowledge graph completion model called MRME-KGC, based on multi-view Riemannian Manifolds fusion to achieve this. Specifically, MRME-KGC simultaneously considers the fusion of four views: two hyperbolic Riemannian spaces with negative curvature, a Euclidean Riemannian space with zero curvature, and a spherical Riemannian space with positive curvature to enhance knowledge graph modeling. Additionally, this paper proposes a contrastive learning method for Riemannian spaces to mitigate the noise and representation issues arising from Multi-view Riemannian Manifolds Fusion. This paper presents extensive experiments on MRME-KGC across multiple datasets. The results consistently demonstrate that MRME-KGC significantly outperforms current state-of-the-art models, achieving highly competitive performance even with low-dimensional embeddings.

Self-supervised Heterogeneous Graph Representation (SSHGRL) learning is widely used in data mining. The latest SSHGRL methods normally use metapaths to describe the heterogeneous information (multiple relations and node types) to learn the heterogeneous graph representation and achieve impressive results. However, establishing metapaths requires lofty computational costs that are too high for the medium and large graphs. To this end, this paper proposes a Reserving-Masking-Reconstruction (RMR) model that can fully consider heterogeneous information without relying on the metapaths. In detail, we propose a reserving method to reserve to-be-masked nodes' (target nodes) information before graph masking. Second, we split the reserved graph into relation subgraphs according to the type of relations that require much less computational overheads than metapath. Then, the target nodes in each relation subgraph are randomly masked with minimal topology information loss. After, a novel reconstruction method is proposed to reconstruct the masked nodes on different relation subgraphs to establish the self-supervised signal. The proposed method requires low computational complexity and can establish a self-supervised signal without deeply changing the graph topology. Experimental results show the proposed method achieves state-of-the-art records on medium and large-scale heterogeneous graphs and competitive records on small-scale heterogeneous graphs. The code is available at https://github.com/DuanhaoranCC/RMR.

Self-supervised graph representation Learning (SSGRL) is an emerging technique for machine learning-based expert applications. SSGRL can effectively encode unlabeled data into machine-understandable knowledge embeddings that can be used in downstream expert tasks. Recently, Masked-Graph-Model (MGM) has achieved extraordinary performances in the field of SSGRL. However, MGM inevitably suffers from a so-called Negative-Migration (NM) problem on unbalanced datasets for specific tasks such as graph classification. The NM problem forces the MGM to predict (reconstruct) the dominant nodes causing the ignoring of the minority but critical nodes. To this end, a novel online prediction module is proposed to dynamically predict the nodes with extended contexts that enable minority nodes can have more weights in the model. Based on the online prediction module, a graph representation learning model is proposed to alleviate the NM problem. Extensive experiments demonstrate that the model outperforms state-of-the-art methods on unbalanced datasets and achieves competitive performance on balanced datasets. Moreover, extra experiments demonstrate that the proposed method requires less computational resource overhead than existing methods. The code is available at: https://github.com/DuanhaoranCC/SimGOP.

Heterogeneous Information Networks (HINs) are crucial in various intelligent systems. The latest advancements in HIN learning aim to combine meta-paths and hypergraphs, capitalizing on their strengths for further success. However, existing methods typically transform meta-paths into hypergraphs by simply removing the original edges from the meta-paths to integrate two semantics. This will inevitably encounter semantic ambiguity, a so-called semantic-shift problem, during the “meta-path → hyperedges” transforming, causing limited improvements. To address this, we introduce a novel fusion framework that distills knowledge from meta-paths into hypergraphs, mitigating such a problem. Specifically, we propose a unique hyperedge extraction method for constructing the hypergraph, incorporating various aspects instead of relying solely on one type of meta-path. Subsequently, we introduce a shallow student model to capture high-order information from the hypergraph, complementing a teacher model that focuses on encoding low-order information from meta-paths. Then, a distillation framework is employed to integrate explicitly multi-order information into the student. Experimental results across diverse datasets demonstrate a substantial improvement in node classification tasks, with an average accuracy increase of 2.1% over existing state-of-the-art methods.

Abstract Self-supervised graph representation learning has been widely used in many intelligent applications since labeled information can hardly be found in these data environments. Currently, masking and reconstruction-based (MR-based) methods lead the state-of-the-art records in the self-supervised graph representation field. However, existing MR-based methods did not fully consider both the deep-level node and structure information which might decrease the final performance of the graph representation. To this end, this paper proposes a node and edge dual-masked self-supervised graph representation model to consider both node and structure information. First, a dual masking model is proposed to perform node masking and edge masking on the original graph at the same time to generate two masking graphs. Second, a graph encoder is designed to encode the two generated masking graphs. Then, two reconstruction decoders are designed to reconstruct the nodes and edges according to the masking graphs. At last, the reconstructed nodes and edges are compared with the original nodes and edges to calculate the loss values without using the labeled information. The proposed method is validated on a total of 14 datasets for graph node classification tasks and graph classification tasks. The experimental results show that the method is effective in self-supervised graph representation. The code is available at: https://github.com/TangPeng0627/Node-and-Edge-Dual-Mask .

Graph representation is a critical technology in the field of knowledge engineering and knowledge-based applications since most knowledge bases are represented in the graph structure. Nowadays, contrastive learning has become a prominent way for graph representation by contrasting positive-positive and positive-negative node pairs between two augmentation graphs. It has achieved new state-of-the-art in the field of self-supervised graph representation. However, existing contrastive graph representation methods mainly focus on modifying (normally removing some edges/nodes) the original graph structure to generate the augmentation graph for the contrastive. It inevitably changes the original graph structures, meaning the generated augmentation graph is no longer equivalent to the original graph. This harms the performance of the representation in many structure-sensitive graphs such as protein graphs, chemical graphs, molecular graphs, etc. Moreover, there is only one positive-positive node pair but relatively massive positive-negative node pairs in the self-supervised graph contrastive learning. This can lead to the same class, or very similar samples are considered negative samples. To this end, in this work, we propose a Virtual Masking Augmentation (VMA) to generate an augmentation graph without changing any structures from the original graph. Meanwhile, a node augmentation method is proposed to augment the positive node pairs by discovering the most similar nodes in the same graph. Then, two different augmentation graphs are generated and put into a contrastive learning model to learn the graph representation. Extensive experiments on massive datasets demonstrate that our method achieves new state-of-the-art results on self-supervised graph representation. The source code of the proposed method is available at https://github.com/DuanhaoranCC/CGRA.

Zero-shot learning (ZSL) aims to predict unseen classes without using samples of these classes in model training. The ZSL has been widely used in many knowledge-based models and applications to predict various parameters, including categories, subjects, and anomalies, in different domains. Nonetheless, most existing ZSL methods require the pre-defined semantics or attributes of particular data environments. Therefore, these methods are difficult to be applied to general data environments, such as ImageNet and other real-world datasets and applications. Recent research has tried to use open knowledge to enhance the ZSL methods to adapt it to an open data environment. However, the performance of these methods is relatively low, namely the accuracy is normally below 10%, which is due to the inadequate semantics that can be used from open knowledge. Moreover, the latest methods suffer from a significant "semantic gap" problem between the generated features of unseen classes and the real features of seen classes. To this end, this paper proposes a multi-view graph representation with a similarity diffusion model, applying the ZSL tasks to general data environments. This model applies a multi-view graph to enhance the semantics fully and proposes an innovative diffusion method to augment the graph representation. In addition, a feature diffusion method is proposed to augment the multi-view graph representation and bridge the semantic gap to realize zero-shot predicting. The results of numerous experiments in general data environments and on benchmark datasets show that the proposed method can achieve new state-of-the-art results in the field of general zero-shot learning. Furthermore, seven ablation studies analyze the effects of the settings and different modules of the proposed method on its performance in detail and prove the effectiveness of each module.