Graph Neural Networks (GNNs) are the rising standard for graph tasks, yet their distributed training in servers or computing power network remains challenging. Cross-machine sampling and data loading often create bottlenecks, leading to inefficient resource utilization. In this paper, we present Hybrid2, a distributed GNN training system that combines full-graph and mini-batch training through a novel hybrid-batch training method. It also adopts hybrid feature extraction, leveraging both local caching and remote access to improve feature retrieval efficiency. The integration of these methods in Hybrid² results in a dual hybrid-gain effect. First, it reduces sampling and loading overhead by pre-aggregating neighbors for each target vertex, minimizing the layers to sample and load. Second, it accelerates data loading by dynamically identifying and locally caching the most frequently accessed vertices during training, maximizing memory efficiency. Experimental results demonstrate that Hybrid² brings substantial performance improvements across key components of distributed GNN training. Network communication overhead is reduced by up to tens of times, while both sampling and loading achieve at least several-fold speedups. These gains contribute to an overall training acceleration exceeding 20× compared to DistDGL, all with comparable GPU memory usage and no loss in accuracy. Compared to the state-of-the-art system, it achieves nearly 3× speedup while using fewer resources.

Graph Neural Networks (GNNs)-based recommendation algorithms typically assume that training and testing data are drawn from independent and identically distributed (IID) spaces. However, this assumption often fails in the presence of out-of-distribution (OOD) data, resulting in significant performance degradation. In this study, we construct a Structural Causal Model (SCM) to analyze interaction data, revealing that environmental confounders (e.g., the COVID-19 pandemic) lead to unstable correlations in GNN-based models, thus impairing their generalization to OOD data. To address this issue, we propose a novel approach, graph representation learning via causal diffusion (CausalDiffRec) for OOD recommendation. This method enhances the model's generalization on OOD data by eliminating environmental confounding factors and learning invariant graph representations. Specifically, we use backdoor adjustment and variational inference to infer the real environmental distribution, thereby eliminating the impact of environmental confounders. This inferred distribution is then used as prior knowledge to guide the representation learning in the reverse phase of the diffusion process to learn the invariant representation. In addition,we provide a theoretical derivation that proves optimizing the objective function of CausalDiffRec can encourage the model to learn environment-invariant graph representations, thereby achieving excellent generalization performance in recommendations under distribution shifts. Our extensive experiments validate the effectiveness of CausalDiffRec in improving the generalization of OOD data, and the average improvement is up to 10.69% on Food, 18.83% on KuaiRec, 22.41% on Yelp2018, and 11.65% on Douban datasets.

The distributionally robust optimization (DRO)-based graph neural network methods improve recommendation systems' out-of-distribution (OOD) generalization by optimizing the model's worst-case performance. However, these studies fail to consider the impact of noisy samples in the training data, which results in diminished generalization capabilities and lower accuracy. Through experimental and theoretical analysis, this paper reveals that current DRO-based graph recommendation methods assign greater weight to noise distribution, leading to model parameter learning being dominated by it. When the model overly focuses on fitting noise samples in the training data, it may learn irrelevant or meaningless features that cannot be generalized to OOD data. To address this challenge, we design a Distributionally Robust Graph model for OOD recommendation (DRGO). Specifically, our method first employs a simple and effective diffusion paradigm to alleviate the noisy effect in the latent space. Additionally, an entropy regularization term is introduced in the DRO objective function to avoid extreme sample weights in the worst-case distribution. Finally, we provide a theoretical proof of the generalization error bound of DRGO as well as a theoretical analysis of how our approach mitigates noisy sample effects, which helps to better understand the proposed framework from a theoretical perspective. We conduct extensive experiments on four datasets to evaluate the effectiveness of our framework against three typical distribution shifts, and the results demonstrate its superiority in both independently and identically distributed distributions (IID) and OOD.

Recently, there has been an emergence of self-supervised recommendation methods that integrate knowledge graphs. Upon conducting a comprehensive review of contrastive learning (CL) in recommender systems, we conclude that existing methods solely focus on data view generation (the first phase) while neglecting the equally pivotal data view alignment (the second phase). However, due to the complexity and variability of real-world graph data, regardless of the graph augmentation strategy employed, it may be unrealistic to expect all entities to benefit from CL. In this article, we propose a H ierarchical E nhanced K nowledge-Aware C ontrastive L earning (HEK-CL) method for recommendation. Overall, we aim to hierarchically carry out enhancement strategies in both the first and second phases of knowledge-aware CL: (1) From the perspective of enhancing data view generation, we focus on combining non-Euclidean representation learning with graph denoising modules. Owing to the unified space’s ability to learn the ideal curvature from data distributions, the quality of embeddings for graph data has seen enhancements; (2) From the perspective of enhancing data view alignment, we propose a hyperbolic robust contrastive loss, named HRCL. Through rigorous theoretical analysis and experiments, we demonstrate that HRCL provides a more balanced and equitable training process for all entities than InfoNCE. Numerous experiments on the three real-world datasets show that our HEK-CL outperforms state-of-the-art baselines.

Graph Contrastive Learning (GCL) leverages data augmentation techniques to produce contrasting views, enhancing the accuracy of recommendation systems through learning the consistency between contrastive views. However, existing augmentation methods, such as directly perturbing interaction graph (e.g., node/edge dropout), may interfere with the original connections and generate poor contrasting views, resulting in sub-optimal performance. In this article, we define the views that share only a small amount of information with the original graph due to poor data augmentation as noisy views (i.e., the last 20% of the views with a cosine similarity value less than 0.1 to the original view). We demonstrate through detailed experiments that noisy views will significantly degrade recommendation performance. Further, we propose a model-agnostic Symmetric Graph Contrastive Learning (SGCL) method with theoretical guarantees to address this issue. Specifically, we introduce symmetry theory into graph contrastive learning, based on which we propose a symmetric form and contrast loss resistant to noisy interference. We provide theoretical proof that our proposed SGCL method has a high tolerance to noisy views. Further demonstration is given by conducting extensive experiments on three real-world datasets. The experimental results demonstrate that our approach substantially increases recommendation accuracy, with relative improvements reaching as high as 12.25% over nine other competing models. These results highlight the efficacy of our method. The code is available at https://github.com/user683/SGCL .

As real-world graph data continues to grow larger and larger, training large graphs in a distributed environment is becoming increasingly prevalent. However, network transmission in a distributed environment can hinder subsequent training steps, resulting in suboptimal training performance. After conducting a comprehensive analysis and experimental demonstration, we have discovered that during the training process, there exist certain data that can be computed once and reused multiple times. In addition, we also found that after a certain iterations of training, the parameter updates during each iteration had minimal effect on the parameters. Based on these findings, we have improved the original implementation and proposed a cache-enhanced distributed graph training system, NeutronCache. It utilizes cached reusable intermediate data and a dynamically adjusted stale embedding reuse strategy, reducing network overhead in distributed systems and accelerating the training process. Through experimental validation, our implementation achieved acceleration ranging from 1.4X to 16.61X on real graph datasets with almost no loss in accuracy.

Popularity bias is a massive challenge for autoencoder-based models, which decreases the level of personalization and hurts the fairness of recommendations. User reviews reflect their preferences and help mitigate bias or unfairness in the recommendation. However, most existing works typically incorporate user (item) reviews into a long document and then use the same module to process the document in parallel. Actually, the set of user reviews is completely different from the set of item reviews. User reviews are heterogeneous in that they reflect a variety of items purchased by users, while item reviews are only related to the item itself and are thus typically homogeneous. In this article, a novel asymmetric attention network fused with autoencoders is proposed, which jointly learns representations from the user and item reviews and implicit feedback to perform recommendations. Specifically, we design an asymmetric attentive module to capture rich representations from user and item reviews, respectively, which solves data sparsity and explainable problems. Furthermore, to further address popularity bias, we apply a noise-contrastive estimation objective to learn high-quality “de-popularity” embedding via the decoder structure. A series of extensive experiments are conducted on four benchmark datasets to show that leveraging user review information can eliminate popularity bias and improve performance compared to various state-of-the-art recommendation techniques.