Graph Neural Networks (GNNs) have achieved remarkable success in node classification tasks on individual graphs. However, existing GNNs trained within a specific domain ( a.k.a., source domain) frequently exhibit unsatisfied performance when transferred to another domain ( a.k.a., target domain), due to the domain gap. To tackle this issue, Few Shot Graph Domain Adaptation (FSGDA) is introduced to the node classification task, facilitating knowledge transfer from a fully labeled source graph to a target graph with minimal annotations for each class. An intuitive solution is directly training the GNN with labeled source and target samples together. Nevertheless, there are two issues in this procedure: (1) When the annotations on the target domain used for training are extremely sparse, the GNN performance may significantly be damaged by nodes with the source-domain bias not aligning with the target-domain distribution. (2) Apart from the biased nodes, the low-value nodes among the remaining nodes impede the GNN learning for the core nodes, like the limited target training nodes. To address the above issues, we propose a new method for FSGDA, named GraphInflu, whose core idea is to grasp the key takeaways from the source domain to facilitate the adaptation process. It contains two characteristic modules, including the Supportive Node Selector and the Soft Logic-Inspired Node Reweighting. The former aims to identify the most influential set of source nodes based on their contribution to improving performance on target nodes. The latter further focuses more on the core nodes in the selected influential set, which closely align with the target nodes especially those presenting challenging predictions. Extensive experiments validate the efficacy of GraphInflu by overcoming the current state-of-the-art methods. Our code is available at https://github.com/lvXiangwei/GraphInflu.git.

Recommender systems are crucial for personalizing user experiences but often depend on implicit feedback data, which can be noisy and misleading. Existing denoising studies involve incorporating auxiliary information or learning strategies from interaction data. However, they struggle with the inherent limitations of external knowledge and interaction data, as well as the non-universality of certain predefined assumptions, hindering accurate noise identification. Recently, large language models (LLMs) have gained attention for their extensive world knowledge and reasoning abilities, yet their potential in enhancing denoising in recommendations remains underexplored. In this paper, we introduce LLaRD, a framework leveraging LLMs to improve denoising in recommender systems, thereby boosting overall recommendation performance. Specifically, LLaRD generates denoising-related knowledge by first enriching semantic insights from observational data via LLMs and inferring user-item preference knowledge. It then employs a novel Chain-of-Thought (CoT) technique over user-item interaction graphs to reveal relation knowledge for denoising. Finally, it applies the Information Bottleneck (IB) principle to align LLM-generated denoising knowledge with recommendation targets, filtering out noise and irrelevant LLM knowledge. Empirical results demonstrate LLaRD's effectiveness in enhancing denoising and recommendation accuracy.

In graph classification, the out-of-distribution (OOD) issue is attracting great attention. To address this issue, a prevailing idea is to learn stable features, on the assumption that they are substructures causally determining the label and that their relationship with the label is stable to the distributional uncertainty. In contrast, the complementary parts termed environmental features, fail to determine the label solely and hold varying relationships with the label, thus ascribed to the possible reason for the distribution shift. Existing generalization efforts mainly encourage the model’s insensitivity to environmental features. While the sensitivity to stable features is promising to distinguish the crucial clues from the distributional uncertainty but largely unexplored. A paradigm of simultaneously exploring the sensitivity to stable features and insensitivity to environmental features is until-now lacking to achieve the generalizable graph classification, to the best of our knowledge. In this work, we conjecture that generalizable models should be sensitive to stable features and insensitive to environmental features. To this end, we propose a simple yet effective augmentation strategy for graph classification: Equivariant and Invariant Cross-Data Augmentation (EI-CDA). By employing equivariance, given a pair of input graphs, we first estimate their stable and environmental features via masks. Then, we linearly mix the estimated stable features of two graphs and encourage the model predictions faithfully reflect their mixed semantics. Meanwhile, by using invariance, we swap the estimated environmental features of two graphs and keep the predictions invariant. This simple yet effective strategy endows the models with both sensitivity to stable features and insensitivity to environmental features. Extensive experiments show that EI-CDA significantly improves performance and outperforms leading baselines. Our codes are available at: https://github.com/yongduosui/EI-GNN .

Masked modeling has recently achieved remarkable success in specific fields of vision and language, sparking a surge of interest in graph-related research. However, Masked Graph Modeling (MGM), which captures fine-grained local information by masking low-level elements such as nodes, edges, and features, limits itself to a sub-optimal position, particularly on tasks requiring high-quality graph-level representations. Such a local perspective disregards the graph's global information and structure. To address these limitations, we propose a novel graph pre-training framework called Graph Contrastive Masked Autoencoder (GCMAE). GCMAE leverages the strengths of both MGM and Graph Contrastive Learning (GCL) to provide a more comprehensive perspective of both local and global. Our frame-work uses instance discrimination to learn global representations of graphs and reconstructs the graph using masked low-level elements. We augment the framework with a novel multi-view augmentation module to further enhance the pre-trained model's robustness and generalization ability. We evaluate GCMAE on real-world biochemistry and social network datasets, conducting extensive experiments on both node and graph classification tasks and transfer learning on downstream graph classification tasks. Our experimental results demonstrate that GCMAE's comprehensive perspective of both local and global benefits model pre-training. Moreover, GCMAE outperforms existing MGM and GCL baselines, proving its effectiveness on downstream tasks. Our code is available at https://github.com/lyc0930/GCMAE.

In the realm of deep learning-based recommendation systems, the increasing computational demands, driven by the growing number of users and items, pose a significant challenge to practical deployment. This challenge is primarily twofold: reducing the model size while effectively learning user and item representations for efficient recommendations. Despite considerable advancements in model compression and architecture search, prevalent approaches face notable constraints. These include substantial additional computational costs from pre-training/re-training in model compression and an extensive search space in architecture design. Additionally, managing complexity and adhering to memory constraints is problematic, especially in scenarios with strict time or space limitations. Addressing these issues, this paper introduces a novel learning paradigm, Dynamic Sparse Learning (DSL), tailored for recommendation models. DSL innovatively trains a lightweight sparse model from scratch, periodically evaluating and dynamically adjusting each weight's significance and the model's sparsity distribution during the training. This approach ensures a consistent and minimal parameter budget throughout the full learning lifecycle, paving the way for "end-to-end" efficiency from training to inference. Our extensive experimental results underline DSL's effectiveness, significantly reducing training and inference costs while delivering comparable recommendation performance. We give an code link of our work: https://github.com/shuyao-wang/DSL.

Graph representation learning on vast datasets, like web data, has made significant strides. However, the associated computational and storage overheads raise concerns. In sight of this, Graph condensation (GCond) has been introduced to distill these large real datasets into a more concise yet information-rich synthetic graph. Despite acceleration efforts, existing GCond methods mainly grapple with efficiency, especially on expansive web data graphs. Hence, in this work, we pinpoint two major inefficiencies of current paradigms: (1) the concurrent updating of a vast parameter set, and (2) pronounced parameter redundancy. To counteract these two limitations correspondingly, we first (1) employ the Mean-Field variational approximation for convergence acceleration, and then (2) propose the objective of Gradient Information Bottleneck (GDIB) to prune redundancy. By incorporating the leading explanation techniques (e.g., GNNExplainer and GSAT) to instantiate the GDIB, our EXGC, the Efficient and eXplainable Graph Condensation method is proposed, which can markedly boost efficiency and inject explainability. Our extensive evaluations across eight datasets underscore EXGC's superiority and relevance. Code is available at https://github.com/MangoKiller/EXGC.

Graph anomaly detection (GAD) has various applications in finance, healthcare, and security. Graph Neural Networks (GNNs) are now the primary method for GAD, treating it as a task of semi-supervised node classification (normal vs. anomalous). However, most traditional GNNs aggregate and average embeddings from all neighbors, without considering their labels, which can hinder detecting actual anomalies. To address this issue, previous methods try to selectively aggregate neighbors. However, the same selection strategy is applied regardless of normal and anomalous classes, which does not fully solve this issue. This study discovers that nodes with different classes yet similar neighbor label distributions (NLD) tend to have opposing loss curves, which we term it as "loss rivalry". By introducing Contextual Stochastic Block Model (CSBM) and defining NLD distance, we explain this phenomenon theoretically and propose a Bi-level optimization Graph Neural Network (BioGNN), based on these observations. In a nutshell, the lower level of BioGNN segregates nodes based on their classes and NLD, while the upper level trains the anomaly detector using separation outcomes. Our experiments demonstrate that BioGNN outperforms state-of-the-art methods on four benchmarks and effectively mitigates "loss rivalry".

Causal effect estimation from networked observational data encounters notable challenges, primarily hidden confounders arising from network structure, or spillover effects that influence unit's outcomes based on neighboring treatment assignments. Existing graph neural network (GNN)-based methods have endeavored to address these challenges, utilizing the GNN's message-passing mechanism to capture hidden confounders or model spillover effects. However, they mainly focus on transductive causal effect learning on a single networked data, limiting their efficacy in inductive settings for real-world applications where networked data often originates from multiple environments influenced by potentially varying time or geographical regions. In light of this, we introduce the principle of invariance to the task of causal effect estimation on networked data, culminating in our Invariant Graph Learning (IGL) framework. Specifically, it first generates multiple networked data to simulate diverse environments from a given observational data. Then it further encourages the model to learn environment-invariant representations for confounders and spillover effects. Such a design enables the model to extrapolate beyond a single observed environment, thereby improving the performance of causal effect estimation in potential new environments. Extensive experiments on two real-world datasets demonstrates the superiority of our approach.

Knowledge graph (KG) demonstrates substantial potential for enhancing the performance of recommender systems. Due to its rich semantic content and associations among interactive entities, it can effectively alleviate inherent limitations in collaborative filtering (CF), such as data sparsity or cold-start issues. However, most existing knowledge-aware recommendation models indiscriminately aggregate all information in KG, without considering information specifically relevant to the recommendation task. Such indiscriminate aggregation could introduce additional noisy knowledge into representation learning, which can distort the understanding of users' genuine preferences, thereby sacrificing the recommendation quality. In this paper, we introduce the principle of invariance to the knowledge-aware recommendation, culminating in our Knowledge Graph Invariant Learning (KGIL) framework. It aims to discern and harness the task-relevant knowledge connections within KG to enhance the recommendation models. Specifically, we employ multiple environment generators to simulate diverse noisy KG-environments. Then we devise a novel attention learning mechanism for KG and user-item interaction graph, aiming to learn environment-invariant subgraphs. Leveraging an adversarial optimization strategy, we enhance the diversity of the environments, meanwhile, promote invariant representation learning across environments. We conduct extensive experiments on three datasets and compare KGIL with state-of-the-art methods. The experimental results further demonstrate the superiority of our approach.

In graph classification, attention- and pooling-based graph neural networks (GNNs) predominate to extract salient features from the input graph and support the prediction. They mostly follow the paradigm of “learning to attend,” which maximizes the mutual information between the attended graph and the ground-truth label. However, this paradigm causes GNN classifiers to indiscriminately absorb all statistical correlations between input features and labels in the training data without distinguishing the causal and noncausal effects of features. Rather than emphasizing causal features, the attended graphs tend to rely on noncausal features as shortcuts to predictions. These shortcut features may easily change outside the training distribution, thereby leading to poor generalization for GNN classifiers. In this article, we take a causal view on GNN modeling. Under our causal assumption, the shortcut feature serves as a confounder between the causal feature and prediction. It misleads the classifier into learning spurious correlations that facilitate prediction in in-distribution (ID) test evaluation while causing significant performance drop in out-of-distribution (OOD) test data. To address this issue, we employ the backdoor adjustment from causal theory—combining each causal feature with various shortcut features, to identify causal patterns and mitigate the confounding effect. Specifically, we employ attention modules to estimate the causal and shortcut features of the input graph. Then, a memory bank collects the estimated shortcut features, enhancing the diversity of shortcut features for combination. Simultaneously, we apply the prototype strategy to improve the consistency of intra-class causal features. We term our method as CAL+, which can promote stable relationships between causal estimation and prediction, regardless of distribution changes. Extensive experiments on synthetic and real-world OOD benchmarks demonstrate our method’s effectiveness in improving OOD generalization. Our codes are released at https://github.com/shuyao-wang/CAL-plus .

With the greater emphasis on privacy and security in our society, the problem of graph unlearning — revoking the influence of specific data on the trained GNN model, is drawing increasing attention. However, ranging from machine unlearning to recently emerged graph unlearning methods, existing efforts either resort to retraining paradigm, or perform approximate erasure that fails to consider the inter-dependency between connected neighbors or imposes constraints on GNN structure, therefore hard to achieve satisfying performance-complexity trade-offs.

In graph classification, attention- and pooling-based graph neural networks (GNNs) prevail to extract the critical features from the input graph and support the prediction. They mostly follow the paradigm of learning to attend, which maximizes the mutual information between the attended graph and the ground-truth label. However, this paradigm makes GNN classifiers recklessly absorb all the statistical correlations between input features and labels in the training data, without distinguishing the causal and noncausal effects of features. Instead of underscoring the causal features, the attended graphs are prone to visit the noncausal features as the shortcut to predictions. Such shortcut features might easily change outside the training distribution, thereby making the GNN classifiers suffer from poor generalization.

Media recommender systems aim to capture users’ preferences and provide precise personalized recommendation of media content. There are two critical components in the common paradigm of modern recommender models: (1) representation learning, which generates an embedding for each user and item; and (2) interaction modeling, which fits user preferences toward items based on their representations. In spite of great success, when a great amount of users and items exist, it usually needs to create, store, and optimize a huge embedding table, where the scale of model parameters easily reach millions or even larger. Hence, it naturally raises questions about the heavy recommender models: Do we really need such large-scale parameters? We get inspirations from the recently proposed lottery ticket hypothesis (LTH), which argues that the dense and over-parameterized model contains a much smaller and sparser sub-model that can reach comparable performance to the full model. In this paper, we extend LTH to media recommender systems, aiming to find the winning tickets in deep recommender models. To the best of our knowledge, this is the first work to study LTH in media recommender systems. With Matrix Factorization and Light Graph Convolution Networks as the backbone models, we found that there widely exist winning tickets in recommender models. On three media convergence data sets—Yelp2018, TikTok and Kwai, the winning tickets can achieve comparable recommendation performance with only 29 % ~ 48 % , 7 % ~ 10 %, and 3 % ~ 17 % of parameters, respectively.