Rationalization, a data-centric framework, aims to build self-explanatory models to explain the prediction outcome by generating a subset of human-intelligible pieces of the input data. It involves a cooperative game model where a generator generates the most human-intelligible parts of the input (i.e., rationales), followed by a predictor that makes predictions based on these generated rationales. Conventional rationalization methods typically impose constraints via regularization terms to calibrate or penalize undesired generation. However, these methods are suffering from a problem called mode collapse, in which the predictor produces correct predictions yet the generator consistently outputs rationales with collapsed patterns. Moreover, existing studies are typically designed separately for specific collapsed patterns, lacking a unified consideration. In this paper, we systematically revisit cooperative rationalization from a novel game-theoretic perspective and identify the fundamental cause of this problem: the generator no longer tends to explore new strategies to uncover informative rationales, ultimately leading the system to converge to a suboptimal game equilibrium (correct predictionsv.scollapsed rationales). To solve this problem, we then propose a novel approach, Game-theoreticPolicyOptimization orientedRATionalization (PoRat), which progressively introduces policy interventions to address the game equilibrium in the cooperative game process, thereby guiding the model toward a more optimal solution state. We theoretically analyse the cause of such a suboptimal equilibrium and prove the feasibility of the proposed method. Furthermore, we validate our method on nine widely used real-world datasets and two synthetic settings, wherePoRatachieves up to 8.1% performance improvements over existing state-of-the-art methods. Code and data are available atanonymous.4open.science/r/Rationalization-PORAT-ECE9.

Cold-start sequential recommendation, where user interaction histories are sparse or minimal, remains a significant challenge in recommendation systems. Current meta-learning-based approaches rely heavily on the interaction histories of regular users to construct meta-tasks, aiming to acquire prior knowledge for cold-start adaptation. However, these methods often fail to account for preference discrepancies between regular and cold-start users, leading to biased preference modeling and suboptimal recommendations. To address this issue, we propose a novel counterfactual task-augmented meta-learning method for cold-start sequential recommendations. Our approach intervenes in user interaction histories to create counterfactual sequences that simulate potential but unrealized user behaviors, establishing counterfactual tasks within a meta-learning framework. Additionally, we aggregate meta-path neighbors to uncover latent relationships between items, enabling more detailed and accurate modeling of user preferences. Moreover, by integrating real and counterfactual task losses, we jointly optimize the model through a combination of global and local updates, enhancing its adaptability to cold-start scenarios. Extensive experiments demonstrate that our method significantly outperforms existing state-of-the-art techniques, achieving superior results in cold-start sequential recommendation tasks.

Graph Neural Networks are powerful tools for modeling graph-structured data but their interpretability remains a significant challenge. Existing model-agnostic GNN explainers aim to identify critical subgraphs or node features relevant to task predictions but often rely on GNN predictions for supervision, lacking ground-truth explanations. This limitation can introduce biases, causing explanations to fail in accurately reflecting the GNN's decision-making processes. To address this, we propose a novel explainer for GNNs with graph segmentation and contrastive learning. Our model introduces a graph segmentation learning module to divide the input graph into explanatory and redundant subgraphs. Next, we implement edge perturbation to augment these subgraphs, generating multiple positive and negative pairs for contrastive learning between explanatory and redundant subgraphs. Finally, we develop a contrastive learning module to guide the learning of explanatory and redundant subgraphs by pulling positive pairs with the same explanatory subgraphs closer while pushing negative pairs with different explanatory subgraphs far away. This approach allows for a clearer distinction of critical subgraphs, enhancing the fidelity of the explanations. We conducted extensive experiments on graph classification and node classification tasks, demonstrating the effectiveness of the proposed method.

Graph contrastive learning (GCL) has become a hot topic in the field of graph representaion learning. In contrast to traditional supervised learning relying on a large number of labels, GCL exploits augmentation techniques to generate multiple views and positive/negative pairs, both of which greatly influence the performance. Unfortunately, commonly used random augmentations may disturb the underlying semantics of graphs. Moreover, traditional GNNs, a type of widely employed encoders in GCL, are inevitably confronted with over-smoothing and over-squashing problems. To address these issues, we propose GNN-Transformer Cooperative Architecture for Trustworthy Graph Contrastive Learning (GTCA), which inherits the advantages of both GNN and Transformer, incorporating graph topology to obtain comprehensive graph representations. Theoretical analysis verifies the trustworthiness of the proposed method. Extensive experiments on benchmark datasets demonstrate state-of-the-art empirical performance.

Multi-hop question answering (MHQA) aims to utilize multi-source intensive documents retrieved to derive the answer. However, it is very challenging to model the importance of knowledge retrieved. Previous approaches primarily emphasize single-step and multi-step iterative decomposition or retrieval, which are susceptible to failure in long-chain reasoning due to the progressive accumulation of erroneous information. To address this problem, we propose a novel Local-tO-Global optimized retrieval method (LOG) to discover more beneficial information, facilitating the MHQA. In particular, we design a pointwise conditional v-information based local information modeling to cover usable documents with reasoning knowledge. We also improve tuplet objective loss, advancing multi-examples-aware global optimization to model the relationship between scattered documents. Extensive experimental results demonstrate our proposed method outperforms prior state-of-the-art models, and it can significantly improve multi-hop reasoning, notably for long-chain reasoning.

Graph contrastive learning has attracted great interest as a dominant and promising self-supervised representation learning approach in recent years. While existing works follow the basic principle of pulling positive pairs closer and pushing negative pairs far away, they still suffer from several critical problems, such as the underlying semantic disturbance brought by augmentation strategies, the failure of GCN in capturing long-range dependence, rigidness and inefficiency of node sampling techniques. To address these issues, we propose Manifold Learning Inspired Lightweight Graph Contrastive Learning (ML$^2$-GCL), which inherits the merits of both manifold learning and GCN. ML$^2$-GCL avoids the potential risks of semantic disturbance with only one single view. It achieves global nonlinear structure recovery from locally linear fits, which can make up for the defects of GCN. The most amazing advantage is about the lightweight due to its closed-form solution of positive pairs weights and removal of pairwise distances calculation. Theoretical analysis proves the existence of the optimal closed-form solution. Extensive empirical results on various benchmarks and evaluation protocols demonstrate effectiveness and lightweight of ML$^2$-GCL. We release the code at https://github.com/a-hou/ML2-GCL.

Dynamic graph representation learning using Spiking Neural Networks (SNNs) exploits the temporal spiking behavior of neurons, offering advantages in capturing the temporal evolution and sparsity of dynamic graphs. However, existing SNN-based methods often fail to effectively capture the impact of latency in information propagation on node representations. To address this, we propose Delay-DSGN, a dynamic spiking graph neural network incorporating a learnable delay mechanism. By leveraging synaptic plasticity, the model dynamically adjusts connection weights and propagation speeds, enhancing temporal correlations and enabling historical data to influence future representations. Specifically, we introduce a Gaussian delay kernel into the neighborhood aggregation process at each time step, adaptively delaying historical information to future time steps and mitigating information forgetting. Experiments on three large-scale dynamic graph datasets demonstrate that Delay-DSGN outperforms eight state-of-the-art methods, achieving the best results in node classification tasks. We also theoretically derive the constraint conditions between the Gaussian kernel’s standard deviation and size, ensuring stable training and preventing gradient explosion and vanishing issues.

Graph Neural Ordinary Differential Equations (GODE) integrate the Variational Autoencoder (VAE) framework with differential equations, effectively modeling latent space uncertainty and continuous dynamics, excelling in graph data evolution and incompleteness. However, existing GODE face challenges in capturing time-varying relationships and nonlinear node state evolution, which limits their ability to model complex dynamic graphs. To address these issues, we propose the ControlSynth Graph ODE (CSG-ODE). In the VAE encoding phase, CSG-ODE introduces an information transmission-based inter-node importance weighting mechanism, integrating it with latent correlations to guide adaptive graph convolutional recurrent networks for temporal node embedding. During decoding, CSG-ODE employs ODE to model node dynamics, capturing nonlinear evolution through sub-networks with nonlinear activations. For scenarios or prediction tasks that require stability, we extend CSG-ODE to stable CSG-ODE (SCSG-ODE) by constraining weight matrices to learnable anti-symmetric forms, theoretically ensuring enhanced stability. Experiments on traffic, motion capture, and simulated physical systems datasets demonstrate that CSG-ODE outperforms state-of-the-art GODE, while SCSG-ODE achieves both superior performance and optimal stability.

Dynamic graph representation learning aims to generate low-dimensional latent vector representations of graphs or nodes at various time points from evolving graph datas, which are then used for downstream tasks like link prediction and node classification. Existing graph neural network-based approaches primarily focus on modeling node and edge interactions, but fail to capture the spatial structure of these interactions, limiting their ability to represent nodes at different time points within the dynamic graph. To overcome the limitations, we propose a novel dynamic graph neural network representation learning method based on motif reconstruction. The method begins with a dynamic motif sampling strategy that considers both temporal and spatial dimensions to capture the evolving interaction patterns of node neighborhoods. Next, a dynamic graph neural network model based on motif reconstruction is developed, using dynamic motif reconstruction to create a new computational graph for aggregating information from node neighborhoods. Additionally, we propose a motif information aggregation method based on neural ordinary differential equations, which enables the representation of nodes at any given time. We conduct comparative experiments on seven publicly available dynamic graph datasets, benchmarking our method against six state-of-the-art dynamic graph representation learning methods. The results demonstrate that our approach outperforms others in both dynamic link prediction and node classification tasks.

Few-shot graph node classification aims to utilize graph structure and node features to make predictions on unlabeled nodes, given only a few labeled nodes. Although existing methods have made progress in alleviating the issue of label scarcity, relatively limited attention has been paid to the domain shift problem. The discrepancy in feature distributions between the source and target domains exacerbates the challenges of transfer learning, while the scarcity of labels restricts the model’s ability to effectively learn class-specific features in the target domain. To address this, this paper proposes a novel meta-learning approach that integrates domain adaptation and information augmentation to tackle the few-shot graph node classification problem. Specifically, we first align the feature distributions of the source and target domains using a Domain-adversarial Variational Autoencoder (DA-VAE) and combine pseudo-label generation and virtual edge addition strategies to enhance inter-class discriminability and mitigate the negative effects of over-alignment. We then design a multi-scale contrastive learning mechanism that optimizes node representations at three levels: feature, node, and subgraph. These strategies are integrated into a meta-learning framework to further enhance cross-domain transferability. Experimental results on five public graph datasets, compared with seven methods, demonstrate that the proposed method significantly improves node classification performance even under extremely weak supervision.

Dynamic graph representation learning aims to capture the evolution of graph structures and obtain accurate node embeddings, a crucial task in graph machine learning. The Hawkes point process, a mathematical framework effective for modeling the influence of historical events on future occurrences, has been validated as a powerful tool for capturing the dynamics of graph evolution in dynamic graph representation learning. However, existing dynamic graph representation learning methods based on the Hawkes point process primarily model excitation at the individual node level, failing to adequately account for structural influences during graph evolution. This limitation restricts their ability to comprehensively capture network evolution patterns. To address this limitation, we propose a Hawkes Point Process-enhanced Dynamic Graph Neural Network (HP-DGNN) model. This model leverages the Hawkes point process to model both individual node histories and structural histories, capturing their respective influences on future node interactions. By integrating individual and structural influences in computing Hawkes conditional intensity, the model comprehensively captures the impacts of both layers on future node interactions. We evaluate our proposed model on two downstream tasks of dynamic graph representation learning: dynamic link prediction and future node degree prediction. Compared to 12 state-of-the-art methods, our model consistently demonstrates superior performance, underscoring its effectiveness in capturing the complexities of graph evolution.

Graph Neural Networks (GNNs) have attracted significant research interest in various graph data modeling tasks. To advance trustworthy, reliable, and safe Artificial Intelligence (AI) systems for practical applications, adversarial robustness learning on GNNs has drawn widespread attention among researchers. Numerous attack methods, including white-box attacks, gray-box attacks, and black-box attacks, have been proposed, but black-box attacks are widely considered to be the most challenging and practical in real-world applications. In this paper, we focus on the challenging and realistic black-box attack scenario on GNNs, where the attacker has no information about the structure and parameters of the target model. We first theoretically demonstrate that the loss changes of the GNNs are related to the node voting matrix, which is subject to the graph topology information and is independent to the structures of GNNs. Then, we propose a novel black-box attack strategy for GNNs based on the theoretical results, i.e., node voting influence-based GNNs black-box adversarial attack, named VoteAttack. Specifically, the VoteAttack algorithm iteratively chooses a group of significant nodes based on mutual voting among nodes (the node voting matrix) and considers the voting weights among nodes. Furthermore, the VoteAttack algorithm modifies the attributes of the selected nodes to create a perturbed graph and ultimately utilizes the perturbed graph to attack GNNs. Experimental results on popular GNNs and graph datasets indicate that the proposed attack strategy outperforms baseline strategies.

Although deep learning has been extensively studied and achieved remarkable performance on single image super-resolution (SISR), existing convolutional neural networks (CNN) mainly focus on broader and deeper architecture design, ignoring the detailed information of the image itself and the potential relationship between the features. Recently, several attempts have been made to address the SISR with graph representation learning. However, existing GNN-based methods learning to deal with the SISR problem are limited to the information processing of the entire image or the relationship processing between different feature images of the same layer, ignoring the interdependence between the extracted features of different layers, which is not conducive to extracting deeper hierarchical features. In this paper, we propose an interlayer feature representation based graph neural network for image super-resolution (LSGNN), which consists of a layer feature graph representation learning module and a channel spatial attention module. The layer feature graph representation learning module mainly captures the interdependence between the features of different layers, which can learn more fine-grained image detail features. In addition, we also unified a channel attention module and a spatial attention module into our model, which takes into account the channel dimension information and spatial scale information, to improve the expressive ability, and achieve high quality image details. Extensive experiments and ablation studies demonstrate the superiority of the proposed model.

Link prediction has become a significant research problem in deep learning, and the graph-based autoencoder model is one of the most important methods to solve it. The existing graph-based autoencoder models only learn a single set of distributions, which cannot accurately represent the mixed distribution in real graph data. Meanwhile, existing learning models have been greatly restricted when the graph data has insufficient attribute information and inaccurate topology information. In this paper, we propose a novel graph embedding framework, termed multi-scale variational graph autoencoder (MSVGAE), which learns multiple sets of low-dimensional vectors of different dimensions through the graph encoder to represent the mixed probability distribution of the original graph data, and performs multiple sampling in each dimension. Furthermore, a self-supervised learning strategy (i.e., graph feature reconstruction auxiliary learning) is introduced to fully use the graph attribute information to help the graph structure learning. Experiment studies on real-world graphs demonstrate that the proposed model achieves state-of-the-art performance compared with other baseline methods in link prediction tasks. Besides, the robustness analysis shows that the proposed MSVGAE method has obvious advantages in the processes of graph data with insufficient attribute information and inaccurate topology information.

Cold-start link prediction is a term for information starved link prediction where little or no topological information is present to guide the determination of whether links to a node will form. Due to the lack of topological information, traditional topology-based link prediction methods cannot be applied to solve the cold-start link prediction problem. Therefore, an effective approach is presented through establishing connections between non-topological and topological information. In the approach, topological information is first extracted by a latent-feature representation model, then a logistic model is proposed to establish the connections between topological and non-topological information, and finally the linking possibility between cold-start users and existing users is calculated. Experiments with three types of real-world social networks Weibo, Facebook, and Twitter show that the proposed approach is more effective in solving the cold-start link prediction problem and establishing connections between topological and non-topological information.

Ru Li, Juan Wu, Zhiqiang Wang, Qinghua Chai. Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2015.