Deep neural networks (DNNs) have achieved extensive progress in time series learning. However, they could suffer from performance degradation when it comes to label noise in the real world. Towards this end, this paper studies an underexplored yet realistic problem of noise-resistant time series learning and proposes a novel data-centric approach named Dual Data-centric Separation with Circular Mixup (DREAM) for this problem. The core of our DREAM is to explore and exploit the noisy data from dual data-centric views for reduced overfitting. On the one hand, we assume that samples with similar features share similar labels and infer the pseudo label of each sample using its affinity graph to capture the corresponding pseudo margin. On the other hand, we monitor the optimization status by simulating the mislabeled data to generate flexible criteria for accurate separation of clean and noisy samples. In addition, we leverage circular Mixup to interpolate between clean and noisy samples in the embedding space. These mixed samples are incorporated into a discrepancy-aware consistency learning framework to ensure robust time series representations of all the separated samples. Experimental results on a wide range of publicly accessible datasets reveal the effectiveness of our DREAM.

This paper investigates the task of unsupervised graph domain adaptation, which facilitates the transfer of knowledge from labeled source graphs to unlabeled target graphs. Recent approaches usually utilize graph contrastive learning and pseudo-labeling to learn from unlabeled target data, which could introduce potential biased representations and supervision of target graphs resulting from serious shifts across two domains. Towards this end, we propose a novel framework named Diffusion-guided Unbiased Discriminative Learning (DisCo) for unsupervised graph domain adaptation. The core of our DisCo is to leverage both feature disentanglement and cross-domain diffusion signals to remove the potential biases for target graphs. In particular, we first utilize adversarial feature disentanglement to extract causal features that are orthogonal to domain biases. More importantly, we retrieve the labels of cross-domain source graphs to generate the conditions, which would be utilized to optimize a diffusion model for label denoising. The consistency between pseudo-labels and denoised labels is measured to reduce the potential biases during domain alignment. Extensive experiments on several real-world benchmarks demonstrate that our proposed DisCo consistently outperforms competing state-of-the-art baselines.

Graph-level classification involves analyzing the property of the whole graph, which is typically solved by using graph neural networks (GNNs). Existing efforts generally assume a balanced class distribution. However, real-world data often exhibit long-tailed distributions, i.e., tail classes have significantly fewer samples than head classes, and thus directly applying GNNs is eventually biased toward the head classes, resulting in limited generalization over the tail classes. Moreover, the predictions of existing algorithms are usually not trustworthy, and the trained classifiers remain ignorant to their predictive confidence. Towards this end, in this paper we develop a principled framework called GraphEVER for long-tailed graph-level classification. Technically, GraphEVER incorporates the beliefs of multiple experts and leverages the idea of subjective logic within the Dempster-Shafer Evidence Theory (DST). It can provide the evidence and uncertainty estimation for each expert, where the evidence is parameterized by a Dirichlet distribution to model class probability distribution, and the uncertainty is quantified via a well-defined theoretical framework. In this way, diverse experts can be integrated under DST to endow the classifier with both reliability and robustness. Moreover, we propose an evidence-based routing mechanism to dynamically assign experts, such that the tail classes can receive more attention, while the head classes can reduce redundant engaged experts, further cutting down the computational cost and improving the efficiency. Extensive experiments on seven datasets verify the superiority of our proposed framework.

Out-of-Distribution (OOD) detection, which seeks to identify samples deviating from the In-Distribution (ID) training distribution at test time, is crucial for building robust machine learning systems. While extensive efforts have been made for Euclidean data, OOD detection on graph-structured data remains relatively underexplored. On the one hand, the specific properties of a graph may be attributed to its substructures. On the other hand, acquiring labeled data for graph learning is typically time-consuming and labor-intensive. Toward this end, in this paper, we propose a novel kernel-enhanced graph substructure learning framework termed KEGOD for unsupervised graph OOD detection. Specifically, we introduce a learnable graph generator to construct the augmented graph view that preserves distinguishable structure information. Then, for both the input graph and augmented view, a graph neural network (GNN) branch and a graph kernel (GK) branch are incorporated to explore graph latent patterns. By performing multi-branch concordance learning on the extracted graph patterns, our KEGOD captures complementary ID structural semantics in both implicit and explicit manners, enabling reliable detection of OOD graphs through semantic inconsistency. Finally, we build a self-adaptive training mechanism to automatically control diverse sensitivities of the graph patterns. Experimental results on several public graph datasets reveal the superiority of our KEGOD. Our code is available at~ https://github.com/jamesyifan/KEGOD.

Graph neural networks (GNNs) have essentially taken over as the de facto model for learning graph-structured data. However, the majority of existing methods perform transductive learning in a known graph, which is unable to tackle abundant in-the-wild unseen graphs with potential domain shifts. Even worse, these graphs, accompanied by domain shifts on structural topology and node attributes, bring in vulnerable data bias and thus a huge drop in performance. To tackle this, we propose a novel GNN method named spectral optimal transport (SPOT) for effective domain generalization on graphs. Our method is motivated by the fact that the high-frequency graph spectrum is more likely to indicate domain differences. In particular, we formulate the structural augmentation as an optimal transport problem to retain low-frequency key knowledge and solve the problem using Sinkhorn-Knopp algorithm. In addition, we incorporate an adaptive perturbation strategy to deep features, where the direction of the additive noise is determined by the homophily degrees to maintain semantic properties. Accordingly, we meticulously construct a collection of real-world benchmark datasets to assess the domain generalization capability of our model on graphs, and extensive experiments confirm the effectiveness of our proposed SPOT.

Out-of-distribution (OOD) detection has garnered increasing concern for identifying test samples that exhibit a distributional shift from the training dataset in practical deep learning applications. With the significant advancements in graph deep learning for graph representation, graph OOD detection has emerged as a research problem. Graph contrastive learning (GCL) is applied to graph OOD detection due to its capacity for learning discriminative representations in a self-supervised manner, thereby eliminating the need for time-consuming and labor-intensive label information. However, existing methods often neglect the explicit consideration of underlying semantics behind graph data distribution for OOD detection. We argue that simple data augmentations for GCL may risk disrupting the intrinsic graph structure while retaining redundant structural information, which hinders semantic discrimination between graphs. Additionally, Euclidean space embedding struggles to maintain hierarchical structural consistency, making it challenging to meaningfully capture the hierarchical semantic distribution of graph data. In response to these issues, we propose a novel framework termed HGOOD-D, which aims to explore latent semantic hierarchies in hyperbolic space for graph OOD detection. Specifically, we design a bottleneck graph extractor grounded in the information bottleneck (IB) principle, which captures the minimal sufficient information to distinguish graph patterns. Based on this, we introduce hierarchical contrastive learning to capture the hierarchical semantics within graph data distribution. These methods are based on hyperbolic space embedding that can preserve complex inter-relationships in graph hierarchies, thereby mitigating data distortion. Comprehensive evaluations on ten widely used benchmark datasets show that HGOOD-D consistently surpasses current state-of-the-art approaches in graph OOD detection.

Graph domain adaptation has emerged as a critical challenge in real-world applications, where labeled graph data is often scarce and expensive to obtain. While existing methods have shown promise, they typically require access to source domain data, which may be restricted due to privacy concerns or data regulations. To address these limitations, we investigate the challenging yet practical problem of source-free graph domain adaptation. We propose a new approach namedRobust CrossSupervision with Target Mining (ROSE) for this problem. ROSE achieves robustness by considering the complementary topology of graphs. The model consists of a message-passing branch for local semantic learning and a graph-kernel branch for global structural capture. Both branches are incorporated into a unified cross-supervision framework. To improve the robustness of the optimization process, we explore the context of the target domain, and divide the target data into discriminant set and anchor set. Then we incorporate the two tasks into a meta-learning optimization framework. Extensive experiments on benchmark datasets have demonstrated that our ROSE, compared with a wide range of baselines, always yields superior performance. The source code is available athttps://github.com/luo-junyu/ROSE.

Graph retrieval (GR), a ranking procedure that aims to sort the graphs in a database by their relevance to a query graph in decreasing order, has wide applications across diverse domains, such as visual object detection and myreddrug discovery. Existing Graph Retrieval (GR) approaches usually compare graph pairs at a detailed level and generate quadratic similarity scores. In realistic scenarios, conducting quadratic fine-grained comparisons is costly. However, coarse-grained comparisons would result in performance loss. Moreover, label scarcity in real-world data brings extra challenges. To tackle these issues, we investigate a more realistic GR problem, namely, efficient graph retrieval (EGR). Our key intuition is that, since there are numerous underutilized unlabeled pairs in realistic scenarios, by leveraging the additional information they provide, we can achieve speed-up while simplifying the model without sacrificing performance. Following our intuition, we propose an efficient model called Dual-TowerModel withDividing, Contrasting andAlignment (TowerDNA). TowerDNA utilizes a GNN-based dual-tower model as a backbone to quickly compare graph pairs in a coarse-grained manner. In addition, to effectively utilize unlabeled pairs, TowerDNA first identifies confident pairs from unlabeled pairs to expand labeled datasets. It then learns from remaining unconfident pairs via graph contrastive learning with geometric correspondence. To integrate all semantics with reduced biases, TowerDNA generates prototypes using labeled pairs, which are aligned within both confident and unconfident pairs. Extensive experiments on diverse realistic datasets demonstrate that TowerDNA achieves comparable performance to fine-grained methods while providing a 10× speed-up.

This paper investigates the problem of learning mesh-based physical simulations, which is a crucial task with applications in fluid mechanics and aerodynamics. Recent works typically utilize graph neural networks (GNNs) to produce next-time states on irregular meshes by modeling interacting dynamics, and then adopt iterative rollouts for the whole trajectories. However, these methods cannot achieve satisfactory performance in long-term predictions due to the failure of capturing long-term dependency and potential error accumulations. To tackle this, we introduce a new future-to-present learning perspective, and further develop a simple yet effective approach named Foresight And Interpolation (FAIR) for long-term mesh-based simulations. The main idea of our FAIR is to first learn a graph ODE model for coarse long-term predictions and then refine short-term predictions via interpolation. Specifically, FAIR employs a continuous graph ODE model that incorporates past states into the evolution of interacting node representations, which is capable of learning coarse long-term trajectories under a multi-task learning framework. Then, we leverage a channel aggregation strategy to summarize the trajectories for refined short-term predictions, which can be illustrated using an interpolation process. Through pyramid-like alternative propagation between the foresight step and refinement step, our proposed framework FAIR can generate accurate long-term trajectories, achieving a significant error reduction compared with the best baseline on four benchmark datasets. Extensive ablation studies and visualization further validate the superiority of our proposed FAIR.

Graph neural networks (GNNs) have shown remarkable success in graph-level classification tasks. However, most of the existing GNN-based studies are based on balanced datasets, while many real-world datasets exhibit long-tailed distributions. In such datasets, the tail classes receive limited attention during training, leading to prediction bias and degraded performance. To address this issue, a range of long-tailed learning strategies have been proposed, such as data re-balancing and transfer learning. However, these approaches encounter several challenges, including insufficient representation capacity for tail classes and their evaluation solely on uniform test data, limiting their capacity to handle unknown class distributions. To tackle these challenges, we introduce a novel framework, namely Knowledge-diverse Experts (KDEX) for long-tailed graph classification. Our KDEX leverages a dynamic memory module to enable the transfer of knowledge from head to tail, which improves the representation ability of the tail. To deal with unknown test distributions, KDEX introduces a knowledge-diverse expert training approach to train experts with different capacities in managing various test distributions. Moreover, we train the hierarchical router in a self-supervised manner to dynamically aggregate each knowledge-diverse expert during testing. Experimental results on multiple benchmarks reveal that our KDEX outperforms current baselines in both standard and test-agnostic long-tailed graph classification.

Collaborative Filtering (CF) is a pivotal research area in recommender systems that capitalizes on collaborative similarities between users and items to provide personalized recommendations. With the remarkable achievements of node embedding-based Graph Neural Networks (GNNs), we explore the upper bounds of expressiveness inherent to embedding-based methodologies and tackle the challenges by reframing the CF task as a graph-signal processing problem. To this end, we propose PolyCF, a flexible graph signal filter that leverages polynomial graph filters to process interaction signals. PolyCF exhibits the capability to capture spectral features across multiple eigenspaces through a series of Generalized Gram filters and is able to approximate the optimal polynomial response function for recovering missing interactions. A graph optimization objective and a pairwise ranking objective are jointly used to optimize the parameters of the convolution kernel. Experiments on three widely adopted datasets demonstrate the superiority of PolyCF over the state-of-the-art CF methods.

Molecular property prediction stands as a cornerstone task in AI-driven drug design and discovery, wherein the atoms within a molecule serve as nodes, collectively forming a graph with bonds acting as edges. Given the crucial role of geometric structures in molecular property prediction, the integration of 3D information with various graph learning methods has been explored to enhance prediction performance. Despite the increasing adoption of the "Graph pre-training and fine-tuning" paradigm to refine molecular representations, a significant challenge persists due to the misalignment between pre-training objectives and downstream tasks. Drawing inspiration from prompt tuning techniques in Natural Language Processing (NLP), several graph prompt-based methods have emerged. However, existing approaches tend to overlook the unique properties inherent in molecular graphs. To address this gap, our paper introduces a novel approach named 3D MO lecul A rpromp T (MOAT) designed specifically for geometric molecules. Specifically, we propose atom-level prompts to capture atom distribution, geometry-level prompts tailored for molecular conformers, where different conformations have distinct chemical properties, and task-level prompts to leverage functional group properties. Results on both 3D and 2D downstream tasks demonstrate its ability to successfully bridge the data gap across diverse settings. To the best of our knowledge, this paper is the first attempt to introduce geometric graph-prompting learning for molecules.

The recently developed unsupervised graph representation learning approaches apply contrastive learning into graph-structured data and achieve promising performance. However, these methods mainly focus on graph augmentation for positive samples, while the negative mining strategies for graph contrastive learning are less explored, leading to sub-optimal performance. To tackle this issue, we propose a Graph Adversarial Contrastive Learning (GraphACL) scheme that learns a bank of negative samples for effective self-supervised whole-graph representation learning. Our GraphACL consists of (i) a graph encoding branch that generates the representations of positive samples and (ii) an adversarial generation branch that produces a bank of negative samples. To generate more powerful hard negative samples, our method minimizes the contrastive loss during encoding updating while maximizing the contrastive loss adversarially over the negative samples for providing the challenging contrastive task. Moreover, the quality of representations produced by the adversarial generation branch is enhanced through the regularization of carefully designed bank divergence loss and bank orthogonality loss. We optimize the parameters of the graph encoding branch and adversarial generation branch alternately. Extensive experiments on 14 real-world benchmarks on both graph classification and transfer learning tasks demonstrate the effectiveness of the proposed approach over existing graph self-supervised representation learning methods.

Graph neural networks have pushed state-of-the-arts in graph classifications recently. Typically, these methods are studied within the context of supervised end-to-end training, which necessities copious task-specific labels. However, in real-world circumstances, labeled data could be limited, and there could be a massive corpus of unlabeled data, even from unknown classes as a complementary. Towards this end, we study the problem of semi-supervised universal graph classification, which not only identifies graph samples which do not belong to known classes, but also classifies the remaining samples into their respective classes. This problem is challenging due to a severe lack of labels and potential class shifts. In this paper, we propose a novel graph neural network framework named UGNN, which makes the best of unlabeled data from the subgraph perspective. To tackle class shifts, we estimate the certainty of unlabeled graphs using multiple subgraphs, which facilities the discovery of unlabeled data from unknown categories. Moreover, we construct semantic prototypes in the embedding space for both known and unknown categories and utilize posterior prototype assignments inferred from the Sinkhorn-Knopp algorithm to learn from abundant unlabeled graphs across different subgraph views. Extensive experiments on six datasets verify the effectiveness of UGNN in different settings.

Sequential recommendation aims at understanding user preference by capturing successive behavior correlations, which are usually represented as the item purchasing sequences based on their past interactions. Existing efforts generally predict the next item via modeling the sequential patterns. Despite effectiveness, there exist two natural deficiencies: (i) user preference is dynamic in nature, and the evolution of collaborative signals is often ignored; and (ii) the observed interactions are often irregularly-sampled, while existing methods model item transitions assuming uniform intervals. Thus, how to effectively model and predict the underlying dynamics for user preference becomes a critical research problem. To tackle the above challenges, in this paper, we focus on continuous-time sequential recommendation and propose a principled graph ordinary differential equation framework named GDERec. Technically, GDERec is characterized by an autoregressive graph ordinary differential equation consisting of two components, which are parameterized by two tailored graph neural networks (GNNs) respectively to capture user preference from the perspective of hybrid dynamical systems. On the one hand, we introduce a novel ordinary differential equation based GNN to implicitly model the temporal evolution of the user-item interaction graph. On the other hand, an attention-based GNN is proposed to explicitly incorporate collaborative attention to interaction signals when the interaction graph evolves over time. The two customized GNNs are trained alternately in an autoregressive manner to track the evolution of the underlying system from irregular observations, and thus learn effective representations of users and items beneficial to the sequential recommendation. Extensive experiments on five benchmark datasets demonstrate the superiority of our model over various state-of-the-art recommendation methods

Next Point-of-Interest (POI) recommendation is a critical task in location-based services that aim to provide personalized suggestions for the user’s next destination. Previous works on POI recommendation have laid focus on modeling the user’s spatial preference. However, existing works that leverage spatial information are only based on the aggregation of users’ previous visited positions, which discourages the model from recommending POIs in novel areas. This trait of position-based methods will harm the model’s performance in many situations. Additionally, incorporating sequential information into the user’s spatial preference remains a challenge. In this article, we propose Diff-POI : a Diffu sion-based model that samples the user’s spatial preference for the next POI recommendation. Inspired by the wide application of diffusion algorithm in sampling from distributions, Diff-POI encodes the user’s visiting sequence and spatial character with two tailor-designed graph encoding modules, followed by a diffusion-based sampling strategy to explore the user’s spatial visiting trends. We leverage the diffusion process and its reverse form to sample from the posterior distribution and optimized the corresponding score function. We design a joint training and inference framework to optimize and evaluate the proposed Diff-POI. Extensive experiments on four real-world POI recommendation datasets demonstrate the superiority of our Diff-POI over state-of-the-art baseline methods. Further ablation and parameter studies on Diff-POI reveal the functionality and effectiveness of the proposed diffusion-based sampling strategy for addressing the limitations of existing methods.

This paper studies the problem of traffic flow forecasting, which aims to predict future traffic conditions on the basis of road networks and traffic conditions in the past. The problem is typically solved by modeling complex spatio-temporal correlations in traffic data using spatio-temporal graph neural networks (GNNs). However, the performance of these methods is still far from satisfactory since GNNs usually have limited representation capacity when it comes to complex traffic networks. Graphs, by nature, fall short in capturing non-pairwise relations. Even worse, existing methods follow the paradigm of message passing that aggregates neighborhood information linearly, which fails to capture complicated spatio-temporal high-order interactions. To tackle these issues, in this paper, we propose a novel model named Dynamic Hypergraph Structure Learning (DyHSL) for traffic flow prediction. To learn non-pairwise relationships, our DyHSL extracts hypergraph structural information to model dynamics in the traffic networks, and updates each node representation by aggregating messages from its associated hyperedges. Additionally, to capture high-order spatio-temporal relations in the road network, we introduce an interactive graph convolution block, which further models the neighborhood interaction for each node. Finally, we integrate these two views into a holistic multi-scale correlation extraction module, which conducts temporal pooling with different scales to model different temporal patterns. Extensive experiments on four popular traffic benchmark datasets demonstrate the effectiveness of our proposed DyHSL compared with a broad range of competing baselines.

Point-of-Interest (POI) recommendation plays a vital role in various location-aware services. It has been observed that POI recommendation is driven by both sequential and geographical influences. However, since there is no annotated label of the dominant influence during recommendation, existing methods tend to entangle these two influences, which may lead to sub-optimal recommendation performance and poor interpretability. In this paper, we address the above challenge by proposing DisenPOI, a novel Disentangled dual-graph framework for POI recommendation, which jointly utilizes sequential and geographical relationships on two separate graphs and disentangles the two influences with self-supervision. The key novelty of our model compared with existing approaches is to extract disentangled representations of both sequential and geographical influences with contrastive learning. To be specific, we construct a geographical graph and a sequential graph based on the check-in sequence of a user. We tailor their propagation schemes to become sequence-/geo-aware to better capture the corresponding influences. Preference proxies are extracted from check-in sequence as pseudo labels for the two influences, which supervise the disentanglement via a contrastive loss. Extensive experiments on three datasets demonstrate the superiority of the proposed model.

In this paper, we study semi-supervised graph classification, a fundamental problem in data mining and machine learning. The problem is typically solved by learning graph neural networks with pseudo-labeling or knowledge distillation to incorporate both labeled and unlabeled graphs. However, these methods usually either suffer from overconfident and biased pseudo-labels or suboptimal distillation caused by the insufficient use of unlabeled data. Inspired by the recent progress of contrastive learning and dual learning, we propose DualGraph, a principled framework to leverage unlabeled graphs more effectively for semi-supervised graph classification. DualGraph consists of a prediction module and a retrieval module to model graphs$G$and their labels$y$from opposite while complementary views (i.e., p(y | G) and p(G | y) respectively). The two modules are jointly trained via posterior regularization, which encourages their inter-module consistency on unlabeled graphs. Moreover, we improve model training for each module with a contrastive learning framework to encourage the intra-module consistency on unlabeled data. Experimental results on a range of publicly accessible datasets reveal the effectiveness of our DualGraph.

Point-of-Interest (POI) recommendation, which benefits from the proliferation of GPS-enabled devices and location-based social networks (LBSNs), plays an increasingly important role in recommender systems. It aims to provide users with the convenience to discover their interested places to visit based on previous visits and current status. Most existing methods usually merely leverage recurrent neural networks (RNNs) to explore sequential influences for recommendation. Despite the effectiveness, these methods not only neglect topological geographical influences among POIs, but also fail to model high-order sequential substructures. To tackle the above issues, we propose a Kernel-Based Graph Neural Network (KBGNN) for next POI recommendation, which combines the characteristics of both geographical and sequential influences in a collaborative way. KBGNN consists of a geographical module and a sequential module. On the one hand, we construct a geographical graph and leverage a message passing neural network to capture the topological geographical influences. On the other hand, we explore high-order sequential substructures in the user-aware sequential graph using a graph kernel neural network to capture user preferences. Finally, a consistency learning framework is introduced to jointly incorporate geographical and sequential information extracted from two separate graphs. In this way, the two modules effectively exchange knowledge to mutually enhance each other. Extensive experiments conducted on two real-world LBSN datasets demonstrate the superior performance of our proposed method over the state-of-the-arts. Our codes are available at https://github.com/ ang6ang/KBGNN.

This paper studies semi-supervised graph classification, which is an important problem with various applications in social network analysis and bioinformatics. This problem is typically solved by using graph neural networks (GNNs), which yet rely on a large number of labeled graphs for training and are unable to leverage unlabeled graphs. We address the limitations by proposing the Kernel-based Graph Neural Network (KGNN). A KGNN consists of a GNN-based network as well as a kernel-based network parameterized by a memory network. The GNN-based network performs classification through learning graph representations to implicitly capture the similarity between query graphs and labeled graphs, while the kernel-based network uses graph kernels to explicitly compare each query graph with all the labeled graphs stored in a memory for prediction. The two networks are motivated from complementary perspectives, and thus combing them allows KGNN to use labeled graphs more effectively. We jointly train the two networks by maximizing their agreement on unlabeled graphs via posterior regularization, so that the unlabeled graphs serve as a bridge to let both networks mutually enhance each other. Experiments on a range of well-known benchmark datasets demonstrate that KGNN achieves impressive performance over competitive baselines.