Despite their success, large language models (LLMs) suffer from notorious hallucination issue.By introducing external knowledge stored in knowledge graphs (KGs), existing methods use paths as the medium to represent the graph information sent into LLMs.However, paths only contain limited graph structure information and are unorganized with redundant sequentially appearing keywords, which are difficult for LLMs to digest.We aim to find a suitable medium that captures the essence of structural knowledge in KGs.Inspired by Neural Message Passing in Graph Neural Networks, we propose Language Message Passing (LMP), which first learns a concise facts graph by iteratively aggregating neighbor entities and transforming them into semantic facts, and then performs Topological Readout that encodes the graph structure information into multi-level lists of texts to augment LLMs.Our method serves as a brand-new innovative framework that brings a new perspective into KG-enhanced LLMs, and also offers humanlevel semantic explainability with significant performance improvements over existing methods on all five knowledge graph question answering datasets.

Although spatio-temporal graph neural networks (STGNNs) become widely used methods in traffic forecasting, they still encounter an issue named short-sightedness. Specifically, due to high model complexity and GPU memory usage, STGNNs are restricted to processing only very short input time series. This limited context often causes STGNNs to focus on local variations and overlook long-term patterns, leading to misinterpretation of time series trends. To tackle this issue, recent studies propose to perform mask reconstruction pre-training on traffic series to enhance STGNNs. However, we argue that mask reconstruction is a suboptimal pre-training paradigm for traffic forecasting, because there exists a great gap between pre-training and downstream forecasting, caused by their inconsistent training targets. To eliminate this gap, we propose a new pre-training paradigm named next patch prediction and prove its advantages from both empirical and theoretical perspectives. Based on this paradigm, we introduce a new framework called Decoder-only Pre-training Enhancement (DoP) to unleash the potential of traffic pre-training model. Specifically, DoP uses Transformer decoders as infrastructure, and leverages next patch prediction as target to conduct pre-training. In addition, we propose a new dual-view temporal embedding to fully capture temporal information and spatial spectral enhancement to model spatial information. After pre-training, DoP enhances existing STGNNs seamlessly with periodic enhancement mechanism. On four real-world traffic benchmarks, we demonstrate its start-of-the-art performance.

Graph Neural Networks (GNNs) have been a powerful tool to acquire high-quality node representations dealing with graphs, which strongly depends on a promising graph structure. In the real world scenarios, it is inevitable to introduce noises in graph topology. To prevent GNNs from the disturbance of irrelevant edges or missing edges, graph structure learning is proposed and has attracted considerable attentions in recent years. In this paper, we argue that current graph structure learning methods still pay no regard to the status of nodes and just judge all of their connections simultaneously using a monotonous standard, which will lead to indeterminacy and instability in the optimization process. We designate these methods as status-unaware models. To demonstrate the rationality of our point of view, we conduct exploratory experiments on publicly available datasets, and discover some exciting observations. Afterwards, we propose a new model named Graph Structure Learning via Progressive Strategy (PROSE) according to the observations, which uses a progressive strategy to acquire ideal graph structure in a status-aware way. Concretely, PROSE consists of progressive structure splitting module (PSS) and progressive structure refining module (PSR) to modify node connections according to their global potency, and we also introduce horizontal position encoding and vertical position encoding in order to capture fruitful graph topology information ignored by previous methods. On several widely-used graph datasets, we conduct extensive experiments to demonstrate the effectiveness of our model, and the source code 1 https://github.com/tigerbunny2023/PROSE is provided.

While Graph Neural Networks (GNNs) have become de facto criterion in graph representation learning, they still suffer from label scarcity and poor generalization. To alleviate these issues, graph pre-training has been proposed to learn universal patterns from unlabeled data via applying self-supervised tasks. Most existing graph pre-training methods only use a single self-supervised task, which will lead to insufficient knowledge mining. Recently, there are also some works that try to use multiple self-supervised tasks, however, we argue that these methods still suffer from a serious problem, which we call it graph structure impairment. That is, there actually exists structural gaps among several tasks due to the divergence of optimization objectives, which means customized graph structures should be provided for different self-supervised tasks. Graph structure impairment not only significantly hurts the generalizability of pre-trained GNNs, but also leads to suboptimal solution, and there is no study so far to address it well. Motivated by Meta-Cognitive theory, we propose a novel model named Core Graph Cognizing and Differentiating (CORE) to deal with the problem in an effective approach. Specifically, CORE consists of cognizing network and differentiating process, the former cognizes a core graph which stands for the essential structure of the graph, and the latter allows it to differentiate into several task-specific graphs for different tasks. Besides, this is also the first study to combine graph pre-training with cognitive theory to build a cognition-aware model. Several experiments have been conducted to demonstrate the effectiveness of CORE.

Epilepsy is one of the most serious neurological diseases, affecting 1-2% of the world's population. The diagnosis of epilepsy depends heavily on the recognition of epileptic waves, i.e., disordered electrical brainwave activity in the patient's brain. Existing works have begun to employ machine learning models to detect epileptic waves via cortical electroencephalogram (EEG), which refers to brain data obtained from a noninvasive examination performed on the patient's scalp surface to record electrical activity in the brain. However, the recently developed stereoelectrocorticography (SEEG) method provides information in stereo that is more precise than conventional EEG, and has been broadly applied in clinical practice. Therefore, in this paper, we propose the first data-driven study to detect epileptic waves in a real-world SEEG dataset. While offering new opportunities, SEEG also poses several challenges. In clinical practice, epileptic wave activities are considered to propagate between different regions in the brain. These propagation paths, also known as the epileptogenic network, are deemed to be a key factor in the context of epilepsy surgery. However, the question of how to extract an exact epileptogenic network for each patient remains an open problem in the field of neuroscience. Moreover, the nature of epileptic waves and SEEG data inevitably leads to extremely imbalanced labels and severe noise. To address these challenges, we propose a novel model (BrainNet) that jointly learns the dynamic diffusion graphs and models the brain wave diffusion patterns. In addition, our model effectively aids in resisting label imbalance and severe noise by employing several self-supervised learning tasks and a hierarchical framework. By experimenting with the extensive real SEEG dataset obtained from multiple patients, we find that BrainNet outperforms several latest state-of-the-art baselines derived from time-series analysis.