Memory-based temporal graph neural networks (MTGNN) use node memory to store historical information, enabling efficient processing of large dynamic graphs through batch parallel training, with larger batch sizes leading to increased training efficiency. However, this approach overlooks the interdependency among edges within the same batch, leading to outdated memory states and reduced training accuracy. Previous studies have attempted to mitigate this issue through methods such as measuring memory loss, overlap training, and additional compensation modules. Despite these efforts, challenges persist, including imprecise coarse-grained memory loss measurement and ineffective compensation modules. To address these challenges, we propose the Refined Batch parallel Training (RBT) framework, which accurately evaluates intra-batch information loss and optimizes batch partitioning to minimize loss, enhancing the training process's effectiveness and efficiency. RBT also includes a precise and efficient memory compensation algorithm. Experimental results demonstrate RBT's superior performance compared to existing MTGNN frameworks like TGL, ETC, and PRES in terms of training efficiency and accuracy across various dynamic graph datasets. Our code is made publicly available at https://github.com/fengwudi/RBT.

Temporal Graph Transformers (TGTs), which incorporates Transformer into the temporal graph learning models, are powerful tools for analyzing and predicting temporal graph data. However, most existing TGT models focus on one-hop interactions due to sequence correlation and computational complexity caused by neighborhood explosion. This limited focus on local subgraph structures restricts the representational power of current TGTs. Additionally, the introduction of higher-order structures exacerbates efficiency issues in TGTs, with the time-consuming feature processing stage often neglected, leading to low training efficiency. To address these challenges, we propose QSFormer (Quick Sense Temporal Graph TransFormer), a solution designed to enhance local sensation ability and accelerate training efficiency in TGTs. QSFormer includes a sense augmentation strategy that incorporates high-order neighbor-hoods with position-differentiated encoding and extends common neighbor. Furthermore, QSFormer implements a quick training framework for TGTs to accelerate feature processing and model convergence, including padded parallel sampling and adaptive mini-batch generation. Extensive experiments demonstrate that QSFormer consistently outperforms existing baselines, including TGTs such as DyGFormer and HOT. Notably, QSFormer surpasses these TGTs in training speed by over four and seven times, respectively. Our code is publicly available at https://github.com/Stephanie0002/QSFormer.

Dynamic graph learning is crucial for accurately modeling complex systems by integrating topological structure and temporal information within graphs. While memory-based methods are commonly used and excel at capturing short-range temporal correlations, they struggle with modeling long-range dependencies, harmonizing long-range and short-range correlations, and integrating structural information effectively. To address these challenges, we present SALoM: Structure Aware Temporal Graph Networks with Long-Short Memory Updater. SALoM features a memory module that addresses gradient vanishing and information forgetting, enabling the capture of long-term dependencies across various time scales. Additionally, SALoM utilizes a long-short memory updater (LSMU) to dynamically balance long-range and short-range temporal correlations, preventing over-generalization. By integrating co-occurrence encoding and LSMU through information bottleneck-based fusion, SALoM effectively captures both the structural and temporal information within graphs. Experimental results across various graph datasets demonstrate SALoM's superior performance, achieving state-of-the-art results in dynamic graph link prediction. Our code is openly accessible at https://github.com/wave5418/SALoM.

Memory-based temporal graph neural network (MTGNN) models are effective for predicting temporal graphs by using node memory and message-passing modules to capture temporal and structural information, respectively. However, distributed training for large graphs presents challenges such as accuracy loss and decreased efficiency due to remote features and memory transmission. Despite improvements in MTGNN system optimizations, issues like dynamic load imbalances, communication overhead, and memory staleness persist. To tackle these challenges, we introduce MemShare, a distributed MTGNN system. MemShare introduces a novel shared node memory paradigm that utilizes a small subset of shared nodes across machines and GPUs to reduce distributed communication for memory management. It incorporates techniques like shared nodes-centric graph partitioning, shared nodes-aware boundary decay sampling, and shared nodes-targeted synchronous smoothing aggregation. Experiments show that MemShare outperforms existing distributed MTGNN systems in accuracy and training efficiency.