Enabling real-time GNN inference services requires low end-to-end latency to meet service level agreements. However, intensive preparation steps and the neighborhood explosion problem pose significant challenges to efficient GNN inference serving. In this paper, we propose FLAG, an FPGA-based GNN inference serving system using vector quantization. To reduce preparation overhead, we introduce offline preprocessing to precompute and compress hidden embeddings for serving. A dedicated FPGA accelerator leverages the precomputed data to enable lightweight aggregation. As a result, FLAG achieves average speedups of $154 \times 176 \times$, and $333 \times$ on three GNN models compared to the baseline system.

Machine Learning Force Fields (MLFFs) have emerged as a key technique in molecular dynamics (MD) simulations, enabling accurate prediction of molecular energies and forces with a level of precision comparable to that of Density Functional Theory (DFT), while significantly reducing computational cost. However, applying MLFFs to real-world scenarios, such as semiconductor process simulations involving massive atomic scales and long simulation times, exposes severe performance limitations, resulting in current MLFF models being inadequate for practical use and emphasizing the necessity of significant computational acceleration. Graph Neural Networks (GNNs) are frequently utilized to construct MLFFs, with tensor product operations constituting a substantial portion of the computational workload. These operations involve high-dimensional tensors whose sizes are determined by the number of channels and the order of rotation. This results in frequent memory accesses and low data reuse. This leads to memory bottlenecks that limit the efficient use of GPU computational resources. In this work, we propose a tensor-based function fusion technique to improve data continuity between functions, thereby increasing on-chip data reuse and reducing off-chip memory access. Furthermore, we identified inefficient utilization of hardware resources and addressed this by restructuring the computation flow and removing redundant operations to enhance computational performance. To optimize performance, we have also designed a dedicated tensor product hardware architecture that is optimized for GNN-based MLFFs. The experimental results demonstrate that the proposed system achieves an average speedup of 6.26× and a 424× improvement in energy efficiency compared to GPU-based execution, while maintaining the original model accuracy.

As online services based on graph databases increasingly integrate with machine learning, serving low-latency Graph Neural Network (GNN) inference for individual requests has become a critical challenge.Real-time GNN inference services operate in an inductive setup, which can handle newly added, previously unseen nodes and their edges.In this setup, the system must prepare the computational graph and input node features for target nodes, followed by GNN inference using the given input data.However, the workflow of a GNN serving system presents two key challenges that hinder low-latency inference.The first challenge arises from the extensive preparation step, which involves heavy memory access in host memory and significant data I/O to devices, constituting the largest portion of end-to-end inference latency.The second challenge is the well-known neighborhood explosion problem in GNN research.As the receptive field for target nodes increases exponentially with the number of layers, this issue exacerbates overall latency.To address these challenges, we propose a Near-Memory Processing (NMP) based low-latency GNN inference serving system named EOD.To ensure low-latency real-time GNN service, we co-design the algorithm and hardware to tackle the aforementioned issues.First, to mitigate the neighborhood explosion problem, we propose a precomputation method for the training node set, reducing memory access and computational complexity from exponential to linear growth.Additionally, we introduce a concatenated ZVC compression method to minimize the overhead of storing precomputed hidden features.Finally, to alleviate heavy host-side memory access and data I/O, we design an NMP architecture that enables efficient aggregation on concatenated ZVC-compressed data.As a result, EOD achieves a geometric mean of 981.1× and 912.0× aggregation speedup over the baseline and the existing architecture for GNN aggregation.Additionally, EOD achieves a geometric mean of 17.9×, and up to 74.3× end-to-end latency speedup over the GPU baseline.