Online learning is a powerful technique that allows models to adjust to concept drift in dynamically changing graphs. This approach is crucial for large mobility-based companies like Grab, where batch-learning methods fail to keep up with the large amount of training data. Our work focuses on scaling graph neural network mixture of expert (MoE) models for real-time traffic speed prediction on road networks, while meeting high accuracy and low latency requirements. Conventional spatio-temporal and incremental MoE frameworks struggle with poor inference accuracy and linear time complexity when scaling experts, for the latter, leading to prohibitively high latency in model updates. To address this issue, we introduce the Indexed Router, a novel method that categorizes experts into a structured hierarchy called the indexed tree. This approach reduces the time to scale and search N number of experts from O(N) to O(log N), making it ideal for online learning under tight service level agreements. Our experiments show that these time savings do not compromise inference accuracy, and our Indexed Router outperforms state-of-the-art spatio-temporal and incremental MoE models in terms of traffic speed prediction accuracy on real-life GPS traces from Grab's database and publicly available records. In summary, the Indexed Router enables MoE models to scale across large numbers of experts with low latency, while accurately identifying the relevant experts for inference.

We present Deep network Dynamic Graph Partitioning (DDGP), a novel algorithm for optimizing the division of large graphs for mixture of expert graph neural networks. Our work is motivated from the observation that real world graphs suffer from spatial concept drift, which is detrimental to neural network training. We answer the question of how we can divide a graph, with vertices in each subgraph sharing a similar distribution, so that an expert network trained over each subgraph may yield the best learning outcome. DDGP is a two pronged algorithm that consists of cluster merging, followed by cluster boundary refinement. We used the training performance of each expert model as feedback to iteratively refine partition boundaries among subgraphs. These partitions are distinct for each model and graph network. We provide theoretical proof of convergence for DDGP boundary refinement as a guarantee for model training stability. Finally, we demonstrate experimentally that DDGP outperforms state-of-the-art graph partitioning algorithms for a regression task on multiple large real world graphs, with GraphSage and Graph Attention as our expert models.

The use of deep learning models for forecasting the resource consumption patterns of SQL queries have recently been a popular area of study. While these models have demonstrated promising accuracy, training them over large scale industry workloads are expensive. Space inefficiencies of encoding techniques over large numbers of queries and excessive padding used to enforce shape consistency across diverse query plans implies 1) longer model training time and 2) the need for expensive, scaled up infrastructure to support batched training. In turn, we developed Prestroid, a tree convolution based data science pipeline that accurately predicts resource consumption patterns of query traces, but at a much lower cost. We evaluated our pipeline over 19K Presto OLAP queries, on a data lake of more than 20PB of data from Grab. Experimental results imply that our pipeline outperforms benchmarks on predictive accuracy, contributing to more precise resource prediction for large-scale workloads, yet also reduces per-batch memory footprint by 13.5x and per-epoch training time by 3.45x. We demonstrate direct cost savings of up to 13.2x for large batched model training over Microsoft Azure VMs.

Interaction-based systems have been widely used in many enterprises like Grab to enable quick and easy analysis of large-scale spatial data. Unlike traditional instruction-based query processing systems, modern interaction-based systems allow users to issue complex queries through simple interactions with a Graphical User Interface (GUI). While such systems have significantly transformed the process of spatial query processing, they still rely on a process-after-query approach for executing the queries. Even though the user is continuously interacting with the GUI, the actual processing is only initiated after the user completes their interactions, thus wasting the opportunities to reduce the response time of query processing.