Large Language Models (LLMs) have demonstrated remarkable capabilities, leading to a significant increase in user demand for LLM services. However, cloud-based LLM services often suffer from high latency, unstable responsiveness, and privacy concerns. Therefore, multiple LLMs are usually deployed at the network edge to boost real-time responsiveness and protect data privacy, particularly for many emerging smart mobile and IoT applications. Given the varying response quality and latency of LLM services, a critical issue is how to route user requests from mobile and IoT devices to an appropriate LLM service (i.e., edge LLM expert) to ensure acceptable quality-of-service (QoS). Existing routing algorithms fail to simultaneously address the heterogeneity of LLM services, the interference among requests, and the dynamic workloads necessary for maintaining long-term stable QoS. To meet these challenges, in this paper we propose a novel deep reinforcement learning (DRL)-based QoS-aware LLM routing framework for sustained high-quality LLM services. Due to the dynamic nature of the global state, we propose a dynamic state abstraction technique to compactly represent global state features with a heterogeneous graph attention network (HAN). Additionally, we introduce an action impact estimator and a tailored reward function to guide the DRL agent in maximizing QoS and preventing latency violations. Extensive experiments on both Poisson and real-world workloads demonstrate that our proposed algorithm significantly improves average QoS and computing resource efficiency compared to existing baselines.

Mobile traffic prediction plays a crucial role in enabling efficient network management and service provisioning. Traditional prediction approaches treat different mobile application services (such as Uber, Facebook, Twitter, etc) as isolated entities, neglecting potential correlation among them. Moreover, such isolated prediction methods necessitate the uploading of historical traffic data from all regions to forecast city-wide traffic, resulting in consuming substantial bandwidth resources and risking prediction failure in the event of data loss in specific regions. To address these challenges, we propose a novel Cross-service Attention-based Spatial-Temporal Graph Convolutional Network (CsASTGCN) for precise and communication-efficient multi-service mobile traffic prediction. Our methodology allows each mobile service to transmit the traffic data of only a fraction of regions for city-wide traffic prediction of all mobile services, which reduces the resource consumption caused by data transmission. Specifically, the sparse traffic data are initially transmitted to the cloud server and the masked graph autoencoder is utilized to roughly reconstruct the traffic volume for regions with missing data. Subsequently, a cross-service attention-based predictor is designed to calculate the data correlation among different mobile services within the same region. Considering the constantly emerging mobile services, we incorporate a novel model-based adaptive transfer learning scheme to extract valuable knowledge from the existing models and expedite the training of a new model for a new service without training from scratch, thereby enhancing the scalability of our framework. Extensive experiments conducted on a large-scale real-world mobile traffic dataset demonstrate that our model greatly outperforms the existing schemes, enhancing both the communication-efficiency and robustness of large-scale multi-service traffic prediction.

Federated Learning (FL) requires frequent exchange of model parameters, which leads to long communication delay, especially when the network environments of clients vary greatly. Moreover, the parameter server needs to wait for the slowest client (i.e., straggler, which may have the largest model size, lowest computing capability or worst network condition) to upload parameters, which may significantly degrade the communication efficiency. Commonly-used client selection methods such as partial client selection would lead to the waste of computing resources and weaken the generalization of the global model. To tackle this problem, along a different line, in this paper, we advocate the approach of model parameter dropout instead of client selection, and accordingly propose a novel framework of Federated learning scheme with Differential parameter Dropout (FedDD). FedDD consists of two key modules: dropout rate allocation and uploaded parameter selection, which will optimize the model parameter uploading ratios tailored to different clients' heterogeneous conditions and also select the proper set of important model parameters for uploading subject to clients' dropout rate constraints. Specifically, the dropout rate allocation is formulated as a convex optimization problem, taking system heterogeneity, data heterogeneity, and model heterogeneity among clients into consideration. The uploaded parameter selection strategy prioritizes on eliciting important parameters for uploading to speedup convergence. Furthermore, we theoretically analyze the convergence of the proposed FedDD scheme. Extensive performance evaluations demonstrate that the proposed FedDD scheme can achieve outstanding performances in both communication efficiency and model convergence, and also possesses a strong generalization capability to data of rare classes.