As large language models (LLMs) are gaining increasing popularity across a wide range of web applications, it is of great importance to optimize service-level objectives (SLOs) for LLM inference services to enhance user satisfaction and improve the competitiveness of cloud vendors. In this paper, we observe that adjusting the parameters of LLM inference engines can improve service performance, and the optimal parameter configurations of different services are different. Therefore, we propose SCOOT, an automatic performance tuning system to optimize SLOs for each LLM inference service by tuning the parameters of the inference engine. SCOOT jointly exploits single-objective and multiple-objective Bayesian optimization (BO) techniques to handle various optimization objectives via exploration and exploitation. Moreover, SCOOT prunes the search space with known constraints and adopts a random forest to learn hidden constraints during the tuning process to mitigate invalid exploration. To improve the tuning efficiency, SCOOT utilizes the parallel suggestion to accelerate the tuning process. Extensive experiments demonstrate that SCOOT considerably outperforms existing tuning techniques in SLO optimization while greatly improving the tuning efficiency. Moreover, SCOOT is universally applicable to various LLM inference engines including vLLM and TensorRT-LLM. Currently, SCOOT has already been implemented in the production environment at Ant Group.

Nowadays, large language models (LLMs) are published as a service and can be accessed by various applications via APIs, also known as language-model-as-a-service (LMaaS). Without knowing the generation length of requests, existing serving systems serve requests in a first-come, first-served (FCFS) manner with a fixed batch size, which leads to two problems that affect batch serving efficiency. First, the generation lengths of requests in a batch vary, and requests with short generation lengths must wait for requests with long generation lengths to finish during the batch serving procedure. Second, requests with longer generation lengths consume more memory during serving. Without knowing the generation lengths of batched requests, the batch size is always set small to avoid the out-of-memory (OOM) error, thus preventing the GPU from being fully utilized. In this paper, we find that a significant number of popular applications in the LMaaS scenario have a positive correlation between the generation length and the length of raw user input. Based on this observation, we propose Magnus, which can accurately predict the request generation length with the user input length, application-level, and user-level semantic features. Accordingly, Magnus can achieve high request throughput by batching requests of similar generation lengths together with adaptive batch sizes. Besides, Magnus can also schedule batches with the highest response ratio next (HRRN) policy to reduce request response time. Experiments conducted on our testbed show that Magnus improves request throughput by up to 234% and reduces response time by up to 89.7% compared to baselines.

The mobility load balancing (MLB) in self-organizing networks (SONs) is designed to automatically resolve the mismatch between network resource distribution and network traffic demand. In this paper, we propose an off-policy deep reinforcement learning (DRL) based MLB framework to balance the load distribution among all the cells. Our main contribution is three-fold. First, we propose to use off-policy RL with multiple behavior policies to autonomously learn the optimal MLB policy without any prior knowledge over the underlying wireless environments. Second, we propose a corresponding DRL-based MLB model by using deep neural networks as the function approximators to improve the generalization ability over complex system states. Third, we propose an asynchronous parallel learning framework for MLB to improve the training efficiency in a collaborative manner. Experimental results show that our proposed DRL-based MLB model can outperform the existing approaches considerably.

In this article, we propose a deep reinforcement learning (DRL)-based mobility load balancing (MLB) algorithm along with a two-layer architecture to solve the large-scale load balancing problem for ultradense networks (UDNs). Our contribution is threefold. First, this article proposes a two-layer architecture to solve the large-scale load balancing problem in a self-organized manner. The proposed architecture can alleviate the global traffic variations by dynamically grouping small cells into self-organized clusters according to their historical loads, and further adapt to local traffic variations through intracluster load balancing afterwards. Second, for the intracluster load balancing, this article proposes an off-policy DRL-based MLB algorithm to autonomously learn the optimal MLB policy under an asynchronous parallel learning framework, without any prior knowledge assumed over the underlying UDN environments. Moreover, the algorithm enables joint exploration with multiple behavior policies, such that the traditional MLB methods can be used to guide the learning process thereby improving the learning efficiency and stability. Third, this article proposes an offline-evaluation-based safeguard mechanism to ensure that the online system can always operate with the optimal and well-trained MLB policy, which not only stabilizes the online performance but also enables the exploration beyond current policies to make full use of machine learning in a safe way. Empirical results verify that the proposed framework outperforms the existing MLB methods in general UDN environments featured with irregular network topologies, coupled interferences, and random user movements, in terms of the load balancing performance.

In this paper, an asynchronous multi-user deep reinforcement learning scheme is developed to control the handover (HO) processes across multiple user equipments (UEs), in the goal of lowering the HO rate while ensuring certain system throughput. In this scheme, we use a deep neural network (DNN) as an HO controller learned by each UE via reinforcement learning in a collaborative fashion. Moreover, we use supervised learning in initializing the DNN controller before the execution of reinforcement learning to exploit what we already know with traditional HO schemes and to mitigate the negative effects of random exploration at the initial stage. Furthermore, we show that the adopted global-parameter-based framework enables us to train faster with more UEs, which could nicely address the scalability issue to support large systems. Finally, simulation results demonstrate that the proposed framework can achieve better performance than the state-of-art on-line schemes, in terms of HO rates.

This paper investigates beamforming design of joint transceiver in full-duplex wireless powered communication networks for throughput maximization, where a multi-antenna hybrid access point (H-AP) broadcasts wireless energy to two distributed dual- antenna users and the users cooperatively transmit information signals via their individually harvested energy. Furthermore, the users can recycle self- energy from their own transmitted signals. H-AP acts as a transceiver to transmit energy and receive information simultaneously and amplified-and-forward relaying channel is applied from users to the H-AP. In this system, we aim to maximize the throughput by jointly optimizing the energy and information beamforming vector of H-AP. As the problem is non- convex, we first apply zero forcing method to cancel the self-interference at H-AP and then employ a series of efficient variable substitutions and a low-complexity algorithm to obtain the global optimum. Simulation results demonstrate that the proposed scheme can effectively improve the achievable throughput in the WPCN.

In this paper, we propose a two-layer framework to learn the optimal handover (HO) controllers in possibly large-scale wireless systems supporting mobile Internet-of-Things users or traditional cellular users, where the user mobility patterns could be heterogeneous. In particular, our proposed framework first partitions the user equipments (UEs) with different mobility patterns into clusters, where the mobility patterns are similar in the same cluster. Then, within each cluster, an asynchronous multiuser deep reinforcement learning (RL) scheme is developed to control the HO processes across the UEs in each cluster, in the goal of lowering the HO rate while ensuring certain system throughput. In this scheme, we use a deep neural network (DNN) as an HO controller learned by each UE via RL in a collaborative fashion. Moreover, we use supervised learning in initializing the DNN controller before the execution of RL to exploit what we already know with traditional HO schemes and to mitigate the negative effects of random exploration at the initial stage. Furthermore, we show that the adopted global-parameter-based asynchronous framework enables us to train faster with more UEs, which could nicely address the scalability issue to support large systems. Finally, simulation results demonstrate that the proposed framework can achieve better performance than the state-of-art online schemes, in terms of HO rates.

Green telecommunication is becoming an important part in designing the wireless communication systems. Small Cell Network (SCN) is considered as one of the most promising techniques in the future cellular networks. In order to reduce the high power consumption caused by dense deployment of small base stations (BSs), we first propose a new decentralized precoding algorithm which aims to maximize the energy efficiency of the SCN and the suboptimal solution can be obtained by two times of iterations. For further consideration, when the quantity of UEs is low in the network, it is a waste of energy to open a BS to serve only a few UEs or provide a low data rate to the UEs. So we introduce a BS closing strategy, which takes advantage of the property of SCN that different BSs cover more overlapping areas. Simulation results show that the proposed decentralized precoding algorithm outperforms the traditional precoding algorithms such as BD precoding, and the sleep strategy does not only take linear time complexity but also improve the energy efficiency of the SCN obviously.

In this paper, we investigate the energy efficient optimization for non-regenerative multiple-input- multiple-output (MIMO) Two-Way Relay (TWR) network. Contrary to existing energy efficiency (EE) optimal precoding designs, we maximize the weighted sum per- source energy efficiencies (WSEE) subject to the maximum transmit power constraints of sources and relay, such that different EE requirements from the sources can be investigated. The resulting optimization problem is a non-convex sum of ratios problem. To address this issue, we first reformulate the objective as a more tractable parametric subtraction form by introducing some auxiliary variables. Then we develop an efficient block coordinate decent method to solve the equivalent problem with respect to the individual optimization variable. It is shown that the proposed iterative algorithm is guaranteed to converge.

Three-dimensional multiple-input multiple-output (3D MIMO) is a promising technology for the future wireless communication systems, since antenna tilt angle can reduce the intercell interference. In this paper, we consider 3D MIMO downlink analysis for a multicell multiuser scenario. Zero-forcing (ZF) precoding technique is applied at BS with perfect channel state information of all users in its cell. We derive the exact analytical expressions for downlink sum rate and outage probability for the multicell multiuser 3D MIMO system. Furthermore, we study the asymptotic system performance in the high signal-to-noise ratio (SNR) and large number of antennas regimes. The relationship between tilt angle and sum rate is given. By analyzing the effects of the tilt angle and distance between users and BS on outage probability, the optimal tilt angle and distance are determined to guarantee the outage probability performance.

In this paper, we provide a Gamma distribution to approximate composite Gamma-Gamma (Generalized-K or KG) distribution by using the moment matching method. Based on the approximate distribution, we propose a novel closed-form approximate upper bound of D-MIMO system. The proposed upper bound, which enable us to go further manipulations, only involves some simple functions. Starting from the approximate upper bound, we perform a high-SNR analysis and investigate the asymptotic behavior of the capacity in the following two cases: i) the number of receive antennas grows into infinity for the fixed average and total transmit power, and ii) the number of antennas at both ends grow large at a fixed ratio. It is demonstrated that the proposed approximate and the asymptotic upper bounds match accurately with the exact analytical expression.