This paper addresses network slicing in a large-scale Multi-Access Edge Computing (MEC)-enabled Radio Access Network (RAN) comprising heterogeneous edge nodes with varying computing and storage resource capacities. These resources are dynamically allocated to slice requests and released when the service of a slice request is completed. Our objective is to optimize the resource allocation for each admitted arriving slice request, considering its demands for computing and storage resources, to maximize the long-run average Earning Before Interest and Taxes (EBIT) of the MEC slicing system. We formulate the optimization problem as a Restless Multi-Armed Bandit (RMAB)-based resource allocation problem with a nonlinear cost rate function. To solve this, we introduce a new policy called Prioritizing-the-Future-Approximated earning per request (PFA) where for each admitted slice request, we always prioritize the allocation of the resource combination that gives the highest achievable earning, considering the future effects of this allocation. PFA is designed to be scalable and applicable to large-scale networks. We numerically demonstrate the superior performance of PFA in maximizing long-run average EBIT through simulations, comparing it with two baseline policies, at various cases of parameter values. Moreover, our findings offer insights for network operators in resource allocation policy selection.

The Internet of Robotic Things (IoRT) merges the capabilities of robotics with the connectivity and computing power of Internet of Things (IoT) technologies, enabling seamless data collection, processing, and exchange. This integration enhances robotic systems with greater intelligence, mobility, and autonomy, unlocking significant potential across various applications, including sustainable agriculture. However, deploying IoRT systems in unpredictable environments poses challenges, such as network instability and hardware failures, which have not been thoroughly explored in the literature. To address these issues, this article introduces Mobility-as-a-Resilience Service (MaaRS), a model that leverages the mobility of active uncrewed aerial vehicles (UAVs), strategically relocating them to critical points of interest in response to potential data collection failures, optimizing resource allocation and enhancing system resilience, particularly in smart farm scenarios. Additionally, a robust multiagent deep deterministic policy gradient (RMADDPG) method is devised to enable efficient task allocation and system recovery in the presence of model uncertainty, observation noise, and reward uncertainty. Extensive simulations demonstrate that the proposed method achieved a significant boost in performance, efficiency, and stability over the state-of-the-art.

The paper addresses the challenge of scheduling multiple output priority queues of a switch in a real-world setting characterized by bursty traffic and diverse traffic priorities. Existing queue scheduling methods primarily employ two types of strategies: priority-based scheduling and weight-based scheduling. However, there is a lack of scheduling methods that can simultaneously handle bursty traffic in a timely manner and maintain priority-based scheduling. This paper addresses this issue by formulating the problem as a restless multi-armed bandit problem, and a queue scheduling method is proposed to balance priority service provisioning and bursty traffic processing. The proposed queue scheduling method operates efficiently in a timely manner based on instant queue length and utilizes the Whittle index method to achieve an asymptotically optimal solution. This design facilitates packet forwarding by considering the instant states of queues, offering improvements over existing methods. Experimental results demonstrate that the proposed method achieves a more efficient balance between priority service provisioning and bursty traffic processing compared to other state-of-the-art methods. Additionally, the proposed method results in better balanced network performance metrics, such as queue length, packet delay, and packet loss, thus efficiently supporting various applications that generate bursty traffic randomly.

In this paper, a non-myopic beam scheduling policy is proposed for multi-target tracking (MTT) in a phased-array radar network, seeking to minimize the discounted sum of tracking error of targets and improve the long-term tracking performance. The Whittle index policy based on the restless multiarmed bandit (RMAB) model can decompose the state space of the underlying optimization problem into independent spaces with reduced sizes. We consider the tracking error covariance (TEC) matrix as the state of each target (arm), which evolves based on the Kalman filter. However, for a real-world MTT, the exact calculation of the Whittle index in multiple dimensions is challenging. The neural network is established to achieve the feature extraction of TEC states and learn the corresponding Whittle index. The deep reinforcement learning (DRL) method is exploited to train the neural network by leveraging the threshold property of the Whittle index policy and engaging in interactions with a single target tracking environment. We propose the DRL-based Whittle index policy, namely DRLWI, aiming to solve the beam allocation problem for MTT with multi-dimensional TEC states. This approach effectively mitigates the exponential computational complexity of classical dynamic programming approaches and the low convergence rate caused by large joint state and action spaces in the simple application of DRL algorithms. Numerical results demonstrate the performance of the proposed DRLWI policy surpasses that of DRL algorithms and myopic policies.

By melding the capabilities of robotics with the agility of edge computing, Robotic Edge System (RES) exemplifies the next generation of Internet Intelligent Service Systems, delivering incredible efficiency and adaptability across diverse real world applications. Inevitably, RES is susceptible to mechanical disruptions of robots, particularly when some tasks are assigned to faulty ones, leading to uncertain failures and performance degradation. Due to communication and latency constraints, it is not always feasible to rely on edge/cloud computing infrastructure for system recovery. To address these issues, a UAV-as-a-Service (UAVaaS) approach is proposed that leverages the mobility of UAVs to enhance system resilience. Specifically, a Markov Decision Process (MDP) is utilized to assign tasks dynamically among active UAVs to achieve system recovery in a livestock monitoring scenario. Additionally, a Dual Noise Deep Deterministic Policy Gradient (DNDDPG)-based mechanism is proposed to minimize system recovery time and energy consumption. The proposed DNDDPG enhances exploration and decision-making during training by integrating parameter noise and behavioral noise into the classic Deep Deterministic Policy Gradient (DDPG) algorithm. The simulation results indicate that the proposed mechanism can achieve convergence within 100 episodes, thereby effectively minimizing the time and energy required for system recovery.

We focus on energy-efficient offloading strategies in a slicing-enabled large-scale edge network, or an "edge slicing" system, with different computing/storage components, on which the service capacities are dynamically released and reused by the incoming user requests. The offloading problem is challenged by its large problem size and the heterogeneity of the service components and user requests, leading to the high-dimensional state space of the underlying stochastic process. We formulate the problem in the manner of the restless-bandit-based (RB-based) resource allocation problem and generalize unrealistic previously made assumptions on specific forms of the power functions, such as linearity, convexity, and taking only binary power states. We adapt the RB-based resource allocation technique to the offloading problem. We quantify actions of selecting certain service components to serve requests through marginal rewards, which take consideration of both the history and future effects of the corresponding action. The marginal rewards exist in closed forms when assuming linear power functions, but it remains an open question in the non-linear case. We approximate the marginal rewards by introducing state-dependent coefficients that compensate for the undesirable effects of non-linearity. We propose a scheduling policy that always prioritizes the service components with the highest marginal rewards, which is simple and applicable in the large-scale case. In the special case with linear power functions, the policy becomes asymptotically optimal - it approaches optimality when the number of components tends to infinity. We numerically demonstrate the effectiveness and robustness of the proposed policy in practical situations with respect to energy efficiency.

We study a job-assignment problem in a large-scale server farm system with geographically deployed servers as abstracted computer components (e.g., storage, network links, and processors) that are potentially diverse. We aim to maximize the energy efficiency of the entire system by effectively controlling carried load on networked servers. A scalable, near-optimal job-assignment policy is proposed. The optimality is gauged as, roughly speaking, energy cost per job. Our key result is an upper bound on the deviation between the proposed policy and the asymptotically optimal energy efficiency, when job sizes are exponentially distributed and blocking probabilities are positive. Relying on Whittle relaxation and the asymptotic optimality theorem of Weber and Weiss, this bound is shown to decrease exponentially as the number of servers and the arrival rates of jobs increase arbitrarily and in proportion. In consequence, the proposed policy is asymptotically optimal and, more importantly, approaches asymptotic optimality quickly (exponentially). This suggests that the proposed policy is close to optimal even for relatively small systems (and indeed any larger systems), and this is consistent with the results of our simulations. Simulations indicate that the policy is effective, and robust to variations in job-size distributions.

Wireless network slicing is a promising technology for next-generation networks to provide tailored on-demand services to mobile users. We consider a scheduling policy for wireless network slicing with the aim to maximize the energy efficiency of the network defined as the ratio of long-run average throughput of user requests to the long- run average power consumption. This gives rise to a problem of extremely high computational complexity which prevents direct application of conventional optimization techniques. We propose a scalable priority-based policy, referred to as the Most Energy-Efficient Resource First (MEERF). MEERF is proved to be asymptotically optimal in the special case appropriate for a local wireless environment with highly dense user population and exponentially distributed service time requirement. The robustness of MEERF to different service time distributions is demonstrated by extensive simulations. We present numerically the effectiveness of MEERF %balancing the QoS and relevant power consumption by comparing it with benchmark policies in a more general network with potentially geographically distributed users and infrastructures. The results show that MEERF outperforms the benchmark policies in most of our experiments and achieves up to 52% improvement in terms of energy efficiency.

We study the problem of job assignment in a large-scale realistically dimensioned server farm comprising multiple processor-sharing servers with different service rates, energy consumption rates, and buffer sizes. Our aim is to optimize the energy efficiency of such a server farm by effectively controlling carried load on networked servers. To this end, we propose a job assignment policy, called Most energy-efficient available server first Accounting for Idle Power (MAIP), which is both scalable and near optimal. MAIP focuses on reducing the productive power used to support the processing service rate. Using the framework of semi-Markov decision process, we show that, with exponentially distributed job sizes, MAIP is equivalent to the well-known Whittle's index policy. This equivalence and the methodology of Weber and Weiss enable us to prove that, in server farms where a loss of jobs happens if and only if all buffers are full, MAIP is asymptotically optimal, as the number of servers tends to infinity under certain conditions associated with the large number of servers, as we have in a real server farm. Through extensive numerical simulations, we demonstrate the effectiveness of MAIP and its robustness to different job-size distributions, and observe that significant improvement in energy efficiency can be achieved by utilizing the knowledge of energy consumption rate of idle servers.

Energy efficiency of server farms is an important design consideration of data centers. One effective approach is to optimize energy consumption by controlling carried load on the networked servers. In this paper, we propose a robust heuristic policy for job assignment in a server farm, aiming to improve the energy efficiency by maximizing the ratio of the long-run average throughput to the expected energy consumption. Our model of the server farm considers parallel processor-sharing queues with finite buffer sizes, heterogeneous server speeds, and an arbitrary energy consumption function. We devise the new energy-efficient (EE) policy in a way that the state distribution of the system depends on the service requirement distribution only through the mean. We show that the state-of-the-art slowest server first (SSF) policy can be obtained as a special case of EE and both policies have the same computational complexity. We provide a rigorous analysis of EE and derive conditions under which EE is guaranteed to outperform SSF in terms of energy efficiency. Extensive numerical results are presented and demonstrate that, in comparison with SSF, EE yields a consistently better system throughput and yet improves the energy efficiency by up to 70%.

Energy efficiency of server farms is an important design consideration of the green datacenter initiative. One effective approach is to optimize power consumption of server farms by controlling the carried load on the networked servers. In this paper, we propose a robust heuristic policy called E* for stochastic job assignment in a server farm, aiming to improve the energy efficiency by maximizing the ratio of job throughput to power consumption. Our model of the server farm considers a parallel system of finite-buffer processor-sharing queues with heterogeneous server speeds and energy consumption rates. We devise E* as an insensitive policy so that the stationary distribution of the number of jobs in the system depends on the job size distribution only through its mean. We provide a rigorous analysis of E* and compare it with a baseline approach, known as most energy-efficient server first (MEESF), that greedily chooses the most energy-efficient servers for job assignment. We show that E* has always a higher job throughput than that of MEESF, and derive realistic conditions under which E* is guaranteed to outperform MEESF in energy efficiency. Extensive numerical results are presented and demonstrate that E* can improve the energy efficiency by up to 100%.

We study the problem of stochastic job assignment in a server farm comprising multiple processor-sharing servers with various speeds and finite buffer sizes. We consider two types of assignment policies: without jockeying, where an arriving job is assigned only once to an available server, and with jockeying, where a job may be reassigned at any time. We also require that the underlying Markov process under each policy is insensitive. Namely, the stationary distribution of the number of jobs in the system is independent of the job size distribution except for its mean. For the case without jockeying, we derive two insensitive heuristic policies: One aims at maximizing job throughput, and the other trades off job throughput for energy efficiency. For the case with jockeying, we formulate the optimal assignment problem as a semi-Markov decision process and derive optimal policies with respect to various optimization criteria. We further derive two simple insensitive heuristic policies with jockeying: One maximizes job throughput, and the other aims at maximizing energy efficiency. Numerical examples demonstrate that, under a wide range of system parameters, the latter policy performs very close to the optimal policy. Numerical examples also demonstrate energy/throughput tradeoffs for the various policies and, in the case with jockeying, they show a potential of substantial energy savings relative to a policy that optimizes throughput.