This paper addresses the challenge of achieving high average User-Perceived Throughput (UPT) in multi-user downlink transmissions, which is essential for immersive experience in emerging 6G applications like extended reality and digital twins. Achieving high UPT involves effectively managing burst data traffic over fluctuating wireless channels, a topic not extensively explored in current literature. We approach this by modeling the system as a Markov Decision Process (MDP), focusing on maximizing average UPT by minimizing the average time users have non-empty queues while maintaining system stability. We introduce a multi-user scheduling scheme for a system with a single Resource Block (RB). Specifically, the scheduling metric is a fluid approximation of the minimum average time with non-empty queues, which is obtained by analytically solving a convex optimization problem. We next extend our scheduling scheme to handle multiple spectrum RBs through successive allocation of available RBs, achieving polynomial complexity. Using a quadratic Lyapunov drift method, we ensure scheduler stability. The scheduler is also modified to operate online, adapting to time-varying channel conditions. Simulation results demonstrate that our scheduler not only achieves near-optimal performance but also significantly outperforms existing benchmarks.

Extended reality (XR) is an emerging 6G application with unique traffic characteristics and requirements, calling for innovative Ultra-Reliable and Low-Latency Communication (URLLC) technologies. This paper investigates multi-user scheduling to meet XR services’ hard-latency constraints. Specifically, we focus on a periodical traffic model, where the latency constraint for transmitting each XR frame is less than the inter-arrival time. We describe the system as a periodic Markov Decision Process (MDP) with the performance metric being the probability of successful transmission within the latency constraint. We then obtain the maximum success probability and the optimal scheduling based on the optimal value function. In the case of homogeneous arrivals, we construct a lower bound of the optimal value function. Based on this, we propose an online multi-user scheduling policy that determines scheduling decisions by solving a series of nonlinear Knapsack Problems (KPs) in polynomial time. Our analysis demonstrates that the scheduling scheme is asymptotically optimal with increasing users. Furthermore, we extend the online scheduling scheme to heterogeneous arrivals and present extensions for practical scenarios with multiple resource blocks, quasi-periodical arrivals, random frame sizes, and time-correlated channel fading. Finally, simulation results show that the proposed scheduler achieves near-optimal performance and outperforms other benchmark schedulers.

In the forthcoming 6G era, Extended reality (XR) is an emerging application with unique traffic characteristics requirements, calling for innovative Ultra-Reliable and Low-Latency Communication (URLLC) technologies. In this paper, we investigate multi-user scheduling to meet hard-latency constraints for XR services. Specifically, we focus on a periodical XR traffic model, where the latency constraint for transmitting each XR frame is less than the inter-arrival time. To find an optimal multi-user scheduling scheme, we first describe the system as a periodic Markov Decision Process (MDP), where the scheduling performance is expressed as the probability of successful transmission within the latency constraint. Then, we obtain the maximum success probability and the optimal scheduling based on the optimal value function. Inspired by the properties of the optimal value function, we construct a lower bound of it and propose an online multi-user scheduling scheme. In particular, scheduling decisions under the proposed scheme are determined by solving a series of nonlinear Knapsack Problem (KP) in polynomial time. Finally, simulation results show that the proposed scheduler achieves nearly optimal performance and outperforms other benchmark schedulers.

Maintaining the freshness of information in the Internet of Things (IoT) is a critical yet challenging problem. In this paper, we study cooperative data collection using multiple Unmanned Aerial Vehicles (UAVs) with the objective of minimizing the total average Age of Information (AoI). We consider various constraints of the UAVs, including kinematic, energy, trajectory, and collision avoidance, in order to optimize the data collection process. Specifically, each UAV, which has limited on-board energy, takes off from its initial location and flies over sensor nodes to collect update packets in cooperation with the other UAVs. The UAVs must land at their final destinations with non-negative residual energy after the specified time duration to ensure they have enough energy to complete their missions. It is crucial to design the trajectories of the UAVs and the transmission scheduling of the sensor nodes to enhance information freshness. We model the multi-UAV data collection problem as a Decentralized Partially Observable Markov Decision Process (Dec-POMDP), as each UAV is unaware of the dynamics of the environment and can only observe a part of the sensors. To address the challenges of this problem, we propose a multi-agent Deep Reinforcement Learning (DRL)-based algorithm with centralized learning and decentralized execution. In addition to the reward shaping, we use action masks to filter out invalid actions and ensure that the constraints are met. Simulation results demonstrate that the proposed algorithms can significantly reduce the total average AoI compared to the baseline algorithms, and the use of the action mask method can improve the convergence speed of the proposed algorithm.

For the emerging Internet of Things (IoT), one of the most critical problems is the real-time reconstruction of signals from a set of aged measurements. During the reconstruction, distortion occurs between the observed signal and the reconstructed signal due to sampling and queuing delay. We focus on minimizing the average distortion defined as the 1-norm of the difference of the two signals under the scenario that a Poisson counting process is reconstructed in real-time on a remote monitor. We consider the reconstruction under three special sampling policies. For each of the policy, we derive the closed-form expression of the average distortion by dividing the overall distortion area into polygons and analyzing their structures. It turns out that the polygons are built up by sub-polygons that account for distortions caused by sampling and queuing delay. The closed-form expressions of the average distortion help us find the optimal sampling parameters that achieve the minimum distortion. In addition, we propose an interpolation algorithm to further decrease the average distortion and give its lower-bound on distortion for one of the three sampling policies. Simulation results are provided to validate our conclusion.

For the emerging Internet of Things (IoT), one of the most important basic problems is how to reconstruct signals in real-time from a set of under-sampled and delayed samples. The sampling omits the details of the signals of interest and the delayed samples against the requirement of real-time. As a result, distortion occurs between the interested signal and the reconstructed signal. In this paper, we focus on minimizing the average distortion defined as the 1-norm of the difference of the two signals under the scenario that a Poisson counting process is reconstructed in real-time on a remote monitor. We derive the average distortion-sampling rate function, with which the optimal sampling rate can be obtained as well as the minimum average distortion. To further decrease the average distortion, an algorithm is proposed by sacrificing the real-time requirement in a small degree.

This paper is motivated by the observation that the average queueing delay can be decreased by sacrificing power efficiency in wireless communications. In this sense, we naturally wonder what the minimum queueing delay is when the available power is limited and how to achieve the minimum queueing delay. To answer these two questions in the scenario where randomly arriving packets are transmitted over multi-state wireless fading channel, a probabilistic cross-layer scheduling policy is proposed in this paper, and characterized by a constrained Markov decision process. Using the steady-state probability of the underlying Markov chain, we are able to derive the mathematical expressions of the concerned metrics, namely, the average queueing delay and the average power consumption. To describe the delay-power tradeoff, we formulate a non-linear programming problem, which, however, is very challenging to solve. By analyzing its structure, this optimization problem can be converted into an equivalent linear programming problem via variable substitution, which allows us to derive the optimal delay-power tradeoff as well as the optimal scheduling policy. The optimal scheduling policy turns out to be dual-threshold-based, which means transmission decisions should be made based on the optimal thresholds imposed on the queue length and the channel state.

In this paper, we study delay optimal scheduling of bursty data traffics over multi-state time-varying wireless channels, where bursty packet arrival in the network layer, queueing behavior in the data link layer, and rate adaptive transmission with flexible modulation in the physical layer are jointly considered from a cross-layer perspective. To achieve a minimum queueing delay under a power constraint, a probabilistic queue-aware and channel- aware cross-layer scheduling policy is proposed, and characterized by a Markov chain model, where the transmission rate, i.e., the number of packets delivered in each slot, is selected with probabilities based on the buffer and channel states in this slot. To reveal the optimal delay-power tradeoff, we formulate a non-linear optimization problem, which, however, is very challenging to solve. To make it tractable, we convert the optimization problem equivalently into a Linear Programming (LP) problem, which helps us achieve the optimal three-dimensional threshold-based scheduling policy analytically. It is found that the source should select one transmission rate jointly based on the channel state and the backlog in the queue.

Communication by wireless portable devices usually has a low energy efficiency and suffers from channel fading. Because of these limitations, assuring Quality of Service(QoS) such as low average transmission delay and packet loss rate under a given energy constraint is becoming an important problem. One efficient solution is considered as cross-layer scheduling which aims to improve the overall performance by combining various layers. With the awareness of random packet arrival in the network layer and channel state in the physic layer, a probabilistic scheduling strategy is proposed in this paper. More specifically, we focus on the arbitrarily random packet arrival distribution when building the system model. Based on the Markov chain model, a linear programming (LP) problem is formulated to minimize the average transmission delay under a given power constraint. Based on the solution of the LP problem, a threshold-based optimal schedule policy can be derived.