In 5G/6G wireless networks, enhanced mobile broadband (eMBB) services provide gigabit-per-second high data rates, while ultra-reliable and low-latency communication (URLLC) services offer extremely low latency and high reliability. Simultaneously supporting these two types of services in the same wireless environment presents a significant challenge due to their fundamentally different service requirements. In this paper, we formulate the bandwidth and power resource allocation problem of multiplexing eMBB and URLLC services in a wireless non-orthogonal multiple access (NOMA) network as a nonlinear programming problem. The objective is to maximize the average supply-demand ratio (SDR) of eMBB users while minimizing the SDR variance among eMBB users, subject to the reliability constraint of URLLC users. We propose a deep reinforcement learning (DRL)-based NOMA resource allocation scheme (DRL-NRAS), which employs two DRL agents at the base station to jointly optimize bandwidth and power allocation for eMBB services, as well as the puncturing and superposition ratios for URLLC transmissions. The proposed DRL-NRAS fully utilizes a DRL technique, twin delayed deep deterministic policy gradient, to effectively handle the time-varying data rate requirements of eMBB users and the stochastic request arrivals of URLLC users. Simulation results demonstrate that the proposed DRL-NRAS reduces the SDR variance by 78.4%, 87.2%, and 72.7% compared to the CVX, B-greedy, and random schemes, respectively, while achieving a 0% delay violation rate under the considered simulation settings. These results indicate that the proposed DRL-NRAS can effectively improve the fairness among eMBB users while ensuring the reliability of URLLC services.

IN the above article [1], in Table II (Continued.), the text is written as “ $R_{m,n}^{e,pun}\left ({ i,j }\right)$ and $R_{m,n}^{e,sup}\left ({ i,j }\right)$ , respectively, in the definition of $R_{m,n}^{e,aff} ({i,j})$ in Table II (Continued.)”.

The immersive experiences offered by the metaverse have been gradually integrated into people’s lives. However, managing the bandwidth and power resources of the base station (BS) in wireless metaverse networks to meet the high data rate requirements of metaverse users has become an important issue. In this paper, we formulate the non-orthogonal multiple access (NOMA) bandwidth and power resources allocation problem in a wireless metaverse network as a nonlinear programming problem, aiming to maximize the average supply-demand ratio (SDR) of NOMA groups while minimizing the SDR variance of NOMA groups and the average delay difference among high-interaction metaverse users. To this end, we propose a deep reinforcement learning-based resource allocation mechanism (DRL-RAM), which utilizes the resource manager within the BS to monitor the bandwidth and power resources usage and intelligently allocate these resources in each time slot. The proposed DRL-RAM fully utilizes a DRL technique, twin delayed deep deterministic policy gradient, to efficiently handle the dynamic nature of time-varying data rate requirements of metaverse users thereby achieving long-term network performance optimization. Simulation results show that the proposed DRL-RAM achieves competitive average SDR performance while significantly reducing SDR variance of NOMA groups and delay differences among high-interaction metaverse users, compared to the P-Greedy and B-Greedy allocation schemes.

With the rapid growth of emerging delay-sensitive and computation-intensive vehicular applications, the integration of vehicle-to-everything (V2X) networks and multi-access edge computing (MEC) has emerged as a promising paradigm. However, how to make appropriate offloading decisions to reduce the overall processing time has become a critical issue. In this paper, we propose a novel V2X collaborative MEC network (VCMN) architecture that enables inter-roadside unit (RSU) collaboration for distributed task offloading. Based on this architecture, we formulate the task offloading problem as a nonlinear programming optimization problem, aiming to minimize the average total processing time of all offloading tasks while satisfying the processing time constraint of each offloading tasks. We propose an event-driven deep reinforcement learning-based task offloading policy (EDRL-TOP), which utilizes a centralized RSU manager to monitor the load conditions of all RSUs and RSU-to-RSU links, and intelligently makes offloading decisions for each event. The proposed EDRL-TOP fully utilizes a DRL technique, deep deterministic policy gradient (DDPG), to effectively handle the stochastic request arrivals thereby achieving long-term network performance optimization. Simulation results show that the proposed EDRL-TOP can decrease the average total processing time of tasks and achieves a higher task success rate compared to the greedy algorithm.

In this paper, we formulate the interference problem among satellites as a nonlinear programming problem, which aims to maximize the average binary scale satisfaction (BSS) of low-earth-orbit (LEO) satellites while satisfying the data rate requirement of the geostationary orbit (GEO) satellite in a GEO-LEO co-existing satellite networks (GLCSNs) through the minimization of frequency band interference among these satellites. We propose an event-driven deep reinforcement learning based frequency bands allocation mechanism (EDRL-FBAM) that utilizes a frequency bands manager in a GEO satellite to monitor the spectrum usage of all LEO satellites and intelligently allocate the frequency bands for each frequency band request from LEO satellites. The proposed EDRL-FBAM fully utilizes a DRL technique, Proximal Policy Optimization, to deal with stochastic arrivals of frequency band requests in the frequency bands manager so as to achieve long-term optimization of the network performance. The simulation results show that the proposed EDRL-FBAM can significantly improve the average data rate of LEO satellites, the average BSS of LEO satellites, and the average data rate of GEO satellite when compared to greedy and random approaches.

Satellite networks provide higher coverage and provide ubiquitous mobile services. However, how to allocate precious satellite spectrum resources to improve better network performance has become an important issue in satellite networks. In this paper, we study the power allocation problem of the Low Earth Orbit (LEO) satellite to maximize the Supply-Demand Ratio (SDR) of the LEO satellite users while minimize the standard deviation (SD) of LEO satellite users’ SDR. We propose an Event-Driven Deep Reinforcement Learning based Power Allocation Mechanism (EDRL-PAM), which utilizes the LEO satellite’s power manager to intelligently allocate the power request for each cell of the LEO satellite. Furthermore, we propose a NOMA-based power allocation algorithm for allocating power to all LEO satellite users within the cells. The proposed EDRL-PAM fully utilizes a Deep Reinforcement Learning (DRL) technique, Deep Deterministic Policy Gradient (DDPG), to deal with stochastic arrivals of power requests in the power manager to achieve long-term optimization of the network performance. The simulation results show that our proposed EDRL-PAM can significantly improve the average data rate, and achieve the long-term optimization and fairness of network performance for satellite users.

With the increasing demands of global communication, integrating the low-earth-orbit (LEO) satellite networks (LSNs) with the existing terrestrial networks can significantly extend the wireless coverage in 6G networks. Since the deployment cost of LEO satellites is extremely high, an energy-efficient routing scheme should be designed to prolong the lifetime of LEO satellites while satisfying the end-to-end delay constraint of each data flow. Thus, the routing problem significantly affects the performance of an LSN. In this paper, we formulate this problem as a nonlinear programming problem, which aims to minimize the energy consumption of an LSN caused by data transmission among LEO satellites while satisfying the end-to-end delay constraint of each data flow. We propose a centralized LSN (CLSN) architecture that utilizes a routing manager in a medium-earth-orbit (MEO) satellite to monitor the condition of LEO satellites and intelligently decide the routing path for each routing request event from LEO satellites. We further propose an energy-efficient event-driven deep reinforcement learning (DRL), Deep Deterministic Policy Gradient (DDPG), enhanced Dijkstra’s algorithm (EEDRL-Dijkstra) in the routing manager to deal with the stochastic routing request event arrivals so as to achieve long-term optimization of the energy consumption performance of an LSN while satisfying the end-to-end delay constraint of each data flow. The simulation results show that in the proposed CLSN architecture, the proposed EEDRL-Dijkstra significantly improves the energy consumption performance as compared to Dijkstra’s algorithm and random routing. Both the proposed EEDRL-Dijkstra and Dijkstra’s algorithm can satisfy the end-to-end delay constraint of each data flow.