In post-disaster scenarios, the rapid deployment of adequate communication infrastructure is essential to support disaster search, rescue, and recovery operations. To achieve this, uncrewed aerial vehicle (UAV) has emerged as a promising solution for emergency communication due to its low cost and deployment flexibility. However, conventional untethered UAV (U-UAV) is constrained by size, weight, and power (SWaP) limitations, making it incapable of maintaining the operation of a macro base station. To address this limitation, we propose a heterogeneous UAV-based framework that integrates tethered UAV (T-UAV) and U-UAVs, where U-UAVs are utilized to enhance the throughput of cell-edge ground user equipments (G-UEs) and guarantee seamless connectivity during G-UEs’ mobility to safe zones. It is noted that the integrated access and backhaul (IAB) technique is adopted to support the wireless backhaul of U-UAVs. Accordingly, we formulate a two-timescale joint user scheduling and trajectory control optimization problem, aiming to maximize the downlink throughput under asymmetric traffic demands and G-UEs’ mobility. To solve the formulated problem, we proposed a two-timescale multi-agent deep deterministic policy gradient (TTS-MADDPG) algorithm based on the centralized training and distributed execution paradigm. Numerical results show that the proposed algorithm outperforms other benchmarks, including the two-timescale multi-agent proximal policy optimization (TTS-MAPPO) algorithm and MADDPG scheduling method, with robust and higher throughput. Specifically, the proposed algorithm obtains up to 12.2% average throughput gain compared to the MADDPG scheduling method.

To provide Uplink Centric Broadband Communication (UCBC), New Radio Unlicensed (NR-U) network has been standardized to exploit the unlicensed spectrum using Listen Before Talk (LBT) scheme to fairly coexist with the incumbent Wireless Fidelity (WiFi) network. Existing access schemes over unlicensed spectrum are required to perform Clear Channel Assessment (CCA) before transmissions, where fixed Energy Detection (ED) thresholds are adopted to identify the channel as idle or busy. However, fixed ED thresholds setting prevents devices from accessing the channel effectively and efficiently, which leads to the hidden node (HN) and exposed node (EN) problems. In this paper, we first develop a centralized double Deep Q-Network (DDQN) algorithm to optimize the uplink system throughput, where the agent is deployed at the central server to dynamically adjust the ED thresholds for NR-U and WiFi networks. Considering that heterogeneous NR-U and WiFi networks, in practice, may not be able to share the raw data with the central server directly due to data privacy, we then develop a vertical federated DDQN algorithm, where two agents are deployed in the NR-U and WiFi networks, respectively. Our results have shown that the uplink system throughput increases by over 100%, where cell throughput of NR-U network rises by 150%, and cell throughput of WiFi network decreases by 30%. To guarantee the cell throughput of WiFi network, we redesign the reward function to punish the agent when the cell throughput of WiFi network is below the threshold, and our revised design can still provide 70% uplink system throughput gain, where cell throughput of NR-U network rises by 100%, and cell throughput of WiFi network rises by 35%.

Cellular-connected unmanned aerial vehicle (UAV) swarm is a promising solution for diverse applications, including cargo delivery and traffic control. However, it is still challenging to communicate with and control the UAV swarm with high reliability, low latency, and high energy efficiency. In this paper, we propose a two-phase command and control (C&C) transmission scheme in a cellular-connected UAV swarm network, where the ground base station (GBS) broadcasts the common C&C message in phase I. In phase II, the UAVs that have successfully decoded the C&C message will relay the message to the rest of UAVs via device-to-device (D2D) communications in either broadcast or unicast mode, under latency and energy constraints. To maximize the number of UAVs that receive the message successfully within the latency and energy constraints, we formulate the problem as a Constrained Markov Decision Process to find the optimal policy. To address this problem, we propose a decentralized constrained graph attention multi-agent Deep-Q-network (DCGA-MADQN) algorithm based on Lagrangian primal-dual policy optimization, where a PID-controller algorithm is utilized to update the Lagrange Multiplier. Simulation results show that our algorithm could maximize the number of UAVs that successfully receive the common C&C under energy constraints.

To cater for the data-hungry Internet of Things (IoT) applications, uplink centric broadband communication (UCBC) has been identified as a new service class in the vision of 5.5G, where the unlicensed spectrum has been regarded as a promising solution to boost the uplink capacity. The new radio unlicensed (NR-U) network adopts category-4 (Cat4) listen before talk (LBT) access scheme to exploit the unlicensed spectrum and fairly coexist with the incumbent wireless fidelity (WiFi) network. However, the existing Cat4 LBT access scheme adopts single fixed energy detection (ED) threshold and backoff speed, which cannot adapt to the sophisticated interference and achieve the expected uplink system throughput. To tackle this issue, in this article, we develop a novel Cat4 LBT access scheme with adaptive backoff procedure for UCBC, which includes instantaneous interference level quantification, instantaneous interference level sharing, and backoff speed determination. The results have shown that our proposed adaptive Cat4 LBT scheme achieves over 70% uplink system throughput performance gain where cell throughput of NR-U network rises by over 100%, and cell throughput of WiFi network increases by 25%.

The sixth-generation (6G) communication research is currently in the early stage, where ultra-reliable low-latency communication (URLLC) is still an important service as in the fifth-generation (5G). Since 6G networks are expected to provide even higher levels of massive connectivity, high spectrum efficiency, high reliability, and low latency than 5G communication, it would confront much more severe spectrum scarcity problems, which make the new radio in unlicensed spectrum (NR-U) technology attractive. However, how to achieve URLLC requirements in NR-U networks is extremely challenging due to interference and collisions among multiple radio access technologies (e.g., WiFi). Therefore, it is urgent to design efficient spectrum-sharing algorithms to support URLLC in emerging 6G networks. In this paper, we develop novel centralized deep reinforcement learning (CDRL) and federated DRL (FDRL) frameworks, respectively, to optimize the downlink URLLC transmission in NR-U and WiFi coexistence systems through dynamically adjusting energy detection (ED) thresholds. Our results show that both CDRL and FDRL approaches have improved the reliability of the NR-U system significantly, but the CDRL framework has sacrificed the reliability of the WiFi system. To guarantee the reliability of the WiFi system while improving the NR-U system, we take fairness into account by redesigning the reward of CDRL.

Grant-free non-orthogonal multiple access (GF-NOMA) is a potential technique to support massive Ultra-Reliable and Low-Latency Communication (mURLLC) service. However, the dynamic resource configuration in GF-NOMA systems is challenging due to random traffics and collisions, that are unknown at the base station (BS). Meanwhile, joint consideration of the latency and reliability requirements makes the resource configuration of GF-NOMA for mURLLC more complex. To address this problem, we develop a novel learning framework for signature-based GF-NOMA in mURLLC service taking into account the multiple access signature collision, the UE detection, as well as the data decoding procedures for the K-repetition GF and the Proactive GF schemes. The goal of our learning framework is to maximize the long-term average number of successfully served users (UEs) under the latency constraint. We first perform a real-time repetition value configuration based on a double deep Q-Network (DDQN) and then propose a Cooperative Multi-Agent learning technique based DQN (CMA-DQN) to optimize the configuration of both the repetition values and the contention-transmission unit (CTU) numbers. Our results show the superior performance of CMA-DQN over the conventional load estimation-based uplink resource configuration approach (LE-URC) in heavy traffic and demonstrate its capability in dynamically configuring in long term for mURLLC service. In addition, with our learning optimization, the Proactive scheme always outperforms the K-repetition scheme in terms of the number of successfully served UEs, especially under the high backlog traffic scenario.

To improve the capacity of cellular systems without additional expenses on licensed frequency bands, the 3rd Gen-eration Partnership Project (3GPP) has proposed New Radio Unlicensed (NR-U). It should be noted that each node in NR-U has to perform the Listen-Before- Talk (LBT) operation before transmission to avoid collisions by other unlicensed radio access technologies (e.g., WiFi). Thus, packets transmissions are prone to delay due to the LBT channel access mechanism. How to achieve Ultra-Reliable and Low-Latency Communications (URLLC) requirements in NR-U networks under the coexistence with WiFi networks is of importance and extremely challenging. In this paper, we develop a novel deep reinforcement learning (DRL) framework to optimize the downlink URLLC trans-mission in the NR-U and WiFi coexistence system through dynamically adjusting the energy detection (ED) thresholds. Our results have shown that the NR-U system reliability has been improved significantly via the DRL compared to that without learning approaches, but with the sacrifice of WiFi system reliability. To address this, we redesigned the reward to take fairness into account, which guarantees the WiFi system reliability while improvina the NR- U system reliability.

Cellular-connected unmanned aerial vehicle (UAV) swarm is a promising solution for diverse applications, including cargo delivery and traffic control. However, it is still challenging to communicate with and control the UAV swarm with high reliability and low latency. In this paper, we propose a two-phase command and control (C&C) transmission scheme in cellular-connected UAV swarm network, where the ground base station (GBS) broadcasts the common C&C message in Phase I, the UAVs that have successfully decoded the C&C message will then relay the message to the rest of UAVs via device-to-device (D2D) communications under individual latency constraint. To maximize the number of UAVs that receive the message successfully within the latency constraint, we formulate the problem as a decentralized and partially observable Markov process for finding the optimal policies. To address this problem, we further develop a fully decentralized graph attention network (GAT)-based reinforcement learning algorithm to optimize the D2D pair selection, where the GAT is utilized to exploit the dynamic topology information of cellular-connected UAV swarm network. Simulation results show that our algorithm outperforms the other two baselines and could achieve a cooperative target under a mobile UAV swarm scenario.

Fifth Generation (5G) New Radio (NR) does not support data transmission during random access (RA) procedures, which results in unnecessary control signalling overhead, especially for small data transmission (SDT). Motivated by this, 3GPP has proposed 4/2-step SDT RA schemes based on the existing grant-based (4-step) and grant-free (2-step) RA schemes, with the aim to enable data transmission during RA procedures in Radio Resource Control (RRC) Inactive state. To compare the 4/2-step SDT RA schemes with the benchmark 4/2-step RA schemes, we provide a spatio-temporal analytical framework to evaluate the RA schemes, which jointly models the preamble detection, Physical Uplink Shared Channel (PUSCH) decoding, and data transmission procedures. Based on this analytical model, we derive the analytical expressions for the overall packet transmission success probability in each RACH attempt. Our results show that 2-step SDT RA scheme provides the highest overall packet transmission success probability, but performance gain decreases with the increase of device intensity.

Fifth Generation (5G) New Radio (NR) does not support data transmission during random access (RA) procedures, which results in unnecessary control signalling overhead and power consumption, especially for small data transmission (SDT). Motivated by this, 3GPP has proposed 4/2-step SDT RA schemes based on the existing grant-based (4-step) and grant-free (2-step) RA schemes, with the aim to enable data transmission during RA procedures in Radio Resource Control (RRC) Inactive state. To compare the 4/2-step SDT RA schemes with the benchmark 4/2-step RA schemes, we provide a spatio-temporal analytical framework to evaluate the RA schemes, which jointly models the preamble detection, Physical Uplink Shared Channel (PUSCH) decoding, and data transmission procedures. Based on this analytical model, we derive the analytical expressions for the overall packet transmission success probability and average throughput in each RACH attempt. We also derive the average energy consumption in each RACH attempt. Our results show that 2-step SDT RA scheme provides the highest overall packet transmission success probability, and the lowest average energy consumption, but the performance gain decreases with the increase of device intensity.