Reconfigurable intelligent surfaces (RISs) have gained considerable interest because of their inherent passive and energy-efficient design. Integrating unmanned aerial vehicles (UAVs) with reconfigurable intelligent surfaces (RIS), known as UAV-RIS, can significantly improve network performance and serve as a crucial enabler for advancements in 6G mobile networks. However, ensuring security in UAV-RIS systems poses notable challenges, particularly in the presence of imperfect channel state information (CSI) and beamforming complexities. In this paper, we identify the critical security requirements for UAV-RIS beamforming in practical scenarios. To address these challenges, we introduce a novel deep deterministic policy gradient with a distributional critic (DDPG-DC)-based beamforming approach aimed at securing UAV-RIS systems while improving the overall secrecy rate. Our proposed secure beamforming solution achieves up to a 48% performance improvement compared to existing state-of-the-art algorithms.

Unmanned aerial vehicles (UAVs) in integration with reconfigurable intelligent surfaces (RIS) play a crucial role in improving wireless communication coverage and enhancing overall performance. However, optimizing the beamforming for the RIS and base station (BS) is critical in improving coverage and ensuring connectivity in densely populated areas. The primary challenge in this process arises from the diverse Quality of Service (QoS) requirements set by user equipment (UEs). To address this complexity, machine learning algorithms are employed to predict the optimal beamforming configurations for both the BS and RIS. However, the traditional supervised learning methods are becoming less effective due to the ever-changing demands of UEs, as these methods rely on fixed data patterns that struggle to adapt to the fluctuating QoS requirements of UEs. Thus, in this paper, we propose an unsupervised learning-based deep learning (DL) approach to jointly predict the optimal beamforming matrix for RIS and BS, enhancing communication coverage and maximizing QoS satisfaction of UEs. The proposed DL-based beamforming adaptively predicts the beamforming matrix, facilitates efficient data exploration during the initial learning phase, and seamlessly scales as the process advances, thereby enhancing overall performance. Numerical results demonstrate that the proposed DL-based RIS and BS beamforming outperforms by up to 89%, compared to the state-of-the-art methods.

Seamless connectivity and efficient mobility management in mobile networks can be facilitated via estimation of the speed and mobility state of user equipment (UE). In this paper, we propose a deep reinforcement learning (DRL) framework for prediction of the UE speed based on the number of performed handovers and density of all base stations (BSs), including both macro base stations (MBSs) and small-cell base stations (SBSs). We leverage the capability of DRL to learn and adapt to complex mobility patterns, enhancing the accuracy of the speed prediction in highly dynamic environments. To demonstrate benefits of the DRL-based UE speed prediction, we further exploit the predicted speed to determine the UEs’ mobility states. The mobility state serves as an input for handover optimization via fuzzy logic. By incorporating the speed prediction, the system proactively anticipates changes in the UEs’ mobility and enables more precise and timely optimization of handover parameters. The simulation with realistic UE mobility traces shows that the proposed DRL improves the accuracy of the speed prediction by up to 35% and reduces the prediction root mean square error by up to 59% compared to the state-of-the-art algorithms. Moreover, using the proposed DRL-based speed prediction for fuzzy logic-based handover optimization improves the UEs’ sum capacity by up to 43% and reduces the number of handovers by up to 39% compared to state-of-the-art works.

In this paper, we introduce a novel framework jointly managing handovers of user equipments (UEs) and Unmanned Aerial Vehicles (UAVs) serving the UEs. The goal is to maximize the sum capacity of the UEs while considering a cost related to the handovers. To this end, we introduce a novel approach based on deep deterministic policy gradient (DDPG) adjusting the Cell Individual Offset (CIO) for handovers of the UEs among the UAVs and ground base stations (GBSs) as well as handovers of the UAVs among the GBSs. The UAVs playing the role of relays often face challenges related to the implementation cost and energy limitations. To address these challenges, the UAVs should operate in a transparent relaying mode. In such mode, unfortunately, the channels between the UEs and the UAVs are unknown as the transparent relays lack any communication control-related functionalities. Therefore, we adopt a deep neural network (DNN) to predict the channel qualities among the UEs and the UAVs for the handover purposes. We demonstrate that the proposal significantly increases the sum capacity of the UEs by dozens of percent and even reduces the number of handovers compared to state-of-the-art works. At the same time, the proposed DDPG-based CIO setting reduces a gap in the sum capacity between the predicted and the optimal (but practically not feasible) case with perfectly known channels among UEs and UAVs. Hence, the proposal is suitable for practical scenarios with not perfectly accurate channel quality information.

We focus on handover management in networks integrating traditional terrestrial ground base stations (GBSs) and non-terrestrial unmanned aerial vehicles (UAVs) serving users. In such scenario, we propose a joint management of handover of users equipment (UEs) between UAVs and GBSs as well as handover of UAVs between GBSs. Our goal is to maximize the sum capacity for UEs while avoiding redundant handovers. As an impact of handover on the future network performance is not explicit, we adopt deep deterministic policy gradient (DDPG). Furthermore, since the UAVs are usually energy constrained, we consider an energy efficient transparent relaying mode for the UAVs. However, in the transparent relaying mode, the access channel quality between the UAV relay and the UE is unknown even if such information is essential for handover. Thus, we further employ deep neural network (DNN) to predict the access channel quality. An incorporation of DDPG for handover management together with DNN for channel quality prediction can impair the sum capacity of the UEs due to an accumulation of inherent small prediction errors of DNN and DDPG. Hence, we also introduce a coordination between DNN and DDPG to suppress the accumulation of the prediction errors. Simulations demonstrate that the proposed handover management and the coordination of DDPG and DNN increase the sum capacity by up to 63% while notably reducing the number of handovers and handover failure ratio compared to the state-of-the-art works.

The evolution of Artificial Intelligence (AI) integrated with the Sixth-generation ($\mathbf{6 G}$) framework poses significant challenges to low-latency applications. Recently, semantic communication has emerged as a promising technique for future intelligent applications. However, the resource management problem combined with semantics is not fully explored. In this paper, we present a deep reinforcement learning-based twin-delayed deep deterministic policy gradient (TD3) for task-centered semantic communication. The proposed TD3 algorithm optimizes bandwidth, and semantic information and prioritizes data with maximum signal-to-noise ratio (SNR) for the efficient transmission of useful information. Simulation results demonstrate the effectiveness of the proposed TD3 scheme compared to state-of-the-art work in terms of transmission efficiency by up to $36 \%$ for varying users and up to $33 \%$ for varying SNR.

Unmanned aerial vehicles (UAVs) have been adopted in various applications, including agriculture, public safety, surveillance, and crucial military missions. However, alongside their advantageous nature, UAVs have also been employed for malicious activities, leading to an increased requirement for timely detection and identification. Despite significant progress in UAV detection, challenges persist, particularly concerning various types of UAVs, the payload carried by UAVs, and the traits of their flight. Employing single machine learning for detection and identification has limitations due to the inability to handle diverse datasets and acquire complex relationships. Therefore, in this paper, we introduce a novel integration of the Visual Geometry Group-based convolutional neural network (VGG-CNN) framework employed for detection with the Mask Region-based convolutional neural network (MR-CNN) for identification of UAVs (jointly termed MR-DCNN). For efficient deployment of MR-DCNN, we add diversity to the dataset by performing data augmentation of new images in the training dataset for the detection of various types of UAVs, payload categories, and flight characteristics. The performance evaluation of the MR-DCNN approach was conducted via simulations, revealing superior detection capabilities for malicious UAVs compared to existing methods.

Multihop communication has gained prominence within the realm of the Internet of Underwater Things (IoUT) owing to its exceptional reliability amidst the challenges posed by the underwater acoustic environment. Despite this, the persistence of limitations caused by propagation delay, high collision rate, and limited energy in underwater communication remains, representing the most formidable hurdles in ensuring the successful transmission of data gathered by sensor nodes. To address these challenges, we employ a machine learning (ML)-based optimal cooperating node selection for each hop, considering the Shortest propagation delay, minimal residual Energy, and a low Collision rate (referred to as SEC). For this purpose, we initially assemble the sensor nodes to create a list of cooperative nodes, considering the aspect of SEC. Then, using an assembled list of cooperating sensor nodes, we employ ML-based algorithms, such as reinforcement learning (RL-SEC), deep Q-networks (DQN-SEC), and deep deterministic policy gradient (DDPG-SEC), to predict the optimal cooperating node for each hop. The simulation results of the DDPG-SEC demonstrate a significant improvement of approximately 56% when compared with RL-SEC, DQN-SEC, and other state-of-the-art techniques.

User applications based on the deep neural networks (DNNs), such as object or anomaly detection, image recognition, or language processing, running on computation- and energy-constrained user equipment (UE) can be partially or fully processed in the edge computing servers to reduce a processing time and save an energy in the UE. To further reduce the processing time and the UE’s energy consumption, DNN with multiple exit points can be incorporated. In this article, we address the problem of the decision on whether the computation should be offloaded from the UE to the edge computing server or processed locally by the UE and we solve this problem jointly and “on-the-fly” together with DNN exit selection. Since the formulated problem is very complex, we exploit the deep deterministic policy gradient for the exit selection and the offloading decisions (labeled DDPG-EOD) for the DNN-based applications. To this end, we first convert the problem into the Markov decision process, and then, we employ an end-to-end learning via DDPG with the actor-critic architecture. Second, we use a knowledge distillation-based technique to efficiently select the DNN’s exit to minimize the delay and energy consumption. Simulation results show that the proposal is highly scalable, converges very quickly, and surpasses the best performing state-of-the-art approach by up to 120% and 100% in terms of the overall DNN processing delay and the energy consumption, respectively.