Mobile edge computing (MEC) is a promising technique to improve the computational capacity of smart devices (SDs) in Internet of Things (IoT). However, the performance of MEC is restricted due to its fixed location and limited service scope. Hence, we investigate an unmanned aerial vehicle (UAV)assisted MEC system, where multiple UAVs are dispatched and each UAV can simultaneously provide computing service for multiple SDs. To improve the performance of system, we formulated a UAV-based trajectory control and resource allocation multi-objective optimization problem (TCRAMOP) to simultaneously maximize the offloading number of UAVs and minimize total offloading delay and total energy consumption of UAVs by optimizing the flight paths of UAVs as well as the computing resource allocated to served SDs. Then, consider that the solution of TCRAMOP requires continuous decision-making and the system is dynamic, we propose an enhanced deep reinforcement learning (DRL) algorithm, namely, distributed proximal policy optimization with imitation learning (DPPOIL). This algorithm incorporates the generative adversarial imitation learning technique to improve the policy performance. Simulation results demonstrate the effectiveness of our proposed DPPOIL and prove that the learned strategy of DPPOIL is better compared with other baseline methods.

In this paper, we explore an unmanned aerial vehicle (UAV)-assisted Internet-of-Things (IoT) data collection system, where multiple UAVs are deployed and each UAV can simultaneously collect data from multiple IoT devices. Specifically, we formulate a UAV-enabled data collection multi-objective optimization problem (UDCMOP) to simultaneously maximize the collected data of UAVs and minimize the total energy consumption of UAVs that contains the moving and hovering energy consumption by optimizing the flight trajectories of UAVs. Given the dynamic nature of the system and the need for coordination among multiple UAVs, we propose an enhanced multi-agent deep reinforcement learning (MADRL) algorithm, namely, multi-agent proximal policy optimization with curiositydriven exploration (MAPPOC). This algorithm incorporates a curiosity-driven exploration mechanism to improve exploration capabilities. Simulation results demonstrate the effectiveness of the proposed MAPPOC and prove that the learned strategy of MAPPOC is better compared with other baseline methods.

With the advancement of smart cities, smart manufacturing, and smart transportation, the Internet of Things (IoT) big data platform operating on wireless networks has emerged as a pivotal sector. In such systems, unmanned aerial vehicles (UAVs) play an indispensable support due to their flexibility and adaptability, but the energy sensitivity and limited communication capabilities pose further challenges. In this paper, we study a UAV-assisted secure communication system, where a UAV-enabled virtual antenna array (UVAA) consisting of multiple UAVs communicates with a remote mobile user (MU) by executing collaborative beamforming (CB), and then an eavesdropper exists for eavesdropping the transmission data from UVAA to MU. Then, a UAV-enabled secure communication optimization problem is formulated to maximize the total secrecy rate between the UVAA and the MU by optimizing the roles, locations and excitation current weights of UAVs. Since the considered scenario is dynamic and the UAVs need to cooperate with each other, we propose a hybrid-action multi-agent deep reinforcement learning (MADRL) algorithm (HMAPPO) to efficiently solve the optimization problem. Simulation results verify the effectiveness of the HMAPPO and illustrate that it learns the best strategy compared with other baseline methods.

In this paper, we investigate an unmanned aerial vehicle (UAV)-assistant air-to-ground communication system, where multiple UAVs form a UAV-enabled virtual antenna array (UVAA) to communicate with remote base stations by utilizing collaborative beamforming. To improve the work efficiency of the UVAA, we formulate a UAV-enabled collaborative beamforming multi-objective optimization problem (UCBMOP) to simultaneously maximize the transmission rate of the UVAA and minimize the energy consumption of all UAVs by optimizing the positions and excitation current weights of all UAVs. This problem is challenging because these two optimization objectives conflict with each other, and they are non-concave to the optimization variables. Moreover, the system is dynamic, and the cooperation among UAVs is complex, making traditional methods take much time to compute the optimization solution for a single task. In addition, as the task changes, the previously obtained solution will become obsolete and invalid. To handle these issues, we leverage the multi-agent deep reinforcement learning (MADRL) to address the UCBMOP. Specifically, we use the heterogeneous-agent trust region policy optimization (HATRPO) as the basic framework, and then propose an improved HATRPO algorithm, namely HATRPO-UCB, where three techniques are introduced to enhance the performance. Simulation results demonstrate that the proposed algorithm can learn a better strategy compared with other methods. Moreover, extensive experiments also demonstrate the effectiveness of the proposed techniques.