The past few years have witnessed a remarkable leap forward in the tactical position of UAV swarm in aerial electronic warfare. Among them, energy-aware packet routing is one of the fundamental problems for cooperation between multiple UAVs to complete combat missions in complex battlefield environments. Recently, deep reinforcement learning (DRL) technique provides a new opportunity to networks related applications, including routing, resource allocation and network access. However, most existing DRL-based routing protocols are difficult to adapt to the changes of network scale, which have weak generalization capabilities and rely on centralized trainers. Thus, these protocols cannot be directly applied to the aerial electronic warfare. In this paper, we propose an adaptive and scalable routing protocol for UAV swarm electronic warfare with graph attention based fully distributed multi-agent reinforcement learning. Besides, a reward function design method based on fuzzy logic is proposed to reduce the probability of abnormal behaviors performed by agents. The simulation results show that our protocol can make effective routing decisions in dynamic wireless multihop networks and enhance the system performances in terms of packet delivery ratio, end-to-end delay and throughput.

Due to its strong flexibility, easy deployment, high maneuverability and extensive connectivity, unmanned aerial vehicle (UAV) swarm has been widely used in the construction of emergency communication network in recent years. Among them, packet routing in a resilient and adaptive manner is one of the fundamental problems for cooperation between multiple UAVs to complete search and rescue tasks. Recently, reinforcement learning (RL) technique has provided a new opportunity for network-related applications, including routing. However, most existing RL-based routing protocols suffer from issues such as local optimum, blind exploration and slow convergence speed. Additionally, the routing protocols based on deep reinforcement learning (DRL) has high computational complexity, making them unsuitable for energy-limited emergency relief scenarios. In this paper, we proposed a Q-learning aided resilient routing protocol with hindsight pre-calculation (QR2HPC) in UAV swarm for the construction of the emergency networks. Firstly, a dynamic exploration and exploitation coefficient is proposed based on the number and speed of neighbors. Secondly, a warm-start mechanism is proposed in the exploration phase that modifies the traditional random next hop selection to a routing approach guided by various indicators. Finally, we introduce a hindsight pre-calculation (HPC) mechanism to improve the robustness of Q-table to traffic flow changes. The experimental results manifest that our protocols can make effective routing decisions in dynamic wireless multi-hop networks, thereby enhancing the system performances in terms of packet delivery ratio, end-to-end delay, throughput and network lifetime.

The Industrial Internet of Things (IIoT) has been viewed as a typical application for the fifth generation (5G) mobile networks. This article investigates the energy efficiency (EE) optimization problem for the Device-to-Device (D2D) communications underlaying unmanned aerial vehicles (UAVs)-assisted IIoT networks with simultaneous wireless information and power transfer (SWIPT). We aim to maximize the EE of the system while satisfying the constraints of transmission rate and transmission power budget. However, the designed EE optimization problem is nonconvex involving joint optimization of the UAV’s location, beam pattern, power control, and time scheduling, which is difficult to tackle directly. To solve this problem, we present a joint UAV location and resource allocation algorithm to decouple the original problem into several subproblems and solve them sequentially. Specifically, we first apply the Dinkelbach method to transform the fraction problem to a subtractive-form one and propose a mulitiobjective evolutionary algorithm based on decomposition (MOEA/D)-based algorithm to optimize the beam pattern. We then optimize UAV’s location and power control using the successive convex optimization techniques. Finally, after solving the above variables, the original problem can be transformed into a single-variable problem with respect to the charging time, which is linear and can be tackled directly. Numerical results verify that significant EE gain can be obtained by our proposed algorithm as compared to the benchmark schemes.

Reconfigurable intelligent surface (RIS) is a promising technique that smartly reshapes wireless propagation environment in the future wireless networks. In this paper, we apply RIS to an unmanned aerial vehicle (UAV)-assisted non-orthogonal multiple access (NOMA) network, in which the transmit signals from multiple UAVs to ground users are strengthened through RIS. Our objective is to minimize the power consumption of the system while meeting the constraints of minimum data rate for users and minimum inter-UAV distance. The formulated optimization problem is non-convex by jointly optimizing the position of UAVs, RIS reflection coefficients, transmit power, active beamforming vectors and decoding order, and thus is quite hard to solve optimally. To tackle this problem, we divide the resultant optimization problem into four independent subproblems, and solve them in an iterative manner. In particular, we first consider the sub-solution of UAVs placement which can be obtained via the successive convex approximation (SCA) and maximum ratio transmission (MRT). By applying the Gaussian randomization procedure, we yield the closed-form expression for the RIS reflection coefficients. Subsequently, the transmit power is optimized using standard convex optimization methods. Finally, a dynamic-order decoding scheme is presented for optimizing the NOMA decoding order in order to guarantee fairness among users. Simulation results verify that our designed joint UAV deployment and resource allocation scheme can effectively reduce the total power consumption compared to the benchmark methods, thus verifying the advantages of combining RIS into the multi-UAV assisted NOMA networks.

This paper investigates a reconfigurable intelligent surface (RIS)-aided unmanned aerial vehicles (UAVs) system with non-orthogonal-multiple access (NOMA), where the transmit signals from multiple UAVs to ground users are strengthened through a RIS. An innovative framework is designed to minimize the total power consumption of the system, by jointly optimizing the position of UAVs, RIS reflection coefficients, active beamforming vectors and decoding order. To solve this problem, we first consider the sub-solution of the UAV’s location which can be achieved via the successive convex approximation (SCA) and maximum ratio transmission (MRT). By applying the Gaussian randomization procedure, we then yield the closed-form solution for RIS phase coefficients. Subsequently, the transmit power is obtained by the standard convex optimization methods. Finally, a dynamic-order decoding scheme is proposed to optimize the decoding order. Simulation results show that the resource allocation scheme can obviously reduce the total power consumption compared to the benchmark schemes.

This article investigates nonorthogonal multiple access (NOMA)-based cloud radio access networks (C-RANs), where edge caching is adopted to cut down the crowdedness of the fronthaul links. We aim to maximize the energy efficiency (EE) by jointly optimizing the power allocation, analog, and digital precoding, which turns out to be an intractable nonconvex optimization problem. To tackle this problem, we first select cluster heads using the selecting cluster-head (SCH) algorithm, where the analog precoding matrix can be resolved by means of maximizing the array gains. Then, the device grouping algorithm is proposed to group devices according to the equivalent channel correlations, and thus, the NOMA devices in the same beam are capable of sharing the same digital precoding vector. Finally, the joint digital precoding design and power allocation algorithm is proposed to decompose the resultant optimization problem into two subproblems and solve them iteratively by applying the Taylor expansion operation and the minimum mean square error (MMSE) detection. Simulation results validate that the proposed NOMA-based C-RANs with a hybrid precoding (HP) scheme can achieve higher spectral efficiency and EE than the traditional orthogonal multiple access (OMA)-based approach and two-stage HP scheme.

Plane spiral orbital angular momentum (PSOAM) mode-groups (MGs) and multiple-input multiple-output non-orthogonal multiple access (MIMO-NOMA) serve as two emerging techniques for achieving high spectral efficiency (SE) in the next-generation networks. In this paper, a PSOAM MGs based multi-user MIMO-NOMA system is studied, where the base station transmits data to users by utilizing the generated PSOAM beams. For such scenario, the interference between users in different PSOAM mode groups can be avoided, which leads to a significant performance enhancement. We aim to maximize the energy efficiency (EE) of the system subject to the constraints of the total transmission power and the minimum data rate. This designed optimization problem is non-convex owing to the interference among users, and hence is quite difficult to tackle directly. To solve this issue, we develop a dual layer resource allocation algorithm where the bisection method is exploited in the outer layer to obtain the optimal EE and a resource distributed iterative algorithm is exploited in the inner layer to optimize the transmit power. Besides, an alternative resource allocation algorithm with Deep Belief Networks (DBN) is proposed to cope with the requirement for low computational complexity. Simulation results verify the theoretical findings and demonstrate the proposed algorithms on the PSOAM MGs based MIMO-NOMA system can obtain a better performance comparing to the conventional MIMO-NOMA system in terms of EE.

This paper investigates the energy efficiency (EE) optimization problem for device-to-device (D2D) communications underlaying non-orthogonal multiple access (NOMA) unmanned aerial vehicles (UAVs)-assisted networks with simultaneous wireless information and power transfer (SWIPT). Our aim is to maximize the energy efficiency of the system while satisfying the constraints of transmission rate and transmission power budget. However, the considered EE optimization problem is non-convex involving joint optimization of the UAV's location, beam pattern, power control and time scheduling, which is difficult to solve directly. To tackle this problem, we develop an efficient resource allocation algorithm to decompose the original problem into several sub-problems and solve them sequentially. Specifically, we first apply the Dinkelbach method to transform the fraction problem to a subtractive-form one, and propose a mulitiobjective evolutionary algorithm based on decomposition (MOEA/D) based algorithm to optimize the beam pattern. We then optimize UAV's location and power control by applying the successive convex optimization techniques. Finally, after solving the above variables, the original problem is transformed into a single-variable problem with respect to the charging time, which is a linear problem and can be solved directly. Numerical results verify that the significant EE gain can be obtained by our proposed method as compared to the benchmark schemes.

Beamforming and non-orthogonal multiple access (NOMA) are two key techniques for achieving spectral efficient communication in the fifth generation and beyond wireless networks. In this paper, we jointly apply a hybrid beamforming and NOMA techniques to an unmanned aerial vehicle (UAV)-carried wireless-powered mobile edge computing (MEC) system, within which the UAV is mounted with a wireless power charger and the MEC platform delivers energy and computing services to Internet of Things (IoT) devices. We aim to maximize the sum computation rate at all IoT devices whilst satisfying the constraint of energy harvesting and coverage. The considered optimization problem is non-convex involving joint optimization of the UAV’s 3D placement and hybrid beamforming matrices as well as computation resource allocation in partial offloading pattern, and thus is quite difficult to tackle directly. By applying the polyhedral annexation method and the deep deterministic policy gradient (DDPG) algorithm, we propose an effective algorithm to derive the closed-form solution for the optimal 3D deployment of the UAV, and find the solution for the hybrid beamformer. A resource allocation algorithm for partial offloading pattern is thereby proposed. Simulation results demonstrate that our designed algorithm yields a significant computation performance enhancement as compared to the benchmark schemes.

Beamforming and non-orthogonal multiple access (NOMA) serve as two potential solutions for achieving spectral efficient communication in the fifth generation and beyond wireless networks. In this paper, we jointly apply a hybrid beamforming and NOMA techniques to an unmanned aerial vehicle (UAV)-carried wireless-powered mobile edge computing (MEC) system, within which the UAV is equipped with a wireless power charger and the MEC platform delivers energy and computing services to Internet of Things (IoT) devices. Our aim is to maximize the sum computation rate at all IoT devices whilst satisfying the constraint of energy harvesting and coverage. The resultant optimization problem is non-convex involving joint optimization of the UAV’s 3D placement and hybrid beamforming matrices as well as computation resource allocation in both partial and binary offloading patterns, and thus is quite difficult to tackle directly. By applying the polyhedral annexation method and the deep deterministic policy gradient (DDPG) algorithm, we develop an effective algorithm to derive the closed-form solution for the optimal 3D deployment of the UAV, and find the solution for the hybrid beamformer. Two resource allocation algorithms for partial and binary offloading patterns are thereby proposed. Simulation results verify that our designed algorithms achieve a significant computation performance enhancement as compared to the benchmark schemes.

This paper considers an unmanned aerial vehicle (UAV)-aided millimeter Wave (mmWave) multiple-input-multiple-output (MIMO) non-orthogonal multiple access (NOMA) system, where a UAV serves as a flying base station (BS) to provide wireless access services to a set of Internet of Things (IoT) devices in different clusters. We aim to maximize the downlink sum rate by jointly optimizing the three-dimensional (3D) placement of the UAV, beam pattern and transmit power. To address this problem, we first transform the non-convex problem into a total path loss minimization problem, and hence the optimal 3D placement of the UAV can be achieved via standard convex optimization techniques. Then, the multiobjective evolutionary algorithm based on decomposition (MOEA/D) based algorithm is presented for the shaped-beam pattern synthesis of an antenna array. Finally, by transforming the original problem into an optimal power allocation problem under the fixed 3D placement of the UAV and beam pattern, we derive the closed-form expression of transmit power based on Karush-Kuhn-Tucker (KKT) conditions. In addition, inspired by fraction programming (FP), we propose a FP-based suboptimal algorithm to achieve a near-optimal performance. Numerical results demonstrate that the proposed algorithm achieves a significant performance gain in terms of sum rate for all IoT devices, as compared with orthogonal frequency division multiple access (OFDMA) scheme.