A novel framework of reconfigurable intelligent surfaces (RISs) enhanced indoor wireless networks is proposed, where an RIS mounted on the robot is invoked to enhance the service quality for mobile users. Meanwhile, non-orthogonal multiple access (NOMA) techniques are adopted to further increase the spectrum efficiency since RISs are capable to provide NOMA with artificially controlled channels, which can be a beneficial condition for NOMA networks. To optimize the sum rate of all users, a federated learning enhanced deep deterministic policy gradient (FL-DDPG) algorithm is proposed to optimize the deployment and phase shifts of the mobile RIS as well as the power allocation policy. Our simulation results indicate that the mobile RIS scheme can provide about three times data rate gain compare to the fixed RIS. Moreover, the NOMA scheme is capable to achieve a significant data rate gain in contrast with the OMA scheme. Finally, the FL-DDPG algorithm has a superior convergence rate and optimization performance than that of the independent training framework.

A novel framework of intelligent reflecting surface (IRS)-aided multiple-input single-output (MISO) non-orthogonal multiple access (NOMA) network is proposed, where a base station (BS) serves multiple clusters with unfixed number of users in each cluster. The goal is to maximize the sum rate of all users by jointly optimizing the passive beamforming vector at the IRS, decoding order, power allocation coefficient vector and number of clusters, subject to the rate requirements of users. In order to tackle the formulated problem, a three-step approach is proposed. More particularly, a long short-term memory (LSTM) based algorithm is first adopted for predicting the mobility of users. Secondly, a K-means based Gaussian mixture model (K-GMM) algorithm is proposed for user clustering. Thirdly, a deep Q-network (DQN) based algorithm is invoked for jointly determining the phase shift matrix and power allocation policy. Simulation results are provided for demonstrating that the proposed algorithm outperforms the benchmarks, while the throughput gain of 35% can be achieved by invoking NOMA technique instead of orthogonal multiple access (OMA).

A novel framework is proposed for cellular offloading with the aid of multiple unmanned aerial vehicles (UAVs), while non-orthogonal multiple access (NOMA) technique is employed at each UAV to further improve the spectrum efficiency of the wireless network. The optimization problem of joint three-dimensional (3D) trajectory design and power allocation is formulated for maximizing the throughput. Since ground mobile users are considered as roaming continuously, the UAVs need to be re-deployed timely based on the movement of users. In an effort to solve this pertinent dynamic problem, a K-means based clustering algorithm is first adopted for periodically partitioning users. Afterward, a mutual deep Q-network (MDQN) algorithm is proposed to jointly determine the optimal 3D trajectory and power allocation of UAVs. In contrast to the conventional deep Q-network (DQN) algorithm, the MDQN algorithm enables the experience of multi-agent to be input into a shared neural network to shorten the training time with the assistance of state abstraction. Numerical results demonstrate that: 1) the proposed MDQN algorithm is capable of converging under minor constraints and has a faster convergence rate than the conventional DQN algorithm in the multi-agent case; 2) The achievable sum rate of the NOMA enhanced UAV network is 23% superior to the case of orthogonal multiple access (OMA); 3) By designing the optimal 3D trajectory of UAVs with the MDON algorithm, the sum rate of the network enjoys 142% and 56% gains than invoking the circular trajectory and the 2D trajectory, respectively.

A novel framework of reconfigurable intelligent surfaces (RISs)-enhanced indoor wireless networks is proposed, where an RIS mounted on the robot is invoked to enable mobility of the RIS and enhance the service quality for mobile users. Meanwhile, non-orthogonal multiple access (NOMA) techniques are adopted to further increase the spectrum efficiency since RISs are capable of providing NOMA with artificial controlled channel conditions, which can be seen as a beneficial operation condition to obtain NOMA gains. To optimize the sum rate of all users, a deep deterministic policy gradient (DDPG) algorithm is invoked to optimize the deployment and phase shifts of the mobile RIS as well as the power allocation policy. In order to improve the efficiency and effectiveness of agent training for the DDPG agents, a federated learning (FL) concept is adopted to enable multiple agents to simultaneously explore similar environments and exchange experiences. We also proved that with the same random exploring policy, the FL armed deep reinforcement learning (DRL) agents can theoretically obtain a reward gain comparing to the independent agents. Our simulation results indicate that the mobile RIS scheme can significantly outperform the fixed RIS paradigm, which provides about three times data rate gain compared to the fixed RIS paradigm. Moreover, the NOMA scheme is capable of achieving a gain of 42% in contrast with the OMA scheme in terms of the sum rate. Finally, the multi-cell simulation proved that the FL enhanced DDPG algorithm has a superior convergence rate and optimization performance than the independent training framework.

A communication enabled indoor intelligent robots (IRs) service framework is proposed, where non-orthogonal multiple access (NOMA) technique is adopted to enable highly reliable communications. In cooperation with the ultramodern indoor channel model recently proposed by the International Telecommunication Union (ITU), the Lego modeling method is proposed, which can deterministically describe the indoor layout and channel state in order to construct the radio map. The investigated radio map is invoked as a virtual environment to train the reinforcement learning agent, which can save training time and hardware costs. Build on the proposed communication model, motions of IRs who need to reach designated mission destinations and their corresponding down-link power allocation policy are jointly optimized to maximize the mission efficiency and communication reliability of IRs. In an effort to solve this optimization problem, a novel reinforcement learning approach named deep transfer deterministic policy gradient (DT-DPG) algorithm is proposed. Our simulation results demonstrate in the following: 1) with the aid of NOMA techniques, the communication reliability of IRs is effectively improved; 2) radio map is qualified to be a virtual training environment, and its statistical channel state information improves training efficiency by about 30%; 3) proposed DT-DPG algorithm is superior to the conventional deep deterministic policy gradient (DDPG) algorithm in terms of optimization performance, training time, and anti-local optimum ability.

A novel reconfigurable intelligent surfaces (RISs)-based transmission framework is proposed for downlink non-orthogonal multiple access (NOMA) networks. We propose a quality-of-service (QoS)-based clustering scheme to improve the resource efficiency and formulate a sum rate maximization problem by jointly optimizing the phase shift of the RIS and the power allocation at the base station (BS). A model-agnostic meta-learning (MAML)-based learning algorithm is proposed to solve the joint optimization problem with a fast convergence rate and low model complexity. Extensive simulation results demonstrate that the proposed QoS-based NOMA network achieves significantly higher transmission throughput compared to the conventional orthogonal multiple access (OMA) network. It can also be observed that substantial throughput gain can be achieved by integrating RISs in NOMA and OMA networks. Moreover, simulation results of the proposed QoS-based clustering method demonstrate observable throughput gain against the conventional channel condition-based schemes.

A novel framework is proposed for integrating reconfigurable intelligent surfaces (RIS) in unmanned aerial vehicle (UAV) enabled wireless networks, where an RIS is deployed for enhancing the service quality of the UAV. Non-orthogonal multiple access (NOMA) technique is invoked to further improve the spectrum efficiency of the network, while mobile users (MUs) are considered as roaming continuously. The energy consumption minimizing problem is formulated by jointly designing the movement of the UAV, phase shifts of the RIS, power allocation policy from the UAV to MUs, as well as determining the dynamic decoding order. A decaying deep Q-network (D-DQN) based algorithm is proposed for tackling this pertinent problem. In the proposed D-DQN based algorithm, the central controller is selected as an agent for periodically observing the state of UAV-enabled wireless network and for carrying out actions to adapt to the dynamic environment. In contrast to the conventional DQN algorithm, the decaying learning rate is leveraged in the proposed D-DQN based algorithm for attaining a tradeoff between accelerating training speed and converging to the local optimal. Numerical results demonstrate that: 1) In contrast to the conventional Q-learning algorithm, which cannot converge when being adopted for solving the formulated problem, the proposed D-DQN based algorithm is capable of converging with minor constraints; 2) The energy dissipation of the UAV can be significantly reduced by integrating RISs in UAV-enabled wireless networks; 3) By designing the dynamic decoding order and power allocation policy, the RIS-NOMA case consumes 11.7% less energy than the RIS-OMA case.

A novel framework is proposed for the deployment and passive beamforming design of a reconfigurable intelligent surface (RIS) with the aid of non-orthogonal multiple access (NOMA) technology. The problem of joint deployment, phase shift design, as well as power allocation in the multiple-input-single-output (MISO) NOMA network is formulated for maximizing the energy efficiency with considering users particular data requirements. To tackle this pertinent problem, machine learning approaches are adopted in two steps. Firstly, a novel long short-term memory (LSTM) based echo state network (ESN) algorithm is proposed to predict users' tele-traffic demand by leveraging a real dataset. Secondly, a decaying double deep Q-network (D3QN) based position-acquisition and phase-control algorithm is proposed to solve the joint problem of deployment and design of the RIS. In the proposed algorithm, the base station, which controls the RIS by a controller, acts as an agent. The agent periodically observes the state of the RIS-enhanced system for attaining the optimal deployment and design policies of the RIS by learning from its mistakes and the feedback of users. Additionally, it is proved that the proposed D3QN based deployment and design algorithm is capable of converging within mild conditions. Simulation results are provided for illustrating that the proposed LSTM-based ESN algorithm is capable of striking a tradeoff between the prediction accuracy and computational complexity. Finally, it is demonstrated that the proposed D3QN based algorithm outperforms the benchmarks, while the NOMA-enhanced RIS system is capable of achieving higher energy efficiency than orthogonal multiple access (OMA) enabled RIS system.

This article proposes a novel framework of resource allocation in multi-cell intelligent reflecting surface (IRS) aided non-orthogonal multiple access (NOMA) networks, where an IRS is deployed to enhance the wireless service. The problem of joint user association, subchannel assignment, power allocation, phase shifts design, and decoding order determination is formulated for maximizing the achievable sum rate. The challenging mixed-integer non-linear problem is decomposed into an optimization subproblem (P1) with continuous variables and a matching subproblem (P2) with integer variables. In an effort to tackle the non-convex optimization problem (P1), iterative algorithms are proposed for allocating transmission power, designing reflection matrix, and determining decoding order by invoking relaxation methods such as convex upper bound substitution, successive convex approximation, and semidefinite relaxation. In terms of the combinational problem (P2), swap matching-based algorithms are developed for achieving a two-sided exchange-stable state among users, BSs and subchannels. Numerical results demonstrate that: i) the sum rate of multi-cell NOMA networks is capable of being increased by 35% with the aid of the IRS; ii) the proposed algorithms for multi-cell IRS-aided NOMA networks can enjoy 22% higher energy efficiency than conventional NOMA counterparts; iii) the trade-off between spectrum efficiency and coverage area can be tuned by judiciously selecting the location of the IRS.

A novel framework of intelligent reflecting surface (IRS)-aided multiple-input single-output (MISO) non-orthogonal multiple access (NOMA) network is proposed, where a base station (BS) serves multiple clusters with unfixed number of users in each cluster. The goal is to maximize the sum rate of all users by jointly optimizing the passive beamforming vector at the IRS, decoding order and power allocation coefficient vector, subject to the rate requirements of users. In order to tackle the formulated problem, a three-step approach is proposed. More particularly, a long short-term memory (LSTM) based algorithm is first adopted for predicting the mobility of users. Secondly, a K-means based Gaussian mixture model (K-GMM) algorithm is proposed for user clustering. Thirdly, a deep Q-network (DQN) based algorithm is invoked for jointly determining the phase shift matrix and power allocation policy. Simulation results are provided for demonstrating that the proposed algorithm outperforms the benchmarks, while the performance of IRS-NOMA system is better than IRS-OMA system.

A novel framework is proposed for enhancing the driving safety and fuel economy of autonomous vehicles (AVs) with the aid of vehicle-to-infrastructure (V2I) communication networks. To solve this pertinent problem, a double deep Q-network (DDQN) algorithm is proposed for making collision-free decisions. Thus, the trajectory and velocity of the AV are determined by receiving real-time traffic information from the base stations (BSs). Compared to the conventional deep Q-network algorithm, the proposed DDQN algorithm is capable of overcoming the large overestimation of action values by decomposing the max-Q-value operation into action selection and action evaluation. Numerical results are provided for demonstrating that the proposed trajectory design algorithms are capable of enhancing the driving safety and fuel economy of AVs. We demonstrate that the proposed DDQN based algorithm outperforms the DQN based algorithm. Additionally, it is also demonstrated that the proposed fuel-economy (FE) based driving policy derived from the DRL algorithm is capable of achieving in excess of 24% of fuel savings over the benchmarks.

A novel framework is proposed for the trajectory design of multiple unmanned aerial vehicles (UAVs) based on the prediction of users' mobility information. The problem of joint trajectory design and power control is formulated for maximizing the instantaneous sum transmit rate while satisfying the rate requirement of users. In an effort to solve this pertinent problem, a three-step approach is proposed which is based on machine learning techniques. Firstly, a multi-agent Q-learning based placement algorithm is proposed for determining the optimal positions of the UAVs based on the initial location of the users. Secondly, in an effort to determine the mobility information of users based on a real dateset, their position data is collected from Twitter to describe the anonymous user- trajectories in the physical world. In the meantime, an echo state network (ESN) based prediction algorithm is proposed for predicting the future positions of users based on the real dataset. Thirdly, a proposed multi-agent Q-learning based algorithm is invoked for predicting the position of UAVs in each time slot based on the movement of users. The algorithm is proved to be able to converge to an optimal state equation. Numerical results are provided to demonstrate that as the size of the reservoir pool increases, the proposed ESN approach improves the prediction accuracy. Finally, we demonstrate that throughput gains of about 17% are achieved.

A novel framework for dynamic 3-D deployment of unmanned aerial vehicle (UAV) in the aerial base station system (ABSS) that based on the machine learning algorithms is proposed. In the framework, the UAV is deployed as an aerial base station to serve a group of ground users and is placed based on the prediction of the users' mobility. The joint problem of prediction of users' track and 3-D deployment of the UAV is formulated for maximizing the sum transmit rate. A two-step approach is proposed for predicting the movement of users and for determining the dynamic 3-D placement of the UAV. Firstly, an echo state network (ESN) based prediction algorithm is utilized for predicting the future positions of users based on the real-world datasets collected from Twitter. Secondly, an iterative K-Means based algorithm is proposed for obtaining the optimal placement of UAV at each time slot based on the output of ESN model. Numerical results are illustrated for showing the superiority of the proposed algorithm over the prevalent algorithm on prediction tasks. The accuracy and efficiency of the proposed framework are also investigated. Additionally, compared with static placement of the UAV, the advantage of dynamic 3-D deployment is demonstrated.