Mobile crowdsensing (MCS) is a novel sensing paradigm by utilizing mobile users (MUs) to collect data from environment. Considering the finite sensing and computing resources of MUs, it is crucial to inspire MUs to take part in crowdsensing willingly. In this study, a multiagent-deep-reinforcement-learning (DRL)-based incentive mechanism is investigated to tackle the joint data sensing and computing issues. Specifically, due to the heterogeneity of sensing tasks, multiple MCS platforms (MCPs) motivate MUs to participate in different tasks. The interaction between MCPs and MUs is modeled as a multileader-multifollower Stackelberg game with Stackelberg equilibrium proved by derivation. Moreover, the Stackelberg game is transformed as a Markov decision process (MDP) to deal with a multiagent DRL method without any prior knowledge. Due to the continuous high-dimensional action space of multiple MCPs and MUs, a multiagent double actors deep deterministic policy gradient (MA-DADDPG) algorithm is proposed to obtain the optimal sensing data size, computing resource, and incentive payment policies. Extensive simulation results illustrate the effectiveness of the proposed crowdsensing incentive mechanism.

Mobile crowdsourcing network is a promising paradigm to leverage mobile users (MUs) to perform large-scale sensing task. Due to limited sensing-computation resource and data security risk, it is necessary to design an efficient incentive mechanism to motivate MUs to complete crowdsourcing task. In this paper, a deep reinforcement learning (DRL)-based contract incentive mechanism is proposed by jointly considering participation contribution, sensing task, and computation resource of MUs. Specifically, considering the heterogeneous willingness of the MUs, we formulate a three-dimensional sensing-computation-reward contract incentive mechanism to obtain the maximum utility of mobile crowdsourcing platform. Moreover, based on the individual rationality and incentive compatibility constraints, we derive the optimal contract under the partial information asymmetry scenario. In the case of the complete information asymmetry scenario, we formulate the contract incentive issue as an Markov decision process. Considering the infinite and continuous action and state spaces, we develop the deep deterministic method to obtain the efficient sensing task, computation resource, and incentive reward policy. Finally, we conduct numerical simulation to demonstrate the feasibility of our DRL-based contract crowdsourcing incentive mechanism.

Vehicular edge computing network emerges as a key technique to offload vehicles’ tasks to the nearby roadside unit (RSU). Considering that the RSU may not always meet computation requirements of task vehicles (TVs), utilizing idle computing resources of the surrounding resource vehicles (RVs) becomes a feasible solution to enhance the TVs’ experience. Due to the self-interested property and limited resource of RVs, an efficient incentive mechanism should be designed to encourage RVs to participate in task offloading and resource cooperation. In this work, a deep reinforcement learning-assisted contract incentive mechanism is investigated by considering TVs’ task offloading requirements, RVs’ computation resources, and the RSU’s transmission time incentive. To truthfully reveal TV-RV task-resource coordination types, a contract is designed with computation task-transmission time contract items. The joint task offloading and resource cooperation optimization issue is formulated to maximize the RSU’s utility with incentive compatible (IR), individual rationality (IC), and task offloading constraints. The optimal transmission time strategy is first derived from IR and IC constraints. To obtain the task offloading and task data size strategies, a Markov decision process is formulated. A multiagent parametrized deep Q-network scheme is developed to handle the discrete-continuous hybrid action space problem. Numerical simulations show the feasibility and effectiveness of our proposed incentive method to solve the joint task offloading and resource cooperation problem.

Mobile edge computing can effectively reduce service latency and improve service quality by offloading computation-intensive tasks to the edges of wireless networks. Due to the characteristic of flexible deployment, wide coverage and reliable wireless communication, unmanned aerial vehicles (UAVs) have been employed as assisted edge clouds (ECs) for large-scale sparely-distributed user equipment. Considering the limited computation and energy capacities of UAVs, a collaborative mobile edge computing system with multiple UAVs and multiple ECs is investigated in this paper. The task offloading issue is addressed to minimize the sum of execution delays and energy consumptions by jointly designing the trajectories, computation task allocation, and communication resource management of UAVs. Moreover, to solve the above non-convex optimization problem, a Markov decision process is formulated for the multi-UAV assisted mobile edge computing system. To obtain the joint strategy of trajectory design, task allocation, and power management, a cooperative multi-agent deep reinforcement learning framework is investigated. Considering the high-dimensional continuous action space, the twin delayed deep deterministic policy gradient algorithm is exploited. The evaluation results demonstrate that our multi-UAV multi-EC task offloading method can achieve better performance compared with the other optimization approaches.

Unmanned aerial vehicle (UAV) is considered to be a key component in the next-generation cellular networks. Considering the non-convex characteristic of the trajectory design and power allocation problem, it is difficult to obtain the optimal joint strategy in UAV-assisted cellular networks. In this paper, a reinforcement learning-based approach is proposed to obtain the maximum long-term network utility while meeting with user equipments' quality of service requirement. The Markov decision process (MDP) is formulated with the design of state, action space, and reward function. In order to achieve the joint optimal policy of trajectory design and power allocation, deep reinforcement learning approach is investigated. Due to the continuous action space of the MDP model, deep deterministic policy gradient approach is presented. Simulation results show that the proposed algorithm outperforms other approaches on overall network utility performance with higher system capacity and faster processing speed.

Traffic offloading is considered to be a promising technology in the Unmanned Aerial Vehicles- (UAVs-) assisted cellular networks. Due to their selfishness property, UAVs may be reluctant to take part in traffic offloading without any incentive. Moreover, considering the dynamic position of UAVs and the dynamic condition of the transmission channel, it is challenging to design a long-term effective incentive mechanism for multi-UAV networks. In this work, the dynamic contract incentive approach is studied to attract UAVs to participate in traffic offloading effectively. The two-stage contract incentive method is introduced under the information symmetric scenario and the information asymmetric scenario. Considering the sufficient conditions and necessary conditions in the contract design, a sequence optimization algorithm is investigated to acquire the maximum expected utility of the base station. The simulation experiment shows that the designed two-stage dynamic contract improves the performance of traffic offloading effectively.

Heterogeneous cellular networks can offload the mobile traffic and reduce the deployment costs, which have been considered to be a promising technique in the next-generation wireless network. Due to the non-convex and combinatorial characteristics, it is challenging to obtain an optimal strategy for the joint user association and resource allocation issue. In this paper, a reinforcement learning (RL) approach is proposed to achieve the maximum long-term overall network utility while guaranteeing the quality of service requirements of user equipments (UEs) in the downlink of heterogeneous cellular networks. A distributed optimization method based on multi-agent RL is developed. Moreover, to solve the computationally expensive problem with the large action space, multi-agent deep RL method is proposed. Specifically, the state, action and reward function are defined for UEs, and dueling double deep Q-network (D3QN) strategy is introduced to obtain the nearly optimal policy. Through message passing, the distributed UEs can obtain the global state space with a small communication overhead. With the double-Q strategy and dueling architecture, D3QN can rapidly converge to a subgame perfect Nash equilibrium. Simulation results demonstrate that D3QN achieves the better performance than other RL approaches in solving large-scale learning problems.

Heterogeneous networks (HetNets) can offload the traffic and reduce the deployment cost, which is regarded as a promising technique in next-generation cellular networks. Because of the non-convex and combinatorial features of the joint issue of user association and resource allocation, it is challenging to achieve an optimal solution. In this paper, a novel method is proposed to maximize the long-term overall network utility while ensuring the user equipments' quality of service requirements in the downlink of HetNets. Multi-agent reinforcement learning approach is developed to obtain the distributed optimal strategy. To solve the computationally expensive issue with the large action space, the multi-user deep reinforcement learning is presented. Double deep Q-network (DDQN) approach is introduced to achieve an optimal policy. Simulation results clearly indicate the better performance of DDQN than that of other reinforcement learning methods.

Cooperative communication is a promising technique to mitigate channel impairment and improve spectrum efficiency. Due to the selfish nature of relay nodes, how to provide proper long-term incentives for relay nodes in dynamic communication environments is an essential issue. In this paper, a two-period dynamic contract is proposed under the dynamic asymmetric information scenario. Considering the relay nodes' types are independent in both periods with identical probability distribution, the contract-theoretic model for ability discrimination relay selection is formulated. And the necessary and sufficient conditions for the optimal contract are systematically characterized. To maximize the source's expected utility, a sequential optimization algorithm is proposed to obtain the optimal relay- reward strategy. Simulation results show that the optimal dynamic contract design scheme is effective in improving system performance for cooperative communication.

Cooperative spectrum sensing (CSS) is a promising technique in cognitive radio networks (CRNs) that utilises multi‐user diversity to mitigate channel instability and noise uncertainty. In this study, the relationship between ‘cooperation mechanisms’ and ‘spatial‐spectral diversity’ over multiple channels jointly sensing is investigated in the presence of an imperfect reporting channel. The multiple channels are sensed at the receiver built on the filter bank‐based multi‐carrier system. The multi‐channel CSS strategies are modelled by the introduced ‘cooperative ratio’ to balance the requirements on ‘sensing accuracy’, ‘efficiency’ and ‘overhead’, which is quantitatively characterised by the energy consumption. The target of CSS is to maximise the aggregate opportunistic throughput of secondary users (SUs) by jointly considering constraints on sensing overhead and the aggregate interference to primary users (PUs). The optimisation is divided into two sequential sub‐optimisation processes, ‘multi‐user diversity optimisation’ and ‘multi‐channel diversity optimisation’. An approach is developed from generic algorithms to solve the two sub‐problems. Numerical results show that the optimal CSS scheme is effective in improving channel utilisation for SUs with low interference to PUs. This study establishes a valuable cooperative model for the design of multi‐channel spectrum sensing algorithms in CRNs.