The utilization of urban underground spaces poses new challenges for underground Internet of Things (IoT), such as energy constraint of sensor nodes (SNs) and channel blockage caused by complex spatial structures. Existing approaches rely on centralized access points for wireless energy transfer (WET), suffering from severe energy loss over long distances. Besides, time duplex energy-data transmission is inefficient. To address these issues, we propose a Reconfigurable Intelligent Surface (RIS)-assisted frequency division duplex (FDD) underground IoT architecture, which facilitates concurrent WET to SNs and uplink heterogeneous data transmission across different frequency bands. To minimize the weighted sum of energy consumption and trans-mission pressure—a metric designed to characterize transmission-task execution situations, we formulate an optimization problem that jointly coordinates the time-frequency-energy resource blocks and RIS phase shifts under heterogeneous Quality of Service (QoS) requirements. Then, we design a parameter-sharing Multi-Agent Proximal Policy Optimization (PS-MAPPO) algorithm which stably converging to the optimal solution of the proposed problem while reducing the complexity of the network. Simulation results demonstrate that the proposed method is superior in energy saving, latency guarantee and improving the task completion rate compared to existing methods.

In the vehicle edge computing network (VECN), how to deal with the computation resources and energy resources shortage problem the roadside units (RSUs) encounter when they are performing delay sensitive computation tasks is an important issue, especially during the peak hours and the situation of VECN is dynamic. To complete the computation tasks on time with the minimum expenditure, in this paper, we investigate the problem of information-energy collaboration among RSUs, where the spectrum management is also involved. For the considered scenario, the RSUs’ strategies of spectrum selection, computation task offloading and energy sharing are derived from the formulated optimization problem. Since the proposed problem is a highly complex mixed-integer nonlinear programming problem and the strategies are coupled with each other, a multi-agent deep deterministic policy gradient (MADDPG) based algorithm is proposed to find the sub-optimal solutions quickly in a dynamic environment. The simulation results show that our approach is superior to the existing schemes in terms of total system expenditure and the spectral efficiency.

For electric vehicles (EVs) that are running out of power during the journey, how to select the charging station (CS) and charge at the minimum cost is an important problem. Taking the computation energy consumption and the carbon tax into consideration, in this paper, the CS and charging power selection problem is formulated into an optimization problem, where the charging expenditure and waiting time are minimized. Since the proposed problem changes dynamically with EV user behaviors, the proposed problem cannot be solved by traditional optimization methods. Then the Multi-Agent Deep Deterministic Policy Gradient (MADDPG) algorithm is used to find the near-optimal solution. Simulation results show that our proposed strategy helps reduce not only the charging costs but also the CO2emissions. In addition, our strategy balances the load of each CS and reduces the peak-to-valley difference in electricity consumption.

To achieve a green heterogeneous 5G network, a promising approach is to shut off the light-loaded low-power nodes (LPNs) and transfer the load to the remote radio heads (RRHs) nearby. However, the RRHs may refuse to cooperate when there is no incentive. Considering a user will stay in an RRH for a period of time, how to provide proper long-term incentives for the potential RRHs and select the best one is an essential issue. Since the offloading capability of an RRH is private information, that is, unavailable to the LPNs, in this article, the RRHs’ collaboration incentive problem under such asymmetric information condition is modeled as a long-term contract design problem. In the formulated problem, the channel condition and traffic load, which represent both RRH’s instantaneous state and long-term state, are combined to capture the RRHs’ offloading capacity and used to classify their types. Due to the dynamic of state, an RRH’s type varies with time, which greatly complicates the contract design. We first study the state transition of RRHs and the long-term utilities of both parties, with which the contract-theoretic framework is formulated. Then, we theoretically analyze and simplify the individual rational and incentive-compatible constraints for a feasible long-term contract. Finally, we propose a low time complexity algorithm to find the optimal contract. Numerical results verify that the long-term contract-based incentive mechanism not only improves the utilities of both cooperation parties, but also is superior to the existing works in reducing handover cost and energy consumption.

To offload the rapidly increasing video traffic in the Internet of Things (IoT), the caching and computing-enabled unmanned aerial vehicles (UAVs) have become a paradigm in alleviating the pressure of wireless backhauls of the mobile network operator (MNO) and improving the Quality of Service (QoS) of subscribers of content providers (CPs), especially, in the situations where fixedly deployed small-cell base stations (SBSs) are not applicable. However, the resources in the UAV are costly and limited, while rational CPs are reluctant to reveal their willingness of leasing. Taking such asymmetric information into consideration, the problem that how the MNO leases the caching and computing resources to different CPs is formulated into a contract design problem, in which the UAV’s 3-D position deployment is also involved. To find the optimal solutions, the proposed problem is decoupled into two subproblems. In the position deployment subproblem, the 3-D position of the UAV is analyzed, and the hovering height and coverage radius are jointly optimized to maximize the offloading volume during the life of the UAV. With the derived optimal position of the UAV, the individual rationality (IR) and incentive compatibility (IC) constraints in the contract design subproblem are simplified, and a low complexity algorithm based on the alternating direction method of multipliers (ADMMs) is proposed to find the optimal contract. Finally, the effectiveness of the proposed scheme is verified by simulation, and a comparative analysis is carried out in terms of save latency, saved bandwidth and utility.

For edge computing (EC) network, one critical problem is how to process computation-intensive task in time with efficient energy usage. However, existing works mainly study one aspect of the problem by computation offloading or energy cooperation. Considering that the computation offloading strategy and the energy cooperation strategy are coupled with each other, in this paper, we propose a new information-energy collaboration model for the EC network powered by renewable energy and stored energy. In this new model, an EC node can collaborate with the other EC nodes and the cloud for computation offloading. At the same time, the EC is also able to store energy and share energy with other EC nodes. In such a case, since the EC nodes can have a stable power supply to finish the computing tasks within the latency limit, we formulate the problem of deriving the information-energy collaboration strategy as the optimization problem of minimizing the cloud computing cost and the power purchase cost. To find the optimal offloading and energy cooperation strategy, we first analyze the sixteen cases of the offloading strategy depending on the computing tasks and the renewable energy. Then we further summarize four cases of energy collaboration strategy according to the different offloading strategies. To derive the collaboration strategy with low complexity, we propose the practical Hybrid Greedy Iterative Algorithm (HGIA) to the optimization problem. Finally, the simulation results demonstrate that our approach is effective and stable.

In the aftermath of disasters, where the communication infrastructure is often impaired or completely unavailable, unmanned aerial vehicle (UAV) assisted Mobile Ad Hoc network(MANET) is a promising choice to recover wireless communication. However, affected by the dynamic topology and time-varying channel quality caused by the nodes' mobility in emergency scenarios, it is difficult to guarantee the quality of service (QoS) of various types of packets in terms of transmission delay. In this paper, a dynamic priority packet scheduling scheme is proposed to maintain high QoS in post-disaster UAV assisted MANET, where not only the occurred packet delay has experienced but also the impact that will occur in the future transmission is taken into consideration on the priority assignment. Specifically, to characterize the dynamic features of nodes, the Gauss-Markov Mobility Model is exploited. Then to incorporate the impacts of the node's movement, as well as the instability of the topology and the time-varying channel quality into the packet's priority assignment, we estimate the packet's transmission delay where the device-to-device(D2D), device-to- UAV(D2U), and UAV-to-UAV(U2U) channels are all specified, and theoretically analyzed the probability of link duration in the future transmission. Simulation results show that the proposed dynamic priority scheme outperforms the static priority and (first in first out) FIFO scheme in terms of the overall packets transmission success ratio and transmission delay.

Deploying unmanned aerial vehicles (UAVs) as flying base stations (BSs) is a promising solution to alleviate the burden of communication infrastructure during the peak-traffic hours. However, when a UAV is deployed as a flying BS to serve the vehicle users in the hotspot, and the vehicle-to-vehicle (V2V) communication is introduced to further improve the network capacity, the performance analysis and optimization problems have not gained well investigated. In this paper, aforementioned problems are carefully studied from a statistical point of view, where the system performance is captured by the users' successful service probability. Specifically, we first derive the successful service probability for the UAV-to-vehicle (U2V) transmission. Meanwhile, taking those important factors, i.e., vehicle mobility and social proximity, into account, we estimate the successful service probability for the V2V transmission. The average successful service probability for the considered scenario is then derived. Based on the mathematical analysis results, we further improve the system performance by adjusting the UAV's altitude position, where the UAV deployment problem is formulated as a service probability maximization problem. To find the optimal solution, a particle swarm optimization algorithm is proposed. Finally, numerical simulations are conducted to verify the theoretical analysis and the efficiency of our proposed scheme.

We examine the problem of Value of Information (VoI) based underwater information collection, which is the essence of many underwater applications such as depth surrounding oil platforms, monitoring of algal blooms and so on. Even if the information collection work has been carried out much in the terrestrial scenario, due to complicated physical, technological and economic differences between the terrestrial and underwater cases, it is not feasible to simply apply the existing terrestrial tricks. The existing cooperative Autonomous Underwater Vehicle (AUV) working paradigms are limited to omniscience of communication channel information among the AUVs, which is yet not practical in such a harsh communication environment. Therefore, we propose a contract based model which has little restriction on the communication channel to overcome information asymmetry and jointly optimizes energy consumption and VoI. Besides, we provide a concrete theoretical proof of the contract items and carry out a performance simulation which shows the mechanism we design is operative. At last, we summarize our work and give an insight of the future research directions.

Recently, the technology of underwater wireless sensors networks (UWSNs) has received more attention on the exploitation of marine resources. However, underwater acoustic communication is still the only reliable means of ocean communication, which is entirely different from the terrestrial scene. In this paper, we propose Q-learning aided ant colony routing protocol (QLACO) to address the issues of energy-efficiency and link instability in UWSNs, which uses both the reward mechanism and artificial ants to determine a global optimal routing selection. QLACO uses the reward function to adapt to the dynamic underwater environment and enhance the packet delivery ratio (PDR). Moreover, we propose an anti-void mechanism to solve the void region dilemma. Simulation results show that QLACO outperforms Q-learning-based energy-efficient and lifetime-aware routing protocol (QELAR) and the depth-based protocol (DBR) in terms of PDR, energy consumption and latency.

Multi-access edge computing (MEC) is viewed as a promising technology to improve the real time video service in vehicular networks. However, in the traditional vehicular networks, the road side units (RSUs) are usually only equipped with communication modules, and the unmanned aerial vehicles(UAVs) are seldom used. In this paper, a new UAV-aided time-varying vehicular network is introduced for vehicle users (VUEs) to obtain better experience, where the RSUs and the UAV are equipped with MEC servers for the real time video transcoding. Considering that the video service always lasts for a period of time, we investigate the user association policy from a long-term perspective. Specifically, to characterize the time-varying features of communication links and the heterogeneity of available resources, we theoretically derive the achievable video chunks and link reliability based on the vehicle mobility model and content caching model. Then, the user association problem is formulated as the utility optimization problem, where both the VUE’s quality of experience (QoE) and handover cost are taken into consideration. Furthermore, we propose an improved Dijkstra algorithm to solve the original NP-hard problem after it is transformed to a shortest path selection problem. Finally, by numerical results, we verify that the proposed scheme outperforms existing schemes in terms of the VUE’s QoE and the handover numbers.

Motivating the potential offloading nodes (PONs) to cooperate with the offloading request nodes (ORNs) is an essential issue of traffic offloading in 5G. Nevertheless, in the coupled traffic offloading case where the PONs' uplink and downlink channel state information (CSI) of both the access and fronthault links is unknown to the ORNs, it is very difficult for the ORNs to provide an appropriate reward which is consistent with the PONs' effort. In this paper, we propose a contract-based framework to tackle this challenge, where the PONs are agents characterized by a two-tuple type, while the ORNs are the principles providing contracts in the form of (offloading quality, monetary reward). Since it is a tradeoff between offloading quality and monetary reward, when designing the contract, the ORN integrates these two factors into the offloading utility function, and then optimizes the utility under the constraints of individual rationality and incentive compatibility conditions. Through mathematical analysis, the necessary and sufficient conditions of the formulated problem for the one-ORN scenario is simplified. Then a gradient descent algorithm is proposed to find the optimal solutions, i.e., the optimal contract. Furthermore, the study on the PON cooperation stimulation problem is extended to the multi-ORN scenario, where the decision-making process of the ORNs and the PONs is formulated into a matching game. To find the stable solution to the matching game, a revised deferred acceptance algorithm is proposed and then proved to be convergent and have low computational complexity. Simulation results demonstrate that the proposed scheme achieves higher offloading utility and energy efficiency compared with the existing schemes.

In order to improve the energy efficiency of the heterogeneous cloud radio access networks (H-CRANs), it is a promising approach to shutting off the small base stations (SBSs) with low traffic load and offload the traffic to the active SBSs nearby. Due to the selfish nature of the SBSs, the nearby SBSs may not cooperate if there is not appropriate incentive. On the other hand, since a user will be served by a SBS for a period of time, during which the SBS's offloading ability, i.e., its own traffic load and the offloading channel conditions, may vary dramatically. In such a case, how to provide proper long-term incentives for the potential SBSs to take over the traffic is an essential issue, especially when these SBSs belong to different service providers. Considering the SBS's offloading ability is the privat information that is unknown to the shutting off SBS, in this paper, the SBSs' cooperation stimulation problem is formulated as a long-term contract design problem, where the shutting off SBS acts as the principal and the offloading SBSs are the agents. A contract-theoretic framework based on the Markov decision process is formulated to study the long-term utilities of both parties. Through theoretical analysis, the feasible conditions for the optimal solution are described. Finally, the optimal contract is obtained through the proposed algorithm, and simulation results validate the effectiveness of our proposed long-term contract-based incentive mechanism for traffic offloading in H-CRANs.

Shutting off the small base stations (SBSs) with low traffic load is a promising approach to save energy, and how to incentive the SBSs nearby to cooperate and take over the traffic is an important problem, especially when these SBSs belong to different service providers and have different benefit targets. In this paper, we assume the channel state information (CSI) of both the access link and the fronthaul link of a potential off-loading SBS (oSBS) is unavailable to the shutting down SBS (sSBS). Under such an asymmetry information condition, contract theory is exploited to encourage the nearby oSBSs to take over the traffic- load and help the sSBS select an appropriate oSBS. Such an off-loading cooperation stimulation problem is formulated into a contract design problem, where the sSBS is the principal and the oSBSs are agents. Through theoretical analysis, we simplify the constraints of the formulated problem, and derive the optimal solution, i.e., the optimal contract, for the scenario where the CSI of the access link and fronthaul link of an oSBS are both quantized into two levels. Numerical results validate that the proposed contract-based incentive scheme is effective for improving the utilities of both the oSBS and sSBS.

To overcome the hidden terminal problem a secondary user (SU) may encounter, cooperative spectrum sensing (CSS) is proposed and gained much attention in the last decades. However, due to the selfish nature, SUs may not cooperate unconditionally as most previous works have assumed. Therefore, how to stimulate SUs to play cooperatively is an important issue. In this paper, we propose a reputation-based CSS incentive framework, where the cooperation stimulation problem is modeled as an indirect reciprocity game. In the proposed game, SUs choose how to report their sensing results to the fusion center (FC) and gain reputations, based on which they can access a certain amount of vacant licensed channels in the future. For the proposed game, we derive theoretically the optimal action rule, according to which the SU will truthfully report its result when the estimated average energy is equal to or higher than the given threshold and vice versa. The decision accuracy of the FC thereby can be greatly improved. Moreover, we derive the condition under which the optimal action rule is evolutionarily stable. Finally, simulation results are shown to verify the effectiveness of the proposed scheme.

Device-to-Device (D2D) communication is a promising technology component towards the fifth generation mobile communication. However, when D2D communication is incorporated into heterogeneous cloud radio access network (H-CRAN), the interference management between the D2D users and current users is a challenge. In this paper, we study how to assign the sub-channels of different bandwidth to multiple D2D pairs and the RRH users (RUEs). In such a case, the sub-channel that has been pre-allocated to a macrocell user (MUE) can be reused, the system performance that accounts for both the overall throughput and the number of admitted users can be maximized and all the users quality of service (QoS) can be guaranteed. Such a resource allocation problem is formulated as a mixed integer nonlinear programming (MINLP) problem which is NP-hard. To obtain the optimal solution, the proposed problem is transformed into a many-to-one matching game with externalizes, and a constrained-DA algorithm is proposed. We prove theoretically that with the constrained-DA algorithm, the matching game can reach the stable matching with a low computational complexity. Numerical results demonstrate that our proposed scheme can significantly improve the system performance in terms of overall throughput and the total number of admitted users compared with existing resource allocation schemes.

Data sharing is one critical step to implement data fusion, and how to encourage sensors to share their data is an important issue. In this paper, we propose a reputation-based incentive framework, where the data sharing stimulation problem is modeled as an indirect reciprocity game. In the proposed game, sensors choose how to report their results to the fusion center and gain reputations, based on which they can obtain certain benefits in the future. Taking the sensing and fusion accuracy into account, reputation distribution is introduced in the proposed game, where we prove theoretically the Nash equilibrium of the game and its uniqueness. Furthermore, we apply the proposed scheme to the cooperative spectrum sensing. We show that within an appropriate cost-to-gain ration, the optimal strategy for the secondary users is to report when the average received energy is above a given threshold and keep silence otherwise. Such an optimal strategy is also proved to be a desirable evolutionarily stable strategy. Finally, simulation results are shown to verify the theoretical results and demonstrate that compared with the existing schemes, our proposed scheme achieves better operating characteristic curve and higher system throughput with convincing performance on fairness.

In cognitive radio networks, secondary users (SUs) are allowed to opportunistically exploit the licensed channels by sensing primary users' (PUs) activities. Once finding the spectrum holes, SUs generally need to share the available licensed channels. Therefore, one of the critical challenges for fully utilizing the spectrum resources is how the SUs obtain accurate information about the PUs' activities and make right decisions of accessing channels to avoid competition from other SUs. In this paper, we formulate SUs' learning and decision making process as a Chinese Restaurant Game by considering the scenario where SUs sense channels simultaneously and make access decisions sequentially. In the proposed game, SUs build the knowledge of the PUs' activities by their own sensing and learning the information from other SUs. They also predict their subsequent SUs' decisions to maximize their own utilities. We analyze the interactions among SUs in the proposed game and study specifically the impact of SUs' prior belief and sensing accuracy on their decisions. We also derive the theoretic results for the two-user two-channel case. Finally, we demonstrate the effectiveness and efficiency of the proposed scheme through simulations.

In this paper, we propose a spectrum access framework to address the efficient allocation of channels of a base station (BS) by stimulating cooperation between primary users (PUs) and secondary users (SUs). We model the cooperation stimulation problem as an indirect reciprocity game, where SUs help PUs relay information and gain reputations to access the vacant channels in the future. We design a reputation updating policy under time-varying channels and prove the existence of stationary reputation distribution. We further formulate the decision making of an SU as a Markov Decision Process (MDP) and use a modified value iteration algorithm to find the optimal action rule. Moreover, we theoretically derive the condition under which the optimal action rule is an evolutionarily stable strategy (ESS). Finally, simulation results are shown to verify the effectiveness of the proposed scheme.

In cognitive radio networks, when the primary user (PU) who has a minimum transmission rate requirement is experiencing a bad channel condition, it can resort to the cooperation with the secondary users (SUs) in exchange for a fraction of its licensed spectrum. However, due to the selfish nature, the SUs may not act as cooperatively as to meet the PU's Quality of Service (QoS) requirement. On the other hand, the SUs may not exploit efficiently the benefit from cooperation if they compete with each other and collaborate with the PU independently. Therefore, how to stimulate the SUs to form a group to cooperate with the PU while guarantee the PU's QoS requirement are critical challenges. In this paper, we propose a two-level bargain framework to address the aforementioned problems. In the proposed framework, the interaction between the PU and the SUs are modeled as the first level bargain game while the second level bargain game is used to formulate the SUs' decision making process on spectrum sharing. We analyze the optimal actions of the PU and the SU and derive the theoretic results for the one-PU one-SU scenario. To find the solutions for the one-PU multi-SU scenario, we propose a revised numerical searching algorithm and prove its convergence. Finally, we demonstrate the effectiveness and efficiency of the proposed scheme through simulations.

Cooperation is a promising approach to simultaneously achieve efficient utilization of spectrum resource and improve the quality of service of primary users in dynamic spectrum access networks. However, due to the selfish nature, secondary users may not act as cooperatively as primary users have expected. Therefore, how to stimulate the secondary users to play cooperatively is an important issue. In this paper, we propose a reputation-based spectrum access framework, where the cooperation stimulation problem is modeled as an indirect reciprocity game. In the proposed game, secondary users choose how to help primary users relay information and gain reputations, based on which they can access a certain amount of vacant licensed channels in the future. By formulating a secondary user's decision making as a Markov decision process, we obtain the optimal action rule, according to which the secondary user will use maximal power to help primary user relay data if the channel is not in an outage, and thus greatly improve the primary user's quality of service as well as the spectrum utilization efficiency. Moreover, we prove the uniqueness of stationary reputation distribution and theoretically derive the condition under which the optimal action rule is evolutionarily stable. Finally, simulation results are shown to verify the effectiveness of the proposed scheme.

This paper discusses on the fair resource allocation in OFDMA two-hop cooperative relaying cellular networks which consist of a single base station (BS), dedicated fixed relay stations (RSs) and user stations (USs). By assuming the relay strategy employed on each subcarrier is adaptively selected among direct transmission, amplify-and-forward (AF) and decode-and-forward (DF) according to different channel conditions, we formulate a fair subcarrier assignment, relay station and relay strategy selection problem with QoS constraints in terms of user's minimum rate requirement. The formulated problem is a binary integer programming (BIP) problem which is NP-complete, so a simple suboptimal algorithm is proposed to manage the network resources. Simulations demonstrate the proposed algorithm obtains near optimal solution with low complexity and achieves a good tradeoff between the overall system performance and the fairness among users.