Deep reinforcement learning (RL) has been applied to traffic signal control recently and demonstrated superior performance to conventional control methods. However, there are still several challenges we have to address before fully applying deep RL to traffic signal control. Firstly, the objective of traffic signal control is to optimize average travel time, which is a delayed reward in a long time horizon in the context of RL. However, existing work simplifies the optimization by using queue length, waiting time, delay, etc., as immediate reward and presumes these short-term targets are always aligned with the objective. Nevertheless, these targets may deviate from the objective in different road networks with various traffic patterns. Secondly, it remains unsolved how to cooperatively control traffic signals to directly optimize average travel time. To address these challenges, we propose a hierarchical and cooperative reinforcement learning method-HiLight. HiLight enables each agent to learn a high-level policy that optimizes the objective locally by selecting among the sub-policies that respectively optimize short-term targets. Moreover, the high-level policy additionally considers the objective in the neighborhood with adaptive weighting to encourage agents to cooperate on the objective in the road network. Empirically, we demonstrate that HiLight outperforms state-of-the-art RL methods for traffic signal control in real road networks with real traffic.

This paper considers the resource allocation in cache-enabled energy-cooperative HetNets, where base stations (BSs) with different cache sizes are powered by both conventional grids and renewable energy sources, and energy can be shared between BSs via the smart grid. Simulation results demonstrate that the proposed joint user association and power control algorithm can significantly enhance the sum data rate and the energy efficiency of the whole network.

This paper focuses on resource allocation in energy-cooperation enabled two-tier heterogeneous networks (HetNets) with non-orthogonal multiple access (NOMA), where base stations (BSs) are powered by both renewable energy sources and the conventional grid. Each BS can serve multiple users at the same time and frequency band. To deal with the fluctuation of renewable energy harvesting, we consider that renewable energy can be shared between BSs via the smart grid. In such networks, user association and power control need to be re-designed, since existing approaches are based on OMA. Therefore, we formulate a problem to find the optimum user association and power control schemes for maximizing the energy efficiency of the overall network, under quality-of-service constraints. To deal with this problem, we first propose a distributed algorithm to provide the optimal user association solution for the fixed transmit power. Furthermore, a joint user association and power control optimization algorithm is developed to determine the traffic load in energy-cooperation enabled NOMA HetNets, which achieves much higher energy efficiency performance than existing schemes. Our simulation results demonstrate the effectiveness of the proposed algorithm, and show that NOMA can achieve higher energy efficiency performance than OMA in the considered networks.

This paper considers power control in energy cooperation enabled heterogeneous networks (HetNets), where each base station (BS) is powered by hybrid energy sources consisting of the conventional power grid and renewable energy sources. Energy can be transferred between BSs with energy loss during the energy transmission process. Transmit power, grid power consumption, and transferred energy are optimized for maximizing the energy efficiency of the whole network. The considered problem is formulated as a non-linear fractional programming problem. To solve it, we propose an energy efficient algorithm, in which the optimal resource allocation policy is obtained by using the lagrangian duality method. Numerical results demonstrate that energy efficiency is substantially improved by using the proposed power control algorithm with energy cooperation, compared with the cases where either power control or energy cooperation are considered.

This paper considers a hybrid heterogeneous network (HetNet), where macro cells adopt massive multiple-input multiple-output (MIMO), and small cells adopt millimeter wave (mmWave) transmissions. We assume that all base stations (BSs) are solely powered by the renewable energy. The implementation of these emerging techniques has a substantial effect on the user association (UA). Motivated by this, we formulate a user association problem to maximize the network utility while the power cost of each BS does not exceed the harvested energy. To solve it, a low complexity distributed UA algorithm is proposed. The results demonstrate that the proposed algorithm achieves higher throughput than the max reference signal received power (RSRP) and max signal-to-interference-plus-noise ratio (SINR) UAs. It also shows that increasing the number of antennas at the macro cell BS with more power consumption, the throughput continues to increase by using the proposed algorithm, compared to the decrease in throughput by using the existing ones. Increasing the number of mmWave BSs, mmWave BS antennas or mmWave bandwidths can significantly improve the throughput. Compared with massive MIMO macro cells, mmWave small cells play a dominant role in enhancing the throughput of the networks due to the larger bandwidths.