In multicell cellular networks, coordinated multiuser scheduling (CMUS) based on block diagonalization precoding, which jointly selects the scheduled users among multiple base stations (BSs), is an efficient method to eliminate inter-user interference. However, most existing CMUS algorithms require global channel state information and many iterations, which are impractical in dynamic wireless networks due to its heavy information overhead and computational complexity. In this paper, we propose a distributed adaptive CMUS algorithm based on multi-agent reinforcement learning (MARL) to maximize the sum rate with less information overhead and computational complexity. In the proposed scheme, an adaptive multiple Deep Q-Network (DQN) architecture is deployed at each BS, aiming to reduce information overhead and computation complexity by identifying suitable CMUS policies with fewer DQNs. Each BS trains its own multi-DQNs and performs appropriate CMUS actions based on local information. Simulation results show that, by executing a subset of the DQNs, the proposed adaptive multi- DQN architecture reduces at least 50% of the information overhead and the computational complexity of the scheme without adaptive multi-DQN architecture. Additionally, compared to the centralized iterative approach, the proposed scheme costs 9.44% of the information overhead and 3.33%0 of the running time of the centralized iterative approach while achieving marginally superior performance.

In dense small cell networks, the coordinated multi-point noncoherent joint transmission (JT) is a key technique to mitigate inter-cell interference and enhance network capacity. However, the capacity-maximizing power control and beamforming problem subject to a total transmit power constraint at each individual small cell base station (BS) is inherently nonconvex and NP-hard. To solve this problem, most existing algorithms require global channel state information (CSI) and a lot of computations, which are infeasible and impractical in dynamic wireless networks with limited computing power and link capacity. In this paper, we characterize a low-dimensional solution structure for the power control of the sum-rate maximization problem in time division duplex (TDD) dense small cell networks. Taking advantage of this low-dimensional structure, a distributed noncoherent JT scheme based on multi-agent reinforcement learning (MARL) is proposed to maximize the sum-rate of the dense small cell networks with reduced information overhead. In the proposed scheme, each BS acts as an agent and makes decisions locally. It is proved that the optimal sum-rate can be achieved for single-transmit-antenna BSs by using the proposed scheme. Compared to the best method presently known, the proposed scheme achieves a similar sum-rate with considerably lower computational complexity and information overhead, which makes it more appealing for practical implementations.