Many of the existing optimization-based transmit power control algorithms suffer from high computational complexity and require instantaneous global channel state information (CSI), both of which hinder their practical implementation. In this paper, we consider a wireless network with multiple transmitter-receiver pairs, where each transmitter only has access to its local CSI fed back from its intended receiver and does not require local CSI exchange with its neighboring transmitters. In such a network scenario, we propose a deep reinforcement learning based decentralized multi-agent power control (DEC-MAPC) algorithm for sum-rate maximization, where each transmitter acts as an intelligent agent. By leveraging the value decomposition technique, we establish a nonlinear mapping from the local reward of each agent to the global reward. Such a design allows each agent to independently control its transmit power based on its local CSI while enabling global collaboration among the agents. The proposed algorithm is scalable to large-scale networks as only local CSI is required, and is robust to the channel and interference variations via interacting with the environment. Simulation results show that the proposed DEC-MAPC algorithm with local CSI achieves comparable sum-rate performance with the centralized optimization algorithms with global CSI, while significantly reducing the computational complexity.

In a time-division duplex (TDD) massive multiple-input multiple-output (MIMO) system, the number of available orthogonal pilot sequences in each cell is limited. In this paper, by assuming Gauss-Markov fading channels, we consider a massive MIMO system where there are less available orthogonal pilots than the number of users. In order to optimize the long-term performance in estimating the uplink (UL) channels, the base station (BS) judiciously decides the allocation of the available pilots to different users in each training phase. The pilot allocation problem is a partially observable Markov decision process with an exploitation-exploration tradeoff that is difficult to analyze. We thus investigate the problem in the framework of restless multi-armed bandits (RMAB) problems and carry out an indexability analysis for the problem. Furthermore, we solve this problem by using the Whittle's index policy with a low complexity. Numerical results demonstrate the superiority of the Whittle's index policy.