We consider a multichannel random access system in which each user accesses a single channel at each time slot to communicate with an access point (AP). Users arrive to the system at random and be activated for a certain period of time slots and then disappear from the system. Under such dynamic network environment, we propose a distributed multichannel access protocol based on multi-agent reinforcement learning (RL) to improve both throughput and fairness between active users. Unlike the previous approaches adjusting channel access probabilities at each time slot, the proposed RL algorithm deterministically selects a set of channel access policies for several consecutive time slots. To effectively reduce the complexity of the proposed RL algorithm, we adopt a branching dueling Q-network architecture and propose an efficient training methodology for producing proper Q-values over time-varying user sets. We perform extensive simulations on realistic traffic environments and demonstrate that the proposed online learning improves both throughput and fairness compared to the conventional RL approaches and centralized scheduling policies.

A multichannel random access system is considered in which each user accesses a single channel among multiple orthogonal channels to communicate with an access point (AP). Users arrive to the system at random and be activated for a certain period of time slots and then disappear from the system. Under such dynamic network environment, we propose a distributed multichannel access protocol based on multi-agent reinforcement learning (RL) to improve both throughput and fairness between users. Unlike the previous approaches adjusting channel access probabilities at each time slot, the proposed RL algorithm deterministically selects a set of channel access policies for several consecutive time slots. To effectively reduce the complexity of the proposed RL algorithm, we adopt a branching dueling Q-network architecture and propose a training methodology for producing proper Q-values under time-varying user sets. Numerical results demonstrate that the proposed scheme significantly improve both throughput and fairness.

In this paper, we consider a joint beam tracking and pattern optimization problem for massive multiple input multiple output (MIMO) systems in which the base station (BS) selects a beamforming codebook and performs adaptive beam tracking taking into account the user mobility. A joint adaptation scheme is developed in a two-phase reinforcement learning framework which utilizes practical signaling and feedback information. In particular, an inner agent adjusts the transmission beam index for a given beamforming codebook based on short-term instantaneous signal-to-noise ratio (SNR) rewards. In addition, an outer agent selects the beamforming codebook based on long-term SNR rewards. Simulation results demonstrate that the proposed online learning outperforms conventional codebook-based beamforming schemes using the same number of feedback information. It is further shown that joint beam tracking and beam pattern adaptation provides a significant SNR gain compared to the beam tracking only schemes, especially as the user mobility increases.