As network architecture becomes complex and the user requirement gets diverse, the role of efficient network resource management becomes more important. However, existing throughput-optimal scheduling algorithms such as the max-weight algorithm suffer from poor delay performance. In this paper, we present reinforcement learning-based network scheduling algorithms for a single-hop downlink scenario which achieve throughput-optimality and converge to minimal delay. To this end, we first formulate the network optimization problem as a Markov decision process (MDP) problem. Then, we introduce a new state-action value function called Q+-function and develop a reinforcement learning algorithm called Q+-learning with UCB (Upper Confidence Bound) exploration which guarantees small performance loss during a learning process. We also derive an upper bound of the sample complexity in our algorithm, which is more efficient than the best known bound from Q-learning with UCB exploration by a factor of γ2where γ is the discount factor of the MDP problem. Finally, via simulation, we verify that our algorithm shows a delay reduction of up to 40.8% compared to the max-weight algorithm over various scenarios. We also show that the Q+-learning with UCB exploration converges to an ε-optimal policy 10 times faster than Q-learning with UCB.

As network architecture becomes complex and the user requirement gets diverse, the role of efficient network resource management becomes more important. However, existing network scheduling algorithms such as the max-weight algorithm suffer from poor delay performance. In this paper, we present a reinforcement learning-based network scheduling algorithm that achieves both optimal throughput and low delay. To this end, we first formulate the network optimization problem as an MDP problem. Then we introduce a new state-action value function called W-function and develop a reinforcement learning algorithm called W-Learning that guarantees little performance loss during a learning process. Finally, via simulation, we verify that our algorithm shows delay reduction of up to 40.8% compared to the max-weight algorithm over various scenarios.