In many reinforcement learning (RL) game tasks, an episode should be interrupted after a certain time, as the agent could sometimes fall into a deadlock state. The learning process is sensitive to the interruption length. So, it is hard to determine an optimal value of interruption. This article presents a novel multiarmed bandit (MAB) model to dynamically interrupt the deadlock state in reinforcement game learning, with the assumption that there are neither prior knowledge of optimal interruption setting, nor prior knowledge that can be used to improve the performance of the RL agent. The proposed MAB model is a nonoblivious adversarial MAB problem with a multisampling process in each round. An efficient algorithm named Exp3.P.MS is proposed for the new bandit setting, achieving an expected regret bound of$\mathcal {O}(\sqrt{nK\ln {(K)}})$. We run the algorithm onSokoban. The experimental results show that the dynamic interruptions can adapt to the weak-to-strong performance of the RL agent and spur a fast learning in game training.

We propose a framework for ensuring safe behavior of a reinforcement learning agent when the reward function may be difficult to specify. In order to do this, we rely on the existence of demonstrations from expert policies, and we provide a theoretical framework for the agent to optimize in the space of rewards consistent with its existing knowledge. We propose two methods to solve the resulting optimization: an exact ellipsoid-based method and a method in the spirit of the "follow-the-perturbed-leader" algorithm. Our experiments demonstrate the behavior of our algorithm in both discrete and continuous problems. The trained agent safely avoids states with potential negative effects while imitating the behavior of the expert in the other states.

We design simple mechanisms to approximate the Gains from Trade (GFT) in two-sided markets with multiple unit-supply sellers and multiple unit-demand buyers. A classical impossibility result by Myerson and Satterthwaite showed that even with only one seller and one buyer, no Bayesian Incentive Compatible (BIC), Individually Rational (IR), and Budget-Balanced (BB) mechanism can achieve full GFT (trade whenever buyer's value is higher than the seller's cost). The same paper also proposed the ``second-best'' mechanism that maximizes the GFT subject to BIC, IR, and BB constraints, which is unfortunately rather complex for even the single-seller single-buyer case. Our mechanism is simple, BIC, IR, and BB and achieves 1/2 of the optimal GFT among all BIC, IR, and BB mechanisms. The result holds for arbitrary distributions of the buyers' and sellers' values and can accommodate any downward-closed feasibility constraints over the allocations. The analysis of our mechanism is facilitated by extending the Cai-Weinberg-Devanur duality framework to two-sided markets.