Text-Based Games (TBGs) have emerged as important testbeds for reinforcement learning (RL) in the natural language domain.Previous methods using LSTM-based action policies are uninterpretable and often overfit the training games showing poor performance to unseen test games.We present SymboLic Action policy for Textual Environments (SLATE), that learns interpretable action policy rules from symbolic abstractions of textual observations for improved generalization.We outline a method for end-to-end differentiable symbolic rule learning and show that such symbolic policies outperform previous stateof-the-art methods in text-based RL for the coin collector environment from 5 -10x fewer training games.Additionally, our method provides human-understandable policy rules that can be readily verified for their logical consistency and can be easily debugged.1

Daiki Kimura, Masaki Ono, Subhajit Chaudhury, Ryosuke Kohita, Akifumi Wachi, Don Joven Agravante, Michiaki Tatsubori, Asim Munawar, Alexander Gray. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. 2021.

This paper proposes a method for modeling event sequences with ambiguous timestamps, a time-discounting convolution. Unlike in ordinary time series, time intervals are not constant, small time-shifts have no significant effect, and inputting timestamps or time durations into a model is not effective. The criteria that we require for the modeling are providing robustness against time-shifts or timestamps uncertainty as well as maintaining the essential capabilities of time-series models, i.e., forgetting meaningless past information and handling infinite sequences. The proposed method handles them with a convolutional mechanism across time with specific parameterizations, which efficiently represents the event dependencies in a time-shift invariant manner while discounting the effect of past events, and a dynamic pooling mechanism, which provides robustness against the uncertainty in timestamps and enhances the time-discounting capability by dynamically changing the pooling window size. In our learning algorithm, the decaying and dynamic pooling mechanisms play critical roles in handling infinite and variable length sequences. Numerical experiments on real-world event sequences with ambiguous timestamps and ordinary time series demonstrated the advantages of our method.