Learning strategies for imperfect information games from samples of interaction is a challenging problem. A common method for this setting, Monte Carlo Counterfactual Regret Minimization (MCCFR), can have slow long-term convergence rates due to high variance. In this paper, we introduce a variance reduction technique (VR-MCCFR) that applies to any sampling variant of MCCFR. Using this technique, periteration estimated values and updates are reformulated as a function of sampled values and state-action baselines, similar to their use in policy gradient reinforcement learning. The new formulation allows estimates to be bootstrapped from other estimates within the same episode, propagating the benefits of baselines along the sampled trajectory; the estimates remain unbiased even when bootstrapping from other estimates. Finally, we show that given a perfect baseline, the variance of the value estimates can be reduced to zero. Experimental evaluation shows that VR-MCCFR brings an order of magnitude speedup, while the empirical variance decreases by three orders of magnitude. The decreased variance allows for the first time CFR+ to be used with sampling, increasing the speedup to two orders of magnitude.

We point out important problems with the common practice of using the best single model performance for comparing deep learning architectures, and we propose a method that corrects these flaws. Each time a model is trained, one gets a different result due to random factors in the training process, which include random parameter initialization and random data shuffling. Reporting the best single model performance does not appropriately address this stochasticity. We propose a normalized expected best-out-of-$n$ performance ($\text{Boo}_n$) as a way to correct these problems.

Several large cloze-style context-questionanswer datasets have been introduced recently: the CNN and Daily Mail news data and the Children's Book Test.Thanks to the size of these datasets, the associated text comprehension task is well suited for deep-learning techniques that currently seem to outperform all alternative approaches.We present a new, simple model that uses attention to directly pick the answer from the context as opposed to computing the answer using a blended representation of words in the document as is usual in similar models.This makes the model particularly suitable for questionanswering problems where the answer is a single word from the document.Ensemble of our models sets new state of the art on all evaluated datasets.

This paper presents two discriminative knowledgebased dialog state trackers and their results on the Dialog State Tracking Challenge (DSTC) 2 and 3 datasets. The first tracker was submitted to the DSTC3 competition and scored second in the joint accuracy. The second tracker developed after the DSTC3 submission deadline gives even better results on the DSTC2 and DSTC3 datasets. It performs on par with the state of the art machine learning-based trackers while offering better interpretability. We summarize recent directions in the dialog state tracking (DST) and also discuss possible decomposition of the DST problem. Based on the results of DSTC2 and DSTC3 we analyze suitability of different techniques for each of the DST subproblems. Results of the trackers highlight the importance of Spoken Language Understanding (SLU) for the last two DSTCs.