While much work on deep latent variable models of text uses continuous latent variables, discrete latent variables are interesting because they are more interpretable and typically more space efficient.We consider several approaches to learning discrete latent variable models for text in the case where exact marginalization over these variables is intractable.We compare the performance of the learned representations as features for lowresource document and sentence classification.Our best models outperform the previous best reported results with continuous representations in these low-resource settings, while learning significantly more compressed representations.Interestingly, we find that an amortized variant of Hard EM performs particularly well in the lowest-resource regimes. 1

Alex Wang, Jan Hula, Patrick Xia, Raghavendra Pappagari, R. Thomas McCoy, Roma Patel, Najoung Kim, Ian Tenney, Yinghui Huang, Katherin Yu, Shuning Jin, Berlin Chen, Benjamin Van Durme, Edouard Grave, Ellie Pavlick, Samuel R. Bowman. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2019.