Jeffrey Lund, Piper Armstrong, Wilson Fearn, Stephen Cowley, Courtni Byun, Jordan Boyd-Graber, Kevin Seppi. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2019.

We propose Labeled Anchors, an interactive and supervised topic model based on the anchor words algorithm (Arora et al., 2013).Labeled Anchors is similar to Supervised Anchors (Nguyen et al., 2014) in that it extends the vector-space representation of words to include document labels.However, our formulation also admits a classifier which requires no training beyond inferring topics, which means our approach is also fast enough to be interactive.We run a small user study that demonstrates that untrained users can interactively update topics in order to improve classification accuracy.

Interactive topic models are powerful tools for understanding large collections of text.However, existing sampling-based interactive topic modeling approaches scale poorly to large data sets.Anchor methods, which use a single word to uniquely identify a topic, offer the speed needed for interactive work but lack both a mechanism to inject prior knowledge and lack the intuitive semantics needed for userfacing applications.We propose combinations of words as anchors, going beyond existing single word anchor algorithmsan approach we call "Tandem Anchors".We begin with a synthetic investigation of this approach then apply the approach to interactive topic modeling in a user study and compare it to interactive and noninteractive approaches.Tandem anchors are faster and more intuitive than existing interactive approaches.

Online topic modeling, i.e., topic modeling with stochastic variational inference, is a powerful and efficient technique for analyzing large datasets, and ADAGRAD is a widely-used technique for tuning learning rates during online gradient optimization.However, these two techniques do not work well together.We show that this is because ADAGRAD uses accumulation of previous gradients as the learning rates' denominators.For online topic modeling, the magnitude of gradients is very large.It causes learning rates to shrink very quickly, so the parameters cannot fully converge until the training ends.

Probabilistic models are a useful means for analyzing large text corpora. Integrating such models with human interaction enables many new use cases. However, adding human interaction to probabilistic models requires inference algorithms which are both fast and accurate. We explore the use of Iterated Conditional Modes as a fast alternative to Gibbs sampling or variational EM. We demonstrate superior performance both in run time and model quality on three different models of text including a DP Mixture of Multinomials for web search result clustering, the Interactive Topic Model, and M OM R ESP , a multinomial crowdsourcing model.

Thang Nguyen, Jordan Boyd-Graber, Jeffrey Lund, Kevin Seppi, Eric Ringger. Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2015.