Touchscreen keyboards rely on language modeling to auto-correct noisy typing and to offer word predictions. While language models can be pre-trained on huge amounts of text, they may fail to capture a user's unique writing style. Using a recently released email personalization dataset, we show improved performance compared to a unigram cache by adapting to a user's text via language models based on prediction by partial match (PPM) and recurrent neural networks. On simulated noisy touchscreen typing of 44 users, our best model increased keystroke savings by 9.9% relative and reduced word error rate by 36% relative compared to a static background language model.

Jiban Adhikary, Jamie Berger, Keith Vertanen. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.

Entering text in virtual environments can be challenging, especially without auxiliary input devices. We investigate text input in virtual reality using hand-tracking and speech. Our system visualizes users' hands in the virtual environment, allowing typing on an auto-correcting midair keyboard. It also supports speaking a sentence and then correcting errors by selecting alternative words proposed by a speech recognizer. We conducted a user study in which participants wrote sentences with and without speech. Using only the keyboard, users wrote at 11 words-per-minute at a 1.2% error rate. Speaking and correcting sentences was faster and more accurate at 28 words-per-minute and a 0.5% error rate. Participants achieved this performance despite half of sentences containing an uncommon out-of-vocabulary word (e.g. proper name). For sentences with only in-vocabulary words, performance using speech and midair keyboard corrections was faster at 36 words-per-minute with a low 0.3% error rate.