Despite their ubiquity in language generation, it remains unknown why truncation sampling heuristics like nucleus sampling are so effective. We provide a theoretical explanation for the effectiveness of the truncation sampling by proving that truncation methods that discard tokens below some probability threshold (the most common type of truncation) can guarantee that all sampled tokens have nonzero true probability. However, thresholds are a coarse heuristic, and necessarily discard some tokens with nonzero true probability as well. In pursuit of a more precise sampling strategy, we show that we can leverage a known source of model errors, the softmax bottleneck, to prove that certain tokens have nonzero true probability, without relying on a threshold. Based on our findings, we develop an experimental truncation strategy and the present pilot studies demonstrating the promise of this type of algorithm. Our evaluations show that our method outperforms its threshold-based counterparts under automatic and human evaluation metrics for low-entropy (i.e., close to greedy) open-ended text generation. Our theoretical findings and pilot experiments provide both insight into why truncation sampling works, and make progress toward more expressive sampling algorithms that better surface the generative capabilities of large language models.

Abstract While recent language models have the ability to take long contexts as input, relatively little is known about how well they use longer context. We analyze the performance of language models on two tasks that require identifying relevant information in their input contexts: multi-document question answering and key-value retrieval. We find that performance can degrade significantly when changing the position of relevant information, indicating that current language models do not robustly make use of information in long input contexts. In particular, we observe that performance is often highest when relevant information occurs at the beginning or end of the input context, and significantly degrades when models must access relevant information in the middle of long contexts, even for explicitly long-context models. Our analysis provides a better understanding of how language models use their input context and provides new evaluation protocols for future long-context language models.

We present Backpacks: a new neural architecture that marries strong modeling performance with an interface for interpretability and control.Backpacks learn multiple non-contextual sense vectors for each word in a vocabulary, and represent a word in a sequence as a contextdependent, non-negative linear combination of sense vectors in this sequence.We find that, after training, sense vectors specialize, each encoding a different aspect of a word.We can interpret a sense vector by inspecting its (non-contextual, linear) projection onto the output space, and intervene on these interpretable hooks to change the model's behavior in predictable ways.We train a 170M-parameter Backpack language model on OpenWebText, matching the loss of a GPT-2 small (124Mparameter) Transformer.On lexical similarity evaluations, we find that Backpack sense vectors outperform even a 6B-parameter Transformer LM's word embeddings.Finally, we present simple algorithms that intervene on sense vectors to perform controllable text generation and debiasing.For example, we can edit the sense vocabulary to tend more towards a topic, or localize a source of gender bias to a sense vector and globally suppress that sense.

Probing experiments investigate the extent to which neural representations make properties-like part-of-speech-predictable.One suggests that a representation encodes a property if probing that representation produces higher accuracy than probing a baseline representation like non-contextual word embeddings.Instead of using baselines as a point of comparison, we're interested in measuring information that is contained in the representation but not in the baseline.For example, current methods can detect when a representation is more useful than the word identity (a baseline) for predicting part-ofspeech; however, they cannot detect when the representation is predictive of just the aspects of part-of-speech not explainable by the word identity.In this work, we extend a theory of usable information called V-information and propose conditional probing, which explicitly conditions on the information in the baseline.In a case study, we find that after conditioning on non-contextual word embeddings, properties like part-of-speech are accessible at deeper layers of a network than previously thought.

Benjamin Newman, Kai-Siang Ang, Julia Gong, John Hewitt. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2021.

Recent work has found evidence that Multilingual BERT (mBERT), a transformer-based multilingual masked language model, is capable of zero-shot cross-lingual transfer, suggesting that some aspects of its representations are shared cross-lingually.To better understand this overlap, we extend recent work on finding syntactic trees in neural networks' internal representations to the multilingual setting.We show that subspaces of mBERT representations recover syntactic tree distances in languages other than English, and that these subspaces are approximately shared across languages.Motivated by these results, we present an unsupervised analysis method that provides evidence mBERT learns representations of syntactic dependency labels, in the form of clusters which largely agree with the Universal Dependencies taxonomy.This evidence suggests that even without explicit supervision, multilingual masked language models learn certain linguistic universals.

Recurrent neural networks empirically generate natural language with high syntactic fidelity.However, their success is not wellunderstood theoretically.We provide theoretical insight into this success, proving in a finiteprecision setting that RNNs can efficiently generate bounded hierarchical languages that reflect the scaffolding of natural language syntax.We introduce Dyck-(k,m), the language of well-nested brackets (of k types) and mbounded nesting depth, reflecting the bounded memory needs and long-distance dependencies of natural language syntax.The best known results use O(k m 2 ) memory (hidden units) to generate these languages.We prove that an RNN with O(m log k) hidden units suffices, an exponential reduction in memory, by an explicit construction.Finally, we show that no algorithm, even with unbounded computation, can suffice with o(m log k) hidden units.

John Hewitt, Percy Liang. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019.

With the advent of conversational assistants like Amazon Alexa, Google Now, etc., dialogue systems are gaining a lot of traction, especially in industrial settings.These systems typically include a Spoken Language understanding component which consists of two tasks: Intent Classification (IC) and Slot Labeling (SL).Generally, these two tasks are modeled together jointly to achieve best performance.However, this joint modeling adds to model obfuscation.In this work, we first design framework for a modularization of joint IC-SL task to enhance architecture transparency.Then, we explore a number of selfattention, convolutional, and recurrent models, contributing a large-scale analysis of modeling paradigms for IC+SL across two datasets.Finally, using this framework, we propose a class of 'label-recurrent' models that are nonrecurrent apart from a 10-dimensional representation of the label history, and show that our proposed systems are highly accurate (achieving over 30% error reduction in SL over the state-of-the-art on the Snips dataset), as well as fast, at 2x the inference and 2/3 to 1/2 the training time of comparable recurrent models, thus giving an edge in critical real-world systems.

John Hewitt, Daphne Ippolito, Brendan Callahan, Reno Kriz, Derry Tanti Wijaya, Chris Callison-Burch. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2018.

Modeling derivational morphology to generate words with particular semantics is useful in many text generation tasks, such as machine translation or abstractive question answering.In this work, we tackle the task of derived word generation.That is, given the word "run," we attempt to generate the word "runner" for "someone who runs."We identify two key problems in generating derived words from root words and transformations: suffix ambiguity and orthographic irregularity.We contribute a novel aggregation model of derived word generation that learns derivational transformations both as orthographic functions using sequence-to-sequence models and as functions in distributional word embedding space.Our best open-vocabulary model, which can generate novel words, and our best closed-vocabulary model, show 22% and 37% relative error reductions over current state-of-the-art systems on the same dataset.

Derry Tanti Wijaya, Brendan Callahan, John Hewitt, Jie Gao, Xiao Ling, Marianna Apidianaki, Chris Callison-Burch. Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. 2017.