Minimum Bayes Risk (MBR) decoding has seen renewed interest as an alternative to traditional generation strategies.While MBR has proven effective in machine translation, where the variability of a language model's outcome space is naturally constrained, it may face challenges in more open-ended tasks such as dialogue or instruction-following.We hypothesise that in such settings, applying MBR with standard similarity-based utility functions may result in selecting responses that are broadly representative of the model's distribution, yet sub-optimal with respect to any particular grouping of generations that share an underlying latent structure.In this work, we introduce three lightweight adaptations to the utility function, designed to make MBR more sensitive to structural variability in the outcome space.To test our hypothesis, we curate a dataset capturing three representative types of latent structuredialogue act, emotion, and response structure (e.g., a sentence, a paragraph, or a list)-and we propose two metrics to evaluate the structural optimality of MBR.Our analysis demonstrates that common similarity-based utility functions fall short by these metrics.In contrast, our proposed adaptations considerably improve structural optimality.Finally, we evaluate our approaches on real-world instruction-following benchmarks, AlpacaEval and MT-Bench, and show that increased structural sensitivity improves generation quality by up to 13.7 percentage points in win rate. 1

In NMT we search for the mode of the model distribution to form predictions.The mode and other high-probability translations found by beam search have been shown to often be inadequate in a number of ways.This prevents improving translation quality through better search, as these idiosyncratic translations end up selected by the decoding algorithm, a problem known as the beam search curse.Recently, an approximation to minimum Bayes risk (MBR) decoding has been proposed as an alternative decision rule that would likely not suffer from the same problems.We analyse this approximation and establish that it has no equivalent to the beam search curse.We then design approximations that decouple the cost of exploration from the cost of robust estimation of expected utility.This allows for much larger hypothesis spaces, which we show to be beneficial.We also show that mode-seeking strategies can aid in constructing compact sets of promising hypotheses and that MBR is effective in identifying good translations in them.We conduct experiments on three language pairs varying in amounts of resources available: English into and from German, Romanian, and Nepali. 1

Recent studies have revealed a number of pathologies of neural machine translation (NMT) systems.Hypotheses explaining these mostly suggest there is something fundamentally wrong with NMT as a model or its training algorithm, maximum likelihood estimation (MLE).Most of this evidence was gathered using maximum a posteriori (MAP) decoding, a decision rule aimed at identifying the highest-scoring translation, i.e. the mode.We argue that the evidence corroborates the inadequacy of MAP decoding more than casts doubt on the model and its training algorithm.In this work, we show that translation distributions do reproduce various statistics of the data well, but that beam search strays from such statistics.We show that some of the known pathologies and biases of NMT are due to MAP decoding and not to NMT's statistical assumptions nor MLE.In particular, we show that the most likely translations under the model accumulate so little probability mass that the mode can be considered essentially arbitrary.We therefore advocate for the use of decision rules that take into account the translation distribution holistically.We show that an approximation to minimum Bayes risk decoding gives competitive results confirming that NMT models do capture important aspects of translation well in expectation.