We present an algorithm for obtaining ϵ-minimax solutions of statistical decision problems. We are interested in problems where i) the statistician is allowed to choose randomly among I decision rules, and ii) the statistical model may have an infinite dimensional parameter space. The minimax solution of these problems admits a convex programming representation over the (I - 1)-simplex, and the algorithm suggested herein to obtain an ϵ-approximation of the minimax solution is a version of mirror subgradient descent, initialized with uniform weights and stopped after a finite number of iterations. The resulting iterative procedure is known in the computer science literature as the hedge algorithm (a particular case of the multiplicative weights update method) and it is used in algorithmic game theory as a practical tool to find approximate solutions of two-person zero-sum games. We apply the suggested algorithm to different minimax problems in the econometrics literature. An empirical application to the problem of optimally selecting sites to maximize the external validity of an experimental policy evaluation illustrates the usefulness of the suggested procedure. The full paper can be found at https://joseluismontielolea.com/epsilon_minimax_v2.pdf.

We present a decision-theoretic justification for viewing the question of how to best choose where to experiment in order to optimize external validity as a k-median (clustering) problem, a popular problem in computer science and operations research. We present conditions under which minimizing the worst-case, welfare-based regret among all nonrandom schemes that select k sites to experiment is approximately equal—and sometimes exactly equal—to finding the k most central vectors of baseline site-level covariates. The k-median problem can be formulated as a linear integer program. Two empirical applications illustrate the theoretical and computational benefits of the suggested procedure.

This paper shows that dropout training in generalized linear models is the minimax solution of a two-player, zero-sum game where an adversarial nature corrupts a statistician's covariates using a multiplicative nonparametric errors-in-variables model. In this game, nature's least favorable distribution is dropout noise, where nature independently deletes entries of the covariate vector with some fixed probability $\delta$. This result implies that dropout training indeed provides out-of-sample expected loss guarantees for distributions that arise from multiplicative perturbations of in-sample data. The paper makes a concrete recommendation on how to select the tuning parameter $\delta$. The paper also provides a novel, parallelizable, unbiased multi-level Monte Carlo algorithm to speed-up the implementation of dropout training. Our algorithm has a much smaller computational cost compared to the naive implementation of dropout, provided the number of data points is much smaller than the dimension of the covariate vector.

$\alpha$-posteriors and their variational approximations distort standard posterior inference by downweighting the likelihood and introducing variational approximation errors. We show that such distortions, if tuned appropriately, reduce the Kullback--Leibler (KL) divergence from the true, but perhaps infeasible, posterior distribution when there is potential parametric model misspecification. To make this point, we derive a Bernstein--von Mises theorem showing convergence in total variation distance of $\alpha$-posteriors and their variational approximations to limiting Gaussian distributions. We use these limiting distributions to evaluate the KL divergence between true and reported posteriors. We show that the KL divergence is minimized by choosing $\alpha$ strictly smaller than one, assuming there is a vanishingly small probability of model misspecification. The optimized value of $\alpha$ becomes smaller as the misspecification becomes more severe. The optimized KL divergence increases logarithmically in the magnitude of misspecification and not linearly as with the usual posterior. Moreover, the optimized variational approximations of $\alpha$-posteriors can induce additional robustness to model misspecification beyond that obtained by optimally downweighting the likelihood.

"Randomized experiments are increasingly central to innovation in many fields. In the tech sector, major platforms run thousands of experiments (called A/B tests) each year on tens of millions of users at any given time and use the results to screen most product innovations. In the policy and academic circles, governments, nonprofit organizations, and academics use randomized control trials to evaluate social programs and shape public policy. Experiments are not only prevalent, but also highly heterogeneous in design. Policy makers and tech giants typically focus on a "go big" approach, obtaining large sample sizes for a small number of experiments to ensure they that can detect even small benefits of a policy intervention. In contrast, many start-ups and entrepreneurs take a different "go lean" approach, running many small tests and discarding any innovation without outstanding success. The idea is to quickly and cheaply experiment with many ideas, abandon or pivot from ideas that do not work, and scale up ideas that do work. In this paper, we study when each of these approaches is appropriate. To do so, we propose a new framework for optimal experimentation that we call the A/B testing problem. The frameworks also yields an optimal strategy of what innovations to implement and methods to calculate the value of data and experimentation. The key insight is that the optimal experimentation strategy depends crucially on the tails of the distribution of innovation quality, and whether these tails have "black swan" outliers, of innovations with a very large positive or negative impact. The A/B testing problem is as follows. A firm has a set of potential innovations i=1,-,I to implement. The quality ..._i of innovation i is unknown and comes from a distribution G. Quality is independently distributed across innovations. The firm selects a number of users n_i to allocate to an A/B test evaluating innovation i. This yields a signal with mean equal to the true quality of idea i and variance a^2/n_i. The firm is subject to the constraint that the total number of users assigned to experiments is no greater than the number N of users available for experimentation. After seeing the realization of the signals, the firm selects a subset S of ideas to implement. The firm's objective is to maximize the expected sum of the true quality of the ideas that are implemented.