"Randomized experiments are increasingly central to innovation in many fields. In the tech sector, major platforms run thousands of experiments (called A/B tests) each year on tens of millions of users at any given time and use the results to screen most product innovations. In the policy and academic circles, governments, nonprofit organizations, and academics use randomized control trials to evaluate social programs and shape public policy. Experiments are not only prevalent, but also highly heterogeneous in design. Policy makers and tech giants typically focus on a "go big" approach, obtaining large sample sizes for a small number of experiments to ensure they that can detect even small benefits of a policy intervention. In contrast, many start-ups and entrepreneurs take a different "go lean" approach, running many small tests and discarding any innovation without outstanding success. The idea is to quickly and cheaply experiment with many ideas, abandon or pivot from ideas that do not work, and scale up ideas that do work. In this paper, we study when each of these approaches is appropriate. To do so, we propose a new framework for optimal experimentation that we call the A/B testing problem. The frameworks also yields an optimal strategy of what innovations to implement and methods to calculate the value of data and experimentation. The key insight is that the optimal experimentation strategy depends crucially on the tails of the distribution of innovation quality, and whether these tails have "black swan" outliers, of innovations with a very large positive or negative impact. The A/B testing problem is as follows. A firm has a set of potential innovations i=1,-,I to implement. The quality ..._i of innovation i is unknown and comes from a distribution G. Quality is independently distributed across innovations. The firm selects a number of users n_i to allocate to an A/B test evaluating innovation i. This yields a signal with mean equal to the true quality of idea i and variance a^2/n_i. The firm is subject to the constraint that the total number of users assigned to experiments is no greater than the number N of users available for experimentation. After seeing the realization of the signals, the firm selects a subset S of ideas to implement. The firm's objective is to maximize the expected sum of the true quality of the ideas that are implemented.

Ride-hailing applications (apps) like Uber and Lyft introduced a matching technology and market design that recent research has found is more efficient than traditional taxi systems [2]. However, unlike traditional street-hailing taxi systems, they are prone to a failure mode first anticipated by [1]. In this paper we model and empirically establish the existence of these dynamics. We then show how surge pricing and, to a lesser extent, other market design interventions can prevent this problem from crippling a ride-hailing market.

Investigating potential purchases, such as a start-up company to acquire, is often a substantial investment under uncertainty. Standard market designs, such as simultaneous or ascending price auctions, compound this with additional uncertainty about the eventual price a bidder will have to pay in order to win. As a result they tend to confuse the process of search by leading to both wasteful information acquisition on goods that have already found a good purchaser and discouraging needed investigations of objects, potentially eliminating all gains from trade. Fully efficient procedures that avoid these problems, such as dynamic Vickrey-Clarke-Groves processes, are extremely complex and fragile. By contrast, we show that the Dutch auction preserves all of its properties from a standard setting without information costs because it guarantees, at the time of information acquisition, a price at which the good can be purchased.

When no agent has substantial influence on a public good outcome, he or she can demand the full surplus. Thus, in rich environments, private (voluntary and self-financing) provision of public goods---or bads such as land assembly---to a large number of self-interested citizens is impossible. This holdout problem is well-known and ubiquitous throughout economics: Holdout was first formalized by Cournot [2], and takes its precise modern form in the work of Mailath and Postelwaite [4]. Unlike in classical auction settings, where increasing competition may offset imperfections in market design ([1,3]), there is no easy way around the necessity of social engineering to solve holdout. Consequently, holdout concerns have informed wide-ranging policy decisions, including eminent domain and corporate takeover laws. In this paper, we study holdout in settings where a good owned by a disparate community of sellers is desired by a buyer only in its entirety; for concreteness, we focus on the particularly salient application of land assembly. In these settings, no mechanism can simultaneously achieve full efficiency and complete individual rationality ([4]). However, as we show, it is possible to strike an attractive balance between these two goals.