When an experimenter has the option of running an adaptive trial, is it admissible to ignore this option and run a non-adaptive trial instead? We provide a negative answer to this question in the best-arm identification problem, where the experimenter aims to allocate measurement efforts judiciously to confidently deploy the most effective treatment arm. We find that, whenever there are at least three treatment arms, there exist simple adaptive designs that universally and strictly dominate non-adaptive completely randomized trials. This dominance is characterized by a notion called efficiency exponent, which quantifies a design's statistical efficiency when the experimental sample is large. Our analysis focuses on the class of batched arm elimination designs, which progressively eliminate underperforming arms at pre-specified batch intervals. We characterize simple sufficient conditions under which these designs universally and strictly dominate completely randomized trials. These results resolve the second open problem posed in Qin [2022]. The full version of this paper is available at https://arxiv.org/pdf/2506.05329.

The time at which renewable energy resources (e.g., solar or wind) produce electricity cannot generally be controlled. However, in many settings, consumers have some flexibility in their energy consumption, and there is growing interest in demand-response programs that leverage this flexibility to shift energy consumption to better match renewable production—thus enabling more efficient utilization of these resources. We study demand response in a setting where consumers use home energy management systems (HEMS) to autonomously adjust their electricity consumption. Our core assumption is that HEMS operationalize flexibility by querying consumers for their preferences and computing the "indifference set"—the set of all energy consumption profiles that can satisfy these preferences. Given an indifference set, HEMS can then respond to grid signals while ensuring user-defined comfort and functionality. For example, if a consumer sets a temperature range, a HEMS can precool and preheat to align with peak renewable production, thus improving efficiency without sacrificing comfort.

Randomized controlled trials are often run in settings with many subpopulations that may have differential benefits from the treatment being evaluated. We consider the problem of sample selection, i.e., whom to enroll in a randomized trial, such as to optimize welfare in a heterogeneous population. We formalize this problem within the minimax-regret framework, and derive optimal sample-selection schemes under a variety of conditions. Using data from a COVID-19 vaccine trial, we also highlight how different objectives and decision rules can lead to meaningfully different guidance regarding optimal sample allocation.

We study randomized experiments in a service system when stochastic congestion can arise from temporarily limited supply or excess demand. Such congestion gives rise to cross-unit interference between the waiting customers, and analytic strategies that do not account for this interference may be biased. In current practice, one of the most widely used ways to address stochastic congestion is to use switchback experiments that alternatively turn a target intervention on and off for the whole system. We find, however, that under a queueing model for stochastic congestion, the standard way of analyzing switchbacks is inefficient, and that estimators that leverage the queueing model can be materially more accurate. We also consider a new experimental design, which we refer to as the length-0 switchback, that can be used to estimate a policy gradient of the dynamic system using only unit-level randomization. This design avoids needing to pre-commit to a switchback length before data collection, and can thus be easier to deploy in settings with nonstationarity.

We investigate properties of Thompson Sampling in the stochastic multi-armed bandit problem with delayed feedback. In a setting with i.i.d delays, we establish to our knowledge the first regret bounds for Thompson Sampling with arbitrary delay distributions, including ones with unbounded expectation. Our bounds are qualitatively comparable to the best available bounds derived via ad-hoc algorithms, and only depend on delays via selected quantiles of the delay distributions. Furthermore, in extensive simulation experiments, we find that Thompson Sampling outperforms a number of alternative proposals, including methods specifically designed for settings with delayed feedback.

We consider the problem of deciding how best to target and prioritize existing vaccines that may offer protection against new variants of an infectious disease. Sequential experiments are a promising approach; however, challenges due to delayed feedback and the overall ebb and flow of disease prevalence make available methods inapplicable for this task. We present a method, partial likelihood Thompson sampling, that can handle these challenges. Our method involves running Thompson sampling with belief updates determined by partial likelihood each time we observe an event. To test our approach, we ran a semi-synthetic experiment based on 200 days of COVID-19 infection data in the US.

We study methods for simultaneous analysis of many noisy experiments in the presence of rich covariate information. The goal of the analyst is to optimally estimate the true effect underlying each experiment. Both the noisy experimental results and the auxiliary covariates are useful for this purpose, but neither data source on its own captures all the information available to the analyst. In this paper, we propose a flexible plug-in empirical Bayes estimator that synthesizes both sources of information and may leverage any black-box predictive model. We show that our approach is within a constant factor of minimax for a simple data-generating model. Furthermore, we establish robust convergence guarantees for our method that hold under considerable generality, and exhibit promising empirical performance on both real and simulated data.

If a concept class can be represented with a certain amount of memory, can it be efficiently learned with the same amount of memory? What concepts can be efficiently learned by algorithms that extract only a few bits of information from each example? We introduce a formal framework for studying these questions, and investigate the relationship between the fundamental resources of memory or communication and the sample complexity of the learning task. We relate our memory-bounded and communication-bounded learning models to the well-studied statistical query model. This connection can be leveraged to obtain both upper and lower bounds: we show strong lower bounds on learning parity functions with bounded communication, as well as upper bounds on solving sparse linear regression problems with limited memory.

We develop a flexible framework for low-rank matrix estimation that allows us to transform noise models into regularization schemes via a simple bootstrap algorithm. Effectively, our procedure seeks an autoencoding basis for the observed matrix that is stable with respect to the specified noise model; we call the resulting procedure a stable autoencoder. In the simplest case, with an isotropic noise model, our method is equivalent to a classical singular value shrinkage estimator. For non-isotropic noise models---e.g., Poisson noise---the method does not reduce to singular value shrinkage, and instead yields new estimators that perform well in experiments. Moreover, by iterating our stable autoencoding scheme, we can automatically generate low-rank estimates without specifying the target rank as a tuning parameter.

NLP models have many and sparse features, and regularization is key for balancing model overfitting versus underfitting.A recently repopularized form of regularization is to generate fake training data by repeatedly adding noise to real data.We reinterpret this noising as an explicit regularizer, and approximate it with a second-order formula that can be used during training without actually generating fake data.We show how to apply this method to structured prediction using multinomial logistic regression and linear-chain CRFs.We tackle the key challenge of developing a dynamic program to compute the gradient of the regularizer efficiently.The regularizer is a sum over inputs, so we can estimate it more accurately via a semi-supervised or transductive extension.Applied to text classification and NER, our method provides a >1% absolute performance gain over use of standard L 2 regularization.

Dropout and other feature noising schemes control overfitting by artificially corrupting the training data. For generalized linear models, dropout performs a form of adaptive regularization. Using this viewpoint, we show that the dropout regularizer is first-order equivalent to an $\LII$ regularizer applied after scaling the features by an estimate of the inverse diagonal Fisher information matrix. We also establish a connection to AdaGrad, an online learner, and find that a close relative of AdaGrad operates by repeatedly solving linear dropout-regularized problems. By casting dropout as regularization, we develop a natural semi-supervised algorithm that uses unlabeled data to create a better adaptive regularizer. We apply this idea to document classification tasks, and show that it consistently boosts the performance of dropout training, improving on state-of-the-art results on the IMDB reviews dataset.

Heterogeneous network 3GPP LTE deployments are considered as a prime candidate to meet increasing demand for mobile broadband service coverage and capacity. Such deployments also need to support mobility that is as robust as in traditional macro deployments. In this paper, we analyze the handover performance in heterogeneous deployments to better understand potential behavior of automatic handover parameter adjustments. Such a Self-Organizing Network (SON) feature is commonly known as Mobility Robustness Optimization (MRO). The analyses indicate difficult challenges with handovers from low power to high power base stations, and an evolved handover procedure that improves the user experience during such times is presented. These results are key for developing an MRO algorithm for heterogeneous networks.

In this paper we present the main concepts of high speed packet access evolution currently being standardized in 3GPP. In general HSPA evolution consists of introduction of MIMO, higher order modulation, and protocol optimizations and optimizations for voice over IP. We describe these improvements in detail and show that HSPA Evolution can reach performance comparable to those of long term evolution of UMTS terrestrial radio access network in a 5 MHz deployment.

In this paper we evaluate the performance of the serving HS-DSCH cell change procedure for delay sensitive services, like voice over IP (VoIP). The impact of the signaling procedure delay on the user plane performance is studied by means of dynamic system simulations. The simulation results show that within the studied range, the duration of the signaling procedure starts to impact performance only for terminals moving 50 km/h, or faster. Parameter tuning and combining the procedures for active set update and change of best HS cell can considerably improve the performance, so that even at 120 km/h, less than 2% of the handovers suffer interruptions longer than 100 ms. Thus, the HSDPA handover procedure in Rel-6 can be made fast enough to accommodate VoIP users traveling at 120 km/h with acceptable conversation quality.

For data transmission WCDMA offers dedicated transport channels (DCH) and common transport channels, i.e. random access channels (RACH) and forward access channels (FACH). Both types of transport channels have different characteristics in terms of transmission performance and radio resource usage. In this paper these characteristics are described and a simulation model for the performance evaluation is presented. The effect of some parameters influencing the performance and efficiency of the different transport channel types is discussed, e.g. channel configuration, traffic load and traffic characteristics. It is demonstrated that common transport channels provide better performance than dedicated channels for a low traffic load of bursty traffic. For low data volumes they are also more resource efficient. However for an increasing traffic load dedicated channels provide a better service quality and radio resource utilisation.