Optimizing recommender systems for objectives beyond accuracy, such as diversity, novelty, and personalization, is crucial for long-term user satisfaction. To this end, industrial practitioners have accumulated vast amounts of structured domain knowledge, which we term human priors (e.g., item taxonomies, temporal patterns). This knowledge is typically applied through post-hoc adjustments during ranking or post-ranking. However, this approach remains decoupled from the core model learning, which is particularly undesirable as the industry shifts to end-to-end generative recommendation foundation models. On the other hand, many methods targeting these beyond-accuracy objectives often require architecture-specific modifications and discard these valuable human priors by learning user intent in a fully unsupervised manner. Instead of discarding the human priors accumulated over years of practice, we introduce a backbone-agnostic framework that seamlessly integrates these human priors directly into the end-to-end training of generative recommenders. With lightweight, prior-conditioned adapter heads inspired by efficient LLM decoding strategies, our approach guides the model to disentangle user intent along human-understandable axes (e.g., interaction types, long- vs. short-term interests). We also introduce a hierarchical composition strategy for modeling complex interactions across different prior types. Extensive experiments on three large-scale datasets demonstrate that our method significantly enhances both accuracy and beyond-accuracy objectives. We also show that human priors allow the backbone model to more effectively leverage longer context lengths and larger model sizes.

The 3rd Workshop on Causal Inference and Machine Learning in Practice at KDD 2025 aims to bring together researchers, industry professionals, and practitioners to explore the application of causal inference within machine learning models. As causal machine learning techniques gain traction across industries, practical challenges related to trustworthiness, robustness, and fairness remain at the forefront. This workshop will provide a forum to discuss methodologies for evaluating causal models in real-world scenarios and explore innovative applications that integrate causal inference with generative AI (GenAI) and large language models (LLMs). Topics of interest include using GenAI and LLMs to facilitate causal inference tasks and leveraging causal inference techniques for evaluating and improving GenAI/LLM models. Building on the success of the previous workshop editions at KDD 2023 and KDD 2024, which attracted over 200 and 250 participants, respectively, this workshop will continue fostering collaboration between academia and industry. Through invited talks, contributed papers, and interactive discussions, we will address key challenges and opportunities at the intersection of causal inference and machine learning. As the field continues to evolve, this workshop serves as a crucial platform for knowledge exchange and innovation, driving forward the application of causal techniques in machine learning and AI.

When large online platforms test a new strategy to implement with their user traffic, the phenomenon of diminishing marginal effects may arise. For example, when a strategy is implemented on 100% of the user traffic, the expected per-user effect can be lower compared to the expected per-user effect when a strategy is implemented on 10% of the user traffic, potentially due to limits of overall budget, resource, attention or content involved with that strategy. This diminishing marginal effect phenomenon brings an additional delicacy to online sequential experiments. In particular, for the classical goal of achieving the largest expected reward, the optimal decision may no longer be assigning 100% traffic to one strategy, but instead a mixture of strategies. We deliver two tasks for online sequential experiments in presence of diminishing marginal effect: (1) Adaptively identify the optimal traffic allocation to maximize the expected cumulative reward and (2) Construct valid central limit theorem (which is critically needed for A/B tests in online platforms) to perform reliable statistical inference for the expected reward under the optimal traffic allocation that is a priori unknown. We show that classical algorithms can fail to deliver the second task, especially because the statistical inference task presents its own difficulty. We develop a new online algorithm that leverages an additional smoothness condition on how the marginal effects change to achieve both tasks. We prove that this algorithm obtains the best achievable expected cumulative reward. Further, crucially for online platforms' need to do trustworthy statistical inference, the algorithm is proved to enjoy a valid central limit theorem. The theoretical findings are illustrated through numerical experiments.

There has been increasing interest in applying multi-armed bandits to adaptive designs in clinical trials. However, most literature assumes that a previous patient’s survival response of a treatment is known before the next patient is treated, which is unrealistic. The inability to account for response delays is cited frequently as one of the problems in using adaptive designs in clinical trials. More critically, the “delays” in observing the survival response are the same as the rewards rather than being external stochastic noise. We formalize this problem as a novel stochastic multi-armed bandit (MAB) problem with reward-dependent delays, where the delay at each round depends on the reward generated on the same round. For general reward/delay distributions with finite expectation, our proposed censored-UCB algorithm achieves near-optimal regret in terms of both problem-dependent and problem-independent bounds. With bounded or sub-Gaussian reward distributions, the upper bounds are optimal with a matching lower bound. Our theoretical results and the algorithms’ effectiveness are validated by empirical experiments.

The workshop's rationale stems from the escalating interest in causal inference and machine learning methodologies within various industrial contexts. This surge in demand underscores the importance for both scholars and practitioners to exchange knowledge and best practices regarding the application of these techniques to tackle real-world challenges. Yet, applying causal machine learning techniques in real-world scenarios presents a range of challenges not addressed in the academic literature. This workshop aims to address the challenges for practical causal machine learning and explore new industry use cases. The workshop will provide a forum for practitioners and researchers to exchange ideas and explore new collaborations. Moreover, this workshop aims to capitalize on the success and achievements of the KDD 2023 Workshop titled "Causal Inference and Machine Learning in Practice".

The increasing demand for data-driven decision-making has led to the rapid growth of machine learning applications in various industries. However, the ability to draw causal inferences from observational data remains a crucial challenge. In recent years, causal inference has emerged as a powerful tool for understanding the effects of interventions in complex systems. Combining causal inference with machine learning has the potential to provide a deeper understanding of the underlying mechanisms and to develop more effective solutions to real-world problems.

The areas of reinforcement learning and multi-armed bandits have recently seen significant innovation, while many application domains, such as e-commerce, are full of problems and challenges to which vanilla RL or MAB methods cannot directly apply. This workshop aims at filling this communication gap by creating a platform for researchers and practitioners from both the method/theory side and application side of the community. Having this platform now instead of at a later time is beneficial to all sides of the community: practitioners and frontline scientists are able to avoid re-inventing existing techniques; theory-oriented researchers can find motivation in industry problems, working within more realistic settings, and making real-world impact. The 2nd Multi-armed Bandits and Reinforcement Learning Workshop was a half day workshop co-located with the 29th ACM SIGKDD Conference on Knowledge Discovery & Data Mining (KDD 2023) in Long Beach, California.

Experimentation has been critical and increasingly popular across various domains, such as clinical trials and online platforms, due to its widely recognized benefits. One of the primary objectives of classical experiments is to estimate the average treatment effect (ATE) to inform future decision-making. However, in healthcare and many other settings, treatment effects may be non-stationary, meaning that they can change over time, rendering the traditional experimental design inadequate and the classical static ATE uninformative. In this work, we address the problem of non-stationary experimental design under linear trends by considering two objectives: estimating the dynamic treatment effect and minimizing welfare loss within the experiment. We propose an efficient design that can be customized for optimal estimation error rate, optimal regret rate, or the Pareto optimal trade-off between the two objectives. We establish information-theoretical lower bounds that highlight the inherent challenge in estimating dynamic treatment effects and minimizing welfare loss, and also statistically reveal the fundamental trade-off between them.

We consider the stochastic multi-armed bandit problem and fully characterize the interplays among three desired properties for policy design: worst-case optimality, instance-dependent consistency, and light-tailed risk. We show how the order of expected regret exactly affects the decaying rate of the regret tail probability for both the worst-case and instance-dependent scenario. A novel policy is proposed to achieve the optimal regret tail risk for any regret threshold. Concretely, for any given $\alpha\in[1/2, 1)$ and $\beta\in[0, 1)$, our policy achieves a worst-case expected regret of $\tilde O(T^\alpha)$ and instance-dependent expected regret of $\tilde O(T^\beta)$, while enjoys a probability of incurring an $\Omega(T^\delta)$ regret that decays exponentially with a polynomial $T$ term. Such decaying rate is proved to be best achievable. We also generalize our analysis to the stochastic multi-armed bandit problem with non-stationary baseline rewards, where in each time period $t$, the decision maker pulls one of $K$ arms and collects a reward which is the sum of three terms: the mean of the pulled arm, an independent noise, and a non-stationary baseline reward as a function of $t$. Our results reveal insights on the trade-off between expected regret and tail risk for both worst-case and instance-dependent scenario, indicating that more sub-optimality and inconsistency leaves space for more light-tailed risk of incurring a large regret.

Contextual decision-making problems have witnessed extensive applications in various fields such as online content recommendation, personalized healthcare, and autonomous vehicles, where a core practical challenge is to select a suitable surrogate model for capturing unknown complicated reward functions. It is often the case that both high approximation accuracy and explicit uncertainty quantification are desired. In this work, we propose a neural network-accompanied Gaussian process (NN-AGP) model, which leverages neural networks to approximate the unknown and potentially complicated reward function regarding the contextual variable, and maintains a Gaussian process surrogate model with respect to the decision variable. Our model is shown to outperform existing approaches by offering better approximation accuracy thanks to the use of neural networks and possessing explicit uncertainty quantification from the Gaussian process. We also analyze the maximum information gain of the NN-AGP model and prove regret bounds for the corresponding algorithms. Moreover, we conduct experiments on both synthetic and practical problems, illustrating the effectiveness of our approach.

We propose gradient-based simulation-optimization algorithms to optimize systems that have complicated stochastic structure. The presence of complicated stochastic structure, such as the involvement of infinite-dimensional continuous-time stochastic processes, may cause the exact simulation of the system to be costly or even impossible. On the other hand, for a complicated system, one can sometimes construct a sequence of approximations at different resolutions, where the sequence has finer and finer approximation resolution but higher and higher cost to simulate. With the goal of optimizing the complicated system, we propose algorithms that strategically use the approximations with increasing resolution and higher simulation cost to construct stochastic gradients and perform gradient search in the decision space. To accommodate scenarios where approximations cause discontinuities and lead path-wise gradient estimators to have an uncontrollable bias, stochastic gradients for the proposed algorithms are constructed through finite difference. As a theory support, we prove algorithm convergence, convergence rate, and optimality of algorithm design under the assumption that the objective function for the complicated system is strongly convex, whereas no such assumptions are imposed on the approximations of the complicated system. We then present a multilevel version of the proposed algorithms to further improve convergence rates, when in addition the sequence of approximations can be naturally coupled. History: Accepted by Bruno Tuffin, Area Editor for Simulation. Supplemental Material: The software that supports the findings of this study is available within the paper and its Supplemental Information ( https://pubsonline.informs.org/doi/suppl/10.1287/ijoc.2023.1279 ) as well as from the IJOC GitHub software repository ( https://github.com/INFORMSJoC/2021.0289 ) at ( http://dx.doi.org/10.5281/zenodo.7485443 ).

A/B tests, also known as online controlled experiments, have been used at scale by data-driven enterprises to guide decisions and test innovative ideas. Meanwhile, nonstationarity, such as the time-of-day effect, can commonly arise in various business metrics. We show that inadequately addressing nonstationarity can cause A/B tests to be statistically inefficient or invalid, leading to wrong conclusions. To address these issues, we develop a new framework that provides appropriate modeling and adequate statistical analysis for nonstationary A/B tests. Without changing the infrastructure for any existing A/B test procedure, we propose a new estimator that views time as a continuous covariate to perform post stratification with a sample-dependent number of stratification levels. We prove central limit theorem in a natural limiting regime under nonstationarity, so that valid large-sample statistical inference is available. We show that the proposed estimator achieves the optimal asymptotic variance among all estimators. When the experiment design phase of an A/B test allows, we propose a new time-grouped randomization approach to make a better balance on treatment and control assignments in presence of time nonstationarity. A brief account of numerical experiments are conducted to illustrate the theoretical analysis.

We consider the classical multi-armed bandit problem and design simple-to-implement new policies that simultaneously enjoy two properties: worst-case optimality for the expected regret, and safety against heavy-tailed risk for the regret distribution. Recently, Fan and Glynn (2021) showed that information-theoretic optimized bandit policies as well as standard UCB policies suffer from some serious heavy-tailed risk; that is, the probability of incurring a linear regret slowly decays at a polynomial rate of $1/T$, as $T$ (the time horizon) increases. Inspired by their result, we further show that any policy that incurs an instance-dependent $O(\ln T)$ regret must incur a linear regret with probability $\Omega(\mathrm{poly}(1/T))$ and that the heavy-tailed risk actually exists for all "instance-dependent consistent" policies. Next, for the two-armed bandit setting, we provide a simple policy design that (i) has the worst-case optimality for the expected regret at order $\tilde O(\sqrt{T})$ and (ii) has the worst-case tail probability of incurring a linear regret decay at an exponential rate $\exp(-\Omega(\sqrt{T}))$. We further prove that this exponential decaying rate of the tail probability is optimal across all policies that have worst-case optimality for the expected regret. Finally, we generalize the policy design and analysis to the general setting with an arbitrary $K$ number of arms. We provide detailed characterization of the tail probability bound for any regret threshold under our policy design. Numerical experiments are conducted to illustrate the theoretical findings. Our results reveal insights on the incompatibility between consistency and light-tailed risk, whereas indicate that worst-case optimality on expected regret and light-tailed risk are compatible.

Motivated by emerging applications such as live-streaming e-commerce, promotions and recommendations, we introduce a general class of multi-armed bandit problems that have the following two features: (i) the decision maker can pull and collect rewards from at most $K$ out of $N$ different arms in each time period; (ii) the expected reward of an arm immediately drops after it is pulled, and then non-parametrically recovers as the idle time increases. With the objective of maximizing expected cumulative rewards over $T$ time periods, we propose, construct and prove performance guarantees for a class of “Purely Periodic Policies”. For the offline problem when all model parameters are known, our proposed policy obtains an approximation ratio that is at the order of $1-\mathcal O(1/\sqrt{K})$, which is asymptotically optimal when $K$ grows to infinity. For the online problem when the model parameters are unknown and need to be learned, we design an Upper Confidence Bound (UCB) based policy that approximately has $\widetilde{\mathcal O}(N\sqrt{T})$ regret against the offline benchmark. Our framework and policy design may have the potential to be adapted into other offline planning and online learning applications with non-stationary and recovering rewards.

We study an extension of the stochastic submodular minimization problem, namely, the stochastic $L^\natural$-convex minimization problem. We develop the first polynomial-time algorithms that return a near-optimal solution with high probability. We design a novel truncation operation to further reduce the computational complexity of the proposed algorithms. When applied to a stochastic submodular function, the computational complexity of the proposed algorithms is lower than that of the existing stochastic submodular minimization algorithms. In addition, we provide a strongly polynomial approximate algorithm. The algorithm execution also does not require any prior knowledge about the objective function except the $L^\natural$-convexity. A lower bound on the computational complexity that is required to achieve a high probability error bound is also derived. Numerical experiments are implemented to demonstrate the efficiency of our theoretical findings.