The digital advertising field has always had challenging ML problems, learning from petabytes of data that is highly imbalanced, reactivity times in the milliseconds, and more recently compounded with the complex user's path to purchase across devices, across platforms, and even online/real-world behavior. The AdKDD workshop continues to be a forum for researchers in advertising, during and after KDD. Our website, which hosts slides and abstracts, continues to receive a large number of monthly visits and active users. In surveys during AdKDD 2019 and 2020, over 60% agreed that AdKDD is the reason they attended KDD, and over 90% indicated they would attend next year. The 2025 edition is particularly timely because of the increasing application of Graph-based NN and Generative AI models in advertising. Coupled with privacy-preserving initiatives, such as those enforced by GDPR or CCPA, the future of computational advertising is at an interesting crossroads. For this edition, we plan to solicit papers that span the spectrum of deep user understanding while remaining privacy-preserving. In addition, we will seek papers that discuss fairness in the context of advertising, to what extent does hyper-personalization work, and whether the ad industry as a whole needs to think through more effective business models such as incrementality. We have hosted several academic and industry luminaries as keynote speakers and have found our invited speaker series hosting expert practitioners to be an audience favorite. We will continue fielding a diverse set of keynote speakers and invited talks for this edition as well. As with past editions, we hope to motivate researchers in this space to think not only about the ML aspects but also to spark conversations about the societal impact of online advertising.

The digital advertising field has always had challenging ML problems, learning from petabytes of data that is highly imbalanced, reactivity times in the milliseconds, and more recently compounded with the complex user's path to purchase across devices, across platforms, and even online/real-world behavior. The AdKDD workshop continues to be a forum for researchers in advertising, during and after KDD. Our website which hosts slides and abstracts receives approximately 2,000 monthly visits and 1,800 active users during the KDD 2021. In surveys during AdKDD 2019 and 2020, over 60% agreed that AdKDD is the reason they attended KDD, and over 90% indicated they would attend next year. The 2024 edition is particularly timely because of the increasing application of Graph-based NN and Generative AI models in advertising. Coupled with privacy-preserving initiatives enforced by GDPR, CCPA the future of computational advertising is at an interesting crossroads. For this edition, we plan to solicit papers that span the spectrum of deep user understanding while remaining privacy-preserving. In addition, we will seek papers that discuss fairness in the context of advertising, to what extent does hyper-personalization work, and whether the ad industry as a whole needs to think through more effective business models such as incrementality. We have hosted several academic and industry luminaries as keynote speakers and have found our invited speaker series hosting expert practitioners to be an audience favorite. We will continue fielding a diverse set of keynote speakers and invited talks for this edition as well. As with past editions, we hope to motivate researchers in this space to think not only about the ML aspects but also to spark conversations about the societal impact of online advertising.

The digital advertising field has always had challenging ML problems, learning from petabytes of data that is highly imbalanced, reactivity times in the milliseconds, and more recently compounded with the complex user's path to purchase across devices, across platforms, and even online/real-world behavior. The AdKDD workshop continues to be a forum for researchers in advertising, during and after KDD. Our website which hosts slides and abstracts receives approximately 2,000 monthly visits and 1,800 active users during the KDD 2021. In surveys during AdKDD 2019 and 2020, over 60% agreed that AdKDD is the reason they attended KDD, and over 90% indicated they would attend next year. The 2023 edition is particularly timely because of the increasing application of Graph-based NN and Generative AI models in advertising. Coupled with privacy-preserving initiatives enforced by GDPR, CCPA the future of computational advertising is at an interesting crossroads. For this edition, we plan to solicit papers that span the spectrum of deep user understanding while remaining privacy-preserving. In addition, we will seek papers that discuss fairness in the context of advertising, to what extent does hyper-personalization work, and whether the ad industry as a whole needs to think through more effective business models such as incrementality. We have hosted several academic and industry luminaries as keynote speakers and have found our invited speaker series hosting expert practitioners to be an audience favorite. We will continue fielding a diverse set of keynote speakers and invited talks for this edition as well. As with past editions, we hope to motivate researchers in this space to think not only about the ML aspects but also to spark conversations about the societal impact of online advertising.

An average consumer spends 8+ hours a day across all devices interacting with online content almost entirely sponsored by advertisements. At over $450B global market size in 2022 and expected to pass $1T by 2027, online advertising has already surpassed traditional ads in global spend. Moreover, computational advertising in particular is perhaps the most visible and ubiquitous application of machine learning and one that interacts directly with consumers. When done right, ads help us enrich our lives and creep us out when done badly. Looking at the published literature over the last few years, many researchers might consider computational advertising as a mature field. Yet, the opposite is true. The field is evolving, however, from ads controlled by monolithic publishers and randomly rotating banner ads to highly personalized content experiences in news feeds on mobile devices and even on TV-all utilizing data amassed from petabytes of stored user data. Ads are far from done.

The digital advertising field has always had challenging ML problems, learning from petabytes of data that is highly imbalanced, reactivity times in the milliseconds and more recently compounded with the complex user's path to purchase across devices, across platforms and even online/real-world behavior. The AdKDD workshop continues to be a forum for researchers in advertising, during and after KDD. Our website which hosts slides and abstracts receives approximately 2,000 monthly visits. In surveys during AdKDD 2019 and 2020, over 60% agreed that AdKDD is the reason they attended KDD and over 90% indicated they would attend next year. The 2021 edition is particularly timely because of ongoing developments in ad tracking. We will aim to discuss notions of privacy and tracking enforced by GDPR and through company policies. In addition, we will seek papers that discuss fairness in the context of advertising, to what extent does hyper-personalization work, and on whether the ad industry as a whole needs to think through more effective business models such as incrementality. Ad tech is in an interesting place of evolution/maturity now and we would like to use the AdKDD forum to get the researchers to think not only about the ML aspects but also spark conversations about the societal ones.

Aggregation is a common technique in data-driven applications for handling issues like privacy, scalability and reliability in a vast range of domains including healthcare, sensor networks and web applications. However, despite the ubiquitousness, extending machine learning methods to the aggregation context is unfortunately not well-studied. In this work, we consider the problem of learning individual level predictive models when the target variables used for training are only available as aggregates. In particular, this problem is a critical bottleneck in designing effective bidding strategies in the context of online advertising where ground-truth cost-per-click (CPC) data is aggregated before being released to advertisers. We introduce a novel learning framework that can use aggregates computed at varying levels of granularity for building individual-level predictive models. We generalise our modelling and algorithmic framework to handle data from diverse domains, and extend our techniques to cover arbitrary aggregation paradigms like sliding windows and overlapping/non-uniform aggregation. We show empirical evidence for the efficacy of our techniques with experiments on both synthetic data and real data from the online advertising domain as well as healthcare to demonstrate the wider applicability of our framework.

Machine learning literature on Computational Advertising typically tends to focus on the simplistic CTR prediction problem which while being relevant is the tip of the iceberg in terms of the challenges in the field. There is also very little appreciation for the scale at which the real-time-bidding systems operate (200B bid requests/day) or the increasingly adversarial ecosystem all of which add a ton of constraints in terms of feasible solutions. In this talk, I'll highlight some recent efforts in developing models that try to better encapsulate the journey of an ad from the first display to a user to the effect on an actual purchase.

Factorization Machines (FM) have been recognized as an effective learning paradigm for incorporating complex relations to improve item recommendation in recommender systems. However, one open issue of FM lies in its factorized representation (latent factors) for each feature in the observed feature space, a characteristic often resulting in a large parameter space. Therefore, training FM (in other words, learning a large number of parameters in FM) is a computationally expensive task. Our work targets to improve the scalability of FM by building it in a distributed environment. We propose a new system framework that integrates Parameter Server (PS) with the Map/Reduce (MR) framework. In addition to the data parallelism achieved via MR, our framework particularly benefits from PS for model parallelism, a critical characteristic for learning with a large number of parameters in FM. We further address two specific challenges in our system, namely, communication cost and parameter update collision. Through both offline and online experiments on recommendation tasks, we demonstrate that the proposed system framework succeeds in scaling up FM for very large datasets, while it also maintains competitive performance on recommendation quality compared to alternative baselines.

Discovering latent structures in spatial data is of critical importance to understanding the user behavior of location-based services. In this paper, we study the problem of geographic segmentation of spatial data, which involves dividing a collection of observations into distinct geo-spatial regions and uncovering abstract correlation structures in the data. We introduce a novel, Latent Poisson Factor (LPF) model to describe spatial count data. The model describes the spatial counts as a Poisson distribution with a mean that factors over a joint item-location latent space. The latent factors are constrained with weak labels to help uncover interesting spatial dependencies. We study the LPF model on a mobile app usage data set and a news article readership data set. We empirically demonstrate its effectiveness on a variety of prediction tasks on these two data sets.

Feature-based collaborative filtering models, such as state-of-the-art factorization machines and regression-based latent factor models, rarely consider features' structural information, ignoring the heterogeneity of inter-type and intra-type relationships. Naïvely treating all feature pairs equally would potentially deteriorate the overall recommendation performance. In addition, human prior knowledge and other hierarchical or graphical structures are often available for some features, e.g., the country-state-city hierarchy for geographic features and the topical taxonomy for article features. It is a challenge to utilize the prior knowledge to further boost performance of state-of-the-art models. In this paper we employ rich features from both user and item sides to enhance latent factors learnt from interaction data, uncovering hidden structures from features' relationships and learning sparse pairwise and tree structural connections among features. Our framework borrows the modeling strengh from both structural sparsity modeling and latent factor models. Experiments on a real-world large-scale recommendation data set demonstrated that the proposed model outperforms several strong state-of-the-art baselines.

Despite having various attractive qualities such as high prediction accuracy and the ability to quantify uncertainty and avoid over-fitting, Bayesian Matrix Factorization has not been widely adopted because of the prohibitive cost of inference. In this paper, we propose a scalable distributed Bayesian matrix factorization algorithm using stochastic gradient MCMC. Our algorithm, based on Distributed Stochastic Gradient Langevin Dynamics, can not only match the prediction accuracy of standard MCMC methods like Gibbs sampling, but at the same time is as fast and simple as stochastic gradient descent. In our experiments, we show that our algorithm can achieve the same level of prediction accuracy as Gibbs sampling an order of magnitude faster. We also show that our method reduces the prediction error as fast as distributed stochastic gradient descent, achieving a 4.1% improvement in RMSE for the Netflix dataset and an 1.8% for the Yahoo music dataset.

Content recommendation systems are typically based on one of the following paradigms: user based customization, or recommendations based on either collaborative filtering or low rank matrix factorization methods, or with systems that impute user interest profiles based on content browsing behavior and retrieve items similar to the interest profiles. All of these systems have a distinct disadvantage, namely data sparsity and cold-start on items or users. Furthermore, very few content recommendation solutions explicitly model the wealth of information in implicit negative feedback from the users. In this paper, we propose a hybrid solution that makes use of a latent factor model to infer user interest vectors. The hybrid approach enables us to overcome both the data sparsity and cold-start problems. Our proposed method is learned purely on implicit user feedback, both positive and negative. Exploiting the information in the negative feedback allows the user profiles generated to be discriminative. We also provide a Map/Reduce framework based implementation that enables scaling our solution to real-world recommendation problems. We demonstrate the efficacy of our proposed approach with both offline experiments and A/B tests on live traffic on Yahoo properties.

Many internet companies, such as Yahoo, Facebook, Google and Twitter, rely on content recommendation systems to deliver the most relevant content items to individual users through personalization. Delivering such personalized user experiences is believed to increase the long term engagement of users. While there has been a lot of progress in designing effective personalized recommender systems, by exploiting user interests and historical interaction data through implicit (item click) or explicit (item rating) feedback, directly optimizing for users' satisfaction with the system remains challenging. In this paper, we explore the idea of using item-level dwell time as a proxy to quantify how likely a content item is relevant to a particular user. We describe a novel method to compute accurate dwell time based on client-side and server-side logging and demonstrate how to normalize dwell time across different devices and contexts. In addition, we describe our experiments in incorporating dwell time into state-of-the-art learning to rank techniques and collaborative filtering models that obtain competitive performances in both offline and online settings.

In this paper, we present a system for clustering the search results of a news search engine. The news search interface includes the relevant news articles to a given query organized in terms of related news stories. Here each cluster corresponds to a news story and the news articles are clustered into stories. We present a system that clusters the search results of a news search system in a fast and scalable manner. The clustering system is organized into three components including offline clustering, incremental clustering and realtime clustering. We propose novel techniques for clustering the search results in realtime. The experimental results with large collections of news documents reveal that our system is both scalable and also achieves good accuracy in clustering the news search results.

Many web applications such as ad matching systems, vertical search engines, and page categorization systems require the identification of a particular type or class of pages on the Web. The sheer number and diversity of the pages on the Web, however, makes the problem of obtaining a good sample of the class of interest hard. In this paper, we describe a successfully deployed end-to-end system that starts from a biased training sample and makes use of several state-of-the-art machine learning algorithms working in tandem, including a powerful active learning component, in order to achieve a good classification system. The system is evaluated on traffic from a real-world ad-matching platform and is shown to achieve high categorization effectiveness with a significant reduction in editorial effort and labeling time.

The explosion of online content has made the management of such content non-trivial. Web-related tasks such as web page categorization, news filtering, query categorization, tag recommendation, etc. often involve the construction of multi-label categorization systems on a large scale. Existing multi-label classification methods either do not scale or have unsatisfactory performance. In this work, we propose MetaLabeler to automatically determine the relevant set of labels for each instance without intensive human involvement or expensive cross-validation. Extensive experiments conducted on benchmark data show that the MetaLabeler tends to outperform existing methods. Moreover, MetaLabeler scales to millions of multi-labeled instances and can be deployed easily. This enables us to apply the MetaLabeler to a large scale query categorization problem in Yahoo!, yielding a significant improvement in performance.