Click-through rate (CTR) prediction lies at the heart of the online advertising ecosystem and recommendation systems, helping to improve user engagement and platform revenue. With the advent of Pretrained Language Models (PLMs), researchers focus on incorporating textual features to enhance semantic understanding in CTR prediction. However, existing methods typically aggregate a wealth of textual features and encode the informative text into a single semantic embedding. This mechanism leads to entangled embedding that fails to capture fine-grained feature interactions, ultimately limiting CTR prediction performance. To address this issue, we propose Multi-faceted Semantic Disentanglement for CTR prediction (MSD-CTR), a novel framework designed to disentangle and leverage multi-faceted textual information. MSD-CTR consists of two key components: Disentangled Semantic Topic Model (DSTopic) and Topic Guided Disentangled Representation Learning (TopicDRL). DSTopic employs a disentangled generative process to extract multi-faceted knowledge from the entangled textual information. Meanwhile, TopicDRL integrates the extracted multi-faceted knowledge into CTR prediction and introduces two alignment losses to guide disentangled semantic embedding learning. Extensive experiments on four real-world datasets demonstrate that MSD-CTR outperforms existing CTR models, highlighting the effectiveness of disentangling textual information for better click-through rate prediction.

Display advertising plays a crucial role in benefiting advertisers, publishers, and users. Traditional display advertising systems employ a multi-stage architecture comprising retrieval, coarse ranking, ranking, and re-ranking. However, conventional retrieval methods primarily rely on ID-based learning-to-rank mechanisms, often underutilizing the content information of ads, like ads' title, and description. This limitation reduces the ability to generate diverse and relevant recommendation lists. To address this challenge, we propose leveraging the extensive world knowledge of large language models (LLMs). However, effectively integrating LLMs into advertising systems presents three key challenges: ( i) How to accurately capture user interests, (ii) How to bridge the knowledge gap between LLMs and advertising systems , and ( iii) How to efficiently deploy LLMs at scale. To overcome these challenges, we introduce LEADRE —the L LM E mpowered Display AD vertisement RE commender system. LEADRE consists of three core components. The Intent-Aware Prompt Engineering module introduces multi-faceted knowledge and constructs intent-aware pairs, fine-tuning LLMs to generate ads tailored to users' personal interests. The Advertising-Specific Knowledge Alignment module incorporates auxiliary fine-tuning tasks and Direct Preference Optimization (DPO) to align LLMs with advertising semantics and business objectives. The Latency-Aware Model Deployment module integrates a hybrid service framework that balances latency-tolerant and latency-sensitive service, ensuring seamless online deployment. Extensive offline experiments validate the effectiveness of LEADRE, demonstrating significant improvements across multiple evaluation metrics. Furthermore, online A/B tests reveal a 1.57% and 1.17% increase in Gross Merchandise Value (GMV) for serviced users on WeChat Channels and Moments, respectively. LEADRE has been successfully deployed on both platforms, handling tens of billions of requests daily.

When using large-batch training to speed up stochastic gradient descent, learning rates must adapt to new batch sizes in order to maximize speed-ups and preserve model quality. Re-tuning learning rates is resource intensive, while fixed scaling rules often degrade model quality. We propose AdaScale SGD, an algorithm that reliably adapts learning rates to large-batch training. By continually adapting to the gradient’s variance, AdaScale automatically achieves speed-ups for a wide range of batch sizes. We formally describe this quality with AdaScale’s convergence bound, which maintains final objective values, even as batch sizes grow large and the number of iterations decreases. In empirical comparisons, AdaScale trains well beyond the batch size limits of popular “linear learning rate scaling” rules. This includes large-batch training with no model degradation for machine translation, image classification, object detection, and speech recognition tasks. AdaScale’s qualitative behavior is similar to that of "warm-up" heuristics, but unlike warm-up, this behavior emerges naturally from a principled mechanism. The algorithm introduces negligible computational overhead and no new hyperparameters, making AdaScale an attractive choice for large-scale training in practice.