In the dynamic landscape of online advertising, decoding user intent remains a pivotal challenge, particularly in the context of query classification. Swift classification models, exemplified by FastText, cater to the demand for real-time responses but encounter limitations in handling intricate queries. Conversely, accuracy-centric models like BERT introduce challenges associated with increased latency. This paper undertakes a nuanced exploration, navigating the delicate balance between efficiency and accuracy. It unveils FastText's latent potential as an 'online dictionary' for historical queries while harnessing the semantic robustness of BERT for novel and complex scenarios. The proposed Distribution-Diverse Multi-Expert (DDME) framework employs multiple teacher models trained from diverse data distributions. Through meticulous data categorization and enrichment, it elevates the classification performance across the query spectrum. Empirical results within the JD ads search system validate the superiority of our proposed approaches.

Multi-agent collaboration is the latest trending method to build conversational recommender systems (CRS), especially with the widespread use of Large Language Models (LLMs) recently. Typically, these systems employ several LLM agents, each serving distinct roles to meet user needs. In an industrial setting, it’s essential for a CRS to exhibit low first token latency (i.e., the time taken from a user’s input until the system outputs its first response token.) and high scalability—for instance, minimizing the number of LLM inferences per user request—to enhance user experience and boost platform profit. For example, JD.com’s baseline CRS features two LLM agents and a search API but suffers from high first token latency and requires two LLM inferences per request (LIPR), hindering its performance. To address these issues, we introduce a Hybrid Multi-Agent Collaborative Recommender System (Hybrid-MACRS). It includes a central agent powered by a fine-tuned proprietary LLM and a search agent combining a related search module with a search engine. This hybrid system notably reduces first token latency by about 70% and cuts the LIPR from 2 to 1. We conducted thorough online A/B testing to confirm this approach’s efficiency.

Re-ranking is a process of rearranging ranking list to more effectively meet user demands by accounting for the interrelationships between items. Existing methods predominantly enhance the precision of search results, often at the expense of diversity, leading to outcomes that may not fulfill the varied needs of users. Conversely, methods designed to promote diversity might compromise the precision of the results, failing to satisfy the users' requirements for accuracy. To alleviate the above problems, this paper proposes a Preference-oriented Diversity Model Based on Mutual-information (PODM-MI), which consider both accuracy and diversity in the re-ranking process. Specifically, PODM-MI adopts Multidimensional Gaussian distributions based on variational inference to capture users' diversity preferences with uncertainty. Then we maximize the mutual information between the diversity preferences of the users and the candidate items using the maximum variational inference lower bound to enhance their correlations. Subsequently, we derive a utility matrix based on the correlations, enabling the adaptive ranking of items in line with user preferences and establishing a balance between the aforementioned objectives. Experimental results on real-world online e-commerce systems demonstrate the significant improvements of PODM-MI, and we have successfully deployed PODM-MI on an e-commerce search platform.

Thompson Sampling (TS) is an effective way to deal with the exploration-exploitation dilemma for the multi-armed (contextual) bandit problem. Due to the sophisticated relationship between contexts and rewards in real- world applications, neural networks are often preferable to model this relationship owing to their superior representation capacity. In this paper, we study the problem of combining neural networks with TS in a plug-and-play manner. The basic idea is to maintain a posterior distribution over the reward mean relying on the prediction and the deep representation of the neural network for any given context. Specifically, our proposed algorithm, PlugTS (Pluggable deep Thompson Sampling), introduces no change into the network training process, but only requires one additional sampling stage during serving - sampling from a univariate Gaussian distribution (by maintaining a positive definite matrix). Theoretically, we prove that PlugTS achieves an regret bound, which matches the state-of-the-art neural network-based TS, while PlugTS enjoys much lower computational overhead for each iteration. Experimental results on public datasets among traditional classification and recommendation tasks validate the effectiveness and efficiency of PlugTS. Furthermore, it is inspiring for real-world applications that a simplified version of PlugTS has been deployed in an industrial advertising recommender system of one of the world's largest e-commerce platforms, JD.com, achieving significant improvement in both RPM (Revenue Per Mille) and CTR (Click-Through Rate) in online A/B testing. The appendix and code are available at https://github.com/adsturing/PlugTS.

Organic recommendation and advertising recommendation usually coexist on e-commerce platforms. In this paper, we study the problem of utilizing data from organic recommendation to reinforce click-through rate prediction in advertising scenarios from a multi-view learning perspective. We propose a novel method, termed LOVF (Layered Organic View Fusion). LOVF implements a multi-view fusion mechanism - for each advertising instance, LOVF derives deep representations layer-by-layer from the organic recommendation view and these deep representations are then fused into the corresponding vanilla representations of the advertising view. Extensive experiments across a variety of backbones demonstrate LOVF's generality, effectiveness and efficiency on a new real-world production dataset. The dataset encompasses data from both the organic recommendation and advertising scenarios. Notably, LOVF has been successfully deployed in the advertising recommender system of JD.com, which is one of the world's largest e-commerce platforms; online A/B testing shows that LOVF achieves impressive improvement on advertising clicks and revenue. Our code and dataset are available at https://github.com/adsturing/lovf for facilitating further research.

CTR prediction is crucial in recommendation systems and online advertising platforms, where user-generated data streams that drift over time can lead to catastrophic forgetting if the model continuously adapts to new data distribution. Conventional strategies for catastrophic forgetting are challenging to deploy due to memory constraints and diverse data distributions. To address this, we propose a novel drift-aware incremental learning framework based on ensemble learning for CTR prediction, which uses explicit error-based drift detection on streaming data to strengthen well-adapted ensembles and freeze ensembles that do not match the input distribution, avoiding catastrophic interference. Our method outperforms all baselines considered in offline experiments and A/B tests.

Industrial systems such as recommender systems and online advertising, have been widely equipped with multi-stage architectures, which are divided into several cascaded modules, including matching, pre-ranking, ranking and re-ranking. As a critical bridge between matching and ranking, existing pre-ranking approaches mainly endure sample selection bias (SSB) problem owing to ignoring the entire-chain data dependence, resulting in sub-optimal performances. In this paper, we rethink pre-ranking system from the perspective of the entire sample space, and propose Entire-chain Cross-domain Models (ECM), which leverage samples from the whole cascaded stages to effectively alleviate SSB problem. Besides, we design a fine-grained neural structure named ECMM to further improve the pre-ranking accuracy. Specifically, we propose a cross-domain multi-tower neural network to comprehensively predict for each stage result, and introduce the sub-networking routing strategy with L0 regularization to reduce computational costs. Evaluations on real-world large-scale traffic logs demonstrate that our pre-ranking models outperform SOTA methods while time consumption is maintained within an acceptable level, which achieves better trade-off between efficiency and effectiveness.