Online game advertising is a prominent class of Web-mediated interactive services, where understanding and predicting player Lifetime Value (LTV) is a core scientific challenge in Web-scale user modeling, personalization, and digital economy optimization. However, the LTV prediction task poses severe challenges to traditional methods, which include data sparsity and complex distribution characteristics (such as zero-inflation, long tail, multimodal distribution, and cross-game). Existing methods struggle to capture the realistic and complex LTV distributions and exhibit limitations in leveraging cross-game data. We propose the first cross-game dynamic mixture framework with weighted sub-distributions for LTV prediction, DynaMoLTV. DynaMoLTV primarily models complex distributions via a zero-inflated mixture of lognormal (ZIMLN) loss, incorporates a game expert for cross-game data adaptation, employs a hierarchical payment classifier to capture consumption pattern variations, and integrates coarse and fine-grained losses to balance high-value user identification with LTV prediction accuracy. We conduct comprehensive experiments. The results demonstrate that DynaMoLTV achieves the best performance compared to five state-of-the-art baselines across metrics, including paid user identification, high-value user recall and LTV prediction accuracy. Specifically on three gaming datasets, DynaMoLTV reduces RMSE by 0.76%–46.65%, improves AUC by 0.94%–7.11%, and improves Norm-GINI by 0.63%–11.77% compared to five state-of-the-art baselines. DynaMoLTV also significantly improves ranking capabilities, with Recall@50K increasing by 17.64%–577.78%. We validate DynaMoLTV's effectiveness through two online A/B tests: (1) In the scenario of churned user re-engagement, DynaMoLTV increases online LTV by 20.3%-142.6% and downloads by 22.6%-37.7%. (2) In the scenario of online game advertising, DynaMoLTV increases GMV by 1.89% and GMV(ROI) by 27.31%. Our method has been fully deployed in a Web-based online game advertising platform, which ensures that LTV predictions remain personalized for online gaming ad delivery, supporting smarter and more inclusive decision-making on the Web.

Graph convolutional networks (GCNs) play a critical role in improving the performance of collaborative filtering. They leverage the concept of aggregating neighbor information to capture user preferences on bipartite graphs by stacking multiple convolutional layers. However, this requirement for layer stacking often leads to a long training time for convergence, and results in indistinguishable representations with significant performance deterioration due to the problem of oversmoothing. Additionally, the noise of interactions will be amplified by the stacking of convolutional layers through message passing. To address these issues, we propose a simple, plug-and-play-Neighborhood Structure -Embedding approach, named NSE, which utilizes first-order adjacency information to construct structural embeddings. By explicitly incorporating local topologically statistical information before message passing, the embeddings propagated at GCNs have better topology-structure awareness. This leads to an improved optimization path and greater robustness against noise propagation. Experimental results demonstrate significant performance improvements by employing our proposed NSE in graph collaborative filtering models. Particularly, the NSE-enhanced LGCN shows performance gains of 5.06% and 4.86% on the Yelp and Amazon-Books datasets, respectively. The average training convergence speed is improved by 204.8%. NSE-enhanced graph collaborative filtering has also demonstrated excellent robustness against both noise and oversmoothing.

Modern internet platforms usually have different scenarios to provide rich recommendation services to meet the diverse demands of users. Cross-domain recommendation (CDR) and multi-domain recommendation (MDR) methods are widely used in such platforms to leverage rich auxiliary information from multiple domains. However, state-of-the-art CDR and MDR methods usually enforce some correlations between source and target embeddings on each user, ignoring the correlations between users in both domains. To address this problem, we adopt a relaxed contrastive loss, that employs the pairwise similarities in the source domain as relaxed labels, enforcing such inter-sample relations are reserved in a weighted manner in the target domain. The basic assumption behind such a design is that users with similar interests should be with similar interacted items in a rec- ommender system, and this work takes a step further to realize and specify such similarity modeling as collaborative signals encoded in both implicit embedding spaces. We validate the effectiveness of the proposed method on a large- scale public dataset and a real production dataset with over 700 million samples. We further experimentally show that the proposed embedding transfer method is generic, and can be plugged into any existing deep neural networks, such as YoutubeDNN and BERT4Rec. Currently, the proposed embedding transfer techniques have been successfully deployed in the Guess You Like in WeTV for the CDR/MDR task.

Cross-domain recommendation is an important method to improve recommender system performance, especially when observations in target domains are sparse. However, most existing techniques focus on single-target or dual-target cross-domain recommendation (CDR) and are hard to be generalized to CDR with multiple target domains. In addition, the negative transfer problem is prevalent in CDR, where the recommendation performance in a target domain may not always be enhanced by knowledge learned from a source domain, especially when the source domain has sparse data. In this study, we propose CAT-ART, a multi-target CDR method that learns to improve recommendations in all participating domains through representation learning and embedding transfer. Our method consists of two parts: a self-supervised Contrastive AuToencoder (CAT) framework to generate global user embeddings based on information from all participating domains, and an Attention-based Representation Transfer (ART) framework which transfers domain-specific user embeddings from other domains to assist with target domain recommendation. CAT-ART boosts the recommendation performance in any target domain through the combined use of the learned global user representation and knowledge transferred from other domains, in addition to the original user embedding in the target domain. We conducted extensive experiments on a collected real-world CDR dataset spanning 5 domains and involving a million users. Experimental results demonstrate the superiority of the proposed method over a range of prior arts. We further conducted ablation studies to verify the effectiveness of the proposed components. Our collected dataset will be open-sourced to facilitate future research in the field of multi-domain recommender systems and user modelling.

Graph neural networks have been widely used in recent recommender systems, where negative sampling plays an important role. Existing negative sampling methods restrict the relationship between nodes as either hard positive pairs or hard negative pairs. This leads to the loss of structural information, and lacks the mechanism to generate positive pairs for nodes with few neighbors. To overcome limitations, we propose a novel soft link-based sampling method, namely MixDec Sampling, which consists of Mixup Sampling module and Decay Sampling module. The Mixup Sampling augments node features by synthesizing new nodes and soft links, which provides sufficient number of samples for nodes with few neighbors. The Decay Sampling strengthens the digestion of graph structure information by generating soft links for node embedding learning. To the best of our knowledge, we are the first to model sampling relationships between nodes by soft links in GNN-based recommender systems. Extensive experiments demonstrate that the proposed MixDec Sampling can significantly and consistently improve the recommendation performance of several representative GNN-based models on various recommendation benchmarks.

In the ecology of short video platforms, the optimal exposure proportion of each video category is crucial to guide recommendation systems and content production in a macroscopic way. Though extensive studies on recommendation systems are devoted to providing the most well-matched videos for each view request, fitting the data without considering inherent biases such as selection bias and exposure bias will result in serious issues. In this paper, we formalize the exposure proportion strategy as a policy-making problem with multi-dimensional continuous treatment under certain constraints from a causal inference point of view. We propose a novel ensemble policy learning method based on causal trees, called Maximum Difference of Preference Point Forest (MDP2 Forest), which overcomes the shortcomings of existing policy learning approaches. Experimental results on both simulated and synthetic datasets show the superiority of our algorithm compared to other policy learning or causal inference methods in terms of the treatment estimation accuracy and the mean regret. Furthermore, the proposed MDP2 Forest method can also adapt to a wide range of business settings such as imposing different kinds of constraints on the multi-dimensional treatment.

For many applications, predicting the users' intents can help the system provide the solutions or recommendations to the users. It improves the user experience, and brings economic benefits. The main challenge of user intent prediction is that we lack enough labeled data for training, and some intents (labels) are sparse in the training set. This is a general problem for many real-world prediction tasks. To overcome data sparsity, we propose a masked-field pre-training framework. In pre-training, we exploit massive unlabeled data to learn useful feature interaction patterns. We do this by masking partial field features, and learning to predict them from other unmasked features. We then finetune the pre-trained model for the target intent prediction task. This framework can be used to train various deep models. In the intent prediction task, each intent is only relevant to partial features. To tackle this problem, we propose a Field-Independent Transformer network. This network generates separate representation for each field, and aggregates the relevant field representations with attention mechanism for each intent. We test our method on intent prediction datasets in customer service scenarios as well as several public datasets. The results show that the masked-field pre-training framework significantly improves the prediction precision for deep models. And the Field-Independent Transformer network trained with the masked-field pre-training framework outperforms the state-of-the-art methods in the user intent prediction.

The estimated time of arrival (ETA) is a critical task in the intelligent transportation system, which involves the spatiotemporal data. Despite a significant amount of prior efforts have been made to design efficient and accurate systems for ETA task, few of them take structural graph data into account, much less the heterogeneous information network. In this paper, we propose HetETA to leverage heterogeneous information graph in ETA task. Specifically, we translate the road map into a multi-relational network and introduce a vehicle-trajectories based network to jointly consider the traffic behavior pattern. Moreover, we employ three components to model temporal information from recent periods, daily periods and weekly periods respectively. Each component comprises temporal convolutions and graph convolutions to learn representations of the spatiotemporal heterogeneous information for ETA task. Experiments on large-scale datasets illustrate the effectiveness of the proposed HetETA beyond the state-of-the-art methods, and show the importance of representation learning of heterogeneous information networks for ETA task.

Customer response prediction is critical in many industrial applications such as online advertising and recommendations. In particular, the challenge is greater for ride-hailing platforms such as Uber and DiDi, because the response prediction models need to consider historical and real-time event information in the physical environment, such as surrounding traffic and supply and demand conditions. In this paper, we propose to use dynamically constructed heterogeneous graph for each ongoing event to encode the attributes of the event and its surroundings. In addition, we propose a multi-layer graph neural network model to learn the impact of historical actions and the surrounding environment on the current events, and generate an effective event representation to improve the accuracy of the response model. We investigate this framework to two practical applications on the DiDi platform. Offline and online experiments show that the framework can significantly improve prediction performance. The framework has been deployed in the online production environment and serves tens of millions of event prediction requests every day.

Search and recommender systems process rich natural language text data such as user queries and documents. Achieving high-quality search and recommendation results requires processing and understanding such information effectively and efficiently, where natural language processing (NLP) technologies are widely deployed. In recent years, the rapid development of deep learning technology has been proven successful for improving various NLP tasks, indicating their great potential of promoting search and recommender systems.

Dialogue summarization extracts useful information from a dialogue. It helps people quickly capture the highlights of a dialogue without going through long and sometimes twisted utterances. For customer service, it saves human resources currently required to write dialogue summaries. A main challenge of dialogue summarization is to design a mechanism to ensure the logic, integrity, and correctness of the summaries. In this paper, we introduce auxiliary key point sequences to solve this problem. A key point sequence describes the logic of the summary. In our training procedure, a key point sequence acts as an auxiliary label. It helps the model learn the logic of the summary. In the prediction procedure, our model predicts the key point sequence first and then uses it to guide the prediction of the summary. Along with the auxiliary key point sequence, we propose a novel Leader-Writer network. The Leader net predicts the key point sequence, and the Writer net predicts the summary based on the decoded key point sequence. The Leader net ensures the summary is logical and integral. The Writer net focuses on generating fluent sentences. We test our model on customer service scenarios. The results show that our model outperforms other models not only on BLEU and ROUGE-L score but also on logic and integrity.

For decades large corporations as well as labor placement services have maintained extensive yet static resume databanks. Online professional networks like LinkedIn have taken these resume databanks to a dynamic, constantly updated and massive scale professional profile dataset spanning career records from hundreds of industries, millions of companies and hundreds of millions of people worldwide. Using this professional profile dataset, this paper attempts to model profiles of individuals as a sequence of positions held by them as a time-series of nodes, each of which represents one particular position or job experience in the individual's career trajectory. These career trajectory models can be employed in various utility applications including career trajectory planning for students in schools & universities using knowledge inferred from real world career outcomes. They can also be employed for decoding sequences to uncover paths leading to certain professional milestones from a user's current professional status. We deploy the proposed technique to ascertain professional similarity between two individuals by developing a similarity measure SimCareers (Similar Career Paths). The measure employs sequence alignment between two career trajectories to quantify professional similarity between career paths. To the best of our knowledge, SimCareers is the first framework to model professional similarity between two people taking account their career trajectory information. We posit, that using the temporal and structural features of a career trajectory for modeling profile similarity is a far more superior approach than using similarity measures on semi-structured attribute representation of a profile for this application. We validate our hypothesis by extensive quantitative evaluations on a gold dataset of similar profiles generated from recruiting activity logs from actual recruiters using LinkedIn. In addition, we show significant improvements in engagement by running an A/B test on a real-world application called Similar Profiles on LinkedIn, world's largest online professional network.