The post-training stage of Large Language Models (LLMs) typically involves Supervised Fine-Tuning (SFT) followed by preference alignment to ensure LLMs generate safe, helpful, and instruction-aligned content.The SFT model critically serves as both the initialization and reference model for subsequent preference alignment.However, an essential yet often neglected question is the optimal selection of the SFT checkpoint for this role.We show that checkpoint selection substantially affects final performance, and that the common practice of choosing the minimum validation-loss checkpoint often fails, due to a fundamental conflict between SFT's focus on imitation and alignment's goal of response discriminability.To this end, we propose RewardRank, a simple, effective, training-free metric for estimating initial implicit alignment between reference model and preference objective.Empirical evidence suggests that, using our selected model as reference can gain up to 67.6% relative increase on length-controlled win rate on the popular Zephyr recipe comparing to baselines.

The value and copyright of training data are crucial in the artificial intelligence industry. Service platforms should protect data providers' legitimate rights and fairly reward them for their contributions. Shapley value, a potent tool for evaluating contributions, outperforms other methods in theory, but its computational overhead escalates exponentially with the number of data providers. Recent studies on Shapley values have proposed various approximation algorithms to address the computational complexity issues inherent in exact calculations. However, they need to retrain for each test sample, leading to intolerable costs. We propose Fast-DataShapley, a one-pass training framework that leverages the weighted least squares characterization of the Shapley value to train a reusable explainer model with real-time reasoning speed. Given new test samples, no retraining is required to calculate the Shapley values of the training data. Additionally, we propose three methods with theoretical guarantees to reduce training overhead from two aspects: the approximate calculation of the utility function and the reduction of the sample space complexity. We analyze time complexity to show the efficiency of our methods. The experimental evaluations on various image datasets demonstrate superior performance and efficiency compared to baselines. Specifically, the performance is improved to more than 2×, and the explainer's training speed can be increased by two orders of magnitude.

High-quality data labeling is essential for training robust machine learning models; however, existing methods often ignore fraud or assume non-negative worker utility, failing to penalize harmful contributions without discouraging participation. To address this, we propose the Negativity-Aware Incentive (NAI) mechanism which introduces two novel components. First, the Ability-Result Characteristic Function (AR-CF) adapts and extends Shapley value theory through signed valuation to explicitly capture both positive and negative contributions, by combining workers' abilities with real-time task results to define contribution values. Second, a dynamic stake pool mechanism employs pre-commitment economics with adaptive dual-control parameters to balance fairness and operational efficiency. Through extensive experiments on multimodal datasets (images, text, audio, video), NAI outperforms state-of-the-art baselines: it improves video labeling accuracy by 16.6%, and reduces fraudulent behaviors by 33.9%. Furthermore, our deployment on the NetEase Youling crowdsourcing platform, serving 430,000 registered workers with 80,000 monthly active workers, validates NAI's real-world viability. Real-time A/B testing shows a 59.6% improvement in labeling quality for beginner tasks and a consistent reduction in fraud rates (14.8%-33.9%) across difficulty levels. This work establishes a paradigm shift in crowdsourcing system design, demonstrating that explicit negative modeling can enhance data quality, optimize costs, and foster participation at scale.

The video game industry has emerged as a significant economic force, driving extensive research on optimizing the gaming environment and improving gaming experiences. Among these endeavors, player behavior representation learning has become a critical way to model valuable player properties and is beneficial for a wide range of downstream tasks. However, some common factors, such as login rewards and daily tasks, can trigger similar behaviors among different players, which are informative and noisy for learning high-quality player behavior representations. Existing methods ignore the low signal-to-noise ratio in player behavior data and waste too much modeling capacity on less informative behaviors, resulting in their learned representations being noisy. In this paper, we propose a novel model for Denoised Player Behavior representation Learning, namely DPBL, which consists of two key modules. The first module extracts various player behavior patterns and isolates them from less informative noise. The second module utilizes the extracted patterns to refine the embedding of each behavior and eliminates noise. To optimize DPBL, two contrastive learning strategies are proposed to identify the noise that should be eliminated and to learn distinguishable representations, respectively. With the above design, DPBL is capable of mitigating the impact of noise in the data and learning high-quality representations that effectively capture player characteristics. We conducted extensive experiments on two real-world datasets, and DPBL outperforms all baselines on various downstream tasks with an improvement of 1.4% ∼ 18.1%. The results also show that DPBL achieves an improvement of 5.6% ∼ 23.0% in the denoising experiments, which proves that DPBL is more robust to noisy behaviors. Code is available at https://github.com/LwbXc/DPBL.

Large language models (LLMs) have demonstrated exceptional performance across a wide range of natural language tasks. However, selecting the optimal LLM to respond to a user query often necessitates a delicate balance between performance and cost. While powerful models deliver better results, they come at a high cost, whereas smaller models are more cost-effective but less capable. To address this trade-off, we propose IRT-Router, a multi-LLM routing framework that efficiently routes user queries to the most suitable LLM. Inspired by Item Response Theory (IRT), a psychological measurement methodology, IRT-Router explicitly models the relationship between LLM capabilities and user query attributes. This not only enables accurate prediction of response performance but also provides interpretable insights, such as LLM abilities and query difficulty. Additionally, we design an online query warm-up technique based on semantic similarity, further enhancing the online generalization capability of IRT-Router. Extensive experiments on 20 LLMs and 12 datasets demonstrate that IRT-Router outperforms most baseline methods in terms of effectiveness and interpretability. Its superior performance in cold-start scenarios further confirms the reliability and practicality of IRT-Router in real-world applications. Code is available at https://github.com/Mercidaiha/IRT-Router.

Mingxuan Xia, Haobo Wang, Yixuan Li, Zewei Yu, Jindong Wang, Junbo Zhao, Runze Wu. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025.

Personality is an important concept in psychology that reflects individual differences in thinking and behavior, and has significant applications across various fields.Most existing personality analysis methods address this issue at the bag level, treating the entire corpus gathered from one individual as a single unit for classification.However, this paradigm presents several challenges.From the data perspective, collecting a large corpus for each individual and performing comprehensive annotations pose significant difficulties in both data collection and labeling.On the application side, concentrating on classifying the entire corpus limits its applicability in more common single-instance scenarios.To address these issues, we propose a new task paradigm in text-based personality representation learning.Specifically, we construct a triplet personality trend comparison dataset to learn single-sentence personality embeddings with desirable metric properties.This approach removes the traditional constraints on data sources, facilitating dataset expansion, and can leverage the transfer capabilities of embeddings to easily adapt to various downstream tasks.Our experiments show that the learned embeddings significantly boost performance by a relative 10% across various applications, including personality detection, personality retrieval, and emotion translation prediction.The code and dataset are available at https://github.com/zjutangk/PTCD.

Within the domain of Massively Multiplayer Online (MMO) economy research, Agent-Based Modeling (ABM) has emerged as a robust tool for analyzing game economics, evolving from rule-based agents to decision-making agents enhanced by reinforcement learning. Nevertheless, existing works encounter significant challenges when attempting to emulate human-like economic activities among agents, particularly regarding agent reliability, sociability, and interpretability.In this study, we take a preliminary step in introducing a novel approach using Large Language Models (LLMs) in MMO economy simulation. Leveraging LLMs' role-playing proficiency, generative capacity, and reasoning aptitude, we design LLM-driven agents with human-like decision-making and adaptability. These agents are equipped with the abilities of role-playing, perception, memory, and reasoning, addressing the aforementioned challenges effectively. Simulation experiments focusing on in-game economic activities demonstrate that LLM-empowered agents can promote emergent phenomena like role specialization and price fluctuations in line with market rules.

Matchmaking is a core task in e-sports and online games, as it contributes to player engagement and further influences the game's lifecycle. Previous methods focus on creating fair games at all times. They divide players into different tiers based on skill levels and only select players from the same tier for each game. Though this strategy can ensure fair matchmaking, it is not always good for player engagement. In this paper, we propose a novel Engagement-oriented Matchmaking (EnMatch) framework to ensure fair games and simultaneously enhance player engagement. Two main issues need to be addressed. First, it is unclear how to measure the impact of different team compositions and confrontations on player engagement during the game considering the variety of player characteristics. Second, such a detailed consideration on every single player during matchmaking will result in an NP-hard combinatorial optimization problem with non-linear objectives. In light of these challenges, we turn to real-world data analysis to reveal engagement-related factors. The resulting insights guide the development of engagement modeling, enabling the estimation of quantified engagement before a match is completed. To handle the combinatorial optimization problem, we formulate the problem into a reinforcement learning framework, in which a neural combinatorial optimization problem is built and solved. The performance of EnMatch is finally demonstrated through the comparison with other state-of-the-art methods based on several real-world datasets and online deployments on two games.

New intent discovery (NID) is an important problem for deploying practical dialogue systems, which trains intent classifiers on a semisupervised corpus where unlabeled user utterances contain both known and novel intents.Most existing NID algorithms place hope on the sample similarity to cluster unlabeled corpus to known or new samples.Lacking supervision on new intents, we experimentally find the intent classifier fails to fully distinguish new intents since they tend to assemble into intertwined centers.To address this problem, we propose a novel GeoID framework that learns geometry-aware representations to maximally separate all intents.Specifically, we are motivated by the recent findings on Neural Collapse (NC) in classification tasks to derive optimal intent center structure.Meanwhile, we devise a dual pseudo-labeling strategy based on optimal transport assignments and semi-supervised clustering, ensuring proper utterances-to-center arrangement.Extensive results show that our GeoID method establishes a new state-of-the-art performance, achieving a +3.49% average accuracy improvement on three standardized benchmarking datasets.We also verify its usefulness in assisting large language models for improved in-context performance.The code is available at https: //github.com/zjutangk/GeoID.

Semi-supervised Learning (SSL) has shown remarkable success in applications with limited supervision. However, due to the scarcity of labels in the training process, SSL algorithms are known to be impaired by the lack of proper model selection, as splitting a validation set will further reduce the limited labeled data, and the size of the validation set could be too small to provide a reliable indication to the generalization error. Therefore, we seek alternatives that do not rely on validation data to probe the generalization performance of SSL models. Specifically, we find that the distinct margin distribution in SSL can be effectively utilized in conjunction with the model’s spectral complexity, to provide a non-vacuous indication of the generalization error. Built upon this, we propose a novel model selection method, specifically tailored for SSL, known as Spectral-normalized Labeled-margin Minimization (SLAM). We prove that the model selected by SLAM has upper-bounded differences w.r.t. the best model within the search space. In addition, comprehensive experiments showcase that SLAM can achieve significant improvements compared to its counterparts, verifying its efficacy from both theoretical and empirical standpoints.

This work studies the novel Crowdsourced Multi-Label Learning (CMLL) problem, where each instance is related to multiple true labels but the model only receives unreliable labels from different annotators. Although a few Crowdsourced Multi-Label Inference (CMLI) methods have been developed, they require both the training and testing sets to be assigned crowdsourced labels and focus on true label inferring rather than prediction, making them less practical. In this paper, by excavating the generation process of crowdsourced labels, we establish the first unbiased risk estimator for CMLL based on the crowdsourced transition matrices. To facilitate transition matrix estimation, we upgrade our unbiased risk estimator by aggregating crowdsourced labels and transition matrices from all annotators while guaranteeing its theoretical characteristics. Integrating with the unbiased risk estimator, we further propose a decoupled autoencoder framework to exploit label correlations and boost performance. We also provide a generalization error bound to ensure the convergence of the empirical risk estimator. Experiments on various CMLL scenarios demonstrate the effectiveness of our proposed method. The source code is available at https://github.com/MingxuanXia/CLEAR.

Multi-label text classification (MLTC) allows a given text to be associated with multiple labels, which well suits many real-world data mining scenarios. However, the annotation effort of MLTC is inevitably expensive and time-consuming. Although multi-label active learning provides a cost-effective solution, it still faces two major challenges: (i) constructing decent feature space to distinguish the confusing semantics of different labels; (ii) defining proper sampling criteria to measure a sample's joint effect over the entire label space. To bridge these gaps, we propose a Contrastive Multi-label Active Learning framework (CoMAL) that gives an effective data acquisition strategy. Specifically, a contrastive decoupling mechanism is introduced to fully release the semantic information of multiple labels into the latent space. Then, we devise a hybrid criterion that balances two data value measures: (i) similarity-enhanced label cardinality inconsistency reflects the uncertainty of data predictions. (ii) positive feature diversity evaluates the positive-propensity semantic diversity to handle the label sparsity. Extensive experiments demonstrate that our CoMAL outperforms the current state-of-the-art multi-label active learning approaches. Code for CoMAL is available at https://github.com/chengzju/CoMAL.

As a core problem of online games, matchmaking is to assign players into multiple teams to maximize their gaming experience. With the rapid development of game industry, it is increasingly difficulty to explicitly model players' experiences as linear functions. Instead, it is often modeled in a data-driven way by training a neural network. Meanwhile, complex rules must be satisfied to ensure the robustness of matchmaking, which are often described using logical operators. Therefore, matchmaking in practical scenarios is a challenging combinatorial optimization problem with nonlinear objective, linear constraints and logical constraints, which receives much less attention in previous research. In this paper, we propose a novel deep learning method for high-quality matchmaking in real-time. We first cast the problem as standard mixed-integer programming (MIP) by linearizing ReLU networks and logical constraints. Then, based on supervised learning, we design and train a multi-modal graph learning architecture to predict optimal solutions end-to-end from instance data, and solve a surrogate problem to efficiently obtain feasible solutions. Evaluation results on real industry datasets show that our method can deliver near-optimal solutions within 100ms.

Reinforcement Learning (RL) has demonstrated substantial potential across diverse fields, yet understanding its decision-making process, especially in real-world scenarios where rationality and safety are paramount, is an ongoing challenge. This paper delves in to Explainable RL (XRL), a subfield of Explainable AI (XAI) aimed at unravelling the complexities of RL models. Our focus rests on state-explaining techniques, a crucial subset within XRL methods, as they reveal the underlying factors influencing an agent's actions at any given time. Despite their significant role, the lack of a unified evaluation framework hinders assessment of their accuracy and effectiveness. To address this, we introduce XRL-Bench, a unified standardized benchmark tailored for the evaluation and comparison of XRL methods, encompassing three main modules: standard RL environments, explainers based on state importance, and standard evaluators. XRL-Bench supports both tabular and image data for state explanation. We also propose TabularSHAP, an innovative and competitive XRL method. We demonstrate the practical utility of TabularSHAP in real-world online gaming services and offer an open-source benchmark platform for the straightforward implementation and evaluation of XRL methods. Our contributions facilitate the continued progression of XRL technology.

In recent years, uplift modeling, also known as individual treatment effect (ITE) estimation, has seen wide applications in online marketing, such as delivering one-time issuance of coupons or discounts to motivate users' purchases. However, complex yet more realistic scenarios involving multiple interventions over time on users are still rarely explored. The challenges include handling the bias from time-varying confounders, determining optimal treatment timing, and selecting among numerous treatments. In this paper, to tackle the aforementioned challenges, we present a temporal point process-based uplift model (TPPUM) that utilizes users' temporal event sequences to estimate treatment effects via counterfactual analysis and temporal point processes. In this model, marketing actions are considered as treatments, user purchases as outcome events, and how treatments alter the future conditional intensity function of generating outcome events as the uplift. Empirical evaluations demonstrate that our method outperforms existing baselines on both real-world and synthetic datasets. In the online experiment conducted in a discounted bundle recommendation scenario involving an average of 3 to 4 interventions per day and hundreds of treatment candidates, we demonstrate how our model outperforms current state-of-the-art methods in selecting the appropriate treatment and timing of treatment, resulting in a 3.6% increase in application-level revenue.

Deep Hashing (DH) has emerged as an indispensable technique for fast image search in recent years. To deploy DH on resource-limited devices, the Binary Neural Network (BNN) offers a solution that significantly reduces computations and parameters compared to CNN. Unfortunately, applying BNN directly to DH will lead to huge performance degradation. To tackle this problem, we first conducted extensive experiments and discovered that the center-based method provides a fundamental guarantee for BNN-DH performance. Subsequently, we delved deeper into the impact of BNNs on center-based methods and revealed two key insights. First, we find reducing the distance between hash codes and hash centers is challenging for BNN-DH compared to CNN-based DH. Second, the evolution of hash code aggregation undergoes two stages in BNN-DH, which is different from CNN-based DH. Based on these findings, we designed a strong and general method called One-bit Deep Hashing (ODH). First, ODH incorporates a semantic self-adaptive hash center module to address the problem of hash codes inadequately converging to their hash centers. Then, it employs a novel two-stage training method to consider the evolution of hash code aggregation. Finally, extensive experiments on two datasets demonstrate that ODH can achieve significant superiority over other BNN-DH models.

In the era of big data, data annotation is integral to numerous applications. However, it is widely acknowledged as a laborious and time-consuming process, significantly impeding the scalability and efficiency of data-driven applications. To reduce the human cost, we demonstrate CORAL, a collaborative automatic labeling system driven by large language models (LLMs), which achieves high-quality annotation with the least human effort. Firstly, CORAL employs LLM to automatically annotate vast datasets, generating coarse-grained labels. Subsequently, a weakly-supervised learning module trains small language models (SLMs) using noisy label learning techniques to distill accurate labels from LLM's annotations. It also allows statistical analysis of model outcomes to identify potentially erroneous labels, reducing the human cost of error detection. Furthermore, CORAL supports iterative refinement by LLMs and SLMs using manually corrected labels, thereby ensuring continual enhancement in annotation quality and model performance. A visual interface enables annotation process monitoring and result analysis.

Cheating is widespread in online games, particularly in competitive games like First-Person Shooter (FPS) games. One of the most common types of cheating is Extrasensory Perception (ESP), which involves illicitly obtaining visual information to gain an unfair advantage over normal players. To protect the gaming experience of legitimate players and the interests of game companies, there is an urgent need for anti-cheating applications. In this paper, we propose a general system for ESP anti-cheating in online FPS games, considering the business characteristics and industrial applications. We present a vision-based anti-cheating framework that incorporates both supervised and unsupervised solutions for comprehensive cheating detection. Based on this framework, we design and deploy a dual-audit human-in-the-loop system for industrial gaming anti-cheating applications. We evaluate our proposed framework from multiple online and offline perspectives and demonstrate its practical significance with superior performance.

Recommender system helps address information overload problem and satisfy consumers’ personalized requirement in many applications such as e-commerce, social networks, and in-game store. However, existing approaches mainly focus on improving the accuracy of recommendation tasks but usually ignore how to improve the interpretability of recommendation, which is still a challenging and crucial task, especially for some complicated scenarios such as large-scale online games. A few previous attempts on explainable recommendation mostly depend on a large amount of a priori knowledge or user-provided review corpus, which is labor consuming as well as often suffers from data deficiency. To relieve this issue, we propose a Multi-source Heterogeneous Graph Attention Network for Explainable Recommendation (MHANER) for the case without enough a priori knowledge or corpus of user comments. Specifically, MHANER employs the attention mechanism to model players’ preference to in-game store items as the support for the explanation of recommendation. Then a graph neural network–based method is designed to model players’ multi-source heterogeneous information, including the players’ historical behavior data, historical purchase data, and attributes of the player-controlled character, which is leveraged to recommend possible items for players to buy. Finally, the multi-level subgraph pattern mining is adopted to combine the characteristics of a recommendation list to generate corresponding explanations of items. Extensive experiments on three real-world datasets, two collected from JD and one from NetEase game, demonstrate that the proposed model MHANER outperforms state-of-the-art baselines. Moreover, the generated explanations are verified by human encoding comprised of hard-core game players and endorsed by experts from game developers.

Massive multiplayer online role-playing games (MMORPG) have been becoming one of the most popular and exciting online games. In recent years, a cheating phenomenon called real money trading (RMT) has arisen and damaged the fantasy world in many ways. RMT is the sale of in-game items, currency, or even characters to earn real money, breaking the balance of the game economy ecosystem and damaging the game experience. Therefore, some studies have emerged to address the problem of RMT detection. However, they cannot well handle the label uncertainty problem in practice, where there are only labeled RMT samples (positive samples) and unlabeled samples, which could either be RMT samples or normal transactions (negative samples). Meanwhile, the trading relationship between RMTers is modeled in a simple way, leading to some normal transactions being falsely classified as RMT. In this article, we propose PU-Detector, a novel framework based on PU learning (learning from positive and unlabeled data) for RMT detection, considering the fact that there are only labeled RMT samples and other unlabeled transactions. We first automatically estimate the likelihood of one transaction being RMT by developing an improved PU learning method and proposing an assessment rule. Sequentially, we use the estimated likelihood as edge weight to construct a trading graph to learn trader representation. Then, with the trader representations and basic trading features, we detect RMT samples by the improved PU learning method. PU-Detector is evaluated on a large-scale real world dataset consisting of 33,809,956 transaction logs generated by 43,217 unique players. Compared with other approaches, it achieves the state-of-the-art performance and demonstrates its advantages in detecting underlying RMT samples.

Self-attention models have achieved the state-of-the-art performance in sequential recommender systems by capturing the sequential dependencies among user–item interactions. However, they rely on adding positional embeddings to the item sequence to retain the sequential information, which may break the semantics of item embeddings due to the heterogeneity between these two types of embeddings. In addition, most existing works assume that such dependencies exist solely in the item embeddings, but neglect their existence among the item features. In our previous study, we proposed a novel sequential recommendation model, i.e., MLP4Rec, based on the recent advances of MLP-Mixer architectures, which is naturally sensitive to the order of items in a sequence because matrix elements related to different positions of a sequence will be given different weights in training. We developed a tri-directional fusion scheme to coherently capture sequential, cross-channel, and cross-feature correlations with linear computational complexity as well as much fewer model parameters than existing self-attention methods. However, the cascading mixer structure, the large number of normalization layers between different mixer layers, and the noise generated by these operations limit the efficiency of information extraction and the effectiveness of MLP4Rec. In this extended version, we propose a novel framework – SMLP4Rec for sequential recommendation to address the aforementioned issues. The new framework changes the flawed cascading structure to a parallel mode, and integrates normalization layers to minimize their impact on the model’s efficiency while maximizing their effectiveness. As a result, the training speed and prediction accuracy of SMLP4Rec are vastly improved in comparison to MLP4Rec. Extensive experimental results demonstrate that the proposed method is significantly superior to the state-of-the-art approaches. The implementation code is available online to ease reproducibility.

Collecting high-quality labeled data for model training is notoriously time-consuming and labor-intensive for various NLP tasks.While copious solutions, such as active learning for small language models (SLMs) and prevalent in-context learning in the era of large language models (LLMs), have been proposed and alleviate the labeling burden to some extent, their performances are still subject to human intervention.It is still underexplored how to reduce the annotation cost in the LLMs era.To bridge this, we revolutionize traditional active learning and propose an innovative collaborative learning framework FreeAL to interactively distill and filter the task-specific knowledge from LLMs.During collaborative training, an LLM serves as an active annotator inculcating its coarse-grained knowledge, while a downstream SLM is incurred as a student to filter out high-quality in-context samples to feedback LLM for the subsequent label refinery.Extensive experiments on eight benchmark datasets demonstrate that FreeAL largely enhances the zero-shot performances for both SLM and LLM without any human supervision.The code is available at https: //github.com/Justherozen/FreeAL.

Learning with Noisy Labels (LNL) has become an appealing topic, as imperfectly annotated data are relatively cheaper to obtain. Recent state-of-the-art approaches employ specific selection mechanisms to separate clean and noisy samples and then apply Semi-Supervised Learning (SSL) techniques for improved performance. However, the selection step mostly provides a medium-sized and decent-enough clean subset, which overlooks a rich set of clean samples. To fulfill this, we propose a novel LNL framework ProMix that attempts to maximize the utility of clean samples for boosted performance. Key to our method, we propose a matched high confidence selection technique that selects those examples with high confidence scores and matched predictions with given labels to dynamically expand a base clean sample set. To overcome the potential side effect of excessive clean set selection procedure, we further devise a novel SSL framework that is able to train balanced and unbiased classifiers on the separated clean and noisy samples. Extensive experiments demonstrate that ProMix significantly advances the current state-of-the-art results on multiple benchmarks with different types and levels of noise. It achieves an average improvement of 2.48% on the CIFAR-N dataset.

Faced with saturated market and fierce competition of online games, it is of great value to analyze the causes of the player churn for improving the game product, maintaining the player retention. A large number of research efforts on churn analysis have been made into churn prediction, which can achieve a sound accuracy benefiting from the booming of AI technologies. However, game publishers are usually unable to apply high-accuracy prediction methods in practice for preventing or relieving the churn due to the lack of the specific decision support (e.g., why they leave and what to do next). In this study, we fully exploit the expertise in online games and propose a comprehensive data-driven decision support framework for addressing game player churn. We first define the churn analysis in online games from a commercial perspective and elaborate the core demands of game publishers for churn analysis. Then we employ and improve the cutting-edge eXplainable AI (XAI) methods to predict player churn and analyze the potential churn causes. The possible churn causes can finally guide game publishers to make specific decisions of revision or intervention in our designed procedure. We demonstrate the effectiveness and high practical value of the framework by conducting extensive experiments on a real-world large-scale online game, Justice PC. The whole decision support framework, bringing interesting and valuable insights, also receives quite positive reviews from the game product and operation teams. Notably, the whole pipeline is readily transplanted to other online systems for decision support to address similar issues.

Semi-supervised learning (SSL) has been a fundamental challenge in machine learning for decades. The primary family of SSL algorithms, known as pseudo-labeling, involves assigning pseudo-labels to confident unlabeled instances and incorporating them into the training set. Therefore, the selection criteria of confident instances are crucial to the success of SSL. Recently, there has been growing interest in the development of SSL methods that use dynamic or adaptive thresholds. Yet, these methods typically apply the same threshold to all samples, or use class-dependent thresholds for instances belonging to a certain class, while neglecting instance-level information. In this paper, we propose the study of instance-dependent thresholds, which has the highest degree of freedom compared with existing methods. Specifically, we devise a novel instance-dependent threshold function for all unlabeled instances by utilizing their instance-level ambiguity and the instance-dependent error rates of pseudo-labels, so instances that are more likely to have incorrect pseudo-labels will have higher thresholds. Furthermore, we demonstrate that our instance-dependent threshold function provides a bounded probabilistic guarantee for the correctness of the pseudo-labels it assigns.

Reinforcement learning based recommender systems (RL-based RS) aim at learning a good policy from a batch of collected data, by casting recommendations to multi-step decision-making tasks. However, current RL-based RS research commonly has a large reality gap. In this paper, we introduce the first open-source real-world dataset, RL4RS, hoping to replace the artificial datasets and semi-simulated RS datasets previous studies used due to the resource limitation of the RL-based RS domain. Unlike academic RL research, RL-based RS suffers from the difficulties of being well-validated before deployment. We attempt to propose a new systematic evaluation framework, including evaluation of environment simulation, evaluation on environments, and counterfactual policy evaluation. In summary, the RL4RS (Reinforcement Learning for Recommender Systems), a new resource with special concerns on the reality gaps, contains two real-world datasets, data understanding tools, tuned simulation environments, related advanced RL baselines, batch RL baselines, and counterfactual policy evaluation algorithms. The RL4RS suite can be found at https://github.com/fuxiAIlab/RL4RS.

Sequential recommender systems aim to predict users’ next interested item given their historical interactions. However, a long-standing issue is how to distinguish between users’ long/short-term interests, which may be heterogeneous and contribute differently to the next recommendation. Existing approaches usually set pre-defined short-term interest length by exhaustive search or empirical experience, which is either highly inefficient or yields subpar results. The recent advanced transformer-based models can achieve state-of-the-art performances despite the aforementioned issue, but they have a quadratic computational complexity to the length of the input sequence. To this end, this paper proposes a novel sequential recommender system, AutoMLP, aiming for better modeling users’ long/short-term interests from their historical interactions. In addition, we design an automated and adaptive search algorithm for preferable short-term interest length via end-to-end optimization. Through extensive experiments, we show that AutoMLP has competitive performance against state-of-the-art methods, while maintaining linear computational complexity.

Online gaming is a multibillion dollar industry that entertains a large, global population. Empowering online games with AI has made a great success, however, ignores the explainability of black-box model makes AI less responsible and hinders its further development. In this article, we introduce and discuss the audience and the concept of XAI (eXplainable AI) in online games. We propose a GXAI workflow, which combines the strong expressiveness of multiview data sources and the clear transparency of multiview black-box models. We present four specific classifiers and explainers in the character portrait view, the behavior sequence view, the client image view, and the social graph view. Experiments conducted on real-world datasets for game cheating detection and player churn prediction show the accuracy of classification and the rationality of explanation. We also discover and present numerous interesting and valuable findings from the individual, local, and global explanations. We implement and deploy three practical applications, including evidence and reason generation, model debugging and testing, and model compression and comparison in NetEase Games and have received quite positive reviews from user studies. More future work is in progress since this is the first work that introduces XAI in online games.

Density-based and classification-based methods have ruled unsupervised anomaly detection in recent years, while reconstruction-based methods are rarely mentioned for the poor reconstruction ability and low performance. However, the latter requires no costly extra training samples for the unsupervised training that is more practical, so this paper focuses on improving reconstruction-based method and proposes a novel O mni-frequency C hannel-selection R econstruction (OCR-GAN) network to handle sensory anomaly detection task in a perspective of frequency. Concretely, we propose a Frequency Decoupling (FD) module to decouple the input image into different frequency components and model the reconstruction process as a combination of parallel omni-frequency image restorations, as we observe a significant difference in the frequency distribution of normal and abnormal images. Given the correlation among multiple frequencies, we further propose a Channel Selection (CS) module that performs frequency interaction among different encoders by adaptively selecting different channels. Abundant experiments demonstrate the effectiveness and superiority of our approach over different kinds of methods, e.g., achieving a new state-of-the-art 98.3 detection AUC on the MVTec AD dataset without extra training data that markedly surpasses the reconstruction-based baseline by +38.1 ↑ and the current SOTA method by +0.3 ↑ . The source code is available in the additional materials.

In this paper, we focus on the problem of stable prediction across unknown test data, where the test distribution might be different from the training one and is always agnostic when model training. In such a case, previous machine learning methods might exploit subtly spurious correlations induced by non-causal variables in training data for prediction. Those spurious correlations are changeable across data, leading to instability of prediction across unknown test data. To address this problem, we propose a conditional independence test based algorithm to screen out part of non-causal features and reduce those spurious correlations for a more stable prediction by leveraging a seed variable. We show, both theoretically and with empirical experiments, that our algorithm can precisely screen out the isolated non-causal variables, which have no causal relationship with other variables, and remove the spurious correlations induced by them, increasing the stability of prediction across unknown test data. Extensive experiments on both synthetic and real-world datasets demonstrate that our algorithm outperforms state-of-the-art methods for stable prediction across unknown test data.

In observational studies, confounder separation and balancing are the fundamental problems of treatment effect estimation. Most of the previous methods focused on addressing the problem of confounder balancing by treating all observed pre-treatment variables as confounders, ignoring confounder separation. In general, not all the observed pre-treatment variables are confounders that refer to the common causes of the treatment and the outcome, some variables only contribute to the treatment (i.e., instrumental variables) and some only contribute to the outcome (i.e., adjustment variables). Balancing those non-confounders, including instrumental variables and adjustment variables, would generate additional bias for treatment effect estimation. By modeling the different causal relations among observed pre-treatment variables, treatment variables and outcome variables, we propose a synergistic learning framework to i) separate confounders by learning decomposed representations of both confounders and non-confounders, ii) balance confounder with sample re-weighting technique, and simultaneously iii) estimate the treatment effect in observational studies via counterfactual inference. Empirical results on synthetic and real-world datasets demonstrate that the proposed method can precisely decompose confounders and achieve a more precise estimation of treatment effect than baselines.

In online games, user profiling plays a vital role in a variety of personalized services. Current solutions typically treat different dimensions or labels (e.g., willing to pay or not, high, medium, or low appetite for some gameplays) of the full user profiles as independent multi-class/binary classification tasks. However, such one-by-one profiling strategy clearly overlooks the implicitly correlations among profiling tasks, which results in a degraded performance. To cope with this issue, we make the first attempt to formalize this problem as a multi-label learning task. Accordingly, we develop a unified Multi-Source Multi-Label learning framework~(MSML) that well utilizes semantically rich features and labels for boosted user profiling in online games. Specifically, we first introduce a multi-source user representation network that exploits multi-source data in online games to obtain informative user representations. Subsequently, to handle multiple labels, we propose a novel embedding-based multi-label network that consists of two variational autoencoders with disentangled latent spaces. Note that our framework can guarantee the consistency of the training and testing phases by a novel dual-tower design to overcome the limitation of existing approaches that use one coupled decoder for both features and labels. Extensive experiments on six public multi-label datasets and one real-world online game dataset from Justice demonstrate that the proposed framework outperforms the state-of-the-art baseline methods. Moreover, our proposed framework has been successfully deployed in several online games, yielding a significant boost in multi-label user profiling.

Online games make up the largest segment of the booming global game market in terms of revenue as well as players. Unlike games that sell games at one time for profit, online games make money from in-game purchases by a large number of engaged players. Therefore, Customer Lifetime Value (CLTV) is particularly vital for game companies to improve marketing decisions and increase game revenues. Nowadays, as virtual game worlds are becoming increasingly innovative, complex, and diverse, the CLTV of massive players is highly personalized. That is, different players may have very different patterns of CLTV, especially on churn and payment. However, current solutions are inadequate in terms of personalization and thus limit predictive performance. First, most methods just attempt to address either task of CLTV, i.e., churn or payment, and only consider the personalization from one of them. Second, the correlation between churn and payment has not received enough attention and its personalization has not been fully explored yet. Last, most solutions around this line are conducted based on historical data where the evaluation is not convincing enough without real-world tests. To tackle these problems, we propose a general system to predict personalized customer lifetime value in online games, named perCLTV. To be specific, we revisit the personalized CLTV prediction problem from the two sub-tasks of churn prediction and payment prediction in a sequential gated multi-task learning fashion. On this basis, we develop a generalized framework to model CLTV across games in distinct genres by heterogeneous player behavior data, including individual behavior sequential data and social behavior graph data. Comprehensive experiments on three real-world datasets validate the effectiveness and rationality of perCLTV, which significantly outperforms other baseline methods. Our work has been implemented and deployed in many online games released from NetEase Games. Online A/B testing in production shows that perCLTV achieves a prominent improvement in two precision marketing applications of popup recommendation and churn intervention.

As an independent social and economic entity, game servers plays a dominant role in building a stable, living, and attractive virtual world in massive multi-player online role-playing games (MMORPGs). We propose and implement a novel intelligent decision support system for server merge (SM) for maintaining the game ecology at the macro level. The services provided by this system include server health diagnosis, server merge assessment, and combination strategy recommendation. Specifically, we design an effective time series prediction algorithm to diagnose the health status of one server (e.g., user activity, online time, daily revenue) based on real game scenarios, and then select the servers with poor status from all servers. Moreover, to dig out the inherent development laws of servers from the historical merge records, we leverage a correlation measurement algorithm to find the historical merged servers that are similar to the servers to be merged and then evaluate the potential trend after merging, which can assist experts to make reasonable decisions. We deploy our system into practice for multiple MMORPGs and achieve sound online performance endorsed by the game operation team.

Market popularity prediction has always been a hot research topic, such as sales prediction and crowdfunding prediction. Most of these studies put the perspective on isolated markets, relying on the knowledge of certain market to maximize the prediction performance. However, these market-specific approaches are restricted by the knowledge limitation of isolated markets and incapable of the complicated and potential relations among different markets, especially some with strong dependence such as the financing market and sales market. Fortunately, we discover potentially symbiotic relations between the financing market and the sales market, which provides us with an opportunity to co-promote the popularity predictions of both markets. Thus, for bridgly learning the knowledge interactions between financing market and sales market, we propose a cross-market approach, namely CATN: Cooperative-competitive Attention Transfer Network, which could effectively transfer knowledge of financing capability from the crowdfunding market and sales prospect from the E-commerce market. Specifically, for capturing the complicated relations especially the cooperation or complement of items and enhancing the knowledge transfer between the two heterogeneous markets, we design a novel Cooperative Attention; meanwhile, for finely computing the relations of items especially the competition in specific same market, we further design Competitive Attentions for the two markets respectively. Besides, we also distinguish aligned features and unique features to adapt the cross-market predictions. With the real-world datasets collected from Indiegogo and Amazon, we construct extensive experiments on three types of datasets from the two markets and the results demonstrate the effectiveness and generalization of our CATN model.

Guild is the most important long-term virtual community and emotional bond in massively multiplayer online role-playing games (MMORPGs). It matters a lot to the player retention and game ecology how the guilds are going, e.g., healthy or not. The main challenge now is to characterize and predict the guild health in a quantitative, dynamic, and multi-dimensional manner based on complicated multi-media data streams. To this end, we propose a novel framework, namely Stability-Aware Multi-task Learning Approach(SAMLA) to address these challenges. Specifically, different media-specific modules are designed to extract information from multiple media types of tabular data, time seriescharacteristics, and heterogeneous graphs. To capture the dynamics of guild health, we introduce a representation encoder to provide a time series view of multi-media data that is used for task prediction. Inspiredby well-received theories on organization management, we delicately define five specific and quantitative dimensions of guild health and make parallel predictions based on a multi-task approach. Besides, we devise a novel auxiliary task, i.e.,the guild stability, to boost the performance of the guild health prediction task. Extensive experiments on a real-world large-scale MMORPG dataset verify that our proposed method outperforms the state-of-the-art methods in the task of organizational health characterization and prediction. Moreover, our work has been practically deployed in online MMORPG, and case studies clearly illustrate the significant value.

Game bots are programmed to automatically play games and illegally obtain profit, seriously affecting game experience of honest players and breaking the balance of game ecosystem. Therefore, bot detection needs to be addressed urgently, especially for MMORPGs, one of the most rapidly expanding genres of games. There have been some studies for bot detection, but the features they used are dependent on specific games and the methods cannot be generalized to other games. In this paper, we propose a trajectory based pre-trained model for game bot detection from game character trajectories and mouse trajectories, named T-Detector, which is independent to specific games and can be generalized to others. More specifically, we propose a pretrain method of LocationTime2Vec to learn representations of trajectories from huge unlabeled samples, which deeply embed spatial and temporal information hidden in trajectories. Moreover, we extract universal features based on behavioral differences in movement trajectories between human players and bots. We design an Angle Pretrain to extract features of turning angle, and propose an attention pooling module to extract features of moving speed and distance. Such features are not dependent on any specific game, enabling T-Detector to be generalized to many MMORPGs. Evaluated by two large-scale real-world datasets of 143,938 samples from two MMORPGs, T-Detector achieves the state-of-the-art performance in bot detection, and demonstrates powerful generalization ability.

Self-attention models have achieved state-of-the-art performance in sequential recommender systems by capturing the sequential dependencies among user-item interactions. However, they rely on positional embeddings to retain the sequential information, which may break the semantics of item embeddings. In addition, most existing works assume that such sequential dependencies exist solely in the item embeddings, but neglect their existence among the item features. In this work, we propose a novel sequential recommender system (MLP4Rec) based on the recent advances of MLP-based architectures, which is naturally sensitive to the order of items in a sequence. To be specific, we develop a tri-directional fusion scheme to coherently capture sequential, cross-channel and cross-feature correlations. Extensive experiments demonstrate the effectiveness of MLP4Rec over various representative baselines upon two benchmark datasets. The simple architecture of MLP4Rec also leads to the linear computational complexity as well as much fewer model parameters than existing self-attention methods.

With the emerging of smartphones, mobile games have attracted billions of players and occupied most of the share for game companies. On the other hand, mobile game cheating, aiming to gain improper advantages by using programs that simulate the players' inputs, severely damages the game's fairness and harms the user experience. Therefore, detecting mobile game cheating is of great importance for mobile game companies. Many PC game-oriented cheating detection methods have been proposed in the past decades, however, they can not be directly adopted in mobile games due to the concern of privacy, power, and memory limitations of mobile devices. Even worse, in practice, the cheating programs are quickly updated, leading to the label scarcity for novel cheating patterns. To handle such issues, we in this paper introduce a mobile game cheating detection framework, namely FCDGame, to detect the cheats under the few-shot learning framework. FCDGame only consumes the screen sensor data, recording users' touch trajectories, which is less sensitive and more general for almost all mobile games. Moreover, a Hierarchical Trajectory Encoder and a Cross-pattern Meta Learner are designed in FCDGame to capture the intrinsic characters of mobile games and solve the label scarcity problem, respectively. Extensive experiments on two real online games show that FCDGame achieves almost 10% improvements in detection accuracy with only few fine-tuned samples.

Recent years have witnessed the great accuracy performance of graph-based Collaborative Filtering (CF) models for recommender systems. By taking the user-item interaction behavior as a graph, these graph-based CF models borrow the success of Graph Neural Networks (GNN), and iteratively perform neighborhood aggregation to propagate the collaborative signals. While conventional CF models are known for facing the challenges of the popularity bias that favors popular items, one may wonder "Whether the existing graph-based CF models alleviate or exacerbate the popularity bias of recommender systems?" To answer this question, we first investigate the two-fold performances w.r.t. accuracy and novelty for existing graph-based CF methods. The empirical results show that symmetric neighborhood aggregation adopted by most existing graph-based CF models exacerbates the popularity bias and this phenomenon becomes more serious as the depth of graph propagation increases. Further, we theoretically analyze the cause of popularity bias for graph-based CF. Then, we propose a simple yet effective plugin, namely r-AdjNorm, to achieve an accuracy-novelty trade-off by controlling the normalization strength in the neighborhood aggregation process. Meanwhile, r-AdjNorm can be smoothly applied to the existing graph-based CF backbones without additional computation. Finally, experimental results on three benchmark datasets show that our proposed method can improve novelty without sacrificing accuracy under various graph-based CF backbones.

This paper studies the task of detecting bots for online mobile games. Considering the fact of lacking labeled cheating samples and restricted available data in the real detection systems, we aim to study the finger operations captured by screen sensors to infer the potential bots in an unsupervised way. In detail, we introduce a Transformer-style detection model, namely FingFormer. It studies the finger operations in the format of graph structure in order to capture the spatial and temporal relatedness between the two hands’ operations. To optimize the model in an unsupervised way, we introduce two contrastive learning strategies to refine both finger moving patterns and players’ operation habits. We conduct extensive experiments under different experimental environments, including the synthetic dataset, the offline dataset, as well as the large-scale online data flow from three mobile games. The multi-facet experiments illustrate the proposed model is both effective and general to detect the bots for different mobile games.

Players of online games generate rich behavioral data during gaming. Based on these data, game developers can build a range of data science applications, such as bot detection and social recommendation, to improve the gaming experience. However, the development of such applications requires data cleansing, training sample labeling, feature engineering, and model development, which makes the use of such applications in small and medium-sized game studios still uncommon. While acquiring supervised learning data is costly, unlabeled behavioral logs are often continuously and automatically generated in games. Thus we resort to unsupervised representation learning of player behavioral data to optimize intelligent services in games. Behavioral data has many unique properties, including semantic complexity, excessive length, etc. A worth noting property within raw player behavioral data is that a lot of it is task-irrelevant. For these data characteristics, we introduce a BPE-enhanced compression method and propose a novel adaptive masking strategy called Masking by Token Confidence (MTC) for the Masked Language Modeling (MLM) pre-training task. MTC is designed to increase the masking probabilities of task-relevant tokens. Experiments on four downstream tasks and successful deployment in a world-renowned Massively Multiplayer Online Role-Playing Game (MMORPG) prove the effectiveness of the MTC strategy1.

With an increasing popularity, Multiplayer Online Battle Arena (MOBA) games where two opposing teams compete against each other, have played a major role in E-sports tournaments. Among game analysis, real-time winning prediction is an important but challenging problem, which is mainly due to the complicated coupling of the overall Confrontation1, the excessive noise of the player’s Movement, and unclear optimization goals. Existing research is difficult to solve this problem in a dynamic, comprehensive and systematic way. In this study, we design a unified framework, namely Winning Tracker (WT), for solving this problem. Specifically, offense and defense extractors are developed to extract the Confrontation of both sides. A well-designed trajectory representation algorithm is applied to extracting individual’s Movement information. Moreover, we design a hierarchical attention mechanism to capture team-level strategies and facilitate the interpretability of the framework. To optimize accurately, we adopt a multi-task learning method to design short-term and long-term goals, which are used to represent immediate state and make end-state prediction respectively. Intensive experiments on a real-world data set demonstrate that our proposed method WT outperforms state-of-the-art algorithms. Furthermore, our work has been practically deployed in real MOBA games, and provided case studies reflecting its outstanding commercial value.

Social relationships are the basis for communication and collaboration between players in many online games. In this article, we propose a machine-learning-based approach to model the relationship between players and guilds in online games. Our approach combines deep learning techniques with useful prior expert knowledge, where the core component is a graph convolutional network that is designed to utilize both social relationships and behavior preferences of players. For each player in the game, the model is trained to estimate the likelihood of whether the player matches the guild, which enables rapid matching of players and guilds via recommendation. The proposed approach is evaluated on an industrial dataset collected from a popular online game and also deployed in the game as a basic component of the social system. Experimental results show that our approach is not only intuitive but also very superior to other baseline methods.

In data mining and machine learning, it is commonly assumed that training and test data share the same population distribution. However, this assumption is often violated in practice because of the sample selection bias, which might induce the distribution shift from training data to test data. Such a model-agnostic distribution shift usually leads to prediction instability across unknown test data. This article proposes a novel balance-subsampled stable prediction (BSSP) algorithm based on the theory of fractional factorial design. It isolates the clear effect of each predictor from the confounding variables. A design-theoretic analysis shows that the proposed method can reduce the confounding effects among predictors induced by the distribution shift, improving both the accuracy of parameter estimation and the stability of prediction across unknown test data. Numerical experiments on synthetic and real-world datasets demonstrate that our BSSP algorithm can significantly outperform the baseline methods for stable prediction across unknown test data.

Instrumental variables (IVs), sources of treatment randomization that are conditionally independent of the outcome, play an important role in causal inference with unobserved confounders. However, the existing IV-based counterfactual prediction methods need well-predefined IVs, while it’s an art rather than science to find valid IVs in many real-world scenes. Moreover, the predefined hand-made IVs could be weak or erroneous by violating the conditions of valid IVs. These thorny facts hinder the application of the IV-based counterfactual prediction methods. In this article, we propose a novel Automatic Instrumental Variable decomposition (AutoIV) algorithm to automatically generate representations serving the role of IVs from observed variables (IV candidates). Specifically, we let the learned IV representations satisfy the relevance condition with the treatment and exclusion condition with the outcome via mutual information maximization and minimization constraints, respectively. We also learn confounder representations by encouraging them to be relevant to both the treatment and the outcome. The IV and confounder representations compete for the information with their constraints in an adversarial game, which allows us to get valid IV representations for IV-based counterfactual prediction. Extensive experiments demonstrate that our method generates valid IV representations for accurate IV-based counterfactual prediction.

In recent years, advances in Graph Convolutional Networks (GCNs) have given new insights into the development of social recommendation. However, many existing GCN-based social recommendation methods often directly apply GCN to capture user-item and user-user interactions, which probably have two main limitations: (a) Due to the power-law property of the degree distribution, the vanilla GCN with static normalized adjacency matrix has limitations in learning node representations, especially for the long-tail nodes; (b) multi-typed social relationships between users that are ubiquitous in the real world are rarely considered. In this article, we propose a novel Bilateral Filtering Heterogeneous Attention Network (BFHAN), which improves long-tail node representations and leverages multi-typed social relationships between user nodes. First, we propose a novel graph convolutional filter for the user-item bipartite network and extend it to the user-user homogeneous network. Further, we theoretically analyze the correlation between the convergence values of different graph convolutional filters and node degrees after stacking multiple layers. Second, we model multi-relational social interactions between users as the multiplex network and further propose a multiplex attention network to capture distinctive inter-layer influences for user representations. Last but not least, the experimental results demonstrate that our proposed method outperforms several state-of-the-art GCN-based methods for social recommendation tasks.

Match outcome prediction in group comparison setting is a challenging but important task. Existing works mainly focus on learning individual effects or mining limited interactions between teammates, which is not sufficient for capturing complex interactions between teammates as well as between opponents. Besides, the importance of interacting with different characters is still largely underexplored. To this end, we propose a novel Neural Attentional Cooperation-competition model (NeuralAC), which incorporates weighted-cooperation effects (i.e., intra-team interactions) and weighted-competition effects (i.e., inter-team interactions) for predicting match outcomes. Specifically, we first project individuals to latent vectors and learn complex interactions through deep neural networks. Then, we design two novel attention-based mechanisms to capture the importance of intra-team and inter-team interactions, which enhance NeuralAC with both accuracy and interpretability. Furthermore, we demonstrate NeuralAC can generalize several previous works. To evaluate the performances of NeuralAC, we conduct extensive experiments on four E-sports datasets. The experimental results clearly verify the effectiveness of NeuralAC compared with several state-of-the-art methods.

In recent years, there are great interests as well as many challenges in applying reinforcement learning (RL) to recommendation systems (RS). In this paper, we summarize three key practical challenges of large-scale RL-based recommender systems: massive state and action spaces, high-variance environment, and the unspecific reward setting in recommendation. All these problems remain largely unexplored in the existing literature and make the application of RL challenging. We develop a model-based reinforcement learning framework, called GoalRec. Inspired by the ideas of world model (model-based), value function estimation (model-free), and goal-based RL, a novel disentangled universal value function designed for item recommendation is proposed. It can generalize to various goals that the recommender may have, and disentangle the stochastic environmental dynamics and high-variance reward signals accordingly. As a part of the value function, free from the sparse and high-variance reward signals, a high-capacity reward-independent world model is trained to simulate complex environmental dynamics under a certain goal. Based on the predicted environmental dynamics, the disentangled universal value function is related to the user's future trajectory instead of a monolithic state and a scalar reward. We demonstrate the superiority of GoalRec over previous approaches in terms of the above three practical challenges in a series of simulations and a real application.

Most cognitive diagnosis research in education has been concentrated on individual assessment, aiming at discovering the latent characteristics of students. However, in many real-world scenarios, group-level assessment is an important and meaningful task, e.g., class assessment in different regions can discover the difference of teaching level in different contexts. In this work, we consider assessing cognitive ability for a group of students, which aims to mine groups’ proficiency on specific knowledge concepts. The significant challenge in this task is the sparsity of group-exercise response data, which seriously affects the assessment performance. Existing works either do not make effective use of additional student-exercise response data or fail to reasonably model the relationship between group ability and individual ability in different learning contexts, resulting in sub-optimal diagnosis results. To this end, we propose a general Multi-Task based Group-Level Cognitive Diagnosis (MGCD) framework, which is featured with three special designs: 1) We jointly model student-exercise responses and group-exercise responses in a multi-task manner to alleviate the sparsity of group-exercise responses; 2) We design a context-aware attention network to model the relationship between student knowledge state and group knowledge state in different contexts; 3) We model an interpretable cognitive layer to obtain student ability, group ability and exercise factors (e.g., difficulty), and then we leverage neural networks to learn complex interaction functions among them. Extensive experiments on real-world datasets demonstrate the generality of MGCD and the effectiveness of our attention design and multi-task learning.

As one of the core components of online games, matchmaking is the process of arranging multiple players into matches, where the quality of matchmaking systems directly determines player satisfaction and further affects the life cycle of game products. With the number of candidate players increases, the number of possible match combinations grows exponentially, which makes the current implementation for multiplayer matchmaking can only obtain locally optimal arrangement in an inefficient fashion. In this paper, we focus on the globally optimized matchmaking problem, in which the objective is to decide an optimal matching sequence for the queuing players. To tackle this challenging problem, we propose a novel data-driven matchmaking framework, called GloMatch, based on machine learning principles. Through transforming the matchmaking problem into a sequential decision problem, we solve it with the help of an effective policy-based deep reinforcement learning algorithm. Quantitative experiments on simulation and online game environments demonstrate the effectiveness of the presented framework.

In the business domain,bundling is one of the most important marketing strategies to conduct product promotions, which is commonly used in online e-commerce and offline retailers. Existing recommender systems mostly focus on recommending individual items that users may be interested in, such as the considerable research work on collaborative filtering that directly models the interaction between users and items. In this paper, we target at a practical but less explored recommendation problem named personalized bundle composition, which aims to offer an optimal bundle (i.e., a combination of items) to the target user. To tackle this specific recommendation problem, we formalize it as a combinatorial optimization problem on a set of candidate items and solve it within a neural combinatorial optimization framework. Extensive experiments on public datasets are conducted to demonstrate the superiority of the proposed method.

Online gaming is one of the most successful applications having a large number of players interacting in an online persistent virtual world through the Internet. However, some cheating players gain improper advantages over normal players by using illegal automated plugins which has brought huge harm to game health and player enjoyment. Game industries have been devoting much efforts on cheating detection with multiview data sources and achieved great accuracy improvements by applying artificial intelligence (AI) techniques. However, generating explanations for cheating detection from multiple views still remains a challenging task. To respond to the different purposes of explainability in AI models from different audience profiles, we propose the EMGCD, the first explainable multi-view game cheating detection framework driven by explainable AI (XAI). It combines cheating explainers to cheating classifiers from different views to generate individual, local and global explanations which contributes to the evidence generation, reason generation, model debugging and model compression. The EMGCD has been implemented and deployed in multiple game productions in NetEase Games, achieving remarkable and trustworthy performance. Our framework can also easily generalize to other types of related tasks in online games, such as explainable recommender systems, explainable churn prediction, etc.

Profiling game players, especially potential churn and payment prediction, is of paramount importance for online games to improve the product design and the revenue. However, current solutions view either churn or payment prediction as an independent task and most of the previous attempts only depend on the single data source, i.e., the tabular portrait data. Based on the data of two real-world online games, we conduct extensive data analysis. On the one hand, there exists a significant correlation between the player churn and payment. On the other hand, heterogeneous multi-source data, including player portrait, behavior sequence, and social network, can complement each other for a better understanding of each player. To this end, we propose a novel Multi-source Data Multi-task Learning approach, named MSDMT, to capture the multi-source implicit information and predict the churn and payment of each player simultaneously in a multi-task learning fashion. Comprehensive experiments on two real-world datasets validate the effectiveness and rationality of our proposed method, which yields significant improvements against other baseline approaches.

In business domains, bundling is one of the most important marketing strategies to conduct product promotions, which is commonly used in online e-commerce and offline retailers. Existing recommender systems mostly focus on recommending individual items that users may be interested in. In this paper, we target at a practical but less explored recommendation problem named bundle recommendation, which aims to offer a combination of items to users. To tackle this specific recommendation problem in the context of the virtual mall in online games, we formalize it as a link prediction problem on a user-item-bundle tripartite graph constructed from the historical interactions, and solve it with a neural network model that can learn directly on the graph-structure data. Extensive experiments on three public datasets and one industrial game dataset demonstrate the effectiveness of the proposed method. Further, the bundle recommendation model has been deployed in production for more than one year in a popular online game developed by Netease Games, and the launch of the model yields more than 60% improvement on conversion rate of bundles, and a relative improvement of more than 15% on gross merchandise volume (GMV).

Win prediction and performance evaluation are two core subjects in the sport analytics. Traditionally, they are treated separately and studied by two independent communities. However, this is not the intuitive way how humans interpret the matches: we predict the match results with the competition carrying on, and simultaneously evaluate each action based on the game context and its downstream impact. Predicting the match outcomes and evaluating the actions are coupled tasks, and the more accurately we predict, the better the evaluation is

Behavior tracing or predicting is a key component in various application scenarios like online user modeling and ubiquitous computing, which significantly benefits the system design (e.g., resource pre-caching) and improves the user experience (e.g., personalized recommendation). Traditional behavior tracing methods like Markovian and sequential models take recent behaviors as input and infer the next move by using the most real-time information. However, these existing methods rarely comprehensively model the low-level temporal irregularity in the recent behavior sequence, i.e., the unevenly distributed time intervals between consecutive behaviors, and the high-level periodicity in the long-term activity cycle, i.e., the periodic behavior patterns of each user.

Matchmaking is a core problem for the e-sports and online games, which determines the player satisfaction and further influences the life cycle of the gaming products. Most of matchmaking systems take the form of grouping the queuing players into two opposing teams by following certain rules. The design and implementation of matchmaking systems are usually product-specific and labor-intensive.

Online gaming is a multi-billion dollar industry that entertains a large, global population. However, one unfortunate phenomenon known as real money trading harms the competition and the fun. Real money trading is an interesting economic activity used to exchange assets in a virtual world with real world currencies, leading to imbalance of game economy and inequality of wealth and opportunity. Game operation teams have been devoting much efforts on real money trading detection, however, it still remains a challenging task. To overcome the limitation from traditional methods conducted by game operation teams, we propose, MVAN, the first multi-view attention networks for detecting real money trading with multi-view data sources. We present a multi-graph attention network (MGAT) in the graph structure view, a behavior attention network (BAN) in the vertex content view, a portrait attention network (PAN) in the vertex attribute view and a data source attention network (DSAN) in the data source view. Experiments conducted on real-world game logs from a commercial NetEase MMORPG( JusticePC) show that our method consistently performs promising results compared with other competitive methods over time and verifiy the importance and rationality of attention mechanisms. MVAN is deployed to several MMORPGs in NetEase in practice and achieving remarkable performance improvement and acceleration. Our method can easily generalize to other types of related tasks in real world, such as fraud detection, drug tracking and money laundering tracking etc.

Collaborative filtering (CF), particularly matrix factorization (MF) based methods, have been widely used in recommender systems. The literature has reported that matrix factorization methods often produce superior accuracy of rating prediction in recommender systems. However, existing matrix factorization methods rarely consider confidence of the rating prediction and thus cannot support advanced recommendation tasks. In this paper, we propose a Confidence-aware Matrix Factorization (CMF) framework to simultaneously optimize the accuracy of rating prediction and measure the prediction confidence in the model. Specifically, we introduce variance parameters for both users and items in the matrix factorization process. Then, prediction interval can be computed to measure confidence for each predicted rating. These confidence quantities can be used to enhance the quality of recommendation results based on Confidence-aware Ranking (CR). We also develop two effective implementations of our framework to compute the confidence-aware matrix factorization for large-scale data. Finally, extensive experiments on three real-world datasets demonstrate the effectiveness of our framework from multiple perspectives.

Recent decades have witnessed the rapid growth of educational data mining (EDM), which aims at automatically extracting valuable information from large repositories of data generated by or related to people’s learning activities in educational settings. One of the key EDM tasks is cognitive modelling with examination data, and cognitive modelling tries to profile examinees by discovering their latent knowledge state and cognitive level (e.g. the proficiency of specific skills). However, to the best of our knowledge, the problem of extracting information from both objective and subjective examination problems to achieve more precise and interpretable cognitive analysis remains underexplored. To this end, we propose a fuzzy cognitive diagnosis framework (FuzzyCDF) for examinees’ cognitive modelling with both objective and subjective problems. Specifically, to handle the partially correct responses on subjective problems, we first fuzzify the skill proficiency of examinees. Then we combine fuzzy set theory and educational hypotheses to model the examinees’ mastery on the problems based on their skill proficiency. Finally, we simulate the generation of examination score on each problem by considering slip and guess factors. In this way, the whole diagnosis framework is built. For further comprehensive verification, we apply our FuzzyCDF to three classical cognitive assessment tasks, i.e., predicting examinee performance, slip and guess detection, and cognitive diagnosis visualization. Extensive experiments on three real-world datasets for these assessment tasks prove that FuzzyCDF can reveal the knowledge states and cognitive level of the examinees effectively and interpretatively.

Diagnosing students' knowledge proficiency, i.e., the mastery degrees of a particular knowledge point in exercises, is a crucial issue for numerous educational applications, e.g., targeted knowledge training and exercise recommendation. Educational theories have converged that students learn and forget knowledge from time to time. Thus, it is necessary to track their mastery of knowledge over time. However, traditional methods in this area either ignored the explanatory power of the diagnosis results on knowledge points or relied on a static assumption. To this end, in this paper, we devise an explanatory probabilistic approach to track the knowledge proficiency of students over time by leveraging educational priors. Specifically, we first associate each exercise with a knowledge vector in which each element represents an explicit knowledge point by leveraging educational priors (i.e., Q-matrix ). Correspondingly, each student is represented as a knowledge vector at each time in a same knowledge space. Second, given the student knowledge vector over time, we borrow two classical educational theories (i.e., Learning curve and Forgetting curve ) as priors to capture the change of each student's proficiency over time. After that, we design a probabilistic matrix factorization framework by combining student and exercise priors for tracking student knowledge proficiency. Extensive experiments on three real-world datasets demonstrate both the effectiveness and explanatory power of our proposed model.