Multi-task learning (MTL) is widely utilized across a variety of real-world applications, including recommendation systems. For instance, in the field of e-commerce, MTL is commonly employed to simultaneously model click, conversion, and user dwelling time. Among a various of MTL models, the Multi-gate Mixture-of-Experts (MMoE) has gained significant popularity. However, MMoE suffers from the polarization issue during training, where the weights of certain experts tend to converge towards 0. To address this issue, we propose a novel method called Bagging-Expert network (BEnet) for multi-task learning. BEnet effectively mitigates the problem of polarization and achieves excellent performance in multi-task learning. It incorporates a bagging layer and an attention mechanism to encourage experts focusing on diverse knowledge domains. Simultaneously, polarization is avoided as different experts execute respective duties and specialize in distinct domains. Experimental results on real-world datasets demonstrate that BEnet has strong robustness and outperforms other state-of-the-art (SOTA) MTL methods.

Accurate prediction of coupon usage is crucial for promoting user consumption through targeted coupon recommendations. However, in real-world coupon recommendations, the coupon allocation process is not solely determined by the model trained with the history interaction data but is also interfered with by marketing tactics desired to fulfill specific commercial goals.This interference creates an imbalance in the interactions, which causes the data to deviate from the user's natural preferences. We refer to this deviation as the matching bias. Such biased interaction data affects the efficacy of the model, and thus it is necessary to employ debiasing techniques to prevent any negative impact. We investigate the mitigation of matching bias in coupon recommendations from a causal-effect perspective. By treating the attributes of users and coupons associated with marketing tactics as confounders, we find the confounders open the backdoor path between user-coupon matching and the conversion, which introduces spurious correlation. To remove the bad effect, we propose a novel training paradigm named Backdoor Adjustment via Group Adaptation (BAGA) for debiased coupon recommendations, which performs intervened training and inference, i.e., separately modeling each user-coupon group pair. However, modeling all possible group pairs greatly increases the computational complexity and cost. To address the efficiency challenge, we further present a simple but effective dual-tower multi-task framework and leverage the Customized Gate Control (CGC) model architecture, which separately models each user and coupon group with a separate expert module. We instantiate BAGA on five representative models: FM, DNN, NCF, MASKNET, and DEEPFM, and conduct comprehensive offline and online experiments to demonstrate the efficacy of our proposed paradigm.

Graph-based Recommendation Systems (GRSs) have gained prominence for their ability to enhance the accuracy and effectiveness of recommender systems by exploiting structural relationships in user-item interaction data. Despite their advanced capabilities, we find GRSs are susceptible to feedback-loop phenomena that disproportionately diminish the visibility of new and long-tail items, leading to a homogenization of recommendations and the potential emergence of echo chambers. To mitigate this feedback-loop issue, exploration and exploitation (E&E) strategies have been extensively researched. However, conventional E&E methods rest on the assumption that recommendations are independent and identically distributed-an assumption that is not valid for GRSs. To forge an effective E&E approach tailored to GRSs, we introduce a novel framework, the GRADient-informed Exploration and Exploitation (GRADE), designed to adaptively seek out underrepresented or new items with promising rewards. Our method evaluates the potential benefit of exploring an item by assessing the change in the system's empirical risk error pre- and post-exposure. For practical implementation, we approximate this measure using the gradients of potential edges and model parameters, alongside their associated uncertainties. We then orchestrate the balance between exploration and exploitation utilizing Thompson sampling and the Upper Confidence Bound (UCB) strategy. Empirical tests on datasets from two industrial environments demonstrate that GRADE consistently outperforms existing state-of-the-art methods. Additionally, our approach has been successfully integrated into actual industrial systems.

Recommendation systems are widely used in e-commerce websites and online platforms to address information overload. However, existing systems primarily rely on historical data and user feedback, making it difficult to capture user intent transitions. Recently, Knowledge Base (KB)-based models are proposed to incorporate expert knowledge, but it struggle to adapt to new items and the evolving e-commerce environment. To address these challenges, we propose a novel Large Language Model based Complementary Knowledge Enhanced Recommendation System (LLM-KERec). It introduces an entity extractor that extracts unified concept terms from item and user information. To provide cost-effective and reliable prior knowledge, entity pairs are generated based on entity popularity and specific strategies. The large language model determines complementary relationships in each entity pair, constructing a complementary knowledge graph. Furthermore, a new complementary recall module and an Entity-Entity-Item (E-E-I) weight decision model refine the scoring of the ranking model using real complementary exposure-click samples. Extensive experiments conducted on three industry datasets demonstrate the significant performance improvement of our model compared to existing approaches. Additionally, detailed analysis shows that LLM-KERec enhances users' enthusiasm for consumption by recommending complementary items. In summary, LLM-KERec addresses the limitations of traditional recommendation systems by incorporating complementary knowledge and utilizing a large language model to capture user intent transitions, adapt to new items, and enhance recommendation efficiency in the evolving e-commerce landscape.

In marketing recommendations, the campaign organizers will distribute coupons to users to encourage consumption. In general, a series of strategies are employed to interfere with the coupon distribution process, leading to a growing imbalance between user-coupon interactions, resulting in a bias in the estimation of conversion probabilities. We refer to the estimation bias as the matching bias. In this paper, we explore how to alleviate the matching bias from the causal-effect perspective. We regard the historical distributions of users and coupons over each other as confounders and characterize the matching bias as a confounding effect to reveal and eliminate the spurious correlations between user-coupon representations and conversion probabilities. Then we propose a new training paradigm named De-Matching Bias Recommendation (DMBR) to remove the confounding effects during model training via the backdoor adjustment. We instantiate DMBR on two representative models: DNN and MMOE, and conduct extensive offline and online experiments to demonstrate the effectiveness of our proposed paradigm.

There exists the cold-start problem in the recommendation systems when observed user-item interactions are insufficient. To alleviate this problem, most existing works aim to learn globally shared prior knowledge across all items and be fast adapted to a new item with few interactions. However, such learning techniques are data demanding and work poorly on new items with no interactions. In this applied paper, we present an industrial framework recently deployed on Alipay to address the item cold-start problem in zero-shot scenarios. The proposed framework provides both efficient and high-quality recommendations for cold items with no log data. Specifically, we formulate the cold-start problem as a zero-shot learning problem and build a highly efficient infrastructure to accomplish online zero-shot recommendations used on large-scale platforms. Extensive offline experiments and online A/B testing demonstrate that the proposed framework has superior performance and recommends cold items to preferred users more effectively than other state-of-the-art methods.

Distributed sparse learning with a cluster of multiple machines has attracted much attention in machine learning, especially for large-scale applications with high-dimensional data. One popular way to implement sparse learning is to use L1 regularization. In this paper, we propose a novel method, called proximal SCOPE (pSCOPE), for distributed sparse learning with L1 regularization. pSCOPE is based on a cooperative autonomous local learning (CALL) framework. In the CALL framework of pSCOPE, we find that the data partition affects the convergence of the learning procedure, and subsequently we define a metric to measure the goodness of a data partition. Based on the defined metric, we theoretically prove that pSCOPE is convergent with a linear convergence rate if the data partition is good enough. We also prove that better data partition implies faster convergence rate. Furthermore, pSCOPE is also communication efficient. Experimental results on real data sets show that pSCOPE can outperform other state-of-the-art distributed methods for sparse learning.

Stochastic gradient descent (SGD) and its variants have attracted much attention in machine learning due to their efficiency and effectiveness for optimization. To handle large-scale problems, researchers have recently proposed several lock-free strategy based parallel SGD (LF-PSGD) methods for multi-core systems. However, existing works have only proved the convergence of these LF-PSGD methods for convex problems. To the best of our knowledge, no work has proved the convergence of the LF-PSGD methods for non-convex problems. In this paper, we provide the theoretical proof about the convergence of two representative LF-PSGD methods, Hogwild! and AsySVRG, for non-convex problems. Empirical results also show that both Hogwild! and AsySVRG are convergent on non-convex problems, which successfully verifies our theoretical results.