In recent years, there has been a trend in the field of recommender systems towards multi-task modeling and multi-scenario modeling. The aim is to enhance the performance of various tasks and scenarios by jointly training on multiple tasks or scenarios to learn common patterns and features. Joint modeling of tasks and scenarios has also received widespread attention recently. However, despite the rich proposals of methods for Multi-Task Learning (MTL), Multi-Scenario Learning (MSL), and Multi-Task-Multi-Scenario Learning (MTMSL) in recent years, there still lacks a comprehensive benchmark to evaluate these methods. Previous studies often employed different datasets, data processing techniques, data partitioning strategies, and hyperparameter settings, making replication of existing research and fair comparison of experimental results challenging. To address this challenge, we introduce MMLRec, the first unified comprehensive benchmark for evaluating MTL, MSL and MTMSL, featuring consistent dataset processing and identical parameter settings. This benchmark implements a range of MTL, MSL, and MTMSL algorithms, and evaluates them on multiple commonly used recommender systems datasets. Through fair comparative experiments, we find that some structurally simplistic recommendation algorithms are underestimated, as they can achieve comparable results to more complex algorithms while maintaining lower complexity. Furthermore, our experimental analysis indicates that more complex methods exhibit better robustness when there are significant differences between tasks or scenarios. By providing a unified framework (MMLRec), our goal is to promote rapid evaluation and inspire innovative research in this continuously evolving field. We hope that our open-source benchmark can facilitate swift, equitable evaluations, while also fostering further breakthrough research in the domains of MTL, MSL, and MTMSL.

Click-through rate (CTR) prediction plays a crucial role in recommendation systems, with significant impact on user experience and platform revenue generation. Despite the various public CTR datasets available due to increasing interest from both academia and industry, these datasets have limitations. They cover a limited range of scenarios and predominantly focus on ID-based features, neglecting the vital role of multi-modal features for effective multi-scenario CTR prediction. Moreover, their scale is modest compared to real-world industrial datasets, hindering robust and comprehensive evaluation of complex models. To address these challenges, we introduce a large-scale Multi-Scenario Multi-Modal CTR dataset named AntM2 C, built from real industrial data from Alipay. This dataset offers an impressive breadth and depth of information, covering CTR data from four diverse business scenarios, including advertisements, consumer coupons, mini-programs, and videos. Unlike existing datasets, AntM2 C provides not only ID-based features but also five textual features and one image feature for both users and items, supporting more delicate multi-modal CTR prediction. AntM2 C is also substantially larger than existing datasets, comprising 100 million CTR data. This scale allows for robust and comprehensive evaluation and comparison of CTR prediction models. We employ AntM2 C to construct several typical CTR tasks, including multi-scenario modeling, item and user cold-start modeling, and multi-modal modeling. Initial experiments and comparisons with baseline methods have shown that AntM2 C presents both new challenges and opportunities for CTR models, with the potential to significantly advance CTR research. The AntM2 C dataset is available at https://www.atecup.cn/OfficalDataSet.

Industrial recommender systems usually need to serve multiple scenarios at the same time. In practice, there are various heterogeneous scenarios, since users frequently engage in scenarios with varying intentions and the items within each scenario typically belong to diverse categories. Existing works of multi-scenario recommendation mainly focus on modeling homogeneous scenarios which have similar data distributions. They equally transfer knowledge to each scenario without considering the diversity of heterogeneous scenarios. In this paper, we argue that the heterogeneity in multi-scenario recommendations is a key problem that needs to be solved. To this end, we propose an industrial framework named Scenario-Aware Model-Agnostic Meta Distillation (SAMD) for the multi-scenario recommendation. SAMD aims to provide scenario-aware and model-agnostic knowledge sharing across heterogeneous scenarios by modeling scenarios' relationship and conducting heterogeneous knowledge distillation. Specifically, SAMD first measures the comprehensive representation of each scenario and then proposes a novel meta distillation paradigm to conduct scenario-aware knowledge sharing. The meta network first establishes the potential scenarios' relationships and generates the strategies of knowledge sharing for each scenario. Then the heterogeneous knowledge distillation utilizes scenario-aware strategies to share knowledge across heterogeneous scenarios through intermediate features distillation without the restriction of the model architecture. In this way, SAMD shares knowledge across heterogeneous scenarios in a scenario-aware and model-agnostic manner, which addresses the problem of heterogeneity. Compared with other state-of-the-art methods, extensive offline experiments, and online A/B testing demonstrate the superior performance of the proposed SAMD framework, especially in heterogeneous scenarios.

In industrial recommendation systems, both data sizes and computational resources vary across different scenarios. For scenarios with limited data, data sparsity can lead to a decrease in model performance. Heterogeneous knowledge distillation-based transfer learning can be used to transfer knowledge from models in data-rich domains. However, in recommendation systems, the target domain possesses specific privileged features that significantly contribute to the model. While existing knowledge distillation methods have not taken these features into consideration, leading to suboptimal transfer weights. To overcome this limitation, we propose a novel algorithm called Uncertainty-based Heterogeneous Privileged Knowledge Distillation (UHPKD). Our method aims to quantify the knowledge of both the source and target domains, which represents the uncertainty of the models. This approach allows us to derive transfer weights based on the knowledge gain, which captures the difference in knowledge between the source and target domains. Experiments conducted on both public and industrial datasets demonstrate the superiority of our UHPKD algorithm compared to other state-of-the-art methods.

Conversion Rate (CVR) prediction is a crucial task in online advertising systems. Existing single-domain CVR prediction models suffer from the data sparsity problem since few users purchase items after clicking. In recent years, a robust and effective technique called fine-tuning can transfer knowledge from a data-rich source domain to enhance the CVR prediction performance in a data-sparse target domain. However, since most CVR prediction models have a large number of parameters, fine-tuning all the parameters on a data-sparse domain may lead to over-fitting. In this paper, we propose a general and efficient transfer learning method called Gradient-Flow based Fine-Tuning (GFlow-FT), which only needs to update a subset of parameters (called child network) via pruning the gradients to restrain gradient norm against over-fitting. In addition, our method employs the gradient-flow based measure via calculating the Hessian-gradient product as the criteria for picking the child network, which is superior to the magnitude-based and loss-based measure from empirical results. Extensive experimental results on three real-world datasets from recommendation systems show that GFlow-FT can significantly improve the performance of CVR prediction compared with state-of-the-art fine-tuning approaches.

Deep learning based face swapping technologies have opened new frontiers for entertainment industries while pose novel threats to identity security. Applying face swapping to real-world products, as well as defending against its misuse, rely on the capacity to generate high quality face swapped images from realistic scenarios where high resolution images are hard to come by. To this end, we need to address the drawbacks of existing methods, especially on their lacking on maintaining the detail attributes and their dependency on high resolution images as inputs. In this paper, we propose a novel Attribute-Conditioned Face Swapping Network (AFSNet) to preserve attributes and handle low resolution images. Specifically, we use an Image Enhancement Network (IEN) to restore high resolution images from low resolution images and a Face Exchange Module (FEM) to swap the faces. In the FEM, we improve the fidelities of the generated images by using a novel multi-domain feature fusion module (MDFFM) to integrate the identity feature, context feature, IEN feature, and attribute vector to obtain the final image. We also design an attribute transfer loss to promote the consistency of the attributes such as beards and youth between the source and swapped images. The experiments demonstrate our method’s superior performance compared with the state-of-the-art methods.