Contextual bandit algorithms are crucial in various decision-making applications, such as personalized content recommendation, online advertising, and e-commerce banner placement. Despite their successful applications in various domains, contextual bandit algorithms still face significant challenges with exploration efficiency compared to non-contextual bandit algorithms due to exploration in feature spaces. To overcome this issue, model selection policies such as MetaEXP and MetaCORRAL have been proposed to interactively explore base policies. In this paper, we introduce a novel counterfactual approach to address the model selection problem in contextual bandits. Unlike previous methods, our approach leverages unbiased Off-Policy Evaluation (OPE) to dynamically select base policies, making it more robust to model misspecification. We present two new algorithms, MetaEXP-OPE and MetaGreedy-OPE, which utilize OPE for model selection policy. We also provide theoretical analysis on regret bounds and evaluate the impact of different OPE estimators. We evaluated our model on synthetic data and a semi-synthetic simulator using a real-world dataset, and the results show that MetaEXP-OPE and MetaGreedy-OPE significantly outperform existing policies, including MetaEXP and MetaCORRAL.

Audience Expansion, a technique for identifying new audiences with similar behaviors to the original target or seed users. The major challenges include a heterogeneous user base, intricate marketing campaigns, constraints imposed by sparsity, and limited seed users, which lead to overfitting. In this context, we propose a novel solution named AudienceLinkNet, specifically designed to address the challenges associated with audience expansion in the context of Rakuten's diverse services and its clients. Our approach formulates the audience expansion problem as a graph problem and explores the combination of a Pre-trained Knowledge Graph Embedding Model and a Graph Convolutional Networks (GCNs). It emphasizes the structural retention properties of GCNs, enabling the model to overcome challenges related to cross-service data usage, sparsity and limited seed data. AudienceLinkNet simplifies the targeting process for small and large marketing campaigns and better utilizes demographics and behavioral attributes for targeting. Extensive experiments on our advertising platform, Rakuten AIris Target Prospecting, demonstrate the effectiveness of our audience expansion model. Additionally, we present the limitations of AudienceLinkNet.

Recommender systems are powerful tools for enhancing customer engagement and driving sales for Rakuten businesses. However, to achieve their full potential, these systems must possess a profound understanding of customer behaviors. This understanding can be gained from a variety of sources, including customer purchase history, customer feedback, and customer behavioral patterns. One of the most important aspects of customer understanding is the ability to identify lookalike customers, understand their behavioral patterns, and predict lifestyles, for example, whether a customer is married or unmarried, owns a car or plays golf, etc. Rakuten provides more than 70 different services and heavily relies on recommendations for many of its products. In our platforms, we can observe groups of customers who share similar interests, needs, or behaviors often end up being attracted to similar products or services. Customer preferences can change over time, so it is important for recommender systems to adapt those changes. This can be achieved by tracking customer behavior, static or dynamic environment changes around targeted customers, and their feedback. We utilize various graph and deep learning based models to address the customer understanding problem.

Lookalike models are based on the assumption that user similarity plays an important role towards product selling and enhancing the existing advertising campaigns from a very large user base. Challenges associated to these models reside on the heterogeneity of the user base and its sparsity. In this work, we propose a novel framework that unifies the customers' different behaviors or features such as demographics, buying behaviors on different platforms, customer loyalty behaviors and build a lookalike model to improve customer targeting for Rakuten Group, Inc. Extensive experiments on real e-commerce and travel datasets demonstrate the effectiveness of our proposed lookalike model for user targeting task.

Existing algorithms aiming to learn a binary classifier from positive (P) and unlabeled (U) data generally require estimating the class prior or label noises ahead of building a classification model. However, the estimation and classifier learning are normally conducted in a pipeline instead of being jointly optimized. In this paper, we propose to alternatively train the two steps using reinforcement learning. Our proposal adopts a policy network to adaptively make assumptions on the labels of unlabeled data, while a classifier is built upon the output of the policy network and provides rewards to learn a better strategy. The dynamic and interactive training between the policy maker and the classifier can exploit the unlabeled data in a more effective manner and yield a significant improvement on the classification performance. Furthermore, we present two different approaches to represent the actions sampled from the policy. The first approach considers continuous actions as soft labels, while the other uses discrete actions as hard assignment of labels for unlabeled examples. We validate the effectiveness of the proposed method on two benchmark datasets as well as one e-commerce dataset. The result shows the proposed method is able to consistently outperform state-of-the-art methods in various settings.

This paper leverages heterogeneous auxiliary information to address the data sparsity problem of recommender systems. We propose a model that learns a shared feature space from heterogeneous data, such as item descriptions, product tags and online purchase history, to obtain better predictions. Our model consists of autoencoders, not only for numerical and categorical data, but also for sequential data, which enables capturing user tastes, item characteristics and the recent dynamics of user preference. We learn the autoencoder architecture for each data source independently in order to better model their statistical properties. Our evaluation on two MovieLens datasets and an e-commerce dataset shows that mean average precision and recall improve over state-of-the-art methods.

We propose an encoder-decoder neural architecture to model user session and intent using browsing and purchasing data from a large e-commerce company.

In this paper, we propose a new system extracting potentially copyright infringement texts from the Web, called EPCI. EPCI extracts them in the following way: (1) generating a set of queries based on a given copyright reserved seed-text, (2) putting every query to search engine API, (3) gathering the search result Web pages from high ranking until the similarity between the given seed-text and the search result pages becomes less than a given threshold value, and (4) merging all the gathered pages, then re-ranking them in the order of their similarity. Our experimental result using 40 seed-texts shows that EPCI is able to extract 132 potentially copyright infringement Web pages per a given copyright reserved seed-text with 94% precision in average.